Expand description
The tunable knobs for persist.
Structs§
- Persist
Config - The tunable knobs for persist.
- Retry
Parameters
Constants§
- BATCH_
BUILDER_ MAX_ OUTSTANDING_ PARTS - The maximum number of parts (s3 blobs) that crate::batch::BatchBuilder will pipeline before back-pressuring crate::batch::BatchBuilder::add calls on previous ones finishing.
- BLOB_
CONNECT_ 🔒TIMEOUT - BLOB_
OPERATION_ 🔒ATTEMPT_ TIMEOUT - BLOB_
OPERATION_ 🔒TIMEOUT - BLOB_
READ_ 🔒TIMEOUT - COMPACTION_
HEURISTIC_ MIN_ INPUTS - In Compactor::compact_and_apply, we do the compaction (don’t skip it) if the number of inputs is at least this many. Compaction is performed if any of the heuristic criteria are met (they are OR’d).
- COMPACTION_
HEURISTIC_ MIN_ PARTS - In Compactor::compact_and_apply, we do the compaction (don’t skip it) if the number of batch parts is at least this many. Compaction is performed if any of the heuristic criteria are met (they are OR’d).
- COMPACTION_
HEURISTIC_ MIN_ UPDATES - In Compactor::compact_and_apply, we do the compaction (don’t skip it) if the number of updates is at least this many. Compaction is performed if any of the heuristic criteria are met (they are OR’d).
- COMPACTION_
MEMORY_ BOUND_ BYTES - The upper bound on compaction’s memory consumption. The value must be at
least 4*
blob_target_size
. Increasing this value beyond the minimum allows compaction to merge together more runs at once, providing greater consolidation of updates, at the cost of greater memory usage. - CONSENSUS_
CONNECTION_ POOL_ MAX_ SIZE - Sets the maximum size of the connection pool that is used by consensus.
- CONSENSUS_
CONNECTION_ 🔒POOL_ MAX_ WAIT - Sets the maximum amount of time we’ll wait to acquire a connection from the connection pool.
- CONSENSUS_
CONNECTION_ 🔒POOL_ TTL - The minimum TTL of a connection to Postgres/CRDB before it is proactively terminated. Connections are routinely culled to balance load against the downstream database.
- CONSENSUS_
CONNECTION_ 🔒POOL_ TTL_ STAGGER - The minimum time between TTLing connections to Postgres/CRDB. This delay is
used to stagger reconnections to avoid stampedes and high tail latencies.
This value should be much less than
consensus_connection_pool_ttl
so that reconnections are biased towards terminating the oldest connections first. A value ofconsensus_connection_pool_ttl / consensus_connection_pool_max_size
is likely a good place to start so that all connections are rotated when the pool is fully used. - CRDB_
CONNECT_ TIMEOUT - The duration to wait for a Consensus Postgres/CRDB connection to be made before retrying.
- CRDB_
TCP_ USER_ TIMEOUT - The TCP user timeout for a Consensus Postgres/CRDB connection. Specifies the amount of time that transmitted data may remain unacknowledged before the TCP connection is forcibly closed.
- GC_
BLOB_ DELETE_ CONCURRENCY_ LIMIT - The maximum number of concurrent blob deletes during garbage collection.
- LTS_
VERSIONS 🔒 - MiB 🔒
- STATE_
VERSIONS_ RECENT_ LIVE_ DIFFS_ LIMIT - The # of diffs to initially scan when fetching the latest consensus state, to determine which requests go down the fast vs slow path. Should be large enough to fetch all live diffs in the steady-state, and small enough to query Consensus at high volume. Steady-state usage should accommodate readers that require seqno-holds for reasonable amounts of time, which to start we say is 10s of minutes.
- USAGE_
STATE_ FETCH_ CONCURRENCY_ LIMIT - The maximum number of concurrent state fetches during usage computation.
- USE_
CRITICAL_ SINCE_ CATALOG - Migrate the catalog to use the critical since when opening a new read handle.
- USE_
CRITICAL_ SINCE_ SNAPSHOT - Migrate snapshots to use the critical since when opening a new read handle.
- USE_
CRITICAL_ SINCE_ SOURCE - Migrate the persist source to use the critical since when opening a new read handle.
- USE_
CRITICAL_ SINCE_ TXN - Migrate the txns code to use the critical since when opening a new read handle.
- USE_
GLOBAL_ TXN_ CACHE_ SOURCE - Migrate the persist source to use a process global txn cache.
Functions§
- all_
dyncfgs - Adds the full set of all persist Configs.
- check_
data_ version - check_
data_ 🔒version_ with_ lts_ versions