Module cfg 

The tunable knobs for persist.

Structs

PersistConfig

The tunable knobs for persist.

RetryParameters

Constants

BATCH_BUILDER_MAX_OUTSTANDING_PARTS

The maximum number of parts (s3 blobs) that crate::batch::BatchBuilder will pipeline before back-pressuring crate::batch::BatchBuilder::add calls on previous ones finishing.

COMPACTION_HEURISTIC_MIN_INPUTS

In Compactor::compact_and_apply, we do the compaction (don’t skip it) if the number of inputs is at least this many. Compaction is performed if any of the heuristic criteria are met (they are OR’d).

COMPACTION_HEURISTIC_MIN_PARTS

In Compactor::compact_and_apply, we do the compaction (don’t skip it) if the number of batch parts is at least this many. Compaction is performed if any of the heuristic criteria are met (they are OR’d).

COMPACTION_HEURISTIC_MIN_UPDATES

In Compactor::compact_and_apply, we do the compaction (don’t skip it) if the number of updates is at least this many. Compaction is performed if any of the heuristic criteria are met (they are OR’d).

COMPACTION_MEMORY_BOUND_BYTES

The upper bound on compaction’s memory consumption. The value must be at least 4*blob_target_size. Increasing this value beyond the minimum allows compaction to merge together more runs at once, providing greater consolidation of updates, at the cost of greater memory usage.

CONSENSUS_CONNECTION_POOL_MAX_SIZE

Sets the maximum size of the connection pool that is used by consensus.

CRDB_CONNECT_TIMEOUT

The duration to wait for a Consensus Postgres/CRDB connection to be made before retrying.

CRDB_KEEPALIVES_IDLE

CRDB_KEEPALIVES_INTERVAL

CRDB_KEEPALIVES_RETRIES

CRDB_TCP_USER_TIMEOUT

The TCP user timeout for a Consensus Postgres/CRDB connection. Specifies the amount of time that transmitted data may remain unacknowledged before the TCP connection is forcibly closed.

GC_BLOB_DELETE_CONCURRENCY_LIMIT

The maximum number of concurrent blob deletes during garbage collection.

STATE_VERSIONS_RECENT_LIVE_DIFFS_LIMIT

The # of diffs to initially scan when fetching the latest consensus state, to determine which requests go down the fast vs slow path. Should be large enough to fetch all live diffs in the steady-state, and small enough to query Consensus at high volume. Steady-state usage should accommodate readers that require seqno-holds for reasonable amounts of time, which to start we say is 10s of minutes.

USAGE_STATE_FETCH_CONCURRENCY_LIMIT

The maximum number of concurrent state fetches during usage computation.

USE_CRITICAL_SINCE_CATALOG

Migrate the catalog to use the critical since when opening a new read handle.

USE_CRITICAL_SINCE_SNAPSHOT

Migrate snapshots to use the critical since when opening a new read handle.

USE_CRITICAL_SINCE_SOURCE

Migrate the persist source to use the critical since when opening a new read handle.

USE_CRITICAL_SINCE_TXN

Migrate the txns code to use the critical since when opening a new read handle.

Functions

all_dyncfgs

Adds the full set of all persist Configs.

code_can_read_data

If persist gets some encoded ProtoState from the future (e.g. two versions of code are running simultaneously against the same shard), it might have a field that the current code doesn’t know about. This would be silently discarded at proto decode time: our Proto library can’t handle unknown fields, and old versions of code might not be able to respect the semantics of the new fields even if they did.

code_can_write_data

Can the given version of the code generate data that older versions can understand? Imagine the case of eg. garbage collection after a version upgrade… we may need to read old diffs to be able to find blobs to delete, even if we no longer have code to generate data in that format.