Module mz_persist_client::cfg

source ·
Expand description

The tunable knobs for persist.

Structs§

Constants§

  • BATCH_BUILDER_MAX_OUTSTANDING_PARTS
    The maximum number of parts (s3 blobs) that crate::batch::BatchBuilder will pipeline before back-pressuring crate::batch::BatchBuilder::add calls on previous ones finishing.
  • COMPACTION_HEURISTIC_MIN_INPUTS
    In Compactor::compact_and_apply, we do the compaction (don’t skip it) if the number of inputs is at least this many. Compaction is performed if any of the heuristic criteria are met (they are OR’d).
  • COMPACTION_HEURISTIC_MIN_PARTS
    In Compactor::compact_and_apply, we do the compaction (don’t skip it) if the number of batch parts is at least this many. Compaction is performed if any of the heuristic criteria are met (they are OR’d).
  • COMPACTION_HEURISTIC_MIN_UPDATES
    In Compactor::compact_and_apply, we do the compaction (don’t skip it) if the number of updates is at least this many. Compaction is performed if any of the heuristic criteria are met (they are OR’d).
  • COMPACTION_MEMORY_BOUND_BYTES
    The upper bound on compaction’s memory consumption. The value must be at least 4*blob_target_size. Increasing this value beyond the minimum allows compaction to merge together more runs at once, providing greater consolidation of updates, at the cost of greater memory usage.
  • CONSENSUS_CONNECTION_POOL_TTL 🔒
    The minimum TTL of a connection to Postgres/CRDB before it is proactively terminated. Connections are routinely culled to balance load against the downstream database.
  • CONSENSUS_CONNECTION_POOL_TTL_STAGGER 🔒
    The minimum time between TTLing connections to Postgres/CRDB. This delay is used to stagger reconnections to avoid stampedes and high tail latencies. This value should be much less than consensus_connection_pool_ttl so that reconnections are biased towards terminating the oldest connections first. A value of consensus_connection_pool_ttl / consensus_connection_pool_max_size is likely a good place to start so that all connections are rotated when the pool is fully used.
  • CRDB_CONNECT_TIMEOUT
    The duration to wait for a Consensus Postgres/CRDB connection to be made before retrying.
  • CRDB_TCP_USER_TIMEOUT
    The TCP user timeout for a Consensus Postgres/CRDB connection. Specifies the amount of time that transmitted data may remain unacknowledged before the TCP connection is forcibly closed.
  • GC_BLOB_DELETE_CONCURRENCY_LIMIT
    The maximum number of concurrent blob deletes during garbage collection.
  • MiB 🔒
  • STATE_VERSIONS_RECENT_LIVE_DIFFS_LIMIT
    The # of diffs to initially scan when fetching the latest consensus state, to determine which requests go down the fast vs slow path. Should be large enough to fetch all live diffs in the steady-state, and small enough to query Consensus at high volume. Steady-state usage should accommodate readers that require seqno-holds for reasonable amounts of time, which to start we say is 10s of minutes.
  • USAGE_STATE_FETCH_CONCURRENCY_LIMIT
    The maximum number of concurrent state fetches during usage computation.
  • USE_CRITICAL_SINCE_CATALOG
    Migrate the catalog to use the critical since when opening a new read handle.
  • USE_CRITICAL_SINCE_SNAPSHOT
    Migrate snapshots to use the critical since when opening a new read handle.
  • USE_CRITICAL_SINCE_SOURCE
    Migrate the persist source to use the critical since when opening a new read handle.
  • USE_CRITICAL_SINCE_TXN
    Migrate the txns code to use the critical since when opening a new read handle.
  • USE_GLOBAL_TXN_CACHE_SOURCE
    Migrate the persist source to use a process global txn cache.

Functions§