Struct mz_persist_txn::txn_cache::TxnsCacheState

pub struct TxnsCacheState<T: Timestamp + Lattice + Codec64> { /* private fields */ }

A cache of the txn shard contents, optimized for various in-memory operations.

Implementation Details

Reads of data shards are almost as straightforward as writes. A data shard may be read normally, using snapshots, subscriptions, shard_source, etc, through the most recent non-empty write. However, the upper of the txns shard (and thus the logical upper of the data shard) may be arbitrarily far ahead of the physical upper of the data shard. As a result, we do the following:

• To take a snapshot of a data shard, the as_of is passed through unchanged if the timestamp of that shard’s latest non-empty write is past it. Otherwise, we know the times between them have no writes and can fill them with empty updates. Concretely, to read a snapshot as of T:
  • We read the txns shard contents up through and including T, blocking until the upper passes T if necessary.
  • We then find, for the requested data shard, the latest non-empty write at a timestamp T' <= T.
  • We wait for T' to be applied by watching the data shard upper.
  • We compare_and_append empty updates for (T', T], which is known by the txn system to not have writes for this shard (otherwise we’d have picked a different T').
  • We read the snapshot at T as normal.
• To iterate a listen on a data shard, when writes haven’t been read yet they are passed through unchanged, otherwise if the txns shard indicates that there are ranges of empty time progress is returned, otherwise progress to the txns shard will indicate when new information is available.

Note that all of the above can be determined solely by information in the txns shard. In particular, non-empty writes are indicated by updates with positive diffs.

Also note that the above is structured such that it is possible to write a timely operator with the data shard as an input, passing on all payloads unchanged and simply manipulating capabilities in response to data and txns shard progress. See crate::operator::txns_progress.

Implementations

impl<T: Timestamp + Lattice + TotalOrder + StepForward + Codec64> TxnsCacheState<T>

pub fn txns_id(&self) -> ShardId

Returns the ShardId of the txns shard.

pub fn registered_at_progress(&self, data_id: &ShardId, ts: &T) -> bool

Returns whether the data shard was registered to the txns set as of the current progress.

Specifically, a data shard is registered if the most recent register timestamp is set but the most recent forget timestamp is not set.

This function accepts a timestamp as input, but that timestamp must be equal to the progress exclusive, or else the function panics. It mainly acts as a way for the caller to think about the logical time at which this function executes. Times in the past may have been compacted away, and we can’t always return an accurate answer. If this function isn’t sufficient, you can usually find what you’re looking for by inspecting the times in the most recent registration.

pub fn data_snapshot(&self, data_id: ShardId, as_of: T) -> DataSnapshot<T>

Returns a token exchangeable for a snapshot of a data shard.

A data shard might be definite at times past the physical upper because of invariants maintained by this txn system. As a result, this method discovers the latest potentially unapplied write before the as_of.

Callers must first wait for TxnsCache::update_gt with the same or later timestamp to return. Panics otherwise.

pub fn data_listen_next(&self, data_id: &ShardId, ts: &T) -> DataListenNext<T>

Returns the next action to take when iterating a Listen on a data shard.

A data shard Listen is executed by repeatedly calling this method with an exclusive progress frontier. The returned value indicates an action to take. Some of these actions advance the progress frontier, which results in calling this method again with a higher timestamp, and thus a new action. See DataListenNext for specifications of the actions.

Note that this is a state machine on self.progress_exclusive and the listen progress. DataListenNext indicates which state transitions to take.

pub fn min_unapplied_ts(&self) -> &T

Returns the minimum timestamp not known to be applied by this cache.

Trait Implementations

impl<T: Debug + Timestamp + Lattice + Codec64> Debug for TxnsCacheState<T>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.