Struct mz_persist_txn::txn_cache::TxnsCacheState
source · pub struct TxnsCacheState<T: Timestamp + Lattice + Codec64> { /* private fields */ }
Expand description
A cache of the txn shard contents, optimized for various in-memory operations.
§Implementation Details
Reads of data shards are almost as straightforward as writes. A data shard may be read normally, using snapshots, subscriptions, shard_source, etc, through the most recent non-empty write. However, the upper of the txns shard (and thus the logical upper of the data shard) may be arbitrarily far ahead of the physical upper of the data shard. As a result, we do the following:
- To take a snapshot of a data shard, the
as_of
is passed through unchanged if the timestamp of that shard’s latest non-empty write is past it. Otherwise, we know the times between them have no writes and can fill them with empty updates. Concretely, to read a snapshot as ofT
:- We read the txns shard contents up through and including
T
, blocking until the upper passesT
if necessary. - We then find, for the requested data shard, the latest non-empty write
at a timestamp
T' <= T
. - We wait for
T'
to be applied by watching the data shard upper. - We
compare_and_append
empty updates for(T', T]
, which is known by the txn system to not have writes for this shard (otherwise we’d have picked a differentT'
). - We read the snapshot at
T
as normal.
- We read the txns shard contents up through and including
- To iterate a listen on a data shard, when writes haven’t been read yet they are passed through unchanged, otherwise if the txns shard indicates that there are ranges of empty time progress is returned, otherwise progress to the txns shard will indicate when new information is available.
Note that all of the above can be determined solely by information in the txns shard. In particular, non-empty writes are indicated by updates with positive diffs.
Also note that the above is structured such that it is possible to write a timely operator with the data shard as an input, passing on all payloads unchanged and simply manipulating capabilities in response to data and txns shard progress. See crate::operator::txns_progress.
Implementations§
source§impl<T: Timestamp + Lattice + TotalOrder + StepForward + Codec64> TxnsCacheState<T>
impl<T: Timestamp + Lattice + TotalOrder + StepForward + Codec64> TxnsCacheState<T>
sourcepub fn registered_at_progress(&self, data_id: &ShardId, ts: &T) -> bool
pub fn registered_at_progress(&self, data_id: &ShardId, ts: &T) -> bool
Returns whether the data shard was registered to the txns set as of the current progress.
Specifically, a data shard is registered if the most recent register timestamp is set but the most recent forget timestamp is not set.
This function accepts a timestamp as input, but that timestamp must be equal to the progress exclusive, or else the function panics. It mainly acts as a way for the caller to think about the logical time at which this function executes. Times in the past may have been compacted away, and we can’t always return an accurate answer. If this function isn’t sufficient, you can usually find what you’re looking for by inspecting the times in the most recent registration.
sourcepub fn data_snapshot(&self, data_id: ShardId, as_of: T) -> DataSnapshot<T>
pub fn data_snapshot(&self, data_id: ShardId, as_of: T) -> DataSnapshot<T>
Returns a token exchangeable for a snapshot of a data shard.
A data shard might be definite at times past the physical upper because
of invariants maintained by this txn system. As a result, this method
discovers the latest potentially unapplied write before the as_of
.
Callers must first wait for TxnsCache::update_gt
with the same or
later timestamp to return. Panics otherwise.
sourcepub fn data_listen_next(&self, data_id: &ShardId, ts: &T) -> DataListenNext<T>
pub fn data_listen_next(&self, data_id: &ShardId, ts: &T) -> DataListenNext<T>
Returns the next action to take when iterating a Listen on a data shard.
A data shard Listen is executed by repeatedly calling this method with an exclusive progress frontier. The returned value indicates an action to take. Some of these actions advance the progress frontier, which results in calling this method again with a higher timestamp, and thus a new action. See DataListenNext for specifications of the actions.
Note that this is a state machine on self.progress_exclusive
and the
listen progress. DataListenNext indicates which state transitions to
take.
sourcepub fn min_unapplied_ts(&self) -> &T
pub fn min_unapplied_ts(&self) -> &T
Returns the minimum timestamp not known to be applied by this cache.
Trait Implementations§
Auto Trait Implementations§
impl<T> Freeze for TxnsCacheState<T>where
T: Freeze,
impl<T> RefUnwindSafe for TxnsCacheState<T>where
T: RefUnwindSafe,
impl<T> Send for TxnsCacheState<T>
impl<T> Sync for TxnsCacheState<T>
impl<T> Unpin for TxnsCacheState<T>where
T: Unpin,
impl<T> UnwindSafe for TxnsCacheState<T>where
T: UnwindSafe + RefUnwindSafe,
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
source§impl<R, O, T> CopyOnto<ConsecutiveOffsetPairs<R, O>> for T
impl<R, O, T> CopyOnto<ConsecutiveOffsetPairs<R, O>> for T
source§fn copy_onto(
self,
target: &mut ConsecutiveOffsetPairs<R, O>
) -> <ConsecutiveOffsetPairs<R, O> as Region>::Index
fn copy_onto( self, target: &mut ConsecutiveOffsetPairs<R, O> ) -> <ConsecutiveOffsetPairs<R, O> as Region>::Index
source§impl<T> FutureExt for T
impl<T> FutureExt for T
source§fn with_context(self, otel_cx: Context) -> WithContext<Self>
fn with_context(self, otel_cx: Context) -> WithContext<Self>
source§fn with_current_context(self) -> WithContext<Self>
fn with_current_context(self) -> WithContext<Self>
source§impl<T> Instrument for T
impl<T> Instrument for T
source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
source§impl<T> IntoRequest<T> for T
impl<T> IntoRequest<T> for T
source§fn into_request(self) -> Request<T>
fn into_request(self) -> Request<T>
T
in a tonic::Request
source§impl<T> Pointable for T
impl<T> Pointable for T
source§impl<P, R> ProtoType<R> for Pwhere
R: RustType<P>,
impl<P, R> ProtoType<R> for Pwhere
R: RustType<P>,
source§fn into_rust(self) -> Result<R, TryFromProtoError>
fn into_rust(self) -> Result<R, TryFromProtoError>
RustType::from_proto
.source§fn from_rust(rust: &R) -> P
fn from_rust(rust: &R) -> P
RustType::into_proto
.