Struct mz_adapter::coord::catalog_oracle::CatalogTimestampOracle

pub struct CatalogTimestampOracle<T, N>where
    T: Debug,
    N: GenericNowFn<T>,{
    timestamp_oracle: InMemoryTimestampOracle<T, N>,
    durable_timestamp: T,
    persist_interval: T,
    timestamp_persistence: Box<dyn TimestampPersistence<T>>,
}

A type that wraps a InMemoryTimestampOracle and provides durable timestamps. This allows us to recover a timestamp that is larger than all previous timestamps on restart. The protocol is based on timestamp recovery from Percolator https://research.google/pubs/pub36726/. We “pre-allocate” a group of timestamps at once, and only durably store the largest of those timestamps. All timestamps within that interval can be served directly from memory, without going to disk. On restart, we re-initialize the current timestamp to a value one larger than the persisted timestamp.

See TimestampOracle for more details on the properties of the timestamps.

Fields

timestamp_oracle: InMemoryTimestampOracle<T, N>

durable_timestamp: T

persist_interval: T

timestamp_persistence: Box<dyn TimestampPersistence<T>>

Implementations

impl<T: TimestampManipulation, N> CatalogTimestampOracle<T, N>where N: GenericNowFn<T>,

pub(crate) async fn new<P>( initially: T, next: N, persist_interval: T, timestamp_persistence: P ) -> Selfwhere P: TimestampPersistence<T> + 'static,

Create a new durable timeline, starting at the indicated time. Timestamps will be allocated in groups of size persist_interval. Also returns the new timestamp that needs to be persisted to disk.

See InMemoryTimestampOracle::new for more details.

async fn maybe_allocate_new_timestamps(&mut self, ts: &T)

Checks to see if we can serve the timestamp from memory, or if we need to durably store a new timestamp.

If ts is less than the persisted timestamp then we can serve ts from memory, otherwise we need to durably store some timestamp greater than ts.

Trait Implementations

impl<T: TimestampManipulation, N> TimestampOracle<T> for CatalogTimestampOracle<T, N>where N: GenericNowFn<T>,

fn write_ts<'life0, 'async_trait>( &'life0 mut self ) -> Pin<Box<dyn Future<Output = WriteTimestamp<T>> + 'async_trait>>where Self: 'async_trait, 'life0: 'async_trait,

Acquire a new timestamp for writing.

fn peek_write_ts<'life0, 'async_trait>( &'life0 self ) -> Pin<Box<dyn Future<Output = T> + 'async_trait>>where Self: 'async_trait, 'life0: 'async_trait,

Peek the current write timestamp.

fn read_ts<'life0, 'async_trait>( &'life0 self ) -> Pin<Box<dyn Future<Output = T> + 'async_trait>>where Self: 'async_trait, 'life0: 'async_trait,

Acquire a new timestamp for reading.

fn apply_write<'life0, 'async_trait>( &'life0 mut self, write_ts: T ) -> Pin<Box<dyn Future<Output = ()> + 'async_trait>>where Self: 'async_trait, 'life0: 'async_trait,

Mark a write at write_ts completed.

fn get_shared( &self ) -> Option<Arc<dyn ShareableTimestampOracle<T> + Send + Sync>>

Get a shared, shallow clone of the oracle. Returns None if this oracle is not shareable.