Struct mz_adapter::coord::timeline::DurableTimestampOracle

pub struct DurableTimestampOracle<T> {
    timestamp_oracle: TimestampOracle<T>,
    durable_timestamp: T,
    persist_interval: T,
}

A type that wraps a TimestampOracle and provides durable timestamps. This allows us to recover a timestamp that is larger than all previous timestamps on restart. The protocol is based on timestamp recovery from Percolator https://research.google/pubs/pub36726/. We “pre-allocate” a group of timestamps at once, and only durably store the largest of those timestamps. All timestamps within that interval can be served directly from memory, without going to disk. On restart, we re-initialize the current timestamp to a value one larger than the persisted timestamp.

See TimestampOracle for more details on the properties of the timestamps.

Fields

timestamp_oracle: TimestampOracle<T>

durable_timestamp: T

persist_interval: T

Implementations

impl<T: TimestampManipulation> DurableTimestampOracle<T>

pub(crate) async fn new<F, Fut>( initially: T, next: F, persist_interval: T, persist_fn: impl FnOnce(T) -> Fut ) -> Selfwhere F: Fn() -> T + 'static, Fut: Future<Output = Result<(), Error>>,

Create a new durable timeline, starting at the indicated time. Timestamps will be allocated in groups of size persist_interval. Also returns the new timestamp that needs to be persisted to disk.

See TimestampOracle::new for more details.

async fn write_ts<Fut>( &mut self, persist_fn: impl FnOnce(T) -> Fut ) -> WriteTimestamp<T>where Fut: Future<Output = Result<(), Error>>,

Acquire a new timestamp for writing. Optionally returns a timestamp that needs to be persisted to disk.

See TimestampOracle::write_ts for more details.

async fn peek_write_ts(&self) -> T

Peek current write timestamp.

pub async fn read_ts(&self) -> T

Acquire a new timestamp for reading. Optionally returns a timestamp that needs to be persisted to disk.

See TimestampOracle::read_ts for more details.

pub async fn apply_write<Fut>( &mut self, lower_bound: T, persist_fn: impl FnOnce(T) -> Fut )where Fut: Future<Output = Result<(), Error>>,

Mark a write at write_ts completed.

See TimestampOracle::apply_write for more details.

async fn maybe_allocate_new_timestamps<Fut>( &mut self, ts: &T, persist_fn: impl FnOnce(T) -> Fut )where Fut: Future<Output = Result<(), Error>>,

Checks to see if we can serve the timestamp from memory, or if we need to durably store a new timestamp.

If ts is less than the persisted timestamp then we can serve ts from memory, otherwise we need to durably store some timestamp greater than ts.