Struct mz_storage::source::reclock::ReclockFollower

pub struct ReclockFollower<FromTime: Timestamp, IntoTime: Timestamp + Lattice + Display> {
    since: Antichain<IntoTime>,
    pub inner: Rc<RefCell<ReclockFollowerInner<FromTime, IntoTime>>>,
}

A “follower” for the ReclockOperator, that maintains a trace based on the results of reclocking and data from the source. It provides the reclock method, which produces messages with their associated timestamps.

Shareable with .share()

Fields

since: Antichain<IntoTime>

The since maintained by the local handle. This may be beyond the shared since

inner: Rc<RefCell<ReclockFollowerInner<FromTime, IntoTime>>>

Implementations

impl<FromTime, IntoTime> ReclockFollower<FromTime, IntoTime>

where FromTime: Timestamp, IntoTime: Timestamp + Lattice + Display,

pub fn new(as_of: Antichain<IntoTime>) -> Self

Constructs a new ReclockFollower

pub fn source_upper(&self) -> Antichain<FromTime>

pub fn initialized(&self) -> bool

pub fn push_trace_batch(&self, batch: ReclockBatch<FromTime, IntoTime>)

Pushes a new trace batch into this ReclockFollower.

pub fn reclock<'a, M: 'a>( &'a self, batch: impl IntoIterator<Item = (M, FromTime)> + 'a, ) -> impl Iterator<Item = (M, Result<IntoTime, ReclockError<FromTime>>)> + 'a

where IntoTime: TotalOrder,

Reclocks a batch of messages timestamped with FromTime and returns an iterator of messages timestamped with IntoTime.

Each item of the resulting iterator will be associated with either the time it should be reclocked to or an error indicating that a reclocking decision could not be taken with the data that we have at hand.

This method is most efficient when the to be reclocked iterator presents data in contiguous runs with the same FromTime.

pub fn reclock_time( &self, src_ts: &FromTime, ) -> Result<Antichain<IntoTime>, ReclockError<FromTime>>

Reclocks a single FromTime timestamp into the IntoTime time domain.

pub fn reclock_time_total( &self, src_ts: &FromTime, ) -> Result<IntoTime, ReclockError<FromTime>>

where IntoTime: TotalOrder,

Reclocks a single FromTime timestamp into a totally ordered IntoTime time domain.

pub fn reclock_frontier( &self, source_frontier: AntichainRef<'_, FromTime>, ) -> Result<Antichain<IntoTime>, ReclockError<FromTime>>

Reclocks a FromTime frontier into a IntoTime frontier.

The conversion has the property that all messages that are beyond the provided FromTime frontier will be relocked at times that will be beyond the returned IntoTime frontier. This can be used to drive a IntoTime capability forward when the caller knows that a FromTime frontier has advanced.

The method returns an error if the FromTime frontier is not beyond the since frontier. The error will contain the offending FromTime.

pub fn source_upper_at_frontier<'a>( &self, frontier: AntichainRef<'a, IntoTime>, ) -> Result<Antichain<FromTime>, ReclockError<AntichainRef<'a, IntoTime>>>

Reclocks an IntoTime frontier into a FromTime frontier. The conversion has the property that all messages that would be reclocked to times beyond the provided IntoTime frontier will be beyond the returned FromTime frontier. This can be used to compute a safe starting point to resume producing an IntoTime collection at a particular frontier.

pub fn compact(&mut self, new_since: Antichain<IntoTime>)

Compacts the trace held by this reclock follower to the specified frontier.

Reclocking has the property that it commutes with compaction. What this means is that reclocking a collection and then compacting the result to some frontier F will produce exactly the same result with first compacting the remap trace to frontier F and then reclocking the collection.

pub fn since(&self) -> AntichainRef<'_, IntoTime>

pub fn share(&self) -> Self

pub fn size(&self) -> usize

The number of remap bindings in the trace

Trait Implementations

impl<FromTime: Timestamp, IntoTime: Timestamp + Lattice + Display> Drop for ReclockFollower<FromTime, IntoTime>

fn drop(&mut self)

Executes the destructor for this type.