Module mz_storage::source::reclock

source ·
Expand description

Operators that transform collections that evolve with some timestamp FromTime into a collections that evolve with some other timestamp `IntoTime.

Reclocking happens in two separate phases, implemented by ReclockOperator and ReclockFollower respectively.

Modules

  • Reclocking compatibility code until the whole ingestion pipeline is transformed to native timestamps

Structs

  • A “follower” for the ReclockOperator, that maintains a trace based on the results of reclocking and data from the source. It provides the reclock method, which produces messages with their associated timestamps.
  • The ReclockOperator is responsible for observing progress in the FromTime domain and consume messages from a ticker of progress in the IntoTime domain. When the source frontier advances and the ticker ticks the ReclockOperator will generate the data that describe this correspondence and write them out to its provided remap handle. The output generated by the reclock operator can be thought of as Collection<G, FromTime> where G::Timestamp is IntoTime.

Enums