Module persist_source

Expand description

A source that reads from an a persist shard.

Structs§

FlowControl: Flow control configuration.
PendingWork 🔒: Pending work to read from fetched parts
Subtime: This opaque token represents progress within a timestamp, allowing finer-grained frontier progress than would otherwise be possible.
SubtimeContainer: Derived columnar container for a struct.
SubtimeReference: Derived columnar reference for a struct.

Backpressureable: A trait representing a type that can be used in backpressure.

backpressure: Apply flow control to the data input, based on the given FlowControl.
decode_and_mfp 🔒: Decode fetched parts and apply map_filter_project, writing ok records into CB’s containers and err records into a separate output.
decode_part 🔒: Read part, apply the MFP, and hand every record to give, stopping once work reaches yield_fuel. Returns whether the part is exhausted.
fetch_parts 🔒: Fetch the parts of a persist shard a dataflow needs, distributing the work of reading them across all timely workers.
filter_result 🔒
persist_source: Creates a new source that reads from a persist shard, distributing the work of reading data to all timely workers.
persist_source_core: Creates a new source that reads from a persist shard, distributing the work of reading data to all timely workers.

ErrBuilder 🔒: Err-side container builder.
RefinedScope 🔒
RefinedTime 🔒: The time a decode operator’s capabilities carry, refined with a Subtime so flow control can pace parts within a millisecond.
RowVecBuilder: Ok-side container builder producing row vectors timestamped with T.