pub type Col2ValPagedBatcher<K, V, T, R> = ColumnMergeBatcher<(K, V), T, R>;Expand description
Pageable counterpart to Col2ValBatcher. Routes every chunk produced
by chunking, merging, or extract through a crate::column_pager::ColumnPager,
so memory pressure can spill chains to a backing store without touching
the merge / extract bodies.
Drop-in shape at the type level: both aliases take (K, V, T, R) and
produce a Batcher<Input = Column<((K, V), T, R)>, Output = Column<((K, V), T, R)>>. Call sites can swap with cargo fix–style renaming once
downstream Trace/Builder impls have been wired up. The pager itself
defaults to crate::column_pager::ColumnPager::disabled; inject a
real one via merge_batcher::ColumnMergeBatcher::set_pager.
Aliased Type§
pub struct Col2ValPagedBatcher<K, V, T, R> {
chains: Vec<VecDeque<PagedColumn<((K, V), T, R)>>>,
lower: Antichain<T>,
frontier: Antichain<T>,
stash: Vec<Column<((K, V), T, R)>>,
pager_override: Option<ColumnPager>,
logger: Option<TypedLogger<CapacityContainerBuilder<Vec<(Duration, DifferentialEvent)>>, DifferentialEvent>>,
operator_id: usize,
}Fields§
§chains: Vec<VecDeque<PagedColumn<((K, V), T, R)>>>§lower: Antichain<T>§frontier: Antichain<T>§stash: Vec<Column<((K, V), T, R)>>Recycled empty Column::Typed chunks. Drained heads and shipped result
buffers feed in here; subsequent merge / extract calls pop from here
instead of starting from a zero-capacity Column::default(). Mirrors
the stash carried by the upstream differential_dataflow merge-batcher
framework, which this type forks. Without it, each shipped chunk
triggers a fresh per-leaf grow cycle and per-merge-round allocation
dominates the inner loop.
pager_override: Option<ColumnPager>Optional override. None means “read column_pager::global_pager
fresh on every use” — the production path, so worker_config dyncfg
changes that re-install the process-global pager take effect on the
very next chunk this batcher processes.
logger: Option<TypedLogger<CapacityContainerBuilder<Vec<(Duration, DifferentialEvent)>>, DifferentialEvent>>§operator_id: usize