pub type Col2ValPagedBatcher<K, V, T, R, Chu, Se> = ColumnMergeBatcher<Chu, (K, V), T, R, Se>;Expand description
Pageable counterpart to Col2ValBatcher. Routes every chunk produced
by chunking, merging, or extract through a crate::column_pager::ColumnPager,
so memory pressure can spill chains to a backing store without touching
the merge / extract bodies.
Drop-in shape at the type level: both aliases take the same parameters and
produce a Batcher<C> over the same chunk type. The pager itself defaults
to crate::column_pager::ColumnPager::disabled; inject a real one via
merge_batcher::ColumnMergeBatcher::set_pager.
Aliased Type§
pub struct Col2ValPagedBatcher<K, V, T, R, Chu, Se> {
chunker: Chu,
chains: Vec<VecDeque<PagedColumn<((K, V), T, R)>>>,
frontier: Antichain<T>,
stash: Vec<Column<((K, V), T, R)>>,
pager_override: Option<ColumnPager>,
logger: Option<TypedLogger<CapacityContainerBuilder<Vec<(Duration, DifferentialEvent)>>, DifferentialEvent>>,
operator_id: usize,
sealer: PhantomData<Se>,
}Fields§
§chunker: ChuMelds raw input containers into sorted, consolidated chunks.
chains: Vec<VecDeque<PagedColumn<((K, V), T, R)>>>§frontier: Antichain<T>§stash: Vec<Column<((K, V), T, R)>>Recycled empty Column::Typed chunks. Drained heads and shipped result
buffers feed in here; subsequent merge / extract calls pop from here
instead of starting from a zero-capacity Column::default(). Mirrors
the stash carried by the upstream differential_dataflow merge-batcher
framework, which this type forks. Without it, each shipped chunk
triggers a fresh per-leaf grow cycle and per-merge-round allocation
dominates the inner loop.
pager_override: Option<ColumnPager>Optional override. None means “read column_pager::global_pager
fresh on every use” — the production path, so worker_config dyncfg
changes that re-install the process-global pager take effect on the
very next chunk this batcher processes.
logger: Option<TypedLogger<CapacityContainerBuilder<Vec<(Duration, DifferentialEvent)>>, DifferentialEvent>>§operator_id: usize§sealer: PhantomData<Se>Seals each extracted chain into a batch.