Skip to main content

Col2ValPagedBatcher

Type Alias Col2ValPagedBatcher 

Source
pub type Col2ValPagedBatcher<K, V, T, R> = ColumnMergeBatcher<(K, V), T, R>;
Expand description

Pageable counterpart to Col2ValBatcher. Routes every chunk produced by chunking, merging, or extract through a crate::column_pager::ColumnPager, so memory pressure can spill chains to a backing store without touching the merge / extract bodies.

Drop-in shape at the type level: both aliases take (K, V, T, R) and produce a Batcher<Input = Column<((K, V), T, R)>, Output = Column<((K, V), T, R)>>. Call sites can swap with cargo fix–style renaming once downstream Trace/Builder impls have been wired up. The pager itself defaults to crate::column_pager::ColumnPager::disabled; inject a real one via merge_batcher::ColumnMergeBatcher::set_pager.

Aliased Type§

pub struct Col2ValPagedBatcher<K, V, T, R> {
    chains: Vec<VecDeque<PagedColumn<((K, V), T, R)>>>,
    lower: Antichain<T>,
    frontier: Antichain<T>,
    stash: Vec<Column<((K, V), T, R)>>,
    pager_override: Option<ColumnPager>,
    logger: Option<TypedLogger<CapacityContainerBuilder<Vec<(Duration, DifferentialEvent)>>, DifferentialEvent>>,
    operator_id: usize,
}

Fields§

§chains: Vec<VecDeque<PagedColumn<((K, V), T, R)>>>§lower: Antichain<T>§frontier: Antichain<T>§stash: Vec<Column<((K, V), T, R)>>

Recycled empty Column::Typed chunks. Drained heads and shipped result buffers feed in here; subsequent merge / extract calls pop from here instead of starting from a zero-capacity Column::default(). Mirrors the stash carried by the upstream differential_dataflow merge-batcher framework, which this type forks. Without it, each shipped chunk triggers a fresh per-leaf grow cycle and per-merge-round allocation dominates the inner loop.

§pager_override: Option<ColumnPager>

Optional override. None means “read column_pager::global_pager fresh on every use” — the production path, so worker_config dyncfg changes that re-install the process-global pager take effect on the very next chunk this batcher processes.

§logger: Option<TypedLogger<CapacityContainerBuilder<Vec<(Duration, DifferentialEvent)>>, DifferentialEvent>>§operator_id: usize