Expand description
Merge-batcher for Column chunks with per-chunk paging.
Forks the differential_dataflow merge-batcher framework so chains can
hold PagedColumn entries — letting the ColumnPager page chunks
out as they’re produced and fetch them back lazily during merge / extract.
Reuses the resident building blocks from super::batcher: the inherent
Column::merge_from / Column::extract methods (per-chunk merge / split).
Input consolidation happens upstream: the chunker
(super::batcher::ColumnChunker) is supplied to the arrange operator
separately, so this batcher receives already-consolidated Column chunks
via PushInto.
Structs§
- Column
Merge Batcher - Drives the merge-batcher over
Columnchunks routed through aColumnPager. - Fetch
Iter - Streaming materializer over a chain of
PagedColumnentries.
Functions§
- extract_
chain - Streaming extract: walks
mergedchunk-by-chunk viaColumn::extract, routing each filled keep/ship chunk through its sink after pageing. Mirrors the per-chunk ship-threshold yield already insideColumn::extract. - merge_
chains - Two-way merge driver. Reuses today’s per-chunk gallop / ship-threshold
logic from
Column::merge_from, but pulls heads fromFetchIterand emits finished output chunks throughsinkafter routing them through the pager exposed byFetchIter::pager.