Skip to main content

merge_batches

differential_dataflow::columnar::arrangement::trie_merger

Function merge_batches 

Source 
pub fn merge_batches<U, I1, I2, S>(list1: I1, list2: I2, sink: S)
where
    U: Update,
    U::Time: 'static,
    I1: IntoIterator<Item = UpdatesTyped<U>>,
    I2: IntoIterator<Item = UpdatesTyped<U>>,
    S: FnMut(UpdatesTyped<U>),
Expand description

A merge implementation that operates batch-at-a-time.

Inputs are taken as IntoIterator so the caller can stream chunks in lazily — e.g. fetching paged-out chunks one group at a time — rather than materializing entire chains up front. Output chunks are emitted via the caller-supplied sink as they become stable, so the caller can apply a spill policy mid-merge rather than buffering the full result.