fn build_bucketed_stage<G>(
    debug_name: &str,
    input: Collection<G, ((Row, u64), Vec<Row>), Diff>,
    aggrs: Vec<AggregateFunc>,
    buckets: u64,
    validating: bool
) -> (Collection<G, ((Row, u64), Vec<Row>), Diff>, Option<Collection<G, DataflowError, Diff>>)where
    G: Scope,
    G::Timestamp: Lattice,
Expand description

Build the dataflow for one stage of a reduction tree for multiple hierarchical aggregates.

buckets indicates the number of buckets in this stage. We do some non obvious trickery here to limit the memory usage per layer by internally holding only the elements that were rejected by this stage. However, the output collection maintains the ((key, bucket), (passing value) for this stage. validating indicates whether we want this stage to perform error detection for invalid accumulations. Once a stage is clean of such errors, subsequent stages can skip validation.