Module columnar 

Columnar dataflow edge support.

Defines CollectionEdge, the columnar batch representation that dataflow edges between Plan nodes carry. Every producer emits this representation.

Within a Plan node, operators may freely materialize Vec collections. Only the collection edge format is constrained. A node that produces a row-based collection re-encodes it to the columnar edge at its output leaf via vec_to_columnar. A node that must consume rows decodes at its input leaf via columnar_to_vec. Both are named operators (VecToColumnar, ColumnarToVec), so those leaf seams stay visible in dataflow introspection.

Functions

columnar_consolidate

Consolidates a ColumnarCollection natively, without a row round-trip.

columnar_negate

Negates the diff of every record in a ColumnarCollection.

columnar_to_vec

Decodes columnar batches into a row-based collection.

concat_many

Concatenates a collection of columnar edges.

flat_map_datums

Applies logic to each record in edge, exposing the record as a borrowed DatumVecBorrow and giving it ok and err output sessions.

negate_column 🔒

Negates the diff of every record in column, rebuilding only the diff column.

vec_to_columnar

Repacks a row-based collection into columnar batches.

Type Aliases

CollectionEdge

A dataflow edge between Plan nodes: a columnar collection of (Row, Diff) updates.

ColumnarCollection

A columnar collection of (D, T, R) updates traveling on a compute dataflow edge.