Module columnar 

Columnar dataflow edge support.

Defines ColCollection, the columnar collection that dataflow edges between Plan nodes carry. Every producer emits this representation.

Within a Plan node, operators may freely materialize Vec collections. Only the collection edge format is constrained. A node that produces a row-based collection re-encodes it to the columnar edge at its output leaf via vec_to_columnar. A node that must consume rows decodes at its input leaf via columnar_to_vec. Both are named operators (VecToColumnar, ColumnarToVec), so those leaf seams stay visible in dataflow introspection.

Functions

columnar_consolidate

Consolidates a ColumnarCollection natively, without a row round-trip.

columnar_leave_dynamic

Leaves a dynamically created scope that has level iteration coordinates.

columnar_negate

Negates the diff of every record in a ColumnarCollection.

columnar_to_vec

Decodes columnar batches into a row-based collection.

concat_many

Concatenates a collection of columnar edges.

flat_map_datums

Applies logic to each record in edge, exposing the record as a borrowed DatumVecBorrow and giving it ok and err output sessions.

negate_column 🔒

Negates the diff of every record in column, rebuilding only the diff column.

truncate_times 🔒

Truncates every time in column to level - 1 iteration coordinates.

vec_to_columnar

Repacks a row-based collection into columnar batches.

Type Aliases

ColCollection

A columnar collection of (Row, Diff) updates, which is what a dataflow edge between Plan nodes carries.

ColumnarCollection

A columnar collection of (D, T, R) updates traveling on a compute dataflow edge.

RecTimestamp

The timestamp of a scope that carries iteration coordinates.