Expand description
Columnar dataflow edge support.
Defines CollectionEdge, a wrapper that lets dataflow edges between Plan
nodes carry either row-based (VecCollection) or columnar
(ColumnarCollection) batches of (D, T, R) updates.
§Migration model
The migration is consumer-first: every Plan-node consumer learns to accept both variants before any producer emits the columnar variant. Producers can then flip to columnar one at a time.
Within a Plan node, operators may freely materialize Vec collections; only
the inter-node edge format is constrained. A decode from columnar to Vec at
a consumer’s input is acceptable only when the consumer would have decoded
Row to mz_repr::Datum anyway. Pure passthrough consumers (Negate,
Union) round-trip the columnar variant without decoding.
Consumers that have not yet learned the columnar form fall back to
CollectionEdge::into_vec, which decodes through the named
ColumnarToVec operator. Repack seams therefore stay visible in dataflow
introspection, so they can be found and retired.
Enums§
- Collection
Edge - A dataflow edge carrying records as either a row-based
VecCollectionor aColumnarCollection.
Functions§
- columnar_
negate - Negates the diff of every record in a
ColumnarCollection. - columnar_
to_ vec - Decodes columnar batches into a row-based collection.
- vec_
to_ columnar - Repacks a row-based collection into columnar batches.
Type Aliases§
- Columnar
Collection - A columnar collection of
(D, T, R)updates traveling on a compute dataflow edge.