Module mz_persist_types::arrow

A protobuf representation of Apache Arrow arrays.

Motivation

Persist can store a small amount of data inline at the consensus layer. Because we are space constrained, we take particular care to store only the data that is necessary. Other Arrow serialization formats, e.g. Parquet or Arrow IPC, include data that we don’t need and would be wasteful to store.

Modules

• proto 🔒

Structs

• ArrayIdx
  A struct representing a particular entry in a particular array. Most useful for its Ord implementation, which can compare entire rows across similarly-typed arrays.
• ProtoArrayData

Enums

• ArrayOrd
  Wraps a single arrow array, downcasted to a specific type.