Module mz_persist::indexed::columnar

A columnar representation of ((Key, Val), Time, i64) data suitable for in-memory reads and persistent storage.

Modules

• arrow
  Apache Arrow encodings and utils for persist data
• parquet
  Apache Parquet encodings and utils for persist data

Structs

• ColumnarRecords
  A set of ((Key, Val), Time, Diff) records stored in a columnar representation.
• ColumnarRecordsBuilder
  An abstraction to incrementally add ((Key, Value), Time, i64) records in a columnar representation, and eventually get back a ColumnarRecords.
• ColumnarRecordsIter
  An Iterator over the records in a ColumnarRecords.
• ColumnarRecordsRef 🔒
  A reference to a ColumnarRecords.
• ColumnarRecordsStructuredExt
  An “extension” to ColumnarRecords that duplicates the “key” (K) and “val” (V) columns as structured Arrow data.

Constants

• BYTES_PER_KEY_VAL_OFFSET 🔒
• KEY_VAL_DATA_MAX_LEN
  The maximum allowed amount of total key data (similarly val data) in a single ColumnarBatch.