Expand description
Code for iterating through one or more parts, including streaming consolidation.
StructsΒ§
- Consolidating
Iter π - Consolidator π
- A tool for incrementally consolidating a persist shard.
- Fetch
Data π - The data needed to fetch a batch part, bundled up to make it easy to send between threads.
- Lower
Bound - An owned lower bound type that can be exchanged for a SortKV and a timestamp.
- Part
Indices π - Indices into a part. For most parts, all we need is a single index to the current entryβ¦ but for parts that have never been consolidated, this would return entries in the βwrongβ order, and itβs expensive to re-sort the columnar data. Instead, we sort a list of indices and then use this helper to hand them out in the correct order.
- PartRef π
- This is used as a max-heap entry: the ordering of the fields is important!
- Structured
Sort - Sort parts ordered by the codec-encoded key and value columns.
- Structured
Updates - An opaque update set for use by StructuredSort.
EnumsΒ§
- Consolidation
Part π
ConstantsΒ§
- MINIMUM_
CONSOLIDATED_ VERSION - Versions prior to this had bugs in consolidation, or used a different sort. However, we can assume that consolidated parts at this version or higher were consolidated according to the current definition.
TraitsΒ§
- RowSort π
FunctionsΒ§
- interleave_
updates π - kv_
lower π - kv_size π
Type AliasesΒ§
- Fetch
Result π - Indices π
- A pair of indices, referencing a specific row in a specific part. In the consolidating iterator, this is used to track the coordinates of some part that holds a particular K and V.
- SortKV π