Module iter

Source
Expand description

Code for iterating through one or more parts, including streaming consolidation.

StructsΒ§

ConsolidatingIter πŸ”’
Consolidator πŸ”’
A tool for incrementally consolidating a persist shard.
FetchData πŸ”’
The data needed to fetch a batch part, bundled up to make it easy to send between threads.
LowerBound
An owned lower bound type that can be exchanged for a SortKV and a timestamp.
PartIndices πŸ”’
Indices into a part. For most parts, all we need is a single index to the current entry… but for parts that have never been consolidated, this would return entries in the β€œwrong” order, and it’s expensive to re-sort the columnar data. Instead, we sort a list of indices and then use this helper to hand them out in the correct order.
PartRef πŸ”’
This is used as a max-heap entry: the ordering of the fields is important!
StructuredSort
Sort parts ordered by the codec-encoded key and value columns.
StructuredUpdates
An opaque update set for use by StructuredSort.

EnumsΒ§

ConsolidationPart πŸ”’

ConstantsΒ§

MINIMUM_CONSOLIDATED_VERSION
Versions prior to this had bugs in consolidation, or used a different sort. However, we can assume that consolidated parts at this version or higher were consolidated according to the current definition.

TraitsΒ§

RowSort πŸ”’

FunctionsΒ§

interleave_updates πŸ”’
kv_lower πŸ”’
kv_size πŸ”’

Type AliasesΒ§

FetchResult πŸ”’
Indices πŸ”’
A pair of indices, referencing a specific row in a specific part. In the consolidating iterator, this is used to track the coordinates of some part that holds a particular K and V.
SortKV πŸ”’