Module iter

Code for iterating through one or more parts, including streaming consolidation.

Structs

ConsolidatingIter 🔒

Consolidator 🔒

A tool for incrementally consolidating a persist shard.

FetchData 🔒

The data needed to fetch a batch part, bundled up to make it easy to send between threads.

PartIndices 🔒

Indices into a part. For most parts, all we need is a single index to the current entry… but for parts that have never been consolidated, this would return entries in the “wrong” order, and it’s expensive to re-sort the columnar data. Instead, we sort a list of indices and then use this helper to hand them out in the correct order.

PartRef 🔒

This is used as a max-heap entry: the ordering of the fields is important!

StructuredSort

Sort parts ordered by the codec-encoded key and value columns.

StructuredUpdates

An opaque update set for use by StructuredSort.

Enums

ConsolidationPart 🔒

Constants

MINIMUM_CONSOLIDATED_VERSION

Versions prior to this had bugs in consolidation, or used a different sort. However, we can assume that consolidated parts at this version or higher were consolidated according to the current definition.

Traits

RowSort 🔒

Functions

interleave_updates 🔒

kv_lower 🔒

kv_size 🔒

Type Aliases

FetchResult 🔒

Indices 🔒

A pair of indices, referencing a specific row in a specific part. In the consolidating iterator, this is used to track the coordinates of some part that holds a particular K and V.

SortKV 🔒