Module fetch 

Fetching batches of data from persist’s backing store

Structs

BatchFetcher

Capable of fetching LeasedBatchPart while not holding any capabilities.

BatchFetcherConfig 🔒

DecodedRows 🔒

Decoded rows of a FetchedPart and the cursor into them.

EncodedPart 🔒

A Blob object that has been fetched, but has no associated decoding logic.

EncodedPartBatches 🔒

A hollow part decoded into normalized updates in batches of at most batch_rows rows.

ExchangeableBatchPart

This represents the serde encoding for LeasedBatchPart. We expose the struct itself (unlike other encodable structs) to attempt to provide stricter drop semantics on LeasedBatchPart, i.e. SerdeLeasedBatchPart is exchangeable (including over the network), where LeasedBatchPart is not.

FetchConfig 🔒

FetchedBlob

A Blob object that has been fetched, but not at all decoded.

FetchedPart

A Blob object that has been fetched, but not yet fully decoded.

Lease

This represents the lease of a seqno. It’s generally paired with some external state, like a hollow part: holding this lease indicates that we may still want to fetch that part, and should hold back GC to keep it around.

LeasedBatchPart

A token representing one fetch-able batch part.

PartNormalization 🔒

The normalization of a fetched part’s updates to its registered description: truncation to the registered bounds and ts_rewrite.

ShardSourcePart

FetchedPart but with an accompanying permit from the fetch mem/disk semaphore.

Enums

FetchBatchFilter 🔒

FetchedBlobBuf 🔒

MissingBlobDiagnosis

The result of BatchFetcher::missing_blob_diagnostics.

PartDecodeFormat

Format we’ll use when decoding a Part.

PartSource 🔒

The updates FetchedPart reads: a part decoded whole, or one decoded in bounded batches on demand.

Constants

FETCH_SEMAPHORE_COST_ADJUSTMENT 🔒

FETCH_SEMAPHORE_PERMIT_ADJUSTMENT 🔒

MISSING_BLOB_DIAGNOSIS_TIMEOUT 🔒

How long missing_blob_diagnostics waits to refresh state.

OPTIMIZE_IGNORED_DATA_FETCH 🔒

PART_DECODE_BATCH_ROWS 🔒

PART_DECODE_FORMAT 🔒

VALIDATE_PART_BOUNDS_ON_READ 🔒

Functions

decode_batch_part_blob 🔒

fetch_batch_part 🔒

fetch_batch_part_blob 🔒

fetch_leased_part 🔒

Trade in an exchange-able LeasedBatchPart for the data it represents.

missing_blob_diagnostics 🔒

Diagnoses a missing-blob fetch failure: refreshes the shard state and reports whether the reader that leased the part is still present in it.