Struct Chunk 

struct Chunk<D: Data> {
    pooled: Mutex<Option<ChunkHandle>>,
    resident: OnceLock<ColumnBody<(D, Timestamp, Diff)>>,
    len: usize,
    body_words: usize,
    first_time: Timestamp,
    last_time: Timestamp,
    depth: u8,
}

A non-empty chunk of updates, backed by a columnar region.

All updates in a chunk are sorted by (time, data) and consolidated.

Chunks are immutable once created. They are produced by ChunkBuilder, which mints a new chunk whenever its in-progress columnar container reaches a fixed serialized byte boundary (~2 MiB, matching the ship granularity used elsewhere in the codebase), so each chunk corresponds to a single, predictably sized allocation.

Fields

pooled: Mutex<Option<ChunkHandle>>

The body in the buffer pool, which may spill it.

Empty when the body stays on the heap, which is what mz_timely_util::columnar::chunk::try_spill_ref decides, and emptied again by Chunk::body: a materialized chunk holds its body on the heap for the rest of its life, so keeping the pool’s copy alive would only double the footprint and hold budget the pool could give to a chunk that still needs it. Chunk::with_view reads a copy and leaves the body here.

A Mutex (not RefCell) keeps the chunk Sync: cursors hold chunks behind a shared Rc, and the iterator returned by CorrectionV2::updates_before borrows them across the persist writer’s await, so &Chunk must be Send. The lock is taken at materialization and for scoped reads, and is otherwise uncontended (the sink runs single-threaded per worker).

resident: OnceLock<ColumnBody<(D, Timestamp, Diff)>>

The materialized form, populated lazily by Chunk::body on first access.

An OnceLock for the same Sync reason as pooled. Once set the slot is never cleared, so its address is stable and Chunk::index can hand out Ref<'_> borrows tied to &self. The allocation is freed when the chunk drops, which bounds resident memory to the chunks under an active merge front.

len: usize

Number of updates, cached so len and chain bookkeeping never page the chunk in.

body_words: usize

Serialized size of the body in words, cached so size accounting never pages it in.

first_time: Timestamp

Time of the first update, cached so boundary checks (split_at_time, can_accept) route a resting chunk without materializing it.

last_time: Timestamp

Time of the last update, cached likewise.

depth: u8

The generational depth: 0 for chunks built from staged updates or written by a read, one more than the deepest input for chunks written by a bucket’s chain merge. The pool treats deeper chunks as colder, and compresses them past the floor set by chunk::set_compress_min_depth.

Implementations

impl<D: Data> Chunk<D>

fn mint(body: &mut ColumnBody<(D, Timestamp, Diff)>, depth: u8) -> Self

Mint a chunk from the given non-empty body, emptying it.

Reads the metadata a resting chunk must answer (length, size, boundary times) while the body is still in hand, then offers it to the buffer pool. A body the pool takes is encoded straight into its slot, which leaves the typed allocation behind for the caller to refill, and materializes lazily on first read. A body the pool declines is serialized out of the typed allocation instead, because a resident chunk has to own it.

Panics

Panics if the body is empty. Chunks are non-empty by construction; ChunkBuilder only ever mints from a populated body.

fn body(&self) -> &ColumnBody<(D, Timestamp, Diff)>

Materialize the chunk’s body, taking it out of the pool on first access.

The returned reference is valid for as long as &self: the OnceLock slot is never cleared once populated, so its contents have a stable address. Taking the body frees the pool’s copy, so the chunk holds exactly one.

For reads by merges and splits, whose output replaces the chunk. A read after which the chunk survives uses Chunk::with_view, which keeps the body spillable.

fn with_view<R>( &self, f: impl FnOnce(<<(D, Timestamp, Diff) as Columnar>::Container as Borrow>::Borrowed<'_>) -> R, ) -> R

Call f with a view of the chunk’s body, leaving the body where it is.

A materialized chunk is read in place. Otherwise the body is copied out of the pool for the call and the copy dropped after it, so the pool keeps its slot and may still spill the body.

fn len(&self) -> usize

Return the number of updates in the chunk.

fn view( &self, ) -> <<(D, Timestamp, Diff) as Columnar>::Container as Borrow>::Borrowed<'_>

Borrow the chunk’s body, paging it in if necessary.

Any caller that touches more than one update must hoist this out of its loop and index the returned view. ColumnBody::borrow on a serialized body rebuilds the struct-of-arrays view from the serialized header on every call, so borrowing per element pays that decode per element.

fn index(&self, idx: usize) -> Ref<'_, (D, Timestamp, Diff)>

Return the update at the given index, paging the chunk in if necessary.

Single-access only. Indexing a hoisted Chunk::view is the loop form.

Panics

Panics if the given index is not populated.

fn first_time(&self) -> Timestamp

Return the time of the first update, without materializing the chunk.

fn last_time(&self) -> Timestamp

Return the time of the last update, without materializing the chunk.

fn find_time_greater_than(&self, time: Timestamp) -> Option<usize>

Return the index of the first update at a time greater than time, or None if no such update exists.

The early-out uses the cached last time, so a chunk whose updates are all at or before time is skipped without paging it in.

fn size(&self) -> usize

Return the serialized size of the chunk’s body in bytes, for use in metrics.

The chunk holds exactly one copy of its body, in the pool or on the heap, so this is its size either way. A scoped read’s copy lives only for the read. It is not a resident-byte number: what the pool has evicted is the pool’s business, and it publishes resident_bytes itself (mz_ore::pool::PoolStats).

Trait Implementations

impl<D: Data> Debug for Chunk<D>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.