Module column_pager 

Column-aware pager. Pages Column instances out via mz_ore::pager, optionally compressing with lz4.

The pager (mz_ore::pager) deals in Vec<u64> blobs and two backends. This module adds:

1. A PagingPolicy trait that decides whether to page out, which backend, and whether to compress. Decisions live in the policy implementation, not in the global atomic the pager exposes.
2. A ColumnPager that drains a Column<C> into a PagedColumn and rehydrates it on demand.
3. Lz4 frame-format compression as an optional codec.

The serialization uses the existing ContainerBytes protocol on Column<C>, so we get a single byte layout that both raw and compressed paths share. See doc/developer/design/20260504_pager.md for background.

Modules

metrics

Prometheus metrics for the column pager.

policy

Concrete PagingPolicy implementations.

Structs

AlwaysResidentPolicy 🔒

Policy that keeps every column resident and discards events. Backs ColumnPager::disabled.

ColumnPager

Pages typed Columns out and back in, driven by a PagingPolicy.

Meta

Sizing metadata captured at pageout time. Stored alongside the payload so take can size buffers.

PageHint

Inputs to a pageout decision.

ResidentTicket

Drop guard that returns budget to a PagingPolicy when a PagedColumn::Resident is destroyed.

Enums

Codec

Compression codec applied to a paged-out column.

CompressedInner

Storage location for the lz4-framed bytes inside a compressed paged column.

PageDecision

Outcome of a policy decision.

PageEvent

Notifications the column-pager sends back to the policy. Implementations typically forward to metrics counters.

PagedColumn

A column whose body may be resident, paged out, or paged out and compressed.

Statics

GLOBAL_PAGER 🔒

Process-global active pager. Defaults to ColumnPager::disabled until worker init calls set_global_pager.

TIERED_POLICY 🔒

Process-wide policy::TieredPolicy singleton.

Traits

PagingPolicy

Decides whether/how to page a column out, and records page events.

Functions

apply_tiered_config

Apply a tiered-pager configuration. Reuses the singleton policy::TieredPolicy so in-flight ResidentTickets remain coherent with the running budget after the operator tunes any of the inputs.

global_pager

Returns the current global pager. Cheap: clones the inner Arc<dyn PagingPolicy>.

pad_u8_to_u64 🔒

Reinterprets bytes as a Vec<u64> by trailing-zero padding to a multiple of 8 and copying. The lz4 frame trailer self-delimits so the trailing pad is invisible to FrameDecoder.

set_global_pager

Install pager as the process-wide active pager. Subsequent global_pager calls return a clone of this value across all threads.

tiered_policy

Returns a reference to the process-wide policy::TieredPolicy singleton.