Skip to main content

Module extent

Module extent 

Source
Available on Unix and crate feature pool only.
Expand description

Swap-backed extents: the backing store for the buffer pool on nodes whose whole disk is provisioned as swap.

An extent is a slot in the pool-owned ExtentArena holding the stored bytes of one chunk, produced by the chunk’s ExtentCodec (lz4 in practice). “Write” encodes into the slot. The slot stays resident, forming the compressed-but-resident middle tier of the pool’s ladder, until the pool’s RSS target forces SwapExtent::pageout, which pushes the pages to the swap device with MADV_PAGEOUT. “Read” issues MADV_WILLNEED ahead of the decode (and makes the pages resident again); “free” returns the slot to the arena with its pages discarded, which also drops any swapped copy for free. Chunk slots never reach the swap device: only these compressed extents are offered to it.

The arena exists so that extent pages never belong to the global allocator: MADV_PAGEOUT over allocator-owned memory leaves swap-entry PTEs behind on freed ranges, which the allocator recycles into unrelated allocations that then major-fault reading dead compressed data. Arena regions are advised MADV_NOHUGEPAGE once at map time, so the reclaim never needs to split a large folio, and slot recycling never re-touches swap. A class whose region is exhausted degrades to a plain heap allocation (counted, never paged out) rather than failing.

Pageout is observed, never assumed: MADV_PAGEOUT may decline any page and still return success (a kernel-internal pin fails isolation, the swap device may be full or absent), so after the advice the page table decides whether the extent left memory, and the extent stays fully resident for accounting until its entire range is unmapped. The observation reads pagemap present bits rather than mincore, which would count the clean swap-cache copies of successfully reclaimed pages as resident.

Structs§

ExtentArena 🔒
Pool-owned arena of anonymous-memory regions backing extents, one region per entry of the extent_classes ladder. Slots are allocated at write, freed cold (pages and any swap copy discarded) at extent drop, and never kept warm: a freed extent’s compressed bytes are dead by definition.
SwapExtent 🔒
One chunk’s compressed backing copy.

Enums§

Backing 🔒
Where an extent’s bytes live.
Scratch 🔒
Retention policy for the thread-local compression scratch across SwapExtent::write calls.

Constants§

PAGEOUT_RETRY_CAP 🔒
Consecutive incomplete pageout passes after which an extent stops being advised out until a read faults it back in. Transient declines are kernel-internal page pins that defeat the reclaim’s isolation step (a folio sitting in another CPU’s LRU batch, a momentary swap-slot allocation failure); they clear as soon as the pin drains, so a retry or two recovers them. Persistent declines (no swap device, an exhausted or unswappable cgroup) never clear, and further advice is pure page-table walking. Three passes covers the transient causes while bounding the wasted advice at two extra passes per extent.

Functions§

extent_classes 🔒
The extent size-class ladder for a page-byte page size: page, then sizes of the form 2^k and 3 * 2^(k-1) bytes, up through the first class that fits max_stored_len, the codec contract’s worst case, over the largest chunk size class. Every class is a multiple of page (the sub-page mid class between page and 2 * page is skipped), so slot-granular paging advice is exact on any kernel page size. Consecutive classes above the smallest are within 1.5x of each other, which bounds a stored payload’s internal fragmentation below 1.5x (page-granular slack at the small end).
heap_layout 🔒
The heap layout of a fallback extent for a compressed payload of comp_len bytes.
max_chunk_bytes 🔒
The largest chunk payload the pool can ask an extent to back.