Module file 

Available on Unix and crate feature pool only.

The file-backed extent store: a userspace extent allocator over one anonymous file per extent class in a scratch directory.

Files. Each class of the ladder the caller passes to FileStore::open gets one file, opened O_TMPFILE so it has no name and the kernel frees its blocks when the process exits, crash included. Where the filesystem refuses O_TMPFILE, the store creates a uniquely named file and unlinks it immediately. A slot (class, index) lives at byte offset index * class_size of its class’s file.

Block ownership. Each class allocates slot indices with a SlotAllocator behind its own mutex. Its warm side holds free slots whose blocks are still allocated, its cold side free slots whose blocks were punched, and slots above its high-water mark were never touched. Freeing a slot pushes it warm with no filesystem call, so slot reuse performs no metadata operation. A slot taken cold or from the high-water mark is fallocated before it is handed out, which turns ENOSPC into an allocation failure before any data is in flight.

Capacity. Every slot with allocated blocks, in use or warm, is charged at its class size against the store’s capacity. When an allocation does not fit, the store punches holes over warm slots of the other class holding the most warm bytes, moving them cold, until it fits. Independently, FileStore::release_warm punches warm slots on its caller’s schedule, so the bytes the store holds follow its live data and space freed in one class is available to any other. A slot it punches pays a fallocate when next allocated.

Alignment. Class sizes are page multiples, and every transfer covers whole pages from a page-aligned buffer, which satisfies O_DIRECT on devices with 512-byte and 4 KiB logical blocks. The store owns that rounding: FileStore::write and FileStore::read return the length they transferred. open probes one page-aligned write and read. A probe failing with EINVAL means the device needs a larger alignment or the filesystem lacks direct I/O, and the store reopens its files buffered: each write is then followed by a synchronous writeback and a page-cache drop over its range. Memory-backed filesystems are rejected by type, since tmpfs accepts the probe and would hold extents in RAM.

Errors. Running out of capacity, or ENOSPC from fallocate, fails the allocation with AllocError::Full. An ENOSPC from a write fails it with WriteError::Full after the store punches and takes back the slot. A short direct write is retried whole first, and its final error is classified the same way. Either ENOSPC additionally lowers the capacity to the bytes allocated at that moment. The volume can be shared with other writers, so FileStore::refresh_capacity re-derives the capacity from the volume’s free space, and room another writer frees becomes usable again. Any other write or fallocate error disables writes for the store’s lifetime. Data already written stays readable. A read error, short read, or checksum mismatch panics: there is no correct value to return, and the pool’s contents are recreatable.

Modules

sys 🔒

The platform seam: every filesystem syscall the store makes.

Structs

AlignedBuf 🔒

Page-aligned, growable byte buffer for O_DIRECT reads.

Class 🔒

One extent class: its file and slot allocator.

Counters 🔒

FileSlot 🔒

A slot in the store: index in the file of extent class class.

FileStore 🔒

The file-backed extent store. See the module documentation.

FileStoreStats 🔒

A snapshot of the store’s counters.

Enums

AllocError 🔒

Why an allocation failed.

IoMode 🔒

Where the store’s I/O goes.

WriteError 🔒

Why a write failed.

Constants

DIRECT_WRITE_ATTEMPTS 🔒

Attempts a direct write makes at a slot before a short write counts as failed.

READ_LATENCY_BUCKETS

Number of read-latency buckets. Bucket i < READ_LATENCY_BUCKETS - 1 counts reads that took less than 32 µs << i and, for i > 0, at least 16 µs << i. The last bucket counts reads of 65.536 ms and more.

Functions

crc 🔒

The checksum the store verifies on every read.

derived_capacity 🔒

The capacity of a new store on a volume with available bytes free, per usable_bytes. None if that cannot hold one slot of largest_class bytes, the largest extent class.

open_files 🔒

Opens classes anonymous files for io_mode and probes the first. In IoMode::Direct, an EINVAL from the open or the probe means direct I/O is unusable here.

probe 🔒

Allocates, writes, reads back, and punches one page at offset 0.

read_latency_bucket 🔒

The bucket of FileStoreStats::read_latency_buckets for a read that took micros microseconds.

usable_bytes 🔒

The bytes a store may allocate on a volume with available bytes free to unprivileged writers: those bytes minus the larger of 1 GiB and 2% of them.

write_all 🔒

Writes all of buf at offset, retrying on EINTR and short writes. In IoMode::Direct, a short write restarts the whole buffer at offset, at most DIRECT_WRITE_ATTEMPTS times, and the final attempt’s error is returned.