Module materialized_view_v2 

Sync Timely operator implementation of the MV sink.

This module provides an alternative implementation of persist_sink that uses sync Timely operators communicating with Tokio tasks via channels, instead of async Timely operators. Gated behind the ENABLE_SYNC_MV_SINK dyncfg.

See the main module for the operator graph and design docs.

Channel ordering requirements

Each operator splits state across a Timely thread (which observes inputs and frontiers) and a Tokio task (which owns persist I/O state). They communicate via mpsc command channels, which preserve send order on a single sender. Each operator instance constructs its own (tx, rx) pair inside render and never clones the sender, so there is exactly one producer per channel — sends are totally ordered. Different worker instances of the same operator never share a channel, so cross-worker ordering is not a concern. The correctness of the operators relies on a few ordering invariants between the messages sent within a single Timely activation:

• mint: the persist_watch Tokio task is the sole producer of persist-frontier updates and emits them in monotonically increasing order, terminated by the empty frontier. The Timely closure drains the receiver each activation; processing in receive order is therefore sufficient. No cross-channel ordering is needed because mint only has the one channel.

• write: per-activation, the Timely closure first appends all observed input data into a single WriteCommand::Batch and only then sends a WriteCommand::WriteBatch (issued from maybe_start_batch after frontier checks). The Tokio task processes commands FIFO, so a WriteBatch is guaranteed to see every Batch from the same activation already applied to the corrections buffer. Reversing this order would let the task write a batch that is missing updates the Timely closure already observed.

• append: per-activation, the Timely closure forwards messages in the order Description → Batch → BatchesFrontier. The first two carry the data the task needs to absorb; BatchesFrontier is the trigger that allows maybe_append_batches to fire. Sending BatchesFrontier after its corresponding Batch messages ensures the task does not append a batch description before all batches contributing to it have been absorbed. If the order were reversed, maybe_append_batches could fire on an incomplete batches set and miss writes.

Modules

append 🔒

Implementation of the append operator.

mint 🔒

Implementation of the mint operator.

write 🔒

Implementation of the write operator.

Functions

persist_sink 🔒

Renders an MV sink writing the given desired collection into the target persist collection.