Expand description
Writing MV sink batches through the process-global SharedBatches.
The workers running in one process coalesce their updates for a given batch interval into a
single batch instead of each writing its own small batch. All workers building an interval
share desc.shared_id, because the mint operator broadcasts one description.
One task per process encodes and uploads a shared batch, so routing large intervals through it serializes work the workers otherwise do in parallel. Each worker therefore stages its updates and hands them to the shared batch only if they stay below a size threshold. Past the threshold, it writes its own batch.
Structs§
- Shared
Batch 🔒Write - One worker’s write for one interval, which contributes to the shared batch until the worker’s updates reach the size threshold and writes the worker’s own batch after. With sharing disabled, all updates go to the worker’s own batch.
- Shared
Batch 🔒Writer - A sink’s handle on the process-global
SharedBatchesfor its output shard.