Struct dataflow::arrangement::manager::TraceManager

pub struct TraceManager {
    pub traces: HashMap<GlobalId, TraceBundle>,
    worker_id: usize,
    maintenance_metrics: HashMap<GlobalId, MaintenanceMetrics>,
    doing_maintenance: DeleteOnDropGauge<'static, AtomicU64, Vec<String>>,
    metrics: TraceMetrics,
}

A TraceManager stores maps from global identifiers to the primary arranged representation of that collection.

Fields

traces: HashMap<GlobalId, TraceBundle>

worker_id: usize

maintenance_metrics: HashMap<GlobalId, MaintenanceMetrics>

doing_maintenance: DeleteOnDropGauge<'static, AtomicU64, Vec<String>>

1 if this worker is currently doing maintenance.

If maintenance turns out to take a very long time, this will allow us to gain a sense that materialize is stuck on maintenance before the maintenance completes

metrics: TraceMetrics

Implementations

impl TraceManager

pub fn new(metrics: TraceMetrics, worker_id: usize) -> Self

pub fn maintenance(&mut self)

performs maintenance work on the managed traces.

In particular, this method enables the physical merging of batches, so that at most a logarithmic number of batches need to be maintained. Any new batches introduced after this method is called will not be physically merged until the method is called again. This is mostly due to limitations of differential dataflow, which requires users to perform this explicitly; if that changes we may be able to remove this code.

pub fn allow_compaction(
    &mut self,
    id: GlobalId,
    frontier: AntichainRef<'_, Timestamp>
)

Enables compaction of traces associated with the identifier.

Compaction may not occur immediately, but once this method is called the associated traces may not accumulate to the correct quantities for times not in advance of frontier. Users should take care to only rely on accumulations at times in advance of frontier.

pub fn get(&self, id: &GlobalId) -> Option<&TraceBundle>

Returns a reference to the trace for id, should it exist.

pub fn get_mut(&mut self, id: &GlobalId) -> Option<&mut TraceBundle>

Returns a mutable reference to the trace for id, should it exist.

pub fn set(&mut self, id: GlobalId, trace: TraceBundle)

Binds the arrangement for id to trace.

pub fn del_trace(&mut self, id: &GlobalId) -> bool

Removes the trace for id.

pub fn del_all_traces(&mut self)

Removes all managed traces.