Struct CollectionState

struct CollectionState<T: ComputeControllerTimestamp> {
    log_collection: bool,
    dropped: bool,
    scheduled: bool,
    shared: SharedCollectionState<T>,
    implied_read_hold: ReadHold<T>,
    warmup_read_hold: ReadHold<T>,
    read_policy: Option<ReadPolicy<T>>,
    storage_dependencies: BTreeMap<GlobalId, ReadHold<T>>,
    compute_dependencies: BTreeMap<GlobalId, ReadHold<T>>,
    introspection: CollectionIntrospection<T>,
    wallclock_lag_histogram_stash: Option<BTreeMap<(WallclockLagHistogramPeriod, WallclockLag, BTreeMap<&'static str, String>), Diff>>,
}

State maintained about individual compute collections.

A compute collection is either an index, or a storage sink, or a subscribe, exported by a compute dataflow.

Fields

log_collection: bool

Whether this collection is a log collection.

Log collections are special in that they are only maintained by a subset of all replicas.

dropped: bool

Whether this collection has been dropped by a controller client.

The controller is allowed to remove the CollectionState for a collection only when dropped == true. Otherwise, clients might still expect to be able to query information about this collection.

scheduled: bool

Whether this collection has been scheduled, i.e., the controller has sent a Schedule command for it.

shared: SharedCollectionState<T>

State shared with the ComputeController.

implied_read_hold: ReadHold<T>

A read hold maintaining the implicit capability of the collection.

This capability is kept to ensure that the collection remains readable according to its read_policy. It also ensures that read holds on the collection’s dependencies are kept at some time not greater than the collection’s write_frontier, guaranteeing that the collection’s next outputs can always be computed without skipping times.

warmup_read_hold: ReadHold<T>

A read hold held to enable dataflow warmup.

Dataflow warmup is an optimization that allows dataflows to immediately start hydrating even when their next output time (as implied by the write_frontier) is in the future. By installing a read capability derived from the write frontiers of the collection’s inputs, we ensure that the as-of of new dataflows installed for the collection is at a time that is immediately available, so hydration can begin immediately too.

read_policy: Option<ReadPolicy<T>>

The policy to use to downgrade self.implied_read_hold.

If None, the collection is a write-only collection (i.e. a sink). For write-only collections, the implied_read_hold is only required for maintaining read holds on the inputs, so we can immediately downgrade it to the write_frontier.

storage_dependencies: BTreeMap<GlobalId, ReadHold<T>>

Storage identifiers on which this collection depends, and read holds this collection requires on them.

compute_dependencies: BTreeMap<GlobalId, ReadHold<T>>

Compute identifiers on which this collection depends, and read holds this collection requires on them.

introspection: CollectionIntrospection<T>

Introspection state associated with this collection.

wallclock_lag_histogram_stash: Option<BTreeMap<(WallclockLagHistogramPeriod, WallclockLag, BTreeMap<&'static str, String>), Diff>>

Frontier wallclock lag measurements stashed until the next WallclockLagHistogram introspection update.

Keys are (period, lag, labels) triples, values are counts.

If this is None, wallclock lag is not tracked for this collection.

Implementations

impl<T: ComputeControllerTimestamp> CollectionState<T>

fn new( collection_id: GlobalId, as_of: Antichain<T>, shared: SharedCollectionState<T>, storage_dependencies: BTreeMap<GlobalId, ReadHold<T>>, compute_dependencies: BTreeMap<GlobalId, ReadHold<T>>, read_hold_tx: ChangeTx<T>, introspection: CollectionIntrospection<T>, ) -> Self

Creates a new collection state, with an initial read policy valid from since.

fn new_log_collection( id: GlobalId, shared: SharedCollectionState<T>, read_hold_tx: ChangeTx<T>, introspection_tx: UnboundedSender<(IntrospectionType, Vec<(Row, Diff)>)>, ) -> Self

Creates a new collection state for a log collection.

fn read_frontier(&self) -> Antichain<T>

Reports the current read frontier.

fn write_frontier(&self) -> Antichain<T>

Reports the current write frontier.

fn storage_dependency_ids(&self) -> impl Iterator<Item = GlobalId> + '_

fn compute_dependency_ids(&self) -> impl Iterator<Item = GlobalId> + '_

fn dependency_ids(&self) -> impl Iterator<Item = GlobalId> + '_

Reports the IDs of the dependencies of this collection.

Trait Implementations

impl<T: Debug + ComputeControllerTimestamp> Debug for CollectionState<T>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.