Struct mz_compute_client::controller::instance::CollectionState

struct CollectionState<T: ComputeControllerTimestamp> {
    log_collection: bool,
    dropped: bool,
    scheduled: bool,
    shared: SharedCollectionState<T>,
    implied_read_hold: ReadHold<T>,
    warmup_read_hold: ReadHold<T>,
    read_policy: Option<ReadPolicy<T>>,
    storage_dependencies: BTreeMap<GlobalId, ReadHold<T>>,
    compute_dependencies: BTreeMap<GlobalId, ReadHold<T>>,
    introspection: CollectionIntrospection<T>,
}

State maintained about individual compute collections.

A compute collection is either an index, or a storage sink, or a subscribe, exported by a compute dataflow.

Fields

log_collection: bool

Whether this collection is a log collection.

Log collections are special in that they are only maintained by a subset of all replicas.

dropped: bool

Whether this collection has been dropped by a controller client.

The controller is allowed to remove the CollectionState for a collection only when dropped == true. Otherwise, clients might still expect to be able to query information about this collection.

scheduled: bool

Whether this collection has been scheduled, i.e., the controller has sent a Schedule command for it.

shared: SharedCollectionState<T>

State shared with the ComputeController.

implied_read_hold: ReadHold<T>

A read hold maintaining the implicit capability of the collection.

This capability is kept to ensure that the collection remains readable according to its read_policy. It also ensures that read holds on the collection’s dependencies are kept at some time not greater than the collection’s write_frontier, guaranteeing that the collection’s next outputs can always be computed without skipping times.

warmup_read_hold: ReadHold<T>

A read hold held to enable dataflow warmup.

Dataflow warmup is an optimization that allows dataflows to immediately start hydrating even when their next output time (as implied by the write_frontier) is in the future. By installing a read capability derived from the write frontiers of the collection’s inputs, we ensure that the as-of of new dataflows installed for the collection is at a time that is immediately available, so hydration can begin immediately too.

read_policy: Option<ReadPolicy<T>>

The policy to use to downgrade self.implied_read_hold.

If None, the collection is a write-only collection (i.e. a sink). For write-only collections, the implied_read_hold is only required for maintaining read holds on the inputs, so we can immediately downgrade it to the write_frontier.

storage_dependencies: BTreeMap<GlobalId, ReadHold<T>>

Storage identifiers on which this collection depends, and read holds this collection requires on them.

compute_dependencies: BTreeMap<GlobalId, ReadHold<T>>

Compute identifiers on which this collection depends, and read holds this collection requires on them.

introspection: CollectionIntrospection<T>

Introspection state associated with this collection.

Implementations

impl<T: ComputeControllerTimestamp> CollectionState<T>

fn new( collection_id: GlobalId, as_of: Antichain<T>, shared: SharedCollectionState<T>, storage_dependencies: BTreeMap<GlobalId, ReadHold<T>>, compute_dependencies: BTreeMap<GlobalId, ReadHold<T>>, read_holds_tx: UnboundedSender<(GlobalId, ChangeBatch<T>)>, introspection: CollectionIntrospection<T>, ) -> Self

Creates a new collection state, with an initial read policy valid from since.

fn new_log_collection( id: GlobalId, shared: SharedCollectionState<T>, read_holds_tx: UnboundedSender<(GlobalId, ChangeBatch<T>)>, introspection_tx: Sender<(IntrospectionType, Vec<(Row, Diff)>)>, ) -> Self

Creates a new collection state for a log collection.

fn read_frontier(&self) -> Antichain<T>

Reports the current read frontier.

fn write_frontier(&self) -> Antichain<T>

Reports the current write frontier.

fn storage_dependency_ids(&self) -> impl Iterator<Item = GlobalId> + '_

fn compute_dependency_ids(&self) -> impl Iterator<Item = GlobalId> + '_

fn dependency_ids(&self) -> impl Iterator<Item = GlobalId> + '_

Reports the IDs of the dependencies of this collection.

Trait Implementations

impl<T: Debug + ComputeControllerTimestamp> Debug for CollectionState<T>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.