Struct mz_compute_client::controller::CollectionState

pub struct CollectionState<T: Timestamp> {
    log_collection: bool,
    dropped: bool,
    scheduled: bool,
    read_capabilities: MutableAntichain<T>,
    implied_read_hold: ReadHold<T>,
    warmup_read_hold: ReadHold<T>,
    read_policy: Option<ReadPolicy<T>>,
    storage_dependencies: BTreeMap<GlobalId, ReadHold<T>>,
    compute_dependencies: BTreeMap<GlobalId, ReadHold<T>>,
    write_frontier: Antichain<T>,
    collection_introspection: CollectionIntrospection<T>,
}

State maintained about individual compute collections.

A compute collection is either an index, or a storage sink, or a subscribe, exported by a compute dataflow.

Fields

log_collection: bool

Whether this collection is a log collection.

Log collections are special in that they are only maintained by a subset of all replicas.

dropped: bool

Whether this collection has been dropped by a controller client.

The controller is allowed to remove the CollectionState for a collection only when dropped == true. Otherwise, clients might still expect to be able to query information about this collection.

scheduled: bool

Whether this collection has been scheduled, i.e., the controller has sent a Schedule command for it.

read_capabilities: MutableAntichain<T>

Accumulation of read capabilities for the collection.

This accumulation contains the capabilities held by all ReadHolds given out for the collection, including implied_read_hold and warmup_read_hold.

NOTE: This field may only be modified by Instance::apply_read_hold_changes and Instance::acquire_read_hold. Nobody else should modify read capabilities directly. Instead, collection users should manage read holds through ReadHold objects acquired through Instance::acquire_read_hold.

TODO(teskje): Restructure the code to enforce the above in the type system.

implied_read_hold: ReadHold<T>

A read hold maintaining the implicit capability of the collection.

This capability is kept to ensure that the collection remains readable according to its read_policy. It also ensures that read holds on the collection’s dependencies are kept at some time not greater than the collection’s write_frontier, guaranteeing that the collection’s next outputs can always be computed without skipping times.

warmup_read_hold: ReadHold<T>

A read hold held to enable dataflow warmup.

Dataflow warmup is an optimization that allows dataflows to immediately start hydrating even when their next output time (as implied by the write_frontier) is in the future. By installing a read capability derived from the write frontiers of the collection’s inputs, we ensure that the as-of of new dataflows installed for the collection is at a time that is immediately available, so hydration can begin immediately too.

read_policy: Option<ReadPolicy<T>>

The policy to use to downgrade self.implied_read_hold.

If None, the collection is a write-only collection (i.e. a sink). For write-only collections, the implied_read_hold is only required for maintaining read holds on the inputs, so we can immediately downgrade it to the write_frontier.

storage_dependencies: BTreeMap<GlobalId, ReadHold<T>>

Storage identifiers on which this collection depends, and read holds this collection requires on them.

compute_dependencies: BTreeMap<GlobalId, ReadHold<T>>

Compute identifiers on which this collection depends, and read holds this collection requires on them.

write_frontier: Antichain<T>

The write frontier of this collection.

collection_introspection: CollectionIntrospection<T>

Introspection state associated with this collection.

Implementations

impl<T: Timestamp> CollectionState<T>

pub fn read_capability(&self) -> &Antichain<T>

Reports the current read capability.

pub fn read_frontier(&self) -> AntichainRef<'_, T>

Reports the current read frontier.

pub fn write_frontier(&self) -> AntichainRef<'_, T>

Reports the current write frontier.

fn storage_dependency_ids(&self) -> impl Iterator<Item = GlobalId> + '_

fn compute_dependency_ids(&self) -> impl Iterator<Item = GlobalId> + '_

fn dependency_ids(&self) -> impl Iterator<Item = GlobalId> + '_

Reports the IDs of the dependencies of this collection.

impl<T: ComputeControllerTimestamp> CollectionState<T>

pub(crate) fn new( collection_id: GlobalId, as_of: Antichain<T>, storage_dependencies: BTreeMap<GlobalId, ReadHold<T>>, compute_dependencies: BTreeMap<GlobalId, ReadHold<T>>, read_holds_tx: UnboundedSender<(GlobalId, ChangeBatch<T>)>, introspection_tx: Sender<(IntrospectionType, Vec<(Row, Diff)>)>, initial_as_of: Option<Antichain<T>>, refresh_schedule: Option<RefreshSchedule>, ) -> Self

Creates a new collection state, with an initial read policy valid from since.

pub(crate) fn new_log_collection( id: GlobalId, read_holds_tx: UnboundedSender<(GlobalId, ChangeBatch<T>)>, introspection_tx: Sender<(IntrospectionType, Vec<(Row, Diff)>)>, ) -> Self

Creates a new collection state for a log collection.

Trait Implementations

impl<T: Debug + Timestamp> Debug for CollectionState<T>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.