Module mz_dataflow_types::client::controller::compute
source · [−]Expand description
A controller that provides an interface to a compute instance, and the storage layer below it.
The compute controller curates the creation of indexes and sinks, the progress of readers through these collections, and their eventual dropping and resource reclamation.
The compute controller can be viewed as a partial map from GlobalId to collection. It is an error to
use an identifier before it has been “created” with create_dataflows(). Once created, the controller holds
a read capability for each output collection of a dataflow, which is manipulated with allow_compaction().
Eventually, a collection is dropped with either drop_sources() or by allowing compaction to the empty frontier.
Created dataflows will prevent the compaction of their inputs, including other compute collections but also collections managed by the storage layer. Each dataflow input is prevented from compacting beyond the allowed compaction of each of its outputs, ensuring that we can recover each dataflow to its current state in case of failure or other reconfiguration.
Structs
State maintained about individual collections.
An immutable controller for a compute instance.
A mutable controller for a compute instance.
Controller state maintained for each compute instance.
Enums
Errors arising from compute commands.