Enum mz_compute_client::command::ComputeCommand

pub enum ComputeCommand<T = Timestamp> {
    CreateTimely {
        config: TimelyConfig,
        epoch: ComputeStartupEpoch,
    },
    CreateInstance(LoggingConfig),
    InitializationComplete,
    UpdateConfiguration(BTreeSet<ComputeParameter>),
    CreateDataflows(Vec<DataflowDescription<Plan<T>, CollectionMetadata, T>>),
    AllowCompaction(Vec<(GlobalId, Antichain<T>)>),
    Peek(Peek<T>),
    CancelPeeks {
        uuids: BTreeSet<Uuid>,
    },
}

Commands related to the computation and maintenance of views.

A replica can consist of multiple clusterd processes. Upon startup, a clusterd will listen for a connection from environmentd. The first command sent to clusterd must be a CreateTimely command, which will build the timely runtime.

CreateTimely is the only command that is sent to every process of the replica by environmentd. The other commands are sent only to the first process, which in turn will disseminate the command to other timely workers using the timely communication fabric.

After a timely runtime has been built with CreateTimely, a sequence of commands that have to be handled in the timely runtime can be sent: First a CreateInstance must be sent which activates logging sources. After this, any combination of UpdateConfiguration, CreateDataflows, AllowCompaction, Peek, and CancelPeeks can be sent.

Within this sequence, exactly one InitializationComplete has to be sent. Commands sent before InitializationComplete are buffered and are compacted. For example a Peek followed by a CancelPeek will become a no-op if sent before InitializationComplete. After InitializationComplete, the clusterd is considered rehydrated and will immediately act upon the commands. If a new cluster is created, InitializationComplete will follow immediately after CreateInstance. If a replica is added to a cluster or environmentd restarts and rehydrates a clusterd, a potentially long command sequence will be sent before InitializationComplete.

Variants

CreateTimely

Fields

config: TimelyConfig

epoch: ComputeStartupEpoch

Create the timely runtime according to the supplied CommunicationConfig. Must be the first command sent to a clusterd. This is the only command that is broadcasted to all clusterd processes within a replica.

CreateInstance(LoggingConfig)

Setup and logging sources within a running timely instance. Must be the second command after CreateTimely.

InitializationComplete

Indicates that the controller has sent all commands reflecting its initial state.

UpdateConfiguration(BTreeSet<ComputeParameter>)

Update compute instance configuration.

CreateDataflows(Vec<DataflowDescription<Plan<T>, CollectionMetadata, T>>)

Create a sequence of dataflows.

Each of the dataflows must contain as_of members that are valid for each of the referenced arrangements, meaning AllowCompaction should be held back to those values until the command. Subsequent commands may arbitrarily compact the arrangements; the dataflow runners are responsible for ensuring that they can correctly maintain the dataflows.

AllowCompaction(Vec<(GlobalId, Antichain<T>)>)

Enable compaction in compute-managed collections.

Each entry in the vector names a collection and provides a frontier after which accumulations must be correct. The workers gain the liberty of compacting the corresponding maintained traces up through that frontier.

Peek(Peek<T>)

Peek at an arrangement.

CancelPeeks

Fields

uuids: BTreeSet<Uuid>

The identifiers of the peek requests to cancel.

Cancel the peeks associated with the given uuids.

Trait Implementations

impl Arbitrary for ComputeCommand<Timestamp>

type Strategy = Union<BoxedStrategy<ComputeCommand<Timestamp>>>

The type of Strategy used to generate values of type Self.

type Parameters = ()

The type of parameters that arbitrary_with accepts for configuration of the generated Strategy. Parameters must implement Default.

fn arbitrary_with(_: Self::Parameters) -> Self::Strategy

Generates a Strategy for producing arbitrary values of type the implementing type (Self). The strategy is passed the arguments given in args.

fn arbitrary() -> Self::Strategy

Generates a Strategy for producing arbitrary values of type the implementing type (Self).

impl<T: Clone> Clone for ComputeCommand<T>

fn clone(&self) -> ComputeCommand<T>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<T: Debug> Debug for ComputeCommand<T>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de, T> Deserialize<'de> for ComputeCommand<T>where
    T: Deserialize<'de>,

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
    __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl<T: Send> GenericClient<ComputeCommand<T>, ComputeResponse<T>> for Box<dyn ComputeClient<T>>

fn send<'life0, 'async_trait>(
    &'life0 mut self,
    cmd: ComputeCommand<T>
) -> Pin<Box<dyn Future<Output = Result<(), Error>> + Send + 'async_trait>>where
    Self: 'async_trait,
    'life0: 'async_trait,

Sends a command to the dataflow server.

fn recv<'life0, 'async_trait>(
    &'life0 mut self
) -> Pin<Box<dyn Future<Output = Result<Option<ComputeResponse<T>>, Error>> + Send + 'async_trait>>where
    Self: 'async_trait,
    'life0: 'async_trait,

Receives the next response from the dataflow server.

fn as_stream<'a>(
    &'a mut self
) -> Pin<Box<dyn Stream<Item = Result<R, Error>> + Send + 'a, Global>>where
    R: 'a + Send,

Returns an adapter that treats the client as a stream.

impl<T: PartialEq> PartialEq<ComputeCommand<T>> for ComputeCommand<T>

fn eq(&self, other: &ComputeCommand<T>) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl<T> Partitionable<ComputeCommand<T>, ComputeResponse<T>> for (ComputeCommand<T>, ComputeResponse<T>)where
    T: Timestamp + Lattice,

type PartitionedState = PartitionedComputeState<T>

The type which functions as the state machine for the partitioning.

fn new(parts: usize) -> PartitionedComputeState<T>

Construct a PartitionedState for the command–response pair.

impl<T> PartitionedState<ComputeCommand<T>, ComputeResponse<T>> for PartitionedComputeState<T>where
    T: Timestamp + Lattice,

fn split_command(
    &mut self,
    command: ComputeCommand<T>
) -> Vec<Option<ComputeCommand<T>>>

Splits a command into multiple partitions.

fn absorb_response(
    &mut self,
    shard_id: usize,
    message: ComputeResponse<T>
) -> Option<Result<ComputeResponse<T>, Error>>

Absorbs a response from a single partition.

impl RustType<ProtoComputeCommand> for ComputeCommand<Timestamp>

fn into_proto(&self) -> ProtoComputeCommand

Convert a Self into a Proto value.

fn from_proto(proto: ProtoComputeCommand) -> Result<Self, TryFromProtoError>

Consume and convert a Proto back into a Self value.

impl<T> Serialize for ComputeCommand<T>where
    T: Serialize,

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>where
    __S: Serializer,

Serialize this value into the given Serde serializer.

impl<T> StructuralPartialEq for ComputeCommand<T>