Enum ComputeResponse

pub enum ComputeResponse<T = Timestamp> {
    Frontiers(GlobalId, FrontiersResponse<T>),
    PeekResponse(Uuid, PeekResponse, OpenTelemetryContext),
    SubscribeResponse(GlobalId, SubscribeResponse<T>),
    CopyToResponse(GlobalId, CopyToResponse),
    Status(StatusResponse),
}

Compute protocol responses, sent by replicas to the compute controller.

Replicas send ComputeResponses in response to ComputeCommands they previously received from the compute controller.

Variants

Frontiers(GlobalId, FrontiersResponse<T>)

Frontiers announces the advancement of the various frontiers of the specified compute collection.

Replicas must send Frontiers responses for compute collections that are indexes or storage sinks. Replicas must not send Frontiers responses for subscribes and copy-tos (#16274).

Replicas must never report regressing frontiers. Specifically:

• The first frontier of any kind reported for a collection must not be less than that collection’s initial as_of frontier.
• Subsequent reported frontiers for a collection must not be less than any frontier of the same kind reported previously for the same collection.

Replicas must send Frontiers responses that report each frontier kind to have advanced to the empty frontier in response to an AllowCompaction command that allows compaction of the collection to to the empty frontier, unless the frontier has previously advanced to the empty frontier as part of the regular dataflow computation. (#16271)

Once a frontier was reported to have been advanced to the empty frontier, the replica must not send further Frontiers responses with non-None values for that frontier kind.

The replica must not send Frontiers responses for collections that have not been created previously by a CreateDataflow command or by a CreateInstance command.

PeekResponse(Uuid, PeekResponse, OpenTelemetryContext)

PeekResponse reports the result of a previous Peek command. The peek is identified by a Uuid that matches the command’s Peek::uuid.

The replica must send exactly one PeekResponse for every Peek command it received.

If the replica did not receive a CancelPeek command for a peek, it must not send a Canceled response for that peek. If the replica did receive a CancelPeek command for a peek, it may send any of the three PeekResponse variants.

The replica must not send PeekResponses for peek IDs that were not previously specified in a Peek command.

SubscribeResponse(GlobalId, SubscribeResponse<T>)

SubscribeResponse reports the results emitted by an active subscribe over some time interval.

For each subscribe that was installed by a previous CreateDataflow command, the replica must emit Batch responses that cover the entire time interval from the minimum time until the subscribe advances to the empty frontier or is dropped. The time intervals of consecutive Batches must be increasing, contiguous, non-overlapping, and non-empty. All updates transmitted in a batch must be consolidated and have times within that batch’s time interval. All updates’ times must be greater than or equal to as_of. The upper of the first Batch of a subscribe must not be less than that subscribe’s initial as_of frontier.

The replica must send DroppedAt responses if the subscribe was dropped in response to an AllowCompaction command that advanced its read frontier to the empty frontier. The DroppedAt frontier must be the upper frontier of the last emitted batch.

The replica must not send a DroppedAt response if the subscribe’s upper frontier (reported by Batch responses) has advanced to the empty frontier (e.g. because its inputs advanced to the empty frontier).

Once a subscribe was reported to have advanced to the empty frontier, or has been dropped:

• It must no longer read from its inputs.
• The replica must not send further SubscribeResponses for that subscribe.

The replica must not send SubscribeResponses for subscribes that have not been created previously by a CreateDataflow command.

CopyToResponse(GlobalId, CopyToResponse)

CopyToResponse reports the completion of an S3-oneshot sink.

The replica must send exactly one CopyToResponse for every S3-oneshot sink previously created by a CreateDataflow command.

The replica must not send CopyToResponses for S3-oneshot sinks that were not previously created by a CreateDataflow command.

Status(StatusResponse)

Status reports status updates from replicas to the controller.

Status responses are a way for replicas to stream back introspection data that the controller can then announce to its clients. They have no effect on the lifecycles of compute collections. Correct operation of the Compute layer must not rely on Status responses being sent or received.

Status responses that are specific to collections must only be sent for collections that (a) have previously been created by a CreateDataflow command and (b) have not yet been reported to have advanced to the empty frontier.

Trait Implementations

impl Arbitrary for ComputeResponse<Timestamp>

type Strategy = Union<BoxedStrategy<ComputeResponse>>

The type of Strategy used to generate values of type Self.

type Parameters = ()

The type of parameters that arbitrary_with accepts for configuration of the generated Strategy. Parameters must implement Default.

fn arbitrary_with(_: Self::Parameters) -> Self::Strategy

Generates a Strategy for producing arbitrary values of type the implementing type (Self). The strategy is passed the arguments given in args.

fn arbitrary() -> Self::Strategy

Generates a Strategy for producing arbitrary values of type the implementing type (Self).

impl<T: Clone> Clone for ComputeResponse<T>

fn clone(&self) -> ComputeResponse<T>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<T: Debug> Debug for ComputeResponse<T>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<T: Send> GenericClient<ComputeCommand<T>, ComputeResponse<T>> for Box<dyn ComputeClient<T>>

fn recv<'life0, 'async_trait>( &'life0 mut self, ) -> Pin<Box<dyn Future<Output = Result<Option<ComputeResponse<T>>, Error>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Cancel safety

This method is cancel safe. If recv is used as the event in a tokio::select! statement and some other branch completes first, it is guaranteed that no messages were received by this client.

fn send<'life0, 'async_trait>( &'life0 mut self, cmd: ComputeCommand<T>, ) -> Pin<Box<dyn Future<Output = Result<(), Error>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Sends a command to the dataflow server.

fn as_stream<'a>( &'a mut self, ) -> Pin<Box<dyn Stream<Item = Result<R, Error>> + Send + 'a>>

where R: Send + 'a,

Returns an adapter that treats the client as a stream.

impl<T> GenericClient<ComputeCommand<T>, ComputeResponse<T>> for SequentialHydration<T>

where T: ComputeControllerTimestamp,

fn recv<'life0, 'async_trait>( &'life0 mut self, ) -> Pin<Box<dyn Future<Output = Result<Option<ComputeResponse<T>>, Error>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Cancel safety

This method is cancel safe. If recv is used as the event in a tokio::select! statement and some other branch completes first, it is guaranteed that no messages were received by this client.

fn send<'life0, 'async_trait>( &'life0 mut self, cmd: ComputeCommand<T>, ) -> Pin<Box<dyn Future<Output = Result<(), Error>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Sends a command to the dataflow server.

fn as_stream<'a>( &'a mut self, ) -> Pin<Box<dyn Stream<Item = Result<R, Error>> + Send + 'a>>

where R: Send + 'a,

Returns an adapter that treats the client as a stream.

impl<T: PartialEq> PartialEq for ComputeResponse<T>

fn eq(&self, other: &ComputeResponse<T>) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl<T> Partitionable<ComputeCommand<T>, ComputeResponse<T>> for (ComputeCommand<T>, ComputeResponse<T>)

where T: ComputeControllerTimestamp,

type PartitionedState = PartitionedComputeState<T>

The type which functions as the state machine for the partitioning.

fn new(parts: usize) -> PartitionedComputeState<T>

Construct a PartitionedState for the command–response pair.

impl<T> PartitionedState<ComputeCommand<T>, ComputeResponse<T>> for PartitionedComputeState<T>

where T: ComputeControllerTimestamp,

fn split_command( &mut self, command: ComputeCommand<T>, ) -> Vec<Option<ComputeCommand<T>>>

Splits a command into multiple partitions.

fn absorb_response( &mut self, shard_id: usize, message: ComputeResponse<T>, ) -> Option<Result<ComputeResponse<T>, Error>>

Absorbs a response from a single partition.

impl RustType<ProtoComputeResponse> for ComputeResponse<Timestamp>

fn into_proto(&self) -> ProtoComputeResponse

Convert a Self into a Proto value.

fn from_proto(proto: ProtoComputeResponse) -> Result<Self, TryFromProtoError>

Consume and convert a Proto back into a Self value.

fn into_proto_owned(self) -> Proto

A zero clone version of Self::into_proto that types can optionally implement, otherwise, the default implementation delegates to Self::into_proto.

impl<T> StructuralPartialEq for ComputeResponse<T>