pub enum ComputeResponse<T = Timestamp> {
    FrontierUpper {
        id: GlobalId,
        upper: Antichain<T>,
    },
    PeekResponse(Uuid, PeekResponse, OpenTelemetryContext),
    SubscribeResponse(GlobalId, SubscribeResponse<T>),
    CopyToResponse(GlobalId, CopyToResponse),
    Status(StatusResponse),
}
Expand description

Compute protocol responses, sent by replicas to the compute controller.

Replicas send ComputeResponses in response to ComputeCommands they previously received from the compute controller.

Variants§

§

FrontierUpper

Fields

§id: GlobalId

TODO(#25239): Add documentation.

§upper: Antichain<T>

TODO(#25239): Add documentation.

FrontierUpper announces the advancement of the upper frontier of the specified compute collection. The response contain a collection ID and that collection’s new upper frontier.

Upon receiving a FrontierUpper response, the controller may assume that the replica has finished computing the given collection up to at least the given frontier. It may also assume that the replica has finished reading from the collection’s inputs up to that frontier.

Replicas must send FrontierUpper responses for compute collections that are indexes or storage sinks. Replicas must not send FrontierUpper responses for subscribes.

Replicas must never report regressing frontiers. Specifically:

  • The first frontier reported for a collection must not be less than that collection’s initial as_of frontier.
  • Subsequent reported frontiers for a collection must not be less than any frontier reported previously for the same collection.

Replicas must send a FrontierUpper response reporting advancement to the empty frontier for a collection in two cases:

  • The collection has advanced to the empty frontier (e.g. because its inputs have advanced to the empty frontier).
  • The collection was dropped in response to an AllowCompaction command that advanced its read frontier to the empty frontier. (#16275)

Once a collection was reported to have been advanced to the empty upper frontier:

  • It must no longer read from its inputs.
  • The replica must not send further FrontierUpper responses for that collection.

The replica must not send FrontierUpper responses for collections that have not been created previously by a CreateDataflow command or by a CreateInstance command.

§

PeekResponse(Uuid, PeekResponse, OpenTelemetryContext)

PeekResponse reports the result of a previous Peek command. The peek is identified by a Uuid that matches the command’s Peek::uuid.

The replica must send exactly one PeekResponse for every Peek command it received.

If the replica did not receive a CancelPeek command for a peek, it must not send a Canceled response for that peek. If the replica did receive a CancelPeek command for a peek, it may send any of the three PeekResponse variants.

The replica must not send PeekResponses for peek IDs that were not previously specified in a Peek command.

§

SubscribeResponse(GlobalId, SubscribeResponse<T>)

SubscribeResponse reports the results emitted by an active subscribe over some time interval.

For each subscribe that was installed by a previous CreateDataflow command, the replica must emit Batch responses that cover the entire time interval from the minimum time until the subscribe advances to the empty frontier or is dropped. The time intervals of consecutive Batches must be increasing, contiguous, non-overlapping, and non-empty. All updates transmitted in a batch must be consolidated and have times within that batch’s time interval. All updates’ times must be greater than or equal to as_of. The upper of the first Batch of a subscribe must not be less than that subscribe’s initial as_of frontier.

The replica must send DroppedAt responses if the subscribe was dropped in response to an AllowCompaction command that advanced its read frontier to the empty frontier. The DroppedAt frontier must be the upper frontier of the last emitted batch.

The replica must not send a DroppedAt response if the subscribe’s upper frontier (reported by Batch responses) has advanced to the empty frontier (e.g. because its inputs advanced to the empty frontier).

Once a subscribe was reported to have advanced to the empty frontier, or has been dropped:

  • It must no longer read from its inputs.
  • The replica must not send further SubscribeResponses for that subscribe.

The replica must not send SubscribeResponses for subscribes that have not been created previously by a CreateDataflow command.

§

CopyToResponse(GlobalId, CopyToResponse)

CopyToResponse reports the completion of an S3-oneshot sink.

The replica must send exactly one CopyToResponse for every S3-oneshot sink previously created by a CreateDataflow command.

The replica must not send CopyToResponses for S3-oneshot sinks that were not previously created by a CreateDataflow command.

§

Status(StatusResponse)

Status reports status updates from replicas to the controller.

Status responses are a way for replicas to stream back introspection data that the controller can then announce to its clients. They have no effect on the lifecycles of compute collections. Correct operation of the Compute layer must not rely on Status responses being sent or received.

Status responses that are specific to collections must only be sent for collections that (a) have previously been created by a CreateDataflow command and (b) have not yet been reported to have advanced to the empty frontier.

Trait Implementations§

source§

impl Arbitrary for ComputeResponse<Timestamp>

§

type Strategy = Union<BoxedStrategy<ComputeResponse>>

The type of Strategy used to generate values of type Self.
§

type Parameters = ()

The type of parameters that arbitrary_with accepts for configuration of the generated Strategy. Parameters must implement Default.
source§

fn arbitrary_with(_: Self::Parameters) -> Self::Strategy

Generates a Strategy for producing arbitrary values of type the implementing type (Self). The strategy is passed the arguments given in args. Read more
source§

fn arbitrary() -> Self::Strategy

Generates a Strategy for producing arbitrary values of type the implementing type (Self). Read more
source§

impl<T: Clone> Clone for ComputeResponse<T>

source§

fn clone(&self) -> ComputeResponse<T>

Returns a copy of the value. Read more
1.0.0 · source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
source§

impl<T: Debug> Debug for ComputeResponse<T>

source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
source§

impl<'de, T> Deserialize<'de> for ComputeResponse<T>
where T: Deserialize<'de>,

source§

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>
where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer. Read more
source§

impl<T: Send> GenericClient<ComputeCommand<T>, ComputeResponse<T>> for Box<dyn ComputeClient<T>>

source§

fn send<'life0, 'async_trait>( &'life0 mut self, cmd: ComputeCommand<T> ) -> Pin<Box<dyn Future<Output = Result<(), Error>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait,

Sends a command to the dataflow server. Read more
source§

fn recv<'life0, 'async_trait>( &'life0 mut self ) -> Pin<Box<dyn Future<Output = Result<Option<ComputeResponse<T>>, Error>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait,

Receives the next response from the dataflow server. Read more
source§

fn as_stream<'a>( &'a mut self ) -> Pin<Box<dyn Stream<Item = Result<R, Error>> + Send + 'a>>
where R: Send + 'a,

Returns an adapter that treats the client as a stream. Read more
source§

impl<T: PartialEq> PartialEq for ComputeResponse<T>

source§

fn eq(&self, other: &ComputeResponse<T>) -> bool

This method tests for self and other values to be equal, and is used by ==.
1.0.0 · source§

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.
source§

impl<T> Partitionable<ComputeCommand<T>, ComputeResponse<T>> for (ComputeCommand<T>, ComputeResponse<T>)
where T: Timestamp + Lattice,

§

type PartitionedState = PartitionedComputeState<T>

The type which functions as the state machine for the partitioning.
source§

fn new(parts: usize) -> PartitionedComputeState<T>

Construct a PartitionedState for the command–response pair.
source§

impl<T> PartitionedState<ComputeCommand<T>, ComputeResponse<T>> for PartitionedComputeState<T>
where T: Timestamp + Lattice,

source§

fn split_command( &mut self, command: ComputeCommand<T> ) -> Vec<Option<ComputeCommand<T>>>

Splits a command into multiple partitions.
source§

fn absorb_response( &mut self, shard_id: usize, message: ComputeResponse<T> ) -> Option<Result<ComputeResponse<T>, Error>>

Absorbs a response from a single partition. Read more
source§

impl RustType<ProtoComputeResponse> for ComputeResponse<Timestamp>

source§

fn into_proto(&self) -> ProtoComputeResponse

Convert a Self into a Proto value.
source§

fn from_proto(proto: ProtoComputeResponse) -> Result<Self, TryFromProtoError>

Consume and convert a Proto back into a Self value. Read more
source§

impl<T> Serialize for ComputeResponse<T>
where T: Serialize,

source§

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>
where __S: Serializer,

Serialize this value into the given Serde serializer. Read more
source§

impl<T> StructuralPartialEq for ComputeResponse<T>

Auto Trait Implementations§

§

impl<T> RefUnwindSafe for ComputeResponse<T>
where T: RefUnwindSafe,

§

impl<T> Send for ComputeResponse<T>
where T: Send,

§

impl<T> Sync for ComputeResponse<T>
where T: Sync,

§

impl<T> Unpin for ComputeResponse<T>
where T: Unpin,

§

impl<T> UnwindSafe for ComputeResponse<T>
where T: UnwindSafe,

Blanket Implementations§

source§

impl<T> Any for T
where T: 'static + ?Sized,

source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
source§

impl<T> Borrow<T> for T
where T: ?Sized,

source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
source§

impl<T, U> CastInto<U> for T
where U: CastFrom<T>,

source§

fn cast_into(self) -> U

Performs the cast.
source§

impl<T> Conv for T

source§

fn conv<T>(self) -> T
where Self: Into<T>,

Converts self into T using Into<T>. Read more
source§

impl<R, O, T> CopyOnto<ConsecutiveOffsetPairs<R, O>> for T
where R: Region<Index = (usize, usize)>, O: OffsetContainer<usize>, T: CopyOnto<R>,

source§

fn copy_onto( self, target: &mut ConsecutiveOffsetPairs<R, O> ) -> <ConsecutiveOffsetPairs<R, O> as Region>::Index

Copy self into the target container, returning an index that allows to look up the corresponding read item.
source§

impl<T> DynClone for T
where T: Clone,

source§

fn __clone_box(&self, _: Private) -> *mut ()

source§

impl<T> FmtForward for T

source§

fn fmt_binary(self) -> FmtBinary<Self>
where Self: Binary,

Causes self to use its Binary implementation when Debug-formatted.
source§

fn fmt_display(self) -> FmtDisplay<Self>
where Self: Display,

Causes self to use its Display implementation when Debug-formatted.
source§

fn fmt_lower_exp(self) -> FmtLowerExp<Self>
where Self: LowerExp,

Causes self to use its LowerExp implementation when Debug-formatted.
source§

fn fmt_lower_hex(self) -> FmtLowerHex<Self>
where Self: LowerHex,

Causes self to use its LowerHex implementation when Debug-formatted.
source§

fn fmt_octal(self) -> FmtOctal<Self>
where Self: Octal,

Causes self to use its Octal implementation when Debug-formatted.
source§

fn fmt_pointer(self) -> FmtPointer<Self>
where Self: Pointer,

Causes self to use its Pointer implementation when Debug-formatted.
source§

fn fmt_upper_exp(self) -> FmtUpperExp<Self>
where Self: UpperExp,

Causes self to use its UpperExp implementation when Debug-formatted.
source§

fn fmt_upper_hex(self) -> FmtUpperHex<Self>
where Self: UpperHex,

Causes self to use its UpperHex implementation when Debug-formatted.
source§

fn fmt_list(self) -> FmtList<Self>
where &'a Self: for<'a> IntoIterator,

Formats each item in a sequence. Read more
source§

impl<T> From<T> for T

source§

fn from(t: T) -> T

Returns the argument unchanged.

source§

impl<T> FromRef<T> for T
where T: Clone,

source§

fn from_ref(input: &T) -> T

Converts to this type from a reference to the input type.
source§

impl<T> FutureExt for T

source§

fn with_context(self, otel_cx: Context) -> WithContext<Self>

Attaches the provided Context to this type, returning a WithContext wrapper. Read more
source§

fn with_current_context(self) -> WithContext<Self>

Attaches the current Context to this type, returning a WithContext wrapper. Read more
source§

impl<T> Instrument for T

source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
source§

impl<T, U> Into<U> for T
where U: From<T>,

source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

source§

impl<T> IntoRequest<T> for T

source§

fn into_request(self) -> Request<T>

Wrap the input message T in a tonic::Request
source§

impl<Unshared, Shared> IntoShared<Shared> for Unshared
where Shared: FromUnshared<Unshared>,

source§

fn into_shared(self) -> Shared

Creates a shared type from an unshared type.
source§

impl<T, U> OverrideFrom<Option<&T>> for U
where U: OverrideFrom<T>,

source§

fn override_from(self, layer: &Option<&T>) -> U

Override the configuration represented by Self with values from the given layer.
source§

impl<T> Pipe for T
where T: ?Sized,

source§

fn pipe<R>(self, func: impl FnOnce(Self) -> R) -> R
where Self: Sized,

Pipes by value. This is generally the method you want to use. Read more
source§

fn pipe_ref<'a, R>(&'a self, func: impl FnOnce(&'a Self) -> R) -> R
where R: 'a,

Borrows self and passes that borrow into the pipe function. Read more
source§

fn pipe_ref_mut<'a, R>(&'a mut self, func: impl FnOnce(&'a mut Self) -> R) -> R
where R: 'a,

Mutably borrows self and passes that borrow into the pipe function. Read more
source§

fn pipe_borrow<'a, B, R>(&'a self, func: impl FnOnce(&'a B) -> R) -> R
where Self: Borrow<B>, B: 'a + ?Sized, R: 'a,

Borrows self, then passes self.borrow() into the pipe function. Read more
source§

fn pipe_borrow_mut<'a, B, R>( &'a mut self, func: impl FnOnce(&'a mut B) -> R ) -> R
where Self: BorrowMut<B>, B: 'a + ?Sized, R: 'a,

Mutably borrows self, then passes self.borrow_mut() into the pipe function. Read more
source§

fn pipe_as_ref<'a, U, R>(&'a self, func: impl FnOnce(&'a U) -> R) -> R
where Self: AsRef<U>, U: 'a + ?Sized, R: 'a,

Borrows self, then passes self.as_ref() into the pipe function.
source§

fn pipe_as_mut<'a, U, R>(&'a mut self, func: impl FnOnce(&'a mut U) -> R) -> R
where Self: AsMut<U>, U: 'a + ?Sized, R: 'a,

Mutably borrows self, then passes self.as_mut() into the pipe function.
source§

fn pipe_deref<'a, T, R>(&'a self, func: impl FnOnce(&'a T) -> R) -> R
where Self: Deref<Target = T>, T: 'a + ?Sized, R: 'a,

Borrows self, then passes self.deref() into the pipe function.
source§

fn pipe_deref_mut<'a, T, R>( &'a mut self, func: impl FnOnce(&'a mut T) -> R ) -> R
where Self: DerefMut<Target = T> + Deref, T: 'a + ?Sized, R: 'a,

Mutably borrows self, then passes self.deref_mut() into the pipe function.
source§

impl<T> Pointable for T

source§

const ALIGN: usize = _

The alignment of pointer.
§

type Init = T

The type for initializers.
source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
source§

impl<T> ProgressEventTimestamp for T
where T: Data + Debug + Any,

source§

fn as_any(&self) -> &(dyn Any + 'static)

Upcasts this ProgressEventTimestamp to Any. Read more
source§

fn type_name(&self) -> &'static str

Returns the name of the concrete type of this object. Read more
source§

impl<P, R> ProtoType<R> for P
where R: RustType<P>,

source§

impl<R, T> PushInto<FlatStack<R>> for T
where R: Region + Clone + 'static, T: CopyOnto<R>,

source§

fn push_into(self, target: &mut FlatStack<R>)

Push self into the target container.
source§

impl<T> PushInto<Vec<T>> for T

source§

fn push_into(self, target: &mut Vec<T>)

Push self into the target container.
source§

impl<T> Same for T

§

type Output = T

Should always be Self
source§

impl<T> Tap for T

source§

fn tap(self, func: impl FnOnce(&Self)) -> Self

Immutable access to a value. Read more
source§

fn tap_mut(self, func: impl FnOnce(&mut Self)) -> Self

Mutable access to a value. Read more
source§

fn tap_borrow<B>(self, func: impl FnOnce(&B)) -> Self
where Self: Borrow<B>, B: ?Sized,

Immutable access to the Borrow<B> of a value. Read more
source§

fn tap_borrow_mut<B>(self, func: impl FnOnce(&mut B)) -> Self
where Self: BorrowMut<B>, B: ?Sized,

Mutable access to the BorrowMut<B> of a value. Read more
source§

fn tap_ref<R>(self, func: impl FnOnce(&R)) -> Self
where Self: AsRef<R>, R: ?Sized,

Immutable access to the AsRef<R> view of a value. Read more
source§

fn tap_ref_mut<R>(self, func: impl FnOnce(&mut R)) -> Self
where Self: AsMut<R>, R: ?Sized,

Mutable access to the AsMut<R> view of a value. Read more
source§

fn tap_deref<T>(self, func: impl FnOnce(&T)) -> Self
where Self: Deref<Target = T>, T: ?Sized,

Immutable access to the Deref::Target of a value. Read more
source§

fn tap_deref_mut<T>(self, func: impl FnOnce(&mut T)) -> Self
where Self: DerefMut<Target = T> + Deref, T: ?Sized,

Mutable access to the Deref::Target of a value. Read more
source§

fn tap_dbg(self, func: impl FnOnce(&Self)) -> Self

Calls .tap() only in debug builds, and is erased in release builds.
source§

fn tap_mut_dbg(self, func: impl FnOnce(&mut Self)) -> Self

Calls .tap_mut() only in debug builds, and is erased in release builds.
source§

fn tap_borrow_dbg<B>(self, func: impl FnOnce(&B)) -> Self
where Self: Borrow<B>, B: ?Sized,

Calls .tap_borrow() only in debug builds, and is erased in release builds.
source§

fn tap_borrow_mut_dbg<B>(self, func: impl FnOnce(&mut B)) -> Self
where Self: BorrowMut<B>, B: ?Sized,

Calls .tap_borrow_mut() only in debug builds, and is erased in release builds.
source§

fn tap_ref_dbg<R>(self, func: impl FnOnce(&R)) -> Self
where Self: AsRef<R>, R: ?Sized,

Calls .tap_ref() only in debug builds, and is erased in release builds.
source§

fn tap_ref_mut_dbg<R>(self, func: impl FnOnce(&mut R)) -> Self
where Self: AsMut<R>, R: ?Sized,

Calls .tap_ref_mut() only in debug builds, and is erased in release builds.
source§

fn tap_deref_dbg<T>(self, func: impl FnOnce(&T)) -> Self
where Self: Deref<Target = T>, T: ?Sized,

Calls .tap_deref() only in debug builds, and is erased in release builds.
source§

fn tap_deref_mut_dbg<T>(self, func: impl FnOnce(&mut T)) -> Self
where Self: DerefMut<Target = T> + Deref, T: ?Sized,

Calls .tap_deref_mut() only in debug builds, and is erased in release builds.
source§

impl<T> ToOwned for T
where T: Clone,

§

type Owned = T

The resulting type after obtaining ownership.
source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
source§

impl<T> TryConv for T

source§

fn try_conv<T>(self) -> Result<T, Self::Error>
where Self: TryInto<T>,

Attempts to convert self into T using TryInto<T>. Read more
source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

§

type Error = Infallible

The type returned in the event of a conversion error.
source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

source§

fn vzip(self) -> V

source§

impl<T> WithSubscriber for T

source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more
source§

impl<T> Data for T
where T: Clone + 'static,

source§

impl<T> Data for T
where T: Send + Sync + Any + Serialize + for<'a> Deserialize<'a> + 'static,

source§

impl<T> DeserializeOwned for T
where T: for<'de> Deserialize<'de>,

source§

impl<T> ExchangeData for T
where T: Data + Data,