Struct mz_compute_client::service::PartitionedComputeState

pub struct PartitionedComputeState<T> { /* private fields */ }

Maintained state for partitioned compute clients.

This helper type unifies the responses of multiple partitioned workers in order to present as a single worker:

• It emits FrontierUpper responses reporting the minimum/meet of frontiers reported by the individual workers.
• It emits PeekResponses and SubscribeResponses reporting the union of the responses received from the workers.

In the compute communication stack, this client is instantiated several times:

• One instance on the controller side, dispatching between cluster processes.
• One instance in each cluster process, dispatching between timely worker threads.

Note that because compute commands, except CreateTimely, are only sent to the first process, the cluster-side instances of PartitionedComputeState are not guaranteed to see all compute commands. Or more specifically: The instance running inside process 0 sees all commands, whereas the instances running inside the other processes only see CreateTimely. The PartitionedComputeState implementation must be able to cope with this limited visiblity. It does so by performing most of its state management based on observed compute responses rather than commands.

Implementations

impl<T> PartitionedComputeState<T>where T: Timestamp,

pub fn observe_command(&mut self, command: &ComputeCommand<T>)

Observes commands that move past, and prepares state for responses.

Trait Implementations

impl<T: Debug> Debug for PartitionedComputeState<T>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<T> PartitionedState<ComputeCommand<T>, ComputeResponse<T>> for PartitionedComputeState<T>where T: Timestamp + Lattice,

fn split_command( &mut self, command: ComputeCommand<T> ) -> Vec<Option<ComputeCommand<T>>>

Splits a command into multiple partitions.

fn absorb_response( &mut self, shard_id: usize, message: ComputeResponse<T> ) -> Option<Result<ComputeResponse<T>, Error>>

Absorbs a response from a single partition.