Struct ActiveComputeState

pub(crate) struct ActiveComputeState<'a, A: Allocate> {
    pub timely_worker: &'a mut Worker<A>,
    pub compute_state: &'a mut ComputeState,
    pub response_tx: &'a mut ResponseSender,
}

A wrapper around ComputeState with a live timely worker and response channel.

Fields

timely_worker: &'a mut Worker<A>

The underlying Timely worker.

compute_state: &'a mut ComputeState

The compute state itself.

response_tx: &'a mut ResponseSender

The channel over which frontier information is reported.

Implementations

impl<'a, A: Allocate + 'static> ActiveComputeState<'a, A>

pub fn handle_compute_command(&mut self, cmd: ComputeCommand)

Entrypoint for applying a compute command.

fn handle_create_instance(&mut self, config: InstanceConfig)

fn handle_update_configuration(&mut self, params: ComputeParameters)

fn handle_create_dataflow( &mut self, dataflow: DataflowDescription<RenderPlan, CollectionMetadata>, )

fn handle_schedule(&mut self, id: GlobalId)

fn handle_allow_compaction( &mut self, id: GlobalId, frontier: Antichain<Timestamp>, )

fn handle_peek(&mut self, peek: Peek)

fn handle_cancel_peek(&mut self, uuid: Uuid)

fn drop_collection(&mut self, id: GlobalId)

Arrange for the given collection to be dropped.

Collection dropping occurs in three phases:

1. This method removes the collection from the ComputeState and drops its CollectionState, including its held dataflow tokens. It then adds the dropped collection to dropped_collections.
2. The next step of the Timely worker lets the source operators observe the token drops and shut themselves down.
3. report_dropped_collections removes the entry from dropped_collections and emits any outstanding final responses required by the compute protocol.

These steps ensure that we don’t report a collection as dropped to the controller before it has stopped reading from its inputs. Doing so would allow the controller to release its read holds on the inputs, which could lead to panics from the replica trying to read already compacted times.

pub fn initialize_logging(&mut self, config: LoggingConfig)

Initializes timely dataflow logging and publishes as a view.

pub fn report_frontiers(&mut self)

Send progress information to the controller.

pub fn report_dropped_collections(&mut self)

Report dropped collections to the controller.

pub fn report_operator_hydration(&self)

Report operator hydration events.

pub(crate) fn report_metrics(&self)

Report per-worker metrics.

fn process_peek(&mut self, upper: &mut Antichain<Timestamp>, peek: PendingPeek)

Either complete the peek (and send the response) or put it in the pending set.

pub fn process_peeks(&mut self)

Scan pending peeks and attempt to retire each.

fn send_peek_response(&mut self, peek: PendingPeek, response: PeekResponse)

Sends a response for this peek’s resolution to the coordinator.

Note that this function takes ownership of the PendingPeek, which is meant to prevent multiple responses to the same peek.

pub fn process_subscribes(&mut self)

Scan the shared subscribe response buffer, and forward results along.

pub fn process_copy_tos(&self)

Scan the shared copy to response buffer, and forward results along.

fn send_compute_response(&self, response: ComputeResponse)

Send a response to the coordinator.

pub(crate) fn check_expiration(&self)

Checks for dataflow expiration. Panics if we’re past the replica expiration time.

pub fn determine_dataflow_expiration( &self, time_dependence: &TimeDependence, until: &Antichain<Timestamp>, ) -> Antichain<Timestamp>

Returns the dataflow expiration, i.e, the timestamp beyond which diffs can be dropped.

Returns an empty timestamp if replica_expiration is unset or matches conditions under which dataflow expiration should be disabled.