Module command_channel

A channel for broadcasting compute commands from worker 0 to other workers.

Compute uses a dataflow to distribute commands between workers. This is to ensure workers retain a consistent dataflow state across reconnects. If each worker would receive its commands directly from the controller, there wouldn’t be any guarantee that after a reconnect all workers have seen the same sequence of commands. This is particularly problematic for CreateDataflow commands, since Timely requires that all workers render the same dataflows in the same order. So the controller instead sends commands only to worker 0, which then broadcasts them to other workers through the Timely fabric, taking care of the correct sequencing.

Commands in the command channel are tagged with an epoch identifying the incarnation of the compute protocol the command belongs to, allowing workers to recognize client reconnects that require a reconciliation.

Structs

• Receiver
  A receiver reading commands from the command channel.
• Sender
  A sender pushing commands onto the command channel.

Functions

• render
  Render the command channel dataflow.
• split_command 🔒
  Split the given command into the given number of parts.