Function mz_storage_operators::s3_oneshot_sink::render_completion_operator

fn render_completion_operator<G, F>(
    scope: G,
    connection_context: ConnectionContext,
    aws_connection: AwsConnection,
    connection_id: CatalogItemId,
    sink_id: GlobalId,
    s3_key_manager: S3KeyManager,
    completion_stream: Stream<G, Result<u64, String>>,
    worker_callback: F,
)
where
    G: Scope<Timestamp = Timestamp>,
    F: FnOnce(Result<u64, String>) + 'static,

Renders the ‘completion’ operator, which expects a completion_stream that it reads over a Pipeline edge such that it receives a single completion event per worker. Then forwards this result to the worker_callback after any cleanup work (see below).

On the leader worker, this operator waits to see the empty frontier for the completion_stream and then does some cleanup work before calling the callback.

This cleanup work removes the INCOMPLETE sentinel file (see description of render_initialization_operator for more details).