Module mz_storage_operators::s3_oneshot_sink
source · Expand description
Uploads a consolidated collection to S3
Structs§
- dyncfg parameters for the copy_to operator, stored in this struct to avoid requiring storage to depend on the compute crate. See
src/compute-types/src/dyncfgs.rs
for the definition of these parameters.
Functions§
- Copy the rows from the input collection to s3.
worker_callback
is used to send the final count of rows uploaded to s3, or an error message if the operator failed. This is per-worker, and these responses are aggregated upstream by the compute client.sink_id
is used to identify the sink for logging purposes and as a unique prefix for files created by the sink.