Struct mz_storage_operators::s3_oneshot_sink::S3KeyManager

struct S3KeyManager {
    pub bucket: String,
    object_key_prefix: String,
}

Helper to manage object keys created by this sink based on the S3 URI provided by the user and the GlobalId that identifies this copy-to-s3 sink. Since there may be multiple compute replicas running their own copy of this sink we need to ensure the S3 keys are consistent such that we can detect when objects were created by an instance of this sink or not.

Fields

bucket: String

object_key_prefix: String

Implementations

impl S3KeyManager

pub fn new(sink_id: &GlobalId, s3_uri: &str) -> Self

fn data_key(&self, batch: u64, file_index: usize, extension: &str) -> String

The S3 key to use for a specific data file, based on the batch it belongs to and the index within that batch.

fn incomplete_sentinel_key(&self) -> String

The S3 key to use for the incomplete sentinel file

fn is_sink_object(&self, object_key: &str) -> bool

Whether the given object key belongs to this sink instance

fn path_prefix(&self) -> &str

The key prefix based on the URI provided by the user. NOTE this doesn’t contain the additional prefix we include on all keys written by the sink e.g. mz-{sink_id}-batch-... This is useful when listing objects in the bucket with this prefix to determine if its clear to upload.

Trait Implementations

impl Clone for S3KeyManager

fn clone(&self) -> S3KeyManager

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.