Struct persist::s3::S3BlobConfig [−][src]
Expand description
Configuration for opening an S3Blob or S3BlobRead.
Fields
client: S3Client
bucket: String
prefix: String
Implementations
Returns a new S3BlobConfig for use in production.
Stores objects in the given bucket prepended with the (possibly empty) prefix. S3 credentials and region must be available in the process or environment.
Returns a new S3BlobConfig for use in unit tests.
By default, persist tests that use external storage (like s3) are
no-ops, so that cargo test
does the right thing without any
configuration. To activate the tests, set the
MZ_PERSIST_EXTERNAL_STORAGE_TEST_S3_BUCKET
environment variable and
ensure you have valid AWS credentials available in a location where the
AWS Rust SDK can discovery them.
This intentionally uses the MZ_PERSIST_EXTERNAL_STORAGE_TEST_S3_BUCKET
env as the switch for test no-op-ness instead of the presence of a valid
AWS authentication configuration envs because a developers might have
valid credentials present and this isn’t an explicit enough signal from
a developer running cargo test
that it’s okay to use these
credentials. It also intentionally does not use the local drop-in s3
replacement to keep persist unit tests light.
On CI, these tests are enabled by adding the scratch-aws-access plugin
to the cargo-test
step in ci/test/pipeline.template.yml
and setting
MZ_PERSIST_EXTERNAL_STORAGE_TEST_S3_BUCKET
in
ci/test/cargo-test/mzcompose.yml
.
For a Materialize developer, to opt in to these tests locally for development, follow the AWS access guide:
https://github.com/MaterializeInc/i2/blob/main/doc/aws-access.md
then running source src/persist/s3_test_env_mz.sh
. You will also have
to run aws sso login
if you haven’t recently.
Non-Materialize developers will have to set up their own auto-deleting bucket and export the same env vars that s3_test_env_mz.sh does.
Only public for use in src/benches.
Returns a clone of Self with a new v4 uuid prefix.
Trait Implementations
Auto Trait Implementations
impl !RefUnwindSafe for S3BlobConfig
impl Send for S3BlobConfig
impl Sync for S3BlobConfig
impl Unpin for S3BlobConfig
impl !UnwindSafe for S3BlobConfig
Blanket Implementations
Mutably borrows from an owned value. Read more
Attaches the provided Subscriber
to this type, returning a
WithDispatch
wrapper. Read more
Attaches the current default Subscriber
to this type, returning a
WithDispatch
wrapper. Read more