Module mz_storage_client::controller
source · Expand description
A controller that provides an interface to the storage layer.
The storage controller curates the creation of sources, the progress of readers through these collections, and their eventual dropping and resource reclamation.
The storage controller can be viewed as a partial map from GlobalId
to collection. It is an error to
use an identifier before it has been “created” with create_source()
. Once created, the controller holds
a read capability for each source, which is manipulated with update_read_capabilities()
.
Eventually, the source is dropped with either drop_sources()
or by allowing compaction to the
empty frontier.
Modules
- A tokio task (and support machinery) for maintaining storage-managed collections.
- Tracks persist shards ready to be finalized, i.e. remove the ability to read or write them. This identifies shards we no longer use, but did not have the opportunity to finalize before e.g. crashing.
- A tokio tasks (and support machinery) for dealing with the persist handles that the storage controller needs to hold.
- Rehydration of storage replicas.
- A tokio task (and support machinery) for producing storage statistics.
Structs
- Describes a request to create a source.
- Metadata required by a storage instance to read a storage collection
- State maintained about individual collections.
- A storage controller for a storage instance.
- Opaque token to ensure
prepare_export
is called beforecreate_exports
. This token proves that compaction is being held back onfrom_id
at least untilid
is created. It should be held while the AS OF is determined. - The subset of
CollectionMetadata
that must be durable stored. - State maintained about individual exports.
- A wrapper struct that presents the adapter token to a format that is understandable by persist and also allows us to differentiate between a token being present versus being set for the first time.
- Controller state maintained for each storage instance.
Enums
- Describes how data is written to the collection.
- Compaction policies for collections maintained by
Controller
. - __Field 🔒
- __Field 🔒
- __Field 🔒
- __Field 🔒
Constants
Statics
- CALLSITE 🔒
- CALLSITE 🔒
- CALLSITE 🔒
- CALLSITE 🔒
- CALLSITE 🔒
- CALLSITE 🔒
- CALLSITE 🔒
- CALLSITE 🔒
- CALLSITE 🔒
- CALLSITE 🔒
- CALLSITE 🔒
- CALLSITE 🔒
- CALLSITE 🔒
- META 🔒
- META 🔒
- META 🔒
- META 🔒
- META 🔒
- META 🔒
- META 🔒
- META 🔒
- META 🔒
- META 🔒
- META 🔒
- META 🔒
- META 🔒
Traits
- A trait that is used to calculate safe resumption frontiers for a source.