Struct mz_persist_client::PersistClient
source · pub struct PersistClient {
pub(crate) cfg: PersistConfig,
pub(crate) blob: Arc<dyn Blob>,
pub(crate) consensus: Arc<dyn Consensus>,
pub(crate) metrics: Arc<Metrics>,
pub(crate) isolated_runtime: Arc<IsolatedRuntime>,
pub(crate) shared_states: Arc<StateCache>,
pub(crate) pubsub_sender: Arc<dyn PubSubSender>,
}
Expand description
A handle for interacting with the set of persist shard made durable at a single PersistLocation.
All async methods on PersistClient retry for as long as they are able, but the returned std::future::Futures implement “cancel on drop” semantics. This means that callers can add a timeout using tokio::time::timeout or tokio::time::timeout_at.
tokio::time::timeout(timeout, client.open::<String, String, u64, i64>(id,
Arc::new(StringSchema),Arc::new(StringSchema),diagnostics, true)).await
Fields§
§cfg: PersistConfig
§blob: Arc<dyn Blob>
§consensus: Arc<dyn Consensus>
§metrics: Arc<Metrics>
§isolated_runtime: Arc<IsolatedRuntime>
§pubsub_sender: Arc<dyn PubSubSender>
Implementations§
source§impl PersistClient
impl PersistClient
sourcepub fn new(
cfg: PersistConfig,
blob: Arc<dyn Blob>,
consensus: Arc<dyn Consensus>,
metrics: Arc<Metrics>,
isolated_runtime: Arc<IsolatedRuntime>,
shared_states: Arc<StateCache>,
pubsub_sender: Arc<dyn PubSubSender>,
) -> Result<Self, ExternalError>
pub fn new( cfg: PersistConfig, blob: Arc<dyn Blob>, consensus: Arc<dyn Consensus>, metrics: Arc<Metrics>, isolated_runtime: Arc<IsolatedRuntime>, shared_states: Arc<StateCache>, pubsub_sender: Arc<dyn PubSubSender>, ) -> Result<Self, ExternalError>
Returns a new client for interfacing with persist shards made durable to the given Blob and Consensus.
This is exposed mostly for testing. Persist users likely want crate::cache::PersistClientCache::open.
sourcepub async fn new_for_tests() -> Self
pub async fn new_for_tests() -> Self
Returns a new in-mem PersistClient for tests and examples.
pub(crate) async fn make_machine<K, V, T, D>( &self, shard_id: ShardId, diagnostics: Diagnostics, ) -> Result<Machine<K, V, T, D>, InvalidUsage<T>>
sourcepub async fn open<K, V, T, D>(
&self,
shard_id: ShardId,
key_schema: Arc<K::Schema>,
val_schema: Arc<V::Schema>,
diagnostics: Diagnostics,
use_critical_since: bool,
) -> Result<(WriteHandle<K, V, T, D>, ReadHandle<K, V, T, D>), InvalidUsage<T>>
pub async fn open<K, V, T, D>( &self, shard_id: ShardId, key_schema: Arc<K::Schema>, val_schema: Arc<V::Schema>, diagnostics: Diagnostics, use_critical_since: bool, ) -> Result<(WriteHandle<K, V, T, D>, ReadHandle<K, V, T, D>), InvalidUsage<T>>
Provides capabilities for the durable TVC identified by shard_id
at
its current since and upper frontiers.
This method is a best-effort attempt to regain control of the frontiers of a shard. Its most common uses are to recover capabilities that have expired (leases) or to attempt to read a TVC that one did not create (or otherwise receive capabilities for). If the frontiers have been fully released by all other parties, this call may result in capabilities with empty frontiers (which are useless).
If shard_id
has never been used before, initializes a new shard and
returns handles with since
and upper
frontiers set to initial values
of Antichain::from_elem(T::minimum())
.
The schema
parameter is currently unused, but should be an object
that represents the schema of the data in the shard. This will be required
in the future.
sourcepub async fn open_leased_reader<K, V, T, D>(
&self,
shard_id: ShardId,
key_schema: Arc<K::Schema>,
val_schema: Arc<V::Schema>,
diagnostics: Diagnostics,
use_critical_since: bool,
) -> Result<ReadHandle<K, V, T, D>, InvalidUsage<T>>
pub async fn open_leased_reader<K, V, T, D>( &self, shard_id: ShardId, key_schema: Arc<K::Schema>, val_schema: Arc<V::Schema>, diagnostics: Diagnostics, use_critical_since: bool, ) -> Result<ReadHandle<K, V, T, D>, InvalidUsage<T>>
Self::open, but returning only a ReadHandle.
Use this to save latency and a bit of persist traffic if you’re just going to immediately drop or expire the WriteHandle.
The _schema
parameter is currently unused, but should be an object
that represents the schema of the data in the shard. This will be required
in the future.
sourcepub async fn create_batch_fetcher<K, V, T, D>(
&self,
shard_id: ShardId,
key_schema: Arc<K::Schema>,
val_schema: Arc<V::Schema>,
is_transient: bool,
diagnostics: Diagnostics,
) -> BatchFetcher<K, V, T, D>
pub async fn create_batch_fetcher<K, V, T, D>( &self, shard_id: ShardId, key_schema: Arc<K::Schema>, val_schema: Arc<V::Schema>, is_transient: bool, diagnostics: Diagnostics, ) -> BatchFetcher<K, V, T, D>
Creates and returns a BatchFetcher for the given shard id.
sourcepub const CONTROLLER_CRITICAL_SINCE: CriticalReaderId = _
pub const CONTROLLER_CRITICAL_SINCE: CriticalReaderId = _
A convenience CriticalReaderId for Materialize controllers.
For most (soon to be all?) shards in Materialize, a centralized “controller” is the authority for when a user no longer needs to read at a given frontier. (Other uses are temporary holds where correctness of the overall system can be maintained through a lease timeout.) To make SinceHandle easier to work with, we offer this convenience id for Materialize controllers, so they don’t have to durably record it.
TODO: We’re still shaking out whether the controller should be the only critical since hold or if there are other places we want them. If the former, we should remove CriticalReaderId and bake in the singular nature of the controller critical handle.
// This prints as something that is not 0 but is visually recognizable.
assert_eq!(
mz_persist_client::PersistClient::CONTROLLER_CRITICAL_SINCE.to_string(),
"c00000000-1111-2222-3333-444444444444",
)
sourcepub async fn open_critical_since<K, V, T, D, O>(
&self,
shard_id: ShardId,
reader_id: CriticalReaderId,
diagnostics: Diagnostics,
) -> Result<SinceHandle<K, V, T, D, O>, InvalidUsage<T>>
pub async fn open_critical_since<K, V, T, D, O>( &self, shard_id: ShardId, reader_id: CriticalReaderId, diagnostics: Diagnostics, ) -> Result<SinceHandle<K, V, T, D, O>, InvalidUsage<T>>
Provides a capability for the durable TVC identified by shard_id
at
its current since frontier.
In contrast to the time-leased ReadHandle returned by Self::open and
Self::open_leased_reader, this handle and its associated capability
are not leased. A SinceHandle does not release its since capability;
downgrade to the empty antichain to hold back the since.
Also unlike ReadHandle
, the handle is not expired on drop.
This is less ergonomic, but useful for “critical” since
holds which must survive even lease timeouts.
IMPORTANT: The above means that if a SinceHandle is registered and then lost, the shard’s since will be permanently “stuck”, forever preventing logical compaction. Users are advised to durably record (preferably in code) the intended CriticalReaderId before registering a SinceHandle (in case the process crashes at the wrong time).
If shard_id
has never been used before, initializes a new shard and
return a handle with its since
frontier set to the initial value of
Antichain::from_elem(T::minimum())
.
sourcepub async fn open_writer<K, V, T, D>(
&self,
shard_id: ShardId,
key_schema: Arc<K::Schema>,
val_schema: Arc<V::Schema>,
diagnostics: Diagnostics,
) -> Result<WriteHandle<K, V, T, D>, InvalidUsage<T>>
pub async fn open_writer<K, V, T, D>( &self, shard_id: ShardId, key_schema: Arc<K::Schema>, val_schema: Arc<V::Schema>, diagnostics: Diagnostics, ) -> Result<WriteHandle<K, V, T, D>, InvalidUsage<T>>
Self::open, but returning only a WriteHandle.
Use this to save latency and a bit of persist traffic if you’re just going to immediately drop or expire the ReadHandle.
The _schema
parameter is currently unused, but should be an object
that represents the schema of the data in the shard. This will be required
in the future.
sourcepub async fn get_schema<K, V, T, D>(
&self,
shard_id: ShardId,
schema_id: SchemaId,
diagnostics: Diagnostics,
) -> Result<Option<(K::Schema, V::Schema)>, InvalidUsage<T>>
pub async fn get_schema<K, V, T, D>( &self, shard_id: ShardId, schema_id: SchemaId, diagnostics: Diagnostics, ) -> Result<Option<(K::Schema, V::Schema)>, InvalidUsage<T>>
Returns the requested schema, if known at the current state.
sourcepub async fn latest_schema<K, V, T, D>(
&self,
shard_id: ShardId,
diagnostics: Diagnostics,
) -> Result<Option<(SchemaId, K::Schema, V::Schema)>, InvalidUsage<T>>
pub async fn latest_schema<K, V, T, D>( &self, shard_id: ShardId, diagnostics: Diagnostics, ) -> Result<Option<(SchemaId, K::Schema, V::Schema)>, InvalidUsage<T>>
Returns the latest schema registered at the current state.
sourcepub async fn compare_and_evolve_schema<K, V, T, D>(
&self,
shard_id: ShardId,
expected: SchemaId,
key_schema: &K::Schema,
val_schema: &V::Schema,
diagnostics: Diagnostics,
) -> Result<CaESchema<K, V>, InvalidUsage<T>>
pub async fn compare_and_evolve_schema<K, V, T, D>( &self, shard_id: ShardId, expected: SchemaId, key_schema: &K::Schema, val_schema: &V::Schema, diagnostics: Diagnostics, ) -> Result<CaESchema<K, V>, InvalidUsage<T>>
Registers a new latest schema for the given shard.
This new schema must be backward_compatible with all previous schemas for this shard. If it’s not, CaESchema::Incompatible is returned.
To prevent races, the caller must declare what it believes to be the latest schema id. If this doesn’t match reality, CaESchema::ExpectedMismatch is returned.
sourcepub async fn is_finalized<K, V, T, D>(
&self,
shard_id: ShardId,
diagnostics: Diagnostics,
) -> Result<bool, InvalidUsage<T>>
pub async fn is_finalized<K, V, T, D>( &self, shard_id: ShardId, diagnostics: Diagnostics, ) -> Result<bool, InvalidUsage<T>>
Check if the given shard is in a finalized state; ie. it can no longer be read, any data that was written to it is no longer accessible, and we’ve discarded references to that data from state.
sourcepub async fn finalize_shard<K, V, T, D>(
&self,
shard_id: ShardId,
diagnostics: Diagnostics,
) -> Result<(), InvalidUsage<T>>
pub async fn finalize_shard<K, V, T, D>( &self, shard_id: ShardId, diagnostics: Diagnostics, ) -> Result<(), InvalidUsage<T>>
If a shard is guaranteed to never be used again, finalize it to delete the associated data and release any associated resources. (Except for a little state in consensus we use to represent the tombstone.)
The caller should ensure that both the since
and upper
of the shard
have been advanced to []
: ie. the shard is no longer writable or readable.
Otherwise an error is returned.
Once finalize_shard
has been called, the result of future operations on
the shard are not defined. They may return errors or succeed as a noop.
sourcepub async fn inspect_shard<T: Timestamp + Lattice + Codec64>(
&self,
shard_id: &ShardId,
) -> Result<impl Serialize, Error>
pub async fn inspect_shard<T: Timestamp + Lattice + Codec64>( &self, shard_id: &ShardId, ) -> Result<impl Serialize, Error>
Returns the internal state of the shard for debugging and QA.
We’ll be thoughtful about making unnecessary changes, but the output of this method needs to be gated from users, so that it’s not subject to our backward compatibility guarantees.
Trait Implementations§
source§impl Clone for PersistClient
impl Clone for PersistClient
source§fn clone(&self) -> PersistClient
fn clone(&self) -> PersistClient
1.0.0 · source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source
. Read moreAuto Trait Implementations§
impl Freeze for PersistClient
impl !RefUnwindSafe for PersistClient
impl Send for PersistClient
impl Sync for PersistClient
impl Unpin for PersistClient
impl !UnwindSafe for PersistClient
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
source§default unsafe fn clone_to_uninit(&self, dst: *mut T)
default unsafe fn clone_to_uninit(&self, dst: *mut T)
clone_to_uninit
)source§impl<T> FutureExt for T
impl<T> FutureExt for T
source§fn with_context(self, otel_cx: Context) -> WithContext<Self>
fn with_context(self, otel_cx: Context) -> WithContext<Self>
source§fn with_current_context(self) -> WithContext<Self>
fn with_current_context(self) -> WithContext<Self>
source§impl<T> Instrument for T
impl<T> Instrument for T
source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
source§impl<T> IntoRequest<T> for T
impl<T> IntoRequest<T> for T
source§fn into_request(self) -> Request<T>
fn into_request(self) -> Request<T>
T
in a tonic::Request
source§impl<T> Pointable for T
impl<T> Pointable for T
source§impl<T> ProgressEventTimestamp for T
impl<T> ProgressEventTimestamp for T
source§impl<P, R> ProtoType<R> for Pwhere
R: RustType<P>,
impl<P, R> ProtoType<R> for Pwhere
R: RustType<P>,
source§fn into_rust(self) -> Result<R, TryFromProtoError>
fn into_rust(self) -> Result<R, TryFromProtoError>
RustType::from_proto
.source§fn from_rust(rust: &R) -> P
fn from_rust(rust: &R) -> P
RustType::into_proto
.source§impl<'a, S, T> Semigroup<&'a S> for Twhere
T: Semigroup<S>,
impl<'a, S, T> Semigroup<&'a S> for Twhere
T: Semigroup<S>,
source§fn plus_equals(&mut self, rhs: &&'a S)
fn plus_equals(&mut self, rhs: &&'a S)
std::ops::AddAssign
, for types that do not implement AddAssign
.