Struct mz_compute_client::controller::CollectionState
source · pub struct CollectionState<T: Timestamp> {
log_collection: bool,
dropped: bool,
scheduled: bool,
read_capabilities: MutableAntichain<T>,
implied_read_hold: ReadHold<T>,
warmup_read_hold: ReadHold<T>,
read_policy: Option<ReadPolicy<T>>,
storage_dependencies: BTreeMap<GlobalId, ReadHold<T>>,
compute_dependencies: BTreeMap<GlobalId, ReadHold<T>>,
write_frontier: Antichain<T>,
collection_introspection: CollectionIntrospection<T>,
}
Expand description
State maintained about individual compute collections.
A compute collection is either an index, or a storage sink, or a subscribe, exported by a compute dataflow.
Fields§
§log_collection: bool
Whether this collection is a log collection.
Log collections are special in that they are only maintained by a subset of all replicas.
dropped: bool
Whether this collection has been dropped by a controller client.
The controller is allowed to remove the CollectionState
for a collection only when
dropped == true
. Otherwise, clients might still expect to be able to query information
about this collection.
scheduled: bool
Whether this collection has been scheduled, i.e., the controller has sent a Schedule
command for it.
read_capabilities: MutableAntichain<T>
Accumulation of read capabilities for the collection.
This accumulation contains the capabilities held by all ReadHold
s given out for the
collection, including implied_read_hold
and warmup_read_hold
.
NOTE: This field may only be modified by Instance::apply_read_hold_changes
and
Instance::acquire_read_hold
. Nobody else should modify read capabilities directly.
Instead, collection users should manage read holds through ReadHold
objects acquired
through Instance::acquire_read_hold
.
TODO(teskje): Restructure the code to enforce the above in the type system.
implied_read_hold: ReadHold<T>
A read hold maintaining the implicit capability of the collection.
This capability is kept to ensure that the collection remains readable according to its
read_policy
. It also ensures that read holds on the collection’s dependencies are kept at
some time not greater than the collection’s write_frontier
, guaranteeing that the
collection’s next outputs can always be computed without skipping times.
warmup_read_hold: ReadHold<T>
A read hold held to enable dataflow warmup.
Dataflow warmup is an optimization that allows dataflows to immediately start hydrating
even when their next output time (as implied by the write_frontier
) is in the future.
By installing a read capability derived from the write frontiers of the collection’s
inputs, we ensure that the as-of of new dataflows installed for the collection is at a time
that is immediately available, so hydration can begin immediately too.
read_policy: Option<ReadPolicy<T>>
The policy to use to downgrade self.implied_read_hold
.
If None
, the collection is a write-only collection (i.e. a sink). For write-only
collections, the implied_read_hold
is only required for maintaining read holds on the
inputs, so we can immediately downgrade it to the write_frontier
.
storage_dependencies: BTreeMap<GlobalId, ReadHold<T>>
Storage identifiers on which this collection depends, and read holds this collection requires on them.
compute_dependencies: BTreeMap<GlobalId, ReadHold<T>>
Compute identifiers on which this collection depends, and read holds this collection requires on them.
write_frontier: Antichain<T>
The write frontier of this collection.
collection_introspection: CollectionIntrospection<T>
Introspection state associated with this collection.
Implementations§
source§impl<T: Timestamp> CollectionState<T>
impl<T: Timestamp> CollectionState<T>
sourcepub fn read_capability(&self) -> &Antichain<T>
pub fn read_capability(&self) -> &Antichain<T>
Reports the current read capability.
sourcepub fn read_frontier(&self) -> AntichainRef<'_, T>
pub fn read_frontier(&self) -> AntichainRef<'_, T>
Reports the current read frontier.
sourcepub fn write_frontier(&self) -> AntichainRef<'_, T>
pub fn write_frontier(&self) -> AntichainRef<'_, T>
Reports the current write frontier.
fn storage_dependency_ids(&self) -> impl Iterator<Item = GlobalId> + '_
fn compute_dependency_ids(&self) -> impl Iterator<Item = GlobalId> + '_
sourcefn dependency_ids(&self) -> impl Iterator<Item = GlobalId> + '_
fn dependency_ids(&self) -> impl Iterator<Item = GlobalId> + '_
Reports the IDs of the dependencies of this collection.
source§impl<T: ComputeControllerTimestamp> CollectionState<T>
impl<T: ComputeControllerTimestamp> CollectionState<T>
sourcepub(crate) fn new(
collection_id: GlobalId,
as_of: Antichain<T>,
storage_dependencies: BTreeMap<GlobalId, ReadHold<T>>,
compute_dependencies: BTreeMap<GlobalId, ReadHold<T>>,
read_holds_tx: UnboundedSender<(GlobalId, ChangeBatch<T>)>,
introspection_tx: Sender<(IntrospectionType, Vec<(Row, Diff)>)>,
initial_as_of: Option<Antichain<T>>,
refresh_schedule: Option<RefreshSchedule>,
) -> Self
pub(crate) fn new( collection_id: GlobalId, as_of: Antichain<T>, storage_dependencies: BTreeMap<GlobalId, ReadHold<T>>, compute_dependencies: BTreeMap<GlobalId, ReadHold<T>>, read_holds_tx: UnboundedSender<(GlobalId, ChangeBatch<T>)>, introspection_tx: Sender<(IntrospectionType, Vec<(Row, Diff)>)>, initial_as_of: Option<Antichain<T>>, refresh_schedule: Option<RefreshSchedule>, ) -> Self
Creates a new collection state, with an initial read policy valid from since
.
sourcepub(crate) fn new_log_collection(
id: GlobalId,
read_holds_tx: UnboundedSender<(GlobalId, ChangeBatch<T>)>,
introspection_tx: Sender<(IntrospectionType, Vec<(Row, Diff)>)>,
) -> Self
pub(crate) fn new_log_collection( id: GlobalId, read_holds_tx: UnboundedSender<(GlobalId, ChangeBatch<T>)>, introspection_tx: Sender<(IntrospectionType, Vec<(Row, Diff)>)>, ) -> Self
Creates a new collection state for a log collection.
Trait Implementations§
Auto Trait Implementations§
impl<T> Freeze for CollectionState<T>
impl<T> !RefUnwindSafe for CollectionState<T>
impl<T> Send for CollectionState<T>
impl<T> Sync for CollectionState<T>
impl<T> Unpin for CollectionState<T>where
T: Unpin,
impl<T> !UnwindSafe for CollectionState<T>
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
source§impl<T> FmtForward for T
impl<T> FmtForward for T
source§fn fmt_binary(self) -> FmtBinary<Self>where
Self: Binary,
fn fmt_binary(self) -> FmtBinary<Self>where
Self: Binary,
self
to use its Binary
implementation when Debug
-formatted.source§fn fmt_display(self) -> FmtDisplay<Self>where
Self: Display,
fn fmt_display(self) -> FmtDisplay<Self>where
Self: Display,
self
to use its Display
implementation when
Debug
-formatted.source§fn fmt_lower_exp(self) -> FmtLowerExp<Self>where
Self: LowerExp,
fn fmt_lower_exp(self) -> FmtLowerExp<Self>where
Self: LowerExp,
self
to use its LowerExp
implementation when
Debug
-formatted.source§fn fmt_lower_hex(self) -> FmtLowerHex<Self>where
Self: LowerHex,
fn fmt_lower_hex(self) -> FmtLowerHex<Self>where
Self: LowerHex,
self
to use its LowerHex
implementation when
Debug
-formatted.source§fn fmt_octal(self) -> FmtOctal<Self>where
Self: Octal,
fn fmt_octal(self) -> FmtOctal<Self>where
Self: Octal,
self
to use its Octal
implementation when Debug
-formatted.source§fn fmt_pointer(self) -> FmtPointer<Self>where
Self: Pointer,
fn fmt_pointer(self) -> FmtPointer<Self>where
Self: Pointer,
self
to use its Pointer
implementation when
Debug
-formatted.source§fn fmt_upper_exp(self) -> FmtUpperExp<Self>where
Self: UpperExp,
fn fmt_upper_exp(self) -> FmtUpperExp<Self>where
Self: UpperExp,
self
to use its UpperExp
implementation when
Debug
-formatted.source§fn fmt_upper_hex(self) -> FmtUpperHex<Self>where
Self: UpperHex,
fn fmt_upper_hex(self) -> FmtUpperHex<Self>where
Self: UpperHex,
self
to use its UpperHex
implementation when
Debug
-formatted.source§impl<T> FutureExt for T
impl<T> FutureExt for T
source§fn with_context(self, otel_cx: Context) -> WithContext<Self>
fn with_context(self, otel_cx: Context) -> WithContext<Self>
source§fn with_current_context(self) -> WithContext<Self>
fn with_current_context(self) -> WithContext<Self>
source§impl<T> Instrument for T
impl<T> Instrument for T
source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
source§impl<T> IntoRequest<T> for T
impl<T> IntoRequest<T> for T
source§fn into_request(self) -> Request<T>
fn into_request(self) -> Request<T>
T
in a tonic::Request
source§impl<T, U> OverrideFrom<Option<&T>> for Uwhere
U: OverrideFrom<T>,
impl<T, U> OverrideFrom<Option<&T>> for Uwhere
U: OverrideFrom<T>,
source§impl<T> Pipe for Twhere
T: ?Sized,
impl<T> Pipe for Twhere
T: ?Sized,
source§fn pipe<R>(self, func: impl FnOnce(Self) -> R) -> Rwhere
Self: Sized,
fn pipe<R>(self, func: impl FnOnce(Self) -> R) -> Rwhere
Self: Sized,
source§fn pipe_ref<'a, R>(&'a self, func: impl FnOnce(&'a Self) -> R) -> Rwhere
R: 'a,
fn pipe_ref<'a, R>(&'a self, func: impl FnOnce(&'a Self) -> R) -> Rwhere
R: 'a,
self
and passes that borrow into the pipe function. Read moresource§fn pipe_ref_mut<'a, R>(&'a mut self, func: impl FnOnce(&'a mut Self) -> R) -> Rwhere
R: 'a,
fn pipe_ref_mut<'a, R>(&'a mut self, func: impl FnOnce(&'a mut Self) -> R) -> Rwhere
R: 'a,
self
and passes that borrow into the pipe function. Read moresource§fn pipe_borrow<'a, B, R>(&'a self, func: impl FnOnce(&'a B) -> R) -> R
fn pipe_borrow<'a, B, R>(&'a self, func: impl FnOnce(&'a B) -> R) -> R
source§fn pipe_borrow_mut<'a, B, R>(
&'a mut self,
func: impl FnOnce(&'a mut B) -> R,
) -> R
fn pipe_borrow_mut<'a, B, R>( &'a mut self, func: impl FnOnce(&'a mut B) -> R, ) -> R
source§fn pipe_as_ref<'a, U, R>(&'a self, func: impl FnOnce(&'a U) -> R) -> R
fn pipe_as_ref<'a, U, R>(&'a self, func: impl FnOnce(&'a U) -> R) -> R
self
, then passes self.as_ref()
into the pipe function.source§fn pipe_as_mut<'a, U, R>(&'a mut self, func: impl FnOnce(&'a mut U) -> R) -> R
fn pipe_as_mut<'a, U, R>(&'a mut self, func: impl FnOnce(&'a mut U) -> R) -> R
self
, then passes self.as_mut()
into the pipe
function.source§fn pipe_deref<'a, T, R>(&'a self, func: impl FnOnce(&'a T) -> R) -> R
fn pipe_deref<'a, T, R>(&'a self, func: impl FnOnce(&'a T) -> R) -> R
self
, then passes self.deref()
into the pipe function.source§impl<T> Pointable for T
impl<T> Pointable for T
source§impl<P, R> ProtoType<R> for Pwhere
R: RustType<P>,
impl<P, R> ProtoType<R> for Pwhere
R: RustType<P>,
source§fn into_rust(self) -> Result<R, TryFromProtoError>
fn into_rust(self) -> Result<R, TryFromProtoError>
RustType::from_proto
.source§fn from_rust(rust: &R) -> P
fn from_rust(rust: &R) -> P
RustType::into_proto
.source§impl<'a, S, T> Semigroup<&'a S> for Twhere
T: Semigroup<S>,
impl<'a, S, T> Semigroup<&'a S> for Twhere
T: Semigroup<S>,
source§fn plus_equals(&mut self, rhs: &&'a S)
fn plus_equals(&mut self, rhs: &&'a S)
std::ops::AddAssign
, for types that do not implement AddAssign
.source§impl<T> Tap for T
impl<T> Tap for T
source§fn tap_borrow<B>(self, func: impl FnOnce(&B)) -> Self
fn tap_borrow<B>(self, func: impl FnOnce(&B)) -> Self
Borrow<B>
of a value. Read moresource§fn tap_borrow_mut<B>(self, func: impl FnOnce(&mut B)) -> Self
fn tap_borrow_mut<B>(self, func: impl FnOnce(&mut B)) -> Self
BorrowMut<B>
of a value. Read moresource§fn tap_ref<R>(self, func: impl FnOnce(&R)) -> Self
fn tap_ref<R>(self, func: impl FnOnce(&R)) -> Self
AsRef<R>
view of a value. Read moresource§fn tap_ref_mut<R>(self, func: impl FnOnce(&mut R)) -> Self
fn tap_ref_mut<R>(self, func: impl FnOnce(&mut R)) -> Self
AsMut<R>
view of a value. Read moresource§fn tap_deref<T>(self, func: impl FnOnce(&T)) -> Self
fn tap_deref<T>(self, func: impl FnOnce(&T)) -> Self
Deref::Target
of a value. Read moresource§fn tap_deref_mut<T>(self, func: impl FnOnce(&mut T)) -> Self
fn tap_deref_mut<T>(self, func: impl FnOnce(&mut T)) -> Self
Deref::Target
of a value. Read moresource§fn tap_dbg(self, func: impl FnOnce(&Self)) -> Self
fn tap_dbg(self, func: impl FnOnce(&Self)) -> Self
.tap()
only in debug builds, and is erased in release builds.source§fn tap_mut_dbg(self, func: impl FnOnce(&mut Self)) -> Self
fn tap_mut_dbg(self, func: impl FnOnce(&mut Self)) -> Self
.tap_mut()
only in debug builds, and is erased in release
builds.source§fn tap_borrow_dbg<B>(self, func: impl FnOnce(&B)) -> Self
fn tap_borrow_dbg<B>(self, func: impl FnOnce(&B)) -> Self
.tap_borrow()
only in debug builds, and is erased in release
builds.source§fn tap_borrow_mut_dbg<B>(self, func: impl FnOnce(&mut B)) -> Self
fn tap_borrow_mut_dbg<B>(self, func: impl FnOnce(&mut B)) -> Self
.tap_borrow_mut()
only in debug builds, and is erased in release
builds.source§fn tap_ref_dbg<R>(self, func: impl FnOnce(&R)) -> Self
fn tap_ref_dbg<R>(self, func: impl FnOnce(&R)) -> Self
.tap_ref()
only in debug builds, and is erased in release
builds.source§fn tap_ref_mut_dbg<R>(self, func: impl FnOnce(&mut R)) -> Self
fn tap_ref_mut_dbg<R>(self, func: impl FnOnce(&mut R)) -> Self
.tap_ref_mut()
only in debug builds, and is erased in release
builds.source§fn tap_deref_dbg<T>(self, func: impl FnOnce(&T)) -> Self
fn tap_deref_dbg<T>(self, func: impl FnOnce(&T)) -> Self
.tap_deref()
only in debug builds, and is erased in release
builds.