Struct mz_storage_types::sources::IngestionDescription
source · pub struct IngestionDescription<S: 'static = (), C: ConnectionAccess = InlinedConnection> {
pub desc: SourceDesc<C>,
pub ingestion_metadata: S,
pub source_exports: BTreeMap<GlobalId, SourceExport<S>>,
pub instance_id: StorageInstanceId,
pub remap_collection_id: GlobalId,
}
Expand description
A description of a source ingestion
Fields§
§desc: SourceDesc<C>
The source description.
§Warning
Any time this field changes, you must recalculate the SourceExport
values in Self::source_exports
.
ingestion_metadata: S
Additional storage controller metadata needed to ingest this source
source_exports: BTreeMap<GlobalId, SourceExport<S>>
Collections to be exported by this ingestion.
§Notes
-
For multi-output sources:
- Add subsources by adding a new
SourceExport
. Look up the appropriate output index usingSourceConnection::output_idx_for_name
, which is available throughSourceDesc::connection
. - Remove subsources by removing the
SourceExport
.
Re-rendering/executing the source after making these modifications adds and drops the subsource, respectively.
- Add subsources by adding a new
-
Any time the
Self::desc
field changes, you must recalculate theSourceExport
values. -
This field includes the primary source’s ID, which might need to be filtered out to understand which exports are ingestion export subsources.
-
This field does not include the remap collection, which is tracked in its own field.
-
This field should not be populated by the storage controller in response to collections being created.
instance_id: StorageInstanceId
The ID of the instance in which to install the source.
remap_collection_id: GlobalId
The ID of this ingestion’s remap/progress collection.
Implementations§
source§impl<S> IngestionDescription<S>
impl<S> IngestionDescription<S>
sourcepub fn subsource_ids(&self) -> impl Iterator<Item = GlobalId> + '_
pub fn subsource_ids(&self) -> impl Iterator<Item = GlobalId> + '_
Return an iterator over the GlobalId
s of self
’s subsources.
Trait Implementations§
source§impl<S: Debug + Eq + PartialEq + AlterCompatible> AlterCompatible for IngestionDescription<S>
impl<S: Debug + Eq + PartialEq + AlterCompatible> AlterCompatible for IngestionDescription<S>
fn alter_compatible( &self, id: GlobalId, other: &IngestionDescription<S> ) -> Result<(), AlterError>
source§impl<S: 'static + Arbitrary, C: ConnectionAccess + Arbitrary> Arbitrary for IngestionDescription<S, C>
impl<S: 'static + Arbitrary, C: ConnectionAccess + Arbitrary> Arbitrary for IngestionDescription<S, C>
§type Parameters = (<SourceDesc<C> as Arbitrary>::Parameters, <S as Arbitrary>::Parameters, <StorageInstanceId as Arbitrary>::Parameters, <GlobalId as Arbitrary>::Parameters)
type Parameters = (<SourceDesc<C> as Arbitrary>::Parameters, <S as Arbitrary>::Parameters, <StorageInstanceId as Arbitrary>::Parameters, <GlobalId as Arbitrary>::Parameters)
arbitrary_with
accepts for configuration
of the generated Strategy
. Parameters must implement Default
.§type Strategy = Map<(<SourceDesc<C> as Arbitrary>::Strategy, <S as Arbitrary>::Strategy, BoxedStrategy<BTreeMap<GlobalId, SourceExport<S>>>, <StorageInstanceId as Arbitrary>::Strategy, <GlobalId as Arbitrary>::Strategy), fn(_: (SourceDesc<C>, S, BTreeMap<GlobalId, SourceExport<S>>, StorageInstanceId, GlobalId)) -> IngestionDescription<S, C>>
type Strategy = Map<(<SourceDesc<C> as Arbitrary>::Strategy, <S as Arbitrary>::Strategy, BoxedStrategy<BTreeMap<GlobalId, SourceExport<S>>>, <StorageInstanceId as Arbitrary>::Strategy, <GlobalId as Arbitrary>::Strategy), fn(_: (SourceDesc<C>, S, BTreeMap<GlobalId, SourceExport<S>>, StorageInstanceId, GlobalId)) -> IngestionDescription<S, C>>
Strategy
used to generate values of type Self
.source§fn arbitrary_with(_top: Self::Parameters) -> Self::Strategy
fn arbitrary_with(_top: Self::Parameters) -> Self::Strategy
source§impl<S: Clone + 'static, C: Clone + ConnectionAccess> Clone for IngestionDescription<S, C>
impl<S: Clone + 'static, C: Clone + ConnectionAccess> Clone for IngestionDescription<S, C>
source§fn clone(&self) -> IngestionDescription<S, C>
fn clone(&self) -> IngestionDescription<S, C>
1.0.0 · source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source
. Read moresource§impl<S: Debug + 'static, C: Debug + ConnectionAccess> Debug for IngestionDescription<S, C>
impl<S: Debug + 'static, C: Debug + ConnectionAccess> Debug for IngestionDescription<S, C>
source§impl<'de, S, C> Deserialize<'de> for IngestionDescription<S, C>
impl<'de, S, C> Deserialize<'de> for IngestionDescription<S, C>
source§fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
source§impl<R: ConnectionResolver> IntoInlineConnection<IngestionDescription, R> for IngestionDescription<(), ReferencedConnection>
impl<R: ConnectionResolver> IntoInlineConnection<IngestionDescription, R> for IngestionDescription<(), ReferencedConnection>
fn into_inline_connection(self, r: R) -> IngestionDescription
source§impl<S: PartialEq + 'static, C: PartialEq + ConnectionAccess> PartialEq for IngestionDescription<S, C>
impl<S: PartialEq + 'static, C: PartialEq + ConnectionAccess> PartialEq for IngestionDescription<S, C>
source§fn eq(&self, other: &IngestionDescription<S, C>) -> bool
fn eq(&self, other: &IngestionDescription<S, C>) -> bool
self
and other
values to be equal, and is used
by ==
.source§impl RustType<ProtoIngestionDescription> for IngestionDescription<CollectionMetadata>
impl RustType<ProtoIngestionDescription> for IngestionDescription<CollectionMetadata>
source§fn into_proto(&self) -> ProtoIngestionDescription
fn into_proto(&self) -> ProtoIngestionDescription
Self
into a Proto
value.source§fn from_proto(
proto: ProtoIngestionDescription
) -> Result<Self, TryFromProtoError>
fn from_proto( proto: ProtoIngestionDescription ) -> Result<Self, TryFromProtoError>
source§impl<S, C> Serialize for IngestionDescription<S, C>
impl<S, C> Serialize for IngestionDescription<S, C>
impl<S: Eq + 'static, C: Eq + ConnectionAccess> Eq for IngestionDescription<S, C>
impl<S: 'static, C: ConnectionAccess> StructuralPartialEq for IngestionDescription<S, C>
Auto Trait Implementations§
impl<S, C> Freeze for IngestionDescription<S, C>where
S: Freeze,
<C as ConnectionAccess>::Kafka: Freeze,
<C as ConnectionAccess>::Pg: Freeze,
<C as ConnectionAccess>::MySql: Freeze,
<C as ConnectionAccess>::Csr: Freeze,
impl<S, C> RefUnwindSafe for IngestionDescription<S, C>where
S: RefUnwindSafe,
<C as ConnectionAccess>::Kafka: RefUnwindSafe,
<C as ConnectionAccess>::Pg: RefUnwindSafe,
<C as ConnectionAccess>::MySql: RefUnwindSafe,
<C as ConnectionAccess>::Csr: RefUnwindSafe,
impl<S, C> Send for IngestionDescription<S, C>where
S: Send,
<C as ConnectionAccess>::Kafka: Send,
<C as ConnectionAccess>::Pg: Send,
<C as ConnectionAccess>::MySql: Send,
<C as ConnectionAccess>::Csr: Send,
impl<S, C> Sync for IngestionDescription<S, C>where
S: Sync,
<C as ConnectionAccess>::Kafka: Sync,
<C as ConnectionAccess>::Pg: Sync,
<C as ConnectionAccess>::MySql: Sync,
<C as ConnectionAccess>::Csr: Sync,
impl<S, C> Unpin for IngestionDescription<S, C>where
S: Unpin,
<C as ConnectionAccess>::Kafka: Unpin,
<C as ConnectionAccess>::Pg: Unpin,
<C as ConnectionAccess>::MySql: Unpin,
<C as ConnectionAccess>::Csr: Unpin,
impl<S, C> UnwindSafe for IngestionDescription<S, C>where
S: UnwindSafe + RefUnwindSafe,
<C as ConnectionAccess>::Kafka: UnwindSafe,
<C as ConnectionAccess>::Pg: UnwindSafe,
<C as ConnectionAccess>::MySql: UnwindSafe,
<C as ConnectionAccess>::Csr: UnwindSafe,
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
source§impl<R, O, T> CopyOnto<ConsecutiveOffsetPairs<R, O>> for T
impl<R, O, T> CopyOnto<ConsecutiveOffsetPairs<R, O>> for T
source§fn copy_onto(
self,
target: &mut ConsecutiveOffsetPairs<R, O>
) -> <ConsecutiveOffsetPairs<R, O> as Region>::Index
fn copy_onto( self, target: &mut ConsecutiveOffsetPairs<R, O> ) -> <ConsecutiveOffsetPairs<R, O> as Region>::Index
source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
source§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
key
and return true
if they are equal.source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
source§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
key
and return true
if they are equal.source§impl<T> FmtForward for T
impl<T> FmtForward for T
source§fn fmt_binary(self) -> FmtBinary<Self>where
Self: Binary,
fn fmt_binary(self) -> FmtBinary<Self>where
Self: Binary,
self
to use its Binary
implementation when Debug
-formatted.source§fn fmt_display(self) -> FmtDisplay<Self>where
Self: Display,
fn fmt_display(self) -> FmtDisplay<Self>where
Self: Display,
self
to use its Display
implementation when
Debug
-formatted.source§fn fmt_lower_exp(self) -> FmtLowerExp<Self>where
Self: LowerExp,
fn fmt_lower_exp(self) -> FmtLowerExp<Self>where
Self: LowerExp,
self
to use its LowerExp
implementation when
Debug
-formatted.source§fn fmt_lower_hex(self) -> FmtLowerHex<Self>where
Self: LowerHex,
fn fmt_lower_hex(self) -> FmtLowerHex<Self>where
Self: LowerHex,
self
to use its LowerHex
implementation when
Debug
-formatted.source§fn fmt_octal(self) -> FmtOctal<Self>where
Self: Octal,
fn fmt_octal(self) -> FmtOctal<Self>where
Self: Octal,
self
to use its Octal
implementation when Debug
-formatted.source§fn fmt_pointer(self) -> FmtPointer<Self>where
Self: Pointer,
fn fmt_pointer(self) -> FmtPointer<Self>where
Self: Pointer,
self
to use its Pointer
implementation when
Debug
-formatted.source§fn fmt_upper_exp(self) -> FmtUpperExp<Self>where
Self: UpperExp,
fn fmt_upper_exp(self) -> FmtUpperExp<Self>where
Self: UpperExp,
self
to use its UpperExp
implementation when
Debug
-formatted.source§fn fmt_upper_hex(self) -> FmtUpperHex<Self>where
Self: UpperHex,
fn fmt_upper_hex(self) -> FmtUpperHex<Self>where
Self: UpperHex,
self
to use its UpperHex
implementation when
Debug
-formatted.source§impl<T> FutureExt for T
impl<T> FutureExt for T
source§fn with_context(self, otel_cx: Context) -> WithContext<Self>
fn with_context(self, otel_cx: Context) -> WithContext<Self>
source§fn with_current_context(self) -> WithContext<Self>
fn with_current_context(self) -> WithContext<Self>
source§impl<T> Instrument for T
impl<T> Instrument for T
source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
source§impl<T> IntoRequest<T> for T
impl<T> IntoRequest<T> for T
source§fn into_request(self) -> Request<T>
fn into_request(self) -> Request<T>
T
in a tonic::Request
source§impl<T, U> OverrideFrom<Option<&T>> for Uwhere
U: OverrideFrom<T>,
impl<T, U> OverrideFrom<Option<&T>> for Uwhere
U: OverrideFrom<T>,
source§impl<T> Pipe for Twhere
T: ?Sized,
impl<T> Pipe for Twhere
T: ?Sized,
source§fn pipe<R>(self, func: impl FnOnce(Self) -> R) -> Rwhere
Self: Sized,
fn pipe<R>(self, func: impl FnOnce(Self) -> R) -> Rwhere
Self: Sized,
source§fn pipe_ref<'a, R>(&'a self, func: impl FnOnce(&'a Self) -> R) -> Rwhere
R: 'a,
fn pipe_ref<'a, R>(&'a self, func: impl FnOnce(&'a Self) -> R) -> Rwhere
R: 'a,
self
and passes that borrow into the pipe function. Read moresource§fn pipe_ref_mut<'a, R>(&'a mut self, func: impl FnOnce(&'a mut Self) -> R) -> Rwhere
R: 'a,
fn pipe_ref_mut<'a, R>(&'a mut self, func: impl FnOnce(&'a mut Self) -> R) -> Rwhere
R: 'a,
self
and passes that borrow into the pipe function. Read moresource§fn pipe_borrow<'a, B, R>(&'a self, func: impl FnOnce(&'a B) -> R) -> R
fn pipe_borrow<'a, B, R>(&'a self, func: impl FnOnce(&'a B) -> R) -> R
source§fn pipe_borrow_mut<'a, B, R>(
&'a mut self,
func: impl FnOnce(&'a mut B) -> R
) -> R
fn pipe_borrow_mut<'a, B, R>( &'a mut self, func: impl FnOnce(&'a mut B) -> R ) -> R
source§fn pipe_as_ref<'a, U, R>(&'a self, func: impl FnOnce(&'a U) -> R) -> R
fn pipe_as_ref<'a, U, R>(&'a self, func: impl FnOnce(&'a U) -> R) -> R
self
, then passes self.as_ref()
into the pipe function.source§fn pipe_as_mut<'a, U, R>(&'a mut self, func: impl FnOnce(&'a mut U) -> R) -> R
fn pipe_as_mut<'a, U, R>(&'a mut self, func: impl FnOnce(&'a mut U) -> R) -> R
self
, then passes self.as_mut()
into the pipe
function.source§fn pipe_deref<'a, T, R>(&'a self, func: impl FnOnce(&'a T) -> R) -> R
fn pipe_deref<'a, T, R>(&'a self, func: impl FnOnce(&'a T) -> R) -> R
self
, then passes self.deref()
into the pipe function.source§impl<T> Pointable for T
impl<T> Pointable for T
source§impl<T> ProgressEventTimestamp for T
impl<T> ProgressEventTimestamp for T
source§impl<P, R> ProtoType<R> for Pwhere
R: RustType<P>,
impl<P, R> ProtoType<R> for Pwhere
R: RustType<P>,
source§fn into_rust(self) -> Result<R, TryFromProtoError>
fn into_rust(self) -> Result<R, TryFromProtoError>
RustType::from_proto
.source§fn from_rust(rust: &R) -> P
fn from_rust(rust: &R) -> P
RustType::into_proto
.source§impl<T> Tap for T
impl<T> Tap for T
source§fn tap_borrow<B>(self, func: impl FnOnce(&B)) -> Self
fn tap_borrow<B>(self, func: impl FnOnce(&B)) -> Self
Borrow<B>
of a value. Read moresource§fn tap_borrow_mut<B>(self, func: impl FnOnce(&mut B)) -> Self
fn tap_borrow_mut<B>(self, func: impl FnOnce(&mut B)) -> Self
BorrowMut<B>
of a value. Read moresource§fn tap_ref<R>(self, func: impl FnOnce(&R)) -> Self
fn tap_ref<R>(self, func: impl FnOnce(&R)) -> Self
AsRef<R>
view of a value. Read moresource§fn tap_ref_mut<R>(self, func: impl FnOnce(&mut R)) -> Self
fn tap_ref_mut<R>(self, func: impl FnOnce(&mut R)) -> Self
AsMut<R>
view of a value. Read moresource§fn tap_deref<T>(self, func: impl FnOnce(&T)) -> Self
fn tap_deref<T>(self, func: impl FnOnce(&T)) -> Self
Deref::Target
of a value. Read moresource§fn tap_deref_mut<T>(self, func: impl FnOnce(&mut T)) -> Self
fn tap_deref_mut<T>(self, func: impl FnOnce(&mut T)) -> Self
Deref::Target
of a value. Read moresource§fn tap_dbg(self, func: impl FnOnce(&Self)) -> Self
fn tap_dbg(self, func: impl FnOnce(&Self)) -> Self
.tap()
only in debug builds, and is erased in release builds.source§fn tap_mut_dbg(self, func: impl FnOnce(&mut Self)) -> Self
fn tap_mut_dbg(self, func: impl FnOnce(&mut Self)) -> Self
.tap_mut()
only in debug builds, and is erased in release
builds.source§fn tap_borrow_dbg<B>(self, func: impl FnOnce(&B)) -> Self
fn tap_borrow_dbg<B>(self, func: impl FnOnce(&B)) -> Self
.tap_borrow()
only in debug builds, and is erased in release
builds.source§fn tap_borrow_mut_dbg<B>(self, func: impl FnOnce(&mut B)) -> Self
fn tap_borrow_mut_dbg<B>(self, func: impl FnOnce(&mut B)) -> Self
.tap_borrow_mut()
only in debug builds, and is erased in release
builds.source§fn tap_ref_dbg<R>(self, func: impl FnOnce(&R)) -> Self
fn tap_ref_dbg<R>(self, func: impl FnOnce(&R)) -> Self
.tap_ref()
only in debug builds, and is erased in release
builds.source§fn tap_ref_mut_dbg<R>(self, func: impl FnOnce(&mut R)) -> Self
fn tap_ref_mut_dbg<R>(self, func: impl FnOnce(&mut R)) -> Self
.tap_ref_mut()
only in debug builds, and is erased in release
builds.source§fn tap_deref_dbg<T>(self, func: impl FnOnce(&T)) -> Self
fn tap_deref_dbg<T>(self, func: impl FnOnce(&T)) -> Self
.tap_deref()
only in debug builds, and is erased in release
builds.