Struct mz_storage_types::sources::IngestionDescription
source · pub struct IngestionDescription<S: 'static = (), C: ConnectionAccess = InlinedConnection> {
pub desc: SourceDesc<C>,
pub ingestion_metadata: S,
pub source_exports: BTreeMap<GlobalId, SourceExport<S>>,
pub instance_id: StorageInstanceId,
pub remap_collection_id: GlobalId,
}
Expand description
A description of a source ingestion
Fields§
§desc: SourceDesc<C>
The source description.
ingestion_metadata: S
Additional storage controller metadata needed to ingest this source
source_exports: BTreeMap<GlobalId, SourceExport<S>>
Collections to be exported by this ingestion.
§Notes
-
For multi-output sources:
- Add exports by adding a new
SourceExport
. - Remove exports by removing the
SourceExport
.
Re-rendering/executing the source after making these modifications adds and drops the subsource, respectively.
- Add exports by adding a new
-
This field includes the primary source’s ID, which might need to be filtered out to understand which exports are explicit ingestion exports.
-
This field does not include the remap collection, which is tracked in its own field.
instance_id: StorageInstanceId
The ID of the instance in which to install the source.
remap_collection_id: GlobalId
The ID of this ingestion’s remap/progress collection.
Implementations§
source§impl IngestionDescription
impl IngestionDescription
pub fn new( desc: SourceDesc, instance_id: StorageInstanceId, remap_collection_id: GlobalId, ) -> Self
source§impl<S> IngestionDescription<S>
impl<S> IngestionDescription<S>
sourcepub fn collection_ids(&self) -> impl Iterator<Item = GlobalId> + '_
pub fn collection_ids(&self) -> impl Iterator<Item = GlobalId> + '_
Return an iterator over the GlobalId
s of self
’s collections.
This will contain ids for the remap collection, subsources,
tables for this source, and the primary collection ID, even if
no data will be exported to the primary collection.
source§impl<S: Clone> IngestionDescription<S>
impl<S: Clone> IngestionDescription<S>
pub fn indexed_source_exports( &self, primary_source_id: &GlobalId, ) -> BTreeMap<GlobalId, IndexedSourceExport<S>>
Trait Implementations§
source§impl<S: Debug + Eq + PartialEq + AlterCompatible> AlterCompatible for IngestionDescription<S>
impl<S: Debug + Eq + PartialEq + AlterCompatible> AlterCompatible for IngestionDescription<S>
fn alter_compatible( &self, id: GlobalId, other: &IngestionDescription<S>, ) -> Result<(), AlterError>
source§impl<S: 'static + Arbitrary, C: ConnectionAccess + Arbitrary> Arbitrary for IngestionDescription<S, C>
impl<S: 'static + Arbitrary, C: ConnectionAccess + Arbitrary> Arbitrary for IngestionDescription<S, C>
§type Parameters = (<SourceDesc<C> as Arbitrary>::Parameters, <S as Arbitrary>::Parameters, <StorageInstanceId as Arbitrary>::Parameters, <GlobalId as Arbitrary>::Parameters)
type Parameters = (<SourceDesc<C> as Arbitrary>::Parameters, <S as Arbitrary>::Parameters, <StorageInstanceId as Arbitrary>::Parameters, <GlobalId as Arbitrary>::Parameters)
arbitrary_with
accepts for configuration
of the generated Strategy
. Parameters must implement Default
.§type Strategy = Map<(<SourceDesc<C> as Arbitrary>::Strategy, <S as Arbitrary>::Strategy, BoxedStrategy<BTreeMap<GlobalId, SourceExport<S>>>, <StorageInstanceId as Arbitrary>::Strategy, <GlobalId as Arbitrary>::Strategy), fn(_: (SourceDesc<C>, S, BTreeMap<GlobalId, SourceExport<S>>, StorageInstanceId, GlobalId)) -> IngestionDescription<S, C>>
type Strategy = Map<(<SourceDesc<C> as Arbitrary>::Strategy, <S as Arbitrary>::Strategy, BoxedStrategy<BTreeMap<GlobalId, SourceExport<S>>>, <StorageInstanceId as Arbitrary>::Strategy, <GlobalId as Arbitrary>::Strategy), fn(_: (SourceDesc<C>, S, BTreeMap<GlobalId, SourceExport<S>>, StorageInstanceId, GlobalId)) -> IngestionDescription<S, C>>
Strategy
used to generate values of type Self
.source§fn arbitrary_with(_top: Self::Parameters) -> Self::Strategy
fn arbitrary_with(_top: Self::Parameters) -> Self::Strategy
source§impl<S: Clone + 'static, C: Clone + ConnectionAccess> Clone for IngestionDescription<S, C>
impl<S: Clone + 'static, C: Clone + ConnectionAccess> Clone for IngestionDescription<S, C>
source§fn clone(&self) -> IngestionDescription<S, C>
fn clone(&self) -> IngestionDescription<S, C>
1.0.0 · source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source
. Read moresource§impl<S: Debug + 'static, C: Debug + ConnectionAccess> Debug for IngestionDescription<S, C>
impl<S: Debug + 'static, C: Debug + ConnectionAccess> Debug for IngestionDescription<S, C>
source§impl<'de, S, C> Deserialize<'de> for IngestionDescription<S, C>
impl<'de, S, C> Deserialize<'de> for IngestionDescription<S, C>
source§fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
source§impl<R: ConnectionResolver> IntoInlineConnection<IngestionDescription, R> for IngestionDescription<(), ReferencedConnection>
impl<R: ConnectionResolver> IntoInlineConnection<IngestionDescription, R> for IngestionDescription<(), ReferencedConnection>
fn into_inline_connection(self, r: R) -> IngestionDescription
source§impl<S: PartialEq + 'static, C: PartialEq + ConnectionAccess> PartialEq for IngestionDescription<S, C>
impl<S: PartialEq + 'static, C: PartialEq + ConnectionAccess> PartialEq for IngestionDescription<S, C>
source§impl RustType<ProtoIngestionDescription> for IngestionDescription<CollectionMetadata>
impl RustType<ProtoIngestionDescription> for IngestionDescription<CollectionMetadata>
source§fn into_proto(&self) -> ProtoIngestionDescription
fn into_proto(&self) -> ProtoIngestionDescription
Self
into a Proto
value.source§fn from_proto(
proto: ProtoIngestionDescription,
) -> Result<Self, TryFromProtoError>
fn from_proto( proto: ProtoIngestionDescription, ) -> Result<Self, TryFromProtoError>
source§fn into_proto_owned(self) -> Proto
fn into_proto_owned(self) -> Proto
Self::into_proto
that types can
optionally implement, otherwise, the default implementation
delegates to Self::into_proto
.source§impl<S, C> Serialize for IngestionDescription<S, C>
impl<S, C> Serialize for IngestionDescription<S, C>
impl<S: Eq + 'static, C: Eq + ConnectionAccess> Eq for IngestionDescription<S, C>
impl<S: 'static, C: ConnectionAccess> StructuralPartialEq for IngestionDescription<S, C>
Auto Trait Implementations§
impl<S, C> Freeze for IngestionDescription<S, C>where
S: Freeze,
<C as ConnectionAccess>::Kafka: Freeze,
<C as ConnectionAccess>::Pg: Freeze,
<C as ConnectionAccess>::MySql: Freeze,
<C as ConnectionAccess>::Csr: Freeze,
impl<S, C> RefUnwindSafe for IngestionDescription<S, C>where
S: RefUnwindSafe,
<C as ConnectionAccess>::Kafka: RefUnwindSafe,
<C as ConnectionAccess>::Pg: RefUnwindSafe,
<C as ConnectionAccess>::MySql: RefUnwindSafe,
<C as ConnectionAccess>::Csr: RefUnwindSafe,
impl<S, C> Send for IngestionDescription<S, C>where
S: Send,
<C as ConnectionAccess>::Kafka: Send,
<C as ConnectionAccess>::Pg: Send,
<C as ConnectionAccess>::MySql: Send,
<C as ConnectionAccess>::Csr: Send,
impl<S, C> Sync for IngestionDescription<S, C>where
S: Sync,
<C as ConnectionAccess>::Kafka: Sync,
<C as ConnectionAccess>::Pg: Sync,
<C as ConnectionAccess>::MySql: Sync,
<C as ConnectionAccess>::Csr: Sync,
impl<S, C> Unpin for IngestionDescription<S, C>where
S: Unpin,
<C as ConnectionAccess>::Kafka: Unpin,
<C as ConnectionAccess>::Pg: Unpin,
<C as ConnectionAccess>::MySql: Unpin,
<C as ConnectionAccess>::Csr: Unpin,
impl<S, C> UnwindSafe for IngestionDescription<S, C>where
S: UnwindSafe + RefUnwindSafe,
<C as ConnectionAccess>::Kafka: UnwindSafe,
<C as ConnectionAccess>::Pg: UnwindSafe,
<C as ConnectionAccess>::MySql: UnwindSafe,
<C as ConnectionAccess>::Csr: UnwindSafe,
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
source§default unsafe fn clone_to_uninit(&self, dst: *mut T)
default unsafe fn clone_to_uninit(&self, dst: *mut T)
clone_to_uninit
)source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
source§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
key
and return true
if they are equal.source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
source§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
key
and return true
if they are equal.source§impl<T> FmtForward for T
impl<T> FmtForward for T
source§fn fmt_binary(self) -> FmtBinary<Self>where
Self: Binary,
fn fmt_binary(self) -> FmtBinary<Self>where
Self: Binary,
self
to use its Binary
implementation when Debug
-formatted.source§fn fmt_display(self) -> FmtDisplay<Self>where
Self: Display,
fn fmt_display(self) -> FmtDisplay<Self>where
Self: Display,
self
to use its Display
implementation when
Debug
-formatted.source§fn fmt_lower_exp(self) -> FmtLowerExp<Self>where
Self: LowerExp,
fn fmt_lower_exp(self) -> FmtLowerExp<Self>where
Self: LowerExp,
self
to use its LowerExp
implementation when
Debug
-formatted.source§fn fmt_lower_hex(self) -> FmtLowerHex<Self>where
Self: LowerHex,
fn fmt_lower_hex(self) -> FmtLowerHex<Self>where
Self: LowerHex,
self
to use its LowerHex
implementation when
Debug
-formatted.source§fn fmt_octal(self) -> FmtOctal<Self>where
Self: Octal,
fn fmt_octal(self) -> FmtOctal<Self>where
Self: Octal,
self
to use its Octal
implementation when Debug
-formatted.source§fn fmt_pointer(self) -> FmtPointer<Self>where
Self: Pointer,
fn fmt_pointer(self) -> FmtPointer<Self>where
Self: Pointer,
self
to use its Pointer
implementation when
Debug
-formatted.source§fn fmt_upper_exp(self) -> FmtUpperExp<Self>where
Self: UpperExp,
fn fmt_upper_exp(self) -> FmtUpperExp<Self>where
Self: UpperExp,
self
to use its UpperExp
implementation when
Debug
-formatted.source§fn fmt_upper_hex(self) -> FmtUpperHex<Self>where
Self: UpperHex,
fn fmt_upper_hex(self) -> FmtUpperHex<Self>where
Self: UpperHex,
self
to use its UpperHex
implementation when
Debug
-formatted.source§impl<T> FutureExt for T
impl<T> FutureExt for T
source§fn with_context(self, otel_cx: Context) -> WithContext<Self>
fn with_context(self, otel_cx: Context) -> WithContext<Self>
source§fn with_current_context(self) -> WithContext<Self>
fn with_current_context(self) -> WithContext<Self>
source§impl<T> Instrument for T
impl<T> Instrument for T
source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
source§impl<T> IntoRequest<T> for T
impl<T> IntoRequest<T> for T
source§fn into_request(self) -> Request<T>
fn into_request(self) -> Request<T>
T
in a tonic::Request
source§impl<T, U> OverrideFrom<Option<&T>> for Uwhere
U: OverrideFrom<T>,
impl<T, U> OverrideFrom<Option<&T>> for Uwhere
U: OverrideFrom<T>,
source§impl<T> Pipe for Twhere
T: ?Sized,
impl<T> Pipe for Twhere
T: ?Sized,
source§fn pipe<R>(self, func: impl FnOnce(Self) -> R) -> Rwhere
Self: Sized,
fn pipe<R>(self, func: impl FnOnce(Self) -> R) -> Rwhere
Self: Sized,
source§fn pipe_ref<'a, R>(&'a self, func: impl FnOnce(&'a Self) -> R) -> Rwhere
R: 'a,
fn pipe_ref<'a, R>(&'a self, func: impl FnOnce(&'a Self) -> R) -> Rwhere
R: 'a,
self
and passes that borrow into the pipe function. Read moresource§fn pipe_ref_mut<'a, R>(&'a mut self, func: impl FnOnce(&'a mut Self) -> R) -> Rwhere
R: 'a,
fn pipe_ref_mut<'a, R>(&'a mut self, func: impl FnOnce(&'a mut Self) -> R) -> Rwhere
R: 'a,
self
and passes that borrow into the pipe function. Read moresource§fn pipe_borrow<'a, B, R>(&'a self, func: impl FnOnce(&'a B) -> R) -> R
fn pipe_borrow<'a, B, R>(&'a self, func: impl FnOnce(&'a B) -> R) -> R
source§fn pipe_borrow_mut<'a, B, R>(
&'a mut self,
func: impl FnOnce(&'a mut B) -> R,
) -> R
fn pipe_borrow_mut<'a, B, R>( &'a mut self, func: impl FnOnce(&'a mut B) -> R, ) -> R
source§fn pipe_as_ref<'a, U, R>(&'a self, func: impl FnOnce(&'a U) -> R) -> R
fn pipe_as_ref<'a, U, R>(&'a self, func: impl FnOnce(&'a U) -> R) -> R
self
, then passes self.as_ref()
into the pipe function.source§fn pipe_as_mut<'a, U, R>(&'a mut self, func: impl FnOnce(&'a mut U) -> R) -> R
fn pipe_as_mut<'a, U, R>(&'a mut self, func: impl FnOnce(&'a mut U) -> R) -> R
self
, then passes self.as_mut()
into the pipe
function.source§fn pipe_deref<'a, T, R>(&'a self, func: impl FnOnce(&'a T) -> R) -> R
fn pipe_deref<'a, T, R>(&'a self, func: impl FnOnce(&'a T) -> R) -> R
self
, then passes self.deref()
into the pipe function.source§impl<T> Pointable for T
impl<T> Pointable for T
source§impl<T> ProgressEventTimestamp for T
impl<T> ProgressEventTimestamp for T
source§impl<P, R> ProtoType<R> for Pwhere
R: RustType<P>,
impl<P, R> ProtoType<R> for Pwhere
R: RustType<P>,
source§fn into_rust(self) -> Result<R, TryFromProtoError>
fn into_rust(self) -> Result<R, TryFromProtoError>
RustType::from_proto
.source§fn from_rust(rust: &R) -> P
fn from_rust(rust: &R) -> P
RustType::into_proto
.source§impl<'a, S, T> Semigroup<&'a S> for Twhere
T: Semigroup<S>,
impl<'a, S, T> Semigroup<&'a S> for Twhere
T: Semigroup<S>,
source§fn plus_equals(&mut self, rhs: &&'a S)
fn plus_equals(&mut self, rhs: &&'a S)
std::ops::AddAssign
, for types that do not implement AddAssign
.source§impl<T> Tap for T
impl<T> Tap for T
source§fn tap_borrow<B>(self, func: impl FnOnce(&B)) -> Self
fn tap_borrow<B>(self, func: impl FnOnce(&B)) -> Self
Borrow<B>
of a value. Read moresource§fn tap_borrow_mut<B>(self, func: impl FnOnce(&mut B)) -> Self
fn tap_borrow_mut<B>(self, func: impl FnOnce(&mut B)) -> Self
BorrowMut<B>
of a value. Read moresource§fn tap_ref<R>(self, func: impl FnOnce(&R)) -> Self
fn tap_ref<R>(self, func: impl FnOnce(&R)) -> Self
AsRef<R>
view of a value. Read moresource§fn tap_ref_mut<R>(self, func: impl FnOnce(&mut R)) -> Self
fn tap_ref_mut<R>(self, func: impl FnOnce(&mut R)) -> Self
AsMut<R>
view of a value. Read moresource§fn tap_deref<T>(self, func: impl FnOnce(&T)) -> Self
fn tap_deref<T>(self, func: impl FnOnce(&T)) -> Self
Deref::Target
of a value. Read moresource§fn tap_deref_mut<T>(self, func: impl FnOnce(&mut T)) -> Self
fn tap_deref_mut<T>(self, func: impl FnOnce(&mut T)) -> Self
Deref::Target
of a value. Read moresource§fn tap_dbg(self, func: impl FnOnce(&Self)) -> Self
fn tap_dbg(self, func: impl FnOnce(&Self)) -> Self
.tap()
only in debug builds, and is erased in release builds.source§fn tap_mut_dbg(self, func: impl FnOnce(&mut Self)) -> Self
fn tap_mut_dbg(self, func: impl FnOnce(&mut Self)) -> Self
.tap_mut()
only in debug builds, and is erased in release
builds.source§fn tap_borrow_dbg<B>(self, func: impl FnOnce(&B)) -> Self
fn tap_borrow_dbg<B>(self, func: impl FnOnce(&B)) -> Self
.tap_borrow()
only in debug builds, and is erased in release
builds.source§fn tap_borrow_mut_dbg<B>(self, func: impl FnOnce(&mut B)) -> Self
fn tap_borrow_mut_dbg<B>(self, func: impl FnOnce(&mut B)) -> Self
.tap_borrow_mut()
only in debug builds, and is erased in release
builds.source§fn tap_ref_dbg<R>(self, func: impl FnOnce(&R)) -> Self
fn tap_ref_dbg<R>(self, func: impl FnOnce(&R)) -> Self
.tap_ref()
only in debug builds, and is erased in release
builds.source§fn tap_ref_mut_dbg<R>(self, func: impl FnOnce(&mut R)) -> Self
fn tap_ref_mut_dbg<R>(self, func: impl FnOnce(&mut R)) -> Self
.tap_ref_mut()
only in debug builds, and is erased in release
builds.source§fn tap_deref_dbg<T>(self, func: impl FnOnce(&T)) -> Self
fn tap_deref_dbg<T>(self, func: impl FnOnce(&T)) -> Self
.tap_deref()
only in debug builds, and is erased in release
builds.