Struct mz_storage_types::sources::IngestionDescription

pub struct IngestionDescription<S: 'static = (), C: ConnectionAccess = InlinedConnection> {
    pub desc: SourceDesc<C>,
    pub ingestion_metadata: S,
    pub source_exports: BTreeMap<GlobalId, SourceExport<S>>,
    pub instance_id: StorageInstanceId,
    pub remap_collection_id: GlobalId,
}

A description of a source ingestion

Fields

desc: SourceDesc<C>

The source description.

ingestion_metadata: S

Additional storage controller metadata needed to ingest this source

source_exports: BTreeMap<GlobalId, SourceExport<S>>

Collections to be exported by this ingestion.

Notes

• For multi-output sources:

  • Add exports by adding a new SourceExport.
  • Remove exports by removing the SourceExport.

  Re-rendering/executing the source after making these modifications adds and drops the subsource, respectively.

• This field includes the primary source’s ID, which might need to be filtered out to understand which exports are explicit ingestion exports.

• This field does not include the remap collection, which is tracked in its own field.

instance_id: StorageInstanceId

The ID of the instance in which to install the source.

remap_collection_id: GlobalId

The ID of this ingestion’s remap/progress collection.

Implementations

impl IngestionDescription

pub fn new( desc: SourceDesc, instance_id: StorageInstanceId, remap_collection_id: GlobalId, ) -> Self

impl<S> IngestionDescription<S>

pub fn collection_ids(&self) -> impl Iterator<Item = GlobalId> + '_

Return an iterator over the GlobalIds of self’s collections. This will contain ids for the remap collection, subsources, tables for this source, and the primary collection ID, even if no data will be exported to the primary collection.

impl<S: Clone> IngestionDescription<S>

pub fn indexed_source_exports( &self, primary_source_id: &GlobalId, ) -> BTreeMap<GlobalId, IndexedSourceExport<S>>

Trait Implementations

impl<S: Debug + Eq + PartialEq + AlterCompatible> AlterCompatible for IngestionDescription<S>

fn alter_compatible( &self, id: GlobalId, other: &IngestionDescription<S>, ) -> Result<(), AlterError>

impl<S: 'static + Arbitrary, C: ConnectionAccess + Arbitrary> Arbitrary for IngestionDescription<S, C>

type Parameters = (<SourceDesc<C> as Arbitrary>::Parameters, <S as Arbitrary>::Parameters, <StorageInstanceId as Arbitrary>::Parameters, <GlobalId as Arbitrary>::Parameters)

The type of parameters that arbitrary_with accepts for configuration of the generated Strategy. Parameters must implement Default.

type Strategy = Map<(<SourceDesc<C> as Arbitrary>::Strategy, <S as Arbitrary>::Strategy, BoxedStrategy<BTreeMap<GlobalId, SourceExport<S>>>, <StorageInstanceId as Arbitrary>::Strategy, <GlobalId as Arbitrary>::Strategy), fn(_: (SourceDesc<C>, S, BTreeMap<GlobalId, SourceExport<S>>, StorageInstanceId, GlobalId)) -> IngestionDescription<S, C>>

The type of Strategy used to generate values of type Self.

fn arbitrary_with(_top: Self::Parameters) -> Self::Strategy

Generates a Strategy for producing arbitrary values of type the implementing type (Self). The strategy is passed the arguments given in args.

fn arbitrary() -> Self::Strategy

Generates a Strategy for producing arbitrary values of type the implementing type (Self).

impl<S: Clone + 'static, C: Clone + ConnectionAccess> Clone for IngestionDescription<S, C>

fn clone(&self) -> IngestionDescription<S, C>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<S: Debug + 'static, C: Debug + ConnectionAccess> Debug for IngestionDescription<S, C>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de, S, C> Deserialize<'de> for IngestionDescription<S, C>

where S: Deserialize<'de> + 'static, C: Deserialize<'de> + ConnectionAccess,

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl<R: ConnectionResolver> IntoInlineConnection<IngestionDescription, R> for IngestionDescription<(), ReferencedConnection>

fn into_inline_connection(self, r: R) -> IngestionDescription

impl<S: PartialEq + 'static, C: PartialEq + ConnectionAccess> PartialEq for IngestionDescription<S, C>

fn eq(&self, other: &IngestionDescription<S, C>) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl RustType<ProtoIngestionDescription> for IngestionDescription<CollectionMetadata>

fn into_proto(&self) -> ProtoIngestionDescription

Convert a Self into a Proto value.

fn from_proto( proto: ProtoIngestionDescription, ) -> Result<Self, TryFromProtoError>

Consume and convert a Proto back into a Self value.

fn into_proto_owned(self) -> Proto

A zero clone version of Self::into_proto that types can optionally implement, otherwise, the default implementation delegates to Self::into_proto.

impl<S, C> Serialize for IngestionDescription<S, C>

where S: Serialize + 'static, C: Serialize + ConnectionAccess,

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl<S: Eq + 'static, C: Eq + ConnectionAccess> Eq for IngestionDescription<S, C>

impl<S: 'static, C: ConnectionAccess> StructuralPartialEq for IngestionDescription<S, C>