Module mz_storage_types::sources

Types and traits related to the introduction of changing collections into dataflow.

Re-exports

• pub use crate::sources::envelope::SourceEnvelope;
• pub use crate::sources::kafka::KafkaSourceConnection;
• pub use crate::sources::load_generator::LoadGeneratorSourceConnection;
• pub use crate::sources::mysql::MySqlSourceConnection;
• pub use crate::sources::mysql::MySqlSourceExportDetails;
• pub use crate::sources::postgres::PostgresSourceConnection;
• pub use crate::sources::postgres::PostgresSourceExportDetails;

Modules

• encoding
  Types and traits related to the decoding of data for sources.
• envelope
  Types related to source envelopes
• kafka
  Types related kafka sources
• load_generator
  Types related to load generator sources
• mysql
  Types related to mysql sources
• postgres
  Types related to postgres sources
• proto_compression
  Nested message and enum types in ProtoCompression.
• proto_ingestion_description
  Nested message and enum types in ProtoIngestionDescription.
• proto_source_connection
  Nested message and enum types in ProtoSourceConnection.
• proto_source_data
  Nested message and enum types in ProtoSourceData.
• proto_source_export_details
  Nested message and enum types in ProtoSourceExportDetails.
• proto_source_export_statement_details
  Nested message and enum types in ProtoSourceExportStatementDetails.
• proto_timeline
  Nested message and enum types in ProtoTimeline.

Structs

• IndexedSourceExport
• IngestionDescription
  A description of a source ingestion
• MzOffset
  Universal language for describing message positions in Materialize, in a source independent way. Individual sources like Kafka or File sources should explicitly implement their own offset type that converts to/From MzOffsets. A 0-MzOffset denotes an empty stream.
• ProtoCompression
• ProtoIngestionDescription
• ProtoMzOffset
• ProtoSourceConnection
• ProtoSourceData
• ProtoSourceDesc
• ProtoSourceExportDataConfig
• ProtoSourceExportDetails
• ProtoSourceExportStatementDetails
  NOTE: this message is encoded and stored as part of a source export statement option (currently only CREATE SUBSOURCE statements) Be extra careful about changes, ensuring that all changes are backwards compatible
• ProtoTimeline
• SourceData
• SourceDataColumnarDecoder
• SourceDataColumnarEncoder
• SourceDesc
  An external source of updates for a relational collection.
• SourceExport
• SourceExportDataConfig
  Defines the configuration for how to handle data that is exported for a given Source Export.
• SourceReferenceResolver
  Stores and resolves references to a &[T: ExternalCatalogReference].

Enums

• Compression
• ExternalReferenceResolutionError
• GenericSourceConnection
• SourceDataRowColumnarDecoder
  A decoder for Rows within SourceData.
• SourceDataRowColumnarEncoder
  An encoder for Rows within SourceData.
• SourceExportDetails
  Details necessary for each source export to allow the source implementations to export data to the export’s collection.
• SourceExportStatementDetails
  Details necessary to store in the Details option of a source export statement (CREATE SUBSOURCE and CREATE TABLE .. FROM SOURCE statements), to generate the appropriate SourceExportDetails struct during planning. NOTE that this is serialized as proto to the catalog, so any changes here must be backwards compatible or will require a migration.
• Timeline
  The meaning of the timestamp number produced by data sources. This type is not concerned with the source of the timestamp (like if the data came from a Debezium consistency topic or a CDCv2 stream), instead only what the timestamp number means.

Constants

• _IMPL_ARBITRARY_FOR_Compression 🔒
• _IMPL_ARBITRARY_FOR_GenericSourceConnection 🔒
• _IMPL_ARBITRARY_FOR_IndexedSourceExport 🔒
• _IMPL_ARBITRARY_FOR_IngestionDescription 🔒
• _IMPL_ARBITRARY_FOR_MzOffset 🔒
• _IMPL_ARBITRARY_FOR_SourceData 🔒
• _IMPL_ARBITRARY_FOR_SourceDesc 🔒
• _IMPL_ARBITRARY_FOR_SourceExport 🔒
• _IMPL_ARBITRARY_FOR_SourceExportDataConfig 🔒
• _IMPL_ARBITRARY_FOR_SourceExportDetails 🔒
• _IMPL_ARBITRARY_FOR_Timeline 🔒

Traits

• ExternalCatalogReference
  Describes how external references should be organized in a multi-level hierarchy.
• SourceConnection
  A connection to an external system
• SourceTimestamp

Functions

• arb_source_data_for_relation_desc
  Given a RelationDesc returns an arbitrary SourceData.