Module mz_storage_client::types::sources
source · Expand description
Types and traits related to the introduction of changing collections into dataflow.
Modules
- Types and traits related to the decoding of data for sources.
- Nested message and enum types in
ProtoCompression. - Nested message and enum types in
ProtoDebeziumSourceProjection. - Nested message and enum types in
ProtoIncludedColumnSource. - Nested message and enum types in
ProtoIngestionDescription. - Nested message and enum types in
ProtoKeyEnvelope. - Nested message and enum types in
ProtoLoadGeneratorSourceConnection. - Nested message and enum types in
ProtoPostgresSourceConnection. - Nested message and enum types in
ProtoSourceConnection. - Nested message and enum types in
ProtoSourceData. - Nested message and enum types in
ProtoSourceEnvelope. - Nested message and enum types in
ProtoTimeline. - Nested message and enum types in
ProtoUpsertStyle.
Structs
- A column that was created via an
INCLUDEexpression - A description of a source ingestion
- Universal language for describing message positions in Materialize, in a source independent way. Individual sources like Kafka or File sources should explicitly implement their own offset type that converts to/From MzOffsets. A 0-MzOffset denotes an empty stream.
- An implementation of PartDecoder for SourceData.
- An implementation of PartEncoder for SourceData.
- An external source of updates for a relational collection.
Enums
- Debezium generates records that contain metadata about the upstream database. The structure of this metadata depends on the type of connection used. This struct records the relevant indices in the record, calculated during planning, so that the dataflow operator can unpack the structure and extract the relevant information.
- Which piece of metadata a column corresponds to
- Whether and how to include the decoded key of a stream in dataflows
SourceEnvelopes describe how to turn a stream of messages fromSourceDescs into a differential stream, that is, a stream of (data, time, diff) triples.- The meaning of the timestamp number produced by data sources. This type is not concerned with the source of the timestamp (like if the data came from a Debezium consistency topic or a CDCv2 stream), instead only what the timestamp number means.
UnplannedSourceEnvelopeis aSourceEnvelopemissing some information. This information is obtained inUnplannedSourceEnvelope::desc, whereUnplannedSourceEnvelope::into_source_envelopecreates a fullSourceEnvelope
Constants
Statics
Traits
- A connection to an external system
Functions
- Computes the indices of the value’s relation description that appear in the key.