The types for the dataflow crate.
These are extracted into their own crate so that crates that only depend on the interface of the dataflow crate, and not its implementation, can avoid the dependency, as the dataflow crate is very slow to compile.
Encoding in Avro format.
A struct to hold more specific information about where a BYO source came from so we can differentiate between topics of the same name across different brokers.
An association of a global identifier to an expression.
Arguments necessary to define how to decode from CSV format
A description of a dataflow to construct and results to surface.
Explanation facilitates pretty-printing of the parts of a
DataflowDesc that are relevant to dataflow rendering.
An index storing processed updates so they can be queried or reused in other computations
In-place restrictions that can be made to rows.
Universal language for describing message positions in Materialize, in a source independent way. Invidual sources like Kafka or File sources should explicitly implement their own offset type that converts to/From MzOffsets. A 0-MzOffset denotes an empty stream.
Encoding in Protobuf format.
A sink for updates to a relational collection.
A source of updates for a relational collection.
A batch of updates to be fed to a local input
Determines the RelationDesc and decoding of CSV objects
A description of how each row should be decoded, from a string of bytes to a sequence of Differential updates.
Whether and how to include the key portion of a stream in dataflows
The response from a
A Source of Object Key names, the argument of the
DISCOVER OBJECTS clause
A description of how to interpret data from various sources
Various responses that can be communicated about the progress of a TAIL command.
The meaning of the timestamp number produced by data sources. This type is not concerned with the source of the timestamp (like if the data came from a Debezium consistency topic or a CDCv2 stream), instead only what the timestamp number means.
Structure wrapping a timestamp update from a source If RT, contains a partition count If BYO, contains a tuple (PartitionCount, PartitionID, Timestamp, Offset), which informs workers that messages with Offset on PartititionId will be timestamped with Timestamp.
A commonly used name for dataflows contain MIR expressions.