Module sources

Logic related to the creation of dataflow sources.

See render_source for more details.

Structs

KV 🔒

After handling metadata insertion, we split streams into key/value parts for convenience

Functions

append_metadata_to_value 🔒

flatten_results_prepend_keys 🔒

Convert from streams of DecodeResult to Rows, inserting the Key according to KeyEnvelope

get_backpressure_max_inflight_bytes 🔒

raise_key_value_errors 🔒

Handle possibly missing key or value portions of messages

render_source

Renders complete differential Collections that represent the final source and its errors as requested by the original CREATE SOURCE statement, encapsulated in the passed SourceInstanceDesc.

render_source_stream 🔒

Completes the rendering of a particular source stream by applying decoding and envelope processing as necessary

split_ok_err 🔒

upsert_commands 🔒

Convert from streams of DecodeResult to UpsertCommands, inserting the Key according to KeyEnvelope