Module sources

Source
Expand description

Logic related to the creation of dataflow sources.

See render_source for more details.

Structsยง

KV ๐Ÿ”’
After handling metadata insertion, we split streams into key/value parts for convenience

Functionsยง

append_metadata_to_value ๐Ÿ”’
flatten_results_prepend_keys ๐Ÿ”’
Convert from streams of DecodeResult to Rows, inserting the Key according to KeyEnvelope
get_backpressure_max_inflight_bytes ๐Ÿ”’
raise_key_value_errors ๐Ÿ”’
Handle possibly missing key or value portions of messages
render_source
Renders complete differential Collections that represent the final source and its errors as requested by the original CREATE SOURCE statement, encapsulated in the passed SourceInstanceDesc.
render_source_stream ๐Ÿ”’
Completes the rendering of a particular source stream by applying decoding and envelope processing as necessary
split_ok_err ๐Ÿ”’
upsert_commands ๐Ÿ”’
Convert from streams of DecodeResult to UpsertCommands, inserting the Key according to KeyEnvelope