Expand description
Logic related to the creation of dataflow sources.
See render_source
for more details.
Structsยง
- KV ๐
- After handling metadata insertion, we split streams into key/value parts for convenience
Functionsยง
- append_
metadata_ ๐to_ value - flatten_
results_ ๐prepend_ keys - Convert from streams of
DecodeResult
to Rows, inserting the Key according toKeyEnvelope
- get_
backpressure_ ๐max_ inflight_ bytes - raise_
key_ ๐value_ errors - Handle possibly missing key or value portions of messages
- render_
source - Renders complete differential
Collection
s that represent the final source and its errors as requested by the originalCREATE SOURCE
statement, encapsulated in the passedSourceInstanceDesc
. - render_
source_ ๐stream - Completes the rendering of a particular source stream by applying decoding and envelope processing as necessary
- split_
ok_ ๐err - upsert_
commands ๐ - Convert from streams of
DecodeResult
to UpsertCommands, inserting the Key according toKeyEnvelope