Module dataflow::render::sources

Logic related to the creation of dataflow sources.

Structs

BoundedCompactionHandle

A handle that allows setting a compaction frontier that is bounded by an upper bound.

KV

After handling metadata insertion, we split streams into key/value parts for convenience

PersistedSourceManager

A PersistedSourceManager stores maps from global identifiers to the one rendered persistent instance of a source.

PersistentSourceConfig

Configuration for persistent sources.

Enums

PersistentEnvelopeConfig

Configuration for the envelope-specific part of persistent sources.

SourceType

A type-level enum that holds one of two types of sources depending on their message type

Functions

append_metadata_to_value

flatten_results_prepend_keys

Convert from streams of DecodeResult to Rows, inserting the Key according to KeyEnvelope

get_persist_config

import_source

Constructs a CollectionBundle and tokens from source arguments.

import_table

Imports a table (non-durable, local source of input).

raise_key_value_errors

Handle possibly missing key or value portions of messages

seal_and_await

Seals both the main stream and the stream of timestamp bindings, allows compaction on underlying persistent streams, and awaits the seal frontier before passing on updates.