Function create_raw_source

Source

pub fn create_raw_source<'scope, 'root, C>(
    scope: Scope<'scope, Timestamp>,
    root_scope: Scope<'root, ()>,
    storage_state: &StorageState,
    committed_upper: StreamVec<'scope, Timestamp, ()>,
    config: &RawSourceCreationConfig,
    source_connection: C,
    start_signal: impl Future<Output = ()> + 'static,
) -> (BTreeMap<GlobalId, VecCollection<'scope, Timestamp, Result<SourceOutput<C::Time>, DataflowError>, Diff>>, StreamVec<'root, (), HealthStatusMessage>, Vec<PressOnDropButton>)where
    C: SourceConnection + SourceRender + Clone + 'static,

Expand description

Creates a source dataflow operator graph from a source connection. The type of SourceConnection determines the type of connection that should be created.

This is also the place where reclocking (https://github.com/MaterializeInc/materialize/blob/main/doc/developer/design/20210714_reclocking.md) happens.

See the source module docs for more details about how raw sources are used.

The resume_stream parameter will contain frontier updates whenever times are durably recorded which allows the ingestion to release upstream resources.

create_raw_source

Function create_raw_source Copy item path

Function create_raw_source