Skip to main content

create_raw_source

Function create_raw_source 

Source
pub fn create_raw_source<'scope, 'root, C>(
    scope: Scope<'scope, Timestamp>,
    root_scope: Scope<'root, ()>,
    storage_state: &StorageState,
    committed_uppers: BTreeMap<GlobalId, StreamVec<'scope, Timestamp, ()>>,
    config: &RawSourceCreationConfig,
    source_connection: C,
    start_signal: impl Future<Output = ()> + 'static,
) -> (BTreeMap<GlobalId, VecCollection<'scope, Timestamp, Result<SourceOutput<C::Time>, DataflowError>, Diff>>, StreamVec<'root, (), HealthStatusMessage>, StreamVec<'scope, Timestamp, ()>, Vec<PressOnDropButton>)
where C: SourceConnection + SourceRender + Clone + 'static,
Expand description

Creates a source dataflow operator graph from a source connection. The type of SourceConnection determines the type of connection that should be created.

This is also the place where reclocking (https://github.com/MaterializeInc/materialize/blob/main/doc/developer/design/20210714_reclocking.md) happens.

See the source module docs for more details about how raw sources are used.

The committed_uppers parameter contains one stream per export whose frontier advances whenever times are durably recorded for that export. They are reclocked into the ResumeUppers the source observes.

Alongside the reclocked exports this returns a no-data stream whose frontier is the remap upper.