Function mz_storage_client::source::util::source

pub fn source<G, D, B, L>(
    scope: &G,
    name: String,
    flow_control_input: &Stream<G, ()>,
    construct: B
) -> (Stream<G, D>, SourceToken)where
    G: Scope<Timestamp = Timestamp>,
    D: Data,
    B: FnOnce(OperatorInfo) -> L,
    L: FnMut(&mut CapabilitySet<Timestamp>, AntichainRef<'_, G::Timestamp>, &mut OutputHandle<'_, G::Timestamp, D, Tee<G::Timestamp, D>>) + 'static,

Constructs a source named name in scope whose lifetime is controlled both internally and externally.

The logic for the source is supplied by construct, which must return a tick function that satisfies L. This function will be called periodically while the source is alive and supplied with a capability to produce data and the output handle into which data should be given. The tick function is responsible for periodically downgrading this capability whenever it can see that a timestamp is “closed”, according to whatever logic makes sense for the source.

If tick realizes it will never produce data again, it should indicate that fact by downgrading the given CapabilitySet to the empty frontier before returning. This will guarantee that tick is never called again.

It is tick’s responsibility to inform Timely of its desire to be scheduled again by chatting with a timely::scheduling::activate::Activator. Holding on to capabilities using the CapabilitySet does not alone cause the source to be scheduled again; it merely keeps the source alive.

The lifetime of the source is also controlled by the returned SourceToken. When the last clone of the SourceToken is dropped, the tick function will no longer be called, and the capability will eventually be dropped.

When the source token is dropped, the timestamping_flag is set to false to terminate any spawned threads in the source operator