Function dataflow::source::util::source

pub fn source<G, D, D2, B, L>(
    scope: &G, 
    name: String, 
    construct: B
) -> (Stream<G, D>, Stream<G, D2>, SourceToken) where
    G: Scope<Timestamp = Timestamp>,
    D: Data,
    D2: Data,
    B: FnOnce(OperatorInfo) -> L,
    L: FnMut(&mut Capability<Timestamp>, &mut Capability<Timestamp>, &mut CapabilitySet<Timestamp>, &mut OutputHandle<'_, G::Timestamp, D, Tee<G::Timestamp, D>>, &mut OutputHandle<'_, G::Timestamp, D2, Tee<G::Timestamp, D2>>) -> SourceStatus + 'static, 

Constructs a source named name in scope whose lifetime is controlled both internally and externally.

The logic for the source is supplied by construct, which must return a tick function that satisfies L. This function will be called periodically while the source is alive and supplied with a capability to produce data and the output handle into which data should be given. The tick function is responsible for periodically downgrading this capability whenever it can see that a timestamp is “closed”, according to whatever logic makes sense for the source.

The tick function is also given a secondary output handle and capability that can be used to emit a stream of data that is separate from the main “data” output. This can, for example, be used to emit and persist the timestamp bindings.

If tick realizes it will never produce data again, it should indicate that fact by returning SourceStatus::Done, which will immediately drop the capability and guarantee that tick is never called again.

Otherwise, tick should return SourceStatus::Alive. It is tick’s responsibility to inform Timely of its desire to be scheduled again by chatting with a timely::scheduling::activate::Activator. Returning SourceStatus::Alive does not alone cause the source to be scheduled again; it merely keeps the capability alive.

The lifetime of the source is also controlled by the returned SourceToken. When the last clone of the SourceToken is dropped, the tick function will no longer be called, and the capability will eventually be dropped.

When the source token is dropped, the timestamping_flag is set to false to terminate any spawned threads in the source operator