pub fn source<G, D, B, L>(
scope: &G,
name: String,
construct: B
) -> (Stream<G, D>, SourceToken)where
G: Scope<Timestamp = Timestamp>,
D: Data,
B: FnOnce(OperatorInfo) -> L,
L: FnMut(&mut CapabilitySet<Timestamp>, &mut OutputHandle<'_, G::Timestamp, D, Tee<G::Timestamp, D>>) + 'static,
Expand description
Constructs a source named name
in scope
whose lifetime is controlled
both internally and externally.
The logic for the source is supplied by construct
, which must return a
tick
function that satisfies L
. This function will be called
periodically while the source is alive and supplied with a capability to
produce data and the output handle into which data should be given. The
tick
function is responsible for periodically downgrading this capability
whenever it can see that a timestamp is “closed”, according to whatever
logic makes sense for the source.
If tick
realizes it will never produce data again, it should indicate that
fact by downgrading the given CapabilitySet
to the empty frontier before
returning. This will guarantee that tick
is never called again.
It is tick
’s responsibility to inform Timely of its desire to be scheduled
again by chatting with a timely::scheduling::activate::Activator
.
Holding on to capabilities using the CapabilitySet
does not alone cause
the source to be scheduled again; it merely keeps the source alive.
The lifetime of the source is also controlled by the returned
SourceToken
. When the last clone of the SourceToken
is dropped, the
tick
function will no longer be called, and the capability will eventually
be dropped.
When the source token is dropped, the timestamping_flag is set to false to terminate any spawned threads in the source operator