pub trait Scope: ScopeParent {
// Required methods
fn name(&self) -> String;
fn addr(&self) -> Rc<[usize]>;
fn addr_for_child(&self, index: usize) -> Rc<[usize]>;
fn add_edge(&self, source: Source, target: Target);
fn allocate_operator_index(&mut self) -> usize;
fn add_operator_with_indices(
&mut self,
operator: Box<dyn Operate<Self::Timestamp>>,
local: usize,
global: usize,
);
fn scoped<T, R, F>(&mut self, name: &str, func: F) -> R
where T: Timestamp + Refines<<Self as ScopeParent>::Timestamp>,
F: FnOnce(&mut Child<'_, Self, T>) -> R;
// Provided methods
fn add_operator(
&mut self,
operator: Box<dyn Operate<Self::Timestamp>>,
) -> usize { ... }
fn add_operator_with_index(
&mut self,
operator: Box<dyn Operate<Self::Timestamp>>,
index: usize,
) { ... }
fn iterative<T, R, F>(&mut self, func: F) -> R
where T: Timestamp,
F: FnOnce(&mut Child<'_, Self, Product<<Self as ScopeParent>::Timestamp, T>>) -> R { ... }
fn region<R, F>(&mut self, func: F) -> R
where F: FnOnce(&mut Child<'_, Self, <Self as ScopeParent>::Timestamp>) -> R { ... }
fn region_named<R, F>(&mut self, name: &str, func: F) -> R
where F: FnOnce(&mut Child<'_, Self, <Self as ScopeParent>::Timestamp>) -> R { ... }
}
Expand description
The fundamental operations required to add and connect operators in a timely dataflow graph.
Importantly, this is often a shared object, backed by a Rc<RefCell<>>
wrapper. Each method
takes a shared reference, but can be thought of as first calling .clone() and then calling the
method. Each method does not hold the RefCell
’s borrow, and should prevent accidental panics.
Required Methods§
sourcefn addr(&self) -> Rc<[usize]>
fn addr(&self) -> Rc<[usize]>
A sequence of scope identifiers describing the path from the worker root to this scope.
sourcefn addr_for_child(&self, index: usize) -> Rc<[usize]>
fn addr_for_child(&self, index: usize) -> Rc<[usize]>
A sequence of scope identifiers describing the path from the worker root to the child
indicated by index
.
sourcefn add_edge(&self, source: Source, target: Target)
fn add_edge(&self, source: Source, target: Target)
Connects a source of data with a target of the data. This only links the two for the purposes of tracking progress, rather than effect any data movement itself.
sourcefn allocate_operator_index(&mut self) -> usize
fn allocate_operator_index(&mut self) -> usize
Allocates a new scope-local operator index.
This method is meant for use with add_operator_with_index
, which accepts a scope-local
operator index allocated with this method. This method does cause the scope to expect that
an operator will be added, and it is an error not to eventually add such an operator.
sourcefn add_operator_with_indices(
&mut self,
operator: Box<dyn Operate<Self::Timestamp>>,
local: usize,
global: usize,
)
fn add_operator_with_indices( &mut self, operator: Box<dyn Operate<Self::Timestamp>>, local: usize, global: usize, )
Adds a child Operate
to the builder’s scope using supplied indices.
The two indices are the scope-local operator index, and a worker-unique index used for e.g. logging.
sourcefn scoped<T, R, F>(&mut self, name: &str, func: F) -> R
fn scoped<T, R, F>(&mut self, name: &str, func: F) -> R
Creates a dataflow subgraph.
This method allows the user to create a nested scope with any timestamp that “refines” the enclosing timestamp (informally: extends it in a reversible way).
This is most commonly used to create new iterative contexts, and the provided
method iterative
for this task demonstrates the use of this method.
§Examples
use timely::dataflow::Scope;
use timely::dataflow::operators::{Input, Enter, Leave};
use timely::order::Product;
timely::execute_from_args(std::env::args(), |worker| {
// must specify types as nothing else drives inference.
let input = worker.dataflow::<u64,_,_>(|child1| {
let (input, stream) = child1.new_input::<String>();
let output = child1.scoped::<Product<u64,u32>,_,_>("ScopeName", |child2| {
stream.enter(child2).leave()
});
input
});
});
Provided Methods§
sourcefn add_operator(&mut self, operator: Box<dyn Operate<Self::Timestamp>>) -> usize
fn add_operator(&mut self, operator: Box<dyn Operate<Self::Timestamp>>) -> usize
Adds a child Operate
to the builder’s scope. Returns the new child’s index.
sourcefn add_operator_with_index(
&mut self,
operator: Box<dyn Operate<Self::Timestamp>>,
index: usize,
)
fn add_operator_with_index( &mut self, operator: Box<dyn Operate<Self::Timestamp>>, index: usize, )
Adds a child Operate
to the builder’s scope using a supplied index.
This is used internally when there is a gap between allocate a child identifier and adding the child, as happens in subgraph creation.
sourcefn iterative<T, R, F>(&mut self, func: F) -> R
fn iterative<T, R, F>(&mut self, func: F) -> R
Creates a iterative dataflow subgraph.
This method is a specialization of scoped
which uses the Product
timestamp
combinator, suitable for iterative computations in which iterative development
at some time cannot influence prior iterations at a future time.
§Examples
use timely::dataflow::Scope;
use timely::dataflow::operators::{Input, Enter, Leave};
timely::execute_from_args(std::env::args(), |worker| {
// must specify types as nothing else drives inference.
let input = worker.dataflow::<u64,_,_>(|child1| {
let (input, stream) = child1.new_input::<String>();
let output = child1.iterative::<u32,_,_>(|child2| {
stream.enter(child2).leave()
});
input
});
});
sourcefn region<R, F>(&mut self, func: F) -> R
fn region<R, F>(&mut self, func: F) -> R
Creates a dataflow region with the same timestamp.
This method is a specialization of scoped
which uses the same timestamp as the
containing scope. It is used mainly to group regions of a dataflow computation, and
provides some computational benefits by abstracting the specifics of the region.
§Examples
use timely::dataflow::Scope;
use timely::dataflow::operators::{Input, Enter, Leave};
timely::execute_from_args(std::env::args(), |worker| {
// must specify types as nothing else drives inference.
let input = worker.dataflow::<u64,_,_>(|child1| {
let (input, stream) = child1.new_input::<String>();
let output = child1.region(|child2| {
stream.enter(child2).leave()
});
input
});
});
sourcefn region_named<R, F>(&mut self, name: &str, func: F) -> R
fn region_named<R, F>(&mut self, name: &str, func: F) -> R
Creates a dataflow region with the same timestamp.
This method is a specialization of scoped
which uses the same timestamp as the
containing scope. It is used mainly to group regions of a dataflow computation, and
provides some computational benefits by abstracting the specifics of the region.
This variant allows you to specify a name for the region, which can be read out in the timely logging streams.
§Examples
use timely::dataflow::Scope;
use timely::dataflow::operators::{Input, Enter, Leave};
timely::execute_from_args(std::env::args(), |worker| {
// must specify types as nothing else drives inference.
let input = worker.dataflow::<u64,_,_>(|child1| {
let (input, stream) = child1.new_input::<String>();
let output = child1.region_named("region", |child2| {
stream.enter(child2).leave()
});
input
});
});