Module differential_dataflow::operators::arrange::arrangement

Arranges a collection into a re-usable trace structure.

The arrange operator applies to a differential dataflow Collection and returns an Arranged structure, provides access to both an indexed form of accepted updates as well as a stream of batches of newly arranged updates.

Several operators (join, reduce, and count, among others) are implemented against Arranged, and can be applied directly to arranged data instead of the collection. Internally, the operators will borrow the shared state, and listen on the timely stream for shared batches of data. The resources to index the collection—communication, computation, and memory—are spent only once, and only one copy of the index needs to be maintained as the collection changes.

The arranged collection is stored in a trace, whose append-only operation means that it is safe to share between the single arrange writer and multiple readers. Each reader is expected to interrogate the trace only at times for which it knows the trace is complete, as indicated by the frontiers on its incoming channels. Failing to do this is “safe” in the Rust sense of memory safety, but the reader may see ill-defined data at times for which the trace is not complete. (All current implementations commit only completed data to the trace).

Structs

• Arranged
  An arranged collection of (K,V) values.

Traits

• Arrange
  A type that can be arranged as if a collection of updates.
• ArrangeByKey
  Arranges something as (Key,Val) pairs according to a type T of trace.
• ArrangeBySelf
  Arranges something as (Key, ()) pairs according to a type T of trace.

Functions

• arrange_core
  Arranges a stream of updates by a key, configured with a name and a parallelization contract.