Expand description

Traits and types related to the distribution of data.

These traits and types are in support of a flexible approach to data distribution and organization, in which we might like to more explicitly manage how certain types are handled. Although the term “hashing” is used throughout, it is a misnomer; these traits relate to extracting reasonably distributed integers from the types, and hashing happens to be evocative of this.

Differential dataflow operators need to co-locate data that are equivalent so that they may have the differences consolidated, and eventually cancelled. The chose approach is to extract an integer from the keys of the data, ensuring that elements with the same key arrive at the same worker, where the consolidation can occur.

The intent is that types should be able to indicate how this integer is determined, so that general data types can use a generic hash function, where as more specialized types such as uniformly distributed integers can perhaps do something simpler (like report their own value).

Traits

  • Types with a hashed method, producing an unsigned output of some type.