Module dataflow

Whole-dataflow optimization

A dataflow may contain multiple views, each of which may only be optimized locally. However, information like demand and predicate pushdown can be applied across views once we understand the context in which the views will be executed.

Structs

CollectIndexRequests 🔒

DataflowMetainfo

Extra information about the dataflow. This is not going to be shipped, but has to be processed in other ways, e.g., showing notices to the user, or saving meta-information to the catalog.

IndexUsageContext 🔒

This struct will save info about parent nodes as we are descending down a MirRelationExpr. We always start with filling in usage_type when we see an operation that uses an arrangement, and then we fill in requested_keys when we see an ArrangeBy. So, the pattern that we are looking for is

Functions

choose_index 🔒

Pick an index from a given Vec of index keys.

inline_views 🔒

Inline views used in one other view, and in no exported objects.

optimize_dataflow

Optimizes the implementation of each dataflow.

optimize_dataflow_demand 🔒

Pushes demand information from published outputs to dataflow inputs, projecting away unnecessary columns.

optimize_dataflow_demand_inner

Pushes demand through views in view_sequence in order, removing columns not demanded.

optimize_dataflow_filters 🔒

Pushes predicate to dataflow inputs.

optimize_dataflow_filters_inner

Pushes filters down through views in view_sequence in order.

optimize_dataflow_monotonic

Propagates information about monotonic inputs through operators, using mz_repr::optimize::OptimizerFeatures from ctx for crate::analysis::Analysis.

optimize_dataflow_relations 🔒

Performs either the logical or the physical optimization pass on the dataflow using the supplied set of indexes.

prune_and_annotate_dataflow_index_imports 🔒

Restricts the indexes imported by dataflow to only the ones it needs. It also adds to the DataflowMetainfo how each index will be used. It also annotates global Gets with whether they will be reads from Persist or an index, plus their index usage types.