Crate mz_transform

source ·
Expand description

Transformations for relation expressions.

This crate contains traits, types, and methods suitable for transforming MirRelationExpr types in ways that preserve semantics and improve performance. The core trait is Transform, and many implementors of this trait can be boxed and iterated over. Some common transformation patterns are wrapped as Transform implementors themselves.

The crate also contains the beginnings of whole-dataflow optimization, which uses the same analyses but spanning multiple dataflow elements.

Re-exports

pub use dataflow::optimize_dataflow;

Modules

Derived attributes framework and definitions.
Canonicalizes MFPs, e.g., performs CSE on the scalar expressions, eliminates identity MFPs.
Transformations based on pulling information about individual columns from sources.
Common subexpression elimination.
Whole-dataflow optimization
Transformation based on pushing demand information about columns toward sources.
Replace operators on constants collections with constant collections.
Transformations that fuse together others of their kind.
Determines the join implementation for join operators.
See if there are predicates of the form <expr>=literal that can be sped up using an index. More specifically, look for an MFP on top of a Get, where the MFP has an appropriate filter, and the Get has a matching index. Convert these to IndexedFilter joins, which is a semi-join with a constant collection.
Hoist literal values from maps wherever possible.
Analysis to identify monotonic collections, especially TopK inputs.
Push non-null requirements toward sources.
Harvests information about non-nullability of columns from sources.
Normalize the structure of Let and LetRec operators in expressions.
Normalize the structure of Let and LetRec operators in expressions.
Pushes predicates down through other operators.
Transform column references in a Map into a Project.
Hoist projections through operators.
Pushes column removal down through other operators.
Removes Reduce when the input has as unique keys the keys of the reduce.
Tries to convert a reduce around a join to a join of reduces. Also absorbs Map operators into Reduce operators.
Remove redundant collections of distinct elements from joins.
Remove semijoins that are applied multiple times to no further effect.
Remove Threshold operators when we are certain no records have negative multiplicity.
Remove TopK operators with both an offset of zero and no limit.
Detects an input being unioned with its negation and cancels them out

Structs

An IndexOracle that knows about no indexes.
A sequence of transformations iterated some number of times.
A sequence of transformations that simplify the MirRelationExpr
A naive optimizer for relation expressions.
Arguments that get threaded through all transforms.

Enums

Errors that can occur during a transformation.

Traits

A trait for a type that can answer questions about what indexes exist.
Types capable of transforming relation expressions.