Derived attributes framework and definitions.
Canonicalizes MFPs, performs CSEs, and speeds up certain filters.
Transformations based on pulling information about individual columns from sources.
Common subexpression elimination.
Transformation based on pushing demand information about columns toward sources.
Transformations that fuse together others of their kind.
Install replace certain
Get operators with their
Determines the join implementation for join operators.
Hoist literal values from maps wherever possible.
Analysis to identify monotonic collections, especially TopK inputs.
Push non-null requirements toward sources.
Harvests information about non-nullability of columns from sources.
Pushes predicates down through other operators.
Transform column references in a
Map into a
Hoist projections through operators.
Pushes column removal down through other operators.
Reduce when the input has as unique keys the keys of the reduce.
Replace operators on constants collections with constant collections.
Tries to convert a reduce around a join to a join of reduces.
Also absorbs Map operators into Reduce operators.
Remove redundant collections of distinct elements from joins.
Remove Threshold operators when we are certain no records have negative multiplicity.
Remove TopK operators with both an offset of zero and no limit.
Detects an input being unioned with its negation and cancels them out
Re-assign type information and identifiers to each
Get to ensure
uniqueness of identifiers.