Struct mz_compute_types::plan::lowering::Context

pub(super) struct Context {
    arrangements: BTreeMap<Id, AvailableCollections>,
    next_lir_id: LirId,
    debug_info: LirDebugInfo,
    enable_reduce_mfp_fusion: bool,
    enable_reduce_unnest_list_fusion: bool,
}

Fields

arrangements: BTreeMap<Id, AvailableCollections>

Known bindings to (possibly arranged) collections.

next_lir_id: LirId

Tracks the next available LirId.

debug_info: LirDebugInfo

Information to print along with error messages.

enable_reduce_mfp_fusion: bool

Whether to enable fusion of MFPs in reductions.

enable_reduce_unnest_list_fusion: bool

Whether to fuse Reduce with FlatMap UnnestList for better window function performance.

Implementations

impl Context

pub fn new(debug_name: String, features: &OptimizerFeatures) -> Self

fn allocate_lir_id(&mut self) -> LirId

pub fn lower<T: Timestamp>( self, desc: DataflowDescription<OptimizedMirRelationExpr>, ) -> Result<DataflowDescription<Plan<T>>, String>

fn lower_mir_expr<T: Timestamp>( &mut self, expr: &MirRelationExpr, ) -> Result<(Plan<T>, AvailableCollections), String>

This method converts a MirRelationExpr into a plan that can be directly rendered.

The rough structure is that we repeatedly extract map/filter/project operators from each expression we see, bundle them up as a MapFilterProject object, and then produce a plan for the combination of that with the next operator.

The method accesses self.arrangements, which it will locally add to and remove from for Let bindings (by the end of the call it should contain the same bindings as when it started).

The result of the method is both a Plan, but also a list of arrangements that are certain to be produced, which can be relied on by the next steps in the plan. Each of the arrangement keys is associated with an MFP that must be applied if that arrangement is used, to back out the permutation associated with that arrangement.

An empty list of arrangement keys indicates that only a Collection stream can be assumed to exist.

fn lower_mir_expr_stack_safe<T>( &mut self, expr: &MirRelationExpr, ) -> Result<(Plan<T>, AvailableCollections), String>

where T: Timestamp,

fn lower_reduce<T: Timestamp>( &mut self, input: &MirRelationExpr, group_key: &Vec<MirScalarExpr>, aggregates: &Vec<AggregateExpr>, monotonic: &bool, expected_group_size: &Option<u64>, mfp_on_top: &mut MapFilterProject, fused_unnest_list: bool, ) -> Result<(Plan<T>, AvailableCollections), String>

Lowers a Reduce with the given fields and an mfp_on_top, which is the MFP that is originally on top of the Reduce. This MFP, or a part of it, might be fused into the Reduce, in which case mfp_on_top is mutated to be the residual MFP, i.e., what was not fused.

pub fn arrange_by<T>( &mut self, plan: Plan<T>, collections: AvailableCollections, old_collections: &AvailableCollections, arity: usize, ) -> Plan<T>

Replace the plan with another one that has the collection in some additional forms.