Struct Context 

pub(super) struct Context {
    arrangements: BTreeMap<Id, AvailableCollections>,
    has_future_updates: BTreeSet<Id>,
    next_lir_id: LirId,
    debug_info: LirDebugInfo,
    enable_reduce_mfp_fusion: bool,
    metrics: Option<LoweringMetrics>,
    single_time: bool,
    source_imports: BTreeSet<GlobalId>,
    source_get_mfps: BTreeMap<LirId, MfpPlan<MirScalarExpr>>,
}

Fields

arrangements: BTreeMap<Id, AvailableCollections>

Known bindings to (possibly arranged) collections.

has_future_updates: BTreeSet<Id>

Ids whose collections may contain updates at future timestamps, e.g., from a temporal MFP using mz_now().

next_lir_id: LirId

Tracks the next available LirId.

debug_info: LirDebugInfo

Information to print along with error messages.

enable_reduce_mfp_fusion: bool

Whether to enable fusion of MFPs in reductions.

metrics: Option<LoweringMetrics>

Metrics recorded during lowering, if any are being collected.

single_time: bool

Whether the current expression is subject to single-time (one-shot SELECT) monotonic operator selection.

Lowering locks in which arrangements a node makes available, and that set changes with the chosen operator variant (e.g. a monotonic TopK/Reduce arranges differently than its non-monotonic form). So the variant must be picked here, during lowering, rather than by a later rewrite that would leave the already-computed AvailableCollections describing the wrong shape.

Initialized from the dataflow’s is_single_time() and forced to false while lowering the recursive bindings of a LetRec, whose values are not restricted to a single time.

source_imports: BTreeSet<GlobalId>

Global ids of the dataflow’s source imports.

source_get_mfps: BTreeMap<LirId, MfpPlan<MirScalarExpr>>

MIR MfpPlans pushed onto Get::Collection reads of source imports, keyed by the Get’s LirId.

refine_source_mfps identifies common parts across sibling reads and pushes the common part into the shared source MFP. That pass runs on MIR because only MIR can utter the mz_now() predicates that temporal bounds fold into. Retaining the MIR form here lets it find common parts without round-tripping the LIR plans back through MIR.

Implementations

impl Context

pub fn new( debug_name: String, features: &OptimizerFeatures, metrics: Option<&LoweringMetrics>, ) -> Self

fn allocate_lir_id(&mut self) -> LirId

pub fn lower( self, desc: DataflowDescription<OptimizedMirRelationExpr>, ) -> Result<DataflowDescription<LirRelationExpr>, String>

fn refine_source_mfps( &mut self, dataflow: &mut DataflowDescription<LirRelationExpr>, )

Identifies common parts of the MapFilterProjects pushed onto sibling Get::Collection reads of each imported source, hoisting the shared prefix into the source’s own MFP.

The reads’ MFPs are lowering artifacts (MIR sees only Get(GlobalId)), so this belongs in lowering. We run on MIR because only MIR can utter the mz_now() predicates that temporal bounds fold into, so it consumes the MIR MfpPlans stashed in Self::source_get_mfps rather than round-tripping the lowered LIR plans back through MIR.

fn lower_mir_expr( &mut self, expr: &MirRelationExpr, ) -> Result<LoweredExpr, String>

This method converts a MirRelationExpr into a plan that can be directly rendered.

The rough structure is that we repeatedly extract map/filter/project operators from each expression we see, bundle them up as a MapFilterProject object, and then produce a plan for the combination of that with the next operator.

The method accesses self.arrangements, which it will locally add to and remove from for Let bindings (by the end of the call it should contain the same bindings as when it started).

The result of the method is both a LirRelationExpr, but also a list of arrangements that are certain to be produced, which can be relied on by the next steps in the plan. Each of the arrangement keys is associated with an MFP that must be applied if that arrangement is used, to back out the permutation associated with that arrangement.

An empty list of arrangement keys indicates that only a Collection stream can be assumed to exist.

fn lower_mir_expr_stack_safe( &mut self, expr: &MirRelationExpr, ) -> Result<LoweredExpr, String>

fn lower_reduce( &mut self, input: &MirRelationExpr, group_key: &Vec<MirScalarExpr>, aggregates: &Vec<AggregateExpr>, monotonic: &bool, expected_group_size: &Option<u64>, mfp_on_top: &mut MapFilterProject, fused_unnest_list: bool, ) -> Result<LoweredExpr, String>

Lowers a Reduce with the given fields and an mfp_on_top, which is the MFP that is originally on top of the Reduce. This MFP, or a part of it, might be fused into the Reduce, in which case mfp_on_top is mutated to be the residual MFP, i.e., what was not fused.

pub fn arrange_by( &mut self, plan: LirRelationExpr, collections: AvailableCollections, old_collections: &AvailableCollections, arity: usize, has_future_updates: bool, ) -> LirRelationExpr

Replace the plan with another one that has the collection in some additional forms.