Enum mz_compute_types::plan::Plan

pub enum Plan<T = Timestamp> {
    Constant {
        rows: Result<Vec<(Row, T, Diff)>, EvalError>,
        lir_id: LirId,
    },
    Get {
        id: Id,
        keys: AvailableCollections,
        plan: GetPlan,
        lir_id: LirId,
    },
    Let {
        id: LocalId,
        value: Box<Plan<T>>,
        body: Box<Plan<T>>,
        lir_id: LirId,
    },
    LetRec {
        ids: Vec<LocalId>,
        values: Vec<Plan<T>>,
        limits: Vec<Option<LetRecLimit>>,
        body: Box<Plan<T>>,
        lir_id: LirId,
    },
    Mfp {
        input: Box<Plan<T>>,
        mfp: MapFilterProject,
        input_key_val: Option<(Vec<MirScalarExpr>, Option<Row>)>,
        lir_id: LirId,
    },
    FlatMap {
        input: Box<Plan<T>>,
        func: TableFunc,
        exprs: Vec<MirScalarExpr>,
        mfp_after: MapFilterProject,
        input_key: Option<Vec<MirScalarExpr>>,
        lir_id: LirId,
    },
    Join {
        inputs: Vec<Plan<T>>,
        plan: JoinPlan,
        lir_id: LirId,
    },
    Reduce {
        input: Box<Plan<T>>,
        key_val_plan: KeyValPlan,
        plan: ReducePlan,
        input_key: Option<Vec<MirScalarExpr>>,
        mfp_after: MapFilterProject,
        lir_id: LirId,
    },
    TopK {
        input: Box<Plan<T>>,
        top_k_plan: TopKPlan,
        lir_id: LirId,
    },
    Negate {
        input: Box<Plan<T>>,
        lir_id: LirId,
    },
    Threshold {
        input: Box<Plan<T>>,
        threshold_plan: ThresholdPlan,
        lir_id: LirId,
    },
    Union {
        inputs: Vec<Plan<T>>,
        consolidate_output: bool,
        lir_id: LirId,
    },
    ArrangeBy {
        input: Box<Plan<T>>,
        forms: AvailableCollections,
        input_key: Option<Vec<MirScalarExpr>>,
        input_mfp: MapFilterProject,
        lir_id: LirId,
    },
}

A rendering plan with as much conditional logic as possible removed.

Variants

Constant

A collection containing a pre-determined collection.

Fields

rows: Result<Vec<(Row, T, Diff)>, EvalError>

Explicit update triples for the collection.

lir_id: LirId

A dataflow-local identifier.

Get

A reference to a bound collection.

This is commonly either an external reference to an existing source or maintained arrangement, or an internal reference to a Let identifier.

Fields

id: Id

A global or local identifier naming the collection.

keys: AvailableCollections

Arrangements that will be available.

The collection will also be loaded if available, which it will not be for imported data, but which it may be for locally defined data.

plan: GetPlan

The actions to take when introducing the collection.

lir_id: LirId

A dataflow-local identifier.

Let

Binds value to id, and then results in body with that binding.

This stage has the effect of sharing value across multiple possible uses in body, and is the only mechanism we have for sharing collection information across parts of a dataflow.

The binding is not available outside of body.

Fields

id: LocalId

The local identifier to be used, available to body as Id::Local(id).

value: Box<Plan<T>>

The collection that should be bound to id.

body: Box<Plan<T>>

The collection that results, which is allowed to contain Get stages that reference Id::Local(id).

lir_id: LirId

A dataflow-local identifier.

LetRec

Binds values to ids, evaluates them potentially recursively, and returns body.

All bindings are available to all bindings, and to body. The contents of each binding are initially empty, and then updated through a sequence of iterations in which each binding is updated in sequence, from the most recent values of all bindings.

Fields

ids: Vec<LocalId>

The local identifiers to be used, available to body as Id::Local(id).

values: Vec<Plan<T>>

The collection that should be bound to id.

limits: Vec<Option<LetRecLimit>>

Maximum number of iterations. See further info on the MIR LetRec.

body: Box<Plan<T>>

The collection that results, which is allowed to contain Get stages that reference Id::Local(id).

lir_id: LirId

A dataflow-local identifier.

Mfp

Map, Filter, and Project operators.

This stage contains work that we would ideally like to fuse to other plan stages, but for practical reasons cannot. For example: threshold, topk, and sometimes reduce stages are not able to absorb this operator.

Fields

input: Box<Plan<T>>

The input collection.

mfp: MapFilterProject

Linear operator to apply to each record.

input_key_val: Option<(Vec<MirScalarExpr>, Option<Row>)>

Whether the input is from an arrangement, and if so, whether we can seek to a specific value therein

lir_id: LirId

A dataflow-local identifier.

FlatMap

A variable number of output records for each input record.

This stage is a bit of a catch-all for logic that does not easily fit in map stages. This includes table valued functions, but also functions of multiple arguments, and functions that modify the sign of updates.

This stage allows a MapFilterProject operator to be fused to its output, and this can be very important as otherwise the output of func is just appended to the input record, for as many outputs as it has. This has the unpleasant default behavior of repeating potentially large records that are being unpacked, producing quadratic output in those cases. Instead, in these cases use a mfp member that projects away these large fields.

Fields

input: Box<Plan<T>>

The input collection.

func: TableFunc

The variable-record emitting function.

exprs: Vec<MirScalarExpr>

Expressions that for each row prepare the arguments to func.

mfp_after: MapFilterProject

Linear operator to apply to each record produced by func.

input_key: Option<Vec<MirScalarExpr>>

The particular arrangement of the input we expect to use, if any

lir_id: LirId

A dataflow-local identifier.

Join

A multiway relational equijoin, with fused map, filter, and projection.

This stage performs a multiway join among inputs, using the equality constraints expressed in plan. The plan also describes the implementation strategy we will use, and any pushed down per-record work.

Fields

inputs: Vec<Plan<T>>

An ordered list of inputs that will be joined.

plan: JoinPlan

Detailed information about the implementation of the join.

This includes information about the implementation strategy, but also any map, filter, project work that we might follow the join with, but potentially pushed down into the implementation of the join.

lir_id: LirId

A dataflow-local identifier.

Reduce

Aggregation by key.

Fields

input: Box<Plan<T>>

The input collection.

key_val_plan: KeyValPlan

A plan for changing input records into key, value pairs.

plan: ReducePlan

A plan for performing the reduce.

The implementation of reduction has several different strategies based on the properties of the reduction, and the input itself. Please check out the documentation for this type for more detail.

input_key: Option<Vec<MirScalarExpr>>

The particular arrangement of the input we expect to use, if any

mfp_after: MapFilterProject

An MFP that must be applied to results. The projection part of this MFP must preserve the key for the reduction; otherwise, the results become undefined. Additionally, the MFP must be free from temporal predicates so that it can be readily evaluated.

lir_id: LirId

A dataflow-local identifier.

TopK

Key-based “Top K” operator, retaining the first K records in each group.

Fields

input: Box<Plan<T>>

The input collection.

top_k_plan: TopKPlan

A plan for performing the Top-K.

The implementation of reduction has several different strategies based on the properties of the reduction, and the input itself. Please check out the documentation for this type for more detail.

lir_id: LirId

A dataflow-local identifier.

Negate

Inverts the sign of each update.

Fields

input: Box<Plan<T>>

The input collection.

lir_id: LirId

A dataflow-local identifier.

Threshold

Filters records that accumulate negatively.

Although the operator suppresses updates, it is a stateful operator taking resources proportional to the number of records with non-zero accumulation.

Fields

input: Box<Plan<T>>

The input collection.

threshold_plan: ThresholdPlan

A plan for performing the threshold.

The implementation of reduction has several different strategies based on the properties of the reduction, and the input itself. Please check out the documentation for this type for more detail.

lir_id: LirId

A dataflow-local identifier.

Union

Adds the contents of the input collections.

Importantly, this is multiset union, so the multiplicities of records will add. This is in contrast to set union, where the multiplicities would be capped at one. A set union can be formed with Union followed by Reduce implementing the “distinct” operator.

Fields

inputs: Vec<Plan<T>>

The input collections

consolidate_output: bool

Whether to consolidate the output, e.g., cancel negated records.

lir_id: LirId

A dataflow-local identifier.

ArrangeBy

The input plan, but with additional arrangements.

This operator does not change the logical contents of input, but ensures that certain arrangements are available in the results. This operator can be important for e.g. the Join stage which benefits from multiple arrangements or to cap a Plan so that indexes can be exported.

Fields

input: Box<Plan<T>>

The input collection.

forms: AvailableCollections

A list of arrangement keys, and possibly a raw collection, that will be added to those of the input.

If any of these collection forms are already present in the input, they have no effect.

input_key: Option<Vec<MirScalarExpr>>

The key that must be used to access the input.

input_mfp: MapFilterProject

The MFP that must be applied to the input.

lir_id: LirId

A dataflow-local identifier.

Implementations

impl<T> Plan<T>

pub fn children(&self) -> impl Iterator<Item = &Self>

Iterates through references to child expressions.

pub fn children_mut(&mut self) -> impl Iterator<Item = &mut Self>

Iterates through mutable references to child expressions.

impl<T> Plan<T>

pub fn lir_id(&self) -> LirId

Return this plan’s LirId.

impl Plan

pub fn pretty(&self) -> String

Pretty-print this Plan to a string.

pub fn explain( &self, config: &ExplainConfig, humanizer: Option<&dyn ExprHumanizer> ) -> String

Pretty-print this Plan to a string using a custom ExplainConfig and an optionally provided ExprHumanizer.

impl<T: Timestamp> Plan<T>

pub fn finalize_dataflow( desc: DataflowDescription<OptimizedMirRelationExpr>, features: &OptimizerFeatures ) -> Result<DataflowDescription<Self>, String>

Convert the dataflow description into one that uses render plans.

Trait Implementations

impl Arbitrary for Plan

type Strategy = BoxedStrategy<Plan>

The type of Strategy used to generate values of type Self.

type Parameters = ()

The type of parameters that arbitrary_with accepts for configuration of the generated Strategy. Parameters must implement Default.

fn arbitrary_with(_: Self::Parameters) -> Self::Strategy

Generates a Strategy for producing arbitrary values of type the implementing type (Self). The strategy is passed the arguments given in args.

fn arbitrary() -> Self::Strategy

Generates a Strategy for producing arbitrary values of type the implementing type (Self).

impl<T: Clone> Clone for Plan<T>

fn clone(&self) -> Plan<T>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<T> CollectionPlan for Plan<T>

fn depends_on_into(&self, out: &mut BTreeSet<GlobalId>)

Collects the set of global identifiers from dataflows referenced in Get.

fn depends_on(&self) -> BTreeSet<GlobalId>

Returns the set of global identifiers from dataflows referenced in Get.

impl<T: Debug> Debug for Plan<T>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de, T> Deserialize<'de> for Plan<T>

where T: Deserialize<'de>,

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl DisplayText<PlanRenderingContext<'_, Plan>> for Plan

fn fmt_text( &self, f: &mut Formatter<'_>, ctx: &mut PlanRenderingContext<'_, Plan> ) -> Result

impl<T> From<Plan<T>> for FlatPlan<T>

fn from(plan: Plan<T>) -> Self

Flatten the given Plan into a FlatPlan.

impl<T: Ord> Ord for Plan<T>

fn cmp(&self, other: &Plan<T>) -> Ordering

This method returns an Ordering between self and other.

fn max(self, other: Self) -> Self

where Self: Sized,

Compares and returns the maximum of two values.

fn min(self, other: Self) -> Self

where Self: Sized,

Compares and returns the minimum of two values.

fn clamp(self, min: Self, max: Self) -> Self

where Self: Sized + PartialOrd,

Restrict a value to a certain interval.

impl<T: PartialEq> PartialEq for Plan<T>

fn eq(&self, other: &Plan<T>) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl<T: PartialOrd> PartialOrd for Plan<T>

fn partial_cmp(&self, other: &Plan<T>) -> Option<Ordering>

This method returns an ordering between self and other values if one exists.

fn lt(&self, other: &Rhs) -> bool

This method tests less than (for self and other) and is used by the < operator.

fn le(&self, other: &Rhs) -> bool

This method tests less than or equal to (for self and other) and is used by the <= operator.

fn gt(&self, other: &Rhs) -> bool

This method tests greater than (for self and other) and is used by the > operator.

fn ge(&self, other: &Rhs) -> bool

This method tests greater than or equal to (for self and other) and is used by the >= operator.

impl<T> Serialize for Plan<T>

where T: Serialize,

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl<T: Eq> Eq for Plan<T>

impl<T> StructuralPartialEq for Plan<T>