Module mz_transform::predicate_pushdown

Pushes predicates down through other operators.

This action generally improves the quality of the query, in that selective per-record filters reduce the volume of data before they arrive at more expensive operators.

The one time when this action might not improve the quality of a query is if a filter gets pushed down on an arrangement because that blocks arrangement reuse. It assumed that actions that need an arrangement are responsible for lifting filters out of the way.

Predicate pushdown will not push down literal errors, unless it is certain that the literal errors will be unconditionally evaluated. For example, the pushdown will not happen if not all predicates can be pushed down (e.g. reduce and map), or if we are not certain that the input is non-empty (e.g. join). Note that this is not addressing the problem in its full generality, because this problem can occur with any function call that might error (although much more rarely than with literal errors). See https://github.com/MaterializeInc/database-issues/issues/4972#issuecomment-1547391011

use mz_expr::{BinaryFunc, MirRelationExpr, MirScalarExpr};
use mz_ore::id_gen::IdGen;
use mz_repr::{ColumnType, Datum, RelationType, ScalarType};
use mz_repr::optimize::OptimizerFeatures;
use mz_transform::{typecheck, Transform, TransformCtx};
use mz_transform::dataflow::DataflowMetainfo;

use mz_transform::predicate_pushdown::PredicatePushdown;

let input1 = MirRelationExpr::constant(vec![], RelationType::new(vec![
    ScalarType::Bool.nullable(false),
]));
let input2 = MirRelationExpr::constant(vec![], RelationType::new(vec![
    ScalarType::Bool.nullable(false),
]));
let input3 = MirRelationExpr::constant(vec![], RelationType::new(vec![
    ScalarType::Bool.nullable(false),
]));
let join = MirRelationExpr::join(
    vec![input1.clone(), input2.clone(), input3.clone()],
    vec![vec![(0, 0), (2, 0)].into_iter().collect()],
);

let predicate0 = MirScalarExpr::column(0);
let predicate1 = MirScalarExpr::column(1);
let predicate01 = MirScalarExpr::column(0).call_binary(MirScalarExpr::column(2), BinaryFunc::AddInt64);
let predicate012 = MirScalarExpr::literal_false();

let mut expr = join.filter(
   vec![
       predicate0.clone(),
       predicate1.clone(),
       predicate01.clone(),
       predicate012.clone(),
   ]);

let features = OptimizerFeatures::default();
let typecheck_ctx = typecheck::empty_context();
let mut df_meta = DataflowMetainfo::default();
let mut transform_ctx = TransformCtx::local(&features, &typecheck_ctx, &mut df_meta, None);

PredicatePushdown::default().transform(&mut expr, &mut transform_ctx);

let predicate00 = MirScalarExpr::column(0).call_binary(MirScalarExpr::column(0), BinaryFunc::AddInt64);
let expected_expr = MirRelationExpr::join(
    vec![
        input1.clone().filter(vec![predicate0.clone(), predicate00.clone()]),
        input2.clone().filter(vec![predicate0.clone()]),
        input3.clone().filter(vec![predicate0, predicate00])
    ],
    vec![vec![(0, 0), (2, 0)].into_iter().collect()],
).filter(vec![predicate012]);
assert_eq!(expected_expr, expr)

Structs

• PredicatePushdown
  Pushes predicates down through other operators.