Enum mz_persist_client::project::ProjectionPushdown

pub enum ProjectionPushdown {
    FetchAll,
    IgnoreAllNonErr {
        err_col_name: &'static str,
        key_bytes: Vec<u8>,
        val_bytes: Vec<u8>,
    },
}

Information about which columns of persist-stored data may not be needed.

TODO: This is mostly a placeholder for real projection pushdown, but in the short-term it allows us a special case of projection pushdown: ignoring all non-Err data. See ProjectionPushdown::try_optimize_ignored_data_fetch.

Variants

FetchAll

Fetch all columns.

IgnoreAllNonErr

For data with a top-level error column in the structured representation, ignore all columns except for data in parts that may contain an error. This may seem like a peculiar set of requirements, but it enables the aggressive Self::try_optimize_ignored_data_fetch optimization and it corresponds to a common query shape: SELECT count(*).

This error bit is certainly a bit of an abstraction breakage in the “persist is independent of mz” story, but it should go away when we implement full projection pushdown.

Fields

err_col_name: &'static str

The name of the top-level error column.

key_bytes: Vec<u8>

The Codec encoded key corresponding to ignored data.

val_bytes: Vec<u8>

The Codec encoded val corresponding to ignored data.

Implementations

impl ProjectionPushdown

pub(crate) fn try_optimize_ignored_data_fetch<T: Timestamp + Codec64>( &self, cfg: &ConfigSet, metrics: &Metrics, as_of: &Antichain<T>, desc: &Description<T>, part: &BatchPart<T>, ) -> Option<BatchPart<T>>

If relevant, applies the Self::IgnoreAllNonErr projection to a part about to be fetched.

If a part contains data from entirely before a snapshot as_of, and the pushed-down MFP projects to an empty list of columns, and we can prove that the part is error free, then we can use the diff sum from stats instead of loading the data. In this case, we return a Some with a replacement BatchPart. In all other cases, a None.

• Summing the diffs in a part is equivalent to projecting the row in each tuple to an empty row and then consolidating. (Which is pretty much how select count(*) queries get compiled today.)
• If the as-of timestamp falls in the middle of a part, we can just fetch and process the part as normal. The optimization can still provide a speedup for other parts. TODO: We could improve this by keeping track in metadata of the largest timestamp(s) in a hollow part.

Trait Implementations

impl Clone for ProjectionPushdown

fn clone(&self) -> ProjectionPushdown

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ProjectionPushdown

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ProjectionPushdown

fn default() -> Self

Returns the “default value” for a type.