Type Alias mz_compute_types::dataflows::DataflowDesc

pub type DataflowDesc = DataflowDescription<OptimizedMirRelationExpr, ()>;

A commonly used name for dataflows contain MIR expressions.

Aliased Type

struct DataflowDesc {
    pub source_imports: BTreeMap<GlobalId, (SourceInstanceDesc<()>, bool)>,
    pub index_imports: BTreeMap<GlobalId, IndexImport>,
    pub objects_to_build: Vec<BuildDesc<OptimizedMirRelationExpr>>,
    pub index_exports: BTreeMap<GlobalId, (IndexDesc, RelationType)>,
    pub sink_exports: BTreeMap<GlobalId, ComputeSinkDesc>,
    pub as_of: Option<Antichain<Timestamp>>,
    pub until: Antichain<Timestamp>,
    pub initial_storage_as_of: Option<Antichain<Timestamp>>,
    pub refresh_schedule: Option<RefreshSchedule>,
    pub debug_name: String,
    pub time_dependence: Option<TimeDependence>,
}

Fields

source_imports: BTreeMap<GlobalId, (SourceInstanceDesc<()>, bool)>

Sources instantiations made available to the dataflow pair with monotonicity information.

index_imports: BTreeMap<GlobalId, IndexImport>

Indexes made available to the dataflow. (id of index, import)

objects_to_build: Vec<BuildDesc<OptimizedMirRelationExpr>>

Views and indexes to be built and stored in the local context. Objects must be built in the specific order, as there may be dependencies of later objects on prior identifiers.

index_exports: BTreeMap<GlobalId, (IndexDesc, RelationType)>

Indexes to be made available to be shared with other dataflows (id of new index, description of index, relationtype of base source/view/table)

sink_exports: BTreeMap<GlobalId, ComputeSinkDesc>

sinks to be created (id of new sink, description of sink)

as_of: Option<Antichain<Timestamp>>

An optional frontier to which inputs should be advanced.

If this is set, it should override the default setting determined by the upper bound of since frontiers contributing to the dataflow. It is an error for this to be set to a frontier not beyond that default.

until: Antichain<Timestamp>

Frontier beyond which the dataflow should not execute. Specifically, updates at times greater or equal to this frontier are suppressed. This is often set to as_of + 1 to enable “batch” computations. Note that frontier advancements might still happen to times that are after the until, only data is suppressed. (This is consistent with how frontier advancements can also happen before the as_of.)

initial_storage_as_of: Option<Antichain<Timestamp>>

The initial as_of when the collection is first created. Filled only for materialized views. Note that this doesn’t change upon restarts.

refresh_schedule: Option<RefreshSchedule>

The schedule of REFRESH materialized views.

debug_name: String

Human-readable name

time_dependence: Option<TimeDependence>

Description of how the dataflow’s progress relates to wall-clock time. None for unknown.