Module pinned 

The durable form of an LIR dataflow.

A PinnedDataflow holds the durable plan for a dataflow. It is the root of the stable LIR schema traced in tests/lir_schema.rs, so every type reachable from it is part of the frozen serialization surface.

We try to avoid pinning things that can be re-derived from the catalog when loading the plan. This means keeping track of assumptions about inputs but only the wiring for outputs. The rest can be recomputed.

Loading a pinned plan is called instantiation, and it is done by means of an InstantiationContext.

Only dataflows behind catalog items can be pinned: indexes, materialized views, and metric sinks. Subscribes and one-shot copies have no durable identity to pin under, and the TryFrom conversion rejects them, along with any export under a transient id (a peek’s index, a curated metric sink) and a dataflow that exports nothing.

TryFrom also rejects a dataflow that imports or exports a builtin collection. Builtins keep their ids across releases while their schemas and definitions change, and the plan inlines the bodies of builtin views, which instantiation cannot check.

Index keys are stored as LirScalarExpr and converted back to MirScalarExpr on instantiation. LIR scalars are a subset of MIR scalars, so that direction is total, and DataflowDescription keeps its MIR-typed index keys for the optimizer’s benefit. Source operators are already an MfpPlan over LIR expressions in a lowered description and are stored as they are.

The stored form is JSON, whose map keys must be strings. Maps keyed by GlobalId serialize their keys through GlobalIdKey, the id’s string form.

Modules

global_id_keys 🔒

Serializes a GlobalId-keyed map with GlobalIdKey keys.

Structs

GlobalIdKey

A GlobalId in its string form (u1, s2, t3), for use as a map key.

IndexInfo

The catalog facts an index import is checked against and an index export is rebuilt from.

MaterializedViewInfo

The catalog facts an exported materialized view sink is rebuilt from.

MetricSinkInfo

The catalog facts an exported metric sink is rebuilt from.

PinnedDataflow

A pinned LIR plan: the optimizer’s decisions for one dataflow.

PinnedIndexImport

The plan’s assumptions about an imported index.

PinnedSink

The wiring of an exported sink. Its descriptor is rebuilt from the catalog.

PinnedSourceImport

The plan’s assumptions about an imported source.

Enums

InstantiateError

Why a PinnedDataflow could not be instantiated.

PinError

Why a DataflowDescription could not be pinned.

PinnedSinkKind

The kinds of sink a pinned dataflow may export.

Traits

InstantiationContext

What PinnedDataflow::instantiate needs from its environment: catalog facts about the plan’s imports and exports, looked up by id, and fresh transient ids. Each lookup returns None when no item of that kind exists.

Functions

replace_global_ids 🔒