Expand description
Dependency extraction and graph assembly from compiled project state.
This module walks the SQL AST of every object to build a project-wide
dependency graph and determine schema types. Query-level traversal is
delegated to mz-sql-parser’s auto-generated Visit trait — a
DependencyVisitor overrides visit_query (for CTE scope management)
and visit_table_factor (to collect table references as dependencies).
All other AST nodes (expressions, set operations, etc.) are handled by
the default traversal.
§CTE Scoping
CTE references must be excluded from the dependency set because they
are query-local names, not database objects. The visitor uses
CteScope to track CTE names across nested queries. Simple CTE names are
introduced incrementally (each body sees only earlier siblings), while
mutually-recursive blocks push all names up front — mirroring Materialize’s
own resolver. A simple CTE that shadows a catalog object therefore still
records a dependency on that object when it references it inside its own
body (e.g. WITH products AS (SELECT * FROM products) ...).
§Statement-Level Dispatch
The top-level extract_dependencies function matches on statement type
and calls visitor.visit_query() only on the relevant subtree (e.g., the
query body of a CREATE VIEW, not the view name itself). Non-query
dependencies (source connections, connection options) are extracted by
dedicated helper functions.
§Dependency Validation
After graph assembly, external_dependencies (the set of references to
objects not defined in the project) is validated against the dependencies
declared in project.toml via validate_dependencies(). This cross-check
produces two sets: undeclared external references (hard error) and declared
dependencies that are never referenced (warning).
Structs§
- Dependency
Validation 🔒 - Result of validating declared dependencies against discovered external references.
- Dependency
Visitor 🔒 - Visitor that collects table reference dependencies from query ASTs.
- Processed
Object 🔒 - The result of processing a single compiled object through dependency extraction.
- Typed
Object 🔒Task - A flattened compiled object with its database/schema context, used as the unit of work for dependency extraction.
Functions§
- determine_
schema_ 🔒type - Determine the schema type based on the objects it contains.
- extract_
connection_ 🔒option_ deps - Extract dependencies from connection options (secrets, other connections).
- extract_
dependencies 🔒 - Extract all dependencies from a statement.
- extract_
external_ 🔒indexes - Find all external indexes on an object. That is, any index that lives on a different cluster than the one where the main object is installed.
- extract_
source_ 🔒connection_ dep - Extract the connection dependency from a source’s connection clause.
- extract_
with_ 🔒option_ value_ deps - Extract dependencies from a single WithOptionValue, recursing into nested structures.
- validate_
dependencies 🔒 - Cross-reference declared dependencies (from project.toml) against discovered external references (from the compiled dependency graph).