Struct mz_coord::catalog::Catalog

pub struct Catalog<S> {
    state: CatalogState,
    storage: Arc<Mutex<Connection<S>>>,
    transient_revision: u64,
}

A Catalog keeps track of the SQL objects known to the planner.

For each object, it keeps track of both forward and reverse dependencies: i.e., which objects are depended upon by the object, and which objects depend upon the object. It enforces the SQL rules around dropping: an object cannot be dropped until all of the objects that depend upon it are dropped. It also enforces uniqueness of names.

SQL mandates a hierarchy of exactly three layers. A catalog contains databases, databases contain schemas, and schemas contain catalog items, like sources, sinks, view, and indexes.

To the outside world, databases, schemas, and items are all identified by name. Items can be referred to by their FullObjectName, which fully and unambiguously specifies the item, or a PartialObjectName, which can omit the database name and/or the schema name. Partial names can be converted into full names via a complicated resolution process documented by the CatalogState::resolve method.

The catalog also maintains special “ambient schemas”: virtual schemas, implicitly present in all databases, that house various system views. The big examples of ambient schemas are pg_catalog and mz_catalog.

Fields

state: CatalogState

storage: Arc<Mutex<Connection<S>>>

transient_revision: u64

Implementations

impl Catalog<Sqlite>

pub async fn open_debug_sqlite(now: NowFn) -> Result<Catalog<Sqlite>, Error>

Opens a debug in-memory sqlite catalog.

See Catalog::open_debug.

impl Catalog<Postgres>

pub async fn open_debug_postgres(
    url: String,
    schema: Option<String>,
    now: NowFn
) -> Result<Catalog<Postgres>, Error>

Opens a debug postgres catalog at url.

If specified, schema will set the connection’s search_path to schema.

See Catalog::open_debug.

impl<S: Append> Catalog<S>

pub async fn open(
    config: Config<'_, S>
) -> Result<(Catalog<S>, Vec<BuiltinTableUpdate>), Error>

Opens or creates a catalog that stores data at path.

Returns the catalog and a list of updates to builtin tables that describe the initial state of the catalog.

async fn load_builtin_types(&mut self) -> Result<(), Error>

Loads built-in system types into the catalog.

Built-in types sometimes have references to other built-in types, and sometimes these references are circular. This makes loading built-in types more complicated than other built-in objects, and requires us to make multiple passes over the types to correctly resolve all references.

fn resolve_builtin_type(
    builtin: &BuiltinType<NameReference>,
    name_to_id_map: &HashMap<&str, GlobalId>
) -> BuiltinType<IdReference>

pub fn transient_revision(&self) -> u64

Returns the catalog’s transient revision, which starts at 1 and is incremented on every change. This is not persisted to disk, and will restart on every load.

pub async fn load_catalog_items<'a>(
    tx: &mut Transaction<'a, S>,
    c: &Catalog<S>
) -> Result<Catalog<S>, Error>

Takes a catalog which only has items in its on-disk storage (“unloaded”) and cannot yet resolve names, and returns a catalog loaded with those items.

This function requires transactions to support loading a catalog with the transaction’s currently in-flight updates to existing catalog objects, which is necessary for at least one catalog migration.

TODO(justin): it might be nice if these were two different types.

pub async fn open_debug(stash: S, now: NowFn) -> Result<Catalog<S>, Error>

Opens the catalog from stash with parameters set appropriately for debug contexts, like in tests.

WARNING! This function can arbitrarily fail because it does not make any effort to adjust the catalog’s contents’ structure or semantics to the currently running version, i.e. it does not apply any migrations.

This function should not be called in production contexts. Use Catalog::open with appropriately set configuration parameters instead.

pub fn for_session<'a>(&'a self, session: &'a Session) -> ConnCatalog<'a>

pub fn for_sessionless_user(&self, user: String) -> ConnCatalog<'_>

pub fn for_system_session(&self) -> ConnCatalog<'_>

async fn storage<'a>(&'a self) -> MutexGuard<'a, Connection<S>>

async fn allocate_system_ids<T, F>(
    &mut self,
    builtins: Vec<T>,
    builtin_lookup: F
) -> Result<AllocatedBuiltinSystemIds<T>, Error> where
    T: Copy + Fingerprint,
    F: Fn(&T) -> Option<(GlobalId, u64)>, 

Allocate new system ids for any new builtin objects and looks up existing system ids for existing builtin objects

pub async fn allocate_user_id(&mut self) -> Result<GlobalId, Error>

pub async fn allocate_oid(&mut self) -> Result<u32, Error>

pub fn resolve_database(
    &self,
    database_name: &str
) -> Result<&Database, SqlCatalogError>

pub fn resolve_schema(
    &self,
    current_database: Option<&DatabaseId>,
    database_name: Option<&str>,
    schema_name: &str,
    conn_id: u32
) -> Result<&Schema, SqlCatalogError>

pub fn resolve_schema_in_database(
    &self,
    database_spec: &ResolvedDatabaseSpecifier,
    schema_name: &str,
    conn_id: u32
) -> Result<&Schema, SqlCatalogError>

pub fn resolve_entry(
    &self,
    current_database: Option<&DatabaseId>,
    search_path: &Vec<(ResolvedDatabaseSpecifier, SchemaSpecifier)>,
    name: &PartialObjectName,
    conn_id: u32
) -> Result<&CatalogEntry, SqlCatalogError>

Resolves name to a non-function CatalogEntry.

pub fn resolve_builtin_table(&self, builtin: &'static BuiltinTable) -> GlobalId

Resolves a BuiltinTable.

pub fn resolve_builtin_log(&self, builtin: &'static BuiltinLog) -> GlobalId

Resolves a BuiltinLog.

pub fn resolve_function(
    &self,
    current_database: Option<&DatabaseId>,
    search_path: &Vec<(ResolvedDatabaseSpecifier, SchemaSpecifier)>,
    name: &PartialObjectName,
    conn_id: u32
) -> Result<&CatalogEntry, SqlCatalogError>

Resolves name to a function CatalogEntry.

pub fn resolve_compute_instance(
    &self,
    name: &str
) -> Result<&ComputeInstance, SqlCatalogError>

pub fn state(&self) -> &CatalogState

pub fn resolve_full_name(
    &self,
    name: &QualifiedObjectName,
    conn_id: Option<u32>
) -> FullObjectName

pub fn try_get_entry_in_schema(
    &self,
    name: &QualifiedObjectName,
    conn_id: u32
) -> Option<&CatalogEntry>

Returns the named catalog item, if it exists.

pub fn item_exists(&self, name: &QualifiedObjectName, conn_id: u32) -> bool

pub fn try_get_entry(&self, id: &GlobalId) -> Option<&CatalogEntry>

pub fn get_entry(&self, id: &GlobalId) -> &CatalogEntry

pub fn get_schema(
    &self,
    database_spec: &ResolvedDatabaseSpecifier,
    schema_spec: &SchemaSpecifier,
    conn_id: u32
) -> &Schema

pub fn get_mz_catalog_schema_id(&self) -> &SchemaId

pub fn get_pg_catalog_schema_id(&self) -> &SchemaId

pub fn get_information_schema_id(&self) -> &SchemaId

pub fn get_database(&self, id: &DatabaseId) -> &Database

pub async fn create_temporary_schema(
    &mut self,
    conn_id: u32
) -> Result<(), Error>

Creates a new schema in the Catalog for temporary items indicated by the TEMPORARY or TEMP keywords.

fn item_exists_in_temp_schemas(&self, conn_id: u32, item_name: &str) -> bool

pub fn drop_temp_item_ops(&mut self, conn_id: u32) -> Vec<Op>

pub fn drop_temporary_schema(&mut self, conn_id: u32) -> Result<(), Error>

pub fn drop_database_ops(&mut self, id: Option<DatabaseId>) -> Vec<Op>

pub fn drop_schema_ops(&mut self, id: Option<(DatabaseId, SchemaId)>) -> Vec<Op>

pub fn drop_items_ops(&mut self, ids: &[GlobalId]) -> Vec<Op>

fn drop_schema_items(
    schema: &Schema,
    by_id: &BTreeMap<GlobalId, CatalogEntry>,
    ops: &mut Vec<Op>,
    seen: &mut HashSet<GlobalId>
)

fn drop_item_cascade(
    id: GlobalId,
    by_id: &BTreeMap<GlobalId, CatalogEntry>,
    ops: &mut Vec<Op>,
    seen: &mut HashSet<GlobalId>
)

fn temporary_ids(
    &mut self,
    ops: &[Op],
    temporary_drops: HashSet<(u32, String)>
) -> Result<Vec<GlobalId>, Error>

Gets GlobalIds of temporary items to be created, checks for name collisions within a connection id.

fn add_to_audit_log(
    &self,
    session: Option<&Session>,
    tx: &mut Transaction<'_, S>,
    builtin_table_updates: &mut Vec<BuiltinTableUpdate>,
    event_type: EventType,
    object_type: ObjectType,
    event_details: EventDetails
) -> Result<(), Error>

fn should_audit_log_item(item: &CatalogItem) -> bool

fn resolve_full_name_detail(
    &self,
    name: &QualifiedObjectName,
    session: Option<&Session>
) -> FullNameV1

fn full_name_detail(&self, name: &FullObjectName) -> FullNameV1

pub async fn transact<F, T>(
    &mut self,
    session: Option<&Session>,
    ops: Vec<Op>,
    f: F
) -> Result<(Vec<BuiltinTableUpdate>, T), CoordError> where
    F: FnOnce(&CatalogState) -> Result<T, CoordError>, 

fn serialize_item(&self, item: &CatalogItem) -> Vec<u8>

fn deserialize_item(&self, bytes: Vec<u8>) -> Result<CatalogItem, Error>

fn parse_item(
    &self,
    create_sql: String,
    pcx: Option<&PlanContext>
) -> Result<CatalogItem, Error>

pub fn uses_tables(&self, id: GlobalId) -> bool

pub fn log_dependencies(&self, id: GlobalId) -> Vec<GlobalId>

Return the names of all log sources the given object depends on.

pub fn dump(&self) -> String

Serializes the catalog’s in-memory state.

There are no guarantees about the format of the serialized state, except that the serialized state for two identical catalogs will compare identically.

pub fn config(&self) -> &CatalogConfig

pub fn entries(&self) -> impl Iterator<Item = &CatalogEntry>

pub fn compute_instances(&self) -> impl Iterator<Item = &ComputeInstance>

pub async fn allocate_introspection_source_indexes(
    &mut self
) -> Vec<(&'static BuiltinLog, GlobalId)>

pub fn pack_item_update(
    &self,
    id: GlobalId,
    diff: Diff
) -> Vec<BuiltinTableUpdate>

Trait Implementations

impl<S> Clone for Catalog<S>

fn clone(&self) -> Self

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<S: Debug> Debug for Catalog<S>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.