Struct coord::catalog::Catalog

pub struct Catalog {
    state: CatalogState,
    storage: Arc<Mutex<Connection>>,
    oid_counter: u32,
    transient_revision: u64,
    config: CatalogConfig,
}

A Catalog keeps track of the SQL objects known to the planner.

For each object, it keeps track of both forward and reverse dependencies: i.e., which objects are depended upon by the object, and which objects depend upon the object. It enforces the SQL rules around dropping: an object cannot be dropped until all of the objects that depend upon it are dropped. It also enforces uniqueness of names.

SQL mandates a hierarchy of exactly three layers. A catalog contains databases, databases contain schemas, and schemas contain catalog items, like sources, sinks, view, and indexes.

To the outside world, databases, schemas, and items are all identified by name. Items can be referred to by their FullName, which fully and unambiguously specifies the item, or a PartialName, which can omit the database name and/or the schema name. Partial names can be converted into full names via a complicated resolution process documented by the Catalog::resolve method.

The catalog also maintains special “ambient schemas”: virtual schemas, implicitly present in all databases, that house various system views. The big examples of ambient schemas are pg_catalog and mz_catalog.

Fields

state: CatalogState

storage: Arc<Mutex<Connection>>

oid_counter: u32

transient_revision: u64

config: CatalogConfig

Implementations

impl Catalog

pub async fn open(
    config: Config<'_>
) -> Result<(Catalog, Vec<BuiltinTableUpdate>), Error>

Opens or creates a catalog that stores data at path.

Returns the catalog and a list of updates to builtin tables that describe the initial state of the catalog.

pub fn transient_revision(&self) -> u64

Retuns the catalog’s transient revision, which starts at 1 and is incremented on every change. This is not persisted to disk, and will restart on every load.

pub fn load_catalog_items(
    tx: &mut Transaction<'_>,
    c: &Catalog
) -> Result<Catalog, Error>

Takes a catalog which only has items in its on-disk storage (“unloaded”) and cannot yet resolve names, and returns a catalog loaded with those items.

This function requires transactions to support loading a catalog with the transaction’s currently in-flight updates to existing catalog objects, which is necessary for at least one catalog migration.

TODO(justin): it might be nice if these were two different types.

pub async fn open_debug(path: &Path, now: NowFn) -> Result<Catalog, Error>

Opens the catalog at path with parameters set appropriately for debug contexts, like in tests.

WARNING! This function can arbitrarily fail because it does not make any effort to adjust the catalog’s contents’ structure or semantics to the currently running version, i.e. it does not apply any migrations.

This function should not be called in production contexts. Use Catalog::open with appropriately set configuration parameters instead.

pub fn for_session<'a>(&'a self, session: &'a Session) -> ConnCatalog<'a>

pub fn for_sessionless_user(&self, user: String) -> ConnCatalog<'_>

pub fn for_system_session(&self) -> ConnCatalog<'_>

fn storage(&self) -> MutexGuard<'_, Connection>

pub fn allocate_id(&mut self) -> Result<GlobalId, Error>

pub fn allocate_oid(&mut self) -> Result<u32, Error>

pub fn resolve_schema(
    &self,
    current_database: &str,
    database: Option<String>,
    schema_name: &str,
    conn_id: u32
) -> Result<&Schema, SqlCatalogError>

pub fn resolve_item(
    &self,
    current_database: &str,
    search_path: &[&str],
    name: &PartialName,
    conn_id: u32
) -> Result<&CatalogEntry, SqlCatalogError>

Resolves name to a non-function CatalogEntry.

pub fn resolve_function(
    &self,
    current_database: &str,
    search_path: &[&str],
    name: &PartialName,
    conn_id: u32
) -> Result<&CatalogEntry, SqlCatalogError>

Resolves name to a function CatalogEntry.

pub fn resolve(
    &self,
    get_schema_entries: fn(_: &Schema) -> &BTreeMap<String, GlobalId>,
    current_database: &str,
    search_path: &[&str],
    name: &PartialName,
    conn_id: u32
) -> Result<&CatalogEntry, SqlCatalogError>

Resolves PartialName into a FullName.

If name does not specify a database, the current_database is used. If name does not specify a schema, then the schemas in search_path are searched in order.

pub fn state(&self) -> &CatalogState

pub fn try_get(&self, name: &FullName, conn_id: u32) -> Option<&CatalogEntry>

Returns the named catalog item, if it exists.

pub fn try_get_by_id(&self, id: GlobalId) -> Option<&CatalogEntry>

pub fn get_by_id(&self, id: &GlobalId) -> &CatalogEntry

pub fn insert_item(
    &mut self,
    id: GlobalId,
    oid: u32,
    name: FullName,
    item: CatalogItem
)

pub fn get_by_oid(&self, oid: &u32) -> &CatalogEntry

pub fn create_temporary_schema(&mut self, conn_id: u32) -> Result<(), Error>

Creates a new schema in the Catalog for temporary items indicated by the TEMPORARY or TEMP keywords.

fn item_exists_in_temp_schemas(&mut self, conn_id: u32, item_name: &str) -> bool

pub fn drop_temp_item_ops(&mut self, conn_id: u32) -> Vec<Op>

pub fn drop_temporary_schema(&mut self, conn_id: u32) -> Result<(), Error>

fn get_schema(
    &self,
    database_spec: &DatabaseSpecifier,
    schema_name: &str,
    conn_id: u32
) -> Option<&Schema>

pub fn drop_database_ops(&mut self, name: String) -> Vec<Op>

pub fn drop_schema_ops(&mut self, name: SchemaName) -> Vec<Op>

pub fn drop_items_ops(&mut self, ids: &[GlobalId]) -> Vec<Op>

fn drop_schema_items(
    schema: &Schema,
    by_id: &BTreeMap<GlobalId, CatalogEntry>,
    ops: &mut Vec<Op>,
    seen: &mut HashSet<GlobalId>
)

fn drop_item_cascade(
    id: GlobalId,
    by_id: &BTreeMap<GlobalId, CatalogEntry>,
    ops: &mut Vec<Op>,
    seen: &mut HashSet<GlobalId>
)

pub fn enable_index_ops(&mut self, id: GlobalId) -> Result<Vec<Op>, Error>

Returns the Ops necessary to enable an index.

Panics

Panics if id is not the id of a CatalogItem::Index.

fn temporary_ids(
    &mut self,
    ops: &[Op],
    temporary_drops: HashSet<(u32, String)>
) -> Result<Vec<GlobalId>, Error>

Gets GlobalIds of temporary items to be created, checks for name collisions within a connection id.

pub fn insert_timestamp_bindings(
    &mut self,
    timestamps: impl IntoIterator<Item = (GlobalId, String, Timestamp, i64)>
) -> Result<(), Error>

Insert timestamp bindings into SQLite, and ignores duplicate timestamp bindings.

Each individual binding is listed as (source_id, partition_id, timestamp, offset) and it indicates that all data from (source, partition) for offsets < offset, can be assigned timestamp iff offset is the minimal such offset (this is a way to encode a [start, end) offset interval without having to duplicate adjacent starts and ends in storage). TODO: we intentionally ignore duplicates because BYO sources can send multiple copies of the same timestamp.

pub fn load_timestamp_bindings(
    &mut self,
    source_id: GlobalId
) -> Result<Vec<(PartitionId, Timestamp, MzOffset)>, Error>

Read all available timestamp bindings for a source

Returns its output sorted by (partition, timestamp)

pub fn compact_timestamp_bindings(
    &mut self,
    source_id: GlobalId,
    frontier: Timestamp
) -> Result<(), Error>

Compact timestamp bindings for a source

In practice this ends up being “remove all bindings less than a given timestamp” because all offsets are then assigned to the next available binding.

pub fn transact<F, T>(
    &mut self,
    ops: Vec<Op>,
    f: F
) -> Result<(Vec<BuiltinTableUpdate>, T), CoordError> where
    F: FnOnce(&CatalogState) -> Result<T, CoordError>, 

fn serialize_item(&self, item: &CatalogItem) -> Vec<u8>

fn deserialize_item(
    &self,
    id: GlobalId,
    bytes: Vec<u8>
) -> Result<CatalogItem, Error>

fn parse_item(
    &self,
    id: GlobalId,
    create_sql: String,
    pcx: Option<&PlanContext>,
    table_persist_name: Option<String>,
    source_persist_details: Option<SerializedSourcePersistDetails>
) -> Result<CatalogItem, Error>

pub fn index_enabled_by_default(&self, id: &GlobalId) -> bool

Returns the default value for an Index’s enabled field.

Note that it is the caller’s responsibility to ensure that the id is used for an Index.

pub fn enabled_indexes(
    &self
) -> &HashMap<GlobalId, Vec<(GlobalId, Vec<MirScalarExpr>)>>

Returns a mapping that indicates all indices that are available for each item in the catalog.

Note that when self.config.disable_user_indexes is true, this does not include any user indexes.

pub fn is_index_enabled(&self, id: &GlobalId) -> bool

Returns whether or not an index is enabled.

Panics

Panics if id does not belong to a CatalogItem::Index.

pub fn get_indexes_on(&self, id: GlobalId) -> Vec<GlobalId>

Returns all indexes on this object known in the catalog.

pub fn default_index_for(&self, id: GlobalId) -> Option<GlobalId>

Returns the default index for the specified id.

Panics if id does not exist, or if id is not an object on which indexes can be built.

pub fn ensure_default_index_enabled(&self, id: GlobalId) -> Result<(), Error>

Returns an error if the object’s default index is disabled.

Note that this function is really only meant to be used with tables.

Panics

Panics if the object identified with id does not have a default index.

pub fn nearest_indexes(
    &self,
    ids: &[GlobalId]
) -> (Vec<GlobalId>, Vec<GlobalId>)

pub fn uses_tables(&self, id: GlobalId) -> bool

pub fn dump(&self) -> String

Serializes the catalog’s in-memory state.

There are no guarantees about the format of the serialized state, except that the serialized state for two identical catalogs will compare identically.

pub fn config(&self) -> &CatalogConfig

pub fn entries(&self) -> impl Iterator<Item = &CatalogEntry>

pub fn schema_adjacent_indexed_relations(
    &self,
    ids: &[GlobalId],
    conn_id: u32
) -> Vec<GlobalId>

Returns all tables, views, and sources in the same schemas as a set of input ids. The indexes of all relations are included.

Trait Implementations

impl Clone for Catalog

fn clone(&self) -> Catalog

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Catalog

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.