pub struct Catalog<S> {
state: CatalogState,
storage: Arc<Mutex<Connection<S>>>,
transient_revision: u64,
}
Expand description
A Catalog
keeps track of the SQL objects known to the planner.
For each object, it keeps track of both forward and reverse dependencies: i.e., which objects are depended upon by the object, and which objects depend upon the object. It enforces the SQL rules around dropping: an object cannot be dropped until all of the objects that depend upon it are dropped. It also enforces uniqueness of names.
SQL mandates a hierarchy of exactly three layers. A catalog contains databases, databases contain schemas, and schemas contain catalog items, like sources, sinks, view, and indexes.
To the outside world, databases, schemas, and items are all identified by
name. Items can be referred to by their FullObjectName
, which fully and
unambiguously specifies the item, or a PartialObjectName
, which can omit the
database name and/or the schema name. Partial names can be converted into
full names via a complicated resolution process documented by the
CatalogState::resolve
method.
The catalog also maintains special “ambient schemas”: virtual schemas,
implicitly present in all databases, that house various system views.
The big examples of ambient schemas are pg_catalog
and mz_catalog
.
Fields
state: CatalogState
storage: Arc<Mutex<Connection<S>>>
transient_revision: u64
Implementations
sourceimpl Catalog<Sqlite>
impl Catalog<Sqlite>
sourcepub async fn open_debug_sqlite(now: NowFn) -> Result<Catalog<Sqlite>, Error>
pub async fn open_debug_sqlite(now: NowFn) -> Result<Catalog<Sqlite>, Error>
Opens a debug in-memory sqlite catalog.
See Catalog::open_debug
.
sourceimpl<S: Append> Catalog<S>
impl<S: Append> Catalog<S>
sourcepub async fn open(
config: Config<'_, S>
) -> Result<(Catalog<S>, Vec<BuiltinTableUpdate>), Error>
pub async fn open(
config: Config<'_, S>
) -> Result<(Catalog<S>, Vec<BuiltinTableUpdate>), Error>
Opens or creates a catalog that stores data at path
.
Returns the catalog and a list of updates to builtin tables that describe the initial state of the catalog.
sourceasync fn load_builtin_types(&mut self) -> Result<(), Error>
async fn load_builtin_types(&mut self) -> Result<(), Error>
Loads built-in system types into the catalog.
Built-in types sometimes have references to other built-in types, and sometimes these references are circular. This makes loading built-in types more complicated than other built-in objects, and requires us to make multiple passes over the types to correctly resolve all references.
fn resolve_builtin_type(
builtin: &BuiltinType<NameReference>,
name_to_id_map: &HashMap<&str, GlobalId>
) -> BuiltinType<IdReference>
sourcepub fn transient_revision(&self) -> u64
pub fn transient_revision(&self) -> u64
Returns the catalog’s transient revision, which starts at 1 and is incremented on every change. This is not persisted to disk, and will restart on every load.
sourcepub async fn load_catalog_items<'a>(
tx: &mut Transaction<'a, S>,
c: &Catalog<S>
) -> Result<Catalog<S>, Error>
pub async fn load_catalog_items<'a>(
tx: &mut Transaction<'a, S>,
c: &Catalog<S>
) -> Result<Catalog<S>, Error>
Takes a catalog which only has items in its on-disk storage (“unloaded”) and cannot yet resolve names, and returns a catalog loaded with those items.
This function requires transactions to support loading a catalog with the transaction’s currently in-flight updates to existing catalog objects, which is necessary for at least one catalog migration.
TODO(justin): it might be nice if these were two different types.
sourcepub async fn open_debug(stash: S, now: NowFn) -> Result<Catalog<S>, Error>
pub async fn open_debug(stash: S, now: NowFn) -> Result<Catalog<S>, Error>
Opens the catalog from stash
with parameters set appropriately for debug
contexts, like in tests.
WARNING! This function can arbitrarily fail because it does not make any effort to adjust the catalog’s contents’ structure or semantics to the currently running version, i.e. it does not apply any migrations.
This function should not be called in production contexts. Use
Catalog::open
with appropriately set configuration parameters
instead.
pub fn for_session<'a>(&'a self, session: &'a Session) -> ConnCatalog<'a>
pub fn for_sessionless_user(&self, user: String) -> ConnCatalog<'_>
pub fn for_system_session(&self) -> ConnCatalog<'_>
async fn storage<'a>(&'a self) -> MutexGuard<'a, Connection<S>>
sourceasync fn allocate_system_ids<T, F>(
&mut self,
builtins: Vec<T>,
builtin_lookup: F
) -> Result<AllocatedBuiltinSystemIds<T>, Error> where
T: Copy + Fingerprint,
F: Fn(&T) -> Option<(GlobalId, u64)>,
async fn allocate_system_ids<T, F>(
&mut self,
builtins: Vec<T>,
builtin_lookup: F
) -> Result<AllocatedBuiltinSystemIds<T>, Error> where
T: Copy + Fingerprint,
F: Fn(&T) -> Option<(GlobalId, u64)>,
Allocate new system ids for any new builtin objects and looks up existing system ids for existing builtin objects
pub async fn allocate_user_id(&mut self) -> Result<GlobalId, Error>
pub async fn allocate_oid(&mut self) -> Result<u32, Error>
pub fn resolve_database(
&self,
database_name: &str
) -> Result<&Database, SqlCatalogError>
pub fn resolve_schema(
&self,
current_database: Option<&DatabaseId>,
database_name: Option<&str>,
schema_name: &str,
conn_id: u32
) -> Result<&Schema, SqlCatalogError>
pub fn resolve_schema_in_database(
&self,
database_spec: &ResolvedDatabaseSpecifier,
schema_name: &str,
conn_id: u32
) -> Result<&Schema, SqlCatalogError>
sourcepub fn resolve_entry(
&self,
current_database: Option<&DatabaseId>,
search_path: &Vec<(ResolvedDatabaseSpecifier, SchemaSpecifier)>,
name: &PartialObjectName,
conn_id: u32
) -> Result<&CatalogEntry, SqlCatalogError>
pub fn resolve_entry(
&self,
current_database: Option<&DatabaseId>,
search_path: &Vec<(ResolvedDatabaseSpecifier, SchemaSpecifier)>,
name: &PartialObjectName,
conn_id: u32
) -> Result<&CatalogEntry, SqlCatalogError>
Resolves name
to a non-function CatalogEntry
.
sourcepub fn resolve_builtin_table(&self, builtin: &'static BuiltinTable) -> GlobalId
pub fn resolve_builtin_table(&self, builtin: &'static BuiltinTable) -> GlobalId
Resolves a BuiltinTable
.
sourcepub fn resolve_builtin_log(&self, builtin: &'static BuiltinLog) -> GlobalId
pub fn resolve_builtin_log(&self, builtin: &'static BuiltinLog) -> GlobalId
Resolves a BuiltinLog
.
sourcepub fn resolve_function(
&self,
current_database: Option<&DatabaseId>,
search_path: &Vec<(ResolvedDatabaseSpecifier, SchemaSpecifier)>,
name: &PartialObjectName,
conn_id: u32
) -> Result<&CatalogEntry, SqlCatalogError>
pub fn resolve_function(
&self,
current_database: Option<&DatabaseId>,
search_path: &Vec<(ResolvedDatabaseSpecifier, SchemaSpecifier)>,
name: &PartialObjectName,
conn_id: u32
) -> Result<&CatalogEntry, SqlCatalogError>
Resolves name
to a function CatalogEntry
.
pub fn resolve_compute_instance(
&self,
name: &str
) -> Result<&ComputeInstance, SqlCatalogError>
pub fn state(&self) -> &CatalogState
pub fn resolve_full_name(
&self,
name: &QualifiedObjectName,
conn_id: Option<u32>
) -> FullObjectName
sourcepub fn try_get_entry_in_schema(
&self,
name: &QualifiedObjectName,
conn_id: u32
) -> Option<&CatalogEntry>
pub fn try_get_entry_in_schema(
&self,
name: &QualifiedObjectName,
conn_id: u32
) -> Option<&CatalogEntry>
Returns the named catalog item, if it exists.
pub fn item_exists(&self, name: &QualifiedObjectName, conn_id: u32) -> bool
pub fn try_get_entry(&self, id: &GlobalId) -> Option<&CatalogEntry>
pub fn get_entry(&self, id: &GlobalId) -> &CatalogEntry
pub fn get_schema(
&self,
database_spec: &ResolvedDatabaseSpecifier,
schema_spec: &SchemaSpecifier,
conn_id: u32
) -> &Schema
pub fn get_mz_catalog_schema_id(&self) -> &SchemaId
pub fn get_pg_catalog_schema_id(&self) -> &SchemaId
pub fn get_information_schema_id(&self) -> &SchemaId
pub fn get_database(&self, id: &DatabaseId) -> &Database
sourcepub async fn create_temporary_schema(
&mut self,
conn_id: u32
) -> Result<(), Error>
pub async fn create_temporary_schema(
&mut self,
conn_id: u32
) -> Result<(), Error>
Creates a new schema in the Catalog
for temporary items
indicated by the TEMPORARY or TEMP keywords.
fn item_exists_in_temp_schemas(&self, conn_id: u32, item_name: &str) -> bool
pub fn drop_temp_item_ops(&mut self, conn_id: u32) -> Vec<Op>ⓘNotable traits for Vec<u8, A>impl<A> Write for Vec<u8, A> where
A: Allocator,
A: Allocator,
pub fn drop_temporary_schema(&mut self, conn_id: u32) -> Result<(), Error>
pub fn drop_database_ops(&mut self, id: Option<DatabaseId>) -> Vec<Op>ⓘNotable traits for Vec<u8, A>impl<A> Write for Vec<u8, A> where
A: Allocator,
A: Allocator,
pub fn drop_schema_ops(&mut self, id: Option<(DatabaseId, SchemaId)>) -> Vec<Op>ⓘNotable traits for Vec<u8, A>impl<A> Write for Vec<u8, A> where
A: Allocator,
A: Allocator,
pub fn drop_items_ops(&mut self, ids: &[GlobalId]) -> Vec<Op>ⓘNotable traits for Vec<u8, A>impl<A> Write for Vec<u8, A> where
A: Allocator,
A: Allocator,
fn drop_schema_items(
schema: &Schema,
by_id: &BTreeMap<GlobalId, CatalogEntry>,
ops: &mut Vec<Op>,
seen: &mut HashSet<GlobalId>
)
fn drop_item_cascade(
id: GlobalId,
by_id: &BTreeMap<GlobalId, CatalogEntry>,
ops: &mut Vec<Op>,
seen: &mut HashSet<GlobalId>
)
sourcefn temporary_ids(
&mut self,
ops: &[Op],
temporary_drops: HashSet<(u32, String)>
) -> Result<Vec<GlobalId>, Error>
fn temporary_ids(
&mut self,
ops: &[Op],
temporary_drops: HashSet<(u32, String)>
) -> Result<Vec<GlobalId>, Error>
Gets GlobalIds of temporary items to be created, checks for name collisions within a connection id.
fn add_to_audit_log(
&self,
session: Option<&Session>,
tx: &mut Transaction<'_, S>,
builtin_table_updates: &mut Vec<BuiltinTableUpdate>,
event_type: EventType,
object_type: ObjectType,
event_details: EventDetails
) -> Result<(), Error>
fn should_audit_log_item(item: &CatalogItem) -> bool
fn resolve_full_name_detail(
&self,
name: &QualifiedObjectName,
session: Option<&Session>
) -> FullNameV1
fn full_name_detail(&self, name: &FullObjectName) -> FullNameV1
pub async fn transact<F, T>(
&mut self,
session: Option<&Session>,
ops: Vec<Op>,
f: F
) -> Result<(Vec<BuiltinTableUpdate>, T), CoordError> where
F: FnOnce(&CatalogState) -> Result<T, CoordError>,
fn serialize_item(&self, item: &CatalogItem) -> Vec<u8>ⓘNotable traits for Vec<u8, A>impl<A> Write for Vec<u8, A> where
A: Allocator,
A: Allocator,
fn deserialize_item(&self, bytes: Vec<u8>) -> Result<CatalogItem, Error>
fn parse_item(
&self,
create_sql: String,
pcx: Option<&PlanContext>
) -> Result<CatalogItem, Error>
pub fn uses_tables(&self, id: GlobalId) -> bool
sourcepub fn log_dependencies(&self, id: GlobalId) -> Vec<GlobalId>ⓘNotable traits for Vec<u8, A>impl<A> Write for Vec<u8, A> where
A: Allocator,
pub fn log_dependencies(&self, id: GlobalId) -> Vec<GlobalId>ⓘNotable traits for Vec<u8, A>impl<A> Write for Vec<u8, A> where
A: Allocator,
A: Allocator,
Return the names of all log sources the given object depends on.
sourcepub fn dump(&self) -> String
pub fn dump(&self) -> String
Serializes the catalog’s in-memory state.
There are no guarantees about the format of the serialized state, except that the serialized state for two identical catalogs will compare identically.
pub fn config(&self) -> &CatalogConfig
pub fn entries(&self) -> impl Iterator<Item = &CatalogEntry>
pub fn compute_instances(&self) -> impl Iterator<Item = &ComputeInstance>
pub async fn allocate_introspection_source_indexes(
&mut self
) -> Vec<(&'static BuiltinLog, GlobalId)>
pub fn pack_item_update(
&self,
id: GlobalId,
diff: Diff
) -> Vec<BuiltinTableUpdate>ⓘNotable traits for Vec<u8, A>impl<A> Write for Vec<u8, A> where
A: Allocator,
A: Allocator,
Trait Implementations
Auto Trait Implementations
impl<S> !RefUnwindSafe for Catalog<S>
impl<S> Send for Catalog<S> where
S: Send,
impl<S> Sync for Catalog<S> where
S: Send,
impl<S> Unpin for Catalog<S>
impl<S> !UnwindSafe for Catalog<S>
Blanket Implementations
sourceimpl<T> BorrowMut<T> for T where
T: ?Sized,
impl<T> BorrowMut<T> for T where
T: ?Sized,
const: unstable · sourcefn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
sourceimpl<T> FutureExt for T
impl<T> FutureExt for T
sourcefn with_context(self, otel_cx: Context) -> WithContext<Self>
fn with_context(self, otel_cx: Context) -> WithContext<Self>
sourcefn with_current_context(self) -> WithContext<Self>
fn with_current_context(self) -> WithContext<Self>
sourceimpl<T> Instrument for T
impl<T> Instrument for T
sourcefn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
sourcefn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
sourceimpl<T> IntoRequest<T> for T
impl<T> IntoRequest<T> for T
sourcefn into_request(self) -> Request<T>
fn into_request(self) -> Request<T>
Wrap the input message T
in a tonic::Request
sourceimpl<T> ProgressEventTimestamp for T where
T: Data + Debug + Any,
impl<T> ProgressEventTimestamp for T where
T: Data + Debug + Any,
sourceimpl<P, R> ProtoType<R> for P where
R: RustType<P>,
impl<P, R> ProtoType<R> for P where
R: RustType<P>,
sourcefn into_rust(self) -> Result<R, TryFromProtoError>
fn into_rust(self) -> Result<R, TryFromProtoError>
See RustType::from_proto
.
sourcefn from_rust(rust: &R) -> P
fn from_rust(rust: &R) -> P
See RustType::into_proto
.
sourceimpl<T> ToOwned for T where
T: Clone,
impl<T> ToOwned for T where
T: Clone,
type Owned = T
type Owned = T
The resulting type after obtaining ownership.
sourcefn clone_into(&self, target: &mut T)
fn clone_into(&self, target: &mut T)
toowned_clone_into
)Uses borrowed data to replace owned data, usually by cloning. Read more
sourceimpl<T> WithSubscriber for T
impl<T> WithSubscriber for T
sourcefn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self> where
S: Into<Dispatch>,
fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self> where
S: Into<Dispatch>,
Attaches the provided Subscriber
to this type, returning a
WithDispatch
wrapper. Read more
sourcefn with_current_subscriber(self) -> WithDispatch<Self>
fn with_current_subscriber(self) -> WithDispatch<Self>
Attaches the current default Subscriber
to this type, returning a
WithDispatch
wrapper. Read more