mz_adapter::catalog::migrate

Function ast_rewrite_sources_to_tables

Source
fn ast_rewrite_sources_to_tables(
    tx: &mut Transaction<'_>,
    now: NowFn,
) -> Result<(), Error>
Expand description

Migrates all sources to use the new sources as tables model

First we migrate existing CREATE SUBSOURCE statements, turning them into CREATE TABLE .. FROM SOURCE statements. This covers existing Postgres, MySQL, and multi-output (tpch, auction, marketing) load-generator subsources.

Second we migrate existing CREATE SOURCE statements for these multi-output sources to remove any subsource-specific options (e.g. TEXT COLUMNS).

Third we migrate existing single-output CREATE SOURCE statements. This includes existing Kafka and single-output load-generator subsources. This will generate an additional CREATE TABLE .. FROM SOURCE statement that copies over all the export-specific options. This table will use to the existing source statement’s persist shard but use a new GlobalID. The original source statement will be updated to remove the export-specific options, renamed to <original_name>_source, and use a new empty shard while keeping its same GlobalId.