Module deployment_ops 

Deployment tracking operations.

This module contains methods for managing deployment records in the database, including creating tracking tables, inserting/querying deployment records, and managing deployment lifecycle (staging, promotion, abort).

Hydration State Machine

During staging, clusters transition through the following states:

  ┌──────────┐    all objects hydrated     ┌───────┐
  │ Hydrating │ ────────────────────────▶  │ Ready │
  └──────────┘    & lag ≤ threshold         └───────┘
       │                                       ▲
       │         lag > threshold          ┌─────────┐
       └────────────────────────────────▶ │ Lagging │
                                          └─────────┘
       │
       ▼ (no replicas OR all replicas OOM-looping)
  ┌─────────┐
  │ Failing  │
  └─────────┘

• Hydrating → objects are being backfilled; progress tracked as hydrated / total via mz_internal.mz_hydration_statuses.
• Ready → all objects hydrated and max wallclock lag ≤ threshold (default 300s / 5 minutes).
• Lagging → all objects hydrated but wallclock lag exceeds threshold.
• Failing → no replicas configured, or all replicas are problematic (3+ OOM kills within 24 hours per mz_cluster_replica_status_history).

Apply State Tracking

The apply (cutover) process uses a pair of schemas in _mz_deploy as a state marker: apply_<id>_pre and apply_<id>_post. During the atomic swap transaction these schemas exchange names, moving the swapped=true comment to the _pre schema. This enables crash recovery:

• Schema absent → NotStarted
• _pre comment = swapped=false → PreSwap (resume pre-swap work)
• _pre comment = swapped=true → PostSwap (resume post-swap work)

SUBSCRIBE Streaming

subscribe_deployment_hydration opens a SUBSCRIBE cursor over the hydration status query. Retractions (mz_diff == -1) are filtered out before yielding updates — only insertions are surfaced to the caller.

Structs

ClusterStatusContext

Full status context for a cluster in a staging deployment.

HydrationStatusUpdate

A hydration status update from the SUBSCRIBE stream.

Enums

ClusterDeploymentStatus

Status of a cluster in a staging deployment.

FailureReason

Reason why a cluster deployment is failing.

Constants

DEFAULT_ALLOWED_LAG_SECS

Default allowed lag threshold in seconds (5 minutes).

Functions

append_deployment_objects 🔒

Append deployment object records (insert-only, never update or delete).

check_deployment_conflicts 🔒

Check for deployment conflicts (schemas updated after deployment started).

create_apply_state_schemas 🔒

Create apply state schemas with comments for tracking apply progress.

delete_apply_state_schemas 🔒

Delete apply state schemas after successful completion.

delete_deployment 🔒

Delete all deployment records for a specific deployment.

delete_deployment_clusters 🔒

Delete cluster records for a staging deployment.

delete_pending_statements 🔒

Delete all pending statements for a deployment.

delete_replacement_mvs 🔒

Delete all replacement MV records for a deployment.

deployment_table_exists 🔒

Check if the deployment tracking table exists.

get_apply_state 🔒

Get the current apply state for a deployment.

get_deployment_clusters 🔒

Get cluster names for a staging deployment.

get_deployment_details 🔒

Get detailed information about a specific deployment.

get_deployment_hydration_status 🔒

Get detailed hydration and health status for clusters in a staging deployment.

get_deployment_metadata 🔒

Get metadata about a deployment for validation.

get_deployment_objects 🔒

Get deployment object records from the database for a specific deployment.

get_pending_statements 🔒

Get pending statements for a deployment that haven’t been executed yet.

get_replacement_mvs 🔒

Get replacement MV records for a deployment.

get_schema_deployments 🔒

Get schema deployment records from the database for a specific deployment.

hydration_status_query 🔒

Build the shared hydration-status SQL query.

insert_deployment_clusters 🔒

Insert cluster records for a staging deployment.

insert_pending_statements 🔒

Insert pending statements for deferred execution (e.g., sinks).

insert_replacement_mvs 🔒

Insert replacement MV records for a deployment.

insert_schema_deployments 🔒

Insert schema deployment records (insert-only, no DELETE).

list_deployment_history 🔒

List deployment history in chronological order (promoted deployments only).

list_production_clusters 🔒

List clusters that host at least one promoted deployment.

list_staging_deployments 🔒

List all staging deployments (promoted_at IS NULL), grouped by deploy_id.

mark_statement_executed 🔒

Mark a pending statement as executed.

update_promoted_at 🔒

Update promoted_at timestamp for a staging deployment.

validate_deployment_clusters 🔒

Validate that all cluster IDs in a deployment still exist in the catalog.