mz_catalog::durable::objects

Module state_update

Source
Expand description

This module contains various representations of a single catalog update and the logic necessary for converting between representations.

The general lifecycle of a single update when read from persist is as follows:

  1. The update is stored in persist as a PersistStateUpdate.
  2. After being read from persist the update is immediately converted into a StateUpdate<StateUpdateKindJson>, which models the update as a JSON.
  3. The StateUpdateKindJson is converted into a protobuf message, proto::StateUpdateKind.
  4. The update is then converted into a StateUpdate<StateUpdateKind>, which is a strongly typed Rust object.
  5. Finally, the update is converted into an Option<memory::objects::StateUpdate>, and Some variants are given to the in-memory catalog. The in-memory catalog is only interested in a subset of catalog updates which is why the Option is necessary.

TLDR: PersistStateUpdate -> StateUpdate<StateUpdateKindJson> -> proto::StateUpdateKind -> StateUpdate<StateUpdateKind> -> Option<memory::objects::StateUpdate>

The process of writing a catalog update to persist is the exact opposite.

When running catalog protobuf upgrades/migrations we may need to take a detour and convert the StateUpdateKindJson to some proto::object_v{x}::StateUpdateKind before applying specific upgrades to get us to a valid proto::StateUpdateKind.

Structs§

Enums§

Constants§

Traits§

Type Aliases§