Skip to main content

Module hydration_history

Module hydration_history 

Source
Expand description

Durable history collection for completed object and replica hydration episodes.

One sweep visits a single user replica, installs a replica-targeted subscribe that diffs that replica’s live hydration timestamps against the durable history tables, and appends what is missing through the timestamped OCC write path. Including each history table in its read expression is what makes the write idempotent across concurrent environmentd processes: two collectors that compute the same row race for one write timestamp, and the loser observes the winner’s append through its own subscribe and finds nothing left to write.

One replica is sampled per interval, so an environment with N eligible replicas revisits each one approximately every N * interval. Lowering the interval improves freshness at the cost of more replica dataflow installs.

Collection is sampling, not an event log. Replica history records only the latest completed episode visible in a sweep. Intermediate episodes and intervals retracted before collection leave no evidence and are not recorded. See the design doc for the resulting semantics.

StructsΒ§

ReplicaTarget πŸ”’
A user replica eligible for one collection step.
Sweep πŸ”’
What one sweep needs to run its mutations against the history table.

ConstantsΒ§

DISABLED_RECHECK_INTERVAL πŸ”’
How often a disabled collector rechecks whether it was enabled.
MUTATION_TIMEOUT πŸ”’
Bound on one replica-targeted mutation.
RETENTION_BATCH_SIZE πŸ”’
Rows retracted per retention step.
SCHEDULE_RECHECK_CAP πŸ”’
Longest a scheduler sleep may run before it rechecks the configuration.

FunctionsΒ§

environment_schedule_offset πŸ”’
Stable offset within interval_ms for one environment id.
next_fire_delay πŸ”’
Milliseconds until the next fire on this environment’s own grid.
next_replica πŸ”’
Picks the replica after cursor, wrapping around at the end.
object_collection_sql πŸ”’
The rows this replica has completed that the history table is missing.
object_retention_sql πŸ”’
A bounded batch of history rows that have aged out.
plan_mutation πŸ”’
Plans sql as the read side of a mutation against target_id.
replica_collection_sql πŸ”’
Returns SQL for the latest completed compute hydration episode and its process resource peaks.
replica_retention_sql πŸ”’
A bounded batch of replica history rows that have aged out.