Expand description
Durable history collection for completed object and replica hydration episodes.
One sweep visits a single user replica, installs a replica-targeted
subscribe that diffs that replicaβs live hydration timestamps against the
durable history tables, and appends what is missing through the timestamped
OCC write path. Including each history table in its read expression is what
makes the write idempotent across concurrent environmentd processes: two
collectors that compute the same row race for one write timestamp, and the
loser observes the winnerβs append through its own subscribe and finds
nothing left to write.
One replica is sampled per interval, so an environment with N eligible
replicas revisits each one approximately every N * interval. Lowering the
interval improves freshness at the cost of more replica dataflow installs.
Collection is sampling, not an event log. Replica history records only the latest completed episode visible in a sweep. Intermediate episodes and intervals retracted before collection leave no evidence and are not recorded. See the design doc for the resulting semantics.
StructsΒ§
- Replica
Target π - A user replica eligible for one collection step.
- Sweep π
- What one sweep needs to run its mutations against the history table.
ConstantsΒ§
- DISABLED_
RECHECK_ πINTERVAL - How often a disabled collector rechecks whether it was enabled.
- MUTATION_
TIMEOUT π - Bound on one replica-targeted mutation.
- RETENTION_
BATCH_ πSIZE - Rows retracted per retention step.
- SCHEDULE_
RECHECK_ πCAP - Longest a scheduler sleep may run before it rechecks the configuration.
FunctionsΒ§
- environment_
schedule_ πoffset - Stable offset within
interval_msfor one environment id. - next_
fire_ πdelay - Milliseconds until the next fire on this environmentβs own grid.
- next_
replica π - Picks the replica after
cursor, wrapping around at the end. - object_
collection_ πsql - The rows this replica has completed that the history table is missing.
- object_
retention_ πsql - A bounded batch of history rows that have aged out.
- plan_
mutation π - Plans
sqlas the read side of a mutation againsttarget_id. - replica_
collection_ πsql - Returns SQL for the latest completed compute hydration episode and its process resource peaks.
- replica_
retention_ πsql - A bounded batch of replica history rows that have aged out.