Struct mz_storage::render::debezium::TrackRange

struct TrackRange {
    pad_start: i64,
    start: i64,
    end: i64,
}

When to start and end full-range tracking

All values are milliseconds since the unix epoch and are meant to be compared to the upstream_time_millis argument to DebeziumDeduplicationState::should_use_record.

We throw away all tracking data after we see the first record past end.

Fields

pad_start: i64

Start pre-filling the seen data before we start trusting it

At some point we need to start trusting the TrackFull::seen_positions map more than we trust the Debezium high water mark. In order to do that, the seen_offsets map must have some data, otherwise all records would show up as new immediately at the phase transition.

For example, consider the following series of records, presented vertically in the order that they were received:

ts  val
-------
1   a
2   b
1   a

If we start tracking at ts 2 and immediately start trusting the hashmap more than the Debezium high water mark then ts 1 will be falsely double-inserted. So we need to start building a buffer before we can start trusting it.

pad_start is the upstream_time_millis at we we start building the buffer, and TrackRange::start is the point at which we start trusting the buffer. Currently pad_start defaults to 1 hour (wall clock time) before start, as a value that seems overwhelmingly likely to cause the buffer to always have enough data that it doesn’t give incorrect answers.

start: i64

end: i64

Trait Implementations

impl Debug for TrackRange

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.