Struct dataflow::render::debezium::TrackRange [−][src]
Expand description
When to start and end full-range tracking
All values are milliseconds since the unix epoch and are meant to be compared to the
upstream_time_millis
argument to DebeziumDeduplicationState::should_use_record
.
We throw away all tracking data after we see the first record past end
.
Fields
pad_start: i64
Start pre-filling the seen data before we start trusting it
At some point we need to start trusting the TrackFull::seen_positions
map more
than we trust the Debezium high water mark. In order to do that, the
seen_offsets
map must have some data, otherwise all records would show up as
new immediately at the phase transition.
For example, consider the following series of records, presented vertically in the order that they were received:
ts val
-------
1 a
2 b
1 a
If we start tracking at ts 2 and immediately start trusting the hashmap more than the Debezium high water mark then ts 1 will be falsely double-inserted. So we need to start building a buffer before we can start trusting it.
pad_start
is the upstream_time_millis at we we start building the buffer, and
TrackRange::start
is the point at which we start trusting the buffer.
Currently pad_start
defaults to 1 hour (wall clock time) before start
,
as a value that seems overwhelmingly likely to cause the buffer to always have
enough data that it doesn’t give incorrect answers.
start: i64
end: i64
Trait Implementations
Auto Trait Implementations
impl RefUnwindSafe for TrackRange
impl Send for TrackRange
impl Sync for TrackRange
impl Unpin for TrackRange
impl UnwindSafe for TrackRange
Blanket Implementations
Mutably borrows from an owned value. Read more
Attaches the provided Subscriber
to this type, returning a
WithDispatch
wrapper. Read more
Attaches the current default Subscriber
to this type, returning a
WithDispatch
wrapper. Read more