Enum interchange::avro::envelope_debezium::deduplication::DebeziumDeduplicationStrategy

pub enum DebeziumDeduplicationStrategy {
    None,
    Ordered,
    Full,
    FullInRange {
        pad_start: Option<NaiveDateTime>,
        start: NaiveDateTime,
        end: NaiveDateTime,
    },
}

Ordered means we can trust Debezium high water marks

In standard operation, Debezium should always emit messages in position order, but messages may be duplicated.

For example, this is a legal stream of Debezium event positions:

1 2 3 2

Note that 2 appears twice, but the first time it appeared it appeared in order. Any position below the highest-ever seen position is guaranteed to be a duplicate, and can be ignored.

Now consider this stream:

1 3 2

In this case, 2 is sent out of order, and if it is ignored we will miss important state.

It is possible for users to do things with multiple databases and multiple Debezium instances pointing at the same Kafka topic that mean that the Debezium guarantees do not hold, in which case we are required to track individual messages, instead of just the highest-ever-seen message.

Variants

None

Do not perform any deduplication

This should mostly be used with envelope upsert

Ordered

We can trust high water mark

Full

We need to store some piece of state for every message

FullInRange

Fields

pad_start: Option<NaiveDateTime>

start: NaiveDateTime

end: NaiveDateTime

Implementations

impl DebeziumDeduplicationStrategy

pub fn full_in_range(
    start: &str,
    end: &str,
    pad_start: Option<&str>
) -> Result<DebeziumDeduplicationStrategy>

Create a deduplication strategy with start and end times

Returns an error if either datetime does not parse, or if there is no time in between them

pub fn render<G: Scope<Timestamp = Timestamp>>(
    self,
    collection: Collection<G, Row, Diff>,
    debug_name: String,
    worker_index: usize,
    arity: usize,
    key_indices: Option<Vec<usize>>
) -> Collection<G, Row, Diff>

Trait Implementations

impl Clone for DebeziumDeduplicationStrategy

fn clone(&self) -> DebeziumDeduplicationStrategy

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DebeziumDeduplicationStrategy

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for DebeziumDeduplicationStrategy

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error> where
    __D: Deserializer<'de>, 

Deserialize this value from the given Serde deserializer.

impl PartialEq<DebeziumDeduplicationStrategy> for DebeziumDeduplicationStrategy

fn eq(&self, other: &DebeziumDeduplicationStrategy) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &DebeziumDeduplicationStrategy) -> bool

This method tests for !=.

impl Serialize for DebeziumDeduplicationStrategy

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error> where
    __S: Serializer, 

Serialize this value into the given Serde serializer.

impl Copy for DebeziumDeduplicationStrategy

impl Eq for DebeziumDeduplicationStrategy

impl StructuralEq for DebeziumDeduplicationStrategy

impl StructuralPartialEq for DebeziumDeduplicationStrategy