Struct UpdateProducer

struct UpdateProducer {
    pi: PartitionKeyIterator,
    batch_size: u64,
    next_offset: u64,
    seed: u64,
    expected_quick_offsets: u64,
    include_offset: bool,
}

A struct that produces batches of data for the post-snapshotting phase.

Fields

pi: PartitionKeyIterator

The key iterator for the partition.

batch_size: u64

The batch size.

next_offset: u64

The next offset to produce updates at.

seed: u64

The source-level seed for the rng.

expected_quick_offsets: u64

The number of offsets we expect to be part of the transactional_snapshot.

include_offset: bool

Whether to include the offset or not.

Implementations

impl UpdateProducer

fn new( partition: u64, next_offset: u64, key_value: KeyValueLoadGenerator, ) -> Self

fn finished_quick(&self) -> bool

If this partition is done producing quick updates.

fn produce_batch<'a>( &'a mut self, value_buffer: &'a mut Vec<u8>, output_indexes: &'a [usize], ) -> (u64, impl Iterator<Item = ((usize, Result<SourceMessage, DataflowError>), MzOffset, Diff)> + 'a)

Produce a batch of message into buffer, of size batch_size. Advances the current batch and round counter. Also returns the frontier after the batch.

The output iterator must be consumed fully for this method to be used correctly.