Struct PeekScan 

pub(super) struct PeekScan<Tr>
where
    Tr: TraceReader<Batch: Navigable>,
{

    peek_timestamp: Timestamp,
    target_id: GlobalId,
    error_phase: ErrorPhase,
    oks: PeekResultIterator<Tr>,
    ended: Option<ScanOutcome>,
    results: Vec<(Row, NonZeroI64)>,
    total_size: usize,
    answer_rows: u64,
    max_result_size: usize,
    stash: StashBounds,
    stash_bound: bool,
    max_results: Option<usize>,
    comparator: Option<RowComparator>,
    pub(super) error_scan_time: Duration,
    pub(super) cursor_setup_time: Duration,
    pub(super) row_iteration_time: Duration,
    pub(super) thinning_time: Duration,
    pub(super) rows_thinned: usize,
}

An index peek’s walk over its error trace and its ok trace.

The walk suspends between any two cursor positions. Both phases spend one budget: the ok walk gets what the error walk leaves.

A stash-eligible scan retains at most the threshold before its first batch and the batch size after, plus the row that crossed either. A scan that cannot use the stash fills no batch, and max_result_size alone bounds its prefix.

Fields

peek_timestamp: Timestamp

The time at which the error trace is read.

target_id: GlobalId

The collection the peek reads, for logging.

error_phase: ErrorPhase

oks: PeekResultIterator<Tr>

The walk over the ok trace, reached only once the error walk reports the error trace clean. Its cursor is opened with the scan, and nothing advances it before then.

ended: Option<ScanOutcome>

The outcome the scan ended with, or None while it can still be stepped. The Ok carries no rows: those left with the ScanOutcome::Finished that reported the end.

results: Vec<(Row, NonZeroI64)>

Rows accumulated since the last batch was taken.

total_size: usize

The byte size of results, as an answer built from them would store them.

answer_rows: u64

The rows the answer holds so far, batches already handed to a driver included, counted in copies because that is what the finishing’s limit counts. Thinning takes its drops back off.

max_result_size: usize

The ceiling on what the scan may hold, above which the peek fails.

stash: StashBounds

stash_bound: bool

Whether a batch has been handed over, which is where the answer stopped being an inline one and the batch size took over from the threshold.

max_results: Option<usize>

A bound on the rows the peek’s finishing needs, limit + offset.

Further limiting happens when the results are collected, so the scan does not have to hold exactly this many rows, just at least those that would have been returned.

comparator: Option<RowComparator>

Orders the rows that thinning keeps. None when the finishing imposes no ordering, in which case the walk ends at the limit rather than thinning at all.

error_scan_time: Duration

Worker time the error walk spent, summed over the slices it was cut into.

cursor_setup_time: Duration

Worker time spent opening the ok cursor.

row_iteration_time: Duration

Worker time the ok walk spent, summed over the slices it was cut into. Includes the time thinning spent.

thinning_time: Duration

Worker time thinning spent, summed over the times it ran.

rows_thinned: usize

Rows handed to thinning, summed over the times it ran.

Implementations

impl<Tr> PeekScan<Tr>

where Tr: TraceReader<Batch: Navigable>, for<'a> BatchCursor<Tr>: Cursor<Key<'a>: ExtendDatums + Eq, KeyContainer: BatchContainer<Owned = Row>, Val<'a>: ExtendDatums, TimeGat<'a>: PartialOrder<Timestamp>, DiffGat<'a> = &'a Diff>,

pub(super) fn new( peek: &Peek, errs_handle: &mut PaddedTrace<TraceAgent<Spine<ArcBatch<OrdKeyBatch<MzStack<((DataflowErrorSer, ()), Timestamp, Diff)>>>>>>, oks_handle: &mut Tr, max_result_size: u64, stash: StashBounds, ) -> Self

Opens a scan of peek over the traces that answer it.

Both cursors are opened here, so that the scan holds everything it reads and needs neither trace handle again. The walks start without a row-iteration limit. The limit in effect is the caller’s to supply to each PeekScan::step.

pub(super) fn step( &mut self, row_iteration_limit: Option<usize>, fuel: &mut usize, ) -> ScanOutcome

Advances the scan until it has an answer for the peek, the accumulated rows make a full batch, or fuel runs out, whichever comes first. Decrements fuel by the number of cursor positions visited, in either phase.

row_iteration_limit is the limit in effect now rather than at the scan’s start, and the count it bounds spans both phases.

ScanOutcome::Suspended is not an end of scan: stepping again resumes where this call stopped. ScanOutcome::Finished is, and stepping past one is a defect in the driver.

pub(super) fn take_batch(&mut self) -> Option<Vec<(Row, NonZeroI64)>>

Takes the accumulated rows once they have crossed the stash threshold.

Returns None while they have not, and for a peek that cannot use the stash at all, so a driver with nowhere to write rows is never handed any.

pub(super) fn rows_processed(&self) -> usize

The number of cursor positions the ok walk has evaluated.

pub(super) fn phases(&self) -> WalkPhases

What the walk has spent so far, in the phases the peek metrics report.

pub(super) fn error_trace_clean(&self) -> bool

Whether the walk over the error trace has ended without finding an error, which is the only way the ok walk runs at all.

False while that walk is under way, and false once it has answered the peek.

pub(super) fn stash_eligible(&self) -> bool

Whether this scan may divert rows to the peek stash, and so whether it ever fills a batch.

A driver reads this rather than deciding eligibility again, so it cannot end up holding a batch it has nowhere to write. A scan holding an untaken batch makes no progress.

pub(super) fn batch_ready(&self) -> bool

Whether the accumulated rows have grown past what this peek may answer with inline, which is when PeekScan::take_batch hands them over.

A scan whose batch is ready stays where it stands until the batch is taken, so this is also whether stepping the scan again can make progress.

fn take_results(&mut self) -> Vec<(Row, NonZeroI64)>

Takes the accumulated rows and the size accounted to them.

Every path that hands rows out goes through here, so total_size stays an account of results.

fn fail(&mut self, error: PeekError) -> ScanOutcome

Fails the peek with error, dropping the rows the scan had accumulated, so that PeekScan::take_batch never hands a driver the prefix of an answer that will not be given.

fn step_error_phase( &mut self, row_iteration_limit: Option<usize>, fuel: &mut usize, ) -> Option<ScanOutcome>

Advances the walk over the error trace.

Returns None once the error trace is known to hold no error at the peek’s timestamp, which is the only way to the ok trace.

fn finishing_satisfied(&self) -> bool

Whether the answer holds every row the peek’s finishing can use.

Only ever true without an ordering: an ordered finishing ranks rows against the whole trace, so no prefix of the walk satisfies it.

fn step_ok_phase( &mut self, row_iteration_limit: Option<usize>, fuel: &mut usize, ) -> ScanOutcome

Advances the walk over the ok trace, accumulating the rows it produces.

fn thin(&mut self) -> Option<ScanOutcome>

Thins the accumulated rows down to the ones an ordered finishing ranks first, once the scan holds many more than it needs.

Does nothing without an ordering: such a scan ends at PeekScan::finishing_satisfied instead of accumulating past its limit.