Struct DatumContainer 

pub struct DatumContainer {
    codec: Option<ColumnsCodec>,
    inner: BytesContainer,
    staging: Vec<u8>,
    stats: Option<ColumnsCodec>,
}

Fields

codec: Option<ColumnsCodec>

Encoder/decoder used to translate between row bytes and the stored bytes. None until enough pushes have been observed (or if compression is disabled).

inner: BytesContainer

The stored, possibly-encoded, row bytes.

staging: Vec<u8>

Staging buffer for ingested Row types.

stats: Option<ColumnsCodec>

Statistics gatherer, used to build a safe codec after enough pushes. None once the codec has been installed or if compression is disabled.

Implementations

impl DatumContainer

pub fn heap_size(&self, callback: impl FnMut(usize, usize))

Visit contained allocations to determine their size and capacity.

pub(crate) fn promote_stats_to_codec(&mut self)

Promote a gathered-but-uninstalled statistics summary into the codec slot.

A container filled via the builder’s push/done path — as the reduce operator does, building batches with Builder::new() + push + done rather than seal — gathers statistics on every push but never reaches seal’s codec install, and only crosses the mid-formation STATS_THRESHOLD install if it grows past it. A smaller such container would otherwise be finalized with no codec at all, even with the flag on.

That is a problem not because this batch needs compressing — its rows are already stored raw and we deliberately do not re-encode them here — but because a codec-less batch poisons future merges: Self::merge_capacity keys off the presence of a codec, so a codec-less input forces the merged container onto the uncompressed path. Moving the gathered statistics into the codec slot leaves the batch carrying a codec whose retained heavy-hitter summary a later merge can rebuild from via ColumnsCodec::new_from, while installing no dictionary: the empty decode map resolves every stored (raw) column through the literal-datum fall-through, so reads stay correct.

We move the summary as-is rather than building a dictionary via new_safe / new_from (which reset the summary): unlike seal and the mid-formation install, done has no further rows to re-observe, so a reset summary would leave the eventual merge nothing to rebuild from.

Trait Implementations

impl BatchContainer for DatumContainer

type Owned = Row

An owned instance of Self::ReadItem<'_>.

type ReadItem<'a> = DatumSeq<'a>

The type that can be read back out of the container.

fn with_capacity(size: usize) -> Self

Creates a new container with sufficient capacity.

fn merge_capacity(cont1: &Self, cont2: &Self) -> Self

Creates a new container with sufficient capacity.

fn index(&self, index: usize) -> Self::ReadItem<'_>

Reference to the element at this position.

fn len(&self) -> usize

Number of contained elements

fn reborrow<'b, 'a: 'b>(item: Self::ReadItem<'a>) -> Self::ReadItem<'b>

Converts a read item into one with a narrower lifetime.

fn into_owned<'a>(item: Self::ReadItem<'a>) -> Self::Owned

Conversion from an instance of this type to the owned type.

fn clone_onto<'a>(item: Self::ReadItem<'a>, other: &mut Self::Owned)

Clones self onto an existing instance of the owned type.

fn push_ref(&mut self, item: Self::ReadItem<'_>)

Push an item into this container

fn push_own(&mut self, item: &Self::Owned)

Push an item into this container

fn clear(&mut self)

Clears the container. May not release resources.

fn get(&self, index: usize) -> Option<Self::ReadItem<'_>>

Reference to the element at this position, if it exists.

fn last(&self) -> Option<Self::ReadItem<'_>>

Returns the last item if the container is non-empty.

fn is_empty(&self) -> bool

Indicates if the length is zero.

fn advance<F>(&self, start: usize, end: usize, function: F) -> usize

where F: for<'a> Fn(Self::ReadItem<'a>) -> bool,

Reports the number of elements satisfying the predicate.

impl PushInto<&Row> for DatumContainer

fn push_into(&mut self, item: &Row)

Push item into self.

impl PushInto<&RowRef> for DatumContainer

fn push_into(&mut self, item: &RowRef)

Push item into self.

impl PushInto<DatumSeq<'_>> for DatumContainer

fn push_into(&mut self, item: DatumSeq<'_>)

Push item into self.

impl PushInto<Row> for DatumContainer

fn push_into(&mut self, item: Row)

Push item into self.