Struct mz_repr::Row

pub struct Row {
    data: CompactBytes,
}

A packed representation for Datums.

Datum is easy to work with but very space inefficient. A Datum::Int32(42) is laid out in memory like this:

tag: 3 padding: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 data: 0 0 0 42 padding: 0 0 0 0 0 0 0 0 0 0 0 0

For a total of 32 bytes! The second set of padding is needed in case we were to write a 16-byte datum into this location. The first set of padding is needed to align that hypothetical decimal to a 16 bytes boundary.

A Row stores zero or more Datums without any padding. We avoid the need for the first set of padding by only providing access to the Datums via calls to ptr::read_unaligned, which on modern x86 is barely penalized. We avoid the need for the second set of padding by not providing mutable access to the Datum. Instead, Row is append-only.

A Row can be built from a collection of Datums using Row::pack, but it is more efficient to use Row::pack_slice so that a right-sized allocation can be created. If that is not possible, consider using the row buffer pattern: allocate one row, pack into it, and then call Row::clone to receive a copy of that row, leaving behind the original allocation to pack future rows.

Creating a row via Row::pack_slice:

let row = Row::pack_slice(&[Datum::Int32(0), Datum::Int32(1), Datum::Int32(2)]);
assert_eq!(row.unpack(), vec![Datum::Int32(0), Datum::Int32(1), Datum::Int32(2)])

Rows can be unpacked by iterating over them:

let row = Row::pack_slice(&[Datum::Int32(0), Datum::Int32(1), Datum::Int32(2)]);
assert_eq!(row.iter().nth(1).unwrap(), Datum::Int32(1));

If you want random access to the Datums in a Row, use Row::unpack to create a Vec<Datum>

let row = Row::pack_slice(&[Datum::Int32(0), Datum::Int32(1), Datum::Int32(2)]);
let datums = row.unpack();
assert_eq!(datums[1], Datum::Int32(1));

Performance

Rows are dynamically sized, but up to a fixed size their data is stored in-line. It is best to re-use a Row across multiple Row creation calls, as this avoids the allocations involved in Row::new().

Fields

data: CompactBytes

Implementations

impl Row

const SIZE: usize = 23usize

pub fn decode_from_proto( &mut self, proto: &ProtoRow, desc: &RelationDesc, ) -> Result<(), String>

A variant of Row::from_proto that allows for reuse of internal allocs and validates the decoding against a provided RelationDesc.

pub fn with_capacity(cap: usize) -> Self

Allocate an empty Row with a pre-allocated capacity.

pub unsafe fn from_bytes_unchecked(data: &[u8]) -> Self

Creates a new row from supplied bytes.

Safety

This method relies on data being an appropriate row encoding, and can result in unsafety if this is not the case.

pub fn packer(&mut self) -> RowPacker<'_>

Constructs a RowPacker that will pack datums into this row’s allocation.

This method clears the existing contents of the row, but retains the allocation.

pub fn pack<'a, I, D>(iter: I) -> Row

where I: IntoIterator<Item = D>, D: Borrow<Datum<'a>>,

Take some Datums and pack them into a Row.

This method builds a Row by repeatedly increasing the backing allocation. If the contents of the iterator are known ahead of time, consider Row::with_capacity to right-size the allocation first, and then RowPacker::extend to populate it with Datums. This avoids the repeated allocation resizing and copying.

pub fn pack_using<'a, I, D>(&mut self, iter: I) -> Row

where I: IntoIterator<Item = D>, D: Borrow<Datum<'a>>,

Use self to pack iter, and then clone the result.

This is a convenience method meant to reduce boilerplate around row formation.

pub fn try_pack<'a, I, D, E>(iter: I) -> Result<Row, E>

where I: IntoIterator<Item = Result<D, E>>, D: Borrow<Datum<'a>>,

Like Row::pack, but the provided iterator is allowed to produce an error, in which case the packing operation is aborted and the error returned.

pub fn pack_slice<'a>(slice: &[Datum<'a>]) -> Row

Pack a slice of Datums into a Row.

This method has the advantage over pack that it can determine the required allocation before packing the elements, ensuring only one allocation and no redundant copies required.

pub fn byte_len(&self) -> usize

Returns the total amount of bytes used by this row.

pub fn data_len(&self) -> usize

The length of the encoded row in bytes. Does not include the size of the Row struct itself.

pub fn byte_capacity(&self) -> usize

Returns the total capacity in bytes used by this row.

pub fn as_row_ref(&self) -> &RowRef

Extracts a Row slice containing the entire Row.

Methods from Deref<Target = RowRef>

pub fn unpack(&self) -> Vec<Datum<'_>>

Unpack self into a Vec<Datum> for efficient random access.

pub fn unpack_first(&self) -> Datum<'_>

Return the first Datum in self

Panics if the RowRef is empty.

pub fn iter(&self) -> DatumListIter<'_>

Iterate the Datum elements of the RowRef.

pub fn byte_len(&self) -> usize

Return the byte length of this RowRef.

pub fn data(&self) -> &[u8]

For debugging only.

pub fn is_empty(&self) -> bool

True iff there is no data in this RowRef.

Trait Implementations

impl Arbitrary for Row

type Parameters = SizeRange

The type of parameters that arbitrary_with accepts for configuration of the generated Strategy. Parameters must implement Default.

type Strategy = BoxedStrategy<Row>

The type of Strategy used to generate values of type Self.

fn arbitrary_with(size: Self::Parameters) -> Self::Strategy

Generates a Strategy for producing arbitrary values of type the implementing type (Self). The strategy is passed the arguments given in args.

fn arbitrary() -> Self::Strategy

Generates a Strategy for producing arbitrary values of type the implementing type (Self).

impl AsRef<RowRef> for Row

fn as_ref(&self) -> &RowRef

Converts this type into a shared reference of the (usually inferred) input type.

impl Borrow<RowRef> for Row

fn borrow(&self) -> &RowRef

Immutably borrows from an owned value.

impl Clone for Row

fn clone(&self) -> Self

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Codec for Row

fn encode<B>(&self, buf: &mut B)

where B: BufMut,

Encodes a row into the permanent storage format.

This perfectly round-trips through Row::decode. It’s guaranteed to be readable by future versions of Materialize through v(TODO: Figure out our policy).

fn decode(buf: &[u8], schema: &RelationDesc) -> Result<Row, String>

Decodes a row from the permanent storage format.

This perfectly round-trips through Row::encode. It can read rows encoded by historical versions of Materialize back to v(TODO: Figure out our policy).

type Storage = ProtoRow

A type used with Self::decode_from for allocation reuse. Set to () if unnecessary.

type Schema = RelationDesc

The type of the associated schema for Self.

fn codec_name() -> String

Name of the codec.

fn decode_from<'a>( &mut self, buf: &'a [u8], storage: &mut Option<ProtoRow>, schema: &RelationDesc, ) -> Result<(), String>

An alternate form of Self::decode which enables amortizing allocs.

fn validate(row: &Self, desc: &Self::Schema) -> Result<(), String>

Checks that the given value matches the provided schema.

fn encode_schema(schema: &Self::Schema) -> Bytes

Encode a schema for permanent storage.

fn decode_schema(buf: &Bytes) -> Self::Schema

Decode a schema previous encoded with this codec’s Self::encode_schema.

fn encode_to_vec(&self) -> Vec<u8>

Encode a key or value to a Vec.

impl ColumnDecoder<Row> for RowColumnarDecoder

fn decode(&self, idx: usize, val: &mut Row)

Decode the value at idx into the buffer val.

fn is_null(&self, idx: usize) -> bool

Returns if the value at idx is null.

fn stats(&self) -> StructStats

Returns statistics for the column. This structure is defined by Persist, but the contents are determined by the client; Persist will preserve them in the part metadata and make them available to readers.

impl ColumnEncoder<Row> for RowColumnarEncoder

type FinishedColumn = StructArray

Type of column that this encoder returns when finalized.

fn goodbytes(&self) -> usize

The amount of “actual data” encoded by this encoder so far.

fn append(&mut self, val: &Row)

Appends val onto this encoder.

fn append_null(&mut self)

Appends a null value onto this encoder.

fn finish(self) -> Self::FinishedColumn

Finish this encoder, returning an immutable column.

impl Columnation for Row

type InnerRegion = RowStack

The type of region capable of absorbing allocations owned by the Self type. Note: not allocations of Self, but of the things that it owns.

impl Debug for Row

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Debug representation using the internal datums

impl Default for Row

fn default() -> Row

Returns the “default value” for a type.

impl Deref for Row

type Target = RowRef

The resulting type after dereferencing.

fn deref(&self) -> &Self::Target

Dereferences the value.

impl<'de> Deserialize<'de> for Row

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Display for Row

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Display representation using the internal datums

impl FromDatumIter for Row

fn from_datum_iter<'a, I, D>(&mut self, datum_iter: I) -> Self

where I: IntoIterator<Item = D>, D: Borrow<Datum<'a>>,

Packs into self the given iterator of datums and returns a clone.

fn try_from_datum_iter<'a, I, D, E>(&mut self, datum_iter: I) -> Result<Self, E>

where I: IntoIterator<Item = Result<D, E>>, D: Borrow<Datum<'a>>,

Packs into self by using the packer’s try_extend method on the given iterator and returns a clone.

impl Hash for Row

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl<'a> IntoIterator for &'a Row

type Item = Datum<'a>

The type of the elements being iterated over.

type IntoIter = DatumListIter<'a>

Which kind of iterator are we turning this into?

fn into_iter(self) -> DatumListIter<'a>

Creates an iterator from a value.

impl IntoRowIterator for Row

type Iter = SingleRowIter

fn into_row_iter(self) -> Self::Iter

impl Ord for Row

fn cmp(&self, other: &Self) -> Ordering

This method returns an Ordering between self and other.

fn max(self, other: Self) -> Self

where Self: Sized,

Compares and returns the maximum of two values.

fn min(self, other: Self) -> Self

where Self: Sized,

Compares and returns the minimum of two values.

fn clamp(self, min: Self, max: Self) -> Self

where Self: Sized + PartialOrd,

Restrict a value to a certain interval.

impl PartialEq for Row

fn eq(&self, other: &Row) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl PartialOrd for Row

fn partial_cmp(&self, other: &Self) -> Option<Ordering>

This method returns an ordering between self and other values if one exists.

fn lt(&self, other: &Rhs) -> bool

Tests less than (for self and other) and is used by the < operator.

fn le(&self, other: &Rhs) -> bool

Tests less than or equal to (for self and other) and is used by the <= operator.

fn gt(&self, other: &Rhs) -> bool

Tests greater than (for self and other) and is used by the > operator.

fn ge(&self, other: &Rhs) -> bool

Tests greater than or equal to (for self and other) and is used by the >= operator.

impl RustType<ProtoRow> for Row

fn into_proto(&self) -> ProtoRow

Convert a Self into a Proto value.

fn from_proto(proto: ProtoRow) -> Result<Self, TryFromProtoError>

Consume and convert a Proto back into a Self value.

fn into_proto_owned(self) -> Proto

A zero clone version of Self::into_proto that types can optionally implement, otherwise, the default implementation delegates to Self::into_proto.

impl Schema2<Row> for RelationDesc

type ArrowColumn = StructArray

The type of column we decode from, and encoder will finish into.

type Statistics = OptionStats<StructStats>

Statistics we collect for a schema of this type.

type Decoder = RowColumnarDecoder

Type that is able to decode values of T from Self::ArrowColumn.

type Encoder = RowColumnarEncoder

Type that is able to encoder values of T.

fn decoder(&self, col: Self::ArrowColumn) -> Result<Self::Decoder, Error>

Returns a type that is able to decode instances of T from the provider column.

fn encoder(&self) -> Result<Self::Encoder, Error>

Returns a type that can encode values of T.

fn decoder_any(&self, col: &dyn Array) -> Result<Self::Decoder, Error>

Returns a type that is able to decode instances of T from a type erased arrow::array::Array, erroring if the provided array is not Self::ArrowColumn.

impl Serialize for Row

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl ToDatumIter for Row

type DatumIter<'a> = DatumListIter<'a>

Datum iterator for Row.

fn to_datum_iter<'a>(&'a self) -> Self::DatumIter<'a>

Borrows self and gets an iterator from it.

impl TryFrom<&ProtoRow> for Row

TODO: remove this in favor of RustType::from_proto.

type Error = String

The type returned in the event of a conversion error.

fn try_from(x: &ProtoRow) -> Result<Self, Self::Error>

Performs the conversion.

impl Eq for Row

impl StructuralPartialEq for Row