Struct repr::Row

pub struct Row {
    data: SmallVec<[u8; 24]>,
}

A packed representation for Datums.

Datum is easy to work with but very space inefficent. A Datum::Int32(42) is laid out in memory like this:

tag: 3 padding: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 data: 0 0 0 42 padding: 0 0 0 0 0 0 0 0 0 0 0 0

For a total of 32 bytes! The second set of padding is needed in case we were to write a 16-byte datum into this location. The first set of padding is needed to align that hypothetical decimal to a 16 bytes boundary.

A Row stores zero or more Datums without any padding. We avoid the need for the first set of padding by only providing access to the Datums via calls to ptr::read_unaligned, which on modern x86 is barely penalized. We avoid the need for the second set of padding by not providing mutable access to the Datum. Instead, Row is append-only.

A Row can be built from a collection of Datums using Row::pack, but it is more efficient to use Row::pack_slice so that a right-sized allocation can be created. If that is not possible, consider using the “packer” pattern: allocate one row, pack into it, and then call Row::finish_and_reuse to receive a copy of that row, leaving behind the original allocation to pack future rows.

Creating a row via Row::pack_slice:

let row = Row::pack_slice(&[Datum::Int32(0), Datum::Int32(1), Datum::Int32(2)]);
assert_eq!(row.unpack(), vec![Datum::Int32(0), Datum::Int32(1), Datum::Int32(2)])

Rows can be unpacked by iterating over them:

let row = Row::pack_slice(&[Datum::Int32(0), Datum::Int32(1), Datum::Int32(2)]);
assert_eq!(row.iter().nth(1).unwrap(), Datum::Int32(1));

If you want random access to the Datums in a Row, use Row::unpack to create a Vec<Datum>

let row = Row::pack_slice(&[Datum::Int32(0), Datum::Int32(1), Datum::Int32(2)]);
let datums = row.unpack();
assert_eq!(datums[1], Datum::Int32(1));

Performance

Rows are dynamically sized, but up to a fixed size their data is stored in-line. It is best to re-use a Row across multiple Row creation calls, as this avoids the allocations involved in Row::new().

Fields

data: SmallVec<[u8; 24]>

Implementations

impl Row

fn try_push_proto(&mut self, x: &ProtoDatum) -> Result<(), String>

impl Row

const SIZE: usize

impl Row

pub fn pack<'a, I, D>(iter: I) -> Row where
    I: IntoIterator<Item = D>,
    D: Borrow<Datum<'a>>, 

Take some Datums and pack them into a Row.

This method builds a Row by repeatedly increasing the backing allocation. If the contents of the iterator are known ahead of time, consider Row::with_capacity to right-size the allocation first, and then Row::extend to populate it with Datums. This avoids the repeated allocation resizing and copying.

pub fn try_pack<'a, I, D, E>(iter: I) -> Result<Row, E> where
    I: IntoIterator<Item = Result<D, E>>,
    D: Borrow<Datum<'a>>, 

Like Row::pack, but the provided iterator is allowed to produce an error, in which case the packing operation is aborted and the error returned.

pub fn with_capacity(cap: usize) -> Self

Allocate an empty Row with a pre-allocated capacity.

pub fn push<'a, D>(&mut self, datum: D) where
    D: Borrow<Datum<'a>>, 

Extend an existing Row with a Datum.

pub fn extend<'a, I, D>(&mut self, iter: I) where
    I: IntoIterator<Item = D>,
    D: Borrow<Datum<'a>>, 

Extend an existing Row with additional Datums.

pub fn try_extend<'a, I, E, D>(&mut self, iter: I) -> Result<(), E> where
    I: IntoIterator<Item = Result<D, E>>,
    D: Borrow<Datum<'a>>, 

Extend an existing Row with additional Datums.

In the case the iterator produces an error, the pushing of datums in terminated and the error returned. The Row will be incomplete, but it will be safe to read datums from it.

pub fn extend_by_row(&mut self, row: &Row)

Appends the datums of an entire Row.

pub fn clear(&mut self)

Clears the contents of the row without de-allocating its backing memory.

pub unsafe fn from_bytes_unchecked(data: Vec<u8>) -> Self

Creates a new row from supplied bytes.

Safety

This method relies on data being an appropriate row encoding, and can result in unsafety if this is not the case.

pub fn pack_slice<'a>(slice: &[Datum<'a>]) -> Row

Pack a slice of Datums into a Row.

This method has the advantage over pack that it can determine the required allocation before packing the elements, ensuring only one allocation and no redundant copies required.

pub fn push_list_with<F, R>(&mut self, f: F) -> R where
    F: FnOnce(&mut Row) -> R, 

Pushes a DatumList that is built from a closure.

The supplied closure will be invoked once with a Row that can be used to populate the list. It is valid to call any method on the Row except for Row::finish_and_reuse or Row::truncate.

Returns the value returned by the closure, if any.

let mut row = Row::default();
row.push_list_with(|row| {
    row.push(Datum::String("age"));
    row.push(Datum::Int64(42));
});
assert_eq!(
    row.unpack_first().unwrap_list().iter().collect::<Vec<_>>(),
    vec![Datum::String("age"), Datum::Int64(42)],
);

pub fn push_dict_with<F, R>(&mut self, f: F) -> R where
    F: FnOnce(&mut Row) -> R, 

Pushes a DatumMap that is built from a closure.

The supplied closure will be invoked once with a Row that can be used to populate the dict.

The closure must alternate pushing string keys and arbitary values, otherwise reading the dict will cause a panic.

The closure must push keys in ascending order, otherwise equality checks on the resulting Row may be wrong and reading the dict IN DEBUG MODE will cause a panic.

The closure must not call Row::finish_and_reuse.

Example

let mut row = Row::default();
row.push_dict_with(|row| {

    // key
    row.push(Datum::String("age"));
    // value
    row.push(Datum::Int64(42));

    // key
    row.push(Datum::String("name"));
    // value
    row.push(Datum::String("bob"));
});
assert_eq!(
    row.unpack_first().unwrap_map().iter().collect::<Vec<_>>(),
    vec![("age", Datum::Int64(42)), ("name", Datum::String("bob"))]
);

pub fn push_array<'a, I, D>(
    &mut self,
    dims: &[ArrayDimension],
    iter: I
) -> Result<(), InvalidArrayError> where
    I: IntoIterator<Item = D>,
    D: Borrow<Datum<'a>>, 

Convenience function to construct an array from an iter of Datums.

Returns an error if the number of elements in iter does not match the cardinality of the array as described by dims, or if the number of dimensions exceeds MAX_ARRAY_DIMENSIONS. If an error occurs, the packer’s state will be unchanged.

pub fn push_list<'a, I, D>(&mut self, iter: I) where
    I: IntoIterator<Item = D>,
    D: Borrow<Datum<'a>>, 

Convenience function to push a DatumList from an iter of Datums

See Row::push_dict_with if you need to be able to handle errors

pub fn push_dict<'a, I, D>(&mut self, iter: I) where
    I: IntoIterator<Item = (&'a str, D)>,
    D: Borrow<Datum<'a>>, 

Convenience function to push a DatumMap from an iter of (&str, Datum) pairs

pub fn finish_and_reuse(&mut self) -> Row

Returns a copy of this Row, clearing the data but not the allocation in self.

The intent is that self’s allocation can be used to pack additional rows, to reduce the amount of interaction with the allocator.

pub unsafe fn truncate(&mut self, pos: usize)

Truncates the underlying storage to the specified byte position.

Safety

pos MUST specify a byte offset that lies on a datum boundary. If pos specifies a byte offset that is within a datum, the row packer will produce an invalid row, the unpacking of which may trigger undefined behavior!

To find the byte offset of a datum boundary, inspect the the packer’s byte length by calling packer.data().len() after pushing the desired number of datums onto the packer.

pub fn truncate_datums(&mut self, n: usize)

Truncates the row to contain at most the first n datums.

Panics

Methods from Deref<Target = RowRef>

pub fn unpack(&self) -> Vec<Datum<'_>>

Unpack self into a Vec<Datum> for efficient random access.

pub fn unpack_first(&self) -> Datum<'_>

Return the first Datum in self

Panics if the Row is empty.

pub fn iter(&self) -> DatumListIter<'_>

Iterate the Datum elements of the Row.

pub fn data(&self) -> &[u8]

For debugging only

pub fn is_empty(&self) -> bool

True iff there is no data in this Row

Trait Implementations

impl Clone for Row

fn clone(&self) -> Self

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Codec for Row

fn encode<B>(&self, buf: &mut B) where
    B: BufMut, 

Encodes a row into the permanent storage format.

This perfectly round-trips through Row::decode. It’s guaranteed to be readable by future versions of Materialize through v(TODO: Figure out our policy).

fn decode(buf: &[u8]) -> Result<Row, String>

Decodes a row from the permanent storage format.

This perfectly round-trips through Row::encode. It can read rows encoded by historical versions of Materialize back to v(TODO: Figure out our policy).

fn codec_name() -> String

Name of the codec.

impl Debug for Row

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Debug representation using the internal datums

impl Default for Row

fn default() -> Row

Returns the “default value” for a type.

impl Deref for Row

type Target = RowRef

The resulting type after dereferencing.

fn deref(&self) -> &Self::Target

Dereferences the value.

impl<'de> Deserialize<'de> for Row

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error> where
    __D: Deserializer<'de>, 

Deserialize this value from the given Serde deserializer.

impl Display for Row

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Debug representation using the internal datums

impl From<&'_ Row> for ProtoRow

fn from(x: &Row) -> Self

Performs the conversion.

impl Hash for Row

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H) where
    H: Hasher, 

Feeds a slice of this type into the given Hasher.

impl Ord for Row

fn cmp(&self, other: &Self) -> Ordering

This method returns an Ordering between self and other.

fn max(self, other: Self) -> Self

Compares and returns the maximum of two values.

fn min(self, other: Self) -> Self

Compares and returns the minimum of two values.

fn clamp(self, min: Self, max: Self) -> Self

Restrict a value to a certain interval.

impl PartialEq<Row> for Row

fn eq(&self, other: &Row) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Row) -> bool

This method tests for !=.

impl PartialOrd<Row> for Row

These implementations order first by length, and then by slice contents. This allows many comparisons to complete without dereferencing memory.

fn partial_cmp(&self, other: &Self) -> Option<Ordering>

This method returns an ordering between self and other values if one exists.

fn lt(&self, other: &Rhs) -> bool

This method tests less than (for self and other) and is used by the < operator.

fn le(&self, other: &Rhs) -> bool

This method tests less than or equal to (for self and other) and is used by the <= operator.

fn gt(&self, other: &Rhs) -> bool

This method tests greater than (for self and other) and is used by the > operator.

fn ge(&self, other: &Rhs) -> bool

This method tests greater than or equal to (for self and other) and is used by the >= operator.

impl Serialize for Row

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error> where
    __S: Serializer, 

Serialize this value into the given Serde serializer.

impl TryFrom<&'_ ProtoRow> for Row

type Error = String

The type returned in the event of a conversion error.

fn try_from(x: &ProtoRow) -> Result<Self, Self::Error>

Performs the conversion.

impl Eq for Row

impl StructuralEq for Row

impl StructuralPartialEq for Row