Enum mz_persist::gen::persist::ProtoBatchFormat

source ·
#[repr(i32)]
pub enum ProtoBatchFormat { Unknown = 0, ArrowKvtd = 1, ParquetKvtd = 2, ParquetStructured = 3, }

Variants§

§

Unknown = 0

§

ArrowKvtd = 1

Arrow, which we’d use for the local blob cache if we use it, gets a structure like \[(K, V, T, D)\] so that we could mmap it and use it directly as our in-mem batches (which have this structure).

§

ParquetKvtd = 2

We have more flexibility with Parquet. Initially we’ll start with the same \[(K, V, T, D)\] as our in-mem batches. Another option would be something like \[(K, [(V, [(T, D)\])])], which would only store each key’s and each val’s data once (this is similar to the [differential_dataflow::trace::layers::Trie] structure of [differential_dataflow::trace::implementations::ord::OrdValBatch]).

Which is better probably comes down to how much duplication we expect of keys and vals in a batch as well as how big the batches are (the trie structure introduces more columns, so has some amount of overhead).

For unsealed batches, we have a better chance of duplicates than trace, but we probably don’t want to pay the cost of converting between the in-mem \[(K, V, T, D)\] representation and anything else (to keep the hot path clean). Unsealed batches are also likely to be our smallest. For this reason, they’ll probably always stay as ParquetKvtd.

For trace batches, we consolidate them before writing them out, so we’re guaranteed to get nothing from the V level of the trie. For duplicate keys, we’ll probably get a good amount of benefit from column specific compression, and I’d like to exhaust that direction first before dealing with a trie-like column structure.

§

ParquetStructured = 3

Parquet format that understands the structure of the inner data. See the metadata field on ProtoBatchPartInline for more information about how this data is structured in Parquet.

For example, the initial use of this format will contain the columns \[(K, V, T, D, K_S, V_S)\] where K_S and V_S are structured versions of the opaque data stored in K and V, respectively. Eventually we’ll stop writing K and V and migrate entirely to K_S and V_S.

Implementations§

source§

impl ProtoBatchFormat

source

pub fn is_valid(value: i32) -> bool

Returns true if value is a variant of ProtoBatchFormat.

source

pub fn from_i32(value: i32) -> Option<ProtoBatchFormat>

👎Deprecated: Use the TryFrom<i32> implementation instead

Converts an i32 to a ProtoBatchFormat, or None if value is not a valid variant.

source§

impl ProtoBatchFormat

source

pub fn as_str_name(&self) -> &'static str

String value of the enum field names used in the ProtoBuf definition.

The values are not transformed in any way and thus are considered stable (if the ProtoBuf definition does not change) and safe for programmatic use.

source

pub fn from_str_name(value: &str) -> Option<Self>

Creates an enum from field names used in the ProtoBuf definition.

Trait Implementations§

source§

impl Clone for ProtoBatchFormat

source§

fn clone(&self) -> ProtoBatchFormat

Returns a copy of the value. Read more
1.0.0 · source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
source§

impl Debug for ProtoBatchFormat

source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
source§

impl Default for ProtoBatchFormat

source§

fn default() -> ProtoBatchFormat

Returns the “default value” for a type. Read more
source§

impl From<ProtoBatchFormat> for i32

source§

fn from(value: ProtoBatchFormat) -> i32

Converts to this type from the input type.
source§

impl Hash for ProtoBatchFormat

source§

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher. Read more
1.3.0 · source§

fn hash_slice<H>(data: &[Self], state: &mut H)
where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher. Read more
source§

impl Ord for ProtoBatchFormat

source§

fn cmp(&self, other: &ProtoBatchFormat) -> Ordering

This method returns an Ordering between self and other. Read more
1.21.0 · source§

fn max(self, other: Self) -> Self
where Self: Sized,

Compares and returns the maximum of two values. Read more
1.21.0 · source§

fn min(self, other: Self) -> Self
where Self: Sized,

Compares and returns the minimum of two values. Read more
1.50.0 · source§

fn clamp(self, min: Self, max: Self) -> Self
where Self: Sized + PartialOrd,

Restrict a value to a certain interval. Read more
source§

impl PartialEq for ProtoBatchFormat

source§

fn eq(&self, other: &ProtoBatchFormat) -> bool

Tests for self and other values to be equal, and is used by ==.
1.0.0 · source§

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.
source§

impl PartialOrd for ProtoBatchFormat

source§

fn partial_cmp(&self, other: &ProtoBatchFormat) -> Option<Ordering>

This method returns an ordering between self and other values if one exists. Read more
1.0.0 · source§

fn lt(&self, other: &Rhs) -> bool

Tests less than (for self and other) and is used by the < operator. Read more
1.0.0 · source§

fn le(&self, other: &Rhs) -> bool

Tests less than or equal to (for self and other) and is used by the <= operator. Read more
1.0.0 · source§

fn gt(&self, other: &Rhs) -> bool

Tests greater than (for self and other) and is used by the > operator. Read more
1.0.0 · source§

fn ge(&self, other: &Rhs) -> bool

Tests greater than or equal to (for self and other) and is used by the >= operator. Read more
source§

impl TryFrom<i32> for ProtoBatchFormat

§

type Error = UnknownEnumValue

The type returned in the event of a conversion error.
source§

fn try_from(value: i32) -> Result<ProtoBatchFormat, UnknownEnumValue>

Performs the conversion.
source§

impl Copy for ProtoBatchFormat

source§

impl Eq for ProtoBatchFormat

source§

impl StructuralPartialEq for ProtoBatchFormat

Auto Trait Implementations§

Blanket Implementations§

source§

impl<T> Any for T
where T: 'static + ?Sized,

source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
source§

impl<T> AsOut<T> for T
where T: Copy,

source§

fn as_out(&mut self) -> Out<'_, T>

Returns an out reference to self.
source§

impl<T> Borrow<T> for T
where T: ?Sized,

source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
source§

impl<T, U> CastInto<U> for T
where U: CastFrom<T>,

source§

fn cast_into(self) -> U

Performs the cast.
source§

impl<T> CloneToUninit for T
where T: Clone,

source§

default unsafe fn clone_to_uninit(&self, dst: *mut T)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dst. Read more
source§

impl<T> CloneToUninit for T
where T: Copy,

source§

unsafe fn clone_to_uninit(&self, dst: *mut T)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dst. Read more
source§

impl<Q, K> Comparable<K> for Q
where Q: Ord + ?Sized, K: Borrow<Q> + ?Sized,

source§

fn compare(&self, key: &K) -> Ordering

Compare self to key and return their ordering.
source§

impl<T> CopyAs<T> for T

source§

fn copy_as(self) -> T

source§

impl<Q, K> Equivalent<K> for Q
where Q: Eq + ?Sized, K: Borrow<Q> + ?Sized,

source§

fn equivalent(&self, key: &K) -> bool

Compare self to key and return true if they are equal.
source§

impl<Q, K> Equivalent<K> for Q
where Q: Eq + ?Sized, K: Borrow<Q> + ?Sized,

source§

fn equivalent(&self, key: &K) -> bool

Checks if this value is equivalent to the given key. Read more
source§

impl<Q, K> Equivalent<K> for Q
where Q: Eq + ?Sized, K: Borrow<Q> + ?Sized,

source§

fn equivalent(&self, key: &K) -> bool

Compare self to key and return true if they are equal.
source§

impl<Q, K> Equivalent<K> for Q
where Q: Eq + ?Sized, K: Borrow<Q> + ?Sized,

source§

fn equivalent(&self, key: &K) -> bool

Checks if this value is equivalent to the given key. Read more
source§

impl<T> From<T> for T

source§

fn from(t: T) -> T

Returns the argument unchanged.

source§

impl<T> FromRef<T> for T
where T: Clone,

source§

fn from_ref(input: &T) -> T

Converts to this type from a reference to the input type.
source§

impl<T> FutureExt for T

source§

fn with_context(self, otel_cx: Context) -> WithContext<Self>

Attaches the provided Context to this type, returning a WithContext wrapper. Read more
source§

fn with_current_context(self) -> WithContext<Self>

Attaches the current Context to this type, returning a WithContext wrapper. Read more
source§

impl<T> Hashable for T
where T: Hash,

§

type Output = u64

The type of the output value.
source§

fn hashed(&self) -> u64

A well-distributed integer derived from the data.
source§

impl<T> Instrument for T

source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
source§

impl<T, U> Into<U> for T
where U: From<T>,

source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

source§

impl<T> IntoRequest<T> for T

source§

fn into_request(self) -> Request<T>

Wrap the input message T in a tonic::Request
source§

impl<Unshared, Shared> IntoShared<Shared> for Unshared
where Shared: FromUnshared<Unshared>,

source§

fn into_shared(self) -> Shared

Creates a shared type from an unshared type.
source§

impl<T> Pointable for T

source§

const ALIGN: usize = _

The alignment of pointer.
§

type Init = T

The type for initializers.
source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
source§

impl<T> PreferredContainer for T
where T: Ord + Clone + 'static,

§

type Container = Vec<T>

The preferred container for the type.
source§

impl<T> ProgressEventTimestamp for T
where T: Data + Debug + Any,

source§

fn as_any(&self) -> &(dyn Any + 'static)

Upcasts this ProgressEventTimestamp to Any. Read more
source§

fn type_name(&self) -> &'static str

Returns the name of the concrete type of this object. Read more
source§

impl<P, R> ProtoType<R> for P
where R: RustType<P>,

source§

impl<T> Same for T

§

type Output = T

Should always be Self
source§

impl<'a, S, T> Semigroup<&'a S> for T
where T: Semigroup<S>,

source§

fn plus_equals(&mut self, rhs: &&'a S)

The method of std::ops::AddAssign, for types that do not implement AddAssign.
source§

impl<T> ToOwned for T
where T: Clone,

§

type Owned = T

The resulting type after obtaining ownership.
source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

§

type Error = Infallible

The type returned in the event of a conversion error.
source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

source§

fn vzip(self) -> V

source§

impl<T> WithSubscriber for T

source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more
source§

impl<T> Allocation for T
where T: RefUnwindSafe + Send + Sync,

source§

impl<T> Data for T
where T: Clone + 'static,

source§

impl<T> Data for T
where T: Data + Ord + Debug,

source§

impl<T> Sequence for T
where T: Eq + Hash,