Struct mz_persist_client::internal::trace::FlatTrace

source ·
pub struct FlatTrace<T> {
    pub(crate) since: Antichain<T>,
    pub(crate) legacy_batches: BTreeMap<Arc<HollowBatch<T>>, ()>,
    pub(crate) hollow_batches: BTreeMap<SpineId, Arc<HollowBatch<T>>>,
    pub(crate) spine_batches: BTreeMap<SpineId, ThinSpineBatch<T>>,
    pub(crate) merges: BTreeMap<SpineId, ThinMerge<T>>,
}
Expand description

This is a “flattened” representation of a Trace. Goals:

  • small updates to the trace should result in small differences in the FlatTrace;
  • two FlatTraces should be efficient to diff;
  • converting to and from a Trace should be relatively straightforward.

These goals are all somewhat in tension, and the space of possible representations is pretty large. See individual fields for comments on some of the tradeoffs.

Fields§

§since: Antichain<T>§legacy_batches: BTreeMap<Arc<HollowBatch<T>>, ()>

Hollow batches without an associated ID. If this flattened trace contains spine batches, we can figure out which legacy batch belongs in which spine batch by comparing the descs. Previously, we serialized a trace as just this list of batches. Keeping this data around helps ensure backwards compatibility. In the near future, we may still keep some batches here to help minimize the size of diffs – rewriting all the hollow batches in a shard can be prohibitively expensive. Eventually, we’d like to remove this in favour of the collection below.

§hollow_batches: BTreeMap<SpineId, Arc<HollowBatch<T>>>

Hollow batches with an associated ID. Spine batches can reference these hollow batches by id directly.

§spine_batches: BTreeMap<SpineId, ThinSpineBatch<T>>

Spine batches stored by ID. We reference hollow batches by ID, instead of inlining them, to make differential updates smaller when two batches merge together. We also store the level on the batch, instead of mapping from level to a list of batches… the level of a spine batch doesn’t change over time, but the list of batches at a particular level does.

§merges: BTreeMap<SpineId, ThinMerge<T>>

In-progress merges. We store this by spine id instead of level to prepare for some possible generalizations to spine (merging N of M batches at a level). This is also a natural place to store incremental merge progress in the future.

Trait Implementations§

source§

impl<T: Clone> Clone for FlatTrace<T>

source§

fn clone(&self) -> FlatTrace<T>

Returns a copy of the value. Read more
1.0.0 · source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
source§

impl<T: Debug> Debug for FlatTrace<T>

source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
source§

impl<T: Timestamp + Codec64> RustType<ProtoTrace> for FlatTrace<T>

source§

fn into_proto(&self) -> ProtoTrace

Convert a Self into a Proto value.
source§

fn from_proto(proto: ProtoTrace) -> Result<Self, TryFromProtoError>

Consume and convert a Proto back into a Self value. Read more
source§

fn into_proto_owned(self) -> Proto

A zero clone version of Self::into_proto that types can optionally implement, otherwise, the default implementation delegates to Self::into_proto.

Auto Trait Implementations§

§

impl<T> Freeze for FlatTrace<T>
where T: Freeze,

§

impl<T> RefUnwindSafe for FlatTrace<T>
where T: RefUnwindSafe,

§

impl<T> Send for FlatTrace<T>
where T: Send + Sync,

§

impl<T> Sync for FlatTrace<T>
where T: Sync + Send,

§

impl<T> Unpin for FlatTrace<T>
where T: Unpin,

§

impl<T> UnwindSafe for FlatTrace<T>

Blanket Implementations§

source§

impl<T> Any for T
where T: 'static + ?Sized,

source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
source§

impl<T> Borrow<T> for T
where T: ?Sized,

source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
source§

impl<T, U> CastInto<U> for T
where U: CastFrom<T>,

source§

fn cast_into(self) -> U

Performs the cast.
source§

impl<T> CloneToUninit for T
where T: Clone,

source§

default unsafe fn clone_to_uninit(&self, dst: *mut T)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dst. Read more
source§

impl<T> From<T> for T

source§

fn from(t: T) -> T

Returns the argument unchanged.

source§

impl<T> FromRef<T> for T
where T: Clone,

source§

fn from_ref(input: &T) -> T

Converts to this type from a reference to the input type.
source§

impl<T> FutureExt for T

source§

fn with_context(self, otel_cx: Context) -> WithContext<Self>

Attaches the provided Context to this type, returning a WithContext wrapper. Read more
source§

fn with_current_context(self) -> WithContext<Self>

Attaches the current Context to this type, returning a WithContext wrapper. Read more
source§

impl<T> Instrument for T

source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
source§

impl<T, U> Into<U> for T
where U: From<T>,

source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

source§

impl<T> IntoRequest<T> for T

source§

fn into_request(self) -> Request<T>

Wrap the input message T in a tonic::Request
source§

impl<Unshared, Shared> IntoShared<Shared> for Unshared
where Shared: FromUnshared<Unshared>,

source§

fn into_shared(self) -> Shared

Creates a shared type from an unshared type.
source§

impl<T> Pointable for T

source§

const ALIGN: usize = _

The alignment of pointer.
§

type Init = T

The type for initializers.
source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
source§

impl<T> ProgressEventTimestamp for T
where T: Data + Debug + Any,

source§

fn as_any(&self) -> &(dyn Any + 'static)

Upcasts this ProgressEventTimestamp to Any. Read more
source§

fn type_name(&self) -> &'static str

Returns the name of the concrete type of this object. Read more
source§

impl<P, R> ProtoType<R> for P
where R: RustType<P>,

source§

impl<T> Same for T

§

type Output = T

Should always be Self
source§

impl<'a, S, T> Semigroup<&'a S> for T
where T: Semigroup<S>,

source§

fn plus_equals(&mut self, rhs: &&'a S)

The method of std::ops::AddAssign, for types that do not implement AddAssign.
source§

impl<T> ToOwned for T
where T: Clone,

§

type Owned = T

The resulting type after obtaining ownership.
source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

§

type Error = Infallible

The type returned in the event of a conversion error.
source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

source§

fn vzip(self) -> V

source§

impl<T> WithSubscriber for T

source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more
source§

impl<T> Allocation for T
where T: RefUnwindSafe + Send + Sync,

source§

impl<T> Data for T
where T: Clone + 'static,