Struct mz_compute::render::context::CollectionBundle

source ·
pub struct CollectionBundle<S: Scope, T = Timestamp>{
    pub collection: Option<(Collection<S, Row, Diff>, Collection<S, DataflowError, Diff>)>,
    pub arranged: BTreeMap<Vec<MirScalarExpr>, ArrangementFlavor<S, T>>,
}
Expand description

A bundle of the various ways a collection can be represented.

This type maintains the invariant that it does contain at least one valid source of data, either a collection or at least one arrangement.

Fields§

§collection: Option<(Collection<S, Row, Diff>, Collection<S, DataflowError, Diff>)>§arranged: BTreeMap<Vec<MirScalarExpr>, ArrangementFlavor<S, T>>

Implementations§

source§

impl<S: Scope, T> CollectionBundle<S, T>

source

pub fn from_collections( oks: Collection<S, Row, Diff>, errs: Collection<S, DataflowError, Diff>, ) -> Self

Construct a new collection bundle from update streams.

source

pub fn from_expressions( exprs: Vec<MirScalarExpr>, arrangements: ArrangementFlavor<S, T>, ) -> Self

Inserts arrangements by the expressions on which they are keyed.

source

pub fn from_columns<I: IntoIterator<Item = usize>>( columns: I, arrangements: ArrangementFlavor<S, T>, ) -> Self

Inserts arrangements by the columns on which they are keyed.

source

pub fn scope(&self) -> S

The scope containing the collection bundle.

source

pub fn enter_region<'a>( &self, region: &Child<'a, S, S::Timestamp>, ) -> CollectionBundle<Child<'a, S, S::Timestamp>, T>

Brings the collection bundle into a region.

source§

impl<'a, S: Scope, T> CollectionBundle<Child<'a, S, S::Timestamp>, T>

source

pub fn leave_region(&self) -> CollectionBundle<S, T>

Extracts the collection bundle from a region.

source§

impl<S: Scope, T> CollectionBundle<S, T>

source

pub fn as_specific_collection( &self, key: Option<&[MirScalarExpr]>, ) -> (Collection<S, Row, Diff>, Collection<S, DataflowError, Diff>)

Asserts that the arrangement for a specific key (or the raw collection for no key) exists, and returns the corresponding collection.

This returns the collection as-is, without doing any unthinning transformation. Therefore, it should be used when the appropriate transformation was planned as part of a following MFP.

source

pub fn flat_map<D, I, C, L>( &self, key_val: Option<(Vec<MirScalarExpr>, Option<Row>)>, constructor: C, ) -> (Stream<S, I::Item>, Collection<S, DataflowError, Diff>)
where I: IntoIterator<Item = (D, S::Timestamp, Diff)>, D: Data, C: FnOnce() -> L, L: for<'a, 'b> FnMut(&'a mut DatumVecBorrow<'b>, &'a S::Timestamp, &'a Diff) -> I + 'static,

Constructs and applies logic to elements of a collection and returns the results.

constructor takes a permutation and produces the logic to apply on elements. The logic conceptually receives (&Row, &Row) pairs in the form of a slice. Only after borrowing the elements and applying the permutation the datums will be in the expected order.

If key_val is set, this is a promise that logic will produce no results on records for which the key does not evaluate to the value. This is used when we have an arrangement by that key to leap directly to exactly those records. It is important that logic still guard against data that does not satisfy this constraint, as this method does not statically know that it will have that arrangement.

source

fn flat_map_core<Tr, K, D, I, L>( trace: &Arranged<S, Tr>, key: Option<K>, logic: L, refuel: usize, ) -> Stream<S, I::Item>
where for<'a> Tr::Key<'a>: ToDatumIter + IntoOwned<'a, Owned = K>, for<'a> Tr::Val<'a>: ToDatumIter, Tr: TraceReader<Time = S::Timestamp, Diff = Diff> + Clone + 'static, K: PartialEq + 'static, I: IntoIterator<Item = (D, Tr::Time, Tr::Diff)>, D: Data, L: for<'a, 'b> FnMut(Tr::Key<'_>, Tr::Val<'_>, &'a S::Timestamp, &'a Diff) -> I + 'static,

Factored out common logic for using literal keys in general traces.

This logic is sufficiently interesting that we want to write it only once, and thereby avoid any skew in the two uses of the logic.

The function presents the contents of the trace as (key, value, time, delta) tuples, where key and value are potentially specialized, but convertible into rows.

source

pub fn arrangement( &self, key: &[MirScalarExpr], ) -> Option<ArrangementFlavor<S, T>>

Look up an arrangement by the expressions that form the key.

The result may be None if no such arrangement exists, or it may be one of many “arrangement flavors” that represent the types of arranged data we might have.

source§

impl<S, T> CollectionBundle<S, T>

source

pub fn as_collection_core( &self, mfp: MapFilterProject, key_val: Option<(Vec<MirScalarExpr>, Option<Row>)>, until: Antichain<Timestamp>, ) -> (Collection<S, Row, Diff>, Collection<S, DataflowError, Diff>)

Presents self as a stream of updates, having been subjected to mfp.

This operator is able to apply the logic of mfp early, which can substantially reduce the amount of data produced when mfp is non-trivial.

The key_val argument, when present, indicates that a specific arrangement should be used, and if, in addition, the val component is present, that we can seek to the supplied row.

source

pub fn ensure_collections( self, collections: AvailableCollections, input_key: Option<Vec<MirScalarExpr>>, input_mfp: MapFilterProject, until: Antichain<Timestamp>, ) -> Self

source

fn specialized_arrange( name: &String, oks: Collection<S, Row, i64>, key: &Vec<MirScalarExpr>, thinning: &Vec<usize>, ) -> (MzArrangement<S>, Collection<S, DataflowError, i64>)

Builds a specialized arrangement to provided types. The specialization for key and value types of the arrangement is based on the bit length derived from the corresponding type descriptions.

Trait Implementations§

source§

impl<S: Clone + Scope, T> Clone for CollectionBundle<S, T>

source§

fn clone(&self) -> CollectionBundle<S, T>

Returns a copy of the value. Read more
1.0.0 · source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more

Auto Trait Implementations§

§

impl<S, T> Freeze for CollectionBundle<S, T>
where <S as ScopeParent>::Timestamp: Sized, S: Freeze,

§

impl<S, T = Timestamp> !RefUnwindSafe for CollectionBundle<S, T>

§

impl<S, T = Timestamp> !Send for CollectionBundle<S, T>

§

impl<S, T = Timestamp> !Sync for CollectionBundle<S, T>

§

impl<S, T> Unpin for CollectionBundle<S, T>
where <S as ScopeParent>::Timestamp: Sized, S: Unpin,

§

impl<S, T = Timestamp> !UnwindSafe for CollectionBundle<S, T>

Blanket Implementations§

source§

impl<T> Any for T
where T: 'static + ?Sized,

source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
source§

impl<T> Borrow<T> for T
where T: ?Sized,

source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
source§

impl<T, U> CastInto<U> for T
where U: CastFrom<T>,

source§

fn cast_into(self) -> U

Performs the cast.
source§

impl<T> CloneToUninit for T
where T: Clone,

source§

default unsafe fn clone_to_uninit(&self, dst: *mut T)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dst. Read more
source§

impl<T> Conv for T

source§

fn conv<T>(self) -> T
where Self: Into<T>,

Converts self into T using Into<T>. Read more
source§

impl<T> DynClone for T
where T: Clone,

source§

impl<T> FmtForward for T

source§

fn fmt_binary(self) -> FmtBinary<Self>
where Self: Binary,

Causes self to use its Binary implementation when Debug-formatted.
source§

fn fmt_display(self) -> FmtDisplay<Self>
where Self: Display,

Causes self to use its Display implementation when Debug-formatted.
source§

fn fmt_lower_exp(self) -> FmtLowerExp<Self>
where Self: LowerExp,

Causes self to use its LowerExp implementation when Debug-formatted.
source§

fn fmt_lower_hex(self) -> FmtLowerHex<Self>
where Self: LowerHex,

Causes self to use its LowerHex implementation when Debug-formatted.
source§

fn fmt_octal(self) -> FmtOctal<Self>
where Self: Octal,

Causes self to use its Octal implementation when Debug-formatted.
source§

fn fmt_pointer(self) -> FmtPointer<Self>
where Self: Pointer,

Causes self to use its Pointer implementation when Debug-formatted.
source§

fn fmt_upper_exp(self) -> FmtUpperExp<Self>
where Self: UpperExp,

Causes self to use its UpperExp implementation when Debug-formatted.
source§

fn fmt_upper_hex(self) -> FmtUpperHex<Self>
where Self: UpperHex,

Causes self to use its UpperHex implementation when Debug-formatted.
source§

fn fmt_list(self) -> FmtList<Self>
where &'a Self: for<'a> IntoIterator,

Formats each item in a sequence. Read more
source§

impl<T> From<T> for T

source§

fn from(t: T) -> T

Returns the argument unchanged.

source§

impl<T> FromRef<T> for T
where T: Clone,

source§

fn from_ref(input: &T) -> T

Converts to this type from a reference to the input type.
source§

impl<T> FutureExt for T

source§

fn with_context(self, otel_cx: Context) -> WithContext<Self>

Attaches the provided Context to this type, returning a WithContext wrapper. Read more
source§

fn with_current_context(self) -> WithContext<Self>

Attaches the current Context to this type, returning a WithContext wrapper. Read more
source§

impl<T> Instrument for T

source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
source§

impl<T, U> Into<U> for T
where U: From<T>,

source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

source§

impl<T> IntoRequest<T> for T

source§

fn into_request(self) -> Request<T>

Wrap the input message T in a tonic::Request
source§

impl<Unshared, Shared> IntoShared<Shared> for Unshared
where Shared: FromUnshared<Unshared>,

source§

fn into_shared(self) -> Shared

Creates a shared type from an unshared type.
source§

impl<T, U> OverrideFrom<Option<&T>> for U
where U: OverrideFrom<T>,

source§

fn override_from(self, layer: &Option<&T>) -> U

Override the configuration represented by Self with values from the given layer.
source§

impl<T> Pipe for T
where T: ?Sized,

source§

fn pipe<R>(self, func: impl FnOnce(Self) -> R) -> R
where Self: Sized,

Pipes by value. This is generally the method you want to use. Read more
source§

fn pipe_ref<'a, R>(&'a self, func: impl FnOnce(&'a Self) -> R) -> R
where R: 'a,

Borrows self and passes that borrow into the pipe function. Read more
source§

fn pipe_ref_mut<'a, R>(&'a mut self, func: impl FnOnce(&'a mut Self) -> R) -> R
where R: 'a,

Mutably borrows self and passes that borrow into the pipe function. Read more
source§

fn pipe_borrow<'a, B, R>(&'a self, func: impl FnOnce(&'a B) -> R) -> R
where Self: Borrow<B>, B: 'a + ?Sized, R: 'a,

Borrows self, then passes self.borrow() into the pipe function. Read more
source§

fn pipe_borrow_mut<'a, B, R>( &'a mut self, func: impl FnOnce(&'a mut B) -> R, ) -> R
where Self: BorrowMut<B>, B: 'a + ?Sized, R: 'a,

Mutably borrows self, then passes self.borrow_mut() into the pipe function. Read more
source§

fn pipe_as_ref<'a, U, R>(&'a self, func: impl FnOnce(&'a U) -> R) -> R
where Self: AsRef<U>, U: 'a + ?Sized, R: 'a,

Borrows self, then passes self.as_ref() into the pipe function.
source§

fn pipe_as_mut<'a, U, R>(&'a mut self, func: impl FnOnce(&'a mut U) -> R) -> R
where Self: AsMut<U>, U: 'a + ?Sized, R: 'a,

Mutably borrows self, then passes self.as_mut() into the pipe function.
source§

fn pipe_deref<'a, T, R>(&'a self, func: impl FnOnce(&'a T) -> R) -> R
where Self: Deref<Target = T>, T: 'a + ?Sized, R: 'a,

Borrows self, then passes self.deref() into the pipe function.
source§

fn pipe_deref_mut<'a, T, R>( &'a mut self, func: impl FnOnce(&'a mut T) -> R, ) -> R
where Self: DerefMut<Target = T> + Deref, T: 'a + ?Sized, R: 'a,

Mutably borrows self, then passes self.deref_mut() into the pipe function.
source§

impl<T> Pointable for T

source§

const ALIGN: usize = _

The alignment of pointer.
§

type Init = T

The type for initializers.
source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
source§

impl<P, R> ProtoType<R> for P
where R: RustType<P>,

source§

impl<T> Same for T

§

type Output = T

Should always be Self
source§

impl<'a, S, T> Semigroup<&'a S> for T
where T: Semigroup<S>,

source§

fn plus_equals(&mut self, rhs: &&'a S)

The method of std::ops::AddAssign, for types that do not implement AddAssign.
source§

impl<T> Tap for T

source§

fn tap(self, func: impl FnOnce(&Self)) -> Self

Immutable access to a value. Read more
source§

fn tap_mut(self, func: impl FnOnce(&mut Self)) -> Self

Mutable access to a value. Read more
source§

fn tap_borrow<B>(self, func: impl FnOnce(&B)) -> Self
where Self: Borrow<B>, B: ?Sized,

Immutable access to the Borrow<B> of a value. Read more
source§

fn tap_borrow_mut<B>(self, func: impl FnOnce(&mut B)) -> Self
where Self: BorrowMut<B>, B: ?Sized,

Mutable access to the BorrowMut<B> of a value. Read more
source§

fn tap_ref<R>(self, func: impl FnOnce(&R)) -> Self
where Self: AsRef<R>, R: ?Sized,

Immutable access to the AsRef<R> view of a value. Read more
source§

fn tap_ref_mut<R>(self, func: impl FnOnce(&mut R)) -> Self
where Self: AsMut<R>, R: ?Sized,

Mutable access to the AsMut<R> view of a value. Read more
source§

fn tap_deref<T>(self, func: impl FnOnce(&T)) -> Self
where Self: Deref<Target = T>, T: ?Sized,

Immutable access to the Deref::Target of a value. Read more
source§

fn tap_deref_mut<T>(self, func: impl FnOnce(&mut T)) -> Self
where Self: DerefMut<Target = T> + Deref, T: ?Sized,

Mutable access to the Deref::Target of a value. Read more
source§

fn tap_dbg(self, func: impl FnOnce(&Self)) -> Self

Calls .tap() only in debug builds, and is erased in release builds.
source§

fn tap_mut_dbg(self, func: impl FnOnce(&mut Self)) -> Self

Calls .tap_mut() only in debug builds, and is erased in release builds.
source§

fn tap_borrow_dbg<B>(self, func: impl FnOnce(&B)) -> Self
where Self: Borrow<B>, B: ?Sized,

Calls .tap_borrow() only in debug builds, and is erased in release builds.
source§

fn tap_borrow_mut_dbg<B>(self, func: impl FnOnce(&mut B)) -> Self
where Self: BorrowMut<B>, B: ?Sized,

Calls .tap_borrow_mut() only in debug builds, and is erased in release builds.
source§

fn tap_ref_dbg<R>(self, func: impl FnOnce(&R)) -> Self
where Self: AsRef<R>, R: ?Sized,

Calls .tap_ref() only in debug builds, and is erased in release builds.
source§

fn tap_ref_mut_dbg<R>(self, func: impl FnOnce(&mut R)) -> Self
where Self: AsMut<R>, R: ?Sized,

Calls .tap_ref_mut() only in debug builds, and is erased in release builds.
source§

fn tap_deref_dbg<T>(self, func: impl FnOnce(&T)) -> Self
where Self: Deref<Target = T>, T: ?Sized,

Calls .tap_deref() only in debug builds, and is erased in release builds.
source§

fn tap_deref_mut_dbg<T>(self, func: impl FnOnce(&mut T)) -> Self
where Self: DerefMut<Target = T> + Deref, T: ?Sized,

Calls .tap_deref_mut() only in debug builds, and is erased in release builds.
source§

impl<T> ToOwned for T
where T: Clone,

§

type Owned = T

The resulting type after obtaining ownership.
source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
source§

impl<T> TryConv for T

source§

fn try_conv<T>(self) -> Result<T, Self::Error>
where Self: TryInto<T>,

Attempts to convert self into T using TryInto<T>. Read more
source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

§

type Error = Infallible

The type returned in the event of a conversion error.
source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

source§

fn vzip(self) -> V

source§

impl<T> WithSubscriber for T

source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more
source§

impl<T> Data for T
where T: Clone + 'static,