pub struct Batch<K, V, T, D>
where T: Timestamp + Lattice + Codec64,
{ /* private fields */ }
Expand description

A handle to a batch of updates that has been written to blob storage but which has not yet been appended to a shard.

A Batch needs to be marked as consumed or it needs to be deleted via Self::delete. Otherwise, a dangling batch will leak and backing blobs will remain in blob storage.

Implementations§

source§

impl<K, V, T, D> Batch<K, V, T, D>
where K: Debug + Codec, V: Debug + Codec, T: Timestamp + Lattice + Codec64, D: Semigroup + Codec64,

source

pub fn shard_id(&self) -> ShardId

The shard_id of this Batch.

source

pub fn upper(&self) -> &Antichain<T>

The upper of this Batch.

source

pub fn lower(&self) -> &Antichain<T>

The lower of this Batch.

source

pub fn rewrite_ts( &mut self, frontier: &Antichain<T>, new_upper: Antichain<T> ) -> Result<(), InvalidUsage<T>>

Efficiently rewrites the timestamps in this not-yet-committed batch.

This Batch represents potentially large amounts of data, which may have partly or entirely been spilled to s3. This call bulk edits the timestamps of all data in this batch in a metadata-only operation (i.e. without network calls).

Specifically, every timestamp in the batch is logically advanced_by the provided frontier.

This method may be called multiple times, with later calls overriding previous ones, but the rewrite frontier may not regress across calls.

When this batch was created, it was given an upper, which bounds the staged data it represents. To allow rewrite past this original upper, this call accepts a new upper which replaces the previous one. Like the rewrite frontier, the upper may not regress across calls.

Multiple batches with various rewrite frontiers may be used in a single crate::write::WriteHandle::compare_and_append_batch call. This is an expected usage.

source

pub async fn delete(self)

Deletes the blobs that make up this batch from the given blob store and marks them as deleted.

source

pub fn into_hollow_batch(self) -> HollowBatch<T>

Turns this Batch into a HollowBatch.

NOTE: If this batch is not eventually appended to a shard or dropped, the data that it represents will have leaked.

source

pub fn into_transmittable_batch(self) -> ProtoBatch

Turns this Batch into a ProtoBatch, which can be used to transfer this batch across process boundaries, for example when exchanging data between timely workers.

NOTE: If this batch is not eventually appended to a shard or dropped, the data that it represents will have leaked. The caller is responsible for turning this back into a Batch using WriteHandle::batch_from_transmittable_batch.

Trait Implementations§

source§

impl<K: Debug, V: Debug, T, D: Debug> Debug for Batch<K, V, T, D>
where T: Timestamp + Lattice + Codec64 + Debug,

source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
source§

impl<K, V, T, D> Drop for Batch<K, V, T, D>
where T: Timestamp + Lattice + Codec64,

source§

fn drop(&mut self)

Executes the destructor for this type. Read more

Auto Trait Implementations§

§

impl<K, V, T, D> !RefUnwindSafe for Batch<K, V, T, D>

§

impl<K, V, T, D> Send for Batch<K, V, T, D>

§

impl<K, V, T, D> Sync for Batch<K, V, T, D>

§

impl<K, V, T, D> Unpin for Batch<K, V, T, D>
where T: Unpin,

§

impl<K, V, T, D> !UnwindSafe for Batch<K, V, T, D>

Blanket Implementations§

source§

impl<T> Any for T
where T: 'static + ?Sized,

source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
source§

impl<T> Borrow<T> for T
where T: ?Sized,

source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
source§

impl<T, U> CastInto<U> for T
where U: CastFrom<T>,

source§

fn cast_into(self) -> U

Performs the cast.
source§

impl<T> From<T> for T

source§

fn from(t: T) -> T

Returns the argument unchanged.

source§

impl<T> FutureExt for T

source§

fn with_context(self, otel_cx: Context) -> WithContext<Self>

Attaches the provided Context to this type, returning a WithContext wrapper. Read more
source§

fn with_current_context(self) -> WithContext<Self>

Attaches the current Context to this type, returning a WithContext wrapper. Read more
source§

impl<T> Instrument for T

source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
source§

impl<T, U> Into<U> for T
where U: From<T>,

source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

source§

impl<T> IntoRequest<T> for T

source§

fn into_request(self) -> Request<T>

Wrap the input message T in a tonic::Request
source§

impl<Unshared, Shared> IntoShared<Shared> for Unshared
where Shared: FromUnshared<Unshared>,

source§

fn into_shared(self) -> Shared

Creates a shared type from an unshared type.
source§

impl<T> Pointable for T

source§

const ALIGN: usize = _

The alignment of pointer.
§

type Init = T

The type for initializers.
source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
source§

impl<P, R> ProtoType<R> for P
where R: RustType<P>,

source§

impl<T> PushInto<Vec<T>> for T

source§

fn push_into(self, target: &mut Vec<T>)

Push self into the target container.
source§

impl<T> Same for T

§

type Output = T

Should always be Self
source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

§

type Error = Infallible

The type returned in the event of a conversion error.
source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

source§

fn vzip(self) -> V

source§

impl<T> WithSubscriber for T

source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more