Trait persist::operators::stream::Persist

pub trait Persist<G: Scope<Timestamp = u64>, K: TimelyData, V: TimelyData> {
    fn persist(
        &self, 
        name: &str, 
        write: StreamWriteHandle<K, V>
    ) -> (Stream<G, ((K, V), u64, isize)>, Stream<G, (String, u64, isize)>);
}

Extension trait for Stream.

Required methods

fn persist(
    &self,
    name: &str,
    write: StreamWriteHandle<K, V>
) -> (Stream<G, ((K, V), u64, isize)>, Stream<G, (String, u64, isize)>)

Passes through each element of the stream and persists it.

This does not wait for persistence before passing through the data. We do, however, wait for data to be persisted before allowing the frontier to advance. In other words, this operator is holding on to capabilities as long as data belonging to their timestamp is not persisted.

Use this together with seal and await_frontier if you want to make sure that data only becomes available downstream when persisted and sealed.

Note: If you need to also replay persisted data when restarting, concatenate the output of this operator with the output of replay().

Implementations on Foreign Types

impl<G, K, V> Persist<G, K, V> for Stream<G, ((K, V), u64, isize)> where
    G: Scope<Timestamp = u64>,
    K: TimelyData + Codec,
    V: TimelyData + Codec, 

fn persist(
    &self,
    name: &str,
    write: StreamWriteHandle<K, V>
) -> (Stream<G, ((K, V), u64, isize)>, Stream<G, (String, u64, isize)>)

Implementors