Struct parquet::file::writer::SerializedRowGroupWriter

source ·
pub struct SerializedRowGroupWriter<'a, W: Write> { /* private fields */ }
Expand description

Parquet row group writer API. Provides methods to access column writers in an iterator-like fashion, order is guaranteed to match the order of schema leaves (column descriptors).

All columns should be written sequentially; the main workflow is:

  • Request the next column using next_column method - this will return None if no more columns are available to write.
  • Once done writing a column, close column writer with close
  • Once all columns have been written, close row group writer with close method - it will return row group metadata and is no-op on already closed row group.

Implementations§

source§

impl<'a, W: Write + Send> SerializedRowGroupWriter<'a, W>

source

pub fn new( schema_descr: SchemaDescPtr, properties: WriterPropertiesPtr, buf: &'a mut TrackedWrite<W>, row_group_index: i16, on_close: Option<OnCloseRowGroup<'a>>, ) -> Self

Creates a new SerializedRowGroupWriter with:

  • schema_descr - the schema to write
  • properties - writer properties
  • buf - the buffer to write data to
  • row_group_index - row group index in this parquet file.
  • file_offset - file offset of this row group in this parquet file.
  • on_close - an optional callback that will invoked on Self::close
source

pub fn next_column(&mut self) -> Result<Option<SerializedColumnWriter<'_>>>

Returns the next column writer, if available; otherwise returns None. In case of any IO error or Thrift error, or if row group writer has already been closed returns Err.

source

pub fn append_column<R: ChunkReader>( &mut self, reader: &R, close: ColumnCloseResult, ) -> Result<()>

Append an encoded column chunk from another source without decoding it

This can be used for efficiently concatenating or projecting parquet data, or encoding parquet data to temporary in-memory buffers

See Self::next_column for writing data that isn’t already encoded

source

pub fn close(self) -> Result<RowGroupMetaDataPtr>

Closes this row group writer and returns row group metadata.

Auto Trait Implementations§

§

impl<'a, W> Freeze for SerializedRowGroupWriter<'a, W>

§

impl<'a, W> !RefUnwindSafe for SerializedRowGroupWriter<'a, W>

§

impl<'a, W> Send for SerializedRowGroupWriter<'a, W>
where W: Send,

§

impl<'a, W> !Sync for SerializedRowGroupWriter<'a, W>

§

impl<'a, W> Unpin for SerializedRowGroupWriter<'a, W>

§

impl<'a, W> !UnwindSafe for SerializedRowGroupWriter<'a, W>

Blanket Implementations§

source§

impl<T> Any for T
where T: 'static + ?Sized,

source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
source§

impl<T> Borrow<T> for T
where T: ?Sized,

source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
source§

impl<T> From<T> for T

source§

fn from(t: T) -> T

Returns the argument unchanged.

source§

impl<T, U> Into<U> for T
where U: From<T>,

source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

§

type Error = Infallible

The type returned in the event of a conversion error.
source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

source§

fn vzip(self) -> V