Struct parquet::file::writer::SerializedRowGroupWriter
source · pub struct SerializedRowGroupWriter<'a, W: Write> { /* private fields */ }
Expand description
Parquet row group writer API. Provides methods to access column writers in an iterator-like fashion, order is guaranteed to match the order of schema leaves (column descriptors).
All columns should be written sequentially; the main workflow is:
- Request the next column using
next_column
method - this will returnNone
if no more columns are available to write. - Once done writing a column, close column writer with
close
- Once all columns have been written, close row group writer with
close
method - it will return row group metadata and is no-op on already closed row group.
Implementations§
source§impl<'a, W: Write + Send> SerializedRowGroupWriter<'a, W>
impl<'a, W: Write + Send> SerializedRowGroupWriter<'a, W>
sourcepub fn new(
schema_descr: SchemaDescPtr,
properties: WriterPropertiesPtr,
buf: &'a mut TrackedWrite<W>,
row_group_index: i16,
on_close: Option<OnCloseRowGroup<'a>>,
) -> Self
pub fn new( schema_descr: SchemaDescPtr, properties: WriterPropertiesPtr, buf: &'a mut TrackedWrite<W>, row_group_index: i16, on_close: Option<OnCloseRowGroup<'a>>, ) -> Self
Creates a new SerializedRowGroupWriter
with:
schema_descr
- the schema to writeproperties
- writer propertiesbuf
- the buffer to write data torow_group_index
- row group index in this parquet file.file_offset
- file offset of this row group in this parquet file.on_close
- an optional callback that will invoked onSelf::close
sourcepub fn next_column(&mut self) -> Result<Option<SerializedColumnWriter<'_>>>
pub fn next_column(&mut self) -> Result<Option<SerializedColumnWriter<'_>>>
Returns the next column writer, if available; otherwise returns None
.
In case of any IO error or Thrift error, or if row group writer has already been
closed returns Err
.
sourcepub fn append_column<R: ChunkReader>(
&mut self,
reader: &R,
close: ColumnCloseResult,
) -> Result<()>
pub fn append_column<R: ChunkReader>( &mut self, reader: &R, close: ColumnCloseResult, ) -> Result<()>
Append an encoded column chunk from another source without decoding it
This can be used for efficiently concatenating or projecting parquet data, or encoding parquet data to temporary in-memory buffers
See Self::next_column
for writing data that isn’t already encoded
sourcepub fn close(self) -> Result<RowGroupMetaDataPtr>
pub fn close(self) -> Result<RowGroupMetaDataPtr>
Closes this row group writer and returns row group metadata.