Struct parquet::file::writer::SerializedFileWriter
source · pub struct SerializedFileWriter<W: Write> { /* private fields */ }
Expand description
Parquet file writer API. Provides methods to write row groups sequentially.
The main workflow should be as following:
- Create file writer, this will open a new file and potentially write some metadata.
- Request a new row group writer by calling
next_row_group
. - Once finished writing row group, close row group writer by calling
close
- Write subsequent row groups, if necessary.
- After all row groups have been written, close the file writer using
close
method.
Implementations§
source§impl<W: Write + Send> SerializedFileWriter<W>
impl<W: Write + Send> SerializedFileWriter<W>
sourcepub fn new(
buf: W,
schema: TypePtr,
properties: WriterPropertiesPtr,
) -> Result<Self>
pub fn new( buf: W, schema: TypePtr, properties: WriterPropertiesPtr, ) -> Result<Self>
Creates new file writer.
sourcepub fn next_row_group(&mut self) -> Result<SerializedRowGroupWriter<'_, W>>
pub fn next_row_group(&mut self) -> Result<SerializedRowGroupWriter<'_, W>>
Creates new row group from this file writer.
In case of IO error or Thrift error, returns Err
.
There is no limit on a number of row groups in a file; however, row groups have
to be written sequentially. Every time the next row group is requested, the
previous row group must be finalised and closed using RowGroupWriter::close
method.
sourcepub fn flushed_row_groups(&self) -> &[RowGroupMetaDataPtr]
pub fn flushed_row_groups(&self) -> &[RowGroupMetaDataPtr]
Returns metadata for any flushed row groups
sourcepub fn finish(&mut self) -> Result<FileMetaData>
pub fn finish(&mut self) -> Result<FileMetaData>
Close and finalize the underlying Parquet writer
Unlike Self::close
this does not consume self
Attempting to write after calling finish will result in an error
sourcepub fn close(self) -> Result<FileMetaData>
pub fn close(self) -> Result<FileMetaData>
Closes and finalises file writer, returning the file metadata.
pub fn append_key_value_metadata(&mut self, kv_metadata: KeyValue)
sourcepub fn schema_descr(&self) -> &SchemaDescriptor
pub fn schema_descr(&self) -> &SchemaDescriptor
Returns a reference to schema descriptor.
sourcepub fn properties(&self) -> &WriterPropertiesPtr
pub fn properties(&self) -> &WriterPropertiesPtr
Returns a reference to the writer properties
sourcepub fn inner_mut(&mut self) -> &mut W
pub fn inner_mut(&mut self) -> &mut W
Returns a mutable reference to the underlying writer.
It is inadvisable to directly write to the underlying writer.
sourcepub fn into_inner(self) -> Result<W>
pub fn into_inner(self) -> Result<W>
Writes the file footer and returns the underlying writer.
sourcepub fn bytes_written(&self) -> usize
pub fn bytes_written(&self) -> usize
Returns the number of bytes written to this instance