parquet::file::writer

Struct SerializedFileWriter

Source
pub struct SerializedFileWriter<W: Write> { /* private fields */ }
Expand description

Parquet file writer API. Provides methods to write row groups sequentially.

The main workflow should be as following:

  • Create file writer, this will open a new file and potentially write some metadata.
  • Request a new row group writer by calling next_row_group.
  • Once finished writing row group, close row group writer by calling close
  • Write subsequent row groups, if necessary.
  • After all row groups have been written, close the file writer using close method.

Implementations§

Source§

impl<W: Write + Send> SerializedFileWriter<W>

Source

pub fn new( buf: W, schema: TypePtr, properties: WriterPropertiesPtr, ) -> Result<Self>

Creates new file writer.

Source

pub fn next_row_group(&mut self) -> Result<SerializedRowGroupWriter<'_, W>>

Creates new row group from this file writer. In case of IO error or Thrift error, returns Err.

There can be at most 2^15 row groups in a file; and row groups have to be written sequentially. Every time the next row group is requested, the previous row group must be finalised and closed using RowGroupWriter::close method.

Source

pub fn flushed_row_groups(&self) -> &[RowGroupMetaData]

Returns metadata for any flushed row groups

Source

pub fn finish(&mut self) -> Result<FileMetaData>

Close and finalize the underlying Parquet writer

Unlike Self::close this does not consume self

Attempting to write after calling finish will result in an error

Source

pub fn close(self) -> Result<FileMetaData>

Closes and finalises file writer, returning the file metadata.

Source

pub fn append_key_value_metadata(&mut self, kv_metadata: KeyValue)

Add a KeyValue to the file writer’s metadata

Source

pub fn schema_descr(&self) -> &SchemaDescriptor

Returns a reference to schema descriptor.

Source

pub fn properties(&self) -> &WriterPropertiesPtr

Returns a reference to the writer properties

Source

pub fn inner(&self) -> &W

Returns a reference to the underlying writer.

Source

pub fn inner_mut(&mut self) -> &mut W

Returns a mutable reference to the underlying writer.

It is inadvisable to directly write to the underlying writer.

Source

pub fn into_inner(self) -> Result<W>

Writes the file footer and returns the underlying writer.

Source

pub fn bytes_written(&self) -> usize

Returns the number of bytes written to this instance

Trait Implementations§

Source§

impl<W: Write> Debug for SerializedFileWriter<W>

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more

Auto Trait Implementations§

§

impl<W> Freeze for SerializedFileWriter<W>
where W: Freeze,

§

impl<W> RefUnwindSafe for SerializedFileWriter<W>
where W: RefUnwindSafe,

§

impl<W> Send for SerializedFileWriter<W>
where W: Send,

§

impl<W> Sync for SerializedFileWriter<W>
where W: Sync,

§

impl<W> Unpin for SerializedFileWriter<W>
where W: Unpin,

§

impl<W> UnwindSafe for SerializedFileWriter<W>
where W: UnwindSafe,

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

impl<T> Allocation for T
where T: RefUnwindSafe + Send + Sync,