Struct parquet::column::writer::GenericColumnWriter
source · pub struct GenericColumnWriter<'a, E: ColumnValueEncoder> { /* private fields */ }
Implementations§
source§impl<'a, E: ColumnValueEncoder> GenericColumnWriter<'a, E>
impl<'a, E: ColumnValueEncoder> GenericColumnWriter<'a, E>
pub fn new( descr: ColumnDescPtr, props: WriterPropertiesPtr, page_writer: Box<dyn PageWriter + 'a>, ) -> Self
sourcepub fn write_batch(
&mut self,
values: &E::Values,
def_levels: Option<&[i16]>,
rep_levels: Option<&[i16]>,
) -> Result<usize>
pub fn write_batch( &mut self, values: &E::Values, def_levels: Option<&[i16]>, rep_levels: Option<&[i16]>, ) -> Result<usize>
Writes batch of values, definition levels and repetition levels. Returns number of values processed (written).
If definition and repetition levels are provided, we write fully those levels and select how many values to write (this number will be returned), since number of actual written values may be smaller than provided values.
If only values are provided, then all values are written and the length of of the values buffer is returned.
Definition and/or repetition levels can be omitted, if values are non-nullable and/or non-repeated.
sourcepub fn write_batch_with_statistics(
&mut self,
values: &E::Values,
def_levels: Option<&[i16]>,
rep_levels: Option<&[i16]>,
min: Option<&E::T>,
max: Option<&E::T>,
distinct_count: Option<u64>,
) -> Result<usize>
pub fn write_batch_with_statistics( &mut self, values: &E::Values, def_levels: Option<&[i16]>, rep_levels: Option<&[i16]>, min: Option<&E::T>, max: Option<&E::T>, distinct_count: Option<u64>, ) -> Result<usize>
Writer may optionally provide pre-calculated statistics for use when computing chunk-level statistics
NB: WriterProperties::statistics_enabled
must be set to EnabledStatistics::Chunk
for these statistics to take effect. If EnabledStatistics::None
they will be ignored,
and if EnabledStatistics::Page
the chunk statistics will instead be computed from the
computed page statistics
sourcepub fn get_total_bytes_written(&self) -> u64
pub fn get_total_bytes_written(&self) -> u64
Returns total number of bytes written by this column writer so far. This value is also returned when column writer is closed.
Note: this value does not include any buffered data that has not yet been flushed to a page.
sourcepub fn get_total_rows_written(&self) -> u64
pub fn get_total_rows_written(&self) -> u64
Returns total number of rows written by this column writer so far. This value is also returned when column writer is closed.
sourcepub fn get_descriptor(&self) -> &ColumnDescPtr
pub fn get_descriptor(&self) -> &ColumnDescPtr
Returns a reference to a ColumnDescPtr
sourcepub fn close(self) -> Result<ColumnCloseResult>
pub fn close(self) -> Result<ColumnCloseResult>
Finalizes writes and closes the column writer. Returns total bytes written, total rows written and column chunk metadata.