Struct parquet::column::reader::GenericColumnReader

pub struct GenericColumnReader<R, D, V> { /* private fields */ }

Reads data for a given column chunk, using the provided decoders:

• R: ColumnLevelDecoder used to decode repetition levels
• D: ColumnLevelDecoder used to decode definition levels
• V: ColumnValueDecoder used to decode value data

Implementations

impl<V> GenericColumnReader<RepetitionLevelDecoderImpl, DefinitionLevelDecoderImpl, V>

where V: ColumnValueDecoder,

pub fn new(descr: ColumnDescPtr, page_reader: Box<dyn PageReader>) -> Self

Creates new column reader based on column descriptor and page reader.

impl<R, D, V> GenericColumnReader<R, D, V>

where R: RepetitionLevelDecoder, D: DefinitionLevelDecoder, V: ColumnValueDecoder,

pub fn read_batch( &mut self, batch_size: usize, def_levels: Option<&mut D::Buffer>, rep_levels: Option<&mut R::Buffer>, values: &mut V::Buffer, ) -> Result<(usize, usize)>

👎Deprecated: Use read_records

Reads a batch of values of at most batch_size, returning a tuple containing the actual number of non-null values read, followed by the corresponding number of levels, i.e, the total number of values including nulls, empty lists, etc…

If the max definition level is 0, def_levels will be ignored, otherwise it will be populated with the number of levels read, with an error returned if it is None.

If the max repetition level is 0, rep_levels will be ignored, otherwise it will be populated with the number of levels read, with an error returned if it is None.

values will be contiguously populated with the non-null values. Note that if the column is not required, this may be less than either batch_size or the number of levels read

pub fn read_records( &mut self, max_records: usize, def_levels: Option<&mut D::Buffer>, rep_levels: Option<&mut R::Buffer>, values: &mut V::Buffer, ) -> Result<(usize, usize, usize)>

Read up to max_records whole records, returning the number of complete records, non-null values and levels decoded. All levels for a given record will be read, i.e. the next repetition level, if any, will be 0

If the max definition level is 0, def_levels will be ignored and the number of records, non-null values and levels decoded will all be equal, otherwise def_levels will be populated with the number of levels read, with an error returned if it is None.

If the max repetition level is 0, rep_levels will be ignored and the number of records and levels decoded will both be equal, otherwise rep_levels will be populated with the number of levels read, with an error returned if it is None.

values will be contiguously populated with the non-null values. Note that if the column is not required, this may be less than either max_records or the number of levels read

pub fn skip_records(&mut self, num_records: usize) -> Result<usize>

Skips over num_records records, where records are delimited by repetition levels of 0

Returns

Returns the number of records skipped