pub struct RowSelection { /* private fields */ }
Expand description
RowSelection
allows selecting or skipping a provided number of rows
when scanning the parquet file.
This is applied prior to reading column data, and can therefore be used to skip IO to fetch data into memory
A typical use-case would be using the PageIndex
to filter out rows
that don’t satisfy a predicate
§Example
use parquet::arrow::arrow_reader::{RowSelection, RowSelector};
let selectors = vec![
RowSelector::skip(5),
RowSelector::select(5),
RowSelector::select(5),
RowSelector::skip(5),
];
// Creating a selection will combine adjacent selectors
let selection: RowSelection = selectors.into();
let expected = vec![
RowSelector::skip(5),
RowSelector::select(10),
RowSelector::skip(5),
];
let actual: Vec<RowSelector> = selection.into();
assert_eq!(actual, expected);
// you can also create a selection from consecutive ranges
let ranges = vec![5..10, 10..15];
let selection =
RowSelection::from_consecutive_ranges(ranges.into_iter(), 20);
let actual: Vec<RowSelector> = selection.into();
assert_eq!(actual, expected);
A RowSelection
maintains the following invariants:
- It contains no
RowSelector
of 0 rows - Consecutive
RowSelector
s alternate skipping or selecting rows
Implementations§
Source§impl RowSelection
impl RowSelection
Sourcepub fn from_filters(filters: &[BooleanArray]) -> Self
pub fn from_filters(filters: &[BooleanArray]) -> Self
Creates a RowSelection
from a slice of BooleanArray
§Panic
Panics if any of the BooleanArray
contain nulls
Sourcepub fn from_consecutive_ranges<I: Iterator<Item = Range<usize>>>(
ranges: I,
total_rows: usize,
) -> Self
pub fn from_consecutive_ranges<I: Iterator<Item = Range<usize>>>( ranges: I, total_rows: usize, ) -> Self
Creates a RowSelection
from an iterator of consecutive ranges to keep
Sourcepub fn scan_ranges(&self, page_locations: &[PageLocation]) -> Vec<Range<usize>>
pub fn scan_ranges(&self, page_locations: &[PageLocation]) -> Vec<Range<usize>>
Given an offset index, return the byte ranges for all data pages selected by self
This is useful for determining what byte ranges to fetch from underlying storage
Note: this method does not make any effort to combine consecutive ranges, nor coalesce
ranges that are close together. This is instead delegated to the IO subsystem to optimise,
e.g. ObjectStore::get_ranges
Sourcepub fn split_off(&mut self, row_count: usize) -> Self
pub fn split_off(&mut self, row_count: usize) -> Self
Splits off the first row_count
from this RowSelection
Sourcepub fn and_then(&self, other: &Self) -> Self
pub fn and_then(&self, other: &Self) -> Self
returns a RowSelection
representing rows that are selected in both
input RowSelection
s.
This is equivalent to the logical AND
/ conjunction of the two
selections.
§Example
If N
means the row is not selected, and Y
means it is
selected:
self: NNNNNNNNNNNNYYYYYYYYYYYYYYYYYYYYYYNNNYYYYY
other: YYYYYNNNNYYYYYYYYYYYYY YYNNN
returned: NNNNNNNNNNNNYYYYYNNNNYYYYYYYYYYYYYNNNYYNNN
§Panics
Panics if other
does not have a length equal to the number of rows selected
by this RowSelection
Sourcepub fn intersection(&self, other: &Self) -> Self
pub fn intersection(&self, other: &Self) -> Self
Compute the intersection of two RowSelection
For example:
self: NNYYYYNNYYNYN
other: NYNNNNNNY
returned: NNNNNNNNYYNYN
Sourcepub fn union(&self, other: &Self) -> Self
pub fn union(&self, other: &Self) -> Self
Compute the union of two RowSelection
For example:
self: NNYYYYNNYYNYN
other: NYNNNNNNN
returned: NYYYYYNNYYNYN
Sourcepub fn selects_any(&self) -> bool
pub fn selects_any(&self) -> bool
Returns true
if this RowSelection
selects any rows
Sourcepub fn iter(&self) -> impl Iterator<Item = &RowSelector>
pub fn iter(&self) -> impl Iterator<Item = &RowSelector>
Returns an iterator over the RowSelector
s for this
RowSelection
.
Sourcepub fn skipped_row_count(&self) -> usize
pub fn skipped_row_count(&self) -> usize
Returns the number of de-selected rows
Trait Implementations§
Source§impl Clone for RowSelection
impl Clone for RowSelection
Source§fn clone(&self) -> RowSelection
fn clone(&self) -> RowSelection
1.0.0 · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source
. Read more