pub struct ColumnIndex {
pub null_pages: Vec<bool>,
pub min_values: Vec<Vec<u8>>,
pub max_values: Vec<Vec<u8>>,
pub boundary_order: BoundaryOrder,
pub null_counts: Option<Vec<i64>>,
pub repetition_level_histograms: Option<Vec<i64>>,
pub definition_level_histograms: Option<Vec<i64>>,
}
Expand description
Optional statistics for each data page in a ColumnChunk.
Forms part the page index, along with OffsetIndex.
If this structure is present, OffsetIndex must also be present.
For each field in this structure, <field>
[i] refers to the page at
OffsetIndex.page_locations[i]
Fields§
§null_pages: Vec<bool>
A list of Boolean values to determine the validity of the corresponding min and max values. If true, a page contains only null values, and writers have to set the corresponding entries in min_values and max_values to byte[0], so that all lists have the same length. If false, the corresponding entries in min_values and max_values must be valid.
min_values: Vec<Vec<u8>>
Two lists containing lower and upper bounds for the values of each page determined by the ColumnOrder of the column. These may be the actual minimum and maximum values found on a page, but can also be (more compact) values that do not exist on a page. For example, instead of storing ““Blart Versenwald III”, a writer may set min_values[i]=“B”, max_values[i]=“C”. Such more compact values must still be valid values within the column’s logical type. Readers must make sure that list entries are populated before using them by inspecting null_pages.
max_values: Vec<Vec<u8>>
§boundary_order: BoundaryOrder
Stores whether both min_values and max_values are ordered and if so, in which direction. This allows readers to perform binary searches in both lists. Readers cannot assume that max_values[i] <= min_values[i+1], even if the lists are ordered.
null_counts: Option<Vec<i64>>
A list containing the number of null values for each page *
repetition_level_histograms: Option<Vec<i64>>
Contains repetition level histograms for each page concatenated together. The repetition_level_histogram field on SizeStatistics contains more details.
When present the length should always be (number of pages * (max_repetition_level + 1)) elements.
Element 0 is the first element of the histogram for the first page. Element (max_repetition_level + 1) is the first element of the histogram for the second page.
definition_level_histograms: Option<Vec<i64>>
Same as repetition_level_histograms except for definitions levels.
Implementations§
Source§impl ColumnIndex
impl ColumnIndex
Trait Implementations§
Source§impl Clone for ColumnIndex
impl Clone for ColumnIndex
Source§fn clone(&self) -> ColumnIndex
fn clone(&self) -> ColumnIndex
1.0.0 · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source
. Read more