Struct parquet_format_safe::PageHeader

source ·
pub struct PageHeader {
    pub type_: PageType,
    pub uncompressed_page_size: i32,
    pub compressed_page_size: i32,
    pub crc: Option<i32>,
    pub data_page_header: Option<DataPageHeader>,
    pub index_page_header: Option<IndexPageHeader>,
    pub dictionary_page_header: Option<DictionaryPageHeader>,
    pub data_page_header_v2: Option<DataPageHeaderV2>,
}

Fields§

§type_: PageType

the type of the page: indicates which of the *_header fields is set *

§uncompressed_page_size: i32

Uncompressed page size in bytes (not including this header) *

§compressed_page_size: i32

Compressed (and potentially encrypted) page size in bytes, not including this header *

§crc: Option<i32>

The 32bit CRC for the page, to be be calculated as follows:

  • Using the standard CRC32 algorithm
  • On the data only, i.e. this header should not be included. ‘Data’ hereby refers to the concatenation of the repetition levels, the definition levels and the column value, in this exact order.
  • On the encoded versions of the repetition levels, definition levels and column values
  • On the compressed versions of the repetition levels, definition levels and column values where possible;
    • For v1 data pages, the repetition levels, definition levels and column values are always compressed together. If a compression scheme is specified, the CRC shall be calculated on the compressed version of this concatenation. If no compression scheme is specified, the CRC shall be calculated on the uncompressed version of this concatenation.
    • For v2 data pages, the repetition levels and definition levels are handled separately from the data and are never compressed (only encoded). If a compression scheme is specified, the CRC shall be calculated on the concatenation of the uncompressed repetition levels, uncompressed definition levels and the compressed column values. If no compression scheme is specified, the CRC shall be calculated on the uncompressed concatenation.
  • In encrypted columns, CRC is calculated after page encryption; the encryption itself is performed after page compression (if compressed) If enabled, this allows for disabling checksumming in HDFS if only a few pages need to be read.
§data_page_header: Option<DataPageHeader>§index_page_header: Option<IndexPageHeader>§dictionary_page_header: Option<DictionaryPageHeader>§data_page_header_v2: Option<DataPageHeaderV2>

Implementations§

source§

impl PageHeader

source

pub fn new<F4, F5, F6, F7, F8>( type_: PageType, uncompressed_page_size: i32, compressed_page_size: i32, crc: F4, data_page_header: F5, index_page_header: F6, dictionary_page_header: F7, data_page_header_v2: F8 ) -> PageHeader

source

pub fn read_from_in_protocol<T: TInputProtocol>( i_prot: &mut T ) -> Result<PageHeader>

source

pub async fn stream_from_in_protocol<T: TInputStreamProtocol>( i_prot: &mut T ) -> Result<PageHeader>

source

pub fn write_to_out_protocol<T: TOutputProtocol>( &self, o_prot: &mut T ) -> Result<usize>

source

pub async fn write_to_out_stream_protocol<T: TOutputStreamProtocol>( &self, o_prot: &mut T ) -> Result<usize>

Trait Implementations§

source§

impl AsyncReadThrift for PageHeader

source§

fn stream_from_in_protocol<'life0, 'async_trait, T>( i_prot: &'life0 mut T ) -> Pin<Box<dyn Future<Output = Result<PageHeader>> + Send + 'async_trait>>
where T: 'async_trait + TInputStreamProtocol, 'life0: 'async_trait,

source§

impl Clone for PageHeader

source§

fn clone(&self) -> PageHeader

Returns a copy of the value. Read more
1.0.0 · source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
source§

impl Debug for PageHeader

source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
source§

impl Hash for PageHeader

source§

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher. Read more
1.3.0 · source§

fn hash_slice<H>(data: &[Self], state: &mut H)
where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher. Read more
source§

impl Ord for PageHeader

source§

fn cmp(&self, other: &PageHeader) -> Ordering

This method returns an Ordering between self and other. Read more
1.21.0 · source§

fn max(self, other: Self) -> Self
where Self: Sized,

Compares and returns the maximum of two values. Read more
1.21.0 · source§

fn min(self, other: Self) -> Self
where Self: Sized,

Compares and returns the minimum of two values. Read more
1.50.0 · source§

fn clamp(self, min: Self, max: Self) -> Self
where Self: Sized + PartialOrd,

Restrict a value to a certain interval. Read more
source§

impl PartialEq for PageHeader

source§

fn eq(&self, other: &PageHeader) -> bool

This method tests for self and other values to be equal, and is used by ==.
1.0.0 · source§

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.
source§

impl PartialOrd for PageHeader

source§

fn partial_cmp(&self, other: &PageHeader) -> Option<Ordering>

This method returns an ordering between self and other values if one exists. Read more
1.0.0 · source§

fn lt(&self, other: &Rhs) -> bool

This method tests less than (for self and other) and is used by the < operator. Read more
1.0.0 · source§

fn le(&self, other: &Rhs) -> bool

This method tests less than or equal to (for self and other) and is used by the <= operator. Read more
1.0.0 · source§

fn gt(&self, other: &Rhs) -> bool

This method tests greater than (for self and other) and is used by the > operator. Read more
1.0.0 · source§

fn ge(&self, other: &Rhs) -> bool

This method tests greater than or equal to (for self and other) and is used by the >= operator. Read more
source§

impl ReadThrift for PageHeader

source§

impl Eq for PageHeader

source§

impl StructuralPartialEq for PageHeader

Auto Trait Implementations§

Blanket Implementations§

source§

impl<T> Any for T
where T: 'static + ?Sized,

source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
source§

impl<T> Borrow<T> for T
where T: ?Sized,

source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
source§

impl<T> From<T> for T

source§

fn from(t: T) -> T

Returns the argument unchanged.

source§

impl<T, U> Into<U> for T
where U: From<T>,

source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

source§

impl<T> ToOwned for T
where T: Clone,

§

type Owned = T

The resulting type after obtaining ownership.
source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

§

type Error = Infallible

The type returned in the event of a conversion error.
source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.