Struct parquet::file::properties::BloomFilterProperties

source ·
pub struct BloomFilterProperties {
    pub fpp: f64,
    pub ndv: u64,
}
Expand description

Controls the bloom filter to be computed by the writer.

Fields§

§fpp: f64

False positive probability, should be always between 0 and 1 exclusive. Defaults to DEFAULT_BLOOM_FILTER_FPP.

You should set this value by calling WriterPropertiesBuilder::set_bloom_filter_fpp.

The bloom filter data structure is a trade of between disk and memory space versus fpp, the smaller the fpp, the more memory and disk space is required, thus setting it to a reasonable value e.g. 0.1, 0.05, or 0.001 is recommended.

Setting to very small number diminishes the value of the filter itself, as the bitset size is even larger than just storing the whole value. You are also expected to set ndv if it can be known in advance in order to largely reduce space usage.

§ndv: u64

Number of distinct values, should be non-negative to be meaningful. Defaults to DEFAULT_BLOOM_FILTER_NDV.

You should set this value by calling WriterPropertiesBuilder::set_bloom_filter_ndv.

Usage of bloom filter is most beneficial for columns with large cardinality, so a good heuristic is to set ndv to number of rows. However it can reduce disk size if you know in advance a smaller number of distinct values. For very small ndv value it is probably not worth it to use bloom filter anyway.

Increasing this value (without increasing fpp) will result in an increase in disk or memory size.

Trait Implementations§

source§

impl Clone for BloomFilterProperties

source§

fn clone(&self) -> BloomFilterProperties

Returns a copy of the value. Read more
1.0.0 · source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
source§

impl Debug for BloomFilterProperties

source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
source§

impl Default for BloomFilterProperties

source§

fn default() -> Self

Returns the “default value” for a type. Read more
source§

impl PartialEq for BloomFilterProperties

source§

fn eq(&self, other: &BloomFilterProperties) -> bool

Tests for self and other values to be equal, and is used by ==.
1.0.0 · source§

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.
source§

impl StructuralPartialEq for BloomFilterProperties

Auto Trait Implementations§

Blanket Implementations§

source§

impl<T> Any for T
where T: 'static + ?Sized,

source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
source§

impl<T> Borrow<T> for T
where T: ?Sized,

source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
source§

impl<T> CloneToUninit for T
where T: Clone,

source§

default unsafe fn clone_to_uninit(&self, dst: *mut T)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dst. Read more
source§

impl<T> From<T> for T

source§

fn from(t: T) -> T

Returns the argument unchanged.

source§

impl<T, U> Into<U> for T
where U: From<T>,

source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

source§

impl<T> ToOwned for T
where T: Clone,

§

type Owned = T

The resulting type after obtaining ownership.
source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

§

type Error = Infallible

The type returned in the event of a conversion error.
source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

source§

fn vzip(self) -> V

source§

impl<T> Allocation for T
where T: RefUnwindSafe + Send + Sync,