Skip to main content

rebuild_data

Function rebuild_data 

Source
fn rebuild_data(
    data: ArrayData,
    nullable: bool,
    metrics: &ColumnarMetrics,
) -> ArrayData
Expand description

Walks the given arrow ArrayData recursively, dropping null buffers from non-nullable fields.

Workaround for https://github.com/apache/arrow-rs/issues/6510: parquet decoding can generate nulls in non-nullable fields that are only masked by, e.g., a grandparent, but some arrow code expects the direct parent to mask its non-nullable children. Dropping the buffer here prevents those validations from failing. (Top-level arrays are always marked nullable, so they’re unaffected.)