Function parquet::arrow::parquet_to_arrow_field_levels
source · pub fn parquet_to_arrow_field_levels(
schema: &SchemaDescriptor,
mask: ProjectionMask,
hint: Option<&Fields>,
) -> Result<FieldLevels>
Expand description
Convert a parquet SchemaDescriptor
to FieldLevels
Columns not included within ProjectionMask
will be ignored.
Where a field type in hint
is compatible with the corresponding parquet type in schema
, it
will be used, otherwise the default arrow type for the given parquet column type will be used.
This is to accommodate arrow types that cannot be round-tripped through parquet natively.
Depending on the parquet writer, this can lead to a mismatch between a file’s parquet schema
and its embedded arrow schema. The parquet schema
must be treated as authoritative in such
an event. See #1663 for more information
Note: this is a low-level API, most users will want to make use of the higher-level
parquet_to_arrow_schema
for decoding metadata from a parquet file.