Expand description
async
API for writing RecordBatch
es to Parquet files
See the crate-level documentation for more details.
The async
API for writing RecordBatch
es is
similar to the sync
API, so please
read the documentation there before using this API.
Here is an example for using AsyncArrowWriter
:
let col = Arc::new(Int64Array::from_iter_values([1, 2, 3])) as ArrayRef;
let to_write = RecordBatch::try_from_iter([("col", col)]).unwrap();
let mut buffer = Vec::new();
let mut writer = AsyncArrowWriter::try_new(&mut buffer, to_write.schema(), None).unwrap();
writer.write(&to_write).await.unwrap();
writer.close().await.unwrap();
let buffer = Bytes::from(buffer);
let mut reader = ParquetRecordBatchReaderBuilder::try_new(buffer.clone())
.unwrap()
.build()
.unwrap();
let read = reader.next().unwrap().unwrap();
assert_eq!(to_write, read);
[object_store
] provides it’s native implementation of AsyncFileWriter
by [ParquetObjectWriter
].
Structs§
- Async
Arrow Writer - Encodes
RecordBatch
to parquet, outputting to anAsyncFileWriter
Traits§
- Async
File Writer - The asynchronous interface used by
AsyncArrowWriter
to write parquet files.