Struct DataFileWriter

pub struct DataFileWriter<B: FileWriterBuilder> { /* private fields */ }

A writer write data is within one spec/partition.

Trait Implementations

impl<B: FileWriterBuilder> CurrentFileStatus for DataFileWriter<B>

fn current_file_path(&self) -> String

Get the current file path.

fn current_row_num(&self) -> usize

Get the current file row number.

fn current_written_size(&self) -> usize

Get the current file written size.

fn current_schema(&self) -> SchemaRef

Get the current schema used by the writer.

impl<B: Debug + FileWriterBuilder> Debug for DataFileWriter<B>

where B::R: Debug,

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<B: FileWriterBuilder> IcebergWriter for DataFileWriter<B>

fn write<'life0, 'async_trait>( &'life0 mut self, batch: RecordBatch, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Write data to iceberg table.

fn close<'life0, 'async_trait>( &'life0 mut self, ) -> Pin<Box<dyn Future<Output = Result<Vec<DataFile>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Close the writer and return the written data files. If close failed, the data written before maybe be lost. User may need to recreate the writer and rewrite the data again.