Skip to main content

do_commit

Function do_commit 

Source
async fn do_commit(
    table: &Table,
    catalog: &dyn Catalog,
    snapshot_properties: Vec<(String, String)>,
    data_files: Vec<DataFile>,
    delete_files: Vec<DataFile>,
) -> Result<Table, CommitError>
Expand description

Build a row delta commit against the given table and send it to the catalog.

We can’t use iceberg-rust’s Transaction::commit (a retry wrapper for Transaction::do_commit) because it automatically rebases the transaction onto the latest state of the table.

That behavior leads to duplicate writes because there’s no way to check:

  1. Have we already committed this data?
  2. Has another (newer) writer taken over?

So this implementation doesn’t do that (details inline).