Function mz_persist_client::internal::state_diff::apply_compaction_lenient

fn apply_compaction_lenient<'a, T: Timestamp + Lattice>(
    metrics: &Metrics,
    trace: Vec<HollowBatch<T>>,
    replacement: &'a HollowBatch<T>,
) -> Result<Vec<HollowBatch<T>>, String>

Apply a compaction diff that doesn’t exactly line up with the set of HollowBatches.

Because of the way Spine internally optimizes only some empty batches (immediately merges them in), we can end up in a situation where a compaction res applied on another copy of state, but when we replay all of the state diffs against a new Spine locally, it merges empty batches differently in-mem and we can’t exactly apply the compaction diff. Example:

• compact: [1,2),[2,3) -> [1,3)
• this spine: [0,2),[2,3) (0,1 is empty)

Ideally, we’d figure out a way to avoid this, but nothing immediately comes to mind. In the meantime, force the application (otherwise the shard is stuck and we can’t do anything with it) by manually splitting the empty batch back out. For the example above:

• [0,1),[1,3) (0,1 is empty)

This can only happen when the batch needing to be split is empty, so error out if it isn’t because that means something unexpected is going on.

TODO: This implementation is certainly not correct if T is actually only partially ordered.