Skip to main content

schema_mismatch

Function schema_mismatch 

Source
fn schema_mismatch(current: &Schema, expected: &Schema) -> Option<String>
Expand description

Why the sink can’t write into a table that already carries current, where expected is the schema the sink would create today, or None if it can. The reason names one field, so it stays short however wide the schema is.

Names and nullability have to match. Types have to match too, except that a column may be fixed[16] where expected has string. Uuid columns were written as fixed binary before, and a table holding one stays writable, because the writer builds its Arrow columns from the table’s own schema rather than from expected. Rejecting it would strand the sink.

NOTE: the tolerance keys on the Iceberg types alone, so it also accepts a fixed[16] column where the relation has text. Such a table fails on the first row written instead of here. Only a table created outside Materialize can be in that state, since the sink never creates one.

Field IDs are not checked as Materialize may generate them differently than the catalog (i.e., DFS vs BFS for nested types), which could result in failures on sink restart.