comdb2 icon indicating copy to clipboard operation
comdb2 copied to clipboard

The multi-table schema changes/partition re-engineer to be atomic

Open dorinhogea opened this issue 1 year ago • 2 comments

The multi-table schema changes (that includes partitions) need a single transactional llmeta entry to save all changes atomicatlly. The block processor for schema changes was revamped and copy&paste code shared with non-DDL transactions was eliminated. The finalize part was consolidated in one function now shared by both block processor runs and resuming schema change code. Old convention of keeping single table schema changes uuid == 0 was eliminated. In the old code, the single table schema changes where finalized by new master, but not the multi-table schema changes. Now the master resumes both types of schema changes. Resuming schema change is done out-of-band, once the proper setup for alters is executed. This will allow enforcing the limit on number of restarted schema changes on resume for large partitions.

NOTE: the old llmeta structures are still present so we can deploy this without breaking the cluster turn. Code will be removed after deployment.

dorinhogea avatar Jun 25 '24 21:06 dorinhogea

cdb2test Aug 4 17:37:21 2024 in-comdb2db multiddl.R20240804.1

dorinhogea avatar Aug 05 '24 13:08 dorinhogea

dhogea Aug 13 10:02:14 2024 in-comdb2db multiddl.R20240813.1

dorinhogea avatar Aug 13 '24 21:08 dorinhogea

@mponomar thank you

dorinhogea avatar Oct 07 '24 15:10 dorinhogea