libsql Schema migration - tracking issue

implements the happy path of schema migration. do not use this feature yet.

Here a list of remaining work to get schema migration working properly:

[x] move all the migration-related stuff from the meta_store to the schema module; let the scheduler use its own connection to the meta store. (#1131)
[x] migration-specific result builder (#1146 )
[x] perform error handling, check migration success status, use custom QueryResultHandler to collect results (#1146)
[x] ensure that the temp table enqueued_tasks is updated corectly after a task is popped when a tasks status changed (maybe use a trigger?) (#1126)
[x] handle abort; first when a dry run failed, then if a user abort the migration before the dry-run is complete (#1163)
[x] prevent a job from being enqueued if there is still a pending job for that namespaces (#1131)
[x] prevent writes while there is are pending jobs in a database (https://github.com/tursodatabase/libsql/pull/1139)
[x] backup database before starting migration (#1140)
[x] backup meta store after enqueue job (#1144)
[x] track version schema
[ ] go through all todos that I left in the code
[x] when performing a task, and that task was already performed, report the status, and don't re-perform the task (#1136)
[x] When a new db is created with a shared schema, bootstrap schema with current schema version. (#1154)
[x] prevent db creation while schema has pending migration it (#1137)
[x] admin API collect status of job (#1134)
[x] shared schema options should only be settable at creation (not possible through config update)
[x] ensure migration doesn't contain any transaction statements (#1149)
[x] deadlock in ns_loaded_with_pending_tasks_writes_is_blocked (#1173)
[x] prevent new migrations if a namespace creation is under way. (i.e. a new namespace creation should not miss fresh migration) (#1183)
[ ] test schema delete

Mar 04 '24 13:03 MarinPostma

what's also missing I think is cloning schema to newly created db that links to existing schema

Mar 04 '24 16:03 haaawk

Add whatever you can think of to the list

Mar 04 '24 20:03 MarinPostma

Can we close this @MarinPostma ?

Aug 07 '24 13:08 haaawk