feat: add process to track and run data migrations
The upcoming changes to normalize some of event payload requires running a data migration a la nosql style. This provides tables to track the daemon version and the state of migrations. They can be run by calling the daemon with the --migrate-data flag. If there are required migrations, the daemon will fail to start.
So I realized the migrations to extract metadata isn't as urgent as I thought (I fixed IOD to not have bugs without it, and it might be more efficient tbh - will have the cleaned up PR today, still need to split it/make it easier to review). I do think we want to do it before GA, but I don't think I have the new table structures schema quite right, so it might make sense to pause this for a couple days. This doesn't really do much, and it doesn't hurt to merge this and store version info, but we may want to track started_at or other data (so we can see if you downgrade in history). And we may want to support running migrations in the background (slowly read events and extract metadata but maybe it's better to just rip through everything for now), which would mean the flag might not be quite right.
Anyway, @stbrody, curious to hear your thoughts/discuss briefly before you spend any more time on this (and I really appreciate how much you've already spent).