ShokoAnime
Add Database export and import as json arrays.
Your cache read everything right? Why not read database to cache and export cache ? that way you wouldn't have to figure out syntax and etc ;-)
But you probably already started it
not everything, in the last commit, you'll see what repositories are cached and whatnot, i think 1/3 of the repositories are cached the others not, anyway, since IRepository is the base interface, to dump, you only need to do GetAll(), and then store as json file. its trivial to do it, but it need some GUI :)
In response to https://github.com/ShokoAnime/ShokoServer/issues/693, this should have the ability to filter what to export and import. The rules will need to be set, as there are ID references and such to consider. For example:
- AniDB Data (in case you want guaranteed fresh data)
- Files (VideoLocals and places, this will need to store with a map of import folders and give the ability to remap on import)
- File states (watched states, MyList, etc. A combination of VideoLocal_User and settings)
- Links (Manual Links, ignored status, variation for multiple files, and CrossRef_File_Episode)
- Community Links (in case you manually linked them and they weren't approved yet)
- Groups (This will be stored with AniDB series IDs to make sure they match)
- Series Data (this will be AnimeSeries, along with overridden names and such)
- Group Filters
- Community site data (in case you want fresh data)
- Ignored Anime for recommendations
- Votes (in case you have it set not to sync, you can back up local votes)
That's all I can think of right now, but it's a start. Obviously, some of these are dependent on others.
I don't know if it needs to be JSON, as JSON is slow, but it'll need serialized one way or another. Protobuf or another binary serialization method could work as well.
JSON is slow because of reflection, and boxing, there isn't a lot that can be done, also in this case, would it be necessary for this to be a lightning quick procedure?
Yes but my assumption is that a reflection operation on every model in the database is going to be slow. If I'm wrong, then thank God, but I doubt that I am.
Why would we need to export every model? We could just export core data like links. And then re-populate the rest?
Also potentially have the option for a full dump?
On Thu., 23 Nov. 2017, 1:56 am Thomas Baer, [email protected] wrote:
Yes but my assumption is that a reflection operation on every model in the database is going to be slow. If I'm wrong, then thank God, but I doubt that I am.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ShokoAnime/ShokoServer/issues/370#issuecomment-346373801, or mute the thread https://github.com/notifications/unsubscribe-auth/AA8rplX-RXmXsMgFivd-7uUIWOXdaC6Wks5s5DYbgaJpZM4JkryW .
Serializing al to json should be fast enough, the whole idea is not over complex the whole thing, just touch, select a directory, and done. Every table will get a [table].json with the all the data in there. i don't expect to be more than few minutes on 3000+ series. The good thing about this, is sql agnostic, so you could use to have a backup, or translate your whole collection to other database. The only thing we should be aware, is the import order, or temporary remove foreign references, autonumbers, etc before import. We could add filters after, but the base functionality should be dump the whole db.
On Wed, Nov 22, 2017 at 11:57 AM, Cayde Dixon [email protected] wrote:
Why would we need to export every model? We could just export core data like links. And then re-populate the rest?
Also potentially have the option for a full dump?
On Thu., 23 Nov. 2017, 1:56 am Thomas Baer, [email protected] wrote:
Yes but my assumption is that a reflection operation on every model in the database is going to be slow. If I'm wrong, then thank God, but I doubt that I am.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub <https://github.com/ShokoAnime/ShokoServer/issues/ 370#issuecomment-346373801>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA8rplX-RXmXsMgFivd- 7uUIWOXdaC6Wks5s5DYbgaJpZM4JkryW>
.
— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/ShokoAnime/ShokoServer/issues/370#issuecomment-346374342, or mute the thread https://github.com/notifications/unsubscribe-auth/ABHDbepiD98j5mJ4HvT4ktwkOpiJqjynks5s5DZwgaJpZM4JkryW .