Proposal todo list

[r0mainK]

Spitballing some ideas, @vmarkovtsev WDYT :

• [ ] Structure
  • [ ] Model stuff could be in a directory
  • [ ] Bakend stuff could be in a directory
  • [ ] Logging could be in a directory
  • [ ] Command line should be in a directory, and split properly (eg tools.py, registry.py)
  • [ ] All the rest should be in utils
• [ ] Command line
  • [ ] Add command to create configuration (modelforgecfg.py file or edit a given pre-existing bash/text file
  • [ ] Add command to edit only the registry (amend), by uploading a modified meta.json. It should be able to edit either a specific model, or a series of model
  • [ ] Add command to add external files to the registry, e.g. tests, travis.yaml file, etc
  • [ ] Include gitbooks, update templates
• [ ] Code
  • [ ] Switch to Pathlib entirely
  • [ ] Update the code, we are not even using the new functionnalities of slogging
  • [ ] Simplify the code as much as possible. For instance, Model().load( ... ) is overly complex, and some features are useless imo, eg loading a model from the model
• [ ] Documentation
  • [ ] Explain configuration, depending on the utilization. Right now it's quite opaque
  • [ ] Explain how metadata can be added in command line, there's no real explanaton apart from the default template and a link to src-d/models
  • [ ] We should talk about slogging since we use it, and it's part of the API
  • [ ] Add doc for new commands
  • [ ] Add an index of the API
  • [ ] List internal methods that should not be overrided in Model

[vmarkovtsev]

Can you please sort these ideas by priority:

• Strictly required to publish the new model without destroying Eiso's work.
• Nice to have and you are really up to it.
• Everything else.

[r0mainK]

High priority:

• command to edit registry, since we need to edit all models -> license issue
• command to add additional files to the registry -> enable adding tests and separate config files
• include gitbooks files
• update template files

Nice to have:

• command to create config, should be easy
• some easy grabs like using slogging
• everything in structure: easy to do, requires updating a couple repos (ml-core, ml-mining)
• doc: for new commands, add index of the API, list internal methods of Model

Everything else:

• updates to the code: pathlib, simplify current structure
• all other doc changes

[vmarkovtsev]

Thanks!

Regarding the amend command, it is fine to edit JSON manually and PR it for now. That's actually the intended use-case which leverages the strong side of maintaining it in a Git repo on GitHub.

Let's do the highest priority points and the low hanging fruits from the "nice to have". We'll return back later because our ultimate goal is to publish the new model ASAP.

[r0mainK]

@vmarkovtsev True, but we also have to update the index pages. Anyway yeah, if I see it takes too long I will skip it and just amend