Submission checker forces to have “gptj-99” and “gpt-99.9” in the code dir
It ends up that we have even three identical directories (see NVIDIA submission) which does not help with the review and understanding the repo by someone not involved in the WG.
Solution: there should be one directory per model
if needed, the submitter can create 99 & 99.9 directories
Now:
- gptj-99.9
- gptj-99
- gptj
After:
- gptj
After - optionally, directories per accuracy category:
- gptj/99.9
- gptj/99
I think we can discuss further on this. The problem with the proposal is that we no longer have the same model names inside the results/measurements directory (like this ) and the ones inside the code directory. Also to avoid duplicates in the code directory we can have softlinks as done here though this can be tricky on some filesystems.