lighteval
lighteval copied to clipboard
Add `ligheval --list-tasks`
This command should list all the available tasks (in extended tasks and the task table)
Hello,
I would like to work on this. Do you need special formatting for the list?
Hi, thanks for your interest! I think providing the tasks grouped by suite would be nice, for example
- lighteval: all the tasks available, sorted alphabetically
- community: all the tasks available, sorted alphabetically
...
We have a lot of tasks so it would be better to have them in a list in that format, this makes reading easier as you can ligheval --list-tasks | grep mmlu for example.
- lighteval:
- task 1
- task 2
- community:
- task 1
- task 2
I understand. I'm working on it now.