lighteval
lighteval copied to clipboard
Data split depending on eval params
This PR needs #158 to be merged first.
The main problem is that splits are no longer by size, but they are however now making sure that all batched generative evals are launched with similar evals (same sampling, same eos tokens)