lighteval icon indicating copy to clipboard operation
lighteval copied to clipboard

Data split depending on eval params

Open clefourrier opened this issue 1 year ago • 0 comments

This PR needs #158 to be merged first.

The main problem is that splits are no longer by size, but they are however now making sure that all batched generative evals are launched with similar evals (same sampling, same eos tokens)

clefourrier avatar Apr 22 '24 16:04 clefourrier