lighteval icon indicating copy to clipboard operation
lighteval copied to clipboard

fix llm as judge warnings

Open NathanHB opened this issue 1 year ago • 2 comments

NathanHB avatar Apr 24 '24 12:04 NathanHB

Now loads openai lazily. the dep is no linger required when using lighteval

NathanHB avatar May 16 '24 13:05 NathanHB

You could add a warning/error when loading metrics at the task level, to check there the deps if needed.

clefourrier avatar May 21 '24 09:05 clefourrier

You could add a warning/error when loading metrics at the task level, to check there the deps if needed.

you mean like I did in task.py to check that the token is set ?

NathanHB avatar May 31 '24 15:05 NathanHB

And for both of these, I mean using the same checks as for all the other optional deps :) (using new functions defined in utils)

clefourrier avatar May 31 '24 16:05 clefourrier

@NathanHB I added the import testing I was thinking about - I'll let you review and if it works for you you can merge

clefourrier avatar Jul 03 '24 14:07 clefourrier