Silvia Colabrese
Silvia Colabrese
@zzhang37, this PR is pretty old, if still needed, could you rebase it?
Closing it until further notice
Ok, thanks @Luca-Calabria. Waiting for @regisss then
Ok, thanks for confirmation!
@yafshar, @alexey-belyakov this is part II
Related to the examples to add, @yafshar, I've added llama-3.2-1B. While for this model (w/o Instruct) the accuracy we get is the same as reported in https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct (i.e., 32.2), I've...
@yafshar ack, I will ping you. In the background I'm adding needed or useful parameters, but I'm blocked on ifeval task, as accuracies on that task are still not replicable...
@veritas9872, very useful comments! In fact, using `gsm8k_cot_llama` really boosted the exact match accuracy! As for the differences in score, I'm fine with tiny variations (as it is hard to...
> I am also curious as to whether this can be merged in time before the next release. Not before 1.16 is out, but shortly after
Chiming in. Tests that were run, and I believe you can use to check, are: ``` cd optimum-habana/ pip install -e . python -m pip install .[tests] pip install pytest...