SafeAILab
Is there scripts to calculate the overall acceptance rate?
I am trying to calculate the acceptance rate for evaluation. I ran:
python -m eagle.evaluation.gen_ea_alpha_vicuna --ea-model-path ~/models/EAGLE-Vicuna-7B-v1.3/ --base-model-path ~/models/vicuna-7b-v1.3/ --model-id ess-vicuna-7b-fp16-alpha --temperature 1
However, this seems to only generates an output decoded file with alpha per question? How do I get the overall alpha rate as in the paper?
After obtaining the result file, you can run the eagle/evaluation/alpha.py file to get the acceptance rate.
After obtaining the result file, you can run the eagle/evaluation/alpha.py file to get the acceptance rate.
eagle/evaluation/gen_ea_alpha_llama2chat.py could't been excuted. It seems to be because the ea_model forward interface has changed.
After obtaining the result file, you can run the eagle/evaluation/alpha.py file to get the acceptance rate.
eagle/evaluation/gen_ea_alpha_llama2chat.py could't been excuted. It seems to be because the ea_model forward interface has changed.
I'm experiencing the same issue. The logits_processor is returning as [], and the following error occurs:
TypeError: forward() got an unexpected keyword argument 'logits_processor'.
Have you found a solution to the problem?
