mhlee

Token-level Probability Always 0.0 When Fine-tuning Llama2-7b Model on Single GPU

1

**Describe the bug** The token-level probabilities consistently appear as 0.0 when fine-tuning the Llama2-7b model using "Ludwig + DeepLearning.ai: Efficient Fine-Tuning for Llama2-7b on a Single GPU.ipynb". https://colab.research.google.com/drive/1Ly01S--kUwkKQalE-75skalp-ftwl0fE?usp=sharing below thing...

Is there no way to inference without training?

3

Hi there, Thank you for the great work! I have some problem. In the Google colab environment ``` !git clone https://github.com/FasterDecoding/Medusa.git %cd Medusa !pip install -e . !python -m medusa.inference.cli...