Jackson comments

Results 4 comments of


                                            Jackson

Saving to GGUF llama.cpp / merging to 16bit for VLLM

Hey @danielhanchen, thanks for the amazing work! However, I tried saving 4-bits model but encounter following error: ``` NotImplementedError: You are calling `save_pretrained` on a 4-bit converted model. This is...

Saving to GGUF llama.cpp / merging to 16bit for VLLM

Ah its okay! I was planning to save it in 16-bit but getting errors, that's why I tried to use 4bit saving. But now I am able to save it...

Hello, I have a problem that the calculated output size is too small (51210）when I train the model, is there anyone who have successfully run the pretrained train_PBAFN_stage1_fs.sh model could give me some suggestions?

You could simply change the ceil mode of maxpool layer from last sequential layer of VGG networks to **True** if you are running on different pytorch version from author's. ![image](https://user-images.githubusercontent.com/51978507/199374709-a88beb92-d916-4a8b-91fc-f6b385806359.png)

Questions on Speculative Decoding in gpt-fast generate.py

Hi, I have a question. Since we are accepting the tokens whenever `q > p`, what happens when there is actually other token that has higher probability than `q`?

Jackson

Saving to GGUF llama.cpp / merging to 16bit for VLLM

Saving to GGUF llama.cpp / merging to 16bit for VLLM

Hello, I have a problem that the calculated output size is too small (512*1*0）when I train the model, is there anyone who have successfully run the pretrained train_PBAFN_stage1_fs.sh model could give me some suggestions?

Questions on Speculative Decoding in gpt-fast generate.py

Hello, I have a problem that the calculated output size is too small (51210）when I train the model, is there anyone who have successfully run the pretrained train_PBAFN_stage1_fs.sh model could give me some suggestions?