Matthew Peyrard comments

Repositories
Issues
Comments

Results 2 comments of


                                            Matthew Peyrard

LLAMA 3.1 8B Quantization failed from BF16 to FP8

I am hitting the same issue. It seems you need to have transformers 4.42.3, but this is impossible because "optimum" requires transformers to be a version

Support for Mistral Nemo

This is working in 0.12. Good job! Does anyone have any advice or documentation that can help to optimize engine builds for Mistral Nemo? I am currently experimenting with fp8...