Juyoung Suk

icon indicating an email [email protected]

icon company @kaistAI icon location Seoul icon location M.S. student at LK Lab @kaistAI

Results 1 issues of Juyoung Suk

Feature Request: Release quantized version of Prometheus models

3 comment

- Support quantization to enable working with Promethues 2 in google colab - Good reference: [AutoGPTQ](https://github.com/AutoGPTQ/AutoGPTQ), [AutoGPTQ Notebook](https://colab.research.google.com/drive/1lSvVDaRgqQp_mWK_jC9gydz6_-y6Aq4A?usp=sharing), [llama.cpp](https://github.com/ggerganov/llama.cpp) - Release the quantized model in Huggingface and Ollama

enhancement