Paul Irolla comments

Results 4 comments of


                                            Paul Irolla

warn("The installed version of bitsandbytes was compiled without GPU support. " - On Linux even tough I have a GPU

Hi, I had this problem, with installation on linux, for the version 58c8ab4c7aaccc50f507fd08cce941976affe5e0 (git checkout 58c8ab4c7aaccc50f507fd08cce941976affe5e0), that is working for the model vicuna-13B-1.1-GPTQ-4bit-128g. The solution was twofold: * first, I...

warn("The installed version of bitsandbytes was compiled without GPU support. " - On Linux even tough I have a GPU

No you don't need to remove it manually, just install on top

warn("The installed version of bitsandbytes was compiled without GPU support. " - On Linux even tough I have a GPU

I ended up using exllama as a ML model execution engine, as I was unable to make it work with latest updates of ooba. Exllama works right off the bat...

Feature suggestions

> Hello, thanks for all of the suggestions! > > We do already have a pretty flexible memory using a vector database called ChromaDB [...] Great ! For now ChromaDB...