Ioannis Karkanias

I encountered the same error, with a similar setup, where I used Chroma and RetrievalQAWithSourcesChain. When using RetrievalQA.from_chain_type and the .run(query) method instead, I can successfully get results. This works:...

I've experimented with the `load_in_8bit` argument in `GPT2LMHeadModel.from_pretrained` in`model.py`'s `from_pretrained()` classmethod, but since I'm using mainly Windows and have an old GTX 1060 locally, I wasn't able to go far....

Hi! Thanks for the updates. Just to restore my sanity, self-hosting without ./setup.sh will require more than just git checkout the main branch and change an nginx env var, right?...

Thank you @ArgoHA for your detailed response. I find that I accept your statements to a satisfactory extent!

Hi @Nuzhny007, could you please share your code or method of comparison - what tools you used to export your models, for example?