su77ungr

Did you try enabling mlock? Does the source install work for you? Hopefully we get you running again soon. Thanks for your patience.

Sorry for the inconvenience. Edit: windows 10 testing now I'm spinning up some VMs in azure now. - [x] Linux (ubuntu 22.04) - [ ] Windows 10 testing - [...

Sure, we are planning to offer a package that works plug-in-play with every python ready machine too. Let's fix this first, run tests and I'll hit you up as soon...

I would recommend sticking with llamacpp. The ReadMe states a model that's over six months in age and was just a test model. The quality of models improved incredible since...

lacking understanding here but this should be seen as a llamacpp port competitor ig... the repo was created a few hours ago so there's still missing a huge chunk. also...

streaming added now https://github.com/marella/ctransformers/releases/tag/v0.1.2

[Outsourced](https://huggingface.co/dnato/ggjt-ready-models/blob/main/README.md) curated list of supported models; later adding to README.md

Maye create setup.py that fetches directly from HF Edit: this does counteract the air-gapped idea ```python from huggingface_hub import hf_hub_download #Download the model hf_hub_download(repo_id="LLukas22/gpt4all-lora-quantized-ggjt", filename="ggjt-model.bin", local_dir=".") ``` Edit: implemented with...

You got it running? We should add benchmark runs so everyone can plot and share results.

Gonna take on this issue tomorrow. Did you test the release v0.1.0 on windows? Edit: not the issue > 1. is it possible to integrate your solution on AWS SageMaker?...