James Olds comments

Results 7 comments of


                                            James Olds

Tags discussion

I would _love_ to have `#tag` support. It would be amazing to sync my notes to mobile and have tags supported, but all apps I've tried don't support reading tags...

Token generation is extremely slow when using 13B models on an M1 Pro with llama.cpp, but it runs at a fine speed with Dalai (which uses an older version of llama.cpp)

Can confirm I had this exact problem on an M1 Pro 16GB ram and rebooting fixed the issue 😄

Token generation is extremely slow when using 13B models on an M1 Pro with llama.cpp, but it runs at a fine speed with Dalai (which uses an older version of llama.cpp)

Aaaaand after loading chrome and doing some other stuff its now back to being extremely slow. Also running this exact setup on an M1 Max with 64GB ram and not...

Token generation is extremely slow when using 13B models on an M1 Pro with llama.cpp, but it runs at a fine speed with Dalai (which uses an older version of llama.cpp)

ah adding `--mlock` actually seems to fix the issue! at the cost of slightly longer initial load

How to access DAnswer through an API interface?

I was able to make a curl request based off of observing the api calls in the browser with dev tools. Get the token from the cookie in the request...

Vicuna (Fine-tuned LLaMa)

This should be possible using llama.cpp now that #2314 landed. I'm able to run vicuna locally with llama.cpp directly, but its hanging when using it with LangChain.

Feature: Support danswer releases through helm charts

I was able to get it to work following [this guide](https://medium.com/@Sushil_Kumar/readwritemany-persistent-volumes-in-google-kubernetes-engine-a0b93e203180) to set up an NFS server and changing the PVCs to `ReadWriteMany` ``` accessModes: - ReadWriteMany ```