Krishna Dubba comments

Repositories
Issues
Comments

Results 4 comments of


                                            Krishna Dubba

Can't run starchat: fails with `AttributeError: module 'global_vars' has no attribute 'gen_config'`

There might be some other error before this. Maybe the model could not be loaded because of memory limits. Could you check the error log?

The cached files aren't being removed after the stale duration is over!

Any updates here? I see the same problem when I use this with CachedNetworkImage. I suspect, the cache manager is not being used, so CachedNetworkImage never refreshes the existing cache.

LLama cpp problem ( gpu support)

The CPU is working fine for the latest version of the server, but the moment I offload layers to GPU, I get gibberish. Something is messing up in the GPU...

LLama cpp problem ( gpu support)

@glaudiston, thanks for the response. I was under the impression if I use "LLAMA_CUBLAS=1 pip install llama-cpp-python", it will take care of finding the .so library.