scavru

Loading big models 4bit

3

### Describe the bug As I understand it, the model is loaded through the cpu. and not directly into the GPU. This makes it impossible to load a large model...

You can add the display of all pictures with a scroll instead of 24 as now.

need to enable the **8-bit cache**, and the number of contexts can be expanded and the context window itself holds 4 digits, the rest is cut off. Thank you for...

`PROMPT @ 2023-08-13 01:21:50 | test ERROR | __main__:generate:3893 - Traceback (most recent call last): File "aiserver.py", line 3880, in generate genout, already_generated = tpool.execute(model.core_generate, txt, found_entries, gen_mode=gen_mode) File "C:\KoboldAI\miniconda3\lib\site-packages\eventlet\tpool.py",...

How do disable summarizer for picture generation? Is it possible to send a raw generated text?

Hello, consider the offer. Add support for LibreTranslate https://github.com/LibreTranslate/LibreTranslate would be great to not be dependent on the internet services to increase privacy. Thank you for your project.

docker

2

Good afternoon, can you tell me where in the prebuild docker container is the tgreddit folder, where to put config. To make it work, I manually wrote the path to...

few wishes

1

Thanks for your project. I have a few wishes: the most important thing is that the models cannot translate more than one sentence (after the dot it does not translate...

Запрос

1

Привет я выскажу свои несвязные мыли и пожелания, я понимаю что это твое время и я ничего не могу требовать, это просто мои хотелки никчему тебя не обязывающие! В связи...

Suggestion

1

Hi, your project is interesting, I found a similar project where many features are implemented. Maybe it will help you. https://github.com/gkvoelkl/python-llm-ratisbona-adventure. There is also a possibility to connect to open...