nazthelizard122

Results 5 comments of nazthelizard122

Scale buf_size linearly with n_ctx

ask chatGPT for a solution lol

Should use `mmap` for model loading

> All tensors and model parameters are "emplaced" into this buffer. There are no extra allocations ocuring. In other words, there is a set memory cap? Like many other people...

GGML_ASSERT: ggml.c:4014: false zsh: abort ./main -m ./models/65B/ggml-model-q4_0.bin -t 16 -n 256 --repeat_penalty 1.0

Idk how to use it

GGML_ASSERT: ggml.c:4014: false zsh: abort ./main -m ./models/65B/ggml-model-q4_0.bin -t 16 -n 256 --repeat_penalty 1.0

idk how to get the check mark

GGML_ASSERT: ggml.c:4014: false zsh: abort ./main -m ./models/65B/ggml-model-q4_0.bin -t 16 -n 256 --repeat_penalty 1.0

now I'm running into another error, it segfaults with the regular script