Noah

custom api

9

is this how u want it? @psyray

CPU SIMD and pipeline optimizations across vec/mmq/ops/kv-cache/repack

1

## Summary I was really bored in some lectures last week so i scoured through the repo for some optimisable/improvable parts so this PR accelerates multiple hot paths in `ggml-cpu`...

displaying context and token stats in real time

4

## Add parsing progress display to webui Fixes #17079 Adds parsing progress during prompt tokenization, similar to token generation progress. **Changes:** - Display `Input: processed / total (percent%)` during prompt...

**Changes** Fixed 12 generator files: garak/generators/guardrails.py garak/generators/nvcf.py garak/generators/nemo.py garak/generators/langchain.py garak/generators/huggingface.py (3 classes: Pipeline, InferenceAPI, LLaVA) garak/generators/openai.py (2 classes: OpenAICompatible, OpenAIGenerator) garak/generators/ggml.py garak/generators/cohere.py garak/generators/litellm.py garak/generators/function.py garak/generators/rest.py (changed to pass uri parameter)...