Jonas

FP8 support in stable fast

6

Is it planned? Currently getting this error when trying to run ComfyUI in fp8 (flags `--fp8_e4m3fn-text-enc --fp8_e4m3fn-unet`): ``` RuntimeError: "addmm_cuda" not implemented for 'Float8_e4m3fn' ```

Prompt validation fails with partially working prompt, leading to stuck main thread

3

### Expected Behavior Prompt validation should fail and return 400 to client as usual ### Actual Behavior Seeing this stack trace and UI is not reacting anymore. Prompt worker is...

Tools support for OpenAI compatible API

1

## Checklist: - [x] I have read the [Contributing guidelines](https://github.com/oobabooga/text-generation-webui/wiki/Contributing-guidelines). This PR is for adding basic tool support, only for the OpenAI compatible API of textgen. I have followed the...

Model aware routing

8

Related issue and conversation: https://github.com/distantmagic/paddler/issues/60 Closes https://github.com/distantmagic/paddler/issues/60