James Ravenscroft

Hi all, I came across this thread and I was inspired by @ryancom16's solution to write a full plugin for FreshRSS which you can find here: https://github.com/ravenscroftj/freshrss-flaresolverr-extension I initially had...

There are a few open PRs for this behaviour - the most recent one being https://github.com/ollama/ollama/pull/3618 it would be amazing to get this merged in. It's a 2 line change...

Yes you can send the grammar as an option when you submit a request with the patch I linked to above enabled. It just isn't documented! Here's an example: POST...

On the roadmap. Can't say for sure when it'll be ready. pRs welcome!

oh that is awesome thanks for the tag @ggerganov - will definitely be looking at adding this as making suggestions much faster will make turbopilot much more usable!

Great question! I will have a look at this as it would be nice for people to be able to use the official plugin. I believe the error you are...

Ok so I was able to fix that problem so that the official autopilot plugin can talk to fauxpilot without causing an error. However, the latest version plugin does seem...

Thanks for your ticket. Looking at the prediction logs you screenshotted there it's taking about 2 minutes to generate a response on your system - I'm not 100% sure but...

Ok great thank you for your report - is it possible that k8s is trying to kill and restart the pod while prediction is happening and the health endpoint is...

I've now enabled multi-threaded serving in [v0.0.4](https://github.com/ravenscroftj/turbopilot/releases/tag/v0.0.4) so hopefully you won't find that the healthcheck endpoint becomes unresponsive while the system is working. I've also forked the fauxpilot plugin for...