guidance
guidance copied to clipboard
[Draft] GPT-OSS Example Usage
Example notebook using guidance to add constraints to GPT-OSS.
GPT-OSS is rather idiosyncratic and currently doesn't fit incredibly well into some of guidance's existing API, so this notebook uses lower-level llguidance grammars alongside openai-harmony and sglang as an inference backend.
This is very helpful. I would suggest adding some introductory prose about what the notebook covers (harmony format examples, tool use, lark grammar) as well as requirements (active GPT-OSS deployment). I thought it was pretty easy to follow along otherwise.
A few other ideas:
- Add an example of running the model locally with llamacpp if possible.
- Add a few sentences explaining that the
ll_grammar()function exports lark format and that it's useful here because the OpenAI client directly supports sending it.