guidance icon indicating copy to clipboard operation
guidance copied to clipboard

[Draft] GPT-OSS Example Usage

Open hudson-ai opened this issue 6 months ago • 1 comments

Example notebook using guidance to add constraints to GPT-OSS.

GPT-OSS is rather idiosyncratic and currently doesn't fit incredibly well into some of guidance's existing API, so this notebook uses lower-level llguidance grammars alongside openai-harmony and sglang as an inference backend.

hudson-ai avatar Aug 22 '25 00:08 hudson-ai

This is very helpful. I would suggest adding some introductory prose about what the notebook covers (harmony format examples, tool use, lark grammar) as well as requirements (active GPT-OSS deployment). I thought it was pretty easy to follow along otherwise.

A few other ideas:

  • Add an example of running the model locally with llamacpp if possible.
  • Add a few sentences explaining that the ll_grammar() function exports lark format and that it's useful here because the OpenAI client directly supports sending it.

nking-1 avatar Aug 22 '25 19:08 nking-1