Built-in guidance for constrained generation and more predictable outputs

[vincentgiraud]

Guidance allows users to constrain generation (e.g. with regex and CFGs) as well as to interleave control (conditionals, loops, tool use) and generation for more predictable outputs.

It's been recently integrated to Phi-3.5-mini in AI Studio and is a powerful paradigm for latency and cost optimization that could also be used within Prompty for hardening and testing prompt controls.

Based on this example, the prompty file could have a guidance section:

guidance: ${file: guidance.json}
---
user: 
"What is the capital of Australia?"
"The capital of Australia is {{reg1}}"

and the guidance.json file to contain:

[{"reg1": "regex('[A-Z][a-z]+')"}]

[sethjuarez]

This is super interesting - will take a look. I do like the idea.