guidance
guidance copied to clipboard
Confusing error message when using capture on stateful function
The bug
When using capture with a stateful function, the error message is not very user-friendly.
To Reproduce Using the stateful function demonstrated in the readme:
from guidance import capture, select
@guidance(stateless=False)
def test(lm):
lm += 'Should I say "Scott"?\n' + select(['yes', 'no'], name='answer') + '\n'
if lm['answer'] == 'yes':
lm += 'Scott'
else:
lm += 'Not Scott'
return lm
lm + capture(test(), "scott")
The user gets this stack trace in return:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Cell In[10], line 1
----> 1 lm + capture(test(), "scott")
File ~/code/laser-rdk/guidance-evaluation/guidance/guidance/models/_model.py:1169, in Model.__add__(self, value)
1167 # run stateless functions (grammar nodes)
1168 elif isinstance(value, GrammarFunction):
-> 1169 out = lm._run_stateless(value)
1171 # run stateful functions
1172 else:
1173 out = value(lm)
File ~/code/laser-rdk/guidance-evaluation/guidance/guidance/models/_model.py:1359, in Model._run_stateless(self, stateless_function, temperature, top_p, n)
1352 logger.debug("start Model._run_stateless")
1354 # This needs to be here for streaming
1355 # if name is not None:
1356 # self[name] = ""
1357
1358 # replace ModelVariables with their actual values (note we save what we replaced so we can restore it later)
-> 1359 replacements = replace_model_variables(stateless_function, self)
1361 # start the generation stream
1362 gen_obj = self.engine(self._current_prompt(), stateless_function)
File ~/code/laser-rdk/guidance-evaluation/guidance/guidance/_grammar.py:544, in replace_model_variables(grammar, model, allowed_vars)
541 continue
543 # Process non-terminal nodes in reverse order to maintain the depth-first order
--> 544 for i in reversed(range(len(current.values))):
545 value = current.values[i]
546 if isinstance(value, ModelVariable):
AttributeError: 'RawFunction' object has no attribute 'values'
It's not clear without internal library knowledge that this error happened because the function was not stateless.
System info (please complete the following information):
- OS (e.g. Ubuntu, Windows 11, Mac OS, etc.): Ubuntu
- Guidance Version (
guidance.__version__): 0.1.13