guidance icon indicating copy to clipboard operation
guidance copied to clipboard

Confusing error message when using capture on stateful function

Open nking-1 opened this issue 1 year ago • 0 comments

The bug When using capture with a stateful function, the error message is not very user-friendly.

To Reproduce Using the stateful function demonstrated in the readme:

from guidance import capture, select

@guidance(stateless=False)
def test(lm):
    lm += 'Should I say "Scott"?\n' + select(['yes', 'no'], name='answer') + '\n'
    if lm['answer'] == 'yes':
        lm += 'Scott'
    else:
        lm += 'Not Scott'
    return lm

lm + capture(test(), "scott")

The user gets this stack trace in return:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[10], line 1
----> 1 lm + capture(test(), "scott")

File ~/code/laser-rdk/guidance-evaluation/guidance/guidance/models/_model.py:1169, in Model.__add__(self, value)
   1167 # run stateless functions (grammar nodes)
   1168 elif isinstance(value, GrammarFunction):
-> 1169     out = lm._run_stateless(value)
   1171 # run stateful functions
   1172 else:
   1173     out = value(lm)

File ~/code/laser-rdk/guidance-evaluation/guidance/guidance/models/_model.py:1359, in Model._run_stateless(self, stateless_function, temperature, top_p, n)
   1352 logger.debug("start Model._run_stateless")
   1354 # This needs to be here for streaming
   1355 # if name is not None:
   1356 #     self[name] = ""
   1357 
   1358 # replace ModelVariables with their actual values (note we save what we replaced so we can restore it later)
-> 1359 replacements = replace_model_variables(stateless_function, self)
   1361 # start the generation stream
   1362 gen_obj = self.engine(self._current_prompt(), stateless_function)

File ~/code/laser-rdk/guidance-evaluation/guidance/guidance/_grammar.py:544, in replace_model_variables(grammar, model, allowed_vars)
    541     continue
    543 # Process non-terminal nodes in reverse order to maintain the depth-first order
--> 544 for i in reversed(range(len(current.values))):
    545     value = current.values[i]
    546     if isinstance(value, ModelVariable):

AttributeError: 'RawFunction' object has no attribute 'values'

It's not clear without internal library knowledge that this error happened because the function was not stateless.

System info (please complete the following information):

  • OS (e.g. Ubuntu, Windows 11, Mac OS, etc.): Ubuntu
  • Guidance Version (guidance.__version__): 0.1.13

nking-1 avatar May 10 '24 21:05 nking-1