semantic-kernel icon indicating copy to clipboard operation
semantic-kernel copied to clipboard

Python: Bug: Python: Potential Bug in FunctionChoiceBehavior.Required() and FunctionChoiceBehavior.Auto()

Open nmoeller opened this issue 1 year ago • 1 comments

Describe the bug So i was playing around with FunctionChoiceBehavior.Required() in a Project and noticed some strange behavior.

  1. When i use kernel.invoke() with FunctionChoiceBehavior.Required(autoinvoke=true) the first time the function is executed, i recieve an FunctionCallContent Object back instead of FunctionResultContent/TextContent Object.
  2. When i use kernel.invoke() with FunctionChoiceBehavior.Required(autoinvoke=true) the second time the function is not executed, i recieve an FunctionCallContent Object back instead of FunctionResultContent/TextContent Object.

I think it has to do with the following line , this will set the auto_invoke_kernel_functions to False after the first execution.

When you do another request then, it will hit the following if statement , this results in the Function is not Executed.

This could potentially also impact FunctionChoiceBehavior.Auto() when 5 Functions are called in the Loop. Then the property is set to false and no more Functions would be executed by the Kernel.

Probably me as a user can get around this when i reset the settings after kernel.invoke(), but i am not sure if this is intended.

To Reproduce Steps to reproduce the behavior:

  1. Go to samples/concepts/function_calling_with_required_type.py
  2. Change Line 55 to stream = False
  3. Use a Debugger or add the Following Lines at 173
        print(type(result.value[0].items[0]))
        print([type(message.items[0]) for message in result.metadata['messages'].messages])
  1. Run the samples/concepts/function_calling_with_required_type.py 2 Times

Expected behavior

  1. The first time running with FunctionChoiceBehavior.Required(autoinvoke=true) i would expect a FunctionResultContent Object or a TextContent Object answer based on the Function Result
  2. The second time running with FunctionChoiceBehavior.Required(autoinvoke=true) i would expect a FunctionResultContent Object or a TextContent Object with the answer based on the Function Result

Screenshots image

Additional context Add any other context about the problem here.

nmoeller avatar Jul 25 '24 08:07 nmoeller

Hi @nmoeller, the behavior you're introducing in #7448 isn't what we'd like to support. The Required FunctionChoiceBehavior forces the model to only perform a function call (it's essentially requiring the caller to handle a manual tool call). We shouldn't be performing a final chat completion while modifying the underlying FunctionChoiceBehavior settings once the max auto invoke attempts are hit.

moonbox3 avatar Jul 25 '24 12:07 moonbox3

This issue is stale because it has been open for 90 days with no activity.

github-actions[bot] avatar Nov 09 '24 01:11 github-actions[bot]

Closing as we're tracking an AzureChatCompletion FunctionChoiceBehavior.Required() bug in a separate issue.

moonbox3 avatar Nov 09 '24 14:11 moonbox3