microsoft
[Bug]: GPTAssistantAgent does not invoke assistant API when called `generate_reply`
Describe the bug
Calling generate_reply without a sender on a GPTAssistantAgent won't invoke Assistant API -- it uses the ChatCompletion API.
Steps to reproduce
import os
from autogen.agentchat.contrib.gpt_assistant_agent import GPTAssistantAgent
from dotenv import load_dotenv
load_dotenv()
assistant_id = os.environ.get("ASSISTANT_ID", None)
llm_config = {
"config_list": [
{
"model": "gpt-4-turbo",
"api_key": os.environ.get("OPENAI_API_KEY", None),
}
],
}
assistant_config = {
"tools": [
{"type": "code_interpreter"},
]
}
code_oai_agent = GPTAssistantAgent(
name="code_oai_agent",
instructions="You are a code assistant tool that can run code in the code interpreter",
llm_config=llm_config,
assistant_config=assistant_config,
)
print(
code_oai_agent.generate_reply(
messages=[
{
"content": "\
Use code interpreter to run a Monty Hall problem simulation 10000 times, and \
tell me how often you win by switching doors.",
"role": "user"
}
]
)
)
Output:
OpenAI client config of GPTAssistantAgent(code_oai_agent) - model: gpt-4-turbo
No matching assistant found, creating a new assistant
Sure, I will simulate the Monty Hall problem 10,000 times and check how often switching doors leads to a win.
In the Monty Hall problem, you choose one of three doors. Behind one door is a car (which you want to win) and behind the other two doors are goats. After selecting a door, the host, who knows what's behind each door, opens another door revealing a goat. You then have the choice to either stick with your initial selection or switch to the other unopened door. The strategy we'll analyze here is always switching.
Let's run the simulation:
```python
import random
def monty_hall_sim(num_trials, switch):
win_count = 0
for _ in range(num_trials):
# There are three doors, one door has the car (win), two have goats (lose)
doors = ['goat', 'goat', 'car']
random.shuffle(doors)
# The player picks a random door
chosen_door = random.randint(0, 2)
# Monty eliminates one of the doors containing a goat
# Monty cannot open the door chosen by the player if it contains the car
remaining_doors = [i for i in range(3) if i != chosen_door and doors[i] == 'goat']
monty_opens = random.choice(remaining_doors)
# Decide whether to switch based on the argument passed
if switch:
# The player switches to the remaining unopened door
possible_doors = [i for i in range(3) if i != chosen_door and i != monty_opens]
chosen_door = possible_doors[0]
# Check if player has won
if doors[chosen_door] == 'car':
win_count += 1
return win_count
# Simulate 10,000 times and always switch
num_trials = 10000
switch = True
wins_by_switching = monty_hall_sim(num_trials, switch)
# Calculate percentage win rate
win_percentage = (wins_by_switching / num_trials) * 100
win_percentage
```
This code will simulate the Monty Hall problem 10,000 times, with the assumption that the player always switches the door after Monty opens a door revealing a goat. It then calculates the percentage of times the player wins the car when using this strategy. Let's execute this simulation.
Expected Behavior
The assistant should use the code interpreter and generate a response using the result of the simulation.
I believe the fix should be just to change this line:
https://github.com/microsoft/autogen/blob/2749e4386f374410051d73efe6c54b01786e784f/autogen/agentchat/contrib/gpt_assistant_agent.py#L175
change Agent to [Agent, None].
@Hk669 @krishnashed I think a separate PR is more suitable. #2677 has some issues and I think will take some time to merge.
@ekzhu we are facing same issues in the new PR as well, https://github.com/microsoft/autogen/pull/2751
The checks still fail
