microsoft
Tried my best to run Magentic One on Ubuntu 22.04, but WebSurfer doesn't work
What happened?
I was following this link to run Magentic-One on Ubuntu 22.04 cloud server.
https://github.com/microsoft/autogen/tree/main/python/packages/autogen-magentic-one#environment-configuration-for-chat-completion-client
there are some missing steps need to be added by following this issue solution:
https://github.com/microsoft/autogen/issues/4079#issuecomment-2460821199
also looks like I need to run this to grant docker permission on ubuntu:
sudo usermod -aG docker $USER
or
sudo chmod 666 /var/run/docker.sock
after all these steps, everything looks good, but I saw one error when I run
(python) ubuntu@oracle-cloud-test:~/autogen/python/packages/autogen-magentic-one$ python examples/example.py --logs_dir ./my_logs
and ask to fetch apple stock price from yahoo finance, the error details are listed as below, please help to check why it happens, and what's the solution.
[2024-11-12T20:55:14.286065], Orchestrator (-> WebSurfer):
Click on the Yahoo Finance link that leads to Apple Inc. (AAPL) stock information and report the current stock price.
[2024-11-12T20:55:14.286287], Orchestrator (thought):
Next speaker WebSurfer [2024-11-12T20:55:19.585917], WebSurfer: click( {"reasoning": "To access the Apple stock page on Yahoo Finance, clicking the provided link will help retrieve detailed stock information.", "target_id": 89} )
[2024-11-12T20:55:19.695422], WebSurfer:
Web surfing error:
Traceback (most recent call last): File "/home/ubuntu/autogen/python/packages/autogen-magentic-one/src/autogen_magentic_one/agents/multimodal_web_surfer/multimodal_web_surfer.py", line 285, in _generate_reply request_halt, content = await self.__generate_reply(cancellation_token) File "/home/ubuntu/autogen/python/packages/autogen-magentic-one/src/autogen_magentic_one/agents/multimodal_web_surfer/multimodal_web_surfer.py", line 594, in __generate_reply return await self._execute_tool(message, rects, tool_names, cancellation_token=cancellation_token) File "/home/ubuntu/autogen/python/packages/autogen-magentic-one/src/autogen_magentic_one/agents/multimodal_web_surfer/multimodal_web_surfer.py", line 349, in _execute_tool await self._click_id(target_id) File "/home/ubuntu/autogen/python/packages/autogen-magentic-one/src/autogen_magentic_one/agents/multimodal_web_surfer/multimodal_web_surfer.py", line 716, in _click_id raise ValueError("No such element.") from None ValueError: No such element.
What did you expect to happen?
I'm expecting not to see this issue.
How can we reproduce it (as minimally and precisely as possible)?
described as above.
AutoGen version
i run in the main branch
Which package was this bug in
Magentic One
Model used
gpt-4
Python version
No response
Operating system
No response
Any additional info you think would be helpful for fixing this bug
No response
Thanks for the report. I think someone else mentioned a no-such element error (#4147).
Does it work for other web tasks? There are some possible race conditions I would like to rule out.
hi @afourney i think the error you are referring to is MultimodalWebSurfer is not defined. That PR didn't fix it.
In the new version of the websurfer on autogen-ext we have eliminated some of these errors with longer timeouts and sleeps, but they still occur randomly because of weird html behavior which might not be caused by the websurfer.
However, these errors don't stop execution, the orchestrator will ask the websurfer try again and most often it will work, have you observed this error every time you run this task or just this one time?
