Introducing AutoGen Deploy

[ekzhu]

Why are these changes needed?

This is a work in progress. Comments and contributions welcome!

AutoGen's north star is to be THE multi-agent application development framework, so a robust and scalable serving solution is a must. Therefore, introducing AutoGen Deploy, a service framework to deploy your AutoGen agents as long-running, distributed workers handling messages on-demand. The agent workers talk to each other through a message broker.

Example usage:

First convert your AutoGen agents to Celery workers:

sample_service.py

assistant = AssistantAgent(
    name="assistant",
    llm_config={
        "config_list": config_list,
    },
)
user_proxy = UserProxyAgent(
    "user_proxy",
    code_execution_config={"work_dir": "coding", "use_docker": False},
    human_input_mode="NEVER",
)

app = Celery("autogen_deploy", backend="rpc://", broker="pyamqp://guest@localhost//")
assistant = CeleryAgent(app, assistant)
user_proxy = CeleryAgent(app, user_proxy)

To start a long-running service on localhost:

celery -A sample_service worker -l INFO

Query the service:

from sample_service import assistant, user_proxy

user_proxy.initiate_chat(
    assistant,
    message="What is the change YTD of the S&P 500?",
)

It is important to note that because currently AutoGen agents do not store state in shared external memory (i.e., a database) the client is storing all agent state, while workers are stateless. Once we have finished the state serialization stuff, we can make the agent workers stateful by synchronizing using shared external memory.

Also, because currently the human input handling is through console, it is not supported right now. So human_input_mode should be set to NEVER.

Related issue number

Checks

• [ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
• [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR.
• [ ] I've made sure all auto checks have passed.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (3680197) 31.92% compared to head (c10c90c) 31.80%. Report is 3 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1175      +/-   ##
==========================================
- Coverage   31.92%   31.80%   -0.12%     
==========================================
  Files          29       29              
  Lines        4097     4112      +15     
  Branches      955      961       +6     
==========================================
  Hits         1308     1308              
- Misses       2695     2710      +15     
  Partials       94       94              

Flag	Coverage Δ
unittests	31.76% <ø> (-0.12%)	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[victordibia]

@ekzhu , I really like the direction this is taking. I'll take a look and give comments later this week.

[randombet]

@ekzhu Could you clarify what AutoGen Deploy is trying to achieve? An application on terminal or a client-server architect design? What message broker is used here?

[ekzhu]

Close as it will be contained by future work.