adk-docs icon indicating copy to clipboard operation
adk-docs copied to clipboard

Agent working in local but not when deployed to Cloud Run, with error "AttributeError: 'coroutine' object has no attribute 'status'"

Open webravolab opened this issue 8 months ago • 3 comments

** Please make sure you read the contribution guide and file the issues in the right place. ** Contribution guide.

Describe the bug I am trying to deploy an agent to Cloud Run, it works fine locally using adk web to test it, but when deployed to Cloud Run (using adk deploy cloud run) it fails with error at first interaction (saying "Hi" or something in the web chat interface).

The agent must connect to a MCP server (using toolbox_core) already deployed and running on the same Cloud Run, with or without authentication. Local instance could connect fine to the MCP server and working fine with it.

The error stack is the following: INFO: 169.254.169.126:41924 - "POST /run_sse HTTP/1.1" 200 OK Error in event_generator: 'coroutine' object has no attribute 'status' (most recent call last): File "/home/myuser/.local/lib/python3.12/site-packages/google/adk/cli/fast_api.py", line 805, in event_generator runner = await _get_runner_async(req.app_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/myuser/.local/lib/python3.12/site-packages/google/adk/cli/fast_api.py", line 978, in _get_runner_async root_agent = await _get_root_agent_async(app_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/myuser/.local/lib/python3.12/site-packages/google/adk/cli/fast_api.py", line 963, in _get_root_agent_async agent_module = importlib.import_module(app_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 935, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 999, in exec_module File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "/app/webravo-agent1/__init__.py", line 1, in <module> from . import agent File "/app/webravo-agent1/agent.py", line 32, in <module> agent_toolset = toolbox_client.load_toolset("merchants-toolset") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/myuser/.local/lib/python3.12/site-packages/toolbox_core/sync_client.py", line 154, in load_toolset async_tools = asyncio.run_coroutine_threadsafe(coro, self.__loop).result() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 456, in result return self.__get_result() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result raise self._exception File "/home/myuser/.local/lib/python3.12/site-packages/toolbox_core/client.py", line 264, in load_toolset header_name: await resolve_value(original_headers[header_name]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/myuser/.local/lib/python3.12/site-packages/toolbox_core/utils.py", line 143, in resolve_value return await source() ^^^^^^^^^^^^^^ File "/home/myuser/.local/lib/python3.12/site-packages/toolbox_core/auth_methods.py", line 50, in aget_google_id_token await creds.refresh(_aiohttp_requests.Request()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/myuser/.local/lib/python3.12/site-packages/google/auth/compute_engine/credentials.py", line 126, in refresh self._retrieve_info(request) File "/home/myuser/.local/lib/python3.12/site-packages/google/auth/compute_engine/credentials.py", line 99, in _retrieve_info info = _metadata.get_service_account_info( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/myuser/.local/lib/python3.12/site-packages/google/auth/compute_engine/_metadata.py", line 342, in get_service_account_info return get(request, path, params={"recursive": "true"}) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/myuser/.local/lib/python3.12/site-packages/google/auth/compute_engine/_metadata.py", line 212, in get if response.status in transport.DEFAULT_RETRYABLE_STATUS_CODES: ^^^^^^^^^^^^^^^ AttributeError: 'coroutine' object has no attribute 'status' /home/myuser/.local/lib/python3.12/site-packages/google/adk/cli/fast_api.py:819: RuntimeWarning: coroutine 'Request.__call__' was never awaited yield f'data: {{"error": "{str(e)}"}}\n\n' RuntimeWarning: Enable tracemalloc to get the object allocation traceback Unclosed client session client_session: <aiohttp.client.ClientSession object at 0x3e019d8deff0> Expected behavior Expected to run as in local environment

Desktop (please complete the following information):

  • OS: Google Cloud Run
  • Python version(python -V): 3.12
  • ADK version(pip show google-adk): 1.0.0

Additional context I've already tried:

  • Try "cloud run deploy" instead of "adk deploy cloud run" with custom Dockerfile - same error
  • Try deploying in another region (both europe-west2 and us-central1, the same of the gemini model) - same error
  • Try to allow unauthenticated access to both agent and mcp server cloud run services - same error

My Agent code (names offuscated): ` from google.adk.agents import Agent from google.adk.runners import Runner from google.adk.sessions import InMemorySessionService from google.adk.artifacts.in_memory_artifact_service import InMemoryArtifactService from toolbox_core import ToolboxSyncClient, auth_methods import os import datetime

os.environ['GOOGLE_GENAI_USE_VERTEXAI'] = 'True' os.environ['GOOGLE_CLOUD_PROJECT'] = 'my-project' os.environ['GOOGLE_CLOUD_LOCATION'] = 'us-central1'

auth_token_provider = auth_methods.aget_google_id_token # can also use sync method toolbox_client = ToolboxSyncClient( "https://my-toolbox-xxxxxxxxxx.us-central1.run.app", client_headers={"Authorization": auth_token_provider}, )

agent_toolset = toolbox_client.load_toolset("my-toolset")

prompt = """ You're a helpful assistant. """

root_agent = Agent( model='gemini-2.5-flash-preview-05-20', name='my_test_agent', description='A helpful AI assistant', instruction=prompt, tools=agent_toolset, ) `

webravolab avatar May 23 '25 14:05 webravolab

Addendum:

I have partially solved the issue: It seems a problem related to service-to-service authorization.

I've tried to enable unauthorized access to the MCP Toolbox server cloud run service and remove the auth_token_provider part in the agent when creating the toolbox_client, as follows:

toolbox_client = ToolboxSyncClient( "https://my-toolbox-xxxxxxxxxx.us-central1.run.app/" )

Now the agent deploy and works fine. Now I need some little help to understand how to handle the service-to-service authentication in Cloud Run to let the Agent to connect to the MCP toolbox server using IAM or other method.

I have to say the the error message is not clear and unrelated to the real issue. Thank you.

webravolab avatar May 23 '25 14:05 webravolab

IIUC, the service account running the agent should be granted access to the service account of running the toolbox.


@seanzhou1023 , I think toolbox needs better documentation about how to deploy.

Jacksunwei avatar May 26 '25 23:05 Jacksunwei

toolbox document can be found here : https://github.com/googleapis/genai-toolbox

seanzhou1023 avatar May 27 '25 22:05 seanzhou1023