Agent unable to read response from custom workflow tools (Anthropic through OpenRouter Amazon Bedrock)

[infinitnet]

Self Checks

• [x] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [x] I have searched for existing issues search for existing issues, including closed ones.
• [x] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• [x] Please do not modify this template :) and fill in all the required fields.

Dify version

15.3

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Create a workflow tool that outputs data. Add workflow as tool to an agent and use Sonnet 3.5 through OpenRouter and verify in the OpenRouter activity log that it's using Amazon Bedrock (works fine if routed through Anthropic!). Include instructions to repeat the tool's output verbatim if called. Call the tool by starting a chat with the agent.

✔️ Expected Behavior

Agent should be able to read and "see" the response from the custom workflow tool and be able to repeat it.

❌ Actual Behavior

The agent is unable to read or process the output although the workflow does output it correctly. There seems to be an issue with the "interation" that's reading and repeating the output of the tool if OpenRouter routes requests to Amazon Bedrock. Works fine if Anthropic is used through OpenRouter or directly.

[dosubot[bot]]

I found a similar unsolved discussion titled "Help：Agent NEVER get any result from workflow tool" which might be related to the issue you're experiencing ^[1].

Regarding your issue, it seems that the agent is unable to process the output from custom workflow tools in Dify version 15.3, even though it worked in version 15.2. This could be due to changes in how Dify handles the output of custom workflow tools. Dify supports multiple message types, including text, links, images, file BLOBs, and JSON ^[2].

To troubleshoot, you might want to ensure that the output from your custom workflow tool is formatted correctly according to these supported types. Additionally, you can utilize Dify's debugging and logging features to trace the interaction between agents and workflow tools. The DifyAgentCallbackHandler class provides methods for logging tool start and end, as well as any errors encountered ^[3].

If the issue persists, it might be beneficial to compare the configurations and outputs between versions 15.2 and 15.3 to identify any discrepancies.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[infinitnet]

Not reproducible - closing bug report. I've just created a new agent, added the same workflow tool and it works fine. This must be related to a damaged agent or part of the agent prompt instructions or some specific setting. Will provide another update and potentially reopen if I'm able to identify the cause.

[infinitnet]

Reopening because although it's not easy to reproduce this, I do keep running into it.

I've created two new agents using the same prompt and the same workflow tool attached. Both using Sonnet 3.5 through OpenRouter. One agent works, while the other one doesn't. As soon as I switch the non working one from OpenRouter to Anthropic, it always works. When I then switch it back to OpenRouter, it fails again but sometimes works.

So it seems that this is specific to OpenRouter, likely depending on through which API provider it routes the requests at the time (might be Amazon Bedrock when the bug gets triggered) and my assumption is that this might be somehow related to #12460

[infinitnet]

Found it! The bug appears only when Sonnet 3.5 is used through OpenRouter AND it routes them to Amazon Bedrock. So depending on current latency/load, OpenRouter would sometimes route to Amazon Bedrock and sometimes to Anthropic or other providers. This is why the bug was so hard to reproduce. Updated title and initial description accordingly.

My current workaround is to add anthropic/claude-3.5-sonnet:beta as custom OpenRouter model and use this instead of anthropic/claude-3.5-sonnet because anthropic/claude-3.5-sonnet:beta always routes to Anthropic and never to Amazon Bedrock. This bypasses the bug.

[dosubot[bot]]

Hi, @infinitnet. I'm Dosu, and I'm helping the Dify team manage their backlog. I'm marking this issue as stale.

Issue Summary:

• The issue involves a bug with an agent failing to process responses when routed through OpenRouter to Amazon Bedrock.
• The problem is specific to Sonnet 3.5 and does not occur when routed through Anthropic.
• You have implemented a workaround by using a custom OpenRouter model to consistently route to Anthropic.
• The issue may be related to another open issue, #12460.

Next Steps:

• Please let us know if this issue is still relevant to the latest version of the Dify repository by commenting on this issue.
• If there is no further activity, this issue will be automatically closed in 15 days.

Thank you for your understanding and contribution!