dify Error with Bedrock and Dify 1.4.0

Self Checks

[x] This is only for bug report, if you would like to ask a question, please head to Discussions.
[x] I have searched for existing issues search for existing issues, including closed ones.
[x] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
[x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
[x] Please do not modify this template :) and fill in all the required fields.

Dify version

1.4.0

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

👋 Hey team! I'm running Dify 1.4.0 self-hosted on EKS with the following setup:

dify-api: langgenius/dify-api:1.4.0

dify-web: langgenius/dify-web:1.4.0

dify-plugin-daemon: main-local

dify-sandbox: main

🔧 We've configured everything using CDK + Helm, with TLS enabled and a custom domain.

🧩 Problem with Amazon Bedrock plugin:

Whenever I try to configure the Amazon Bedrock model provider using the UI (leaving the credentials empty to use IAM role-based access from the pod), it throws an Internal Server Error.

Here's the POST request that's failing: POST https://knowledge-ai.widergydev.com/console/api/workspaces/current/model-providers/langgenius/bedrock/bedrock Status: 500 Internal Server Error

🪵 Logs from the API container: PluginInvokeError: {"args":{},"error_type":"StopIteration","message":""}

✅ What I tried:

Downgrading bedrock plugin versions (tried older ones from GitHub).

Tried different Bedrock models (Titan, Claude 3.5, etc.).

Confirmed model access in the AWS Bedrock console.

Gave the EKS node role full bedrock:* permissions and access to the specific model ARN.

💡 My hypothesis:

It seems like the plugin daemon is returning an unexpected empty response and raising StopIteration. Maybe the plugin is not properly handling the validate_provider_credentials() call when no explicit key/secret is provided and it relies on IAM role?

📎 Extra context:

No proxy.

Dify runs on EKS, ALB Ingress Controller with TLS.

Access to Bedrock is confirmed in AWS UI.

Any ideas or is this a known issue with the plugin backend?

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

May 19 '25 18:05 sanacorea

Hey @sanacorea! 👋 I'm Dosu, your friendly bot here to help you tackle bugs, answer questions, and become a contributor while we wait for a human maintainer. Let's get this sorted out together!

The issue you're encountering with the Amazon Bedrock plugin in Dify 1.4.0 seems related to how the validate_provider_credentials() function handles IAM role-based access when no explicit key/secret is provided. The function is expected to validate credentials using the validate_credentials method of the model instance, and if validation fails, it raises a CredentialsValidateFailedError or logs an exception and raises it ^[1]^[2].

Given your setup, it appears that the plugin might not be correctly handling IAM role-based access, leading to the StopIteration error. This could be due to the function expecting credentials in the credentials dictionary parameter, which are not provided when relying on IAM roles.

To address this issue, you might want to:

Verify if there are any specific configuration settings or environment variables required for the Amazon Bedrock plugin to function correctly in a self-hosted setup on EKS. Unfortunately, I couldn't find specific details on this ^[1]^[2].
Check if there are any updates or patches available for the plugin that address IAM role-based access issues.
Consider reaching out to the maintainers for further insights or potential fixes, as this might be a known issue with the plugin backend.

These steps might help in resolving the error you're facing.

To continue talking to Dosu, mention @dosu.

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

May 19 '25 18:05 dosubot[bot]

hI @dosu Just to add more context:

✅ The pod is running on EKS. We tried two configurations:

No credentials provided — relying on IAM role-based access via the node's instance profile.

Access Key + Secret Key explicitly provided — with full permissions to Bedrock (bedrock:*).

In both cases, when trying to add the Bedrock provider, we get this error: PluginInvokeError: {"args":{},"error_type":"StopIteration","message":""}

We also confirmed that:

The account has access to the model (e.g. amazon.titan-embed-text-v2:0) — it works via the AWS console.

The plugin appears to start fine (main-local build of dify-plugin-daemon).

All services are running inside Kubernetes and reachable.

From the traceback, it seems like the plugin backend is calling validate_provider_credentials(), and when it doesn’t get what it expects (either credentials or something else), it crashes internally with StopIteration, without returning a proper error message.

Is it possible that the plugin doesn’t support environment-based credential resolution (IAM role or EC2 metadata) yet?

Or maybe the Bedrock plugin expects credentials to always be present in the payload, even if they’re not needed in an EKS setup?

We’re happy to test any fix or debug further if needed. Let me know if there’s a workaround or if this is something that could be patched in the plugin backend.

Thanks again for the help!

May 19 '25 19:05 sanacorea

Can you take a look at this issue? @ybalbert001

May 20 '25 01:05 crazywoola

Which helm repo are you using, I can try to reproduce it, and we'll run the cluster on eks as well

May 26 '25 10:05 reckless-huang

I encountered this issue too with Dify 1.2

May 29 '25 06:05 ytnobody

Hi, @sanacorea. I'm Dosu, and I'm helping the Dify team manage their backlog and am marking this issue as stale.

Issue Summary:

You reported a 500 Internal Server Error caused by a StopIteration exception when using the Amazon Bedrock model provider with IAM role-based access (no explicit credentials) in Dify 1.4.0 self-hosted on EKS.
The error originates from the plugin daemon's validate_provider_credentials() function, which does not handle empty credentials gracefully.
You confirmed that using explicit keys avoids the error, indicating the plugin may not support environment-based credential resolution yet.
Maintainers and community members have been notified and are investigating the issue, but no resolution has been posted yet.

Next Steps:

Please confirm if this issue is still relevant with the latest version of Dify; if so, you can keep the discussion open by commenting here.
Otherwise, this issue will be automatically closed in 15 days.

Thank you for your understanding and contribution!

Sep 01 '25 16:09 dosubot[bot]