Token quantity limit error

[ocaicaio]

Self Checks

• [x] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [ ] I have searched for existing issues search for existing issues, including closed ones.
• [x] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• [x] Please do not modify this template :) and fill in all the required fields.

Dify version

1.0.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

This is the deepseek-r1 model communication in the local ollama that I call through the dify tool. There was no problem with the same configuration before, but now this error message is displayed. The token I set is 4096, which should be supported by deepseek-r1 32b (14b is the same). Of course, I also have a system prompts, which is about 1,000 words. The problem scenario is that I will get this error message even if I send "Hello". Is this a problem with deepseek or dify? Because I have the same setting on another cherry ai client and it is normal!

✔️ Expected Behavior

Set a system prompts, about 1000 words. Set the token max to 4096, and this error message will be reported for any question. Is this a problem with deepseek or dify? Because I have the same setting on another cherry ai client and it works fine!

❌ Actual Behavior

[dosubot[bot]]

Hello, @ocaicaio! I'm here to help you with bug reports, questions, and becoming a contributor. Let me know how I can assist you!

The issue you're encountering with the deepseek-r1 model in Dify might be related to the model's configuration or compatibility with Dify. Here are a few points to consider:

1. Token Limit Configuration: The deepseek-r1 model has a maximum generation length of 32,768 tokens, but the context length should be set to 4096 when using the OpenAI-API-compatible provider ^[1][2]. Ensure that your configuration aligns with these settings.

2. System Prompts: The deepseek-r1 model does not support system prompts. You should move any system prompts to the 'user' role instead ^[3][4].

3. Compatibility Issues: There are known compatibility issues with the deepseek-r1 model, such as the instability of the DeepSeek endpoint, which might cause null responses or errors in Dify ^[3].

4. Error Code 400: The error message you received indicates a PluginInvokeError due to an API request failure with status code 400. This might be related to the model not supporting certain features or tools as indicated in the error message.

5. Comparison with Other Clients: Since the same configuration works with another client (Cherry AI), it suggests that the issue might be specific to how Dify interacts with the model or the specific setup in your Dify environment.

To resolve the issue, ensure that your configuration matches the recommended settings, especially regarding token limits and system prompts. If the problem persists, consider checking for any updates or patches for Dify that might address compatibility issues with the deepseek-r1 model.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[ocaicaio]

I deployed locally, and the same settings were normal in the previous 0.15 version, but this error message appeared in the later 1.0 and 1.0.1 with the same configuration!

[dosubot[bot]]

Hi, @ocaicaio. I'm Dosu, and I'm helping the Dify team manage their backlog. I'm marking this issue as stale.

Issue Summary:

• Encountering a token quantity limit error with the deepseek-r1 model in a self-hosted Docker environment.
• The issue arises in versions 1.0 and 1.0.1, but not in version 0.15.
• Suggested to check model configuration, especially token limits and system prompts.
• The issue seems specific to Dify's interaction with the model, as it works with Cherry AI.

Next Steps:

• Please confirm if this issue is still relevant with the latest version of Dify. If so, you can keep the discussion open by commenting here.
• If there is no further activity, this issue will be automatically closed in 15 days.

Thank you for your understanding and contribution!