dify icon indicating copy to clipboard operation
dify copied to clipboard

HTTP Request: Text size is too large, max size is 0.1MB

Open alexknows opened this issue 1 year ago • 3 comments

Self Checks

  • [X] I have searched for existing issues search for existing issues, including closed ones.
  • [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [X] Pleas do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing?

I need to be able to make an http request (using the HTTP Request node) and the JSON body I need to pass exceeds 0.1MB. Is there a way around this limitation? Why is the JSON body max size only 0.1MB?

Core feature: we are generating JSON objects and we need to POST this in the body to an external database.

image

2. Describe the feature you'd like to see

The http node should be able to work with JSON body that exceed 0.1 MB.

3. How will this feature improve your workflow or experience?

Most of the endpoints we need to call their JSON body exceeds 0.1 MB.

4. Additional context or comments

No response

5. Can you help us with this feature?

  • [ ] I am interested in contributing to this feature.

alexknows avatar May 03 '24 15:05 alexknows

We limit it to 0.1MB by default to avoid malicious resource consumption, as you know, all of the data will be stored into our database, and for now, there are about 30~50GB increment of storage in our SaaS service, too much space to be wasted is not our expectation, but you can break the limitation if you are self-deploying Dify, just refer to core.workflow.nodes.http_request.http_executor.py

Yeuoly avatar May 04 '24 13:05 Yeuoly

Thank you!

alexknows avatar May 06 '24 06:05 alexknows

Please expose the config for docker deploy instead of hard code in the source code. 😄

qianlifeng avatar May 06 '24 08:05 qianlifeng

it has been fixed #4137

Yeuoly avatar May 07 '24 07:05 Yeuoly

Good job.

qianlifeng avatar May 07 '24 07:05 qianlifeng

it has been fixed #4137 @Yeuoly

[google] Server Unavailable Error, 504 Deadline Exceeded

image

It seems there is an issue; please check. When I use the Model Gemini-1.5-pro and the data I input into the prompt system is large, and when I begin to test debug, I receive the error message shown in the picture.

chalitbkb avatar May 09 '24 17:05 chalitbkb

it has been fixed #4137已修复 #4137

Is the fix for this issue that the environment variable is not exposed to Docker? Does it only exist in api/.env?

zhuiyue132 avatar Sep 24 '24 07:09 zhuiyue132

Current version 0.8.3 (latest version), still exists [Text size is too large, max size is 1.00 MB, but current size is 1.69MB] problem. The HTTP_REQUEST_NODE_MAX_TEXT_SIZE environment variable does not seem to be exposed to Docker.

zhuiyue132 avatar Sep 24 '24 07:09 zhuiyue132