dify Uploading files to the knowledge base can cause the Dify service to crash.

Self Checks

[x] This is only for bug report, if you would like to ask a question, please head to Discussions.
[x] I have searched for existing issues search for existing issues, including closed ones.
[x] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
[x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
[x] Please do not modify this template :) and fill in all the required fields.

Dify version

1.0.0

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

When I upload files to the knowledge base, the entire Dify service crashes, showing a "cannot allocate memory" error. The system can only be restored by rebooting the machine. The error is as follows:

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

Mar 10 '25 01:03 gongshaojie12

The "cannot allocate memory" error you're experiencing when uploading files to the knowledge base in a self-hosted Docker environment is likely due to insufficient memory resources allocated to Docker. Here are some steps you can take to address this issue:

Increase Memory Allocation: Ensure your Docker environment has at least 8GB of total memory and a minimum of 2 virtual CPUs. Running the worker service with less than 3GB of memory may cause server crashes ^[1].
Separate Worker Service: Consider deploying the worker service separately on another machine to distribute the load and prevent memory overload ^[1].
File Size and Batch Limits: Ensure that the files you are uploading do not exceed the limit of 15MB per document and a maximum of 20 files per batch upload. Exceeding these limits could contribute to memory allocation issues ^[2].
Optimize Docker Configuration: Check your Docker configuration to ensure it is optimized for resource management. This includes verifying Docker and Docker Compose versions (Docker 19.03 or later, Docker Compose 1.25.1 or later) ^[1].

By following these recommendations, you should be able to mitigate the memory allocation errors and prevent the Dify service from crashing during file uploads.

To continue talking to Dosu, mention @dosu.

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

Mar 10 '25 01:03 dosubot[bot]

The current machine has 64GB of memory, which should be sufficient.
With 64GB of memory, it seems unnecessary to separate the worker service.
Only one file of 6MB was uploaded.
The Docker version is: Docker version 26.1.4, build 5650f9b.

All four of your recommendations have been met, but the issue still persists. How should this problem be resolved?

Mar 10 '25 01:03 gongshaojie12

The specific error message is as follows:

[root@05htdsfssdiZfdhfn docker]# docker ps
runtime/cgo: pthread_create failed: Resource temporarily unavailable
SIGABRT: abort
PC=0x7f2f1a538387 m=0 sigcode=18446744073709551610

goroutine 0 [idle]:
runtime: g 0: unknown pc 0x7f2f1a538387
stack: frame={sp:0x7ffe337c7968, fp:0x0} stack=[0x7ffe32fc8e10,0x7ffe337c7e20)
0x00007ffe337c7868:  0x00007ffe337c7928  0x00005629248be45a <runtime.(*pageAlloc).update+0x000000000000039a>
0x00007ffe337c7878:  0x00007f2f1b040c00  0x00007ffe337c78c0
0x00007ffe337c7888:  0x000056292489710b <runtime.mmap.func1+0x000000000000004b>  0x0000000000000000
0x00007ffe337c7898:  0x00007ffe337c7958  0x00005629248be45a <runtime.(*pageAlloc).update+0x000000000000039a>
0x00007ffe337c78a8:  0x00007f2f1b040c00  0x00007f2f1afa0000
0x00007ffe337c78b8:  0x0000000000000000  0x00007ffe337c7980
0x00007ffe337c78c8:  0x00005629248be45a <runtime.(*pageAlloc).update+0x000000000000039a>  0x000740003a000000
0x00007ffe337c78d8:  0x0000000000000000  0x00007ffe337c79a0
0x00007ffe337c78e8:  0x0000000000000012  0x0000000000000003
0x00007ffe337c78f8:  0x0000000000000000  0x0000000000002030
0x00007ffe337c7908:  0x0000000000002031  0x0000000000000000
0x00007ffe337c7918:  0x0000000000000000  0x0000000000002031
0x00007ffe337c7928:  0x00005629248b7339 <runtime.(*scavengeIndex).alloc+0x0000000000000079>  0x0000000002030000
0x00007ffe337c7938:  0x00007f2f1a8ca868  0x00005629255c2201
0x00007ffe337c7948:  0x00005629281f2050  0x0000000000000000
0x00007ffe337c7958:  0x0000562926340d60  0x0000000000000006
0x00007ffe337c7968: <0x00007f2f1a539a78  0x0000000000000020
0x00007ffe337c7978:  0x0000000000000000  0x0000000000000000
0x00007ffe337c7988:  0x0000000000000000  0x0000000000000000
0x00007ffe337c7998:  0x0000000000000000  0x0000000000000000
0x00007ffe337c79a8:  0x0000000000000000  0x0000000000000000
0x00007ffe337c79b8:  0x0000000000000000  0x0000000000000000
0x00007ffe337c79c8:  0x0000000000000000  0x0000000000000000
0x00007ffe337c79d8:  0x0000000000000000  0x0000000000000000
0x00007ffe337c79e8:  0x0000000000000000  0x000000000000000d
0x00007ffe337c79f8:  0x00007f2f1a5f1bc9  0x0000000000000000
0x00007ffe337c7a08:  0x00007f2f1a57c2f3  0x000000000000000d
0x00007ffe337c7a18:  0x00007f2f1a8ca1c0  0x0000000000000001
0x00007ffe337c7a28:  0x00007f2f1a8ca243  0x00007f2f1a8c6380
0x00007ffe337c7a38:  0x00007f2f1a57db0e  0x00007f2f1a8ca1c0
0x00007ffe337c7a48:  0x000000000000000a  0x00005629281f2050
0x00007ffe337c7a58:  0x0000000000000000  0x0000562926340d60
runtime: g 0: unknown pc 0x7f2f1a538387
stack: frame={sp:0x7ffe337c7968, fp:0x0} stack=[0x7ffe32fc8e10,0x7ffe337c7e20)
0x00007ffe337c7868:  0x00007ffe337c7928  0x00005629248be45a <runtime.(*pageAlloc).update+0x000000000000039a>
0x00007ffe337c7878:  0x00007f2f1b040c00  0x00007ffe337c78c0
0x00007ffe337c7888:  0x000056292489710b <runtime.mmap.func1+0x000000000000004b>  0x0000000000000000
0x00007ffe337c7898:  0x00007ffe337c7958  0x00005629248be45a <runtime.(*pageAlloc).update+0x000000000000039a>
0x00007ffe337c78a8:  0x00007f2f1b040c00  0x00007f2f1afa0000
0x00007ffe337c78b8:  0x0000000000000000  0x00007ffe337c7980
0x00007ffe337c78c8:  0x00005629248be45a <runtime.(*pageAlloc).update+0x000000000000039a>  0x000740003a000000
0x00007ffe337c78d8:  0x0000000000000000  0x00007ffe337c79a0
0x00007ffe337c78e8:  0x0000000000000012  0x0000000000000003
0x00007ffe337c78f8:  0x0000000000000000  0x0000000000002030
0x00007ffe337c7908:  0x0000000000002031  0x0000000000000000
0x00007ffe337c7918:  0x0000000000000000  0x0000000000002031
0x00007ffe337c7928:  0x00005629248b7339 <runtime.(*scavengeIndex).alloc+0x0000000000000079>  0x0000000002030000
0x00007ffe337c7938:  0x00007f2f1a8ca868  0x00005629255c2201
0x00007ffe337c7948:  0x00005629281f2050  0x0000000000000000
0x00007ffe337c7958:  0x0000562926340d60  0x0000000000000006
0x00007ffe337c7968: <0x00007f2f1a539a78  0x0000000000000020
0x00007ffe337c7978:  0x0000000000000000  0x0000000000000000
0x00007ffe337c7988:  0x0000000000000000  0x0000000000000000
0x00007ffe337c7998:  0x0000000000000000  0x0000000000000000
0x00007ffe337c79a8:  0x0000000000000000  0x0000000000000000
0x00007ffe337c79b8:  0x0000000000000000  0x0000000000000000
0x00007ffe337c79c8:  0x0000000000000000  0x0000000000000000
0x00007ffe337c79d8:  0x0000000000000000  0x0000000000000000
0x00007ffe337c79e8:  0x0000000000000000  0x000000000000000d
0x00007ffe337c79f8:  0x00007f2f1a5f1bc9  0x0000000000000000
0x00007ffe337c7a08:  0x00007f2f1a57c2f3  0x000000000000000d
0x00007ffe337c7a18:  0x00007f2f1a8ca1c0  0x0000000000000001
0x00007ffe337c7a28:  0x00007f2f1a8ca243  0x00007f2f1a8c6380
0x00007ffe337c7a38:  0x00007f2f1a57db0e  0x00007f2f1a8ca1c0
0x00007ffe337c7a48:  0x000000000000000a  0x00005629281f2050
0x00007ffe337c7a58:  0x0000000000000000  0x0000562926340d60

goroutine 1 [running]:
runtime.systemstack_switch()
        /usr/local/go/src/runtime/asm_amd64.s:474 +0x8 fp=0xc000072740 sp=0xc000072730 pc=0x5629248fbb68
runtime.main()
        /usr/local/go/src/runtime/proc.go:169 +0x6d fp=0xc0000727e0 sp=0xc000072740 pc=0x5629248cce0d
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000727e8 sp=0xc0000727e0 pc=0x5629248fdb61

rax    0x0
rbx    0x7f2f1a8ca868
rcx    0xffffffffffffffff
rdx    0x6
rdi    0x4877
rsi    0x4877
rbp    0x5629255c2201
rsp    0x7ffe337c7968
r8     0xa
r9     0x7f2f1b120740
r10    0x8
r11    0x206
r12    0x5629281f2050
r13    0x0
r14    0x562926340d60
r15    0x6
rip    0x7f2f1a538387
rflags 0x206
cs     0x33
fs     0x0
gs     0x0
[root@iZuf6f7eny4wu2j05htsdiZ docker]# docker ps
runtime/cgo: pthread_create failed: Resource temporarily unavailable
runtime/cgo: pthread_create failed: Resource temporarily unavailable
SIGABRT: abort
PC=0x7fd1a55fe387 m=3 sigcode=18446744073709551610

goroutine 0 [idle]:
runtime: g 0: unknown pc 0x7fd1a55fe387
stack: frame={sp:0x7fd15e3b0938, fp:0x0} stack=[0x7fd15dbb12a0,0x7fd15e3b0ea0)
0x00007fd15e3b0838:  0x0000000000000000  0x0000000000000000
0x00007fd15e3b0848:  0x0000000000000000  0x0000000000000000
0x00007fd15e3b0858:  0x00007fd1a6026000  0x0000558c78af10c0 <runtime.mmap.func1+0x0000000000000000>
0x00007fd15e3b0868:  0x00007fd15e3b0858  0x0000000000000000
0x00007fd15e3b0878:  0x0000000000040000  0x0000002200000003
0x00007fd15e3b0888:  0x00000000ffffffff  0x00007fd15e3b08c8
0x00007fd15e3b0898:  0x0000558c78b04065 <runtime.sysAllocOS+0x0000000000000025>  0x0000000000000000
0x00007fd15e3b08a8:  0x0000000000000000  0x0000000000000000
0x00007fd15e3b08b8:  0x0000000000000000  0x0000000000082a92
0x00007fd15e3b08c8:  0x00007fd15e3b08e8  0x0000000078b03eb5
0x00007fd15e3b08d8:  0x0000000000000000  0x000000000007eaea
0x00007fd15e3b08e8:  0x00007fd15e3b0938  0x0000558c78afad65 <runtime.persistentalloc1+0x00000000000001c5>
0x00007fd15e3b08f8:  0x0000000000040000  0xffffffffffffc058
0x00007fd15e3b0908:  0x00007fd1a5990868  0x0000558c7981c201
0x00007fd15e3b0918:  0x00007fd1580008c0  0x0000000000000000
0x00007fd15e3b0928:  0x000000c000006ea0  0x0000000000000001
0x00007fd15e3b0938: <0x00007fd1a55ffa78  0x0000000000000020
0x00007fd15e3b0948:  0x0000000000000000  0x0000000000000000
0x00007fd15e3b0958:  0x0000000000000000  0x0000000000000000
0x00007fd15e3b0968:  0x0000000000000000  0x0000000000000000
0x00007fd15e3b0978:  0x0000000000000000  0x0000000000000000
0x00007fd15e3b0988:  0x0000000000000000  0x0000000000000000
0x00007fd15e3b0998:  0x0000000000000000  0x0000000000000000
0x00007fd15e3b09a8:  0x0000000000000000  0x0000000000000000
0x00007fd15e3b09b8:  0x0000000000000000  0x000000000000000d
0x00007fd15e3b09c8:  0x00007fd1a56b7bc9  0x0000000000000000
0x00007fd15e3b09d8:  0x00007fd1a56422f3  0x000000000000000d
0x00007fd15e3b09e8:  0x00007fd1a59901c0  0x0000000000000001
0x00007fd15e3b09f8:  0x00007fd1a5990243  0x00007fd1a598c380
0x00007fd15e3b0a08:  0x00007fd1a5643b0e  0x00007fd1a59901c0
0x00007fd15e3b0a18:  0x000000000000000a  0x00007fd1580008c0
0x00007fd15e3b0a28:  0x0000000000000000  0x000000c000006ea0
runtime: g 0: unknown pc 0x7fd1a55fe387
stack: frame={sp:0x7fd15e3b0938, fp:0x0} stack=[0x7fd15dbb12a0,0x7fd15e3b0ea0)
0x00007fd15e3b0838:  0x0000000000000000  0x0000000000000000
0x00007fd15e3b0848:  0x0000000000000000  0x0000000000000000
0x00007fd15e3b0858:  0x00007fd1a6026000  0x0000558c78af10c0 <runtime.mmap.func1+0x0000000000000000>
0x00007fd15e3b0868:  0x00007fd15e3b0858  0x0000000000000000
0x00007fd15e3b0878:  0x0000000000040000  0x0000002200000003
0x00007fd15e3b0888:  0x00000000ffffffff  0x00007fd15e3b08c8
0x00007fd15e3b0898:  0x0000558c78b04065 <runtime.sysAllocOS+0x0000000000000025>  0x0000000000000000
0x00007fd15e3b08a8:  0x0000000000000000  0x0000000000000000
0x00007fd15e3b08b8:  0x0000000000000000  0x0000000000082a92
0x00007fd15e3b08c8:  0x00007fd15e3b08e8  0x0000000078b03eb5
0x00007fd15e3b08d8:  0x0000000000000000  0x000000000007eaea
0x00007fd15e3b08e8:  0x00007fd15e3b0938  0x0000558c78afad65 <runtime.persistentalloc1+0x00000000000001c5>
0x00007fd15e3b08f8:  0x0000000000040000  0xffffffffffffc058
0x00007fd15e3b0908:  0x00007fd1a5990868  0x0000558c7981c201
0x00007fd15e3b0918:  0x00007fd1580008c0  0x0000000000000000
0x00007fd15e3b0928:  0x000000c000006ea0  0x0000000000000001
0x00007fd15e3b0938: <0x00007fd1a55ffa78  0x0000000000000020
0x00007fd15e3b0948:  0x0000000000000000  0x0000000000000000
0x00007fd15e3b0958:  0x0000000000000000  0x0000000000000000
0x00007fd15e3b0968:  0x0000000000000000  0x0000000000000000
0x00007fd15e3b0978:  0x0000000000000000  0x0000000000000000
0x00007fd15e3b0988:  0x0000000000000000  0x0000000000000000
0x00007fd15e3b0998:  0x0000000000000000  0x0000000000000000
0x00007fd15e3b09a8:  0x0000000000000000  0x0000000000000000
0x00007fd15e3b09b8:  0x0000000000000000  0x000000000000000d
0x00007fd15e3b09c8:  0x00007fd1a56b7bc9  0x0000000000000000
0x00007fd15e3b09d8:  0x00007fd1a56422f3  0x000000000000000d
0x00007fd15e3b09e8:  0x00007fd1a59901c0  0x0000000000000001
0x00007fd15e3b09f8:  0x00007fd1a5990243  0x00007fd1a598c380
0x00007fd15e3b0a08:  0x00007fd1a5643b0e  0x00007fd1a59901c0
0x00007fd15e3b0a18:  0x000000000000000a  0x00007fd1580008c0
0x00007fd15e3b0a28:  0x0000000000000000  0x000000c000006ea0

goroutine 1 [running, locked to thread]:
        goroutine running on other thread; stack unavailable

goroutine 2 [runnable]:
runtime.forcegchelper()
        /usr/local/go/src/runtime/proc.go:313 fp=0xc000072fe0 sp=0xc000072fd8 pc=0x558c78b272a0
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000072fe8 sp=0xc000072fe0 pc=0x558c78b57b61
created by runtime.init.6 in goroutine 1
        /usr/local/go/src/runtime/proc.go:310 +0x1a

goroutine 3 [GC sweep wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000073778 sp=0xc000073758 pc=0x558c78b274ce
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:404
runtime.bgsweep(0x0?)
        /usr/local/go/src/runtime/mgcsweep.go:280 +0x94 fp=0xc0000737c8 sp=0xc000073778 pc=0x558c78b12414
runtime.gcenable.func1()
        /usr/local/go/src/runtime/mgc.go:200 +0x25 fp=0xc0000737e0 sp=0xc0000737c8 pc=0x558c78b07585
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000737e8 sp=0xc0000737e0 pc=0x558c78b57b61
created by runtime.gcenable in goroutine 1
        /usr/local/go/src/runtime/mgc.go:200 +0x66

goroutine 4 [GC scavenge wait]:
runtime.gopark(0xc00007a000?, 0x558c798043c0?, 0x1?, 0x0?, 0xc0000071e0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000073f70 sp=0xc000073f50 pc=0x558c78b274ce
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:404
runtime.(*scavengerState).park(0x558c7a59a980)
        /usr/local/go/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc000073fa0 sp=0xc000073f70 pc=0x558c78b0fce9
runtime.bgscavenge(0x0?)
        /usr/local/go/src/runtime/mgcscavenge.go:653 +0x3c fp=0xc000073fc8 sp=0xc000073fa0 pc=0x558c78b1027c
runtime.gcenable.func2()
        /usr/local/go/src/runtime/mgc.go:201 +0x25 fp=0xc000073fe0 sp=0xc000073fc8 pc=0x558c78b07525
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000073fe8 sp=0xc000073fe0 pc=0x558c78b57b61
created by runtime.gcenable in goroutine 1
        /usr/local/go/src/runtime/mgc.go:201 +0xa5

rax    0x0
rbx    0x7fd1a5990868
rcx    0xffffffffffffffff
rdx    0x6
rdi    0x482a
rsi    0x4791
rbp    0x558c7981c201
rsp    0x7fd15e3b0938
r8     0xa
r9     0x7fd15e3b1700
r10    0x8
r11    0x202
r12    0x7fd1580008c0
r13    0x0
r14    0xc000006ea0
r15    0x1
rip    0x7fd1a55fe387
rflags 0x202
cs     0x33
fs     0x0
gs     0x0

Mar 10 '25 06:03 gongshaojie12

upgrade to v1.0.1 fixes the problem

Mar 12 '25 07:03 entropy2333

I am running the 1.0.1, self-hosted docker, having the kind of the same problem. every time I upload a document to knowledge base, when it start to index the work container will restart. ollama, or different api all the same.

2025-03-15 04:44:54.101 INFO [Dummy-3] [document_indexing_task.py:59] - Start process document: 3f287495-6309-48dd-b308-3436fca58f4c �2025-03-15 04:44:54.234 WARNING [Dummy-3] [warnings.py:112] - /app/api/.venv/lib/python3.12/site-packages/pypdfium2/_helpers/textpage.py:80: UserWarning: get_text_range() call with default params will be implicitly redirected to get_text_bounded() warnings.warn("get_text_range() call with default params will be implicitly redirected to get_text_bounded()")

Mar 15 '25 04:03 holynuts

Hi, @gongshaojie12. I'm Dosu, and I'm helping the Dify team manage their backlog. I'm marking this issue as stale.

Issue Summary:

You reported a "cannot allocate memory" error when uploading files to a self-hosted Dify service with 64GB of memory.
Suggestions to increase Docker's memory allocation and optimize configurations were confirmed as already met by you.
The error persists, and a detailed stack trace was provided.
User entropy2333 mentioned that upgrading to version 1.0.1 resolved the issue, but holynuts reported similar problems with version 1.0.1 during document indexing.

Next Steps:

Please let us know if this issue is still relevant to the latest version of the Dify repository. If so, you can keep the discussion open by commenting on this issue.
Otherwise, this issue will be automatically closed in 15 days.

Thank you for your understanding and contribution!

Apr 15 '25 16:04 dosubot[bot]