uwsgi Triple-Usage of disk space in TMPDIR on file uploads (post)

I am using CKAN behind uwsgi+nginx and I am encountering an unexpected issue when uploading large (20GB) files. I set the TMPDIR environment variable to a dedicated mount point /cache (100GB block storage) and when I upload a 20GB file, uwsgi fetches the post-data from nginx and creates three (two too many) 20GB files. Since nginx and uwsgi write to deleted files, here is the output of lsof /cache immediately before uwsgi finishes consuming the upload (the #504, #505, #506 files are created one after another):

COMMAND  PID     USER   FD   TYPE DEVICE    SIZE/OFF NODE NAME
nginx    978 www-data   19u   REG   0,55 22124009256  503 /cache/nginx/client_body/1/00/0000000001 (deleted)
uwsgi   1099 www-data   11u   REG   0,55 22124009256  504 /cache/uwsgi/#504 (deleted)
uwsgi   1099 www-data   12u   REG   0,55 22124008746  505 /cache/uwsgi/#505 (deleted)
uwsgi   1099 www-data   13u   REG   0,55 22124008746  506 /cache/uwsgi/#506 (deleted)

My problem with that is that for a 20GB file upload, I now need 80GB of disk space. I understand that nginx is helping alot by caching the post request, but I don't understand why uwsgi has to parse the post request so many times. Is there some protocol conversion taking place (e.g. http->uwsgi) or is this actually a CKAN issue? How do I find out?

/etc/ckan/default/ckan-uwsgi.ini:

[uwsgi]

http11-socket   =  127.0.0.1:8080
uid             =  www-data
gid             =  www-data
wsgi-file       =  /etc/ckan/default/wsgi.py
virtualenv      =  /usr/lib/ckan/default
module          =  wsgi:application
master          =  true
pidfile         =  /tmp/%n.pid
harakiri        =  7200
max-requests    =  5000
vacuum          =  true
callable        =  application
buffer-size     =  32768

post-buffering  =  0
strict          =  true

workers         =  2
lazy-apps       =  true

For more background:

https://github.com/ckan/ckan/issues/5933
https://dc.readthedocs.io/en/latest/sec_self_hosting/installation.html#configuring-nginx-ssl-and-uwsgi-proxy (nginx and uwsgi configuration)

Apr 06 '21 22:04 paulmueller

FYI: Using uwsgi-socket instead of http11-socket (and changing the corresponding configuration in nginx) leads to the same behavior.

Apr 07 '21 10:04 paulmueller

Does anyone have any idea why this is happening?

Dec 13 '21 08:12 paulmueller

Hi, unfortunately I do not have a solution to your problem. Just want to state, that I have the same issue on ckan 2.9.8 and ckan 2.9.9.

Jun 22 '23 12:06 ChristianF88