owasp-modsecurity
segmentation fault with SecStreamInBodyInspection and chunked encoding
On Redhat/CentOS/Rocky 8, httpd crashes with a segmentation fault when enabling SecStreamInBodyInspection and sending a chunked encoded request. This happens with the platform httpd & mod_security2. Same with latest mod_security2 (v2/master 2.9.4).
Note that the crash doesn't happen in CentOS 7, nor in Fedora. I guess there's a memory problem that corrupts the memory, but not always in a critical location.
This can be reproduced with this simple request: curl -v -H "Transfer-Encoding: chunked" -d hello http://x/a
Minimal config to reproduce the problem:
LoadModule mpm_worker_module modules/mod_mpm_worker.so
LoadModule systemd_module modules/mod_systemd.so
LoadModule unixd_module modules/mod_unixd.so
LoadModule authn_core_module modules/mod_authn_core.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule unique_id_module modules/mod_unique_id.so
LoadModule security2_module modules/mod_security2.so
User apache
Group apache
CoreDumpDirectory "/tmp"
PidFile /var/run/httpd/httpd.pid
DocumentRoot /var/www/htdocs
ServerName x.y.z
LogLevel warn
SecRuleEngine On
SecAuditLogType Serial
SecAuditLog /var/log/httpd/security.log
SecAuditLogParts ABIFHZ
SecTmpDir "/tmp/modsecurity/TmpDir"
SecUploadDir "/tmp/modsecurity/UploadDir"
SecDataDir "/tmp/modsecurity/DataDir"
SecHashKey rand KeyOnly
SecResponseBodyAccess off
SecRequestBodyAccess On
# This causes the segmentation fault
SecStreamInBodyInspection On
Listen 80
<VirtualHost "*:80">
ServerName x
<Location />
ProxyPreserveHost Off
ProxyPass "http://127.0.0.1:80/notexist/"
</Location>
</VirtualHost>
Example of stack trace:
Thread 3 "httpd" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fffda59e700 (LWP 3202)] ap_proxy_http_prefetch (url=0x7fffcc325dc0 "/a", uri=0x7fffcc325c80, req=0x7fffcc325ce0) at mod_proxy_http.c:640 640 origin->keepalive = AP_CONN_CLOSE; (gdb) bt #0 ap_proxy_http_prefetch (url=0x7fffcc325dc0 "/a", uri=0x7fffcc325c80, req=0x7fffcc325ce0) at mod_proxy_http.c:640 #1 proxy_http_handler (r=0x7fffcc011840, worker=
, conf= , url=0x7fffd8492a6e "http://127.0.0.1/a", proxyname= , proxyport= ) at mod_proxy_http.c:2076 #2 0x00007fffeeb84094 in proxy_run_scheme_handler (r=r@entry=0x7fffcc011840, worker=0x5555f8139920, conf=conf@entry=0x5555f8139778, url=0x7fffd8492a6e "http://127.0.0.1/a", proxyhost=proxyhost@entry=0x0, proxyport=proxyport@entry=0) at mod_proxy.c:3102 #3 0x00007fffeeb84e49 in proxy_handler (r=0x7fffcc011840) at mod_proxy.c:1266 #4 0x000055555559cad8 in ap_run_handler (r=r@entry=0x7fffcc011840) at config.c:170 #5 0x000055555559d096 in ap_invoke_handler (r=r@entry=0x7fffcc011840) at config.c:444 #6 0x00005555555b4243 in ap_process_async_request (r=r@entry=0x7fffcc011840) at http_request.c:453 #7 0x00005555555b43b2 in ap_process_request (r=r@entry=0x7fffcc011840) at http_request.c:488 #8 0x00005555555b05d5 in ap_process_http_sync_connection (c=0x7fffd4003c68) at http_core.c:210 #9 ap_process_http_connection (c=0x7fffd4003c68) at http_core.c:251 #10 0x00005555555a6ab8 in ap_run_process_connection (c=c@entry=0x7fffd4003c68) at connection.c:42 #11 0x00005555555a7038 in ap_process_connection (c=c@entry=0x7fffd4003c68, csd=csd@entry=0x7fffd4003a50) at connection.c:219 #12 0x00007ffff392b41f in process_socket (bucket_alloc=0x7fffcc000b88, my_thread_num=0, my_child_num=0, sock=0x7fffd4003a50, p=0x7fffd40039d8, thd=0x5555566286a0) at worker.c:479 #13 worker_thread (thd=0x5555566286a0, dummy= ) at worker.c:808 #14 0x00007ffff6b0a14a in start_thread (arg= ) at pthread_create.c:479 #15 0x00007ffff6635dc3 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
Hi @marcstern ,
I rather suspect that this has nothing to do with ModSecurity.
For one thing, there are no ModSecurity functions in the call stack. That alone is only an indicator rather than a guarantee of any sort, however ...
There have been other reports of a comparable failure using Redhat 8 unrelated to ModSecurity, see for examples these items from a thread ... https://www.mail-archive.com/[email protected]/msg75786.html https://www.mail-archive.com/[email protected]/msg75809.html https://www.mail-archive.com/[email protected]/msg75788.html
What do you think?
Hi @martinhsv ,
"SecStreamInBodyInspection on" generates the core dump, "SecStreamInBodyInspection off" works correctly. This really looks like a problem linked to ModSecurity (maybe the way it register as a filter?).
Hi @marcstern ,
If the on/off setting toggles the outcome reliably, then that does suggest some interaction between ModSecurity and the non-ModSecurity call stack that you provided in your third comment. However, that doesn't necessarily mean that the actual code bug is in ModSecurity code -- it could equally be in non-ModSecurity apache code that is engaging in some memory misuse.
Nevertheless, I will re-open this at least temporarily.
However, given that:
- it's not certain that the bug is in ModSecurity
- it is only seen in some O/S environments
- it involves a setting that I doubt is widely used
- even if someone were inclined to use that setting, it can be turned off and worked-around
... I doubt I can justify spending a whole lot of time on this item (especially in the near term).
But perhaps some member of the community will feel inclined to dig into this further.