coreruleset
Rule 942200 triggers a substantial number of false positives.
Description
Not sure what rule 942200 is supposed to do exactly (as it's not docummented nor has tests), but it yields a lot false-positives. For example, the simple payload like this gets blocked: ?text=yes, bob's.
For the corpus wikipedia_2016_1M it yields 28342 results.
$ grep -i -E -c -f regexes.txt ./eng_wikipedia_2016_1M/eng_wikipedia_2016_1M-sentences.txt
28342
The the most critical expression from 942200.data is ,.*?[)\da-f\"'`][\"'`][^\"'`]+ which alone yields 27983 results.
TBF, the rule doesn't seem to be doing anything useful (except \Wselect.+\W*?from which doesn't fit the rule and should be moved somewhere else) so I'd consider disabling it. One thing for sure it needs a docummentation and tests.
The usage of the word space also seems to be a bug.
msg:'Detects MySQL comment-/space-obfuscated injections and backtick termination'
space is a function in MysQL, apparently.
The SLQi rules are a mess and we're in the process of cleaning them up. It will probably be a while before we can address this particular issue, so I've added a new sqli label.
We noticed another strange issue related to this rule ID, which is that when applied as part of sqli-stable or sqli-v33-stable rules via GCP Cloud Armor to a backend API endpoint, a request to a path under that endpoint can be made within the web app without any issues, but when one attempts to access that same path directly in another browser tab, 403 Forbidden is returned. The logs show that this rule is the culprit:
...
enforcedSecurityPolicy: {
configuredAction: "DENY"
name: "block-malicious-requests"
outcome: "DENY"
preconfiguredExprIds: [
0: "owasp-crs-v030001-id942200-sqli"
]
...
Then, puzzlingly, when the same API path is accessed in an incognito window (or a via cURL in the terminal), we get a different response determined by our own app code (not the Cloud Armor rule): 401 Unauthorized - this is what we would expect to see if that path is accessed by an unauthenticated user.
It seems the rule is triggered only when the API path is accessed within the same browser session as the web app, but not when it is accessed outside of that session. But we can't seem to figure out why this may be the case. This is not really a problem since it is not (yet) interfering with the functioning of the app in any way. But we would like to understand the root of this behavior, if possible.
@alex-scratch from what you describe it sounds like cookies could be at fault. Looking at the rule, you will see that it checks a bunch of fields, including cookies and cookie names:
REQUEST_COOKIES|!REQUEST_COOKIES:/__utm/|REQUEST_COOKIES_NAMES|ARGS_NAMES|ARGS|XML:/*
Do you have any more data? Which string has been matched, in which argument etc.?
@theseion Thanks for your quick response! That's it indeed. After clearing the relevant cookies, I'm now getting 401 Unauthorized in another tab in the same browser session, as elsewhere. It's interesting (and in our case reassuring!) however that even repeated requests coming from within the app are not blocked by the rule. In any case, if we ever observe that, we can simply exclude the particular rule ID. Thanks again.
Upon some further investigation, it's now clear what's going on: when the requests are submitted from the web app, no cookies are submitted in the request headers, but when they are submitted from another browser tab, a number of Hubspot cookies are included in the request headers. The presence of these cookies only in the latter requests triggers the rule.
Thanks for the update @alex-scratch. Could you possibly share (redacted) data from those cookies that trigger the rule? We can use it to write a test case.
Sure!
These were all the cookies present in the header (from Chrome):
cookie: hubspotutk=[redacted]; __hssrc=[redacted]; _gcl_au=[redacted]; _delighted_web=[redacted]; __hstc=[redacted]; __hssc=[redacted]
However, I observed in another browser (Firefox) that including only _delighted_web=[redacted] also triggered the rule. Various other combinations of these cookies would elicit either 401 Unauthorized or 403 Forbidden responses (e.g. hubspotutk=[redacted]; __hssrc=[redacted]; __hstc=[redacted]; __hssc=[redacted] returns 403 but any combination of two or fewer of these (the values of each also did not seem to effect this) seems to return 401. So I'm assuming the overall value of the cookie header (rather than any particular cookie value) is checked for a match with the regex.
This issue has been open 120 days with no activity. Remove the stale label or comment, or this will be closed in 14 days
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Getting this on a lot of ocasions
Random copy/paste with some redactions
--847c7876-A--
[09/Nov/2022:17:00:07.618421 +0100] Y2vOhxaWGiqL8dJ2iSX92gAAhAc x.x.x.x 56562 x.x.x.x 443
--847c7876-B--
GET /image?f=favicon.png HTTP/2.0
User-Agent: Mozilla/5.0 (Linux; Android 12; SAMSUNG SM-G973F) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/19.0 Chrome/102.0.5005.125 Mobile Safari/537.36
Dnt: 1
Accept: image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8
Sec-Fetch-Site: same-origin
Sec-Fetch-Mode: no-cors
Sec-Fetch-Dest: image
Referer: https://x.x/page?utm_source=Google&utm_medium=Display&utm_campaign=campaing1&gclid=EAIaIQobChMIwdnh5e2g-wIVSuGaCh1SeQv6EAEYASAAEgJkRPD_BwE
Accept-Encoding: gzip, deflate, br
Accept-Language: nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
Cookie: keuvelaar=icdi9m2vj2ghjqdci8mtj9i011; csrf-token=ae294d9b3ff8a8384ffb09351516e4321903692c3ebb2415f10302030090e81537818627d62ef955a18b95e394d4e3fe497453096747f4d3b1b8ecc576bdcd5f; bicycle-attack-token=NTE2MzUzNDAxMDIwMTQzMTQwMDYzMjM2MjIxNDQzOTQ1NTM4NDQyNDMzNTEzOTQ3NTIzNTExNTQ2NjI2NTI1MTI5MjA5NTM2MTI0NDk0MTQzODIwNTUyMDU4MTg0NTIwMzU1ODQ1MjQyNzYyNTI0OTQ2MTI1OTk1OTQ3NDUxNTIxNTY0Nzk0NjQ5MzUzNDE1MTk2MTQ2NTE3NjExMjQ5; _pk_ref.10281.d9ab=%5B%22Bretling%22%2C%22%22%2C1668009606%2C%22https%3A%2F%2Fgoogleads.g.doubleclick.net%2F%22%5D; _pk_id.10281.d9ab=07f37fad903ad845.1668009606.; _pk_ses.10281.d9ab=1; _fbp=fb.1.1668009606419.3614615
Host: x.x
--847c7876-F--
HTTP/1.1 403 Forbidden
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Expect-CT: max-age=300, report-uri="https://x.x"
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Content-Length: 306
Connection: close
Content-Type: text/html; charset=iso-8859-1
Date: Wed, 09 Nov 2022 16:00:07 GMT
Server: Apache
--847c7876-E--
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access this resource.</p>
<hr>
<address>Apache Server at <a href="mailto:[email protected]">keuvelaar.nl</a> Port 443</address>
</body></html>
--847c7876-H--
Message: Access denied with code 403 (phase 2). Pattern match "(?i:(?:(?:(?:(?:trunc|cre|upd)at|renam)e|(?:inser|selec)t|de(?:lete|sc)|alter|load)\\s*?\\(\\s*?space\\s*?\\(|,.*?[)\\da-f\"'`][\"'`](?:[\"'`].*?[\"'`]|(?:\\r?\\n)?\\z|[^\"'`]+)|\\Wselect.+\\W*?from))" at REQUEST_COOKIES:_pk_ref.10281.d9ab. [file "/usr/share/modsecurity-crs/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf"] [line "690"] [id "942200"] [msg "Detects MySQL comment-/space-obfuscated injections and backtick termination"] [data "Matched Data: ,\x22\x22,1668009606, found within REQUEST_COOKIES:_pk_ref.10281.d9ab: [\x22Bretling\x22,\x22\x22,1668009606,\x22https://googleads.g.doubleclick.net/\x22]"] [severity "CRITICAL"] [ver "OWASP_CRS/3.3.4"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-sqli"] [tag "OWASP_CRS"] [tag "capec/1000/152/248/66"] [tag "PCI/6.5.2"] [tag "paranoia-level/2"]
Apache-Error: [file "apache2_util.c"] [line 271] [level 3] [client x.x.x.x] ModSecurity: Access denied with code 403 (phase 2). Pattern match "(?i:(?:(?:(?:(?:trunc|cre|upd)at|renam)e|(?:inser|selec)t|de(?:lete|sc)|alter|load)\\\\\\\\s*?\\\\\\\\(\\\\\\\\s*?space\\\\\\\\s*?\\\\\\\\(|,.*?[)\\\\\\\\da-f\\\\"'`][\\\\"'`](?:[\\\\"'`].*?[\\\\"'`]|(?:\\\\\\\\r?\\\\\\\\n)?\\\\\\\\z|[^\\\\"'`]+)|\\\\\\\\Wselect.+\\\\\\\\W*?from))" at REQUEST_COOKIES:_pk_ref.10281.d9ab. [file "/usr/share/modsecurity-crs/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf"] [line "690"] [id "942200"] [msg "Detects MySQL comment-/space-obfuscated injections and backtick termination"] [data "Matched Data: ,\\\\x22\\\\x22,1668009606, found within REQUEST_COOKIES:_pk_ref.10281.d9ab: [\\\\x22Bretling\\\\x22,\\\\x22\\\\x22,1668009606,\\\\x22https://googleads.g.doubleclick.net/\\\\x22]"] [severity "CRITICAL"] [ver "OWASP_CRS/3.3.4"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-sqli"] [tag "OWASP_CRS"] [tag "capec/1000/152/248/66"] [tag "PCI/6.5.2"] [tag "paranoia-level/2"] [hostname "keuvelaar.nl"] [uri "/image"] [unique_id "Y2vOhxaWGiqL8dJ2iSX92gAAhAc"]
Action: Intercepted (phase 2)
Stopwatch: 1668009607607776 11118 (- - -)
Stopwatch2: 1668009607607776 11118; combined=4246, p1=606, p2=3490, p3=0, p4=0, p5=150, sr=95, sw=0, l=0, gc=0
Response-Body-Transformed: Dechunked
Producer: ModSecurity for Apache/2.9.6 (http://www.modsecurity.org/); OWASP_CRS/3.3.4.
Server: Apache
WebApp-Info: "SITEIDX" "-" "-"
Engine-Mode: "ENABLED"
--847c7876-Z--
If I'm correct the offending cookie decodes to: ["Bretling","",1668009606,"https://googleads.g.doubleclick.net/"] Which doesn't appear to be harmfull in any way.
Yes, that is correct. JSON in Cookies is really a nightmare. There's nothing we can do on our side, at least not short-term. You will have to tune those rules. Deactivate the problematic ones or, better, dynamically exclude things that you know are benign.
At this point we simply disabled these rules, simply googling for the site gives these cookies and blocks all visitors causing severe issues.
In my opinion these should be disabled by default or at the very least come with some decent warnings (and guide how to disable them, modsec 3.x allows for SecRuleRemoveById xxxxx directly in crs.conf, but doesn't honor them).
modsec 3.x allows for SecRuleRemoveById xxxxx directly in crs.conf, but doesn't honor them
This is probably loaded before the rule is declared. Config-time rule exclusions need to be declared after the rule itself.
Adding them to RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf works, learning every day :)
For people finding this ticket:
SecRuleUpdateTargetById 942260 !REQUEST_COOKIES
SecRuleUpdateTargetById 942340 !REQUEST_COOKIES
SecRuleUpdateTargetById 942200 !REQUEST_COOKIES
SecRuleUpdateTargetById 942370 !REQUEST_COOKIES
SecRuleUpdateTargetById 920300 !REQUEST_HEADERS
You may want to take a closer look at my ModSec tutorials on https://www.netnea.com/apache-tutorials/. There is also a cheat sheet that is meant to prevent exactly this error (which happens to me as well regularly, btw).
Yes, that is correct. JSON in Cookies is really a nightmare. There's nothing we can do on our side, at least not short-term. You will have to tune those rules. Deactivate the problematic ones or, better, dynamically exclude things that you know are benign.
Anything being done in the long-term?
Anything being done in the long-term?
What exactly do you mean?
Any changes to the regular expressions to handle JSON in Cookies to prevent countless false positives.
That will largely depend on engine support. There's really nothing we can do with rules alone to.
Our own quantiative testing lists 942200 PL2 as a problematic rule too, but it's not one of the worst offenders. So we'll probably accept it as is for CRSv4 and deal with it afterwards.
See #3392.