Azure-Sentinel When this Rule Update To adapt OWASP CRS ruleSet Version 2.0?

1.As we update application firewall Rule ,The logs have some change. Since OWASP CRS ruleSet update to 3.2, the application firewall log contents in Message Field can be parse results like SQLI_Score ,XSS_Score ,RFI_Score... 2.So Update WAF XSS and SQLI rule that can parse Attack behavior make sure detection work. WXWorkCapture_17025540006919

Thanks.

Dec 14 '23 11:12 ql3xHd630

Thank you for submitting an Issue to the Azure Sentinel GitHub repo! You should expect an initial response to your Issue from the team within 5 business days. Note that this response may be delayed during holiday periods. For urgent, production-affecting issues please raise a support ticket via the Azure Portal.

Dec 14 '23 11:12 github-actions[bot]

Hi @ql3xHd630, could you please share more details about this issue? like which solution or content you are getting this issue. Thanks!

Dec 14 '23 12:12 v-sudkharat

Thanks fir your reply, Next i will show a details: Issue: Since Sentinel build-in WAF Analytics Rule "Application Gateway WAF - XSS Detection" and "Application Gateway WAF - SQLi Detection" detect Attack behavier by Parse Message Field in logs. When the OSWAP_CRS rule version is 3.1.0 , the query in Analytics Rule work, it can parse XSS_Core, SQLi Core, Blocked_Reason from "Message" field. 3 1 But when OWASP_CRS verions update to 3.2 ,the content in "Message" field seems change a lot, so that query in WAF analytics rule can not parse XSS_Core, SQLi Core, Blocked_Reason and so on from "Message". 3 2 Expect Soluction: Update WAF Analytics Rule that can parse like XSS_Score, SQLiScore, Blocked_Reason and so on.

Dec 15 '23 02:12 ql3xHd630

Hi @ql3xHd630 , Thanks for flagging this issue, we will investigate this issue and get back to you with some updates by 21Dec2023. Thanks!

Dec 15 '23 07:12 v-muuppugund

Hello @v-muuppugund, Thanks for your feedback! I look forward to the reply then. thanks!

Dec 15 '23 07:12 ql3xHd630

Hi @ql3xHd630 ,I tried to run the Analytic rule in our work spaces as don't have data,unable to replicate the issue ,so could you please share the sample data in csv format over an email to ([email protected] ),so can proceed further on analysis.

Dec 21 '23 14:12 v-muuppugund

Hi @ql3xHd630 , Gentle Reminder, I tried to run the Analytic rule in our work spaces as don't have data,unable to replicate the issue ,so could you please share the sample data in csv format over an email to ([email protected] ),so can proceed further on analysis.

Dec 27 '23 11:12 v-muuppugund

Hi @v-muuppugund, i have send the waf logs to you. please check. If you have any questions about my mail. please contact me at any time. Thanks!

Dec 28 '23 06:12 ql3xHd630

Hi @ql3xHd630 ,I have checked my emails didn't receive any email checked till wednesday,Could you please check and share the details to ([email protected] )

Dec 29 '23 08:12 v-muuppugund

SInce i research the previous WAF XSS and SQLI rule in sentinel analytics rule kql query(https://analyticsrules.exchange/analyticrules/d2bc08fa-030a-4eea-931a-762d27c6a042/).

Not difficute to find that the attack behavor it detect by Message Field in WAF logs.Becase i have only WAF logs for prod env, so i download many logs with only "Message" and "ruleId_s" Field.

If you have any question, please relpy me at any time. Thanks!

@v-muuppugund i upload file here wafLogs (3).csv

Jan 03 '24 05:01 ql3xHd630

Hi @ql3xHd630 ,thanks for the data, will do further analysis and update you.

Jan 04 '24 08:01 v-muuppugund

Hey @ql3xHd630, just want to check from your provided log's, Could you please run below query's and share the result or log's? - As per the query - the parse operator is extracting various scores and a reason from the Message field of the logs. So i want to see the Message for ruleSetVersion = 3.1 and 3.2. In your provided screenshot. i can see it query parse for 3.1 which may having those values for Blocked_Reason and XSS_Score. 1st -

   AzureDiagnostics
   | where Category == "ApplicationGatewayFirewallLog"
   | where action_s == "Matched" or action_id_s ==  "Detected"
   | where ruleSetVersion_s =="3.1"
   | parse Message with MessageText 'Total Inbound Score: ' TotalInboundScore ' - SQLI=' SQLI_Score ',XSS=' XSS_Score ',RFI=' RFI_Score ',LFI=' LFI_Score ',RCE=' RCE_Score ',PHPI=' PHPI_Score ',HTTP=' HTTP_Score ',SESS=' SESS_Score '): ' Blocked_Reason '; individual paranoia level scores:' Paranoia_Score
    | project Message

2nd-

   AzureDiagnostics
   | where Category == "ApplicationGatewayFirewallLog"
   | where action_s == "Matched" or action_id_s ==  "Detected"
   | where ruleSetVersion_s =="3.2"
   | parse Message with MessageText 'Total Inbound Score: ' TotalInboundScore ' - SQLI=' SQLI_Score ',XSS=' XSS_Score ',RFI=' RFI_Score ',LFI=' LFI_Score ',RCE=' RCE_Score ',PHPI=' PHPI_Score ',HTTP=' HTTP_Score ',SESS=' SESS_Score '): ' Blocked_Reason '; individual paranoia level scores:' Paranoia_Score
    | project Message

Thanks!

Jan 05 '24 11:01 v-sudkharat

Hi @ql3xHd630, I hope you are doing well. We are waiting for your response to the above comment. It would be great if you could check on that and share your response with us. Thanks!

Jan 09 '24 06:01 v-sudkharat

i'm sorry for not replying in time. Now i upload search result for two query. The one is for ruleSetVersion_s =="3.1", the oether is for ruleSetVersion_s =="3.2" 3.2.csv

3.1.csv

Jan 12 '24 01:01 ql3xHd630

Hi @ql3xHd630 ,Thanks for providing response,will work on it and update you

Jan 12 '24 05:01 v-muuppugund

@v-muuppugund got it. thanks.

Jan 12 '24 06:01 ql3xHd630

Hi @ql3xHd630 ,I have done the initial analysis and based on messages data unable to see the messages for the issue Query shared by you i.e. in 3.1.csv and 3.2.csv with message contains "Mandatory Rule" ,Please find below attaching same for reference

We can have a call over teams meeting to discuss further on this issue.

Jan 16 '24 11:01 v-muuppugund

hi, @v-muuppugund，our WAF update the OWASP_CRS ruleSetVersion_s to 3.2.0 , so we need to get if the logs means actually attack behavior from from logs that ruleSetVersion equal 3.2.0

Jan 17 '24 02:01 ql3xHd630

hi ,@v-muuppugund The key to the problem is that we cannot parse the Block Reason, XSS _SCore, Total Inbound and other fields from Message Field in log using the query you provided. Because the message field with rule_set_version equal to 3.2 has changed. The purpose of parsing these fields in the 3.1 version of the rule is to identify attack behaviors, so it is not necessary to parse these fields to obtain them in version 3.2. We just need to identify the real attack behaviors through logs. Hope you can provide a solution. Thanks.

Jan 19 '24 01:01 ql3xHd630

Hi @ql3xHd630 ,Apologies for the delayed response,Will check on it and come back to you with an update.

Jan 21 '24 10:01 v-muuppugund

Hi @ql3xHd630 ,Still need some more time for further analysis and investigation,will update you.

Jan 25 '24 04:01 v-muuppugund

@v-muuppugund
got it ,thanks

Jan 26 '24 00:01 ql3xHd630

Hi @ql3xHd630 , Could you please share convenient time slots for teams meeting as having some queries to discuss in detail on this issue to ([email protected]).

Jan 30 '24 22:01 v-muuppugund

Hi @ql3xHd630 ,Gentle Reminder,Could you please share convenient time slots for teams meeting as having some queries to discuss in detail on this issue to ([email protected]).

Feb 05 '24 17:02 v-muuppugund

Hi @ql3xHd630 ,Gentle Reminder,Could you please share convenient time slots for teams meeting as having some queries to discuss in detail on this issue to ([email protected]).

Feb 08 '24 10:02 v-muuppugund

Since we have not received a response in the last 5 days, we are closing your issue (https://github.com/Azure/Azure-Sentinel/issues/9610) as per our standard operating procedures. If you still need support for this issue, feel free to re-open at any time. Thank you for your co-operation!

Feb 14 '24 02:02 v-muuppugund