Adding more Analytics rules

[jctaillandier]

Required items, please complete

Change(s):

• See guidance below

Reason for Change(s):

• See guidance below

Version Updated:

• Required only for Detections/Analytic Rule templates
• See guidance below

Testing Completed:

• See guidance below

Checked that the validations are passing and have addressed any issues that are present:

• See guidance below

Guidance <- remove section before submitting

---

Before submitting this PR please ensure that you have read the following sections and filled out the changes, reason for change and testing complete sections:

Thank you for your contribution to the Microsoft Sentinel Github repo.

Details of the code changes in your submitted PR. Providing descriptions for pull requests ensures there is context to changes being made and greatly enhances the code review process. Providing associated Issues that this resolves also easily connects the reason.

Change(s):

• Updated syntax for XYZ.yaml

Reason for Change(s):

• New schema used for XYZ.yaml
• Resolves ISSUE #1234

Version updated:

• Yes
• Detections/Analytic Rule templates are required to have the version updated

The code should have been tested in a Microsoft Sentinel environment that does not have any custom parsers, functions or tables, so that you validate no incorrect syntax and execution functions properly. If your submission requires a custom parser or function, it must be submitted with the PR.

Testing Completed:

• Yes/No/Need Help

Note: If updating a detection, you must update the version field.

Before the submission has been made, please look at running the KQL and Yaml Validation Checks locally. https://github.com/Azure/Azure-Sentinel#run-kql-validation-locally

Checked that the validations are passing and have addressed any issues that are present:

• Yes/No/Need Help

Note: Let us know if you have tried fixing the validation error and need help.

References:

• Guidance for Detection checks
• General contribution guidance
• PR validation troubleshooting

---

[jctaillandier]

I keep getting:

                    Errors: The name 'risk_reasons_s' does not refer to any known column, table, variable or function., Code: 'KS142', Severity: 'Error', Location: '20..34'

Although .script/tests/KqlvalidationsTests/CustomTables/Firework_CL.json does contain risk_score_d as a declared variable.

[jctaillandier]

@v-atulyadav could you help?

[v-prasadboke]

Hello @jctaillandier please change the unique id of analytic rules. All the analytic rules contains the same unique id

[v-prasadboke]

Hello @jctaillandier Please update your sample data ' risk_reasons_cl ' is working fine for me

[v-prasadboke]

The Firework_cl table you have uploded does not contain the colum ' risk_reasons_s ' whereas it contains ' risk_source_d ' that is the reason ' risk_source_d ' is working fine

[jctaillandier]

@v-prasadboke I did but now get errors saying I change the ID value of my analytics rule

[v-prasadboke]

I keep getting:

                    Errors: The name 'risk_reasons_s' does not refer to any known column, table, variable or function., Code: 'KS142', Severity: 'Error', Location: '20..34'

Although .script/tests/KqlvalidationsTests/CustomTables/Firework_CL.json does contain risk_score_d as a declared variable.

is this working now or you are getting the same error message

[jctaillandier]

Although .script/tests/KqlvalidationsTests/CustomTables/Firework_CL.json does contain risk_score_d as a declared variable.

is this working now or you are getting the same error message

This one seems fine

[jctaillandier]

Now failures are on files that don't belong to me (ExcessiveNXDOMAINDNSQueriesStaticThresholdBased):

2023-04-19T12:39:03.9654395Z Starting test execution, please wait...
2023-04-19T12:39:04.0187054Z A total of 1 test files matched the specified pattern.
2023-04-19T12:54:18.2674677Z [xUnit.net 00:15:13.42]     Kqlvalidations.Tests.KqlValidationTests.Validate_DetectionQueries_SkippedTemplatesDoNotHaveValidKql(fileName: "ExcessiveNXDOMAINDNSQueriesStaticThresholdBased.ya"..., encodedFilePath: "L2hvbWUvdnN0cy93b3JrLzEvcy9Tb2x1dGlvbnMvRE5TIEVzc2"...) [FAIL]
2023-04-19T12:54:18.2757074Z   Failed Kqlvalidations.Tests.KqlValidationTests.Validate_DetectionQueries_SkippedTemplatesDoNotHaveValidKql(fileName: "ExcessiveNXDOMAINDNSQueriesStaticThresholdBased.ya"..., encodedFilePath: "L2hvbWUvdnN0cy93b3JrLzEvcy9Tb2x1dGlvbnMvRE5TIEVzc2"...) [368 ms]
2023-04-19T12:54:18.2758303Z   Error Message:
2023-04-19T12:54:18.2761631Z    Template Id:4ab8b09e-3c23-4974-afbe-7e653779eb2b is valid but it is in the skipped validation templates. Please remove it from the templates that are skipped since it is valid.
2023-04-19T12:54:18.2764622Z Expected: False
2023-04-19T12:54:18.2766480Z Actual:   True

[jctaillandier]

Seems good to go

[v-prasadboke]

All looks good

[v-prasadboke]

Hello @jctaillandier please discard the changes from ' Create-Azure-Sentinel-Solution/v2/input ' folder as well as from ' Create-Azure-Sentinel-Solution/input '

[jctaillandier]

Done @v-prasadboke

[v-prasadboke]

Hello @jctaillandier is there any changes in the zip 2.0.2

[v-prasadboke]

Please update the version of analytic rule FlareDetectionLeaks.yaml

[v-prasadboke]

Hello @jctaillandier please update the api version of ' Microsoft.Resources/templateSpecs ' ' 2021-05-01 ' to ' 2022-02-01 '. And please update the version of detection " FlareDetectionLeaks.yaml " as well, Thank you.

[jctaillandier]

done @v-prasadboke

[v-prasadboke]

Going through this

[v-prasadboke]

Hello @jctaillandier have you repackaged the solution after incrementing the version of flarecredentailleaks

[jctaillandier]

No, I can do it. Is there a list I should look at of things to do ?

[v-prasadboke]

Sorry @jctaillandier for the changes i have requested multiple times, But we have to go through every instance so that the solution doesnt crash or contain any bugs or issues which would lead to extra work.

[v-prasadboke]

Hello @jctaillandier please update the api version of ' Microsoft.Resources/templateSpecs ' ' 2021-05-01 ' to ' 2022-02-01 '.

[v-prasadboke]

And please remove the files from v2/input folder

[jctaillandier]

done..

[v-prasadboke]

done..

Ok going through this

[v-prasadboke]

Hello @jctaillandier Delete the zip version 2.1.0 and create a new one with 2.0.4 and this zip should only include createui and maintemplate, Thank you

[v-prasadboke]

And please discard the changes the v2/input folder

[jctaillandier]

I want this version to be 2.1.0, is that ok ?

Could you make a list of things I need to do and share it with me once. We have been going at this for weeks with 1 change at the time.

[v-prasadboke]

Hello @jctaillandier sorry for the incovenience you are facing, But whatever changes you make in the maintemplate or createui should reflect in the zip as well. The maintemplate and create ui in the folder and should be the same with the maintemplate and createui in the zip package of the latest version. As the main template from the zip and outside the zip are not identical, Please update the zip package with the maintemplate which is in the package folder.

The Analytic rule ' FlareCredentialLeaks.yaml ' contains a column ' data_new_leaks_s ' which is not present in the current sample data. I request you to share the latest sample data, Or update the sample data for the specific column.

These are the only things which needs to change / modify, Other than this analytic rules, playbooks, and workbook is working fine.

Thank you, Prasad.

[jctaillandier]

Thanks.

I just fixed the package, but I do see data_new_leaks_s in my sample (see screenshot). Not sure where else you mean?