[FR] Add white space checking for KQL parse
Issues
https://github.com/elastic/detection-rules/issues/2700
Summary
This addresses an issue where lark parses KQL queries without whitespace around certain tokens, where KQL does not.
E.g. "Get-NetComputerSiteName" or "Get-NetLocalGroup" vs "Get-NetComputerSiteName" or"Get-NetLocalGroup". Both of which parse via lark/ANTLR, but the second fails in Kibana.
Some notes about alternative implementations:
- We don't want to update the grammar because we need to ignore white space
- We don't want to use regex pre-processing because it will catch things like "category" as invalid unavoidably
This approach adds a post-processing step to the lark parsing to tell us where the and and or tokens are in the original string, then compare to see if those tokens locations have the appropriate spacing.
Note since this PR updates the KQL lib please make sure to update the KQL lib version appropriately.
Contributor checklist
- Have you signed the contributor license agreement?
- Have you followed the contributor guidelines?
This should probably be handled in the grammar instead
This should probably be handled in the grammar instead
The current grammar requires white space to be ignored. I think the way you are suggesting would require a refactor of both the grammar and the parsing to handle this. This would not only be a refactor/overhaul but in effect a full replacement as most of not all of the code would need to be updated compared to the relatively minor change I am suggesting.
Enhancement - Guidelines
These guidelines serve as a reminder set of considerations when addressing adding a feature to the code.
Documentation and Context
- [ ] Describe the feature enhancement in detail (alternative solutions, description of the solution, etc.) if not already documented in an issue.
- [ ] Include additional context or screenshots.
- [ ] Ensure the enhancement includes necessary updates to the documentation and versioning.
Code Standards and Practices
- [ ] Code follows established design patterns within the repo and avoids duplication.
- [ ] Ensure that the code is modular and reusable where applicable.
Testing
- [ ] New unit tests have been added to cover the enhancement.
- [ ] Existing unit tests have been updated to reflect the changes.
- [ ] Provide evidence of testing and validating the enhancement (e.g., test logs, screenshots).
- [ ] Validate that any rules affected by the enhancement are correctly updated.
- [ ] Ensure that performance is not negatively impacted by the changes.
- [ ] Verify that any release artifacts are properly generated and tested.
- [ ] Conducted system testing, including fleet, import, and create APIs (e.g., run
make test-cli,make test-remote-cli,make test-hunting-cli)
Additional Checks
- [ ] Verify that the enhancement works across all relevant environments (e.g., different OS versions).
- [ ] Confirm that the proper version label is applied to the PR
patch,minor,major.