Bug: Contentctl adds spaces in annotations when outputting to savedsearches.conf in ESCU

[TheLawsOfChaos]

When creating a correlation search manually via the GUI in Splunk Enterprise Security, the annotations are created without spaces. An example:

{"cis20":["CIS 10"],"kill_chain_phases":["Exploitation"],"mitre_attack":["T1003.001","T1003"],"nist":["DE.CM"],"confidence":90,"impact":90,"analytic_story":["Credential Dumping"]}

The same query done via ESCU has the following annotations: {"analytic_story": ["Credential Dumping"], "cis20": ["CIS 10"], "confidence": 90, "impact": 90, "kill_chain_phases": ["Exploitation"], "mitre_attack": ["T1003.001", "T1003"], "nist": ["DE.CM"], "type": "TTP"}

After every 'annotation name', there is a space after the colon. No one probably even caught this, but I was working on rex to extract the analytic stories, mitre_attack techniques, confidence, and impact from the annotations which causes the problem to appear.

My initial code, with ESCU only correlation searches worked just fine:

| rex field=action.correlationsearch.annotations "\"analytic_story\": \[(?<analytic_story>[^\]]+)\]" 
| rex field=action.correlationsearch.annotations "\"mitre_attack\": \[(?<mitre_attack>[^\]]+)\]" 
| rex field=action.correlationsearch.annotations "\"confidence\": (?<confidence>[^,]+)," 
| rex field=action.correlationsearch.annotations "\"impact\": (?<impact>[^,]+),"

And to get it to work with custom created content I merely had to modify it to:

| rex field=action.correlationsearch.annotations "\"analytic_story\":\s?\[(?<analytic_story>[^\]]+)\]" 
| rex field=action.correlationsearch.annotations "\"mitre_attack\":\s?\[(?<mitre_attack>[^\]]+)\]" 
| rex field=action.correlationsearch.annotations "\"confidence\":\s?(?<confidence>[^,]+)," 
| rex field=action.correlationsearch.annotations "\"impact\":\s?(?<impact>[^,]+),"

Ultimately not a huge deal on my part, after realizing it. That said, I imagine contentctl's point is to make the savedsearches.conf to be in the same standards with Splunk Enterprise Security, so I'm pointing this bug out.

I am including a screenshot of proof. In it there are three results : ESCU - Create Remote Thread into LSASS - Rule is the correlation search by default from ESCU. BSDE - Create Remote Thread into LSASS - Rule is the same correlation search cloned via the ES GUI. Manual Creation - Create Remote Thread into LSASS is the same correlation search, but done via the New Correlation search menu, with the data copy/pasted into it.

This can also be recreated by creating any manual correlation search with annotations to find the lack of space.

This discovery was done in collaboration with @rivosyke

The code block from my screenshot is provided, so you don't have to handjam to recreate it if needed:

| rest splunk_server=local count=0 /servicesNS/-/-/saved/searches 
| search action.correlationsearch.label=* 
| rex field=action.correlationsearch.annotations "\"analytic_story\":\s?\[(?<analytic_story>[^\]]+)\]" 
| rex field=action.correlationsearch.annotations "\"mitre_attack\":\s?\[(?<mitre_attack>[^\]]+)\]" 
| rex field=action.correlationsearch.annotations "\"confidence\":\s?(?<confidence>[^,]+)," 
| rex field=action.correlationsearch.annotations "\"impact\":\s?(?<impact>[^,]+),"
| table action.correlationsearch.label  eai:acl.app analytic_story confidence impact mitre_attack action.correlationsearch.annotations

[ljstella]

Thanks for bringing this up @TheLawsOfChaos, we'll take a look. In the meantime, you should be able to avoid that by not using regex and relying on spath instead:

| rest /servicesNS/-/-/saved/searches count=0 splunk_server=local 
| search action.correlationsearch.label=* 
| spath input=action.correlationsearch.annotations
| table action.correlationsearch.label eai:acl.app analytic_story{} confidence impact mitre_attack{} action.correlationsearch.annotations