Fix bug: changed reference to references and updated cwe info to fit into semgrep required format

[nitinNayar]

Fixed 4 issues related to metadata:

• the semgrep rules have reference key. This should be references to be compliant with semgrep required format
• the semgrep rules reference key should be a list. This is semgrep required format
• the semgrep rules have cwe key. This should be in the format `CWE-XXX: CWE_TITLE' to be compliant with semgrep required format
• the semgrep rules cwe key should be a list. This is semgrep required format

I have updated these metadata field - no changes to the rule patterns or messages

-- nitin (Semgrep Solution Engineering, EMEA)

[abhinavsejpal-mns]

@ajinabraham - It would be helpful if you could review proposed changes and share your feedback 👍

[ajinabraham]

Have you tested this with mobsfscan? To simplify things mobsfscan will substitute cwe-xxx with the correct CWE description at runtime.

[nitinNayar]

Hi @ajinabraham This should not have any impact on your rule as this is only metadata. This is the recommended syntax / format so that the references and CWE show up correctly in the Semgrep UI.

The reference to references change is blocking Abhinav right now. Can you please approve. If it makes sense, lets get on a quick 5-10 minute call

[ajinabraham]

We pin to an older version of semgrep internally and it works well with that. With the changes, the tests are failing.

collecting ... collected 5 items

tests/unit/test_dotfile.py::test_mobsfscan_dotfile FAILED                [ 20%]
tests/unit/test_matcher.py::test_kotlin PASSED                           [ 40%]
tests/unit/test_matcher.py::test_ios PASSED                              [ 60%]
tests/unit/test_mobsfscan.py::test_patterns_and_semgrep FAILED           [ 80%]
tests/unit/test_xml.py::test_xml PASSED                                  [100%]

=================================== FAILURES ===================================
____________________________ test_mobsfscan_dotfile ____________________________

    def test_mobsfscan_dotfile():
        paths = get_paths()
        files = paths['dot_file']
>       res = scanner([files])

tests/unit/test_dotfile.py:17: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/unit/setup_test.py:8: in scanner
    return MobSFScan(paths, True).scan()
.tox-mobsf/py/lib/python3.11/site-packages/mobsfscan/mobsfscan.py:104: in scan
    result = scanner.scan()
.tox-mobsf/py/lib/python3.11/site-packages/libsast/scanner.py:65: in scan
    self.options).scan(valid_paths)
.tox-mobsf/py/lib/python3.11/site-packages/libsast/core_sgrep/semantic_sgrep.py:41: in scan
    self.format_output(sgrep_out)
.tox-mobsf/py/lib/python3.11/site-packages/libsast/core_sgrep/semantic_sgrep.py:68: in format_output
    self.expand_mappings(smatches[rule_id])
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <libsast.core_sgrep.semantic_sgrep.SemanticGrep object at 0x7f3038c587d0>
meta = {'files': [{'file_path': '/home/runner/work/mobsfscan/mobsfscan/tests/assets/src/dot_mobsf/scan_but_ignore.java', 'mat... 'description': 'A hardcoded password in plain text is identified.', 'masvs': 'storage-14', 'owasp-mobile': 'm9', ...}}

    def expand_mappings(self, meta):
        """Expand libsast standard mappings."""
        meta_keys = meta['metadata'].keys()
        for mkey in meta_keys:
            if mkey not in self.standards.keys():
                continue
            to_expand = meta['metadata'][mkey]
>           expanded = self.standards[mkey].get(to_expand)
E           TypeError: unhashable type: 'list'

.tox-mobsf/py/lib/python3.11/site-packages/libsast/core_sgrep/semantic_sgrep.py:77: TypeError
__________________________ test_patterns_and_semgrep ___________________________

    def test_patterns_and_semgrep():
        paths = get_paths()
>       res = scanner([paths['java'], paths['kotlin']])

tests/unit/test_mobsfscan.py:28: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/unit/setup_test.py:8: in scanner
    return MobSFScan(paths, True).scan()
.tox-mobsf/py/lib/python3.11/site-packages/mobsfscan/mobsfscan.py:104: in scan
    result = scanner.scan()
.tox-mobsf/py/lib/python3.11/site-packages/libsast/scanner.py:65: in scan
    self.options).scan(valid_paths)
.tox-mobsf/py/lib/python3.11/site-packages/libsast/core_sgrep/semantic_sgrep.py:41: in scan
    self.format_output(sgrep_out)
.tox-mobsf/py/lib/python3.11/site-packages/libsast/core_sgrep/semantic_sgrep.py:68: in format_output
    self.expand_mappings(smatches[rule_id])
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <libsast.core_sgrep.semantic_sgrep.SemanticGrep object at 0x7f303979c510>
meta = {'files': [{'file_path': '/home/runner/work/mobsfscan/mobsfscan/tests/assets/src/java/java_vuln.java', 'match_lines': ...sly acquired from an otherwise unimpeachable certificate authority.', 'masvs': 'network-4', 'owasp-mobile': 'm3', ...}}

    def expand_mappings(self, meta):
        """Expand libsast standard mappings."""
        meta_keys = meta['metadata'].keys()
        for mkey in meta_keys:
            if mkey not in self.standards.keys():
                continue
            to_expand = meta['metadata'][mkey]
>           expanded = self.standards[mkey].get(to_expand)
E           TypeError: unhashable type: 'list'

.tox-mobsf/py/lib/python3.11/site-packages/libsast/core_sgrep/semantic_sgrep.py:77: TypeError
=============================== warnings summary ===============================
tests/unit/test_dotfile.py::test_mobsfscan_dotfile
  /home/runner/work/mobsfscan/mobsfscan/.tox-mobsf/py/lib/python3.11/site-packages/semgrep/semgrep_core.py:21: DeprecationWarning: path is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.
    with importlib.resources.path("semgrep.bin", exec_name) as path: