sapp
sapp copied to clipboard
Add SARIF schema test
Pre-submission checklist
- [x] I've ran the following linters locally and fixed lint errors related to the files I modified in this PR
- [x]
black . - [x]
usort format . - [x]
flake8
- [x]
- [x] I've installed dev dependencies
pip install -r requirements-dev.txtand completed the following:- [x] I've ran tests with
./scripts/run-tests.shand made sure all tests are passing
- [x] I've ran tests with
Summary
This PR adds initial tests to SARIF output from SAPP. Currently the SARIF output has no test coverage at all. This API add a schema test that the output SARIF passes the defined schema.
Test Plan
./scripts/run-tests.sh
----------------------------------------------------------------------
Ran 111 tests in 1.719s
...
and running the specific new tests
(venv) ielsayed@ibrahims-mbp-2 ~/s/r/personal-sapp (add-sarif-schema-validation-tests) [SIGINT]> python3 -m unittest -v sapp.ui.tests.sarif_test
testSarifSchemaCheckWithIssuesNoTraces (sapp.ui.tests.sarif_test.SarifTest.testSarifSchemaCheckWithIssuesNoTraces) ... ok
testSarifSchemaCheckWithIssuesWithTraces (sapp.ui.tests.sarif_test.SarifTest.testSarifSchemaCheckWithIssuesWithTraces) ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.134s
OK
I think the failing test is just a flaky test. I don't think the failure is from this PR
/cc @arthaud not sure if you guys missed this PR :D