microsoft
'sb result diagnosis' should exit with valid error code when input data format has error
What's the issue, what's expected?: This is superbench release/0.6. The 'sb result diagnosis' command didn't exit with error code properly when input data file has wrong format. e.g. for an empty data file, it has the right error detection which can be told from stdout/stderr, but the exit code is 0. see repro steps.
While empty file is just a rare case, this problem shows a general issue in superbench.
How to reproduce it?: Create an empty data file. Create valid rule and baseline files. Then run (For privacy reason, the timestamp and hostname were removed from the following log)
$ sb result diagnosis --data-file outputs/b/results-summary.jsonl --rule-file rule1.y --baseline-file baseline1.json --output-file-format json --output-all --output-dir diag
...[file_handler.py:41][ERROR] Analyzer: invalid raw data fomat - 'node' ...[rule_base.py:106][ERROR] RuleBase: empty raw data ...[data_diagnosis.py:405][INFO] DataDiagnosis: Begin to process 0 nodes ...[data_diagnosis.py:111][ERROR] DataDiagnosis: get criteria failed ...[data_diagnosis.py:407][INFO] DataDiagnosis: Processed finished ...[data_diagnosis.py:428][INFO] DataDiagnosis: Output results to diag1/diagnosis_summary.json
$ echo $? 0
