Evaluation suite - output

[csudre]

Context Following the evaluation process on a set of references / output pairs, an aggregated report should be produced

Describe the solution you'd like 2 separate aggregations should be proposed:

• Raw aggregation - All the calculated metrics for each reference/output pair should be made available
• Population aggregation - Core statistics of the metrics over the population of reference/output pair should be produced including
  • central tendency and variation
  • indication of range
  • indication of cases at 5, 25, 50, 75, 95 percentile of performance for each metric
• The reports should be made available as pandas data frame with the option to save in a specified format (csv, xlsx, html)

Additional context https://github.com/Project-MONAI/MONAI/wiki/Evaluation-metrics-task-force @jenspetersen @danieltudosiu @AReinke

[Nic-Ma]

For the first step, I will try to implement a summary report tool based on ignite handler.

Thanks.