Dilip Sequeira

The directory structure in 5.6 for inference doesn't conform with the de facto structure in the inference submission repo.

Does end_on_device make sense?

14

The rationale for start_from_device is that submissions should not need to incur the overhead of transfer from system DRAM if there is a mechanism whereby network inputs can be delivered...

Timed preprocessing

10

For the March '21 round (1.0?) we would like to see consideration of more timed preprocessing in the datacenter scenarios. specifically for the image models and for 3D-UNet. For edge,...

Address issue #122 according to Inf WG discussion

23

This also adds a definition of a Reproducible software component so that RDI does not become a venue for submissions that are not reproducible and can never be reproducible.

Schedule deadline for public-facing materials

2

MLPerf's public-facing material (i.e. press releases, slide decks for press etc) should be available for review by submitters as part of the submission schedule. Let's add this for future rounds.

Require disclosure of Accuracy when quoting Open division results

2

Propose replacing _When comparing Open and Closed division results any ways in which the Open result would not qualify as a Closed result must be identified._ with _When comparing Open...

Originally in https://github.com/mlperf/training_policies/pull/258 before the file moved.

Remove the requirement to submit trace.json

1

Generating the trace can bottleneck loadgen, The log can be gigabytes - or when compressed, hundreds of megabytes - for a single benchmark/scenario combination. We suggest it be removed from...