Ritika Borkar

Currently SD rcps are using steps which decrease with increasing batch-size. This will break rcp-pruning logic. Please use samples/epochs in RCP json

training rule described in https://github.com/mlcommons/training_policies/blob/master/training_rules.adoc#91-hyperparameters require num_image_candidates to be (1000 or 2000) or (1000 * batches per chip). Current check only looks for multiple of 1000: https://github.com/mlcommons/logging/blob/master/mlperf_logging/compliance_checker/training_2.0.0/closed_maskrcnn.yaml#L64

Document RCPs method of production

6

It would be useful if each RCP point also notes the method of production that was used to generate it, in the RCP jsons. Intention would be to have a...

We would like to finalize the submission review calendar events by the rules freeze deadline.

[BERT] Pre-trained checkpoint conversion script across frameworks

1

As per #317 , we would like all references that use pre-trained weights or start from a checkpoint to provide conversion scripts for use across different frameworks (Pytorch, TF, &...

[Minigo] Pre-trained checkpoint conversion script across frameworks

4

As per #317 , we would like all references that use pre-trained weights or start from a checkpoint to provide conversion scripts for use across different frameworks (Pytorch, TF, &...

Please tag v3.0 reference code.

Experimental Tag for benchmarks

5

Agile tag is a way for MLPerf to stay agile & make early bets on upcoming hot benchmarks. It allows adopting viral benchmarks while also providing a way to update,...

Update withdrawal rules

1

Objective: Penalize submitters who participate in the private review period but do not contribute any result to the public table. Edit New objective: based on discussion in WG, spell out...