inference icon indicating copy to clipboard operation
inference copied to clipboard
verified publisher icon mlcommons

Must preview results be followed by a published result or is a submission + withdrawal acceptable?

Open pgmpablo157321 opened this issue 9 months ago • 2 comments

  • There have been cases where competitive results got changed and even though available submissions were ready, the company was not happy publishing them.
  • One potential solution here will be for the preview system submitter to show the performance to the review committee and make the code and hardware available in the subsequent round. Whether the preview system must now be audited is a question to be answered.

Post mortem task proposed by @arjunsuresh

pgmpablo157321 avatar Apr 22 '25 23:04 pgmpablo157321

Would be good to clarify before the next round. Need a rules proposal in order to move forward.

Arjun to draft a proposal.

swasson488 avatar May 06 '25 16:05 swasson488

Draft PR given here: https://github.com/mlcommons/policies/pull/194/files

arjunsuresh avatar May 13 '25 15:05 arjunsuresh