Vijay Vasudevan

This is great, thanks for organizing this! One thing I've also been thinking about like @daviddao is how to validate that the models are actually computing the same thing --...

1) eh, software is tricky, nothing to do with standardization 2) this conversation has ratholed from its original intent to discuss benchmarks, though the discussion is interesting.

My comment on ratholing was about standardization of APIs and frameworks -- I do think convergence speed and accuracy tests are important and we should eventually have them.

Step 1: define important models / benchmarks Step 2: make sure people's definitions of models match so the numbers are actually comparable Step 3: care and compare multi-step convergence I...

@bilalsattar Try again -- I asked the Waymo Open Dataset team to ensure access to registered users, so your job should hopefully have access now. Please keep in mind that...

Hm, the only thing I can think is to verify that the account *********[email protected] has been correctly registered on the Waymo Open Dataset website; the account used to access the...

okay at this point it probably makes sense to file an issue at https://github.com/waymo-research/waymo-open-dataset pointing at this issue for more directed help (we don't control that bucket or registration, so...

Only thing I can think of is to use your user account to copy the data into a bucket that you control, and use that bucket instead (since you can...

We used a Cloud TPU v3-32 and it trained for about 15 hours to get to the numbers reported in the paper, but we haven't trained it recently on GCP...

The training settings in waymo.py should describe what we train for: 150 epochs at effective batch size 128 (split across 32 TPU cores): I believe this corresponds to around 92500...