William Ljungbergh

Thanks a lot for the clarification. I don't know how I missed that issue... sorry about that. I now see that you define the metric similarly to ST-P3. However, upon...

No worries, glad to help. I'd be happy to make a pull request, but I will take up this again sometime after March 7th for obvious reasons. :writing_hand: :page_facing_up:

Alright! Now I have some time on my hands. Do you want to change the evaluation to match those of SPT3 and VAD? Or do you want it to be...

Yes. They do the same for L2 and collision-rate (i.e., cumulative). As you can see [here](https://github.com/hustvl/VAD/blob/1ede800a517653837faf78b5ec1c3b5fc3ec77b5/projects/mmdet3d_plugin/VAD/VAD.py#L624C1-L629C1), they create one planning metric for each of the timesteps they are using and...

Added a PR #173. Cross-posting the table here: | Method | L2 1s (m) | L2 2s (m) | L2 3s (m) | CR 1s (%) | CR 2s (%)...

If you want to create your sensor-realistic closed-loop simulator, have a look at [NeuRAD](https://github.com/georghess/neurad-studio) and the soon-to-be-released [NeuroNCAP](https://github.com/atonderski/neuro-ncap)