UniAD
UniAD copied to clipboard
Add option to use legacy planning metric definition
As described in #166, there is a discrepancy between how planning metrics (L2 and collision rate) are computed in this repo vs previous works, such as VAD and ST-P3.
However, in the paper, these values are compared in the same table, leading to confusion. This PR enables computation of the UniAD metrics using the legacy definition used by VAD/ST-P3, making comparison easier. However, it still defaults to the "original" UniAD way of evaluating. This results in a ranking shift when comparing the methods, see the table below
| Method | L2 1s (m) | L2 2s (m) | L2 3s (m) | CR 1s (%) | CR 2s (%) | CR 3s (%) |
|---|---|---|---|---|---|---|
| ST-P3 | 1.33 | 2.11 | 2.90 | 0.23 | 0.62 | 1.27 |
| UniAD (your current metric) | 0.51 | 0.98 | 1.65 | 0.10 | 0.15 | 0.61 |
| UniAD (legacy ST-P3 metric) | 0.42 | 0.64 | 0.91 | 0.07 | 0.10 | 0.22 |
| VAD-Tiny | 0.46 | 0.76 | 1.12 | 0.21 | 0.35 | 0.58 |
| VAD-Base | 0.41 | 0.70 | 1.05 | 0.07 | 0.17 | 0.41 |