shift-dev
shift-dev copied to clipboard
[WIP] Add SHIFT reference evaluators
| Task | Metrics | Status |
|---|---|---|
| Semantic segmentation | mIoU, mAcc, fwIoU, fwAcc | [x] |
| Object detection 2D | COCO (mAP, AP50 AP75) | [x] |
| Instance segmentation | COCO (mAP, AP50 AP75) | [x] |
| Depth estimation | MAE, Delta_{1,2,5}, SILog | [x] |
| Optical flow | EPE | [x] |
| MOT | MOTA, HOTA, IDSw | [ ] |
| Object detection 3D | nuScene (mAP, AP50 AP75) | [ ] |
| MOT 3D | nuScene (MOTA, HOTA, IDSw) | [ ] |
| Trajectory forecasting | EPE, APE | [ ] |
| Pose estimation (human) | PCK | [ ] |
| Dump to eval.ai format | [x] |