Performance gap in reproducing results

[ShangGL123]

The reproduced results show that DS is much lower than the numerical value in the paper. Is this result jitter normal?

[jiaxiaosong1002]

Hi, thanks for your interests.

This codebase has encountered a major refactor and it is mixed with some implementations in our next work (accepted by ICCV 2023 and this time we will release both the original 189K and 2M checkpoints, please stay tuned!). without thorough inspection and tuning due to limited time and computational resource. The model is also retrained.

Thus, there might be some bugs. I check out one of our old team_agent files and write a similar one for this codebase: old_team_agent.py Could you please try this one? (Just replace the original_team_agent) In the meantime, I will cross-check with the private codebase when having time.

Thanks for your time.

[Naive-Bayes]

The reproduced results show that DS is much lower than the numerical value in the paper. Is this result jitter normal?

Hello, which scenarios file do you use, just the "all_towns_traffic_scenarios_no256.json" provided in the codebase?

[raozhongyu]

Hi, I'd like to ask can you provide the inference time when you evaluate this code. How long it takes ?