decisionintelligence
About the results of Crossformer on PEMS dataset
Hi, I tried the script of the model Crossformer on the PEMS dataset, but the results I got are so different (considerable drops) compared with those reported in the paper, is there something wrong? By the way, why do you tune the parameters so much with this model, it seems that settings are very different among different prediction horizons.
Hello, Thank you for your attention and questions. Sorry, due to some updates in our code, some results may not be recoverable, causing inconvenience to you. We will take some time to retest the performance of certain algorithms with the new code. And update their results in OpenTS: https://decisionintelligence.github.io/OpenTS/ and the arXiv. Regarding the issue of different hyperparameters, we initially used the initial parameters of the traffic dataset from the crossformer as the initial parameters for PEMS, and adjusted 8 sets of parameters for each prediction horizon based on this. Other algorithms also adjust the hyperparameters 8 times separately for each prediction horizon.
Hello, Thank you for your attention and questions. Sorry, due to some updates in our code, some results may not be recoverable, causing inconvenience to you. We will take some time to retest the performance of certain algorithms with the new code. And update their results in OpenTS: https://decisionintelligence.github.io/OpenTS/ and the arXiv. Regarding the issue of different hyperparameters, we initially used the initial parameters of the traffic dataset from the crossformer as the initial parameters for PEMS, and adjusted 8 sets of parameters for each prediction horizon based on this. Other algorithms also adjust the hyperparameters 8 times separately for each prediction horizon.
Thank you for your prompt reply, I think the main problem is the PEMS08 dataset. I tried temporally the models of PatchTST and Crossformer, and the results of both the two differ from those in the paper. Looking forward to your modification.
@Luoauoa Apologies for the delayed response and thank you for your patience. The previous issue was due to incorrect environment configurations and our use of a look-back window length of 720 in Crossformer, as specified in the paper, which led to discrepancies in the results. We have since corrected the relevant results in OpenTS and updated the corresponding scripts. Additionally, we recently tested several forecasting algorithms for 2024 and 2025. You can review the updated results and scripts in the TFB codebase and OpenTS. Thank you again for your understanding!
@Luoauoa Apologies for the delayed response and thank you for your patience. The previous issue was due to incorrect environment configurations and our use of a look-back window length of 720 in Crossformer, as specified in the paper, which led to discrepancies in the results. We have since corrected the relevant results in OpenTS and updated the corresponding scripts. Additionally, we recently tested several forecasting algorithms for 2024 and 2025. You can review the updated results and scripts in the TFB codebase and OpenTS. Thank you again for your understanding!
Thanks for your efforts