Gerald Woo

## moirai-1.0-R-Base ### with fix: |index|dataset|test\_metrics/MSE\[mean\]|test\_metrics/MSE\[0\.5\]|test\_metrics/MAE\[0\.5\]|test\_metrics/MASE\[0\.5\]|test\_metrics/MAPE\[0\.5\]|test\_metrics/sMAPE\[0\.5\]|test\_metrics/MSIS|test\_metrics/RMSE\[mean\]|test\_metrics/NRMSE\[mean\]|test\_metrics/ND\[0\.5\]|test\_metrics/mean\_weighted\_sum\_quantile\_loss| |---|---|---|---|---|---|---|---|---|---|---|---|---| |0|electricity|1708712\.5|1711201\.625|164\.1307373046875|0\.7915405631065369|0\.10031454265117645|0\.11076250672340393|6\.184873580932617|1307\.17724609375|0\.5480201840400696|0\.06881006807088852|0\.054687876254320145| |1|solar-energy|1011\.0943603515625|1108\.408935546875|16\.981399536132812|1\.2911229133605957|2\.296311855316162|1\.4095485210418701|7\.017038345336914|31\.797710418701172|1\.0324212312698364|0\.5513591766357422|0\.41874560713768005| |2|walmart|26299296\.0|19072352\.0|2049\.69384765625|0\.9657745957374573|0\.23114198446273804|0\.1677016019821167|8\.415294647216797|5128\.28369140625|0\.29334932565689087|0\.117247074842453|0\.09353204816579819| |3|istanbul\_traffic|37\.16828918457031|40\.77793884277344|4\.562923431396484|0\.5369675755500793|0\.2586391270160675|0\.2553446292877197|3\.8279149532318115|6\.0965800285339355|0\.1627856343984604|0\.12183525413274765|0\.09833786636590958| |4|turkey\_power|473797\.125|474377\.5625|295\.6066589355469|0\.8949130177497864|0\.16863861680030823|0\.37849825620651245|6\.532022476196289|688\.3292236328125|0\.11860048770904541|0\.050933610647916794|0\.040024567395448685| ### without fix: |index|dataset|test\_metrics/MSE\[mean\]|test\_metrics/MSE\[0\.5\]|test\_metrics/MAE\[0\.5\]|test\_metrics/MASE\[0\.5\]|test\_metrics/MAPE\[0\.5\]|test\_metrics/sMAPE\[0\.5\]|test\_metrics/MSIS|test\_metrics/RMSE\[mean\]|test\_metrics/NRMSE\[mean\]|test\_metrics/ND\[0\.5\]|test\_metrics/mean\_weighted\_sum\_quantile\_loss| |---|---|---|---|---|---|---|---|---|---|---|---|---| |0|electricity|7544522\.5|6037268\.0|300\.3915710449219|1\.4005271196365356|0\.1897597312927246|0\.17114804685115814|8\.438410758972168|2746\.7294921875|1\.1515370607376099|0\.12593597173690796|0\.09828455746173859| |1|solar-energy|6939\.76318359375|3234\.031982421875|46\.51622772216797|3\.5624678134918213|15\.757192611694336|1\.4608030319213867|40\.661155700683594|83\.30523681640625|2\.704789161682129|1\.5103082656860352|1\.162361741065979| |2|walmart|20425108\.0|19084788\.0|1990\.737548828125|0\.9465618133544922|0\.22026395797729492|0\.16590428352355957|8\.317343711853027|4519\.41455078125|0\.25852063298225403|0\.1138746440410614|0\.09144116938114166| |3|istanbul\_traffic|122\.82603454589844|160\.76173400878906|8\.947684288024902|1\.0514392852783203|1\.8842673301696777|0\.375012069940567|5\.328226566314697|11\.082691192626953|0\.2959204614162445|0\.23891335725784302|0\.17149078845977783| |4|turkey\_power|7537457\.0|2658505\.75|731\.1760864257812|1\.9673036336898804|5\.040554523468018|0\.4480009973049164|29\.197540283203125|2745\.44287109375|0\.4730452299118042|0\.1259830892086029|0\.11150600761175156|

https://github.com/SalesforceAIResearch/uni2ts/blob/main/cli/conf/finetune/model/moirai_small.yaml Does using a yaml file like this solve your issue? You can just create it for your use case.

I recommend using something like the linked yaml file to solve the issue. The default should not be to upload model weights to huggingface hub. There are many reasons why...

Sure, please feel free to make a PR on this. I don't see any drawback since it's modular and the user can decide whether or not to use the callback...

This can go into `src/uni2ts/callback/lightning`.

https://github.com/SalesforceAIResearch/uni2ts/blob/main/cli/conf/finetune/model/moirai_1.0_R_small.yaml https://github.com/SalesforceAIResearch/uni2ts/blob/main/cli/conf/finetune/model/moirai_small.yaml Please see these 2 files, depending if you want to load the huggingface_hub file or the lightning checkpoint.

Hi, could you edit the bug report according to the template? It's quite hard to understand what is the error from just the above.

what about the yaml file in uni2ts/cli/conf/finetune/data?

Hey, it might be due to the data being in the wrong directory. The recommended approach is: `huggingface-cli download Salesforce/lotsa_data --repo-type=dataset --local-dir PATH_TO_SAVE` which would download the data into `lotsa_data/CMIP6_1855/data-00001-of-00096.arrow`...

Thanks @awaelchli for the detailed notebook, its really helpful! One question I have is, does this only work with PyTorch's inbuilt parallel strategies, or will it work for other strategies...