facebook
How to determine the best period for cross validation?
Hi, I have 2 years of daily data (one data per day from Monday to Friday), plus the current year and I want to forecast up to a month and a half (45 days horizon). I set as initial one year of data but my problem is with the period, I don't know what would be the optimal period to have the lowest possible error and the correct calculation for the hyperparameters.
I know that in the documentation they explain well the horizon, initial and period and its default setting of initial = 3*horizon and period = 0.5 *horizon. I have tested with 5, 10 and 23 days and get a similar average MAPE with all, but their MAPE over the 45 day forecast are very different. This is what I get:
With period = 23 days
With period = 10 days
With period = 5 days
I tend to think that the right thing to do is to choose period = 5 because it is constant over the horizon, or even use a lower value, but I want to be sure of this. Or maybe answer what is a better criterion for choosing period?
Period is how often you are making those forecasts 45 days out. So, it depends entirely on your business case for what you should do.
For instance, I create a new forecast every week for 5 weeks out. So, I have my initial window (initial) I want to train my model on, then I make new models every 7 days (period), and then I have my 35 day horizon.
So, I would pick period for how often you are making these forecasts and use that. if you're only making the forecast once a month, then you should probably go with a number around 30. If you're doing it every day, then a number around 1.
