Problem when Pretrainig the model with past_feat_dynamic_real

[bkoyuncu]

Discussed in https://github.com/SalesforceAIResearch/uni2ts/discussions/169

^{Originally posted by bkoyuncu January 8, 2025} Hello,

I have created my data with past_feat_dynamic_real, however, I believe the current implementation in pretrain.py does not include them in training as:

https://github.com/SalesforceAIResearch/uni2ts/blob/cadebd82106e32409b7854b033dbd7a68de87fc0/src/uni2ts/model/moirai/pretrain.py#L485

where self.seq_fields are defined as

https://github.com/SalesforceAIResearch/uni2ts/blob/cadebd82106e32409b7854b033dbd7a68de87fc0/src/uni2ts/model/moirai/pretrain.py#L66-L72

I believe past_feat_dynamic_real are not being output in the end of processing the input data.

Is this an expected behaviour? How can I make sure past_feat_dynamic_real are used in training?

[bkoyuncu]

I am thinking if past_feat_dynamic_real needs to be truncated with other inputs as these are known context features, which should not be masked at all.

https://github.com/SalesforceAIResearch/uni2ts/blob/cadebd82106e32409b7854b033dbd7a68de87fc0/src/uni2ts/model/moirai/pretrain.py#L447-L454

[liuxu77]

Hi @bkoyuncu, I think past_feat_dynamic_real should be involved in the "target", and at the same time, their prediction_mask are all set to False.