Improve Future Temporal Features Implementation

[IAlgarve]

Background

According to the README update (2024.05), TimeMixer now supports using future temporal features for prediction through the use_future_temporal_feature parameter. While this functionality has been well-received by the community, several aspects of the implementation can be enhanced.

Current Limitations

1. The current implementation treats all future temporal features equally, without considering their varying predictive power
2. There's limited documentation on how to best leverage this feature
3. No specialized mechanisms for different types of temporal features (e.g., cyclical vs. trend indicators)
4. Limited unit tests for this functionality
5. No integration with external calendar or event data sources

Proposed Enhancements

1. Feature Importance-Based Selection

Implement an automatic selection mechanism that identifies which future temporal features provide the most predictive value:

class FutureFeatureSelector(nn.Module):
    def __init__(self, feature_dim, hidden_dim=64):
        super().__init__()
        self.importance_scorer = nn.Sequential(
            nn.Linear(feature_dim, hidden_dim),
            nn.ReLU(),
            nn.Linear(hidden_dim, feature_dim),
            nn.Sigmoid()
        )
        
    def forward(self, future_features):
        # Generate importance scores for each feature
        importance = self.importance_scorer(future_features)
        # Apply importance as weights
        weighted_features = future_features * importance
        return weighted_features, importance

2. Specialized Feature Encoders

Add dedicated encoders for different types of temporal features:

class TemporalFeatureEncoder(nn.Module):
    def __init__(self, configs):
        super().__init__()
        self.cyclical_encoder = CyclicalFeatureEncoder(configs)
        self.trend_encoder = TrendFeatureEncoder(configs)
        self.event_encoder = EventFeatureEncoder(configs)
        
    def forward(self, x_mark):
        # Split features by type
        cyclical_features = x_mark[:, :, :self.cyclical_dims]
        trend_features = x_mark[:, :, self.cyclical_dims:self.cyclical_dims+self.trend_dims]
        event_features = x_mark[:, :, -self.event_dims:]
        
        # Process each feature type separately
        encoded_cyclical = self.cyclical_encoder(cyclical_features)
        encoded_trend = self.trend_encoder(trend_features)
        encoded_event = self.event_encoder(event_features)
        
        # Combine encodings
        return torch.cat([encoded_cyclical, encoded_trend, encoded_event], dim=-1)

3. External Calendar Integration

Add support for integrating external calendar data as future temporal features:

def load_calendar_features(start_date, end_date, region='US'):
    """
    Load calendar features like holidays, fiscal periods, etc.
    
    Args:
        start_date: Start date for features
        end_date: End date for features
        region: Country/region code for locale-specific calendars
        
    Returns:
        DataFrame with calendar features
    """
    # Implementation details

Implementation Plan

1. Refactor the current future temporal feature handling in models/TimeMixer.py
2. Add feature importance estimation and selection
3. Implement specialized encoders for different feature types
4. Create utilities for external calendar data integration
5. Add comprehensive documentation and examples
6. Develop unit tests for new functionality

Benefits

• Better Predictions: More sophisticated handling of future temporal features
• Explainability: Understanding which temporal features contribute most
• Flexibility: Supporting different types of future information
• Usability: Clear documentation on how to leverage this capability

Related Components

• models/TimeMixer.py
• utils/timefeatures.py (needs enhancement)
• New file: utils/external_features.py

[kevinliu2000]

Thank you very much for your suggestion regarding Future Temporal Features. In my recent work, I have also found that Future Temporal Features are crucial for real-world forecasting. Unlike theoretical research, in practical applications, the main influence on future prediction actually comes from future-known information, which has often been ignored in previous time-series model studies—leading to suboptimal performance in real-world scenarios. From what I see, part of this functionality is already supported in the current code implementation. If you have a better approach, you are very welcome to submit a pull request.