Data-Mining-Project icon indicating copy to clipboard operation
Data-Mining-Project copied to clipboard

Enhanced Data Handling and Multi-Sensor Integration

Open ZohaiAli opened this issue 3 months ago • 0 comments

This PR upgrades the existing Random Forest–based activity recognition pipeline with robust data-handling improvements and multi-sensor fusion capabilities. It modernizes the codebase for Python 3 + latest scikit-learn (≥1.3) and improves data reliability, model accuracy, and maintainability.

**Key Enhancements

🧠 Data Handling**

Added automatic missing value detection and imputation using SimpleImputer(strategy='median').

Implemented feature scaling/normalization with StandardScaler to improve model convergence and consistency.

Added robust preprocessing pipeline shared across all sensors for reproducibility.

⚙️ Model Improvements

Integrated GridSearchCV for automatic hyperparameter tuning (n_estimators, criterion, max_depth).

Implemented feature importance extraction and visualization for better interpretability.

Added normalized confusion matrix heatmaps for performance diagnostics.

🔄 Multi-Sensor Fusion

Combined wrist, hip, thigh, ankle, and upper arm datasets into a unified multi-sensor model.

Trains and evaluates this fusion model alongside individual sensor models.

Generates summary report comparing cross-sensor performance.

🧩 Code Modernization

Fully updated deprecated scikit-learn modules (cross_validation, grid_search → model_selection).

Structured into modular, reusable functions for easier maintenance.

Added model persistence via joblib for future inference.

Added warnings suppression, error handling, and clean console output.

Results

Stronger model robustness due to proper handling of missing data and feature scaling.

Improved interpretability with automatic feature importance visualization.

Cleaner, reproducible training pipeline for all sensors and combined models.

ZohaiAli avatar Oct 27 '25 08:10 ZohaiAli