Electricity Forecasting Framework

Comprehensive electricity load and demand forecasting framework. Supports statistical methods (ARIMA, SARIMA), machine learning (XGBoost, LightGBM, Random Forest), and deep learning (LSTM, GRU, Transformer, TFT). Use when building short-term load forecasting (STLF) systems, predicting electricity demand for energy trading, analyzing consumption patterns, integrating weather features, evaluating forecasts with MAPE/RMSE/MAE, or deploying production pipelines with uncertainty quantification.

Overview

This skill provides end-to-end support for electricity load/demand forecasting projects, from data preprocessing to model deployment. It covers traditional statistical methods, modern machine learning approaches, and state-of-the-art deep learning architectures.

Quick Start

1. Define Your Forecasting Task

Horizon	Type	Typical Use
1-48 hours	Short-term (STLF)	Grid operations, unit commitment
1 week - 1 month	Medium-term	Maintenance scheduling, fuel planning
1-12 months	Long-term (LTLF)	Capacity planning, infrastructure investment

2. Prepare Your Data

bash

# Run the data preparation script
python scripts/prepare_data.py --input raw_load.csv --output processed/

Required data columns:

• timestamp: Datetime index (hourly or sub-hourly)
• load: Target variable (MW or kWh)
• temperature: Weather feature (°C)
• Optional: humidity, wind_speed, solar_radiation, holiday_flag

3. Select Your Model

See references/model-selection.md for detailed guidance.

Quick recommendation:

• Baseline: Start with persistence or seasonal-naive
• Production STLF: Use XGBoost or LightGBM with weather features
• Research/SOTA: Try Temporal Fusion Transformer (TFT) or iTransformer

4. Train and Evaluate

bash

python scripts/train_model.py --model xgboost --data processed/ --horizon 24

Key metrics to track:

• MAPE (%): Mean Absolute Percentage Error - business interpretability
• RMSE (MW): Root Mean Square Error - penalizes large errors
• MAE (MW): Mean Absolute Error - robust to outliers
• Coverage (%): Prediction interval coverage probability

Core Workflows

Data Preprocessing

1. Load raw data with proper datetime parsing
2. Handle missing values: Forward-fill for short gaps, interpolate for longer
3. Feature engineering:
   • Temporal: hour, day_of_week, month, is_weekend, is_holiday
   • Lag features: load_t-1, load_t-24, load_t-168 (weekly)
   • Rolling stats: rolling_mean_24h, rolling_std_7d
   • Weather: temperature, humidity, apparent_temperature
4. Normalization: RobustScaler or MinMaxScaler for deep learning models

See references/feature-engineering.md for complete feature list.

Model Training

python

# Example training workflow
from electricity_forecasting import ForecastPipeline

pipeline = ForecastPipeline(
    model_type="xgboost",
    horizon=24,
    lookback=168  # 1 week of history
)

pipeline.fit(train_data, val_data)
predictions, uncertainty = pipeline.predict(test_data)
metrics = pipeline.evaluate(predictions, actuals)

Hyperparameter Tuning

Use scripts/hyperparameter_search.py for automated tuning:

bash

python scripts/hyperparameter_search.py \
  --model lightgbm \
  --data processed/ \
  --n-trials 50 \
  --study-name stlf-tuning

Uncertainty Quantification

For risk-aware decision making:

• Quantile Regression: Predict multiple quantiles (0.1, 0.5, 0.9)
• Conformal Prediction: Distribution-free uncertainty bounds
• Ensemble Methods: Model disagreement as uncertainty proxy
• Monte Carlo Dropout: For neural networks

See references/uncertainty.md for implementation details.

Model Reference

Statistical Models

Model	Best For	Pros	Cons
ARIMA	Stable series	Interpretable, fast	Assumes linearity
SARIMA	Strong seasonality	Captures daily/weekly patterns	Manual parameter tuning
Prophet	Multiple seasonalities	Handles holidays well	Less accurate for STLF
TBATS	Complex seasonality	Automatic parameter selection	Slower training

Machine Learning Models

Model	Best For	Pros	Cons
XGBoost	Production STLF	Fast, accurate, handles missing	No native uncertainty
LightGBM	Large datasets	Faster than XGBoost, memory efficient	Sensitive to hyperparameters
Random Forest	Baseline ML	Robust, easy to tune	Lower accuracy than boosting
CatBoost	Categorical features	Handles categoricals natively	Slower training

Deep Learning Models

Model	Best For	Pros	Cons
LSTM	Sequential patterns	Captures long-term dependencies	Slow training, hard to tune
GRU	Similar to LSTM	Faster convergence	Similar limitations
Transformer	Long sequences	Parallel training, attention	Data-hungry, complex
TFT	Multi-horizon	Interpretable attention, uncertainty	Complex implementation
N-BEATS	Pure deep learning	Strong baseline, interpretable	Less flexible than TFT
iTransformer	SOTA performance	Inverted transformer architecture	Recent, less battle-tested

See references/deep-learning-models.md for architecture details and PyTorch implementations.

Evaluation Best Practices

Time Series Cross-Validation

Never use random k-fold! Use expanding or sliding window:

python

# Expanding window CV
from sklearn.model_selection import TimeSeriesSplit

tscv = TimeSeriesSplit(n_splits=5, test_size=168)  # 1 week test
for train_idx, test_idx in tscv.split(data):
    train, test = data[train_idx], data[test_idx]
    # Train and evaluate

Backtesting Framework

bash

python scripts/backtest.py \
  --model xgboost \
  --data processed/ \
  --cv-splits 5 \
  --horizon 24 \
  --metrics mape,rmse,mae

Benchmark Comparison

Always compare against:

1. Persistence: load_t = load_t-1
2. Seasonal Naive: load_t = load_t-24 (for hourly data)
3. Weekly Naive: load_t = load_t-168

Deployment

Production Pipeline

1. Model serialization: Save with joblib or ONNX
2. Feature pipeline: Ensure identical preprocessing at inference
3. Scheduling: Cron or Airflow for automated forecasts
4. Monitoring: Track forecast drift and retrain triggers

See references/deployment.md for MLOps patterns.

Real-time Inference

python

from electricity_forecasting import DeploymentModel

model = DeploymentModel.load("models/xgboost-stlf.joblib")
features = prepare_features(latest_data)
prediction = model.predict(features, return_uncertainty=True)

Common Pitfalls

1. Data leakage: Ensure no future information in features
2. Holiday handling: Special days need explicit modeling
3. Temperature nonlinearity: Use heating/cooling degree days
4. Concept drift: Retrain quarterly or when MAPE degrades >20%
5. Peak prediction: Models often under-predict peaks - consider quantile loss

Resources

• Feature Engineering Guide
• Model Selection Guide
• Deep Learning Architectures
• Uncertainty Quantification
• Deployment Patterns
• Datasets Reference

Scripts

Script	Purpose
scripts/prepare_data.py	Data cleaning and feature engineering
scripts/train_model.py	Model training with validation
scripts/hyperparameter_search.py	Automated hyperparameter optimization
scripts/backtest.py	Time series cross-validation
scripts/evaluate.py	Comprehensive metric calculation
scripts/deploy_model.py	Export model for production

Example Usage

bash

# Complete workflow example
# 1. Prepare data
python scripts/prepare_data.py --input data/load_2024.csv --output data/processed/

# 2. Train model
python scripts/train_model.py --model lightgbm --data data/processed/ --horizon 48

# 3. Hyperparameter tuning
python scripts/hyperparameter_search.py --model lightgbm --data data/processed/ --n-trials 100

# 4. Backtest
python scripts/backtest.py --model lightgbm-best --data data/processed/ --cv-splits 5

# 5. Deploy
python scripts/deploy_model.py --model lightgbm-best --output models/production/