Electricity Forecasting Framework

Comprehensive electricity load and demand forecasting framework. Supports statistical methods (ARIMA, SARIMA), machine learning (XGBoost, LightGBM, Random Fo...

Security Scan

The skill's files, instructions, and requirements are coherent with an electricity forecasting framework and do not request disproportionate credentials or install arbitrary third‑party code.

Purpose & Capability

Name/description match the included scripts and reference docs (data prep, training, evaluation, deployment). Nothing in the manifest asks for unrelated cloud credentials, binaries, or config paths.

Instruction Scope

SKILL.md directs the agent to run local Python scripts, prepare data, train models, and run FastAPI/batch jobs. Instructions reference weather/data APIs as examples (with placeholder API keys) but do not instruct reading unrelated system files or exfiltrating secrets.

Install Mechanism

No install spec is provided (instruction-only). All code is included in the package; nothing is downloaded from external URLs or extracted to disk during an install step.

Credentials

The skill declares no required environment variables or credentials. Example snippets show usage of third‑party APIs that would need user-supplied API keys, which is expected for such data sources and not requested by the skill itself.

Persistence & Privilege

always:false and normal agent invocation settings. The skill does not request persistent system privileges or attempt to modify other skills or global agent config in the provided files.

Assessment

This package appears coherent for electricity forecasting. Before running it: (1) review the full scripts (especially deploy_model.py, batch_forecast/send_to_downstream and any truncated functions) to confirm any network endpoints they call; (2) supply API keys only for trusted weather/data providers and store them securely (env vars or secret manager), since example code shows placeholders; (3) run in an isolated virtual environment or container and avoid running as root (batch job writes to /var/log and reads/writes models/ and forecasts/ directories); (4) pin dependencies and audit third‑party libraries (PyTorch, LightGBM, joblib, FastAPI, requests, etc.) before deploying to production. If you want, I can scan the specific scripts (deploy_model.py, batch_forecast, send_to_downstream) for any network calls or unexpected behavior.

Overview

This skill provides end-to-end support for electricity load/demand forecasting projects, from data preprocessing to model deployment. It covers traditional statistical methods, modern machine learning approaches, and state-of-the-art deep learning architectures.

Quick Start

1. Define Your Forecasting Task

Horizon	Type	Typical Use
1-48 hours	Short-term (STLF)	Grid operations, unit commitment
1 week - 1 month	Medium-term	Maintenance scheduling, fuel planning
1-12 months	Long-term (LTLF)	Capacity planning, infrastructure investment

2. Prepare Your Data

# Run the data preparation script
python scripts/prepare_data.py --input raw_load.csv --output processed/

Required data columns:

• timestamp: Datetime index (hourly or sub-hourly)
• load: Target variable (MW or kWh)
• temperature: Weather feature (°C)
• Optional: humidity, wind_speed, solar_radiation, holiday_flag

3. Select Your Model

See references/model-selection.md for detailed guidance.

Quick recommendation:

• Baseline: Start with persistence or seasonal-naive
• Production STLF: Use XGBoost or LightGBM with weather features
• Research/SOTA: Try Temporal Fusion Transformer (TFT) or iTransformer

4. Train and Evaluate

python scripts/train_model.py --model xgboost --data processed/ --horizon 24

Key metrics to track:

• MAPE (%): Mean Absolute Percentage Error - business interpretability
• RMSE (MW): Root Mean Square Error - penalizes large errors
• MAE (MW): Mean Absolute Error - robust to outliers
• Coverage (%): Prediction interval coverage probability

Core Workflows

Data Preprocessing

1. Load raw data with proper datetime parsing
2. Handle missing values: Forward-fill for short gaps, interpolate for longer
3. Feature engineering:
   • Temporal: hour, day_of_week, month, is_weekend, is_holiday
   • Lag features: load_t-1, load_t-24, load_t-168 (weekly)
   • Rolling stats: rolling_mean_24h, rolling_std_7d
   • Weather: temperature, humidity, apparent_temperature
4. Normalization: RobustScaler or MinMaxScaler for deep learning models

See references/feature-engineering.md for complete feature list.

Model Training

# Example training workflow
from electricity_forecasting import ForecastPipeline

pipeline = ForecastPipeline(
    model_type="xgboost",
    horizon=24,
    lookback=168  # 1 week of history
)

pipeline.fit(train_data, val_data)
predictions, uncertainty = pipeline.predict(test_data)
metrics = pipeline.evaluate(predictions, actuals)

Hyperparameter Tuning

Use scripts/hyperparameter_search.py for automated tuning:

python scripts/hyperparameter_search.py \
  --model lightgbm \
  --data processed/ \
  --n-trials 50 \
  --study-name stlf-tuning

Uncertainty Quantification

For risk-aware decision making:

• Quantile Regression: Predict multiple quantiles (0.1, 0.5, 0.9)
• Conformal Prediction: Distribution-free uncertainty bounds
• Ensemble Methods: Model disagreement as uncertainty proxy
• Monte Carlo Dropout: For neural networks

See references/uncertainty.md for implementation details.

Model Reference

Statistical Models

Model	Best For	Pros	Cons
ARIMA	Stable series	Interpretable, fast	Assumes linearity
SARIMA	Strong seasonality	Captures daily/weekly patterns	Manual parameter tuning
Prophet	Multiple seasonalities	Handles holidays well	Less accurate for STLF
TBATS	Complex seasonality	Automatic parameter selection	Slower training

Machine Learning Models

Model	Best For	Pros	Cons
XGBoost	Production STLF	Fast, accurate, handles missing	No native uncertainty
LightGBM	Large datasets	Faster than XGBoost, memory efficient	Sensitive to hyperparameters
Random Forest	Baseline ML	Robust, easy to tune	Lower accuracy than boosting
CatBoost	Categorical features	Handles categoricals natively	Slower training

Deep Learning Models

Model	Best For	Pros	Cons
LSTM	Sequential patterns	Captures long-term dependencies	Slow training, hard to tune
GRU	Similar to LSTM	Faster convergence	Similar limitations
Transformer	Long sequences	Parallel training, attention	Data-hungry, complex
TFT	Multi-horizon	Interpretable attention, uncertainty	Complex implementation
N-BEATS	Pure deep learning	Strong baseline, interpretable	Less flexible than TFT
iTransformer	SOTA performance	Inverted transformer architecture	Recent, less battle-tested

See references/deep-learning-models.md for architecture details and PyTorch implementations.

Evaluation Best Practices

Time Series Cross-Validation

Never use random k-fold! Use expanding or sliding window:

# Expanding window CV
from sklearn.model_selection import TimeSeriesSplit

tscv = TimeSeriesSplit(n_splits=5, test_size=168)  # 1 week test
for train_idx, test_idx in tscv.split(data):
    train, test = data[train_idx], data[test_idx]
    # Train and evaluate

Backtesting Framework

python scripts/backtest.py \
  --model xgboost \
  --data processed/ \
  --cv-splits 5 \
  --horizon 24 \
  --metrics mape,rmse,mae

Benchmark Comparison

Always compare against:

1. Persistence: load_t = load_t-1
2. Seasonal Naive: load_t = load_t-24 (for hourly data)
3. Weekly Naive: load_t = load_t-168

Deployment

Production Pipeline

1. Model serialization: Save with joblib or ONNX
2. Feature pipeline: Ensure identical preprocessing at inference
3. Scheduling: Cron or Airflow for automated forecasts
4. Monitoring: Track forecast drift and retrain triggers

See references/deployment.md for MLOps patterns.

Real-time Inference

from electricity_forecasting import DeploymentModel

model = DeploymentModel.load("models/xgboost-stlf.joblib")
features = prepare_features(latest_data)
prediction = model.predict(features, return_uncertainty=True)

Common Pitfalls

1. Data leakage: Ensure no future information in features
2. Holiday handling: Special days need explicit modeling
3. Temperature nonlinearity: Use heating/cooling degree days
4. Concept drift: Retrain quarterly or when MAPE degrades >20%
5. Peak prediction: Models often under-predict peaks - consider quantile loss

Resources

• Feature Engineering Guide
• Model Selection Guide
• Deep Learning Architectures
• Uncertainty Quantification
• Deployment Patterns
• Datasets Reference

Scripts

Script	Purpose
scripts/prepare_data.py	Data cleaning and feature engineering
scripts/train_model.py	Model training with validation
scripts/hyperparameter_search.py	Automated hyperparameter optimization
scripts/backtest.py	Time series cross-validation
scripts/evaluate.py	Comprehensive metric calculation
scripts/deploy_model.py	Export model for production

Example Usage

# Complete workflow example
# 1. Prepare data
python scripts/prepare_data.py --input data/load_2024.csv --output data/processed/

# 2. Train model
python scripts/train_model.py --model lightgbm --data data/processed/ --horizon 48

# 3. Hyperparameter tuning
python scripts/hyperparameter_search.py --model lightgbm --data data/processed/ --n-trials 100

# 4. Backtest
python scripts/backtest.py --model lightgbm-best --data data/processed/ --cv-splits 5

# 5. Deploy
python scripts/deploy_model.py --model lightgbm-best --output models/production/