Pareto Optimization
Pareto optimization deals with multi-objective optimization where you want to optimize multiple conflicting objectives simultaneously.
Key Concepts
Pareto Dominance
Point A dominates point B if:
- A is at least as good as B in all objectives
- A is strictly better than B in at least one objective
Pareto Frontier (Pareto Front)
The set of all non-dominated points. These represent optimal trade-offs where improving one objective requires sacrificing another.
Computing the Pareto Frontier
Using the paretoset Library
from paretoset import paretoset
import pandas as pd
# Data with two objectives (e.g., model accuracy vs inference time)
df = pd.DataFrame({
'accuracy': [0.95, 0.92, 0.88, 0.85, 0.80],
'latency_ms': [120, 95, 75, 60, 45],
'model_size': [100, 80, 60, 40, 20],
'learning_rate': [0.001, 0.005, 0.01, 0.05, 0.1]
})
# Compute Pareto mask
# sense: "max" for objectives to maximize, "min" for objectives to minimize
objectives = df[['accuracy', 'latency_ms']]
pareto_mask = paretoset(objectives, sense=["max", "min"])
# Get Pareto-optimal points
pareto_points = df[pareto_mask]
Manual Implementation
import numpy as np
def is_dominated(point, other_points, maximize_indices, minimize_indices):
"""Check if point is dominated by any point in other_points."""
for other in other_points:
dominated = True
strictly_worse = False
for i in maximize_indices:
if point[i] > other[i]:
dominated = False
break
if point[i] < other[i]:
strictly_worse = True
if dominated:
for i in minimize_indices:
if point[i] < other[i]:
dominated = False
break
if point[i] > other[i]:
strictly_worse = True
if dominated and strictly_worse:
return True
return False
def compute_pareto_frontier(points, maximize_indices=[0], minimize_indices=[1]):
"""Compute Pareto frontier from array of points."""
pareto = []
points_list = list(points)
for i, point in enumerate(points_list):
others = points_list[:i] + points_list[i+1:]
if not is_dominated(point, others, maximize_indices, minimize_indices):
pareto.append(point)
return np.array(pareto)
Example: Model Selection
import pandas as pd
from paretoset import paretoset
# Results from model training experiments
results = pd.DataFrame({
'accuracy': [0.95, 0.92, 0.90, 0.88, 0.85],
'inference_time': [150, 120, 100, 80, 60],
'batch_size': [32, 64, 128, 256, 512],
'hidden_units': [512, 256, 128, 64, 32]
})
# Filter by minimum accuracy threshold
results = results[results['accuracy'] >= 0.85]
# Compute Pareto frontier (maximize accuracy, minimize inference time)
mask = paretoset(results[['accuracy', 'inference_time']], sense=["max", "min"])
pareto_frontier = results[mask].sort_values('accuracy', ascending=False)
# Save to CSV
pareto_frontier.to_csv('pareto_models.csv', index=False)
Visualization
import matplotlib.pyplot as plt
# Plot all points
plt.scatter(results['inference_time'], results['accuracy'],
alpha=0.5, label='All models')
# Highlight Pareto frontier
plt.scatter(pareto_frontier['inference_time'], pareto_frontier['accuracy'],
color='red', s=100, marker='s', label='Pareto frontier')
plt.xlabel('Inference Time (ms) - minimize')
plt.ylabel('Accuracy - maximize')
plt.legend()
plt.show()
Properties of Pareto Frontiers
- Trade-off curve: Moving along the frontier improves one objective while worsening another
- No single best: All Pareto-optimal solutions are equally "good" in a multi-objective sense
- Decision making: Final choice depends on preference between objectives
