Token Cost Optimization

Use Cases

User mentions token savings, API cost optimization, prompt compression, cache strategy, model downgrade, cost analysis.

Quick Start

Token Calculator

Run the calculation script, input conversation scale, and quickly estimate current token consumption and optimization potential:

python scripts/token_calculator.py

The script will prompt for:

• Number of conversation history items / average length
• Model and pricing used
• Current optimization status

Output: Current cost, optimized cost, savings percentage.

Three-Tier Optimization Strategy

Ranked by effect / implementation cost:

Tier	Strategy	Effect	Implementation Cost
L1	Prompt compression & output truncation	10-30%	Low
L2	Conversation summary caching	30-50%	Medium
L3	Model downgrade + task routing	50-70%	High

Priority Recommendation: Implement in order L1 → L2 → L3, verifying results at each stage before proceeding.

Detailed strategies, configuration guides, and pitfalls → See references/tier-strategies.md

Phased Implementation Guide

Phase 1: L1 Compression (Immediate Effect)

• Clean up redundant descriptions in system prompt
• Set max_tokens limits for long responses
• Remove outdated/unused messages from conversation history

Phase 2: L2 Caching (1-3 Days)

• Establish FAQ shortcuts for high-frequency repeat questions
• Add summary compression at the beginning of conversations (execute every N rounds)

Phase 3: L3 Routing (1-2 Weeks)

• Route simple tasks to cheaper models (e.g., 4o-mini / Haiku)
• Retain strong models for complex tasks
• Configure model routing rules

Quantifiable Comparison Example

See the "Quantified Comparison" section in references/tier-strategies.md for details.