Which LLM? Deterministic model selection for agents

Deterministic decision-ranking API with HTTP 402 payments and outcome credits.

Which‑LLM: Outcome‑Driven Decision Optimizer

Overview

Use this skill when you need a deterministic recommendation for which LLM to use under explicit constraints such as cost and quality.

This skill is for calling the Which‑LLM API. It does not run an LLM itself. For paid requests, the AI bot needs access to a crypto wallet.

When to use it

• Pick the cheapest model that still meets a quality target
• Choose a fallback model if the preferred one fails
• Keep model selection deterministic and auditable
• Report execution results and earn credits for later requests

Quick Reference

• API base URL: https://api.which-llm.com
• Primary paid endpoint: POST /decision/optimize
• Outcome endpoint: POST /decision/outcome
• Free discovery endpoints: GET /capabilities, GET /pricing, GET /status
• Payment asset: USDC
• Supported chains: Base, Ethereum, Arbitrum, Optimism, Avalanche

Registry Summary Expectations

• Primary credential: WALLET_CREDENTIALS
• Install mechanism: instruction-only
• Writes to disk on install: false
• Downloads code on install: false

Prerequisites

Before using this skill, have the following ready:

• Ability to make HTTPS requests to https://api.which-llm.com
• Ability to send and receive JSON
• AI bot access to a crypto wallet for paid requests

What this skill does

• Sends requests to the Which‑LLM API
• Uses POST /decision/optimize to get a recommended model and fallback plan
• Uses POST /decision/outcome to report real execution results
• Explains the request and response shapes for the public API
• Describes how to retry requests after payment has been satisfied externally

What this skill does not do

• It does not call an LLM directly
• It does not execute arbitrary code from your prompt
• It does not require unrelated files or system access beyond API usage and payment handling

Runtime Scope

• always_on: false: the skill is not force-installed and does not run continuously
• disable_model_invocation: false: an agent may invoke it on demand
• The skill is limited to outbound HTTP API usage and response handling
• For paid requests, the AI bot needs access to a crypto wallet

Payment Model

Paid calls may use HTTP 402 Payment Required.

High-level flow:

1. Call POST /decision/optimize
2. If the API returns 402, inspect fields such as required_amount, accepts, and payment_reference
3. Satisfy the payment requirement outside this skill using host- or client-managed infrastructure
4. Retry the same request with payment proof headers if available

This skill documents the API behavior and payment-related response handling. The wallet requirement is limited to paid requests.

Endpoints

GET /capabilities

Use this to discover supported constraints, decision types, and payment behavior.

GET /pricing

Use this to check current pricing and supported chains before making a paid request.

GET /status

Use this for service-health checks.

POST /decision/optimize

This is the main endpoint. Send the goal and constraints, then receive:

• recommended_model
• fallback_plan
• decision metadata and explainability fields

Typical request shape:

json

{
  "goal": "Summarize customer feedback emails into a 5-bullet executive summary",
  "constraints": {
    "min_quality_score": 0.8,
    "max_cost_usd": 0.01
  },
  "workload": {
    "input_tokens": 1200,
    "output_tokens": 300,
    "requests": 1
  },
  "task_type": "summarize"
}

If payment is required, the API may first return 402 with fields such as:

• required_amount
• currency
• accepts[].chain
• accepts[].pay_to
• payment_reference

Retry the request after external payment handling with:

• X-Payment-Chain
• X-Payment-Tx
• X-Payer
• X-Payment-Amount
• X-Payment-Asset

If you have a valid credit token, also send:

• X-Credit-Token

POST /decision/outcome

Use this after running the recommended model. Report what actually happened so the system can issue a credit token for future use.

Typical request shape:

json

{
  "decision_id": "d4e5f6a7-b8c9-0d1e-2f3a-4b5c6d7e8f90",
  "option_used": "openai/gpt-4o-mini",
  "actual_cost": 0.008,
  "actual_latency": 650,
  "quality_score": 0.86,
  "success": true
}

Typical response includes:

• status
• decision_id
• outcome_hash
• refund_credit.credit_token

Request Strategy For Agents

• Call GET /capabilities or GET /pricing first if you need to discover current payment behavior
• Use POST /decision/optimize only when you actually need model selection help
• Reuse the returned decision data rather than repeatedly asking the same question
• After running the chosen model, call POST /decision/outcome to earn credits
• Use the host or client payment flow when a paid request requires wallet-backed settlement

Troubleshooting

PAYMENT_REQUIRED

The endpoint needs payment first. Read the 402 response, satisfy the payment requirement externally, then retry with payment proof headers if available.

PAYMENT_INVALID

Check:

• exact amount sent
• correct chain
• correct recipient
• confirmed transaction
• headers match the actual transaction

NO_FEASIBLE_OPTIONS

Your cost and quality constraints are too strict for the available models. Relax the budget or quality threshold and retry.

RATE_LIMIT_EXCEEDED

Back off and retry later. Use an idempotency key for safe retries.