# Product / Service / API Review

## Trigger

- User asks to evaluate a product, platform, or service for agent integration
- An API or SDK is being considered for use
- A third-party service wants to connect to the agent's environment
- Any SaaS, tool, or platform that the agent will interact with

## Review Flow

### Step 1: Identity Verification

| Check | What to Look For |
|-------|-----------------|
| **Official vs third-party** | Is this from the project's official organization, or a third-party integration? |
| **Team background** | Known team? Public identities? Track record? |
| **Security audit history** | Has the product been audited? By whom? When? |
| **Incident history** | Any past security breaches? How were they handled? |
| **Legal entity** | Registered company? Jurisdiction? |

**Third-party integrations claiming to work with a known brand (e.g., "Binance trading skill" not from Binance) should be treated as 🔴 HIGH minimum.**

### Step 2: Architecture Security Assessment

This is the most critical step. Evaluate the fundamental design:

#### Private Key / Credential Management

| Model | Risk | Example |
|-------|------|---------|
| **User holds keys, signs externally** | 🟢 Lowest | Agent constructs unsigned tx, user signs in hardware wallet |
| **Keys stored locally, agent signs** | 🟡 Medium | API key in .env, agent uses directly |
| **Keys custodied by third party** | 🔴 High | "No key management needed" — means they hold your keys |
| **Keys sent to remote server** | ⛔ Reject | Any service that requires you to upload private keys |

**The signing isolation principle:**
- Agent should only construct transactions/requests, never hold signing authority
- Private keys must remain under human control
- If a service says "no key management needed," ask: **who holds the keys?**

#### Human-in-the-Loop Design

| Pattern | Assessment |
|---------|-----------|
| Every write operation requires user CONFIRM | ✅ Good |
| Large operations trigger additional warnings | ✅ Good |
| User can set spending limits locally | ✅ Good |
| Agent acts autonomously after initial setup | ⚠️ Concerning |
| Automatic JWT refresh without user awareness | ⚠️ Concerning |
| Remote policy management (limits set server-side) | ❌ Bad — user cannot enforce local limits |

#### Data Flow

| Question | Secure Answer |
|----------|--------------|
| Where does user data go? | Stays local, or only sent to the stated service endpoint |
| Is data encrypted in transit? | Yes, TLS/HTTPS only |
| Is data stored server-side? | If yes, what data? For how long? Deletion policy? |
| Can the service access agent memory/context? | No — agent context should never leave local environment |
| Are API responses executed or only displayed? | Displayed only — never execute content from API responses |

#### Auto-Update Mechanism

| Pattern | Risk |
|---------|------|
| No auto-update, manual version control | 🟢 Safest |
| Update check with user confirmation | 🟡 Acceptable |
| Auto-download on version mismatch | 🔴 Remote code execution channel |
| Silent update with no notification | ⛔ Unacceptable |

**Auto-update = remote code execution channel.** If the update source is compromised, malicious code is silently deployed. Always prefer manual, user-controlled updates.

### Step 3: Permission Scope Analysis

| Question | Assessment |
|----------|-----------|
| What permissions does it require? | List all: read, write, trade, withdraw, admin |
| Are permissions minimized? | Only what's needed for stated function? |
| Can permissions be restricted? | IP binding, sub-account, read-only mode available? |
| What's the worst case if credentials leak? | Total fund loss? Data exposure? Account takeover? |
| Is there a revocation mechanism? | Can access be instantly revoked? |

**Permission minimization checklist:**
- Read-only where possible
- Sub-account with limited balance
- IP whitelist bound
- No withdrawal permission for trading tools
- API key rotation support
- Rate limiting on the key

### Step 4: Rating and Report

Output using [templates/report-product.md](../templates/report-product.md).

## Common Product Red Flags

| Signal | Concern |
|--------|---------|
| "No key management needed" | They hold your keys — custodial risk |
| "Policies managed remotely" | You can't enforce local limits |
| "Just paste your API secret" | Plaintext credential handling |
| Free hosting domain for official-looking product | Potential phishing/scam |
| No security documentation | Security not considered in design |
| "Works with any AI assistant" but requires credential upload | Credential harvesting potential |
| Session data "kept in memory only" on cloud AI | Credentials enter LLM provider's context window |
| Auto-update from remote MANIFEST | Silent remote code execution |