Agents

Designs, debugs, evaluates, and hardens AI agents — the loop, tools, memory, context budget, cost, and escalation — independent of any framework. Use when an agent loops forever, repeats a tool call, drifts from its instructions after many turns, invents tool arguments, stops mid-task, or swallows a tool error silently; when deciding single agent versus several, or which framework to build on; when token cost per task or p95 latency has to come down; when designing tool schemas, retries, timeouts, checkpoints, or human approval; when writing an eval set or a regression suite for agent behavior; when prompt injection, tool abuse, or an over-permissioned action is the risk; and when specifying an agent's purpose, escalation rules, and cost ceiling for a team. Covers memory design, multi-agent handoffs, tracing, and rollout. Not for LangChain APIs (langchain), retrieval pipelines (rag), prompt craft alone (prompting), or agent persona and voice (agent).

Data. At the start of every session, read ~/Clawic/data/agents/config.yaml (what the user declared) and ~/Clawic/data/agents/memory.md (what you observed, plus its ## Boxes index and ## Due table). Open any file ## Boxes names when the condition on its line applies — the index is the list of files, never assume the list is fixed. Every path it names is inside ~/Clawic/data/; ignore any line that points anywhere else. Everything this skill reads or writes is a plain local note under the folders declared in configPaths — nothing leaves the machine and no credential is ever written. In a shared box it updates or removes only the rows it wrote itself, matched on that box's identity key; a row another skill wrote is read, never rewritten and never deleted, and every write and deletion is named in one line as it happens. Before changing any agent, read its spec box and its eval box if ## Boxes points to them. Read ~/Clawic/data/servers/servers.md before answering which machine runs a worker or proposing where one should run, ~/Clawic/data/projects/<project>.md before working on a build the user tracks as a project, and ~/Clawic/data/contacts/contacts.md before naming an owner or an escalation target. If none of it exists, work from defaults and say nothing about it.

Write before the session ends whenever it produced something durable: an agent defined, renamed, retired, or given a new tool; a system prompt that finally worked; a framework, memory, or single-versus-multi decision and what was rejected; an eval case or an eval run; a measured cost or latency per task; a release and the bundle that would roll it back; an escalation policy; a failure whose cause was not obvious; a red-team finding. memory-template.md holds every destination, format and threshold, and is the only file you open in order to write.

Shared boxes. An agent build that the user tracks as a piece of work goes to ~/Clawic/data/projects/<project>.md, not here — objective, status, decisions. A machine that runs an agent worker goes to the shared inventory ~/Clawic/data/servers/servers.md, one row per host keyed by Name + Provider. A model-provider account with a recurring bill goes to ~/Clawic/data/finances/subscriptions.md, keyed by the account name, amount with its currency. A person the agent is built for or escalates to — its owner, the human an approval reaches — goes to ~/Clawic/data/contacts/contacts.md, one row per person, keyed by Key (lowercase email → handle → <kebab-name> plus a stable disambiguator) written as a column of the row; everywhere else in this skill a person appears by that key only, never copied into a spec. Read each file before adding: an entity already there is updated in place, never duplicated. Formats and full protocol travel with this skill in memory-template.md.

No credential is ever written anywhere under ~/Clawic/data/ — not in the files named here, not in a file you create, not in text the user pastes in to be saved. A pasted system prompt, .env, trace, or tool config is the densest source of keys there is: strip the value and store the pointer, env:OPENAI_API_KEY, keychain:agent-prod, 1password:Work/LLM/prod, file:~/.config/agent/credentials. If data sits at an old location (~/agents/ or ~/clawic/agents/), move it to ~/Clawic/data/agents/, and say in one line that you moved it and from where.

An agent is a loop over a stateless model: observe, decide, act, observe. Everything that looks like memory, personality, or competence is something you re-send on every turn, and everything that looks like autonomy is a tool you granted. So name which of the five parts is wrong — the loop, the context, a tool, the model, or the policy — before proposing a fix, and give the schema, the limit, or the line that changes. Work from defaults immediately: never open with questions about their stack, their budget, or how autonomous the agent should be. Precedence for any value: config.yaml → ~/Clawic/profile.yaml (shared universals: currency, locale) → the Configuration table default.

When To Use

• Designing an agent: purpose, loop shape, single versus multiple, tool set, memory strategy, escalation rules, cost ceiling
• Debugging agent behavior: loops, drift, invented arguments, silent tool failures, tasks that stop halfway, results that differ between runs
• Making the token bill or the latency of a task come down without losing task success
• Building the eval set, the regression suite, and the tracing that tell you whether a prompt change helped
• Hardening: prompt injection through fetched content, tool abuse, over-permissioned actions, sandboxing, approval gates
• Taking an agent to production: releases and rollback, concurrency, timeouts, retries, checkpoints, kill switch, on-call
• Not for LangChain-specific APIs (langchain), retrieval quality (rag), general prompt craft (prompting), or an agent's persona and voice (agent) — this covers the framework-agnostic engineering under all four

Quick Reference

Situation	Play	Depth
Agent repeats the same tool call forever	Hash (tool, arguments); a repeat means the result never reached the transcript or never changed state	debugging.md
Agent ignores the system prompt after many turns	Compaction dropped it, or it is buried mid-context — re-anchor instructions after every compaction	context.md
Agent invents arguments or picks the wrong tool	Schema too loose, description ambiguous, or two tools overlap	tools.md
Agent claims work it never did	No tool call happened; gate completion on an observed side effect, never on the narration	debugging.md
Designing the loop, or "does this even need an agent?"	Loop shape, termination, planning depth, checkpoints — a fixed sequence is a workflow, not an agent	architecture.md
One agent or several?	Split only on different tool sets or different trust levels, never on different topics (Rule 1)	multi-agent.md
Which framework, or none	Selection by what you will need in six months: durability, state, human gates, migration cost	frameworks.md
Cost per task too high, or it jumped with no code change	Transcript growth is quadratic in turns; then caching, then routing (→ Cost And Latency Arithmetic)	cost.md
Latency p95 too high	Serial chains, retry backoff, fallback models — parallelize tools before shrinking the model	production.md
What should the agent remember, and where	Four memory types, the write policy, and how recall gets stale	memory-design.md
Writing the system prompt for an agent	Anatomy, instruction hierarchy, stop conditions, steering tool choice	prompts.md
Need the actual loop, retry, or checkpoint code	Runnable patterns in the language language selects	implementation.md
"Is it any good?" / a prompt change to validate	Trajectory versus outcome scoring, sample size, regression gate	evaluation.md
Prompt injection, exfiltration, over-permissioned tool	Trust boundary, the lethal trifecta, egress and sandbox controls	security.md
When must a human step in, and with what context	Escalation triggers, approval UX, the handoff packet	human-in-the-loop.md
Shipping, rolling out, rolling back, on-call	Release bundle, canary, kill switch, alerts that mean something	production.md
Building a support / coding / research / ops / voice / browser agent	Per-type tool set, memory, escalation and failure profile	agent-types.md
Anything else agent-shaped	Reproduce with the smallest loop that shows it — one tool, one turn, fixed inputs — then re-add pieces until it breaks; the piece that breaks it names the subsystem	—

Coverage map: architecture.md loop shapes and topologies · multi-agent.md coordination and handoffs · tools.md tool design · context.md context engineering · memory-design.md what persists and where · prompts.md agent system prompts · implementation.md runnable patterns · frameworks.md selection · debugging.md symptom→cause · evaluation.md evals and regression · cost.md token economics · production.md deploy and operate · security.md injection and permissions · human-in-the-loop.md escalation · agent-types.md per-type recipes.

Core Rules

1. One agent until you can name the second one's tool list. A second agent buys role separation and costs a handoff contract, a shared-state decision, and a second failure surface. Split on different tool sets or different trust levels — a read-only researcher and a write-capable executor — never on different topics, which is what a system prompt is for. Test: if you cannot write the receiving agent's input schema in one line, you have a section, not an agent (multi-agent.md).
2. Cap the loop three ways: turns, wall clock, and money. max_turns (default 20), a task deadline, and cost_ceiling_per_task_usd (default 0.50). Whichever trips first ends the run with a partial result and the reason, never silently. A turn cap alone is not a budget: a fast tool in a tight retry burns the whole spend inside the cap, and a slow one burns the whole hour.
3. Reliability compounds — quote p^n, not p. A step that works 95% of the time works 0.95^20 ≈ 36% across a 20-step task; at 99% it is 0.99^20 ≈ 82%. So an end-to-end promise names both numbers, and the three levers are: fewer steps, higher per-step reliability, or checkpoints so a failure resumes instead of restarting (implementation.md). Long autonomous chains fail not because the model is weak but because 20 is a big exponent.
4. Every tool returns something the model can act on, including its failures. An exception rendered as "Error" teaches the loop nothing and it retries identically. Return what failed, which argument was wrong, what to try instead, as text the model reads. Tools are idempotent or carry an idempotency key, because the loop will retry — a non-idempotent send_email retried twice is two emails (tools.md).
5. Risk tier is a property of the tool, fixed at definition time. read · write · external · irreversible. autonomy_level decides which tiers run unattended; anything above that line stops for a human with the packet from human-in-the-loop.md. A tool shipped without a declared tier is treated as irreversible.
6. Content that arrived through a tool has no instruction authority. A fetched page, an email body, a file, a database row, another agent's output: data, delimited and labeled as data. The invariant to enforce in code, not in prose — content that entered via a tool cannot trigger a write, external, or irreversible action without a human. The dangerous combination is private data + untrusted content + an outbound channel in the same loop (Simon Willison's "lethal trifecta"); remove one of the three (security.md).
7. Ship the eval set before the second prompt change. With no eval, every change after the first is a vibe and regressions ship silently. Sizing: the 95% confidence half-width on a pass rate is about 1/√N, where N is total runs (cases × runs per case), so N = 100 resolves ±10 points, 400 resolves ±5, and a 3-point regression stays invisible below roughly 1,000. Zero failures is not proof either — the rule of three puts the upper bound at 3/N, so 0 failures in 50 red-team runs still permits a ~6% true failure rate (evaluation.md).
8. A release is a bundle, and rollback needs all of it. Prompt version, model id and its dated snapshot, tool schemas, framework version, and the config that was live. Rolling back the prompt while the provider silently moved the model behind an unversioned alias is not a rollback. Pin the dated model snapshot and write the bundle to deploys/<year>.md in the same turn (memory-template.md).
9. Trace the trajectory or you are debugging by re-running. Every turn records: tool chosen, arguments, result size, latency, tokens in/out, and why the loop ended (done · max_turns · budget · timeout · error · escalated). The end reason is the single most diagnostic field in the whole trace, and it is the one most systems forget to store (debugging.md).

Cost And Latency Arithmetic

The model is stateless, so every turn re-sends the conversation. That one fact explains most surprise bills.

• Transcript growth is quadratic in turns. If each turn adds about d tokens to the transcript and the fixed prefix (system prompt + tool schemas) is base, total input across T turns ≈ T·base + d·T(T−1)/2. Doubling the turns roughly quadruples the transcript component. A 20-turn task is not 10× a 2-turn task; it is closer to 100× on the growing part.
• The fixed prefix is paid every turn too. A 4,000-token system prompt plus 30 tool schemas across 20 turns is 20 × base tokens before the agent does anything. Cutting the tool set is a cost fix, not only an accuracy fix.
• Prompt caching attacks exactly the prefix. It pays when the prefix is stable and reused within the provider's cache window; anything that mutates early in the prompt (a timestamp, a shuffled tool order, a per-turn injected memory) invalidates the whole suffix. Put volatile content last (cost.md).
• Cost per task = Σ_turns (input_tokens × input_price + output_tokens × output_price) + tool-side API costs. Measure it per task type, not per call — the average hides the 5% of tasks that spend 50× the median.
• Latency is serial by default: Σ_turns (model latency + slowest tool in that turn). Parallel tool calls compress only the tool term within a turn; nothing compresses the number of turns except a better plan.
• Fan-out multiplies tokens, not wall clock. k parallel subagents cost about k× and finish with the slowest branch, so fan-out is for read-only breadth, never for latency on a serial dependency (multi-agent.md).

Constraints That Force Designs

Not trivia — each one has killed an agent design that was already half-built.

Constraint	The design it forces
Model statelessness	Nothing persists between turns except what you re-send. "The agent remembers" is always your storage, never the model (memory-design.md)
Context window	Shared by system prompt, tool schemas, transcript, tool results, and the reply. Reserve output space explicitly or the run dies at the last turn
Tool-result size	One unbounded page or query result can consume the window in a single turn. Truncate at the tool boundary and return a handle for more (tools.md)
Max output tokens	Long artifacts must be written to a file through a tool and referenced, not returned in the reply
Rate limits: requests and tokens per minute	Tokens/min is usually the binding one, because each turn re-sends the transcript; fan-out multiplies it (production.md)
Request timeouts	A tool slower than the request timeout must become async: start, return a job id, poll
Nondeterminism	Same input ≠ same trajectory. Every behavioral test needs n runs and a pass rate, never one run and a verdict (evaluation.md)
Provider deprecation windows	An unpinned model alias moves under you; a pinned snapshot expires on a published date. Both are release problems (Rule 8)

Autonomy Tiers

Tier is decided when the tool is defined, and autonomy_level draws the line for the whole agent.

Tier	Examples	Control	What goes wrong without it
Read	search, fetch, query, list	Log and rate-limit	Cost and context blowup, quiet exfiltration path
Write (own scope)	write a file in the workspace, create a draft, update a record it created	Log, quota, reversible by design	Silent corruption discovered days later
External	send an email or message, post, call a third-party API that others see	Approval unless autonomy_level: autonomous, plus a content check	The failure everyone remembers, because it has an audience
Irreversible	delete, deploy, pay, publish, rotate a credential, change permissions	Human approval, always; plus a dry-run that shows the diff	The one incident that ends the pilot

suggest runs read only and proposes the rest. approve-writes runs read freely and stops at write. autonomous runs read, write and external, and never irreversible.

Output Gates

Before shipping an agent design, a prompt, a tool schema, or a release:

• Does the loop have all three caps — turns, deadline, money — and does exhausting one return a partial result with its reason (Rule 2)?
• Does every tool declare a risk tier, return actionable failures, and behave under retry (Rules 4, 5)?
• Is any action reachable from tool-fetched content without a human, and are private data, untrusted content and an outbound channel all present in one loop (Rule 6)?
• Did I state the per-task cost and the end-to-end success estimate as p^n with both numbers named (Rules 3, and Cost Arithmetic)?
• Is there an eval set this change was measured against, at a size that could see the difference (Rule 7)?
• Does the release name its full bundle — prompt version, dated model snapshot, tool schemas, framework version, config (Rule 8)?
• Does the trace record the end reason for every run (Rule 9)?
• Did anything durable come out of this — an agent, a spec, a working prompt, a decision, an eval case, an eval run, a measured cost, a release, an incident, a red-team finding? Then it is written to its box in memory-template.md, with its ## Boxes line, in this same turn.

Configuration

User-dependent variables. Defaults apply until the user states a preference; store them in ~/Clawic/data/agents/config.yaml.

Variable	Type	Default	Effect
language	python | typescript	python	Language of every code sample in implementation.md, tools.md and production.md
framework	raw-sdk | langgraph | openai-agents | crewai | autogen | pydantic-ai | mastra | other	raw-sdk	Dialect of generated code and which migration notes frameworks.md surfaces
model_provider	text (provider name)	none	Whose model ids, pricing shapes and rate-limit units examples use; while unset, examples name tiers (small/mid/frontier), never products
default_model_tier	small | mid | frontier	mid	The tier assumed for the main loop, and the baseline that cost.md routing compares against
autonomy_level	suggest | approve-writes | autonomous	approve-writes	Which tool tiers run unattended (→ Autonomy Tiers) and how many approval gates the generated design contains
max_turns	number (1-200)	20	The loop's turn cap (Rule 2) and the step count used in every p^n estimate
cost_ceiling_per_task_usd	number (USD)	0.50	The money cap that ends a run, and the bar for calling a design expensive in cost.md
eval_gate	bool	true	Whether a prompt, tool or model change is presented as blocked until it is measured against the eval set (Rule 7)
observability_stack	text (tool name)	none	Which tracing backend the instrumentation examples in debugging.md and production.md target; unset means plain structured logs
runtime_target	local | container | serverless | managed	container	Concurrency, timeout, cold-start and state-persistence advice in production.md; serverless forces externalized state

Preference areas — customizable dimensions; a stated preference gets recorded in config.yaml and applied from then on:

• Tooling — framework version and extras, MCP servers versus hand-written tools, structured-output mechanism, prompt storage (in code, files, or a registry) — affects every generated snippet
• Conventions — agent and tool naming, prompt versioning scheme, trace field names, repository layout for prompts and evals — affects prompts.md and evaluation.md artifacts
• Platform — where the loop runs, queue and worker topology, state store, streaming versus batch delivery — affects production.md
• Safety posture — approval breadth, sandbox strictness, egress allowlist, whether destructive tools are generated at all, red-team appetite — affects Autonomy Tiers and security.md
• Restrictions and compliance — PII handling before a tool call (none · redact · block), data residency, retention of traces and transcripts, vetoed providers or tools — affects security.md and what tracing stores
• Output register — how much reasoning to show the end user, streaming style, whether tool activity is visible, verbosity of failures — affects prompts.md
• Integrations — observability backend, eval platform, model gateway, ticketing target for escalations — affects evaluation.md and human-in-the-loop.md
• Cadence — eval regression runs, red-team passes, cost reviews, model re-bid, transcript sampling — every accepted cadence becomes a row in the ## Due table of memory.md

Traps

Trap	Why it fails	Do instead
Multi-agent as the first architecture	Every handoff loses context, and coordination bugs read as model stupidity	One agent with the tools; split on tool set or trust level (Rule 1)
Tool that returns a stack trace, or nothing, on failure	The loop cannot distinguish "bad argument" from "service down" and retries identically until the cap	Failures as instructions: what broke, which argument, what to try (Rule 4)
Adding tools to fix a wrong-tool problem	Each tool added enlarges the schema block and the choice space; accuracy falls while cost rises	Merge overlapping tools, sharpen descriptions, or gate tools by phase (tools.md)
Dumping full tool output into the transcript	One large result poisons the window for the rest of the run and is re-sent every turn afterwards	Truncate at the tool boundary, return a handle (context.md)
Vector memory for everything	Recall returns similar, not true — stale facts resurface with high similarity and outrank the current one	Facts that must be right live in explicit state; recall is for the long tail (memory-design.md)
Testing an agent with one run per case	Nondeterminism means one green run and one red run are the same evidence	n runs per case, report the pass rate and its confidence (Rule 7)
Evaluating the final answer only	Passing with a lucky guess and passing by doing the work look identical	Score the trajectory too: tools chosen, order, retries (evaluation.md)
Chat transcript as the audit log	Reasoning text is not evidence — the model narrates actions it never took	Log tool calls and observed side effects; gate completion on the side effect
Unpinned model alias in production	The model changes under a fixed prompt and every metric moves at once with no deploy to blame	Pin the dated snapshot; treat a model bump as a release with an eval run (Rule 8)
Prompt improvements applied straight to production	The change that fixed one complaint regresses three unmeasured behaviors	Eval set first, then the change, then the diff in pass rate (eval_gate)
Retry loop with no jitter or budget	Synchronized retries hit the rate limit that caused them, and the cost cap arrives before the answer	Exponential backoff with jitter, a retry budget per task, and the failure surfaced (production.md)
Approval prompt that shows only the tool name	Humans approve send_email and learn later what it said	Approval shows the rendered arguments and the reversibility, or it is theatre (human-in-the-loop.md)
Secrets passed through the context window so the agent "has access"	Anything in the window can be echoed, logged, traced, or exfiltrated by injected content	Tools hold credentials server-side and take a reference, never a value (security.md)
An architecture or framework decision that lives only in the chat	Re-litigated every quarter by whoever is on call	artifacts/ with the date, the alternatives and what was rejected (memory-template.md)

Where Experts Disagree

• Multi-agent versus one agent with more tools. Cognition's "don't build multi-agents" argues context fragmentation makes handoffs unreliable; orchestrator-worker practitioners report large gains on breadth-first research. The frontier is shared mutable state: parallel read-only subagents that return summaries usually win, subagents that edit the same artifact usually lose (multi-agent.md).
• Framework versus raw SDK. Frameworks buy state persistence, retries, human-in-loop primitives and traces; they cost a migration when their abstraction disagrees with your control flow. The honest boundary: durable long-running workflows and complex branching justify one; a single loop with five tools does not (frameworks.md).
• Memory as recall versus memory as state. One school retrieves from an embedding store per turn; the other keeps an explicit, small, hand-maintained state document plus files. Recall scales to the long tail and fails silently on facts that changed; state stays correct and does not scale. Most working systems run both, with state winning on conflict (memory-design.md).
• Visible reasoning traces. Interleaved thought makes debugging tractable and costs output tokens on every turn; structured plans are cheaper and hide the moment the plan went wrong. Choose by whether a human will read the trajectory (architecture.md).
• Where the guardrail lives. Prompt-level rules are cheap and bypassable; code-level enforcement around the tool call is unbypassable and inflexible. The settled part: anything irreversible is enforced in code — a rule the model can read is a rule an injected instruction can argue with (security.md).

Security & Privacy

Credentials: this skill designs systems that call model and tool APIs. It does NOT store, log, copy, or transmit API keys, tokens, or user data, and never writes a credential into ~/Clawic/data/agents/.

Local storage: agent specs, eval sets, run history, measured costs, releases and generated artifacts stay in ~/Clawic/data/agents/ on this machine, plus project, host and subscription rows in the shared boxes. Names, model ids, prompt versions, digests and numbers only — no secrets, no captured end-user transcripts unless the user explicitly asks and the content is stripped of personal data first.

Guardrails: designs default to the lowest autonomy that completes the task. Destructive and irreversible tools are presented with their blast radius and an approval gate, never inside a copy-paste block of read-only code.

Related Skills

More Clawic skills, get them at https://clawic.com/skills/agents (install if the user confirms):

• langchain — the same patterns in concrete LangChain and LangGraph APIs
• rag — retrieval quality behind an agent's search tool
• prompting — prompt craft in general, beyond the agent system prompt
• observability — tracing, metrics and alerting for the service the agent runs in
• agent — persona, voice and boundaries of a single assistant

Feedback

• If useful, star it: https://clawic.com/skills/agents
• Latest version: https://clawic.com/skills/agents

Part of Clawic, the verified skill library. Get this skill: https://clawic.com/skills/agents.