Hermes Agent Health Check

Audit a NousResearch/hermes-agent checkout or fork for Hermes-specific runtime-contract drift, command-surface splits, memory/skill/gateway health, and agent...

Audit the architecture and health of a Hermes Agent checkout, fork, or deployment support repo.

Hermes Agent has a connected runtime: agent loop, command registry, CLI, TUI, gateway, skills, memory, cron, tools, plugins, and terminal environments. hermescheck helps keep those surfaces aligned.

When to Use

• You are preparing a Hermes Agent PR and want a repeatable architecture review
• A Hermes fork works in CLI but not gateway, TUI, skills, cron, or plugins
• A new slash command risks drifting across surfaces
• A tool or environment change needs clearer capability boundaries
• Memory, session search, or skill behavior regressed after a refactor
• Startup paths or background jobs became hard to reason about

Quick Start

pip install hermescheck
hermescheck /path/to/hermes-agent

Produces audit_results.json and audit_report.md.

The 12-Layer Stack

#	Layer	What Goes Wrong
1	System prompt	Conflicting instructions, instruction bloat
2	Session history	Stale context from previous turns
3	Long-term memory	Pollution across sessions
4	Distillation	Compressed artifacts re-entering as pseudo-facts
5	Active recall	Redundant re-summary layers wasting context
6	Tool selection	Wrong tool routing, model skips required tools
7	Tool execution	Hallucinated execution — claims to call but doesn't
8	Tool interpretation	Misread or ignored tool output
9	Answer shaping	Format corruption in final response
10	Platform rendering	UI/API/CLI mutates valid answers
11	Hidden repair loops	Silent fallback/retry agents running second LLM pass
12	Persistence	Expired state or cached artifacts reused as live evidence

Audit Scanners

#	Scanner	Severity	What It Catches
1	Hardcoded Secrets	critical	API keys, tokens, credentials in source code
2	Tool Enforcement Gap	high	"Must use tool X" in prompt but no code validation
3	Hidden LLM Calls	high	Secret second-pass LLM calls in fallback/repair loops
4	Unrestricted Code Execution	critical	exec(), eval(), subprocess(shell=True) without sandbox
5	Static Bug Inference	high	Code-level bug patterns inferred without runtime execution
6	Token Usage Budget	high	Large default context windows, full-history prompts, missing thrift controls
7	Memory Lifecycle Governance	medium	Memory without types, lifecycle, retrieval budgets, decay, or evidence pointers
8	RAG Pipeline Governance	medium	Retrieval without chunk, top-k, rerank, ingestion, or context budget controls
9	Self-Evolution Capability	high	Learning loops without external signals, source reading, constraint fit, safe landing, or verification
10	Loop Safety Budget	high	Tool/agent loops without max-iteration, retry budget, stuck-job, or duplicate-call controls
11	Plugin / Remote Tool Boundary	high	Executable plugins and MCP/OpenAPI tools without sandbox, schema, allowlist, or approval boundaries
12	Output Pipeline Mutation	medium	Response transformation corrupting correct answers
13	Missing Observability	medium	No tracing, logging, cost tracking, or audit trail

Severity Model

Level	Meaning
critical	Agent can confidently produce wrong operational behavior
high	Agent frequently degrades correctness or stability
medium	Correctness usually survives but output is fragile or wasteful
low	Mostly cosmetic or maintainability issues

Fix Strategy

Default fix order (code-first, not prompt-first):

1. Code-gate tool requirements — enforce in code, not just prompt text
2. Remove or narrow hidden repair agents — make fallback explicit with contracts
3. Reduce context duplication — same info through prompt + history + memory + distillation
4. Tighten memory admission — user corrections > agent assertions
5. Tighten distillation triggers — don't compress what shouldn't be compressed
6. Reduce rendering mutation — pass-through, don't transform
7. Convert to typed JSON envelopes — structured internal flow, not freeform prose

Report Schema

Reports follow a formal JSON Schema (see references/report-schema.json) with:

• overall_health: critical_risk | high_risk | medium_risk | low_risk
• findings: array of severity-ranked issues with evidence refs
• maturity_score: positive signal ledger, penalty ledger, score formula, and expected recovery directions
• ordered_fix_plan: prioritized fix steps with rationale

Anti-Patterns to Avoid

• ❌ Saying "the model is weak" without falsifying the wrapper first
• ❌ Saying "memory is bad" without showing the contamination path
• ❌ Letting a clean current state erase a dirty historical incident
• ❌ Treating markdown prose as a trustworthy internal protocol
• ❌ Accepting "must use tool" in prompt text when code never enforces it

Related

• GitHub: https://github.com/huangrichao2020/hermescheck