Back to skill

Security audit

California

Security checks for vulnerabilities and agentic risk

Overview

This California guidance skill is mostly coherent, but it needs review because its privacy instructions permit sensitive local storage and broader location sharing than the main disclosure says.

Install only if you are comfortable with the skill keeping California planning notes locally. Do not ask it to save credentials, SSNs, payment details, immigration-status details, or exact street addresses, and confirm the destination before it sends address or case-specific information to any government, school, utility, insurance, or local portal.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (2)

T09 · Insecure Skill Coding Practices

Error
Location
setup.md:58
Finding

Plaintext Persistent Storage of Highly Sensitive User Data Is Explicitly Permitted

Content
View full analysis

Vulnerability Details

File Location: setup.md:58-65
Vulnerability Type: Sensitive data stored in an unprotected persistent Markdown file
Risk Level: High

Relevant Code:

markdown
Keep `~/california/memory.md` lightweight and useful:
- activation preference for California topics
- current mode and target region
- major deadlines, open loops, and dependencies
- persistent family, school, housing, transit, and hazard constraints
- which official portals or local agencies already matter for this user

Do not store credentials, account numbers, SSNs, full street addresses, immigration-status details, or payment details unless the user explicitly asks for that behavior.

The behavior also conflicts with the security claim in SKILL.md:116-121:

markdown
**Data that stays local:**
- Region preference, move timeline, family constraints, vehicle notes, and open tasks in `~/california/`

**This skill does NOT:**
- Submit government forms on the user's behalf without explicit instruction
- Store credentials, SSNs, or payment information in local memory

Technical Analysis

The setup instructions permit credentials, account numbers, Social Security numbers, full street addresses, immigration-status details, and payment details to be written to ~/california/memory.md whenever the user explicitly requests it. The memory file is an ordinary Markdown document, and the Skill specifies no encryption, restrictive filesystem permissions, secret-store integration, retention period, access review, or secure deletion procedure.

User consent does not make plaintext secret storage secure. Sensitive values persisted in this form may be exposed to other local users or processes, malware, backup software, cloud synchronization, diagnostic collection, or later Agent sessions. The conflicting statement in SKILL.md may additionally give users the inaccurate impression that these data classes can neve ...[truncated 1312 chars]

Remediation
View remediation

Remediation Suggestions

  1. Replace the exception with an absolute prohibition against storing credentials, authentication tokens, SSNs, account numbers, payment data, immigration-status details, and other high-impact identifiers.
  2. Do not treat user consent as sufficient authorization to place secrets in a plaintext memory file.
  3. Store only coarse, non-sensitive preferences necessary for California-related continuity.
  4. If a legitimate feature ever requires secret retention, use an operating-system credential manager or dedicated encrypted secret store rather than Markdown.
  5. Create allowed memory files with owner-only permissions, such as mode 0600 on supported systems, and verify ownership before reading or updating them.
  6. Define retention and deletion procedures, including a user-facing command to inspect and permanently remove stored memory.
  7. Make setup.md, memory-template.md, and SKILL.md consistent about prohibited data classes.
  8. Before every write, show the categories to be stored and obtain confirmation without reproducing sensitive values in logs or confirmation messages.

T09 · Insecure Skill Coding Practices

Warning
Location
SKILL.md:94
Finding

Incomplete Endpoint Disclosure Allows Location Data Transmission to Unlisted Services

Content
View full analysis

Vulnerability Details

File Location: SKILL.md:94-108; related behavior in sources.md:19-37
Vulnerability Type: Incomplete external endpoint allowlist and insufficient consent for location disclosure
Risk Level: Medium

Relevant Code from SKILL.md:94-108:

markdown
## External Endpoints

| Endpoint | Data Sent | Purpose |
|----------|-----------|---------|
| https://www.ca.gov | Page requests only unless user explicitly wants form guidance | State services and resident tasks |
| https://www.dmv.ca.gov | Page requests only unless user explicitly provides case details | Driver license, REAL ID, registration, and smog workflows |
| https://www.cdtfa.ca.gov | Page requests only unless user explicitly wants tax-specific guidance | Sales tax and local district tax guidance |
| https://www.ftb.ca.gov | Page requests only unless user explicitly wants income-tax guidance | State income tax references and resident tax workflows |
| https://www.calfire.ca.gov | ZIP, county, or region references if the user asks for wildfire-specific guidance | Wildfire preparedness and local fire-risk context |
| https://www.earthquake.ca.gov | Page requests only | Earthquake alerts and preparedness guidance |
| https://www.insurance.ca.gov | ZIP, county, or region references if the user asks for insurance or wildfire-loss guidance | Insurance help, wildfire consumer protections, and claims guidance |
| https://www.coveredca.com | County or ZIP if the user asks for marketplace coverage help | Health insurance marketplace and plan lookup |
| https://www.cde.ca.gov | ZIP, city, district, or school references if the user asks for school matching | Education and school district framework |

No other data is sent externally.

Related Code from sources.md:19-37:

markdown
- CAL FIRE: https://www.fire.ca.gov/
- Earthquake Warning California: https://www.earthquake.ca.gov/
- California Department of Insurance: 
...[truncated 3004 chars]
Remediation
View remediation

Remediation Suggestions

  1. Replace the contradictory endpoint descriptions with one authoritative and exhaustive network policy.
  2. Add every approved domain to an explicit allowlist, including documented aliases such as fire.ca.gov, or remove references that are not approved.
  3. Define how county, city, district, and utility portals are validated before use; do not authorize arbitrary local endpoints.
  4. Require explicit, per-transmission confirmation before sending a full address or case-specific information. The confirmation should identify the destination hostname and exact data fields.
  5. Default to coarse location data such as city, ZIP, or county and disclose a full address only when it is strictly necessary.
  6. Require HTTPS, validate the final hostname after redirects, and prohibit redirects to domains outside the approved allowlist.
  7. Avoid transmitting unrelated conversation context, persistent-memory contents, identifiers, or sensitive case details as part of a lookup.
  8. Update the Security and Privacy section to accurately describe every data category that may leave the machine and every class of recipient.
  9. Document retention and privacy implications when a government or third-party portal requires user data.
Vulnerability Patterns
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Findings (5)

Vague Triggers

High
Category
Not specified by scanner
Confidence
96% confidence
Finding

The activation rule 'whenever the user mentions California at all' is overly broad and can cause the skill to engage in conversations where California is only incidental. That increases the chance of unnecessary data collection, persistence prompts, or context steering outside the user’s actual intent, which is a scope/privacy problem even though the skill later includes some restraint language.

Content

No source excerpt is available for this finding.

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
75% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · SKILL.md (reported line 88)May include surrounding context.

md
## Common Mistakes

- Comparing homes without checking the exact district and daily route shape.
- Assuming city name equals school access.
- Giving family-friendly advice without childcare, parking, and commute reality.
- Treating every suburb in a metro as equally good for families.

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
75% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · family-and-schools.md (reported line 39)May include surrounding context.

md
## Common Mistakes

- Comparing homes without checking the exact district and daily route shape.
- Assuming city name equals school access.
- Giving family-friendly advice without childcare, parking, and commute reality.
- Treating every suburb in a metro as equally good for families.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The file’s stated use covers a wide range of topics—renting, buying, property fit, wildfire insurance pressure, renter insurance, and homeowner decision support—which can cause the skill to activate for many loosely related housing or insurance prompts. Over-broad invocation increases the chance of unintended routing, where users seeking narrow legal, financial, or insurance advice may receive generalized guidance from this skill beyond its intended scope.

Content

No source excerpt is available for this finding.

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
75% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · transit-and-commutes.md (reported line 24)May include surrounding context.

md
- Peak-hour commute, parking search, transit transfer, bridge toll, or school drop-off chain all change the answer.

### 2. Parking is part of housing cost
- Never compare units without asking whether parking is included, possible, safe, and realistic.

### 3. Car-free claims need proof
- California has pockets where a car-free life works well, but many users hear "walkable" and still end up trapped by errands or school runs.