Testing Strategy

Deep testing strategy workflow—risk mapping, test pyramid, levels of isolation, flakiness, data, CI gates, and quality signals beyond coverage %. Use when de...

Security Scan

Instruction-only workflow guidance about testing strategy; nothing in the package attempts to access credentials, run code, or perform actions outside its stated purpose.

Purpose & Capability

The name and description match the SKILL.md content: a human-oriented testing strategy workflow. There are no unexpected requirements (no binaries, env vars, or config paths).

Instruction Scope

SKILL.md contains high-level guidance and checklists only. It does not instruct the agent to read files, call external endpoints, access secrets, or execute commands beyond advising test practices.

Install Mechanism

No install spec and no code files are present; nothing will be downloaded or written to disk by installation.

Credentials

The skill requests no environment variables, credentials, or config paths. Mentions of anonymized prod-like data are advisory and do not translate into required access.

Persistence & Privilege

always is false and the skill does not request persistent privileges or modify agent/system configuration. Autonomous invocation is allowed by default but not problematic here because the skill is read-only guidance.

Assessment

This is a guidance-only skill (no code, no installs, no credentials). It's internally consistent and low-risk. Before using, verify it aligns with your org's policies (especially around suggestions to use prod-like datasets — ensure proper governance and anonymization). As with any guidance, adapt recommendations to your stack and don't execute any commands or share secrets based solely on the document.

Testing Strategy (Deep Workflow)

Testing strategy answers: what failures would hurt users, what’s cheap to catch, and what signals we trust in CI. Coverage percentage alone is a weak proxy—risk alignment matters.

When to Offer This Workflow

Trigger conditions:

• New service or major refactor; “what should we test?”
• Flaky CI, long runtimes, or tests nobody trusts
• Debate: unit vs integration vs e2e; QA headcount vs automation

Initial offer:

Use six stages: (1) risk & quality goals, (2) pyramid & layers, (3) design per layer, (4) data & environments, (5) CI & gates, (6) observability of test health. Confirm release cadence and regulatory needs.

---

Stage 1: Risk & Quality Goals

Goal: Connect tests to user impact and business risk.

Questions

1. Worst failure categories: payments wrong, data leak, outage, wrong advice (AI)?
2. SLO for critical paths—what must never break silently?
3. Change velocity—how fast must PRs merge safely?

Output

Risk register → test priorities (not every line equally important).

Exit condition: Top 5 risks have explicit test intent.

---

Stage 2: Pyramid & Layers

Goal: Many fast tests, some integration, few e2e—proportion tuned to risk.

Layers (typical)

• Unit: pure logic, cheap, deterministic
• Integration: DB, queue, real dependencies in containers—slower but valuable
• Contract: between services—consumer-driven contracts when decoupled teams
• E2E: full stack—expensive; minimal happy path + critical regressions

Anti-patterns

• E2E-only (slow, flaky)
• Mock everything (misses real integration bugs)

Exit condition: Written policy: what belongs in each layer for this codebase.

---

Stage 3: Design Per Layer

Goal: Tests are readable, stable, and debuggable.

Unit

• Given/when/then clarity; avoid testing implementation details
• Property-based tests for tricky invariants (dates, money, parsers)

Integration

• Testcontainers or docker-compose in CI; migrations applied
• Parallel safe—unique DB schemas or transactions

E2E

• Stable selectors (data-testid); retry policy disciplined—fix flakes, don’t hide them
• Seed data minimal; idempotent setup

Exit condition: Flake classification process exists (quarantine + ticket).

---

Stage 4: Data & Environments

Goal: Representative data without PII leakage.

Practices

• Fixtures versioned; factories for variations
• Anonymized prod-like datasets for perf tests—governance for access
• Env parity: staging behaves like prod enough for meaningful e2e

Exit condition: Data generation documented; secrets not in tests.

---

Stage 5: CI & Gates

Goal: Fast feedback on PRs; nightly heavier suites if needed.

Tiers

• PR: lint, unit, fast integration subset
• Main: full integration; optional e2e against ephemeral env
• Release: smoke + canary in prod

Metrics

• Flake rate, duration, quarantined tests count—visible

Exit condition: Merge policy tied to green checks; exceptions process defined.

---

Stage 6: Test Health & Culture

Goal: Tests are owned like features.

Practices

• Ownership per suite; on-call for CI when org size supports
• Delete tests that don’t pay rent—or fix them

---

Final Review Checklist

• Risks mapped to test layers
• Pyramid policy documented
• Flake management process exists
• CI tiers match team velocity
• Data/fixture strategy safe and maintainable

Tips for Effective Guidance

• Recommend testing seams: boundaries where contracts are stable.
• Warn against snapshot abuse for large UI—diff noise kills trust.
• For AI/LLM, discuss eval harnesses beyond classic unit tests.

Handling Deviations

• Legacy untestable code: characterization tests then refactor seams.
• Startup speed: smoke + critical path first; expand as pain appears.