Testing Strategy

Deep testing strategy workflow—risk mapping, test pyramid, levels of isolation, flakiness, data, CI gates, and quality signals beyond coverage %. Use when designing test approach, fighting flaky CI, or restructuring QA vs dev ownership.

Testing Strategy (Deep Workflow)

Testing strategy answers: what failures would hurt users, what’s cheap to catch, and what signals we trust in CI. Coverage percentage alone is a weak proxy—risk alignment matters.

When to Offer This Workflow

Trigger conditions:

• New service or major refactor; “what should we test?”
• Flaky CI, long runtimes, or tests nobody trusts
• Debate: unit vs integration vs e2e; QA headcount vs automation

Initial offer:

Use six stages: (1) risk & quality goals, (2) pyramid & layers, (3) design per layer, (4) data & environments, (5) CI & gates, (6) observability of test health. Confirm release cadence and regulatory needs.

---

Stage 1: Risk & Quality Goals

Goal: Connect tests to user impact and business risk.

Questions

1. Worst failure categories: payments wrong, data leak, outage, wrong advice (AI)?
2. SLO for critical paths—what must never break silently?
3. Change velocity—how fast must PRs merge safely?

Output

Risk register → test priorities (not every line equally important).

Exit condition: Top 5 risks have explicit test intent.

---

Stage 2: Pyramid & Layers

Goal: Many fast tests, some integration, few e2e—proportion tuned to risk.

Layers (typical)

• Unit: pure logic, cheap, deterministic
• Integration: DB, queue, real dependencies in containers—slower but valuable
• Contract: between services—consumer-driven contracts when decoupled teams
• E2E: full stack—expensive; minimal happy path + critical regressions

Anti-patterns

• E2E-only (slow, flaky)
• Mock everything (misses real integration bugs)

Exit condition: Written policy: what belongs in each layer for this codebase.

---

Stage 3: Design Per Layer

Goal: Tests are readable, stable, and debuggable.

Unit

• Given/when/then clarity; avoid testing implementation details
• Property-based tests for tricky invariants (dates, money, parsers)

Integration

• Testcontainers or docker-compose in CI; migrations applied
• Parallel safe—unique DB schemas or transactions

E2E

• Stable selectors (data-testid); retry policy disciplined—fix flakes, don’t hide them
• Seed data minimal; idempotent setup

Exit condition: Flake classification process exists (quarantine + ticket).

---

Stage 4: Data & Environments

Goal: Representative data without PII leakage.

Practices

• Fixtures versioned; factories for variations
• Anonymized prod-like datasets for perf tests—governance for access
• Env parity: staging behaves like prod enough for meaningful e2e

Exit condition: Data generation documented; secrets not in tests.

---

Stage 5: CI & Gates

Goal: Fast feedback on PRs; nightly heavier suites if needed.

Tiers

• PR: lint, unit, fast integration subset
• Main: full integration; optional e2e against ephemeral env
• Release: smoke + canary in prod

Metrics

• Flake rate, duration, quarantined tests count—visible

Exit condition: Merge policy tied to green checks; exceptions process defined.

---

Stage 6: Test Health & Culture

Goal: Tests are owned like features.

Practices

• Ownership per suite; on-call for CI when org size supports
• Delete tests that don’t pay rent—or fix them

---

Final Review Checklist

• Risks mapped to test layers
• Pyramid policy documented
• Flake management process exists
• CI tiers match team velocity
• Data/fixture strategy safe and maintainable

Tips for Effective Guidance

• Recommend testing seams: boundaries where contracts are stable.
• Warn against snapshot abuse for large UI—diff noise kills trust.
• For AI/LLM, discuss eval harnesses beyond classic unit tests.

Handling Deviations

• Legacy untestable code: characterization tests then refactor seams.
• Startup speed: smoke + critical path first; expand as pain appears.