Prompt Engineering (Deep Workflow)

Prompts behave like natural-language programs: they need specs, tests, and version control—especially in production.

When to Offer This Workflow

Trigger conditions:

• Prompt or system message change; quality regressions
• Structured outputs (JSON), tool use, or RAG grounding requirements
• Safety or policy alignment needs

Initial offer:

Use six stages: (1) define task & success, (2) constraints & format, (3) few-shot & style, (4) build eval set, (5) iterate with discipline, (6) ship, monitor, regress). Confirm model family and latency budget.

---

Stage 1: Define Task & Success

Goal: Clear user-visible outcome and failure modes (hallucination, omission, tone).

Exit condition: Success rubric in plain language; out-of-scope cases listed.

---

Stage 2: Constraints & Format

Goal: Must/must-not rules; output schema (JSON Schema, bullet structure); length limits.

Practices

• Separate system (policy, role) from user (task instance)
• Ask model to cite sources when grounding matters

---

Stage 3: Few-Shot & Style

Goal: Use examples only when they reduce ambiguity—avoid huge prompt bloat.

Practices

• Diverse examples; avoid overlong negative examples that confuse

---

Stage 4: Build Eval Set

Goal: Frozen inputs with expected properties (not always exact text match).

Practices

• Adversarial and multilingual slices if relevant
• Regression suite in CI for critical prompts

---

Stage 5: Iterate With Discipline

Goal: Change one major variable at a time when debugging quality.

Practices

• Compare with same temperature settings when A/B testing wording
• Log prompt version id with outputs in production

---

Stage 6: Ship, Monitor, Regress

Goal: Canary prompt changes; watch implicit signals (thumbs, edits, task completion).

---

Final Review Checklist

• Task and rubric defined
• Constraints and output format explicit
• Eval set versioned; regression path exists
• Iteration log disciplined; prompt versions tracked
• Production monitoring and rollback plan

Tips for Effective Guidance

• Clarity beats cleverness—short explicit instructions often win.
• Chain-of-thought: use when reasoning helps; hide chain from end users if needed.
• Align with llm-evaluation skill for larger harness design.

Handling Deviations

• Chat vs batch: batch can use stricter structure and lower temperature.
• Multimodal: specify how image details may be used or ignored.