Prompts

Deep prompt engineering workflow—task spec, constraints, examples, evaluation sets, iteration protocol, regression testing, and safety alignment. Use when improving LLM outputs, shipping prompt changes, or building reusable prompt templates.

Prompt Engineering (Deep Workflow)

Prompts behave like natural-language programs: they need specs, tests, and version control—especially in production.

When to Offer This Workflow

Trigger conditions:

• Prompt or system message change; quality regressions
• Structured outputs (JSON), tool use, or RAG grounding requirements
• Safety or policy alignment needs

Initial offer:

Use six stages: (1) define task & success, (2) constraints & format, (3) few-shot & style, (4) build eval set, (5) iterate with discipline, (6) ship, monitor, regress). Confirm model family and latency budget.

---

Stage 1: Define Task & Success

Goal: Clear user-visible outcome and failure modes (hallucination, omission, tone).

Exit condition: Success rubric in plain language; out-of-scope cases listed.

---

Stage 2: Constraints & Format

Goal: Must/must-not rules; output schema (JSON Schema, bullet structure); length limits.

Practices

• Separate system (policy, role) from user (task instance)
• Ask model to cite sources when grounding matters

---

Stage 3: Few-Shot & Style

Goal: Use examples only when they reduce ambiguity—avoid huge prompt bloat.

Practices

• Diverse examples; avoid overlong negative examples that confuse

---

Stage 4: Build Eval Set

Goal: Frozen inputs with expected properties (not always exact text match).

Practices

• Adversarial and multilingual slices if relevant
• Regression suite in CI for critical prompts

---

Stage 5: Iterate With Discipline

Goal: Change one major variable at a time when debugging quality.

Practices

• Compare with same temperature settings when A/B testing wording
• Log prompt version id with outputs in production

---

Stage 6: Ship, Monitor, Regress

Goal: Canary prompt changes; watch implicit signals (thumbs, edits, task completion).

---

Final Review Checklist

• Task and rubric defined
• Constraints and output format explicit
• Eval set versioned; regression path exists
• Iteration log disciplined; prompt versions tracked
• Production monitoring and rollback plan

Tips for Effective Guidance

• Clarity beats cleverness—short explicit instructions often win.
• Chain-of-thought: use when reasoning helps; hide chain from end users if needed.
• Align with llm-evaluation skill for larger harness design.

Handling Deviations

• Chat vs batch: batch can use stricter structure and lower temperature.
• Multimodal: specify how image details may be used or ignored.