prompt-ab-lab

Design, log, compare, and score prompt experiments so users can systematically improve outputs instead of guessing.

Security Scan

The skill's requested resources, instructions, and bundled script are coherent with its stated purpose (prompt A/B testing); it uses a local, auditable Python helper and declares no unexpected credentials or network installs.

Purpose & Capability

Name/description match the delivered assets: an instruction-only skill plus a small Python helper to generate CSV logs. Declared runtime dependency python3 is appropriate and sufficient for the bundled script.

Instruction Scope

SKILL.md confines actions to designing experiments, generating rubrics, and optionally using the local script and resource file. It does not instruct reading unrelated system files, accessing hidden env vars, or sending data to external endpoints.

Install Mechanism

No install spec is present (instruction-only). The only code is a small, local, readable Python script; there are no remote downloads, package installs, or extraction steps.

Credentials

No environment variables, credentials, or config paths are required. The skill's functionality does not demand any secrets or external service access.

Persistence & Privilege

always:false (default) and no post-install actions or modifications to other skills are present. The skill does not request elevated or persistent platform privileges.

Assessment

This skill appears low-risk and internally consistent. Before installing or invoking: (1) confirm python3 is on PATH; (2) review scripts/prompt_experiment_logger.py (it's small and local) and run it manually on a sample JSON to see outputs; (3) when using the skill in an agent session, be conscious that the agent could run the local script and write files to the working directory—only allow that if you trust the session context; (4) if you need network isolation, ensure the agent runtime does not have outbound network access (the skill itself does not require it).

Purpose

Design, log, compare, and score prompt experiments so users can systematically improve outputs instead of guessing.

Trigger phrases

• 比较两个提示词
• prompt ab test
• 提示词实验
• 哪个 prompt 更好
• 建一个评测表

Ask for these inputs

• prompt A and B
• task
• evaluation criteria
• test set
• weights if any

Workflow

1. Define what success looks like before comparing prompts.
2. Generate an evaluation rubric and structured test table.
3. Log outputs per test case and compute weighted scores.
4. Summarize tradeoffs instead of declaring a winner too early.
5. Recommend the next experiment iteration.

Output contract

• experiment plan
• scored comparison table
• rubric
• next-iteration suggestions

Files in this skill

• Script: {baseDir}/scripts/prompt_experiment_logger.py
• Resource: {baseDir}/resources/eval_rubric.md

Operating rules

• Be concrete and action-oriented.
• Prefer preview / draft / simulation mode before destructive changes.
• If information is missing, ask only for the minimum needed to proceed.
• Never fabricate metrics, legal certainty, receipts, credentials, or evidence.
• Keep assumptions explicit.

Suggested prompts

• 比较两个提示词
• prompt ab test
• 提示词实验

Use of script and resources

Use the bundled script when it helps the user produce a structured file, manifest, CSV, or first-pass draft. Use the resource file as the default schema, checklist, or preset when the user does not provide one.

Boundaries

• This skill supports planning, structuring, and first-pass artifacts.
• It should not claim that files were modified, messages were sent, or legal/financial decisions were finalized unless the user actually performed those actions.

Compatibility notes

• Directory-based AgentSkills/OpenClaw skill.
• Runtime dependency declared through metadata.openclaw.requires.
• Helper script is local and auditable: scripts/prompt_experiment_logger.py.
• Bundled resource is local and referenced by the instructions: resources/eval_rubric.md.