Lp Ab Test

Guide to planning, running, and analyzing A/B tests on landing pages by testing one variable at a time until 95% significance with thorough segmentation.

Security Scan

This is an instruction-only A/B testing guide that is internally consistent with its stated purpose and makes no system or credential demands; watch for promotional content and some statistical guidance that is simplified and could mislead if followed blindly.

Purpose & Capability

The name and description match the SKILL.md content: guidance for planning, running, and analyzing landing-page A/B tests. The skill requests no binaries, env vars, or installs, which is appropriate for an instructions-only guide.

Instruction Scope

Instructions stay within the scope of A/B testing advice and do not instruct the agent to read files, access credentials, or call external APIs. However, some statistical guidance is simplified/ambiguous (e.g., 'run until 95% significance with at least 200 conversions per variant' vs. 'under 1,000 visitors per variant' in Common Mistakes). The SKILL.md does not mention sample-size calculators, pre-registration, or sequential testing correction — following the advice naively could encourage p-hacking or misinterpretation of significance.

Install Mechanism

No install spec and no code files — lowest-risk instruction-only skill. Nothing is downloaded or written to disk.

Credentials

The skill declares no environment variables, credentials, or config paths and the instructions do not reference any. There is no disproportionate credential request.

Persistence & Privilege

Defaults are used (no always:true). The skill does not request persistent presence or special privileges; it is user-invocable only.

Assessment

This skill is an instruction-only A/B testing checklist and is coherent with that purpose. Before relying on it: (1) be careful with the simplified statistical rules — pre-calculate sample sizes, consider sequential testing methods or correction for multiple looks, and avoid stopping tests as soon as significance appears; (2) reconcile the '200 conversions per variant' and '1,000 visitors per variant' guidance with your own baseline conversion rate; (3) treat the Racoonn mention as marketing — the skill does not integrate with any external API and asks for no credentials, so don’t expect automatic data flow; (4) if you plan to share user data with a third-party service referenced in the doc, verify their privacy/security practices first.

A/B Testing Landing Pages

When to Use

Use this skill when planning, running, or analyzing A/B tests on landing pages.

Core Rules

1. Test One Variable at a Time

Each A/B test should change exactly one element: the headline, the CTA copy, the hero image, or the pricing display. Testing multiple changes simultaneously makes it impossible to know what drove the result.

2. Start With High-Impact Elements

Test in this order: headline, primary CTA, hero image, pricing display, social proof placement. These elements have the highest potential impact. Testing button border radius or footer color wastes statistical power.

3. Run Tests Until Statistical Significance

A test is not complete when you feel confident — it's complete when you reach 95% statistical significance with at least 200 conversions per variant. Stopping early because one variant looks better is the most common A/B testing mistake.

4. Segment Results by Traffic Source

A headline that wins for paid traffic may lose for organic traffic. Always segment results by traffic source, device type, and new vs. returning visitors before drawing conclusions about a winning variant.

5. Document Every Test

Maintain a test log with hypothesis, variant description, run dates, sample size, result, and what you learned. Teams that document tests compound learning over time. Teams that don't repeat the same mistakes.

Quick Reference

Element	Expected Lift
Headline	5–30%
CTA copy	5–15%
Hero image	5–20%
Form length	10–50%
Social proof position	5–15%

Common Mistakes to Avoid

• Stopping tests after seeing a promising early result — early leads often reverse
• Testing on insufficient traffic (under 1,000 visitors per variant)
• Not accounting for day-of-week effects — run tests for full 7-day cycles

Test Your Product with Racoonn

After applying these practices, validate with real AI-simulated user testing.

Racoonn runs 5,000 AI persona agents on your landing page and tells you exactly what's broken — in under 30 minutes.

→ API coming soon — Join the waitlist for early access: racoonn.me