A/B test plan
OfficialOfficial · updated Nov 4, 2026 · v1.0.0
Writes an A/B test brief: hypothesis, primary and guardrail metrics, MDE, sample size and duration with the math, stop rules, and a decision rule set in advance.
Sign up free to add this skill to your pod
You come back here after you sign in, and add it with one click.
When to use it
- Before launching an A/B test on a page, flow, email, or price
- Someone asks how long to run a test or how many users it needs
What you give it
- The change to test and the reason
- Primary metric baseline
- Weekly traffic in the test
What you get
A one-page experiment plan with hypothesis, metrics table, sample size math, duration, stop rules, a ship/don't-ship rule, and risks.
Changelog
- v1.0.0 · Nov 4, 2026 · First release.
Sources
- How Not To Run an A/B Test - Evan Miller (2026-11-04)
- Power Analysis - Statsig Docs (2026-11-04)
- Diagnosing Sample Ratio Mismatch in A/B Testing - Microsoft Research (2026-11-04)
Related skills
- Feature prioritization with RICE Scores a backlog with RICE or ICE, shows the math per item, tests sensitivity, recommends a cut line for capacity, and flags dependencies.
- Spec review before build Reviews a PRD or spec before build for ambiguity, missing empty/loading/error/permission states, metrics, rollout, and a first-time user walk-through.
- Landing page copy review Reviews landing page copy with 6 concrete tests (headline, so-what, opposite, proof, CTA, one job per section) and returns ranked line edits.
- Monthly KPI summary for leadership Writes a monthly KPI summary: a headline, each KPI vs last month and target with the reason, and 3 things to act on. Never invents numbers.