Does my WooCommerce store have enough traffic to A/B test?

How To Implement

• 1 Define the exact WooCommerce surface and the exact primary metric

  Write down whether you are testing a single product template, a category archive, the cart, the checkout, or a sitewide/lading-page surface. If it is cart or checkout, note whether the store is using Cart & Checkout Blocks or the classic shortcode pages, because the implementation surface is different even though the sample-size maths is the same. WooCommerce’s own docs show that Cart and Checkout blocks are edited in the editor, can be transformed to classic shortcodes, and have different extensibility rules from the shortcode flow.

• 2 Check that the metric is actually measured in GA4 and available on the tested surface

  For retail stores, GA4’s ecommerce model supports add_to_cart, begin_checkout, and purchase. If you cannot trust the event, you cannot trust the baseline. Measurement note: if you mark a new event as a key event today, that changes reporting from the time of creation and does not backfill historic data, so do not size a test from a partial or newly-created key event history.

• 3 Pull the real baseline from GA4 using the same denominator your testing setup uses

  For page-entry tests, use the Landing page report and filter to the actual page path or path pattern; GA4 classifies this report as session-scoped and lets you add dimensions such as Session source / medium. For step-specific journeys, build a Funnel exploration in Explore → Funnel exploration. If you need key-event counts by surface, GA4 reports and explorations support Key events and Session key event rate.

• 4 Keep the denominator consistent across GA4 and your test tool

  GA4 exposes both sessionKeyEventRate:event_name and userKeyEventRate:event_name. Nelio, by contrast, documents its test conversion rate as conversions ÷ page views after variant assignment. If you size from a GA4 session-based baseline but judge results in a page-view-based tool metric, you are mixing units and your sample estimate will be wrong. If you cannot align them exactly, note the mismatch in the test brief and use the same denominator all the way through decision-making.

• 5 Choose the smallest relative lift that would genuinely matter to the business

  This is your MDE. Use a relative value, not an absolute percentage-point wish. In practical terms, the question is not “what uplift would be nice?”, but “what is the smallest swing worth shipping, QA’ing and keeping?” VWO’s help explicitly notes that smaller detectable effects require more precision and therefore more sample.

• 6 Calculate the per-variant sample before launch

  For a two-variant equal split, two-sided 5% alpha and 80% power, the standard large-sample two-proportion formula simplifies to the very usable approximation n ≈ 15.7 × (1 − p) / (p × MDE²) per variant, where p is the baseline rate as a decimal and MDE is the relative lift as a decimal. Worked examples:

  • p = 0.02, MDE = 0.10 ⇒ about 76,930 per variant by approximation, which is close to the ~80k territory returned by exact calculators.
  • p = 0.02, MDE = 0.20 ⇒ about 19,233 per variant by approximation, roughly the ~20k rule of thumb.
  • p = 0.10, MDE = 0.10 ⇒ about 14,130 per variant, which shows why add-to-cart tests are often far more feasible than purchase tests.

  You can run this in a standalone calculator such as Evan Miller, or in a vendor calculator such as VWO. VWO’s public calculator is explicit that its main version is tied to its enhanced SmartStats engine and links to a separate classic calculator.

• 7 Convert the sample into a planned run-length and write it down

  Divide required visitors per variant by the page’s expected eligible visitors per variant per day, not by total site traffic. Then write down the sample, the expected duration, and the stop rule in the brief before the test starts. VWO’s calculator and help centre both frame sample size and duration together for exactly this reason.

• 8 If the duration is impractical, redesign the test before you launch

  The usual moves are: test a higher-frequency metric such as add_to_cart or begin_checkout; test a broader surface with more eligible traffic; or accept a larger MDE. What usually does not work is launching a low-baseline purchase test and hoping the maths becomes kinder later.

• 9 Launch with guardrails, not with “wait and see”

  On a normal fixed-horizon design, only read the main result when the pre-set sample is reached or a pre-declared stop condition fires. If you are using Nelio, remember its Required Sample Size and Required Confidence settings are thresholds in the UI, not a substitute for store-specific power planning. If you are using VWO, be explicit whether the analysis is fixed-horizon or sequential/SmartStats.

• 10 QA the split and event firing during the test, but do not call winners early

  If the allocation looks off, treat it as a data-quality problem first. Microsoft’s experimentation team documents sample ratio mismatch as a real failure mode caused by assignment, execution, logging, or biased analysis issues.

How To Measure

Pitfalls

• Mistake: mixing denominators across tools. GA4 can report session-based or user-based key event rates, while Nelio’s test conversion rate is conversions divided by page views. A clean-looking sample estimate can still be wrong if the baseline and the reporting denominator do not match.
• Mistake: chasing a tiny uplift on a low-baseline metric. At a 2% purchase baseline, moving from a 20% MDE to a 10% MDE does not double the sample need; it roughly quadruples it. That is why low-traffic stores often get far more value from testing add_to_cart or begin_checkout first.
• Myth: you can just stop when one variant “looks significant”. In a fixed-horizon test, naïve peeking and early stopping raise the false-positive rate. If you want optional stopping, you need a sequential method that explicitly supports it.
• Myth: if you leave the test running long enough, the tool will eventually reveal the winner. Not necessarily. Long duration does not repair a badly sized test, and VWO explicitly distinguishes between fixed-horizon and sequential approaches because the analysis rules are different.
• Edge case: large GA4 numbers are not always exact counts. Google states that Active Users and Sessions in GA4 are approximated with HyperLogLog++ at scale, so on very large properties, exact exports or BigQuery may be better for baselines where precision matters.

Examples

FAQs

Sources & Further Reading

• Sample Size Calculator — Evan’s Awesome A/B Tools – Update date: Undated page Note: Widely used standalone calculator for fixed-horizon A/B test planning; useful as a neutral cross-check of baseline rate, MDE, power and significance assumptions.
• Understanding Sample Size Calculations — VWO Help Centre – Update date: 28 November 2024 Note: Vendor source that clearly explains why lower MDEs require more sample and how sample size, error rate and power trade off.
• Estimating Your Campaign Duration and Sample Size — VWO Help Centre – Update date: 8 August 2025 Note: Vendor explanation of duration planning and why higher baselines reduce required sample size.
• A/B Test Sample Size & Duration Calculator — VWO – Update date: Current public calculator, page copyright 2026 Note: Useful for planning duration and visitor requirements, with an explicit note that the main calculator is tied to VWO’s enhanced SmartStats and not the classic engine.
• Measure ecommerce — Google Analytics for Developers – Update date: 4 May 2026 Note: Primary GA4 reference for ecommerce events including add_to_cart, begin_checkout, and purchase.
• API Dimensions & Metrics — Google Analytics for Developers – Update date: 4 May 2026 Note: Primary reference for sessionKeyEventRate:event_name, userKeyEventRate:event_name, and the fact that marking an event as a key event does not alter historic data.
• About key events — Analytics Help – Update date: Undated help page Note: Explains how key events appear in reports, how to narrow to a specific event, and where GA4 exposes key-event reporting.
• [GA4] Landing page report — Analytics Help – Update date: Undated help page Note: Confirms the report is session-scoped, shows how to filter to relevant landing pages, and supports secondary dimensions such as Session source / medium.
• [GA4] Funnel exploration — Analytics Help – Update date: Undated help page Note: Primary Google reference for step-based funnel analysis, useful when sizing add-to-cart, checkout-start or checkout-completion tests.
• Customizing the Cart and Checkout Pages — WooCommerce documentation – Update date: Undated documentation page Note: Shows the editor paths for block-based Cart/Checkout pages and how to transform them back to classic shortcodes when relevant.
• Getting started with Cart and Checkout extensibility — WooCommerce developer docs – Update date: Undated developer documentation page Note: Explains that Cart & Checkout Blocks have their own extensibility model and that some shortcode hooks work, but not all.
• High Performance Order Storage — WooCommerce developer docs – Update date: 18 June 2026 Note: Current primary reference for HPOS, including defaults, compatibility implications and the location of incompatible-plugin warnings.

About This Page

• Written By: Eliot Webb – Founder & WooCommerce CRO Consultant
• Last Reviewed: 18 Jun 2026
• Last Updated: 18 Jun 2026