01Which A/B Testing Tool Should an Online Shop Choose in 2026?
This page is the hub of our tool comparison. It explains how to choose, gives every one of fifteen tools a short profile, and sends you to the deeper pages: the 15-tool deep dive with full fact tables, twelve Apex vs pages, and head-to-head comparisons between third-party tools.
02Which Tool Fits Which Use Case?
| Use case | Pick | Why | Watch out for |
|---|---|---|---|
| Tests keep coming back flat | Apex by DRIP | Likelihood to Win per idea before launch | Price not published; book a call |
| Price, shipping and offer tests on Shopify | Intelligems | Commercial tests read as profit and COGS | Price plans: 3-month minimum, no trial |
| Lower-cost price tests on Shopify | Elevate | Price and shipping tests from $99 a month | No hosting region or DPA published |
| Shopify tests billed on test orders | ABConvert | Nine test types, profit per visitor | Price tests start on Growth, $199 |
| Theme tests in the Shopify editor | Shoplift | Variants built in the Shopify Theme Editor | No checkout, headless or A/B/n tests |
| No-code tests, any shop, flat price | Varify.io | €199 a month, unlimited traffic, Frankfurt | One-sided default statistics; one domain |
| Tests, personalisation and upsells on Shopify | Visually.io | One app, priced by orders from $19 | Price tests need 600+ monthly orders |
| A free native start on Shopify | Shopify Rollouts | Included from Grow, no app to install | No revenue metrics; method not published |
| Warehouse-native or self-hosted testing | GrowthBook | Open source, free to self-host | Needs a data warehouse; no price tests |
| Strict data protection, developer team | ABlyft | Hosting in Germany, no stored personal data | Paid plans on quote; no Shopify app |
| EU hosting, certificates, public prices | Convert Experiences | Frankfurt, ISO 27001, SOC 2 Type 2 | Shopify app has no reviews yet |
| Agents across the whole test cycle | VWO (Wingify) | Agents from research to rollout, approval gates | No public prices; price tests “coming soon” |
| Prompt-built variants inside Wingify | AB Tasty (Wingify) | Evi turns prompts into HTML, CSS, JavaScript | No public prices; plans follow Wingify |
| Enterprise statistics and server-side tests | Optimizely | Sequential Stats Engine, SDKs, certifications | On quote; third-party median $81,447 a year |
| European vendor with agents and flags | Kameleoon | PBX agents, MCP server, server-side flags | From $495 a month; Shopify guide for Plus |
Each pick is a documented strength, not a claim that the tool beats the others overall.
03How Do You Find the Constraint That Should Decide?
Feature tables make fifteen tools look alike: nearly all have a visual editor, a code path, targeting, a results page and, since 2026, some form of AI. What separates them is the problem each was built to remove. Name your problem and the shortlist shrinks to two or three tools.
- Choosing tests. If tests keep coming back flat, the problem sits before the build, and a faster editor only ships more flat tests.
- Building tests. If the backlog is full and little ships, check what a non-developer on your team can build alone.
- Test scope. Prices, shipping fees, discounts and checkout need deeper shop access, and some tools offer them, often on a higher plan.
- Budget and compliance. Both remove tools from a shortlist faster than any feature does, so each has its own section below.
04What Must an A/B Testing Tool Do for an Online Shop?
An A/B testing tool splits visitors at random between the current version and one or more variants and measures what each group does: add to cart, checkout, order, revenue. Before a trial, check these points against your own shop:
- It tests what you need on your shop system and plan. On Shopify, checkout tests usually need Shopify Plus, and price tests often need a higher plan of the tool.
- Results show revenue, and ideally profit, per visitor, not only conversion rate.
- QA covers every device, with a preview link you can share before launch.
- Parallel tests are possible, with exclusion groups when two tests touch the same element.
- A guardrail stops a test that loses money without waiting for someone to look.
- The script fits your page speed budget, and flicker is handled.
- Results can be exported and kept when you leave.
- If you work in ChatGPT or Claude, check what the tool’s MCP server or API may read and change. Most tools in this guide now offer one.
Shopify Rollouts shows where a native tool ends. It tests theme, checkout and customer account changes from the Grow plan, but its experiments report no revenue or AOV, its statistical method is not published, and prices cannot be tested. Those gaps are where dedicated tools start.
05Which Statistics Method Should the Tool Use?
Fixed-horizon tests fix the sample size in advance and read the result once. Bayesian tests report a probability that a variant is better. Sequential tests are built to be watched while they run. Each fails when used as another, most often when a fixed-horizon test is stopped the day it looks good.
The tools in this guide cover the whole range (vendor documentation, checked October 2026). Optimizely’s default Stats Engine is sequential with false discovery rate control. Shoplift, Intelligems, Visually.io and VWO use Bayesian methods. Varify.io defaults to “a frequentist method with one-sided tests”. ABlyft defaults to frequentist tests with a Bayesian option. Convert, Kameleoon and GrowthBook offer frequentist, Bayesian and sequential methods. Elevate declares a winner only when four rules are met. Apex uses fixed-horizon statistics with confidence. Shopify does not publish the method behind Rollouts.
Sample ratio checks catch broken splits, where one variant gets more traffic than it should and the result means nothing. Optimizely, Kameleoon, Convert and GrowthBook document automatic checks; for many Shopify apps we found none.
06Where Does the Tool Keep Your Data?
| Data location | Tools |
|---|---|
| Hosting in Germany | ABlyft |
| Frankfurt | Varify.io (AWS plus Cloudflare), Convert Experiences |
| EU plus UK | Apex by DRIP |
| EU data centers for “the majority” of products | Optimizely |
| EU region for event data, application in the US | GrowthBook Cloud, or self-hosted where you choose |
| United States | Shoplift |
| Transfers to Canada and the US | Intelligems, Visually.io |
| Region not published | Kameleoon, ABConvert, Elevate, Shopify Rollouts |
| Not covered in our research | VWO, AB Tasty |
Certificates differ as much as regions. Optimizely, Kameleoon and Convert document ISO 27001 and SOC 2 Type 2. Varify.io marks SOC 2 as “SOON”, Visually.io says its SOC 2 and GDPR programs are “in the process”, and for Apex, ISO 27001 and SOC 2 are in progress, with no certification yet.
Consent is the third question. ABlyft states “We do not store any personal data. Nothing.” and uses localStorage instead of cookies by default; Varify.io states “In principle, no cookies are set”. Ask every vendor what happens to tests and tracking when a visitor declines, and get the DPA before the contract.
07How Do A/B Testing Tools Price?
| Price model | Tools | Entry point |
|---|---|---|
| Monthly orders per store | Intelligems, Elevate, Visually.io | $69, $49 and $19 a month |
| Test orders only | ABConvert | $99 a month, $74 billed yearly |
| All site visitors | Shoplift | $99 a month, $74 billed annually |
| Tested users | Convert Experiences | $299 a month billed annually |
| Monthly tracked users | Kameleoon, VWO, AB Tasty | From $495 a month; Wingify not published |
| Flat per domain | Varify.io | €199 a month billed annually |
| Per seat | GrowthBook | Free to self-host; Pro $40 per seat |
| Part of the shop plan | Shopify Rollouts | Experiments from Grow, €79 a month yearly |
| On quote | Optimizely, ABlyft paid plans | Not published; both have free plans |
| Not published | Apex by DRIP | Card-free free tier for one store |
Model the bill on your peak month, because plans priced by visitors, orders or tracked users all rise with a good sale.
Read the terms next to the price (checked October 2026). Intelligems’ price plans have a 3-month minimum and no trial. Convert’s annual plans “don’t auto-renew”. Shoplift pauses tests above the visitor limit. Kameleoon sells the code editor, CUPED and bandits as add-ons on Starter. The price of Apex is not published; a card-free free tier for one store exists, and beyond it the route is to book a call.
08Can a Tool Predict Which Tests Will Win Before Launch?
Selection matters because most tests lose: with the industry’s 10 to 20 percent win rate, per Ronny Kohavi, 12 tests a year find about one or two winners.
- Idea generators. Optimizely’s Idea Builder, ABlyft’s Lumen and Wingify’s Hypothesis Agent generate ideas; Intelligems’ Atlas “ranks them by estimated profit impact”.
- Simulators. VWO’s Synthetic Testing Agent tests changes on “synthetic visitors”. Shopify’s SimGym simulates shoppers, with Shopify’s caveat that “results might differ from actual buyer behavior”.
- Scores from past tests. Kameleoon’s Ideate is “Trained on 20,000+ experiments” and claims “63% accuracy on high-confidence predictions” (vendor-stated, method not published). Apex scores ideas against observed A/B tests.
Apex scores each idea against 4.3 million observed A/B tests from 151,000 shops and shows how many tests back each score. The nearest published figure in this comparison is Kameleoon’s Ideate, trained on more than 20,000 experiments; Varify cites 10,000 or more test patterns; no other vendor in this comparison publishes a count of tests behind its recommendations (vendor pages checked October 2026).
The score is called Likelihood to Win: one percent figure per test idea, shown with how many tests back it. In a holdout check within a shop, the top third of ideas won 1.4 to 1.6 times as often (press release, September 29, 2026).
Think of it as a weather forecast: it does not decide the weather, but it tells you whether to pack an umbrella. Whatever tool you look at, ask what evidence ranks an idea and how many tests stand behind it.
09What Does Each of the 15 Tools Do?
Each profile uses the vendor’s own pages, read in October 2026 (VWO and AB Tasty on October 6). Quoted text is the vendor’s wording. Where a vendor publishes nothing, the profile says so. Review ratings, evidence levels and the full capability grid are in the 15-tool deep dive.
Apex by DRIP: best for shops whose tests keep coming back flat
- Best for: shops whose tests keep coming back flat: the constraint is choosing which tests to run, not building them.
- Can test: pages and page elements, parallel tests with optional exclusion groups, A/A tests, redirect tests, global holdout, feature flags and staged rollouts.
- Shop systems: installs as a Shopify app (two-click install via link, server-side tracking, first-party data).
- Data location: hosting in the EU plus UK; ISO 27001 and SOC 2 in progress.
- Entry price: pricing is not published; a card-free free tier for one store exists.
- Main strength: the memory of millions of observed tests behind every idea, so selection improves before anything is built.
- Main limitation: the price is not published.
Three layers: a test memory of 4.3 million A/B tests from 151,000 shops, collected since 2011, with a Likelihood to Win for every idea; a testing tool with heat maps per variant, profit per test and the chat; and managed execution with the DRIP team.
Apex is not the right choice for a team that already picks winning tests well, which gets less from the prediction layer, or for anyone who needs a published price list before a conversation. A shop that cannot ship a test a week yet should build that habit on a lighter tool first; buying prediction before you can execute is the wrong order. More on Apex: What is Apex by DRIP.
Intelligems: best for price, shipping and offer tests on Shopify
- Best for: Shopify merchants testing prices, shipping and offers, read as profit.
- Can test: prices, discounts, bundles, shipping, themes, content, checkout blocks (Plus), post-purchase offers.
- Shop systems: Shopify; headless via an npm package.
- Data location: region not published; transfers “to Canada and the United States”.
- Entry price: Smart Content $69 a month (checked October 2026).
- Main strength: commercial tests with profit and COGS reporting.
- Main limitation: Shopify only; price tests need a team-built integration.
Bayesian statistics, an Atlas agent that builds content tests, and an MCP server. Price plans carry a 3-month minimum without a trial.
Not the right choice for buyers who need EU data documentation up front. Compared in detail: Apex vs Intelligems and Apex vs Shoplift vs Intelligems.
Shoplift: best for theme and template tests on Shopify
- Best for: Shopify brands testing themes in the editor they already use.
- Can test: templates, theme elements, prices on Advanced and Pro; no checkout pages.
- Shop systems: Shopify and Plus; headless “not currently supported”.
- Data location: “transferred to and processed in the United States”.
- Entry price: Core $99 a month, $74 billed annually (checked October 2026).
- Main strength: variants built in Shopify’s own Theme Editor.
- Main limitation: no headless, checkout or A/B/n tests.
Bayesian Probability to Win, with a guideline of “at least 10,000 visitors per variant and 14 days”. Tests pause above the plan’s visitor limit.
Not the right choice for headless stores or checkout tests. Compared in detail: Apex vs Shoplift and Shoplift vs Intelligems.
Varify.io: best for no-code tests on any shop system at a flat price
- Best for: no-code tests at a flat price with unlimited traffic.
- Can test: product, collection, cart, home and landing pages; Plus checkout via a separate app.
- Shop systems: any shop that loads JavaScript.
- Data location: AWS in Frankfurt plus Cloudflare.
- Entry price: Growth €199 a month billed annually, one domain (checked October 2026).
- Main strength: a flat price per domain, unlimited traffic.
- Main limitation: one-sided default statistics; Bayesian needs Pro.
Results come from your own analytics (“We do not have our own tracking.”), and an MCP server can “create, start, pause experiments”.
Not the right choice for sequential testing today or many domains on one plan. Compared in detail: Apex vs Varify and ABlyft vs Varify.io.
VWO (Wingify): best for agents across the whole testing lifecycle
- Best for: agents from research to rollout, with approval gates.
- Can test: pages; Shopify themes via split URL; prices “coming soon”.
- Shop systems: a Shopify app without reviews (September 2026); others not researched.
- Data location: not covered in our research of October 6, 2026.
- Entry price: not published; a third party reports $665 a month (Growth, annual).
- Main strength: one platform of agents with guardrails.
- Main limitation: no public prices; no price, offer or checkout tests live.
VWO and AB Tasty merged in January 2026 as Wingify. Its Wingz agents run the cycle, and “Nothing goes live without approval”.
Not the right choice if you need a public price or a managed service. Compared in detail: Apex vs VWO, VWO vs Optimizely and ABlyft vs VWO.
AB Tasty (Wingify): best for prompt-built variants on the Wingify platform
- Best for: prompt-built variants on the Wingify platform.
- Can test: as VWO; prices “coming soon”.
- Shop systems: not covered in our research.
- Data location: not covered in our research.
- Entry price: not published; Growth, Pro and Enterprise plans.
- Main strength: Evi turns prompts into HTML, CSS and JavaScript.
- Main limitation: no public prices; platforms converge “over time”.
Evi launched in November 2025 and is now part of Wingz; existing contracts stay unchanged after the merger.
The VWO profile covers the shared platform. Compared in detail: VWO vs AB Tasty and Kameleoon vs AB Tasty.
Optimizely: best for enterprise experimentation with deep statistics
- Best for: enterprise tests across web, backend, mobile and edge.
- Can test: the page DOM; server-side via 20+ SDKs; bandits; feature flags.
- Shop systems: any site via snippet; no Shopify App Store testing app.
- Data location: EU data centers for “the majority” of products.
- Entry price: not published; Vendr median $81,447 a year (third-party).
- Main strength: sequential Stats Engine, sample ratio detection, bandits.
- Main limitation: no public pricing; enterprise-sized contracts.
Opal agents added an Idea Builder and a Build agent in 2026, and ISO 27001:2022 and SOC 2 Type 2 are documented. The snippet cannot run in Shopify’s checkout.
Not the right choice for small budgets. Compared in detail: Apex vs Optimizely and ABlyft vs Optimizely.
Kameleoon: best for European enterprise teams that want agents and server-side flags
- Best for: prompt-based building, server-side flags and an MCP server.
- Can test: page elements via snippet; server-side features via flags.
- Shop systems: snippet and SDKs; Shopify guide “exclusively” for Plus.
- Data location: region not stated; ISO/IEC 27001:2022, SOC 2 Type 2.
- Entry price: PBX Starter “Starting from $495/month” (checked October 2026).
- Main strength: client-side tests and server-side flags in one platform.
- Main limitation: Shopify integration documented for Plus only.
PBX agents find ideas, build variations from prompts or Figma, and ship winners behind feature flags. CUPED and bandits are add-ons on Starter.
Not the right choice for Shopify stores below Plus. Compared in detail: Apex vs Kameleoon and ABlyft vs Kameleoon.
ABlyft: best for developer teams with strict data protection needs
- Best for: developer teams wanting German hosting and no stored personal data.
- Can test: page content, redirects, Shopify draft themes and templates.
- Shop systems: any site via snippet; no Shopify App Store app.
- Data location: “Hosting and Storage in Germany”.
- Entry price: free plan; paid plans on quote (checked October 2026).
- Main strength: no stored personal data or IP addresses, no cookies by default.
- Main limitation: native price, discount or checkout tests are not documented.
Made in Kiel for developers: custom code per variation, a Git connection, the Lumen AI and an MCP server.
Not the right choice for teams without developers. Compared in detail: Apex vs ABlyft, ABlyft vs VWO and ABlyft vs Kameleoon.
Shopify Rollouts: best for a free native start on Shopify
- Best for: scheduling and testing changes inside the Shopify admin.
- Can test: theme, checkout, customer accounts, discounts; not prices or Liquid templates.
- Shop systems: Shopify Online Store; no headless storefronts.
- Data location: not published for experiments.
- Entry price: no separate fee; experiments from Grow, €79 a month yearly (checked October 2026).
- Main strength: built into the admin at no listed extra fee.
- Main limitation: conversion-type metrics only; method not published.
Since June 5, 2026 it also covers checkout and customer accounts. On reporting, Shopify states: “You can't customize the available metrics”.
Not the right choice for teams that judge tests on revenue or profit. Compared in detail: Apex vs Shopify Rollouts.
ABConvert: best for Shopify tests billed on test orders
- Best for: growing Shopify brands and CRO teams.
- Can test: price, shipping, offer, redirect, template, theme, visual, checkout, combined.
- Shop systems: Shopify; no headless Shopify.
- Data location: region not published.
- Entry price: Starter $99 a month, $74 billed yearly (checked October 2026).
- Main strength: nine test types, read on profit per visitor.
- Main limitation: Shopify only; price tests start on Growth, $199.
It bills on test orders, “a unique order ABConvert records while a test or personalization runs”, and computes profit per visitor from your own costs.
Not the right choice off Shopify. Compared in detail: Apex vs ABConvert.
Convert Experiences: best for EU hosting with certificates and public prices
- Best for: teams wanting a full tool “without enterprise pricing”.
- Can test: pages, themes, prices via Cart Transform Functions.
- Shop systems: a Shopify app plus WooCommerce, BigCommerce, Magento integrations.
- Data location: Frankfurt, Germany; ISO 27001 and SOC 2 Type 2.
- Entry price: Growth $299 a month billed annually (checked October 2026).
- Main strength: public prices with EU hosting and certificates.
- Main limitation: Shopify app without reviews; checkout prices need Plus.
Frequentist and Bayesian statistics, sequential testing from Pro, sample ratio checks and heat maps; annual plans “don’t auto-renew”.
Not the right choice for checkout price tests without Plus. Compared in detail: Apex vs Convert.
GrowthBook: best for warehouse-native or self-hosted testing
- Best for: engineers, data, product and growth teams.
- Can test: page content, redirects, any code path via flags.
- Shop systems: any site via script or SDK; Shopify theme snippet.
- Data location: self-hosted, or Cloud with event data in Ireland.
- Entry price: free to self-host; Pro $40 per seat a month (checked October 2026).
- Main strength: open source, unlimited traffic.
- Main limitation: no documented Shopify price or checkout tests.
Analysis runs on your own data warehouse, with Bayesian, frequentist and sequential engines and CUPED.
Not the right choice without a data warehouse. Compared in detail: Apex vs GrowthBook and ABlyft vs GrowthBook.
Visually.io: best for testing, personalisation and upsells in one Shopify app
- Best for: e-commerce teams on Shopify.
- Can test: themes, discounts, prices, shipping, carts, checkouts, upsells.
- Shop systems: Shopify and Plus; headless via Hydrogen.
- Data location: transfers “to Canada and the United States”.
- Entry price: from $19 a month, up to 200 orders (checked October 2026).
- Main strength: testing, personalisation and upsells in one app.
- Main limitation: no documented server-side option; SOC 2 “in the process”.
Bayesian statistics and a Global Control Group as holdout. Price tests need 600+ monthly orders on Scale, or Plus with 7,500+.
Not the right choice for smaller shops that want price tests. Compared in detail: Apex vs Visually.io.
Elevate: best for lower-cost price and shipping tests on Shopify
- Best for: Shopify stores, from new stores to teams that need unlimited testing.
- Can test: pages, themes, prices, shipping, offers, checkout (Plus), images.
- Shop systems: Shopify; headless via SDK on Premium.
- Data location: not published.
- Entry price: Basic $49 a month; price tests from Standard, $99 (checked October 2026).
- Main strength: price and shipping tests from $99 a month.
- Main limitation: no published hosting region, DPA or certificate.
Price tests work “on any Shopify plan, including non Plus”, and a winner needs four rules met, including three days of runtime.
Not the right choice if you need a signed DPA first. No Apex vs page yet. Full profile: Elevate in the 15-tool deep dive.
10Where Should You Go Next?
The full comparison
- A/B Testing Tools for Online Shops: 15 Platforms Compared: the deep dive, with full fact tables.
- A/B Testing Software Comparison: the five dimensions that decide.
Apex against one tool
Each Apex vs page compares the documented facts and says where each tool is not the right choice.
- Apex vs Intelligems: prediction against price and offer tests.
- Apex vs Shoplift: prediction against Theme Editor tests.
- Apex vs Shoplift vs Intelligems: all three side by side.
- Apex vs Varify: prediction against flat-price no-code tests.
- Apex vs VWO: a test memory against lifecycle agents.
- Apex vs Optimizely: a shop program against an enterprise platform.
- Apex vs Kameleoon: two ways to score ideas.
- Apex vs ABlyft: prediction against a developer tool.
- Apex vs ABConvert: prediction against nine Shopify test types.
- Apex vs Convert Experiences: prediction against EU hosting and public prices.
- Apex vs GrowthBook: prediction against open source.
- Apex vs Visually.io: prediction against testing plus upsells.
- Apex vs Shopify Rollouts: a dedicated tool against the built-in one.
Third-party tools head to head
- Shoplift vs Intelligems
- VWO vs Optimizely
- Kameleoon vs AB Tasty
- ABlyft vs VWO
- ABlyft vs Kameleoon
- ABlyft vs Varify.io
- ABlyft vs Optimizely
By constraint
- A/B testing tools for Shopify: the Shopify view, including native apps.
- A/B testing tools for enterprise e-commerce: security, scale and total cost.
- A/B testing tools in Europe: tools with documented EU hosting.
- Free A/B testing tools: where free plans stop.
Prediction and Apex
- What is Apex by DRIP?: the three layers, explained.
- Predictive A/B testing: what a prediction can and cannot do.
- AI A/B testing tools: what the AI in each tool does.
Not sure which constraint you have? See if your shop is a fit→




