A/B Testing Tools for Online Shops in 2026: 15 Platforms Compared

01Which A/B Testing Tool Should an Online Shop Pick in 2026?

The quick verdict, one sentence per constraint:

1. If your tests keep coming back flat, your constraint is choosing tests, and Apex by DRIP is built for it: every test idea gets a Likelihood to Win before launch, backed by a database of observed A/B tests.
2. If your backlog is full and little ships, your constraint is building tests, and Varify.io (any shop system, flat price per domain) or Shoplift (Shopify themes, edited in the Shopify Theme Editor) take that work off developers.
3. If you want to test prices, shipping fees and offers on Shopify, Intelligems covers all three with profit and COGS reporting, and ABConvert and Elevate start at lower prices.
4. If budget is the constraint, start with Shopify Rollouts, which is included from the Grow plan, or self-host GrowthBook, which is open source and free.
5. If compliance decides, ABlyft (hosting in Germany, no personal data stored) and Convert Experiences (servers in Frankfurt, ISO 27001, SOC 2 Type 2) document their data handling in detail.

Behind the five constraints sits one pattern. What many shops end up needing is not another editor. It is one place where ideas, tests and results live, so the next test is chosen from what the last ones showed. We come back to that in the section on testing programs.

Every profile carries an evidence level: 1 for product claims documented by the vendor, 2 for third-party documentation or directories, 3 for named case studies with figures, 4 for review aggregators, 5 for figures the vendor states without documentation. A level says how a claim is backed, not whether it is true.

02Which Tool Fits Which Use Case?

03What Are the 15 Tools at a Glance?

The order of this table is the order of the profiles below, not a ranking. Review counts come from different directories, so compare them within one source, not across sources.

04What Decides This More Than Feature Lists?

Feature tables make fifteen tools look alike. What separates them is which problem they were built to remove, so name your constraint before you compare features.

Constraint 1: choosing tests

If tests keep coming back flat, the problem sits before the build: in which ideas reach the backlog and which of them run. A faster editor does not fix that; it ships more flat tests. Tools differ here more than anywhere else: some generate ideas, a few rank them, and Apex shows a prediction per idea backed by a published database of observed tests.

Constraint 2: shipping tests

If the backlog is full and little ships, the constraint is build capacity. Here the editor, the QA tools, the AI builders and the developer path decide. Check in the trial what a person without coding skills on your team can build alone, and what still needs a developer.

Constraint 3: budget

Pricing models differ more than prices: per visitor, per tested user, per order, per seat, per domain, or on quote. A plan that looks cheap at normal traffic can change tier when a sale month lifts sessions or orders. Model the bill on your peak month, not your average one.

Constraint 4: compliance

Where data is processed, which certifications exist and how consent is handled vary widely in this list, from hosting in Germany to processing in the United States. If your legal team needs a DPA, a subprocessor list or an ISO 27001 certificate before signing, several tools drop out. Ask for documents, not badges.

Why selection comes before speed

Win rates show why. Industry data puts the share of experiments that win at 10 to 20 percent, per Ronny Kohavi. In DRIP's own testing, the win rate went from 27 percent in 2024 (808 tests) to 56 percent between April and June 2026 (278 tests). A win rate also depends on the statistics behind it: a loose threshold produces more winners on paper and fewer in revenue. Compare win rates by how they were called, not by the headline figure.

Running more tests at a 15 percent win rate still means that most of them lose, and every losing test costs build time and traffic. Velocity pays off once selection improves, not before.

05What Is an A/B Testing Tool for an Online Shop?

The tool splits visitors at random between a control and one or more variants and records what each group does: add to cart, checkout, order, revenue. Because both groups see the same traffic at the same time, the difference can be attributed to the change, within the limits of the statistics.

For an online shop, the tool has three jobs.

• Choose: decide which ideas to test and in which order. This covers research, the idea backlog, prioritisation and, in some tools, a prediction or a ranking before launch.
• Build and run: create the variant in a visual or code editor, check it on all devices, target the right visitors, split the traffic and launch. Price, shipping and checkout tests need deeper access to the shop system than page tests.
• Read: decide whether the variant won, on which metric and with what confidence. For a shop, revenue per visitor and profit per visitor matter more than clicks, and heat maps per variant explain why a result happened.

Nearly every tool in this comparison is strong at building and running. Fewer cover reading in shop terms such as profit per test, and fewer still help with choosing. That gap is where the difference between a testing tool and a testing program sits.

Where Shopify Rollouts ends and dedicated tools begin

Shopify Rollouts is built into the Shopify admin. From the Grow plan, it schedules and A/B tests changes to the main theme, checkout and customer accounts, discounts and product catalogs, at no listed extra fee.

The limits are documented by Shopify itself. Experiments report conversion-type metrics and no revenue or AOV metrics; the statistical method is not published; prices are not offered as a test; Liquid template changes cannot be part of a rollout; and headless storefronts are not covered. A dedicated tool starts to pay when you need revenue or profit per test, price or shipping tests, a documented statistics method, heat maps per variant or help choosing what to test.

06Why Does the Current Setup Cost More Than It Looks?

A typical shop testing program has four parts that do not talk to each other. An agency or freelancer writes ideas and builds variants. A testing tool splits the traffic. An analytics tool holds the numbers the team trusts. And the backlog lives in a spreadsheet or a project board, with the reasons behind past results in slide decks.

The hidden cost sits between the parts. Results in the testing tool disagree with the analytics tool, so meetings go to reconciling numbers. Learnings sit in decks nobody opens, so ideas get tested twice, and when the agency changes, the history leaves with it.

The cost that adds up is the losing tests. With an industry win rate of 10 to 20 percent, a program of 12 tests a year finds about one or two winners, and a program of 38 tests a year finds about four to eight. Every test that loses still used design time, build time, QA and weeks of traffic that could have run a better idea.

07How Do the 15 Tools Compare, Fact by Fact?

Both tables use the vendors' own pages, read in October 2026 (VWO and AB Tasty on October 6, the rest on October 10). Where a vendor says nothing, the table says so.

How to read the capability grid. Yes: documented by the vendor. Add-on: available for an extra charge. Enterprise: available on the enterprise plan. No: the vendor says it is not supported. Not published: the vendor does not say. None found: we did not find it in the documentation we read, which does not mean it does not exist. For Apex, Not published means the feature is not part of the product description on drip-apex.com that this comparison uses. Column meanings: Price tests covers price and offer tests, Heat maps means heat maps per variant, Prediction means a prediction from past tests, AI builds means AI builds variants, MCP includes any ChatGPT or Claude connection, Auto-stop means a guardrail auto-stop, Cookieless means a cookieless mode.

Notes on cells that need context:

• Apex: tracking runs server-side; the Server-side column asks about server-side test delivery. AI builds refers to the MCP connection, through which a shop owner builds and reads tests from ChatGPT or Claude.
• Shoplift: variants are edited in Shopify's own Theme Editor. Price tests need Advanced or Pro. Lift Assist inserts a styled theme section without content. Claude and ChatGPT connect through an HTTP API; an MCP server was not found.
• Varify.io: AI building uses credits, bought as a package or through your own API key. Server-side delivery and auto-stop are on Enterprise.
• VWO and AB Tasty: our research of October 6, 2026 covered the Wingify agent lifecycle, pricing and the Shopify app. Editors, server-side delivery and data location were not part of it, so those cells read None found. Auto-stop refers to the Guardrail Agent, which reverses breaches.
• Kameleoon: the code editor is an add-on on the Starter plan. Prediction refers to the vendor-stated idea scoring of PBX Ideate, trained on “20,000+ experiments”.
• Shopify Rollouts: discounts can be part of a rollout, prices cannot. SimGym, a separate Shopify app, simulates shoppers before launch; it is trained on commerce sessions, not on past A/B tests.
• GrowthBook: the AI Visual Editor is on Pro and Enterprise. Auto-stop refers to auto-rollback in safe rollouts on Pro.
• Visually.io: price tests need the Scale plan with at least 600 monthly orders, or Shopify Plus with 7,500 or more on Pro.
• Elevate: the AI generates product images for image tests. The MCP server reads data but cannot change experiments.

08What Does Each of the Nine Main Tools Do, in Detail?

Each profile draws on vendor pages, documentation, pricing pages and review directories, read in October 2026. Vendor claims are marked as such.

Apex by DRIP: best for shops whose tests keep coming back flat

• Best for: shops whose tests keep coming back flat: the constraint is choosing which tests to run, not building them.
• Can test: pages and page elements, parallel tests on the same traffic with optional exclusion groups, A/A tests, redirect tests, global holdout, feature flags and staged rollouts.
• Shop systems: installs as a Shopify app (two-click install via link, server-side tracking, first-party data).
• Data location and compliance: GDPR in place; hosting in the EU plus UK; DPAs for US tools; ISO 27001 and SOC 2 in progress, with no certification yet; consent supported for most Shopify GDPR apps.
• Entry price and commitment: pricing is not published; a card-free free tier for one store exists; book a call.
• Main strength: the memory of millions of observed tests behind every idea, so selection improves before anything is built.
• Main limitation: the price is not published.
• Evidence level: product claims documented on drip-apex.com (1); database and holdout figures vendor-stated in the September 29, 2026 press release (5); no third-party reviews, as there is no public review profile yet.

Core capabilities. Apex by DRIP is an A/B testing platform for companies selling online that predicts which tests will win before they go live, built on one of the largest A/B test databases in e-commerce. DRIP calls the method Prediction-Based Experimentation. The plain version: we know what shops like yours tested and which tests won. Apex has three layers: a test memory, a testing tool and managed execution.

The test memory holds 4.3 million A/B tests from 151,000 shops, collected since 2011. Every test idea gets a Likelihood to Win before launch: one percent figure per test idea, shown with how many tests back it. In a holdout check within a shop, the top third of ideas won 1.4 to 1.6 times as often (press release, September 29, 2026). DRIP also scored and froze predictions for 33 brands in early September 2026; the results are published from the launch on.

The testing tool builds, launches and evaluates tests directly in the shop. Experiments can be configured as code, edited in a code editor, or built in a visual builder called Operator, with CLI preview links and QA via preview. The CLI is the developer path. An MCP connection lets a shop owner build and read tests from ChatGPT or Claude without a terminal.

Results use fixed-horizon statistics with confidence. Each test shows the main metric, the lift and the revenue during its runtime. On Shopify, Apex shows profit per test, so a shop can optimise for profit: discount share, refund share and profit per sale. Guardian stops a test automatically at a revenue loss threshold. Clients see four QA checks: all devices, tracking and targeting, responsive, and matches design.

Heat maps and behaviour are shown per variant: clicks, scroll depth, average fold and funnel steps. An Engagement view shows bounce rate, read depth and whether visitors keep browsing. Integrations with Clarity and Hotjar are in beta. An analytics area covers acquisition, behaviour, audiences and sources, including ChatGPT. The Apex chat in the app answers what to test next, why a test won and how visitors moved, and its answers cite sources. The backlog classifies ideas automatically and gives each a score.

The third layer is managed execution: tests built, checked, launched and read together with the DRIP team. DRIP has run over 4,000 own tests and has served over 250 online shops, among them SNOCKS, AG1, Hornbach and KoRo.

Apex scores each idea against 4.3 million observed A/B tests from 151,000 shops and shows how many tests back each score. The nearest published figure in this comparison is Kameleoon’s Ideate, trained on more than 20,000 experiments; Varify cites 10,000 or more test patterns; no other vendor in this comparison publishes a count of tests behind its recommendations (vendor pages checked October 2026).

Limitations. The price is not published. A card-free free tier for one store exists, and beyond it the route to a price is to book a call. If your buying process needs a price list before any conversation, that is a real cost of choosing Apex.

Where Apex is not the right choice:

• A team that already picks winning tests well gets less from the prediction layer.
• A shop that cannot ship a test a week yet should build that habit on a lighter tool beforehand; buying prediction before you can execute is the wrong order.
• Anyone who needs a published price list before a conversation.

Summary. Apex by DRIP addresses the constraint before the build: choosing which tests to run. It combines a test memory of 4.3 million A/B tests from 151,000 shops, a testing tool with heat maps per variant and profit per test, and managed execution with the DRIP team. Its limitation is that the price is not published; a card-free free tier for one store exists, and the next step is to book a call.

Intelligems: best for price, shipping and offer tests on Shopify

• Best for: Shopify merchants who want to test prices, shipping rates and offers alongside content, and read the result as profit.
• Can test: prices (per SKU, subscription, multi-currency), discounts, gifts, bundles, upsells, shipping rates and thresholds, themes, templates, landing pages, page content, checkout blocks (Shopify Plus), post-purchase offers.
• Shop systems: Shopify, every plan; checkout tests need Plus and shipping rate tests need Advanced or Plus. Headless Hydrogen and React storefronts via the @intelligems/headless package.
• Data location and compliance: hosting region not published; the privacy policy says data “will be transferred outside of Europe, including to Canada and the United States”. DPA, subprocessors page, SOC 2, ISO 27001, cookieless mode: none found.
• Entry price and commitment: Smart Content $69 a month or $660 a year at the entry tier (0 to 1,000 orders); Unlimited from $349 a month with a 3-month minimum and no trial; 14-day trial on the other plans; no free tier; priced per store by monthly order volume, “no per-visitor charges and no overages” (checked October 2026).
• Main strength: one Shopify app tests prices, shipping, offers, checkout and content with profit and COGS reporting.
• Main limitation: it supports Shopify and no other shop system, and price testing needs a team-built integration with a 3-month minimum.
• Evidence level: 1 for product, documentation and pricing; 2 for third-party price reports; 4 for reviews; 3 and 5 for vendor claims.

Core capabilities. Intelligems describes itself with “Decide Confidently. Grow Profitably.” and “Most tools optimize for clicks. We optimize for profit.” The New York company launched on the Shopify App Store in December 2021 and states “100,000+ Tests run” and “5,000+ Stores” (vendor-stated). Its focus is commercial tests: prices, discounts, bundles, shipping rates and thresholds, checkout blocks and post-purchase offers, next to theme, template and content tests.

Tests are built in a Visual Builder, an AI Visual Builder, with CSS and JavaScript, or through JavaScript and an External API. The Atlas Experience Agent builds content, theme, template, redirect and JS or CSS tests from a text description; Intelligems states that offer, checkout, post-purchase, price and shipping experience types are not yet supported by the agent. An Atlas QA agent checks the setup before launch on all accounts. A test can have up to five groups, parallel tests have “no limit” (one shipping test at a time), and mutually exclusive experiments are in beta on request.

Statistics use “a Bayesian statistical model and Monte Carlo simulations” with 95 percent intervals. Intelligems recommends “300+ orders per test group” and “7+ days”. Each experiment reports revenue, profit and COGS, plus profit per visitor. Auto-stop, guardrails and sample ratio checks are not published. Integrations include GA4, Segment, Klaviyo, Clarity, Heatmap.com and Hotjar, plus “direct connections to Snowflake and your data warehouse”. Own heat maps or recordings: none found.

On AI, Smart Test Recommendations (beta) surface test ideas from the store's own performance, and Atlas “ranks them by estimated profit impact”. Activation stays manual. An MCP server with OAuth connects Claude, ChatGPT, Gemini and Grok; it reads results and can create drafts and start, pause or end tests, with a confirmation step. MCP, Atlas and the AI Visual Builder are on every paid plan. On the Shopify App Store, Intelligems holds 4.8 from 198 reviews (October 10, 2026).

Limitations. Intelligems states that it currently supports Shopify stores and no other system. Price testing needs a team-built integration that “typically” takes “3-5 business days”, and Smart Pricing and Unlimited carry a 3-month minimum without a trial. Support runs through Slack and email without live chat, and low reviews mention slow support, setup that needs code, and price. A hosting region, a DPA or security certifications were not found.

Where Intelligems is not the right choice:

• Stores on Shopware, WooCommerce, Magento or any other shop system.
• Teams that want to try price testing before committing: Smart Pricing and Unlimited have no trial.
• Teams that need live chat support.
• Buyers who need SOC 2, a DPA or EU hosting documented up front: none found.

Summary. Intelligems is a Shopify app built around commercial tests: prices, shipping, offers and checkout, read as profit. It pairs that with a visual content editor, AI building agents and an MCP server. It suits Shopify stores with the volume for price tests and the patience for a 3-month minimum, and it does not suit buyers who need EU data documentation up front.

Shoplift: best for theme and template tests on Shopify

• Best for: Shopify brands that want to test themes and templates in the editor they already use.
• Can test: home, collection, product, page, blog, article, cart and search templates; header, footer, navigation, global styles and cart type; prices on Advanced and Pro (single-currency stores); subscription prices via Recharge 2.0; shipping thresholds via the JavaScript API; product images via redirect tests. Checkout pages: not supported.
• Shop systems: Shopify and Shopify Plus. Headless storefronts (Hydrogen, Next.js) are “not currently supported”.
• Data location and compliance: data “is transferred to and processed in the United States”; DPA with EEA SCC Module 2 and UK IDTA; consent through Shopify's Customer Privacy API. ISO 27001 and SOC 2: none found.
• Entry price and commitment: Core $99 a month, or $74 a month billed annually, up to 50,000 monthly visitors; Advanced $399 monthly or $299 annually; Pro billed annually, from $699 a month. Billed on site-wide unique visitors, “not tested visitors”; 14-day trial; no free tier (checked October 2026).
• Main strength: tests run on Shopify's own themes and templates, edited in the Theme Editor, at a vendor-stated page speed cost of 1 to 4 points.
• Main limitation: Liquid-based Shopify storefronts alone, with no headless, checkout or A/B/n testing.
• Evidence level: 1 for product and pricing, with pricing conflicts noted; 2 for company data; 4 for reviews; 3 and 5 for vendor claims.

Core capabilities. Shoplift calls itself “The CRO Platform Built for Shopify” and a platform “built for high-growth DTC brands”. The New York company has been in the Shopify Plus Certified App Program since December 2024. Its approach is to stay inside Shopify: variants are edited in the Shopify Theme Editor rather than in an editor of its own, with Liquid or JavaScript for code changes and a JavaScript API for custom tests.

Tests come with a preview mode and a QA checklist, and parallel tests have “no limit”, separated automatically by test type. Each test runs two groups at a default 50/50 split; A/B/n is not supported. URL redirect tests handle product images and other cases the theme cannot. Price testing joined Advanced and Pro in September 2025, for single-currency stores and not on products built with Replo, PageFly and other listed page builders.

Shoplift uses “Bayesian statistical methods” with a main metric called Probability to Win. A result has to hold “for multiple days” before Shoplift labels it Leaning, High Confidence, Clear Winner, or No Change after two weeks; another vendor page says it “requires 95% statistical significance before declaring a winner”. The guideline is “at least 10,000 visitors per variant and 14 days”. Metrics include revenue, orders, cart adds and revenue per visitor. Profit metrics, sample ratio checks, holdouts and auto-stop: none found. Heat maps come through integrations with Heatmap.com, Hotjar and Clarity; GA4 is on Advanced and Pro.

Lift Assist is “Shoplift's recommendation engine which dynamically provides theme sections”: it duplicates the template and inserts a section styled with the brand's imported styles, but it “does not automatically add content”. Shoplift says it draws on “test ideas from hundreds of top brands” and “millions of shopper sessions” (vendor-stated). An HTTP API lets teams “Command Your CRO Program From Claude or ChatGPT”; an MCP server was not found. Shoplift gives three page speed statements: 2 to 4 points, about 2 points, and 1 to 2 Lighthouse points. Pro adds an account manager and strategy sessions twice a month. On the Shopify App Store it holds 4.9 from 133 reviews (October 10, 2026).

Limitations. Shoplift does not test checkout pages, does not support headless storefronts and does not run tests with more than two groups. Price tests need Advanced or Pro and a single-currency store. The price is tied to all site visitors, and over the plan limit tests pause. Low reviews criticise price and reliability. Data is processed in the United States, and the pricing page and App Store differ on some plan prices.

Where Shoplift is not the right choice:

• Headless Shopify or any shop that does not run on Shopify.
• Tests with three or more variants.
• Checkout page tests.
• Price tests in multi-currency stores or on products built with Replo or PageFly.

Summary. Shoplift turns Shopify's own Theme Editor into a testing tool, which keeps the learning curve short for teams already working there. It covers templates, theme elements and, on higher plans, prices, with Bayesian statistics and a vendor-stated page speed cost of a few points. It stops at checkout, headless and A/B/n tests, and it bills on all site visitors.

Varify.io: best for no-code tests on any shop system at a flat price

• Best for: teams on any shop system that want no-code tests at a flat price with unlimited traffic.
• Can test: product, collection, cart, home and landing pages via the snippet; Shopify Plus checkout via the separate app “Varify: Checkout A/B Testing”; split URL, multi-page and personalisation on all plans. Price, shipping threshold and recommendation tests are not published as features.
• Shop systems: any shop that loads JavaScript; the vendor names Shopify, WooCommerce, Magento, Shopware and custom builds, plus React and Vue single-page apps. One snippet, installed “Via GTM or directly”.
• Data location and compliance: AWS in Frankfurt plus Cloudflare; the vendor claims “GDPR, CCPA, HIPAA and revDSG”; “In principle, no cookies are set”; downloadable DPA. SOC 2 is marked “SOON”; ISO 27001: none found.
• Entry price and commitment: Growth €199 a month billed annually (€2,388 a year) or €249 monthly, one domain, five active experiments; Pro €249 a month annually or €349 monthly; Enterprise from €799 a month. AI credits cost extra. 15-day trial, “No credit card details, no active cancellation”; no free tier (checked October 2026).
• Main strength: a flat price per domain with unlimited traffic, from €199 a month billed annually.
• Main limitation: default statistics are one-sided frequentist tests; two-sided and Bayesian methods need Pro.
• Evidence level: 1 for product, pricing and compliance; 4 for reviews; 5 for the performance figures and the Claude connector listing.

Core capabilities. Varify.io calls itself “The A/B testing platform that scales with you” and promises “Transparent pricing. Unlimited Traffic. Powerful testing.” The Munich-area company states “2,704 teams worldwide”. The pricing model is its clearest difference: a flat fee per domain, “Same price, unlimited traffic”, where other tools in this comparison charge by visitors, orders, seats or tracked users.

Every plan includes the visual and code editors, preview links and a QA mode. Growth allows five active experiments; Pro removes the limit. Exclusion groups and server-side delivery (an API since April 2026) are on Enterprise, and server-side tests still need a client-side tracking block for reporting. On Shopify, the snippet runs on any plan, and since May 6, 2026 a separate app tests Shopify Plus checkouts for $119 a month.

Varify states: “We do not have our own tracking.” Results come from the customer's analytics, such as GA4, Clarity or Hotjar on all plans and BigQuery, Adobe Analytics or Mixpanel on Pro. Since September 18, 2026 an optional cookieless real-time Varify Tracking exists. The default statistics are “a frequentist method with one-sided tests”, chi-square for rates and a t-test for revenue, significant above 95 percent. Revenue significance and ARPU are on every plan. Auto-stop arrived in July 2026 on Enterprise; sequential testing and bandits are marked “SOON”.

Varify added AI through 2026: an AI audit for ideas, prompt-based variant creation, bring-your-own-key for Anthropic, Google or OpenAI, a Varify MCP server that can “create, start, pause experiments”, and AI experiment analysis. The vendor says its recommendations “are based on CRO patterns from real experiments” and cites “10,000+ test patterns”. Varify states a snippet of 11.5 KB that usually loads in under 30 milliseconds (vendor-stated). Reviews: 4.8 from 113 on OMR Reviews and 4.9 from 57 on G2 (October 10, 2026), with praise for ease of use, the visual editor, flat pricing and fast support.

Limitations. The default statistics are one-sided frequentist tests, and the two-sided and Bayesian options need Pro. Each plan covers one domain. Profit metrics, guardrails, sample ratio checks and holdouts: none found. SOC 2 is planned, not done. OMR reviewers mention “a partly cluttered user interface”, documentation “that needs improvement”, analysis in Google Analytics and targeting that “requires technical know-how”.

Where Varify.io is not the right choice:

• Teams that need sequential testing or bandits today: both are marked “SOON”.
• Multi-brand or multi-country setups on many domains: each plan covers one domain.
• Buyers who require SOC 2 or ISO 27001 today.
• Teams that want profit or guardrail metrics inside the testing tool: none found.

Summary. Varify.io is a no-code testing tool for any shop system, hosted in Frankfurt and priced flat per domain with unlimited traffic. It added AI building and an MCP server in 2026 and relies on your own analytics for results. Watch the one-sided default statistics and the one-domain plans.

VWO (Wingify): best for agents across the whole testing lifecycle

• Best for: teams that want agents from research to rollout, with human approval gates, on one platform.
• Can test: pages; on Shopify, theme tests via split URL, with the script not running on checkout pages. Price testing is “coming soon”; offer testing and a landing page builder are not in the product list (October 6, 2026).
• Shop systems: a Shopify app since March 1, 2024: free to install, paid subscription required, zero reviews as of September 3, 2026. Other shop systems were not part of our research.
• Data location and compliance: not covered in our research of October 6, 2026; ask the vendor.
• Entry price and commitment: Growth, Pro and Enterprise plans without public prices, scaled by monthly tracked users. A third party reports in-app VWO Testing prices of $665 a month (Growth, annual) and $1,336 (Pro) at 100k tracked users (mida.so, May 19, 2026). The free Starter plan is gone; 30-day trial without card.
• Main strength: agents for research, ideas, build, QA, launch, rollout and guardrails in one platform, with autonomy set as “a dial, not an all or nothing”.
• Main limitation: no public prices, no price, offer or checkout testing live, and a Shopify app without reviews.
• Evidence level: 1 for product pages and releases; 2 for third-party prices; 4 for G2 and Trustpilot; 5 for revenue and customer figures.

Core capabilities. VWO and AB Tasty announced their merger on January 20, 2026, with “ARR surpassing $100 million” and “more than 4,000 customers”. On September 16, 2026 the combined company relaunched as Wingify, “the Agentic Experience Optimization Platform”. VWO and AB Tasty stay as product brands, vwo.com and abtasty.com now redirect to wingify.com, and the company says: “Nothing about this launch asks our customers to start over.” Wingify is headquartered in New Delhi and has an office in Cologne.

The product is now built around agents under the name Wingz. An Insight Agent “reads funnels, replays, forms, and surveys; finds the segments that behaved differently and why”. A Hypothesis Agent “turns evidence into ranked, testable ideas and drafts the variations”, and the recommendations “are presented for human review”. A Synthetic Testing Agent “tests a change on synthetic visitors built from your real segments” before launch. Wingz “interprets the design intent and builds the variation” from text, Figma, a screenshot or a PDF, and the result is “visible and editable before anything ships”.

The Experiment Agent prepares audience, variations, metrics and launch, and “Nothing goes live without approval”. Statistics run on Bayesian SmartStats, and revenue projections come with confidence intervals. A Rollout Agent “stages a feature out, holds on a guardrail, reverts, resumes”, and a Guardrail Agent reverses breaches. Autonomy ranges from “propose-and-approve” to autopilot, and “the safeguards never switch off: an approval gate, guardrail metrics, traffic caps, a kill switch, and a permanent holdout”. Wingz connects via MCP to Claude, ChatGPT, Cursor and Gemini, and enterprise customers can bring their own model. The agents are “trained on AB Tasty's proprietary data and user insights”; a corpus size or a published win prediction was not found.

Wingify lists “200+ Customer Success Experts” and a dedicated success manager on Enterprise. A managed experimentation service, where the vendor's team builds and runs the tests, was not found. On G2, VWO Testing holds 4.4 from 929 reviews and AB Tasty 4.4 with Leader badges for Summer 2026. On Trustpilot, VWO holds 2.2 from 93 reviews, with complaints about renewal pricing and billing (as reported on September 4, 2026).

Limitations. Prices are not public and the free plan is gone. Price testing is “coming soon”, and offer and checkout tests are not live. On Shopify, the app has no reviews, the script does not run on checkout pages and theme tests go through split URLs. After the merger, existing contracts stay unchanged and the platforms converge “over time”; one third party expects prices for the mid-market to rise. Our research did not cover data location, so ask for it.

Where VWO is not the right choice:

• Teams that need a public price before talking to sales.
• Shops that want to test prices, offers or checkout now.
• Shopify merchants who choose apps by App Store reviews.
• Teams that want the vendor to build and run the tests: a managed service was not found.

Summary. VWO, now part of Wingify with AB Tasty, sells the whole cycle from research to rollout as a set of agents with approval gates and guardrails. It is a platform for teams with the budget for an enterprise contract and the people to supervise agents. Commercial tests on Shopify, public prices and a managed service are not part of it as of October 2026.

Optimizely: best for enterprise experimentation with deep statistics

• Best for: enterprise teams that run experiments across web, backend, mobile and edge and need documented statistics and certifications.
• Can test: anything in the page DOM through the snippet; server-side logic through 20+ SDKs; A/B, multivariate, multi-page and bandit tests; feature flags with progressive rollouts and kill switches. Shopify checkout is measured through a custom pixel, not changed. Price, shipping and discount features: none found.
• Shop systems: any site through a JavaScript snippet in the page head, including Shopify themes; no testing app on the Shopify App Store; a Shopify Plus connector syncs catalogue and sales data into Optimizely Data Platform.
• Data location and compliance: ISO 27001:2022, ISO 27017, ISO 27018, “SOC 2 Type 2 Attestation”, PCI DSS; “Your data will be stored in EU data centers”, with “the majority” of products in the EU Data Boundary; DPA with SCCs plus UK and Swiss addendums; public subprocessor list.
• Entry price and commitment: not published: “Every Optimizely plan is individually packaged.” A free Feature Experimentation plan runs one concurrent experiment. Vendr reports a median of $81,447 a year across 126 purchases, with a range of $26,460 to $261,075, for the whole Optimizely portfolio (third-party, checked October 2026).
• Main strength: a broad, documented statistics and agent stack in one product: sequential Stats Engine, sample ratio detection, bandits, an MCP server and test-building agents shipped through 2026.
• Main limitation: no public pricing and a third-party median contract of about $81,000 a year.
• Evidence level: 1 for product, statistics and compliance; 2 for company data and pricing; 4 for reviews; 5 for vendor claims.

Core capabilities. Optimizely, founded in 2010, was acquired by Episerver in 2020, which took the Optimizely name in 2021 and bought NetSpring for warehouse-native analytics in 2024. Web Experimentation is sold as “Agentic Experimentation” under the line “Test everything, regret nothing”. Feature Experimentation adds SDKs for backend, mobile and edge.

In 2026 the visual editor gained drag-and-drop, inline editing, templates and a Monaco code editor synced with it. A Variation Development agent (March) and “one-click variation creation” (June) lead to the Build agent (September), which “turns a saved or draft idea into a complete A/B test”.

Statistics are Optimizely's strong point. The default Stats Engine uses sequential testing with false discovery rate control, built with Stanford researchers, so teams can “monitor results continuously”. Fixed-horizon frequentist and Bayesian methods exist in beta through a Customer Success Manager. Multi-armed bandits, Stats Accelerator and contextual bandits are available, CUPED is described, and automatic sequential sample ratio mismatch detection gives “immediate detection at the beginning of an experiment's lifecycle”. Analytics are warehouse-native; heat maps and recordings come through Hotjar and Contentsquare integrations tagged by variation.

Opal, Optimizely's agent platform, added a Program Overview agent (January), an Idea Builder that “generates experiment ideas based on your page, goals, and research” (April) and the Build agent (September). An MCP server connects Claude, Cursor and VS Code to experimentation data via OAuth (April 2026). Opal uses “foundational LLMs including Gemini and Claude”, and Optimizely states: “Your data is never used to train the model or shared across customers.” Opal runs on credits whose price is not published. The snippet holds every running experiment and its definitions, and Optimizely recommends a synchronous install in the head because async loading “increases the risk of flashing”. Reviews: G2 4.2 from 411 (search snippet), TrustRadius 8.7 of 10 from 599 (October 10, 2026); one G2 criticism reads “robust but expensive, suited for larger organizations”.

Limitations. Prices are on quote, and third-party reports put typical contracts in the tens of thousands of dollars a year. Optimizely has no testing app on the Shopify App Store, and its snippet cannot run in Shopify's checkout, which is measured through a custom pixel instead. Price, shipping and discount tests have no dedicated feature.

Where Optimizely is not the right choice:

• Small budgets: prices are on quote and Vendr's logged contracts start at $26,460 a year.
• Shopify checkout tests: the snippet cannot run in Shopify's checkout.
• Teams that want a Shopify App Store install: there is no testing app there.
• Sites sensitive to page weight: the snippet grows with each running experiment and is recommended synchronous in the head.

Summary. Optimizely documents a sequential statistics engine, sample ratio checks and bandits, plus server-side SDKs, broad certifications and a growing set of agents. It is built for enterprise programs that test across web and product, and it is priced for them. For a Shopify store that mainly wants to test themes, prices or checkout, it is more platform than the job needs.

Kameleoon: best for EU enterprise teams that want agents and server-side flags

• Best for: enterprise teams that want prompt-based building, server-side feature flags and an MCP server, backed by ISO 27001 and SOC 2 Type 2.
• Can test: any page element the snippet can change, plus server-side features through feature flags. On Shopify, checkout conversions are tracked through a web pixel; changing the checkout itself: none found. Prices, shipping thresholds and discounts as dedicated features: none found.
• Shop systems: a JavaScript snippet loaded async from Cloudflare and SDKs for feature experimentation. A Shopify app exists since June 2021; the integration guide applies “exclusively to users with both a Shopify Plus and Kameleoon account”. Other shop systems were not checked.
• Data location and compliance: ISO/IEC 27001:2022, SOC 2 Type 2, PCI DSS, HIPAA, GDPR, CCPA, CyberVadis Platinum; dedicated servers that Kameleoon personnel alone can access; subprocessors include OVHcloud, Hetzner, Cloudflare and OpenAI; hosting region not stated on the pages read; consent modes including IAB TCF 2.3; a public DPA was not found.
• Entry price and commitment: PBX Starter “Starting from $495/month”, billed monthly, up to 10 experiments and 50,000 monthly tracked users with 30, 100 or 200 prompt credits a month; Enterprise on quote. 30-day free trial without card, up to three experiments. Code editor, CUPED, bandits, feature flags, personalisation and mobile are add-ons on Starter (checked October 2026).
• Main strength: one platform covers client-side tests, server-side flags and an MCP server that turns winning tests into production code, with ISO 27001 and SOC 2 Type 2.
• Main limitation: the Shopify integration is documented for Shopify Plus, and the App Store listing has no reviews after more than five years.
• Evidence level: 1 for product, statistics and compliance; 2 for company data; 4 for reviews; 5 for vendor claims such as prediction accuracy.

Core capabilities. Kameleoon, based in Paris, leads with “Build experiments in minutes by chatting with AI”. PBX means prompt-based experimentation, and PBX 2.0 (April 16, 2026) is “a suite of AI agents designed to support every stage of the experimentation process”. Ideate analyses a website for test opportunities and returns “up to eight test ideas for each page”. Build, Configure and Analyze create variations from prompts or Figma designs, set up goals and segments, and keep a library of takeaways from completed tests. Ship (May 2026) moves winning variations into server code behind feature flags through the MCP server.

Kameleoon's idea agent is trained on past experiments, and the vendor gives two figures: the launch post says a database “of over 25,000 experiments”, the Ideate page says “Trained on 20,000+ experiments”. It also states “63% accuracy on high-confidence predictions, compared to an industry average of around 20%” and that Ideate ideas are “three times more likely to convert than the industry average” (both vendor-stated, without published method). The MCP server offers 50+ tools, documented for Claude Code and Cursor, and a Claude connector works “no IDE required”. A ChatGPT connector was not found.

For building without AI, there is a Graphic Editor, a Widget Studio for no-code widgets, and a JS and CSS code editor, which is an add-on on Starter. Starter includes a “Mutually Exclusive Group/Holdout” and “40+ segmentation criteria”. Statistics cover fixed-sample frequentist, Bayesian and sequential methods, with multiple testing correction and real-time sample ratio mismatch detection; CUPED and multi-armed and contextual bandits are add-ons. A significance threshold, guardrails and auto-stop were not found on the pages read. Integrations include GA4, Hotjar, Clarity, Contentsquare, BigQuery, Snowflake and Klaviyo; own heat maps: none found.

The script weighs “29 KB (compressed with Brotli)” and loads async, with an anti-flicker default of 1,000 milliseconds. Reviews: OMR Reviews 4.5 from 192 and G2 4.6 from 137 (search excerpt), October 10, 2026. Reviewers praise the interface and support; criticism centres on complex tests needing developers, a learning curve and price.

Limitations. The Shopify integration guide is written for Shopify Plus, and the App Store app has had no reviews since June 2021. Native price, shipping and checkout tests were not found. Starter starts at $495 a month with 50,000 tracked users, and the code editor, CUPED and bandits cost extra. The two training figures for Ideate differ, and the accuracy claim has no published method.

Where Kameleoon is not the right choice:

• Shopify stores not on Plus: the integration guide applies “exclusively” to Plus.
• Teams that want to test prices, shipping thresholds or checkout layouts natively: none found.
• Small budgets: the entry plan starts at $495 a month, with key statistics features as paid add-ons.
• Teams without developer time for complex tests, a frequent criticism in reviews.

Summary. Kameleoon combines a full set of AI agents, an MCP server and server-side flags with broad certifications from a European vendor. It suits enterprise teams that test across web and product and want agents to build and ship. For Shopify stores below Plus, or for price and checkout tests, the documentation points elsewhere.

ABlyft: best for developer teams with strict data protection needs

• Best for: developer-led teams that want hosting in Germany and no stored personal data.
• Can test: page content through visual or code variations, redirect tests, Shopify draft themes and alternate templates. Native price, shipping threshold, discount or checkout tests: none found.
• Shop systems: a JavaScript snippet on any site, installed in the head, via Google Tag Manager or self-hosted; Shopify via theme.liquid, with orders from a webhook and events from a custom Web Pixel; no Shopify App Store app found; single-page apps supported.
• Data location and compliance: “Hosting and Storage in Germany”, snippet via Amazon CloudFront; “We do not store any personal data. Nothing.” and “We do not store IP addresses.”; localStorage by default, cookies off unless switched on. ISO 27001 and SOC 2: not published; a public DPA was not found.
• Entry price and commitment: free plan at €0, “No Credit Card needed”, limits not published; paid plans on quote (“Pricing tailored to your needs”) with a proof of concept; OMR reviewers mention annual contracts (checked October 2026).
• Main strength: data handling: German hosting, no stored personal data or IP addresses, and localStorage instead of cookies by default.
• Main limitation: the Shopify integration is a theme snippet without an App Store app, and native price, discount or checkout tests are not documented.
• Evidence level: 1 for product, data protection and pricing; 2 for company size and third-party price reports; 4 for reviews; 5 for performance claims.

Core capabilities. ABlyft is made by Conversion Expert GmbH in Kiel, founded in 2018, with a team of 2 to 10 people and no funding rounds. Its homepage names TUI, tesa, Deutsche Telekom and HORNBACH among its customers. Store Leads counted 961 shops using ABlyft on October 2, 2026, 75.5 percent of them on Shopify and 68.6 percent in Germany.

The tool is built for developers. Each variation can carry custom JavaScript and CSS with automatic minification and helper functions, and a GIT connection stores snippet changes in GitHub or Bitbucket as a revision history. A web-based visual editor handles text, images, styles, HTML and links; it stores edits as CSS selectors, so changed class names can break them. Parallel tests, exclusion groups, “Intelligent Redirect Experiments” and single-page application targeting by URL and DOM changes are documented.

Statistics default to frequentist methods (a z-test on conversion rates, Welch's t-test on values), with a Bayesian Beta-Binomial option. The threshold is configurable from 50 to 99.9 percent, with defaults of 100 conversions and two days. Revenue goals support capping of outliers, and a sample ratio warning is referenced. Profit metrics, guardrails, auto-stop and holdouts are not published. Any tool with a JavaScript tracker can be connected, and a REST API serves results.

ABlyft's AI layer, Lumen, generates “five AI-generated experiment ideas with hypotheses” from a URL, a goal and an audience, recommends “Ship, Don't Ship, or Keep Running” on results, and writes variation CSS and JavaScript, each on credits. An MCP server with OAuth connects Claude Desktop, ChatGPT, Cursor and Windsurf with “17 tools” to read and write experiments, goals, status and results. ABlyft states that “your data is never used for AI training”, and AI is opt-in per team admin. The vendor gives a 13 KB initial snippet that loads in 30 milliseconds (vendor-stated). On OMR Reviews, ABlyft holds 4.9 from 129 reviews with a “Leader - A/B Testing Q4/26” badge (October 10, 2026); reviewers praise responsive support, the interface and fair pricing.

Limitations. Paid prices are on quote. ABlyft has no Shopify App Store app, and native price, discount or checkout tests are not documented. OMR reviewers say complex tests need programming skills, the visual editor is limited, contracts are annual and documentation has gaps. ISO 27001 and SOC 2 are not published, and the security documentation says of itself: “It is not a compliance statement.”

Where ABlyft is not the right choice:

• Teams that want public prices: paid plans are on quote.
• Teams without developer capacity: complex tests need programming skills.
• Shopify merchants who want an App Store install or native price and offer tests.
• Buyers who need public ISO 27001 or SOC 2 evidence.

Summary. ABlyft is a lean, developer-led testing tool from Kiel with careful data handling: German hosting, no stored personal data and no cookies by default. It added AI ideas, code generation and an MCP server and keeps a free plan. It suits teams with developers who build tests in code, not teams that want price tests or a Shopify app.

Shopify Rollouts: best for a free native start on Shopify

• Best for: Shopify merchants who want to schedule and A/B test theme, checkout and discount changes inside the admin at no extra fee.
• Can test: theme edits or a full replacement theme, including translations; checkout and customer accounts (since June 5, 2026); discount activation, up to 250 per rollout; catalog status. Not supported: prices, Liquid template changes, vintage themes.
• Shop systems: Shopify Online Store, native in the admin under Markets, no app to install. Headless and custom storefront checkouts are not supported.
• Data location and compliance: Shopify is “certified Level 1 PCI DSS compliant” and has “issued SOC 2 Type II and SOC 3 reports”. Hosting region, visitor assignment, cookies and consent handling for experiments: not published.
• Entry price and commitment: no separate fee. Scheduling from Basic (€36 a month, or €27 billed yearly); experiments from Grow (€105 a month, or €79 billed yearly); market-specific targeting on Advanced (€384, or €289) and Plus (from €2,100). European pricing page, checked October 2026. SimGym: free install, then $0.75 per single-theme simulation and $1.50 per comparison beyond a monthly allowance.
• Main strength: testing built into the admin at no listed extra fee from Grow, covering theme, checkout and customer accounts since June 2026.
• Main limitation: experiments report conversion-type metrics, with no published statistical method and no revenue or AOV metrics.
• Evidence level: 1 for the help center, changelog and pricing; 2 for third-party analyses; 4 for SimGym reviews; 5 for the SimGym training claim.

Core capabilities. Shopify announced Rollouts in the Winter '26 Edition on December 10, 2025, opened scheduling and A/B tests to Basic and above on March 31, 2026, and extended them to checkout and customer accounts on June 5, 2026. Shopify's help center defines a rollout as “a scheduled set of changes to your online store's main theme, checkout and accounts pages, discounts, product catalogs, or a combination of these changes.” There are three types: Launch (permanent), Event (time-boxed, then rolled back) and Experiment (control against treatment).

Changes are made in Shopify's own theme editor and checkout settings, and Shopify creates a copy of the published theme automatically, so the live theme stays editable. The default split is 50 percent control and 50 percent treatment, with an end date 90 days after the start. Each experiment gets its own mutually exclusive segment of traffic, and a visitor is assigned to one rollout at a time for a given resource. At the end, the merchant rolls back or applies the changes. Market-specific targeting needs Advanced or Plus.

The reporting is narrow by design. Theme experiments show conversion rate over time, bounce rate, reached checkout rate and add to cart rate; checkout experiments show checkout conversion rate. “You can't customize the available metrics.” Gross sales, average order value and orders appear for Launch rollouts, not for experiments. Shopify does not publish its statistical method, and third parties disagree on whether confidence intervals are shown. WeltPixel (July 2026) found no documented way to see variant assignment outside the rollout's results card, including in GA4.

SimGym is a separate Shopify app in “AI Research Preview” that “uses AI-powered shoppers to simulate buyer behavior on your online store”. Shopify says the models are trained on “real commerce sessions across our platform” and adds its own caveat: “results might differ from actual buyer behavior”. SimGym holds 2.8 from 40 reviews on the Shopify App Store (October 10, 2026), with criticism that results cannot be exported and simulated shoppers cannot be customised.

Limitations. Experiments need the Grow plan or higher. They report conversion-type metrics without revenue, AOV or profit, and the statistical method is not published. Prices cannot be tested, Liquid template changes cannot be part of a rollout, vintage themes are excluded and headless storefronts are not covered. Heat maps, segments, AI building and an MCP connection are not published for Rollouts.

Where Shopify Rollouts is not the right choice:

• Stores on Basic that want A/B tests: experiments need Grow or higher.
• Headless or custom storefronts: experiments do not cover them.
• Teams that judge tests on revenue, AOV or profit per visitor, which experiments do not report.
• Price tests or Liquid template tests, which rollouts do not support.

Summary. Shopify Rollouts makes scheduling and simple A/B tests a native part of the Shopify admin, now including checkout, at no listed extra fee from the Grow plan. It is a sound place to start testing theme and checkout changes. Once you need revenue or profit per test, price tests or a documented statistics method, you need a dedicated tool next to it.

09Which Six Other Platforms Are Worth Knowing?

ABConvert

ABConvert calls itself “The A/B testing platform built for Shopify merchants” and runs nine test types: price, shipping, offer, URL redirect, template, theme, visual editor, checkout and combined. Profit per visitor is calculated from the merchant's own product, shipping and transaction costs. Hosting runs on MongoDB Atlas, Linode and Google Cloud, and the region is not published. It runs on Shopify and not on headless Shopify, and an MCP server connects to Claude on Growth and above. Starter costs $99 a month or $74 a month billed yearly; price tests start on Growth at $199 (checked October 2026). What it does that the nine above do not: it bills on test orders, “a unique order ABConvert records while a test or personalization runs”, rather than on all store traffic.

Convert Experiences

Convert presents itself as a “Proven A/B Testing Tool”, “Reliable & Affordable”, with “Carbon-Neutral Servers in Frankfurt, Germany”, ISO 27001 and SOC 2 Type 2. It switches between fixed-horizon frequentist, Bayesian and sequential statistics, checks sample ratio mismatch on every plan, and includes heat maps and session recordings on all plans. On Shopify, prices can be tested through Cart Transform Functions on every paid plan, with checkout-level prices on Plus; the Shopify app has no reviews yet. Convert's July 2025 documentation says direct checkout A/B testing “isn't currently possible” under Shopify's Checkout Extensibility, and shipping thresholds are tested through JavaScript variations. Beyond Shopify, Convert lists integrations for WooCommerce, BigCommerce, Magento, NopCommerce, PrestaShop and Volusion. Growth costs $299 a month billed annually or $399 monthly for 100,000 tested users (checked October 2026). What it does that the nine above do not: its annual plans “don't auto-renew”.

GrowthBook

GrowthBook is “The warehouse-native platform used by modern product teams for experimentation, feature flags, and product analytics.” Analysis runs on your own warehouse, with Bayesian and frequentist engines, sequential testing, CUPED and automatic sample ratio checks. On Shopify it is a theme snippet plus an AI Visual Editor Chrome extension, with no App Store app and no documented price or checkout tests. Cloud Starter is free for up to three users, Pro is $40 per seat a month, and since September 2026 an EU region in Ireland stores event data while the application stays in the US (checked October 2026). What it does that the nine above do not: the full platform is open source and free to self-host, with no seat limit and unlimited traffic.

Visually.io

Visually.io describes itself as a “One-stop, AI powered A/B testing & personalization platform for ecommerce” on Shopify and Shopify Plus, with headless support through Hydrogen. Its statistics are Bayesian, a Global Control Group serves as a holdout, and an MCP server answers reporting questions in Claude and ChatGPT. It tests homepages, product pages, carts, checkouts and post-purchase upsells. Its privacy policy says data “will be transferred outside of Europe, including to Canada and the United States”, and the hosting region is not published. Price tests are limited to the Scale plan with 600+ monthly orders or Shopify Plus stores with 7,500+ orders. Pricing runs by monthly orders from $19 a month (up to 200 orders) on the website, while the App Store lists $15 for up to 100 orders; 14-day trial, no contract (checked October 2026). SOC 2 and GDPR programs are “in the process”. What it does that the nine above do not: product recommendations and upsells sit in the same app as testing and personalisation.

Elevate

Elevate sells “All-in-one AB testing: optimize prices, shipping, themes, content and more to maximize store profit!” as a Shopify app, with headless support on Premium. Price tests work “on any Shopify plan, including non Plus” without duplicate products, alongside shipping, offer, checkout (Plus) and product image tests. It also tests templates, whole themes, content, subscription prices and landing pages. Basic starts at $49 a month and Standard, which adds price and shipping tests, at $99 for stores up to 1,000 orders; yearly billing saves 40 percent and every plan has a 14-day trial (checked October 2026). Hosting region, DPA and certifications are not published, and data is removed 48 hours after uninstall. What it does that the nine above do not: it declares a winner when four rules are met together (confidence threshold, visitor minimum, conversion minimum and a three-day minimum runtime) and reports power and minimum detectable effect per test.

AB Tasty (Wingify)

AB Tasty merged with VWO in January 2026 and is now a product brand of Wingify; abtasty.com redirects to wingify.com, and existing contracts stay unchanged. Its AI, Evi, launched in November 2025 (vendor-stated: “3,000 early adopters”, “73% reduction in campaign setup time”) and turns prompts into HTML, CSS and JavaScript; Wingz now unifies Evi with VWO's Copilot. Plans and prices follow Wingify: Growth, Pro and Enterprise without public prices. What it adds to the VWO profile above: Wandz.ai, which AB Tasty bought in November 2025 and which is stated to process “3.7 trillion data points monthly”.

10What Changes When You Move From a Testing Tool to a Testing Program?

A testing tool runs tests. A testing program decides which tests to run, runs them, and gets better at deciding.

Many shops end up stitching that program together from parts: ideas in a spreadsheet, variants in a testing tool, numbers in an analytics tool, reasons in slide decks, and the context in the heads of whoever ran the tests. Each handover loses something. The six steps a program needs are simple to name:

1. Backlog: every idea in one list, classified, with its source.
2. Prediction: an estimate before the build of which ideas are likely to win.
3. Build: variants built and checked on every device.
4. Launch: traffic split, targeting and guardrails set.
5. Results: read in shop terms, revenue and profit per visitor, not clicks.
6. Learning: what the result means for the next idea, kept where the next idea is chosen.

Several vendors in this comparison move in that direction. Wingify's agents run from research to rollout. Kameleoon keeps a library of takeaways from completed tests. Optimizely's Program Overview agent reports on the program, and its Idea Builder generates ideas from pages, goals and research. The question to ask each of them is where the evidence for choosing an idea comes from.

Apex by DRIP is built around that question, in three layers. The test memory holds 4.3 million A/B tests from 151,000 shops, collected since 2011, one of the largest A/B test databases in e-commerce, and every test idea gets a Likelihood to Win before launch. The testing tool builds, launches and evaluates tests directly in the shop, with heat maps per variant, profit per test and the Apex chat. Managed execution means tests are built, checked, launched and read together with the DRIP team.

Think of Likelihood to Win as a weather forecast. A forecast does not decide the weather, but it tells you whether to plan the picnic or pack an umbrella. Likelihood to Win does the same for a backlog: it ranks an idea against the shop's own backlog, so the next build goes to the idea that ranks higher. It is not a win probability in the strict sense, and it does not replace the test.

11How Should a Shop Choose an A/B Testing Tool?

Demos show the editor. These eight questions show the rest; ask every vendor on your shortlist, including us.

1. Where is our data processed and stored? Ask for the hosting region, a DPA and a subprocessor list. In this comparison, answers range from hosting in Germany to processing in the United States.
2. How does the tool handle consent? Ask what happens to tests and tracking when a visitor declines, and who decides whether consent is needed.
3. How heavy is the script, and how does it load? Ask for the size, whether it loads synchronously or async, and how flicker is prevented. Vendors publish figures from 11.5 KB (Varify.io, vendor-stated) to a snippet that grows with every running experiment (Optimizely).
4. What can it test on our shop system? Pages, theme sections, prices, shipping, checkout, landing pages: check each against your plan. Checkout tests on Shopify usually need Plus, and price tests often need a higher plan or a team-built integration.
5. Which statistics method does it use? Ask for the method, the default threshold, the minimum sample, and whether it checks sample ratio. Defaults vary from one-sided frequentist tests to Bayesian probability to win to sequential testing, and some tools do not publish theirs.
6. How are winners chosen before launch? Ask what evidence ranks an idea: the shop's own data, a generic framework, an AI model, or a database of observed tests. Ask how many tests stand behind a recommendation and whether the vendor publishes it.
7. What happens to our results? Ask whether results show revenue and profit per visitor, whether you can export them, and what happens when you leave. Elevate, for example, removes data 48 hours after uninstall; Intelligems offers a $9 Pause Plan that keeps data.
8. How long is the contract? Ask for the minimum term, the renewal rule and what happens when traffic exceeds the plan. Intelligems price plans have a 3-month minimum, Convert's annual plans do not auto-renew, and Shoplift pauses tests above the visitor limit.

12What Is the Verdict for 2026?

If you are not sure which constraint you have, look at the last twelve months. If tests ran but few won, the constraint is choosing. If good ideas waited for months, it is building. If the tool bill grew faster than the wins, it is budget. If legal held up the contract, it is compliance.

If your constraint is choosing tests, Apex by DRIP was built for it. The price is not published, a card-free free tier for one store exists, and the next step is to book a call.