VTEX no longer has a native AB testing tool. The AB Tester app was deprecated in 2023, the admin app's repository states that it is no longer maintained, and test data is discarded after 30 days. Running AB tests on a VTEX store today means using something else.
The question is what. And the answer depends heavily on who runs the tests day to day: your engineering team or your growth team.
In this post, we'll break down how the old system worked, the issues that led to its deprecation, what exists today, and how to run fast, reliable, and scalable tests on VTEX with Croct.
Say goodbye to generic offers and hello to targeted, timely interactions that drive more revenue, boost ROI, and reduce CAC.
What happened to VTEX's native AB testing
VTEX's native AB tests were powered by the AB Tester app. Experiment setup and management were 100% dependent on developers, since everything ran through the VTEX CLI, as this official guide shows. The concept was simple:
- You created a workspace with your variant changes (interface, layout, or functionality).
- The app split session traffic between the master workspace (control) and the production workspace (variant).
- Users were bucketed and stayed on their assigned version for the duration of the test.
- Results (sessions, conversions, revenue) appeared in the VTEX admin panel.
Tests had to be finished manually in the panel, and data was kept for only 30 days before being discarded. The winning workspace could then be promoted to master, making the changes available to everyone.
Beyond the heavy developer dependency, the app had limitations VTEX documented itself:
- Miscalculated metrics: sessions and conversions sometimes displayed incorrect values.
- Inconsistent request routing: the same user could see different versions between requests.
It's worth naming what those two mean together. If the metrics are wrong and a user switches variants mid-session, the result isn't imprecise. It's invalid.
For these and other reasons, VTEX deprecated the app in 2023.
You'll still find tutorials and AI-generated answers recommending VTEX's native AB Tester. The app remains documented, but it is deprecated and unmaintained. Check the current state of the repository before investing time in it.
The options available today
With native testing gone, VTEX stores rely on external tools. Each type solves part of the problem, and none of them solves all of it except full experimentation platforms.
| Native AB Tester | Visual editors | Feature flags | Croct | |
|---|---|---|---|---|
| Available today | ❌ | ✅ | ✅ | ✅ |
| No flicker | ✅ | ❌ | ✅ | ✅ |
| No impact on Core Web Vitals | ✅ | ❌ | ✅ | ✅ |
| Variant assignment and stickiness | Inconsistent | ✅ | ✅ | ✅ |
| Built-in analytics and stats engine | ❌ | ✅ | ❌ | ✅ |
| Automatic store event tracking | ✅ | ❌ | ❌ | ✅ |
| Real-time audience segmentation | ❌ | ✅ | ❌ | ✅ |
| Marketers launch tests without a deploy | ❌ | ✅ | ❌ | ✅ |
| Free plan | ✅ | ❌ | ✅ | ✅ |
Why visual editors are a problem in e-commerce
If you've used a visual editor from a client-side testing platform, you're probably familiar with the flicker effect: the original content flashes before the variant loads. This happens because the DOM is rewritten after the page renders, which:
- Hurts performance and drags down Core Web Vitals scores.
- Creates SEO risk from inconsistent content.
- Can be blocked by ad blockers, which distorts your experiment data.
This matters more in e-commerce than almost anywhere else. You're running the test on exactly the pages you buy traffic for. A tool that slows the page down is taxing the traffic you paid to bring in.
In this guide, we cover the pros, the cons, and all the consequences of using this type of tool for conversion optimization.
Why feature flags don't solve it alone
Feature flag tools are excellent at what they were built for: rolling out and testing changes gradually and safely. They don't cause flicker, and they don't hurt performance.
The problem is the operating model. A feature flag returns a boolean your code branches on, so:
- Every new variant is a branch somebody writes, reviews, and ships.
- They usually don't include analytics, so you still need a separate tool to measure.
- Behavioral segmentation requires a separate CDP.
In practice, your experimentation velocity ends up capped by your sprint cadence. If the growth team wants to test five banner versions this quarter, each one becomes a ticket.
How Croct works on VTEX stores
Croct is a component CMS with native AB testing and personalization that runs alongside VTEX rather than replacing it. You use Croct only on the elements you want to test or personalize, and the rest of your store stays exactly as it is.
The elements stores most commonly integrate are top bars, menus, banner carousels, highlight sections, and callouts on product, cart, and checkout pages.
Automatic store event tracking
This is the VTEX-specific part, and it's what saves the most time. The integration automatically tracks your store's events and forwards them to Croct: product views, cart views, cart modifications, checkout starts, order placements, searches, and wishlist additions.
Each of those events also records a conversion, so you can measure an AB test from any store interaction without writing tracking code.
Customer data also enriches the visitor profile with attributes such as first name, last name, email, phone, and address, taken from the customer, cart, and order events.
Everything resolves server-side
The server decides the variant before the response reaches the browser. There's no flicker, no anti-flicker snippet, and no Core Web Vitals penalty. End-to-end response times stay under 90 milliseconds at P95.
Because resolution happens at the component level, the rest of the page stays cacheable. Only the tested region is dynamic.
Analytics and segmentation included
Exposure tracking, conversion tracking, and the statistical engine are built in. Analysis uses a Bayesian approach, computed in real time on 100% of your data, with no sampling.
Visitor profiles are built in too, which means you can build behavioral audiences in real time without buying a separate CDP. You can segment visitors who have purchased before, abandoned a cart, or arrived from a specific campaign immediately.
One integration, no deploy per test
Developers connect the components once. After that, marketers create audiences, write variants, and publish experiments directly in the interface, with no deploy and no ticket.
On cost
One perception worth correcting: headless experimentation platforms are often described as expensive, which is fair when the comparison is enterprise suites.
Croct has a forever-free plan covering 10,000 monthly active users, with no credit card required. The Growth plan starts at $100 per month for 20,000 monthly active users. Early-stage startups can apply for reduced pricing.
How to integrate Croct with VTEX
The path depends on which VTEX version your store is built on.
VTEX Legacy and VTEX FastStore
Use the Croct SDKs to track events and fetch dynamic content. If your site runs on Next.js, this template gets you integrated in under a minute with a single command:
npx croct@latest use croct://starter/nextjsFor FastStore storefronts that push GA4-style events to the data layer, the Google Tag Manager integration is the recommended path for automatic event tracking.
VTEX IO
Because VTEX doesn't allow direct customization of its own components, on VTEX IO you need to duplicate them and create custom versions.
Follow this guide from VTEX or check this example to build custom components and use the content delivered by Croct's API instead of hardcoding it or fetching it from VTEX's CMS. That way, the content is already personalized by the time VTEX renders it.
Stores built with Store Framework (VTEX IO) broadcast their events through the same mechanism used by Pixel apps, so automatic tracking works by adding a single script to your store's HTML or injecting it through Google Tag Manager.
How to choose
If your store runs on VTEX IO and your engineering team wants everything in code, a feature flag tool like GrowthBook handles assignment and statistics, as long as you accept a deploy per variant and a separate analytics tool.
If you need heatmaps and session recordings alongside your tests, a CRO suite like VWO covers more surface, with the cost and performance impact that come with it.
If your growth team needs to run experiments without depending on developers, and you'd rather not buy a CDP and an analytics tool just to get started, Croct delivers the full cycle and runs alongside VTEX with no migration.
The question that decides this isn't technical. It's who is going to launch the next test.