VTEX killed the native A/B Tester, how are people running tests now?
we leaned on VTEX's native A/B Tester for a couple years until it got deprecated. the whole thing was workspace splitting, which meant we were splitting entire deployments, not components. and the session and conversion counts it reported never lined up with our real order data, so every readout turned into an argument.
now that the Croct integration exists I want to know what a reliable setup actually looks like. what replaces the workspace-splitting model
2 answers
We made this exact move earlier this month, so I can tell you what changed for us.
There is no workspace splitting anymore. Croct connects through the single GTM tag, so nothing forks your deployment. Your store events feed audiences, and order placements record conversions automatically as Goal Completed, so the conversion count comes from the same order signal you already trust.
Tests target components and experiences, not entire deployments, which means you can run one on a single PDP block without touching the rest of the site. And the analytics are Bayesian and unsampled across 100% of the data, not a 30-day-purge tool like the old one. After the native tester's session counts never lining up with our orders, having unsampled conversions that actually matched was the relief we needed.
Adding a technical note on why the numbers behave better. Because the conversion is the order placement event itself rather than a routing-based guess, you are not depending on which workspace a session happened to land in. The old splitting model attributed by deployment, which is where a lot of the session and conversion drift came from.
If you do want to reconcile against store analytics later, the usual causes of small deltas are worth knowing up front, similar to what people hit comparing Croct conversions against storefront order counts.
that matches what we are seeing so far, conversions tracking the actual orders instead of the routing. much less to argue about. appreciate it