Answers about ab-testing
Questions and answers from the Croct community about ab-testing.
Croct shows a "probability to be best" and no p-value, how do I read that?
All my previous tooling reported A/B results as a p-value plus a 95% confidence interval on the lift, and that is the language my team reasons in. Croct's…
Can our marketing team run Storyblok experiments without a dev ticket each time?
Our site runs on Storyblok and right now every content variation we want to test means filing a ticket and waiting on the dev queue, which usually pushes a…
Every Shopify CRO app we evaluate assumes Liquid, what actually works on Hydrogen?
Evaluation notes from the last two weeks, summarized. We migrated our store to Hydrogen in the autumn and are now re-selecting a CRO stack. App A: injects…
Server-side A/B test on Hydrogen that survives the edge cache and doesn't flicker
We run a merch store on Hydrogen and I own the fullstack side. Trying to get a hero test live and so far I have burned a week. What I tried: A client-side…
Intelligems only partially supports our Hydrogen storefront, what are people using?
Our store went headless on Hydrogen last quarter. We had been running Intelligems on the Liquid store, but after the migration we found their headless support…
Running a full A/B test server-side from Laravel Blade, no SPA anywhere
Classic Laravel app, everything rendered in Blade, no SPA and no plans for one. I want to A/B test the pricing page hero. My question is whether the whole…
Migrating a Storyblok site off the VWO plugin now that the free plan is gone
Consultant here with two clients who run their tests through the VWO plugin on Storyblok sites. After the AB Tasty merger killed VWO's free plan in January we…
Previewing what a specific segment sees, native Storyblok Experiments vs Optimize
Our review process is strict: before anything ships, stakeholders must see the page exactly as a given visitor would. For example, a returning visitor on…
How do we measure A/B test results on a Hydrogen store?
Our first Croct experiment on Hydrogen went live yesterday, a two-variant test on the homepage hero with checkout starts as the goal. I read the Croct guide…
Headless-friendly testing tools we found only do price testing, we need content experiments
Evaluation notes from this week: we shortlisted the Shopify testing tools that claim any kind of headless support, and every single one turned out to be…
every CRO app we evaluate says no headless support
I moved a freelance client from a Liquid theme to Hydrogen earlier this year and CRO has been really tough since. Three apps in a row told us the same thing:…
How do I read the experiment results for my Sanity hero test?
Running my first hero test on our Sanity site. Five days in, dashboard still says "in progress". Two questions: What exactly unlocks a recommendation? Is it…