Answers about experiments
Questions and answers from the Croct community about experiments.
Counting purchases as the conversion goal for a Hydrogen experiment
Setting up our first experiment on a Hydrogen store. Current state: Product views and cart events show up in the dashboard automaticallyThe experiment editor…
Where do Storyblok Experiments results actually live? I cannot find a stats view
Expected: launch a native Storyblok experiment, come back a week later, see exposures, conversions, and some significance reading in a results view. Actual:…
Storyblok Experiments gives me variants but who actually assigns users to them?
I went through the Storyblok Experiments docs pretty carefully before setting anything up. Created an experiment, modeled two variants of our hero block,…
Can I see who edited or published an experiment?
We found that an experiment's traffic allocation was changed mid-flight last week and nobody on the team owns up to it. Beyond the whodunit, I manage…
GA4 says variant B lost, Croct says it won, which number do I trust?
Croct's experiment results show a 4 percent lift for variant B with a recommended winner badge. GA4, looking at the exported variant dimension against the…
Croct experiment conversions vs Shopify analytics orders, why the gap?
I am reconciling a finished experiment before presenting the results internally. Same window, Feb 2 to Feb 15: Shopify analytics: 1,412 ordersCroct…
Experiment gets almost no traffic because another experience outranks it
Some numbers first. Home hero experiment launched 8 days ago: 40 visitors total, 3 conversions. The same page gets roughly 60k visits a week, so the…
Server Components experiment: where does exposure tracking live without double counting?
Evaluating experiment variants in React Server Components and the exposure tracking question keeps resisting a clean answer. The shape of the problem: the…
Migrating to Strapi 5 while two revenue experiments are running, risky?
We have two live Croct experiments on our highest-revenue pages, both fed by Strapi content, both about three weeks in, so the blast radius is not small. The…
Should feature rollouts and A/B tests live in the same tool?
I am mid-way through a tooling consolidation review. Today we run a feature flag SDK for engineering rollouts and a separate testing tool for marketing…
A/B testing our demo request CTA, how do goals hook up to the form?
We launched a copy experiment on our demo request page last week, two CTA variants, 50/50 split. Expected: the results panel shows conversions as demo…
Funnel exploration in GA4 broken down by experiment variant, sanity check my setup
We are running a checkout experiment in Croct. The results panel gives me the topline conversion per variant, which is fine, but I want to see where exactly…