Answers about experiments
Questions and answers from the Croct community about experiments.
My test has run a week and still says "in progress", when does it actually conclude?
My experiment has been live for exactly a week now and the status still reads "in progress". I am trying to work out whether that is expected or whether…
Once an experiment picks a winner, how do I ship it to 100% of traffic without a deploy?
my headline test just flagged a clear winner. now I want everyone to see it. do I need a dev to push that live, or can I do the rollout myself from the…
Is it worth running an A/A test first to check my Croct setup?
before i trust any real results i want to sanity-check the plumbing, so my plan is an A/A test: two identical variants, same content, and just confirm that…
Counting purchases as the conversion goal for a Hydrogen experiment
Setting up our first experiment on a Hydrogen store. Current state: Product views and cart events show up in the dashboard automaticallyThe experiment editor…
Where do Storyblok Experiments results actually live? I cannot find a stats view
Expected: launch a native Storyblok experiment, come back a week later, see exposures, conversions, and some significance reading in a results view. Actual:…
Storyblok Experiments gives me variants but who actually assigns users to them?
I went through the Storyblok Experiments docs pretty carefully before setting anything up. Created an experiment, modeled two variants of our hero block,…
GA4 says variant B lost, Croct says it won, which number do I trust?
Croct's experiment results show a 4 percent lift for variant B with a recommended winner badge. GA4, looking at the exported variant dimension against the…
Croct experiment conversions vs Shopify analytics orders, why the gap?
I am reconciling a finished experiment before presenting the results internally. Same window, Feb 2 to Feb 15: Shopify analytics: 1,412 ordersCroct…
Can I see who edited or published an experiment?
We found that an experiment's traffic allocation was changed mid-flight last week and nobody on the team owns up to it. Beyond the whodunit, I manage…
Experiment gets almost no traffic because another experience outranks it
Some numbers first. Home hero experiment launched 8 days ago: 40 visitors total, 3 conversions. The same page gets roughly 60k visits a week, so the…
Server Components experiment: where does exposure tracking live without double counting?
Evaluating experiment variants in React Server Components and the exposure tracking question keeps resisting a clean answer. The shape of the problem: the…
Migrating to Strapi 5 while two revenue experiments are running, risky?
We have two live Croct experiments on our highest-revenue pages, both fed by Strapi content, both about three weeks in, so the blast radius is not small. The…