Answers about experiments
Questions and answers from the Croct community about experiments.
Can I use VTEX order placement as the primary goal of an experiment?
I set up an experiment on a VTEX landing experience and I want the purchase to count as the conversion. The problem is the order event was not showing up as a…
Starting my first experiment on a Strapi hero, what traffic split should I begin with?
I wired our Strapi hero block to a slot and built two variants, so now I have to actually set the experiment up. I am not sure whether to send all my traffic…
Do logged-in shoppers stay in the same variant across devices on Hydrogen?
lots of our shoppers browse on mobile and then buy later on desktop. We run an experiment on the product page and I do not want someone to see variant A on…
Can I bump a variant from 30% to 50% of traffic while the test is already running?
variant B is looking strong quite early and I want to reach significance faster by sending it more traffic. Currently it is a 30/70 split and I am tempted to…
What does Croct mean by "recommended winner", what confidence level is that?
Coming from a tool that reported everything as frequentist p-values, so the "recommended winner" label is new to me and I want to map it to something I…
Can I bump the traffic allocation partway through a Hydrogen experiment?
I started an experiment on a product page on our Hydrogen storefront at a conservative traffic share because I wanted to limit exposure early. It is behaving…
Once an experiment picks a winner, how do I ship it to 100% of traffic without a deploy?
my headline test just flagged a clear winner. now I want everyone to see it. do I need a dev to push that live, or can I do the rollout myself from the…
Is it worth running an A/A test first to check my Croct setup?
before i trust any real results i want to sanity-check the plumbing, so my plan is an A/A test: two identical variants, same content, and just confirm that…
My test has run a week and still says "in progress", when does it actually conclude?
My experiment has been live for exactly a week now and the status still reads "in progress". I am trying to work out whether that is expected or whether…
Counting purchases as the conversion goal for a Hydrogen experiment
Setting up our first experiment on a Hydrogen store. Current state: Product views and cart events show up in the dashboard automaticallyThe experiment editor…
Where do Storyblok Experiments results actually live? I cannot find a stats view
Expected: launch a native Storyblok experiment, come back a week later, see exposures, conversions, and some significance reading in a results view. Actual:…
Storyblok Experiments gives me variants but who actually assigns users to them?
I went through the Storyblok Experiments docs pretty carefully before setting anything up. Created an experiment, modeled two variants of our hero block,…