Answers about experiments
Questions and answers from the Croct community about experiments.
Can a HubSpot form submission be the primary goal of an A/B experiment?
I am setting up an experiment on a landing page and I want the conversion to count when someone actually submits our HubSpot demo form, not just when they…
How long should I let an experiment run before trusting the numbers?
this is my first experiment and after two days variant B is already ahead and it looks really promising. part of me wants to just call it and ship, but i do…
Do purchase and sign-up events from GTM count as conversions for an experiment?
I push purchase and sign-up events into the data layer and want to use one of them as the conversion goal of an experiment I am setting up. Does the Croct tag…
Starting my first experiment on a Strapi hero, what traffic split should I begin with?
I wired our Strapi hero block to a slot and built two variants, so now I have to actually set the experiment up. I am not sure whether to send all my traffic…
Ran an A/B test on a Sanity hero, how long before Croct calls a winner?
first experiment here. I set up an A/B test on our Sanity hero component and it has been running a few days, but it still shows as in progress and I am…
Do logged-in shoppers stay in the same variant across devices on Hydrogen?
lots of our shoppers browse on mobile and then buy later on desktop. We run an experiment on the product page and I do not want someone to see variant A on…
Can I use VTEX order placement as the primary goal of an experiment?
I set up an experiment on a VTEX landing experience and I want the purchase to count as the conversion. The problem is the order event was not showing up as a…
What does Croct mean by "recommended winner", what confidence level is that?
Coming from a tool that reported everything as frequentist p-values, so the "recommended winner" label is new to me and I want to map it to something I…
Can I bump a variant from 30% to 50% of traffic while the test is already running?
variant B is looking strong quite early and I want to reach significance faster by sending it more traffic. Currently it is a 30/70 split and I am tempted to…
Is it worth running an A/A test first to check my Croct setup?
before i trust any real results i want to sanity-check the plumbing, so my plan is an A/A test: two identical variants, same content, and just confirm that…
Can I bump the traffic allocation partway through a Hydrogen experiment?
I started an experiment on a product page on our Hydrogen storefront at a conservative traffic share because I wanted to limit exposure early. It is behaving…
Once an experiment picks a winner, how do I ship it to 100% of traffic without a deploy?
my headline test just flagged a clear winner. now I want everyone to see it. do I need a dev to push that live, or can I do the rollout myself from the…