# Ran an A/B test on a Sanity hero, how long before Croct calls a winner?

Asked by lena_ab on 2026-09-11. Tags: sanity, experiments, ab-testing.

first experiment here. I set up an A/B test on our Sanity hero component and
it has been running a few days, but it still shows as in progress and I am
getting a little impatient for a call.

What are the actual thresholds Croct waits for before it recommends a winner?
I would like the concrete numbers so I know whether I am looking at days or
weeks, and whether I have enough traffic to get there at all.

## 2 answers

### Accepted answer from rajpatel (2026-09-12)

There are hard gates before it will even consider a result. An experiment
stays "in progress" until every variant has at least 1000 visitors, at least
25 conversions, and it has run for at least 1 week. All three, per variant.

Once those are met, a recommended winner appears when the probability to be
best goes above 95 percent and the potential loss drops below 0.1 percent.
The stats are Bayesian and unsampled, so it is looking at 100 percent of your
data rather than a sample.

So if it is only a few days in, the one-week floor alone means it cannot call
it yet regardless of how good the numbers look. Check where each variant sits
against the visitor and conversion counts and you will know if traffic is the
constraint.

#### Reply from lena_ab (2026-09-13)

that explains the wait, we are past the visitor count but not the week
yet. will leave it alone. thanks

### Answer from Priya Venkataraman (2026-09-13)

One warning while you wait: do not touch the traffic allocation or add or
remove variants mid-test. Changing the split or the variant set during a
running experiment is strongly discouraged because it invalidates the
statistics, and then the clock effectively resets on trusting the result. If
a variant is clearly bad, pause the whole experiment or let it finish and
start a fresh one rather than editing it live.