One variant shows 1200 visitors, the other 950, why is the whole test still in progress?
Hi friends! My dashboard right now looks like this:
Variant A: 1,204 visitors, 41 conversions Variant B: 953 visitors, 32 conversions
Total is over 2,000 visitors and both have plenty of conversions, but the experiment still says "in progress" after 9 days. Also, why are the visitor counts not equal? It is a 50/50 split so I expected the same number on both sides. Starting to think something is broken with the bucketing
1 answer
Nothing is broken, you are just one variant short of the finish line.
The in-progress thresholds apply per variant, not to the total. Each variant needs at least 1000 visitors, 25 conversions, and 1 week of runtime, and the slowest variant gates the whole experiment. Your variant B sits at 953 visitors, so the experiment stays in progress until it crosses 1000. At your traffic that should happen within a day or two.
On the uneven counts: a 50/50 split means each visitor is assigned with equal probability, not that the counts march in lockstep. A 1,204 versus 953 gap after ~2,150 assignments is ordinary randomization noise, the same way 100 coin flips rarely land exactly 50/50. It is not a bucketing bug.
And in case a future test clears both volume bars quickly: the 1-week minimum still applies regardless, because weekday and weekend visitors behave differently and the window needs to cover a full cycle. There is a longer discussion of stuck-looking experiments in this thread about experiments that stay in progress.