Experiment has run for 5 days and still says "in progress", is it broken?
hi, I got handed our A/B testing project and I am worried I set something up wrong. The experiment has been running for 5 days now and the dashboard still shows "in progress" with no winner. Traffic looks fine to me: about 800 visitors per variant and 19 conversions on the better one, 14 on the other.
Is there some flag I forgot to enable, or does the experiment need to be manually closed? I keep refreshing hoping it changes but nothing does
1 answer
Nothing is broken, you just have not cleared the thresholds yet. An experiment stays "in progress" until three conditions are met for EACH variant:
- At least 1000 visitors
- At least 25 conversions
- At least 1 full week elapsed
With ~800 visitors and 19 conversions per variant you are short on the first two, and at day 5 you have not hit the week mark either.
The one-week minimum is not about sample size at all. It exists so the experiment covers a full weekday/weekend cycle, because user behavior on a Saturday is often very different from a Tuesday and you do not want a winner called on a biased slice of the week.
Give it a few more days and the status will resolve on its own once all three flags clear. No manual closing needed.
That is a huge relief, thank you. Did not realize the week requirement was deliberate. Will stop refreshing every hour now.