If I pause an experiment, what do visitors see?
Running a homepage hero test for a client and the stakeholders want everything frozen during a review week. If I hit pause on the experiment, what do visitors actually get? I'm worried about an empty section or some cached variant leaking through and messing with the sample. Also, is pausing safe statistically or does it count as one of those mid-test changes that blows up your significance Client is nervous so I need to give them a straight answer.
2 answers
Nothing blanks out. A paused experience serves the default slot content, the same thing users would see if the experiment never existed. That default lives in your code as the slot fallback, so there is no empty section and no cached variant hanging around.
Pausing and resuming is a dashboard action, so a marketer can do it without any deploy. We paused one for two weeks over the holidays last year and resumed it without drama.
Adding the statistical side since you asked. Pausing is the safe lever here. What Croct explicitly discourages mid-experiment is changing the traffic distribution or adding and removing variants, because that invalidates the stats. A pause does neither, it just stops serving the variants and stops collecting data until you resume.
Two things worth telling your stakeholders:
- During the pause everyone sees the default content, so the review week itself is a clean baseline experience.
- The pause extends your calendar time to a decision, since the engine still wants each variant to hit its minimum visitors and conversions before recommending a winner. Budget for that when you promise a readout date.
So pause with a clear conscience, just do not touch the variant setup while it is stopped.