Running two experiments on different slots at the same time, do they interfere?
I ran experimentation programs at my previous company where we serialized everything out of interaction-effect paranoia, and it throttled the roadmap badly. Now I want a hero test and a pricing-page test live during the same weeks.
Two questions. Does Croct do anything to prevent the same visitor from entering both experiments, or is overlap allowed? And in your experience, how much do interaction effects actually matter in practice versus in theory
1 answer
Overlap is allowed and expected. Experiments attach to experiences on specific slots, so tests on different slots run independently, and a visitor can be in both simultaneously. There is no cross-experiment exclusion mechanism because for most programs you do not want one.
On the practical question: under randomized assignment, interaction effects mostly wash out. Visitors from both hero variants are distributed evenly across both pricing variants, so any interaction shows up symmetrically and does not bias either experiment's comparison. The theoretical concern is real but the measured impact is usually noise.
The case where serializing is still worth it is when both tests touch the same funnel step, for example two experiments both trying to change checkout behavior, where a true interaction could matter for the decision itself.
If you want to inspect it rather than trust it, results drill down by device, audience, time window, and custom events, so you can check each experiment's segments separately and see whether anything looks asymmetric.