What traffic allocation for an experiment on a niche audience?
We run ABM personalization and I want to experiment on an audience of enterprise accounts that represents roughly 4% of total traffic. The general advice I keep seeing is to allocate around 30% of traffic to an experiment to limit risk.
Does that guidance apply here? If allocation is a percentage of the eligible audience rather than of total traffic, 30% of an already tiny segment seems like it would take months to reach anything conclusive. What is the correct setting for this case
1 answer
Your reading is right: traffic allocation is the percentage of the ELIGIBLE audience that gets entered into the experiment. If you set 30%, then 30% of the visitors matching your enterprise audience see variants and the remaining 70% see the default content. It is not a percentage of overall site traffic.
For a niche audience like your 4% segment, use 100% allocation. Every variant needs to reach at least 1000 visitors and 25 conversions before a winner can be called, and if you feed the experiment only a third of an already small segment, hitting those thresholds takes prohibitively long.
Partial allocation exists mainly to de-risk experiments on high-traffic pages, where you do not want a bad variant exposed to everyone at once. In your case the audience definition is already the risk limiter: only 4% of visitors can ever see the experiment, so there is little left to hedge.