# Are experiment results sampled, and do bots pollute conversion numbers?

Asked by rcalloway on 2026-04-21. Tags: ab-testing, analytics, bot-filtering.

Two questions before we commit to running experiments here.

1. Are results computed on sampled sessions above some quota? Previous tool
   we used sampled everything past a monthly cap and it made segment
   drill-downs useless.
2. How is bot/crawler traffic handled? We once had a scraper hammer exactly
   one arm of a test and it inflated that variant's traffic numbers for a
   week before anyone noticed

Want the actual behavior, not the marketing page version.

## 1 answer

### Answer from Nadia Osman (2026-04-21)

On sampling, results are unsampled. Analytics are computed on 100% of
the data in real time, there is no session quota past which it starts
extrapolating. So the drill-downs (by device, audience, time window, and
custom events) operate on the full dataset, which is exactly what you
want for the sanity checks you are describing.

On bots, filtering is built in on the Growth plan and above. Known bot
traffic is excluded, so a crawler hitting one arm does not inflate that
variant the way it did in your old setup. A related detail that matters
for billing: known bots do not count toward MAU on paid plans either, so
scraper spikes are not eating your user quota.

For your specific scraper scenario I would still keep the habit of
drilling into results by device and time window when something looks off.
Filters catch known bots, but slicing the data is how you spot anything
unusual regardless of tooling.