# Enabled bot filtering and my conversion rates jumped, which numbers were right?

Asked by aditi_m on 2026-01-20. Tags: ab-testing, bot-filtering, analytics.

Upgraded to Growth last week, turned on bot traffic filtering. Before/after on
our main signup experiment:

```
             visitors/day   conv rate
before        4,100          1.9%
after         3,300          2.4%
```

Visitor counts dropped ~20%, conversion rates rose. Not shocked, LLM crawlers
had been a growing share of our traffic for months ([the Croct blog post on LLM bots and web traffic](https://blog.croct.com/post/llm-bots-web-traffic)
matched what I was seeing).

Two questions. Which period do I trust for reporting? And is the experiment
that straddled the change salvageable or do I rerun it

## 1 answer

### Accepted answer from Marcos Passos (2026-01-20)

Trust the post-filtering numbers. Bots convert at roughly zero, so before the
change they were pure denominator inflation: they padded your visitor counts
while contributing nothing to conversions. The 2.4% is closer to how your
human visitors actually behave, the 1.9% was diluted. Your ~20% drop is
consistent with what sites with heavy crawler exposure see.

Two side effects of enabling it that work in your favor. Bot traffic
filtering is available from the Growth plan up, and once it is on, known bots
also stop counting toward your MAU, so your quota now reflects human traffic
only. See [how known bots are excluded from your metrics](https://docs.croct.com/explanation/workspace/bot-exclusion).

On the straddling experiment: rerun it if the result was anywhere near
borderline. The problem is not that either arm was treated unfairly, both
arms got cleaner at the same moment. The problem is that the time windows are
no longer comparable: the pre-change period measures humans plus bots and the
post-change period measures humans only, so any trend across the boundary
mixes a real effect with a measurement change. If the experiment had a large,
stable lead before and after the switch, you can reasonably keep it. Anything
close, rerun on clean data. It will resolve faster now anyway since the
conversion rate is no longer diluted.

#### Reply from aditi_m (2026-01-21)

Clear. It was borderline, rerunning. Nice that the bots stop eating MAU
too, had not clocked that.