# Ad blockers silently drop a chunk of visitors from our client-side tests, how do you keep samples honest?

Asked by Arvind Nair on 2025-10-29. Tags: ab-testing, ad-blockers, methodology, data-quality.

Framing this as a sampling bias problem because that is what it is. Our
client-side testing tool's script is on common blocklists, so visitors running
ad blockers never enter experiments at all. No error, no log line, they just do
not exist in the results.

The bias is not random either. Blocker usage correlates with technical
audiences, certain browsers, desktop over mobile. So every experiment we run is
measured on a sample that systematically underrepresents part of our
marketplace's users, and we have no way to quantify the gap from inside the
tool itself.

How do teams keep experiment traffic representative? Is this solvable at the
architecture level or does everyone just accept the skew

## 2 answers

### Accepted answer from Yusuf Demir (2025-10-29)

It is solvable at the architecture level, and the split is clean once you see
it. A blocklist can only filter what the browser requests. If the variant
decision happens on your server during the render, the visitor receives the
chosen variant in the HTML regardless of what extensions they run. Server-side
content resolution is unaffected by client blocklists, so exposure to the
experiment is complete by construction. That is the part that fixes your
sampling frame.

The client side does not disappear entirely though, event tracking still
involves browser requests. We use Croct with server-side resolution, and for
the client SDK calls the docs are upfront that EasyList and EasyPrivacy block
some of its endpoints. The documented remedy is first-party proxying: route
the calls through your own domain via the `baseEndpointUrl` and
`cidAssignerEndpointUrl` options, so requests go to your origin instead of a
listed third-party host. Setup details are in this thread on
[ad blockers blocking Croct endpoints](/answers/ad-blockers-blocking-croct-endpoints).

With both pieces in place the blocker population is back in your experiments,
assignment and measurement alike.

#### Reply from Arvind Nair (2025-10-30)

The assignment vs measurement decomposition is very helpful, we had been
treating it as one problem. Will estimate our blocker share first to size
the skew.

### Answer from claire_b (2025-10-30)

To quantify the gap before you migrate anything: compare sessions in your
server logs (or CDN analytics) against sessions your testing tool reports for
the same period and pages. The delta is roughly your invisible population. We
did this exercise and found 11% overall, but 19% on desktop Chrome, which
matched your intuition about the bias being structured. Having that number
made the architecture conversation with leadership very short.