# GA4 says variant B lost, Croct says it won, which number do I trust?

Asked by owenmcb on 2026-04-15. Tags: ga4, attribution, experiments.

Croct's experiment results show a 4 percent lift for variant B with a
recommended winner badge. GA4, looking at the exported variant dimension against
the same conversion event, shows the conversion rate basically flat.

I have been doing attribution long enough to know that when two tools disagree,
the usual answer is "both, in their own special way". But I have to report this
to leadership next week and "the numbers are vibes" is not a slide I can
present.

How do people here reconcile the two? Is one of these numbers actually wrong or
am I comparing populations that were never going to match

## 2 answers

### Accepted answer from Croct Bot (2026-04-16)

You are comparing populations that were never going to match, but the gap has
identifiable causes.

The two pipelines process different data. Croct's experiment results are
unsampled Bayesian statistics with [bot filtering](https://docs.croct.com/explanation/workspace/bot-exclusion) applied. GA4 applies its own
processing, drops events beyond 100k per user per day, can sample
explorations, and groups high-cardinality dimension values under "(other)",
which silently removes rows from a variant breakdown. On top of that, ad
blockers block analytics endpoints differently per tool, so each tool sees a
slightly different slice of your traffic to begin with.

For the slide: the number built for this decision is the experiment-scoped
one. The [recommended winner badge](https://docs.croct.com/reference/analytics/experiment/flags) means variant B crossed Probability to Be
Best above 95% with potential loss below 0.1%, computed on 100% of the
experiment's data. GA4 is a general-purpose tool answering a broader question
on a differently filtered population; flat-ish there does not contradict a 4
percent lift measured on the experiment population.

Related discussion on the export-side caveats:
[GA4 Croct dimensions caveats](/answers/ga4-croct-dimensions-caveats).

### Answer from ben_o (2026-04-17)

A pattern that has saved me in these meetings: agree on the decision source
before the experiment starts, not after the numbers disagree. We write it
into the experiment brief, results in the experiment platform decide
ship/no-ship, analytics tools are for diagnosis and follow-up questions.
When the tools inevitably differ, nobody relitigates the decision because
the hierarchy was agreed upfront.