Funnel exploration in GA4 broken down by experiment variant, sanity check my setup
We are running a checkout experiment in Croct. The results panel gives me the topline conversion per variant, which is fine, but I want to see where exactly variant B loses people between basket and payment. The obvious tool is a GA4 funnel exploration.
Two questions. First, can the variant data Croct exports be used as a breakdown dimension in a funnel exploration, or is it limited to standard reports? Second, should I expect the funnel's conversion numbers to match what Croct reports, or will there be some drift between the two
I have read the analytics documentation but it does not cover funnels specifically.
2 answers
Yes to the first question. The exported variant assignments arrive in GA4 as event data, and once registered they work as a breakdown dimension in funnel explorations the same way as any other dimension. So you can rebuild your basket-to-payment funnel and break every step down by variant. That is exactly the kind of analysis the export is meant for.
On the second question: expect small discrepancies, and do not chase them. GA4 drops events beyond 100k per user per day and explorations can be sampled, so the populations will never line up perfectly with Croct's numbers. Croct's own results are unsampled, so treat those as the source of truth for the win/lose decision and use the GA4 funnel for the diagnostic question of where the drop happens.
That distinction is helpful. Decision from Croct, diagnosis from GA4. Rebuilt the funnel this morning and variant B loses people at the delivery options step. Thanks.
Adding one thing before you dig deeper in GA4: Croct's results panel has drill-downs of its own. The experiment results are Bayesian and unsampled, and you can break them down by device, audience, time window, and custom events. If you track an event at each funnel step and hook them up as goals, you can often answer the "where does B lose people" question without leaving Croct. There is a thread on tracking a goalCompleted event on a button click that covers the wiring.