Croct experiment conversions vs Shopify analytics orders, why the gap?
I am reconciling a finished experiment before presenting the results internally. Same window, Feb 2 to Feb 15:
- Shopify analytics: 1,412 orders
- Croct experiment: 1,287 conversions across both variants
That is a delta of 125 orders, about 8.8%. Before I put these numbers in front of the team I want to understand what each system counts. Is this expected, and what are the usual causes of teh difference?
1 answer
A single-digit gap between two independent collection systems is normal, and in your case there are three likely contributors:
-
Ad blockers. Croct analytics are unsampled, so nothing is dropped on Croct's side, but browser extensions using lists like EasyPrivacy can block the tracking endpoints entirely. Those visitors convert in Shopify but never send events to Croct. The remedy is first-party proxying: route requests through your own domain via the baseEndpointUrl option so the blocklists no longer match.
-
Bot filtering. On paid plans, known bots are excluded from Croct metrics. Storefront analytics may still count some of that traffic, which inflates the other side of the comparison.
-
Session boundaries. Croct sessions close after 30 minutes of inactivity by default. If Shopify attributes an order to a visit that Croct already considers a separate session, the conversion can land outside the experiment window on Croct's side.
For the presentation, report the Croct numbers for the variant comparison (both variants lose ad-blocked traffic equally, so the lift is unbiased) and note the reconciliation reasons as a footnote.