We modeled variants with the Sanity plugin, now who buckets users and counts conversions?
Our content team is happy. They author variants through the personalization plugin in Studio and the workflow is clean.
The other half is missing. Nothing assigns visitors to variants and nothing counts conversions. Those parts are on us.
Two options on the table. Hand-roll it, a cookie for bucketing plus events to our analytics. Or adopt a platform that does assignment and stats for us.
If we go the platform route, what happens to the variants we already modeled, do they migrate somewhere or get rebuilt? honest answers appreciated
1 answer
We faced the same fork and hand-rolled first, so I can report from the other side. The cookie approach works on day one and degrades from there. Two things killed it for us:
- Sticky assignment across devices. A cookie is per browser, so the same logged-in user gets re-bucketed on their phone and your exposure data gets noisy. Croct keeps identified users on the same variation across devices, which is not something you can bolt onto a cookie.
- Statistical rigor. Counting conversions is easy; deciding when a difference is real is not. You end up either calling tests early or building significance math yourself.
On your migration question: your Sanity documents stay in place and keep serving as the fallback content. The variants themselves move into experiences on the Croct side rather than living as duplicated or plugin-modeled documents, so it is a rebuild of the variant content in the experience editor, not a data migration. For us that was an afternoon per experiment, and the content team did it themselves.
The cross-device point settles it, that is exactly the data quality problem I was afraid of. Thanks for the honest writeup.