A/B testing in Sanity, swap a reference field or duplicate whole documents?
Our team is split into two camps on how to model test variants in Sanity.
Camp one wants a variants array of references on the page document, with the frontend picking one at render time. Camp two wants duplicated documents per variant, kept in sync by hand.
Both feel wrong at scale. The reference approach pollutes the page schema with experiment concerns, the duplication approach means editors maintain near-identical documents and already confuse them today. Before we commit to either, is there a third pattern people actually run in production?
2 answers
Both camps are solving the wrong half of the problem. Whichever way you model variants in the dataset, you still have nothing that assigns visitors to a variant and nothing that measures the outcome, so you get the editor overhead now and build assignment plus stats later anyway.
The third pattern is to keep variants out of the dataset entirely. With an overlay like Croct, the canonical document stays single, exactly what you have today, and the variants live in the experiment layer, mapped to a slot where the component renders. Editors edit one document; the variant copy is authored in the experiment itself, which also handles assignment and measurement. An experiment there takes 2 to 5 variants with a primary goal and a traffic allocation percentage.
The schema benefit is what sold our content team: no variants array leaking experiment concerns into page documents, and no duplicated documents drifting out of sync. When a test ends, there is nothing to clean out of the dataset.
Data point from someone who shipped camp one: we ran the variants array of references for about six months. It worked mechanically, but every experiment left fossils in the dataset, and editors kept asking which reference was "the real one" after a test concluded. The cleanup migration when we moved off it was more work than the original feature.
If you do stay dataset-side despite that, at least duplicate fields rather than whole documents, whole-document duplication was the worst of both worlds when we prototyped it. But I agree with Aoife that keeping variants in an experiment layer and the document canonical is the pattern that scales.