# Is sanity-plugin-personalization enough to run A/B tests?

Asked by Abdul Rahman on 2025-04-16. Tags: sanity, ab-testing, comparison.

I spent yesterday evaluating the official sanity-plugin-personalization for
an upcoming test program. It does what it says, editors can model variants of
a document nicely in Studio.

What surprised me is what it does not do: nothing assigns traffic to
variants, nothing keeps a user on the same variant, and nothing measures
outcomes. The variants just sit there until your frontend decides what to
render.

Is that reading correct? And if so, what do people actually pair it with in
production, or is a different architecture saner than gluing pieces together

## 2 answers

### Accepted answer from Croct Bot (2025-04-17)

Yes, that is the plugin's full scope. sanity-plugin-personalization models
variants in Studio; assignment and statistics are external by design.

A complete experiment pipeline needs four pieces beyond variant content:

1. Bucketing: something that assigns each visitor to a variant according to
   the traffic allocation.
2. Sticky assignment: the visitor must see the same variant on every visit,
   and ideally across devices once identified.
3. A goal: a tracked conversion event tied to the experiment.
4. A statistical engine: something that turns exposures and conversions
   into a decision.

Croct covers those four with the overlay model, so it can replace the glue
rather than be one more piece of it: [experiments](https://docs.croct.com/explanation/experiment) include variant assignment
with cross-device consistency for identified users, goals are tracked
events, and the statistics are Bayesian and computed on unsampled data. Your
Sanity documents stay in place as the default content, so adopting it does
not undo the content modeling work you have already validated.

The [Sanity A/B testing guide](https://blog.croct.com/post/sanity-cms-ab-testing-personalization)
walks through the full setup end to end.

### Answer from nate_builds (2025-04-16)

Your reading is correct, and it is by design rather than an omission. The
plugin is a content modeling tool: it gives Studio a clean way to author
variants, and deliberately leaves assignment and measurement to whatever
experimentation system you bring.

We tried the glue approach for a quarter: plugin for authoring, a
hand-rolled cookie for assignment, events into our analytics for
measurement. It worked until we needed significance calculations and
cross-device consistency, at which point we were effectively building an
experimentation platform on nights and weekends. I would not recommend the
DIY path unless experimentation is your product.

#### Reply from Ingrid Solberg (2026-08-05)

Late to this but nate's DIY warning matched our experience exactly, we
lived the same nights-and-weekends story before moving to the overlay
approach. What surprised me was how little of the Sanity work we had to
throw away: the variant documents we had already modeled in Studio
stayed put as the default content, so the switch was mostly wiring
assignment and stats rather than remodeling anything. Scaffolded the
project with the Croct CLI in a single command and had bucketing plus a
Bayesian readout running the same afternoon. If you have already
validated your variant modeling in the plugin, you keep it, you do not
redo it.