All questions

We built A/B tests with middleware rewrites and cookies, is a slot-based approach actually simpler?

Asked by Nia Griffiths on

NGNia Griffiths

Some context first because I think it matters. I consult for a mid-size B2B company running the classic Vercel pattern: middleware picks a variant, rewrites the request to a variant page, and a cookie keeps the assignment sticky. It worked fine with one test. Now they run two concurrent tests and the repo has four page permutations, plus hand-rolled exposure events wired into their analytics. Marketing cannot change a headline without a deploy, so every iteration goes through an engineering ticket.

Before I recommend moving them to Croct slots I want to sanity check the model. Does a slot-based setup genuinely remove the permutation problem, and what happens to assignment stickiness that the cookie currently handles?

Was this helpful?

2 answers

DODan OkaforAccepted answer

It removes the permutation problem, yes, because the unit of variation changes. With rewrites, the unit is a page, so n concurrent tests means up to 2^n prebuilt pages. With slots, the unit is a placeholder inside one page: the experiment swaps which component content the slot serves, and two concurrent tests are just two slots on the same route. No page matrix.

On the workflow side, experiments live in the dashboard. Marketers publish, pause, and roll out a winner without a deploy, which directly addresses your ticket problem. Goal tracking and Bayesian stats are built in, so you can delete the hand-rolled exposure events instead of maintaining them around rewrites.

Stickiness: variant assignment is handled by the platform, so you drop your own cookie logic. For identified users the assignment is also consistent across devices, which cookies alone never gave you. The Next.js SDK announcement post walks through how content resolves server side, worth a read before you scope the migration.

Was this helpful?
NGNia Griffiths

That matches what I hoped. I will pilot it on the pricing page test first and keep the rewrite setup running in parallel for one cycle.

Kkylemp

One caveat from having done a similar migration: rewrites still make sense when the variants are genuinely different pages, like a full landing page redesign with a different layout and components. Slots shine when variants share the page skeleton and differ in sections. Also note that fetchContent reads the request, so routes using it render dynamically. For a marketing site that is usually fine, but check it against any ISR assumptions before you move everything over.

Was this helpful?
Still have questions?