Server-side A/B test on Hydrogen that survives the edge cache and doesn't flicker
We run a merch store on Hydrogen and I own the fullstack side. Trying to get a hero test live and so far I have burned a week. What I tried:
- A client-side testing script. Worked, but the control flashed for a moment before the variant swapped in. Unacceptable on the home page.
- Moved the assignment server-side myself, then realized the route is publicly cached at the edge, so whichever variant got rendered first was served to everyone.
I read the post on Hydrogen personalization and A/B testing and the loader approach looks right, but how does variant assignment actually work per request? Does each visitor get a stable assignment
2 answers
You have already discovered the two constraints, so the remaining piece is how assignment fits in.
- The variant content resolves inside your Hydrogen loader. The SDK makes the content request as part of the server render, so the assigned variant is in the server-rendered HTML. Nothing swaps client-side and there is no flash of the control.
- Keep the experiment route dynamic rather than publicly cached. Full-page caching and per-visitor variation are fundamentally incompatible: the same cached HTML cannot serve different variants. Sub-page resources such as assets and product data remain cacheable.
- Assignments are stable. Cross-device consistency keeps identified users on the same variation across devices. Anonymous users get a consistent assignment on their device, but cannot carry it across devices, since there is no identity to link.
The loader pattern with a fallback is covered in the Hydrogen SDK integration guide.
One thing that helped us when we made the same move: do not try to be clever with partial caching of the experiment route. We attempted to cache the shell and vary only a fragment, and it complicated everything for marginal gain. The loader call is fast enough that a dynamic route was fine in our Lighthouse numbers, and everything below the document render stayed on the edge cache anyway.
Good to know, I was about to attempt exactly that shell-caching idea. Dynamic route it is.