# How do I keep a holdout group to measure our personalization program overall?

Asked by Ingvar Thorsen on 2026-07-07. Tags: ab-testing, methodology, personalization.

I run growth at a media group in the Nordics, and individual test results are
not my problem. The board question is: does the personalization program as a whole
pay for itself. For that I need a slice of visitors who see none of it, a
global holdout, so I can report program-level lift rather than a stack of
individual test wins that may overlap.

Does Croct support a holdout group as a concept, or do I need to construct one
myself? And if I construct it, what is the recommended mechanism so the
comparison holds up in an org-level report

## 1 answer

### Answer from samwhit (2026-07-08)

Traffic allocation is the built-in lever for this. When you configure an
experiment you set what share of traffic participates, for example allocating
30% to the experiment means the remaining 70% sees the default content. That
untouched share is functionally your holdout: a randomly assigned group that
experiences none of the variants, giving you the comparison baseline.

Allocation is set as part of experiment configuration, alongside the primary
Goal ID and the 2 to 5 variants. One rule matters a lot for the kind of
report you are writing: do not change the allocation once the experiment is
running. Adjusting the split mid-flight invalidates the statistics, so pick
the holdout size you can defend and leave it alone for the full run.

For the program-level question specifically, look at journey-level
experiments. Croct can run experiments across full multi-page journeys rather
than a single slot, so your holdout comparison can span the whole funnel a
visitor moves through instead of measuring one hero at a time. That maps much
more cleanly onto "does the program pay" than aggregating per-slot results,
because it captures how the experiences interact along the path.