Should feature rollouts and A/B tests live in the same tool?
I am mid-way through a tooling consolidation review. Today we run a feature flag SDK for engineering rollouts and a separate testing tool for marketing experiments. The flag vendor bills experimentation as a separately metered add-on on top of our seat tier, which is what triggered this review in the first place.
I see Croct lists feature flags in its feature set. What I cannot tell from the marketing pages: do those flags actually replace a dedicated flag vendor for gradual rollouts, or are they only there to support the experimentation side? And if both live in one tool, how do teams keep the engineering and marketing workflows from stepping on each other?
2 answers
Feature flags and server-side testing are part of Croct's A/B testing feature set, alongside experiment scheduling and bot filtering. The practical consequence of that packaging is the thing you are actually buying: gradual rollouts and experiments share one assignment and analytics layer. A rollout and a test bucket users the same way and report into the same unsampled analytics, so you are not reconciling two vendors' numbers when a rollout becomes a test.
On workflow separation, the split that works in practice is by surface, not by tool. Marketers own experiences and experiments in the dashboard, where they can publish, pause, and roll out winners without code changes. Engineering keeps flags in code. Both evaluate against the same audiences, so "beta users" means the same population whether a flag or an experience references it.
Whether it fully replaces your flag vendor depends on what you use beyond basic gating, so I would inventory your flag usage before deciding.
The shared assignment and analytics layer is the argument I was missing for the consolidation doc. Running the flag usage inventory now.
One more input for the review: the two workflows genuinely differ in a few places (lifecycle, ownership, what "done" means), and it helps to name those before consolidating rather than after. There is a good breakdown in Croct's post on feature flags versus A/B testing that we used to decide which of our flags were really experiments in disguise. About a third of ours were, which changed the math on keeping a separate flag vendor at all.