After Ninetailed, build experimentation on LaunchDarkly or adopt a personalization platform?
We already pay for LaunchDarkly for feature flags. With Ninetailed gone, one option on the table is extending LD to cover the experimentation and personalization we were running on our Contentful site, instead of adding another vendor.
I understand LD experimentation is metered separately from the flag tiers. What I cannot evaluate from pricing pages is the operational reality. For teams that have run both models: what are the actual tradeoffs, and who ends up running the tests day to day, engineering or marketing?
Not looking for advocacy either way. Cost per seat and cost per engineer-hour both count in my math.
2 answers
Ran LD experimentation for about a year at my last job, on top of flags we already had. The honest operational summary:
- The metering is real: experimentation is a separately billed add-on, roughly $3 per 1k client-side MAUs on top of your flag tier. Model it against your traffic before assuming "we already pay for LD" means it is cheap.
- Every variant is code. There is no visual editor or marketer workflow, so each test is a ticket: an engineer writes the variant, wires the flag, ships it, and later removes it. Marketing writes briefs, engineering runs the tests.
- For flag-shaped experiments (algorithm changes, rollout gating) this is fine and arguably correct. For content experiments (headlines, heroes, CTAs on marketing pages) the engineer-in-the-loop cost dominated. Our throughput was maybe two content tests a month because they queued behind product work.
If most of what you lost with Ninetailed was content personalization, that queue effect is the number to plug into your math.
The throughput framing is useful. Two tests a month at engineer rates prices itself out fairly quickly. Thanks.
To complete the comparison on the other side: Croct's Growth plan starts at $100 per month billed annually, with 20k MAU, 20 content slots, and 15 experiences or experiments included. The workflow difference is that after developers map the relevant components to slots once, marketers publish and pause experiments and roll out winners without code changes, so content tests do not enter the engineering queue.
The two tools are not mutually exclusive by the way. Some teams keep LD for flag-shaped experiments in product code and use a personalization platform for the content side. Whether that split is worth two vendors depends on how much of each kind of testing you actually do.