First A/B test on a hero inside a Strapi dynamic zone, am I doing this right?
hi all, intern here and I got handed the A/B testing project. I followed the Strapi A/B testing guide and got this far:
- Mapped our homepage hero (a component in a Strapi dynamic zone) to a Croct slot
- Created an experience in the dashboard
- Attached an experiment with 3 variants of the hero copy
- Set the primary Goal ID to hero-cta-click
The variants are serving, I can see them rotate in incognito windows. But the results page shows zero conversions even though people are definitely clicking the CTA. Do variants and goals connect somewhere I missed? Sorry if this is obvious, its my first experiment ever.
2 answers
Your setup order is right, experiments attach to experiences, so steps 1-3 are fine. The missing piece is step 4: the primary Goal ID must reference an event that is already being tracked. Setting hero-cta-click in the dashboard does not create the event, it just tells the experiment which event to count.
So you need to fire the event yourself when the CTA is clicked:
croct.track('goalCompleted', {goalId: 'hero-cta-click'});Once that lands, conversions start attributing to whichever variant the visitor was assigned. Clicks that happened before you added the tracking call are gone, the experiment cannot count events it never received.
One more thing since it is your first experiment: do not expect a verdict fast. Even with conversions flowing, results take time to become meaningful.
that was it, I never fired the event at all. added the track call and conversions are showing now. one follow up, variant B is already ahead, can I shift more traffic toward it while the test runs?
Please don't. Changing traffic distribution or adding/removing variants during an experiment is strongly discouraged, it invalidates the statistics. Let it run to completion, then roll out the winner.
Adding a note on timing expectations, because this trips up a lot of first-timers. An experiment stays "In progress" until each variant has at least 1000 visitors, at least 25 conversions, and at least one week has passed. With 3 variants that is a decent amount of traffic, so if your homepage is not that busy, consider whether two variants would get you an answer faster. The supported range is 2 to 5 variants, but fewer variants means each one accumulates data quicker.