All questions

First A/B test on a VTEX product page, what does the flow look like end to end?

Asked by Diego Salamanca on

DSDiego Salamanca

The tag is installed and events are flowing on our VTEX store, so I am ready to run my first real A/B test on a product page. I have never set one up before and I would love a clear step-by-step, from picking the slot all the way to reading the result.

Sorry if this is obvious, I just want to get the sequence right the first time.

Was this helpful?

2 answers

MPMarcos PassosAccepted answer

Not obvious at all the first time, so here is the whole sequence.

Start by wiring the slot on the product page, the specific placeholder where the variant content will render. Then build an experience for that slot, and attach an experiment to it. On the experiment you set the primary goal; because your VTEX order placement records a conversion automatically as Goal Completed, that is the natural conversion goal for a product-page test. You also set your traffic allocation and add your variants there.

Then let it run. The experiment stays in progress until each variant reaches at least 1000 visitors, at least 25 conversions, and at least one week, so do not read into it before those thresholds. The results are Bayesian and unsampled, and a recommended winner appears once a variant's probability to be best passes 95%.

The experiment setup steps walk through attaching it to the experience, and reading experiment results covers the analytics view once traffic is in.

Was this helpful?
RARenata Alencar

For anyone else running their first one: do not touch traffic allocation or add and remove variants once it is live. Changing those mid-flight invalidates the stats and you effectively restart the reads. Set them at the start and leave them.

Was this helpful?
Still have questions?