Answers about bayesian
Questions and answers from the Croct community about bayesian.
Moving off Optimizely, how different is Croct's Bayesian approach in practice?
We are finally leaving Optimizely and I am the one who has to retrain the team on reading results. Everything we did there was frequentist: wait for…
Croct shows a "probability to be best" and no p-value, how do I read that?
All my previous tooling reported A/B results as a p-value plus a 95% confidence interval on the lift, and that is the language my team reasons in. Croct's…
Does 90% probability to be best mean the same as 90% confidence?
I support a marketing team that is used to seeing "significant at 95% confidence" in every results deck. Before Croct I would run their raw numbers through an…
What do "probability to be best" and "potential loss" actually mean?
Our previous experimentation setup was a LaunchDarkly add-on (billed separately on top of our flag seats), and everything there was p-values and confidence…
Variant B is clearly losing, can I kill the test before the thresholds clear?
Client experiment I am consulting on, 9 days in, variant B is converting well below control and the gap has not narrowed once. Every day it runs the client…