Answers about statistics
Questions and answers from the Croct community about statistics.
Croct shows a "probability to be best" and no p-value, how do I read that?
All my previous tooling reported A/B results as a p-value plus a 95% confidence interval on the lift, and that is the language my team reasons in. Croct's…
How do I read the experiment results for my Sanity hero test?
Running my first hero test on our Sanity site. Five days in, dashboard still says "in progress". Two questions: What exactly unlocks a recommendation? Is it…
Does 90% probability to be best mean the same as 90% confidence?
I support a marketing team that is used to seeing "significant at 95% confidence" in every results deck. Before Croct I would run their raw numbers through an…
What do "probability to be best" and "potential loss" actually mean?
Our previous experimentation setup was a LaunchDarkly add-on (billed separately on top of our flag seats), and everything there was p-values and confidence…
Is A/B testing even worth it on 8k visitors a month?
My site does about 8k visitors a month and roughly 120 conversions, and that is not a typo, we sell a niche B2B tool with a long sales motion. If I run a…
Experiment has run for 5 days and still says "in progress", is it broken?
hi, I got handed our A/B testing project and I am worried I set something up wrong. The experiment has been running for 5 days now and the dashboard still…
One variant shows 1200 visitors, the other 950, why is the whole test still in progress?
Hi friends! My dashboard right now looks like this: Variant A: 1,204 visitors, 41 conversions Variant B: 953 visitors, 32 conversions Total is over 2,000…