# Croct shows a "probability to be best" and no p-value, how do I read that?

Asked by Cody Marsh on 2026-07-26. Tags: ab-testing, bayesian, statistics.

All my previous tooling reported A/B results as a p-value plus a 95%
confidence interval on the lift, and that is the language my team reasons in.
Croct's experiment view instead gives me a "probability to be best" and a
"potential loss" per variant, and I cannot find a p-value or an interval
anywhere.

Can someone tell me what those two numbers actually mean, and what specifically
makes the tool call a winner?

## 2 answers

### Accepted answer from Croct Bot (2026-07-27)

Croct's experiment statistics are Bayesian rather than frequentist, so there
is deliberately no p-value and no confidence interval to look for. The two
figures you see answer more direct questions:

- Probability to be best is exactly what it sounds like: given the data
  collected so far, the probability that this variant is the top performer.
  It is not one minus a p-value, and it does not depend on a fixed sample
  horizon.
- Potential loss estimates how much you stand to give up if you ship this
  variant and it turns out not to be the best. It bounds the cost of a wrong
  call rather than the chance of one.

You do not set a threshold yourself. Croct flags a recommended winner only
when probability to be best is above 95% and potential loss is below 0.1%,
which together mean "very likely the best, and cheap even if wrong". Until
then the experiment keeps reporting and the estimates sharpen as data
accumulates.

The [Bayesian experiment results](https://docs.croct.com/reference/analytics/experiment)
reference explains each metric, and the
[recommended winner criteria](https://docs.croct.com/reference/analytics/experiment/flags)
cover exactly when the call is made. Results are computed on 100% of your
data and update in real time.

### Answer from elias_k (2026-07-27)

The intuition that helped me most coming from p-values: potential loss is an
expected value, not a worst case. It already weighs how likely the variant
is to be behind by how far behind it would be. So a tiny potential loss is
the tool saying that even in the worlds where you picked wrong, the damage is
negligible, which is often a more useful thing to tell a stakeholder than a
significance flag.