Skip to the lab

Open research instrumentReference implementation

Probability of backtest overfitting calculator

A deflated Sharpe ratio asks whether one result survives the search that produced it. This asks a sharper question: across every configuration you tried, does the one that ranked best on your in-sample data also rank well out of sample, or did it just win a popularity contest against the noise in that one split? Paste a matrix of per-period returns, one column per configuration, and it runs Combinatorially Symmetric Cross-Validation (CSCV) entirely in your browser. Canli Capital, the research house that publishes ALPHAC, runs this calculator free, with no account or key required.

What this is. This runs the production Combinatorially Symmetric Cross-Validation implementation on whatever variant returns you submit. It does not know where your matrix came from, cannot detect lookahead in how it was built, and a low probability of overfitting is not proof a strategy is deployable. It is one diagnostic about whether an in-sample ranking predicts an out-of-sample one, nothing more.

The calculator

Probability of backtest overfitting
...
Splits evaluated
...
Sampler
...
Block length
...
Mean in-sample Sharpe of the winner
...
Mean out-of-sample Sharpe of the winner
...
Share of splits where it degraded out of sample
...

Plain reading

Run CSCV to see the plain-language reading of the result above.

What CSCV does, step by step

Step 1

Split the T rows into S contiguous equal blocks (S even). Drop any remainder rows so every block has exactly T div S rows.

Step 2

For each way of choosing S/2 blocks as in-sample, form the in-sample matrix from those blocks and the out-of-sample matrix from the rest.

Step 3

Take the in-sample winner: the configuration with the highest in-sample Sharpe.

Step 4

Find that same configuration's rank among all configurations out-of-sample, ascending, and take the logit of its relative rank.

Step 5

PBO is the fraction of splits where the in-sample winner lands in the worse half out-of-sample.

Same core as the API. a line-for-line port of alphaforge.validation.pbo.pbo_cscv, the same core the validation API's POST /api/v1/validate/overfitting route imports, so a browser result and an API result on the same matrix agree. when C(n_splits, n_splits / 2) is at most the combinations ceiling, every split is enumerated and the result is bit-comparable to the Python reference. Above that ceiling, n_splits / 2 sized subsets are sampled without replacement by this repo's mulberry32 generator, and the result is reported as exhaustive: false, agreeing with the Python reference within sampling noise rather than bit for bit.

Sandbox contract. the matrix and parameters you enter never leave your browser; nothing is uploaded. no write path to any ledger, artifact, broker or published record exists. export downloads a JSON receipt of exactly what you entered and what it computed.

Rendered from backtest_overfitting_calculator_contract.json, whose worked example is computed at build time from the published golden vector and checked against the production expectation before this page is written. Reference: Bailey, Borwein, Lopez de Prado and Zhu (2017), The Probability of Backtest Overfitting, Journal of Computational Finance 20(4), 39 to 69. Run the same matrix through the validation API, read the arithmetic behind the search in the arithmetic of not fooling yourself, or check one Sharpe at a time with the deflated Sharpe calculator.