Skip to content

Methodology

How to validate a backtest

By , Canli Capital

Each step below runs through one of this site's own instruments, in the order they are meant to be used. Every calculator also runs as a validation API route where one is published, so the same check can run in a browser or inside a pipeline.

  1. Count the trials

    A Sharpe ratio means little without knowing how many variants were tried to find it, so start by counting every hypothesis in the search.

    Trial accounting explorerTrials summary endpoint

  2. Deflate the Sharpe

    Put the observed Sharpe ratio and that trial count through the same deflation the search should have been judged by from the start.

    Deflated Sharpe ratio calculatorDeflated Sharpe validation route

  3. Estimate overfitting probability

    Check whether the in-sample winner predicts anything out of sample, rather than assuming a high Sharpe ratio implies it does.

    Probability of backtest overfitting calculatorOverfitting validation route

  4. Price execution assumptions

    Reprice the same strategy under different fill, delay and impact assumptions, because a backtest that fills at the mid is pricing a trade nobody could place.

    Execution assumptions cost calculatorNo validation API route publishes this step yet

  5. Check breadth

    Ask what this strategy actually adds to a book of others once its correlation to them is accounted for, not just what it earns alone.

    Portfolio breadth calculatorBreadth validation route

  6. Keep a receipt

    Every validation call above returns a content-hashed receipt naming the exact source it ran; keep it, and verify a signed record stays unchanged rather than trusting a screenshot of it.

    Signed evidence chain verifierHow the validation API issues a receipt

What this page does not establish

None of these steps establish that a strategy is profitable, admissible for capital, or a forecast of anything. Each is a diagnostic about a search and a return series exactly as submitted; a strategy can clear every step here and still fail on data quality, execution realism the calculators do not model, or plain bad luck. What counts as a passing result is a question for the reader's own research programme; this page states what to check and in what order, not what a good number looks like.

Where to go next

Every calculator, in one place · The validation API quickstart · Methodology · How to check the published record itself