Skip to content

Methodology

How to validate a backtest

By , Canli Capital

Six checks, in the order to run them. Each has a free calculator, and most also run through the API or from an AI assistant, so the same check works in a browser, in your code or in a chat.

  1. Count every version you tried

    A good Sharpe ratio means little until you know how many versions of the strategy were tried to find it. Write down every variant, parameter set and idea you tested, including the ones you dropped.

    Trial accounting explorerTrials summary endpoint

  2. Shrink the Sharpe for that count

    The more versions you try, the better the best one looks by luck alone. The deflated Sharpe ratio corrects for that: it is the chance your Sharpe beats the best that luck would give across that many tries.

    Deflated Sharpe ratio calculatorDeflated Sharpe validation route

  3. Check the winner on unseen data

    Split the history into pieces, pick the best version on some of them, and see how it ranks on the rest. If the winner often lands in the bottom half, the backtest is probably overfit.

    Probability of backtest overfitting calculatorOverfitting validation route

  4. Price in realistic trading costs

    Run the same strategy again with slower fills, wider spreads and market impact. A backtest that always trades at the midpoint is pricing trades nobody could place.

    Execution assumptions cost calculatorNo validation API route publishes this step yet

  5. See what it adds to a portfolio

    A strategy that moves with the ones you already run adds less than its own Sharpe suggests. Check what it adds once its correlation with them is counted.

    Portfolio breadth calculatorBreadth validation route

  6. Keep a receipt

    Each check through the API returns a signed receipt that names the exact code it ran. Keep it, so anyone can recheck the result later instead of trusting a screenshot.

    Signed evidence chain verifierHow the validation API issues a receipt

Short on time? From an AI assistant, audit_backtest runs the first three checks in one call, and backtest_strategy tests a rule over a whole grid of settings and counts every version for you. See the validation MCP server.

What this page does not establish

None of these checks proves a strategy will make money, and none is a forecast. Each one describes your search and your returns exactly as you sent them. A strategy can pass every step and still fail on bad data, costs the calculators do not model, or plain bad luck. This page tells you what to check and in what order, not what a good number looks like.

Where to go next

Every calculator, in one place · The validation API quickstart · Methodology · How to check the published record itself