Deflated Sharpe Ratio: Test If Your Best Result Is Luck
The deflated Sharpe ratio answers a question most backtest reports ignore: how many attempts did it take to get this result? Test 50 versions of a strategy and the best one will look impressive, even if none has any edge. A Sharpe ratio of 1.8 means something very different after one test than after fifty.
What Is the Deflated Sharpe Ratio?
The deflated Sharpe ratio (DSR) is a probability. It estimates the chance that a strategy’s true Sharpe ratio is above zero, once you allow for how many variants were tested to find it. David Bailey and Marcos López de Prado published the method in The Journal of Portfolio Management in 2014.
The ordinary Sharpe ratio measures return per unit of volatility. The deflated version takes that figure and adjusts it for four things:
- Number of trials: how many variants you tested.
- Spread of results: how far apart those variants’ Sharpe ratios were.
- Sample length: how many return observations you have.
- Return shape: whether returns have a negative skew or fat tails.
A DSR of 0.95 means about a 95% probability that the result reflects more than selection luck. That is a common bar, though it is a convention rather than a law.
Why the Best Backtest Is Usually Flattered
Picking the top result from a list is a selection process. The winner gets credit for its genuine quality, if it has any, plus all of its good luck. Its luck will not carry over into live data.
You can estimate the size of this effect. Take two years of daily returns from strategies with no edge at all. Their annualised Sharpe ratios still scatter around zero by about 0.7, purely from noise. Now look at the expected best score as the number of attempts rises:
| Independent variants tested | Expected best Sharpe from pure noise |
|---|---|
| 10 | 1.11 |
| 20 | 1.34 |
| 50 | 1.61 |
| 100 | 1.79 |
| 500 | 2.16 |
Our calculation, using the expected-maximum approximation from the DSR method. It assumes two years of daily returns, independent variants and no true edge.
So fifty attempts can produce a Sharpe above 1.6 from nothing. Traders who would dismiss that as luck in a single test often celebrate it after a long optimisation run. This is data snooping expressed as a number.
How Does the Deflation Work?
The calculation has two stages, and both make sense without the formula.
First, it sets a higher hurdle. Instead of asking whether your Sharpe beats zero, it asks whether it beats the expected best score from your number of trials. That is the table above.
Second, it asks how confident you can be about clearing that hurdle. Longer samples raise confidence. Negative skew and fat tails reduce it, because they make the measured Sharpe less reliable.
A Worked Example
Suppose your best variant has a Sharpe of 1.8 over two years of daily returns. Here is the same result under different testing histories, using our own illustrative calculation:
| Variants tested | Noise hurdle | DSR |
|---|---|---|
| 1 | 0 | 0.99 |
| 10 | 1.11 | 0.83 |
| 50 | 1.61 | 0.61 |
| 200 | 1.96 | 0.41 |
The backtest report is identical in all four rows. Only the history behind it has changed. At 200 variants, the same 1.8 is more likely luck than not.
Return shape barely moved this example. Setting skew to minus 1 and kurtosis to 10 took the 50-variant DSR from 0.61 to about 0.60. Shape matters more with fewer observations, such as a low-frequency strategy.
What Counts as a Trial?
This is the hard part, and the answer decides whether the number means anything.
Count every variant you evaluated, including the ones you abandoned. Changing a lookback from 14 to 21 is a trial. Swapping an entry rule is a trial. Trying the idea on another market and keeping the better one is a trial too.
Variants are rarely independent, though. An RSI of 14 and an RSI of 15 are close to the same test. Counting them separately overstates your search. Later work summarised here groups similar strategies into clusters to estimate an effective number of trials.
A practical approach is to record both numbers:
- Raw count: every variant you ran.
- Effective count: the number of genuinely different ideas.
If the result only survives using the lower count, treat it with suspicion.
AI Makes the Count Climb Faster
Connected AI assistants can run dozens of variants in an afternoon. That is useful, but it can multiply the problem just as quickly. Ask the assistant to report every variant it tried, not just the winner.
Where the DSR Falls Short
The DSR is an estimate that rests on honest inputs. You can only count the trials you recorded. Also, a daily sample with long stretches of no position has fewer useful observations than its length suggests. A strategy with 30 trades over two years deserves extra caution, whatever the DSR says.
It also cannot replace new data. A high DSR makes luck less likely, but it says nothing about whether the market will change.
How to Apply the Deflated Sharpe Ratio in Arrow Algo
The drag-and-drop visual builder makes variants cheap to create, so the discipline has to come from your process. Treat the DSR as a calculation you run alongside the report:
- Fix the hypothesis. Write down the idea and the settings you plan to vary before you connect any visual blocks.
- Reserve untouched data. Keep a final period out of all testing. Our in-sample vs out-of-sample guide explains how to split it.
- Log every backtest. Record the settings and Sharpe ratio in a simple spreadsheet, including the failures.
- Compare the best with the noise hurdle. Use the trial count and the spread of Sharpe ratios in your log.
- Inspect the neighbours. A real effect usually holds up in nearby settings. An isolated spike is a warning.
- Run the single survivor once on untouched data. If you adjust it afterwards, that data no longer counts as untouched.
A connected AI assistant can help with the arithmetic if you give it the log. Check its assumptions as you would any other calculation.
If you later choose to deploy a strategy, your rules signal your own exchange account and the exchange decides whether to execute.
What Are the Key Takeaways?
- The deflated Sharpe ratio estimates whether a result beats what your number of tests would produce by luck.
- Fifty independent no-edge variants over two years can be expected to produce a best Sharpe near 1.6.
- The same Sharpe of 1.8 can look strong or weak depending on the testing behind it.
- Count every trial, including abandoned ones, and record a separate effective count for similar variants.
- Treat the DSR as a filter. Untouched data still has the final say.
Disclaimer: This content is for educational purposes only and does not constitute financial advice. Trading involves significant risk and you should only trade with capital you can afford to lose. Past performance is not indicative of future results. Always conduct your own research before making any trading decisions.
Ready to build your own automated trading strategies without writing a single line of code? Start for free at Arrow Algo and join thousands of traders who've made the switch to systematic trading.