The cause is simple. Every setting you adjust and every filter you add gives a strategy one more way to fit the past. Fit enough of them and the strategy will match the history almost perfectly, including the parts of the history that were pure chance and will never repeat. Testing tools make this worse: when a backtest takes seconds, it is easy to try hundreds of versions and keep the best. This guide explains how overfitting happens, how researchers measured it, the warning signs, the tests that catch it, two worked examples, how it shows up in Indian option strategies, and the habits that keep strategies robust.
Overfitting at a glance
| Part | What it means |
|---|---|
| What it is | A strategy fitted to the noise of past data rather than to a lasting pattern |
| How it happens | Too many settings, too many versions tried, the best one kept |
| The symptom | A strong backtest followed by weak live results |
| The main tests | Out-of-sample testing, walk-forward testing, parameter stability, other periods and markets |
| The main defence | Few rules, a reason for each, a test plan fixed in advance, data held back |
| Related traps | Look-ahead bias, survivorship bias, ignoring costs |
A short history of the problem
Scientists have long known that a model with enough adjustable parts can fit any set of observations. The statistician's phrase for it is "curve fitting": draw a line through enough points and it will pass through all of them, while telling you nothing about the next point.
In finance the problem became known as data snooping. In 1990 Andrew Lo and Craig MacKinlay showed how results in academic studies of stock returns could be inflated when researchers chose what to test after looking at the data.1 In 1999 Ryan Sullivan, Allan Timmermann and Halbert White tested thousands of technical trading rules on a century of US stock index data and found that much of the apparent success of the best rules disappeared once the number of rules tried was taken into account.2 White's "Reality Check", published in 2000, gave a formal test for whether the best of many strategies is better than chance.3
The problem returned in a new form as computers and data grew. In 2014 David Bailey, Jonathan Borwein, Marcos López de Prado and Qiji Jim Zhu published "Pseudo-Mathematics and Financial Charlatanism", showing how quickly a researcher who tries many strategy versions will find one with an impressive backtest by luck alone.4 With the same group they proposed ways to measure the probability that a backtest is overfitted, and Bailey and López de Prado proposed the deflated Sharpe ratio, which marks a backtest's result down for the number of versions tried.5 In 2016 Campbell Harvey, Yan Liu and Heqing Zhu argued that, because so many factors had been tested in academic finance, a new finding should clear a much higher statistical bar than the traditional one.6
For individual traders the lesson arrived with cheap backtesting software. A trader today can run more tests in an afternoon than a research desk ran in a year in the 1980s, which makes discipline, rather than computing power, the scarce resource.
How overfitting happens
Every strategy has settings: entry times, indicator periods, stop levels, targets, filters. Each setting you tune gives the strategy another way to fit the past. With enough settings, a strategy can be made to fit almost any history, including patterns that were pure chance.
Consider testing 100 versions of a strategy on the same data. Even if none of them has a real edge, some will look good by luck. If you choose the best one and trust its backtest, you have selected luck, not skill. The more versions you test, the better the best one will look, and the less its result means.
Researchers have shown how strong this effect is. Bailey and his co-authors worked out that, with five years of daily data, trying more than about 45 independent strategy variations makes it likely that at least one shows an attractive result in the backtest even if none has any real edge.4
Overfitting also creeps in without a formal search. A trader looks at a backtest, sees a bad month, adds a filter that would have avoided it, and runs the test again. The curve looks better. They repeat this five or six times. Each step felt like learning; together they have fitted the strategy to the specific bad months of that history. The next bad month will be a different one, and none of the filters will catch it.
Why fitting noise looks like skill
Market prices contain a small amount of signal and a large amount of noise. A rule that captures the signal will show a modest, uneven edge in a backtest. A rule that captures the noise of one particular history can show a large, smooth edge, because it is matching the exact ups and downs of that history. On new data the noise is different, so the fitted part of the rule stops working, and often the rule does worse than a simple version would have, because its extra conditions now make wrong decisions.
How many versions is too many?
A simple calculation shows how fast luck builds up. Suppose you test strategies that have no edge at all, each on five years of daily data, and measure each one's Sharpe ratio, the return per unit of risk. Because of noise, each strategy's measured Sharpe ratio will be scattered around zero. The best of many will look good purely by chance:
| Versions tried | Expected best Sharpe ratio, with no real edge |
|---|---|
| 10 | About 0.7 |
| 45 | About 1.0 |
| 100 | About 1.1 |
| 1,000 | About 1.5 |
A Sharpe ratio of 1 is often treated as the sign of a good strategy. After 45 tries on five years of data, you should expect to find one by chance alone. The figures are approximate, from a simulation of strategies with no edge and independent results, and real strategy versions are often related to each other, which lowers the effective number of tries. But the lesson holds: the best of many tries is inflated, and the inflation grows with every try.
Warning signs of an overfitted strategy
- Many conditions. Five filters and three exceptions, each added to make the equity curve smoother.
- Precise, unusual settings. An RSI of 13 and a stop of 37 points, when 14 and 35 fail.
- Neighbours that fail. Small changes to a setting turn a profit into a loss.
- Results concentrated in a few trades or in one period.
- A smooth equity curve with very few losing months, far better than similar strategies.
- No clear reason why the strategy should work.
- A large number of versions tried before this one was chosen.
- Rules that refer to specific dates or events, such as "skip the second week of March".
One warning sign is not proof. Several together are a strong reason to doubt the backtest.
Tests that catch overfitting
Out-of-sample testing
Split your history into two parts. Design and tune the strategy on the first, then test it once on the second, which it has never seen. If performance collapses, the strategy was fitted to the first part. The test only works if you do not go back and tune on the second part afterwards; once you have adjusted the rules after seeing the out-of-sample result, that data is no longer out of sample.
Walk-forward testing
Repeat the out-of-sample test over time. Design on one window, test on the next, then move forward and repeat. The combined test periods give a fairer estimate of live performance, and show whether the strategy's settings stay stable as time passes. See walk-forward testing.
Parameter stability
Run the strategy across a grid of nearby settings. A robust strategy shows a broad region of reasonable results. An overfitted one shows a single sharp peak.
| Stop loss | 20% | 25% | 30% | 35% |
|---|---|---|---|---|
| Robust strategy | Profit | Profit | Profit | Small profit |
| Overfitted strategy | Loss | Loss | Large profit | Loss |
This is an illustration. The robust strategy works across its neighbourhood; the overfitted one works only at one setting. If you must choose a setting, choose one in the middle of a broad profitable region, not the single best point.
Different markets and periods
A real edge should appear, at least partly, in related markets or other periods. A Nifty strategy that also works, perhaps more weakly, on Sensex or Bank Nifty, or in earlier years, is more likely to be robust. One that works only on one index in one year is suspect.
Monte Carlo simulation
Reshuffle or resample the strategy's trades thousands of times to see the range of drawdowns and results the same trades could have produced. A result that depends on the exact order of a few trades is fragile. See Monte Carlo simulation.
Adjust for the number of trials
Statistical methods such as White's Reality Check and the deflated Sharpe ratio adjust a backtest's performance for how many versions were tried. Even without the maths, keep a record of every version you test, including the ones you discarded, and be most sceptical of the best one. The number of versions you tried is part of the result.
Randomised data
Run the strategy on data with no pattern in it, such as prices with their daily changes shuffled. If the strategy still looks profitable on noise, the testing process itself, not the market, is producing the profit.
Overfitting's close relatives
Three other traps produce the same symptom, a backtest that is better than live trading:
| Trap | What happens | Example |
|---|---|---|
| Look-ahead bias | The test uses information that was not known at the time | Entering at the day's low, or using a candle's close before it has closed |
| Survivorship bias | The test uses only stocks that exist today | Leaving out stocks that were delisted, merged or dropped from an index |
| Ignoring costs and slippage | The test fills at ideal prices with no charges | A scalping strategy that earns ₹3 a trade before ₹8 of costs |
They often occur together with overfitting, and they are checked in the same way: by asking whether every piece of information the rule used was available, at that price, at that moment. See look ahead bias and our guide to backtesting mistakes.
Habits that keep strategies robust
- Start with a reason. Write down why the strategy should work before testing it: who is on the other side, and why they keep losing to this rule.
- Keep rules few. Each extra condition should earn its place across many periods, not just the one you are looking at.
- Use round, common settings unless there is a reason for something else. A 20-period average is less likely to be fitted than a 23-period one.
- Fix the test plan in advance: the data, the period, the measures and what counts as success.
- Hold back recent data for a final test, and use it once.
- Count trades. Results based on fewer than 100 trades are fragile.
- Keep a log of every version tested, with the result.
- Forward test before going live, and compare the forward test with the backtest for the same weeks.
- Prefer a strategy that is good across many settings to one that is excellent at one.
A worked example: two traders, one idea
This is a hypothetical example.
A trader tests a Nifty straddle with 6 entry times, 5 stop levels, 4 re-entry rules and 3 day filters: 360 combinations. The best shows a high return with a tiny drawdown. On the following year, which the trader did not use, it loses money.
A second trader tests only 3 entry times and 3 stop levels, chosen in advance, and finds that 8 of the 9 combinations are profitable, with the best and worst not far apart. On the following year, results are lower than the backtest but still positive. The second strategy is less impressive on paper and far more trustworthy.
The first trader also has a harder time deciding what went wrong. With 360 combinations, there is no way to tell which of the choices, the entry time, the stop, the re-entry rule or the day filter, was fitted to noise. The second trader's nine combinations can be read at a glance, and the next change can be tested one setting at a time.
The difference was not the idea: both traders sold the same straddle. It was the search. The first trader's best combination was the one that happened to fit the noise of the test years best. The second trader's small, fixed search left little room for noise to decide the result.
A worked example: how a filter fits a history
This is a hypothetical example.
A trader's breakout rule on Bank Nifty futures shows 180 trades over three years and a modest profit after costs. Looking at the equity curve, they see that most of the losses came in two months of choppy, sideways trading.
- First change: they add a filter that skips trades when the 14-period ADX is below 18. The two bad months disappear from the test, and the profit doubles.
- Second change: a few remaining losses came on Fridays, so they skip Fridays. The profit rises again.
- Third change: they move the stop from 0.6% to 0.55% because it removes three losing trades.
The final backtest is excellent. But each change was chosen after seeing which trades lost, and each removed specific losses from that specific history. An ADX threshold of 18 was picked because it happened to sit just above the ADX on those two months; any other choppy period might sit at 19 or 22. The honest test is to fix the three changes and run them on a fourth year, untouched. Very often the profit falls back to where it started, or below, because the new filters now skip some good trades and keep some bad ones.
A simple test plan you can follow
Write this down before you run the first backtest, and keep to it:
- The idea and its reason, in two sentences.
- The rules, with every setting you will allow to vary and the values you will try. Keep the total number of combinations small, ideally under 20.
- The data: the instrument, the candle size, and the dates. Split them into a design period and a held-back period, for example the first three years and the last one.
- The measures: net profit after costs, maximum drawdown, number of trades, worst month, and how many of the tried combinations were profitable.
- The pass mark, decided now: for example, at least 100 trades, most nearby settings profitable, and a held-back result at least half as good as the design result.
- The held-back test: run once, on the chosen setting, and record the result whatever it is.
- The forward test: at least four weeks on live prices with test money before any real order.
A plan like this turns testing from a search into an experiment. It does not guarantee a good strategy; it guarantees that a bad one is found out before it costs money.
What to do when a strategy fails out of sample
A failed held-back test is not a reason to adjust the rules and run it again on the same held-back data. That simply turns the held-back data into more design data. Instead:
- Accept the result and record it.
- Ask why. Did the market change between the two periods, for example a new expiry day or a rise in costs? Was the design result driven by a few trades? Was the idea weak to begin with?
- If the idea still seems sound, design a new version on the old design data, and wait for new data, such as the next three to six months, to test it. Use Forward test in the meantime.
- If not, drop it. Most ideas fail. Finding that out cheaply is the purpose of testing.
Overfitting in Indian option strategies
Option strategies in India have their own ways of fitting the past:
- Entry times to the minute. Testing every minute from 9:16 to 10:30 for a straddle entry will find a "best" minute, almost always by chance. Choose a few round times in advance.
- Expiry-day rules built on a changing calendar. NSE weekly expiries moved to Tuesday from 1 September 2025,7 and the number of weekly expiries was cut from 20 November 2024.8 A rule tuned on the old calendar may describe a market that no longer exists.
- Contract sizes and lot sizes have been revised, which changes the rupee results of every past trade and the effect of fixed rupee stops.
- Tax and charges. STT on futures and options rose from 1 April 2026.9 A strategy whose small edge was measured under the old rates may have none under the new ones.
- Volatility regimes. A strategy tested only through a calm period, when India VIX stayed low, has not met the conditions that cause the largest option-selling losses.
Overfitting and AI-written strategies
AI assistants can now turn a sentence into a strategy and a backtest in seconds. That is useful, and it changes the arithmetic of overfitting. Ask an assistant for twenty variations of a breakout rule, test them all, and keep the best, and you have run twenty trials whether or not it felt like a search. The same discipline applies:
- Count every version the assistant produces and you test, not only the one you keep.
- Ask for the reason behind a rule, not only the rule. A rule with no reason is a candidate for chance.
- Hold back data before you start asking for variations, and test the final choice on it once.
- Be wary of very specific suggestions, such as a 17-minute window or a 0.43% stop, that improve the backtest without a reason you can explain.
An assistant that is honest about being software, not an adviser, will help you build and test your own rules; it cannot tell you which rule will work, because nobody can.
Testing on QuantPower without fooling yourself
QuantPower's tools help with testing, but the discipline is yours:
- The unseen-days check. An option strategy backtest holds back the last 30% of the period as unseen days and compares them with the first 70%, the days the strategy was built on. It gives a plain verdict: held up, weaker, fell apart, lost money even in the period it was built on, or too few days to judge. A strategy that fails it is not worth a longer test.
- Every trade shown. Backtests list each trade after charges, so you can check that the rules did what you meant and see whether a few trades carry the result.
- Plain warnings. The report notes when the test period is short, when a few large losses outweigh many small wins, when the worst day took more than half the profit, and when charges take a large share of the gross.
- Forward test. The same algo runs on live prices with test money, so you can compare it with the backtest for the same weeks before risking money.
The first 10 backtests each week are free and later runs cost 2 tokens each. That is cheap enough to test properly and a gentle reminder that each run is a trial you should count.
Common mistakes
- Tuning on the out-of-sample data after seeing it, and still calling it out of sample.
- Trusting the best of many runs without asking how many runs there were.
- Adding a filter for every bad month.
- Treating a smooth equity curve as proof. Real strategies have rough patches.
- Testing on too short a history, or only one kind of market.
- Ignoring changes in the market's rules between the test period and today.
- Throwing away the record of failed versions, which hides how many tries the winner came from.
Where the fight against overfitting is going
Two forces pull in opposite directions. Testing keeps getting faster and cheaper, and AI tools can now write and test many strategies from a sentence, which multiplies the number of trials a trader runs without noticing. At the same time, methods that account for the number of trials, such as the deflated Sharpe ratio and combinatorial cross-validation, are moving from research papers into ordinary testing tools, and held-back checks like the unseen-days test are becoming standard parts of backtest reports. The traders who do well will be the ones who treat each test as a trial to be counted, not as a free look at the past.
Start with 500 free tokens
Connect your broker, build a strategy and test it on paper before a single live order goes out.
Frequently asked questions
What is overfitting in trading?
Tuning a strategy so closely to past data that it captures random noise. It performs well in the backtest and poorly on new data.
How do I know if my strategy is overfitted?
Check whether nearby settings also work, whether it holds up on data it was not designed on, whether its results depend on a few trades, and how many versions you tried before choosing it.
Is optimisation always overfitting?
No. Choosing reasonable settings is part of design. It becomes overfitting when you search many combinations and trust the best one without testing it on new data.
How much data do I need to avoid overfitting?
Enough for many trades across different market conditions, plus a separate period for out-of-sample testing. More rules and settings need more data, roughly in proportion to how many you add.
Can a strategy with only two settings be overfitted?
Yes, if you tried many values of those two settings and kept the best pair. The number of versions you tried matters as much as the number of settings.
Should I re-optimise my strategy every month?
Frequent re-tuning on the latest data is a slow form of overfitting: each month the settings chase the last month's noise. If you re-tune, do it on a fixed schedule with a fixed method, such as a walk-forward test, and judge the method by its results on data it had not seen.
Does forward testing remove the risk of overfitting?
It is the best check, because the data is genuinely new. But a short forward test can still be lucky; compare it with the backtest over several weeks and many trades before trusting it.
Related guides
- Backtesting: testing rules on past data with the unseen-days check
- Sharpe ratio calculator: measuring return against risk
- Backtest trading strategies: the method, step by step
References
- A. W. Lo and A. C. MacKinlay, "Data-Snooping Biases in Tests of Financial Asset Pricing Models", *Review of Financial Studies* 3(3), 431–467 (1990)
- R. Sullivan, A. Timmermann and H. White, "Data-Snooping, Technical Trading Rule Performance, and the Bootstrap", *Journal of Finance* 54(5), 1647–1691 (1999)
- H. White, "A Reality Check for Data Snooping", *Econometrica* 68(5), 1097–1126 (2000)
- D. H. Bailey, J. M. Borwein, M. López de Prado and Q. J. Zhu, "Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance", *Notices of the American Mathematical Society* 61(5), 458–471 (2014)
- D. H. Bailey and M. López de Prado, "The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting and Non-Normality", *Journal of Portfolio Management* 40(5), 94–107 (2014)
- C. R. Harvey, Y. Liu and H. Zhu, "… and the Cross-Section of Expected Returns", *Review of Financial Studies* 29(1), 5–68 (2016)
- NSE, circular on the revision of expiry days of derivatives contracts, effective 1 September 2025 (nseindia.com)
- SEBI, circular "Measures to strengthen Equity Index Derivatives framework for increased investor protection and market stability", 1 October 2024 (sebi.gov.in)
- Union Budget 2026-27: securities transaction tax on futures and options raised from 1 April 2026 (indiabudget.gov.in)