What is survivorship bias in a backtest?

Testing rules on today's stock list silently excludes every company that was delisted, merged or collapsed along the way, so results are computed only on the winners. An honest backtest includes dead companies and admits the bias can never be fully scrubbed out.

What is a look-ahead leak?

Any simulated decision that uses information which did not yet exist at that moment — a moving average computed with a close the rule then 'buys' before, or ranks built on later-revised data. Even small leaks compound into fantasy returns.

Can a backtest predict future returns?

No. At best it shows how a fixed set of rules behaved in past conditions, before brokerage, taxes and slippage. KALMAT publishes its results as hypothetical simulations — evidence about the method's character, not a forecast.

Guide · 2026-08-05

Backtesting honestly: what a simulated record can and cannot say

A backtest replays a set of rules against historical data to see how they would have behaved. Done honestly, it is the best available antidote to trading folklore. Done carelessly, it is a machine for manufacturing confidence in things that never worked. This guide covers the four classic ways backtests lie — and how to read one that is trying to tell the truth.

Lie #1: survivorship bias

Test a strategy on "all NSE stocks" using today's list and you have silently excluded every company that was delisted, merged, or collapsed along the way — precisely the stocks your strategy might have bought before they vanished. The survivors did well by definition; that is why they survived. Results computed only on survivors flatter every strategy, sometimes wildly. An honest backtest includes dead companies, and an honest report admits the bias can never be fully scrubbed out — which is why KALMAT's results carry a standing survivorship-bias caveat rather than a pretence of purity.

Lie #2: look-ahead leaks

A backtest must only use information that existed at the moment of each simulated decision. The leaks are sneakier than they sound: computing a moving average with the full day's close and then "buying" earlier that same day; ranking relative strength with data revised weeks later; defining a base using bars that hadn't printed yet. Even small leaks compound into fantasy returns, because the simulation keeps peeking one card ahead. KALMAT's engine evaluates each historical day using only that day's completed data — the same discipline the live scan runs under every evening.

Lie #3: ignoring costs

Every real trade pays brokerage, securities transaction tax, exchange charges, and — the big one — slippage, the gap between the printed price and your actual fill. A strategy that trades often can lose its entire paper edge to these frictions. KALMAT's simulated results are stated plainly as before costs and taxes, so you can apply your own assumptions instead of trusting invisible ones. When comparing any two backtests, always ask what the trade count is: a 20%-a-year system making 400 trades is a very different animal from one making 40.

Lie #4: curve-fitting

Add enough parameters and any dataset will confess to anything. Tune a stop to 7.83%, a base length to 23 days and an RS floor of 76 because those exact numbers maximised the historical result, and you have not discovered an edge — you have memorised the past. The tells of an honest system are the opposite: few rules, round numbers, and stability. If the results are good at an 8% stop, they should be decent at 7% and 9% too. A strategy that only works at one magic setting works at none. This is why the backtest lab exposes a handful of coarse dials rather than fifty fine ones — and why trying to break the rules yourself is the entire point of the tool.

How to read a backtest report

CAGR and max drawdown together, never apart. A 30% CAGR with a 55% drawdown means the account was more than halved on the way — would you truly have kept following the rules at the bottom? The drawdown is the price of the return; decide if you can pay it.
Win rate with payoff. A 35% win rate is excellent when winners average three times losers, and ruinous when they don't. Neither number means anything alone.
Trade count. Twenty trades prove nothing; a few hundred begin to. Small samples are where luck hides.
Year-by-year spread. One monster year propping up a flat decade is a warning, not a track record. Look for the edge showing up repeatedly across different market regimes.

What a good backtest is actually for

Not prediction. Markets change, and the disclaimer is not decoration: hypothetical results do not guarantee anything about the future. What an honest backtest genuinely gives you is calibration — realistic expectations for losing streaks, drawdown depth and the base rate of failure — so that when the live rules hit a rough patch, you recognise it as normal instead of abandoning the method at the worst moment. It also builds the conviction that rules deserve: KALMAT publishes its failed setups next to the winners because a record with the losses removed is not a record at all.

Test the rules, doubt the results, respect the caveats — and then hold the method to the same standard live that you demanded from it on paper.

Educational content only. KALMAT Screener is not SEBI-registered and nothing here is a recommendation to buy or sell any security. All backtested results are hypothetical, before costs and taxes, and subject to survivorship bias.