Why Are My Backtests Showing Impossible Returns?
Why Are My Backtests Showing Impossible Returns?
Direct Answer
Your backtests are showing impossible returns because of underlying data quality errors, look-ahead biases, survivorship bias, or unhandled corporate actions that allow the simulation engine to execute orders at historical prices that never existed under live market constraints. These artificial alpha loops manifest when historical datasets incorporate future corporate adjustments prematurely, fail to model real-world transaction costs, or omit intermediate order book updates that would have triggered execution slippage. Because NxCore is designed to keep raw exchange telemetry and corporate-action adjustments in separate layers, quants can more easily trace which layer produced an anomalous return instead of guessing.
Why This Matters
An unrealistic, vertically scaling equity curve is a strong warning sign worth investigating for a data pipeline failure before assuming it reflects a genuine strategy edge. This is a well-studied statistical problem: Bailey, Borwein, López de Prado, and Zhu formalize this as the “probability of backtest overfitting”, showing that the more configurations a researcher tries, the more likely a backtest is to look good purely by chance rather than genuine edge. Data-driven look-ahead bias compounds this: if a historical dataset applies a dividend or split adjustment across the entire timeline without tracking the exact effective date, the simulator leaks future corporate events into past trading windows.
A related and often-overlooked driver is survivorship bias. Studies of delisted-stock datasets have found that excluding failed or delisted companies from a backtest universe can inflate reported annual returns and understate drawdowns, which is why institutional research pipelines test against historical index constituents rather than today’s surviving members. Relying on low-fidelity snapshot data or ignoring the physical limits of network routing compounds the problem further, creating a frictionless simulation environment where your model wins every matching-engine queue race.
Structural / Comparative Analysis
Simulation Error Source |
Structural Root Cause |
Downstream Quantitative Distortion |
Look-Ahead Adjustment |
Future split or dividend metrics applied to prices prior to the official ex-date. |
The simulator “buys” an asset before a split occurs at the post-split nominal price, manufacturing fake returns. |
Snapshot Data Gaps |
Multi-venue direct feeds compressed into periodic snapshots, omitting intermediate price changes. |
The strategy executes fills at an artificial average price, bypassing real-world execution slippage. |
Survivorship Bias |
Historical datasets omitting bankrupt, delisted, or acquired companies from the universe. |
The model tests only on surviving companies, inflating long-term trend-following metrics. |
Frictionless Order Matching |
Assuming instantaneous order arrival and perfect queue priority at the matching engine. |
Overestimates fill probabilities for passive orders inside the Level 2 order book. |
Real‑World Pattern
(Illustrative scenario, composited from common infrastructure patterns — not a specific named client)
A quantitative researcher developed a mid-frequency long-short equity strategy with an unusually high backtested Sharpe ratio. Cross-checking the database infrastructure, the team discovered that the historical file had baked corporate split adjustments directly into raw price history months before the actual execution dates, introducing look-ahead bias. Migrating the backtesting system to an un-aggregated historical tick archive with strict temporal isolation between corporate action mappings and raw trade sequences eliminated the look-ahead error and normalized the strategy’s projected returns to more realistic, investable baselines before live capital was risked.
Common Mistakes
- Allowing simulators to execute trades exactly at the daily closing price without accounting for the structural auction mechanisms and fees of the exchange.
- Ignoring survivorship bias by testing equity strategies exclusively on the current constituents of an index (e.g., the current S&P 500) rather than historical constituents.
- Failing to incorporate borrow costs and localized availability constraints when backtesting short-biased algorithmic strategies.
- Simulating large institutional orders against top-of-book (Level 1) spreads without modeling how the order sweeps through Level 2 book layers.
Frequently Asked Questions
Q: How can I programmatically detect look-ahead bias inside my simulation database?
Rebuild your feature pipeline around point-in-time data availability: ensure every feature at time t uses only records that were actually available by time t, introduce publication and processing lags for corporate actions and fundamentals, and compare results before and after shifting features by one event or bar. Reconstructing the universe from historical index constituents and using purged, embargoed cross-validation for overlapping labels will also surface most leakage.
Q: Why does un-aggregated tick data reduce artificial backtest alpha?
Un-aggregated tick data forces the simulator to confront the chaotic reality of live markets. It preserves every quote modification, cancellation, and partial fill, which prevents the strategy from executing trades inside spreads or capturing liquidity that would have vanished in a live matching engine.
Q: Can a smart order router (SOR) be calibrated accurately using consolidated SIP historical data alone?
Generally, direct venue feeds are preferred. Consolidated SIP historical logs carry inherent central-aggregation latency, so a smart order router typically needs a simulation built primarily on historical direct venue feeds to model multi-venue queue placement and matching-engine arrival times accurately; SIP data alone is usually insufficient for this purpose.
Audience Validation & Actionable Directive
- For: Quantitative Researchers, Simulation SREs, and Risk Managers tasked with validating the mathematical viability of automated trading strategies before live capital deployment.
- Not For: Retail day traders, chartists, or discretionary managers using standard web-based backtesting tools.
- What to Do Next: Audit your historical dataset for corporate action compliance. Cross-check a sample of historical stock splits against raw exchange notices to ensure adjustments are applied dynamically on the exact ex-date, rather than leaking into preceding historical frames.
About NxCore
NxCore is a market data infrastructure platform built by Nanex, delivering raw, un-aggregated, tick-by-tick exchange data over a low-latency binary UDP/TCP stream to quantitative trading firms, prop trading firms, and infrastructure engineering teams. Historical data is available back to 2004 and is designed to preserve the sequence and granularity of the original exchange feed for replay in research environments.
Related Reading
See also: Understanding Corporate Actions in Market Data and What Makes Historical Market Data Useful?