The first reaction to finding a high-Sharpe strategy in simulation should be to assume there’s a mistake. Markets are adversarial, data is messy, and small modeling errors compound into misleading results. Common mistakes in training systematic trading models and some solutions:
Omitting fees and slippage: Nearly all equities, options, futures, and digital asset exchanges charge higher fee amounts for removing liquidity than for adding liquidity. HFT strategies that hold positions for only seconds or minutes frequently see their expected edge disappear once realistic fees are applied. Another common mistake is assuming a liquidity-taking order is fully filled at the best bid/ask instead of walking the book, consuming successive price levels up to available size. This both systematically underestimates cost and overestimates the capacity of the strategy. Accurately modeling fees and slippage requires building automated reconciliation between predicted costs and exchange-reported fees and fill prices.
In-sample contamination: Statistical arbitrage strategies that fit model parameters to historical data need to avoid evaluating performance on the period used for training. The naive method of splitting data is to hold back the most recent days from the training set, which may result in training the strategy on a different market volatility and momentum regime than the recent past. Simple K-fold cross validation, a common fix from machine learning, is inappropriate for time series as this method leaks future information into the past. The most successful data splitting techniques involve purged K-fold, walk-forward optimization, or combinatorial purged cross-validation.
Incompatible clocks: Backtesters that reference timestamps based on the market data capture machine’s clock overestimate achievable fill ratios. A partial solution is to refer to exchange timestamps for order book events to determine whether liquidity still exists when orders are sent. However that solution is also unreliable as exchanges have wide distributions of latencies between matching engine actions and book update dissemination. A robust heuristic is to re-run the backtest under a range of artificial order-submission delays and examine the sensitivity of both fill ratio and P&L.
Survivorship bias: The choice of instrument universe itself introduces survivorship bias. For example, choosing the constituents from today’s S&P 500 index systematically selects historical winners only, as those names that underperformed have already been excluded from the basket. One solution is to record point-in-time universes to use for future historical studies, and another is to fix today’s universe and implement a walk-forward simulation until enough data has been collected.