---
title: Where Backtests and Live Trading Part Ways: Six Defects from One Intraday System
url: https://www.ml-quant.com/papers/arxiv/2610.21846/
site: ML-Quant (https://www.ml-quant.com)
updated: 2026-10-16
license: Summaries CC BY 4.0; links go to the original sources
index: https://www.ml-quant.com/llms.txt
identifier: arXiv:2610.21846
source_url: https://arxiv.org/abs/2610.21846
featured: 2026-10-16
citations: unknown
topic: Trading, Microstructure & Execution
---


# Where Backtests and Live Trading Part Ways: Six Defects from One Intraday System

Six defects from data, code, and configuration caused live intraday trading to diverge from backtest replay, none visible in summary statistics, each requiring specific controls.

- Source: https://arxiv.org/abs/2610.21846
- Identifier: arXiv:2610.21846
- Released: 2026-10-16
- First featured: Quant Letter No. 135 (2026-10-16): https://www.ml-quant.com/issues/2026-10-16/
- Citations (Semantic Scholar): not tracked
- Published in: not yet
- Topic: Trading, Microstructure & Execution
- Authors: Alex Vidovich, Zivana Zerjal

## Abstract (arXiv, CC0)

Backtests of intraday strategies can drift away from what the live system actually does; in our case, the drift never showed in summary statistics. We report six defects found while running one intraday US equity system live and comparing its live days with their backtest replay. Three came from data: a default feed that carried a small fraction of real opening volume, historical prices adjusted for dividends that live orders never see, and data the live system downloaded but the backtest never did. Two came from code: a sizing step that exceeded a limit the live account enforces, and UTC timestamps read as New York time. One came from configuration: settings that were switched on but never reached the code that places orders. For each we give the symptom, the size of the error, the cause and the test or control that now guards against it. All six sat where two paths that should agree had drifted apart. We close with a ten-point checklist. We make no claim about whether the system is profitable.

## Related

- [The bias of IID resampled backtests for rolling window mean-variance portfolios](https://www.ml-quant.com/papers/arxiv/2505.06383/): A study finds that resampling techniques in backtests can cause bias in Sharpe Ratio estimates, suggesting a need for structure-preserving resampling methods.
- [Optimizing Portfolio Performance through Clustering and Sharpe Ratio-Based Optimization: A Comparative Backtesting Approach](https://www.ml-quant.com/papers/arxiv/2501.12074/): The article presents a new method for improving portfolio performance using clustering-based segmentation and Sharpe ratio-based optimization, tested with historical data from various asset classes.
- [Forecasting and backtesting gradient allocations of expected shortfall](https://www.ml-quant.com/papers/arxiv/2401.11701/): The study introduces a new semiparametric model for predicting dynamic Expected Shortfall contributions, which has shown excellent results in analyzing stock returns.
- [Backtest overfitting in the machine learning era: A comparison of out-of-sample testing methods in a synthetic controlled environment](https://www.ml-quant.com/papers/ssrn/4686376/): The Combinatorial Purged Cross-Validation (CPCV) method is superior in financial analytics for reducing overfitting risks, outperforming traditional methods like K-Fold and Walk-Forward.
- [LIGHT Benchmark - Comprehensive Backtesting Framework for Market Risk Models Comparison](https://www.ml-quant.com/papers/ssrn/4586897/): Market Risk Backtesting: The article presents LIGHT Benchmark, a tool for comparing market risk models, including a scoring system for evaluating Value at Risk and Expected Shortfall models.
- [E-backtesting](https://www.ml-quant.com/papers/arxiv/2209.00991/): A new backtesting procedure for Expected Shortfall forecasts is proposed using e-values and e-processes.