Can We Stop the Backtest Yet? Anytime-Valid Promotion Tests for Adaptive Financial Research Agents

Financial research agents can generate and monitor trading rules faster than a human can document the resulting selection process. Candidates appear at data-dependent times, weak rules are abandoned, and promising ones are watched until they cross a reporting threshold. We introduce PROMOTE, a protocol that assigns every candidate predictable testing wealth at birth and accumulates a cost-aware e-process only from subsequent returns. If lifetime allocations sum to at most one, a weighted Ville argument bounds the probability of ever promoting a rule whose declared bounded net score has always had nonpositive conditional mean. The result permits dependent candidates, adaptive births, and indefinite monitoring. We audit the implementation in two settings. A retrospective pseudo-out-of-sample replay contains 24 panel--rule hypotheses, 3,000 shared-sign null paths, injected alternatives, and public ETF and industry data. Repeated unadjusted tests produce a family crossing in 94.5% of null paths, versus 0.0% for PROMOTE; at a 15-bp aligned ETF-component perturbation, PROMOTE has 70.1% target power and median crossing session 2,327. A separate exact mean-zero experiment lets an agent select candidate identities and birth times from past scores; the funded gate has 1.77% false-family crossings across 3,000 programs. The market replay produces zero promotions after costs. The certificate controls historical false promotion under the declared null; it is not a certificate that an earlier edge remains profitable at deployment. AI-use disclosure. Generative AI tools, including OpenAI Codex, assisted manuscript preparation. For this preprint version, Codex assisted with LaTeX editing, author and venue metadata, compilation, and layout checks.

1. 14 May 2025

   The bias of IID resampled backtests for rolling window mean-variance portfolios

   A study finds that resampling techniques in backtests can cause bias in Sharpe Ratio estimates, suggesting a need for structure-preserving resampling methods.

   arXivIn Investment Analysts Journal

   1cites
2. 23 Jan 2025

   Optimizing Portfolio Performance through Clustering and Sharpe Ratio-Based Optimization: A Comparative Backtesting Approach

   The article presents a new method for improving portfolio performance using clustering-based segmentation and Sharpe ratio-based optimization, tested with historical data from various asset classes.

   arXiv

   1cites
3. 23 Jan 2024

   Forecasting and backtesting gradient allocations of expected shortfall

   The study introduces a new semiparametric model for predicting dynamic Expected Shortfall contributions, which has shown excellent results in analyzing stock returns.

   arXivIn Insurance: Mathematics and Economics

   1cites
4. 9 Jan 2024

   Backtest overfitting in the machine learning era: A comparison of out-of-sample testing methods in a synthetic controlled environment

   The Combinatorial Purged Cross-Validation (CPCV) method is superior in financial analytics for reducing overfitting risks, outperforming traditional methods like K-Fold and Walk-Forward.

   SSRNIn Knowl. Based Syst.

   21cites
5. 4 Oct 2023

   LIGHT Benchmark - Comprehensive Backtesting Framework for Market Risk Models Comparison

   Market Risk Backtesting: The article presents LIGHT Benchmark, a tool for comparing market risk models, including a scoring system for evaluating Value at Risk and Expected Shortfall models.

   SSRN

   1cites
6. 24 May 2023

   E-backtesting

   A new backtesting procedure for Expected Shortfall forecasts is proposed using e-values and e-processes.

   arXiv

   28cites