Hard to Fool: A Bounded LLM Research Loop for Quantitative Finance, and What Two Dozen Campaigns Found

Agentic research loops that let a language model propose a change, run a fixed experiment and keep whatever improves the metric work well in machine learning. In trading research the same loop is a machine for producing false discoveries: a backtest that "improves the metric" is, by default, overfit, look-ahead, a multiple-comparisons artifact, cost-free, or market drift wearing a strategy. We describe Autonomous Quant Researcher, an open-source loop built on the opposite premise: the model proposes, and only trusted code decides. Proposals are strict declarative JSON over a fixed vocabulary, so there is no model-written code to sandbox. Evaluation runs on immutable chronological stores split into discovery, validation and a final holdout that can be opened exactly once, after a durable receipt is written. Acceptance is a conjunction of absolute gates (positive edge and net P&L after costs, a positive weekly-cluster bootstrap lower bound, survival at elevated costs, per-year stability and sample floors), never a single score. Duplicates are refused before they spend a trial, and every formulation, refusal, crash and result is kept in a hash-chained, append-only ledger. We report every campaign run to date, across 0DTE options priced from traded option bars, overnight SPX verticals, daily ETF rules, technical-analysis premises, Kalshi crypto binaries, earnings options, off-exchange volume and a pre-registered alternative-data campaign. Most hypotheses fail, and we report them. Two results survive their pre-registered bars. The clearest example of the loop working is a failure: a language-model survivor that passed discovery and validation with Sharpe ratios near 3 and fell to 0.26 on its single holdout use. The value of such a loop lies less in what it finds than in how hard it is to make it report something that is not there.

1. 9 Oct 2026

   How Execution Assumptions Change Short-Horizon Sharpe Rankings: Evidence from a Synthetic Trading Benchmark

   Varying execution realism from ideal fills to latency and impact reshuffles rankings of LLM and classical trading policies, showing how backtest conventions affect headline results.

   arXiv

   0cites
2. 16 Oct 2026

   SwingGuard: Risk-Conditioned Multi-Agent Swing Trading with Dual-Timescale Governance

   A dual-timescale multi-agent framework generates trading strategies conditioned on current risk and maintains ongoing position governance between strategy revisions on cryptocurrency, preserving returns after transaction costs.

   arXiv

   1cites
3. 7 Aug 2025

   ContestTrade: A Multi-Agent Trading System Based on Internal Contest Mechanism

   The article introduces a new multi-agent system for financial trading that uses competition to analyze market data and make decisions, proving more effective than traditional methods in tests.

   arXiv

   10cites
4. 30 May 2025

   Classifying and clustering trading agents

   The research suggests using an agent-based model to create synthetic data for categorizing financial investors by their behavior, emphasizing the difficulties in validating and interpreting machine learning methods.

   arXivIn Quantitative Finance

   2cites
5. 14 May 2025

   The bias of IID resampled backtests for rolling window mean-variance portfolios

   A study finds that resampling techniques in backtests can cause bias in Sharpe Ratio estimates, suggesting a need for structure-preserving resampling methods.

   arXivIn Investment Analysts Journal

   1cites
6. 23 Jan 2025

   Optimizing Portfolio Performance through Clustering and Sharpe Ratio-Based Optimization: A Comparative Backtesting Approach

   The article presents a new method for improving portfolio performance using clustering-based segmentation and Sharpe ratio-based optimization, tested with historical data from various asset classes.

   arXiv

   1cites