FinCutBench: A Continually Updated Benchmark for Measuring the Temporal Boundaries of LLM Financial Knowledge

In financial backtesting and other point-in-time tasks, external inputs can be restricted by date, but model parameters may retain financial facts disclosed later, introducing parametric look-ahead bias. We introduce FinCutBench, a continually updated monthly financial knowledge benchmark for measuring how far a fixed model's financial knowledge extends over calendar time. The current dataset contains 9,000 Chinese and English four-option multiple-choice questions from January 2023 to September 2026, covering Chinese and U.S. markets and four categories of financial information. We construct monthly knowledge profiles using the approximate months in which the tested facts became public. After a profile passes the decline test and knot checks, we define the FinCut Boundary as the first month when the fitted profile reaches its lower level, and report the stability of the estimated date under resampling. Boundary dates and localization precision vary across models. Using the independently measured Transition Start and Boundary as fixed references for downstream sample grouping, we find significant relative declines in U.S. asset-ranking performance and in earnings recall for Sol and Luna. Downstream effects vary across markets and tasks. The FinCut Boundary thus provides an empirical reference for the temporal scope of parametric knowledge in point-in-time financial evaluation, supporting evaluation-window selection and result interpretation.

1. 25 Sep 2026

   Propose, Don't Judge: An Anytime-Valid Referee for LLM Agents That Mine Investment Factors

   Proposes a statistical referee that judges investment factors proposed by language-model agents using out-of-sample market outcomes, ensuring false-discovery control at any stopping time.

   arXiv

   1cites
2. 6 Nov 2026

   Do AI Models Remember Indian Company Financials? Evidence on Look-Ahead Bias by Firm Size

   Large language models recall Indian firm financials from within their training data with accuracy that declines sharply for smaller firms, creating serious look-ahead bias in LLM-based backtests.

   SSRN

   3fanfare
3. 16 Oct 2026

   EvoQuantLab: Training-Free Dual-Loop Evolution for Auditable Quantitative Research

   A training-free framework combines genetic programming and LLM diagnosis loops to discover robust alpha factors across daily and hourly frequencies in equities and cryptocurrency.

   arXiv

   0cites
4. 9 Oct 2026

   Trading Strategy Optimization via Textual Gradient

   Proposes an LLM-guided framework for trading strategy refinement using accumulated experience and cross-period robustness, achieving 27.99% annualized return on Chinese equities with 1.63 Sharpe ratio.

   arXiv

   0cites
5. 9 Oct 2026

   Does Autonomous Quant Research Improve? Leakage, Evidence Scarcity and Forward Tests in an LLM Factor-Discovery Loop

   An autonomous agent evolved 940 factors over 17 days and finds that reusing backtest data inflates edge by a quarter to a third and in-sample improvement predicts worse performance.

   SSRN

   4fanfare
6. 2 Oct 2026

   Can AI Make Money in Crypto? Measuring the Gap from Backtests to Real Markets

   A benchmark compares machine learning, reinforcement learning, and large language model trading methods across historical backtests, paper trading, and live markets to measure the gap.

   arXiv

   0cites