---
title: Reading or Remembering? Measuring Temporal Contamination in Language Models
url: https://www.ml-quant.com/papers/arxiv/2611.21554/
site: ML-Quant (https://www.ml-quant.com)
updated: 2026-11-20
license: Summaries CC BY 4.0; links go to the original sources
index: https://www.ml-quant.com/llms.txt
identifier: arXiv:2611.21554
source_url: https://arxiv.org/abs/2611.21554
featured: 2026-11-20
citations: unknown
topic: LLMs & Text
---


# Reading or Remembering? Measuring Temporal Contamination in Language Models

Compares modern language models to chronologically consistent baselines on FOMC statements, finding lookahead premiums in real-time GDP nowcasts are undetectable at macroeconomic sample sizes.

- Source: https://arxiv.org/abs/2611.21554
- Identifier: arXiv:2611.21554
- Released: 2026-11-17
- First featured: Quant Letter No. 140 (2026-11-20): https://www.ml-quant.com/issues/2026-11-20/
- Citations (Semantic Scholar): not tracked
- Published in: not yet
- Topic: LLMs & Text
- Authors: Arunkumar V, Gangadharan G. R., S. Senthilkumar

## Abstract (arXiv, CC0)

Language models are now used to read central-bank text for economic forecasting, but most were trained on text written after the documents they read. A model reading a 2021 FOMC statement may already know that inflation rose in 2022, so part of its apparent skill can be memory of the outcome, a temporal form of test-set contamination. We measure this memory on FOMC statements and minutes from 2000 to 2026 by comparing modern open models with chronologically consistent models, which are trained only on text from before a fixed date. Asked about past GDP outcomes given only a date, models that cannot know the answer still reach an AUC of up to 0.64 from priors on dates and eras, so chance is the wrong baseline. Against these memory-free models on the same questions, most modern models recall which years had strong growth. Membership-inference scores sort documents by era and document type rather than by whether a model saw them, as exact counts in an open pretraining corpus show. In real-time nowcasts, every lookahead premium we estimate on US GDP is smaller than the forecast comparison can detect at macroeconomic sample sizes, and none on GDP or inflation survives correction for multiple testing. Lookahead audits should therefore measure memory directly against memory-free models on the same items.

## Related

- [From Tone to Trajectory: Continuous Sentiment and the Shape of Monetary Policy Communication](https://www.ml-quant.com/papers/arxiv/2609.25034/): The study shows that how monetary policy sentiment unfolds across a press conference, not just its average tone, predicts rate changes and shapes forecaster expectations at the ECB and Fed.
- [Interpreting the Interpreter: Can We Model post-ECB Conferences Volatility with LLM Agents?](https://www.ml-quant.com/papers/arxiv/2508.13635/): A new method using a Large Language Model can predict financial market responses to European Central Bank press conferences, aiding in maintaining financial stability.
- [Identifying economic narratives in large text corpora - An integrated approach using Large Language Models](https://www.ml-quant.com/papers/arxiv/2506.15041/): A study finds that Large Language Models like GPT-4o can interpret economic narratives from texts, but their performance is not yet on par with experts, based on analysis of articles about inflation.
- [Uncertainty and financial market resilience: evidence from China](https://www.ml-quant.com/papers/arxiv/2409.18422/): The article evaluates the impact of external shocks, including climate policy uncertainty, on China's financial markets, revealing that such uncertainty affects investor sentiment, commercial banks' non-performing loan ratio, and the capital and financial account balance.
- [Breaking the Bank with ChatGPT: Few-Shot Text Classification for Finance](https://www.ml-quant.com/papers/arxiv/2308.14634/): Conversational GPT models are suggested for efficient text classification in finance, providing a practical solution for tasks with limited labels and achieving top-tier results.
- [ECB perception of non-banks: a machine learning-based approach](https://www.ml-quant.com/papers/repec/ecb-ecbwps-20263304/): A fine-tuned language model reveals the ECB holds predominantly negative views of non-banks since the financial crisis, driven by stability and regulatory concerns.
