---
title: BuySideFlow: A Point-in-Time Benchmark for LLM Agents in Investment Research
url: https://www.ml-quant.com/papers/arxiv/2611.12966/
site: ML-Quant (https://www.ml-quant.com)
updated: 2026-11-14
license: Summaries CC BY 4.0; links go to the original sources
index: https://www.ml-quant.com/llms.txt
identifier: arXiv:2611.12966
source_url: https://arxiv.org/abs/2611.12966
featured: 2026-11-13
citations: unknown
topic: ML & AI Methods
---


# BuySideFlow: A Point-in-Time Benchmark for LLM Agents in Investment Research

Evaluates LLM agents on 408 investment research tasks, finding that half of workflows passing result checks contain temporal violations, revealing a 50.8% gap between output agreement and temporal validity.

- Source: https://arxiv.org/abs/2611.12966
- Identifier: arXiv:2611.12966
- Released: 2026-11-11
- First featured: Quant Letter No. 139 (2026-11-13): https://www.ml-quant.com/issues/2026-11-13/
- Citations (Semantic Scholar): not tracked
- Published in: not yet
- Topic: ML & AI Methods
- Authors: Dong Bo, Fanjin Zhang, Ke Cheng, Xinxin Zhao, Xiaowei Su, Yang Wang, Jinghang Xu, Cuiping Li, Hong Chen, Jing Zhang

## Abstract (arXiv, CC0)

Large language models (LLMs) can generate SQL and SQL--Python workflows for investment research, but ensuring point-in-time (PIT) validity remains challenging. Agents must enforce disclosure cutoffs and select the appropriate historical states and record versions throughout data retrieval and financial analysis. We introduce \textsc{BuySideFlow}, a benchmark of 408 practitioner-derived Chinese-language tasks spanning equities, funds, fixed income, and macroeconomics over a fixed snapshot from a licensed production financial database. We provide executable references and a taxonomy of temporal errors, pairing result checks with expert audits of executed workflows. We evaluate nine agent configurations on all tasks, with result accuracy ranging from 16.2\% to 36.8\%. An audit of 900 runs on 100 shared tasks confirms PIT violations in 193 of the 380 runs that pass result checks (50.8\%), showing that output agreement alone does not establish temporal validity.

## Related

- [Propose, Don't Judge: An Anytime-Valid Referee for LLM Agents That Mine Investment Factors](https://www.ml-quant.com/papers/arxiv/2609.27051/): Proposes a statistical referee that judges investment factors proposed by language-model agents using out-of-sample market outcomes, ensuring false-discovery control at any stopping time.
- [AlphaDiverse: Post-Training Local Quantitative Research Agents for Diverse Exploration in Alpha Factor Mining](https://www.ml-quant.com/papers/arxiv/2609.29014/): Proposes a multi-agent system with post-training that automates alpha factor mining locally, using diverse research paths and joint optimization to broaden exploration while maintaining prediction quality.
- [Can Generative AI agents behave like humans? Evidence from laboratory market experiments](https://www.ml-quant.com/papers/arxiv/2505.07457/): Large Language Models (LLMs) have potential in mimicking human behavior in economic markets, but need more research for improved diversity and accuracy.
- [Fake News, Real Crashes: Information Injection Attacks on LLM Financial Trading Agents](https://www.ml-quant.com/papers/arxiv/2610.26438/): Fabricated news reaches 62% attack success against LLM trading agents, far exceeding prompt injection, with surface-form credibility cues mattering more than content accuracy.
- [Quant-Arena: Benchmarking LLM Agents for Quantitative Trading Research](https://www.ml-quant.com/papers/arxiv/2610.30113/): A public benchmark tests whether large language models can complete end-to-end quantitative research with real trading constraints; no frontier model beats the equal-weight market in 2024.
- [VQuant: An Agentic Large Language Model Architecture for Automated Quantitative Financial Analysis and Document Generation: Reasoning-Augmented Tool Orchestration, In-Loop Numerical Computation, and Multi-Format Reporting](https://www.ml-quant.com/papers/ssrn/7650999/): A platform couples a language model with deterministic execution to turn natural-language requests into auditable reports where every figure is from live data or reproducible code, not model hallucination.