---
title: LiveSearch: Questions as Executable Verifier Code for a Live Benchmark of Professional-Source Financial Search Agents
url: https://www.ml-quant.com/papers/arxiv/2610.37166/
site: ML-Quant (https://www.ml-quant.com)
updated: 2026-10-30
license: Summaries CC BY 4.0; links go to the original sources
index: https://www.ml-quant.com/llms.txt
identifier: arXiv:2610.37166
source_url: https://arxiv.org/abs/2610.37166
featured: 2026-10-30
citations: unknown
topic: LLMs & Text
---


# LiveSearch: Questions as Executable Verifier Code for a Live Benchmark of Professional-Source Financial Search Agents

A live benchmark stores financial questions as executable code so answers update daily against real data, preventing memorization and revealing that 70% of daily queries change their correct answer.

- Source: https://arxiv.org/abs/2610.37166
- Identifier: arXiv:2610.37166
- Released: 2026-10-28
- First featured: Quant Letter No. 137 (2026-10-30): https://www.ml-quant.com/issues/2026-10-30/
- Citations (Semantic Scholar): not tracked
- Published in: not yet
- Topic: LLMs & Text
- Authors: Sinuo Wang, Wentao Zhao, Jiacheng Lu, Rui Sun, Beidi Luan, Qiming Shi, Jing Li, Daxin Jiang, Zuo Bai

## Abstract (arXiv, CC0)

In finance, the answer that matters is usually the current one: prices, rates, and disclosures hold only as of a given day, so retrieving up-to-date facts is a core capability of a financial agent. Yet most financial benchmarks overlook timeliness. They fix each answer at annotation time, so the gold goes stale as markets move and, once leaked into training data, rewards recall rather than search and analysis. We introduce LiveSearch, a live benchmark that stores each question as executable verifier code: a typed program of calls to a professional financial data service, followed by a declarative extractor. The question is frozen, while its gold answer is determined by executing the program against professional financial data at evaluation time, and a daily job re-executes it with no human or language model in the loop. A memorized answer is thus likely outdated, and a model must reach the current one through tools and analysis. Construction is equally automatic: a language model only verbalizes programs enumerated from real tool responses and never chooses an answer. Under general web search, six contemporary tool-calling models reach 70.9% to 76.7% answer accuracy. Over three consecutive days, 70% of served questions change their gold value, and even under tolerance-aware matching, a literally reused earlier snapshot fails on a majority of questions within a single day. Per-question tracking uncovers dynamic shifts in correctness masked by aggregate accuracy.

## Related

- [GPT-4o as the Gold Standard: A Scalable and General Purpose Approach to Filter Language Model Pretraining Data](https://www.ml-quant.com/papers/arxiv/2410.02755/): Data Filtering System with GPT-4o Accuracy: The article introduces SIEVE, a cost-effective method for filtering web-scale data that matches the accuracy of GPT-4o and is efficient in curating large datasets for language model training.
- [The Financial Uncertainty of Climate-Related Assets](https://www.ml-quant.com/papers/ssrn/4753434/): The paper identifies a factor linked to climate news indices and commodity markets that contributes to the excess volatility of climate-sensitive assets.
- [FinInteract: Benchmarking Clarification and Intent Integration in Ambiguous Financial Question Answering](https://www.ml-quant.com/papers/arxiv/2609.24002/): A benchmark reveals that language models answer financial questions above 90 percent with clarification but only 28.9 percent when they must elicit it themselves, exposing model ambiguity resolution.
- [Investor Sentiment in Malaysian Government Bonds](https://www.ml-quant.com/papers/ssrn/4955228/): The COVID-19 pandemic has changed investment strategies in bond markets, with Malaysian government bonds showing equity-like traits and deviating from their usual safe-haven role, with gold prices being the key factor affecting bond yields.
- [Media Hype and Fake News Impact on Commodity Prices](https://www.ml-quant.com/papers/repec/eee-finlet-v-59-y-2024-i-c-s1544612323010309/): The study shows that media hype and fake news greatly influence commodity prices, especially during COVID-19. It also found that bi-directional long-short-term memory is useful in predicting these impacts.
- [Large Datasets and Hybrid Models](https://www.ml-quant.com/papers/repec/gam-jjrfmx-v-16-y-2023-i-6-p-298-d-1167483/): Study explores using machine learning and sentiment analysis to forecast foreign exchange rates and commodity prices.