LiveOption: Evaluating LLM Agents in Structured Option Trading with Nonlinear Payoffs

Large language models (LLMs) and multi-agent systems (MAS) have shown promise in financial decision-making, yet existing evaluations focus on equity trading and primarily assess directional prediction, overlooking the structural complexity of derivative markets. Option trading introduces fundamentally different challenges, including nonlinear payoffs and multi-leg strategy construction, requiring structured decisions rather than simple directional bets. We introduce LiveOption, an evaluation framework for LLM-based agents in option trading. LiveOption formulates the problem as structured sequential decision-making under realistic execution and capital constraints, and provides a reproducible environment with standardized interaction protocols. The framework includes three task suites covering portfolio overlays, event-driven earnings trading, and 0DTE intraday trading. We further propose a hierarchical metric suite that evaluates action validity, decision quality, risk characteristics, and outcome-level performance. Experiments show that current agents often fail to achieve competitive returns in most scenarios. LiveOption offers a principled testbed for evaluating structured decision-making beyond outcome-based metrics.

1. 14 May 2025

   Can Generative AI agents behave like humans? Evidence from laboratory market experiments

   Large Language Models (LLMs) have potential in mimicking human behavior in economic markets, but need more research for improved diversity and accuracy.

   arXiv

   19cites
2. 2 Oct 2026

   Can AI Make Money in Crypto? Measuring the Gap from Backtests to Real Markets

   A benchmark compares machine learning, reinforcement learning, and large language model trading methods across historical backtests, paper trading, and live markets to measure the gap.

   arXiv

   4fanfare
3. 25 Sep 2026

   Propose, Don't Judge: An Anytime-Valid Referee for LLM Agents That Mine Investment Factors

   Proposes a statistical referee that judges investment factors proposed by language-model agents using out-of-sample market outcomes, ensuring false-discovery control at any stopping time.

   arXiv

   0cites
4. 25 Sep 2026

   AlphaDiverse: Post-Training Local Quantitative Research Agents for Diverse Exploration in Alpha Factor Mining

   Proposes a multi-agent system with post-training that automates alpha factor mining locally, using diverse research paths and joint optimization to broaden exploration while maintaining prediction quality.

   arXiv

   0cites
5. 15 May 2024

   AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments

   AI Evaluation in Clinical Environments: The paper introduces AgentClinic, a benchmark for assessing large language models in simulated clinical environments, highlighting the significant impact of biases on diagnostic accuracy and patient interactions.

   Machine learning

   263cites
6. 1 May 2024

   Learning Performance-Improving Code Edits

   The research presents a framework for optimizing programs using large language models, achieving a mean speedup of 6.86, outperforming average individual programmers.

   Machine learningIn International Conference on Learning Representations

   167cites