THEATRON: A Benchmark for Evaluating Persona Persistence in Long-Horizon LLM Decision-Making

Existing evaluations often test persona expression rather than whether persona information persists in extended decision-making. We introduce THEATRON, a controlled benchmark that uses financial replay to evaluate persona persistence in long-horizon sequential decision-making. In these environments, models conditioned on fixed behavioral-finance personas observe shared daily market evidence and trajectory-specific account-state feedback and make sequential decisions over multi-month trajectories. We evaluate Zero-Shot, One-Shot, and Chain-of-Thought persona injection across four LLMs, using cumulative-return and action trajectories to diagnose within-persona stability, cross-persona separability, and action collapse. Persona conditioning produces systematic behavioral shifts beyond a no-persona baseline, and persona-level separation generally exceeds the variation induced by semantically equivalent descriptions. Nevertheless, these shifts do not reliably form coherent persona-specific policies: apparently stable separation can arise from stereotyped action collapse, while diverse policies can exhibit noisy trajectory switching. These findings expose a gap between observable persona-induced behavior and persistent, context-sensitive persona-conditioned decision policies. Code and benchmark data are publicly available at: https://github.com/yuyan12138/theatron

1. 15 May 2024

   AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments

   AI Evaluation in Clinical Environments: The paper introduces AgentClinic, a benchmark for assessing large language models in simulated clinical environments, highlighting the significant impact of biases on diagnostic accuracy and patient interactions.

   Machine learning

   293cites
2. 1 May 2024

   Learning Performance-Improving Code Edits

   The research presents a framework for optimizing programs using large language models, achieving a mean speedup of 6.86, outperforming average individual programmers.

   Machine learningIn International Conference on Learning Representations

   186cites
3. 13 Dec 2023

   Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordia

   Concordia is a library designed to help build and operate Generative Agent-Based Models (GABMs), using Large Language Models (LLMs) to simulate physical or digital environments.

   Machine learning

   189cites
4. 28 Aug 2024

   Jamba-1.5: Hybrid Transformer-Mamba Models at Scale

   Transformer-Mamba Models: Jamba-1.5 is a new large language model with enhanced conversational and instruction-following capabilities, featuring a unique quantization technique for cost-effective inference.

   Machine learning

   57cites
5. 31 Jul 2024

   MindSearch: Mimicking Human Minds Elicits Deep AI Searcher

   Mimicking Human Minds for Search: MindSearch is a Large Language Model-based framework that simulates human cognitive processes for web information seeking, greatly enhancing response quality.

   Machine learningIn International Conference on Learning Representations

   91cites
6. 16 Oct 2026

   What Do Foundation Models for Market Microstructure Learn? A Controlled Study with Synthetic Limit Order Books

   Autoregressive transformers trained on limit-order-book message streams learn amortized Bayesian inference over market regimes, matching optimal prediction and enabling accurate parameter calibration.

   arXiv

   1cites