---
title: Understanding the Inner Workings of a State-Space Foundation Model for Limit Order Books
url: https://www.ml-quant.com/papers/arxiv/2610.28753/
site: ML-Quant (https://www.ml-quant.com)
updated: 2026-10-23
license: Summaries CC BY 4.0; links go to the original sources
index: https://www.ml-quant.com/llms.txt
identifier: arXiv:2610.28753
source_url: https://arxiv.org/abs/2610.28753
featured: 2026-10-23
citations: unknown
topic: Trading, Microstructure & Execution
---


# Understanding the Inner Workings of a State-Space Foundation Model for Limit Order Books

A Mamba sequence model trained on limit-order-book messages learns order-flow imbalance as an internal representation with R² ≈ 0.59 and generalizes across stocks and microstructure metrics.

- Source: https://arxiv.org/abs/2610.28753
- Identifier: arXiv:2610.28753
- Released: 2026-10-22
- First featured: Quant Letter No. 136 (2026-10-23): https://www.ml-quant.com/issues/2026-10-23/
- Citations (Semantic Scholar): not tracked
- Published in: not yet
- Topic: Trading, Microstructure & Execution
- Authors: Valentin Mohl, Alexandre Bismuth, Aramis Marti-Shahandeh, George Nigmatulin, Kang Li, Stefan Zohren, Jakob Foerster, Mihai Cucuringu, Anisoara Calinescu

## Abstract (arXiv, CC0)

Autoregressive foundation models offer state-of-the-art capabilities to simulate the market by generating the message flow of limit order books (LOBs). Yet, without understanding their inner workings, research directions are hard to narrow and practical use in real-world trading environments remains uncertain. This study therefore asks what these models actually learn about market microstructure. We find that, similarly to games where sequence models build an internal representation of the board, an 8.2M-parameter Mamba-3 state-space model trained autoregressively on the tokenized message stream of eight large-cap US equities learns an internal representation of the market and its dynamics. In particular, order-flow imbalance, a central driver of short-horizon price moves, is decodable at $R^2 \approx 0.59$, far above controls and baselines. This representation extends to unseen stocks, suggesting that these models learn general representations rather than ticker-specific patterns. The model also generalizes across metrics beyond order-flow imbalance: results hold across traded volume and volatility. Additionally, the representation is causally steerable: pushing the state along a feature's axis shifts the model's next-message beliefs as that feature implies, monotonically in dose, consistent with the emergence of a world model.

## Related

- [Adversarial Post-Training of Foundation Models for High-Frequency Trading](https://www.ml-quant.com/papers/arxiv/2610.25604/): Wasserstein GAN post-training on limit order book foundation models cuts error accumulation and improves realism by 17%, outperforming continued pre-training with far lower compute cost.
- [TradeFM: A Generative Foundation Model for Trade-flow and Market Microstructure](https://www.ml-quant.com/papers/arxiv/2602.23784/): A Model for Trade-Flow in Market Microstructure: TradeFM is a new AI model that improves the analysis of market structures by studying billions of trade events in stocks, leading to better simulations of financial returns.
- [Painting the market: generative diffusion models for financial limit order book simulation and forecasting](https://www.ml-quant.com/papers/arxiv/2509.05107/): A new method for simulating financial market data transforms limit order book data into an image format and uses diffusion models to predict future states, showing top performance on LOB-Bench.
- [Bimodal dynamics of the artificial limit order book stock exchange with autonomous traders](https://www.ml-quant.com/papers/arxiv/2508.17837/): The paper uncovers the inherent bistability and complex dynamics of an artificial stock market exchange, which emerge from micro-level trading rules.
- [Detecting Multilevel Manipulation from Limit Order Book via Cascaded Contrastive Representation Learning](https://www.ml-quant.com/papers/arxiv/2508.17086/): The study suggests a learning framework to enhance the detection of trade-based manipulation in financial markets, with Transformer-based architectures proving most successful.
- [Reinforcement learning for trade execution with market and limit orders](https://www.ml-quant.com/papers/arxiv/2507.06345/): The paper presents a reinforcement learning framework for optimal trade execution, using multivariate logistic-normal distributions, which outperforms traditional benchmark strategies.