---
title: Cooperative Multi-Agent Reinforcement Learning for Market Making on Prediction-Market Order Books
url: https://www.ml-quant.com/papers/arxiv/2610.26789/
site: ML-Quant (https://www.ml-quant.com)
updated: 2026-10-23
license: Summaries CC BY 4.0; links go to the original sources
index: https://www.ml-quant.com/llms.txt
identifier: arXiv:2610.26789
source_url: https://arxiv.org/abs/2610.26789
featured: 2026-10-23
citations: unknown
topic: Trading, Microstructure & Execution
---


# Cooperative Multi-Agent Reinforcement Learning for Market Making on Prediction-Market Order Books

Decomposing a market maker into quoter and risk-manager agents trained with cooperative IPPO on Polymarket reduces held-out losses and improves quoting frequency versus baselines.

- Source: https://arxiv.org/abs/2610.26789
- Identifier: arXiv:2610.26789
- Released: 2026-10-20
- First featured: Quant Letter No. 136 (2026-10-23): https://www.ml-quant.com/issues/2026-10-23/
- Citations (Semantic Scholar): not tracked
- Published in: not yet
- Topic: Trading, Microstructure & Execution
- Authors: Skander Lejmi

## Abstract (arXiv, CC0)

A market maker provides liquidity by quoting prices to buy and sell an asset, earning the spread while managing inventory risk. We study market makers trained with multi-agent reinforcement learning on prediction markets, specifically Polymarket, whose thin order books pose distinct challenges. Three challenges underpin this research: adverse selection may cause a policy trained on real data to stop quoting; the consequences of a quote can arrive many steps after the action; and prediction-market L2 data does not identify individual orders needed for simulation. We address these challenges by reconstructing order-level LOBSTER-format data from Polymarket and training agents in JaxMARL-HFT on 417 real markets. We enrich the baseline market maker's observation space with the book and recent flow, redesign its actions and inventory controls, and propose a cooperative multi-agent decomposition into a quoter and a risk manager trained using recurrent Independent Proximal Policy Optimisation (IPPO) with RNN policies. We investigate reward shaping, particularly reward blending and post-fill markout, to mitigate credit-assignment difficulties between the agents. Our redesigned single-agent market maker quotes more frequently and incurs smaller held-out losses than the retrained JaxMARL-HFT baseline, although the contributions of jointly changed observations, actions and risk constraints are not separately identified. Decomposition alone performs poorly; careful reward design substantially improves the desk relative to the unshaped split and yields our best result. Models are evaluated on realised end-of-episode portfolio value on held-out markets. None establishes positive average portfolio value in periodic held-out evaluation, and each configuration was trained with one seed. The environment, data and protocol lay a foundation for makers that quote many linked markets at once.

## Related

- [Robust Market Making with Hawkes Order Flow and Price Impact via Adversarial Reinforcement Learning](https://www.ml-quant.com/papers/arxiv/2609.22785/): The research extends adversarial reinforcement learning for market making to handle self-exciting order arrivals and price impact, using an LSTM module to improve robustness in complex microstructure environments.
- [Consistent time travel for realistic interactions with historical data: reinforcement learning for market making](https://www.ml-quant.com/papers/arxiv/2408.02322/): The article discusses the use of consistent data time travel in offline reinforcement learning for market making in limit order books.
- [Reinforcement learning for trade execution with market and limit orders](https://www.ml-quant.com/papers/arxiv/2507.06345/): The paper presents a reinforcement learning framework for optimal trade execution, using multivariate logistic-normal distributions, which outperforms traditional benchmark strategies.
- [Event-Based Limit Order Book Simulation under a Neural Hawkes Process: Application in Market-Making](https://www.ml-quant.com/papers/arxiv/2502.17417/): An event-driven Limit Order Book model using a Neural Hawkes process is proposed to simulate high-frequency dynamics in financial markets, offering a more accurate depiction of trade execution.
- [Market Making with Exogenous Competition](https://www.ml-quant.com/papers/arxiv/2407.17393/): A study reveals that a 'reference market maker' who optimizes her posted depths can achieve a near perfect solution, which is compared against other solutions using an Euler scheme or reinforcement learning techniques in a competitive environment.
- [The Negative Drift of a Limit Order Fill](https://www.ml-quant.com/papers/arxiv/2407.16527/): The research identifies a negative drift in market making models, particularly in limit order fills, using the 10 Year US Treasury Bond futures for empirical simulation.