---
title: Towards Robust Reinforcement Learning Market Making: Lessons from Loss Function Experimentation
url: https://www.ml-quant.com/papers/arxiv/2611.09012/
site: ML-Quant (https://www.ml-quant.com)
updated: 2026-11-14
license: Summaries CC BY 4.0; links go to the original sources
index: https://www.ml-quant.com/llms.txt
identifier: arXiv:2611.09012
source_url: https://arxiv.org/abs/2611.09012
featured: 2026-11-13
citations: unknown
topic: Trading, Microstructure & Execution
---


# Towards Robust Reinforcement Learning Market Making: Lessons from Loss Function Experimentation

Adding inventory and cash management penalties to reinforcement learning market makers improves resilience and adaptability compared to pure profit-maximizing approaches.

- Source: https://arxiv.org/abs/2611.09012
- Identifier: arXiv:2611.09012
- Released: 2026-11-09
- First featured: Quant Letter No. 139 (2026-11-13): https://www.ml-quant.com/issues/2026-11-13/
- Citations (Semantic Scholar): not tracked
- Published in: not yet
- Topic: Trading, Microstructure & Execution
- Authors: Siddharth Nand, Bora Guney

## Abstract (arXiv, CC0)

Market makers play a vital role in ensuring efficient trade within financial markets. Recent literature has delved into training market-making systems using reinforcement learning. This paper delves into the impact of various loss function formulations on risk management behavior within reinforcement-based market making. Through empirical evaluation, we contrast profit-maximizing strategies with risk-averse approaches, finding that penalties for inventory and cash management foster more resilient and adaptable market-making strategies. Our results underscore the significance of striking a balance between profit maximization and risk management in algorithmic trading, providing actionable insights for refining strategies in practical settings.

## Related

- [Market Making with Exogenous Competition](https://www.ml-quant.com/papers/arxiv/2407.17393/): A study reveals that a 'reference market maker' who optimizes her posted depths can achieve a near perfect solution, which is compared against other solutions using an Euler scheme or reinforcement learning techniques in a competitive environment.
- [Risk-Sensitive Reinforcement Learning for an Options Market Maker: An Empirical Study with Real S&P 500 Options](https://www.ml-quant.com/papers/arxiv/2611.10956/): A reinforcement learning agent trained on historical S&P 500 options data reduces tail losses by 3.1% versus delta hedging, reaching 8.9% in volatile downturns.
- [Cooperative Multi-Agent Reinforcement Learning for Market Making on Prediction-Market Order Books](https://www.ml-quant.com/papers/arxiv/2610.26789/): Decomposing a market maker into quoter and risk-manager agents trained with cooperative IPPO on Polymarket reduces held-out losses and improves quoting frequency versus baselines.
- [Reinforcement Learning with Hawkes-Structured Networks for Market Making](https://www.ml-quant.com/papers/arxiv/2610.16524/): The study proposes a neural network architecture for market making that models self-exciting order arrivals through learnable exponential features and trains via reinforcement learning to optimize quoting policy.
- [Robust Market Making with Hawkes Order Flow and Price Impact via Adversarial Reinforcement Learning](https://www.ml-quant.com/papers/arxiv/2609.22785/): The research extends adversarial reinforcement learning for market making to handle self-exciting order arrivals and price impact, using an LSTM module to improve robustness in complex microstructure environments.
- [Consistent time travel for realistic interactions with historical data: reinforcement learning for market making](https://www.ml-quant.com/papers/arxiv/2408.02322/): The article discusses the use of consistent data time travel in offline reinforcement learning for market making in limit order books.