Towards Robust Reinforcement Learning Market Making: Lessons from Loss Function Experimentation

Market makers play a vital role in ensuring efficient trade within financial markets. Recent literature has delved into training market-making systems using reinforcement learning. This paper delves into the impact of various loss function formulations on risk management behavior within reinforcement-based market making. Through empirical evaluation, we contrast profit-maximizing strategies with risk-averse approaches, finding that penalties for inventory and cash management foster more resilient and adaptable market-making strategies. Our results underscore the significance of striking a balance between profit maximization and risk management in algorithmic trading, providing actionable insights for refining strategies in practical settings.

1. 31 Jul 2024

   Market Making with Exogenous Competition

   A study reveals that a 'reference market maker' who optimizes her posted depths can achieve a near perfect solution, which is compared against other solutions using an Euler scheme or reinforcement learning techniques in a competitive environment.

   arXivIn SIAM J. Financial Math.

   12cites
2. 13 Nov 2026

   Risk-Sensitive Reinforcement Learning for an Options Market Maker: An Empirical Study with Real S&P 500 Options

   A reinforcement learning agent trained on historical S&P 500 options data reduces tail losses by 3.1% versus delta hedging, reaching 8.9% in volatile downturns.

   arXiv

   3fanfare
3. 23 Oct 2026

   Cooperative Multi-Agent Reinforcement Learning for Market Making on Prediction-Market Order Books

   Decomposing a market maker into quoter and risk-manager agents trained with cooperative IPPO on Polymarket reduces held-out losses and improves quoting frequency versus baselines.

   arXiv

   0cites
4. 16 Oct 2026

   Reinforcement Learning with Hawkes-Structured Networks for Market Making

   The study proposes a neural network architecture for market making that models self-exciting order arrivals through learnable exponential features and trains via reinforcement learning to optimize quoting policy.

   arXiv

   0cites
5. 25 Sep 2026

   Robust Market Making with Hawkes Order Flow and Price Impact via Adversarial Reinforcement Learning

   The research extends adversarial reinforcement learning for market making to handle self-exciting order arrivals and price impact, using an LSTM module to improve robustness in complex microstructure environments.

   arXiv

   0cites
6. 7 Aug 2024

   Consistent time travel for realistic interactions with historical data: reinforcement learning for market making

   The article discusses the use of consistent data time travel in offline reinforcement learning for market making in limit order books.

   arXiv

   0cites