---
title: Reinforcement Learning with Hawkes-Structured Networks for Market Making
url: https://www.ml-quant.com/papers/arxiv/2610.16524/
site: ML-Quant (https://www.ml-quant.com)
updated: 2026-10-16
license: Summaries CC BY 4.0; links go to the original sources
index: https://www.ml-quant.com/llms.txt
identifier: arXiv:2610.16524
source_url: https://arxiv.org/abs/2610.16524
featured: 2026-10-16
citations: unknown
topic: Trading, Microstructure & Execution
---


# Reinforcement Learning with Hawkes-Structured Networks for Market Making

The study proposes a neural network architecture for market making that models self-exciting order arrivals through learnable exponential features and trains via reinforcement learning to optimize quoting policy.

- Source: https://arxiv.org/abs/2610.16524
- Identifier: arXiv:2610.16524
- Released: 2026-10-14
- First featured: Quant Letter No. 135 (2026-10-16): https://www.ml-quant.com/issues/2026-10-16/
- Citations (Semantic Scholar): not tracked
- Published in: not yet
- Topic: Trading, Microstructure & Execution
- Authors: Fernando Acero, Philipp Plank, Yufei Zhang

## Abstract (arXiv, CC0)

Market making under realistic order-flow dynamics is challenging due to the clustered, self-exciting nature of order arrivals, which are naturally modeled by non-Markovian Hawkes processes. We propose a model-free reinforcement learning (RL) framework for market making with general Hawkes order-flow dynamics. We first establish non-asymptotic performance bounds that quantify the loss incurred by approximating the infinite-dimensional Hawkes state using a finite collection of exponentially weighted features. Guided by this analysis, we introduce the Hawkes-Structured Network (HSN), a novel neural architecture that encodes historical order flow through a learnable mixture-of-exponentials filter. HSN is trained end-to-end using model-free RL, jointly optimizing the feature representation and quoting policy directly from reward feedback. Experiments on single- and multi-asset market-making problems with mixture-of-exponentials, power-law, and regime-switching kernels demonstrate that HSN outperforms general-purpose neural architectures, achieving higher trading rewards while implicitly recovering the underlying order-flow intensities.

## Related

- [Robust Market Making with Hawkes Order Flow and Price Impact via Adversarial Reinforcement Learning](https://www.ml-quant.com/papers/arxiv/2609.22785/): The research extends adversarial reinforcement learning for market making to handle self-exciting order arrivals and price impact, using an LSTM module to improve robustness in complex microstructure environments.
- [Event-Based Limit Order Book Simulation under a Neural Hawkes Process: Application in Market-Making](https://www.ml-quant.com/papers/arxiv/2502.17417/): An event-driven Limit Order Book model using a Neural Hawkes process is proposed to simulate high-frequency dynamics in financial markets, offering a more accurate depiction of trade execution.
- [Market Making with Exogenous Competition](https://www.ml-quant.com/papers/arxiv/2407.17393/): A study reveals that a 'reference market maker' who optimizes her posted depths can achieve a near perfect solution, which is compared against other solutions using an Euler scheme or reinforcement learning techniques in a competitive environment.
- [Consistent time travel for realistic interactions with historical data: reinforcement learning for market making](https://www.ml-quant.com/papers/arxiv/2408.02322/): The article discusses the use of consistent data time travel in offline reinforcement learning for market making in limit order books.
- [Reinforcement Learning for Arbitrage in Decentralized Exchanges](https://www.ml-quant.com/papers/ssrn/4708173/): The article presents a game-theory model to explain the tactics of various players in decentralized exchanges with automated market makers.
- [Deep Reinforcement Learning for Active High Frequency Trading](https://www.ml-quant.com/papers/arxiv/2101.07107/): A new Deep Reinforcement Learning framework has been developed for high frequency stock trading, showing potential for profitable long-term strategies.