---
title: Grounding Long Financial Text with Semantically-Enriched Tables via Multimodal Learning
url: https://www.ml-quant.com/papers/arxiv/2610.37913/
site: ML-Quant (https://www.ml-quant.com)
updated: 2026-10-30
license: Summaries CC BY 4.0; links go to the original sources
index: https://www.ml-quant.com/llms.txt
identifier: arXiv:2610.37913
source_url: https://arxiv.org/abs/2610.37913
featured: 2026-10-30
citations: unknown
topic: LLMs & Text
---


# Grounding Long Financial Text with Semantically-Enriched Tables via Multimodal Learning

A transformer framework integrates semantically enriched table embeddings with unstructured text via cross-attention, showing structured data dominates but text adds incremental signal.

- Source: https://arxiv.org/abs/2610.37913
- Identifier: arXiv:2610.37913
- Released: 2026-10-30
- First featured: Quant Letter No. 137 (2026-10-30): https://www.ml-quant.com/issues/2026-10-30/
- Citations (Semantic Scholar): not tracked
- Published in: not yet
- Topic: LLMs & Text
- Authors: Xiaoyu Hu

## Abstract (arXiv, CC0)

We propose a multimodal transformer framework for efficiently modeling long1 financial documents that combine structured tabular data with noisy, unstructured2 textual disclosures. The framework represents financial tables as semantically3 enriched token sequences and integrates them with long-form text through a cross-4 attention mechanism that explicitly captures cross-modal dependencies. In par-5 ticular, we augment each numerical variable with a semantic embedding derived6 from its textual definition, enabling the model to encode both numerical magnitude7 and economic meaning. This design yields a more expressive and parameter-8 efficient representation of tabular data compared to value-only embeddings. Using9 a large-scale dataset of financial statements and MD&A disclosures, we show that10 structured numerical features remain the dominant source of predictive signal;11 however, textual information provides consistent incremental gains when incor-12 porated via cross-attention rather than simple concatenation. Our results further13 demonstrate that noisy textual disclosures are most useful when conditioned on a14 stable, semantically structured tabular representation. In particular, using frozen15 variable-level tabular representations as cross-attention queries enables the model16 to weight MD&A chunks differently for each financial variable and observation.17 These findings suggest that cross-attention provides an efficient mechanism for18 table-conditioned aggregation of long-form narrative evidence.

## Related

- [SliceGPT: Compress Large Language Models by Deleting Rows and Columns](https://www.ml-quant.com/papers/arxiv/2401.15024/): Compressing Language Models: The paper introduces SliceGPT, a post-training sparsification scheme for large language models that reduces the network's embedding dimension, maintains high performance, reduces inference computation, and reveals computational invariance in transformer networks.
- [Transformer-based CoVaR: Systemic Risk in Textual Information](https://www.ml-quant.com/papers/repec/bri-uobdis-26-840/): Integrating financial news embeddings from large language models with market data, the study improves systemic risk forecasts using conditional value-at-risk without requiring large datasets.
- [Efficient Adversarial Training in LLMs with Continuous Attacks](https://www.ml-quant.com/papers/arxiv/2405.15589/): CAdvUL, a new adversarial training algorithm, enhances the resilience of large language models against adversarial attacks by efficiently calculating attacks in the continuous embedding space.
- [You Only Cache Once: Decoder-Decoder Architectures for Language Models](https://www.ml-quant.com/papers/arxiv/2405.05254/): YOCO architecture improves large language models by reducing GPU memory usage and speeding up the prefill stage, outperforming the Transformer model.
- [Gecko: Versatile Text Embeddings Distilled from Large Language Models](https://www.ml-quant.com/papers/arxiv/2403.20327/): Compact Text Embeddings: Gecko is a new text embedding model that improves knowledge extraction from large language models, surpassing other models in the Massive Text Embedding Benchmark.
- [From News to Returns: A Granger-Causal Hypergraph Transformer on the Sphere](https://www.ml-quant.com/papers/arxiv/2510.04357/): The research suggests the CSHT, a new architecture for financial time-series forecasting that considers the impact of financial news and sentiment on asset returns, providing robust generalisation across market regimes and clear attribution pathways.