---
title: Learning the RoPEs: Better 2D and 3D Position Encodings with STRING
url: https://www.ml-quant.com/papers/arxiv/2502.02562/
site: ML-Quant (https://www.ml-quant.com)
updated: 2026-09-26
license: Summaries CC BY 4.0; links go to the original sources
index: https://www.ml-quant.com/llms.txt
identifier: arXiv:2502.02562
source_url: https://arxiv.org/abs/2502.02562
featured: 2025-04-02
citations: 20
topic: ML & AI Methods
---


# Learning the RoPEs: Better 2D and 3D Position Encodings with STRING

The article introduces STRING, an extension of Rotary Position Encodings, which offers exact translation invariance and low computational footprint, proving beneficial in robotics and Vision Transformers.

- Source: https://arxiv.org/abs/2502.02562
- Identifier: arXiv:2502.02562
- Released: 2025-02-04
- First featured: Quant Letter No. 91 (2025-04-02): https://www.ml-quant.com/issues/2025-04-02/
- Citations (Semantic Scholar): 20
- Published in: International Conference on Machine Learning
- Topic: ML & AI Methods

## Related

- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://www.ml-quant.com/papers/arxiv/2312.00752/): Sequence Modeling: Mamba, a neural network architecture that doesn't use attention or MLP blocks, provides faster inference and better performance in language, audio, and genomics than Transformers.
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://www.ml-quant.com/papers/arxiv/2405.21060/): The research identifies a link between state-space models and Transformers in deep learning, leading to the creation of a faster language modeling architecture, Mamba-2.
- [Octo: An Open-Source Generalist Robot Policy](https://www.ml-quant.com/papers/arxiv/2405.12213/): Octo is a large transformer-based policy for robotic manipulation, trained on a vast dataset, that can be instructed via language or images and adapted to new domains.
- [Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction](https://www.ml-quant.com/papers/arxiv/2404.02905/): The article discusses Visual AutoRegressive modeling (VAR), a new image learning method that outperforms diffusion transformers in terms of speed, image quality, and scalability.
- [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://www.ml-quant.com/papers/arxiv/2310.00426/): PIXART-$\alpha$, a Transformer-based text-to-image model, generates high-quality images at a low cost, reducing CO2 emissions and offering a cost-effective solution for the AIGC community.
- [Gated Linear Attention Transformers with Hardware-Efficient Training](https://www.ml-quant.com/papers/arxiv/2312.06635/): Efficient Training of Gated Linear Attention Transformers: The research introduces a more hardware-efficient version of gated linear attention Transformers that performs well against other models, especially in training on longer sequences.