---
title: EdgeRAG: Online-Indexed RAG for Edge Devices
url: https://www.ml-quant.com/papers/arxiv/2412.21023/
site: ML-Quant (https://www.ml-quant.com)
updated: 2026-09-26
license: Summaries CC BY 4.0; links go to the original sources
index: https://www.ml-quant.com/llms.txt
identifier: arXiv:2412.21023
source_url: https://arxiv.org/abs/2412.21023
featured: 2025-01-08
citations: 27
topic: LLMs & Text
---


# EdgeRAG: Online-Indexed RAG for Edge Devices

Online RAG: EdgeRAG is a system proposed for deploying Retrieval Augmented Generation on devices with limited resources, reducing latency and memory usage by pruning and generating embeddings as needed.

- Source: https://arxiv.org/abs/2412.21023
- Identifier: arXiv:2412.21023
- Released: 2024-12-30
- First featured: Quant Letter No. 81 (2025-01-08): https://www.ml-quant.com/issues/2025-01-08/
- Citations (Semantic Scholar): 27
- Published in: not yet
- Topic: LLMs & Text

## Related

- [SliceGPT: Compress Large Language Models by Deleting Rows and Columns](https://www.ml-quant.com/papers/arxiv/2401.15024/): Compressing Language Models: The paper introduces SliceGPT, a post-training sparsification scheme for large language models that reduces the network's embedding dimension, maintains high performance, reduces inference computation, and reveals computational invariance in transformer networks.
- [RAFT: Adapting Language Model to Domain Specific RAG](https://www.ml-quant.com/papers/arxiv/2403.10131/): Retrieval Augmented FineTuning (RAFT) is a new training method that enhances large language models' ability to answer domain-specific questions by training them to ignore irrelevant documents and cite relevant ones.
- [HybridRAG: Integrating Knowledge Graphs and Vector Retrieval Augmented Generation for Efficient Information Extraction](https://www.ml-quant.com/papers/arxiv/2408.04948/): Q&A Systems for Financial Data: HybridRAG, a new method combining Knowledge Graphs and VectorRAG techniques, improves question-answer systems for extracting information from financial documents, offering better accuracy and answer generation.
- [Efficient Adversarial Training in LLMs with Continuous Attacks](https://www.ml-quant.com/papers/arxiv/2405.15589/): CAdvUL, a new adversarial training algorithm, enhances the resilience of large language models against adversarial attacks by efficiently calculating attacks in the continuous embedding space.
- [ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilities](https://www.ml-quant.com/papers/arxiv/2407.14482/): Bridging the Gap: ChatQA 2 is a model that improves long-context understanding and retrieval-augmented generation, matching the accuracy of top proprietary models.
- [Gecko: Versatile Text Embeddings Distilled from Large Language Models](https://www.ml-quant.com/papers/arxiv/2403.20327/): Compact Text Embeddings: Gecko is a new text embedding model that improves knowledge extraction from large language models, surpassing other models in the Massive Text Embedding Benchmark.