Machine learningLLMs & Text
Understanding Synthetic Context Extension via Retrieval Heads
The paper finds that fine-tuning long-context language models with synthetic data improves performance in retrieval and reasoning tasks, with attention heads predicting performance.
Featured in No. 72 on 31 Oct 2024 · 2 days after release · 13 citations today · published in International Conference on Machine Learning
- Released
- 29 Oct 2024
- First featured
- No. 72 · 31 Oct 2024
- Citations (Semantic Scholar)
- 13
- Influential citations
- 0
- Published in
- International Conference on Machine Learning
- Shares when featured
- 9
- Identifier
- arXiv:2410.22316
Citations and venue from Semantic Scholar (ODC-BY), refreshed weekly. Summary: Quant Letter (CC BY 4.0).