Machine learningML & AI Methods
Transformers are Multi-State RNNs
The research shows that decoder-only transformers can be seen as infinite multi-state RNNs and introduces a new policy, TOVA, which performs better in long-range tasks and uses less memory.
Featured in No. 33 on 17 Jan 2024 · 6 days after release · 128 citations today · published in Conference on Empirical Methods in Natural Language Processing
- Released
- 11 Jan 2024
- First featured
- No. 33 · 17 Jan 2024
- Citations (Semantic Scholar)
- 128
- Influential citations
- 21
- Published in
- Conference on Empirical Methods in Natural Language Processing
- Shares when featured
- 152
- Identifier
- arXiv:2401.06104
Citations and venue from Semantic Scholar (ODC-BY), refreshed weekly. Summary: Quant Letter (CC BY 4.0).