Machine learningLLMs & Text
MoMa: Efficient Early-Fusion Pre-training with Mixture of Modality-Aware Experts
The MoMa model is a new architecture designed for pre-training mixed-modal language models, providing improved efficiency in processing images and text in any order.
Featured in No. 60 on 7 Aug 2024 · 7 days after release · 78 citations today
- Released
- 31 Jul 2024
- First featured
- No. 60 · 7 Aug 2024
- Citations (Semantic Scholar)
- 78
- Influential citations
- 4
- Published in
- Not yet, as far as Semantic Scholar knows
- Shares when featured
- 348
- Identifier
- arXiv:2407.21770
Citations and venue from Semantic Scholar (ODC-BY), refreshed weekly. Summary: Quant Letter (CC BY 4.0).