Machine learningML & AI Methods
Scaling Laws for Fine-Grained Mixture of Experts
The research introduces a new hyperparameter, granularity, to Mixture of Experts models, improving training optimization and outperforming dense Transformers.
Featured in No. 37 on 14 Feb 2024 · 2 days after release · 181 citations today · published in International Conference on Machine Learning
- Released
- 12 Feb 2024
- First featured
- No. 37 · 14 Feb 2024
- Citations (Semantic Scholar)
- 181
- Influential citations
- 12
- Published in
- International Conference on Machine Learning
- Shares when featured
- 120
- Identifier
- arXiv:2402.07871
Citations and venue from Semantic Scholar (ODC-BY), refreshed weekly. Summary: Quant Letter (CC BY 4.0).