Machine learningML & AI Methods
An Empirical Study of $\mu$P Learning Rate Transfer
Neural Network Scaling Rules: A study has found that the μ-Parameterization (μP) is generally effective in determining the best learning rates for large neural network models, although it doesn't work in all situations.
Featured in No. 45 on 17 Apr 2024 · 9 days after release · 7 citations today
- Released
- 8 Apr 2024
- First featured
- No. 45 · 17 Apr 2024
- Citations (Semantic Scholar)
- 7
- Influential citations
- 0
- Published in
- Not yet, as far as Semantic Scholar knows
- Shares when featured
- 109
- Identifier
- arXiv:2404.05728
Citations and venue from Semantic Scholar (ODC-BY), refreshed weekly. Summary: Quant Letter (CC BY 4.0).