Machine learningML & AI Methods
Grokking at the Edge of Numerical Stability
The study investigates 'grokking' in deep learning, introduces Softmax Collapse and naïve loss minimization concepts, and suggests a new activation function and training algorithm for grokking without regularization.
Featured in No. 82 on 15 Jan 2025 · 7 days after release · 41 citations today · published in International Conference on Learning Representations
- Released
- 8 Jan 2025
- First featured
- No. 82 · 15 Jan 2025
- Citations (Semantic Scholar)
- 41
- Influential citations
- 6
- Published in
- International Conference on Learning Representations
- Shares when featured
- 50
- Identifier
- arXiv:2501.04697
Citations and venue from Semantic Scholar (ODC-BY), refreshed weekly. Summary: Quant Letter (CC BY 4.0).