Machine learningLLMs & Text
Optimizing Large Language Model Training Using FP4 Quantization
The research presents the first FP4 training framework for large language models (LLMs), using low-bit arithmetic operations to lessen computational demands, achieving similar accuracy to BF16 and FP8 with slight degradation.
Featured in No. 84 on 5 Feb 2025 · 8 days after release · 61 citations today · published in International Conference on Machine Learning
- Released
- 28 Jan 2025
- First featured
- No. 84 · 5 Feb 2025
- Citations (Semantic Scholar)
- 61
- Influential citations
- 7
- Published in
- International Conference on Machine Learning
- Shares when featured
- 52
- Identifier
- arXiv:2501.17116
Citations and venue from Semantic Scholar (ODC-BY), refreshed weekly. Summary: Quant Letter (CC BY 4.0).