Machine learningLLMs & Text

Adapt-Pruner: Adaptive Structural Pruning for Efficient Small Language Model Training

The research explores ways to speed up small language models, discovering that layer-wise adaptive pruning (Adapt-Pruner) is effective in large language models and outperforms existing pruning methods.