Machine learningLLMs & Text

LightTransfer: Your Long-Context LLM is Secretly a Hybrid Model with Effortless Adaptation

SimLayerKV is a technique that minimizes memory usage in large language models by identifying and reducing cache in lazy layers, achieving significant cache compression with minimal performance loss.