Machine learningLLMs & Text

CuMo: Scaling Multimodal LLM with Co-Upcycled Mixture-of-Experts

CuMo, a model that integrates Co-upcycled Top-K sparsely-gated Mixture-of-experts blocks into the vision encoder and the MLP connector, improves multimodal LLMs with minimal additional activated parameters during inference, outperforming other multimodal LLMs across various benchmarks.