---
title: RapVerse: Coherent Vocals and Whole-Body Motion Generation from Text
url: https://www.ml-quant.com/papers/arxiv/2405.20336/
site: ML-Quant (https://www.ml-quant.com)
updated: 2026-09-26
license: Summaries CC BY 4.0; links go to the original sources
index: https://www.ml-quant.com/llms.txt
identifier: arXiv:2405.20336
source_url: https://arxiv.org/abs/2405.20336
featured: 2024-06-05
citations: 7
topic: LLMs & Text
---


# RapVerse: Coherent Vocals and Whole-Body Motion Generation from Text

Vocals and Motions: A new task uses a multimodal transformer model to generate 3D body motions and singing vocals from textual lyrics, ensuring a realistic blend of vocals and human motions.

- Source: https://arxiv.org/abs/2405.20336
- Identifier: arXiv:2405.20336
- Released: 2024-05-30
- First featured: Quant Letter No. 52 (2024-06-05): https://www.ml-quant.com/issues/2024-06-05/
- Citations (Semantic Scholar): 7
- Published in: 2025 IEEE/CVF International Conference on Computer Vision (ICCV)
- Topic: LLMs & Text

## Related

- [SliceGPT: Compress Large Language Models by Deleting Rows and Columns](https://www.ml-quant.com/papers/arxiv/2401.15024/): Compressing Language Models: The paper introduces SliceGPT, a post-training sparsification scheme for large language models that reduces the network's embedding dimension, maintains high performance, reduces inference computation, and reveals computational invariance in transformer networks.
- [You Only Cache Once: Decoder-Decoder Architectures for Language Models](https://www.ml-quant.com/papers/arxiv/2405.05254/): YOCO architecture improves large language models by reducing GPU memory usage and speeding up the prefill stage, outperforming the Transformer model.
- [player2vec: A Language Modeling Approach to Understand Player Behavior in Games](https://www.ml-quant.com/papers/arxiv/2404.04234/): Player Behavior in Games: A new technique for learning hidden user profiles from player behavior data in video and mobile games is presented, utilizing a long-range Transformer model from natural language processing, showing promising results in matching behavior event distribution.
- [Transformer Explainer: Learning LLM Transformers with Interactive Visual Explanation and Experimentation](https://www.ml-quant.com/papers/arxiv/2408.04619/): Transformer Explainer, an interactive tool, is unveiled to help non-experts understand Transformers through the GPT-2 model, allowing real-time user input experimentation.
- [No Train, all Gain: Self-Supervised Gradients Improve Deep Frozen Representations](https://www.ml-quant.com/papers/arxiv/2407.10964/): The FUNGI method improves transformer encoders' features using self-supervised gradients, enhancing performance in vision, natural language processing, and audio tasks and datasets.
- [A Single Transformer for Scalable Vision-Language Modeling](https://www.ml-quant.com/papers/arxiv/2407.06438/): SOLO is a unified transformer for vision-language modeling, addressing scalability issues in large models and providing an open-source training blueprint.