YuE
YuE2: frontier music generation with symbolic planning, zero-shot covers, and agentic music editing.
What is YuE?
YuE2: frontier music generation with symbolic planning, zero-shot covers, and agentic music editing.
Overview of YuE
What it is
YuE2 is a music generation system that unifies symbolic and audio generation. It takes lyrics and a style prompt, writes a melody-and-chord plan, and realizes it as a complete song with vocals and accompaniment. It is the successor to YuE, with its code and license preserved on the YuE-v1 branch.
Key features
- Frontier quality: competitive with Suno v5/v6 on WildSongBench; achieves 6.9632 SongBench Avg (best-of-8).
- White-box music generation through symbolic planning: melody and chords are explicit controls that can be inspected and edited.
- Zero-shot covers and agentic editing: transcribe a source recording and reinterpret it in a new style, or refine a song through conversation about its score, arrangement, and lyrics.
- One AR–NAR Mixture-of-Transformers backbone predicts score and semantic tokens autoregressively, then generates acoustic latents with flow matching.
- Staged Python API: plan() → generate_semantic() → synthesize() → decode().
- Agent skill (yue2-music) enables instrumental generation and covers from description, ABC score, or reference recording.
Use cases
- Create a full song from lyrics and a style prompt.
- Cover a song: transcribe a source recording with SheetSage2, provide new lyrics or target style, and generate a new interpretation.
- Edit a composition: export the plan, revise the musical details (harmony, melody, tempo, form), and render the edited score.
- Agentic editing: an agent can follow a conversation to refine a song through multiple steps.
Getting started
bash git clone https://github.com/multimodal-art-projection/YuE.git cd YuE python3.12 -m venv .venv source .venv/bin/activate python -m pip install --upgrade pip python -m pip install . python examples/generate.py --output outputs/first-song Requires Linux, Python 3.12, NVIDIA GPU with BF16 support and 24 GB VRAM. Model files download from Hugging Face on first use.
License and pricing
Licensed under Apache-2.0. Open source.
Based on the project's public README and repository metadata.