YuE

YuE2: frontier music generation with symbolic planning, zero-shot covers, and agentic music editing.

What is YuE?

YuE2: frontier music generation with symbolic planning, zero-shot covers, and agentic music editing.

Overview of YuE

What it is

YuE2 is a music generation system that unifies symbolic and audio generation. It takes lyrics and a style prompt, writes a melody-and-chord plan, and realizes it as a complete song with vocals and accompaniment. It is the successor to YuE, with its code and license preserved on the YuE-v1 branch.

Key features

  • Frontier quality: competitive with Suno v5/v6 on WildSongBench; achieves 6.9632 SongBench Avg (best-of-8).
  • White-box music generation through symbolic planning: melody and chords are explicit controls that can be inspected and edited.
  • Zero-shot covers and agentic editing: transcribe a source recording and reinterpret it in a new style, or refine a song through conversation about its score, arrangement, and lyrics.
  • One AR–NAR Mixture-of-Transformers backbone predicts score and semantic tokens autoregressively, then generates acoustic latents with flow matching.
  • Staged Python API: plan() → generate_semantic() → synthesize() → decode().
  • Agent skill (yue2-music) enables instrumental generation and covers from description, ABC score, or reference recording.

Use cases

  • Create a full song from lyrics and a style prompt.
  • Cover a song: transcribe a source recording with SheetSage2, provide new lyrics or target style, and generate a new interpretation.
  • Edit a composition: export the plan, revise the musical details (harmony, melody, tempo, form), and render the edited score.
  • Agentic editing: an agent can follow a conversation to refine a song through multiple steps.

Getting started

bash git clone https://github.com/multimodal-art-projection/YuE.git cd YuE python3.12 -m venv .venv source .venv/bin/activate python -m pip install --upgrade pip python -m pip install . python examples/generate.py --output outputs/first-song Requires Linux, Python 3.12, NVIDIA GPU with BF16 support and 24 GB VRAM. Model files download from Hugging Face on first use.

License and pricing

Licensed under Apache-2.0. Open source.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/multimodal-art-projectionyue.svg)](https://yunodb.com/tools/multimodal-art-projectionyue?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/multimodal-art-projectionyue?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/multimodal-art-projectionyue.svg" alt="Featured on YunoDB" height="20"></a>