YuE2

License Requirement

YuE2 model weights are under CC‑BY‑NC‑4.0 (attribution required, non‑commercial only), while project code uses Apache 2.0 license.

Basic Project Information

YuE2 is developed by HKUST and Multimodal‑Art‑Projection (M‑A‑P). It is an open‑source full‑song music generation model. ‑ Application scenarios: music creation, short video soundtracks, lyrics‑to‑song conversion, AI voice cover, music research. ‑ Underlying technology: Two‑stage architecture combining symbolic planning and acoustic flow matching. It first generates editable ABC musical scores (melody and chords), then renders 48kHz stereo vocals and accompaniment via flow matching model.

Main Functions (for general users)

1. Generate full songs with vocal and accompaniment from lyrics and style prompts, supporting multiple languages.
2. Editable ABC musical score: modify melody and chords before audio rendering.
3. Reference audio style transfer for AI voice cover.
4. Customize verse/chorus structure and music genres.

Advantages (with supporting references)

1. Editable symbolic score layer for controllable music structure Reference: Its pipeline outputs ABC notation score before audio generation. Users or agents can edit chords and melody before rendering audio, unlike end‑to‑end waveform music models.
2. Strong benchmark performance among open‑source music models Reference: On WildSongBench, YuE2 best-of-8 reaches 6.9632, outperforming Suno v6 and approaching Suno v5. It belongs to top-tier open music generation models (self-reported benchmark).
3. Native long song generation; RTX4090 can generate a 3.6‑minute song in 71 seconds Reference: Official benchmark shows RTX4090 can finish planning and rendering a 3.6‑minute song within 71 seconds, no manual segment splicing required.
4. 48kHz stereo output, separated vocal/accompaniment modeling, multi‑language and multi‑genre support Reference: Outputs 48kHz stereo audio. Supports Mandarin, English, Cantonese, Japanese, Korean and various music styles, with AI cover capability based on reference accompaniment.

Disadvantages (with supporting references)

1. Official native support for Linux + NVIDIA GPUs only; difficult native deployment on Windows Reference: Official environment recommendation is Linux with NVIDIA GPU. No native Windows support. Windows users need WSL or third-party packaged builds.
2. Occasionally weakens musical melody and produces ambient sound effects despite music‑oriented prompts Reference: Community user tests show that in some prompt cases, the model drifts toward ambient sound instead of melody; it requires highly detailed prompt description.
3. 24GB VRAM GPU recommended for full‑precision inference; low‑VRAM GPUs need quantization with audio quality degradation Reference: Official recommendation is 24GB VRAM GPU for BF16 weights. Quantized models are required for smaller VRAM at the cost of audio quality.
4. WildSongBench is a self-built dataset from the research team; no independent third-party validation Reference: WildSongBench is constructed by the M-A-P team. Benchmark results are self-reported and lack extensive independent external evaluation.
5. Lyric-vocal alignment errors may occur under complex music styles Reference: Community testing shows misaligned singing, such as rushed or delayed syllables, in fast tempo and complex arrangements.