NotaGen

Test Tool: ABC Notation Online Player

NotaGen is a symbolic music generation model jointly developed by the Central Conservatory of Music, Beihang University, Tsinghua University, and the ElectricAlexis team, dedicated to exploring the potential of generating high-quality classical sheet music.

Technical Features

• Three-stage training paradigm: Drawing inspiration from the success of Large Language Models (LLMs), it adopts a three-stage training paradigm. It first undergoes pre-training on 1.6 million musical pieces, then fine-tuning on approximately 9,000 classical works with "period-composer-instrumentation" prompts, and finally Reinforcement Learning using the novel CLaMP-DPO method, which requires no human annotations or pre-defined rewards.
• Multiple model scales: It provides pre-trained models of different scales, including NotaGen-small (110M parameters), NotaGen-medium (244M parameters), and NotaGen-large (516M parameters), to meet different usage requirements.
• NotaGen-X optimization: It introduces a post-training stage between pre-training and fine-tuning, refining the model with a classical-style subset of the pre-training dataset; removes the key augmentation in the fine-tune stage to make the instrument range of the generated compositions more reasonable; after RL, it utilizes the resulting checkpoint to gather a new set of post-training data, and starts another round of post-training, fine-tuning, and reinforcement learning from the pre-trained checkpoint.

Functional Advantages

• High-quality music generation: It can generate music with a classical style. Users can input "Period-Composer-Instrumentation" as a prompt to have NotaGen generate music, preview the audio/PDF scores, and download them.
• Flexible local deployment: It provides a local Gradio demo. Deploying NotaGen-X inference may require 8GB of GPU memory, and users can run and generate music locally.
• Rich extended support: It has extended support such as Google Colab notebooks and ComfyUI nodes. The ComfyUI node can automatically convert the generated.abc format to.xml,.mp3, and.png formats, making it convenient for users to listen to and view the sheet music.
• Complete training process: It provides a complete training process from data preprocessing to pre-training, fine-tuning, and reinforcement learning, as well as detailed examples. Users can use their own data for model training and optimization. NotaGen is a symbolic music generation model developed by the ElectricAlexis team, dedicated to exploring the potential of generating high-quality classical sheet music.

Technical Features

• Three-stage training paradigm: Drawing inspiration from the success of Large Language Models (LLMs), it adopts a three-stage training paradigm. It first undergoes pre-training on 1.6 million musical pieces, then fine-tuning on approximately 9,000 classical works with "period-composer-instrumentation" prompts, and finally Reinforcement Learning using the novel CLaMP-DPO method, which requires no human annotations or pre-defined rewards.
• Multiple model scales: It provides pre-trained models of different scales, including NotaGen-small (110M parameters), NotaGen-medium (244M parameters), and NotaGen-large (516M parameters), to meet different usage requirements.
• NotaGen-X optimization: It introduces a post-training stage between pre-training and fine-tuning, refining the model with a classical-style subset of the pre-training dataset; removes the key augmentation in the fine-tune stage to make the instrument range of the generated compositions more reasonable; after RL, it utilizes the resulting checkpoint to gather a new set of post-training data, and starts another round of post-training, fine-tuning, and reinforcement learning from the pre-trained checkpoint.

Functional Advantages

• High-quality music generation: It can generate music with a classical style. Users can input "Period-Composer-Instrumentation" as a prompt to have NotaGen generate music, preview the audio/PDF scores, and download them.
• Flexible local deployment: It provides a local Gradio demo. Deploying NotaGen-X inference may require 8GB of GPU memory, and users can run and generate music locally.
• Rich extended support: It has extended support such as Google Colab notebooks and ComfyUI nodes. The ComfyUI node can automatically convert the generated.abc format to.xml,.mp3, and.png formats, making it convenient for users to listen to and view the sheet music.
• Complete training process: It provides a complete training process from data preprocessing to pre-training, fine-tuning, and reinforcement learning, as well as detailed examples. Users can use their own data for model training and optimization.