AudioX-Turbo

AudioX-Turbo is a unified multimodal audio generation framework developed by the NoizAI team at the Hong Kong University of Science and Technology (HKUST). As an optimized successor to AudioX, this project focuses on delivering ultra-fast text-to-audio, text-to-music, and video-to-audio generation.

⚖️ Licensing & Restrictions (Most Important)

The model is released under the CC-BY-NC 4.0 license and contains embedded watermarks. It is strictly restricted to non-commercial use only—this is the core requirement users must comply with.

🎯 Core Features & Use Cases

AudioX-Turbo supports the following generation tasks:

• Text-to-Audio (T2A): Convert text descriptions like "typing on a keyboard" into sound effects
• Text-to-Music (T2M): Generate music from text prompts like "music with piano and violin"
• Video-to-Audio (V2A): Synthesize matching ambient sound for video content
• Video-to-Music (V2M): Create background music synchronized with video
• Hybrid-Condition (TV2A/TV2M): Generate audio using both video and text cues

Applicable scenarios include: short-form video creation, game audio design, film post-production, podcast production, and more.

🔬 Technical Architecture

• Foundation: Built on Multimodal Diffusion Transformer architecture
• Key Innovations:
  • Multimodal Adaptive Fusion module aligns diverse input modalities
  • Distribution Matching Distillation achieves efficient 4-step generation
  • Integrated GAN discriminator ensures output quality
• Training Data: IF-caps-Pro dataset with approximately 9.2M high-quality samples
• Generation Speed: Requires only 4 diffusion steps—approximately 25× faster than traditional multi-step approaches

✅ Key Advantages

1. Exceptional Speed: 4-step generation with minimal inference cost
2. Unified Multimodal Framework: Flexibly combine text, video, and audio conditions
3. Superior Quality: Outperforms baselines on Inception Score and other metrics
4. Strong Instruction Following: Excellent comprehension of user prompts
5. Open Source: Includes Gradio demo and Python API

❌ Key Limitations

1. Audio Length Constraint: Currently supports maximum 10 seconds of audio generation—insufficient for longer content
2. Music Generation Degradation: When users request music generation, the model sometimes weakens musical elements while emphasizing environmental sounds, resulting in misaligned outputs
3. Watermark Embedded: Models contain watermarks that may restrict commercial applications
4. Non-Commercial Restriction: Strict CC-BY-NC licensing allows only research and personal use