MOSS-Transcribe-Diarize

MOSS-Transcribe-Diarize is an open-source, next-generation audio understanding large model developed by the OpenMOSS Team at Fudan University.

Core Features & Highlights

• All-in-One Transcription & Speaker Diarization: Upload a multi-speaker audio file, and it simultaneously converts speech to text while automatically identifying who is speaking (labeling speakers as [S01], [S02], etc.).
• Precise Timestamps: Automatically tags the start and end times for every sentence, making it easy to export directly into standard subtitle files.
• Acoustic Event Recognition: Beyond speech recognition, it detects background environmental sounds such as laughter, applause, and coughing.
• Long-Form & Multilingual Support: Processes up to 90 minutes of continuous audio in a single pass across 50+ languages.

Ideal Use Cases

• Meeting Minutes: Automatically processes multi-party discussions and generates transcripts with clear speaker attributions.
• Interviews & Podcasts: Converts dialogues and interviews into well-structured, speaker-labeled transcripts.
• Video Subtitle Generation: Automatically creates precise, timestamped subtitle files for long-form video content.
• Lectures & Classrooms: Efficiently transcribes long educational recordings into text notes.

Underlying Technology

• End-to-End Speech LLM Architecture: Unlike traditional pipelines that chain separate ASR (speech recognition) and Diarization models together, this project utilizes a compact 0.9B parameter end-to-end multimodal speech LLM. It generates text, speaker labels, and timestamps simultaneously in a single forward pass.
• Lightweight & High Efficiency: With only 0.9B parameters, it achieves state-of-the-art accuracy while maintaining low GPU memory usage, making it ideal for local and lightweight deployment.