What it is
OpenVoice is an instant voice cloning system developed by researchers at MIT, Tsinghua University and MyShell. It clones a reference speaker's tone color and generates speech in multiple languages and accents. V2, released in April 2024, adds better audio quality and native multilingual support. It has powered the voice cloning feature on myshell.ai since May 2023.
Who it's for
- Developers who need voice cloning in their applications
- Researchers working on voice cloning and cross-lingual speech generation
- Teams that need a commercially usable voice cloning model under the MIT License
Pros & cons
Pros
- Pro:Accurate tone color cloning that reproduces the reference voice in multiple languages and accents
- Pro:Granular style control over emotion, accent, rhythm, pauses and intonation
- Pro:Zero-shot cross-lingual cloning: neither the generated language nor the reference language has to appear in the training dataset
- Pro:MIT licensed for both V1 and V2, free for commercial and research use
Cons
- Con:The README has no install or setup commands; it only points to docs/USAGE.md for instructions
- Con:Native language support in V2 is limited to English, Spanish, French, Chinese, Japanese and Korean
Images
Related
Repo RVC-Boss/GPT-SoVITS
1 min voice data can also be used to train a good TTS model! (few shot voice cloning)
Repo index-tts/index-tts
An Industrial-Level Controllable and Efficient Zero-Shot Text-To-Speech System
Repo babysor/MockingBird
🚀Clone a voice in 5 seconds to generate arbitrary speech in real-time
Repo nari-labs/dia
A TTS model capable of generating ultra-realistic dialogue in one pass.
Repo debpalash/VoiceStudio
VoiceStudio is the open-source, fully-local ElevenLabs alternative — voice cloning, voice design, video dubbing, dictation, transcription & audiobook creation in 646 languages.
Repo coqui-ai/TTS
🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production