Voice Cloning In Multiple Languages - Open Source

In this video, we will look at Bark, the state of the art Open Source Text to Speech model from Sun AI. I will show you how to set it up locally and generate audio from your text prompt. You will also learn how to CLONE voices with Bard by utilizing the Conqui TTS.
Have fun :)
#voicecloning #voice #ai
▬▬▬▬▬▬▬▬▬▬▬▬▬▬ CONNECT ▬▬▬▬▬▬▬▬▬▬▬
☕ Buy me a Coffee: ko-fi.com/promptengineering
|🔴 Support my work on Patreon: Patreon.com/PromptEngineering
🦾 Discord: / discord
▶️️ Subscribe: www.youtube.com/@engineerprom...
📧 Business Contact: [e-mail]
💼Consulting: calendly.com/engineerprompt/c...
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
LINKS:
Suno Bark: github.com/suno-ai/bark
Suno HuggingFace: huggingface.co/suno/bark
Coqui-ai: github.com/coqui-ai/TTS
Bark in Coqui: tts.readthedocs.io/en/dev/mod...
Long-Form Generation: github.com/suno-ai/bark/blob/...
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Timestamps:
Intro: [0:00]
Bark: [0:17]
Bark installation: [3:25]
Running locally: [5:10]
TTS - Examples: [7:45]
Longer Audio: [9:50]
Voice Cloning: [10:50]
Conqui AI: [11:51]
Suno Bark repo: [12:50]
Voice Cloning: [15:30]
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
All Interesting Videos:
Everything LangChain: • LangChain
Everything LLM: • Large Language Models
Everything Midjourney: • MidJourney Tutorials
AI Image Generation: • AI Image Generation Tu...

[@CS-hh1mn]

Great video! The second clone audio sounded more like the actual one. How did you train the model?

[@agenticmark]

Great work! I have been deep in voice cloning for awhile and Bark does pretty damn good when compared to tacotron2 and the like!

[@basbasmounir3943]

🎯 Key Takeaways for quick navigation: 00:35 🌐 Bark, an Open Source Text-to-Speech model from Sun AI, stands out due to its high audio quality and support for multiple languages. 00:58 🎶 Bark can generate non-speech sounds like laughs, music, and emphasis, making the generated audio more natural. 01:25 💻 You can run Bark on both GPUs and CPUs, but the generated audios are typically around 13-14 seconds. 02:20 🌍 Bark supports multiple languages and even allows mixing languages in a single audio prompt. 03:29 🛠️ To set up Bark locally, create a virtual environment, install the Bark package, and the Transformers package. 07:53 🔊 Bark can generate high-quality, natural-sounding audio that can be customized with various options. 10:08 📜 To generate longer audios, you can split the text into sentences, generate audio for each sentence, and combine them. 11:02 🗣️ While not directly available in Bark, you can clone voices using the Kukui AI package, which supports Bark integration. 14:56 🎙️ Cloning voices involves importing the Bark model, configuring speaker and text information, and generating the cloned audio. 16:30 🤖 Cloning voices with Bark is probabilistic, and the quality of the output may vary depending on the input audio quality. Made with HARPA AI

[@okachobe1]

I dont think its Kukui AI is it? I think its Conqui AI

[@bardaiart]

Thank you for making this video! I'm hoping for a true competitor to ElevenLabs -- even if it's a commercial one. Thing is, Bark is very unpredictable, even when choosing the same speaker -- making it kinda useless for lengthy content :(

[@kevinehsani3358]

Thanks for the video. Is this really doing voice cloning or rather just use different voices for tts. Coqui studio claims to clone any voice from 3 seconds of recording. Would this repository do the same. I did not hear an option where it would allow you to clone a voice first and then place it in the voices directory.

[@MasterBrain182]

This is pure gold 🔥🔥🔥

[@piclezwd]

Great video, thanks! What is a good amount of recorded voice to use to train?

[@ozzy1987mr]

tengo tiempo usando bark pero no sabia como hacer la clonación de voces, excelente video

[@engineerprompt]

Thank you 😊

[@user-pm5bc9wb7j]

Thanks you for this very valuable video. I have just 2 questions, can I save the voice ? and can I retrain a voice that i trained with other audios so that it become better. ( without the old audios )

[@equilibriointeligente]

gracias por compartir.” is: This video is very useful for me since I was looking for a free or open source AI tool to clone my voice, thank you for sharing.

[@ronaldoromerovergel8373]

pudiste clonar tu voz??

[@equilibriointeligente]

@@ronaldoromerovergel8373 si pero me salió en español con acento americano jaja.

[@ronaldoromerovergel8373]

bro me puedes ayudar, a mi no me funciona el programa jajaj@@equilibriointeligente

[@mequavis]

ive been using bark on my chan for months now, its great

[@engineerprompt]

Nice, that's great to hear.

[@moviesexplained5129]

Channel name

[@user-ub5cx4tw9x]

did you train your voice in coqui ai?

[@mequavis]

@@user-ub5cx4tw9x i remember we tried coqui, but it never worked right. We ended up using a modified version of Bark that had voice cloning hacked into it. it was very hit and miss to make a good npz file, but when you got a good one they were pretty dang good. We did a bunch of celebrities' and stuff but then decided to cool down because youtube was complaining :P

[@IO-fz2sm]

Awesome! you just earned a subscriber...

[@engineerprompt]

Thank you 😊

[@trendkillsp]

The small model takes almost 3 minutes to generate audio using Google Collab Pro. I think it is a good model, but it has to improve generation time in order to be useful. I doubt this runs almost live in GPU, as they promise in Github

[@Jwoodill2112]

it generates voice clips for me in about 15 seconds on GPU

[@ESGamingCentral]

@@Jwoodill2112what GPU?

[@Jwoodill2112]

​@@ESGamingCentral rtx 3060. it's a laptop so this one only has 6gb vram

[@ESGamingCentral]

@@Jwoodill2112 thank you. I’m trying to find good software to upscale audio files to a higher audio quality

[@mcarwa]

Very interesting project. Can it do dubbing from English to Russian? I've already trained English native speaker dataset at RVC, but when I'm trying to speak in Russian with trained model it has English accent. How to remove English accent and save voice speaker identity?

[@metaveta]

waiting for dataset video🙌

[@engineerprompt]

Recording is done, editing soon :)

[@RikusLategan]

1:12 Limitation = Maximum length of generated audios = 15 seconds; but there are ways around this

[@ilanser]

Great video, you are an amazing content creator! keep on the good job. BTW, you sound very shy (speaking about voice), try to be more clear and happy when you voice over, like the American vlogers. Good Luck!

[@revengeua]

Help please, could you share a working file for long text? I can't get it to work with the GPU, it's always an error. I have a 3070 8 gigabyte. I would be very grateful to you, thank you!

[@MichealAngeloArts]

The description is a bit confusing as it says Bard, not Bark, so I've initially thought we can do TTS with Google Bard (which was surprising!) 😃

[@engineerprompt]

Oh, fixed it. Thanks for pointing it out :)

[@Lawh]

How do I use the terminal to type down the first command? it's not working.

[@abli00346]

i like ur voice

[@guiavisualdearmas]

you are the best

[@nhexplorers]

How do you determine what version of python to use in your virtual environment? Is this specified somewhere?

[@engineerprompt]

Yes, for different projects you will usually see that it's specified. If not, 3.10 is a safe bet in most of the cases unless the project is using a very old version.

[@BilalKhan-pv7vi]

It is difficult to download the bark files one by one. i have cloned the bark github repo but the files are different in github repo as compare to the hugging face.Kindly tell us how you have downloaded the files .

[@nibo100]

there is no link to the longer audio method.

[@engineerprompt]

Sorry, just updated the description.

[@bluebeam287]

what is vocoder model in Coqui tts. how to generate that? and how to genrate pth,index file in tts

[@nirsarkar]

What about Mac? Does wav work there? Great Video though!

[@engineerprompt]

Yes.

[@93simongh]

is the voice cloning part in your script running on gpu or cpu?

[@engineerprompt]

CPU of M2 Max

[@grillodon]

When i run the .py file I have this error: "line 1, in from transformers import AutoProcessor, BarkModel ModuleNotFoundError: No module named 'transformers' Obviously, I installed the transformers module has you explained. Thx

[@mtnmecca_ej]

Thanks for the video. How do you specify using gpu in the code?

[@cgd1602]

Pytorch: Firstly you have to check if cuda is available by this line of code: device = "cuda" if torch.cuda.is_available() else "cpu" .... mostly you wil use the GPU for these lines of code: model = BarkModel.from_pretrained(model_id, torch_dtype=torch.float32).to(device) audio_array = model.generate(**inputs, do_sample=True).to(device)

[@BrillianceSpark]

can we clone our own voice using this and how long audio we can get

[@user-cn6ur5mq7j]

Do you have colab version? It took me a few hours to download everything!

[@lilianburdianov8494]

Here it is: colab.research.google.com/drive/1dWWkZzvu7L9Bunq9zvD-W02RFUXoW-Pd?usp=sharing#scrollTo=FwXohRuBAfBN

[@TheJoepanelli]

Thank you, but How can we use our voice into other languages ? using our SAME voice. Thank you

[@zoybot2254]

tbh, bark still has a lot of instability. especially in languages other than English. unfortunately it is not a reality yet

[@elvismorellidigitalvisuala6211]

Is a TTS? there's a way to use audio as input? and get another voice as ouput?

[@engineerprompt]

Yes, you will need to add another layer into it. Something like whisper that takes your audio and convert it into text. That can be combined with the code in the video

[@elvismorellidigitalvisuala6211]

@@engineerprompt thanks for your answer, but I was thinking about something different, is still a TTS output, I need to preserve intonation, cadence, musicality of the input audio, think to an audio from an actor, with emotions, a tts output remove all emotions

[@mipiaceiltubo]

@@elvismorellidigitalvisuala6211 like rask ai? i'm actually looking for same

[@virtuakamp]

Hi, I followed all the steps but I'm getting the error: line 1246, in _legacy_load magic_number = pickle_module.load(f, **pickle_load_args) _pickle.UnpicklingError: invalid load key, '

[@marcosgodinhofilho8700]

in load_model.py file, comment these lines: if ( not config.USE_SMALLER_MODELS and os.path.exists(ckpt_path) and _md5(ckpt_path) != config.REMOTE_MODEL_PATHS[model_type]["checksum"] ): logger.warning(f"found outdated {model_type} model, removing...") os.remove(ckpt_path) if not os.path.exists(ckpt_path): logger.info(f"{model_type} model not found, downloading...") _download(config.REMOTE_MODEL_PATHS[model_type]["path"], ckpt_path, config.CACHE_DIR)

[@harshmishra5331]

@@marcosgodinhofilho8700 Not helping. Same error again.

[@sebastianmoraugalde1997]

This means that your .pt files from /bark are corrupted, make sure that every single by the is downloaded as in the Hugging Face page of Bark

[@mundoartitech]

link Google clave?

[@m.rr.c.1570]

after training the model on a specific voice how can i make say something in hindi rather than in english

[@rodrigocozta]

should have win11 new pc or old win8 old cpu run it,thanks

[@qwertyntarantino1937]

Voice generation didn't work for me. I've got error `_pickle.UnpicklingError: invalid load key, '

[@harshmishra5331]

How have you solved it?

[@qwertyntarantino1937]

@@harshmishra5331 I didn’t. This is still a new technology with tons of bugs so it’s not ready for wide usage yet

[@harshmishra5331]

I am not using it for wide use .

[@harshmishra5331]

Just a small project. But am not able to get across this error

[@rsunghun]

Waiting for GUI 🤗

[@JOHNSMITH-sj3lg]

hello my friend great video. my only problem is that is has a english accent. try to clone my german voice

[@yahya_elistinsary]

Do Bark also support TTS for Arabic language?

[@engineerprompt]

I don't think so

[@vivekakaviv]

Thanks for the video! But the cloning is terrible. The code works but the cloned voice does not sound anything like the original. Anyone facing similar issue? Any solutions?

[@harshmishra5331]

which python version are you using?

[@chanansiegel834]

and you can run all this in cpu?

[@engineerprompt]

Yes

[@vivekkarumudi]

its extremely slow to the point where it doesnt make any sense to use it.. It would rather stick to my other options.

[@mr2octavio]

Description says BARD

[@engineerprompt]

fixed it :)

[@Hollywood1127]

Is BARK able to speak more than 13 seconds of text?

[@DavidBrown-tv8fx]

of course

[@gRosh08]

Uncle Amos said he is going to make a sandwich and try again later.

[@lusineharutyunyan4523]

can you do this for the Armenian language?

[@engineerprompt]

I think it only supports english.

[@billyindrajaya]

Too short audio !

[@nathanjohnson4721]

is dutch also there

[@engineerprompt]

I believe so.

[@nathanjohnson4721]

@@engineerprompt i want to do text to speech with dutch voices

[@docxonglabiet]

Language Vietnamese

[@magnus948]

Coqui ai shut down not working anymore!!

[@kitlt8476]

not any language!!!!

[@QHawk7]

*No Arabic?* :(

[@mekkicharfi5454]

Dark on dark , we see nothing !

[@henrylawson430]

Lots of scams can be generated with this…

[@c01000100]

Ugg... Cringe each time I hear visual code studio (vs visual studio code). Appreciate the content though.

[@antonpictures]

Your channel is great, your voice and accent are bad, this is perfect. Use it! My voice is Annoying too so,

[@danieletorrigiani]

there is nothing wrong with his voice or accent, you are free to not appreciating your own voice, but please be considerate when making comments on other people attributes.