DEEPFAKE Tutorial: Beginner guide: cloning voice and lip-synching in video (tortoise TTS & wav2lip)

Hi, I am Alex Borg and in this tutorial, I will guide you step by step on how to generate a deepfake voice using Tortoise TTS and Elon Musk's voice, and how to synchronize it to a video spoken by Elon Musk.
I am a virtual KZbin AI and my body and voice are virtual.
We will look at how to create a deepfake by changing the words spoken by a person in a video. For example, let's use Elon Musk as a celebrity. Here is the text for Elon Musk to read: "I had said that the Doge cryptocurrency was cool, but I just changed my mind, I'm going to sell everything and instead buy Elrond Gold (EGLD), it's a very promising cryptocurrency with low transaction fees, just like Ethereum."
Check out the direct result video here: • Elon Musk's Investment...
Follow me on:
Twitter: / alexborg0101
Facebook: / alexborg0101
Tiktok: / alexborg0101
For cloning and generating voice, we use Tortoise TTS.
Source code and explanations can be found here :
link : github.com/neo...
To use Google research Colab directly online, you can use this :
link : colab.research...
For generating videos with lip-synching, we use wav2lip.
My Wav2lip +folder GUI modified by Romain Baker / full ready with "start.bat" to launch app under windows 10 64 bits (tested with RTX 2060 ok, but not ok with RTX 3060) : surl.li/cylbs
link : github.com/Rud...
You have to add a model in model folder, here is the
link : iiitaphyd-my.s...
- - - - - - - - - - - - - - - - - -
The solutions I show in my video are forks of tortoise TTS and wav2lip.
If you are interested for a link of these modified versions, working as stand alone without any other installation : here are the direct links below :
Tortoise TTS : My folder full ready with "start.bat" to launch app under windows 10 64 bits : surl.li/cylaf
Wav2lip + GUI modified by Romain Baker / full ready folder with "start.bat" to launch app under windows 10 64 bits (tested with RTX 2060 ok, but not ok with RTX 3060) :
surl.li/cylbs

[@alexborg0101]

my twitter : twitter.com/alexBorg0101 my facebook : facebook.com/AlexBorg0101 my tiktok : www.tiktok.com/@alexborg0101 For cloning and generating voice, we use Tortoise TTS. Source code and explanations can be found here : github.com/neonbjb/tortoise-tts To use Google research Colab directly online, you can use this : colab.research.google.com/drive/1wVVqUPqwiDBUVeWWOUNglpGhU3hg_cbR?usp=sharing For generating videos with lip-synching, we use wav2lip. github.com/Rudrabha/Wav2Lip The solutions I show in my video are forks of tortoise TTS and wav2lip. If you are interested for a link of these modified versions, working as stand alone without any other installation : here are the direct links below : -------------------------------------------------- Tortoise TTS : My folder full ready with "start.bat" to launch app under windows 10 64 bits : surl.li/cylaf Tutorial if you have troubles to use it after installation on your computer : -install Cuda 11.3 (2,7 gb) ! Here is the link you have to download (Windows -> x86_64 -> V10 -> .exe (local) developer.nvidia.com/cuda-11.3.0-download-archive?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exe_local - run "start.bat", wait for automatic download of models, - write your text (enter), choose voice "halle" (enter), enter 2 (to get 2 versions of result) Wav Files could then be found in subfolder : "C:\conda3 esults\longform\halle" (halle is the name of the choosed voice for me) PS : when you choose another voice after entering your text, voices names are the folders into "C:\conda3\tortoise\voices" -------------------------------------------------- Wav2lip + GUI modified by Romain Baker / full ready folder with "start.bat" to launch app under windows 10 64 bits (tested with RTX 2060 ok, but not ok with RTX 3060) : surl.li/cylbs

[@WhimKR]

"I won't dwell on this solution, which may put most of you off." Literally the only reason I clicked this video.

[@SpicedFuture]

Really awesome to see how far the deepfake is going on. I am sure they will master it one day that you cant see a difference of real or fake. Great video!

[@alexborg0101]

Thank you very much !!

[@SpicedFuture]

@@alexborg0101 Hello Alex, i try now since many hours to get your offline TTS running. I had success over collab and then i had no volume left 🙂 Then i started to install Anaconda. made a env and copied your download able file in. It starts, i can input the text, have to choose random bec. names it dont find and gives me errors, and differnt versions i type in 1. Then it makes a process without errors, but i cant find any wav or mp3 fine. Do you maybe have a tutorial how to use your TTS file in the right way? I love to work with it on collab the complete night, but now i cant do it anymore. I have a 3080Ti, so it should work i think. Do you maybe have discord?

[@alexborg0101]

@@SpicedFuture Hi, I have a new laptop with a RTX 3080. As I didn't tried yet Tortoise TTS to work on it, let me already copy my downloaded link version its drive c:\conda3 and see if I get errors or problemes to run it. I will let you know 😊

[@tamgaming9861]

@@alexborg0101 great - thank you very much 🙂

[@SpicedFuture]

@@alexborg0101 Awesome. Played so long and it was very funny. But now i cant get it to run... it sad 🙂

[@Ilovepapayas]

Hi! Thank you for that great Tutorial! And thank you for providing that downloadable Folder.

[@aleph2d]

That was insane, and thoroughly entertaining. Thank you.

[@alexborg0101]

Thank you very much for your very cool comment ! I will try to make another amazing entertaining soon ! Don't miss it ! :)

[@lazerusmfh]

What an awesome program I installed it in anaconda on windowd and it’s FANTASTIC! :)

[@lBioHaZarDl]

Merci, thank you Alex Borg! Is there any way to make it sound more realistic by any chance? For example, if you train it with more .wav files? What do you think Subbed btw

[@alexborg0101]

Hi You can alter many parameters if you use api . You can even change quality from low to ultra so you should have again chance to improve render à little more I think

[@ekkamailax]

the audio is so out of sync with the lip movements in this entire video

[@alexborg0101]

Really? Do you have the same feeling with my more recent videos?) I tried to improve this sync in recent ones. And next will be even better

[@yvankoabiloa9490]

Je suis un intéressé par un lien vers ces versions modifiées, fonctionnant en stand alone sans aucune autre installation, stp ? J’aimerai bien voir comment tu les as utilisé. Je suis bloqué sur un point

[@johannex.]

Hello, cool video and powerful program. But is it possible to create your own voice package so that it takes fractions of a second to play text-to-speech, as it happens in pyttsx3 ?

[@misstatazen]

Verry cool job

[@alexborg0101]

Thanks Miss Tata Zen, you must be a very cool person !!

[@mycloudvip]

Nice WORK! Keep it up!

[@Ecoute_AI]

I want to train voice model in hindi language, how to do this, please help....

[@prinks1993]

Very good question if you get the "yes" please tell me too bhai

[@fabianschierz]

How did you generate the human avatar for this video? I’m currently using Synthesia but it’s quite expensive. Wondering if there’s a better tool.

[@alexborg0101]

Search and download a "chroma key woman speaking" video, then, generate audio vocal with tortoise TTS, and import all in Wav2lip. Finally, to change face, use deepfacelive and export new video into your video editor and change green background by what you want ;)

[@fabianschierz]

@@alexborg0101 Thanks for the detailed answer. I'll take note of it for future reference.

[@stevecommand77]

Got this error.. nvrtc: error: invalid value for --gpu-architecture (-arch)

[@chasepoundher336]

Any luck on this? I tried re-extracting a fresh copy and still got that error too. It was working fine yesterday...

[@pavlynavr393]

Thank you Alex for these explanations, I will be able to pass myself off as the President of the Republic! Haha ✨✨🔥

[@alexborg0101]

Yes of course, you can have a try ! lol

[@gaborjuhasz2260]

I always get this error: ['utf-8' codec can't decode byte 0xa0 in position 20: invalid start byte] when I try wav2lip Could you please give me some advice?

[@hengkyyudhiwijaya3402]

Please provide minimum instructions on the specifications of the laptop that can be used to run this tool

[@hengkyyudhiwijaya3402]

very useful.. is there a faster voice clone process? this is good but too slow

[@alexborg0101]

you can lower quality in python file. Or you can changer your gpu. For example, just changing from RTX 2060 to 3080 is X10 faster for me

[@hellasenpai]

How can I make it read from a .txt file instead than putting my text?

[@alexborg0101]

Hi Try this : surl.li/dmydl Put it in main folder of installation. Look at text file named : text_to_read.txt and start bat script by launching : start_read_txt_file.bat It will execute a file that I created in tortoise folder, named read_file.py. Instead of asking text, it ask you relative path to txt file to read. Simply enter : text_to_read.txt if you want to try my text file in the root folder. Let me know if it works. Be careful with returns to lines. I am not sure that it works very well.

[@kylergeston]

@@alexborg0101 What would I have to do so I can use this method with the Colab ? Can you explain please ?

[@adexkadex3316]

I just download the two link just now but did not understand which app to open both the link and is there anyone that can put me through it cause no matter how hard I try I still couldn’t do it even after watching the tutorial video over and over again

[@beyondmax1]

Let's link up

[@adexkadex3316]

@@beyondmax1 okay But how dear

[@atruebiblicalchurch]

I keep getting this when I run the sart.bat. Traceback (most recent call last): File "tortoise/read_ask.py", line 30, in tts = TextToSpeech(models_dir=args.model_dir) File "C:\Users\willw\Desktop\VoiceClone\tortoiseTTS\conda3\tortoise\api.py", line 246, in __init__ self.vocoder.load_state_dict(torch.load(get_model_path('vocoder.pth', models_dir), map_location=torch.device('cpu'))['model_g']) File "C:\Users\willw\Desktop\VoiceClone\tortoiseTTS\conda3\lib\site-packages\torch\serialization.py", line 705, in load with _open_zipfile_reader(opened_file) as opened_zipfile: File "C:\Users\willw\Desktop\VoiceClone\tortoiseTTS\conda3\lib\site-packages\torch\serialization.py", line 242, in __init__ super(_open_zipfile_reader, self).__init__(torch._C.PyTorchFileReader(name_or_buffer)) RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory

[@alexborg0101]

Hi William, Not sure if that could help, but did you already tried to move conda3 folder and subfolders to c:\conda3 ?

[@michaelbishop813]

Appeared to have a error come up with the pre setup version in the download link when running the voice generator. failed to open nvrtc-builtins64_113.dll. not sure how to resolve this?

[@alexborg0101]

Hi Michael, did you already installed Nvidia Cuda on your computer ?

[@alexborg0101]

install this and it will work : developer.nvidia.com/cuda-11.3.0-download-archive?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exe_local

[@sfonetwo]

Your links are not working

[@alexborg0101]

You have to copy link and remove space between http and : KZbin has forbidden me to add links because my Channel is New.

[@alexborg0101]

The solutions I show in my video are forks of tortoise TTS and wav2lip. If you are interested for a link of these modified versions, working as stand alone without any other installation : here are the direct links below : Tortoise TTS : My folder full ready with "start.bat" to launch app under windows 10 64 bits : surl.li/cylaf Wav2lip + GUI modified by Romain Baker / full ready folder with "start.bat" to launch app under windows 10 64 bits (tested with RTX 2060 ok, but not ok with RTX 3060) : surl.li/cylbs

[@jasonwhite3178]

the links don't work put the links in the comments for everyone can use them, the wav2lipGUI don't work the one i founded on the internet. had it been updated or something where is the stand along wav2lip GUI

[@alexborg0101]

Let me find a little time (before this week end) to pack all my files in one archive and upload them somewhere. I will update links in the my first comment in this video later. Stay connected ;)

[@alexborg0101]

The solutions I show in my video are forks of tortoise TTS and wav2lip. If you are interested for a link of these modified versions, working as stand alone without any other installation : here are the direct links below : Tortoise TTS : My folder full ready with "start.bat" to launch app under windows 10 64 bits : surl.li/cylaf Wav2lip + GUI modified by Romain Baker / full ready folder with "start.bat" to launch app under windows 10 64 bits (tested with RTX 2060 ok, but not ok with RTX 3060) : surl.li/cylbs

[@joaquinmellamo6669]

hello,cloning voice spanish? thank

[@alexborg0101]

Hello Actually only english is possible For multi langage , try coqui Tts .

[@James-ip1tc]

Do you know what the command is to train a voice. It's a utility but it's not specific

[@alexborg0101]

you don't need to train, because existing model uses samples from wav source you provide as input for reference.