Stable Diffusion FREE talking avatar! (Wav2Lip, Thin-Plate-Spline-motion model and more)...

FREE talking avatar. Learn how to create a FREE talking avatar with Stable Diffusion (Wav2Lip, Thin-Plate-Spline-motion model, Google Colab version and D-id). AI upscale your video with HitPaw Video Enhancer: bit.ly/3TmLqmp (Coupon Code: YT-30OFF). Free generate AI artwork online: bit.ly/3yLdJ4D. Now you can create a cool talking avatar for your KZbin videos or home videos.
This video is created on the Wacom Cintiq Pro 32 with Stable DIffusion, Photoshop, HitPaw Video Enhancer, RunDiffusion and Macbook Pro, Atem Mini Pro, After Effects and Adobe Premiere Pro.
-------------------------------
SUBSCRIBE
▶ handle: / @levendestreg
▶ You can subscribe to our channel here: kzbin.info...
▶ Thin-Plate-Spline-motion model: replicate.com/yoyo-nb/thin-pl...
▶ Wav2Lip: bhaasha.iiit.ac.in/lipsync/up...
▶ Thin-Plate-Spline-motion model: huggingface.co/spaces/CVPR/Im...
▶ Google Colab: colab.research.google.com/dri...
▶ Github local install: github.com/Fictiverse/Thin-Pl...
▶ RunDiffusion Promo Code: levendestreg15
▶ RunDiffusion reference: bit.ly/RunDiffusionLevendeStreg
▶ HitPaw Video Enhancer: bit.ly/3TmLqmp (Coupon Code: YT-30OFF)
-------------------------------
My setup
Macbook Pro M1 Max: amzn.to/3Znd44l
Wacom Cintiq 32 inch (couldn't find the 32 inch - so links is for 27 inch): amzn.to/40ubLSl
Atem Mini Pro: amzn.to/3nAe9s4
Stream Deck XL: amzn.to/3Zun5N9
Razer Tartarus v2: amzn.to/3TV239a
Logitech MX master: amzn.to/3TSFK43
-------------------------------
00:00:32
All you need for the two first free versions to work is an image that has 512 pixels by 512 pixels
or an aspect ratio of 1 to 1.
00:00:54
Step two is to record a video of yourself talking.
00:01:11
Avoid the PITFALL of not aligning your images with your video. The image and video need to be aligned to look good in the Thin-Plate-Spline-motion model. Crop your image or video inside Photoshop in a square format like this. Remember to name it "driving.mp4".
00:03:21
The first way to make an animated avatar - replicate.com.
00:03:51
The best alternative to the Thin-Plate-Spline-motion model is the wave2lip model.
00:04:31
So let me just show you how I upscale this video with the HitPaw Video Encoder.
00:06:56
Combine your audio and video with Adobe Premiere Pro.
00:08:17
Go to the link in the description below for the Google Colab solution for creating your talking avatar. This is also the Thin-Plate-Spline-motion model.
00:11:15
But what about D-ID? Here I compare the talking avatars with the one I create in D-ID.

[@jeremycointin1996]

Great tutorial. That chin tip is gold!

[@LevendeStreg]

Thank you kindly! I’m so glad you like it! And thanks for watching!🙌

[@zippythinginvention]

Great tutorial of my rather hit-and-miss workflow of this exact thing. Thanks for making it so concise and clear.

[@LevendeStreg]

Thank you I really appreciate it, ZippyThing Invention. So glad you liked the video🥳

[@You-and-AI]

Thank you very much for your precious suggestions!

[@LevendeStreg]

You’re very welcome - thank you watching and commenting! 🙌

[@ivandesiberia]

Es increíble como este video no tiene mas visitas,te felicito

[@LevendeStreg]

Thank you for your kind words, Ivan. I appreciate it. And yes, I am trying to gain more view and subs - so I'm pushing forward hoping more people will view my videos🙌

[@RhapsHayden]

I love you. Thanks for this! It worked great just with the my model and the donald trump video. Going to try making more!

[@876Abb]

D-ID seems very popular considering that they're charging between $5 to $49 a month for 10-15 mins for a month and even though you're paying for a subscription, they still will have a watermark on the videos as if you're using a free version.

[@LevendeStreg]

Yeah I know. The Watermark thing is a bit weird. Don’t know why they do that!

[@musumo1908]

Awesome. Thanks! Finally got a colab version running! Er do you have a link to the github used for this colab? I seem to be picking up one that looks different to your video as cant see the github source. Thanks

[@LevendeStreg]

Thank you - glad you liked the video. Once at my laptop, I’ll find the GitHub link for you!🙌

[@LevendeStreg]

Hi there again. I used this one: colab.research.google.com/drive/1DREfdpnaBhqISg0fuQlAAIwyGVn1loH_?usp=sharing#scrollTo=OzYZDRk8_ikB

[@eri7-11]

Is this something that won't be easy to do when I only have 15gb Google space and do I need to buy pro version of Google Colab to get this to work, cuz when I tried something similar, I kept getting errors, like ran out of memory (I'm on a MacBook Pro 2020, average one). Thanks.

[@LevendeStreg]

Great question. Yes you need to have about 25GB of Google Space to get it work. BUT... the code might also be broken. The code breaks every month or so - with the different updates of A1111 for instance. So that might also be why you're getting errors. When using cloud colab you're not using the working memory of your own computer, so that doesn't matter with the Mac. If you are doing LOCAL installs of SD it will give you a lot of problems. If you want to do a local install maybe try diffusionbee. Hope it works out for you!🙌

[@ivandesiberia]

Can i make streaming on twich with D-ID and this tool? How i make live video with this tool for example an call-video or take me a photo??

[@LevendeStreg]

Oh What a great question, Ivan. No you Cannon live stream with it. But I Will check out your request for a future video🙌

[@LevendeStreg]

There are some streaming solution links on this page. I’m going to check them out - but just wanted to share them with you too: democreator.wondershare.com/video-editor/best-talking-avatars.html

[@DamianWampler]

Great video! In one video did you mention a website where the platform actually pays some top creators who are making really great art?

[@LevendeStreg]

Thank you kindly. I don't recall having said that. Maybe the site www.123rf.com/ does so. But I don't use that my self. Also there are sites where you can sell your prompts to make art from (promptbase.com). I think that might be, what you heard me say🙌

[@googlefuuplayad9055]

very nice 👌 Thank you

[@LevendeStreg]

Thank you kindly. I really appreciate it 🙌

[@sam_sands]

thank you so much!!

[@LevendeStreg]

Thank you for your kind comment and thanks for watching.🙌

[@LevendeStreg]

By the way - The upscale for the Google Colab version is broken at the moment. Instead you can use HitPaw Video Enhancer: bit.ly/3TmLqmp (Coupon Code: YT-30OFF) Free generate AI artwork online: bit.ly/3yLdJ4D

[@LouisGedo]

👋

[@LevendeStreg]

@@LouisGedo hey Louis! 🙌thanks. I really appreciate it 🙏

[@jesusleguizamon6566]

Hi, this can make in like real time video generation with more gpu? for conversation, regards

[@LevendeStreg]

Hi there. No, for live conversation you need other solutions (like this one: www.chat-avatar.com/).

[@jesusleguizamon6566]

@@LevendeStreg Hi, Thanks you, is there some documentation? In link just is for chat

[@LevendeStreg]

Hmmm - actually I'm not sure@@jesusleguizamon6566 . I'll keep an eye out for it. I came across some other solutions, but I can't find the links or remember the names of them.

[@itslilsalami]

You are a badass human being

[@LevendeStreg]

Haha. Thank you kindly for your comment. 🙌

[@generalweb2957]

excelent...

[@LevendeStreg]

Thank you kindly for watching and commenting. 🙌

[@jonathanerich]

helped me a lot

[@LevendeStreg]

So glad to hear it. Thank you for watching and commenting!

[@jonathanerich]

@@LevendeStreg you're welcome

[@INGLES.Gratis]

Hey, how did you move your head up and to the sides... it was WAY BETTER than D-ID. D-ID doesn't move freely your head like a real video of your own. You showed too fast the demo in the beginning of the video! Please, could you make an updated version of this video with the free method of having that realistic look and movments WAY BETTER than D-ID?

[@LevendeStreg]

Hey there. I just did a video of myself and then used one of the free versions of "Wave2lip" in Stable diffusion. It's an extension. But just find one of the free versions on Huggingface. Hope it works for you. Cheers - Maria!

[@INGLES.Gratis]

@@LevendeStreg many thanks for returning back! i looked for a video explaining this, and really there's a gap. It would be extremely interesting if you could make a provokative video like "FORGET ABOUT TALKING PHOTOS WEBSITES - this is the next level!" and then, you show a real application with a demo to blow our minds. The video would get easy views and would help us visualize that in real applications for serious; not just 3 seconds of a quick demo :)

[@INGLES.Gratis]

P.s: this is the future of monetizing talking avatars. KZbin will crush on it soon in the future. A video of yourself with a mask on it and a new cloned voice is way more authoral (and original) than simple talking photos from D-ID and similars.

[@INGLES.Gratis]

P.s. 2: it would help people who got banned from KZbin and want a second chance playing fair; and people who doesn't want to show their faces but don't want to make ordinary faceless videos like everyone, having artistic appeal and humanity to make a strong connection to the audience like a real person. Talking photos are cool but c'mon: humans won't "buy it" after a time and won't create a real bond to the creator.

[@LevendeStreg]

Yes@@INGLES.Gratis I agree. KZbin is already crushing on it. Nerdy Rodent (KZbinr) has been doing it for months now.🙌

[@MarkHilldacreative1]

Have we ever met? Possibly in LA at a music conference Musepo?

[@LevendeStreg]

Hahaha. Nope, but nice guess. I am an old musician. But haven’t attended a music conference in LA though.

[@MarkHilldacreative1]

@@LevendeStreg you have twin roaming this world

[@LevendeStreg]

@@MarkHilldacreative1 actually it’s said that we all have a doppelgänger somewhere☺️

[@bluetreacle]

none of the links work

[@LevendeStreg]

Sorry to hear it. It's a rather old video, so I'm guessing that some of the solutions are updated now. 🙌

[@mosog8829]

But Photoshop is not free. How can you explain how to use free Ai avatar by using a paid software?

[@LevendeStreg]

Most people working professionally with AI and visuals have Photoshop in some capacity. For instance I just did a live course in AI (Midjourney and Stable Diffusion) - and 7 out of 9 participants had access to PS and Creative Cloud.

[@AaliDGr8]

but D-ID has a very bad one thing and that is we can't create our own custom facial expression like SD did hmmm ???

[@LevendeStreg]

That’s true. D-ID does not copy facial expression from your own video!

[@AaliDGr8]

@@LevendeStreg yes bb that's proof D-ID failed in front of TP and plz make a seperate video on google collap ThinPlate with its own upscale video option which down below ... u missed it in this video plz!

[@LevendeStreg]

@@AaliDGr8 thanks - the Google Colab version with upscaling is broken at the moment - the code for upscaling is broken at the moment. I mentioned that in the video😜

[@AaliDGr8]

@@LevendeStreg yes yes i heard that madam i heard ......... understand it plz that i want if plz u make another tutorial in future when collab upscale version restore that was my advance request swan lady !!

[@LevendeStreg]

@@AaliDGr8 Hahahaha... Okay got it. Not doing one at the moment. But got some good stuff coming up with Adobe Firefly and Midjourney Cheat sheet.

[@Inferencer]

I use thin-plate spline but I upscale with gfpganv1.4 I also generate my stable diffusion image based on the first frame of my video rather than manually alighning at you did kzbin.info/www/bejne/j2aVpJKenLJpac0

[@LevendeStreg]

Nice - thank you for sharing, Semi Deus (nice name by the way 😉). So with that method you have to upscale every image singularly?

[@Inferencer]

@@LevendeStreg correct, I used "ChaiNNer" repo which allowed me to just load an upscaling model & my video file without messing with compliling or extracting frames, my next step is to try frame interpolation after removing any bad frames and potentioally every second or third frame which should in theory provide smoother results, "enhancr" by mafiosnik777 which is currently considered state of the art should be releasing a free/non-patreon within 2 weeks

[@LevendeStreg]

@@Inferencer Oh nice. I'll check that out Semi Deus. Thank you for sharing that knowledge with me🙌