Animate ANY Face From A SINGLE IMAGE (Thin Plate Spline Motion Model)

Head to squarespace.com/cgmatter to save 10% off your first purchase of a website or domain using code CGMATTER
google collab colab.research.google.com/dri...
patreon and stuff / cg_matter
my website www.cgmatter.com
#ai #walterwhite #breakingnews

[@sasodoma]

I was very confused why I was getting notifications for a random yt channel I had never seen before. Then finally you included your face on a thumbnail.

[@ads_mello]

Same dude!!!!!!

[@kleber1983]

the fact that you narrate this hole video using the technique that you are teaching us is just amazing...lol

[@douchymcdouche169]

It's very holesome.

[@smeggers]

Phenomenal job, Mr.White!

[@yajastudio8241]

Thanks so much Walter!

[@HoldenArt]

OK, I'm only a couple of minutes in and I can tell that the speech pattern, choice of words, pauses etc is very telling. "Either way", "I find it works best this way", "again, you can...", "we're good to go...", "in other words...", "before we begin"

[@seriouslycoolful]

"push this buh-tin"

[@nullvoid3545]

I wasn't expecting it to fix something like choice of words, but I was expecting it to recognize his lisp, manage to interpret what he was trying to say like we can, and replace it appropriately. One can conceivably compensate for their choice of words with A script, but an integral part of how someone pronounces syllables that not only cant be changed, but wouldn't be properly mapped if someone were to for example: use cgmatters voice to do this. You would have to compensate for his lisp when impersonating him to make the voice seem convincing.

[@douchymcdouche169]

@@nullvoid3545 I don't think you quite understand what "lisp" means.

[@nullvoid3545]

@@douchymcdouche169 Your right, after some research I have discovered I was incorrectly using the term lisp as A shortened way of saying speech impediment when the word I should have been using for this specific one is rhotacism. Thanks for calling me out, I learned something new today.👍

[@MrMargaretScratcher]

@@nullvoid3545 *wotacism

[@bubanga]

Thanks for sharing this, Walter!

[@JeanEdouardKevin]

Wow that's crazy stuff, i'm interested in the full body

[@darksydeflow]

Hey how would I go about changing the resolution of the outputted video? I'm trying to get a higher quality looking video, similar in resolution to the original photo.

[@taoluanti7816]

Thank you so much for sharing this video😊😊

[@RhapsHayden]

Oh my god. I'm alive thank you :) I tried the webui thin plate but had no way of upscaling the resolution. This works great!

[@humildao5450]

omg, thank you, just thank you for teaching all this incredible knowledge

[@MadLabZ]

Thanks for the video, works like a charm :)

[@Hyperboid]

thank you walter matter very cool

[@methadonmanfred2787]

this is a huge step up form first order model

[@don-dextv4383]

if you haven't run all of the green ticks in the correct order will it matter? And if so is there a way to refresh the process? Thanks

[@fayyazb]

Super! So you did this whole YT Video using this same method? And used Voice cloning too?

[@aleksandersanya1817]

You're goddamn right!

[@Awokenmystic]

this is insane man

[@sanjayptlsp22]

Nicely explained. I would like u to make a video on full body to......

[@vijaykanhai9775]

any way to do this locally on m1 and do a batch process of this ?

[@bestmeme8902]

yo can you do a meth tutorial next pleaseeeeeee

[@violentpixelation5486]

🤣

[@sofuckinstylish4313]

hello i need help i keep getting the runtime error: Sizes of tensors must match except in dimension 1. Expected size 32 but got size 33 for tensor number 1 in the list.

[@user-yc8ci2gh7z]

Top notch love the use of Bryan Cranston from breaking bad. So cool!

[@DLlama]

Oh wow! I can think of all kinds of fun ways to use this!! 😈

[@zachluker8708]

Its not woking for me, even when I don't even change the source files. I keep getting the error [Errno 2] No such file or directory: 'checkpoints/vox.pth.tar' Anyone know how to resolve this?

[@Melisa-vo9sp]

hey, I got same error, did you solve it?

[@christopherwilliams242]

Same :(

[@LHMATIAS]

I would like to do something similar to what exists in the iphone. Where should I go? is there anything already built for it? Thank you for the content

[@lehaison1236]

Thank you very much, may I ask what software you are using to record live chat water white videos?

[@stephanedavi]

Hi, thanks a lot for your video. Do you know if it's possible to change the frame rate at 25fps and the image size?

[@BeatSlayer]

In the final result, the mouth is barely opening at all. Yet I've shot a pretty clear video in which you can see all facial details. Any idea why?

[@pkay3399]

Thank you. I'm getting weird artifacts around the mouth similar to another app. Not sure how to fix it.

[@swalihstudios]

It's dmn awesome.

[@colibrian]

Thank you for tutorial! How do I get a higher resolution video?

[@user-eu4hx6ze3p]

When I reach the part of "Try" it gives a error

[@ve7da94]

i tried with diffrent images and videos and it didn't work pass the step with the two in a video.

[@bfrankob]

Can you please make full body one please 🙏. Thank you so so much

[@MegaChris456]

I learned this so much faster knowing Walter White is my teacher. Thanks!

[@-.......-]

Can you do it in blender?

[@jtjames79]

Do it with an anime face.

[@TheRealJerseyJoe]

Nice !

[@yo80090]

after getting your avatar video how do you make it to talk and mimic all your expressions in real time?

[@crimester]

before we begin tho (chunk of meat in superposition appears to the right of heisenburger)

[@iseahosbourne9064]

what are you using for the voice?

[@sandeepsingh-xg1rk]

I am thinking to apply this technic to a photo of 3d model then i dont need to do facial animation

[@adekolaopeyemi7874]

Please do realse the one with full body . Thanks

[@gemini9775]

amazing wow ^^ how did you get walter white voice ?

[@zirufe]

Hi. I am also interested how your dubbing voice also turn into Mr. White's Voice. Is it also AI or just voice filter in any application?

[@XTripleXCrossX]

How did you do the voice too?

[@DamianWampler]

I was able to generate a video but how do I then download it? I refreshed the screen and it vanished.

[@CTLN7]

How do you increase the resolution?

[@z33r0now3]

How to do the voice is coming up as the next vid I hope? Gonna use this for my mums birthday, doing it with her favorite artist, wishing her a nice birthday.

[@helphelphelphelpmehelphelphelp]

It's here! kzbin.info/www/bejne/aarNc5KGaLqpnKc

[@z33r0now3]

@@helphelphelphelpmehelphelphelp thx mate

[@goatpepperherbaltea7895]

The really trippy part about is is when the eyes look down for a second it’s crazy😂

[@JJ-yw3ug]

It's so strange. I followed the exact same steps in the video. At the start, I put in roughly 4K images and 4K videos, and after clicking on the "try" column, the output image still comes out as Jackie Chan (I also changed the names of 'driving' and 'source' files in the folder). I thought maybe the image and video size were too large, so I adjusted them to 480X720 resolution. However, even after doing this, once I hit generate, it still comes out as Jackie Chan. I checked the code, all the paths are correct, but it still generates Jackie Chan...

[@hblnutritionskinvietnamlag6408]

Got u, do exactly the same the author did, u might by passed things. I got the same result like yours, then I checked it, did step by step, then got the result I wanted

[@drawnimo]

"bryan cranston here, representing Squarespace..." hmm... already seeing how this tech could be controversial! neat!

[@dany-tu1qg]

is it possible to do that using a faceless image , for example spider man or dead pool ?

[@AaliDGr8]

plz tutorial for ted full body animation things also needed more then that.................

[@yadav-r]

Will this work, if I have a full body shot, my pose will remain static, but the face animate.

[@victoreze3362]

amazing what you i would like to learn a lot from you sir with respects

[@vivektyagi6848]

Can a 3D FBX Model of the Animated Final Video be generated too ?

[@blissfulbreeze5966]

So you can do live streams with this?

[@zestaahbad5464]

the mp4 and picture i imported not changing when i click on try its still showing the old one

[@Briskybroski77]

for some reasons I can't get one of the steps to run...It always says "demo" not found. Can you help?

[@DaniiJardim]

Can anyone tell me where I can find the moving face video to download?

[@devaccount661]

Really cool video and I'm interested to try it out, but I'm a bit nervous about using this model for safety and privacy reasons. What do you guys think?

[@ClarksvilleCares]

I don’t know code so I’m spooked to run it too

[@abiyyupanggalih854]

I just find gold here

[@Tellitasitis72Joe]

Hi and Thanks.. Can we run this offline?

[@ahmesrhindeng]

Is really handsome Bryan Carton.

[@janholecek2692]

Hey, I wasnt able to change the resolution. How do you do that? When i change it i get error in the last code block

[@miracoolgabrell]

Can it be used for live video?

[@michaelk.hinrichs9224]

das ist ja Hammerhart

[@rucksackcookbook5657]

How exactly do I change the resolution?

[@lehaison1236]

What does the video resolution depend on? My video is blurry and not as clear as yours

[@ericfoster5267]

S-tier content

[@briandives7271]

what about the full body version??

[@tanyav2077]

give a link if you have a more detailed analysis, even if it is paid, thank you!

[@beyondthepeel_]

Did this break for anyone else today? Was using it yesterday fine but now I get errors at: from demo import... And can't move beyond it

[@ginfreecz]

but how did you make it as your avatar? and using a voice changer

[@optimus6858]

how can i run it locally on pc crashes every time due to low ram

[@____-gy5mq]

"Cgmatter? Uh aah I'm the Walter"

[@kelvinjorge-se4ky]

Can we use this for video call

[@juaninbits]

I get this error, does someone knows how to fix it? FileNotFoundError: [Errno 2] No such file or directory: 'checkpoints/vox.pth.tar' It happens on step 6 (5:28)

[@sam_sands]

me too! if you found a solution please let me know

[@Melisa-vo9sp]

@@sam_sands hey i got same error, did you solve it?

[@brendankeogh4669]

Does this work for a non human face? For example an image of a Pixar style cartoon bear face 🐻? or is it only able to detect “ human “ faces?

[@nickexel2425]

I also have this question....

[@degibocu]

ohhh mannnnn

[@user-qv1is6yp4z]

Very interesting, even superbly interesting, but some added info should be provided. Ex: Careful about mp4 size or a Ram crash will certainly ensue. The code editor is the worst, it doesn't even have a right click paste function. I also notice that the end result provide an output of the 3 assets instead of what everyone is looking for: the animated face. Has anyone been able to output the single animated face? ;)

[@gonzaloquispefernandez8303]

Hello bro, the animated face video is inside the "Thin-Plate-Spline-Motion-Model" folder, and it's name is genearted.mp4.

[@TheShazExperience]

Wouldn’t DeepFaceLab be a much simpler software to use than this one?

[@relaxchill888]

how make it for online streem?

[@dougiejones628]

Is there a way to download the Git and run it locally via Pinokio?

[@Awokenmystic]

you are good

[@KieranShort]

What's going on in the output when you open your mouth?

[@frenchie_leguminati]

It's cool but 256x256 pixel limitation + looks like I can't do more than 5seconds, I don't know why. Too bad :)

[@danalex5458]

How to do that? Where are programs? You give only 10% of info, how to do it live? In real-time?

[@MichaelWarfCA]

I’ve been messing around with this technique but I gave up due to what I thought was the 256x256 pixel limitation for output. Have you had success with higher resolution output? If so I’d love to give it another go with higher resolution assets.❤😊

[@ChrisEldridgeKino]

I export the video to png sequence and upscale those using GFPGAN, then re-render the video.

[@belenparedes2085]

@@ChrisEldridgeKino hey, can you explain me how to do that? If I export the video to png sequence then I have like 1000 pics

[@ChrisEldridgeKino]

@@belenparedes2085 it will be a lot of images. Just look up how to use gfpgan or even a tool like upscayl to process all of the images in a directory. Then convert those 1000 images back into a video with ffmpeg

[@mrnekoartcraft]

It would be nice if you mentioned that this method won't work if you don't have the NVIDIA driver. Cheers!

[@ohschneider]

This looks interesting, I send a photo and a video and what about the voice issue, does the video need to contain the right speech?

[@user-rq6jx9uv6f]

simply u r 👍

[@Ogg30631]

will it work on an amd gpu?

[@j5545]

Can I do it locally?

[@pile333]

It's weird when your mouth is open wide though. Sort of Alien Xenomorph effect. 😄

[@krivadnaaiservices]

Spline available on mobile

[@wearegeeks]

Yeaaaah! science beeeech!

[@cameronlazecko5040]

Does not work. Colab will not run the checkpoint code shown at the 5:28 mark in the KZbin video. Tried about 25 times.