Animate ANY Face From A SINGLE IMAGE (Thin Plate Spline Motion Model)

Head to squarespace.com/cgmatter to save 10% off your first purchase of a website or domain using code CGMATTER
google collab colab.research.google.com/dri...
patreon and stuff / cg_matter
my website www.cgmatter.com
#ai #walterwhite #breakingnews

[@sasodoma]

I was very confused why I was getting notifications for a random yt channel I had never seen before. Then finally you included your face on a thumbnail.

[@ads_mello]

Same dude!!!!!!

[@kleber1983]

the fact that you narrate this hole video using the technique that you are teaching us is just amazing...lol

[@douchymcdouche169]

It's very holesome.

[@smeggers]

Phenomenal job, Mr.White!

[@HoldenArt]

OK, I'm only a couple of minutes in and I can tell that the speech pattern, choice of words, pauses etc is very telling. "Either way", "I find it works best this way", "again, you can...", "we're good to go...", "in other words...", "before we begin"

[@seriouslycoolful]

"push this buh-tin"

[@nullvoid3545]

I wasn't expecting it to fix something like choice of words, but I was expecting it to recognize his lisp, manage to interpret what he was trying to say like we can, and replace it appropriately. One can conceivably compensate for their choice of words with A script, but an integral part of how someone pronounces syllables that not only cant be changed, but wouldn't be properly mapped if someone were to for example: use cgmatters voice to do this. You would have to compensate for his lisp when impersonating him to make the voice seem convincing.

[@douchymcdouche169]

@@nullvoid3545 I don't think you quite understand what "lisp" means.

[@nullvoid3545]

@@douchymcdouche169 Your right, after some research I have discovered I was incorrectly using the term lisp as A shortened way of saying speech impediment when the word I should have been using for this specific one is rhotacism. Thanks for calling me out, I learned something new today.👍

[@MrMargaretScratcher]

@@nullvoid3545 *wotacism

[@zachluker8708]

Its not woking for me, even when I don't even change the source files. I keep getting the error [Errno 2] No such file or directory: 'checkpoints/vox.pth.tar' Anyone know how to resolve this?

[@Melisa-vo9sp]

hey, I got same error, did you solve it?

[@christopherwilliams242]

Same :(

[@don-dextv4383]

if you haven't run all of the green ticks in the correct order will it matter? And if so is there a way to refresh the process? Thanks

[@user-eu4hx6ze3p]

When I reach the part of "Try" it gives a error

[@vijaykanhai9775]

any way to do this locally on m1 and do a batch process of this ?

[@darksydeflow]

Hey how would I go about changing the resolution of the outputted video? I'm trying to get a higher quality looking video, similar in resolution to the original photo.

[@fayyazb]

Super! So you did this whole YT Video using this same method? And used Voice cloning too?

[@yajastudio8241]

Thanks so much Walter!

[@BeatSlayer]

In the final result, the mouth is barely opening at all. Yet I've shot a pretty clear video in which you can see all facial details. Any idea why?

[@humildao5450]

omg, thank you, just thank you for teaching all this incredible knowledge

[@pkay3399]

Thank you. I'm getting weird artifacts around the mouth similar to another app. Not sure how to fix it.

[@RhapsHayden]

Oh my god. I'm alive thank you :) I tried the webui thin plate but had no way of upscaling the resolution. This works great!

[@bubanga]

Thanks for sharing this, Walter!

[@sofuckinstylish4313]

hello i need help i keep getting the runtime error: Sizes of tensors must match except in dimension 1. Expected size 32 but got size 33 for tensor number 1 in the list.

[@yo80090]

after getting your avatar video how do you make it to talk and mimic all your expressions in real time?

[@sandeepsingh-xg1rk]

I am thinking to apply this technic to a photo of 3d model then i dont need to do facial animation

[@JeanEdouardKevin]

Wow that's crazy stuff, i'm interested in the full body

[@lehaison1236]

Thank you very much, may I ask what software you are using to record live chat water white videos?

[@LHMATIAS]

I would like to do something similar to what exists in the iphone. Where should I go? is there anything already built for it? Thank you for the content

[@colibrian]

Thank you for tutorial! How do I get a higher resolution video?

[@Hyperboid]

thank you walter matter very cool

[@MadLabZ]

Thanks for the video, works like a charm :)

[@zestaahbad5464]

the mp4 and picture i imported not changing when i click on try its still showing the old one

[@southmoltondancecentre1401]

This was working fine for a while but now no longer works. When running the "from demo import load_checkpoints" module, I get the folowing error message: FileNotFoundError: [Errno 2] No such file or directory: 'checkpoints/vox.pth.tar' even though the file seems to be in the checkpoint directory

[@Melisa-vo9sp]

hey, i got same error. could you solve it?

[@stephanedavi]

Hi, thanks a lot for your video. Do you know if it's possible to change the frame rate at 25fps and the image size?

[@JJ-yw3ug]

It's so strange. I followed the exact same steps in the video. At the start, I put in roughly 4K images and 4K videos, and after clicking on the "try" column, the output image still comes out as Jackie Chan (I also changed the names of 'driving' and 'source' files in the folder). I thought maybe the image and video size were too large, so I adjusted them to 480X720 resolution. However, even after doing this, once I hit generate, it still comes out as Jackie Chan. I checked the code, all the paths are correct, but it still generates Jackie Chan...

[@hblnutritionskinvietnamlag6408]

Got u, do exactly the same the author did, u might by passed things. I got the same result like yours, then I checked it, did step by step, then got the result I wanted

[@XTripleXCrossX]

How did you do the voice too?

[@user-yc8ci2gh7z]

Top notch love the use of Bryan Cranston from breaking bad. So cool!

[@iseahosbourne9064]

what are you using for the voice?

[@DamianWampler]

I was able to generate a video but how do I then download it? I refreshed the screen and it vanished.

[@beyondthepeel_]

Did this break for anyone else today? Was using it yesterday fine but now I get errors at: from demo import... And can't move beyond it

[@methadonmanfred2787]

this is a huge step up form first order model

[@z33r0now3]

How to do the voice is coming up as the next vid I hope? Gonna use this for my mums birthday, doing it with her favorite artist, wishing her a nice birthday.

[@helphelphelphelpmehelphelphelp]

It's here! kzbin.info/www/bejne/aarNc5KGaLqpnKc

[@z33r0now3]

@@helphelphelphelpmehelphelphelp thx mate

[@taoluanti7816]

Thank you so much for sharing this video😊😊

[@-.......-]

Can you do it in blender?

[@zirufe]

Hi. I am also interested how your dubbing voice also turn into Mr. White's Voice. Is it also AI or just voice filter in any application?

[@yadav-r]

Will this work, if I have a full body shot, my pose will remain static, but the face animate.

[@bestmeme8902]

yo can you do a meth tutorial next pleaseeeeeee

[@violentpixelation5486]

🤣

[@ve7da94]

i tried with diffrent images and videos and it didn't work pass the step with the two in a video.

[@Awokenmystic]

this is insane man

[@DLlama]

Oh wow! I can think of all kinds of fun ways to use this!! 😈

[@aleksandersanya1817]

You're goddamn right!

[@DaniiJardim]

Can anyone tell me where I can find the moving face video to download?

[@CTLN7]

How do you increase the resolution?

[@janholecek2692]

Hey, I wasnt able to change the resolution. How do you do that? When i change it i get error in the last code block

[@vivektyagi6848]

Can a 3D FBX Model of the Animated Final Video be generated too ?

[@drawnimo]

"bryan cranston here, representing Squarespace..." hmm... already seeing how this tech could be controversial! neat!

[@lehaison1236]

What does the video resolution depend on? My video is blurry and not as clear as yours

[@sanjayptlsp22]

Nicely explained. I would like u to make a video on full body to......

[@optimus6858]

how can i run it locally on pc crashes every time due to low ram

[@Briskybroski77]

for some reasons I can't get one of the steps to run...It always says "demo" not found. Can you help?

[@juaninbits]

I get this error, does someone knows how to fix it? FileNotFoundError: [Errno 2] No such file or directory: 'checkpoints/vox.pth.tar' It happens on step 6 (5:28)

[@sam_sands]

me too! if you found a solution please let me know

[@Melisa-vo9sp]

@@sam_sands hey i got same error, did you solve it?

[@gemini9775]

amazing wow ^^ how did you get walter white voice ?

[@TheRealJerseyJoe]

Nice !

[@dougiejones628]

Is there a way to download the Git and run it locally via Pinokio?

[@dany-tu1qg]

is it possible to do that using a faceless image , for example spider man or dead pool ?

[@crimester]

before we begin tho (chunk of meat in superposition appears to the right of heisenburger)

[@jtjames79]

Do it with an anime face.

[@MegaChris456]

I learned this so much faster knowing Walter White is my teacher. Thanks!

[@rucksackcookbook5657]

How exactly do I change the resolution?

[@ginfreecz]

but how did you make it as your avatar? and using a voice changer

[@miracoolgabrell]

Can it be used for live video?

[@devaccount661]

Really cool video and I'm interested to try it out, but I'm a bit nervous about using this model for safety and privacy reasons. What do you guys think?

[@ClarksvilleCares]

I don’t know code so I’m spooked to run it too

[@brendankeogh4669]

Does this work for a non human face? For example an image of a Pixar style cartoon bear face 🐻? or is it only able to detect “ human “ faces?

[@nickexel2425]

I also have this question....

[@danalex5458]

How to do that? Where are programs? You give only 10% of info, how to do it live? In real-time?

[@bfrankob]

Can you please make full body one please 🙏. Thank you so so much

[@swalihstudios]

It's dmn awesome.

[@adekolaopeyemi7874]

Please do realse the one with full body . Thanks

[@briandives7271]

what about the full body version??

[@AaliDGr8]

plz tutorial for ted full body animation things also needed more then that.................

[@TheShazExperience]

Wouldn’t DeepFaceLab be a much simpler software to use than this one?

[@KieranShort]

What's going on in the output when you open your mouth?

[@relaxchill888]

how make it for online streem?

[@blissfulbreeze5966]

So you can do live streams with this?

[@MichaelWarfCA]

I’ve been messing around with this technique but I gave up due to what I thought was the 256x256 pixel limitation for output. Have you had success with higher resolution output? If so I’d love to give it another go with higher resolution assets.❤😊

[@ChrisEldridgeKino]

I export the video to png sequence and upscale those using GFPGAN, then re-render the video.

[@belenparedes2085]

@@ChrisEldridgeKino hey, can you explain me how to do that? If I export the video to png sequence then I have like 1000 pics

[@ChrisEldridgeKino]

@@belenparedes2085 it will be a lot of images. Just look up how to use gfpgan or even a tool like upscayl to process all of the images in a directory. Then convert those 1000 images back into a video with ffmpeg

[@Tellitasitis72Joe]

Hi and Thanks.. Can we run this offline?

[@cameronlazecko5040]

Does not work. Colab will not run the checkpoint code shown at the 5:28 mark in the KZbin video. Tried about 25 times.

[@tanyav2077]

give a link if you have a more detailed analysis, even if it is paid, thank you!

[@kelvinjorge-se4ky]

Can we use this for video call

[@TheMaskedGirl95]

Pls I need app on Android that I can use it from phone

[@____-gy5mq]

"Cgmatter? Uh aah I'm the Walter"

[@ohschneider]

This looks interesting, I send a photo and a video and what about the voice issue, does the video need to contain the right speech?

[@goatpepperherbaltea7895]

The really trippy part about is is when the eyes look down for a second it’s crazy😂

[@degibocu]

ohhh mannnnn

[@abedieckman7894]

No matter how many times I try this step-by-step, it always just pulls back in Trump and Jackie Chan at the "try" step. I've deleted and re-uploaded source and driving five or six times. What am I missing?

[@ProyectoSanAndres]

The same thing happens to me, I don't know what to do.

[@ericfoster5267]

S-tier content

[@user-qv1is6yp4z]

Very interesting, even superbly interesting, but some added info should be provided. Ex: Careful about mp4 size or a Ram crash will certainly ensue. The code editor is the worst, it doesn't even have a right click paste function. I also notice that the end result provide an output of the 3 assets instead of what everyone is looking for: the animated face. Has anyone been able to output the single animated face? ;)

[@gonzaloquispefernandez8303]

Hello bro, the animated face video is inside the "Thin-Plate-Spline-Motion-Model" folder, and it's name is genearted.mp4.

[@kampey8590]

how do i reset the google collab so I can redo the process

[@kampey8590]

figured it out "go to runtime, reset runtime, delete your generated file on the left, delete the checkpoints, and you should be good".

[@Ogg30631]

will it work on an amd gpu?

[@azzedine00]

I have an error message for the fifth step "image file is truncated", can u help me please

[@azzedine00]

my picture was way too big lol, I shrunk it and it's fine now !!

[@victoreze3362]

amazing what you i would like to learn a lot from you sir with respects

[@ahmesrhindeng]

Is really handsome Bryan Carton.

[@abiyyupanggalih854]

I just find gold here

[@frenchie_leguminati]

It's cool but 256x256 pixel limitation + looks like I can't do more than 5seconds, I don't know why. Too bad :)

[@hass2588]

I tried it yesterday an it worked just fine, but today it is giving errors!

[@j5545]

Can I do it locally?

[@asalpeydaei5854]

Any idea of why I can't upload the video?

[@johan9936]

are you solved it yet? I've got the same problem