Combine FaceID and Facial Expressions with IPAdapter & Controlnet

Learn to combine facial expressions with your reference face by combining FaceId IPadater with Controlnet in less than 10 minutes.
🔍 *What You'll Learn:*
- Step-by-step instructions on using a workflow to apply expressions to your reference face using controlnet and IPadapter.
- Demonstrations of IPAdapter troubleshooting to get your desired result.
🔗 *Useful Links:*
- Join our Discord Community: [Join Discord]( / discord )
- Models & Workflows:
- [Workflows](endangeredai.com/transferring...)
🌟 *Support Endangered AI:*
- Patreon: [Support on Patreon]( / endangeredai )
- Buymeacoffee: [Buy me a coffee](www.buymeacoffee.com/endangered)
🔔 *Stay Connected:*
- Visit our Website: [Endangered AI](endangeredai.com)
- Subscribe to our Newsletter: [Sign Up Here](endangeredai.com/sign-up/)
👍 *Engage with Our Content:*
Don't forget to like, subscribe, and hit the notification bell to stay updated with our latest tutorials and guides. Your engagement greatly supports our channel and helps us keep producing high-quality content!
#IPAdapter #TechTutorial #AIWorkflow #IPAdapterGuide #stabilityai #stablediffusion #comfyui #controlnet

[@pixelcounter506]

Straight to the point, well done! First, I used openpose/dwpose, later in combination with canny, but I wasnt happy about the results. For me, the combination of canny and Midas depth map (I prefer it over Zoe) works the best.

[@Merlinvn82]

You can combine canny with depth or (and) openpose. Adjust strength accordingly.

[@EndangeredAI]

Hmmm interesting, have you tested it out? What would be the benefit of depth? As typically it wouldn’t catch facial features right? Are you suggesting this to compensate for canny taking too much detail?

[@Merlinvn82]

@@EndangeredAIThe idea come from 3d render with 3 maps: bump, depth and normal. Those combined maps will make the render more realistic. I tried those combinations for facial render long time ago when the ipadapter was not invented yet 😅.

[@pixelcounter506]

@@EndangeredAII tested all combinations of canny, openpose, depth masks, but esp. for faces and poses. Depth seems to be good for landscapes and structures (for sub-/objects which aren't so much in focus like a face), but gives not enough information about the face and its contours. Therefore canny to get a better facial structure. I would define it the other way round: depth gives not enough information (the same goes with openpose/dwpose).

[@florianbecke3076]

Was looking for some guide like this. Thanks alot!!!

[@EndangeredAI]

Glad it helped!

[@goodie2shoes]

nice! I'm experimenting with a way that involves merging 3 diffrent, manipulated faces through ipadapter plus.I use a 3d toon model to basically make a caricature of a person. And then feed that result back in the loop and 'tune it down' with a normal model and the regular image's. It's a great way to get hilarious and kinda extreme results that still look 'natural'. Of course I will give your method a try! and perphaps combine it with my attempts.

[@EndangeredAI]

I’d love it if you came by the discord and shared your work in the share your work channel! Would love to see!

[@goodie2shoes]

@@EndangeredAI Thank you and I might take you up on that offer. First I need to perfect the workflow. It still has a lot of manual tweaking involved :-)

[@stuartrae2253]

Thanks so much for this! I am having problems when executing IPAdapterUnifiedLoaderFaceID - Is it beacuse its presets and how can i fix this?

[@EndangeredAI]

I need to know more about the issue, please come by the discord and drop some screenshots shots, we’ll sort you out

[@NokaNokai]

I've been following you for a long time and I love your videos. I would like you to do, if you can, as a way to remove freckles or moles from your face.

[@EndangeredAI]

Interesting! I’ll consider it for a 2 minute Tuesday video

[@Make_a_Splash]

Very interesting. Thanks for the workflow. UNfortunately brings me an error when running it in SD1.5 : ClipVision model not found. Any ideas how to fix this? I have updated everything

[@EndangeredAI]

Are you using unified loader, and you have the clip vision files downloaded and renamed correctly? If there’s any spacing issues it might cause problems. Regardless you can always grab the ipadspter advanced node +model loader, and clip vision loader to select the clip vision model manually.

[@ImAlecPonce]

Hi, nice vid!! if we erase the man's hair... would it respect the IPadapter a bit more or would she be bald?

[@EndangeredAI]

It would respect the ipadapter more, as where there’s no lines the model will do what it wants

[@francescodetommaso-yc4il]

awesome tutorials! but why in the preview image of openpose i see all black?

[@EndangeredAI]

That means it hasn’t picked up a pose from your image.

[@francescodetommaso-yc4il]

@@EndangeredAI thank you!

[@mr.entezaee]

The link to download Workflows does not work for me. Can you check and see, right?

[@EndangeredAI]

Let me double check that, thanks for letting me know

[@mr.entezaee]

@@EndangeredAI I hope it will be fixed as soon as possible because I really need it. Thankful

[@EndangeredAI]

@@mr.entezaee Sorry for the late response, I was away over the last week. The link is in the description, you can find it at my website here. endangeredai.com/transferring-facial-expressions-using-ipadapter-faceid-controlnet/ you might just need to click the download button twice, as the ad platform I'm currently using is a little annoying when used with Wordpress. I'm planning to change it soon.

[@mr.entezaee]

@@EndangeredAI It was finally downloaded and I am very happy about it. really thank you

[@PawFromTheBroons]

Aaaand another channel calling this a 2 Minutes Tuesday. Because the KZbin Gods know that if you want to succeed on the platform, you need to have Peter McKinnon's style topped with MrBeast's font. 😆

[@EndangeredAI]

I actually didn’t know that was Mr. Beast’s font 😂, and they always say… learn from the greats.

[@PawFromTheBroons]

@@EndangeredAI You didn't use it, but there are scores of aspiring KZbinrs who do, as if it has some sort of magical incantation property. If find it quite depressing.

[@EndangeredAI]

Well for me, I'm working on my storytelling and general ability to improve the quality of my tutorials while keeping viewer engagement. Following a format from an experienced and well known KZbinr and storyteller helps me focus on trying to improve that craft while having something to benchmark against. I know many people take it as a gimmick, but I'd love to have the ability to craft a tutorial on the level of Peter McKinnon or Networkchuck. One of the best pieces of advice I ever got was Immitate. Then Innovate. I'm still at the imitation stage, but I do hope you stick around and let me know if you see an improvement every few videos. I am always trying to new formats and new ways of getting the message across while maintaining clear explanations in combination with engaging storytelling.

[@PawFromTheBroons]

@@EndangeredAI You're doing quite well, in a very crowded space.

[@TheSickness]

Sus pop-ups on that site, hard pass

[@EndangeredAI]

You can go ahead and close off the ads, AdWords has stopped accepting new sites, and the one I’m currently using is annoying I will admit. Im working on making it less obtrusive. But it helps pay for things albeit slightly. You can close it off, or alternatively the workflows are available at Patreon ad free