Master Character Design: Create Consistent Faces with Stable Diffusion!

*** important note regarding the controlnet models. it was brought to my attention by one of our viewers that Stability Matrix/Forge no longer comes pre loaded with diffusers_xl_canny_full.saftensors or ip-adapter_xl.pth
these can be downloaded directly from huggingface.co/lllyasviel/sd_...
and placed in the controlnet\models folder
so this is awesome, the person who originally did the document tutorial on this and provided the original character sheet responded on Reddit and gave me the link to the tutorial he was doing. here's the link.
cobaltexplorer.com/2023/06/ch...
character sheet template can be found in the share linked below under the "Templates" folder.
Welcome to our latest tutorial, where we delve into the art of creating consistent character faces from various angles using Stable Diffusion and the powerful Automatic 1111 Forge interface! 🎨✨ In this step-by-step guide, I'll show you how to leverage a character template sheet to achieve highly detailed and consistent character features, enhancing your storytelling and character design process.
Discover the optimal settings in Automatic 1111 Forge that will bring your characters to life with precision and consistency. Whether you're working on a comic book, a novel, or any project requiring character consistency, this tutorial is your key to unlocking incredible potential in character creation.
🔗 Essential Links:
Access FREE resources on my Google Share Drive: share.xerophayze.com
Try XeroGen Online Prompt Generator: shop.xerophayze.com/xerogenlite
👉 Plus, stay tuned till the end for an exciting update on our XeroGen online prompt generator! This tool is about to get even better, and you won't want to miss what's coming.
Whether you're a seasoned artist or just starting out, this video will equip you with the tools and knowledge to revolutionize how you design characters. Let's create together!
check out these other videos about consistent characters:
• Consistent Characters ...
• Easy Consistent Charac...
• The Truth About Consis...

[@fishpickles1377]

Been trying to get this to work for three days. Nothing. It looks like my issue was that Stability Matrix/Forge no longer comes pre loaded with diffusers_xl_canny_full.saftensors or ip-adapter_xl.pth Had to download them. SO far, everything seems to be working as this tutorial shows. May want to maybe sticky a message or something on here to let folks know.

[@AIchemywithXerophayze-jt1gg]

That's great info, I'll get it posted right away. Thanks for the insight. It's been a long time since I've had to actually install it.

[@fishpickles1377]

Ill keep you posted if i come across anything else.​@@AIchemywithXerophayze-jt1gg

[@gimperita3035]

Great approach to character consistency. Looking into it. Thank you!

[@AIchemywithXerophayze-jt1gg]

👍

[@RickHenderson]

OMG there is so much to this in the first 7minutes I have WAY more to learn.

[@AIchemywithXerophayze-jt1gg]

Keep going. 👍

[@bakablitz6591]

great video, this honestly makes controlnet look fun to use for once 😆

[@vanteal]

Thanks for the lesson, and thank you for the resources.

[@AIchemywithXerophayze-jt1gg]

My pleasure!

[@feilox]

great work! I'll have a re-look over when I have time!

[@AIchemywithXerophayze-jt1gg]

Great

[@user-gl9qb3un6h]

Thank you very much. Each of your video lessons is a gem. Your attention to detail is simply invaluable. I would like to wish you good luck, thank you again.

[@AIchemywithXerophayze-jt1gg]

You are welcome!

[@marilynlucas5128]

@@AIchemywithXerophayze-jt1gg You're incredible!

[@DavidRoyPennCollazos]

This is probably the most comprehensive video currently out there on the topic. I have seen something similar, and have been practicing with a 2*2 grid - splitting then upscaling. But your approach seems to give far better results. I appreciate your time, you've saved me so much work! 😁

[@AIchemywithXerophayze-jt1gg]

Wow, thank you!

[@RonnieMirands]

A comfyui workflow would be great, since its very popular now

[@AIchemywithXerophayze-jt1gg]

I don't use comfyUI myself but others on my discord do and they might be able to help. discord.com/invite/yDytUNpJ

[@dahozabich]

Cool stuff Man

[@AIchemywithXerophayze-jt1gg]

Thanks!

[@oscarorellana3884]

I come from Facebook XD. Nice Results!!!

[@AIchemywithXerophayze-jt1gg]

Thanks for coming

[@kevz1532]

Great work thanks for sharing, any ideas for ComfyUI?

[@AIchemywithXerophayze-jt1gg]

8 haven't learned enough of comfyUI to even begin to perform the same process. Someone on my discord might know as a lot of them use it. discord.com/invite/yDytUNpJ

[@kevz1532]

@@AIchemywithXerophayze-jt1gg Thanks for your prompt reply, what you shared is already much valuable I'll try to work on it in comfyUI according to what you've showed. Thanks again

[@soinov_yar]

Hi! Cool video! Where did you get this SDXL IP-Adapters?

[@AIchemywithXerophayze-jt1gg]

They were just there. I think it's how forge works.

[@nabilalhusail4731]

How about using the character sheet to generate other things? Are you planning to create a video for that to close the loop?

[@AIchemywithXerophayze-jt1gg]

What other things would you like to generate? I think I understand what you're saying, and I have used other types of "character sheets" just using either scribbles or what I'll do is change the coloring or add a shape in the area to represent what it is I want there, and then it's just a matter of messing with the controlnet strength and denoise.

[@nabilalhusail4731]

@AIchemywithXerophayze-jt1gg i mean like using the character sheet i generated and have SD generate it in a pose I tried Pulid and IpAdapter FaceID, and they usually take the front center picture of the face, not the entire sheet So i guess my question is, how do i take on of these characters and have it walk in the street, stand at a market stall, sleep in bed ...etc

[@ridaennasry2558]

brilliant, would u be able to train a lora on that dataset?

[@AIchemywithXerophayze-jt1gg]

I don't see why not with a little setup you probably could.

[@lifeinsienna]

This is very cool - how do you create profiles within the webui that load all of your settings each time you restart the webui if using Automatic1111 or a fork like SDForge?

[@AIchemywithXerophayze-jt1gg]

Yeah it's one of my favorite extensions. It's called config presets. github.com/Zyin055/Config-Presets.git

[@paulbourasseau6219]

Hey, thanks a lot for this video. One question, did you try to create a DFL dataset with consistent faces ?

[@AIchemywithXerophayze-jt1gg]

Lol, sorry I don't know what that is. I'll have to look it up.

[@kaiqhang3482]

It seems that this can only be used for generating synthetic images, not for creating a specific facial dataset.

[@MMrostamiani]

For insightface it's better to crop the image

[@axeldenardi5214]

I don't understand how to make the same face every time after we make the grid with our character.. We just need to import the new grid on the first controlnet and disable the second controlnet ?

[@AIchemywithXerophayze-jt1gg]

Make sure you're using the template that I linked to in the description. The template should go in the first controlnet, The template helps to keep the character consistent.

[@TechnBlog]

Great. Where I can find the control image used?

[@AIchemywithXerophayze-jt1gg]

Share.xerophayze.com In the template folder

[@TechnBlog]

@@AIchemywithXerophayze-jt1gg thx, but I'm using Fooocus, no template xD

[@yeahthatsme4863]

@@TechnBlog the image is in the folder

[@AIchemywithXerophayze-jt1gg]

Yeah sorry didn't see this until now. The control image is in a folder in my Google share called templates. Just go to share.xerophayze.com

[@TechnBlog]

@@AIchemywithXerophayze-jt1gg thanks

[@gjohgj]

Amazing! What if I want to use faceswap on it? Does that work?

[@AIchemywithXerophayze-jt1gg]

Not sure it would probably work for most of them. The ones that are turned all the way to the side might have a little more difficulty. Only one way to find out.

[@gjohgj]

@@AIchemywithXerophayze-jt1gg Thanks! Gonna try it out. Maybe InstantID works better

[@nabilalhusail4731]

How is the adetailer so fast? Currently using comfyui, and I'll need to find the equivalent there for all the turbo stuff

[@AIchemywithXerophayze-jt1gg]

I don't remember how fast it was under automatic 1111, but I'm using WebUI forge edition which has been optimized in a lot of different ways. That may be part of it, the other part is I'm using a RTX 3080 TI with 12 gigs of VRAM and I have 192 gigs of RAM on a 10th gen i9 processor system. So a lot of things run quick for me.

[@nabilalhusail4731]

@AIchemywithXerophayze-jt1gg love the setup I waited and bought the 4080 16GB vram, but paired with i7 and 128 GB of ram, which i thought was plenty for AI, but it seems you can never have enough

[@ulasyunusoglu9630]

Adetailer changes the character's portrait too much and turns him into a completely different person, is there any way to adjust this?

[@AIchemywithXerophayze-jt1gg]

Adetailer does have the ability to adjust the cfg scale when it does the inpainting for the face, if you lower that down you'll see less of a change in the face. But at the same time it may not correct issues with the face like it normally would. It also has its own built-in control net, and as long as you have the models loaded into that you can also add in like a reference image and use IP adapter to retain the same face

[@camoQuandoi]

how do I get : preprocessor : insightface +clip-H ( ipadapter)

[@AIchemywithXerophayze-jt1gg]

All I know is that it just kind of showed up because I'm using stable diffusion automatic 1111 Forge edition. It has its own controlnet and it automatically downloads the preprocessors as we need them.

[@TheGunjanshah]

Thanks joe rogan

[@AIchemywithXerophayze-jt1gg]

Lol

[@friezaniger5178]

How to do this with full body?

[@AIchemywithXerophayze-jt1gg]

Similar process but with a full body character in different poses. Using open pose.

[@wstolk]

It seems to completely drop the character grid on my end and just show me three versions of the character all looking directly into the lens. Any idea what I'm missing?

[@wstolk]

I seem to not be able to select any models of the preprocessors of the controlnet.

[@wstolk]

Yep, that was it. I needed to download the controlnet models. Sorry bout that!

[@wstolk]

Ok, for anyone reading this, I tried the sharing models between a1111 and forge route and that's where it borked up.

[@AIchemywithXerophayze-jt1gg]

Make sure you're not turning the control nut strength down too far. Anything below .25 or .30 and you're going to get effects like that.

[@digor72noname86]

Nothing works. Where can I get the preprocessor InsightFace+Clip-H and where should I put it?

[@AIchemywithXerophayze-jt1gg]

I'm going to be honest with you, I think the easiest thing to do would be to just install stability matrix, then install the automatic 1111 forge edition, then just install the controlnet extension and this will provide all the necessary features.

[@digor72noname86]

@@AIchemywithXerophayze-jt1gg Thanks for the answer! I'm sorry about newby guy. What is a stable matrix and where to get it?

[@digor72noname86]

@@AIchemywithXerophayze-jt1gg i find it

[@JadeEmperorTales]

Does this work for fooocus?

[@AIchemywithXerophayze-jt1gg]

You would probably have to use a special version called fooocus-controlnet I haven't tried it, so I'm not 100% sure on that.

[@badis103]

bro i don't have "DPM++2M SDE TURBO" and i don't have that "Insightface +clip+H" ??? even i updated automatic1111

[@AIchemywithXerophayze-jt1gg]

Use stability matrix and then install a1111 forge edition

[@neoninja4352]

Nice. now do it with Fooocus.

[@AIchemywithXerophayze-jt1gg]

Never used it. Maybe I'll give it a try.

[@neoninja4352]

@@AIchemywithXerophayze-jt1gg it's easier to use.. less capable thought.

[@Jetsetts]

@@AIchemywithXerophayze-jt1gg That would be great, its pretty popular at the moment for low entry barrier use.

[@tstone9151]

Foocus can't do this man

[@MilesBellas]

Automatic 1111 isn't deprecated ?

[@AIchemywithXerophayze-jt1gg]

😂 not the Forge edition.

[@Disha-yq7ph]

not working , all pics are malformed with ip adapter

[@AIchemywithXerophayze-jt1gg]

Yeah definitely double check all your settings. Something's not set right.

[@moonduckmaximus6404]

i didnt know white people could have a ''dark complexion''

[@AIchemywithXerophayze-jt1gg]

Lol, I know, that's why I thought twice about it and changed it to tan. 😂

[@wanglinda2212]

Hi,AIchemy with Xerophayze,reat work! 🙋‍♀question: "canny/diffusers_xl_canny_full" and "IP-Adapter/ CLIP-ViT-bigG" where to get it?

[@AIchemywithXerophayze-jt1gg]

So I'm using a version of automatic 1111 called Forge, it kind of comes with that when it's updated.

[@fishpickles1377]

I realize this is a month old post, but in case you never found them, you can get them at huggingface, just google the individual file names, and you should go right to it. I had the same issue. (obviously disregard this if you've already found them.) :)