How to use Face Analysis to improve your workflows

[@goodie2shoes]

Ì was in the middle of performing open heart surgery but then I heard the new Latent Vision video dropped. Patient will have to wait.

[@latentvision]

even better invite the patient over

[@AndyHTu]

lol its worth it. The heart has to come after.

[@context_eidolon_music]

I was watching Latent Vision when the new Latent Vision dropped. I obviously died.

[@latentvision]

@@context_eidolon_music 🤣

[@jordibasss]

You can always create better face for the patient 😂

[@TheDocPixel]

Matt3o... you're an absolute gem of a teacher, developer, and entertainer as well! Your casual humor really makes this mountain of info so much easier to climb! Thank you so very much, and wishing you joy in everything you do!

[@radsy5821]

You are a man of taste and distinction choosing Ms Ferguson as your subject.

[@tartwinkler1711]

Thank you for explaining how this works. This type of information is very useful when delivered by someone that is also a great teacher.

[@lhovav]

Thanks!

[@davidb8057]

Fantastic job, Matteo. The amount of knowledge you're sharing in your videos is always amazing.

[@starzilla2975]

FANTASTIC! Just what the doctor ordered! These align nodes will be very nice to play with too! This may be a long shot with the current technology, but I am hoping at some point I can find a way to get my faces to take a reference expression and copy that expression to another face or just straight up better control expressions in some way. unfortunately, right now, stable diffusion doesn't seem very flexible with expressions and it changes the face a lot so this will be helpful to help me constrain the face to be the same person. With lukc, there may be a way though so I am gunning for it.

[@latentvision]

it's possible to change expression but it only works with extremes (very angry, very happy... or neutral)... we miss everything in between

[@starzilla2975]

@@latentvision You should consider looking at a new paper that came out this week titled "Face Adapter for Pre-Trained Diffusion Models with Fine-Grained ID and Attribute Control", it published with code and a huggingface model. It does exactly what I hoped, takes a reference image and copies the expression to your target face, it isn't perfect but it is definitely an improvement and the best I have seen so far for Stable Diffusion. Unfortuantely I think it only works for SD 1.5 but they have a demo and I did a bunch of test faces on it and it is pretty great. I don't know how related it is to IP-Adapter, but considering your stint on face analysis and swapping, I figured I would comment on here and let you know in case you missed it because I think it is something you would be interested in.

[@starzilla2975]

@@latentvision In case you missed it, I definitely think you should take a look at the paper with it's associated hugging face model and githubio titled "Face Adapter for Pre-Trained Diffusion Models with Fine-Grained ID and Attribute Control". It came out right before this video and it's exacty what I hoped, even if it only works with sd15

[@YING180]

You are the real master and you are so generous to share those techniques. Thank you so much.

[@crazydiamond3757]

Complimenti! ...Lei è davvero un cervello in gamba!

[@leolis78]

Mateo your work is excellent! you always surprise all of us with your progress! The workflow you show in the video is uploaded somewhere? Thanks a lot!!!

[@latentvision]

it's in the discord but... I pasted it here too pastebin.com/4tYRceiK

[@leolis78]

@@latentvision thanks so much!!!! 😃

[@2PeteShakur]

brilliant and fresh as ever, keep up the great work matteo! ;)

[@d4n87]

Grande Matteo 😁👍 anche per le spiegazioni tecniche dettagliate su queste cose difficili spesso da comprendere

[@unsivilaudio]

"It will hurt your feelings" 😂 happens to me alot when I thought I got a good likeness

[@Firespark81]

Thank you for making this. It was very helpful.

[@Skyn3tD1dN0th1ngWr0ng]

It's free... Did I mentioned its free?... Well is accessible and free very informative and helpful, thank a ton.

[@kalicromatico]

Woowwww!!! this video is a masterclass

[@tbolton9247]

Love your videos and tutorials....best out there and I learn a lot

[@mehradbayat9665]

...that last picture 🤣

[@gnsdgabriel]

Thank you 🙏

[@weirdscix]

Another excellent video, thank you

[@marshy..]

Now I just need a faster GPU... Excellent video - thank you.

[@michail_777]

Hi, Mateo. I understand you are the one creating the nodes for the IPAdapter. There is a node IPAdapter Encoder, it has a mask. I tried to connect the image and image as a mask, but did not understand what it is for. If possible, please tell me how to apply this mask correctly in this particular node. Thank you again.

[@OlivierLutaud]

Instant ID won't install (import failed) any ideas on that issue? Thanks

[@HiProfileAI]

I got that working but only with insight face. How do we install Dlib?

[@bwheldale]

If you only got as far as including the models (as I had) it still has more to do. This is how I got it working with portable windows Go to the provided link for "Face Analysis for ComfyUI" Installation Section: For DLIB download (4 models) Check which 'python_embedded' python --version in ComfyUI portable (mine was 3116) Download the Precompiled Dlib (for Windows e.g., 311) (into ComfyUI portable root folder) run the command python -m pip install 'the exact spelling of the downloaded wheel file' UPDATE: Well this does get the dlib into the dropdown, but I celebrated too soon: Error occurred when executing FaceEmbedDistance: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (4,) + inhomogeneous part. Sorry, if someone else can help both of us that would be appreciated!

[@HiProfileAI]

@@bwheldale I always have problems with the pip well install stuff I'll try to figure it out and then give an update on how to do it.

[@bwheldale]

I can verify the Dlib was installed succesfully, the other issue was something else and has been resolved with the latest update.

[@JoeyD786]

Does anyone know how I can get the value and dist data shown in the image preview?

[@kytbaba67]

Is it possible to make caricature image with similar workflow?

[@sergetheijspartner2005]

Do you know that meme where someone is in mathclass and the professor writes 1 + 1 = 2 and he thinks it is easy and then he is distracted for a second and the board is full with the most complex formulaes? That is what your videos feel like, luckily you can pause this and return if you missed something😂

[@latentvision]

don't worry... happens to me all the times

[@petneb]

Thank you.

[@riccardobiagi7595]

Great video as always! a question.. Where do I find the "Face segmentation" node? I don't have it and If it's a custom node to install I'm not finding it :( Thank you

[@no-handles]

update then refresh, you will see it when you search for faceseg..

[@giusparsifal]

Hello and thanks, I am on your discord on workflow section but I can't find how to download it...

[@adriantang5811]

Thank you Teacher!

[@KINGLIFERISM]

Just waiting for this to be as good as photomaker.

[@ChanhDucTuong]

What an insightful video, I learn a lot interesting and useful stuff from this. Thank you so much. There are 2 problem I haven’t been able to fix without training a Lora: 1. Can this workflow or any other workflow could ensure the proportion of (head vs body) or (head vs shoulder width)? In the past I usually got problems of head too small or too big. 2. How can we control the age and other feature of the face without Lora? I mean I got complains about the face look too young/old or lack some marks/moles of the original face. I can manually fix them for a few image but If I want to scale then it would take a lot more time compare to just train a Lora for that face.

[@latentvision]

well for #1 you need a controlnet, for #2 I'd say with good prompting you can improve things considerably

[@ChanhDucTuong]

@ Thank you again. I just remember that in many beauty apps like SODA there are many sliders to adjust the size/length/width/color… of face/nose/mouth… etc. so it’s a quick fix for #1 issue. I’m not sure if there is similar tool for Comfy.

[@eucharistenjoyer]

Thanks for the video Matteo! But doesn't FaceID load insightface already by default? Why use dlib then?

[@latentvision]

not all face models require insightface

[@eucharistenjoyer]

@@latentvision Thank you, that's what I imagined. It's just my bias and the fact that FaceId is so good that I end up using it in almost all the images I do haha.

[@Homopolitan_ai]

Top 👍

[@IOVART]

Error occurred when executing FaceAnalysisModels: Can you tell me what's wrong? Error occurred when executing IPAdapterUnifiedLoaderFaceID: IPAdapter model not found.

[@contrarian8870]

Can you explain the concept of "embeddings" in general? What function do they serve? What do these numbers represent?

[@lucagenovese7207]

Maestro... grazie!!

[@sharpwater2]

Great video! Anyone know where I can find the 'Image Random Transform' node?

[@latentvision]

ComfyUI Essentials, link in the description

[@mattm7319]

face align could be really funny to work with ^^ :D

[@runebinder]

Downloaded the workflow but getting this error on the FaceEmbedDistance node, any ideas? I've tried to pictures of the same person that were taken on the same shoot and look very similar. "Error occurred when executing FaceEmbedDistance: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (4,) + inhomogeneous part."

[@bwheldale]

Me too

[@bwheldale]

MIne is working now with the latest update of Face Analysis

[@runebinder]

@@bwheldale I reinstalled Windows over the weekend (File Explorer has been playing up and been putting it off for ages), working fine for me after setting Comfy back up . Would have installed the updated version you mentioned.

[@fayezhang6085]

Which node in maneger is “Face Warp", or github address? I can't find it.

[@latentvision]

it's in the face analysis extension. You probably need to upgrade it if you have it already

[@jomiller7332]

trouble changing within the node "face Analysis Models" from insightface library to Dlib library, (I did make a Dlib folder within Comfyui Models )

[@anandwadodkar]

I am stuck too

[@dfriis]

As far as I understand, Insight Face is for non-commercial use - what are alternatives?

[@latentvision]

depends what you want to do. for face analysis you can use dlib

[@dfriis]

@@latentvision Thanks for the reply and thanks for the amazing videos! You are an excellent communicator! For doing "face transfer", is InsightFace the only option?

[@AnNguyen-pd2xi]

Hello Latent Vision! What's the best method to get the most similar and realisic face? Using nodes as Controlnet, PulID or training LORA for this face?

[@latentvision]

depends on the source material and the kind of image you want to generate (eg: can the colors and pose be the same as the original?).

[@AnNguyen-pd2xi]

@@latentvision I'm focusing on similarity. When I try to generate images of babies in cute scenes and costumes, I realize that maintaining facial similarity is harder with babies than with adults. My idea is to incorporate scenes and costumes in Lora to create the style I want and then use your PulID workflow to achieve the best facial results. My goal is to get the realistic face details combine with my desired photography style.

[@adityaghalibhendryan4211]

Can FaceID node use dlib too instead insightface so we have free commercial option for FaceID IPAdapter?

[@latentvision]

unfortunately not at the moment

[@GES1985]

Is there a node or workflow that takes 2 images of the same ai person and generates an image that represents the halfway point between the two? Idea being to make frames for video / gif

[@latentvision]

you may need a controlnet of the face turning

[@GES1985]

@latentvision does control net have head position points for faces that are turned beyond 90 degrees (like back of head/hair for turning around)

[@banzai316]

Very useful

[@rambapat588]

I tried to install instant id and face id from comfy ui manager but is shows import fail error, could it be because of my low vram, cuz I have only 4gb vram or could there be any other reason?

[@latentvision]

4GB won't be enough for InstandID but it could work with SD1.5 models and faceID. check the repository for installation instructions

[@voxtyz]

can you share the workflow?

[@mr.entezaee]

how to install Dlib ?

[@latentvision]

check the repository issue tracker or my discord

[@sirmeon1231]

My Manager says dlib is installed, but i can't select it :/ The things you can do with it are pretty awesome!

[@latentvision]

check the repository, you'll find installation instructions

[@rinpsantos]

@@latentvision The same for me... I installed the dlib models into dlib directory and installed dlib library too. However, I can´t select dlib (only insightface).

[@rinpsantos]

Well, I installed the dlib library using the ConfyUI portable environment and, now, I can choose dlib.

[@latentvision]

@@rinpsantos please check the discord for help or the official repository

[@raysun5889]

@@rinpsantos Hey man, can you elaborate on that, I was hoping to find a "venv" for virtual environment but I can't..., please help, thanks a lot!

[@AmitShq]

ty where is the workflow?

[@clavarx11]

Why not use face alignment by default in all ipadapter nodes? This node is awesome but should be a default thing in ipadapter.

[@latentvision]

we can't foresee all scenarios, up to the user to pick the best workflow for their needs

[@melihdalar6610]

😍

[@thevoid6756]

just fyi "likeliness" is a probability. "likeness" describes similarity.

[@ismgroov4094]

Thx sir q

[@AI.Absurdity]

🖖

[@lijiang-g2s]

Great, I have a request, now facial analysis can only process one person's image, I think three people together, analysis of the face may produce three different images,Not in the same picture

[@Kikoking-y9b]

What's the name of that beautiful girl

[@latentvision]

that would be Rebecca Ferguson