ControlNet Guidance tutorial. Fixing hands?

[@timm1638]

I think the quality of your hand/starting image makes all the difference here. If you can create or find a quality model at the correct angle for your picture, that's when this will shine.

[@Jojo2]

1:00 I'm looking for something more like this though, keeping it almost exact but just a better hand.

[@ryanshi4565]

I saw some one using depth map with real hand picture so it can paint correctly for shape of hands. maybe u want to try that.

[@sebastiankamph]

Yes, I provided that in video description.

[@AscendantStoic]

There is a way to do it with ControlNet + Inpainting I think, Aitrepreneur did manage to mix two different images by inpainting part of one image into another, if you apply that to a hand image/render/sketch it could get more consistent results.

[@weizhenghuang6452]

This is what I did - I Photoshopped a good hand over the generated bad hand, then I exported just the hand and used it on Canny.

[@AscendantStoic]

@@weizhenghuang6452 Yeah, that's definitely a valid way to do it 👍

[@LewGiDi]

Yes, I'm using the second method he explained in his video to have same picture first photoshoped and second using inpainting

[@Zurilo_]

Have you tried with inpainting? So would be like this: txt2txt > get an image > send to inpainting > painting over the hand > on ControlNet put the hand image as you did > generate

[@michaelleue7594]

If you look at the image, you can still "see" the original hand in the modifications, which suggests that the "idea" of the hand's position is forming pretty early in the process compared to when you're introducing the modification. I think you're likely to have better success if you adjust the start time earlier until it no longer seems like the new hand is a modification of the old hand.

[@NotY0urHeroo]

Seems to me that Guidance End would primarily be used as a sort of img2img with more control. You could have very simple shapes like rectangles in the background to turn into a cityscape background that only need to be held for a short while before you let your prompt take over for more variance across batches. Nothing we couldn't do before with extra steps but now it's conveniently tucked into text2img.

[@sebastiankamph]

Good point!

[@theaiplaybook]

It does save some time indeed. :)

[@Tcgtrainer]

Great tip man, i was wondering yesterday about it.

[@harrycslau]

Really love your videos and especially thanks for tapping into this topic! The hand/fingers problem in SD is really a big pain for me. Inpaint, controlnet, different models... all are of no avail. It is particularly difficult if the hand or finger is covering a part of the body, say the shh gesture (i.e. index finger on lips, which also happens in this video). I really wish that you could publish an ultimate guide on perfect hands and fingers fix soon! Have a good one 😉

[@fabgeb667]

you can do a lot by using negative prompts. and words like "beautiful", perfect hands, five fingered hands

[@harrycslau]

@@fabgeb667 Unfortunately, I've tried them all but of no avail.

[@allgouda4u]

in ControlNet, how do you get additional tabs for control models like he has in the video? i only have one

[@sebastiankamph]

See my video on Multi-controlnet. kzbin.info/www/bejne/mX-seY1_o7tmo8k

[@vanteal]

Photopea is awesome. Its developer is awesome! Wish there were more devs like them.

[@sebastiankamph]

Agreed! Now I do mostly use Photoshop, but since all the shortcuts are the same, I use Photopea for videos so everyone can join in 🌟

[@welton26]

Enable Guess Mode and put 50 steps. the result will be much better

[@sebastiankamph]

Interesting, thanks for the tip! 🌟

[@amj2048]

this will be really interesting for animations, thanks for sharing!

[@pastuh]

Helpful :) I will try to add rain drops

[@ysy69]

Thanks for this share. We're certain this will only get better and we will have perfect hand control sometime this year

[@123b3d]

Cool idea, Sebastian. Did you try that with Openpose Hands? Also i downloaded DAZ 3D to get poses, maybe that helps you 👌

[@saeruji]

Did you see the LoRA thing? I used some ready LoRA models successfully, but couldn't train one myself. Would love to see a video review of it from you!

[@JohnToridas]

which LoRA models did you use?

[@saeruji]

@@JohnToridas Various character LoRA.

If they somehow figure it out to lock parts of the prompt to the region handled by a specific ControlNet instance (or specify separate prompt for each ControlNet section), that would probably again bump up the quality a notch.

[@Antraxus11]

Its definetly a interesting feature and I will try it out but so far I feel that fixing hands with inpaint might be better at least when you dont want a specific hand gesture.

[@sebastiankamph]

Inpaint + ControlNet probably is the way to go

[@Nova-ul3vv]

Great video like always. It would even be better we could use this during inpainting somehow as i always want to change a specific thing in my images. Maybe also for next time you could change cloth or hair but keeping original input

[@TheSchwarzKater]

For some reason the resolution of the controlnet hand **inside Inpaint** is way off and I can't get it to work anymore. An update must have broken it. It still works in text2img though.

[@mschf_nft2581]

Use gues mode and do 4 iterations, instead of adjusting values

[@sebastiankamph]

Interesting, good tip!

[@ncturnal23]

this man jokes always got me 🤣

[@sebastiankamph]

I see you're a fellow dad enjoyer 🌟

[@antisony2008]

which GPU is he using? is blazing fast

[@sebastiankamph]

Rtx 3080

[@TheCopernicus1]

Click invert Input color too! you have a white background :)

[@morizanova]

So in theory we can fix the text or adding UI text in SD using this feature right ?

[@sebastiankamph]

I did a video on text workflow, check that.

[@leejacques8170]

I like your videos! I can’t seem to install controller properly on my Mac. I see it there and openpose as well but there’s no send to controller button under the stick man in openpose

[@testales]

Shouldn't the "invert color" box be checked? I mean it literally says "Invert colors if your image has a white background"? Quite confusing if this isn't actually necessary.

[@sebastiankamph]

You would think so, and I've tried it and can't make sense out of it. Sometimes it works, sometimes it's in reverse. Either I'm doing something wrong or there's a bug somewhere.

[@sownheard]

why is your control net UI ControlNet - 0 and ControlNet - 1 next to each other instead of below?

[@sebastiankamph]

They keep changing the layout with new updates

[@Name-sl3bm]

I saw "can fix hands" -> i click xD

[@12njr]

on tiktok i see videos where art is moving and has many fotos, i want to learn it but i dont know how its called or what technique it is. the tiktok creator is not responding aswell 😅😮‍💨

[@sebastiankamph]

Deforum. Check my animation tutorials.

[@conrifor]

Magic poser has hand presets you can use

[@sebastiankamph]

Good tip!

[@gabriellekamph]

⭐🙌

[@dkemil]

I think the only way hands will get fixed is a model or LORA specifically made to fix hands by inpainting.

[@ShawnFumo]

It seems like using depth maps with skeleton from the beginning is decently consistent. Using either that blender rig or now there's an extension with a library of hand depth maps, plus can use openpose editor or the new 3d skeleton editor to get the main pose. Part of the issue in this particular case was trying to change the hand of an existing render without changing everything else.

[@paperbea]

lol I just want to point out 'peace sign 2' in the hands pack is an offensive jesture in the UK.

[@DanielPartzsch]

I'm not sure but maybe guidance end helps reducing producing too much details for eg background elements similar to the weighting described in this video: kzbin.info/www/bejne/fpyvhqCKf9Ssntk

[@Airbender131090]

wtf. what is the point of the tutorial? there was no perfect hand till the end

[@thibaudherbert3144]

So now we will logically finally be able to create real star wars lightsaber fight or knight fight with proper handle/hands etc

[@nothappyz]

2:13 💀

[@califmike2003]

We shouldn't even have to deal with hand problems there's got to be a way that they can fix in the software.

[@TheJtrg]

o_0 bad hand was, bad hand is... time for nothing

[@ElHongoVerde]

Finally. Hands! As the kids say nowadays... first comment

[@TeunSegers]

As the kids say nowadays... fail

[@ChristmasNeo]

It is so fast… with my amd it would took me 5-10 mimutes

[@sebastiankamph]

Vroom! 😂

[@ChristmasNeo]

@@sebastiankamph sry, don't rly understand this ^^" - but still a great video

[@guangxing9010]

File "C:\Users\lenovo\AppData\Roaming\Python\Python311\site-packages\torch\cuda\__init__.py", line 284, in _lazy_init raise AssertionError("Torch not compiled with CUDA enabled") AssertionError: Torch not compiled with CUDA enabled

[@nova-thecreator]

am i the only one who's controlnet has been broken since 1.1 update?

[@abdellahla6159]

chatGPT jokes 🤣🤣🤣🤣

[@OldToby53]

thats one frustrating thing about AI rendering...hands!

[@Beyondarmonia]

Scribble is shit for hands. Use depth or canny.

[@rosiegonzalez2322]

you talk so slow bro lol