WOW! NEW ControlNet feature DESTROYS competition!

With a new major update to ControlNet for Stable diffusion, Reference only literally changed the game, again.
Prompt styles here:
/ sebs-hilis-79649068
Support me on Patreon to get access to unique perks! / sebastiankamph
Chat with me in our community discord: / discord
My Weekly AI Art Challenges • Let's AI Paint - Weekl...
My Stable diffusion workflow to Perfect Images • Revealing my Workflow ...
ControlNet tutorial and install guide • NEW ControlNet for Sta...
Famous Scenes Remade by ControlNet AI • Famous Scenes Remade b...
LIVE Pose in Stable Diffusion • LIVE Pose in Stable Di...
Control Lights in Stable Diffusion • Control Light in AI Im...
Ultimate Stable diffusion guide • Stable diffusion tutor...
Inpainting Tutorial - Stable Diffusion • Inpainting Tutorial - ...
The Rise of AI Art: A Creative Revolution • The Rise of AI Art - A...
7 Secrets to writing with ChatGPT (Don't tell your boss!) • 7 Secrets in ChatGPT (...
Ultimate Animation guide in Stable diffusion • Stable diffusion anima...
Dreambooth tutorial for Stable diffusion • Dreambooth tutorial fo...
5 tricks you're not using in Stable diffusion • Top 5 Stable diffusion...
Avoid these 7 mistakes in Stable diffusion • Don't make these 7 mis...
How to ChatGPT. ChatGPT explained in 1 minute • How to ChatGPT? Chat G...
This is Adobe Firefly. AI For Professionals • This Is Adobe Firefly....
Adobe Firefly Tutorial • Adobe Firefly Tutorial...
ChatGPT Playlist • ChatGPT

[@sebastiankamph]

Download Prompt styles: www.patreon.com/posts/sebs-hilis-79649068 Please support me on Patreon for early access videos. It will also help me keep creating these guides: www.patreon.com/sebastiankamph

[@mizutofu]

how do you get two controlnet unit on your gui?

[@UnBknT]

You have to add the styles to the prompt, btw. In the video you just selected them from the dropdown, but they're not added to the prompt until you click on "add style to prompt"

[@sebastiankamph]

@@UnBknT Not needed to click the button. They are still applied.

[@142vids]

Why pay your monthly patreon while I can watch your free youtube videos with adblock on? We were beating the competition I thought no?

[@sebastiankamph]

@@142vids you're free to do whatever you want. The people supporting me does it from the kindness of their heart, helping me keep doing these videos.

[@Rustmonger]

Started playing with it a few hours ago. It is insane. It's nearly as good as training but without the training. It pulls faces, poses, lighting, art style, everything. I cannot believe this is only the first iteration, it is already so good. I thought Shuffle was dope but this is on a whole new level.

[@Mocorn]

Exactly, "almost as good as training" is the scary part. I've been able to get better likeness out of this reference_only model than I've had with pretty much every early training attempt. There's been a bit of cherry picking but in some cases I've gotten 2 extremely good hits from a 4 batch render. It's crazy how good this is already!

[@derek5634]

@@Mocorn Strange - casue I still cannot get it to create a decent copy of the original face. Always makes the new image look younger and very different from original face

[@tamask001]

To me it looks like this method only works for people "coming out of the model". For example if you take the seed image from this video and try to generate other images from that without Sebastian's "Digital/Oil Painting" and "Easy Negative" styles, the results are very unimpressive. I'm not saying that this new ControlNet is not super cool for some use cases, but I though he could have been clearer about the limitations.

[@ggoddkkiller1342]

I couldn't make it work with v1.1.174, txt2img is completely broken even hair colour doesn't match. While img2img kinda works better at least matching hair and clothes but faces are out of a horror movie twisted etc. Im using exactly same styles and settings

[@kimjohn3877]

how to access free trial?

[@ckblue154]

this is actually the very definition of game changing

[@sebastiankamph]

💯

[@FarbachViBrittanniaBuildPeace]

Can create a character and make a whole tv show or anime out of the character

[@illusivec]

Wow. CN guys are on a roll. They are innovating faster than OpenAI and Google. Hopefully they can keep up the momentum.

[@rproctor83]

Ha, every day there is a dozen new breakthroughs!

[@jan.kowalski]

@@AG-ur1lj Thats why the battle for those brilliant mind is not based on ambition but depravation. The big ones will acquire what they can, and the rest will be depraved and obscured. As always.

[@jan.kowalski]

​@@AG-ur1lj powerful how? Will it scale to millions of users? Will it be safe from lawsuits or flexible enough to attract business users? I doubt that. Microsoft or Google could wait and buy anything viable and you, even with your brilliance will have nothing to say. As always in the history.

[@jan.kowalski]

@@AG-ur1lj you didn't realized that this technology is already paywalled and regulated. You will not profit of it - above certain level of course - because you will not have a resources to train those tools or licenses to use copyrighted source data. As of now, it is not a problem for big corporations, because they just take the best solutions and use them with their data. You probably will be happy, but once more, you will not profit of that. Even if you will be able to train the best of the art algorithm it will be WORSE than their, because they have access to all those data and resources.

[@jan.kowalski]

@@AG-ur1lj you downloaded terabytes of images and text and all copyrighted books and proprietary magazines from internet? I doubt that. Yet, Google or Microsoft works on that scale. Since you will NEVER have access to data, you will just become a giver of ideas to big corporations with your improvements to "open" algorithms. Without data, those algorithms just dont work. Even algorithms I enclosed in parantheses because when open source community will produce some breakthrough algorithm, big corporations WILL patent some small improvement and you will be barred from using them. That is the reality based on history. I'm amazed of your idealistic view of business.

[@deepfakeboy]

Dude... You're litrally faster than me clicking update button in SD... Have my sub!

[@sebastiankamph]

Thank you kindly! 😊🌟

[@timpruitt7270]

Unfortunately, it doesn't work for me. The generated images all look like the same person but they dont resemble the person in my original image. It's like my image is completely ignored.

[@PKBO173]

Totaly same. Iam getting whole different face..

[@ВладимирМороз-т5ъ]

Do you use mac m processors? Cause i have and there is a bug when it tryies to catch uploaded face.

[@kamillorek5087]

The Open Pose 3D extension is great for posing - you can run it in the GUI tab, set the skeleton in three-dimensional space, together with hands and feet and generate 3 images: canny, depth and openpose.

[@sherpya]

please raise your volume, I almost had an heart attack when ad kicked in lol

[@hoting666]

This seems like something they could really use to do multi-frame rendering for txt2video

[@kallamamran]

I've tried it... I don't get it to render anything even close to the likeness of the input image 😥

[@forestlong1]

Very strange! I updated everything, turned everything on exactly the same way, uploaded a picture but the result is completely random. It does NOT WORK!

[@TheFarmingAngels]

the same. it works only with some demo pictures (perfect face, no sobtle expressions, no background). And openpose miss the front/back pose 70% of time

[@DedBruzon]

Yes, same thing.

[@lioncrud9096]

I LOVE THIS FEATURE. Already got some awesome results on first few minutes fooling with it.

[@ADMNtek]

I have only started using Stble Diffusion a bit over a week ago and your videos are such a big help.

[@GooseAlarm]

This is insanely useful. Like I've been trying for the last week to collect images for a Lora. It can be tricky as hell because keeping characters consistent is HARD. Change just a few words and suddenly the whole piece looks like a different style. It will be SOOO easy to make Lora now thanks to this. What will they come up with next because google and openai in my opinion are doing a pretty "meh" job.

[@TTarragon]

Yeah, this was my first thought too. By itself, it's great, but it can be SO useful for training Loras, which I suppose, are more accurate

[@scottyfityoga]

Hey can you please tell me how this makes it easier for training a Lora?

[@benjaminjako]

@@scottyfityoga Easier to source images of a certain person, for example.

[@jasonmarktemple]

i have 1.66 - but it will only copy the pose - the person looks nothing like the original photo...any ideas why?

[@Maltebyte2]

Same issue here! does exactly motivate me at all to continue this! have you found out why?

[@MaxKrovenOfficial]

Just wow, GAME CHANGER is the right set of words for this... just tried it and am uttelry impressed, thanks for reporting on this!!

[@michaelleue7594]

I would love to see how they pulled this off. It seems like if they can do this, then a lot of other things we don't have yet ought to be possible, like maintaining outfits or architecture. This is perfect for making comics, though, with character coherence between frames. Maybe they could even fix the coherency issue of tiling a high res image, depending on what they did, exactly. This is pretty crazy,

[@wykydytron]

You can maintain outfit with it, just promy that outfit or maybe use just outfit here and face in separate controlnet .. you know what gonna check that today

[@skittlzboi]

@@wykydytron did you figure out how to do it? I try to use a CN for reference and one for open pose but cant seem to figure out how to get good results

[@eranfeit]

Hi , very good tutorial . I tried my own image is input for the controlnet with refrence_only , and a simple promt like "man is smiling" in the face are totally different. how can I preserve the face ? Thanks Eran

[@ElHongoVerde]

This is quite amazing. Even better than using LORAs and the chance to combine LORAs, seeds and ControlNet with reference methods, NICE... BTW... I was specting my "Wonderwall" dad joke. I'm very disapointed, mister Kamph (read it in a beautiful british Sean Connery angry tone).

[@sebastiankamph]

You posted it after I recorded this. But I did find it very good! 😂😘

[@bjrnard.d9516]

Waiting for my new computer with beefy vram to arrive, watching your vids to prep, and I'm loving what I'm seeing! Thanks so much for these!

[@Elwaves2925]

If you keep injuring yourself, it's time to book an appointment to learn some shelf improvement. This looks amazing, I keep meaning to look into ControlNet more but never seem to get around to it. Cheers.

[@DedBruzon]

Something wrong, i have latest version of ControlNet, but images come out absolutely different from my control image.

[@Deljron777]

It's still very clunky but we can see the future here. I want to be able to adjust like making a MMO character, then dress how ever i see fit, then put the character in any scene I want, in any pose i want, talking / singing / dancing /what ever. we are so close to that now it is so exciting!

[@wykydytron]

There is controlnet that allows for easy outfit swap, my poor memory can't handle it's name it has 3 versions first end on 20 in name if that helps, anyway it detects what's on picture and paints it in corresponding color, then you just say you want person to have x outfit and it will change clothes but rest will remain unchanged

[@aguyfromnothere]

@@wykydytron Segmentation.

[@Melodias102]

Sebastian big thanks for providing your styles. I mostly use them right at the beginning before even prompting and they provide beautiful results.

[@sebastiankamph]

Happy to help!

[@audiogus2651]

Really curious how this could also work with inpainting and img2img at the same time. exciting!

[@moe_joe_man]

Thanks for the video, I love watching how you present it. Keep it up!

[@sebastiankamph]

Thank you for the support! 😊

[@wrayday7149]

Well, this wasn't quite what I was looking for but holy hell I got something good. I accidently wiped my prompts and I didn't know how to get them back... loading the image into png brought up my prompts/settings. So, thank you for that!!!

[@fireguy9931]

is it just me or are none of these working for me, like the reference and pose combination and everything shown in this video isn't as good as my results

[@TheWorldAccordingToArf]

"I only have my shelf to blame." What a super fine hack joke. I bow, and thanks for the quality info.

[@polystormstudio]

I don't get it. Your character looks very similar but when I try it, I get a totally different person. It's not even a slight difference like in your results, it's completely unrecognizable. I'm using CN 1.1.166.

[@TheFarmingAngels]

me too. it works only with demo picture. with real faces, no way to.

[@mercantigo]

I've been trying this for 3 hours and no good results with a pre-existent image. dont know if this is a mindblowing ...

[@sebastiankamph]

Make sure you update to the latest one. They fixed some of the bugs just an hour ago.

[@QueenAddison]

Same issue here, no luck at all. I even triple check that its up to date

[@neverend9302]

Are you using a png without a prompt embedding? If I use a jpg the generated image is unrelated to the jpg input image entered into ControlNet (though if I put it on ControlNet is more import at least the hair changes color). However if I use a PNG image with a prompt/seed embedding etc then (even though I don't have clean workable results yet) it is reproducing a version of the input Image in Control Net. I am still trying to get decent results...

[@not-alot-of-options]

I updated moments ago, and I'm getting very mixed results. Occasionally it will loosely resemble the reference image, but for the most part it's getting it as close as a police radio description.

[@DedBruzon]

Same thing, bro. My version is latest, but results is very lame. I'm using my wife portrait photos and absolutely different woman appears.

[@eskim21]

6:11 can this be applied to the full body pose? I noticed the generated image only focuses on the half of the body.

[@theaussiegeek]

which one of the videos above explains the style details you refered saying they are in the description?

[@sebastiankamph]

The free prompt styles link. Also top pinned comment.

[@forestlong1]

Control net v 1.1.166 this function is NOT WORKING . There must be some kind of error.

[@mr.entezaee]

If it can reach a stage where the face is completely the same, but in different situations, I think we have entered a sensitive stage.

[@blackvx]

Thank you for this news update!

[@AubreyWatt]

been trying to figure this out. my outputs always look completely different from the reference

[@sebastiankamph]

Try using an input that's made from a similar prompt to the output.

[@davidm8966]

What a gamechanger...my goodness!

[@GuestUser-jf8uj]

it took me 15 seconds to laugh at "I only have my shelf to blame"

[@sebastiankamph]

Haha, but you laughed! 😊

[@letouristedunet3921]

Try the multi ControNet : Unit 0 = depth and Unit 1 = reference_only.

[@leeren_]

How do you access controlnet units on the UI? I don't see those two tabs. Is it another extension?

[@jeremycointin1996]

Love this!!! I need this. Character consistency is my biggest problem.

[@wykydytron]

With faces result vary, from very impressive almost 1:1 copy to completely messy IMG, but if you add proper Lora for that person it has about 90% of accuracy but what i love about it most is that you can use it as style definition, you don't need Lora, just put IMG in style you aim for and your done, doesn't even matter what's on that image it will do great job in doing style. It's also very easy to achieve dark low light images when you put dark low light img as source. Honestly CN will dominate everything untill someone makes similar ai but one that does understand math, gender, individuals and how human body can or cannot band.

[@leeren_]

What do you recommend then for creating new characters that only exist as a single image to start? I was thinking of using CN and cherry-picking the good results to create a LoRa out of.

[@islandersean2213]

my control net model show none, how do i load model into control net? thank you

[@leosieczka3724]

Amazing! Thanks you for making these tutorials

[@cryptolover827]

Many thanks for sharing tutorials, its a massive time saver ;D

[@cloudkicker9593]

Is there some sort of trick to this? Using your EXACT settings and prompts I end up with images that are in no way similar to the input image. I'd have better results leaving controlnet off and using wishful thinking instead.

[@curious_about_AI-we9lt]

Same here. I downloaded the same model Sebastian used, hoping that might fix things, but I got nowhere. Suspect it only works with very specific types of images.

[@saschasutorius526]

I just get this error message every time (mac m1) "RuntimeError: Input type (MPSFloatType) and weight type (MPSHalfType) should be the same"

[@trsd8640]

Deleting the Controlnet extension folder and reinstall solved it for me. Anyway,.. reference_only ist not any alternative to lora or dreambooth. The results are as bad as with img2img. The demo pictures Sebastian uses are not the results you get with other pictures.

[@SouthbayCreations]

AMAZING!! Fantastic video! Thank you for sharing it!!

[@sebastiankamph]

Glad you liked it, you superstar, you! 😊🌟

[@Silver_Nomad]

There's actually one thing, I was thinking about... After version 1.1, ControlNet started to implement something new almost every week. First of all, new preprocessors. So, I'm pretty much curious about, when there is going to be an actual counterpart of Midjourney's Remix mode...?

[@harshvardhansinha5267]

Great videos. But I always have to crank up the sound to max to listen. 😊

[@gjohgj]

This is what I was waiting for! My goodness

[@Silver_Nomad]

There's one thing about this pre-processor... It's more resource consuming. I'm generating a 512x768 image, and setting a Hires Fix to 2x. As soon, as it starts to render a Hires image, "NansException: A tensor with all NaNs was produced in Unet" error occurs. It starts to render an upscaled image only if I lower an upscaler to 1,6.

[@nurlanhacizade]

I've tried a few too. Don't wink at the camera, don't send a kiss. But it doesn't do that.

[@hardflame6486]

Bro is the best, thank you so much for saving a ton of time

[@JonatanE]

I can't get this to work, it just send a bunch of error messages my way and end with ''TypeError: unhashable type: slice

[@monjenvely]

CN1.69, my (shocked woman mouth opened:1.4) = open mouth vampire teeth distorted images.... bruh I had to use canny as a secondary to get it under control

[@shanshanshans5167]

it is a hit or miss. And the hit rate is very low for me...

[@MugiwaraRuffy]

This video metaphorically "saved my life". Because I was just on the look for a way of having the possibility of using the same character over and over without the hassle of creating my own model in dreambooth or whatever this is called.

[@sebastiankamph]

Happy to hear it! Also check my latest video on Roop for this.

[@DIARYOFGOAT]

Doesn't really work for me tbh

[@remi8118]

I actually have troubleshooting with the first part, with reference_only I got an image totally different. Anyone know about resolving this issue ?

[@pablomax9376]

Nice video, but why do you say that the styles are "free to download" when you can only access them after you signup and pay for your patreon? Since you have to pay, that is literally the opposite of free.

[@reallybigname]

I'm actually doing even more crazy things with tile. But yeah, reference ones are great too.

[@rproctor83]

Cant wait to try it, thx!

[@Radarhacke]

Thank you but how can i enable multiple controlnet units? in my version ControlNet v1.1.214 is only one called Single Image. 😭

[@sebastiankamph]

Settings - controlnet - multicontrolnet

[@Radarhacke]

@@sebastiankamph THX, it was alread set to 3, i set it to 2, and now it works.

[@1gagana101]

How can I make ControlNet Units? Mine only shows the CN version and then Single Image | Batch.

[@alexandercato7400]

Settings - controlnet - multicontrolnet: It is set to 3 by default. If you set it to 2, it should work.

[@Distty]

Wow, that is amazing, great video as always.

[@sebastiankamph]

Glad you liked it! 😊

[@SethSmith354]

I just started stable diffusion, how do I get that UI and feature because I don't see it? Is there a video on how to set it up?

[@nqvo]

For some reason, I've never ever been able to generate images with controlnet. Like literally, I'll press generate and it will run but not generate images.

[@sebastiankamph]

Should work pretty well out of the box. Tried reinstalling both SD and ControlNet?

[@TheAquabears]

How did you get your styles menu subdivided like that? Is there an extension that does that or what?

[@phalcon23]

How do you get that expanded bar on the top of the screen?

[@emilsvfx]

sad, dont work for me : AttributeError: 'NoneType' object has no attribute 'to' any idea how to fix

[@ColoNihilism]

10x for the update! Looks reassuring as not to have to learn how to train and fine tune. Wonder if you can just keep using the same reference face with ANY different scenario, thus we got ourself a character mapped by seed only

[@ZENIXTH_KUN]

wow great, by the way what gpu are you using?

[@sebastiankamph]

Thanks! RTX 4090

[@Epulsenow]

Hello Sir, can you help with my query about Hand and fingers deform ho when creating artwork like extra legs extra fingers. sometime the went missing or not corrects as per human anatomy. I tried negative prompts, but still issue remains.

[@FantasyArtworkAI]

Whenever I try this, it works well EXCEPT I keep getting instances where the body of the person looks like it's covered in sand or other patterns. The face area gets cleaned up during the swap and face fix, but the body just gets completely wrecked. The most recent example is it looks like they got wet then laid down in the sand before standing up to take a picture.

[@fred4838]

Thnak you!

[@ShinichiKudoQatnip]

Wow, just wow 😲😲

[@sutek484]

Great tutorial and I have it working but how do i get the ControlNetUnits to show up , they dont appear on mine so i can only use an image or a pose not both?

[@sebastiankamph]

Settings tab

[@victoria1520]

I can't get it working, every time I try to use it I get "ValueError: too many values to unpack (expected 3)", what is this error and how could I fix it?

[@VaibhavShewale]

isnt this already old thing? like 4 week or something?

[@ozstockman]

has anybody figured out while there are no models coming with ControlNet v1.1.234? I tried to use it on this version and nothing worked -ControlNet just ignored for anything(canny, pose etc). I could not select any models for any pre-processor as Models dropdown list was empty. I have downloaded one model for openPose, put it in models folder in extensions. Now I can select this model for pose and it is all started working. I have installed ControlNet from Automatic1111 but it only puts yml files for models in the required folder but not actually models itself.

[@jesontodd2615]

Folks, if I want to generate a character in a background image and have the character blend into the background, how do I do this?

[@shadealarbed1415]

is the og pic was generated using the same model deliberate??

[@lakislambrianides7619]

Do I have to follow this procedure if I want to take one image from image 2 image window and apply on it open pose to get different variations? Or is there a simpler way!

[@RavenMobile]

Your AI did have the "left arm" going up... only it doesn't understand the difference between hair and an arm. You can see it generating what looks like an arm in the air for the first frame, then it updates it to being hair hanging down.

[@TheAquabears]

nuts. I'm on 1.1.174 and it looks like the Unit 0 / Unit 1 feature was removed. I only get a single instance of controlnet now. No way to add an openpose to an existing image anymore.

[@sebastiankamph]

Settings - ControlNet - Multi Controlnet

[@CabrioDriving]

Best channel. period.

[@orgax]

I have "FileNotFoundError: [Errno 2] No such file or directory: 'C:\\New Folder\\tmp\\gradio\\" error, what's can be wrong?

[@OMSHEWALE]

i cant see controlnet option there i m on latest stable diffusion n from India but idk any reason why I cant see it

[@sebastiankamph]

Install from extensions tab

[@dreamersrealm6389]

Do you have a video for training models or how to set up loras and the such? im lil dumb, Ty

[@Archalternative]

Nice video🎉...update...two new choices for reference in controlnet....interesting method to get nice jobs without Lora

[@GlennDeBacker]

Man what is that with the clickbaity titles? Your content is good enough that it does not need it

[@sebastiankamph]

I wish it worked that way. Thank you though! 🌟

[@swannschilling474]

This is the biggest update yet!! 🎉🎉🎉

[@alexanderpina5913]

Hello, how do you increase the importance or weight of the prompt so quickly? Do you use keyboard shortcuts? Ass seen in the video: 02:07

[@sebastiankamph]

Ctrl + arrow up/down

[@daquan99999]

I see new job market opening and old job market closing quickly. New ero, some old folks will be really panic now. I feel them, even though I am not one of them. It might come to my industry soon, who knows.

[@sebastiankamph]

I'm panicking already!

[@Thejigholeman]

How are you generating pics so fast? What kind of graphics card do you have?

[@sebastiankamph]

RTX 3080 when I recorded this. Now RTX 4090. Which do you use?

[@user-pc7ef5sb6x]

Consistent character problem is now solved. I wonder if this will solve the flickering problem in animations too.

[@sebastiankamph]

Let's hope for a big jump in animations soon. It's been a while!

[@pencilgame]

How did you get multi control net? It looks like I only have the one .

[@kritikusi-666]

do you have any tutorials on how to create professional self portraits? I want to look pretty on Linkedin lol

[@dusteye1616]

Thanks, great video and straight to the point. Liked, subbed and commented !!!

[@sebastiankamph]

The holy trinity! You're the real mvp 🌟