ComfyUI : NEW Official ControlNet Models are released! Here is my tutorial on how to use them.

It's official! Stability.ai has now released the first of our official stable diffusion SDXL Control Net models. In this ComfyUI tutorial we will quickly cover how to install them as well as how to use the preprocessors to get things all set for the Control Net node. We will also cover installing the manager, since it makes things a lot easier going forward. I did cover that in Episode 2 in-depth, so if you need a refresher, check out that video as well. Please note, these are now the official models, not the ones released by others.
Want to help support the channel and get this graph? WOOT!
/ @sedetweiler
You can grab the official Control Net LoRA (OFFICIAL) files from here: huggingface.co/stabilityai/co... (Remember, these are Control Net files and are LoRA in name only!)
Grab the manager custom node from here: github.com/ltdrdata/ComfyUI-M...
SDXL model from here (OFFICIAL): (bonus LoRA also here)
huggingface.co/stabilityai/st...
Control Net Preprocessors are available here:
github.com/Fannovel16/comfyui...
Remember, you will have to restart the ComfyUI interface for these to start loading, and the models will also download as they are requested from the Controlnet processor.
Twitter: / sedetweiler
Threads: www.threads.net/sedetweiler

[@s.anonyme6855]

Thank you to spend time creating those so didactic videos. Building up the worflow from nothing, explaining the settings, the whys.. I have been using MJ for months and shifted to A1111 to have more control but ComfyUI is exactly the place I craved all along. And with your videos the learning process is so enjoyable. Thank you. Happy to have subscribed.

[@sedetweiler]

My pleasure!

[@lelio87]

Thanks for the video! I'm just getting started with ComfyUI and your video has helped me a lot! Thanks again!

[@coda514]

You are a wonderful teacher, I not only learn how to use new features but also how they work. Thank you.

[@sedetweiler]

Happy to hear that!

[@SachavaV]

it's a pleasure to absorb knowledge from such a mentor.

[@sedetweiler]

Thank you!

[@iresolvers]

finally someone who can explain comfyui the way I can understand, thank you so much!!

[@AmateurSpecialist]

I think you are the only Stable Diffusion video creator who actually pronounces Euler correctly. My poor ears thank you.

[@sedetweiler]

Dude, I am with ya! .🤣

[@CoconutPete]

do people still prefer Euler over DPM++ 2M Karras?

[@GlassHexagonalColumbus]

Thanks for your lessons. Unfortunately, your videos are not popular among ALL Internet users, but know that for those who share your interests, you do a lot and inspire new things

[@sedetweiler]

I appreciate that!

[@redgear5647]

@@sedetweilerhey bro do you know how can i use the control net in comfyUI to select an image, then using another to merge only the style of the image, without changing pose, details, ecc? Basically a style transfer, thanks

[@COMM1317]

Hey, Scott. Would you consider making us a short video about your Top 5 or Top 10 Stable Diffusion MYTHS (number of steps, schedulers, negative prompts, etc.)? 🙏

[@user-kl1bk4wx4v]

Thank you so much for being out there :)

[@sedetweiler]

Always!

[@eliasdelreal3863]

Happy trees! 🤣 I love it. I think I know who your inspiration for good video tutorials is.

[@sedetweiler]

Yeah, that came out of nowhere! Lol!

[@androgynastronaut]

Very good. I need to learn to keep faces similar in various character images for plot consistency.

[@Vestu]

You are helping so much with these videos. Joined as channel sponsor (it's way way worth it folks)

[@sedetweiler]

Thank you!

[@sedetweiler]

Thank you!

[@UNOCASTILLO]

this is what i call a reeeeaaally good tut. thank you

[@sedetweiler]

Glad you think so!

[@UNOCASTILLO]

@@sedetweiler I would like to suggest you a topic. could you make a tutorial on how to colorize black and white pictures using the new RECOLOR? could you teach us how to use various controlNet modules? Thanks

[@AlexKellyArtUK]

Thanks - very well explained!

[@sedetweiler]

Glad it was helpful!

[@zaffarhayyat3608]

Your tutorial videos on Comfy are fantastic! I'm truly enjoying them. How can I support your channel? Also, it would be great if you could add clear subtitles to minimize any language and accent barriers. Thank you!

[@sedetweiler]

Thank you! There should be a join button under the video or on my main page to help support the channel. Lots of great goodies as well as a sign of appreciation. Cheers!

[@simonmcdonald446]

I have the control - loras but nothing apprears in the "load control net model"box. whats gone wrong there?

[@Mediiiicc]

same

[@norcoreanodelsur]

Hi, this might be a dumb question but why it doesn't work with other checkpoints?, I have a setup to create pixel art, and I get an error in the KSampler "moduleList object doesn't have attribute 1", I tried with different models but the only one that works is the base model. Thanks for the tutorials I am binge watching your videos.

[@Xavi-Tenis]

perfect explanation

[@vilainm99]

Hi, in my controlnet/control-lora directory I only have the rank128 and rank256 of canny, recolor, depth and sketch. any idea what happened? (Windows system here with ComfyUI from Pinokio)

[@aunalise7091]

I like to blend depth and canny or lineart 🙂 But once it is all set up, I can easily choose whether to use all canny or all depth, or a certain ratio.

[@jasondulin7376]

I want to try the same thing. Probably too large a question, but if I wanted to add another controlnet can I just duplicate a few modules and plug them into the same place?

[@sedetweiler]

Use them in a chain. The order isn't important.

[@uk3dcom]

Might be worth mentioning the clone will take a while and the comand window won't show progress continuously, you just have to be patient and wait for the Filtering content line to hit 100%. First time round I quit out of the cmd window to early. 🙃

[@sedetweiler]

Thank you, that is a good observation!

[@seanpretorius718]

I really struggled with this part. I was running Comfy via Swarm. When Control-Lora loaded, and Terminal/PowerShell said "done," I thought I was finished. I shut it down and restarted Comfy, hoping it would work. However, I didn't have any models when I restarted Comfy, and I started overthinking things. Despite having read this thread earlier, I continued to overcomplicate things. After several hours of installs and uninstalls with no success, I took a lunch break, and then this comment began to blossom in the overgrown garden that is my brain. So, thank you for this. Also Big thanks to Scott Detweiler for these fantastic tutorials!

[@mhill88ify]

Good shot m8, I totally did this also.

[@mccloudmedia2686]

You should highlight this comment, I must have closed too early also and the controlnet models where empty.

[@lioncrud9096]

How would you set up multiple control nets? I tried to chain 2 Apply Control Net Nodes together but I think it’s not working. For example, let’s say first Apply Control Net is for a depth map, set to run from 0.0 steps to 0.2. Then, I chain that into another Apply Control Net that has a Canny, and set for start at 0.0 steps and end at 0.8…….wouldn’t the most recent Apply Control Net steps settings override the first Control Net? Since I passed the first into the second?

[@Renzsu]

Scott, do you know if there are plans to bring the lineart preprocessors to SDXL? I'm a product designer and for me it's one of the most exciting things about SD. Load in a clean sketch and see where SD takes it. I need more detail though for things like bicycles.. SD1.5 does cars really well, but fine details get lost and that's what makes up most of a bicycle :) I'm trying lineart with SD1.5, then taking the output of that into depth and XL, but it's a bit of a roundabout way of doing things with less accurate results than I'd like.

[@SaadKhanAhmed]

I'm using SD 1.5 and using controlnet to give my character a pose as you described (without Reactor). Can I somehow control and define my own background that the subject (my character) blends in?

[@brandonflores4]

i get the following error when it reaches the ksampler node. The ksampler is also outlined in purple at this step/error. Any idea how to fix? Error occurred when executing KSampler: 'ModuleList' object has no attribute '1' EDIT: was using the soapmix load checkpoint from earlier videos, unsure why that one did not work. reverted to the SDXL load checkpoint, problem solved.

[@jac001]

Nice job, appriciate it. 😀

[@sedetweiler]

Thanks! 😃

[@Potts2k8]

I'm hoping you can help, I'm trying to use the ControlNet Repository and OpenPose - bit I keep running into errors when at the Control Net Loader step, something about py lines 151, 81, etc. I've tried updating and reinstalling control net repository but I'm still running into the same issue.

[@OneMinuteMeds]

Now we cookin with FIRE!

[@sedetweiler]

Heck yes!

[@MacS7n]

Thanks a lot. Please talk about the pipe in your next video.

[@sedetweiler]

Sure thing!

[@royjones5790]

is there any stereoscopic/stereogram creating depth map in ComfyUI yet?

[@alecubudulecu]

small suggestion with these tutorials. and particularly these intro tutorials - slow down. like... a lot. I totally get you know what you doing... but most of us have to switch screens and hunt and peck for where things are... we don't even know menu buttons. we have to pause the video every few seconds.... rewind constantly. especially at the end... you move so quick. and I say that - because your tutorials are excellent. you communicate well and show very good details of what you are doing....just so fast! haha

[@sedetweiler]

Yeah, sorry about that. I have found that if I am to slow, people just leave the video, so I tend to just get it down to the actual points you need to know. I often suggest watching it first, and then doing it again with pauses as needed. I do the same thing with tutorials on products like zBrush, as those can be brutal to keep up with, so I totally understand what you mean. Since you are a subscriber, you will also see a recent live stream and you might really enjoy the speed of those. Thank you for your support!

[@user-yo8pw8wd3z]

Can I ask, since we are using SDXL, why is the standard Clip Text Encode node used instead of the CLIPTextEncodeSDXL node?

[@sedetweiler]

I just didn't need both G and L for this demo.

[@STaSHZILLA420]

Hi there. I used the manager to download comfyui-extensions to get the custom color nodes and titled reroutes etc, and it isnt workig. The extra options didnt even show up. Is there something i did wrong? Is there a fix for this or a step to solve this? thanks in advance.

[@WARweezy]

thank you!!!

[@sedetweiler]

You're welcome!

[@Nexus-16]

Hi i cannot get past the install custom nodes bit this update sign comes up and then just doesnt disappear what can i do???

[@dasmatthews]

Hey Scott and community! First up, thank you so much for the very well and not too quickly explained tutorial!! I'm having a little problem though and I don't know exactly what to change/do to correct it... At 8:19 you're showing the long list of ControlNet Models and you say that we'll see the list if we did it correctly and I followed your video step by step but I still don't get that list... any idea or hint on what might've gone wrong? Thanks so much in advance!

[@CamiloMonsalve]

Awesome démonstration. Do you have any playlist to start from baby steps?

[@sedetweiler]

Yup! kzbin.info/aero/PLIF38owJLhR1EGDY4kOnsEnMyolZgza1x

[@miguelfigueiredo9814]

im using the comfyUi portable version and when i clone the comfyui manager to the custom nodes it wont appear the manager button on the ui

[@arvinsim]

I am getting a conflicted nodes warning when I search for that model in manager. What should I do?

[@TomasWittelsbach]

a touch-off subject, in the sdlx UIs, there is a tile button; where can I select the ability to make seamless tileable textures in comfy? I'm a sculptor seeing if I can maybe train a checkpoint or lora or something to create tiling patterns like snake scales and hopefully train in on alphas or depth maps so we can create interesting new stuff as opposed to my students always downloading the same old crap. I can imagine snake scales with mouths or eyes on them. but no one is training texture models. thank you t p.s. Thank you greatly you are making it possible for someone who can't c-prompt his way out of a paper bag to explore what these new technologies can take our art. I went from my best tools being an 800-year-old set of chisels to a computer when I saw Zbrush 18 years ago. Your videos are wonderful thank you again

[@sedetweiler]

Hmm, I know we have some methods for doing that in-house. Let me see if there is a method I can get out here that you can use.

[@TomasWittelsbach]

I joined last night, I don't want to take advantage of your generosity. I teach Zbrush if I can figure out how to train texture and seamless tileable texture models or Loras or something lol, it will be an amazing tool. I'm so excited I found your page. such powerful tools. thank you greatly @@sedetweiler

[@timreutemann9223]

Thanks for that. There is still one big big missing piece for narrative work with SD - namely LORAs for the refiner - we can get consistent characters with base LORAs via Kohya-ss, but refining screws them up, so quality of custom characters is barely better than SD1.5 at this point...

[@wsippel]

I use Impact to fix that. Generate the image as usual, with the refiner, then run it through the Impact Face Detailer and regenerate the face with just the base model and character LoRA, without the refiner. The nice thing about Impact is that it detects the face automatically, so once you have your workflow set up, you don't have to worry about it anymore. Just let it do its thing.

[@timreutemann9223]

​@@wsippel Thank you for the idea! Will give it a try tonight.

[@rishatpepsi4333]

Hey thanks for the tutorial! I have a question: i have no problem using this with SDXL, but the moment i try to pair it with sd 1.5 checkpoints comfyUI sees an error It says: Error occurred when executing KSampler: 'ModuleList' object has no attribute '1' What should I do?

[@sedetweiler]

you cannot mix the 2 models or loras. 😞

[@rishatpepsi4333]

@@sedetweiler Can I use controlnet with sd 1.5 checkpoints in comfyui and if so, how?

[@VenkmanZero]

7:54 Late comment, but you actually can override the default css font-size: values in ComfyUI/web/style.css by adding your preferred settings in the ComfyUI/web/user.css file. I did this once for the multiline textboxes to make them more legible on zoom out but it was awhile ago and I've reinstalled ComfyUI once or twice since then so I don't have that modified user.css anymore. Was only a matter of copying and pasting the relevant style section from styles.css into user.css and then modifying the part I needed to change.

[@francotobon]

I'm encountering an issue while trying to use the 'INTER_LINEAR' attribute from the 'cv2' module in my Python code. When I attempt to execute my script, I receive the following error message: Error: module 'cv2' has no attribute 'INTER_LINEAR' I'm using OpenCV (cv2) for performing image processing operations, and I need to use the 'INTER_LINEAR' method for image interpolation in my project. I have checked the OpenCV documentation, and I know that this attribute should be available, but I don't understand why I'm receiving this error.

[@corknakovastein]

will these work for automatic 11? Both of my gaming rigs im unsuccessful at installing warpfusion which I want to use for better consistency in videos, especially to redo the last realism test I put on my youtube, as well as comfyui I cant install. I always run into some sort of issue with the installation on both

[@sedetweiler]

I think in the beta, but I have not checked with the most recent pull.

[@CalebHoernschemeyer]

Hey Scott, great video, i got most of the way there but keep encountering a "header too large" error. Running Apple Macbook Pro M2 24gb Ram. Is this hardware related? Im searching for a lighterweight model with little luck. Control net works, and I can run stable diffusion with control net on the webui, but not ComfyUI. Followed your tut to a t twice. Any direction or resources you can reccomend would be great, thanks for the informative video!

[@basti11097]

I´m always getting an error on the ControlNetLoader saying Error occurred when executing ControlNetLoader: 'input_blocks.0.0.weight' any idea what that could be. Setup should be the same. Thanks! Awesome Tutorials by the way!

[@christophermoonlightproduction]

You lost me at the 15:55 mark. How do you place a model in the folder with a code? I'm equally confused as to what you were doing in Notepad. Is there a walkthrough where I can see it being done? Thanks.

[@jasondulin7376]

Thank you so much for these wonderful tutorials. I'm very new to this and have a newb question: Can this same workflow be used with other models? Any other model I try gives me a litany of errors, the first being - "Error occurred when executing KSampler: 'ModuleList' object has no attribute '1' Should I be changing a setting in the KSampler? Thanks!

[@sedetweiler]

Yes, the workflow will be the same for the models. If one is giving you an issue figure out if it is the preprocessor, controlnet model, or the inference model.

[@jasondulin7376]

@@sedetweiler thanks so much!

[@TheKMB787]

@@jasondulin7376 Did you ever figure out what the issue was for you? I get the same error, I tried changing everything @sedetweiler mentioned and nothing helped or even changed the error.

[@ruihan6583]

hello! same error with you, How do you fixed it?

[@joeyhandles]

easy to follow very helpful euler gang euler gang

[@sedetweiler]

woot! woot!

[@yoniwoker]

Friend, I want to add realistic lights and shadows to a photo, without changing the characters, is it possible?

[@stef2020]

Is it worth switching from a1111 to comfyui regarding performance? I am working with sd1.5 in a1111 but I want to try sdxl. So maybe a good Point to Switch.

[@sedetweiler]

I find it much faster since it is only loading what you need and not that huge interface.

[@bordignonjunior]

hello Scott, how can y ou make the reroute nodes get the name of whaterver they are rerouting?

[@sedetweiler]

right click on the tiny dot and choose rename. Mine are that way because of the pythongossss node set I believe.

[@Darkfredor]

Waouu,super merci c'est genial

[@0A01amir]

Thanks for these awesome tuts. Why don't you use Link Render Mode: Straight ? it fixes the noodle problem. it's in the option (gear button on top right of Queue Prompts). I installed a bunch of custom nods now SDXL1 wont work anymore, before it was slow now i'm getting not enough memory error. well i guess XL is no for me.

[@sedetweiler]

Yea, I used straight for a bit, but prefer the noodles :-)

[@kduxyz]

hi, you can help me? Error occurred when executing ControlNetLoader:

[@ronnykhalil]

wonderful news!

[@sedetweiler]

:-)

[@MultiOmega1911]

Good tutorial! I got a workflow that mixes SDXL with SD1.54 (i think as a refiner), does it makes sense? I use it like "a giant lora" as i make the denoise super low so it changes the features of the image with the model i choose but not changing the image as whole (es. A-Zovya has a particular style i like so i create an image with sdXL model i like, then the refiner will implement some features from the Zvya 1.5 model)

[@sedetweiler]

Sounds like fun! That is one of the reasons why I love this node method so much, as you can do whatever you want!

[@38dedo]

thank you for these videos but i need a little help! im a 3d artist that is transition to learning some stable diffusion, im an absolute boor when it comes to python, i use gpt to write maya scripts for me because looking at code makes my head spin. anyway, when i try to use the 2 pose recognition nodes installed in this video, one return an error that im missing something called "media pipe" and the other node says im missing "matplotlib". how can i install these? the manages doesnt show anything missing, and before this video i didnt even understand how to install stuff from huggingface or github because im used to a big green "download" button. cheers for the help :)

[@sedetweiler]

You might want to ask on the git for the nodes throwing the errors. Issues are so unique it would be hard to troubleshoot.

[@alexlindgren1]

Would this work for replacing for example a sofa in a room with another sofa? Or would you suggest some other approach?

[@sedetweiler]

Sure! That would be a good example.

[@alexlindgren1]

@@sedetweiler alright, but how would one know what is the sofa in the image and replace that part and not something else?

[@carstenli]

Thanks for the great tutorial, Scott. How to get the VAE label on the reroute node btw?

[@sedetweiler]

It is automatic with some of the latest updates.

[@CPDCT]

you can right clock on the reroute and set to 'show type by default'

[@MVLonden]

Im stuck on getting my loras into the control net. I have git LFS installed. But when I past into my control net folder. Nothing happens. Am I doing something wrong?

[@miracletraveler2835]

I’m new to this so I’m not sure what you mean but I managed to get a file called “control-Lora” to appear by opening in terminal and using the git clone

[@ahmedaborady7883]

in Load controlnet model node i get this Error that i can't understand why Error while deserializing header: HeaderTooLarge I did everything i reinstalled comfyui :) I'm using same model and same controlnet-lora you're using here any idea what this could mean?

[@InfiLoopAlchemist]

same problem you find anything?

[@NHCH]

thanks dude, ur a dude!

[@sedetweiler]

Sweet!

[@TheRMartz12]

Hey can you help me out? I cloned the repository with git clone like mentioned at 3:40 and everything was cloned except for the models 😓 can you tell me why this is? I always have to download models manually because of this

[@TheRMartz12]

I have been trying to use any depth preprocessor and for the life of me I cant! I don't know why. i've been searching all the tutorials. I have all the auxilary preprocessors and the models in the right folders and it keeps telling me I dont :(

[@CerbyBite]

Even if i run a workflow from the control Lora folder I get the error: 'CFGDenoiser' object has no attribute 'model_config' on the Ksampler. Can someone help? But 1.5 controller models work.

[@sedetweiler]

It isn't a LoRA in the traditional sense. In the video I show where to place it.

[@CerbyBite]

@@sedetweiler Yeah, i got that. The control Loras are in my a1111 installation/extension/model folder. Comfyui got the path to that, and 1.5 Controlnet files work on the XL model and vae, but the control lora gives the CFGDenoiser error. Welp, it's something that i don't need/want for now. Maybe it will fix itself with an update or something.

[@kallamamran]

Why does the section in extra_model_paths.yaml say a111 and not a1111? 🤔

[@gregoryh2os]

Where can I find help with a Depthmap error?

[@user-sl3ri4lt9l]

hi can someone tell me how can i add multiple preprocessor ?

[@Doomer35]

Forgive me I may be a bit slow but when I type in git clone and paste link in the cmd window it says "git not recognized as an internal or external command" etc. I also don't have admin privilege in the window, how do I fix this?

[@sedetweiler]

You will need to install git. Most of us already had it, so I made an assumption here that was apparently incorrect.

[@Doomer35]

thanks@@sedetweiler

[@eliasherreradg5966]

How do I apply this same tutorial but for openpose?

[@joesheble5458]

I have followed this to the letter but when I connect my ControlNet node to my KSampler and run it, I get an error that I have no clue about, as I'm just a user not a developer... Error occurred when executing KSampler: 'ModuleList' object has no attribute '1' Any ideas at all as to who I can ask, or what I can do?

[@sedetweiler]

what kind of control net are you using?

[@joesheble5458]

@sedetweiler The exact same ones shown in the video. For the Midas depth map, but also tried the Zoe as well.

[@Raulikien]

You are using a SD 1.5 model, control loras are for SDXL. You have to download a control net model for 1.5, from lllyasviel/ControlNet-v1-1 Hugging Face

[@SergeGolikov]

1:18 Struggling to understand the very first part, command line prompt? can you slow down? thanks 🍷

[@TransformXRED]

I would love to have the ability to split the screen, and then have one part that is fixed (on the image generation node), and the second part where we can move freely

[@sedetweiler]

Oh, that would be pretty sexy!

[@TransformXRED]

​@@sedetweilerBtw, I remember you talking about chaiNNer a while back. The recent versions have the ability to use automatic1111, and nodes specific to stable diffusion

[@sedetweiler]

Interesting! If you check the ComfyUI folders, you will also find chaiNNer is part of it's implementation as well.

[@galdocstutorials]

You mentioned Blender -- here's a fun question: do you happen to know if there's ever going to be plans to port ComfyUI nodes in Blender? I recognize that these two programs both are VRAM hogs and this might limit the utility of it for a lot of people -- myself included; I def need a new card, hah -- but there seems to be a TON of potential for workflows, particularly with both the depth and normal maps, but also more creatively with the other bits, like rigged poses, training it to make textures for UVs, etc. :) Thanks!

[@sedetweiler]

We do have a stability API tie in for Blender, so we can get part of the way there. We do have some new goodies coming that I think you are going to enjoy!

[@Juliangonzi]

Conflicted node warning when installing that Controlnet thing, why?

[@MLKL2011]

amazing.

[@sedetweiler]

Thank you! Cheers!

[@SaadKhanAhmed]

Hi @Scott, your videos are amazing! I am very new in this space and trying to do a PoC where I need to take person's photo with clear face and use the face and give a slightly different expression (maybe use controlnet openpose or canny) and more importantly create a comic style. I know we can do really good comic style using SDXL 1.0 but I can't use my custom face in such situation. Is my only way to achieve it is using training model or lora? Can you please guide me?

[@sedetweiler]

You can try roop or training a custom LoRA. You have a few options there, but roop or reactor would be your best bet.

[@SaadKhanAhmed]

​@@sedetweiler , I am currently trying to use controlnet canny with SDXL 1.0 and positive comment "color comic book..." and it seems to work ok-ish. So roop/reactor would make sense to put before this flow to swap person's face into the picture and pose I want it to be and then do the same process? This eventually has to be hooked up on a website for people to try so training a Lora would be time taking process and probably not the best too?!

[@sedetweiler]

I am sure those would work. However, be careful what you allow people to use as training material!

[@Skettalee]

I get so lost trying to follow stuff like this. Mine has errors all over the place, both prompts and the ksaple and vae all are red and i seem to have all the same things you got. Where do i get your workflow files?

[@sedetweiler]

Be sure to change the model to things you have on your machine. The graphs for all of the videos are available on the Community tab here in KZbin for channel sponsors. Have you done the earlier videos? This is not a good one to start with, as this is assuming you have done the previous episodes.

[@Skettalee]

i have at this point, thanks man! @@sedetweiler

[@DarenEilert]

How about a quick basic overview on recommended environment, directory structure, to manage in an efficient way - storage, update mgmt, image mgmt, workflow...

[@sedetweiler]

Sure! I can do that. I use a NAS so not sure how many others will be interested in using what I have.

[@CALIGARIS-xe1pp]

Hi, I followed all your steps for installing clone models, but even after copying "clone this model repository " in huggingface (should I paste it somewhere?), then I can't find them in "controlnet" folder, but i only have the file "put_controlnets_and_t2i_here". What did I do wrong? Thanks if someone will answer me.

[@sedetweiler]

You need to download the models from huggingface and put them into the controlnet folder next to that "put_contronnets_and_2ti_here" file.

[@CALIGARIS-xe1pp]

Be patient, but for me this is a completely new thing, where do I find the files to download, can you give me the link please? Thank you@@sedetweiler

[@thehowlingfantodz]

Check again, mine too a while to load in folder, but let me know if they show up in controlnet checkpoint loader mine didn't@@CALIGARIS-xe1pp

[@CALIGARIS-xe1pp]

Sorry but i didn't find the files "control-lora and controlNet-v1-1 in hugginface, please could you help me with a link to download them? Thanks@@sedetweiler

[@dirolawal8415]

would be much appreciated @sedetweiler @@CALIGARIS-xe1pp

[@MartinDenStore]

I had some issues getting the Load ControlNet Model node to work, as well as getting the Apply ControlNet (Advanced) node to show up. I managed to solve them, and if anyone need help, this is what I found out: You have to update ComfyUI, and recreate the node. ||| Updating wasn't as simple as running update_comfyui.bat; I also had to handle a merge conflict. I asked ChatGPT, and it gave me two commands to run from the command prompt in the \ComfyUI folder: "git fetch" & "git reset --hard origin/master" ||| After that, the update_comfyui.bat worked again (though I'm not sure it was necessary to run it afterwards). ||| And as mentioned: Recreate the Load ControlNet Model node after the update and restart of ComfyUI - it seem to have changed. Cheers!

[@sedetweiler]

Just use "git pull" when you want to update. It keeps you at the current versions, which my videos will always use unless otherwise noted.

[@subcreation4341]

@@sedetweiler I have the same problem, but they never load regardless of what I do. I tried "git fetch" & "git reset" and "git pull" (Already up to date). I see the files are where they belong but they will not load in ComfyUI. I've restarted ComfyUI dozens of times and restarted my computer several as well. I tried putting them both in comfy and a1111 (which is where its configured to point to in extra_model_paths.yaml, and that's where all my models and loras are loading from). Nothing works

[@___x__x_r___xa__x_____f______]

UPDATE: Found that the error came from my cuda12.1 installation. I reverted to 11.8. All good thanks: Hello Scott, when I install Controlnet_aux i get dw Pose error, unable to install, is there a known issue? I tried installing both from Manager and git clone.

[@sedetweiler]

I think DW pose uses onxy, so you might want to check their git for install tips.

[@yoniwoker]

friend I have extra_model_paths.yaml.example but I don't have extra_model_paths.yaml. What do I do, where can I get off?

[@sedetweiler]

You rename the .example file and edit it so it is pointing to the correct place. The example file is just an example.

[@blacksage81]

I got it working just fine, and I did a test gen with the Canny model, and I was wondering what was taking my render so long... I realized that I loaded 6gb diffusion model and not the control Lora. I once again was setting my laptop's gpu on fire.

[@sedetweiler]

Don't start a fire! :-)

[@opensourceradionics]

Why is it not possible to use ControlNet with other models than SD?

[@mkthedoctor]

I get an error at 6:13 that it can't find a file in custom_nodes\\comfyui_controlnet_aux\\ckpts\\lllyasviel/Annotators\\cache\\models--lllyasviel--Annotators\\snapshots\\982e7edaec38759d914a963c48c4726685de7d96'.

[@-Belshazzar-]

hey thanks for the video, unfortunately the git clone command dose not download the models themselves. Also a bit confusing is the the fact the old controlenet in AUtomatic1111 is actually installed under extentions > controlnet and not at Models > Controlnet

[@sedetweiler]

The models get pulled in for the preprocessors when you use them the first time. The scripts and the nodes are pulled in with the git.

[@MartinDenStore]

It did install them for me, but it took a long time, and the cmd window didn't update for like 10 minutes at one point.

[@MartinDenStore]

I don't have the Apply ControlNet (Advanced) version. Only the non-advanced version. Any clue? Thanks for the videos btw! :)

[@MartinDenStore]

Erm nvm, turns out the ControlNet model loader doesn't work for me either, so it doesn't matter. :P

[@MartinDenStore]

I'll add this comment here too: I had some issues getting the Load ControlNet Model node to work, as well as getting the Apply ControlNet (Advanced) node to show up. I managed to solve them, and if anyone need help, this is what I found out: You have to update ComfyUI, and recreate the node. ||| Updating wasn't as simple as running update_comfyui.bat; I also had to handle a merge conflict. I asked ChatGPT, and it gave me two commands to run from the command prompt in the \ComfyUI folder: "git fetch" & "git reset --hard origin/master" ||| After that, the update_comfyui.bat worked again (though I'm not sure it was necessary to run it afterwards). ||| And as mentioned: Recreate the Load ControlNet Model node after the update and restart of ComfyUI - it seem to have changed.

[@sedetweiler]

Use "git pull" when you start your new session. Using an old build will probably not go well.

[@bluelantern5241]

I just installed comfyui. I'm having difficulty finding a tutorial on how to train my own images for use. Do you have a tutorial for that?

[@sedetweiler]

I don't use comfy for training, just for inference.

[@bluelantern5241]

@@sedetweiler is there a way to drop in a random photograph and have comfy tweak it?

[@caricaturecebuAI]

Great Tuts, but unfortunately mine always got this error: input_blocks.0.0.weight'

[@sedetweiler]

Sounds like the preprocessors. I would restart comfy and it should locate the missing model.

[@saymen881]

What if i dont see the giant list at 8:19?

[@katjanka22]

Can you make a tutorial how to solve Clip Vision not appearing in preprocessors? It is something about yaml changing names. But No video about it. I’m Lost. 😄

[@sedetweiler]

Clip vision has it's own loader

[@pogiman]

cant find your video that made the Manager button appear

[@victorhansson3410]

You're watching it..

[@pogiman]

@@victorhansson3410 thanks man.. I totally missed it... rewatched it and saw it..

[@sedetweiler]

glad you got it working!

[@tiffanyw3794]

I have a1111 and a 4060 Nvidia graphic card takes 45 sec to 1min for images . Is comfy faster

[@sedetweiler]

Yup! It's a lot leaner application. It won't be a ton faster, but as you get your workflow in place, you can optimize it. For example you might find you can cut the steps on your first pass, or between control nets.

[@Grit_Ai_Art-vq8xb]

❤ty

[@sedetweiler]

yw!

[@lucademarco5969]

Hi Scott! Is it possible to do face swap with comfyui? If yes, can you a,e a vjdeo explaining how? Thanks in advance!

[@sedetweiler]

Yes, there are quite a few ways. I have a tutorial in the works for that!

[@lucademarco5969]

@@sedetweiler Thank you very much!

[@tompom3513]

[@sedetweiler]

:-)

[@97BuckeyeNut]

I think we need a video showing how to add multiple LoRas to a ComfyUI workflow.

[@lukeovermind]

Comfy Roll Nodes has a Lora Stacker

[@sedetweiler]

It's super easy. Just chain them together

[@LouisGedo]

Hi

[@sedetweiler]

Well, hi there!