ControlNET 1.1 - What you NEED to know!

[@neverend9302]

Thanks Olivio, this helps me so much being able to see someone working with these systems in video format.

[@sebastien6592]

Hey Olivio, Several methods allow to use the "Tiling". In my eyes, the best remains the "TiledVAE" function of the "Multidiffusion" extension, a marvel, usable in all cases, and with the txt2img, it is usable with the HiresFix It therefore allows to generate a low resolution image (with or without ControlNet), then to keep the seed and check the HiresFix. It avoids passing on the img2img

[@funkfz6312]

does this work on defoum??

[@OlivioSarikas]

#### Links from the Video #### Controlnet 1.1 github.com/Mikubill/sd-webui-controlnet Controlnet 1.1 Model Download: huggingface.co/lllyasviel/ControlNet-v1-1/tree/main Controlnet Models explained: github.com/lllyasviel/ControlNet-v1-1-nightly/blob/main/README.md #### Join and Support me #### Buy me a Coffee: www.buymeacoffee.com/oliviotutorials Join my Facebook Group: facebook.com/groups/theairevolution Joint my Discord Group: discord.gg/XKAk7GUzAW

[@relaxation_ambience]

Please, Olivio, could you stop making enormous sudden screen zooming ? I understand, that you think that it's better somehow, but in reality it's directly opposite. It confuses a lot, while when you see full screen as you, it's easier to find things.

[@SpiralingOut23]

Very cool - thanks for the update, the new open pose stuff is pretty sweet!

[@OlivioSarikas]

Thank you. Was a lot of fun to experiment with this

[@mexa31416]

I think this video is one of the most relevant ones regarding the latest ControlNet update...👌🏻

[@steffen_r]

Thanks for the video. I just tried the tile_resample in combination with Ultimate SD Upscale. The results are really stunning. It's crazy how many possibilities you have in SD/ the webui. That's being said after playing with SD since last friday ;) (05/05)..

[@athossimoes]

Thank you for every video you make. You are helping me a lot. I can't pay you a coffee in this time, but some day i'll. You deserve it, thanks.

[@rproctor83]

Very helpful, thank you! I literally just installed controlnet a few days ago and was wondering why I could not find information on some of the features, didn't realize it was so new. I also didn't realize the models were new, so I need to double check I have the correct versions. Thanks!

[@OlivioSarikas]

Thank you. Happy i could help

[@rproctor83]

​@@OlivioSarikas I seem to be having problems using the tile model. I just did a fresh install to make sure I wasn't missing anything, updated to all the latest models. Following your instructions, double and triple checking the settings, model numbers, etc, I can not get it to work. - render an image, 512 x 512, send to inpaint. - Set inpaint to 1024 x 1024, denoising 0.5 - Controlnet, enable, pixel perfect, processor tile_resample, model control_v11f1e_sd15_tile, weight 1, weight 1, down sample 2 No matter what I set the denoise and downsample to, the image never changed except making a larger lower quality version. Thought maybe I needed inpaint not masked checked, which does make changes, but it has same effect as img2img, which just makes the image appear blurry like it took a blurry version of the image and just overlayed it on top.

[@nachoferrando3591]

@@rproctor83It's happening the same to me. Pls help.

[@rproctor83]

@@nachoferrando3591 Unfortunately I was not able to fix it, may try reddit here this afternoon.

[@rproctor83]

@@nachoferrando3591 I've resolved the issue, however I am unable to use it as described in this video. I've not found another guide describing this method, only bits and pieces of people discussing it with no discernable instructions. I could only get it to work under these conditions. With denoising set to 1 (for clearest impact, but you can use whatever): - Either normal img2img (not inpaint) with Resize mode set to Just resize (latent upscale) Provided the best results. - Or with the Ultimate SD Upscale script. When using the script the tile model will (more) accurately generate the tiles for the upscale. Without controlnet the tiles are typically far removed from each other (too random). It can seemingly produce higher detailed results, but the seams can be a killer if they land on say, an eyeball.

[@nealmenhinick]

Awesome these vids always ensure I install updates correctly! can't wait to use the version

[@OlivioSarikas]

Thank you, my pleasure :)

[@steve09219]

at 1:08 you typed with underscores. they should be dashes. it kinda tripped me up for a couple minutes. beginner here! so dont know if someone else might get tripped up as well.

[@AlexS3]

Would love to know what Checkpoint you're using for this

[@thanksfernuthin]

Having the eyes look at the camera to show it tracked the eyes are the worst example you can pick! I have to beat my computer with an axe to get it to show someone looking at anything BUT the camera. Everything else looks great. Can't wait to try it.

[@kalleyneves2]

nice job olivio this new update 1.1 is amazing

[@100_Percent_Best_Memes_Ever]

Amazing. I'm so excited. Great video, and so succinct.

[@TheDontletmedie]

Olivio, you do an amazing job with your videos! Thank you so much

[@RJMCTV]

at 12:30 I cannot find the _tile model anywhere. It is not in my controller install, and not available to download from Huggingface controlnet models

[@chrislannon]

Olivio, Thanks so much for doing such great content! Very helpful. So exciting to be learning this stuff!

[@szzzak999]

Thank you Olivio for your work. A lot knowledge.

[@OlivioSarikas]

My Pleasure :) Love to share all these infos with you :)

[@karter.design]

As always, great content and tutorial, thanks for posting!

[@dziku2222]

Not working for me - when I put input image for preprocessing, I end up with black square as a preview instead of face dots/body joints. Help!

[@digitalpabs]

Amazing amazing, thanks for all your explanations

[@nalisten]

Im definitely Trying this Olivio🙏🏽🙏🏽🙏🏽

[@hoks888]

at 11:55 you have selected "inpaint masked" but you havn't masked anything, I selected "inpaint not masked" and it worked, with "inpaint masked" selected it wasn't doing anything

[@Amelia_PC]

The "explosion" button is a blessing for low VRAM graphics cards like mine.

[@Hatkld]

I got a small problem when I want to use the model for ( ControlNet) I can't find it, it says ( none ) even though I already downloaded the models so what should I do?

[@xd-vf1kx]

Oli strikes again! thanks for this hack!

[@RamonGuthrie]

Heads up some negative embeddings might cause errors, so remove them from your prompts before you use the Tiled preprocessor with controlnet

[@AB-wf8ek]

For the inpaint tile controlnet upscale trick, it didn't seem to work for me unless I switched the mask mode to "Inpaint not masked"

[@TrueMeridian]

i was wondering if you can take a nice game illustration and render it to look more realistic, or with more detail.

[@cdrocrossdiscovery]

Mr. Olivio, can you control and adjust skin tone this program?

[@ARNABOSS]

AI is result of human endeavor for excellence thanks for making these concepts easily understandable for all humans

[@sebastianarriola4369]

thanks for your content, its awesome

[@cleanartcollective]

For some reason, i followed your instructions on how to install controlnet, but when i use the drop down menu to select a model, it shows that i have none, even tho i go to the folder and it shows all of the models that come installed. i can't seem to get it to show me any even after reinstalling and redownloading models??! any suggestions?

[@HarryXiao88]

What exactly does the "tiled upscale method" do better comparing to others? And how to fix the obvious "hand not recognised" problem?

[@Mocorn]

I'm also curious about what the difference in the tile upscaling actually is.

[@matthewjmiller07]

I got ControlNet 1.1 but previews aren't working for me now

[@unisolitario]

I was animated to update the automatic1111 in my 4 Gb VRAM notebook and run the ControlNet 1.1 to note this new update simply can't run in my machine. Now I want to know how I can return to old version. Otherwise, I will no longer be able to run Stable Diffusion again.

[@olternaut]

So what is the difference of using Controlnet on txt2img vs img2img? You did not make that clear in your video. In fact, you had the video so zoomed in that I couldn't see what section of the GUI you were.

[@BecauseBinge]

Using ControlNet with img2img tends to retain colors, style of the input image and using it with text2img it generate vastly different styles compared to the input image (using only the pose data as the reference).

[@calvinnguyen1451]

Olivio: thanks for this awesome tutorial. At 14:32 mark, the upscale image has even more warped hands, still with 6 fingers like the original image. Would inserting the "bad hands" negative embedding fix this problem?

[@Leo-qc004]

Thank you, but after testing, I cannot use the tile option of the controlnet to get a high resolution result, the image got 2x, but the quality is bad, don't know why

[@OlivioSarikas]

it seems to be buggy at the moment. that is why i used in my video the inpaint. try instead to use the img2img. try to restart A1111. make sure controlnet is enabled. try to load the image into controlnet too and see if that helps

[@Leo-qc004]

@@OlivioSarikas I've tried with the latest version of A1111 and Vlad webui, this method doesn't work on my local at all.

[@OlivioSarikas]

@@Leo-qc004 update Controlnet again and try again. it's buggy

[@uk3dcom]

Whatever these new control net models have changed is resulting in a massive improvement for many of my renderings, allowing me to get even closer to the output style I'm chasing. The upscale trick looks to work for me too though I am now looking to Adobe Photoshop's neural filters for scaling beyond the render size in Auto1111.

[@OlivioSarikas]

also check out my "ultrasharp" video on that

[@dikmugget]

@Weird Lighthouse yes. read the developer's notes on how further training and refinement of the models has removed many of the old controlNET issues

[@godned74]

What I find the most interesting you can achieve all these great results with 1.5 if you mess with the prompt, steps resolution and so on. It seems the basic stable can do all the above , Just more tricky to figure it out.

[@tdfilmstudio]

Tiled controlnet still not working for me, using your method. Everything else works just fine.

[@HanAdventures793]

I followed your instructions step-by-step but when you took me to the Models drop down list at 2:46, there was nothing in that list for me. What am I doing wrong? Please help!

[@arturocrazy]

Try using the refresh button next to the model drop down list (the blue square with 2 arrows making a circle)

[@OlivioSarikas]

yes, refresh or restart A1111

[@BritBox777]

Since my machine can't handle highres fix of 1024 or greater I'm now experimenting with using control tile instead. Normally I can't do IMG2IMG above 800x size but with this, I can! Even with 1 Denoise strength and 4 Down sampling the results are always very close to the original though, which is unlike IMG2IMG. Anyone know any tricks?

[@BloxEzio3]

What is the base app??? It is not stated anywhere

[@andyone7616]

Zoe depth doesn't works it ask me that timm model is not installed

[@joephillygizzie3319]

I love this guy.. i made.beautiful pictures because of you

[@xX_SweetAngel_Xx]

Hoping that because I didn't have the yaml files that maybe that is the reason why it isn't working.

[@braykvn]

Hello Olivio thank you for your help, I wanted to ask you and everyone that could help me... since I have updated controlNET I have this error message and I don't know how to fix it: RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument index in method wrapper__index_select) Do you know how I can make SD run normally again?

[@rescuehelly271]

what would happen if I set the upscale size which is not exactly 2x of original one? Let's say 3x, or even a set a 16:9 upscale size with a 4:3 input picture?

[@twilightfilms9436]

Do we have to get rid of the old controller 1.0? If so, how? Or can we keep the old models?

[@OlivioSarikas]

no, you can keep them if you have the space

[@HarryXiao88]

delete old models files...they are useless

[@TheDoggyChannelVideo]

HELP!!! Hi there, thanks for your videos ive been doing amazing things, thanks a lot... Now, i have a problem using controlnet: "TypeError: Script.process() missing 18 required positional arguments: 'is_enabled', 'is_combinatorial', 'combinatorial_batches', 'is_magic_prompt', 'is_feeling_lucky', 'is_attention_grabber', 'min_attention', 'max_attention', 'magic_prompt_length', 'magic_temp_value', 'use_fixed_seed', 'unlink_seed_from_prompt', 'disable_negative_prompt', 'enable_jinja_templates', 'no_image_generation', 'max_generations', 'magic_model', and 'magic_blocklist_regex'" ... and even though deforum generate images, it does not follow the video int... do you have any idea how i can fix this? TIA

[@TheDoggyChannelVideo]

i found a "solution" turn off sd-dynamic-prompts and then it started working with no errors

[@YoIomaster]

i followed your tutorial step by step but my upscaled tile images are blurry, as if i would simply stretch them into the desired resolution. what am i doing wrong?

[@Mrnobody___]

when i try to load in extension. im having error

[@havemoney]

This is good, thanks!

[@OlivioSarikas]

Thank you very much

[@rickraposo89]

I got ControlNet v1.1.116 but it only shows single image and batch. How do I get it to show unit 0, 1 and 2?

[@sipsi6207]

I can’t install this local host anything else you know as alternative ai on web?

[@Sappydance]

mine doesn't work ): mine just gets more blur-er instead

[@whatthefunction9140]

What tool will allow me to define a face structure to have one character reusable

[@Beetlebomb3D]

Where do you find those stock images to use as a base for ControlNet? Seems VERY useful!

[@Kamod90]

I get error on extensions what happened don't know

[@MAKdotCZ]

hi, Olivio! your instructions help me a lot, I just want to ask, is it really necessary to use the screen zoom? it will make the orientation very confusing. ;)

[@stas_lu]

I didn 't notice any difference at all when control_vl1fle_sd15_tile. is this exactly what you need to put in inpaint?

[@OlivioSarikas]

it seems to be buggy at the moment. try to use it in img2img instead and make sure controlnet is enabled. also close A1111 and start it again to try again

[@johnaspaul]

@@OlivioSarikas Hello Olivio, why are the new models only 1.5Gb large? The old ones were over 5GB each. Are the models in the link pruned?

[@RamonGuthrie]

@Kohlendioxid 2019 no the tech just got better optimised

[@tr0ublem4kerWZ]

Hi, excellent video as always! I need some helps, most of the time I can't generate a preview for my control net, I have allow preview enabled. I can confirm that it controlnet is working but I just can't preview it before generate which is kinda annoying. Is there any fix for that?

[@ramboti6402]

Is the preprocessor only connected to preview? If I want to simply generrate an image can I leave it on None?

[@uqoF]

could you help me? when i generate with controlnet it also generates the preview of controlnet how should i disable that? this is a huge problem for me because i'm remaking kpop music videos to anime looking ai and i dont want to delete these preview files later in my output folder.

[@StSpoiler]

I have no model in control net. even I checked with every Preprocessor

[@JayVP]

As always great video. Wanted to ask. How come you have multiple control net units. I only have one.. can anyone help?

[@OlivioSarikas]

go to settings - controlnet and there is a multicontrolnet slider

[@XIIOSRS]

Great video, could you please tell me what model was used for the thumbnail? I’m the author of hyper v1. Thanks

[@Steamrick]

I'm not sure if I'm being an idiot, but if I try to replicate your inpaint upscaling trick, all I get is the same image, with double the resolution but unchanged quality, which is useless. I've watched the section thrice and I'm not sure what I'm missing. Could you maybe do a dedicated video on the subject? I think that upscaling while adding detail but basically leaving the base image untouched is of very high interest to everyone. That said, usign tile resample and img2img is (also?) highly promising in this regard.

[@OlivioSarikas]

try the img2img instead. make sure your controlnet is enabled. try to load the origianl txt2img image into controlnet and see if that helps. it can be a bit buggy

[@Steamrick]

@@OlivioSarikas img2img worked, as stated above. I did try loading the original into the controlnet, but that didn't make any difference. I suppose it might also have had an unexpected interaction with Tiled Diffusion since I was using that to get past the 2048 pixel size maximum.

[@OlivioSarikas]

@@Steamrick i see. yes, extensions can have a bit of a fight with each other when used together

[@KC_79]

I would suggest using with Ultimate SD upscale. I got an amazing result (IMO).

[@OlivioSarikas]

@@KC_79 Awesome, i will give it a try too :)

[@LHITShappy]

This is looking pretty awesome thanks for the video. Just one question will this work with SD2.1…or is it just for SD1.5?

[@kenrock2]

for some reason i could not update my current existing controlnet ... how to uninstall and reinstall the updated one?

[@OlivioSarikas]

move the controlnet folder out of your A1111 folder and install it again from the extensions tab. then move the models from the old controlnet folder into the new one

[@hoangcan91]

Thank you!

[@kimjohn3877]

is this a chrome extension?

[@saltygamer8435]

The update is not working, I already had control net and when i tried to update nothing changed?

[@OlivioSarikas]

move the controlnet folder out of A1111 (but don't delete it), install it again from extensions folder and then move the models files back into the folder

[@swannschilling474]

Thanks a lot!! 🎉😊

[@OlivioSarikas]

you are welcome

[@AlessioRegal]

Thank you very much for these videos! Is there any version of the control net with safetensor files instead of pth files? Just wondering, since as far as I was aware the safetensor models are the most trusted ones for security

[@danielodom8245]

Hey Olivio! Love your videos! I'm getting a weird error with the v2-1_768-ema-pruned model and control net to play nicely. Keep getting this error: RuntimeError: mat1 and mat2 shapes cannot be multiplied (154x1024 and 768x320) I've put the .yaml in the path in control net settings, but no luck. Any ideas?! Thank you!

[@tomschuelke7955]

Hi , i got some questions.. I only got an older geforce 1080 Card.. does this all still work? also, i dont know where to beginn.. Do you also have a video , how to install Automatic 11 11 in the first place? WIth everything nessecair, do i for example have to have account on huggin faces, what ever this is and so on? and Controlnet would be the next step.. nice content here.

[@tomschuelke7955]

ah yes.. and whats about invoke ai? would this be better? sounds as if automatic 1111 has more features.. and i also read, that 1080 gforce graphiccards ar to outdated for invoke..

[@OlivioSarikas]

Yes, if you look at my channel there is also a A1111 install guide. a 1080 might work, but it might take a lot longer per image to render and you might be limited to lower resolutions like 512x512. alternatively you can check out my video about playground AI. there you can't upload your own models, but you can upload images and they will soon add a better version of their outpainting canvas

[@Pauluz_The_Web_Gnome]

Hi Olivio, if you move yourself further away if possible from the green screen, there is less, or no spill around you! :D So the green screen will not reflect on yourself.

[@HarryXiao88]

Or just use a real background wall with some blur effect...

[@rebelape4257]

WHAT IF IM USING COLLAB

[@dschroeder7354]

So can I delete the older models now?

[@stanleywtang]

Looks very similar to the result using Tile diffusion

[@pascal_dop]

ControlNet is still not working in my Deforum Section 😓

[@OlivioSarikas]

interesting. thank you for letting me know

[@PhilippSeven]

I don’t get how to properly use the “Tile” model… in theory you use it with SD upscaler script in img2img but when I do so SD generates the same picture in each tile…

[@bryan98pa]

Doesn't works with SD Upscale but you can use Ultimate SD Upscale.

[@OlivioSarikas]

in each tile? what do you mean? It only creates one image. you don't need to set your batch to a higher number. It will do the tile resampling inside of controlnet

[@PhilippSeven]

@@bryan98pa ok, I’ll try it too.

[@Steamrick]

@@OlivioSarikas He meant in the scripts, using the SD Upscale and/or Ultimate SD Upscale scripts. It's easy to forget they exists, I'd also done so until the reminder just now...

[@OlivioSarikas]

@@Steamrick ah, yes. I will try that :)

[@thecat8513]

This is still very confusing. You are saying it, and I am following but nothing is working.

[@Gh0sty.14]

For some reason I keep getting an error saying I don't have the .yaml files but I definitely do have them.

[@OlivioSarikas]

strange. Maybe restart A1111. You could also move the controlnet folder out of A1111, install again from extensions tab and move the model files back

[@gregoryscaux]

Another thing, as far as im concerned, is that I had issues with the Shuffle preprocessor in Controlnet 1.0. but 1.1 solved them :)

[@OlivioSarikas]

I'm still a bit fuzzy on what Shuffle is actually good for. it creates variations, but they don't look as good as the original. why not just use a different seed instead?

[@voEovove]

Awesome!

[@OlivioSarikas]

thank you :)

[@zupremo9141]

You look like Techland's Tymon Smektała.

[@marcelomiguelbazan]

Hi Olivio: Your videos are great. There's just one thing you should change: The "lable" part in "Available" is not pronounced as the "label" of a tag, or form input in HTML, it's pronounced as "lab" in "laboratory" with an "L" sound at the end. Thanks and greets from Argentina (Hope it is from England someday) Marcelo Miguel Bazan.

[@maddercat]

nice thanks!

[@bobjohnson9354]

Make a backup before installing Controlnet guys, it just killed my Stable Diffusion : "RuntimeError: Cannot add middleware after an application has started"

[@OlivioSarikas]

interesting. if all fails, you can simply delete the venv folder and A1111 will do a fresh install, but will all your models, extensions and images in place. like a reset

[@bobjohnson9354]

@@OlivioSarikas thanks, I did not know this, did a full new install ahah.

[@shanshanshans5167]

12:00 i think it should be inpaint not masked

[@vhanla]

softedge hed is awesome

[@jibcot8541]

I don't normal say people are talking too fast as I watch a lot of youtube on 2x speed. but when you are showing the folder paths you go way too quickly and I have to watch the same bit 5 times to get the files in the correct place, if you just went more slowly it would save time. thanks.

[@OlivioSarikas]

really? I guess i cut it a bit too fast this time

[@Rasukix]

do I have to download the models one by one?

[@WallyMahar]

What?? Download all of them …i mean… i have no idea. Are you serious? Can you give us a basic essentials. I don’t know what a shuffle is or a normal bay or a canny😢

[@antonioanonimo2521]

Está mal estructurado el vídeo, para hacer el proceso deben marcar inpaint not masked.

[@enricoblanc]

Hes explaining how to install it without explaining where he got the actual application from...

[@avidlearner8117]

What's wrong with you, you can't Google?!?! Jesus!