How To Inpaint Anything in Stable Diffusion WebUI - Just 3 clicks!

Inpainting can be fun, but making the masks... not so much. Segment Anything to the rescue! Pick anything you like from the segmentation map, and have the mask created for you, ready for ControlNet Inpainting. Easily change clothes, faces, dogs, cats - segment anything! No need to try and draw masks by hand, which is often tedious and inaccurate.
Just give whatever prompt you like as normal, then as if by magic, the thing you wanted to change is transformed before your very eyes!
No complex or tedious ControlNet setups required as you have a ready-to-go section given to you in the Inpaint Anything tab. Unless you like complex setups, because you can do that too thanks to the mask tab :)
== Links! ==
* Inpaint Anything - github.com/Uminosachi/sd-webu...
* Automatic1111 Web UI - github.com/AUTOMATIC1111/stab...
* ControlNet Extension - github.com/Mikubill/sd-webui-...
* How do I create an animated SD avatar? - • Create your own animat...
* Installing Anaconda for MS Windows Beginners - • Anaconda - Python Inst...
== Stable Diffusion Playlists ==
* The MASSIVE Stable Diffusion Playlist! - kzbin.info/aero/PLj...
* Interested in ADDING NEW THINGS to your AI Art? Try these!
** Dreambooth Playlist - • Stable Diffusion Dream...
** Textual Inversion Playlist - • Stable Diffusion Textu...

[@banzai316]

I’m glad you are still covering Stablediffusion. Very useful!

[@NerdyRodent]

If it's AI, I cover it ;)

[@4.0.4]

I love how many generally useful tools A1111 has, even for non-SD tasks. Like I could install this separately, but here it's just an extension.

[@zakblue]

like I needed another reason to be more obsessed with auto111. So goood!

[@zakblue]

update: this crashed webui I had to spend some time with a workaround to get it started again. What worked: reinstalling stable diffusion in a separate folder from scratch then copying the 'venv' folder and replacing the old one. It had to do some reinstalling but now seems to work fine.

[@Sol-zp6kc]

You have been the most consistent channel of ai related tutorials and stuff! Appreciate everything!

[@worldsofconfusion]

Dude! So good to see you not only still going, but nailing it! I don't spend any time on KZbin these days but I happen too see this and it made me very happy to see you doing such great stuff. Big love Nerdy!

[@NerdyRodent]

Hello again! Good to see you still around as well 😉 Big love back at ya!

[@swannschilling474]

Another great addon and another great Nerdy Rodent explaining!! 🎉❤🎉

[@xmattar]

The power of the sun In the palm of my hands

[@Aaron-fi2ef]

Been looking for an hour on a video that gives simple instructions and goes through the basics. Thank you!

[@NerdyRodent]

Glad it helped!

[@tonywhite4476]

This is a really helpful tool. I hope they make a swap anything next. Like masking two images and choose a mask to swap, like the face masks, clothes mask or even the background masks. That would be awesome.

[@jason-sk9oi]

Totally badass! Thanks for the heads up!

[@KiZzymesl]

Omg, you packed so much in this video. Workflow should be more harmonious. (or a Harmonica Workflow)! Ha, totally funny! I laughed so hard at the end !

[@pastuh]

This is most powerful tool. Would be nice to have segmentation before model/lora training

[@JohnVanderbeck]

Robert is your Father's Brother! LOL love it!

[@AliasArketer]

Fascinating stuff!

[@NerdyRodent]

Ikr! 😉

[@aa-xn5hc]

Brilliant thank you!

[@encapsulatio]

Do you know of any local AI voice enhancing options out there? Removing noise, reverb without denaturing the original voice quality is kind of hard.

[@831digital]

Can you run this in batch mode to correct a specific feature in a series of images?

[@ssapienn]

Did everything but when I run segment anything after it runs it give an error under download model tab and on the left output window. Also I do not have any Yaml file in the folder. How do I fix this pray tell please.

[@1Garrett2010]

Please I need a tutorial on how to put the same exact face in another "body" or Controlnet pose, if its possible with Stable Diffusion. With Controlnet I see the "reference" mode makes possible something similar, but not permits to choose the exact body (or Controlnet) pose.

[@randomlikeu]

love it!!!!

[@facelessvoice]

Thank you 😌

[@ricardoseeyou]

Great tutorial thanks, i use stable diffusion with amd gpu and i have issue with inpaint anything, when running "run segment anything" it search and produce a black image on the right. Any idea? Thanks

[@NerdyRodent]

I don’t use AMD GPUs unfortunately

[@polystormstudio]

Just a tip, I think you have to quit and restart webui and A1111 after installing Inpaint Anything as I got some weirdness going on with my controlnet inpaint. It seemed to work afterwards.

[@KACOC]

Thank you very much. Such a helpful video. Where can I find that document. Please

[@NerdyRodent]

The extension can be found via the automatic1111 extensions tab

[@waterwater5931]

Is is possible to do virtual-try on with this method? I am thinking of getting the mask first and make the target cloth to fit in the mask.

[@NerdyRodent]

Sounds like you may like this: Stable Diffusion Face + Pose + Outfit Swap - NO training required! kzbin.info/www/bejne/kJSml6qKoL6GsJo

[@Snaaaaaaaaaaaaake]

Thank youuuuuu~

[@user-gq2bq3zf1f]

When I run inpaintanything in StableDiffusionUI, especially when I run inpainting, I keep getting error Unexpected end of JSON input.I ran it through Google Labs, what should I do?

[@mistraelify]

Just one thing that didn't catch my eye and it's pretty important. You can segment lower resolutions then afterwards add an upscaled one, then create mask and create controlnet inpainting. I suggest you guys doing that instead of segmenting an already upscaled image which take too much time to be done.

[@vrMoleck]

bravo!

[@lilillllii246]

I use StableDiffusion locally and when I press RUN SEGMENTANYTHING in INPAINT ANYTHING, it doesn't generate a masking image. what should I do?

[@metanulski]

Hi. I get a "ControlNet inpaint model is not available. Requires the ControlNet-v1-1 inpaint model in the extensions\sd-webui-controlnet\models directory." error. I have the Inpant Model in the folder, but I noticed that you downloaded a 723MB savetensor model, but If I follow the link in the description I the inpaint file is a 1,4GB pth file.

[@NerdyRodent]

Either size model is fine, I just like to use the smaller files to save disk space :)

[@metanulski]

@@NerdyRodent thanks, ir :-)

[@sb6934]

Thanks!

[@NerdyRodent]

No problem!

[@lifemarketing9876]

Are you able to create a segmentation from anything and then using that segmentation sketch your own image from stratch?

[@NerdyRodent]

Sure, go for it!

[@Aristocle]

However, if I want to fill an empty room with interior furnishings, what methodology could I use?

[@NerdyRodent]

Photobashing would be one way

[@Aristocle]

@@NerdyRodent the 'problem' is that I would like it to furnish the room but randomly, so that it would give me some ideas on the layout through many images in batches.

[@ZeemZoomtv]

I like to see how creative my computer is.. I will invest in it and teach it to get more out of itself...

[@elarcadenoah9000]

how can u make a head bigger and put some text on the wall or anywhere'

[@bigboss97]

I can't get the segmentation work. Always get a single color image. What am I doing wrong?

[@lenoillionel2342]

hello.. does anyone has a problem with creating the mask?? mine creates a mask a second then it disappear

[@fernandomasotto]

happens the same to me 🥲

[@twilightfilms9436]

Can you run it with batch sequences?

[@escaperoutebritish]

Hi Nerdy. When I create a mask it automatically disappears. Any ideas?

[@NerdyRodent]

Not had that happen as yet!

[@tr0ublem4kerWZ]

I have a 10gb 3080 and after creating mask, I can't impaint anything due to "out of CUDA memory". Is there a fix?

[@NerdyRodent]

Try going for a much lower resolution image.

[@troystreet]

having so much trouble. I have everthing installed. Running a gtx 1060 16gb card and 16gb ram ssd hard drive but for some reason mine takes almost an hour to finish. and can't figure out why. I didnt change any settings past what you showed.

[@NerdyRodent]

Takes a few seconds for me. Check your system’s resource usage

[@troystreet]

@@NerdyRodent thanks I'll try. I should mention there's almost nothing on the computer since it's a pretty new Windows install. Only other thing I could figure is it's trying to use the onboard intel hd 3000 graphics

[@wenowinter]

Hi! Nice video, but can anyone help me with this error?: ControlNet inpaint model is not available. Requires the ControlNet-v1-1 I've already installedthe controller, but still have this error...

[@Sol-zp6kc]

Was wondering what are the ram requirements for this?

[@12Jerbs]

No idea how you got the ControlNet Inpaint to work properly. I selected a white t-shirt and set the prompt to "red t-shirt". It gave me a white t-shirt with red trim and another time a pink t-shirt. I switched to the default Inpainting tab and only got a red t-shirt after turning the Guidance Scale to max. Seeing the video, I had high hopes for this, but it seems rather "meh" on my end.

[@NerdyRodent]

Always worked great for me! Perhaps play with denoise rather than the classifier free guidance?

[@tolly_HD]

I do always get an error when clicking segment anything. But I do think the reason for that is that I use an amd gpu, with the direct-ml lshqqytiger's fork. The Error has do to something with cuda

[@1Know1tHurts]

Try making the resolution of your initial image lower

[@AB-wf8ek]

"Just like Robert is your father's brother..." I'm not British, but I got it 😂

[@IzeIzeBaby]

Can I use segment anything on Videos somehow? Would be usefull to blur faces and license plates in my dashcam Videos instead of doing it by hand for hours 😂

[@TVdosPatriotas]

I downloaded the inpainting extension through A1111 but I don't see the tab.

[@NerdyRodent]

Make sure you’re running the latest version of a1111 and restart?

[@martinkaiser5263]

Would it be possible to colour b&w pictures ? And how to do it ?

[@NerdyRodent]

While you can easily make full colour images from b&w sources, they tend to change substantially from the source image. For the most part, things like deoldify etc. are better at adding colour but retaining the original image composition.

[@Bockiiey]

AFter pressing "Run Segment Anything" the segmented image flashes for a moment and dissapears :(

[@NerdyRodent]

Could be a browser thing, maybe?

[@alekseyzhuravlov]

Very cool! But I guess a direct Automatic1111 plugin in Photoshop is still better? (more tools)

[@NerdyRodent]

Better = free and open source 😉

[@alekseyzhuravlov]

Thank you! @@NerdyRodent

[@lars337]

cooool

[@zafferflower]

Hi Nerdy Rodent, Thanks so much for making this video. I especially appreciated the step-by-step instructions on how to install -- I needed that. However, (there's always a however), I'm pretty sure I got everything installed correctly, but ControlNet Inpaint doesn't color in the lines! It changes everything in the image, not just the masked area, even with ControlNet Preprocessor set on inpaint-only. By "installing everything correctly," I mean I'm running the latest version of Automatic1111on a Windows 10 machine with plenty of processing power. I'm using anything-v4.5.safetensor [1d1e459f9f] for the checkpoint and sam_hq_vit_h.pth as the Model ID, and reference_adain+attn as the Reference Type. Both extensions, sd-webui-inpaint-anything.git, and sd-webui-segment-anything.git seem to be installed correctly. Segment Anything runs beautifully and gives a great mask. It's just that ControlNet Inpaint won't stay in the mask. Any suggesstions will be greatly appreciated. Thanks, Zaffer

[@NerdyRodent]

Does your controlnet inpaint work in img2img?

[@zafferflower]

@@NerdyRodent yes

[@zafferflower]

I've got a workaround, I make the seperator mask and save it, then upload it using img2img upload paint. That works pretty well, but is extra steps. Thanks for your help

[@zafferflower]

Hi Nerdy, I got everything working! :-) I did install of Automatic1111, controlNet and Inpaint anything and now, the inpainting will stay inside the mask. I love controlNet. It seems to have so much more "imagination" than plain Inpainting.

[@NerdyRodent]

@@zafferflower 😀

[@pacify1750]

unable to find the fp16 controlnet inpaint model

[@NerdyRodent]

As mentioned, any controlnet inpainting model will do so feel free to use the 1.4gb file instead of the half sized one!

[@user-ep6ig6un6o]

Please tell me what location to install sam_hq_vit h,l,b?

[@illusivec]

6:50 pretty sure that's not how you use the outpainting feature. For one thing, if you're using Preprocessor, you need to have something in the canvas of the control net.

[@NerdyRodent]

And yet there it is working really well 😉

[@chrisbraeuer9476]

Now the remove anything stuff has to work.

[@speaktruthtopower3222]

im selecting "Inpaint only" and its still changing the entire image,

[@masterzed1]

same

[@Cutieplus]

Anyone know where the inpaint model download by this extension located ? It seem that the inpaint models download by this extension are not located within the stable diffusion folder.

[@alonsogarrote8898]

it should be in models/ControlNet

[@NerdyRodent]

Check out my control net video for more info!

[@Cutieplus]

@@NerdyRodent I meant the inpaint models that automatically download by Inpaint Anything. It is not located within any folder inside stable diffusion folder. The extension has download over 10GB of file but the total stable diffusion folder size haven't change at all.

[@Cutieplus]

@@alonsogarrote8898 Thank you for your reply, but it is not. The models/ControlNet is empty. The extensions\sd-webui-inpaint-anything\models folder only have those SAM files. It doesn't include those inpaint model file.

[@NerdyRodent]

@@Cutieplus The diffusers models go into the default location (your home directory cache)

[@abdullahsulaymaan9085]

why do you sound like matt from @DIYPerks

[@alex.nolasco]

"..feels like traditional art" this guy... lol

[@NerdyRodent]

😉

[@gui205]

Seem to not work on AMD Gpu

[@NerdyRodent]

Linux + Nvidia is your best bet for compatibility & performance

[@thomassynths]

You sound like Techmoan

[@dragonmares59110]

My issue with A1111 is that so much functionnality are being created, updated, changed that is is becoming way too hard to keep up. I see tons of this kinds of video but we really need a written ressource. Videos are not a good format for this kind of fast changing tech.

[@NerdyRodent]

Some people really don’t do well with the written format and much prefer videos - and they tend to use youtube 😉

[@user-bs3jd2hj3z]

To be honest it much difficult then photoshop, but more advanced , I agreed. And its not just 3 clicks

[@dogvandog]

CAUTION ☢ Extension can ruin you current SD installer. Many people even cant run SD after this addon.

[@dogvandog]

I deleted Venv folder from my "stable diffusion" and runned SD (to redownload and fix problem). I think this kind of problem can happen when you installed SD long time ago. Sometimes better to do clean instal SD.

[@NerdyRodent]

Yup, it's best to make sure you've got a working SD install first ;)

[@2PeteShakur]

yeh just delete venv folder and run sd again to fix apparently - I'm using vlad's version of a1111 and seems to work fine, just requires a few restarts and the tab appears! ;)

[@NerdyRodent]

@@2PeteShakur nice. I just use anaconda myself - less hassle! Let the segmentation commence 😀

[@alonsogarrote8898]

@@2PeteShakur I tried many times to install this extension but the tab never appeared, I will try restarting multiple times

[@Smashachu]

Your youtube notification just got me killed in fortnite. downvote.

[@NerdyRodent]

git gud ;)

[@DoppsPkin]

Subscribed

[@NerdyRodent]

Welcome! 😉

[@AiNomadArt]

I found inpaint_only+lama to do outpainting is insane

[@NerdyRodent]

Yeah, it’s a cool model and set of preprocesses!