Regional Prompt Control with Tiled Diffusion/MultiDiffusion, Compose your Image Like A Boss! [A1111]

#aiart, #stablediffusiontutorial, #generativeart
This tutorial will show you how to use Regional Prompt Control within the Tiled Diffusion / Multi Diffusion extension to compose your images the way you want it! No, this isn't quite the same as Regional Prompter, i think it's actually easier to use. Plus, you can use LoRAs, Adetailer, and Control Net with this extension to give you even more control of your images.
Chapters:
00:00 - Intro
01:36 - How to Use Regional Prompt Control, Simple Example
05:45 - About the Seed values in Regions
07:22 - Working with more than 2 regions
08:17 - How to save your configurations into .json files
08:41 - How to load your saved config files
09:55 - Feather value
11:31 - How to use LoRAs with Regional Prompt Control
12:40 - Using Adetailer with Regional Prompt Control
13:50 - Using Control Net Open Pose with Regional Prompt Control
14:40 - Tiled Diffusion plus Regional Prompt Control Use Case
Useful links:
Regional Prompt Control github
github.com/pkuliyi2015/multid...
Tifa LORA
civitai.com/models/87167/tifa...
Aerith LORA
civitai.com/models/63013?mode...
Open Poses from CivitAI
civitai.com/models/76882?mode...
**If you enjoy my videos, consider supporting me on Ko-fi**
ko-fi.com/keyboardalchemist

[@rod-me8ey]

This is the best A1111 channel hands down.

[@KeyboardAlchemist]

Thank you for your kind words!

[@ImmersionImagery]

You do a great job with your videos. No fluff just clear to the point explanations. Thanks

[@KeyboardAlchemist]

Thank you! I appreciate that.

[@farcfoxguerrilha]

Very good! Thanks!

[@KeyboardAlchemist]

Glad you liked it! Thanks for watching!

[@jhontitor9332]

You and your guides are amazing, hands down

[@KeyboardAlchemist]

Thank you! I'm glad you like my tutorials.

[@nikosurfingYT]

Super clear explanation 🎉, thank you

[@KeyboardAlchemist]

You’re welcome 😊

[@hornet_play9234]

Thx man, cool! FF7 one love😊

[@KeyboardAlchemist]

You're welcome! Thanks for watching!

[@HorseyWorsey]

That final image is ridiculous. How much VRAM and I guess what GPU are you running? Super Demo!

[@KeyboardAlchemist]

Thank you, I like this final image a lot! I have a 8GB 3060Ti, it took all of the 8GB of VRAM and almost all of my 16GB shared RAM to run the upscaling. I left it overnight to run, and came back the next morning, it was still running. Total of 12 hours to generate that last upscaling.😂

[@drpepper7749]

hi can you apply LORA to spesific region?For example from your last example can you apply LORA to like region1 : Church region2:Tavern

[@KeyboardAlchemist]

Yes, using a LORA to help control what is generated in that region is a great idea.

[@shtorm7267]

Thanks! does regional prompt support dynamic prompts?

[@KeyboardAlchemist]

I have not tried it with dynamic prompts, but I don’t see why not.

[@an-ul8zp]

Need Help！！ I encountered a problem when the progress bar of the browser page was completed and the image was displayed, and the progress bar of the “MultiDiffusion Sampling” in the console was stuck at 48% or 75%, causing the image in that area to not blend well.

[@progeman]

it seems both these add-ons have changed their name and functionality. Could you make an updated video of this?

[@bathtime1989]

i think i found the solution: the version he used is located here on the guthub /releases/tag/sd-webui-v1.1.0

[@AhKaj-jw8of]

Thanks to this video, my doodle 1024 images have become much cleaner! Thank you! But at 18:20, I want to make it 8192x4096 as you show, but I don't have enough memory. No matter how many times I imitate your settings, I get similar results. When image generation exceeds 50%, it suddenly goes over 24bg and starts using around 13gb of shared memory. When it starts using shared memory, the generation never finishes as you know. I can't figure out what the cause is. upscaling from 2048x1024 to 4096x2048 finishes at a normal speed, so I don't think automatic1111 is broken.

[@AhKaj-jw8of]

So I installed a new Stable Diffusion, ONLY added tiled VAE and ControlNet, and it worked!. Probably some extension was messing things up before. Sorry for the hassle! By the way, in the last video(girls one), you had the noise inversion setting on, did you turn it on in this video as well?

[@KeyboardAlchemist]

Hi, I'm glad to hear that it worked out in the end. Sorry for the delay in my response. I did not have noise inversion in this regional prompt control video, it was only just tiled diffusion and control net.

[@Kendisputed]

Hi. Great video! As i'm trying to install multidiffusion I ran into a problem "ModuleNotFoundError: No module named 'llama_cpp'" Can you help me resolve this? Thanks.

[@TheBlazerGOLD]

bruh, i cant understand why cant i see that extension in forge? Forge is much more faster than 1111 on my 3080, like 3times more faster, so i would like to keep it on forge, but i cant find it in my extension after downloading

[@KeyboardAlchemist]

Unfortunately, this extension doesn't work on Forge. You can download it, but it won't enable.

[@TheBlazerGOLD]

sad, any good alternatives for forge?@@KeyboardAlchemist

[@Aresmar]

It has a bug with SDXL models :( cant use it.

[@KeyboardAlchemist]

Yes, this is unfortunate.

[@nickaww]

Hey, when I try to upscale and add detail I lose detail instead when my denoising strength goes too high, even at 0,4 I lose detail and the image becomes "Soft". I also run into this issue when using the Ultimate SD upscaler script. I tried multiple upscalers too. Do you by any chance know what could cause this problem? Can't find it on the github either. I thought I'd shoot my shot here :D Thanks

[@KeyboardAlchemist]

I'll take a blind stab at this because it's hard to know what the issue was without some additional info. I have noticed that if you've set the Resize Mode to be "Just resize (latent upscale)" and try to upscale it, then your image will come out with a soft and blurry kind of look (looks kind of like a Renaissance painting). Choosing "Just Resize" will fix this particular issue.

[@nickaww]

I will give this a try! Thank you for your answer! @@KeyboardAlchemist

[@nickaww]

Unfortunately that wasn't it. Here are my settings hyper detailed, intricate details, extreme quality, 8k, HDR, Negative prompt: cartoon, painting, illustration, (worst quality, low quality, normal quality:2) Steps: 25, Sampler: Euler, CFG scale: 7, Seed: 2922028837, Size: 2048x2048, Model hash: 338b85bc4f, Model: juggernaut_reborn, Denoising strength: 0.6, Tiled Diffusion upscaler: 4x-UltraSharp, Tiled Diffusion scale factor: 2, Tiled Diffusion: {"Method": "Mixture of Diffusers", "Tile tile width": 96, "Tile tile height": 96, "Tile Overlap": 48, "Tile batch size": 4, "Upscaler": "4x-UltraSharp", "Upscale factor": 2, "Keep input size": true, "NoiseInv": true, "NoiseInv Steps": 10, "NoiseInv Retouch": 1, "NoiseInv Renoise strength": 1, "NoiseInv Kernel size": 64}, ControlNet 0: "Module: tile_resample, Model: control_v11f1e_sd15_tile [a371b31b], Weight: 1, Resize Mode: Crop and Resize, Low Vram: False, Threshold A: 1, Guidance Start: 0, Guidance End: 1, Pixel Perfect: True, Control Mode: ControlNet is more important, Hr Option: Both, Save Detected Map: True", Lora hashes: "add_detail: 7c6bad76eb54", Downcast alphas_cumprod: True, Noise multiplier: 0, Version: v1.8.0-2-gc89c8a63 I tried different models, noise inversion on/off different tile sizes. Different upscalers, R-ESRGAN 4x and 4x Ultrasharp

[@nickaww]

It's fixed, by copying the settings from your other video. I think the noise inversion settings were wrong.

[@KeyboardAlchemist]

@@nickawwI'm glad it worked out in the end! I noticed that you have the add_detail LoRA in your prompt, which may have some unforeseen interactions with Noise Inversion. Just guessing here, I have to test it out later.

[@an-ul8zp]

Need Help!!! I encountered a problem when the progress bar of the browser page was completed and the image was displayed, and the progress bar of the MultiDiffusion Sampling in the console was stuck at 48% or 75%, causing the image in that area to not blend well.

[@KeyboardAlchemist]

Can't say that I've seen this issue before. It could be that you are running out of memory. Sorry, I won't be much help in this case.

[@masterjames1856]

Hey i'm getting this error; "must be real number, not NoneType" and i don't know what to do :c

[@KeyboardAlchemist]

Sorry I don't think I've seen an error like that before. But one general tip, if you are using an SDXL model with the Regional Prompt Control function, it might not work very well. I've heard people having issues with SDXL. SD1.5 works very well though.

[@rockysikander.8177]

doesnt work with sdxl gives TypeError: must be real number, not NoneType

[@KeyboardAlchemist]

Thank you for this note! Someone else was having the same issue. Seems SDXL and this extension are not compatible.

[@LouisGedo]

👋

[@KeyboardAlchemist]

Thanks for tuning in!

[@Skydam33hoezee]

Very clear explanation, but I just can't seem to get this to work. Getting a 'TypeError: must be real number, not NoneType' as error.

[@KeyboardAlchemist]

Sorry to hear that you are running into errors. I have not seen this error before.

[@KeyboardAlchemist]

Actually, I just saw another comment about this. The extension doesn't work on SDXL models. It'll work with SD1.5 models.

[@omegablast2002]

its hot garbage. it caps out ram usage on a 4090 when doing anything except 2 region, and it is applying the negative prompt AS POSITIVE the entire generation.

[@KeyboardAlchemist]

Sorry to hear you are having issues. When I'm using it, I can generate a 2048 x 1024 image with 4 Foreground regions and 1 Background region in 1min 30seconds, and VRAM usage was only 5GB out of 8GB. It's possible this is caused by a driver version issue. Also, I never ran into any issues where the negative prompt mixes with the positive prompt.

[@psteels]

Reeeeeee Unknown technology Uga boga hot garbage