Segmentation - Controlnet 1.1 - I LOVE THIS!!!!

Segmentation in Controlnet 1.1 is really powerful and fun.
#### Links from the Video ####
ControlNET 1.1 Guide: • ControlNET 1.1 - What ...
Color Code: docs.google.co...
#### Join and Support me ####
Buy me a Coffee: www.buymeacoff...
Join my Facebook Group: / theairevolution
Joint my Discord Group: / discord
Support my Channel:
/ @oliviosarikas
Subscribe to my Newsletter for FREE: My Newsletter: oliviotutorial...
How to get started with Midjourney: • Midjourney AI - FIRST ...
Midjourney Settings explained: • Midjourney Settings Ex...
Best Midjourney Resources: • 😍 Midjourney BEST Reso...
Make better Midjourney Prompts: • Make BETTER Prompts - ...
My Facebook PHOTOGRAPHY group: / oliviotutorials.superfan
My Affinity Photo Creative Packs: gumroad.com/sa...
My Patreon Page: / sarikas
All my Social Media Accounts: linktr.ee/oliv...

[@OlivioSarikas]

#### Links from the Video #### ControlNET 1.1 Guide: kzbin.info/www/bejne/sKOqfXidfKt9j5Y Color Code: docs.google.com/spreadsheets/d/1se8YEtb2detS7OuPE86fXGyD269pMycAWe2mtKUj2W8/edit?usp=sharing #### Join and Support me #### Buy me a Coffee: www.buymeacoffee.com/oliviotutorials Join my Facebook Group: facebook.com/groups/theairevolution Joint my Discord Group: discord.gg/XKAk7GUzAW

[@history3042]

Is this the best tutorial to still use for segmentation 9 months later?

[@Seetvael]

You can use latent couple + Composable lora, and replace objects in the scene on request

[@ahtoshkaa]

Can you please explain what that means for a newbie? What is latent couple?

[@Seetvael]

@@ahtoshkaa these are two add-ons that need to be installed additionally. It allows a color identifier to be used as a mask. By adding a text query to each mask.

[@j.c.4123]

@@Seetvael that sounds interesting, can you share a link to these? Thanks!

[@Seetvael]

@@j.c.4123 I left a link but it gets deleted, find MultiDiffusion region control

[@gustavosaliola]

Is this something similar to Cryptomattes used in VFX and Digital Composition?

[@jonmichaelgalindo]

The "sticks too close" / "too much information" problem can be solved by lowering the duration over which controlnet runs. Let it run for maybe 30% to set up a "foundation", then let the AI take over and decide what it wants to do from there. The same applies with segment. If you want more variety and creativity, use less control.

[@fnorgen]

Yeah. I found that a gentle nudge by the depth-net early in the generation process is really helpful for preserving overall composition without locking down the image too strictly. Especially when combined with the openPose-net.

[@minimalfun]

I can't believe it, I was composing a shot and I was pulling my hair and the answer was right here !!!! As if this video was made for me!!! Thank you very much Olivio, truly awesome channel and as always awesome info.

[@AB-wf8ek]

I've been experiencing this for the past year. Everytime I reach some limitation, I think, well, give it a few months to a year for something to be developed. But then the next day there's a new solution. The development cycle on these tools has been unprecedented, there has to be a plateau at some point, but this has to be one of the most intense periods of sustained development for an open source project, and it's still progressing at a steady pace. For sure Blender has been around for decades and has seen similar benefit as an open source project, but I don't think it ever reached this intensity of development. The barrier to entry for image generation is so much lower, it's really been mind blowing to witness the capabilities of open source development as decentralized collective cognition.

[@minimalfun]

@@AB-wf8ek I couldn't agree more, it has been really amazing, the pace is insane and I can't wait to see what is next, but one things is for sure. We ain't seen nothing yet!

[@76abbath]

Thanks a lot for this video about seg with the code colors. Anothers possibilities! Argh! 😮 And thanks to you for all the videos you post! I follow you every day!

[@OlivioSarikas]

My pleasure :)

[@MaisnerProductions]

great update!

[@OlivioSarikas]

Thank you ♥

[@dk_2405]

Olivio you are SD master

[@OlivioSarikas]

thank you 😍

[@mikerhinos]

This channel needs so much more attention and subscribers...

[@R67K]

Stabiles Video du alter Haudegen!!! Thank you for your tips and tricks.

[@OlivioSarikas]

Danke 🔥

[@MDLEUA]

Great desciption for this modes! Now I know when to use this 3, please more content like this!

[@Carolingio]

Bravo Olivio. 👏👏👏👏👏👏👏👏

[@MrDebranjandutta]

Great tut olivio, gonna try this out. Being a blender user the seg maps reminded me of cypromattes

[@frazuppi4897]

As computer vision engineer, I have to say that your explanation is really really good! Love your content, thanks a lot :)

[@wboumans]

Nice explanation between seg and depth!

[@AIMusicExperiment]

This has been taken to the next level with Inpaint anything!

[@TheMaxlizzy]

What you mean bro? Say like someone (not me) wants to undress a certain person can it do that?

[@renwar_G]

this is super creative

[@Maisonier]

Thank you, amazing video, as usual.

[@ToniCorvera]

Me: How many times are you going to blow my mind? ControlNet: Yes

[@-invisio]

perfect, as usual. Thanks

[@Comic_Book_Creator]

thanks, you are doing a lot of nice tutorials.

[@OlivioSarikas]

Thank you very much

[@Sinistar74]

Thanks for the death map info! 😉

[@amj2048]

I think this method would be really good with animation too, I need to play with that. Thanks for sharing this!

[@xXMsCarlyXx]

the next renovations in my home are going to be wild

[@boya8719]

Another great video

[@jibcot8541]

Very nice, I never knew about this!

[@OlivioSarikas]

I'm also surprised i didn't use this earlier

[@sarpsomer]

Brilliant!

[@perschistence2651]

Very nice! For the future I would wish to be able to click on a segment and edit the promt for the area. Or directly edit it in A1111

[@oaahmed7515]

♥

[@OlivioSarikas]

😍

[@keisaboru1155]

"A house with a death map" 😂

[@nekoeko500]

If I remember correctly, In the ancient past of a couple months ago, Google released a paper and a demo of simillar capabilities for imagen. This looks way better though

[@in2thedark]

Game changer 👌 thanks

[@OlivioSarikas]

Thank you 🔥

[@hugoruix_yt995]

great video, thanks!

[@loubakalouba]

This is a game changer for me, thank you so much! Is there any way to separate 2 or more different people in a scene, old, young, male, female?

[@wykydytron]

I don't think there is reliable way to do it as AI doesn't understand numbers, separation, individuals etc. Best you can do is using control net with few open pose skeletons or img2img to lead it to what you want. There are some Lora's that try to do that but result as always with Lora are varied depending on checkpoint and how well Lora was trained.

[@Dante02d12]

Regional Prompter. With that, you can "break" the prompt into multiple little prompts (one per zone on the image ; you define the zone yourself), the AI will understand which character has which features. It gives very reliable results, but each character must stay on the side of its own prompt. Sadly, it doesn't "break" the LoRAs, so you can't use multiple LoRAs for multiple zones.

[@michaelsnow7440]

Does ControlNet use that spreadsheet to determine what each color should be or is that just saying what is inside the seg process? Can I change the objects assigned to each color?

[@teambellavsteamalice]

This is awesome, I imagined there would be something like this. Can you use this to inpaint more precisely?

[@shadowdemonaer]

This looks like it will make great sense to me and should work well. I have been tired for a long time of the way it doesn't allow me to do what i need very specifically. Two shot was never helpful for me. i may very well uninstall it.

[@pastuh]

Waiting for something like "reverse original" which generates objects from previous seed... It would solve blinking animations for sure..

[@OlivioSarikas]

wasn't there a script that made videos very stable?

[@pastuh]

I mean blinking, which appears from random generated objects.. It's like 4 frames ok, but next is totally different.. Lack of consistency.

[@jcims]

One quick note, I've been tinkering with this Feb 2024 and am unable to get it to work at all unless I match the current version of the checkpoint and VAE you are using (I'm new to all of this and don't actually know what that means). I don't know if that's expected or just a big in the most recent releases but just figured I'd share.

[@jonmichaelgalindo]

Ack! I wish I could just plug words of a *prompt* into pixels of a map like this. (And no, composable lora / two-shot doesn't do this. It gives you a "mask drawing" feature for prompt regions, but then it's just converting what you draw to rectangles behind the scenes, which absolutely never works.)

[@sasaj9420]

Hello Olivio, thanks for the great video. My question is, can we divide something into more segments? For me, it turns the girl's image into one segment, but it would mean if I could divide it into several segments, such as head, suit, hands... Is that possible?

[@magimyster]

Hello Olivio, very interesting!... Do you think it's possible to add other objects with different custom colors to the list?

[@y2washere]

amazing

[@swannschilling474]

Thanks a lot for this tutorial!! 😊

[@Dante02d12]

Segmentation is clearly the way to go with AI generation. It gives complete control over detail.

[@johnhendersonmusic]

This is fantastic. One question though... the semantic categories is pretty short and limited right now... is this getting expanded?

[@reyP]

amazing video. Is this Controlnet 1.1 installed on Stabel diffusion or in Vlade diffusion ? I have no experience and will jump into this but want to make sure which local to install, Vlade or stable?

[@alexlindgren1]

Thanks for this. I'm wondering how I would go about to know which of the elements that are the walls, maybe use gpt 4 in some way? I want to be able to have an image of a livingroom and generate a new version of this livingroom in a specific style but with the specific sofa. Now sure on how to create this workflow. Any ideas?

[@OlivioSarikas]

you could try controlnet with canny or hed. but getting Ai to render something specific in a very specific design is almost impossible. Loras can help you with that, but it's not exact. AI is more of a cooperation between AI and a human, not like a camera. maybe some day

[@coolbinchen]

Is there a good way to replace a specific part of the previous image (eg cha ge the chair) while preserving everything else? I think that can be accomplished via inpaint with masks but is there a simpler workflow via segmentation?

[@konraddobson]

You can mask it.

[@user-ey1nf9xl2j]

Hi Olivio, your videos are amazing!! Thanks very much, i have a question for you. Do you think it's possible to make an Ai model wear a real dress? For example if I have the ghost mannequin photo of a dress can i generate a worn photo with an AI model? Please let me know, i'm new in this fild anch I think this could be very usefull

[@metasamsara]

I don't understand when I pick a preprocessor I don't have models that go with it. Is there a video where you explain how to fully install controlnet without anything missing?

[@walidflux]

still waiting lora trainning with colab, please do it!

[@lioncrud9096]

if they can do segmentation, why don't they natively support Multidiffusion Region Control? I can't get it to work, seems it was shut down

[@dogvandog]

Is this full list of color code objects that segmentation can understand or someone have more detailed? Would be cool to segment as example jeans and underwear.

[@digitalgym]

This is so cool, how could this be used to render specific materials swatches in these areas? Say I want to add a concrete floor instead of a wooden floor but I want the concrete floor to be based off of a sample swatch?

[@alexlindgren1]

Also wondering this. Have you figured it out?

[@Eugenlch]

I love u bro ❤️

[@OlivioSarikas]

Thank you 😍

[@Eugenlch]

@@OlivioSarikas I really do not have time to test all the features, and you do it as quickly and efficiently as possible. Thank you. best content

[@OlivioSarikas]

@@Eugenlch my pleasure :)

[@TheMaxlizzy]

You down bad bro

[@ponponych2]

this thing doesn't separate two characters? people of the same color

[@LegendD112]

Hi Olivio, let's say that I use seg and it makes me a mask of the subject, how do I indicate in the prompts that I only want changes in the person? I try it and it changes the whole image!, thanks

[@mohsenalfaraj207]

Hi, is there a way to train stable diffusion on a logo?

[@AstroPinoy]

for some reason on my MacOS the preview is not working...

[@simonmcdonald446]

Hi - is there a way to use segmantation with SDXL on comfy?

[@caseyj789456]

Please do you have this for comfyUI ?

[@TheRealZachTV]

Could you use this to remove a person from a image?

[@user-qb5ky4qw1i]

2:53 how did you get that excel sheet?

[@Meftur]

Rendering in architecture with expensive hardwares has officialy died

[@matthewjmiller07]

For my controlnet, the preview button sometimes works and sometimes doesn't - any reccs for a fix?

[@matthewjmiller07]

worked for ufade20k but not ofade20k, for example

[@hartdr8074]

Do you just refer to the color mapping? I feel like this is counter intuitive because it doesnt link the control net seg map into the prompt.

[@hartdr8074]

How do you assign different stuff to replace the colors?

[@captainsalmonslayer]

painbucket in image editing software, just replace with whatever colour code you want from the list he linked.

[@TheUselessgeneration]

This has so much potential, just fails on the last step. If you could select each object and give an individual prompt, now that would be great.

[@buddy2665]

Bro I can't generate images above 512×768 after upgrading to torch 2.0 , it says cuda out of memory everytime, do you know how to fix this , i also tried to reinstall automatic 1111 but now it installs torch 2.0 , it all started when I updated my webui , than i thought upgrading to torch 2.0 might help me but it got even worse.😢😢

[@peterwake5265]

Most likely your xformers install is broken. I had to update many dependencies manually after going to torch 2.0

[@gameplayfirst6548]

Buddy get a 3090. Regards, Bro.

[@buddy2665]

​@@peterwake5265bro can you give me the links? Or just tell me what dependencies i need to update , I'll find them myself.