NEXT-GEN MULTI-CONTROLNET INPAINTING! You’ve NEVER SEEN THIS BEFORE!

[@Aitrepreneur]

HELLO HUMANS! Thank you for watching & do NOT forget to LIKE and SUBSCRIBE For More Ai Updates. Thx

[@Leto2ndAtreides]

Have you seen Corridor Crew's video "Did We Just Change Animation Forever?", that seems super worth experimenting with.

[@dimitrishow_D]

i have some cartoon characters , see my profile ....i tried training stable diffusion with this but it cant seem to replacate this...the training only seems to work with things it created itself ..am i doing something wrong or is it that my character is unique and it doesnt have enough reference...when i did kinda get it to worrk with i think astria...it drew it like a 5 year old ..if it is possible to train with your own 2d characters PLEASE make a video about that

[@long20014]

Hi, could you share the excel file about color?

[@MissAIchan]

How did you enable the guess mode?

[@richardgreaney]

This is getting so mind blowing now. There are just so many different possibilities for how to make a good image now. I almost feel like I need to take a step back for a bit, and think about what I would really like to create, then see what technologies are now available that could help me achieve this.

[@kyoko703]

With so much "tuning" and manual work, it's pretty safe to say that these "generative" works are more human than anything else and demonstrates that these are just tools at the end of the day.

[@dv6165]

In this case very cumbersome tools for a pretty simple operation

[@gara8142]

@@dv6165 True, but only if we take it at face value, if you wanted to change an image from realisitc to , idk, helltaker style manually, it would take way longer. As with most tools it's a matter of use cases.

[@user-zi6rz4op5l]

With Generative AI, you never get the image you wanted, but end with some image which looks pretty after a million random pushing of parameters. Can you get deterministic result on the next attempt. NO. It would be better if these models churn out 2k+ resolution as the upscaling is again a million guess on the parameter buttons 😔😔

[@markusm108]

@@user-zi6rz4op5l you essentially describe what a customer feels when he hires an artist or designer. what these tools do is they put you into a different role - not the artist but the art director.

[@vi6ddarkking]

We Are Getting So Much Control So Fast. I Honestly Don't Think even 6 month ago anyone thought we would have advanced thim much this quickly.

[@wakegary]

and it's exponentially growing. video is a month or two away (or less). then come the apps, the corporate control, the tightening laws on models, the underground trade, the bootleg versions blah blah. it will be fun to watch - we're in the wild wild west phase right now

[@cdreid9999]

This reminds me of when personal computers became available and a lot of us learned to program

[@Mocorn]

I can see a quality difference in my output folder from just two weeks back! This is crazy.

[@Dex_1M]

dude, i need to watch this video 10 times with some personal experimentations, this new multi control net is a masterpiece

[@kinnectar820]

Holy shit, so much progress in AI image gen in so little time, my dreams of doing my own graphic novels using ML are pretty much fully within grasp using these techniques. What a time to be alive!

[@Gxbbzee]

I've been looking at separate tutorials for the past few days, and even those combined didn't give me as good and precise information as this single video did... amazing! Very much appreciated!

[@Real28]

I called it. 6 months ago I said that while it seems like this kind of tech should take a year, it would take half the time. There's just too many people grinding away at these tools simultaneously. Just crazy.

[@johannvandebron986]

Nice! You have the best Stable Diffusion / Automatic 1111 content on YT. Thank you so much for letting us know all this!

[@Mocorn]

That trick to crudely draw lines to generate edge lighting worked way better than I thought it would. How on earth does it bleed onto the nose like that? This is some wild shit!

[@dreamzdziner8484]

For someone who had been playing with Controlnet from day one just to blend a foreground character perfectly to a desired background perfectly; this video is a treasure. I tried many combinations but never ever thought of that inpainting trick you showed. You are the absolute best dear friend. 👌💪👍👏🧡

[@jurandfantom]

have you managed to make it working? for me it change whole picture and "masked only" other hand make things incorrect.

[@DESIGNISTASTY]

This is why I love SD because you have total control of what you want to do and how your image will be.

[@uk3dcom]

I have been trying to do the things you have now solved for hours. Thank you for your experimentation and sharing. The fact that you and a couple of other guys on KZbin are feeding off each other and pushing this forward to simplify our efforts is really appreciated. Please keep up the good work.☺

[@설리-o2w]

Can't wait WHAT A TIME TO BE ALIVE AND FIRST!!!

[@fnorgen]

Now this is the level of control I was badly missing back in January!

[@oblivionronin]

15:41 That image is absolutely insane, i love it !

[@winkletter]

This gives me an idea for an experiment: Comics made with multi ControlNets. One for the frame. Add characters in with OpenPose. Then segmentation for specific objects.

[@f0kes32]

how would you draw the same character in different poses?

[@anastasiaklyuch2746]

@@f0kes32 see previous video ;)

[@M.I.F..]

Charturner....(civitai)

[@wakegary]

I just madea Steve Urkel in the style of EC Comics (Tales from the Crypt) and I'm very weirded out. AKA it's awesome

[@Strangepaper]

@@wakegary what model for ec comics?

[@schenier]

I find that all this shows that this tool can be used by real artists and create little by little an image, from background, to pose, all the way to the lighting

[@coda514]

Controlnet is amazing, and thanks to you I understand it more and more every day. Thank you from the bottom of my heart. Sincerely, your loyal subject

[@Artishtic]

Thanks for the epic tutorial! If there were only a photopea version of After Effects too.

[@travelwell6049]

Interesting. I have not seen anyone talking about controlnet. So it’s great to see. Thanks.

[@bentp4891]

Amazing. Probably the most impressive new features since Dreambooth.

[@human-error]

Best controlnet video on the net. Tnx NON-HUMAN !

[@MrBlitzpunk]

I haven't tried the multiCN, but with a single CN if you use the hed model it's already good for changing the style of a single image, it works kinda like depth map but preserves the detail and lines of the image

[@TrancorWD]

Very cool seeing the multi-control net stuff. Last few days I've been adding in support for interactive visualization of control net outputs as a pre-pre-processor. Since it's annoying poking at values without knowing what the outcome will actually be. And I figure, if I'm doing that for fun, other people are probably doing this as well on the sd-webui-controlnet extension team. Cause the capability is all there for feedback from the ControlNet preprocessor in A1111, just a matter of connecting up the hooks for it.

[@PhilippSeven]

You can Inpaint character on the background much easier: there is a tab for that - “inpaint upload”. Just use depth pic as a mask. No need to draw something, and result is much cleaner.

[@krz9000]

dude...what a treasure chest of a video!!

[@bolbolzaboon]

This video right here worth so much that I'm gladly going to join Patreon.

[@Nickknows00]

Awesome man!!! So much new I didn’t realise you could do!

[@evylrune]

Damn, the sketch lighting trick is pretty cool.

[@autonomousreviews2521]

You never waste my time :D Thank you for the depth and detail.

[@garen591]

Thats really cool. Could you talk about how to change an object perspective too? Say you want a front, top, side or isometric view of an object

[@lioncrud9096]

damn, this was like 20 tutorials in one! Awesome content Mr. Aitrepreneur

[@cruhstin]

Fantastic tips! It would be awesome if you add checkpoints/timestamps to this video so I can quickly go to a spot in the video if I want to review a specific trick you showed off. Keep up the great work 😀

[@AscendantStoic]

Fantastic collection of tips & tricks, nice work ;)

[@hexemeister]

Your channel is awesome, but I am overwhelmed with so much info. Is there a newbie playlist to start from beginning to catch it up?

[@joshuajaydan]

You sir are insanely talented. Thanks for sharing.

[@maddercat]

wow, i had no idea this was possible, that's insane...I gotta wrap my head around it, it's difficult even seeing you do it. lol

[@benjamininkorea7016]

Brilliant. The segmentation index is completely crazy, and I never would have found it. I gave up on segmentation almost immediately as useless, but I was wrong! Do you have any workflow yet for altering the appearance of exisitng human faces not from prompts? Especially, I want to use my face in a new scene but not with the same lighting taken in my living room-- but add god rays etc. to it.

[@muuuuuud]

Unless I'm mistaken i think Lora is your answer

[@Mocorn]

​@@muuuuuudor dreambooth or textual inversion or hypernetwork.

[@sunlightg]

I agree with the previous commentator. Just train your own lora model with your own face and then use it as you wish. There is a good video in this channel about training LoRa, so just follow the instructions. Just don't forget to read the comments under the video because there are some important things to add :D

[@ashokp9260]

Hey this is so incredible... tools for infinite creativity at 0 cost. Also, I feel I am light years behind, if I forget to follow develpments even for a week.

[@devnull_]

Like you didn't already have possibility for "infinite creativity" with pen and paper or photoshop.

[@ashokp9260]

@@devnull_ Like everyone on earth is so artistic like Davinci.

[@thethiny]

Why did you turn off the pre processor for Open Pause while making the man dancing in the living room?

[@aaronhhill]

"Because I'm a madman." Hahahahaha! Love you man, you're great!

[@TheAiConqueror]

troll king 👑 no seriously, i love your video, your workflow tricks. I could watch you for hours 😁👍

[@HolidayAtHome]

remember the time where txt2 image with 1.4 was the only thing we could do back then =D ?

[@estrangeiroemtodaparte]

I remember discovering disco diffusion and thinking it was magical! lol Things are changing fast!

[@IceMetalPunk]

Yeah, less than a year ago 😂

[@luke.perkin.inventor]

Great video, so many useful tips!

[@OriBengal]

whoa! Totally stands out from the other guys :) -- Great techniques!

[@Apothis1]

Thankyou so much for all these helpful vids, maes this super easy to understand, really appreciate it

[@ColoNihilism]

awesome where do we get some here do we get some lighting examplars ?

[@vi6ddarkking]

Would it be possible to use Multiple Hypernetworks With Multi-Controlnet? To for example, compose an image with multiple different characters with specific outfits?

[@moneyjuice]

that's insane how quick the AI tech is evolving

[@6666daf]

Really good techniques.

[@mariokotlar303]

Thank you so much! This was very helpful!

[@karl-heinzbiederbick87]

Wow! Thank you for sharing.

[@victorwijayakusuma]

Thank you for this! YOu are wonderful! by the way how to do the excel thing or is there any other program to see the segmentation list?

[@Kryptonic83]

awesome collection of tips, I've been loving playing around with controlnet lately. keep up the great work!

[@lucstep]

Really cool! But is it possible to combine photos, like shown in the video, but not originating from SD (for example a real photo of a background with a real photo of someone) ?

[@Tsero0v0]

What if we use multiply controlnet and all of them with guess mode or without it? Is there any different?

[@asciikat2571]

Craaaaaazzzzzzy! Love it, You are the strongest

[@rincondesalva]

Awesome video!... Could you please (in any control-net video) tell us which requirements about VRAM specially) should everyone need in order to run in local properly this addon?...I guess my rtx3070 mobile with 8gb is almost uncapable...

[@haan3388]

do you have an in depth vid on the inpaint trick you explained around 5:00?

[@alexb6969]

You r the best! This stuff is so cool! I can't stop to admire this new tool)

[@KonstantinRozumny]

Great video! How to add two different characters to the same background?

[@RyanBigNose]

can you do a totorial on "Inpaint batch mask directory (required for inpaint batch processing only)" there is no tutorials for it

[@fredmcveigh9877]

Very informative and inspiring .Thankyou .

[@zhizui]

omg!!这真的太有用了！看一次可能都学不完里面的知识

[@gameswithoutfrontears416]

Thanks. Wow. Pretty cool stuff. So many possibilities 👍

[@Actual_CT]

finally 3d texture workflow assistance...with precision

[@Eagleshadow]

Super useful, thanks!

[@leowei771]

Good lord, this is moving so fast that I can barely keep up with all this new stuff.

[@cienciaemutopia]

Nice tips and video,from brazil

[@hatonafox5170]

You sir are my hero!

[@OndrejL]

wow bruh great discoveries and thx for the share

[@jimdelsol1941]

Absolutly amazing. Thank you.

[@megumin4625]

The multi controlnets are now tabbed. Niiice

[@GerwaldJensRadsma]

WooooOOOooooW Good as always!

[@mistercapitale]

Aitrepreneur, is it possible to use a logo and place it on a shirt with any of control net models?

[@muuuuuud]

Thanks K for the powerful knowledge, have a wonderful weekend! ^__^

[@lntcmusik]

This is great! Thanks for sharing 🙂 Do you have an idea on how to create an animated, let's say, sticker for any messanger using Stable Diffusion? That would be interesting.

[@danilo_88]

This is pretty cool. I love your videos. You could use more models other than just anime and cartoon.

[@velly027]

I don't know if it's possible yet because I lost the overview of the newest developments. I have 2 trained models of 2 different characters. I want place both characters in one image with a chosen background. Both characters should be arranged with open pose. Then the whole image should be created with diffusion in one step so that the lightnings and the shadows are correct over the whole image. Is this possible yet? Maybe I must wait another week :)

[@goldenknowledge5914]

Its crazy. So much things to learn just for one feature. It is indeed the age of AI

[@doobertoob4266]

Lol finally. Guess mode is pretty much what Midjourney does by default

[@jtmcdole]

Now we just need a node-based flow plugin to make the rotoscoping easier.

[@Nickknows00]

Now all I want to is to have SD generate my character in new poses

[@FleischYT]

great tip, appreciated

[@Dex_1M]

i think using 2 control nets, one with the canny or depth, and one with the pose, and playing only with the pose around and not changing the canny or depth, well make you control the characters movement while keeping the details, i assume this is true and if you can get this right, you can make a lot of images, and thus having an animation.

[@fernandoz6329]

Awesome tricks

[@in2thedark]

You are the man! 🤖 thanks for the tips

[@mandapanda5252]

I have been going through this one by one, not getting any of the same results, feeling a bit defeated. I was most excited about transferring a character onto a background, but no matter what I do it changes the character completely. I have adjusted the denoise and weight so many times. Perhaps it struggles with full body characters?

[@cameron7814]

If you figure this out please lmk, also having trouble transferring characters into a background. The depth and canny models seem to be working fine but the character always shows up non detailed and almost transparent to the background no matter which settings I change.

[@thanksfernuthin]

Do we have anything to aim the eyes with yet? It's driving me nuts! All this control over poses and other things and my characters are always looking at the damned camera. I am super happy with everything they've added. So cool. Being able to say where the character is looking would be huge.

[@EpochEmerge]

Get the final image, copy HED preview and redraw the irises in the hed map and then use it with control net

[@EpochEmerge]

If you can't draw, get a photo of yourself in the same camera perspective and then Photoshop your eyes on the final image and again he'd + img2img with final image and denoise somewhere 0.4-0.5

[@sunlightg]

I would recommend you to use inpaint sketch option to change pupil location with the "only masked" option and low denoise.

[@thanksfernuthin]

@@EpochEmerge I don't know anything about HED. You got me all excited!!! I thought it might be like the pose one but for the head and eyes. Looks kind of like a sketch. To get the best stuff I pose a scene in Poser and render an image to bring into img2img complete with premade depth map. So the eyes are pointing exactly where I want them from the start. Or did I try the wrong thing?

[@EpochEmerge]

@@thanksfernuthin I think with control net there are sooo many workflows. We are only scratching the surface From my experience depth map for precision, normal maps for more 'artisic' and free output, same as hed gives more "freedom" to stable than canny

[@justinyuvilla8944]

How can you show a photograph into the reference ouput with the correct exact lighting style you want and have the ai match that very same lighting on your original image as well?

[@ItsBrody]

Can you add chapters to your videos? Sometimes I want to come back to specific parts but I can't remember where they are

[@philipZhang86]

great tutorial~~

[@cesar4729]

You missed the best sketch trick, were you use colors to add things or transform the picture. For that you need the same picture in controlnet and play around. Is way more powerful than the segmentation trick.

[@resonantone3284]

Installed it yesterday, and boy... I'm still struggling to figure it out. If it's going to do what I think, wow... Why do all the cool toys have to come out when I'm on a deadline?!??!? It's a plot I tell you!

[@jurandfantom]

Interesting thing. ControlNet f up, upscale extension. Even when not active, it still break SD and ultimate SD upscaler. Lucky just turn extension off fix everything :)

[@earthpond8043]

Bro you keep blowing my mind damn gj

[@flonixcorn]

Very nice!

[@Krougher]

That s insane !

[@nganpoiis8961]

Is there a contolnet rig available for blender?