ComfyUI - FreeU: You NEED This! Upgrade any model, no additional time, training, or cost!

I want to introduce a brand new node that was just added by Comfy to his stable diffusion system this morning, it's called FreeU. The concept here is you are able to change some of the underlying contribution mechanisms of the u-net, and this is the core of stable diffusion. The results tend to be much better, and it doesn't slow us down or cost any additional GPU load! #freeu #stablediffusion #comfy #comfyui #aiart
You can read all out the methods behind this node here: chenyangsi.top/FreeU
Interested in the finished graph and in supporting the channel? Come on over to the dark side! :-)
/ @sedetweiler

[@roktecha]

repost from Reddit: I focused mostly on context so far. E.g. I was trying to get my model to drive a fantasy scene but as a photo realistic model it was struggling -- I was seeing concrete, glass, etc in the mid and background. To try to balance this out I: - Increased weight of B1 which controls higher concepts like "fantasy", reducing S1 (i.e. B1 skip factor), increased S2 (i.e. B2 skip factor) - My settings: B1:1.3, B2:1, S1.2, S2: 3.3

[@LeKhang98]

2:42 I found a copy paste trick - Ctrl C to copy a node then Ctrl+Shift+V to paste that node with all of its connection so you don't have to connect them again and again. You can also do this for multiple nodes too. Thank you for this useful video.

[@sedetweiler]

Yes, I do that in a lot of these videos. I just do it the hard way as I am thinking through the process from time to time.

[@edsonjr-dev]

Thank you, as always with the best tips 💜💜💜

[@sedetweiler]

You are so welcome!

[@TR-707]

this gets super funky if you run some sd turbo lcm combo and have near realtime feedback.. super nice to see the generation change

[@murphylanga]

Thanks for your great videos. It seems the B values control the style and image quality. With values around one for both I get very good results. the S-values control the quality/number (e.g. number of legs of an animal) of the objects on the image, here I get the best results with values around and higher for S1 and with low values between 0-0.3 for S2. When the distance between S1 and S2 becomes smaller, more and more objects are generated.

[@sedetweiler]

think of it like tiny details and textures are the S, and the larger blobs are B. Normally the S and B interplay in a way that tends to find balance, but perhaps that balance isn't ideal for your task or the model in general. This is a nice way to balance the high (S) and low (B) frequencies of the image.

[@marjolein_pas]

I like it a lot. Not sure what is does, but it does something and I love some variation

[@Vestu]

Gotta try this today!

[@sedetweiler]

It's pretty interesting!

[@courtneyb6154]

Great video and thanks for the headsup!! Care to share a comfy image link in the description using this new stuff so that us noobs can just drag the image into comfy to see how everything works? I am so new that I don't even know how to create the "model" node that you have in your pipeline lol

[@sedetweiler]

I do, but that is in the community area for members supporting the channel. Quite a collection of those in there now.

[@prattner67]

Seems to be showing a blurrier background and a better detailed foreground with the settings you have here. However, it is also coming with a "happy kodak color" which is both good and bad. I think the background greenery is too green, though.

[@JpMidnight]

Thank you very much for the great videos. Watching your videos, now I can build nodes to generate images and I switched from A1111. This node makes me feel like I'm using a different checkpoint. Fix value 0.9/0.2, S1/S2. B1/B2, 1.1/1.3 makes the image very colour saturated, extremetly detail acentuated (i.e. abs are more visible). Skins are more like painted. 3D checkpoint is like 2.8D one. lowering down to B1/B2 1.0/1.1 is still the same, but less acentuated. However, 0.9/0.9 makes images rather less sharp in my case (this probably is different for different checkpoints). I'm gonna try B1/B2 0.9/1.0 to see what's up. I'm generating portrait images only.

[@JpMidnight]

I think where to put the node is also very challenging. If you have multiple segments to iteratively upscale, then the effect is accentuated a lot if you put the node at the beggining of the flow.

[@tsutsen1412]

I was wondering is there a way to easily generate tiling images in Comfy? I quite miss this feature after switching from A1111

[@AI.ImaGen]

Really interesting node. I was looking for something like that. Other and easy way: put only FreeU node between checkpoint model and KSampler model and you get something like A1111 extras but with more power. Perfect to keep the global composition and make little variations. *** best values removed because depends of...many things :) ***

[@sedetweiler]

It gets even more interesting when you split the samplers and use different settings for the latter noise. I keep finding new ways to use this. It's really interesting!

[@AI.ImaGen]

You see, now it's FreeU Advanced and this one got a lot of new parameters... and look very hard to understand. @@sedetweiler

[@sedetweiler]

I sure hope they start to annotate these, as I agree it is a lot of mumbling if we cant understand them.

[@TransformXRED]

With 1.5, I have good results with these - 0.9, 0.9, 0.9, 0.2 It adds some contrast (which makes the images visually sharper), and a tad of more saturation... but not too much. With some seeds, there are more differences with and without FreeU. It makes euler_a more "realistic" too in my opinion, less plastic

[@sedetweiler]

Good to know! I think there is a lot of benefit here, and I am also pondering some advanced ideas already. Pretty great node! Thanks for sharing your findings!

[@sedetweiler]

It looks like the values you have are quite different from this person. Often the model and goal can shift these, so be careful not to over generalize settings. tinyurl.com/yn9gj4c8

[@lukeovermind]

@@sedetweiler sigh the link above is the exact setting I gave on your live stream last week. I am still using SD1.5, but I knew about this website page and thought I can quickly post it lol but that backfired spectacularly lol

[@jdsguam]

Using your settings on an SD1.5 Model and it improved the background, added some detail and the subject is looking at the viewer instead of mindless looking elsewhere.

[@sedetweiler]

That sounds like a great end result!

[@alwilson6471]

Du you prefer the output from this node to using the refiner? Can the refiner be used in addition to FreeU?

[@DoozyyTV]

Is that math node only for XL or also 1.5? Been wanting something like that

[@CMak3r]

Seems to be greatly improving consistency of text and lines in generations, and makes results more coherent. Try it with text and vector designs, I am blown away (Backbone: 1.1, skip forwarding: 1.1)

[@sedetweiler]

oh, that is interesting for sure! I have not tried it with text yet. thanks for the update and settings!

[@appolonius4108]

nice video! but im a bit confused because ive skipped a few videos. what video are you referencing that added comfymath node?

[@blacksage81]

It's most likely the one where he goes over upscaling as he used a scale by side node.

[@sedetweiler]

Yup! It is just used for the SDXL resolution, so we don't have to guess. You can skip it, but I highly recommend using it or something similar.

[@pedrogorilla483]

Have you used SAG on A1111? I’m wondering if there’s a SAG implementation for ComfyUI. Also wondering what it would do if both were coupled together.

[@sedetweiler]

SAG as in segment anything?

[@burghardvonkarger5718]

Does this go before or after LORAs? My first results: FreeU radically changed the style of my results from pastel and painterly to crisp and dramatic (not what I was looking for).

[@sedetweiler]

That is a great question! This is shiny and new, so we all have to play with it to see what works best. Also, you will need to play with those B and S number to see what works best. In a few days I am sure we will have more of an idea on where this sits in a common workflow, but for now, it's time to science the $hlT out of it!

[@WASasquatch]

It doesn't matter where it goes at this time.

[@ysy69]

Hi Scott, I noticed my SDXL template workflow I created from your previous tutorial is different and trying to match. Problem is that I am not able to find the Batch Size node anywhere. I tried dragging the batch_size and it doesn't show up in the menu. Thanks

[@ColdLinoleum]

You need to right click the Empty Latent Image node and convert batch size to input. after that, you can drag a noodle out from the “batch size” that’s now at the top of your empty latent and connect it to a primitive node. Just rename the primitive node to “batch size” and recolor it however you like.

[@Vestu]

By the way, why the width and height are 4096 on the clip text encoders?

[@sedetweiler]

That is the size they were trained on, but you can use something smaller if accuracy at this point of the workflow isn't important.

[@alpharay7519]

" just added by Comfy" - i can not find this node. Installed an extended FreeU node that is now available, but this one is actually not working. Where can i find this node?

[@sedetweiler]

Just have to be sure you always do an update and restart. so much changes every single day.

[@gordonbrinkmann]

Hmmm... I can't really find any settings to use this FreeU with SDXL. No matter which values I set going up and down, the results are always less detailed, more blunt... do not know how to say it. And faces and hands are the same at best, often worse than just the standard version. I tried it with the SDXL base model and with the Dreamshaper model which I use mostly. Can't get it to work for me as a tool of improvement...

[@sedetweiler]

I think it will be a bit before we know ideal settings for each situation. However, it is nice to have a tool that can mess with things at this level.

[@puoiripetere]

For SDXL: Final parameters selected: s1 = 0.6 s2 = 0.4 b1 = 1.1 b2 = 1.2

[@ryuugajailani]

the image result is too vibrant i think

[@sedetweiler]

Final based on your testing, or from a larger group coming to consensus?

[@lionhearto6238]

your video says there are parameters for sdxl but i cannot find any. do you know them?

[@sedetweiler]

There are not parameters for SDXL yet, I am sure we will have some soon.

[@JackTorcello]

At which level of Community can I d/load the graph?

[@sedetweiler]

Sponsor level or higher. Thank you for your support!

[@nonemous5]

what dictates how fast comfyui runs? i'm trying to improve processing speed. thank you

[@sedetweiler]

Just keep your steps to around 30 and don't load models you don't use. It will run as fast as possible if you just keep the processing down.

[@beatemero6718]

How do I install it? Did a git clone, but comfy says couldnt import custom node freeU.

[@sedetweiler]

It is a core node. Just update comfy as you would normally.

[@touristhawk]

where can i get that dropdown sdxl res node?

[@sedetweiler]

There are a few of them. I think I was using the Comfy Math nodes for this video.

[@touristhawk]

@@sedetweiler yeah i found it by luck

[@frazuppi4897]

is it for sdxl as well?

[@sedetweiler]

Yes, any model.

[@LouisGedo]

👋

[@sedetweiler]

:-)

[@THEAITIPS]

How do you do that model line handle?

[@sedetweiler]

Not sure I understand.

[@THEAITIPS]

@@sedetweiler I've found out. I was mentioning the rerouter.

[@hilbrandbos]

Resolutions being 'safe'? what is that about?

[@sedetweiler]

The model is trained at certain shapes and if you use something outside of that it will not be ideal. Most models early on were only square, but this one will allow other resolutions as long as you use one it can handle without effort.

[@badis103]

doesn't support torch directml :(

[@sedetweiler]

It's day 1. Give them a moment to broaden its capability. 😉

[@hleet]

So nobody knows what B and S stands for ? (yeah except BullSh*t haha...). It kinds of change something on the picture, I don't know what it really does more or less to the final result. I tought at first that the fingers seems better, but not as better as I would like to. There is the FreeU Advanced custom node that is hell more confusing with other numbers to play with. At the end, I would rather prefer a "Ksampler" that incorporate this freeu so you don't need to scratch your head upon what to do with B and S (...)

[@sedetweiler]

If you look at the paper for this, one is the baseline and one is for the skip-factor. Read on and you can level up. :-) github.com/ChenyangSi/FreeU

[@MikevomMars]

When it comes to AI image generation, we already have to deal with 1 million "placebo parameters" - do we really need MORE of them? Come on, I mean... it's really just a placebo, isn't it? 🤔

[@osten222312]

Nope, it obviously does something. What you expect is for people to fully understand the technology, which isn't the case. We try things. We don't know, anyone, what the hell is going on in these models

[@MikevomMars]

@@osten222312 I might give it a try but every time such a tool or method comes up, males (which seem to be very sensible for "magic numbers") are posting their "magic numbers" throughout the entire web, so it quickly turns more into superstition instead of science. But ok, I'll do a comparison since it is easy to do with the node workflow shown here.

[@sedetweiler]

I remember when you had to direct email through different hubs to reach your destination. Maybe you want to step back for a few years, or maybe you enjoy this bleeding edge, it's not for everyone.

[@JordanPNW]

This adds seconds to my diffusions. And only minor differences, no improvements.