ComfyUI Just Got Better: Finally, a Fix for Flux Upscaling and 2 more Mindblowing ControlNet Models

[@TheSneakyRobot]

Here's some freebies: Workflow: openart.ai/workflows/sneakyrobot/sneaky_robot-gguf_fp8-workflow-plus-controlnet-upscaler-and-controlnet/wUHlwiibPMgVxf7RPZsU Model list. Flux Dev GGUF: huggingface.co/city96/FLUX.1-dev-gguf/tree/main Flux Dev fp8: huggingface.co/XLabs-AI/flux-dev-fp8/blob/main/flux-dev-fp8.safetensors T5 GGUF Model: huggingface.co/city96/t5-v1_1-xxl-encoder-gguf/blob/main/t5-v1_1-xxl-encoder-Q4_K_M.gguf T5 FP8 Model: huggingface.co/stabilityai/stable-diffusion-3-medium/tree/main NB: you'll need to login or sign and agree to the terms of stabilityai to download the fp8 model Clip VIT: huggingface.co/zer0int/CLIP-GmP-ViT-L-14/blob/main/ViT-L-14-TEXT-detail-improved-hiT-GmP-TE-only-HF.safetensors Flux hyper model Lora: huggingface.co/ByteDance/Hyper-SD Flux dev add detail lora: huggingface.co/Shakker-Labs/FLUX.1-dev-LoRA-add-details jasperai/Flux.1-dev-Controlnet-Upscalerr: huggingface.co/jasperai/Flux.1-dev-Controlnet-Upscaler/blob/main/diffusion_pytorch_model.safetensors jasperai/Flux.1-dev-Controlnet-Depth: huggingface.co/jasperai/Flux.1-dev-Controlnet-Depth/blob/main/diffusion_pytorch_model.safetensors jasperai/Flux.1-dev-Controlnet-Surface-Normals: huggingface.co/jasperai/Flux.1-dev-Controlnet-Surface-Normals/blob/main/diffusion_pytorch_model.safetensors Controlnet Depth: huggingface.co/Shakker-Labs/FLUX.1-dev-ControlNet-Depth/blob/main/diffusion_pytorch_model.safetensors Contorlnet Canny: huggingface.co/InstantX/FLUX.1-dev-Controlnet-Canny/blob/main/diffusion_pytorch_model.safetensors Mistoline Controlnet: huggingface.co/TheMistoAI/MistoLine_Flux.dev/blob/main/mistoline_flux.dev_v1.safetensors Mistoline GitHub: github.com/TheMistoAI/MistoControlNet-Flux-dev

[@yiluwididreaming6732]

ahhh...there we go....TA!!!!

[@john2stai]

I think for the upscaler controlnet, it is best to paired it with tiled diffusion! Also, when I got grainy result, I usually tried to lower the controlnet strength or the end time. The grainy results to me, look like overcooked images.

[@WhatsThisStickyStuff]

How do you use tiled diffusion with flux? I just get the image repeating a bunch of times

[@john2stai]

@@WhatsThisStickyStuff I am using regular ksampler and denoise strength around 0.65

[@john2stai]

@@WhatsThisStickyStuff also, it's best to leave the prompts empty and since I am using regular ksampler, I left the cfg to 1 without the use of cfg guidance.

[@ultimategolfarchives4746]

Great stuff man! Thanks for the detailed guide. One potential enhancement suggestion: considering adding a brief, basic demo of a raw workflow at the beginning to set the foundation, before diving into the more detailed workflow. Would help new viewers Great job 👌

[@TheSneakyRobot]

Thanks for the suggestion, will do that next time.

[@gloorbit5471]

Installing the missing nodes and totally messed up my comfy, I guess I'm going to have to do a clean install. For some reason there are no snapshots in the scripts folder in comfy manager. Note to self. Make a snapshot whenever making changes on this scale.

[@TheSneakyRobot]

Busy with a solution that requires les nodes will make it available soon

[@Cingku]

Same here. The issue seemed to be with the ComfyUI layer style node by chflame. It really messed up my ComfyUI installation. Reinstalling everything took a whole day, mostly because I used the portable version. I didn’t expect any problems with installing a new node, as I had never faced issues before, but this one node was different. So Lesson learned: always back up your ComfyUI folder before installing new nodes or updating ComfyUI.

[@TheSneakyRobot]

I Finally found the Group of Nodes responsible For the Problem. Will rectify in the next video

[@TheSneakyRobot]

@cpttiles2266 I've removed half the packages, check the new video

[@haqthat]

I keep getting "Key Error 947 Impact Switch" which is attached to the dual clip loader, I can't even get an image to generate. 2024-10-10 10:56:44,950 - root - ERROR - * ImpactSwitch 947: 2024-10-10 10:56:44,951 - root - ERROR - - Required input is missing: select

[@thibaudherbert3144]

After a fresh install of comfyui your workflow is using many corruped nodes apparently, many nodes are conflicting or can't be used even though i install all the missing nodes, update everything, starting fresh. it's sad cause your worfklow was the best. I even tried to install all the custom nodes manually with git clone but nothing changes. Everything is conflicting

[@TheSneakyRobot]

I will try and test it on a clean install of ComfyUI and let you know what conflicts there are and how to fix it, Thanx for letting me know

[@TheSneakyRobot]

Opologies for the lengthy explanation. I installed comfy on a different drive, and this is what I found: 1. when you install the nodes don't install all of them at once. - I found that installing 3 nodes at a time works perfectly. - another thing I've noticed is that it takes a while to install some nodes. 2. there are two nodes that's not part of the comfyUi manager and that the perfection styler and mistoline nodes. Install perfection styler: - Go to the custom nodes folder, right click on empty space in the folder and then click on open in terminal. - next copy the following in the terminal and press enter: git clone github.com/TripleHeadedMonkey/ComfyUI_MileHighStyler.git - Restart comfyUI and it will be installed automatically. Install Mistoline: First thing you need to do is to Navigate to the custom note's folder, (ComfyUI_windows_portable\ComfyUI\custom_nodes), right click on empty space inside the folder and select open in terminal. Next paste the following in the terminal that's now open, git clone github.com/TheMistoAI/MistoControlNet-Flux-dev.git . Press enter and wait for it to install. After it's done installing close the terminal and start comfyUI again, it will auto install the note automatically on startup. 3. As for the conflicts: There really isn't any. the only thing I've seen is that some packages have nodes inside them that conflict with each other. you'll be fine as long as those are not in the workflow. I'll clarify in the next video.

[@thibaudherbert3144]

@@TheSneakyRobot ok i'll reinstall comfy and try again, thanks for your time, ill let u know if it fixed it or not

[@thibaudherbert3144]

@@TheSneakyRobot Hi, so i followed your instructions and it worked ! But after some tests, there is something wierd that made me go back to my old workflow : the outpu quality. I don't know what causes this (certainly the ksampler block), but the output of each image is bad. I've ran dozen of comparisons with same prompts, loras, format, resolutions across your worflows and mine and it seems the prompt understanding and final image isn't the same, the image does not look sharp, and the ouput is way less structured than my other worflow. I guess it's do to the sampler

[@npgolding]

This is so well done!

[@TheSneakyRobot]

Thanks'

[@rodszera]

Great job! I downloaded all the files and everything seems to be in the right folder because I can select and change the files. When I ran for the first time Florence models seems to be downloaded to the LLM folder but the process stucks in the Switch (Any) returning the error ImpactSwitch 'select'. The log show that it is expecting some models in the LLM folder and didn't find the vae and other models. But the models are there, not with the same subfolder like FLUX/Lora... I placed directly in the lora's folder and my schnell vae is just ae.safetensors ... I selected the files in the node graph but I am stuck in this Switch node. What I am doing wrong?

[@DrMacabre]

is it normal behavior for the llm in auto prompt to use the CPU instead of GPU?, when using searge llm in my own workflow, i have not experienced this.

[@defidigest9]

Does it use hiresfix?

[@gloorbit5471]

diffusion_pytorch_model.safetensors. This file keeps popping up for different uses. Are we supposed to rename this file? Update and installing the missing nodes made a right mess of my comfyui. And the workflow is now just sitting there blinking at me.

[@TheSneakyRobot]

Sorry for that. every model called diffusion_pytorch_model.safetensors should be renamed before you safe it. for example, if you download the ControlNet upscaler it should be renamed to "Flux.1-dev-Controlnet-Upscaler.safetensors" this will make it easy for you easily recognize the files. I made the same mistake. as for installing the nodes don't install them all at once. I am trying to come up with a simpler solution with less notes for the next video.

[@mal-avcisi9783]

hey bro, do you use another comfpyUI version, your toolsbars etc on top etc. looks different ?

[@TheSneakyRobot]

It's the new comfyUI front end. All you have to do is to update comfy, and in the settings change the menu to top.

[@mal-avcisi9783]

@@TheSneakyRobot thanks

[@cemilhaci2]

thanks, workflow link not working

[@TheSneakyRobot]

Fixed it. here's the workflow. Please let me know how I can improve on it. openart.ai/workflows/sneakyrobot/sneaky_robot-gguf_fp8-workflow-plus-controlnet-upscaler-and-controlnet/wUHlwiibPMgVxf7RPZsU

[@cemilhaci2]

Sorry, I solved the problem, false alarm :D

[@TheSneakyRobot]

glad it's working.

[@kamaelfleur]

thanks for the workflow!. The best so far of all I have tried. the author deserves more subscriptions for quality content

[@TheSneakyRobot]

Thank you too! Much appreciated.

[@kamaelfleur]

@@TheSneakyRobotsome suggestions that surfaced during testing: may be we can optimize the process of selecting the downscale factor and upscale multiplier, perhaps with the help of mathematical nodes, make the calculation of these factors automatically based on the original resolution of the image. Based on the specifics of the upscaler, the final resolution for the upscale should not exceed 2k. So you can create switches presets to select for example: 1k1-repaint original, 1500 and 2048 pixels. The idea is that the upscale factor would be set automatically based on the uploaded image. It may be necessary to fix the downscale factor at 0.5.

[@TheSneakyRobot]

Thats a great Idea. Don't know why I haven't thought of that. Thanx for the Idea!

[@AInfectados]

Better than SUPIR?

[@TheSneakyRobot]

As far as I've tested Supir is GPU intensive, Plus it requires a second Checkpoint, preferably a sdxl checkpoint, which will take more or less an hour on a 8GB GPU, and way longer if you use flux to upscale. So Yes, For Low VRam Systems Its better.

[@gnsdgabriel]

Those 'upscale by' and 'downscale by' are not a very smart way to do things.

[@JarppaGuru]

no panix yet again same another comes. flux not get missed. it is just sdxl what was sd

[@thibaudherbert3144]

Honnestly your workflow is certainly the most well thought / usable / versatile. Most of comfyui workflows on youtube are either not complete or uselessely complexe for no real benefits. Yours is the right balance. The only problem is the upscaler, to me it's not usable for now, way too grainy, the output is worse than original. It's not like a 8x tile super SD upscaler, but thank you for your workflow !

[@TheSneakyRobot]

Thanx, Try 4 x upscale with 8 steps. takes a bit longer but removes the grain completely.

[@thibaudherbert3144]

@@TheSneakyRobot ok ill try that thx

[@thibaudherbert3144]

@@TheSneakyRobot sadely there is a problem with the node under the sdxl prompt styler in text 2 img. It's impossible to download the node even after updating everything etc. I've even fresh install comfyui still does not work i don't know how to correct this or find an alternative. I think it's the perfection styler block

[@thibaudherbert3144]

it's apparently caused by ComfyUI-Universal-Style, many people have this problem, but i can't find a way to replace all the missing nodes in your workflow without this node ='( hope you can find an alternative

[@TheSneakyRobot]

This is the perfection styler which is part Mile high styler package. you need to manually install it. - Go to the custom nodes folder, right click on empty space in the folder and then click on open in terminal. - next copy the following in the terminal and press enter: git clone github.com/TripleHeadedMonkey/ComfyUI_MileHighStyler.git - Restart comfyUI and it will be installed automatically. The other is the Mistoline which also has to be manualy installed: First thing you need to do is to Navigate to the custom note's folder, (ComfyUI_windows_portable\ComfyUI\custom_nodes), right click on empty space inside the folder and select open in terminal. Next paste the following in the terminal that's now open, git clone github.com/TheMistoAI/MistoControlNet-Flux-dev.git . Press enter and wait for it to install. After it's done installing close the terminal and start comfyUI again, it will auto install the note automatically on startup,

[@KalLif-k3i]

Kinda looks like tenofas workflow...