L2: Cool Text 2 Image Trick in ComfyUI - Comfy Academy

In this Part of Comfy Academy we build our very first Workflow with simple Text 2 Image. But then I will also show you some cool tricks that use Laten Image Input and also ControlNet to get stunning Results and Variations with the same Image Composition.
Find my Workflow here: openart.ai/workflows/oliviosa...
#### Join and Support me ####
Buy me a Coffee: www.buymeacoffee.com/oliviotu...
Join my Facebook Group: / theairevolution
Joint my Discord Group: / discord
AI Newsletter: oliviotutorials.podia.com/new...
Support me on Patreon: / sarikas

[@OlivioSarikas]

Update: I limited the Workflow to only what is shown in the video, so it is more focused on the Lecture

[@ceelentzpotter2110]

the ComfyUI can't find AV_ControlNetPreprocessor even after I fully install and update as the Manage tab said.

[@CoolSongsweekly]

Same issue! Even on the openart server@@ceelentzpotter2110

[@TheGalacticIndian]

@@ceelentzpotter2110 I have the same issue unfortunately🤷‍♂Solved it by and replacing AV_ControlNetPreprocessor with (right clicking on empty canvas space) Add Node/ControlNet Preprocessors/AIO Aux Preprocessor. Works like a charm👍

[@LunarFerns]

@@TheGalacticIndian thank you so much!

[@TheYules]

same here@@ceelentzpotter2110

[@madsnylarsen]

ohh man, i wish you had shown the entire build flow, the suddenly jumping in progression it's very confusing, especially when you are learning, while you point at a node and explaining what it does my focus is struggling with figure out how we got there, how i can get to that point, where each of the lines goes, what is an VAE, what is a controlnode etc. meaning there's very little attention on what you are trying to teach, if you instead had build it together with us,, isolated the function like explaining controlnet before using it in a larger setting, it would be way easier to follow along and learn... sorry if i am a bit harsh in my critique, i am just a little dissapointed since i looked forward to follow this series, i enjoy your work and have a ton of fun following how you explore the realm of AI :), now i'll watch the video again a few times.

[@MikevomMars]

4:00 - It'd be helpful if you explained WHY one should use a custom VAE instead of a checkpoint's built-in. Actually, NO ONE ever explained this so I am wondering about hat until today 🤔

[@ethanhorizon]

This. Anyone?

[@coldbeyond]

Custom VAE is by choise of course. Some models have baked VAE, so they don't require a custom VAE. But still you can use it. I'm new to Comfyui, but let's say you use Dreamshaper 6.31 VAE baked checkpoint in A1111; you can either use an additional VAE or you can set the VAE to Automatic.

[@MikevomMars]

@@coldbeyond That should be clear to everyone - the question, however, is: are there any ADVANTAGES in using different VAE's and why not using the baked-in ones?

[@nikgrid]

I'm starting to see the benefit of comfy. Thanks Olivio! Excellent stuff

[@Pguzpet]

This video is pure gold, thank you very much Olivio Sarikas.

[@jesterjunk]

(( 4:24 )) Preview images DO save, but they save to the /temp directory instead of the /output directory. The saved preview images also still include the Workflow metadata. They remain there until ComfyUI is restarted, then that folder is deleted with all its contents.

[@tyegold2020]

Nice! Really getting into this now, love how calm and easy your voice is, makes it so easy to follow.

[@amarmayekar7795]

Thank you very much Olivio Sarikas

[@imagineArtsLaboratory]

Thanks a Billion Olivio! Helping Artists and Creatives become more efficient in technology

[@user-uh7vb4so4k]

Merci pour ce partage !!!! Vous êtes génial !!!!

[@chiptaylor1124]

Olivio, thank you so much for your tutorials!

[@kirillkhomenko_]

I like the way you explain it. It's very simple and straightforward. Thanks

[@aivideos322]

If using aux preprocessors just use aio preprocessor and choose midas or the other depth maps. Olivio here is using default controlnet. They can both do the same thing.

[@SaavedraCG]

I don't think Node Preprocessor

[@Alex-yi3yw]

Thank you very much for the lesson!

[@JustMe-ty2rp]

Definitely some assumptions left to be made by the people jumping in at this point, but I like learning that way. Rather than download the workflow, I challenged myself to fill in the gaps and figure it all out on my own and learned quite a bit in the process. Took me a while to figure out how to get the preprocessors and controlnet models installed - especially given my setup is a little different as I am running ComfyUI via the ModelSurge Auto1111 extension, and I have my models in a non-standard area outside of my WSL2 filesystem; but here I am 3 days later and I got it all working swimmingly, with extras!!! Thank you for the wonderful tutorials @Olivio. You are appreciated. :)

[@TheGalacticIndian]

Please share this apparently secret knowledge - how to get the preprocessors and install ControlNet models?

[@majinvegeta5393]

Thanks for your lesson, it helps a lot, thanks !

[@gwcstudio]

Super useful, Olivio!

[@fabiotgarcia2]

How can I add the Controlnet Preprocessor since its not appearing on the list of nodes?

[@Shisgara77]

I had the same problem. But I think I solved it - in Manager in Custom Nodes I uninstalled and reinstalled the Art Ventura Custom Node used in this tutorial and all preprocessors appeared after refresh)))

[@bonechieftain]

@@Shisgara77 this seemed to work for me as well. instead of double clicking and typing the search field for controlnet preprocessor, i right-clicked art venture,->loaders-.controlnet preprocessor

[@vannoo67]

@@Shisgara77 I had a tough time finding this node. I downloaded Olivio's workflow, expecting to have the node marked as missing, but no, it was there. It's just tricky to find in the AddNode dialog when searching for ControlNet. It's actually called 'AV_ControlNetPreprocessor' (I guess AV is Art Venture) Update: I also needed to install "ComfyUI's ControlNet Auxiliary Preprocessors" before I could see more than 3 Preprocessors in the drop down list

[@sethmunich4062]

What is Art Ventura?@@Shisgara77

[@sethmunich4062]

Oh Art Venture!

[@admireruk]

Nice vid, If you set the seed to the same fixed value on each sampler and set to progress by one each time. You are going to get more consistency between images.

[@evertonfagner]

fantastico. obrigado por compartilhar conosco.

[@LunarFerns]

Thank you for the video! It was tricky to understand how to install everything, but I did it

[@sethmunich4062]

How did you install Controlnet preprocessor and the control_depth-fp16 model?

[@LunarFerns]

@@sethmunich4062 to instail model you need to put the file here ...\ComfyUI\models\controlnet and for controlnet preprocessor you need to open manager and find custom node ComfyUI's ControlNet Auxiliary Preprocessors from Fannovel16 Let me know if this doesn't work

[@pavi013]

Very nice, i learned the basic use pretty quickly.

[@jccluaviz]

Thanks for your awesome video, my friend. Just playing around with nodes. Connected the latent output from the first Ksampler to the input latent from a second Ksampler and lowered a bit the denoise from this second Ksampler.Found that gives me some similar results to use contronet adapters. The question is ... using those contronet adapters gives us more consistency making the result look more similar at the original image ?. Thanks for all

[@kuromiLayfe]

definitely recommend getting the anything anywhere nodes to help keep workflows clean (you connect a node output to the anything anywhere node and it will automatically connect that output to all inputs of the same type (very useful with assigning models and VAE without having a bazillion lines in complex workflows). Also recommend coloring the text-encode nodes : Green for Positive and Red for Negative (will make it a bit clearer at a distance which is what)

[@OlivioSarikas]

That's nice for your private workflows, but not good for when you want to share them or teach them, because then people can't see anymore where your cables are going. That's why i'm not going to use it in any of my workflows

[@kuromiLayfe]

@@OlivioSarikas It adds a toggle to show/hide the cables (even animates them depending in the type of node it is transferring data from and to) like i said it is for the much more complex workflows that have like a massive logic group going on using multiple ksamplers and controlnets etc)

[@tonyblack1981]

No no no! for this level of information I must both like, comment and subscribe

[@cemilhaci2]

Thank you for an understandable and fluent education. How can I preview the recorded images in the interface? They appeared in the section below in some of the trainings I watched.

[@zizhdizzabagus456]

great course, thank u so much. i was alright with autmatic 1111 but comfy was not easy to understand

[@helloRick618]

so cool

[@ZaindTV]

hey! i love youre content and channel! Thank you for work and keep going!

[@coldbeyond]

5:37 I wrote the same prompts and almost got the exact results like you, how cool is that 😄

[@phaylali]

this video explained perfectly the superiority of comfy ui , and made me give up on vladmandic automatic

[@annavendemia9127]

Thank you for these lessons Olivio , they are very clear and quite easy to follow for a beginner. There is a problem with the ControlNet Processor in the workflow, both launched on the clud and run locally. It doesn't seem to be working anymore, so after the first generated image the othes can not follow. can it be reaplced with something else?

[@Fiercesnake1994]

It would be really helpful if you would show how to set up comfyUI properly. Like install the models and comfyUI itself. :)

[@Bullitain]

this is the 2nd lesson, he did all that in the 1st

[@arieldrach809]

@OlivioSarikas I followed the first video and I'm trying to follow this video building the node but I don't have any checkpoints. What I'm missing

[@canehoyer]

Where do I get the dreamshaper models? Can't find them in the manager. And what models would you suggest today?

[@dxnxz53]

Thank you for the video! Where can i get the ControlNet Preprocessor Node ?

[@ogrekogrek]

Thank you Olivio for tutorial. What is difference between Batch_size and batch count.

[@lefourbe5596]

Batch size : generate images in a all in one batch. Use more VRAM but is faster. Batch count : generate a number of image one after the other. Use the same amount of VRAM but is slower. You can combine them both.

[@ogrekogrek]

@@lefourbe5596 thank you

[@pepschp5355]

There is another instance, where your workflow may be very important. Sometimes seemingly small prompts can change your image in a way you don't want to, because the model was trained that way and has some prejudicess. With your workflow you can circumvent the models prejudices and can approach your imagined outcome step by step.

[@shtogie5972]

I'm following along but running into a speedbump @9:00. I can't figure out which node to load in place of the "Controlnet Preprocessor" Node. That node does not exist in my environment and I instead have a package called Controlnot Auxillary Preprocessors, yet I still can't find any node similar to the Controlnet Preprocessor Node with the CNET_Name output.

[@freakyninjaman3]

I ran into the same problem. With ControlNet Auxiliary Preprocessors installed, search for "MiDaS Depth Map" (or right click the workspace > Add Node > ControlNet Preprocessors > Normal and Depth Estimators > MiDaS Depth Map). This node will accomplish the same result as the tutorial (I just kept the default settings). I'm not sure what Olivio used to get that ControlNet Preprocessors node. Let me know if you figure it out :)

[@sunl3583]

use MiDas Depth Map as substitute for the ControlNet Preprocessor, the rest is the same

[@moky4mido]

@@sunl3583 and where the output goes?

[@nowerameditationmusic3256]

@@sunl3583 Thank you, this worked for me. Note that you also need to google the control depth fp16 model safetensors and download it to get it to work.

[@fabiotgarcia2]

MiDas is not connecting with the VAE Decode@@sunl3583

[@dimitrilucy]

I love your tutorials, thanks for that ! sooo Useful ! I have a issue and I don't know how to fix it : When I try to open your graph, I have a error : When loading the graph, the following node types were not found: AV_ControlNetPreprocessor This 3 packages are installed : ComfyUI's ControlNet Auxiliary Preprocessors ComfyUI_Comfyroll_CustomNodes ControlNet Auxiliar

[@pavi013]

Problem i been having is not only to get consistent face but also a body, i guess depth map can help with that 👍

[@walidflux]

how can i add my own lora to the workflow

[@shshsh-zy5qq]

hey Olivio thank you for the amazing tutorials! I am following this tutorial and I keep getting Error occured when executing CheckpointLoaderSimple: Error while deserializing header : HeaderTooSmall

[@acezgaming2793]

how can i install checkpoint? I have a mac

[@Vestu]

UPDATE: It started working when I uninstalled some Aux...somethingsomething... preprocessor custom node I had installed some time earlier. Then restarted ComfyUI, then REINSTALLED it, restarted ComfyUI and now it works. I get an error saying "Error occurred when executing AV_ControlNetPreprocessor: 'resolution' is not in list" on the lower-most workflow.

[@workplaydie]

It says I don't have the AV_ControlNetPreprocessor node when I load the workflow? Edit: It might be really nice for the beginners for you to note that you are using custom nodes & that openart lists custom nodes and links the repositories. It took me a while to it figure out!

[@_carsonjones]

Following along with this and the Controlnet Preprocessor seems to be obsolete. Any suggestions for an updated workflow that achieves the same / similar output?

[@matale66]

Same here. Trying to install comfyui-art-venture custom nodes but install is failing. Did you find a solution?

[@ethanhorizon]

9:37 Apply ControlNet - the strength is tone down to 0.83, any special reason? What if I use 1 or more than 1?

[@RadTechDad]

My KSampler keeps showing the preview. How did you hide it?

[@iulianbabulea4997]

I do not have the manager option visible, how can I fix this?

[@MrKairot]

I downloaded your workflow from OpenArt, but the images are coming out very blurry. I didn't change any settings.

[@yingyingdeng1293]

don't know why the image is strange, eg: the car has more than four lights or wheels. is it the problem for the prompt text? appreciate for your kind reply ...

[@nuejidub]

i can't find the controlnet preprocessor - instead i sent it to MiDas Depth Map directly. The preview image looks similar, but it won't work after that.

[@OlivioSarikas]

you need the preprocessor, otherwise it is useing the color image and not a depth map. if you have the comfyui manager installed, you can install missing nodes with it very easily

[@ist1jano]

@@OlivioSarikas The ControlNet Preprocessor got a red border indicating an issue, which is "depth_midas" is missing. How to fix it please? I run it in the cloud. Thanks and for the great tutorials too !

[@nuejidub]

@@OlivioSarikas thanks! that worked for me!

[@bobacks]

@@OlivioSarikas I just can't get this to work. It shows RED in the manager.

[@samwalker4442]

Might be a dumb question as just learning...Why would you use a different VAE and not the one within the model? Just for variation?

[@samwalker4442]

p.s. great lecture...thank you

[@shubenoprod]

hi! I got an error whe i use the control net " deth-fp-16" 😢😢

[@kevinwang7340]

I can not find contronet Preproccessor in my comfy UI, can you please explain how that part is done ?

[@pastuh]

Someone mentioned, install extension by: comfyui-art-venture And choose: AV_ControlNetPreprocessor Also required: ComfyUI's ControlNet Auxiliary Preprocessors It will allow to choose preprocessor models

[@sickvr7680]

Gracias Olivioooooooo

[@TheLoreLabs]

ooof control net preprocessor nodes missing, checked reddit, don't even have the folder, downloaded via pinokio... anyone have a fix

[@ChristopherLensing]

I had the problem with the preprocessor but solved it via reinstalling art venture. But now, the only preprocessor I am able to select is "tile". I installed the models, also linked my A1111 controlnet-models and also copied them into "models/controlnet". Nothing helps. No preprocessors. Any ideas?

[@lennoyl]

I had the same problem. I solved it ( thanks to the guys in the comments on Olivio's workflow download page). you have to install, in addition of the art venture node you installed, the 'ComfyUI's ControlNet Auxiliary Preprocessors' all the controlnet models will show after that.

[@sethmunich4062]

Worked for me too. this needs more likes!@@lennoyl

[@evertonfagner]

so frustrating. I'm stuck trying to get any ControlNet Preprocessor to work, and to no avail. one error after another, several installations and solutions, and nothing.

[@Jaysunn]

Solid tutorial otherwise

[@64kernel]

As a side comment, it's useful to use a different seed for every instance of the Ksampler or you'll get practically identical results.

[@MyAI6659]

Hi Olivio, may I ask you to change your UI of choice to Invoke. It has everything, A1111 style UI, workflow UI, Cavas, almost everything. Is there a real reason why AI youtubers prefer comfyui to invokeai?

[@JavierPortillo1]

I don't know whats different about my workflow (i built it by hand, not downloaded) but the images get different backgrounds, for example, for the asian woman the trees are japanese cherry trees, and for the african woman, the park has dirt paths and african looking trees. Your backgrounds look very similar.

[@JavierPortillo1]

I get better results if I reduce the denoising to 0.7 for the controlnet images. 👍

[@sethmunich4062]

I have the same result.

[@sethmunich4062]

I tried that and it made the backgrounds more similar but it made the woman's clothing less similar.@@JavierPortillo1

[@sethmunich4062]

try changing the strength of the Apply ControlNet node to .83 like he has it in the video@@JavierPortillo1

[@Kal-el23]

This is obviously for pros lol. Where do you recommend learning more about SD in general?

[@GamingDaveUK]

One bit of feedback, perhaps focus on sdxl (especially when you get to stuff like control net) 1.5 should really be religated to the past.

[@aivideos322]

tbh it should but its not advancing as fast as 1.5 . 1.5 is still getting more support for most things.

[@GamingDaveUK]

@@aivideos322 I find 1.5 prompt crafting to be a nightmare compared to SDXL. Problem is that 1.5 gets mentioned in guides like this more than SDXL and then people learn 1.5 and are reluctant to go to SDXL so it becomes a self fulfilling loop. i would say that SDXL has already suparssed 1.5 in everyway though

[@aivideos322]

​@@GamingDaveUK It has for sure, I love xl and use it A LOT, but still resort to 1.5 for animations, and for the controlnet support(tile in particular) My best workflows use XL for bases and 1.5 for upscales/enhancing because of TILE controlnet. Somone needs to make this for xl... or it cant compete.

[@Pizzaguy2]

I give up with this tutorial, AV_ControlNetPreprocessor missing , how to install, I try ComfyUI's ControlNet Auxiliary Preprocessors and didn't work, why would make video tutorial and skip steps... :/

[@MylesRobinette]

Restart the server and then refresh your browser like it says to do. Worked for me after installing that and refreshing both.

[@Pizzaguy2]

@@MylesRobinette i think I finally fixed it, comfyui-art-venture was missing.

[@sickvr7680]

Oops, invitation not valid when try to join Discord!

[@acezgaming2793]

cannot click ckpt_name nothing is happening 1:07

[@OlivioSarikas]

That means that you don't have any checkpoints in your model folder. you need to download them to your drive to be able to load them.

[@alexanderhermann8299]

Are there any instructions where I can get them and is there a library where you can search for specific features for the models?@@OlivioSarikas

[@alexanderhermann8299]

Are there any instructions where I can get them and is there a library where you can search for specific features for the models? I am new to the subject and find that this point is not clear. It would be nice to get another tutorial on this. Thank you@@OlivioSarikas

[@bonechieftain]

i'm having the same issue @shtogie5972 is having loading the "Controlnet Preprocessor"

[@shtogie5972]

Did you get this solved? I'd be happy to help.

[@maxpavlov1]

I also wonder if you found a solution@@shtogie5972

[@bonechieftain]

@@shtogie5972 i did not get this resolved, would appreciate any help.

[@coryreeve1]

@@shtogie5972 Not sure about him, but I'm having the same issue. I even tried using Manager to "Install Missing Custom Nodes" and it fails that as well. (Still red on restart.) Did you find a viable workaround?

[@groucho828]

@@coryreeve1 I had the same issue, went to install custom nodes through manager and installed Fannovel16's "ComfyUI's ControlNet Auxiliary Preprocessors" you'll then get the controlnet preprocessor node option with MiDas Depth Map