Lora Training using only ComfyUI!!

We show you how to train Loras exclusively in ComfyUI
Github
github.com/LarryJane491/Lora-...
Join and Support me ###
Support me on Patreon:
/ aifuzz
Let’s be Instagram friends:
/ aifuzz1
Discord

[@picobyte2732]

Thanks. Not yet tested but I think N-nodes GPT-loader simple and GPT-sampler could also be included in the captioning process.

[@user-ef4df8xp8p]

Thank you so much for making the video on this interesting topic. Will try out the technique....

[@AIFuzz59]

No problem! Have fun with it!

[@thrillhouse500]

PRO TIP: make sure your images are in PNG format. If not, you may get an error during the tagging phase.

[@asyrofmuzakki]

THANK YOU BUDDY! YOU TOTALLY SAVED MY LIFE :)

[@fredrik241]

Awesome thank you for this tutorial! I'm going to try it! :)

[@AIFuzz59]

Sure thing! Let us know how it goes

[@osmanic5593]

Awesome! Thanks for the vid

[@AIFuzz59]

No problem!

[@LunarFerns]

Thank you so much for the tutorial! One question, how to set the trigger word for lora? Or you don't need it for these loras?

[@AIFuzz59]

I have found that the Loras do work without trigger words however you can use the name Of the Lora as a trigger word and it will Increase the impact it has on the image

[@GamingDaveUK]

I have wd14 and training nodes installed, my comfyui uses cu 121 however I do not see lora caption load or lora caption save?

[@Xavi-Tenis]

i instaled in the custom nodes, but are missing the load and save node of lora training, any advice? thanks

[@iresolvers]

do you have to link to the pytourch cu121 ???

[@bubb31gum]

How do I get the 1.21 version of pytorch for a separate portable install?

[@WhySoBroke]

Very cool!! Is it possible to train SDXL Loras?

[@AIFuzz59]

Yes!

[@AIFuzz59]

Just make sure the base model You train on is an sdxl model

[@urieldeveaud]

you get a new follower :) nice video! ... do you know how to merge loras in comfyui ? thx

[@AIFuzz59]

Yes, there are a few Lora merge custom nodes out there. You can even use a Lora stacker which works well. I’ll find some and do a video on the options

[@urieldeveaud]

@@AIFuzz59 nice!

[@gequ3104]

thank you for this tutorial. Are there any downsides in comparison to Kohya training?

[@AIFuzz59]

So far the only downside is that with some models, it doesn’t click with the Lora

[@jsnavely76]

One downside is it doesn't appear to have very many options that you can configure like what max size to scale the images down to or what network rank & alpha you want. You're stuck with whatever defaults it picked for you, but I haven't tried it so perhaps you might be able to adjust those settings in a configuration file somewhere. If you're already going to the trouble of making an alternate install of ComfyUI just for LoRA training, I'm not sure why you wouldn't just use Kohya or another dedicated training app instead and have full control over all of the settings. If it's a matter of wanting a simplied app, then I don't know why you are using ComfyUI at all because it is probably the highest learning curve of any AI image generator due to how you have to have a lot of knowledge to be able to wire everything up from scratch or be limited to only using pre-made workflows, whereas other apps allow you to do many complex things with the click of a few buttons.

[@AIFuzz59]

@@jsnavely76 it’s time. I have trained Loras on Khoya and it takes several hours. This is a quicker way to get Lota training done. It can be done in less than 30 minutes. That also allows you to experiment with training and trying different things out.

[@AIFuzz59]

@@jsnavely76 I know many people who have several installations of ComfyUI. It’s not uncommon. It is very helpful as well since there Are times custom nodes do clash with one another

[@HiProfileAI]

​@@AIFuzz59yup I had my comfy completely crash after loading someone's workflow. Couldn't get out of it and had to reinstall. Thank goodness I had a backup alternate comfy install.

[@Mr0rginalet]

I seem to have a issue when I start the training, the prompt is executed in less then a second and nothing is generated: "Error while finding module specification for 'accelerate.commands.launch' (ModuleNotFoundError: No module named 'accelerate')"

[@AIFuzz59]

You may need to install accelerate

[@luislozano2896]

So what version of torch was the bad one ??? and how would you roll back to the right version?

[@AIFuzz59]

It would be 1.18

[@Halfgawd_Halfdevil]

shouldn't it install the updated python code when comfy UI updates? and how would you know if the version installed has the correct version? @@AIFuzz59

[@teealso]

I've been trying for two days to get this running and to no avail. I've narrowed it down to the xformer not finding the GPU. I've done all the uninstalls of torch, making sure I'm using 1.2.1 but I can't seen to it to engage the GPU, it only wants to use the CPU which causes it to fail. I'm open to any suggestions. I've gone all through the developer's site as well regarding making sure my I have CUDA in my python path, still no dice.

[@AIFuzz59]

Have you tried to uninstall xformers and install it fresh? Also are you using a new fresh ComfyUI install? You also should not update ComfyUi

[@teealso]

Hey thanks for the reply. What finally got it working was adding the CUDA path to my env variables. ChatGTP was able to help me out. I'm getting a lot of little click away errors while it's starting up, but once it's going on the epochs it's finishing. Thanks for help!

[@ImAlecPonce]

I've installed, reinstalled and run updates.... only 3 nodes apear for me

[@blacktilebluewall]

ayo whats that background noise 🌚

[@helix8847]

I am sure he is using AI Voice.

[@falconbmstutorials6496]

It's called an "hallucination" when you use a trained voice

[@teealso]

AssertionError: Torch not compiled with CUDA enabled I keep getting this error on the clean comfyUI install everytime. I've followed the instructions on what to do about that on the main ComfyUI Github page about uninstalling the Torch an re-installing it, still get the error. Has anyone else had this issue? Also, after downloading and installing the LJRE nodes, I don't see the loader or saver nodes, just the Training and Training Advanced options.

[@amorgan5844]

Did you figure it out?

[@teealso]

@@amorgan5844 hi yes! Thanks. Took a while and I’m not even sure what I did but it works now.

[@leonardkim690]

Thanks for your tutorial. but I've got an error. 'C:\StabilityMatrix-win-x64\Data\Assets\Python310\python.exe: Error while finding module specification for 'accelerate.commands.launch' (ModuleNotFoundError: No module named 'accelerate')' , could you let me know how to fix it?

[@AIFuzz59]

Can you try to reinstall accelerate?

[@NgocNguyen_NDN]

Can you guide me on how to accelerate?@@AIFuzz59

[@iccuesang]

@@AIFuzz59 How to reinstall accelerate?

[@AInomaly600]

audio is sooo weird

[@AIFuzz59]

Thanks!

[@captainadventures]

Rude

[@Wurmhouse]

I don’t understand the ‘comfy version’ thing at the beginning… someone can help me?

[@thedobrobro]

SDXL models training lora is not working with that node((

[@Keklmao]

Does comfy ui also work with fine tuning LLMs?

[@AIFuzz59]

Not yet but hopefully soon

[@PlayerGamesOtaku]

hi, unfortunately in the final part when you show all the windows are connected, I get stuck because I don't know how to connect them. Then on Lora name I get undefined. towards the end your tutorial becomes a mess

[@AIFuzz59]

Ok I apologize, I’m going to upload an updated version soon

[@PlayerGamesOtaku]

@@AIFuzz59 hi, let me know when the new updated tutorial comes out :)

[@umarcga7823]

I am issues installing the requirement files.... giving me error

[@oreiasccp]

me too, u solve this ?

[@cyril1111]

so 38min for 19 images - Slower than Khoya - but seems to get a good quality, maybe worth a try.

[@AIFuzz59]

For quick lora training it works well. I am sure if the data set were larger, outcome would improve but so will time. I do have a slower machine so also keep that in mind

[@amorgan5844]

​@AIFuzz59 shit ive trained dfl models for 3 weeks, this is so much easier.😂

[@dr.gurn420]

@@AIFuzz59 what machine are you running? I'm trying to workout roughly how quick this will be on a NVIDIA L4 32 GB

[@NgocNguyen_NDN]

I'm getting this error, can you help me?, (cannot access local variable 'image1' where it is not associated with a value.)

[@AIFuzz59]

All the images need to be in png format. You will get that error if they are not

[@NgocNguyen_NDN]

Thank you very much, I have done it.@@AIFuzz59

[@NgocNguyen_NDN]

c:\users\admin\appdata\local\programs\python\python311\python.exe:error finding module parameters for speed.commands.launch (modulenotfounderror:no module named `accelerate`) ship is completed I'm having this error, can you help me? Thank you very much. @@AIFuzz59

[@Muhilica]

I must ask...how much VRAM for 1.5 vs SDXL?

[@AIFuzz59]

I have 8gb VRAM and am able to train both Sd 1.5 and SDXL so I would say at least 8

[@Muhilica]

@@AIFuzz59 I see. Well, tomorrow I'll try training with my 2070QMax on an SDXL model and get back with the results. ThankS!

[@Muhilica]

I just throws an error as soon as I start training so...I'll get back to You on github :D

[@AIFuzz59]

@@Muhilica what error are you getting? The version of Pystorch is pretty important and throws a lot of errors

[@Muhilica]

@@AIFuzz59 Indeed, I noticed :) Tried a fresh Comfyui install under windows 11 only for the purpose of training and the UI worked, then, after installing the training node all hell broke loose. Eh, I'll try again in a couple of days and keep you posted in Git :)

[@asyrofmuzakki]

please help me, i get an error like this: C:\My Program Files\AI\ComfyUI_windows_portable\ComfyUI\custom_nodes\Lora-Training-in-Comfy/sd-scripts/train_network.py C:\My Program Files\AI\ComfyUI_windows_portable\python_embeded\python.exe: Error while finding module specification for 'accelerate.commands.launch' (ModuleNotFoundError: No module named 'accelerate') how to fix it?

[@seaviwtf9796]

It seems that you can't give us an explanation, like you I also have the same problem and through chatgpt (by asking and pasting the errors) I have managed to solve some errors but it's still the same, the training ends in less than 15 seconds because now It gives me errors with synchronization libraries like Aio.lib and Libaio

[@sirmeon1231]

Hm, as long as we have to make a second installation of comfy, i don't know why i should prefer this to kohya...

[@AIFuzz59]

You don’t have to but I did because my main ComfyUI has many custom nodes and slows up start time. Having a several ComfyUI installs is pretty common especially for testing different nodes

[@placeholder_Utoob]

How do I install the node?

[@AIFuzz59]

You would go to their GitHub page and install it to your custom nodes folder in ComfyUI

[@umarcga7823]

it's data base not betabase

[@tierlistboss]

All my captions were generated, and at the moment of execution, after having correctly set up the advanced LoRA training node in Comfy, I started the training and received the following error message: Error occurred when executing LoRA Caption Save: list index out of range File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\Image-Captioning-in-ComfyUI\LoRAcaption.py", line 53, in save_text_file filename = self.generate_filename(path, namelistsplit, file_extension) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\Image-Captioning-in-ComfyUI\LoRAcaption.py", line 65, in generate_filename filename = f"{namelistsplit[counter-1]}{extension}" ~~~~~~~~~~~~~^^^^^^^^^^^

[@AIFuzz59]

Did the captions save as txt files in the folder?

[@tierlistboss]

@@AIFuzz59 in fact everything is ok I even optimized and cleaned all the captions one by one, I installed all the dependencies which were necessary and indicated in the document "requirements_win.txt" and the training does not want to start

[@chintansheta]

I got this error, please help to fix it. Error occurred when executing LoRA Caption Load: cannot access local variable 'image1' where it is not associated with a value File "E:\Stable\ComfyUI_win\ComfyUI\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\Stable\ComfyUI_win\ComfyUI\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\Stable\ComfyUI_win\ComfyUI\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\Stable\ComfyUI_win\ComfyUI\custom_nodes\Image-Captioning-in-ComfyUI\LoRAcaption.py", line 148, in captionload return text, path, image1, len(images)

[@RahulGupta1981]

something similar here too

[@Bruno1890]

same here

[@SonKrals]

same here!!!

[@davidedifrancostudiotemp4139]

same

[@davidedifrancostudiotemp4139]

when using png (not jpg) it does work

[@emiliomartinez7132]

failed to download tagger. why'?

[@AIFuzz59]

Is this downloading it from the GitHub?

[@emiliomartinez7132]

@@AIFuzz59 i already had the node in comfy. it stops at 97 percent of load. i am trying to make a folder named "database" and put there the folder with the png files. it didn't work. the console writes timeout error from none. as you can see i am quite slow in this. very grateful for your answer.

[@emiliomartinez7132]

finally it works ! great nodes @@AIFuzz59

[@user-ge6gc9ti9c]

Not working, after done nothing its created.

[@none76ui]

Is this AI narrated?

[@falconbmstutorials6496]

DON'T install this dependency, it destroyed my ComfyUI installation! Now it says it cannot find the torch Cuda.

[@AIFuzz59]

It works fine. I suggested using a fresh install of ComfyUI if you are nervous and don’t know what you are doing.

[@kinenbib]

@@AIFuzz59 same here every time, after a fresh install ComfyUI If I just install the node the same error appears what could it be? win11?

[@AIFuzz59]

@@kinenbib what error is it?

[@caseyj789456]

Thx btw but not work for me. Tons for errors... Back to old method

[@alex.puliatti]

hey i get this error when tryhing to generate caption: Error occurred when executing LoRA Caption Load: cannot access local variable 'image1' where it is not associated with a value File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\Image-Captioning-in-ComfyUI-main\LoRAcaption.py", line 148, in captionload return text, path, image1, len(images) ^^^^^^ what can i do?

[@AIFuzz59]

All the images need to be in PNG format. You will get that error if they are in jpg or any other format

[@emiliomartinez7132]

I got that error because I hadn't set the location of the png file correctly.

[@alfaruqasri1357]

LORA CAPTION LOAD, i got this error, can you help me Error occurred when executing LoRA Caption Load: cannot access local variable 'image1' where it is not associated with a value File "D:\Stable Diffusion\ComfyUI_windows_portabless\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Stable Diffusion\ComfyUI_windows_portabless\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Stable Diffusion\ComfyUI_windows_portabless\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Stable Diffusion\ComfyUI_windows_portabless\ComfyUI\custom_nodes\Image-Captioning-in-ComfyUI\LoRAcaption.py", line 148, in captionload return text, path, image1, len(images) ^^^^^^

[@saidmoya12]

Works only with png files

[@emiliomartinez7132]

I got that error because I hadn't set the location of the png file correctly.