Style LoRA Training guide for Stable diffusion 1.5 and SDXL Concepts Results and Conclusion

[@sarahthecolorist2429]

Thank you! A big help getting started with styles and enough info to play with the numbers and test things out.

[@AI-HowTo]

Very welcome!

[@swannschilling474]

Thanks for sharing!

[@4848kitty]

Hello! This was an extremely useful video! I'm currently a graduate student in ML and I'm working on training a model for style then using that in img2img. Do you think it's possible to use LoRA with img2img?

[@AI-HowTo]

great to know... yes ofcourse, especially for Video generation, the use of a LoRA can increase the consistency of the video generated... and depending on the denoising level, the output would change to become more affected by the LoRA or less.

[@lefourbe5596]

I'm aware about lora. Mmmmh... I've yet to see a tuto of someone train a full SDXL checkpoint.

[@workplaydie]

Thanks for this video! Must all the training images share the same resolution and aspect ratio? I didn't really catch what happened when you chose to only square images. I think you said cropping is unnecessary. I'm attempting to train on a set of med to high res images that have different aspect ratios. I think they are all at least 1024x1024, many are closer to 4k. I don't mind cropping a dozen or so images for training, but I don't understand how/why this is important.

[@AI-HowTo]

You are welcome, not it is not necessary, if we select the option (Enable buckets) it becomes possible to train different image resolutions.

[@RyokoChanGamer]

hello, I'm happy to see another video, I was really wanting to learn about creating loras with styles... but I'm still breaking my head with the characters... I'm trying to create a character but between 1200 and 1500 steps, it starts to overfitting , but still haven't "learned" all the characteristics of the character, what can I do to learn more details without starting to overfit? thank you

[@AI-HowTo]

check this video kzbin.info/www/bejne/rHKVp2V_qsx-bNk it is really good, hight quality images/ hight qualiity regularization images are really important, in many cases 1500 steps can lead to a good looking character...overfitting early means most likely you are using hight network rank such as 128 .... so use 64 or 32 instead ... use regularization... lower your alpha value to half or quarter of network rank or even lower that will make learning smoother... and just experiment and best of luck... the process is experimental, and results may differ from one data set to another.

[@RyokoChanGamer]

@@AI-HowTo Thanks again I will continue doing more tests, I will use your tips

[@LALA-dw2ez]

Danke!

[@AI-HowTo]

Thank you, that was very kind of you, much apprecciated.

[@paveloconnor7759]

Thank you!

[@AI-HowTo]

You're welcome.

[@kallamamran]

So now we can train styles, object/subject... Now the question remains how we trains a design language. Ty making a refridgerator in the design of Volvo or Guchi for example!?

[@AI-HowTo]

This is not something we train as indivisuals using a LoRA for instance, because it is basically part of the Stable diffusion Architecture, or part of the image generation architecture in general and done through prompting (create A in style of B), so we only train a style for instance using many many images, then prompt for it and SD will diffuse it randomly.

[@K-A_Z_A-K_S_URALA]

Hello friend, is there going to be anything new on Lora training program for people? I haven't heard anything in a long time, maybe there are some new software chips

[@AI-HowTo]

I have not been keeping up lately with what's happening in this area, but I think before we get a new training method (That is faster, more accurate, and resource efficient and that is better than LoRA and Dreambooth), there wont be anything new for a while

[@yhuna3365]

Hey! I see you're using runpod, could you tell us how much time it took to train the lora? to know more or less the amount of money needed. Also I want to train a new style wich is used in a videogame of a friend, and I have like more than 1000 images (characters, hand with phone, stuff like that but not backgrounds), how many epochs, repeats, and steps. And actually, if you could explain me better what the differences are between these 3. Nice video by the way and Thanks!

[@AI-HowTo]

You are welcome. for SDXL yes, I used runpod, usually SDXL training takes less number of steps than SD 1.5. each step took around 1 second, I tried a model for 4000 steps, usually 3500 steps is enough, more or less.... this can differ depending on data/regularization... in some models it can takes more than 10000 steps with large number of images.... without regularization, steps will be alot less, sometimes you might converge to suitable results in 1500 steps only. if you have all your data prepared, and links prepared, training for a LoRA can be completed quickly on RunPod in 1 hour or two hours with RTX 3090 for SDXL that will be around $1 and 2$ depending on how fast you setup your installation and download the models and how responsive the pod is in terms of upload and download speeds.

[@Ziko675]

I am confused about one thing. During captioning do we need to caption all the keywords that we want to train in a model or do we have to remove those tags if we want them to train in a model. Which one of them is correct?

[@AI-HowTo]

yes, anything that we want to be part of our LoRA must not be captioned... for example, if our subject always have green eyes, then we better not include that in the captions... if she has blonde hair, then we better not include that in the captions either ... we caption only the things that changes, for example: her clothes, the background...etc.... it is a bit confusing... but this is the best way to caption things and improve accuracy for the LoRA model.

[@Ziko675]

@@AI-HowTo hmm that’s interesting. So you saying that anything which is not consistent among images should be not be captioned/tagged. For example if I am training some background images as a style whose theme is neon light cyberpunk, so I should not caption cyberpunk or neon light as it will be consistent among the images but I could caption a group of people for example or tall building because they will not be always there

[@AI-HowTo]

exactly...we can also add a trigger word that refers to your style (which absorbs all things related to your style common features in all the images) for example which may help increase the chance of bring that style out when prompting too.

[@Ziko675]

@@AI-HowTo I think I understand some of it even if it sounds somewhat confusing. Thanks mate😀

[@AI-HowTo]

you are welcome.

[@Th3Shooter]

Hey could you please let me know where to get the regularization images?

[@AI-HowTo]

you can generate them from SD for example... I used these which you can download (800 style class images size 1024x1024 huggingface.co/datasets/AIHowto/460RegImages1024x1024/resolve/main/jpg%20style%20class%20images%201024x1024.7z ) another data set containing 1200 class images (768x768 is huggingface.co/datasets/AIHowto/460RegImages1024x1024/resolve/main/jpg%20768X768%20STYLE%20class%20images.7z)

[@riggitywrckd4325]

Thanks for these. Do you have any hints for what to use as a prompt when creating your own? I'll be downloading these and then using png info but you know maybe it won't work. Thanks for all your help.

[@AI-HowTo]

You are welcome, i uploaded jpg class images because the PNG are huge in size, jpg is smaller, so png info wont show up. to generate style image we can use something like: artwork style with Negative prompt: text or illustration style or drawing style etc.

[@riggitywrckd4325]

@@AI-HowTo thank you very much for all the effort and help.

[@AI-HowTo]

you are most welcome @@riggitywrckd4325

[@ricardoc9436]

Sorry but I don’t follow you. I think is a good video, but I don’t understand you very well…