Stable Diffusion: High-Resolution Image Synthesis with Latent Diffusion Models | ML Coding Series

[@deepblender]

Very cool video, thanks a lot for that! Whenever I train neural networks, it is still surprising to me that this whole process actually works and is even quite flexible. I just got started with diffusion models and this video is invaluable to solidify my understanding. To me, one of the critical milestones in learning something new is to confidently know when something is not important for my current toy project and which snippets are worth trying out.

[@myksmith]

This is an amazing video - you start with the paper, not pulling punches on the tech, then dive into practical applications. Wonderful.

[@TheAIEpiphany]

Thanks Michael!

[@Kutlutr]

Wonderful video, to the point, starting with the paper, understanding the necessary background and moving forward with practical exercise, very nice, keep up the good work bro.

[@TheAIEpiphany]

Check out my high level how to get started with diffusion video here: kzbin.info/www/bejne/m6HOpX6qgbyafrM If you want to understand how Stable Diffusion works behind the scenes in this video I walk you through the codebase step by step explaining: 1. First stage autoencoder training (autoencoder with KL regularization) 2. Latent Diffusion Model training (UNet + conditioning model) 3. Sampling using PLMS scheduler and how a link to differential equations enables us to sample much faster

[@arcosin3861]

If I was a king or other monarch of authority, I would Knight you. Great tutorial. Peace brother.

[@junpyohong2132]

Amazing work. It helps me a lot to understand code! Trying to figure PLM out. Not an EZ thing...

[@lukadjokovitch7040]

Your Chanel is gold

[@TheAIEpiphany]

Thanks man

[@Ayayron1998]

More videos on stable/latent diffusion please

[@manikantabandla3923]

What is the difference between stable diffusion and latent diffusion repo?

[@marcusleong7604]

This video is pure gold!

[@user-uc1ky6ce5n]

This video really help me to fix my latent diffusion model ,thanks!

[@Vikram-wx4hg]

Super thanks for this video, Aleksa. And just to let you know I sat through the whole of it - in two sittings (with a lunch break in between). 🙂 And it was so well done. 👏

[@TheAIEpiphany]

Haha niceee!! Thanks, glad to hear that!

[@nekoeko500]

A few minutes into the vid and I see this guy using Irfanview. I need to subscribe

[@TheAIEpiphany]

Lol 🤣

[@abhishekdhiman5719]

i went through LDM paper, i didnt understand what is the ground truth author is using. The output should be conditioned on the input text and for comparing this we need a ground truth. How to obtain that ground truth for training the model

[@masoudkarimi8211]

Great job, way to go :)

[@chyldstudios]

Amazing, I was just looking for a video on this specific topic. Thanks for sharing!

[@TheAIEpiphany]

Nice, you're welcome!

[@sairampenjarla]

bro's common knowledge is my entire knowledge.

[@giovanith]

Hello Aleksa, what a great & legit lecture about this issue. Thank you

[@abhijeetnarharshettiwar6175]

Thank you Aleksa.

[@TheAIEpiphany]

You are welcome!

[@sacramentofwilderness6656]

Thanks, Aleksa for the video, very insightful and thorough series. By the way, you have encouraged me to use debugger instead of print statements :)

[@mananshah2140]

This is the best part! Use the debugger!

[@tresuvesdobles]

Many thanks for the video! It is helping quite a bit to understand their codebase.

[@tresuvesdobles]

Help! At around 53:40 I am getting an Exception: FileNotFoundError: [Errno 2] No such file or directory: 'configs/first_stage_models/vq-f8/model.yaml'

[@maramrashad3403]

Hello, I'm getting the same error. Could you please share how you fix that, if you did. Many thanks.

[@trideeprath1991]

Great video. But kinda got confusing on the conditioning part especially with the clip based conditioning, why there were two networks, one with empty text and another with text and then add them up. Would be good if you can point out to understand that part better ?

[@objectobjectobject4707]

Ovo je kidanje !

[@vincentaxm8322]

thank you so much!!! your videos really help me a lot.

[@user-lb1bs8iv3f]

would you please explain where to put the downloaded imagenette and how to config it to work.

[@leshanwang4090]

may I ask how many 3090GPUs do we need to train this latent diffusion model for super resolution task，cos the stable diffusion V1 is very expensive and needs hundreds and thousands of A100 GPU, it's terrifying

[@akhandatripathi9897]

instead of text prompt as condition, how can we use image as condition in cross-attention to develop image to image translation model?

[@dimitrismit6714]

Propably a stupid question, but why do we sample from the distribution since we have the decoder of vq gan?

[@tylersnard]

This is amazing, thank you! Would you mind doing a post about Enformer?

[@TheAIEpiphany]

I unfortunately can't cover DeepMind research due to NDA restrictions.

[@user-lb1bs8iv3f]

Thank you so much. I finally got the code to work. But I am confused with the classifier-free guidance. In the original classifier-free paper, they said that during training they randomly remove the condition. However, I haven't see this operation in any codes in DDPM series. Got any ideas？Many Thanks。

[@kallamamran]

I wish I could follow this 😳👍

[@user-lb1bs8iv3f]

Amazing video

[@TheAIEpiphany]

Thanks!

[@commandobrandolive]

I get lost at the beginning of the code part cause I cloned the project but there is no .vscode file and I have show hidden files on. Not sure how to get or make the appropriate launch.json file

[@mananshah2140]

Just add this to configuration: { "name": "training", "type": "python", "request": "launch", "program": "${file}", "console": "integratedTerminal", "justMyCode": true, "env": { "PYDEVD_WARN_SLOW_RESOLVE_TIMEOUT": "5" }, "args": [ "--base", "configs/autoencoder/autoencoder_kl_64x64x3.yaml", "-t", "--gpus", "0," ] }

[@Jake-om9no]

@@mananshah2140 Hi, what do you mean exactly by saying 'add this to configuration'? May I know which specific file should I input those lines? Really appreciate it if you can help me with this.

[@dcdcdc5469]

Hello, a very interesting video. Do you have any content on how to use this type of models to upscale low-resolution images to high resolution? Or do you know any website with examples of this? Thank you very much for your content.

[@SanatBatra]

Hey, did you get any solution to this problem ? Let me know ASAP !

[@catfood7859]

Huge thanks for the crystal clear explanation! May I ask what's the difference between the latent diffusion vs stable diffusion? (I roughly browsed two github repos and found they are literally the same)

[@elkwang4357]

I think stable diffusion is the updated version of ldm. They are almost the same.

[@awa8766]

From my research, stable diffusion is just a name for the architecture released by Stability AI and the lab they worked with. The entire network falls within the latent diffusion domain. Therefore, stable diffusion is a part of latent diffusion.

[@convolutionalnn2582]

Berkeley Reinforcement Learning or David Silver course or Coursera Reinforcement Learning....Which one is best to get start in Reinforcement Learning?

[@TheAIEpiphany]

Yes ♥️

[@andreyzakharov3651]

I'd recommend Hugging face's course on Reinforcement learning

[@mobi02]

Hi, Would you explain this paper in your video? "First Order Motion Model for Image Animation"

[@raroca23]

Hi, thanks for your video. I'm learning a lot but I still don't have enought programming skills in DL and Pytorch. I have a doubt/question that could be very useful for me and I think for other people if can be solved. Do you think there is a way of adding local labeled images (for example your own photos) to the model without training all the dataset?. I see a lot of commertial applications in this area if can be done with a local GPU.

[@cl1mbat1ze]

It's not exactly what you want but you may find Textual Inversion interesting. You can create an embedding that biases the model towards including the concept or object represented in the embedding when it generates pictures. The embedding is created by training the model on 3-5 images of your choosing (representing the concept/object you want to bias the model towards) over 5000+ timesteps AND it can be run locally, albeit with a lot of Vram and slowly. It can definitively be run from a collab, though, if you splurge for the pro version.

[@quecksilber457]

How do i change the resolution of my images? Is there an easy way?

[@wenfangsun]

Hello, I have the same question

[@danwood4171]

Assuming sd-v1-4.ckpt is the ?model?/?weights? for a decent stable diffusion program is the weights/model for stability.ai downloadable given that you said we can do local generation. I assume the quality is better with stability.ai versus something like sd-v1-4.ckpt???

[@Vikram-wx4hg]

Same quality.

[@suntanudipto2228]

Little did I know that my dumapp self creating drum softs by making a bunch of different soft rolls was far from the best way of doing

[@joseantonioapariciogallego5620]

I don't understand it, so where is the source code, can you see it when you become a patreon?

[@mananshah2140]

blah blah blah, we don't care about model checkpointing. There are some conv layers, configs 🤣. This guy simply shrugs off complex code with insouciance. Just so much fun to watch and follow along. "Nothing Fundamental!"🤣

[@dingran]

1:30:53 the `len(old_eps)==2` branch is the Adams-Bashforth three-step method see en.wikipedia.org/wiki/Linear_multistep_method (search for "The Adams-Bashforth methods with ")

[@navissivan]

The loss part is still very confusing to me, why add the same loss twice with different weights, and as you said in the end going to zero? Anyone knows?