The Math Behind Generative Adversarial Networks Clearly Explained!

[@marinafuster7005]

I almost never leave comments... this is an amazing summary of the mathematical aspect and quirks of the paper. Thank you!

[@sharvani_0779]

This video is a perfect and most explanatory video on this topic, absolutely love it.

[@KaniskaM-yg9wq]

This is the Best resource fo gan i've come across so far. Very detailed explanation with complicating the terms. You are a life saver! . thank you

[@vijayshankar102]

This is a well made and well explained video, one can be extremely grateful to you for this

[@NormalizedNerd]

Thanks a lot!

[@jourytasnim7107]

you deserve Best explanation award

[@mitalihalder1]

Amazing.. Just no words. Standing ovation to you.

[@NormalizedNerd]

means a lot ❤

[@bhattbhavesh91]

Hey Sujan, I just started learning GANs and I happen to stumble upon your video & your channel ! Great work :) You have a bright future ahead :)

[@wolfywolfgang2498]

Woah !!!! I consider myself bad at math, but this video was like a hot knife in my dense but butterish brain! thank you !

[@parisdettorre3008]

Congratulation....one of best marh explanations of GAN ever🎉🎉

[@ruju-tt5sy]

Truly an exceptional and informative video! Literally made me understand the concept very well!!

[@danielwamriew9614]

Thanks Sujan for the excellent tutorial on the math behind GANs. I just started learning about GANs today, and cleared my eyes on the subject.Kudos!

[@JaishreeramCoder]

This is exactly what I wanted. Your explanation was amazing and very clear.

[@Lena-of7wd]

This is the best explanation I’ve found on GANs, thank you!! I’m currently training a DCGAN, however in terms of theory, are there any differences between DCGAN vs GAN as from my understanding, the difference is DCGAN utilizes deep convolution networks and GAN utilizes fully connected layers, however is the theory described in this video also applied to DCGAN? Thanks for your help, appreciate it!

[@NormalizedNerd]

Yeah, same theory. As you correctly mentioned, just replace fully connected layers with conv and pooling layers.

[@staffankonstholm3506]

You are my favorite person on the internet right now

[@NormalizedNerd]

😁😁

[@nch77884]

Great explanation for something as complex as GAN.

[@gamerbattle2554]

The first well explained GAN I've ever seen, thanks

[@NormalizedNerd]

Thanks! :D

[@rxzin7201]

Thanks, got saved for Deep Learning exam

[@muhammadwaseem_]

Why is your channel so underrated ?!!!

[@RambutanLaw]

I like how he pronounce "z" as "zed" but "G of Z" as "G of zee".

[@yashbhambhu6633]

I present you the indian english : )

[@jourytasnim7107]

focus on the message he is explaining not the pronunciation

[@porku5059]

This was a fantastic video, Im might actually pass my class now!

[@roro-v3z]

really good explanation!! Understood clearly

[@diby4283]

I'm really thankful. Great explanation!

[@rivetinggull2596]

Thank you! Very well explained. Mentioned all the underlying mathematical concepts on which GAN is based on.

[@NormalizedNerd]

Most welcome!

[@OmerBoehm]

Brilliant presentation - simplifies the math using intuitive explanations and examples (same for the video about the binary cross entropy ) - thank you for this

[@NormalizedNerd]

❤️❤️

[@stevea8201]

Really nice, I really like the part at 11:52, made that part so much easier to understand with that visual example

[@NormalizedNerd]

Thanks mate :D

[@lcslima45]

How do you get the equation of binary cross entropy from the cross entropy definition?

[@cs2466]

Thanks for the great effort in making the videos. God bless you

[@petsart3792]

Thanks for explaining this advanced topic!

[@NormalizedNerd]

Glad you liked it. Keep supporting the channel :D

[@utsavbandyopadhyaymaulik9006]

Clear mathematical explanation

[@amirhosseinboreiri3360]

Wow.this was very educational. Please make more videos on gans.

[@yashmore3525]

This is a great explanation! I'd love to see more in depth videos! If you could cover autoencoders that'd be really cool too!

[@NormalizedNerd]

Thanks! I have one for autoencoders: kzbin.info/www/bejne/o2Okqp-gea2Mm9U

[@alexandrakogan2840]

Thank you so much for this video! *I think the JS divergence equation needs a second ln sign after ...+1/2 Ex pg ? The equation appears at 12:53. Thank you again!

[@NormalizedNerd]

Oh...You are right. I forgot the ln sign. Thanks for pointing this out :D

[@alexandrakogan2840]

@@NormalizedNerd Great, just wanted to make sure I understood this correctly :)) Thank you!!

[@greenufo_0108]

Dude. Awesome!! Literally you explain better than medium "how to"'s :) expecting awesome content

[@greenufo_0108]

Nice explanation next lets create a neuro network from scratch

[@NormalizedNerd]

@Green UFO_010 thank you man! Yeah more interesting videos are on the way. Keep supporting :D

[@NormalizedNerd]

Neural Network from scratch is definitely in my bucket list!

[@demoredemore933]

Wow wow Thank you! Well explained.

[@vivekkandeyang6175]

Thanks for such a great explanation

[@arshamafsardeir2692]

Good explanation. Thank you!

[@prakashdey7356]

Best video on Gan

[@teetanrobotics5363]

Amaizng video bro..Could you please make a playlist for Deep learning(include this video) and/or reinforcement learning.

[@NormalizedNerd]

Ok I'll try to create a playlist.

[@victitova3811]

Thank you very much for the video! Can someone help me and explain why 1 - was dropped at 12:30

[@SJ23982398]

What would really help is if you add links to your videos if some concepts in a video have been discussed more in depth. So if I don't understand some concept that is glossed over here I can scroll down and click the video that explains it in more depth.

[@NormalizedNerd]

Feedback noted!

[@tanmoym6241]

Nice tutorial. Which software you are using for writing on the board here ?

[@nitinkumarmittal4369]

Thank you for posting this!

[@NormalizedNerd]

My pleasure!

[@danielnenov3882]

Do you provide personal lessons? @normalized nerd

[@NormalizedNerd]

Sorry, but I don't.

[@Rajkumar-sm6bi]

Great, dont stop! Keep making such nice videos.

[@NormalizedNerd]

More to come!

[@MovieTheater69]

Great work thank you very very much❤

[@johntzimiskes1480]

Very nice explanation

[@anonymousvector729]

Amazing video. I'm still confuse that what is difference between normal GAN vs GAN CLS? Can you explain a little bit

[@NormalizedNerd]

In GAN CLS, the input is a sentence vector + some noise. In normal GAN it's just noise.

[@anonymousvector729]

@@NormalizedNerd would love to see your video on GAN vs GAN CLS if you make one.

[@archiexzzz]

was very easy to understand. thank you

[@shashanksharma21]

Incredibly well made !

[@10xGarden]

wow, gan er gan beregelo amar thanks for that.

[@NormalizedNerd]

Hee Hee ❤️❤️

[@FRANKONATOR123]

Such a good explanation, man! Thank you so much!!

[@Marco-m7o7b]

What about the sign he told us to forget? should it not be considered at the end? why?

[@stanislavzamecnik3049]

Extremely good explanation!!!!

[@NormalizedNerd]

Thanks! :D

[@casualcomputer6544]

Best explanation ever!

[@felixz7273]

Thanks! Wonderful explanation. But it seems that 9:58 needs to adding a sum sign in front of the square brackets.

[@NormalizedNerd]

The summation is taken care by the inner loop

[@goldfishjy95]

This is so.. GOOD! Thank you so much!!!!

[@NormalizedNerd]

❤❤

[@sefika9825]

Very informative. Thanks for this clear explanation!

[@NormalizedNerd]

You are welcome!

[@TejasPatil-fz6bo]

Would like to see Use of Regularization functions/terms in loss function through equations....Plz make VDO on this

[@NormalizedNerd]

Btw I talked about l1, l2 regularization a bit here: kzbin.info/www/bejne/fJq2qmmwjKmZn6M

[@onlyumangsri]

Very well explained. Can you once try gan inversion as well?

[@HappinessYata]

I didn't manage to understand starting from Binary Crossentropy Function :(

[@mithilgaonkar7676]

Well, thanks for the transfer learning🤭... You have explained it in a very crisp manner.. Keep up the good work 👍

[@NormalizedNerd]

My pleasure 😊

[@ahmadatta66]

thank you. Great explanation

[@NormalizedNerd]

You are welcome!

[@theupsider]

Amazing video man!

[@amisha4891]

Outstanding content

[@harshraj22_]

Hey ! What does it mean, when people say, data points/ images/ texts (on which we train our model) belong to a distribution ? What is its inuitive meaning about belonging to a distribution and how are they sure about the real life data belonging to a distribution ?

[@NormalizedNerd]

I really liked your question. So here's the thing... I hope you are comfortable with the distributions in 1 or 2 dimensions e.g. distribution of height and weight of a population. Now imagine we are talking about images. Can we represent an image with 1 or 2 dimensions? No. For a 256px*256px RGB image we need 256*256*3 dimensions. Suppose you have 1000 such images of flowers. Now you can plot the pixel values in each dimension right? If you do this for 1000 images you will get the pixel distribution or simply the distribution of your dataset. Then the goal of your ML model will be to capture this distribution. I talked about pixels but the idea can be used in words (text data) also. And something belonging to a distribution means it follows (looks similar) the training dataset. Obviously in Statistics we can mathematically say if something belongs to a distribution or not. But intuitively it means "looks similar".

[@arisceznyk]

Awesome explaination

[@NormalizedNerd]

Glad you think so!

[@theankitkurmi]

Nicely explained. Do make a playlist of regression classification nlp deep learning so that we can easily follow up. Great job 👌👌👌

[@NormalizedNerd]

Thanks! Actually, I have playlists for NLP, ML from scratch. Will try to make one for Deep Learning.

[@adityarajora7219]

9:29 in gradient update part, why would generator try to make good images.....as D(g(z)) will be close to zero imples nothing to learn for generator...I dint get ascent and descent at all from any video

[@olympics3948]

Great job!!

[@Menor55672]

what whiteboard software is that ?

[@NormalizedNerd]

Microsoft OneNote

[@subratswain6775]

What platform you're using for the videos?

[@NormalizedNerd]

For this video I used Microsoft OneNote

[@subratswain6775]

@@NormalizedNerd no for the animation

[@NormalizedNerd]

@@subratswain6775 For the animations I use manim (open source python library)

[@subratswain6775]

@@NormalizedNerd can you send the installation steps. I tried to create but couldn't

[@NormalizedNerd]

@@subratswain6775 I'll suggest you to follow youtube tutorials for installing manim. It's not very easy to set up.

[@KUMAR-ne2mb]

It was great explanation

[@mk-wh6mv]

Great Explanation!!

[@NormalizedNerd]

Glad you liked that :D

[@alexandermoralespanitz8772]

Excelent video!

[@sandipandhar4143]

Good initiative but very similar to Ahlad Kumar's explanation.

[@NormalizedNerd]

Thanks for your feedback. I actually didn't know about that.

[@user-cc8kb]

Cool video, thank you very much :)

[@mdnahiduzzaman2719]

Spectacular

[@TheAcujlGamer]

Loved that intro 👌

[@sourabhbhattacharya9133]

sera porali bhai

[@NormalizedNerd]

onek dhonnyobad bhai!

[@bSharpHacker]

Great video, thanks! The label of 0 for the reconstructed image. Is that correct? According to another reference I have, it should set the labels to 1 to fool the discriminator into thinking the image is real? Edit, my bad. Yes, you are correct, feeding y = 0 into the discriminator is correct. The label 1 is then used to train the generator :)

[@DungPham-ai]

great job ! thank so much

[@NormalizedNerd]

@Dung Pham You're welcome! Support this channel for more videos :D

[@garyzhai9540]

I believe the presenter is knowledgeable. However, some details are not well explained and not consistent, such as 11:46, he mentioned that this formula and there is no intuitive explanation, as this type of KZbin presentation is for the general public who has no in-depth of understanding of either maths or deep-learning.

[@rakshitverma5016]

great video!

[@homakashefiamiri3749]

it was wonderful.

[@gisellerodrigues571]

So I heard someone saying it's easier for the discriminator to predict a fake data than It is for the generator to create a fake data who could pass as original. That would make the system unbalaced. Is It true and If so, Is there a way to fix It?

[@NormalizedNerd]

@Giselle Rodrigues Yes, it is true especially at the initial stages of the training. As a matter of fact, GANs are very unstable. It generally requires a lot of trial and error to find the best architecture (and other hyper-parameters) for a given dataset. But luckily, researchers have found some ways to improve the training. Here, you can find some of them. machinelearningmastery.com/how-to-train-stable-generative-adversarial-networks/

[@gisellerodrigues571]

@@NormalizedNerd thank you for the reply!!! I wasn't expecting It to be so fast! Haha I am gonna read It and maybe come back with more questions. Hahaha

[@NormalizedNerd]

Haha. Sure. Keep supporting.

[@sulemanshehzad6205]

Amazing, keep up the good work (y)

[@NormalizedNerd]

Thanks man!

[@jonsnow9246]

Do you use tablet to make these notes?

[@NormalizedNerd]

Yeah

[@alaa.abuqtaish]

Thank you

[@RamanKumar-dh8iu]

🎯 Key Takeaways for quick navigation: 00:00 *🧠 Overview of Generative Adversarial Networks (GANs)* - GANs consist of two models: a generative model (G) and a discriminative model (D). - Generative models learn the joint probability distribution of input and output variables, while discriminative models learn the conditional probability of the target variable given the input variable. - GANs use an adversarial setup where the generator produces fake data points, and the discriminator distinguishes between real and fake data, leading to both models improving over time. 02:13 *📊 Structure and Components of GANs* - GANs consist of multi-layered neural networks representing the generator (G) and discriminator (D). - Theta G and theta D represent the weights of the respective networks. - GANs utilize a noise distribution as input to the generator to produce data points similar to the original distribution. 05:26 *🔢 Understanding the Value Function of GANs* - The value function represents the objective of G (minimize) and D (maximize) in the GAN setup. - The value function resembles the binary cross-entropy function, crucial for training GANs. - Expectation (E) is used to calculate the average value over the entire dataset, essential for continuous distributions. 08:37 *🔄 Training Process and Optimization of GANs* - GAN training involves an iterative process where the generator and discriminator alternate updates. - Stochastic gradient descent is used to optimize the loss function. - The discriminator is updated to maximize the value function, while the generator is updated to minimize it. 10:42 *🎯 Convergence and Guarantee of GANs* - The goal is to prove that the generator's distribution converges to the original data distribution. - Jensen-Shannon divergence is a method used to measure the difference between two distributions. - At the global minimum of the value function, the generator's distribution becomes indistinguishable from the original data distribution. 14:48 *⚙️ Phases of GAN Training* - GAN training progresses through phases where initially, both generator and discriminator perform poorly. - As training continues, the discriminator becomes adept at distinguishing real and fake data, while the generator's distribution approaches that of the original data. - At convergence, the discriminator cannot differentiate between real and generated data, achieving the desired outcome. Made with HARPA AI

[@jamesang7861]

Thank you!

[@NormalizedNerd]

Welcome!

[@ChandraPrakash-yj4vx]

Thanks Man

[@henrrymendoza]

It's not clear why you replace p_z(z) by p_g(x) when showing global optimality. x and z are on a different space.

[@buihung3704]

correct, i still don't get this part, how can he do that? it's true that they can have the same range of values (both are images with same width x height dimension) but that doesn't mean they can swapped each other's places?

[@jrt6722]

Sorry please teach me, I don’t understand how the function V(G,D) corresponds to the loss of generator and discriminator…

[@adityakushal8905]

V(G, D) is total loss of the model(fake image loss + real image loss), u then do partial derivation with respect to generator and discriminator

[@MmmD-jv4ec]

Awesome explanation thank you very much. Subscripted

[@quocanhnguyen7275]

SOOO GOOOD

[@pouryapouryeganeh4280]

thanks bro

[@NormalizedNerd]

You're welcome :)

[@madhuvarun2790]

at 0.22, you pronounced content wrong. sound of "con" in content should be like "con" in con man

[@vinuvs4996]

mathematically convincing

[@chadgregory9037]

AWW, I'm disappointed..... 22,322 views...... so close to 22,222 =]

[@jan.kowalski]

"geee izz da virgin ass" lol