Layer Normalization in Transformers | Layer Norm Vs Batch Norm

Layer Normalization is a technique used to stabilize and accelerate the training of transformers by normalizing the inputs across the features. It adjusts and scales the activations, ensuring consistent output distributions. This helps in reducing training time and improving model performance, making it a key component in transformer architectures.
Share your thoughts, experiences, or questions in the comments below. I love hearing from you!
============================
Did you like my teaching style?
Check my affordable mentorship program at : learnwith.campusx.in
DSMP FAQ: docs.google.com/document/d/1O...
============================
📱 Grow with us:
CampusX' LinkedIn: / campusx-official
CampusX on Instagram for daily tips: / campusx.official
My LinkedIn: / nitish-singh-03412789
Discord: / discord
E-mail us at [e-mail]
✨ Hashtags✨
#deeplearning #campusx #transformers #transformerarchitechture
⌚Time Stamps⌚
00:00 - Intro
02:20 - What is Normalization
03:50 - What do we normalize?
05:30 - Benefits of Normalization in DL
07:10 - Internal Covariate Shift
12:49 - Batch Normalization Revision
22:56 - Why don't we use Batch Norm in Transformers?
38:25 - How does Layer Normalization works?
43:00 - Layer Normalization in Transformer

[@abhisheksaurav]

This playlist is like a time machine. I’ve watched you grow your hair from black to white, and I’ve seen the content quality continuously improve video by video. Great work!

[@animatrix1631]

I feel the same as well but I guess he's not that old

[@zerotohero1002]

Courage comes at a price ❤

[@muhammadsheraz177]

Please end this playlist as early as possible

[@RamandeepSingh_04]

Another student added in the waiting list demanding for next video. Thank you sir.

[@ayushrathore2570]

This whole playlist is the best thing I discovered on KZbin! Thank you so much, sir

[@yashshekhar538]

Respected Sir, your playlist is the best. Kindly increase the frequency of videos.

[@AidenDsouza-ii8rb]

Your DL playlist is like a thrilling TV series - can't wait for the next episode! Any chance we could get a season finale soon? Keep up the awesome work!

[@akeshagarwal794]

Congratulations for building a 200k Family you deserve even more reach🎉❤ We love you sir ❤

[@rajnishadhikari9280]

Thanks for this amazing series.

[@amitmehraa]

please complete this this playlist and add transformers tutorials as soon as possible

[@shreeyagupta5720]

Congratulations for 200k sir 👏 🎉🍺

[@vinaykumar-xh5pi]

please release the next video very curious to complete ...... loved your content as always

[@sahil5124]

this is really important topic. Thank you so much. Please cover everything about Transformer architecture

[@user-nc8nc3lj1c]

Sir try to complete this playlist as early as possible , you are the best teacher and we want to learn the deep learning concept from you

[@krisharora2959]

Next video is awaited more than anything

[@arpitpathak7276]

Thank you sir I am waiting for this video ❤

[@GanitSikho-xo2yx]

Well, I am waiting for your next video. It's a gem of learning!

[@ghousepasha4172]

Please sir update videos regularly, we wait a lot for your videos

[@ai_pie1000]

Congratulations Brother for 200k users Family ... 👏👏👏

[@mayyutyagi]

Amazing series full of knowledge...

[@rb4754]

Congratulations for 200k subscribers!!!!!!!!!!!!!!!!!!

[@bmp-zz9pu]

SIr krdo pls iss playlist ko poora!!!!!!!!!

[@sachink9102]

Thank you, NitishJi, Eeagerly waiting to attend your Transformers sessions. Please complete this series.

[@nvnurav1892]

Sir one small suggestion, aap apni videos pe speech to speech translation laga ke english mai convert kar lo and upload it on Udemy/youtube. it will help a lot of people jinko hindi nhi aati and will help your hard work get more and more attraction.🙂🙂. We are really very lucky that we are getting such rich content for free.. God bless you.

[@znyd.]

Congrats on the 200k subs, love from Bangladesh ❤.

[@dharmendra_397]

Very nice video

[@muhammadsheraz177]

Sir kindly can you tell that when this playlist will complete.

[@hassan_sid]

It would be great if you make a video on RoPE

[@shibrajdeb5177]

sir please upload regular video . This videos help me a lot. please sir upload regular videos

[@AmitBiswas-hd3js]

Please cover this entire Transformer architecture as soon as possible

[@1111Shahad]

Thank you Nitish, Waiting for your next upload.

[@not_amanullah]

This is helpful 🖤

[@taseer12]

Sir I can't describe your efforts Love from Pakistan

[@rose9466]

Can you give an estimate by when this playlist will be completed

[@not_amanullah]

Thanks ❤

[@saurabhbadole821]

I am glad that I found this Channel! can't thank you enough, Nitish Sir! One more request: If you could create one-shot revision videos for machine learning, deep learning, and natural language processing (NLP).🤌

[@princekhunt1]

Sir, Please complete this series.

[@Amanullah-wy3ur]

thanks ❤

[@gurvgupta5515]

Thanks for this video sir. Can you also make a video on Rotary Positional Embeddings (RoPE) that is used in Llama as well as other LLMs for enhanced attention.

[@sachinpatodia4593]

Can you pls explain what is the add in add and norm layer?

[@advaitdanade7538]

Sir please end this playlist fast placement season is nearby😢

[@physicskiduniya8054]

Bhaiya! Awaiting for your course upcoming videos please try to complete this playlist asap bhaiya

[@29_chothaniharsh62]

Sir can you please continue the 100 interview questions on ML playlist?

[@WIN_1306]

at 46:10 ,why it is zero? as beta is added so it will prevent it from becoming zero?

[@WIN_1306]

i am the 300th person to like this video sir plzz upload next vidoes we are eagerly waiting

[@MrSat001]

Great 👍

[@SulemanZeb.]

Please start MLOPs playlist as we are desperately waiting for.......

[@darkpheonix6592]

please upload remaining videos quickly

[@teksinghayer5469]

when will you code transformer from scratch in pytorch

[@technicalhouse9820]

Sir love you so much from Pakistan

[@sagarbhagwani7193]

thanks sir plse complete this playlist asap

[@intcvn]

complete jaldi sir waiting asf

[@shubharuidas2624]

Please also continue with vision transformer

[@manojprasad6781]

Waiting for the next video💌

[@virajkaralay8844]

Absolute banger video again. Appreciate the efforts you're taking for transformers. Cannot wait for when you explain the entire transformer architecture.

[@virajkaralay8844]

Also, congratulations for 200k subscribers. May you reach many more milestones

[@aksholic2797]

200k🎉

[@anonymousman3014]

Sir, is transformer architecture completed as I want to cover it ASAP, I have covered the topics till attention mechanism. I want to cover the topic in one go. Sir please tell please. And, sir I request to upload all video asap. I want to learn a lot. And thanks for the amazing course at 0 cost. God bless you.

[@space_ace7710]

Yeah!!

[@oden4013]

sir please upload next video please its almost a month

[@anonymousman3014]

Sir, is transformer architecture completed as I want to cover it ASAP, I have covered the topics till attention mechanism. I want to cover the topic in one go. Sir please tell please. And, sir I request to upload all video asap. I want to learn a lot. And thanks for the amazing course at 0 cost.

[@ishika7585]

Kindly make video on Regex as well

[@WIN_1306]

what is regex?

[@peace-it4rg]

sir mera doubt that ki mai agar transformer architecture mai batchnorm use karoon kunki jo values matrix mai hai un sabka apna learning rate and bias factor hai to jo bias hai uskai karan to zero chala hi jayega fir layer norm kyun. kyunki ham ((x-u)/var)*lambda+bias krtai hi hain to bias to apne aap usko zero nhi hone dega. Please help sir

[@RamandeepSingh_04]

still it will be a very small number and will affect the result and not represent the true picture of the feature in batch normalization.

[@WIN_1306]

@@RamandeepSingh_04 compared to others who are without padding it will be small, but still sir wrote zero but zero to nhi hi hoga

[@vikassengupta8427]

Sir next video ❤❤

[@zerotohero1002]

one month ho gya sir please upload eagarly waiting🥺🥺🥺

[@barryallen5243]

Just ignoring padded rows while performing batch normalization should also work, I feel like it that padded zeros are not the only reason we layer normalization instead of batch normalization.

[@WIN_1306]

how would you ignore padding cols in batch normalisation?

[@user-mw9ny7wc6l]

Jldi next video dalo sir

[@SANJAYTYAGI-bk6tx]

Sir In batch normalization , in your example we have three mean and three variance along with same number of beta and gamma i.e. 3. But in layer normalization , we have eight mean and eight variance along with 3 beta and 3 gamma. That means number of beta and gamma are same in both batch and layer normalization. Is it correct? Pl elaborate on it .

[@campusx-official]

Yes

[@WIN_1306]

mean and variance are used for normalisation ,beta and gamma are used for scaling

[@ESHAANMISHRA-pr7dh]

Respected sir, I request you to please complete the playlist. I am really thankful to you for your amazing videos in this playlist. I have recommended this playlist to a lot of my friends and they loved it too. Thanks for providing such content for free🙏🙏

[@adarshsagar9817]

sir please complete the NLP playlist

[@WIN_1306]

which one? how many videos does it have?

[@not_amanullah]

🖤🤗

[@titaniumgopal]

Sir PDF Update karo

[@ghousepasha4172]

Sir please complete playlist I will pay 5000 for that

[@DarkShadow00972]

Bring some coding example bro

[@gauravbhasin2625]

Nitish, please relook at your covariate shift funds... yes, you are partially correct but how you explained covariate shift is actually incorrect. (example - Imagine training a model to predict if someone will buy a house based on features like income and credit score. If the model is trained on data from a specific city with a certain average income level, it might not perform well when used in a different city with a much higher average income. The distribution of "income" (covariate) has shifted, and the model's understanding of its relationship to house buying needs to be adjusted.)

[@WIN_1306]

ig , the explanation that sir gave and your explanation are same with different example of covariate shift

[@bmp-zz9pu]

A video after 2 weeks in this playlist.....itna zulam mat karo.....thoda tez kaam kro sirji..............

[@ashutoshpatidar3288]

please be a little fast!

[@WIN_1306]

sir can u tell that around how many and which topics are left?

[@Amanullah-wy3ur]

this is helpful 🖤

[@anonymousman3014]

Sir, is transformer architecture completed as I want to cover it ASAP, I have covered the topics till attention mechanism. I want to cover the topic in one go. Sir please tell please. And, sir I request to upload all video asap. I want to learn a lot. And thanks for the amazing course at 0 cost. God bless you.

[@anonymousman3014]

Sir, is transformer architecture completed as I want to cover it ASAP, I have covered the topics till attention mechanism. I want to cover the topic in one go. Sir please tell please. And, sir I request to upload all video asap. I want to learn a lot. And thanks for the amazing course at 0 cost. God bless you.

[@anonymousman3014]

Sir, is transformer architecture completed as I want to cover it ASAP, I have covered the topics till attention mechanism. I want to cover the topic in one go. Sir please tell please. And, sir I request to upload all video asap. I want to learn a lot. And thanks for the amazing course at 0 cost. God bless you.

[@anonymousman3014]

Sir, is transformer architecture completed as I want to cover it ASAP, I have covered the topics till attention mechanism. I want to cover the topic in one go. Sir please tell please. And, sir I request to upload all video asap. I want to learn a lot. And thanks for the amazing course at 0 cost. God bless you.

[@anonymousman3014]

Sir, is transformer architecture completed as I want to cover it ASAP, I have covered the topics till attention mechanism. I want to cover the topic in one go. Sir please tell please. And, sir I request to upload all video asap. I want to learn a lot. And thanks for the amazing course at 0 cost. God bless you.

[@anonymousman3014]

Sir, is transformer architecture completed as I want to cover it ASAP, I have covered the topics till attention mechanism. I want to cover the topic in one go. Sir please tell please. And, sir I request to upload all video asap. I want to learn a lot. And thanks for the amazing course at 0 cost. God bless you.

[@anonymousman3014]

Sir, is transformer architecture completed as I want to cover it ASAP, I have covered the topics till attention mechanism. I want to cover the topic in one go. Sir please tell please. And, sir I request to upload all video asap. I want to learn a lot. And thanks for the amazing course at 0 cost. God bless you.

[@anonymousman3014]

Sir, is transformer architecture completed as I want to cover it ASAP, I have covered the topics till attention mechanism. I want to cover the topic in one go. Sir please tell please. And, sir I request to upload all video asap. I want to learn a lot. And thanks for the amazing course at 0 cost. God bless you.

[@anonymousman3014]

Sir, is transformer architecture completed as I want to cover it ASAP, I have covered the topics till attention mechanism. I want to cover the topic in one go. Sir please tell please. And, sir I request to upload all video asap. I want to learn a lot. And thanks for the amazing course at 0 cost. God bless you.

[@anonymousman3014]

Sir, is transformer architecture completed as I want to cover it ASAP, I have covered the topics till attention mechanism. I want to cover the topic in one go. Sir please tell please. And, sir I request to upload all video asap. I want to learn a lot. And thanks for the amazing course at 0 cost. God bless you.

[@anonymousman3014]

Sir, is transformer architecture completed as I want to cover it ASAP, I have covered the topics till attention mechanism. I want to cover the topic in one go. Sir please tell please. And, sir I request to upload all video asap. I want to learn a lot. And thanks for the amazing course at 0 cost. God bless you.

[@anonymousman3014]

Sir, is transformer architecture completed as I want to cover it ASAP, I have covered the topics till attention mechanism. I want to cover the topic in one go. Sir please tell please. And, sir I request to upload all video asap. I want to learn a lot. And thanks for the amazing course at 0 cost. God bless you.