Layer Normalization in Transformers | Layer Norm Vs Batch Norm

[@abhisheksaurav]

This playlist is like a time machine. I’ve watched you grow your hair from black to white, and I’ve seen the content quality continuously improve video by video. Great work!

[@animatrix1631]

I feel the same as well but I guess he's not that old

[@zerotohero1002]

Courage comes at a price ❤

[@sachink9102]

Thank you, NitishJi, Eeagerly waiting to attend your Transformers sessions. Please complete this series.

[@RamandeepSingh_04]

Another student added in the waiting list demanding for next video. Thank you sir.

[@muhammadsheraz177]

Please end this playlist as early as possible

[@ayushrathore2570]

This whole playlist is the best thing I discovered on KZbin! Thank you so much, sir

[@nvnurav1892]

Sir one small suggestion, aap apni videos pe speech to speech translation laga ke english mai convert kar lo and upload it on Udemy/youtube. it will help a lot of people jinko hindi nhi aati and will help your hard work get more and more attraction.🙂🙂. We are really very lucky that we are getting such rich content for free.. God bless you.

[@yashshekhar538]

Respected Sir, your playlist is the best. Kindly increase the frequency of videos.

[@sahil5124]

this is really important topic. Thank you so much. Please cover everything about Transformer architecture

[@sharangkulkarni1759]

जबरदस्त,! मजा आगया, जिस तरह से padding के zeroes को लपेटे मे ले लिया, मजा आगया

[@akeshagarwal794]

Congratulations for building a 200k Family you deserve even more reach🎉❤ We love you sir ❤

[@ShivamSuri-lz5it]

Excellent deep learning playlist , highly recommended !!

[@PrathameshKhade-j2e]

Sir try to complete this playlist as early as possible , you are the best teacher and we want to learn the deep learning concept from you

[@SBhupendraAdhikari]

Thanks a Lot Sir, Really enjoying the learning of Transformers

[@udaysharma138]

Thanks a lot Nitish Sir , best Explanation

[@praneethkrishna6782]

@campusx Hi Nitish, thanks a lot for the elaborated explanation. But I have a query, Is it really that the values '0' representing the padding tokens really the reason (or the only reason) which is stopping from using Batch Normalization. because it can be internally handled to not consider '0' which calulating the mean and stadard deviation while calulating z across features. on the other hand I think, this technique (Batch Norm) is clubbing the embeddings of different sentences while calulating Z which seems little odd to me. and that is the reason for not using this technique. please correct me if I am wrong here

[@AKSHATSHAW-tf3ow]

Same doubt, I think there is both the reasons for this.

[@vinaykumar-xh5pi]

please release the next video very curious to complete ...... loved your content as always

[@ryannflin1285]

bhai literally mujhe samajh nhi aa rha hai ki mujhe samajh kaise aa rha, koi itna accha kaise padha sakta hai yrr,love u sir ( from IITJ)

[@AidenDsouza-ii8rb]

Your DL playlist is like a thrilling TV series - can't wait for the next episode! Any chance we could get a season finale soon? Keep up the awesome work!

[@just__arif]

Top-quality content!

[@Xanchs4593]

Can you pls explain what is the add in add and norm layer?

[@ai_pie1000]

Congratulations Brother for 200k users Family ... 👏👏👏

[@GanitSikho-xo2yx]

Well, I am waiting for your next video. It's a gem of learning!

[@Fazalenglish]

I really like the way you explain things ❤

[@taseer12]

Sir I can't describe your efforts Love from Pakistan

[@saurabhbadole821]

I am glad that I found this Channel! can't thank you enough, Nitish Sir! One more request: If you could create one-shot revision videos for machine learning, deep learning, and natural language processing (NLP).🤌

[@WIN_1306]

at 46:10 ,why it is zero? as beta is added so it will prevent it from becoming zero?

[@dilippokhrel4009]

Initially the gama value is kept 1 and beta is kept zero, hence initially the value will be zero. But during training process may be value will be other than zero

[@Schrödinger3]

please complete this this playlist and add transformers tutorials as soon as possible

[@shibrajdeb5177]

sir please upload regular video . This videos help me a lot. please sir upload regular videos

[@ghousepasha4172]

Please sir update videos regularly, we wait a lot for your videos

[@Shisuiii69]

Question: Sir agr kia ho ky jo padding wala vector hai isme B¹ ki value 0 ky bjae khuch aur ajae ky wo update hoti rehti hai to is se padding vector 0 nhi rhe ga to kia ye model me affect nhi kre ga ?

[@slaypop-b5n]

Bro Did u find the answer ? Had the same doubt

[@Ishant875]

Same doubt

[@slaypop-b5n]

@@Ishant875 any updates , bro ? Did u get the answer ?

[@AmitBiswas-hd3js]

Please cover this entire Transformer architecture as soon as possible

[@1111Shahad]

Thank you Nitish, Waiting for your next upload.

[@ESHAANMISHRA-pr7dh]

Respected sir, I request you to please complete the playlist. I am really thankful to you for your amazing videos in this playlist. I have recommended this playlist to a lot of my friends and they loved it too. Thanks for providing such content for free🙏🙏

[@gurvgupta5515]

Thanks for this video sir. Can you also make a video on Rotary Positional Embeddings (RoPE) that is used in Llama as well as other LLMs for enhanced attention.

[@AkashSingh-oz7qx]

please also cover Generative and diffusion models.

[@muhammadsheraz177]

Sir kindly can you tell that when this playlist will complete.

[@shreeyagupta5720]

Congratulations for 200k sir 👏 🎉🍺

[@bmp-zz9pu]

SIr krdo pls iss playlist ko poora!!!!!!!!!

[@mayyutyagi]

Amazing series full of knowledge...

[@krisharora2959]

Next video is awaited more than anything

[@arunkrishna1036]

Sir what if Beta value is updated during learning process? Then it will get added along with the padded zeros making it a non zero value in the further iterations

[@Shisuiii69]

Same confusion, did you find the answer?

[@physicskiduniya8054]

Bhaiya! Awaiting for your course upcoming videos please try to complete this playlist asap bhaiya

[@znyd.]

Congrats on the 200k subs, love from Bangladesh ❤.

[@darkpheonix6592]

please upload remaining videos quickly

[@anonymousman3014]

Sir, is transformer architecture completed as I want to cover it ASAP, I have covered the topics till attention mechanism. I want to cover the topic in one go. Sir please tell please. And, sir I request to upload all video asap. I want to learn a lot. And thanks for the amazing course at 0 cost. God bless you.

[@WIN_1306]

sir can u tell that around how many and which topics are left?

[@princekhunt1]

Sir, Please complete this series.

[@SulemanZeb.]

Please start MLOPs playlist as we are desperately waiting for.......

[@SaurabhKumar-t4s]

At 46:04, if sigma4 is 0 then how do we divide with this value.

[@Shisuiii69]

Mujhe bhi same confusion thi chatgpt se pta kiya usne kha ky hum ek error value add krty jo very close to zero hota hai to isliye hum zero likh dety hai after normalization

[@ishika7585]

Kindly make video on Regex as well

[@WIN_1306]

what is regex?

[@himansuranjanmallick16]

thank you sir................

[@UCS_B_DebopamDey]

thank you sir

[@rajnishadhikari9280]

Thanks for this amazing series.

[@peace-it4rg]

sir mera doubt that ki mai agar transformer architecture mai batchnorm use karoon kunki jo values matrix mai hai un sabka apna learning rate and bias factor hai to jo bias hai uskai karan to zero chala hi jayega fir layer norm kyun. kyunki ham ((x-u)/var)*lambda+bias krtai hi hain to bias to apne aap usko zero nhi hone dega. Please help sir

[@RamandeepSingh_04]

still it will be a very small number and will affect the result and not represent the true picture of the feature in batch normalization.

[@WIN_1306]

@@RamandeepSingh_04 compared to others who are without padding it will be small, but still sir wrote zero but zero to nhi hi hoga

[@technicalhouse9820]

Sir love you so much from Pakistan

[@intcvn]

complete jaldi sir waiting asf

[@rb4754]

Congratulations for 200k subscribers!!!!!!!!!!!!!!!!!!

[@sagarbhagwani7193]

thanks sir plse complete this playlist asap

[@hassan_sid]

It would be great if you make a video on RoPE

[@anonymousman3014]

Sir, is transformer architecture completed as I want to cover it ASAP, I have covered the topics till attention mechanism. I want to cover the topic in one go. Sir please tell please. And, sir I request to upload all video asap. I want to learn a lot. And thanks for the amazing course at 0 cost.

[@WIN_1306]

i am the 300th person to like this video sir plzz upload next vidoes we are eagerly waiting

[@29_chothaniharsh62]

Sir can you please continue the 100 interview questions on ML playlist?

[@shubharuidas2624]

Please also continue with vision transformer

[@advaitdanade7538]

Sir please end this playlist fast placement season is nearby😢

[@arpitpathak7276]

Thank you sir I am waiting for this video ❤

[@SuperRia33]

Thanks

[@virajkaralay8844]

Absolute banger video again. Appreciate the efforts you're taking for transformers. Cannot wait for when you explain the entire transformer architecture.

[@virajkaralay8844]

Also, congratulations for 200k subscribers. May you reach many more milestones

[@vinayakbhat9530]

excellent

[@not_amanullah]

This is helpful 🖤

[@oden4013]

sir please upload next video please its almost a month

[@rose9466]

Can you give an estimate by when this playlist will be completed

[@khatiwadaAnish]

Awesome 👍👍

[@manojprasad6781]

Waiting for the next video💌

[@not_amanullah]

Thanks ❤

[@barryallen5243]

Just ignoring padded rows while performing batch normalization should also work, I feel like it that padded zeros are not the only reason we layer normalization instead of batch normalization.

[@WIN_1306]

how would you ignore padding cols in batch normalisation?

[@SANJAYTYAGI-bk6tx]

Sir In batch normalization , in your example we have three mean and three variance along with same number of beta and gamma i.e. 3. But in layer normalization , we have eight mean and eight variance along with 3 beta and 3 gamma. That means number of beta and gamma are same in both batch and layer normalization. Is it correct? Pl elaborate on it .

[@campusx-official]

Yes

[@WIN_1306]

mean and variance are used for normalisation ,beta and gamma are used for scaling

[@teksinghayer5469]

when will you code transformer from scratch in pytorch

[@adarshsagar9817]

sir please complete the NLP playlist

[@WIN_1306]

which one? how many videos does it have?

[@vikassengupta8427]

Sir next video ❤❤

[@gauravbhasin2625]

Nitish, please relook at your covariate shift funds... yes, you are partially correct but how you explained covariate shift is actually incorrect. (example - Imagine training a model to predict if someone will buy a house based on features like income and credit score. If the model is trained on data from a specific city with a certain average income level, it might not perform well when used in a different city with a much higher average income. The distribution of "income" (covariate) has shifted, and the model's understanding of its relationship to house buying needs to be adjusted.)

[@WIN_1306]

ig , the explanation that sir gave and your explanation are same with different example of covariate shift

[@dharmendra_397]

Very nice video

[@zerotohero1002]

one month ho gya sir please upload eagarly waiting🥺🥺🥺

[@harshgupta-w5y]

Jldi next video dalo sir

[@MrSat001]

Great 👍

[@turugasairam2886]

sir, why dont you translate it to english and upload , making a new channel like campusX english, i am sure it will attract more audience and reach. I am sure you would have thought of this already

[@titaniumgopal]

Sir PDF Update karo

[@aksholic2797]

200k🎉

[@bmp-zz9pu]

A video after 2 weeks in this playlist.....itna zulam mat karo.....thoda tez kaam kro sirji..............

[@ghousepasha4172]

Sir please complete playlist I will pay 5000 for that

[@faizack]

😂😂😂🎉

[@space_ace7710]

Yeah!!

[@not_amanullah]

🖤🤗

[@DarkShadow00972]

Bring some coding example bro

[@ashutoshpatidar3288]

please be a little fast!

[@anonymousman3014]

Sir, is transformer architecture completed as I want to cover it ASAP, I have covered the topics till attention mechanism. I want to cover the topic in one go. Sir please tell please. And, sir I request to upload all video asap. I want to learn a lot. And thanks for the amazing course at 0 cost. God bless you.

[@Amanullah-wy3ur]

this is helpful 🖤

[@Amanullah-wy3ur]

thanks ❤

[@anonymousman3014]

Sir, is transformer architecture completed as I want to cover it ASAP, I have covered the topics till attention mechanism. I want to cover the topic in one go. Sir please tell please. And, sir I request to upload all video asap. I want to learn a lot. And thanks for the amazing course at 0 cost. God bless you.

[@anonymousman3014]

Sir, is transformer architecture completed as I want to cover it ASAP, I have covered the topics till attention mechanism. I want to cover the topic in one go. Sir please tell please. And, sir I request to upload all video asap. I want to learn a lot. And thanks for the amazing course at 0 cost. God bless you.

[@anonymousman3014]

Sir, is transformer architecture completed as I want to cover it ASAP, I have covered the topics till attention mechanism. I want to cover the topic in one go. Sir please tell please. And, sir I request to upload all video asap. I want to learn a lot. And thanks for the amazing course at 0 cost. God bless you.

[@anonymousman3014]

Sir, is transformer architecture completed as I want to cover it ASAP, I have covered the topics till attention mechanism. I want to cover the topic in one go. Sir please tell please. And, sir I request to upload all video asap. I want to learn a lot. And thanks for the amazing course at 0 cost. God bless you.