Vision Transformers explained

[@jayp9158]

Your explanation is one for the best I’ve heard about ViT, thank you very much

[@CodeWithAarohi]

Glad it was helpful!

[@naziahossain3950]

i agree

[@emirhanbilgic2475]

Aarohi, I am watching you for 3 years now, and each time I understand the subject. You're literally the best

[@CodeWithAarohi]

Thank you so much for your incredibly kind words! It means a lot to me😊

[@patis.IA-AI]

Thanks again for this very well explained tuto.

[@CodeWithAarohi]

Glad it was helpful!

[@ariouathanane]

What's about the extra class? and i think that only the extra class is used for the classification. Please could you explain this point?

[@煎饼果子爱学习]

I am a new learner of vision transformer and your explaination is so simple to understand and is also so informative! Thank you!

[@CodeWithAarohi]

Glad it was helpful!

[@CodeWithAarohi]

I have recently posted another video which is even the detailed video on vision transformer. You can check that to understand the concepts in depth: kzbin.info/www/bejne/iXrOiqNsmdiWgtk

[@Rakesh_Seerla]

@@CodeWithAarohi i have a doubt arohi u made 3 videos on vit so also u did 1.11 hr video on vit is both are same or we differ from each other? can u clarify please thank you

[@CodeWithAarohi]

@@Rakesh_Seerla They are different. 1.11 hr video is detailed video where concepts like linear projection, query, key and value are explained in detail.

[@sreekalakishore8422]

Very nice Presentation

[@CodeWithAarohi]

Thanks a lot

[@sohaibahmed9165]

V helpful for beginners❤

[@CodeWithAarohi]

Glad this helped you!

[@_seeker423]

beautifully explained!

[@CodeWithAarohi]

Glad it was helpful!

[@sukumarane2302]

Explanation is very clear .. excellent!

[@CodeWithAarohi]

Glad it was helpful!

[@vikashverma7893]

Excellent explanation

[@CodeWithAarohi]

Glad it was helpful!

[@saireddy7628]

Great insights and easy to consume.

[@CodeWithAarohi]

Glad it helped!

[@SubhranilPaul-gi2jx]

can i get the ppt?

[@manojtelrandhe174]

Great....crystal clear the concepts greatly explained 😊

[@CodeWithAarohi]

Glad it helped!

[@beat-x8794]

short and crisp. studying one day before exam. thank you ma'am

[@CodeWithAarohi]

You are welcome. Hope your exam went well :)

[@beat-x8794]

@CodeWithAarohi nailed it. A question was asked about vit which I properly explained with diagram, thanks to your teaching. Keep making videos on ai topics. It really helps. Will explore your channel more for learning new topics. Thank you again ma'am.

[@soravsingla6574]

Hello Ma’am Your AI and Data Science content is consistently impressive! Thanks for making complex concepts so accessible. Keep up the great work! 🚀 #ArtificialIntelligence #DataScience #ImpressiveContent 👏👍

[@CodeWithAarohi]

My pleasure 😊

[@anugaur2672]

Awesome explanation mam

[@CodeWithAarohi]

Glad you liked it

[@SS-zq5sc]

Your tutorials are always the best, thank you very much. I hope you would create tutorials on Segformer soon.

[@CodeWithAarohi]

Thank you, I will

[@tauseefkhan6901]

How dimension is reduced for each 1D vector when each pixel of 1D vector is multiplied by weights? Can u clear it?

[@QubitBrain]

Matrix multiplication! Let's assume an image is split into 3x3 pixel and each pixel has 16x16 vector embedding which is flattened to 256x1 (means 256 rows and 1 column). Because we have 3x3 pixel size of image it means we have total 9 pixels. Hence if we combine the vector embedding of all the pixels (means if each pixel embedding is 256x1, then for 9 pixels it will become 256x9 i.e 256 rows and 9 columns. Now we have to pass this through linear layer. Linear layer let's say has 5 neurons. so shape for each neuron will be 256 x 1 (means 256 rows and 1 column) and for 5 neurons it will become 256x5 (menas 256 rows and 5 columns). Now we have to do matrix Multiplication of Input with Linear layer, but wait, we cannot multiply the matrix because shape of input is 256x9 and shape of linear is 256x5. In order to multiply the matrices, the columns of Matrix A must be equal to the number of rows of Matrix B. So we will transpose the input matrix of shape 256x9 to 9x256. Now, Let's take this as Matrix A of 9x256 and Matrix B of size 256x5. Because column of Matrix A is same as row of Matrix B, hence, dot product is possible which will result in new matrix of size 9x5. We can see that the original matrix of patch was of size 9x256 which is reduced to 9x5. So we will get the 3 matrices of size 9x5 each for Key, Query and Value. Now based on attention model we can see that we have to do the matrix multiplication of Query and Key and to do so we again have to do the transpose of Matrix because both Key and Query are of same shape (Query Matrix - 9x5 , Key Matrix - 9x5). So if we transpose Key Matrix it will become 5x9 and then the matrix multiplication will be possible between these two matrices (9x5 and 5x9). The dot product output of these matrices will be a matrix of size (9x9) and this output matrix is called as Attention Filter. Then after training we have the final updated values of this attention filter which we have to scale between 0 and 1 using softmax activation function. This scaled attention matrix (9x9) is then multiplied with Value matrix (9x5) which will give the filtered value of Matrix (9x5). Hence based on attention matrix we get the important feature of an image. This is the process of single attention head to extract feature. We use multi-head attention to extract various important features of an image. Each head focuses on different combinations of features.

[@Rahul-vl1no]

Can you please suggest how to use vision transformer for Text classification? Please

[@AbdulNaffiAhanger]

very good explanation.

[@berfin923]

The content is amazing! Very informative, short, and to the point, which is great for beginners. Thank you for these amazing videos 😍I have only one small feedback for your future videos. The audio quality is a little bit bad and noisy. You might consider checking your microphone.

[@CodeWithAarohi]

Thank you for the feedback. I will take care of noise.

[@ShubhamSharma-bo3ot]

Thank You, can you explain difference between CNN and ViT side by side.

[@DeerajRManjaray]

Great Content!!

[@CodeWithAarohi]

Thank you!

[@naziahossain3950]

you are a genius ma Shaa Allah, thanks a lot

[@CodeWithAarohi]

You are most welcome

[@devanshlakshitha7424]

Thanks for this vedio.this tutorial is very clear and explaining and we had learning to how to split the pattern

[@CodeWithAarohi]

You are welcome 😊

[@Unskilledcow30]

Thanks very much the videos are awesome and genuine.

[@CodeWithAarohi]

Glad you like them!

[@yabezD]

Kindly post a video for Deit

[@CodeWithAarohi]

Noted!

[@sm-pz8er]

Very well explained. Thanks alot

[@CodeWithAarohi]

Glad it was helpful!

[@soudaminipanda]

very nice video.

[@CodeWithAarohi]

Many many thanks

[@NISHAKAREEM-g9k]

very good explanation. Thank you

[@CodeWithAarohi]

You are welcome!

[@soravsingla6574]

Good one ma’am

[@CodeWithAarohi]

Thanks a lot

[@kadapallanithin]

Thanks for making the video

[@CodeWithAarohi]

My pleasure!

[@sanathspai3210]

Hi, it is nice content btw I have one doubt. If u divide 224*224 image into patch size = 16, that means there will be 16 grids as shown in 2:33 each patch having 14 pixels? Is my understanding correct?

[@CodeWithAarohi]

You have a total of 14×14=196 patches. Each patch is 16×16 pixels in size. So, the number of grids (patches) is 196, and each patch has 16x16 pixels along each side, not 14x14.

[@sanathspai3210]

@@CodeWithAarohi Oh okay. I got confused by looking onto image which had 16 cells. I hope that's a mistake right it should have been 14 cells right? And each cell has 16*16 pixels along its width and height

[@Unskilledcow30]

Please can you explain or give a series about the Vanilla Vision transformers from the paper to the to the programming side🙏🙏

[@CodeWithAarohi]

The terms "Vanilla Vision Transformers" and "Vision Transformers" are often used interchangeably, and both refer to the same fundamental concept which is applying the Transformer architecture directly to image data for computer vision tasks.

[@RAZZKIRAN]

how to know the feature importance which are generated from ViT ? which features causes classification

[@CodeWithAarohi]

While ViT doesn't inherently provide feature importance scores like some other models, you can analyze the importance of different features in the classification process by examining the attention maps generated by the model. Attention maps in ViT represent the importance of each image patch in relation to the final prediction. Higher attention values indicate greater importance. By visualizing these attention maps, you can gain insights into which image regions contribute most to the classification decision.

[@RAZZKIRAN]

@@CodeWithAarohi please make video on it madam, for one classification task , dog vs cat classification example

[@salmatiru8797]

why we we are using 16x16 patches

[@CodeWithAarohi]

Because in original paper of ViT, they have used patch size 16*16. You can work with some other patch size also. Eg: Patch Size 8×8 - which will give you 784 patches if your image size is 224*224. Another example: Patch Size 32×32 will give you 49 patches if image size is 224*224

[@cleverestidiot4636]

the video was awesome . And can i know the transformer model of all the 6 encoders and 6 decoders , as I have confusion in the input architecture of decoders . Thank you mam

[@mohamedahmed-kd8ue]

Thanks for this tutorials its simple and deep

[@CodeWithAarohi]

You're welcome 😊

[@Sunil-ez1hx]

Thank you soo much mam for this amazing video

[@CodeWithAarohi]

Most welcome 😊

[@sanjoetv5748]

please create a ViT on landmark detection

[@CodeWithAarohi]

I will try

[@sanjoetv5748]

@@CodeWithAarohi thank you so much you are the best

[@anantmohan3158]

Nicely Explained..!

[@CodeWithAarohi]

Thank you

[@mostafamarwanmostafa9975]

can you make a video on SegFormer? thanks in advance for the amazing explanation!

[@CodeWithAarohi]

I will try!

[@BlessingRasheed-nv5tm]

Can these be apply in bank cheque processing

[@CodeWithAarohi]

Yes, vision transformers (ViTs) can be applied to bank cheque processing tasks.

[@beratcokhavali]

excellent explanation. I wanna make a sugesstion. Maybe you should buy a microphone. There are lots of noise in background.

[@CodeWithAarohi]

Thank you, I will

[@karthickkuduva9819]

Waiting for your fusion transformer tutorial mam

[@CodeWithAarohi]

Ok sure

[@ervinperetz5973]

I came to this video to learn how to do positional encoding for 2D images -- the precise math. When you come to that portion, you simply reference your intro video, re Transformers for linear text (and in which even the linear positional encoding isn't really explained).

[@CodeWithAarohi]

Sorry for inconvenience. I will try to cover the math's in separate video.

[@ervinperetz5973]

@@CodeWithAarohi Thanks for responding. Your videos are terrific otherwise. Thanks for sharing your work and insights.

[@MP-sx6tg]

‘The precise math for encoding’ Bro it’s deep learning and you talk about precise math 😂 Literally those people encoded 1,2,…256 for each patch.

[@Reem.alhamimi]

WOW WOW WOW!!!!

[@CodeWithAarohi]

Thank you :)

[@vimalshrivastava6586]

Thank you for making this video. Please make a python code of ViT, if possible. Thank you.

[@CodeWithAarohi]

Working on it!

[@umarjibrilmohd8660]

Please, do it on how to train ViT on semantic segmentation tasks.

[@aluissp]

Thanks a lot! it helps me :3

[@CodeWithAarohi]

I'm glad!

[@fayezalhussein7115]

thank you so much Aarohi, please,could you explain SWIN transformer too with its with coding ?

[@CodeWithAarohi]

Sure, I have started a playlist on transformers and will try to cover every important topic which comes under transformers

[@fayezalhussein7115]

@@CodeWithAarohi thank you again, waiting for it

[@AhmedAbdelAzizeMoahmed]

nice, can you share slide with me?

[@soravsingla6574]

Code with Aarohi is Best KZbin channel for Artificial Intelligence #CodeWithAarohi

[@Pradeep...87]

Can you provide code

[@CodeWithAarohi]

In this video, I have explained Vision transformer theory. You can check next video and Code link is mentioned in description section of that video.

[@alis5893]

will you do vision transformers with tensorflow?

[@CodeWithAarohi]

I will try.

[@alis5893]

@@CodeWithAarohi Thank you. Your method of teaching is amazing. But i am never comfortable with torch. Tensorflow is so natural for deep learning. I look forward to this .