Transformers for beginners | What are they and how do they work

Over the past five years, Transformers, a neural network architecture, have completely transformed state-of-the-art natural language processing.
*************************************************************************
For queries: You can comment in comment section or you can mail me at [e-mail]
*************************************************************************
The encoder takes the input sentence and converts it into a series of numbers called vectors, which represent the meaning of the words. These vectors are then passed to the decoder, which generates the translated sentence.
Now, the magic of the transformer network lies in how it handles attention. Instead of looking at each word one by one, it considers the entire sentence at once. It calculates a similarity score between each word in the input sentence and every other word, giving higher scores to the words that are more important for translation.
To do this, the transformer network uses a mechanism called self-attention. Self-attention allows the model to weigh the importance of each word in the sentence based on its relevance to other words. By doing this, the model can focus more on the important parts of the sentence and less on the irrelevant ones.
In addition to self-attention, transformer networks also use something called positional encoding. Since the model treats words as individual entities, it doesn't have any inherent understanding of word order. Positional encoding helps the model to understand the sequence of words in a sentence by adding information about their position.
Once the encoder has calculated the attention scores and combined them with positional encoding, the resulting vectors are passed to the decoder. The decoder uses a similar attention mechanism to generate the translated sentence, one word at a time.
Transformers are the model behind GPT, BERT, and T5
#transformers #naturallanguageprocessing #nlp

[@lyeln]

This is the only video around that REALLY EXPLAINS the transformer! I immensely appreciate your step by step approach and the use of the example. Thank you so much 🙏🙏🙏

[@CodeWithAarohi]

Glad it was helpful!

[@eng.reemali9214]

exactly

[@mdfarhadhussain]

Very nice high level description of Transformer

[@CodeWithAarohi]

Glad you think so!

[@exoticcoder5365]

Very well explained ! I can instantly grab the concept ! Thank you Miss !

[@CodeWithAarohi]

Glad it was helpful!

[@PallaviPadav]

Accidentally I came across this video, very well explained. You are doing an excellent job .

[@CodeWithAarohi]

Glad it was helpful!

[@BharatK-mm2uy]

Great Explanation, Thanks

[@CodeWithAarohi]

Glad it was helpful!

[@MrPioneer7]

I had watched 3 or 4 videos about transformers before this tutorial. Finally, this tutorial made me understand the concept of transformers. Thanks for your complete and clear explanations and your illustrative example. Specially, your description about query, key and value was really helpful.

[@CodeWithAarohi]

You're very welcome!

[@aditichawla3253]

Great explanation! Keep uploading such nice informative content.

[@CodeWithAarohi]

Thank you, I will

[@user-kx1nm3vw5s]

Its great. I have only one query as whats the input of the masked multi-head attention as its not clear to me kindly guide me about it?

[@MAHI-kj5tg]

Just amazing explanation 👌

[@CodeWithAarohi]

Thanks a lot 😊

[@harshilldaggupati]

Very well explained, even with such niche viewer base, keep making more of these please

[@CodeWithAarohi]

Thank you, I will

[@VishalSingh-wt9yj]

Well explained. before watching this video i was very confused in understanding how transformers works but your video helped me alot

[@CodeWithAarohi]

Glad my video is helpful!

[@servatechtips]

This is a fantastic, Very Good explanation. Thank you so much for good explanation

[@CodeWithAarohi]

Glad it was helpful!

[@animexworld6614]

Great Content

[@CodeWithAarohi]

Thanks!

[@user-mv5bo4vf2v]

Hello and Thank you so much. 1 question: I don't realize where the numbers in word embedding and positional encoding come from?

[@user-oo2co6xb8u]

Wow.. you are amazing. Thank you for the clear explanation

[@CodeWithAarohi]

You're very welcome!

[@soravsingla6574]

Hello Ma’am Your AI and Data Science content is consistently impressive! Thanks for making complex concepts so accessible. Keep up the great work! 🚀 #ArtificialIntelligence #DataScience #ImpressiveContent 👏👍

[@CodeWithAarohi]

Thank you!

[@imranzahoor387]

best explanation i saw multiple video but this provide the clear concept keep it up

[@CodeWithAarohi]

Glad to hear that

[@debarpitosinha1162]

Great Explanation mam

[@CodeWithAarohi]

Glad you liked it

[@pandusivaprasad4277]

excellent explanation madam... thank you so much

[@CodeWithAarohi]

Thanks and welcome

[@soravsingla6574]

Very well explained

[@CodeWithAarohi]

Thanks for liking

[@satishbabu5510]

thank you very much for explaining and breaking it down 😀 comparatively so far, your explanation is easy to understand compared to other channels thank you very much for making this video and sharing to everyone❤

[@CodeWithAarohi]

Glad it was helpful!

[@TheMayankDixit]

Nice explanation Ma'am.

[@CodeWithAarohi]

Thank you! 🙂

[@AbdulHaseeb091]

Ma'am, we are eagerly hoping for a comprehensive Machine Learning and Computer Vision playlist. Your teaching style is unmatched, and I truly wish your channel reaches 100 million subscribers! 🌟

[@CodeWithAarohi]

Thank you so much for your incredibly kind words and support!🙂 Creating a comprehensive Machine Learning and Computer Vision playlist is an excellent idea, and I'll definitely consider it for future content.

[@bijayalaxmikar6982]

excellent explanation

[@CodeWithAarohi]

Glad you liked it!

[@user-dl4jq2yn1c]

Best video ever explaining the concepts in really lucid way maam,thanks a lot,pls keep posting,i subscribed 😊🎉

[@CodeWithAarohi]

Thanks and welcome

[@blindprogrammer]

very high level but perfect!

[@CodeWithAarohi]

Thanks!

[@vasoyarutvik2897]

Very Good Video Ma'am, Love from Gujarat, Keep it up

[@CodeWithAarohi]

Thanks a lot

[@minalmahala5260]

Really very nice explanation ma'am!

[@CodeWithAarohi]

Glad my video is helpful!

[@manishnayak9759]

Thanks Aaroh i😇

[@CodeWithAarohi]

Glad it helped!

[@vimalshrivastava6586]

Thanks for making such an informative video. Please could you make a video on the transformer for image classification or image segmentation applications.

[@CodeWithAarohi]

Will cover that soon

[@afn8370]

your video is good, explanation is excellent , only negative I felt was the bg noise. pls use a better mic with noise cancellation. thankyou once again for this video

[@CodeWithAarohi]

Noted! I will take care of the noise :)

[@sahaj2805]

The best explanation of transformer that I have got on the internet , can you please make a detailed long video on transformers with theory , mathematics and more examples. I am not clear about linear and softmax layer and what is done after that , how training happens and how transformers work on the test data , can you please make a detailed video on this?

[@CodeWithAarohi]

I will try to make it after finishing the pipelined work.

[@sahaj2805]

@@CodeWithAarohi Thanks will wait for the detailed transformer video :)

[@_Who_u_are]

Thank you so much

[@CodeWithAarohi]

Welcome!

[@thangarajerode7971]

Thanks. Concept explained very well. Could you please add one custom example (e.g finding similarity questions)using Transformers?

[@CodeWithAarohi]

Will try

[@sairampenjarla]

hi, Good explanation but at the end, when you explained what would be the input to the decoder's masked multi-head attention, you fumbled and didn't explain clearly. But the rest of the video was very good.

[@CodeWithAarohi]

Thank you for the feedback!

[@akshayanair6074]

Thank you. The concept has been explained very well. Could you please also explain how these query, key and value vectors are calculated?

[@CodeWithAarohi]

Sure, Will cover that in a separate video.

[@digambar6191]

Thank you mam

[@CodeWithAarohi]

Most welcome 😊

[@burerabiya7866]

can you please upload the presentation

[@user-wh8vy9ol8w]

Can you please let us know I/p for mask multi head attention. You just said decoder. Can you please explain. Thanks

[@_seeker423]

Can you also talk about the purpose of the 'feed forward' layer. looks like its only there to add non-linearity. is that right?

[@abirahmedsohan3554]

Yes you can say that..but mayb also for make key, quarry and value trainable

[@nikhilrao20]

Didn't understand what is the input to the masked multi head self attention layer in the decoder, Can you please explain me?

[@CodeWithAarohi]

In the Transformer decoder, the masked multi-head self-attention layer takes three inputs: Queries(Q), Keys(K) and Values(V) Queries (Q): These are vectors representing the current positions in the sequence. They are used to determine how much attention each position should give to other positions. Keys (K): These are vectors representing all positions in the sequence. They are used to calculate the attention scores between the current position (represented by the query) and all other positions. Values (V): These are vectors containing information from all positions in the sequence. The values are combined based on the attention scores to produce the output for the current position. The masking in the self-attention mechanism ensures that during training, a position cannot attend to future positions, preventing information leakage from the future. In short, the masked multi-head self-attention layer helps the decoder focus on relevant parts of the input sequence while generating the output sequence, and the masking ensures it doesn't cheat by looking at future information during training.

[@sahaj2805]

Can you please make a detailed video explaining the Attention is all you need research paper line by line, thanks in advance :)

[@CodeWithAarohi]

Noted!

[@_seeker423]

Question about query, key, value dimensionality Given that query is a word that is looking for other words to pay attention to key is a word that is being looked at by other words shouldn't query and word be a vector of size the same as number of input tokens? so that when there is a dot product between query and key the word that is querying can be correctly (positionally) dot product'd with key and get the self attention value for the word?

[@CodeWithAarohi]

The dimensionality of query, key, and value vectors in transformers is a hyperparameter, not directly tied to the number of input tokens. The dot product operation between query and key vectors allows the model to capture relationships and dependencies between tokens, while positional information is often handled separately through positional embeddings.

[@sukritgarg3175]

Great Video ma'am could you please clarify what you said at <a href="#" class="seekto" data-time="1340">22:20</a> once again... I think there was a bit confusion there.

[@AyomideFagoroye-oe2hd]

same here

[@techgirl6451]

hello maa is this transform concept same for transformers in NLP?

[@CodeWithAarohi]

The concept of "transform" in computer vision and "transformers" in natural language processing (NLP) are related but not quite the same.

[@akramsyed3628]

can you please explain <a href="#" class="seekto" data-time="1327">22:07</a> onward

[@mahmudulhassan6857]

maam can you please make one video of classification using multi-head attention with custom dataset

[@CodeWithAarohi]

Will try

[@KavyaDabuli-ei1dr]

Can you please make a video on bert?

[@CodeWithAarohi]

I will try!

[@user-gf7kx8yk9v]

how to get pdfs mam

[@palurikrishnaveni8344]

Could you make a video on image classification for vision transformer, madam ?

[@CodeWithAarohi]

Sure, soon

[@kadapallavineshnithinkumar2473]

Could you explain with python code which would be more practical. Thanks for sharing your knowledge

[@CodeWithAarohi]

Sure, will cover that soon.

[@saeed577]

I thought it's transformers in CV. all explanations were in NLP

[@CodeWithAarohi]

I recommend you to understand this video first and then check this video: kzbin.info/www/bejne/qpy9fp1ni7B3b9U After watching these 2 videos, you will understand properly the concept of transformers used in computer vision. Transformers in CV are based on the idea of transformers in NLP. SO its better for understanding if you learn the way I told you.

[@Red_Black_splay]

Gonna tell my kids this was optimus prime.

[@CodeWithAarohi]

Haha, I love it! Optimus Prime has some serious competition now :)

[@jagatdada2.021]

Use mic, background noise irritate

[@CodeWithAarohi]

Noted! Thanks for the feedback.

[@_Who_u_are]

Speaking in Hindi would be more better

[@CodeWithAarohi]

Sorry for inconvenience

[@niluthonte45]

thank you mam

[@CodeWithAarohi]

Most welcome 😊