Visual Guide to Transformer Neural Networks - (Episode 2) Multi-Head & Self-Attention

[@HeduAI]

*CORRECTIONS* A big shoutout to the following awesome viewers for these 2 corrections: 1. @Henry Wang and @Holger Urbanek - At (10:28), "dk" is actually the hidden dimension of the Key matrix and not the sequence length. In the original paper (Attention is all you need), it is taken to be 512. 2. @JU PING NG - The result of concatenation at (14:58) is supposed to be 7 x 9 instead of 21 x 3 (that is to so that the concatenation of z matrices happens horizontally and not vertically). With this we can apply a nn.Linear(9, 5) to get the final 7 x 5 shape. Here are the timestamps associated with the concepts covered in this video: 0:00 - Recaps of Part 0 and 1 0:56 - Difference between Simple and Self-Attention 3:11 - Multi-Head Attention Layer - Query, Key and Value matrices 11:44 - Intuition for Multi-Head Attention Layer with Examples

[@amortalbeing]

Where's the first video?

[@HeduAI]

​@@amortalbeing Episode 0 can be found here - kzbin.info/www/bejne/amnKc4OCbMZ7gLs

[@amortalbeing]

@@HeduAI thanks a lot really appreciate it:)

[@omkiranmalepati1645]

Awesome...So dk value is 3?

[@jasonwheeler2986]

@@omkiranmalepati1645 d_k = embedding dimensions // number of heads

[@thegigasurgeon]

Need to say this out loud, I saw Yannic Kilcher's video, read tonnes of materials on internet, went through atleast 7 playlists, and this is the first time I really understood the inner mechanism of Q, K and V vectors in transformers. You did a great job here

[@HeduAI]

This made my day :,)

[@afsalmuhammed4239]

True

[@exciton007]

Very intuitive explanation!

[@EducationPersonal]

Totally agree with this comment

[@VitorMach]

Yes, no other video actually explains what the actual input for these are

[@nitroknocker14]

All 3 parts have been the best presentation I've ever seen of Transformers. Your step-by-step visualizations have filled in so many gaps left by other videos and blog posts. Thank you very much for creating this series.

[@HeduAI]

This comment made my day :,) Thanks!

[@bryanbaek75]

Me, too!

[@lessw2020]

Definitely agree. These videos really crystallize a lot of knowledge, thanks for making this series!

[@Charmente2014]

ش

[@ML-ok9nf]

Absolutely underrated, hands down one of the best explanations I've found on the internet

[@nurjafri]

Damn. This is exactly what a developer coming from other backgrounds need. Simple analogies for a rapid understanding. Thanks a ton. Keep uploadinggggggggggg plss

[@Xeneon341]

Agreed, very well done. You do a very good job of explaining difficult concepts to a non-industry developer (fyi I'm an accountant) without assuming a lot of prior knowledge. I look forward to your next video on masked decoders!!!

[@HeduAI]

@@Xeneon341 Oh nice! Glad you enjoyed these videos! :)

[@wireghost897]

Finally a video on transformers that actually makes sense. Not a single lecture video from any of the reputed universities managed to cover the topic with such brilliant clarity.

[@devchoudhary8892]

best, best best explanation on transformer, you are adding so much value to the world.

[@rohtashbeniwal9202]

this channel needs more love (the way she explains is out of the box). I can say this because I have 4 years of experience in data science, she did a lot of hard work to get so much clarity in concepts (love from India)

[@HeduAI]

Thank you Rohtash! You made my day! :) धन्यवाद

[@alankarmisra]

3 days, 16 different videos, and your video "just made sense". You just earned a subscriber and a life-long well-wisher.

[@nizamphoenix]

Being a professional in this field for ~5years can say this is by far the best explanation of attention. Amused as to why this doesn't pop up on YT's recommendation for attention at the top. Probably, YT's attention needs some attention to fix its Q, K, Vs

[@HeduAI]

You made my day :)

[@malekkamoua5968]

I've been stuck for so long trying to get the Transformer Neural Networks and this is by far the best explanation ! The examples are so fun making it easier to comprehend. Thank you so much for you effort !

[@HeduAI]

Cheers!

[@HuyLe-nn5ft]

The important detail that set you apart from the other videos and websites is that not only did you provide the model's architecture with numerous formulas but you also demonstrated them in vectors and matrixes, successfully walked us through each complicated and trivial concept. You really did a good job!

[@EducationPersonal]

This is one of the best Transformer videos on KZbin. I hope KZbin always recommends this Value (V), aka video, as a first Key (K), aka Video Title, when someone uses the Query (Q) as "Transformer"!! 😄

[@HeduAI]

😄

[@ja100o]

I'm currently reading a book about transformers and was scratching my head over the reason for the multi-headed attention architecture. Thank you so much for the clearest explanation yet that finally gave me this satisfying 💡-moment

[@rohanvaidya3238]

Best explanation ever on Transformers !!!

[@adscript4713]

As someone NOT in the field reading the Attention paper, after having watched DOZENS of videos on the topic this is the FIRST explanation that laid it out in an intuitive manner without leaving anything out. I don't know your background, but you are definitely a great teacher. Thank you.

[@HeduAI]

So glad to hear this :)

[@MGMG-li6lt]

Finally! You delivered me from long nights of searching for good explanations about transformers! It was awesome! I can't wait to see the part 3 and beyond!

[@HeduAI]

Thanks for this great feedback!

[@HeduAI]

“Part 3 - Decoder’s Masked Attention” is out. Thanks for the wait. Enjoy! Cheers! :D kzbin.info/www/bejne/nXucnJSAqKpqbs0

[@chaitanyachhibba255]

Were you the one who wrote transformers in the fist place, because no one explained it like you did. This is undoubtfully the best info I have seen. I hope you please keep posting more videos. Thanks a lot.

[@HeduAI]

This comment made my day! :) Thank you.

[@shubheshswain5480]

I went through many videos from Coursera, youtube, and some online blogs but none explained so clear about the Query, key, and values. You made my day.

[@HeduAI]

Glad to hear this Shubhesh :)

[@sebastiangarciaacosta5468]

The best explanation I've ever seen of such a powerful architecture. I'm glad of having found this Joy after searching for positional encoding details while implementing a Transformer from scratch today. Valar Morghulis!

[@HeduAI]

Valar Dohaeris my friend ;)

[@andrewbrowncopper]

I'm a grad student currently applying NLP - this is literally the best explanation of self-attention I have ever seen. Thank you so much for a great vid!

[@Srednicki123]

I just repeat what everybody else said: these videos are the best! thank you for the effort

[@Clammer999]

I’ve gone through dozens of videos on transformers and the multi-head attention is one of the most complex mechanisms that require not only a step-by-step explanation, but be accompanied with a step-by-step animation, which many videos tend to skip over but this video really nails it. Thanks so much!

[@Abhi-qf7np]

You are the best😄😄, This is THE Best explanation I have ever seen on KZbin for Transformer Model, Thank you so much for this video.

[@sujithkumar5415]

This is quite literally the best attention mechanism video out there guys

[@rayxi5334]

Better than the best Berkeley professor! Amazing!

[@giridharnr6742]

Its one of the best explainations of Transformers. Just mind blowing.

[@freaknextdoor9040]

Hands down, this series is the best one explaining the essence of transformers I have found online!! Thanks a lot, you are awesome!!!!

[@HeduAI]

Cheers! 🙌

[@darkcrafteur165]

Never posting but right now I need to thank you, I really don't believe that it exists a better way to understand self attention than watching your video. Thank you !

[@frankietank8019]

Hands down the best video on transformers I have seen! Thank you for taking your time to make this video.

[@wayneqwele8847]

Thank you for taking the time explain from a linear algebra perspective what actually happens. Many teachers on youtube are comfortable just leaving it at math symbols and labels. Showing what actually happens to matrice values has sharpened my intuition of what actually happens under the hood. Thank you.🙏

[@MCMelonslice]

This is the best resource for an intuitive understanding of transformers. I will without a doubt point everyone towards your video series. Thank you so much!

[@jasonpeloquin9950]

Hands down the best explanation of the use of Query, Key and Value matrices. Great video with an easy example to understand.

[@jboyce007]

If only I saw your videos earlier. As everyone in the comments says, these are THE BEST videos on the subject matter found anywhere! Thank you so very much for helping us all!

[@HeduAI]

Cheers! :)

[@ghostvillage1]

Hands down the best series I've found on the web about transformers. Thank you

[@mrkshsbwiwow3734]

This is the best explanation of transformers on KZbin.

[@naveenpoliasetty954]

This video is underated, I have saw a bunch of videos in transformers but they you related cosine similarity and self-attention no one else did that. Thank you so much ....

[@forresthu6204]

Self-attention is a villain that has struck me for a long time. Your presentation has helped me to better understand this genius idea.

[@karannesh7700]

Really underrated VIDEO ! Thanks! Sister! What a great work!🙏🙏🙏🙏🙏

[@rishidixit7939]

Extremely Amazing, after scouring through videos and articles finally found a gem. Great Understanding

[@abdot604]

brilliant explanation , your chanel deserve way more ATTENTION.

[@tokareuv]

Bloody hell, that was so much helpful! Saw the multi-head attention on the coursera course, but it was so poorly explained that I had to look for other references. Now I've got more useful information in this video than on the entire course I took. Nice!

[@jamesshady5483]

This explanation is incredible and better than 99% of what I found on the Internet. Thank you!

[@laodrofotic7713]

This video is amazing, finally someone intelligent that can actually teach things! I believe this creator is truly talented and fully understands the concept, unlike many others. She also not only understand but is very good at communicating the information in a clear and intuitive way, which I haven't seen before. I have only watch this video up to 12:33 at the time of this comment but I am truly impressed, will watch the others. Thank you Hedu for sharing this.

[@bendarodes61]

I've watched many video series about transformers, this is by far the best.

[@rishiraj8225]

Coming back after a year, just to revise the basic concepts. It is still the best video on YT. Thanks Hedu AI

[@persianform]

The best explanation of attention models on the earth!

[@madhusharath]

Wow! Truly wow! Your ability to explain complex stuff in layman terms + reference to well known series/anime shows how in-depth your understanding actually is!

[@HeduAI]

Your comment made my day :)

[@humayounkhan7946]

That is the best explanation I've seen as a beginner in this field(I have been reading tons of articles and went through a lot of videos). Thanks a lot for your hard work. Because of my renewed understanding thanks to this video, I can now better explain this concept to others by borrowing your analogy, I hope you know that your video continues to bring compounding value to society even after 2 years of posting them. Thanks a lot Hedu!

[@HeduAI]

Thank you so much! Loved reading your comment.

[@charlesgormley9075]

I agree, this is great for starting out with transformers

[@madhu1987ful]

Wow. Just wow !! This video needs to be in the top most position when searched for content on transformers and their explanation

[@HeduAI]

So glad to see this feedback! :)

[@MikeAirforce111]

My goodness, you have talent as a teacher!! :-) This builds a very good intuition about what is going on. Very impressed. Subscribed!

[@nicholasabad8361]

By fair the best explanation of Multi-Head Attention I've ever seen on KZbin! Thanks!

[@HeduAI]

Glad to hear this :)

[@danielarul2382]

One of the best explanations on Attention in my opinion.

[@jackziad]

Your videos are so good at getting complex ideas across in an intuited way. You are like the 3Blue1Brown equivalent for AI. Keep it up and keep producing high-quality video content, at your own pace of course 😋

[@HeduAI]

3Blue1Brown is one of my favorite channels! Therefore, you comparing these videos to that channel is one of the best compliments ever. Thank you! :)

[@rishiraj8225]

@@HeduAI yes.. this is awesome explanation comparable to 3Blue1Brown.. make more..

[@chenlim2165]

Bravo! After watching dozens of other explainer videos, I can finally grasp the reason for multi-headed attention. Excellent video. Please make more!

[@endgameisthejoke]

Absolutely beautifully and accurate guide on Multi-head Attention

[@jirasakburanathawornsom1911]

Hand down the best transformer explanation. Thank you very much!

[@krishnakumarprathipati7186]

The MOST MOST MOST MOST ..........................useful and THE BEST video ever on Multi head attention........Thanks a lot for your work

[@HeduAI]

So glad you liked it! :)

[@vanhell966]

Amazing work. Really appreciate you, making complex topics into simple language with the touch of anime and series. Amazing.

[@kennethm.4998]

You have a gift for explanations... Best I've seen anywhere online. Superb.

[@mysterybox109]

I streamed dozens of video for this paper and you had the best explanation so far. It answer my every little question. For instance, I was so confused what queries, keys and value vectors are. You simply answered in a line saying that those are the same. Thanks a ton !!!!!!!

[@HeduAI]

You are very welcome Hitarth! :)

[@mysterybox109]

@@HeduAI can you please also explain "An Image is Worth 16x16 Words" paper?

[@fernandonoronha5035]

I don't have words to describe how much these videos saved me, thank you!

[@artukikemty]

Thanks for posting, by far this is the most didactic Transformer presentation I've ever seen. AMAZING!

[@davidlazaro3143]

This video is GOLD, it should be everywere! Thank you so much for doing such an amazing job 😍😍

[@andrerocha155]

Congrats! Fantastic explanation. You really used your Transformer skills to "pay attention" to the most important issues in this matter and explained it as simply as needed to be understood even by a 10 years old child!

[@marcosmartinez9241]

These are the best serie of videos where I finally can find a good explanation about the Transformer network. Thanks a lot!!

[@HeduAI]

Cheers! 🙌

[@EngineeredFemale]

Most underrated video about transformers. Going to recommend this to everyone. Thankyou

[@陳建文-k5x]

The best video I've ever seen for explaining transformer.

[@skramturbo8499]

I really like the fact that you ask questions within the video. In fact those are the same questions one has and first reading about transformers. Keep up the awesome work!

[@shivam6565]

Finally I understood the concept of query, key and value. Thank you.

[@adityaghosh8601]

Blown away by your explanation . You are a great teacher.

[@RafidAslam]

Thank you so much! This is by far the clearest explanation that I've ever seen on this topic

[@jackderrida]

Holy crap, this tutorial is good! I've had GPT-4 generate me so many analogies to refresh my understanding of the same concepts you perfectly explain here.

[@an_experienced_guy]

This is by far the best and the most clear and insightful explanation of transformers, I've tried to understand it through multiple blogs and videos and stack exchange answers, this is the first time every component became clear to me and how they all work in conjunction. Thanks a lot for this series. Amazing explanations.

[@geetanshkalra8340]

This is by far the best video to understand Attention Networks. Awesome work !!

[@jeremyhofmann7034]

I’ve watched dozens of these and read as many articles and none have been able to explain in detail what self-attention is doing as well as this one. Finally I get it! Great work.

[@HeduAI]

I feel so glad upon reading your comment! :) Mission served.

[@kafaayari]

I won't say this is the best explanation so far, but this is the only explanation. Others are just repeating the original paper.

[@mrmuffyman]

You are awesome!! I watched Yannic Kilcher's video first and was still confused by the paper, probably because there's so much detail skipped over in the paper and Kilcher's video. However, your video goes much slower and in depth so the explanations were simple to understand, and the whole picture makes sense now. Thank you!

[@adithyakaravadi8170]

You are so good, thank you for breaking down a seemingly scary topic for all of us.The original paper requires lot of background to understand clearly, and not all have it. I personally felt lost. Such videos help a lot!

[@DeepakSadulla]

The KZbin example helps a lot with understanding QKVs. Really good explanation... Thanks!!

[@hubertkanyamahanga2782]

I am just speechless, this is unbelievable! Bravo!

[@weihongchin8140]

I seldom post comments to youtube channel. By far the best ever video of transformer explanation! Please keep going release more videos!

[@ClaudiaAcquistapace]

Totally in love with your explanations.. You are the light at the end of my personal tunnel trying to understand transformers in preparation of the lecture I have to give on this topic. I will mention your videos all the way through my lecture. Thanks so much for explaining it so clearly.

[@sowmendas812]

This is literally the best explanation for self-attention I have seen anywhere! Really loved the videos!

[@cihankatar7310]

This is the best explanation of transformers architecture with a lot of basic analogy ! Thanks a lot!

[@jikin91]

I've read and watched tonnes of materials to understand the role of Q,K,V in transformers.. this is hands down the BEST I have seen. Also, this is the first time i placed any comment on any KZbin video in my 15 years of watching. Your video made me do this! Please produce more awesome content👏

[@HeduAI]

You truly made my day! :) Thank you!

[@JDechnics]

Holy shit was this a good explanation! Other blogs literally copy what the paper states (which is kinda confusing), but you explained it in such a intuitive and fun way! Thats what I called talent!!

[@oliverhu1025]

Probably the best explanation of transformers I’ve found online. Read the paper, watched Yannic’s video, some paper reading videos and a few others, the intuition is still missing. This connects the dots, keep up the great work!

[@pythondev2631]

The best video on multihead attention by far!

[@andybrice2711]

This really is an excellent explanation. I had some sense that self-attention layers acted like a table of relationships between tokens, but only now do I have more sense of how the Query, Key, and Value mechanism actually works.

[@aritamrayul4307]

Ohh why I get to know this channel now . This channel is criminally underrated!!

[@prashantchauhan1742]

This is Gold. I was confused after going through the paper. And boom this cleared it..

[@HeduAI]

Cheers! :D

[@minruihu]

it is impressive, you explain so complicated topics in a vivid and easy way!!!

[@ariasardari8588]

Your ability to convey concepts is quite impressive! Probably the best tutorial video I've ever seen. From now on, every time I open KZbin, I first check if you have a new video It was fantastic! I greatly appreciate it.

[@HeduAI]

Thanks a lot Aria! Really means a lot :)

[@bochengxiao1352]

Thank you so much! It's the best Transformer video ever! Really hope more on other models.

[@HeduAI]

Glad to hear that! :) Do let me know if there are certain models that you would like to see covered in future videos.

[@cw9249]

you are amazing. ive watched other videos and read materials but nothing compares to your videos

[@pedroviniciuspereirajunho7244]

To visualize the matrices helped me to understand better transformers. Again, thank you very much!

[@MANUSHEORANBEE]

The best explanation for this topic. So much detail for the most crucial part, just loved that. And also liked your concept of referring to web-series/movies/memes while explaining the concept. It sort of balances the knowledge and entrainment part. Great job, waiting for 3rd part!!!!

[@HeduAI]

Thanks for this awesome feedback!

[@HeduAI]

“Part 3 - Decoder’s Masked Attention” is out. Thanks for the wait. Enjoy! Cheers! :D kzbin.info/www/bejne/nXucnJSAqKpqbs0

[@bhavyaghai1924]

Educational + Entertaining. Nice examples and figures. Loved it!

[@wolfie6175]

This is an absolute gem of a video.