Transformer Neural Networks - EXPLAINED! (Attention is all you need)

Please subscribe to keep me alive: kzbin.info...
BLOG: / dataemporium
PLAYLISTS FROM MY CHANNEL
⭕ Reinforcement Learning: • Reinforcement Learning...
Natural Language Processing: • Natural Language Proce...
⭕ Transformers from Scratch: • Natural Language Proce...
⭕ ChatGPT Playlist: • ChatGPT
⭕ Convolutional Neural Networks: • Convolution Neural Net...
⭕ The Math You Should Know : • The Math You Should Know
⭕ Probability Theory for Machine Learning: • Probability Theory for...
⭕ Coding Machine Learning: • Code Machine Learning
MATH COURSES (7 day free trial)
📕 Mathematics for Machine Learning: imp.i384100.net/MathML
📕 Calculus: imp.i384100.net/Calculus
📕 Statistics for Data Science: imp.i384100.net/AdvancedStati...
📕 Bayesian Statistics: imp.i384100.net/BayesianStati...
📕 Linear Algebra: imp.i384100.net/LinearAlgebra
📕 Probability: imp.i384100.net/Probability
OTHER RELATED COURSES (7 day free trial)
📕 ⭐ Deep Learning Specialization: imp.i384100.net/Deep-Learning
📕 Python for Everybody: imp.i384100.net/python
📕 MLOps Course: imp.i384100.net/MLOps
📕 Natural Language Processing (NLP): imp.i384100.net/NLP
📕 Machine Learning in Production: imp.i384100.net/MLProduction
📕 Data Science Specialization: imp.i384100.net/DataScience
📕 Tensorflow: imp.i384100.net/Tensorflow
REFERENCES
[1] The main Paper: arxiv.org/abs/1706.03762
[2] Tensor2Tensor has some code with a tutorial: www.tensorflow.org/tutorials/...
[3] Transformer very intuitively explained - Amazing: jalammar.github.io/illustrated...
[4] Medium Blog on intuitive explanation: / what-is-a-transformer
[5] Pretrained word embeddings: nlp.stanford.edu/projects/glove/
[6] Intuitive explanation of Layer normalization: mlexplained.com/2018/11/30/an...
[7] Paper that gives even better results than transformers (Pervasive Attention): arxiv.org/abs/1808.03867
[8] BERT uses transformers to pretrain neural nets for common NLP tasks. : ai.googleblog.com/2018/11/ope...
[9] Stanford Lecture on RNN: cs231n.stanford.edu/slides/201...
[10] Colah’s Blog: colah.github.io/posts/2015-08...
[11] Wiki for timeseries of events: en.wikipedia.org/wiki/Transfo...)

[@CodeEmporium]

For more details and code on building a translator using a transformer neural network, check out my playlist "Transformers from scratch": kzbin.info/www/bejne/h3Stgnpqedp7ipI

[@ThisNameWasntTaken]

what a hugely underrated video. You did such a better job at explaining this on multiple abstraction layers in such a short video than most videos I could find on the topic which were more than twice as long.

[@CodeEmporium]

Thanks a ton Jeffrey! Means a lot. I've come to realize (fairly recently) that only speaking in jargon isn't going to help. Pealing it down from highly abstract to more technical goes a long way for viewers and myself. I understand more when I break the jargon down. Using this more in future videos

[@ThisNameWasntTaken]

@@CodeEmporium Of course everyone has a different approach to understanding a topic. I recently had to get into a few topics quite quickly and fore me the best way of getting there fast was to start out with very general videos to get a sort of feel for the general ideas and how everything works together on a high level. Then I would watch some more detailed videos or switch to reading more detailed articles until finally reading the actual papers and looking at the formulas and all that stuff. Having an understanding of the bigger picture helped me comprehend the details better. I Also think no explanation can ever be "too simple" cause sometimes when explanations try to save time by glossing over parts or taking things for granted you spend way more time rewinding trying to wrap your head about some small detail just because you might be missing some needed knowledge. I think in an explanation it's like with spices on food. better keep it simple and easy. Individuals can always skip parts for themselves. same with the spices: Better not add too much thinking everyone will be able to take it, rather add a little and if it's not enough for someone they can add it themselves.

[@bhargavyagnik]

So true man !! i scraped the net to find a simple explanation !! you are a genius :)

[@joosthorskamp1736]

True, I did not understand the use of attention until watching this video

[@sindhuorigins]

@@CodeEmporium Great approach that you've taken. A high level understanding followed by deeper understanding of the topic pretty much clears up the concept. Subscribed.

[@ShotReverseShot]

2:03 I died at the Vsauce reference. Well played.

[@kekmeister42]

Me too, I had to scroll back around 30s because he just continued explaining and I completely lost focus :D

[@ahmedshahriarsakib5728]

:v

[@PhilbertLin]

Great video! Watched it a few times already so these timestamps will help me out: 0:00 Problems with RNNs and LSTMs 3:34 First pass overview over transformer architecture 8:10 Second pass with more detail 10:34 Third pass focusing on attention networks and normalization 11:57 Other resources (code & blog posts)

[@CodeEmporium]

Thanks for this! It'll help others watching too.

[@akkipant]

@@CodeEmporium Pin this comment.

[@mattcoakes5682]

Thank you so much! I planned to watch this a few times for reference as I delve into transformer code. This will be very useful.

[@Elanus19]

Incredibly well explained and concise. I can't believe you pulled off such a complete explanation in just 13 minutes!

[@CodeEmporium]

Thank you for the kind words. Super glad you liked it :)

[@danbochman]

Wow. I've seen lectures that are 45m+ long trying to understand this architecture, even lectures from the original authors. Your video was hands-down the best, really helped me piece some key missing intuition pieces together. You have a gift for teaching and explaining -- I wholeheartedly hope you're able to leverage that in your professional career!

[@gearoidmurphy4988]

The multi-pass approach to progressively explaining the internals worked well. Thanks for your content!

[@ajcosta]

The understanding converges!

[@himeshph]

@2:02 that vsauce thing was cool

[@ShenalPerera]

Ikr 😅

[@blakef.8566]

...or was it?

[@aryamaanjena1706]

Love that so many people got the vsauce ref

[@ozgurakpinar_gr]

I came here to say that. Excelsior

[@lingding77]

I love the multi-pass way of explanation so that the viewer can process high level concepts and then build upon that knowledge, great job.

[@HIMANSHUKUMARSINHA7]

I bought udemy course for Transformer and BERT but with no help and wasted my time, money and energy. This video and your BERT video made my day. thanks. I may explain in my interview well. :)

[@MrKfirlevi]

Great video!! I am taking a course in my university and one of the lectures was about RNNs and transformers. Your video of 13 mins explains way better than the 100 mins lecture i attended. Thank you!

[@seungjungjin9217]

Great video! I love how you go through multiple passes, each getting into deeper specifics!

[@frederikbrammer]

This is by far the best explanation of the Transformers architecture that I have ever seen! Thanks a lot!

[@vahekassardjian5032]

Outstanding explanations: to the point and well illustrated. Thank you.

[@aram69420]

I don't know how many attention video I have watched. But I swear I'm getting there. Repetition is the key to understanding these shit

[@lonewolf2547]

I cant believe you explained so complicated things in crystal clear format. Excellent job dude

[@PRUTHVIRAJRGEEB]

This is heavily underrated!! Such an awesome video! Thanks Man!

[@vtrandal]

With videos like this one you should be having 100,000+ subscribers soon. Adding a bit of humor to uncompromising technical content is a very good way to go.

[@cPho3nix]

That was the best way of explaining thins in my opinion. Start big picture, getting more detailed over time.

[@newginsam670]

Bro TBH no words to appreciate such a well structured video in a short time and the explanation was easly understandable even for people with less knowledge. Thanks for the video man.

[@KrazeeKrab]

This was a phenomenal video. You managed to explain transformers in 13 minutes better than my professor could in three hours. Thank you and keep on creating content!

[@rodrigoklosowski8219]

Amazing! The best explanation on youtube about Transformer Neural networks, the matrices visual representations helps a lot!

[@adwaitpatil8300]

One of the cleanest explanation for transformers without dabbling too much into the theory!! Thanks man

[@moneyinahurry]

One of the best or probably the best explanation I've seen. Thank you very much for the effort.

[@logicloudy2851]

Thanks, man. This is a really clear and high-level explanation. Really helpful for some guys like me who just stepped into this area. I read many explanations online. They give tons of details but fail in explaining these abstract items. These explanations always use other abstract definitions to explain this one. This problem happens again in the explanation of the "other abstract item". Sometimes I just forgot originally what I want to understand. Or even worse, they form a circulation... Thank you so much! This video helped me a lot in understanding the paper

[@snehashishpaul2740]

I had to make 2 passes of your video to fully understand and appreciate the underlying mathematics and working of the model. You have put a great effort in making it simpler to understand with illustration and animation.

[@jonathanburrell7055]

This is awesome!!! Thank you for breaking it down concisely, understandably, and deeply! It’s hard to find explanations that aren’t so simplistic they’re useless, or so involved they don’t save time in achieving understanding. Thank you!!

[@CodeEmporium]

My pleasure! If you are into building the transformer piece by piece from scratch, I suggest checking out the “Transformers from scratch” playlist.

[@Ashwin436]

This was very helpful! You broke it down into such simpler concepts. I'm sure I'll be needing you again. Please keep at it. Thanks!

[@ScriptureFirst]

KEEP IT UP! Please! This is outstanding :) love the diagrams, dry crisp active speaking, & overview technique.

[@simonevagnoni1758]

Your way to breaking down step by step is very effective! Congrats and thanks. School systems should use it more

[@paragrk1]

Went through several videos on 'Attention is all you need' paper before this, all the details you managed to cover in thirteen minutes is amazing. Could not find explanation that is so easy to understand anywhere else. Great job!

[@enjakuro7048]

right? I couldn't believe this video is only 13 minutes! That's a very good talent to have.

[@klam77]

BEAUTIFUL......deep, concise, pithy, each word is meaningful.....well done.

[@eashwaraerahan861]

Great work. I really like the info graphics. I’m a person with no background on NLP but I was still able to follow till the second pass, thanks to your great work.

[@TheJonathanLugo]

Wow, I am so glad I found your channel. The concept is clearly explained and assumes an intelligent audience. Well done!

[@utsabkhakurel9742]

Simple and easy to understand. Great job!

[@imagiro1]

Thank you very much for you effort! You just adjusted my attention layer, the pieces start to fall in place and I have a much better understanding of why TNNs are so revolutionary.

[@mattcoakes5682]

LOVE the multipass strategy for explaining the architecture. I don't think I've seen this approach used with ML, and it's a shame as this is an incredibly useful strategy for people like me trying to play catch up. I hopped on the ML train a little late, but stuff like this makes me feel not nearly as lost.

[@dannysuarez6265]

How is it possible such a beautiful video don't have more views/likes? Thank you CodeEmporium:)

[@gajeshladhar3777]

it is really straight forward video which talks exactly on what we need to learn !!

[@navinahmed]

Didnt know what a transformer hype was until I landed on this video. Thanks a lot ! Subscribed. Gotta check more content on this channel now

[@ApprovingSeal]

Finally a basic explanation I can understand. I tried reading the original "Attention is all you need" paper, but it felt like it was assuming I was already familiar with the basics of NLP, like the encoder-decoder setup. Which I wasn't.

[@yahyasowti9711]

Great job in explaining such a complicated concept!

[@Sidnv]

Really great video. As someone transitioning from pure math into machine learning and AI, I find the language barrier to be the biggest hurdle and you broke down these concepts in a really clear way. I love the multiple layer approach you took to this video, I think it worked really well to first give a big picture overview of the architecture before delving deeper.

[@CodeEmporium]

Super happy this helped!

[@PasseScience]

Question2: I do not see well what we should "repeat/extend" when we varie the input length in a transformer. (by exemple with a sequence to vector, it's easy, ou have one recurrent bloc per element you add in your sequence, and you plug it to the next recurrent bloc, but in transformer, from the schema, I do not see what should be repeated, what should vary whith different input size. Do you have a figure showing that?)

[@d63810728]

This is by far the most comprehensive yet short video i haave seen

[@tictacX1]

Good job CodeEmporium! Very well made overview. thanks.

[@ryanhewitt9902]

Thank you for making this! As a curious outsider I have been anxious about falling behind in recent years and this was perfect to bring me up to speed - at least enough to follow the conversation.

[@CodeEmporium]

You are very welcome! Thanks for watching! :)

[@Kasper-ev3zh]

Thank you so much for uploading this, appreciate the work you put into making it! Your explanation really helped my understanding, and much more so than a lot of other videos covering the subject.

[@CodeEmporium]

Absolutely! Thanks for taking the be time to watch. Will be making more videos on the subject. So stay tuned

[@farnazfaramarzi4137]

Super informative, thorough explanation! Thanks!

[@derrxb]

This is one of the best explanations for transformers I've come across online! Awesome job, man! Thanks. I'll totally recommend your channel to some classmates!! :)

[@CodeEmporium]

Thanks! And Glad it's helpful! Spreading the word of my channel is the best thing you can do to help :)

[@Themojii]

Very well, clearly explained the concepts, and nice visualization video. I spent a couple of hours and read multiple blog/tutorial about transformers, but I learned a lot more from your 13 mins video compared to those tutorials. Great job. I subscribed to your channel after watching this. Keep up the good work

[@theneilpowers]

This earned a subscription! Excellent explanation!

[@shaina2231]

very good presentation indeed!! explains even complex concepts in very simple and easy way, well done

[@paramveersingh2919]

Watched Andrew Ng, watched this, you got me to stick through the video and Andrew who i consider one of the best in this field did not manage to express as clearly as you did! Cheers man, amazing video!

[@emeralde3761]

Just want to say this video is amazing. Watched like three other 30+ mins videos but they all failed to train my stupid brain. This 13 minutes video is intuitive, detailed, and beginner-friendly. Thank you :3

[@gurudevilangovan]

Wow. Amazing video. Better than anything I've watched on the topic, all in thirteen minutes.

[@ahmedelayek2110]

what a guide for the transformer in just 13 min. thanks a lot for this simplicity.

[@yonistoller1]

This is one of the best tutorial videos I've watched on any subject, thank you!

[@CodeEmporium]

Glad to hear that. Thanks so much !

[@gokuson6399]

Best explanation so far! Keep up the good work!

[@piyalikarmakar5979]

What an explanation!!! You have made such complex concepts so easy..Thank you so much...

[@lisandrocesaratto3012]

Best video on Transformers I have seen so far! The examples really help to understand how the architecture works. Subscribed.

[@CodeEmporium]

Thank you for watching!

[@pipe_runner_lab]

I saw Yanik's explaination and now I saw yours. Yanik does a terrible job at explaining papers, he usually just jokes around. Your explanation is probably one of the best I have seen so far. Thanks man.

[@bauwndule]

Best explanation ever. I have an interview with Microsoft tomorrow. This was the best brushing up I could get.

[@mikashaw7926]

this is literally the best explanation ever, thank you!

[@Fruchtkotzekiddy]

this video was one of the best learning videos i EVER SAW first you give a high level overview, then u step in deeper every step with an understandable example THANK YOU SO MUCH!!!

[@CodeEmporium]

You are very welcome. Thank you for the compliments :)

[@ritwikdubey5331]

I was searching for this particular explanation from a long tym! thanks for this!

[@ayushkant392]

This channel is so under-rated. Amazing video

[@VaibhavPatil-rx7pc]

TOP OF TOP clear explanation you provided !!!

[@shipan5940]

By far, the MOST comprehensible explaination on Transformer available in the whole internet space.

[@shipan5940]

You deserve 1M subscribers at least.

[@CodeEmporium]

Thank you for the kind words! Maybe one day

[@karakadir8860]

dude you absolutely deserve each and any subsriber. thank you very much for your highly helpful and quality content.

[@CodeEmporium]

Thanks so much for the lovely comment! And also for subscribing! More to come!

[@leoisikdogan]

Very well explained! Great video, as always.

[@fahdciwan8709]

Thanks a lot man!! i know u explained is clearly but it'll still take me a few more times to watch the video and digest the concept

[@cristianarteaga]

What a great explanation! Thanks a lot and please keep creating material like this. It's such a great help.

[@sairamsubramaniam8316]

This is the best explanation! I came in search for transformers but I found Gold.

[@QuanNguyen-vo2xh]

This video deserves more views. It's awesome. Good job buddy.

[@arshiavashisht3481]

Had to watch it like 2-3 times to get the entire thing, but worth it. Somehow so concise, yet every relevant detail is included.

[@CodeEmporium]

Thanks a lot for watching and commenting! Super happy this video gave you value

[@wangy01]

I watched this video four times. After each time, I feel I understand this topic better than the previous one.

[@prateek4546]

Wow, the best explaination on youtube ! Had to subscribe after watching !

[@praveenrajagopal9161]

Brilliant piece of work! You nailed it in few mins!

[@CodeEmporium]

Thanks for watching!

[@rexma9625]

Good job on explaining the transformer! Thanks!

[@jasminelee1720]

Really great video. I have taken so many classes that have discussed this model and never understood it until now. Thanks!

[@CodeEmporium]

Perfect! Glad this helped!

[@anishpratheepkumar4184]

Nice work man, that's a pretty nice explanation of transformers. Thanks!

[@rashedulhasanrijul5506]

Very simple explanation. Thank you

[@hareshwedanayake7427]

Great video. Definitely struggled to understand this concept but I think I understand it a bit now. Will definitely have to read up on this

[@srinivasanthiruvengadathan2177]

Excellent Video. Clear and easy to understand. Thanks

[@abhishekswain2502]

thank you, bro! this a very well made and precise video. Helped me a lot !

[@tvanpeer]

Great video! I love the layered approach for explaining the concepts. Very well done. Thank you!

[@CodeEmporium]

I am super glad that approach helped

[@Rishabh199494]

Amazing way of explaining complex things in simple terms.

[@CodeEmporium]

Thanks !

[@guygirineza4001]

Your explanations are just superb !! Thank you very much, you earned a new subscriber !

[@langlansrobert8599]

The easiest understanding video about this topic, as I can see! Thank you

[@CodeEmporium]

You are very welcome :)

[@gabrielcournelle3055]

That was an awesome explanation. I have a question about the Add & Norm block. Do you add the embedded vector before or after performing normalization ? Is there even a difference if we do one instead of the other ?

[@user-uc5xc5sb9z]

Very helpful and insightful, nice job!

[@danielpwagner]

Outstanding. Best explanation of transformers that I’ve seen by far.

[@CodeEmporium]

Thanks a ton for watching :)

[@k.sladkina872]

Wow!! Thank you for this video. It really makes things clear.

[@sshubam]

Wow ! Truly magnificient video ! Really understood transformers very well. Thank you so much. Please keep making these videos. You are very good at it. Thanks again.

[@TusharKale9]

Superb explanation in 13 minutes. I have been watching videos over 1 hour long to get this concept. Well done and keep it up. Regards

[@usamahussain4461]

Man! Brilliant video. I saw a 27 mins video and was totally spent out and didn't even understand much. But this was just awesome, and in half the time! The only thing lacking might be the examples of keys, values and queries but i mostly got the hang of it.

[@poulamisinhamahapatra8104]

Lucid explanation for such a complex network. Great job!

[@CodeEmporium]

Thanks for watching :)

[@ishwarinalgirkar1193]

great job putting attention in simple words. Very intuitive!

[@CodeEmporium]

Thanks so much!

[@NoOne-sy5fg]

Great video bro! You're underrated af. One of the best if not the best explanation of some neural network architecture. Keep up the good work. Kudos!

[@CodeEmporium]

Thanks so much! I am making more related videos. So do check ‘em out :)