What are Transformer Models and how do they work?

[@zafersahinoglu5913]

Luis Serrano, this set of 3 videos to explain how LLMs and transformers works is truly the best explanation available. Appreciate your contribution to the literature.

[@anipacify1163]

Best playlist on transformers and attention. Period . There is nothing better on KZbin. Goated playlist. Thank you soo much !

[@tonym4953]

This is what you get when the teacher is concerned with actually imparting knowledge and learning as opposed to showing off how sophisticated their own knowledge is. I am incredibly appreciative of his humility and passion. An example to follow in my own life.

[@RuliManurung]

What an awesome series of lectures. I spent 10 years teaching undergraduate-level Artificial Intelligence and NLP courses, so I can really appreciate the skill in breaking down and demystifying these concepts. Great job! I would say the only thing missing from these videos is that you don't really cover how the learning/training process works in detail, but presumably that would detract from the focus of these videos, and you cover it elsewhere.

[@BunnyLaden]

Thank you Luis! This series is THE BEST explanation I've seen on attention and transformers. Your gift of clear explanation is truly a gift to those of us seeking knowledge.

[@anupamjain345]

Thanks! Never came across anyone explaining anything in such a great detail, you are amazing !!!

[@SerranoAcademy]

@anupamjain345, thank you so much for your really kind contribution, and for your nice words!

[@YannickBurky]

This video is incredible. I've been looking for material to help me understand this mess for quite some time now, but everything about this video is perfect: the tone, the speed of speech, the explanations, the hierarchy of knowledge... I'm screaming with joy!

[@lightninghell4]

I've seen many videos on transformers but this series os the first where I understood the topic at a deep enough level to appreciate it.

[@rudyfigaro1861]

Luis has a talent for breaking down complex problems into simple steps and then build the whole thing back so that ordinary people can understand.

[@Aleks-ng3pp]

In the previous video you said you would explain how to compute Q, K, and V matrices in this one. But I don't see.

[@Glomly]

This is the BEST explanation ever you can find on the internet. I'm serious

[@analagunapradas3840]

Agree, definitly the BEST, as always ;) Luis Serrano

[@SourjyaMukherjee]

Hard to believe that someone came up with better content on transformers than 3B1B

[@matin2021]

All my friends and I have watched every single one of your videos Great content, appropriate sentences and... really great Keep making great videos 🙌

[@nc2581]

Thank you for this fantastic video series on Transformers! The first two videos were particularly enlightening. I'm fascinated by how the query, key, and value vectors evolve before each attention module. It would be wonderful to gain a deeper understanding of the encoder-decoder architecture, particularly why the first attention belongs to the encoder while the subsequent ones are part of the decoder. Also, I'm intrigued by the visualization of linear transformations at each step during training, especially when outputs are recycled back into the decoder. Eagerly awaiting more insights!

[@jazznomad]

Very clear. You are a natural teacher

[@sandhiyar8763]

Absolutely worth the watch! The clarity in Luis's explanation truly reflects his solid grasp on the content.👍

[@MohamedHassan-pv1xl]

I feel super lucky to come across your videos, normally I don't comment, but I saw the 3 videos of the series and I'm amazed on how you explain complicated topics. You're efforts are highly appreciated.

[@zihaoli962]

such a wonderful video! it lays a quite good foundation and during watching the 3 videos a lot of questions occur to my mind e.g., 1) number-of-heads vs. accuracy/computation/space, 2) what's the trainable size of QKV, 3) for the whole framework what are the data shapes of all intermediate steps, and how do we perform back-propagation and so on. then after watching the video, you could proactively go search n find those answers then your level of understanding could be further boosted! that's the good thing about this video - it takes decent amount of effort to understand while it does not act like a show-stopper (I'd say the paper "attention is all you need"). it starts you off and you could pick up more along the learning journey. a big thx to Serrano!

[@narasimmanramiah3062]

The set of 3 videos is excellent. I could understand the transformer architecture very well with visuals. Kudos to Luis Serrano !!

[@alidanish6303]

Finally the 3rd video of the series and as usual with the same clarity of concepts as expected from Serrano. The way you have perceived these esoteric concepts have produced pure gold. I am following you and jay from Udacity and you guys have made real contribution in explaining a lot of black magic. Any plans to update grokking series...?

[@jaeen7665]

This is a fantastic explanation of transformers!!!

[@SkyRiderJavelin]

What an excellent series on Transformers, really did the trick !!! The penny has finally dropped. Thanks very much for posting this is very useful content. I wish I came across this channel before spending 8 hours doing a course and still not understanding what happens under the hood.

[@luizabarguil5214]

A clear, concise and conclusive way of explaining Transformers! Congrats and Thank you so much for sharing it!

[@shaktisd]

Amazing series on Transformer. Never ever imagined the true rationale behind Q,K,V ... it is actually clear after watching your video. Thanks a lot.

[@patriciasilvaoliveira6130]

Amazingly clear and encouraging to learn more. Thanks, maestro!

[@vishalmishra3046]

25:17 Hello Luis, *Positional Encoding* enables a Transformer to handle multiple words in parallel since ordering of words are built-in due to PE. Before transformers, the Reinforcement Learning using Long Short-Term Memory (LSTM) models were slow because words had to be processed serially since order and sequence matters. In addition to Attention, PE was a big innovation in Transformer models making them highly performant and so successful today.

[@johnny1966m]

Thank you Mr. Serrano, it was very educated and lectured in very good way. In relation to Positioning, the example with arrows gave me the idea that the purpose of this stage is to make sure that only the correct positions of words in the sentence cluster together, while the incorrect ones diverge them, thus the neural network distinguishes their position in the sentence during training.

[@GameDevCoffeeBreak]

This is the best video I have watched about the subject; it explains the concepts in such a way everyone can understand them! Thumbs up and subscribing right now!

[@aminemharzi7222]

The best explanation I found

[@ehsanpartovi8279]

This was such a thorough and precise explanation. And the visualizations! Great job. We would like to see more videos like this, particularly about other generative models such as those used for image and video generation, i.e. Canva.

[@aljebraschool]

Delighted to learn Transformer from my mentor today! Thanks for a very clear overview!!!

[@nileshkikle8112]

Dr. Luis - Thank you for taking all the effort for creating these #3 videos. Explaining complex things in the simplest way is an art! And you have that knack! Great job! Been following you ML videos for years now and I always enjoy them. PS- Funny enough, for typing these comments, I'm being prompted for selecting the next prediction word! 🙂

[@vimalshrivastava6586]

Thanks for such a wonderful explanation of a complex model. If possible, please make videos on Vision Transformer as well, if possible.

[@johnschut164]

Your explanations are truly great! You have even understood that you sometimes have to ‘lie’ first to be able to explain things better. My sincere compliments! 👊

[@JohnDeBrittoL-n6x]

Thanks!

[@SerranoAcademy]

Thank you so much for your kindness! Very appreciated. :)

[@BhuvanDwarasila-y8x]

That''s fire! One quick thought I had about finetuning is a good model should be able to respond to human behaviors really well and in the way the business wants! For example, the model should not simply respond to someone sad and say oh thats great, keep crying!, but you never know what the majority of the web context may be like! However at the same time this is something to be very cautious about from a user perspective because businesses can easily manipulate the model to encourage nation, business, other motives that may be unclear to us!

[@KC-dr4gj]

For word2vec, my understanding is that the embeddings are the weights from the input layer to the hidden layer, not the hidden layer itself as you menioned in 24:57?

[@anuragdh]

Thanks.. for the first time (not that I've gone through a lot of them :)), I was able to appreciate how the different layers of a Neural Network fit together with their weights. Thanks for making this video with the example used

[@karunamudliyar5625]

The best video, that I watched on Transformer. Very clear explanation

[@Tuscani2005GT]

Seriously some of the best videos on the topic. Thank you!

[@faraazmohammed3693]

Liked the video while watching at 7:15. crystal clear explanation. Good job, thank you Serrano and I appreciate your work.

[@nazmulhaque8533]

Excellent presentation. Waiting to see more videos like this. I would request you to make a series about aspect based sentiment analysis. Best wishes...

[@AmanBansil]

Incredible - just found this channel and I am about to pour over all videos. Thank you so much for your effort.

[@incognito3k]

As always amazing content. We need a book from Luis on GenAI!

[@GiovanneAfonso]

The best explanation available

[@markuskaukonen3903]

Very nice stuff!, This first time somebody explained clearly what large language models are. Especially the second video was very valuable for me!

[@jamesgeller9975]

This is VERY GOOD. I also watched several videos and this helped most. The one thing I don't understand is how the system decides that there are two apples. I felt that one Apple will be pulled back and forth between phone and orange.

[@utkarshkapil]

Beautifully explained. Loved how you went ahead to also teach a bit of the pre-requisites!

[@AboutOliver]

You have a skill for teaching! Thanks so much for this series.

[@Nerdimo]

8:53 do you think it’s a plain feed forward neural network, or something like an RNN, LSTM to be specific? Just a thought.

[@lakshminarayanan5486]

Hi Luis, Excellent material and you know how to deliver it to perfection. Thanks a lot. Could you please explain a bit more on positional encoding and how the residual connections & layer normalization, encoder-decoder components fit into the very same example.

[@samirelzein1095]

the great Luis! i am recommending you in my job posts, your content is a prerequisite before working for us

[@SerranoAcademy]

Wow thanks Samir, what an honor! And great to hear from you! I hope all is well on your end!

[@samirelzein1095]

@@SerranoAcademy honor is mine! you are the artist of going inside it, seeing the wiring and connections and delivering them as seen to all people. that s the job of prophets and saints. bless you. i am doing great, plenty of text and image processing currently :) digitizing the undigitized!

[@ikheiri]

Best video i've come across that explains concepts simply. Helped tremendously in my learning endeavor to create a mental model for neural networks (there's a joke there somewhere)

[@SerranoAcademy]

Thanks! Lol, I see what you did there! :)

[@edwinma9933]

this is amazing, it deserves 10M views.

[@poussinet2]

Thank you for these really high quality videos and explanations.

[@jukebox419]

You're the greatest teacher ever lived in the history of mankind. Can you please do more videos regularly?

[@SerranoAcademy]

Thank you so much! Yes I'm definitely working hard at it. Some videos take me quite a while to make, but I really enjoy the process. :) If you have suggestions for topics, please let me know!

[@harithummaluru3343]

great explanation. perhaps one of the best videos

[@skbHerath]

finally, i managed to understand the concept clearly, Thanks

[@timothyjoubert8543]

thank you for this series - wonderfully explained. 💯

[@sohamlakhote9822]

Thanks a lot man!!! You did a fantastic job explaining these concepts 🙂

[@usefbob]

This series was great! Appreciate all the time and effort you've put into them, and laid out the concepts so clearly 🙏🙏

[@dragolov]

Deep respect, Luis Serrano! Thank you so much!

[@abdelrhmanshoeeb7159]

Finally ,i am waiting it from a month. Thank you alot.

[@stephengibert4722]

Emmy Noether did not "invent" abstract algebra. It is very disconcerting that when I googled that question as a test I came up with, in great big capital letters, "NOETHER..." etc. Once these models are relied upon, we are in big trouble, because such errors will be embedded in the embeddings, and so on. Of course this is really peripheral to the topic at hand, and I very much appreciate your excellent job at teaching the basic ideas of transformers. Thanks very much!

[@wanggogo1979]

Finally, I waited until this video was released.

[@19AKS58]

Clearly the best & clearest explanations. One question: after the "write a story." prompt, when the Attention block selects possible words such as Once or There, does it go through the ENTIRE vocabulary to do so? Doesn't that take a very long time? Are there shortcuts to make it more efficient? Thx

[@SerranoAcademy]

Great question! Yes, transformers do go through all the words, but in a really efficient way. First of all, matrix multiplications are very optimized and parallelized, so these are done very fast. Also, sometimes shortcuts are applied, like taking topk max, in order to not look at the whole vector of all words each time. You may lose a word here and there, but it goes much faster.

[@chrisogonas]

Well illustrated! Thanks for sharing.

[@blueberryml]

excellent -- clear and concise explanations

[@jorovifi89]

Great work as always, thank you keep them coming

[@vasimshaikh9857]

Finally 3rd video is here 😮😅 thank you sir I have been waiting for this video since last month , everyday I check your channel for the 3rd video sir , thank you so much sir You're doing great work 👍

[@VerdonTrigance]

24:59 - again... who and how defines these layers and network to set words vectors? All it comes to it. How do we know that cherry and apple has a similar 'properties' ?

[@danieltiema]

Thank you for explaining this so well.

[@silvera1109]

Great video, hugely appreciated, thank you Luis! 🙏

[@leenabhandari5949]

Your videos are gold.

[@dayobanjo3870]

Great video, speaking from Abuja capital of Nigeria

[@SerranoAcademy]

ohhhh greetings to Abuja!!! Nigerians are the kindest people, I hope to visit sometime!

[@joaomontenegro]

These videos are great!! I would love to see one about the intuition of cross attention in, for example, the context of translation between two languages.

[@SerranoAcademy]

Thanks, great suggestion!

[@mikelCold]

Where does context length come in? Why can some models be longer than others?

[@amrapalisamanta5085]

Very good playlist

[@SatyaRao-fh4ny]

This is a great video, clarifying a number of concepts. However, I am still not finding any answer to some of my questions. e.g in this video, when the user enters "Write a story.", these are 4 tokens. But the "model" spits out a NEW word "Once". Where is this NEW word coming from? How does the "model" even "KNOW" about such a word? Is it saved in some database/file? Is there a dictionary of ALL the words (or tokens) that the "model" has access to? And I guess the other question what does "training a model" actually mean- on the ground- not just conceptually? After training, is the end result some data/words/tokens/embeddings that are save in some file that the "model" "reads/processes" when it is used later on? What are parameters? I have watched several hours of videos, but have not found answers to these questions! Thanks for any help for experts!

[@SerranoAcademy]

Thanks, great questions! Yes, there is a database of tokens, and what the model does is output a list of probabilities, for each token. The ones with high probability are the ones that are very likely to be the next in the sentence. So then one can pick a token at random based on this probabilities, and very likely you'll pick one that has a high probability (and that way, the model will not always answer the questions in the exact same way, but it'll have variety). The training part is very similar to a neural network. It consists on updating the weights so that the model does a better job. So for example, if the next word in a sentence should be "apple", and the model gives "apple" a very low probability, then the backpropagation process updates the weights so that the probability of "apple" increases, and all the other ones decrease. The parameters are the parameters of the neural network + the parameters of the attention matrices. If you'd like to learn more about neural networks and the training process, check out this video: kzbin.info/www/bejne/eIOcmWdtf9mkr9k

[@khaledbouzaiene3959]

nice wow but please i still have a question, you didn’t mentioned how the words with similarities are placed close in embedding, i know after we assign the mechanism attention score but don’t get do the embedding is a separate neural network as in video

[@tantzer6113]

That closeness is achieved automatically in the end result because it’s more efficient. It isn’t something that the human designer plans for.

[@SerranoAcademy]

Yes, great question! The idea is to train a neural network to learn the neighboring words to a particular word. So in principle, words with similar neighbors will be close in the embedding, because the neural network sees them similarly. Then the embedding comes from looking at the penultimate layer in the neural network, which has a pretty good description of the words. So for example, the word 'apple' and the word 'pear' have similar neighboring words, so the neural network would output similar things. Therefore, at the penultimate layer, we'd imagine that the neural network must be carrying similar numbers for each of the words. The embeddings come out of here, so that's why the embeddings for 'apple' and 'pear' would be similar.

[@khaledbouzaiene3959]

@@SerranoAcademythanks for clarifying i got confused coz i just want this huge neural network composed of multiple layers of smaller neural networks where the first one is the embedding layer not separate one , but generally everything now make sense now no matter the design

[@MohamedKeddache-r1o]

what is the true value of V,K,Q matrix ?

[@alexanderzikal7244]

Thank You very much for all your videos! Whats Software do You use for your presentations? All looks really nice, all pictures…

[@panoskolyvakis4075]

you re an absolute legend

[@sathyanukala3409]

Excellent explanation. Thanks you.

[@nafisanawrin2901]

While training model if it shows wrong answer how it is corrected?

[@karstenhannes9628]

Thanks for the video! I particularly liked the previous video about attention, super nice explanation! However, I thought most transformers simply use a linear layer that is also trained to create the embedding instead of using a pre-trained network like word2vec.

[@vankram1552]

This is a fantasitc video, by far the best on youtube. My only feedback would be the guitar music you use between chapters is a little abrasive and can you take you out of the learning process. Maybe some calmer more thought provocing music along with more interesting title cards would be better.

[@jamesgeller9975]

Also the third video shoes the attention blocks in the neural network but I don't see how that layer implements K Q V.

[@techchanx]

Excellent video, will be good to indicate what encoder - decoder model is in transformers. Couldnt figure that out here.

[@SerranoAcademy]

Thanks! Yes that's something I'm trying to make sense of, perhaps in a future video. In the meantime, this blog post is the best place to go for that: jalammar.github.io/illustrated-transformer/

[@TemporaryForstudy]

Your videos are rocking as always. Hey, do you have any remote internship opportunities in your team or in your organisation? I would love to learn and work with you guys.

[@SerranoAcademy]

Thank you so much! Yes we have internships, check them out here! jobs.lever.co/cohere

[@samarthseksaria2587]

Could you describe the feedforward step more, with respect to the variable input length? Is it an RNN?

[@william_8844]

I like the attention explanation

[@k.i.a7240]

Again, I am speechless with the awesomeness of this video, There are many who understands these concepts but there are actually very few who can convey them in such a clear and precise way. With that said, I have another question(:D): In the Word2Vec Method, this nueral netwrok which helps us find the similarity and put the words which are more similar in closer coordinates, how it is trained? I understood that it is pre-trained, but at some point for calculation the error it should have had a True Y label right to compare the result with right ? So they need to have a reference Y labels for pre-training this nueral network ?

[@SerranoAcademy]

@@k.i.a7240 thank you! Great question! There are many ways to train word2vec, but I’m a nutshell, both the features and the label come from the text. For example, if the sentence is “hello how are you”, you could use as features “hello how are”, and as label “you”, since the NN is meant to find the next word in a sentence. (You could also reuse this sentence to use as features “hello how”, and as label “are”, and continue the rolling window. Other ways to train word2vec, is to remove a word from the sentence, and guess it using the other ones, so for example in the sentence “hello how are you”, you have four possible words to remove, say, you can use as features “hello, are, you”, and make the NN predict the label “how”. Or even viceversa, from the feature “how”, you can train the network to predict the labels “hello, how, are”. This way, the neural network gets used to words that are similar (since for example, guessing the neighbors of “Apple” would be similar than the neighbors of “pear”, thus, the neural network treats them similarly, implying that the penultimate layer of the NN would fire similar numbers with “Apple” and with “pear”. Hope that helped, lemme know if there are any doubts! :)

[@k.i.a7240]

​@SerranoAcademy, thank you again,It is cristal clear now and I appreciate the time you put for reading and answering our questions, many thanks 🙏🙏

[@ksprdk]

Great video, thanks

[@htchtc203]

Sir, thank you for very clear and informative series of presentations. Excellent job? May I ask something about embedding or word2vectors. How is a NN trained for words in order to cluster words for some kind of similarity grouos in multidimensional vector space? Is this training proceas guided or is it like self organizing map or process?

[@maethu]

I am happy like a zygote about this video! Great work, thanks a lot!

[@maneeshajainz]

I like your videos. Can you post a quiz after each of your videos?

[@sriharsha580]

Thanks for the wonderful presentation. In the previous video of the series while discussing the relationship between query and key(building the context relationship between words), it is mentioned the relationship QK and v(predicting next word) will be covered in this video, may I know whether it will be covered in another video or not.

[@rasmusnordstrom9947]

Is there any good explanation out there for the "second" input into the transformer structure: Outputs (shifted right) in the original paper?

[@SerranoAcademy]

Thanks for the question! Not sure exactly what second input. The one coming out of the attention mechanism and into the transformer? I would say that that's a 'enhanced' vector for the input text. Namely, one that carries context on it. Lemme know if that's what you meant, or if it was a different one.

[@rasmusnordstrom9947]

​@@SerranoAcademy If I understand correctly when looking at Figure 1 in the original paper (attention is all you need), the initial prompt is first fed into the encoder and then inserted halfway into the decoder, which finally yields the first token. As far as I understand, to generate the next token, we don't simply append the first token to the initial prompt and run it through both the encoder and decoder. Instead, we insert the newly generated token directly into the decoder (or runs it through a different encoder?). I'm somewhat confused about this part.

[@ColinTimmins]

Very nice! Love your work and visuals. =]

[@atriplehero]

Does the whole "Once upon a time" already built gets fed again into the whole process again as a 'input' in order to get attached its next word/token? In order words, is it like a cycling again and again untill a "seemengly complete" answer is generated? If this is the case, it would be a whole lot of inefficiency and explains why so much electricity is consumed!! Please answer this crucial detail.