18- Long Short Term Memory (LSTM) Networks Explained Easily

[@sholabenedict6125]

i honestly did not think i could understand this at my first watch, this is amazing.

[@williamashbee1140]

Every time i've attempted to understand rnn and lstm my brain went to mush. you did the best job of explaining this of anyone i've seen. thanks.

[@ValerioVelardoTheSoundofAI]

Thank you William!

[@sivadasanet7966]

Thank you so much sir for this wonderful class.

[@jamesadler4859]

Wow. After a week of being confused by these things, watching videos, and reading articles, you just totally cleared my vision in this 30 minute video. THANK YOU!

[@ValerioVelardoTheSoundofAI]

Glad I could help James :)

[@jewbaby9143]

This is an outstanding video. Great job! I really like that you include examples along with your explanation of the steps. That really helps, and you can't find that anywhere else :)

[@danny_p466]

It's really impressive how you simplify such complex topics. Being a Udacity DL Nanodegree graduate some months ago, I came here to refresh these topics and your explanation was exactly on point! Will continue with your music generation series, thanks :)

[@ValerioVelardoTheSoundofAI]

Thank you Danny!

[@Virtualexist]

BEST SIMPLIFIED EXPLANATION OF LSTM. I WATCHED 7-8 VIDEOS BEFORE THIS. BUT UNDERSTOOD ENOUGH ONLY TO SAY HMMMM... THIS ONE MAKES ME HAVE A CONVERSATION WITH MYSELF ABOUT THE CONCEPT. BEAUTIFUL SIRRRR !

[@ValerioVelardoTheSoundofAI]

Thank you!

[@user-uv6ri7qb4g]

Im currently studying MAsters of Applied Data Analytics at one of the top universities, and your explanation is much more superior. Thanks so much !

[@ValerioVelardoTheSoundofAI]

Glad you liked it! Stay tuned for more ;)

[@karmawangchuk283]

Thank you so much for the precise explanation. Now, it is forever tattooed in my mind.

[@knowandthink4960]

I really try to find the best video of LSTM, and I wanna to say one thing about this video. That is fucking best video to understand LSTM. I don't want to say thank you Valerio Velardo because This video deserve more than that....

[@T4l0nITA]

Wow, I remember studying this years ago but understanding close to nothing, your explanation made everything clear

[@ketaki9633]

Amazing channel!!! Every doubt solved! Great playlists, theory and implementation! Kudos to you for helping hundreds of people!

[@ValerioVelardoTheSoundofAI]

Thank you!

[@bhrz123]

Amazing explanation. There are many tutorials on lstm out there that have shown only the equations but haven't actually explained how lstm remembers or forgets an information and your video has filled up those gaps in those tutorials. Thank you for your amazing videos. I have liked and subscribed. Looking forward to more amazing tutorials from you.

[@atNguyen-jv5yc]

There has not been much material on audio processing so I'm a big fan with this series. Really appreciate your hard work. :)

[@ValerioVelardoTheSoundofAI]

Thank you Dat!

[@vaibhavsingh8715]

One of the best explanation of LSTM working. Thank you so much.

[@vasanthdamera5896]

I am doing my project on hourly electricity price forecasting (using Python)... There's a need to learn about LSTM. As it is the main concept of the prediction models... found this a lot helpful... now i can easily explain my peers about how an LSTM works..Thank you bro.. I hope u make much more content like this.

[@jainrohit0123]

Awesome Explanation!! The title "Explained Easily" is really justified.

[@ValerioVelardoTheSoundofAI]

Thanks Rohit :)

[@artyomgevorgyan7167]

A great explanation by comparing simple RNNs to the modified LSTM!

[@ValerioVelardoTheSoundofAI]

Thank you!

[@reshmadevidas8380]

Wow. I happened to be reading that blogpost yesterday and quickly realised the diagram is from that blog post before you mentioned it.

[@neuralmist3548]

What a fantastic and simple explanation. Thank you!

[@leopoldodellaporta3831]

This video is pure gold

[@Mrnobody-qj7zl]

very well explained. i had few misconceptions and this awesome video just cleaned up. thanks a lot. May Lord Krishna bless you.

[@alirezamirhabibi8039]

Very Very good explained, Thanks a lot dear Valerio.

[@ValerioVelardoTheSoundofAI]

Thanks!

[@aashishagarwal8470]

Cleared a lot of doubts! Thank you. :)

[@sanderwood]

A really good tutorial! I feel LSTM will be more suitable for music generation task.

[@ValerioVelardoTheSoundofAI]

Yes, LSTMs have been used extensively for music generation. Indeed, I have a series on that!

[@70ME3E]

I like your energetic way of explaining it 🙂

[@Kraft_Funk]

Thank you John Lennon for a great explanation of LSTMs!

[@ValerioVelardoTheSoundofAI]

Ahahahah... sometimes they come back ;)

[@manpreetkaur8587]

Beautifully explained!

[@justinwong1111]

Thank you very much! Looking for unsupervised training series.

[@timuk2008]

Amazing job at explaining complex stuff! Thanks a lot

[@merakid2129]

Thank you man. You made it simple and interesting.

[@ValerioVelardoTheSoundofAI]

Thanks - I'm happy you liked the video!

[@hackercop]

These videos are really good, thanks, you are a really good teacher.

[@christianmitrache6317]

Dude, you are literally a lifesaver!! My professors don't go into ANY details on RNNs or LSTM. Do you have any videos/github posts for transformers?

[@ValerioVelardoTheSoundofAI]

Thanks! I don't have videos on transformers yet, but I'll cover them in the future. Stay tuned!

[@Rotnisi]

Great video! Very good explanation! :)

[@orcunkoraliseri9214]

Why don't you put early stopping and is there any other video for LSTM tuning? Thank you. Great tutorial

[@isaasricardovaldiviahernnd7736]

Great video!! Help me a lot to understand LSTM

[@BencFenc]

Great video - fantastic explanation. Thanks!

[@ValerioVelardoTheSoundofAI]

Thank you!

[@Merucury]

I understood well. Thanks :)

[@10mpmy10]

15:46 best part

[@EngRiadAlmadani]

great work sir but what is the advantage of output filter in lstm cell because cell state forgot the unimportant information in the beginning

[@withzmh]

You are awesome ! Thank you for sharing.

[@i_am-ki_m]

Nice, overtime! I interest to LSTM (because this method it's largest usually in engineering/programming), so what you indicates for a possible future pontual study? Keep walking to finish other series, tkx so much and chers!

[@Sawaedo]

Thank you Valerio! My question would be, how long does it takes to train a LSTM network vs a RNN, and what are the sizes comparisons between the two?

[@sachinkuntal5421]

nicely explained!

[@draxd3045]

Excellent video. Like it so much

[@kchan8878]

Great tutorial. Thanks.

[@ValerioVelardoTheSoundofAI]

I'm glad you liked it!

[@Magistrado1914]

Excellent course 15/11/2020

[@SparklingCupcakez]

Great great great video!

[@ValerioVelardoTheSoundofAI]

Thank you Alexandra - glad you appreciated it :)

[@olanmalde9312]

great explanation! :)

[@ValerioVelardoTheSoundofAI]

Thanks!

[@nhactrutinh6201]

If we need long term memory, why shouldn't we make RNN the a stack or a queue to store? Why we need such a complex LSTM?

[@badnewswade]

Thank you very much sensei! Do these things have their OWN biases, or do they use a common bias as well as weights / layers?

[@fuweirao5770]

Do all the weighted matrices in the dense layers keep constant during a whole time series?

[@razterizer]

So we just concatenate h_{t-1} with x_i in every cell and no splitting afterwards? Wouldn't then the vectors used in the cell grow for each latterally and forward connecting cell? I'm surprised that no one explains the dimensionalities. The linear algebra aspect is just as important to understand in order to be able to make an implementation.

[@levran4ik]

Чётко все так разложил, красава!

[@ITelefonmanI]

Great explanation, thank you very much. However, I do have a question regarding the Forget Gate Layer. You say that the sigmoid function will render the values of the ft matrix to be between 0 and 1, not 0 or 1 - hence 0.45 or 0.55 are possible values in the ft matrix. So how does the next step forget (set them to 1 or 0 by elementwise multiplication) values in Ct-1?

[@shruthimahalingam8936]

Great video explanation thanks a lot!! As a student I am doing a project for evaluation of students answers based on reference answers. In that project I want to add LSTM model. Can I use LSTM for comparing similarity between two sentences(student answer and reference answer)? If so can you please suggest me one LSTM model suitable for that? It would be great if you could clarify my doubt.Thanks!

[@rafsunahmad4855]

You are awesome❤️

[@grjesus9979]

thank u a lot man

[@amirasad5348]

hi thanks for your amazing channel please help me to find a good data set for genre classification except GTZAN. I need a larger dataset

[@fatirali884]

Great Video Valerio! Just one question, could you explain further why we use tanh for the output layer?

[@beshosamir8978]

Hi , i need some help here why we decide to make the next hidden state = the long memory after filter it ? why not the next hidden layer not = the long memory (Ct)

[@shubhamchauhan6916]

Sir I am facing a problem in predicting Y if I am giving my X values and I'm getting this error - expected dense_1_input to have 2 dimensions, but got array with shape (278, 1, 1) why???? I have send request to your linkedin group can I show my code there ?

[@elseklinge]

hey valerio, thanks so much for this video! Are "cells" the same as "neurons"?

[@ヨママ]

I think it's the entire neutral network at one time step

[@fujiawang4326]

Hi! why do LSTMs work well with MFCC data?

[@ivanatora]

You tricked me with "Explained Easily"

[@ValerioVelardoTheSoundofAI]

Lol - wasn't it though?

[@ivanatora]

@@ValerioVelardoTheSoundofAI I got lost too many times, but its just me :) I love your videos, man, you deal with super interesting field and you are also a charming speaker. One question - let's suppose I want to prepare training dataset with audio data. I have to manually classify lots of different segments. Do you know of an app that can provide a nice UI for doing it?

[@ValerioVelardoTheSoundofAI]

@@ivanatora thanks :) Unfortunately, I don't know any such UI - but I know that there are several companies that do music/audio classification as a service (e.g., TagTeam). If you're familiar with Python. you could create a prototype using for example a simple Flask, MySQL, HTML stack.

[@badnewswade]

On a subsequent viewing I'm REALLY confused. Wikipedia doesn't say anything about concatenation, it uses addition - wouldn't concatenation cause you to end up with unfeasably huge vectors for subsequent layers?

[@Kraft_Funk]

I just thought of this. But they are actually the same. Suppose you have a hidden state of size 128 and embedding vectors of size 10. If you use addition, you would multiply your embedding vector with a matrix of size (128, 10) to match with the hidden state length. And you would multiply your hidden state with a matrix of size (128, 128) so as to keep its size constant. Then you add them up. However, instead of all these, you can just concatenate your embedding vector with the hidden state, obtaining a vector of length 138, and also concatenate these two matrices yielding a size of (128, 138), and then do a single matrix multiplication. Just get a pen and paper and try these out, they are equivalent.

[@jonjon-xh7xj]

Someone should really tell colah to change the part in the image where ht=ot*tanh(Ct). In his legend there's curved lines for intersection or divergence, but in the image it's a T joint. Also, where does ht go to ? There's a split into 2 outputs of ht.

[@ValerioVelardoTheSoundofAI]

h_t plays two roles. It becomes the new hidden state for the current time step that's fed back into the cell at the next time step (h_t in the lower section of the image). h_t is also the output of the cell that, if we have one LSTM layer only, usually gets fed into a softmax dense layer for classification (h_t in purple circle).

[@JustinMitchel]

Yes nice work

[@ValerioVelardoTheSoundofAI]

Glad you liked it!

[@lawan8349]

amazing and funny tutorial

[@ValerioVelardoTheSoundofAI]

Thanks!

[@凌璃-b8z]

tks

[@gb7586]

This helps

[@saeedullah5365]

Why LSTM have more accuracy than Bi directional LSTM though is noval concept

[@ValerioVelardoTheSoundofAI]

There are certain tasks where bidirectional LSTM layers perform better than simple LSTM layers and vice-versa. It depends on the task.

[@Amsardm]

sigma(Wi[Ht-1,Xt]+Bi) need a bit of clarification on concatenating Wi[Ht-1,Xt] please :)

[@TheFedonMan]

Concatenation is just putting the two matrices side by side either horizontally or vertically. If you concatenate horizontally the matrices must have an equal number of rows, and if you concatenate vertically an equal number of columns. For example: |10, 5| |1| |10, 5, 1| |20, 6| , |2| = |20, 6, 2| |30, 7| |3| |30, 7, 3| These two matrices cannot be concatenated vertically because the number of columns is different.

[@Amsardm]

@@TheFedonMan thank you :)

[@Kajahzao]

hard to overfit these things and train ...

[@oueslatiamine3843]

If I meet you one day, I'll make you a sandwich!

[@ckoegl]

Very wordy explanation attempts. Fails to shed light to the influence of x_i and h_i-1 on any of the computations. Does not explain why it is important that h_i is squashed by using tanh but why C_i is not. Fails to provide any explanation why the cell's computations actually make h_i capture short term info and C_i capture long term info. More like a slow walk through the low-level operations rather than connecting them to the high-level purpose of the components.

[@boriscrisp518]

video could be 1/3 of the lengths if you stoped saying "kinda" ever other word

[@TearsOnYT]

Why don't you try making videos on machine learning as good as him then?

[@marsgalaxy6734]

Too Slow. Better come to point in time. Time matters

[@amansinghal5908]

I am not one to be spiteful, but you wasted 30 minutes of my time! This is a walk-through, not an explanation - understand the difference. 1. Don't watch this video if you want to build an intuition 2. Watch this video is you want a walkthrough, having said that - there are shorter videos out there

[@ayo4757]

Hi velario, i am trying to repoduce the ai model used bay moises.ai page (tracks separaton ->> song = voice, bass, guitar, piano, batery,etc )! do you have some video or any recomendation to inroduce me in this journy ? thx you are the best!