L15.5 Long Short-Term Memory

Slides: sebastianraschka.com/pdf/lect...
-------
This video is part of my Introduction of Deep Learning course.
Next video: • L15.6 RNNs for Classif...
The complete playlist: • Intro to Deep Learning...
A handy overview page with links to the materials: sebastianraschka.com/blog/202...
-------
If you want to be notified about future videos, please consider subscribing to my channel: / sebastianraschka

[@virozz1024]

Best videos i have ever seen about ai you clarify everything and it's pretty easy to understand.

[@SebastianRaschka]

Thanks a lot!

[@hungerbear4858]

You're so good at explaining complex topics. Thanks!

[@yogeshrampariya8346]

Thanks professor for detailed explanation, I have been following deep learning playlist. I have not found such in depth explanation and code anywhere.

[@harishpl02]

Clear explanation, Thanks

[@rishavkumar8341]

Thanks for the great explanation. Neverthless, note that at 11:06 *i sub t* is "input gate" whereas at 12:51 it is denoted as "input node". It is a minor errata.

[@SebastianRaschka]

Oops, good catch!

[@ibrahimwalid416]

Thank you so much, subbed.

[@736939]

Your explanation is awesome. The only thing, that always makes me confused, is how does forget gate know what to forget and what to save? I understand that there is a sigmoid [-1, 1] function, but how does it learn what to save or forget? Thank you.

[@SebastianRaschka]

That's an interesting question. Its range is [0, 1] where 0 would be forget and 1 would be not forget (and values in between are "forget a bit" lol). It does depend on the input values, and my guess is that it learns to shape the inputs in a certain way over the time steps. Honestly, why it works so well I have no idea. There is a relatively recent paper though that revisits the LSTM, and apparently the forget gate is one of the most important components: arxiv.org/abs/1503.04069

[@736939]

@@SebastianRaschka Thank you very much. This question I asked so many times in forums but it works like a magic )) Also interesting to know the back propagation in steps of elementwise multipication and elementwise sum how they are calculated. BTW your newest book is awesome, I bought it from Amazon. Thank you.