Recurrent Neural Networks (RNNs), Clearly Explained!!!

[@statquest]

To learn more about Lightning: github.com/PyTorchLightning/pytorch-lightning To learn more about Grid: www.grid.ai/ Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/

[@rathnakumarv3956]

whether the book include LSTM and RNN for bi variate input time series data as well?

[@statquest]

@@rathnakumarv3956 Nope, just basic neural networks. My next book on deep learning will have more stuff about fancy neural networks.

[@rathnakumarv3956]

@@statquest please include CNN, RNN and LSTM for multivariate time series as input and continuous variable as out put - problems, these are very much useful in climate change studies.

[@statquest]

@Tech What time point, minutes and seconds, is confusing? And have you watched the entire series on Neural Networks before watching this one?

[@Lyr00]

@statquest I have a question about the vanishing gradient problem. I understand that input1 from the first timestep is having less and less impact on the output the more steps we take but isn’t the gradient also relying on the new inputs of every timestep? I don’t understand why the gradient is vanishing if the new inputs aren’t that heavily discounted as the timesteps that are older. I imagined it’s more like the old inputs are less impactful and the network is more focused on the newer inputs but can still train normally. Is there something I’m missing?

[@s0meus3r]

The only place on the internet where you can actually grasp a complex topic before diving deeper into the topic. I am so grateful people like you exist. Thank you!

[@statquest]

Thanks!

[@lucutes2936]

@@statquest np

[@wildpjah]

I'm in a deep learning class right now and the amount of straight math that my teacher throws at me is overwhelming. Videos like yours are incredible and I'm so thankful for the help and the color coding and the fun that makes it worth watching! It is super helpful as I'm studying for my midterm and just want to get a more definite grasp of what all this math actually means without reading someone's Mathematics PhD dissertation or something

[@statquest]

Good luck on your midterm! :)

[@mohammadhamed-ro9tx]

Everytime I watch on of your lessons, I become sooo happy, because you make all the subjects easy to be understood in magical way. Thank you for your effort

[@statquest]

Wow, thank you!

[@artem_isakow]

Thanks to your series of videos on neural networks, I was able to pass the entrance exam for PhD program at St. Petersburg State University.

[@statquest]

Congratulations!!! TRIPLE BAM!!! :)

[@jamesyoun1143]

One of the most underrated channels. Never once have I had trouble understanding the intuition of whatever you explain. I'd donate money to you if I weren't a broke college student.

[@statquest]

Thanks!

[@toddgillies3380]

Never quite understood RNNs until I watched this video, thank you! A hand-calculated example of a one-to-one RNN is extremely hard to find online, so this was perfect. The only one out there, I believe.

[@statquest]

Thanks!

[@kirannbhavaraju5978]

This is the CLEAREST explanation of RNNs.

[@statquest]

Thank you!

[@elmehditalbi8972]

You can't understand how good this is. I've spent all of yesterday trying to understand these concepts but I couldn't grasp them. THANK YOU!!!

[@statquest]

Glad it helped!

[@sushankmishra53]

With this level of simplicity in teaching, even a high schooler could grasp these concepts, probably quicker than me! Scared of the future now....

[@statquest]

:)

[@thecomputerpal221]

Josh, I found your channel yesterday and have been binge watching. Incredible work in democratizing knowledge. Thankful for your work.

[@statquest]

Thank you!

[@sylwiagotzman5422]

democratizing knowledge -- exactly !!!

[@charlescoult]

This explanation covers some very important points that were missed in several other lectures I've watched on this subject. Thank you for clearing things up for me.

[@charlescoult]

For example, the note at 10:31

[@statquest]

Thank you! Yes, that little detail is important, but often glossed over.

[@LuizHenrique-qr3lt]

Josh!!!! I love u!!! I can't wait to learn about the Transformers!! thank you very much for your content

[@statquest]

Thank you!

[@capyk5455]

Josh teaching about transformers would be a blessing

[@statquest]

@@capyk5455 I'm looking forward to it!

[@shaiguitar]

Transformers out yet or some ETA to expect?

[@statquest]

@@shaiguitar LSTMs comes out in the next week or so. Then I'll start working on transformers.

[@shivanshgupta934]

wow, just wow! 2 days of headache solved by a 17 min video! thank you for existing.😊

[@statquest]

bam!

[@CodingwithRayyan]

This is not fair, I literally am addicted to your style of teaching and find it quite hard to learn from other sources now.

[@statquest]

Hopefully, in the long term, what you learn from these videos will make it easier to understand other sources. At least, that's starting to happen for me. It's still hard, though.

[@ZaidAlhusainy]

Your channel should be mandatory for all universities teaching AI 💖

[@statquest]

Maybe one day!

[@guptafamily1981]

It will be for sure 😊

[@rw7154]

StatQuest’s stylized scalar-based mumerical examples are amazing even for learning beyond the introductory level. To get the full vectorized-matrix version of the algorithms, I just mentally swap x, w, b etc with the corresponding vectors and matrices, then it’s golden!

[@statquest]

Bam! :)

[@svensvensson3679]

Our lecturer at the uni recommended us this video. I am amazed how simply it is put. Great job! Both funny and informative ❤

[@statquest]

Thank you!

[@Ahmad_Alhasanat]

I was looking for a small thing in RNN, but your way of explanation forced me to keep watching the entire video! and I subscribed to your channel!!

[@statquest]

Hooray! Thank you very much! :)

[@samama251]

First time commenting on youtube video. You are a living legend. It has been two days I am trying to understand RNNs and came across your video and Baam. RNNs concept got cleared.

[@statquest]

Glad I could help!

[@carleanoravelzawongso9786]

I'm just in love with your content. I've watched your neural network series and it was just so easy to understand. You really deserve more subs and views Josh!

[@statquest]

Thank you very much! :)

[@chaitanyasharma6270]

you definitely are the best teacher for machine learning and deep learning

[@statquest]

Thank you!

[@DavidBlayvas-wo4lj]

You're gonna carry me through my neural networks class, what a godsend

[@statquest]

You can do it!

[@wongkitlongmarcus9310]

Josh, you are the person who make ML theory so understandable!

[@statquest]

Thank you!

[@pixel_yeast]

i come to listen to " peep poop poop"

[@statquest]

bam! :)

[@anonymousertugrul5858]

I like the way how clearly and easily you explain concepts. Thank you very much!

[@statquest]

You're very welcome!

[@qwertz2167]

True Hero. I have an exam on 29th about rnns, lstms and transformers.

[@statquest]

Good luck! :)

[@v-sig2389]

Thank you so much 😭 People like you are the real mvp of humanity !

[@statquest]

Thanks!

[@santiagocalvo]

honestly your channel is one of if not the best channel on all youtube, thank you so much for this!

[@statquest]

Wow, thank you!

[@deepikasheshabutter4790]

i literally was having a menatal breakdown coz i was unable to understand things. your video helped me a lot and also brought a smile on my face :))

[@statquest]

Glad I could help!

[@TheChair610]

OH MY GOD THIS IS EXACTLY WHAT I WAS LOOKING FOR THANK YOU SO MUCH

[@statquest]

bam! :)

[@manishnarang6490]

Really looking forward to your LSTM video.. You are a very good teacher !!

[@statquest]

Thank you!

[@WonPeace94]

@@statquest when will you make the next vid ? i have exam in two weeks and i need your LSTM video

[@statquest]

@@WonPeace94 :)

[@curiosityspace8635]

Oh man i literally watch his videos like a web series its very fun and very easy to understand thank you very much sir !!!!😭😭

[@statquest]

Thanks!

[@cat-a-lyst]

you are vry very very very very very brilliant teacher ! you are my low variance and low bias position.

[@statquest]

bam! :)

[@huruynegash4847]

Hello Guys let's make this man happy always as he did for us!!!!!!!!!!!!! Nothing to say just thanks a lot.

[@statquest]

BAM! Thanks!

[@anashaat95]

Very high level explanation. Waiting for the next video on "Long-short Term Networks". Thank you so much.

[@statquest]

Thanks! :)

[@BrunoJr09]

OMG, Finally I understand Vanishing Exploding Gradient, Thank you StatQuest!

[@statquest]

HOORAY!!! Thanks for supporting StatQuest!!! TRIPLE BAM! :)

[@waizwafiq9481]

Amazing video as always, professor! I cant wait for the video on LSTM

[@statquest]

You and me both!

[@usamsersultanov689]

@@statquest when it comes to application of RNN the LSTM is sometimes a must to have:) that’s why it would be great to have clearly explanation of LSTM. But these are little things. In any case, thank you very much for the valuable knowledge that we can get here.

[@statquest]

@@usamsersultanov689 I hope to have a video on LSTMs soon.

[@james199879]

@@statquest One on Transformers and their variations would be even greater :D

[@statquest]

@@james199879 That's the master plan, but we'll get there one step at a time.

[@saharshayegan]

This was the best explanation I've heard for RNNs!

[@statquest]

Thank you! :)

[@brandonso5477]

why are you master of everything???? I have been watching your video for two years through out my university course

[@statquest]

Ha! Thank you! :)

[@IsaacKLusuku]

Hey Josh...the way you made this so easy is mind blowing, I love you man, keep being awesome 😊

[@statquest]

Thank you so much 😀!

[@KayYesYouTuber]

Hi Josh, You are the best. Nobody has explained exploding gradient like you have, Thank you

[@statquest]

Wow, thanks!

[@zytriesthings4540]

Thank you brother, this was really intuitive and easy to understand

[@statquest]

Thanks!

[@broccoli322]

Im a simple man, I see statquest, I click like. Can't wait for the videos on transformers.

[@statquest]

bam! :)

[@CharlieYou823]

PENTA BAM!!! The best pre-course !

[@statquest]

Thanks!

[@solotop5916]

Dude that DOUBLE BAMM and TRIPLE BAMMM kills me. Actually fun way to get info. Also greate video very easy to understand

[@statquest]

Thank you! :)

[@gabbye165]

12:22 is probably the cutest bam I've heard Also thank you for your videos! They have definitely been helping me get through my Bioinformatics grad course. You are AWESOME

[@statquest]

Thank you so much and good luck with your course! :)

[@8bit-ascii]

They way you explained RNNs made me so excited for LSTMs. Can‘t wait to see it!

[@statquest]

bam!

[@lore10010]

Excelente proyecto! no pense que con dibujos fuera tan entretenido e informativo. Definitivamente un muy buen video para comenzar!

[@statquest]

Muchas gracias! :)

[@HEYTHERE-ko6we]

Those tones won won bam double bam kaboom n d fun way of learning, opens up mind for grasping things real quick as well as we can think freely wdout bcming nervous. U lord🙌

[@statquest]

Thanks!

[@sheltonsong6120]

Clearly explained the difference between RNN and normal network, gradient vanishing/exploding! Looking forward to the LSTM and Transformer videos!!!

[@statquest]

Thanks!

[@osamahabdullah3715]

literally before I see your video, I made a like, that much I trust your information and knowledge , thank you for your time and effort to explain this to us

[@statquest]

Thanks!

[@ai_station_fa]

Best explanation I've seen from RNNs. Thanks.

[@statquest]

Thank you!

[@AI_ML_DL_LLM]

Great video, you must become the President of the ClearlyExplainedLand

[@statquest]

:)

[@felipela2227]

El video estuvo muy bien explicado, lo entendí facilmente. Gracias

[@statquest]

Muchas gracias! :)

[@chrism3440]

Oh my gosh that was an amazing explanation. I'm quite literally flabbergasted. Thanks, mate!!

[@statquest]

Glad you liked it!

[@indusairaman2126]

U have an amazing way of explaining with adlibs loved it and thank you so much as I was not able to understand at all but now it is very clear

[@statquest]

Glad I could help!

[@AhmadAbuNassar]

Any darn fool can make something complex; it takes a genius to make something simple (" Albert Einstein"), and you made it very very simple. Thanks!

[@statquest]

Thank you very much! :)

[@mahu1203]

Thank you so much. You make great videos... Just great teaching. Thanks alot.

[@statquest]

Glad you like them!

[@loreii1982]

I saw a light turning on my head! great video

[@statquest]

bam! :)

[@safiyajd]

Josh, you are amazing! Thank God you exist

[@statquest]

Thank you!

[@CarlosCaetanoJr]

Anxious for the LSTM, Transformers and Attention StatQuests!

[@statquest]

It's already available to early access patreon supporters and will be available to everyone else soon.

[@justLu__]

Hey, hopefully this will safe my Deep Learning exam. And... love the sound effects.

[@statquest]

Best of luck!

[@TheGibberingGoblin]

... you sir are a timeless legend!

[@statquest]

Thanks!

[@iReaperYo]

You don’t understand how good the timing of this is. Been struggling to explain the concept in detail on my MSc project. Are you are doing a video on LSTM / GRU soon ??

[@statquest]

Yes, I hope so.

[@giorda77]

Clear as day!!! Hooray!!!! Thank you Josh

[@statquest]

Thank you!

[@harishbattula9881]

Thanks a lot Josh. Every concept explained by you is a BAM!!!!!!!!!!

[@statquest]

Glad you think so!

[@exxzxxe]

Your professorial ability is only exceeded by your singing!!

[@statquest]

Thanks!

[@exxzxxe]

@@statquest When are you going to release your, surely to be a best-selling, book on singing lessons?

[@statquest]

@@exxzxxe Just as soon as I win Eurovision.

[@shofiyabootwala2094]

the vanishing/exploding gradient problem is synonymous to choosing a right value for alpha (learning rate) as choosing a greater value would leave us bouncing and choosing a lower value would lead to more iterations of gradient descent

[@statquest]

Nice

[@sarahk13peace]

OMG I love you for your teaching style

[@statquest]

Thank you! 😃

[@JulietNovember9]

Oh man! This has been super tough for me to wrap my head around. I knew this was going to be a great weekend! Thank you for the drop! :D

[@statquest]

BAM! :)

[@WALID0306]

Gracias !! Estuvo excelente ✨✨✨✨ Bendiciones

[@statquest]

Muchas gracias!

[@mohammadahmedbasri3067]

Thank you for this amazing explanation! Waiting for the video on LSTM! :)

[@statquest]

Coming soon!

[@mostafamarwanmostafa9975]

Thank YOUUU Clearly explained !! I have been struggling with it !

[@statquest]

BAM! :)

[@NockyLucky]

Really liked the video. Quite creative and straight to the point!

[@statquest]

Thanks!

[@hamzaahmad1224]

Thank you for the video. I believe it was a clear explanation.

[@statquest]

Glad it was helpful!

[@lukastoral5059]

Thank you for making these videos! They are very helpful.

[@statquest]

TRIPLE BAM!!! Thank you so much for supporting StatQuest!!! :)

[@AuthsKads]

u are king my friend perfect explanation with simple example

[@statquest]

Thanks!

[@luistalavera6392]

Man, this is awesome. I wasn't understanding anything about RNNs in my course but thanks to this video is all clear now. Thank you Josh Stamer :D

[@statquest]

BAM! :)

[@sinaro93]

KA-BAAAM! Thank you for all these amazing videos. I wish you had different series about CNNs and RNNs separately.

[@statquest]

If you want to learn about CNNs, see: kzbin.info/www/bejne/fnjac4t6gKueb6s

[@sinaro93]

@@statquest How could I not have seen this?! By different series, I mean it would be great if you could create more videos covering each of these topics in more detail. But of course, you've already done so far, and I'm so grateful to you for sharing your knowledge in such a good way.

[@Tapsthequant]

Been waiting and waiting, the waiting is BAM!!!

[@statquest]

:)

[@dikshagupta3500]

Your book on Machine Learning was excellent. I am looking forward to reading your book on deep learning.

[@statquest]

Thank you very much!

[@Xayuap]

gonna be honest, I get here looking for backprop. I didn't, instead found myself doing the whole course. Now I'm taking the selective courses 🧘🏽 I do feel like Neo wanting more Xaolin.

[@statquest]

That's awesome!

[@chelvynchristsonimmanuel1453]

Amazing explaination with simple, easy-absorb, attractive method and but still pursue clear concept. 🙂 Kabaam... nice job

[@statquest]

Thank you! :)

[@chriskong7418]

Love the little embarrassing singings during the videos. Subscribed. Great videos!

[@statquest]

Thanks!

[@maximus2978]

You are insane Man, very clear and understandable explaination!! Thanks a ton 🎉

[@statquest]

Happy to help!

[@Raulvic]

Thank you for sharing 🙂 super excited for the transformers statsquest!

[@statquest]

Thanks!

[@harshshah3797]

Summary Problem with regular neural network It takes fixed length of input. Here comes RNN to help. It can take variable length of input. How RNN is made? Input + Previous Output == Output Why RNN is not popular? As it has one problem call Vanishing / Exploding Gradient Problem. As we have long chains,it is natural to this problem will arise. Lets say we have weight that multiplies with previous output greater than 1. As we have long chain,we will be multiplying many times numbers will become large. (Output will very too much) If we have less than 1 it will become very small. (Output will not change at all) Here is an analogy that might help you understand the Vanishing / Exploding Gradient Problem: Imagine that you are trying to find your way through a forest. You have a map, but the map is very old and the trails are not marked very well. As you walk through the forest, you make a lot of small decisions about which way to turn. These decisions are based on the gradients of the map. If the gradients are very large, you might make a big turn and end up very far away from your destination. If the gradients are very small, you might make a small turn and not make much progress. The Vanishing / Exploding Gradient Problem is like having a map with very large or very small gradients. In this case, it would be very difficult to find your way through the forest.

[@statquest]

bam

[@mermich]

You graced upon us as a stats saviour :))) Send love from Australia

[@statquest]

Awesome! Thank you!

[@alekseishkurin4590]

Omg, that intro jingle is gold!

[@statquest]

bam!

[@krishnaphanindra1841]

Beautiful and succinct explanations!! So glad I found your channel....lots of love

[@statquest]

Thanks!

[@TPLCompany]

Great video!! I can't wait for LSTM and transformer videos!

[@statquest]

Coming soon!

[@ClemensPutz-ist-der-beste]

I wish you were my math teacher! The whole class would have sang like you while calculating🤣

[@statquest]

That would be awesome!

[@kanisrini01]

Great video & explanation 👏🌟. Thank you so much.

[@statquest]

Glad it was helpful!

[@mohammedjaddoa9783]

appreciate your effort & work,THANK YOU

[@statquest]

Thank you!

[@zeljkobalanovic8900]

That intro was sick. I smashed like button immediatelly :D

[@statquest]

bam! :)

[@Naveedahmed-bq5iz]

Amazing, This is one best and coolest learning tutorial i have watched ever, great work Josh, keep it up. Thanks

[@statquest]

Thanks, will do!

[@FahimAhmed-gq4rh]

Awesome explaination by the creator ❤

[@statquest]

Thank you!

[@BlizzgamesAlive]

this videos just get better and better

[@statquest]

Thank you!

[@hasansoufan]

You're the best, thanks from the heart❤

[@statquest]

Thank you! :)