How Does Deep Learning Work? | Two Minute Papers #24

Artificial neural networks provide us incredibly powerful tools in machine learning that are useful for a variety of tasks ranging from image classification to voice translation. So what is all the deep learning rage about? The media seems to be all over the newest neural network research of the DeepMind company that was recently acquired by Google. They used neural networks to create algorithms that are able to play Atari games, learn them like a human would, eventually achieving superhuman performance.
Deep learning means that we use artificial neural network with multiple layers, making it even more powerful for more difficult tasks. These machine learning techniques proved to be useful for many tasks beyond image recognition: they also excel at weather predictions, breast cancer cell mitosis detection, brain image segmentation and toxicity prediction among many others.
In this episode, an intuitive explanation is given to show the inner workings of deep learning algorithms.
________________________
Original blog post by Christopher Olah (source of many images):
colah.github.io/posts/2014-03-...
You can train your own deep neural networks on Andrej Karpathy's website:
cs.stanford.edu/people/karpath...
Images used in this video:
Bunny by Tomi Tapio K (CC BY 2.0) - flic.kr/p/8EbcEk
Train by B4bees (CC BY 2.0) - flic.kr/p/6RzHe4
Train with bunny by Alyssa L. Miller (CC BY 2.0) - flic.kr/p/5WPeRN
The knot theory blackboard image was created by Clayton Shonkwiler (CC BY 2.0) flic.kr/p/64FYv
The tangled knot image was created by Mikael Hvidtfeldt Christensen (CC BY 2.0) flic.kr/p/beYG9D
The thumbnail image is a work of Duncan Hull (CC BY 2.0) - flic.kr/p/98qtJB
Subscribe if you would like to see more of these! - kzbin.info_c...
Splash screen/thumbnail design: Felícia Fehér - felicia.hu
Károly Zsolnai-Fehér's links:
Patreon → / twominutepapers
Facebook → / twominutepapers
Twitter → / karoly_zsolnai
Web → cg.tuwien.ac.at/~zsolnai/

[@manoelfernandotenorio9339]

In decades of teaching and research I found that this intuitive explanation of this episode to be the clearest demonstration of NN as a classifier.

[@TwoMinutePapers]

Heartwarming message of the day. Thanks very much for the kind words! :)

[@nrood5646]

I really admit this

[@TwoMinutePapers]

This episode has some pretty cool figures and animations (thanks for Christopher Olah!), but since this wasn't about pure computer graphics, the visuals are not as spectacular as in the fluid simulation papers. Did you still find it as interesting? Please let me know here in the comments section. :)

[@achakhar]

+Károly Zsolnai-Fehér (Two Minute Papers) i find the last demonstartion very interesting. Can i have a link of the demonstration to play with or can you give me the tools please

[@TwoMinutePapers]

It is available in the video description box. Here you go! cs.stanford.edu/people/karpathy/convnetjs/demo/classify2d.html

[@karotkiller]

As a non-scientific person, I found this video verry clear and interesting. Do you know where I can find other 3D representation of data, used for data sorting? Like the ones we see at the end (the intricated TOR and the spiral-like donut) thanks !

[@luldebehanger506]

You managed to capture one of the most breathtakingly beautiful ideas behind neural networks in a five minute video, and conveyed it with simplicity and elegance. Very well done.

[@BehindYouIsMe]

This is THE ACTUAL way that deep learning models works. This intuition is so powerful in understanding neural networks and architectures. If you don't view them like this, then you only have vague calculus with trial & error to work on and you'll be far behind. This vid is super underrated

[@gishgos]

This is absolutely fascinating. Everyone just normally shows the diagram of the neural networks and I never really understood what was going on, but the way you showed how the graph gets manipulated with subsequent layers made things much clearer. Thanks!

[@TwoMinutePapers]

+gishgos Hey there gishgos, very happy to hear that it helped to understand a bit better what is going on inside these neural networks. :)

[@anjishnu8643]

Came here from 3 blue 1 brown. This amazing visualization helped me understand the basic concept of ML really well!

[@nasirhaji5585]

you know your stuff when you can explain a complex subject it in the simplest form, for anyone, anywhere to understand AND appreciate. Kudos.

[@shreeyaksajjan1200]

One of the most difficult tasks is to take a complex subject and simplify it to where anyone can understand. And to do it 2 min? Your work is absolutely amazing, thank you! I can only imagine the effort put into each video. And to think you present papers from so many different fields, you are truly a inspiration.

[@AhmetSezginDuran]

Simply the best explanation of NNs and DNNs I've ever seen. Keep this awesomeness up!

[@TwoMinutePapers]

+Ahmet Sezgin Duran Thanks! :) Happy to hear you enjoyed it.

[@saifullah4127]

This is by far the best video of illustration of ANN I have seen on the internet.. keep making this stuff

[@TwoMinutePapers]

Thank you so much for the kind words!

[@CafeDelMaar]

Really crisp and to the point explanation.

[@MarkJay]

never seen these visualisation with deep networks. very cool video!

[@Haylomeni]

I have to say, this was one of the best, clear and easy to understand videos I've ever watched about this subject. Exactly what I searched for, very well done, thanks a ton!

[@TwoMinutePapers]

Thanks so much for the kind words and happy to have you in our growing club of Fellow Scholars! :)

[@jamesdunbar2386]

Still holds up as a great introductory explanation years later. Thank you!

[@glumb8968]

How could I have missed this awesome video for so long? As I mathematician, I really liked the explanation: "... homeomorphisms, which is a term that mathematicians like to use.". ;)

[@TwoMinutePapers]

;)

[@SaadTaameOfficial]

You wouldn't find such a description of Neural Nets normally. Great video !

[@cepi24]

I've just discovered this channel and and I have to say big thank you for your work. Man you are doing amazing stuff, I could not imagine how much effort have you put inside each video but the result is 10000000x better.

[@TwoMinutePapers]

Thanks so much, really happy to hear that you liked the series! :)

[@RelatedGiraffe]

Wow, your videos are really great and professional! I thought this was gonna be all stuff I already knew since it was so short and just an introduction, but you managed to stick some parts in there which I had not previously seen, like those visualizations and that thing about knot theory. Great work!

[@TwoMinutePapers]

+RelatedGiraffe Thanks so much for the kind words. :) It's great to hear you're enjoying the channel and I'm very happy to have you around!

[@phamvinh5424]

It's amazing. Thank you.

[@waelhussein4606]

Simply the best simple explanation for neural networks and deep learning. Thank you very much for this very nice and outstanding piece of work!

[@onetouchtwo]

Very much appreciate such a clear and direct explanation on deep learning.

[@TwoMinutePapers]

+Mark Canlas Thanks so much for watching the videos, happy to have you around! :)

[@covertimpala]

Definitely the best explanations I have ever seen for NN

[@dr.michaelr.alvers17]

Very very well done! That is the way students can get difficult matter in minutes. Thanks for this ...

[@TwoMinutePapers]

+Dr. Michael R. Alvers Thanks for watching! :)

[@lc11s]

Awesome explanation - all the vector and linear algebra makes sense now - thank you ;)

[@MrNimbus420]

Maybe the best video on youtube. I watch 4:30 hours a day.

[@aonoymousandy7467]

Thanks a lot for this video, you are the only one who actually provides interesting presentations with clear explanations

[@Noone-of-your-Business]

Honestly, I still don't understand diddly squat. But I am glad that there are people out there who do. Just make sure that SkyNet has a sense of humor before flipping the switch, please.

[@GaryMcKinnonUFO]

Good stuff, liked and subbed. I was heavily into NNs, GAs, classifiers etc many years ago, i think it's time to jump back in :)

[@TwoMinutePapers]

Thank you very much for the kind words, happy to hear you've been enjoying the series! :)

[@raghudeep6555]

Thank you for explaining it beautifully.

[@a9raag]

That is the best explanation I have ever heard for NN. Thank you

[@akhilkumar8233]

the best one I've seen till now, really cool! thanks sir!

[@TwoMinutePapers]

+Akhil Kumar Thanks for watching. Happy to have you around! :)

[@Sherlockarim]

what a great channel! keep up the good work man.

[@jach8952]

Awesome Explanation!! Thanks for the video

[@ervins775]

Karoly I have to tell you, I can barely follow the topics you present, as I am neither privately nor through my job involved in any kind of mathematics, physics or computer science. But the way you present that stuff is just brilliant. And your accent is really likeable. Keep up the good work

[@TwoMinutePapers]

You are very kind, thank you so much!

[@diete103]

I thought it was very interesting and the animations really helped eith understanding! Also, loved the sound of your mind being blown. Pretty much the same sound I make.

[@TwoMinutePapers]

+diete103 Thank you! Lots of more mind blowing stuff to come very soon. :)

[@ghanshyamsahu4375]

Best explanation ever.just Amazing

[@nik4799]

Great work!

[@Funzelwicht]

Amazing explanation and visualization, thank you!

[@TwoMinutePapers]

Happy to hear you've enjoyed it. Thanks for watching! :)

[@sandeepsrikonda7352]

Awesome visualisations.Really helped a lot in understanding the NNs.Can u do some more videos explaining what kind of models can a NN take care of and what it depends on?

[@luis96xd]

Wow, excellent video! I learned a bit more of Deep Learning, I saw a linear transformation, visually, in data classification

[@lmotaribeiro]

Really like your explanation about these stuff. Keep up with the good work :)

[@TwoMinutePapers]

+Lucas Mota Hey there Lucas. Thanks and welcome to our club of Fellow Scholars! :)

[@AndrewCarlson005]

Love this stuff! Reminds me of 3blue1brown.

[@Kram1032]

Great to see this :) By the way, on Olah's blog even the comments occasionally hold a gem worthy to be checked out. Some people link interesting papers down there. That being said, those papers probably aren't typically well suited for this channel. They often are abstract and not very visual. Still, if you are interested in digging deeper, this is highly recommended. And maybe there even are one or two papers that actually are visual enough for this channel hidden in there.

[@TwoMinutePapers]

+Kram1032 I remember reading the comments there before. Definitely some of the highest level discussion in a comment section I've seen in a while. :) Regarding the appealing visual content, it's always a good question. This was one of the first episodes with a bit less visuals, and I'm quite interested in the reception. It seems to be positive so far! :)

[@Kram1032]

Károly Zsolnai-Fehér I think the visuals of distorting a space to separate regions in it with a line worked well enough :)

[@aamir122a]

hats off, you have done a great job. My question is do deep neural networks transform space or transforms data. From what I gather they transform space.

[@araelnllamas183]

this is truly amazing

[@suchitrabasak8054]

OMG This was so beautiful! Thank you!

[@suchitrabasak8054]

It was perfectly concise

[@TwoMinutePapers]

+Suchitra Basak Thanks for watching. Very happy to hear that you also liked it! :)

[@psy901]

wow... fascinating channel. Thanks for the clearest explanation ever..

[@TwoMinutePapers]

Thank you for watching and for the kind words!

[@ntcool123]

These videos are brilliant!!

[@TwoMinutePapers]

Thanks for the kind words, happy to hear you're enjoying them! :)

[@jayeshrathod94]

Nice explanation

[@M0481]

This was amazing! I am a new subscriber and I found this video very interesting!

[@TwoMinutePapers]

Excellent, thank you very much and happy to have you in the club! :)

[@gc4152]

I'm not a "scholar" but I find it fascinating, thanks!

[@TwoMinutePapers]

+Guillaume Combot Hey there Guillaume. Are you watching Two Minute Papers regularly? If yes, then you are! :) Happy to have you around!

[@wavecutter69]

You are absolutely amazing!!! I am joining your channel.. This was an amazing video

[@TwoMinutePapers]

You are too kind, thank you so much for the kind words and for your generous support Robert! 🙏

[@beepboopgpt1439]

Whaaa' don't leave yet, i want to learn more!

[@shabeer821]

Concise yet very meaningful. Keep up the great work and useful work. Subscribing to your channel.

[@TwoMinutePapers]

Thank you, happy to have you in our growing club of Fellow Scholars! :)

[@attentiondeficitdisorder]

I think now it's more common to use non-linear (Rectified Linear) activation functions. It no longer needs to be a straight line and the line can warp to fit to the data.

[@larryng1]

that. was. awesome!

[@ahmetgokdayi7854]

Thanks, keep up the good work :)

[@TwoMinutePapers]

+Ahmet Gokdayi Thanks for watching Ahmet! :)

[@RawanLaz]

So many new informations 😮😮😮

[@danc3229]

It's pretty good in that it avoids getting lost in too much depth (pun intended) but it severely lacks in that the only explanation for deep learning is that it involves neural nets with a number of hidden layers. Which doesn't differ from basic feed forward supervised learning neural nets.

[@spandanhetfield]

Epic stuff. Keep it up :)

[@TwoMinutePapers]

+Spandan Madan Thanks. Lots of cool new stuff to come soon! :)

[@karotkiller]

As a non-scientific person, I found this video verry clear and interesting. Do you know where I can find other 3D representation of data, used for data sorting? Like the ones we see at the end (the intricated TOR and the spiral-like donut) thanks !

[@divakarramachandramurthi2849]

Really the best and simple explanation sir.. I am wondering can we apply deep learning( concept of warp of dimensions )to the problem of string theory which mainly deals with the multi dimensional objects like torus..knots...

[@karagiozhs]

BRAVO!

[@MrZ1234]

Great video

[@TwoMinutePapers]

Thanks for watching! :)

[@arjunpukale3310]

Please make a video on universal approximation theorem of Deep neural networks

[@epistemocrat]

great indeed thanks! I subscribe! just one passage I do not have clear: why it must draw a straight line? can't it draw a curved line in the first place or a circle in the bull eye example?

[@myAutoGen]

Hey, really interesting video. Could you recommend any resources for learning more about the mathematics surrounding neural networks?

[@marc6775]

Great video. Can you please clarify what you mean at 5:00 with 'increasing the number of neurons'? I understand that if you increase the dimensions, you will be able to warp and manipulate your graph to find a suitable place for your straight line, and then scale back down to original dimension, but what do you mean by increasing the neurons? I'm also assuming by layers you mean dimensions.

[@camillembiakob4862]

I have been looking for easy ways to understand Deep learning for a class project, since we only working on Multi Layers Neural networks. And i find your video easy to understand. but now i am thirsty for more. Please advise with resources or links i can use to gain deep knowledge on this topic. Thanks in advance

[@TwoMinutePapers]

+Camille Mbiakob No problem Camille, we got you covered! There is this video on exactly this topic, hope it will help. Note that there are tons of materials in the video description box there as well. - kzbin.info/www/bejne/apmTpnZve7WJg7M

[@secondhandmemory]

In terms of the classification problem, is there an advantage of using this over the K nearest neighbors algorithm?

[@bluedragonflyish]

Brilliant presentation! I am (K)not kidding :D

[@TwoMinutePapers]

Thanks for the kind words. Very happy to hear that you liked it! :)

[@FindMultiBagger]

Great

[@MM-fv1pi]

What is that software u used to create NN with such convinient and fast results?

[@transhumanistmemes9131]

If we can solve a "2d entanglement" using 3 dimensions, could we solve a 3D entanglement using 4 dimensions?

[@shin-ishikiri-no]

Good observation.

[@NM-jq3sv]

How exactly the data be if images when represented as points in higher dimensional space looks like a knot.Awesome.

[@zagyex]

Great video, thanks. I would be really thankful if you could share your opinion on something: Does this at all aimed to represent the inner workings of the human brain? I have an example: I have a 2-year old and it is enough to show her an image of a crocodile once or any new object, lets say a crane ONCE. She then basically recognizes ALL crocodiles and cranes if she sees. So it is not about comparing lots of images and having a more sharp line between objects but feels like understanding the basic concept of a crocodile. It may sound a bit platonic, but hopefully get what I mean. She doesn’t need a lot of data to draw a line between the two objects. In extreme cases she needs only one example to recognize that type of object. I am thinking of this a lot, but i may totally miss the point as a laic. Köszönöm!

[@TwoMinutePapers]

+zagyex Hey there zagyex, you are indeed right. A human can look at two mugs and immediately learn the concept of a mug. A deep learning system requires the very least thousands of samples to achieve the same feat. We usually say the term that it "kind of" simulates the inner workings of the human brain. It is not even close to efficient as the brain is, but it is doing something that is somewhat similar. There is a great example of this here: kzbin.info/www/bejne/qHS6hYKFnLuLr6c The human brain part is important because all the computer sees on an image is pixels. It does not have depth perception or any kind of morphological sense like our brain does. Hence, as the first step in creating a program that can recognize images, people tried to reproduce what our brain does - this is the only example we know of. After all, this is what we use our brains to see things around us. :)

[@zagyex]

+Károly Zsolnai-Fehér (Two Minute Papers) Thanks for the answer. If we would like to estimate the future of deep learning neural networks, is there any trend pointing towards the ability to recognize images with a minimal set of samples? I mean: will a very well developed form of todays deep learning algorithms be able to recognize a mug after it sees it once or only a few times? Is the number of samples needed decreasing as these algorithms develop? Or may it be that it will never be possible to do it this way and we need a whole different approach (and deep learning will be just a small part of the solution? At least for me as a laic it seems that what these neural networks do is usually approximating using big amounts of data, and getting better all the time, but does this mean they will work well with "little data" like seeing a mug once?

[@LabGecko]

It is also important to remember that when training a human, it has already been trained in things like mugs and cranes. When training a deep mind (as I understand it), it is starting from embryonic stage.

[@zagyex]

I was thinking of objects and drawings a human never seen before, but recognizes all of them by observing only one example. It is true however that humans have seen many many objects before that particular object.

[@dospy1]

you are missing one point tho. You may have given only one example to the 2-year old, HOWEVER, i suppose one example in 3D (basically seeing the mug from different perspectives/angles etc.) accounts for lack of other examples whereas a neural network fed with only one picture of course can't compete with that.

[@dopaminecircuit]

Mind.....blown. Wait, and this was 4 years ago??? I'm not sure, but I either just sh@t a brick or a 3rd testicle just descended...

[@a.s.9145]

wow, so it was all homotopy topological morphisms down all the way to the turtles

[@blackdedo93]

but how does it decide from all of the edges that a combination of edges make a certain face, even u let it learn for 10000000 years it wont know that thats me unless i told it at least once am i right ?? what i mean is the out put already known we just want to filter to the output ! correct me if am wrong please

[@atomnous]

give it a goal, and it'll learn something. (eg: we as a babies need to recognize faces in order to know where to ask for foods)

[@joshuarawls2613]

SOMEONE HELP!!!! 😭 ... I'm the type of person who (litterally) struggles 4 attempts per math course! I'm trying to make sense of NN's & Deep Learning Algorithms so I can survive the A.I era ... HELP!!!!!!😭😭

[@dr.michaelr.alvers17]

One minor thing: may be a (better) motivation should be given, why a straight "line" is necessary in the first place.

[@TwoMinutePapers]

+Dr. Michael R. Alvers Agreed. This was as much information as I could cram in there in this amount of time without making it tedious. Maybe in a followup video! :)

[@dr.michaelr.alvers17]

But seriously ... why. I never thought about it how to motivate ... baffled ...

[@isectoid9454]

Did you ever make that follow up video?

[@shin-ishikiri-no]

Straight line is probably easier to deal with mathematically. There may be better or more efficient options out there, but they are possibly impractical to apply for mere mortals like us.

[@Invalid571]

* Insert mind being blown sound here. *

[@mrvk699]

This is so concerning , how Artificial Intelligence - Deep learning etc. can start mimic human :/ The Danger to human species from A.I. depicted in Hollywood Movies is not too far .

[@nullnull6032]

We gonna fail CVMA UofG #glasgow

[@mzkabn]

[@ngpeehock]

?

[@ksztyrix]

Absolutely heretical