How to Do Mathematics Easily - Intro to Deep Learning #4

Let's learn about some key math concepts behind deep learning shall we? We'll build a 3 layer neural network and dive into some key concepts that makes deep learning give us such incredible results.
Coding challenge for this video:
github.com/llSourcell/how_to_...
Jovian's Winning Code:
github.com/jovianlin/siraj-in...
Vishal's Runner up Code:
github.com/erilyth/DeepLearni...
Linear Algebra cheatsheet:
www.souravsengupta.com/cds2016...
Calculus cheatsheet:
tutorial.math.lamar.edu/pdf/Ca...
Statistics cheatsheet:
web.mit.edu/~csvoss/Public/usa...
And if you have never had experience with any of these 3 and want to learn from absolute scratch, I'd recommend the respective KhanAcademy courses:
www.khanacademy.org/math
More Learning Resources:
people.ucsc.edu/~praman1/stat...
www.vision.jhu.edu/tutorials/I...
datascience.ibm.com/blog/the-m...
Join us in our Slack channel:
wizards.herokuapp.com/
And Part I of this book is so dope, seriously:
www.deeplearningbook.org/
Please subscribe! And like. And comment. That's what keeps me going.
And please support me on Patreon:
www.patreon.com/user?u=3191693
Follow me:
Twitter: / sirajraval
Facebook: / sirajology Instagram: / sirajraval Instagram: / sirajraval
Signup for my newsletter for exciting updates in the field of AI:
goo.gl/FZzJ5w
Hit the Join button above to sign up to become a member of my channel for access to exclusive content! Join my AI community: chatgptschool.io/ Sign up for my AI Sports betting Bot, WagerGPT! (500 spots available):
www.wagergpt.co

[@offchan]

4:32 That's not a normal distribution. It's a uniform one. A normal distribution should have bell curve as its probability density function instead of a rectangle-like function.

[@SirajRaval]

ur right i'll annotate thx

[@jacobsanchez1587]

I came for the Math. I stayed for the Memes.

[@SirajRaval]

yesss

[@dharmendragurve9468]

I like the "Sanskari James bond" meme

[@jimmy4282]

That part about learning gradients on Friday night hit me right in the feels

[@SirajRaval]

i didn't mean for it to put anyone down, it was just a weirdly relevant meme i found for gradient descent. For the record, i spend many weekend nights learning. No shame at all in that, better than getting wasted!

[@tinkeringengr]

Smart people be learning on the weekends :)

[@akinsamuel2007]

Siraj you are brilliant! You have truly found your calling. You explain things clearly in an amusing and dynamic way - and your singing is 😂 I've not met anyone make maths so engaging. Thank you!

[@eriksundell1400]

Siraj - the work you do is very meaningful to me - you help me as a math teacher make it clear to my students why math is awesome and abolish questions like "why should we learn math?". Keep it up!

[@SussyBacca]

Holy. Mother. Fucking. Shit. You just unlocked deep learning to people who don't even know shit about math... are you Jesus?!?! How are you performing these miracles?!? Remember me when you get to the promise land :D

[@karius85]

Yeah if promise land then remember generic guy who did nothing! This is definite success!

[@SussyBacca]

karius85 This is a Jesus reference from the bible... chill, heathen ;)

[@SirajRaval]

lol this is my all time fav comment thanks

[@DuhBroadcaster]

Siraj, I had to brush up on my calculus.. but from what I understand: x * ( 1 - x) is not the derivative to 1 / ( 1 + e ^ -x ). Can you explain what's going on here?

[@art9phy]

If s(x) = 1 / ( 1 + e ^ -x ), then ds/dx = s(x) * (1 - s(x)). He's calculating s' by using s, not x directly.

[@umar11ism]

I got it. It involved numbers

[@SirajRaval]

yup

[@gregisaacson660]

Your an absolute life saver! I gave been searching for extra maths teachers and I couldn't find any decent tutorial videos! Thanks

[@Xaminn]

Dude... The humor and memes in your videos are on point and SO educational. Please keep making these!

[@johnhammer8668]

This episode is for bottom up people. I really liked this format. This gives the intuition about deep learning. Thanks so much.

[@iasoto]

You have no idea how I'm enjoying your videos. Thanks man, you're amazing

[@sudharsanravichandiran3461]

My God! You just made math so easy with great intuition. This is more than awesome. Can't thank you more.

[@50quid11]

This is the one video on you tube that changes everything!

[@cellularmitosis2]

You are my favorite youtuber of all time

[@SarcSaus]

I’m phenomenally bad at maths, no content has ever explained how this works, as well as broken down the areas of study I need. Thank you!

[@johnhammer8668]

So entertaining with memes and educational at the same time. Considering the video is one guys work is damn impressive.

[@FranklinVanNes]

Your commitment to helping people learn is phenomenal! This is fantastic. Keep up the good work

[@SirajRaval]

thx Franklin

[@TheCypherwars]

I cannot believe how much effort you put into this video, all of them really ! Thank you Siraj :)

[@TheAlderFalder]

Your videos are a great summary of the course Machine Learning from Prof. Ng, which I'm currently taking. They give me back the quick overview, when I need it.

[@samha1513]

I like how fast and to the point you are and not wasting time bit I wish I understand what is all this

[@mason6662006]

Love what you're doing dude. coming from an investigatory background, I have applied some of the methods you have outlined and the results look ever more promising! keep delivering the message!

[@SirajRaval]

thanks!

[@smellybathroom]

I learn so much from you. probably going to change my life watching your channel.

[@JasonBunnell]

Equally funny and interesting. I have always called tensors cubes while realizing it was not a perfect description because more than three arrays or dimensions. Very cool.

[@ammar0466]

everytime I just enjoyed your video without knowing what to do with what I watched, just like how you put together all information and memes and rap music and your face, thanks for making this video

[@party10277]

been waiting for this video for a long time........... as always great work Siraj

[@kirill4531]

great links in description for learning deep learning math from basic level! thank you

[@bikashprasad6446]

Everything related to ML in one channel.. Thank you Siraj..

[@SirajRaval]

thanks Bikash

[@Fcpuru95]

" backpropogate to get cost corrected".. loved it

[@Nealpa]

Agree with audience. You definitely cleared small confusions. Thanks...

[@SirajRaval]

thanks Neal

[@menzithesonofhopehlope7201]

Indeed, it's Friday and I'm sitting here learning about Gradient Descent. (Well, technically it's Saturday morning from where I'm sitting.) Thanks Siraj.

[@shaminmohammed6157]

Man... U r awesome...we should support you....

[@colocolo49]

Siraj, you bloody rock.

[@berathan90]

So excited about new vid :)

[@abhisheksinghchauhan6115]

Good topic !!! Siraj bro , i need this too much and finally i gotted, thank you so much....

[@pocketman5510]

Wow. I actually have a good idea on what's going on with the maths now. Thank you Siraj!

[@SirajRaval]

yessss. thanks Pocket man!

[@mehmetedex]

your videos are better than most videos on the net

[@SirajRaval]

thanks Mehmet!

[@guiray2000]

Super good tutorials and very entertaining and clear.

[@nederlandas]

7:35 - Best meme I've ever seen

[@YEASTY_COMMIE]

most accurate meme I've ever seen

[@theghostmachine]

Literally me right now

[@maheshkanojiya4858]

absolutely amazing video , thanks and keep making more

[@NoOffenseAnimation]

This really helped me understand neural networks. Thanks!

[@Dylan-qk8ss]

I loved the intro!

[@tobias2688]

I want to give one million thumbs up! Very well done!

[@SirajRaval]

thx tobias

[@MrTk3435]

Watch it for the 3rd time, You Rock Siraj!!

[@hiteshvaidya3331]

This is actually the best way to teach. Instead of directly jumping to code, this approach gives a clear idea of the concept. You had created videos on building NN before as well, however this is better than all of them. So hereon please explain theory along with code (like this video) which will give clear understanding. PEACEmax

[@SirajRaval]

thanks Hitesh u r a tru fan

[@Ant-xz6he]

Oh wow, the cheatsheets you provided are legit helpful. I was looking for something like that for quite some time (especially for lin. algebra). Thanks!

[@SirajRaval]

woot

[@SkynerGaming]

I just love this guy videos

[@Hustada]

Thanks for this Siraj. Helps a lot. The most challenging part for me at this point is my rustiness in math.

[@SirajRaval]

practice makes perfect thanks

[@UsmanAhmed-sq9bl]

awesome as always Siraj

[@SirajRaval]

thanks Usman

[@tuxmode]

That flow is sick, bruh. You should make an album of it.

[@iamshridhar1]

Awesome video as always!

[@swapanjain892]

finally,I was waiting for this..

[@gspitz01]

I really enjoy the musical learning stuff. Tapping into more sense memory. It's cheesy for sure, but I'm also sure it helps people remember. Well done, sir.

[@SirajRaval]

thanks Greg

[@dannyiskandar]

my brain wants to explode ...man ...i understand this now ..awesome!

[@SirajRaval]

woooooo nice

[@spenhouet]

Really good explanations in there! Good work!

[@SirajRaval]

thanks Spen!

[@silleetram]

YAY! just what i needed!

[@offchan]

7:14 When Gradient is 0 that doesn't mean the error is 0. It doesn't guarantee anything. You can be at a top of a parabola curve that points upward when the gradient is 0. Or you can be at the bottom of the bowl, but it still just says that the error is minimized, it's not going to be 0 unless the network fits the data perfectly (which is impossible because of the sigmoid that squeezes output to never be equal to 0 or 1). The dub is correct though, but the image is not.

[@CrashPreinsertion]

Hurmm,.. is it possible for a parabola that looks like an upside-down "U" or a lower-case "n" to show up as a representation of possible error values? I don't believe so, because the possible values are converted into a sigmoid, so I believe the graph of possible error values will always look like a "U" parabola. Also, I think the odds of a perfect 0 (at the top of that parabola) are impossible (as you state in your second point). Eventually gradient descent would make the error start to move down that hill & inch closer to a valley & toward an error of zero. And as for your second point, yes, error checking will never end up at a true zero. It will eternally search for the lowest error. Why? Because when we look at a graph of a sigmoid, everyone should remember that the lines continue out to infinity. However, as more epochs are applied, the probability becomes lower and lower. It never touches zero, but continually drifts toward it. David Hume's problem of induction still holds. :)

[@krazy8903]

THE FIRST EVER VIDEO where I needed to play it in half the speed to get the smallest grip. Damn MATHS!

[@arsnakehert]

This is better than expected, you should rename it "what math you need to know for neural networks"

[@AbangJalan2]

Best.Intro.Ever

[@chukidadiz]

Good intro for neural network keepit up brother.

[@kingpopaul]

Pretty good basic intro, for people that are more into being bombarded by formulas, I highly recommend Hinton's course about neural network on Coursera. Maybe it's just my math brain that's rusty or my eternal disdain for calculus, but this course is pretty hardcore.

[@SirajRaval]

good call

[@iamdurgeshk]

you're the coolest teacher I have ever seen. Man, your memes are right to the point. and thanks for the video. helped a lot.!!!! Could you start a series telling about A to Z of Data Science with Python and Deep Learning?

[@SirajRaval]

thanks! i have one see learn python for data science

[@ricardofilipegomes4396]

u are amazing sir Siraj Raval pahah really wanting to get your course in udacity when i got the money! :D

[@SirajRaval]

thanks Ricardo

[@alynnem0z]

going to offer it again??.. at lower cost would be super awesome. anyway slow down the video .75 instead of toooo slow .5?

[@TobiasWeg]

Thank you for this nice introduction! makes things much clearer. One note so, an integer is not proper python variable. So 11 = 'something' will most likely produce a syntax error! you can use _11 if you like. As, I see u corrected this in the Github link;)

[@BigAsciiHappyStar]

I once wrote and performed a song poking fun at a number of Australian politicians, using Richard Clayderman's "Les Premiers Sourires De Vanessa" as Karaoke-style backing, but Mozart's Symphony in G minor K550 is taking it to the next level :D Siraj literally radiates awesomeness and makes me look like a beginner

[@harshitaarora6319]

Amazing videos! You're a great teacher! :D

[@Councildude]

intense, awesome.. unlocked for me .. thanks this has started the path ... you have an amazing gift with your offbeat/hip easy to remember fun way of conveying this information... definitely on the god scale :-> ... thanks for this!

[@SirajRaval]

thanks Byron!

[@DanielWeikert]

I simply do not understand how anyone could rate this negative. This is awesome content and the explanation is perfect. Please keep going Siraj, especially with these kind of in depth explanation videos. They are perfect for newjoiners like me. Maybe you could also do this for the other kind of neural networks. I also would highly appreciate a session, where we take a look at really "messy" data where we need to put a lot of effort to prepare the data properly. (e.g. Scaling, outlier removal,...)

[@SirajRaval]

thanks so much, more to come

[@randyswagmonster499]

Not sure how helpful this would be for someone not familiar with the process already, but still, nice video! I appreciate all the work you put into the channel!

[@SirajRaval]

thanks Randy

[@ardianmoreless1462]

Siraj you are great! :)

[@larryteslaspacexboringlawr739]

thank you for deep learning video

[@star-dj3xf]

This is a great video!! Can you do another with more details and backup info on each of the components? The math gets lost easily and no one really puts it all together the way you do :)

[@sarahhsu3857]

Mozart no.40

[@justinfrancis5834]

This is probably one of your best videos imho! Should do a video on derivatives!

[@SirajRaval]

thx Justin

[@AncoraImparoPiper]

Utterly. Awesome.

[@harperjmusic]

Great video!

[@ruskodudesko9679]

Linear algebra, statistics, and calculus? My three favorites lol - although I am only just learning lin now, and only took lower division calculus - I suppose I should look up the proofs that said "the proof for this is outside the scope of this course", because advanced calculus basically just does all those proofs.

[@USBEN.]

I don't understand everything in this , but i oddly enjoy watching these videos .

[@666aron]

You sir, are an amazing and amusing teacher, which is rare noways. Thanks for another awesome video. By the way... some insights on creating/training/using convolutional or wavelet neural networks would be really nice.

[@SirajRaval]

thanks this will come

[@1mrGhost]

Why could this video not come out last week? I have just finished a weeks vacation... this could have been a fun thing to spend my vacation on.

[@c0mpuipf]

this is how it went for me: memes -> formal naming of stuff in math -> gibberish What if you'd do a video about each of the math concepts behind it and how it applies to DL (especially tensors, there isn't anything about that on khan academy yet)

[@zangarkhan]

Your epic and a godsend! TY

[@dimabreezy2000]

By the way, if you wanna test this neural net, use this function def think(input, syn0, syn1): return nonlin(np.dot(np.dot(input, syn0), syn1)) print(int(think(np.array([]), syn0, syn1)))

[@WilsonMar1]

I noticed that the mouse moves up to click the icon to run. An alternative is to press Shift+Enter to run.

[@chinmaydas4053]

u r boss siraj...

[@zeusgagarin7089]

if only my instructors could teach like you then maybe I wouldn't have any problem passing a test.

[@thizzlefurly]

Very useful and the memes provided are hilarious.

[@SirajRaval]

thx

[@v8metal]

siraj is a cool guy. thanks for the vid

[@SirajRaval]

thanks BSS

[@fcarriedo]

Man. You're good.

[@evrider9063]

so amazing, thanks!!!

[@SirajRaval]

thanks!

[@AkashMishra23]

This is Amazing, I so Needed this right now, I'm good with Linear Algebra, Statistics and Calculus but sometimes Multivariable Definite Integrals can be a Big Headache....

[@AkashMishra23]

Hamchun Kou actually, I watch siraj's videos for some quick DL tricks, and I've taken University level Maths in high school the ones you get college credit for, sometimes Multivariable Definite Integrals give me a hard time but other than that I'm pretty well with Maths, and I'm just trying to encourage people that's all....

[@AkashMishra23]

Hamchun Kou I didn't list anything that's in my syllabus and knowing the name of a Topic and Understanding them to their depths are two very different things.

[@drnesr]

Thank you Siraj fir the nice descriptions I advice not use the small letter L in the programs you show, as it looks very similar to number 1, so it may be conflicting to read l0, l1, or l2, as could be read 10, 11, and 12 respectively. You can use capital L or the letter r for layer instead.

[@SirajRaval]

good point thx

[@griesrt]

Do you ever hold Machine Learning Meetups in SF? Also, I haven't checked yet but do you have any recommendations on where to look for the latest in content/CF recommendation engines? The best paper I've found is the "Collaborative filtering for implicit feedback data sets" paper written by Koren. I'm very interested in a paper which factors in negative implicit interactions. Great videos!

[@kartiksibal]

I love how you gradually increase the learning curve, from absolute zero to the peak. Amazing work! PS: What do you think about, how in depth, should one go into the maths of different ML algorithms. Not for research, but applied purposes. Thanks :-)

[@SirajRaval]

thank you! for production grade code, doing intro courses in the 3 subjects will be sufficient

[@kartiksibal]

+Siraj Raval thank you :-)

[@umairrehman4299]

Can you make a video on Kriging (KG) Method for parameter estimation?

[@danielketterer9683]

When you sung the mechanism of a neural network in a few verses I lost my shit. Seriously funny and informative

[@YoulooDingDongKenth]

What nodepad program are you using?

[@wengeance8962]

Hi, whats a good way to start learning Calculus/Linear Algebra and Statistics as someone who has not learned this math before? Or computer science fundamentals

[@atrumluminarium]

What I have trouble with is the choice of activation function. Why is the logistic function preferred over other sigmoids such as transformations of tanh(x), arctan(x), etc. ?

[@anujbeatles]

Fuck yes! IPython! Thank you, Siraj!