Neural Networks from Scratch - P.4 Batches, Layers, and Objects

Neural Networks from Scratch book: nnfs.io
NNFSiX Github: github.com/Sentdex/NNfSiX
Playlist for this series: • Neural Networks from S...
Neural Networks IN Scratch (the programming language): • Neural Networks in Scr...
Python 3 basics: pythonprogramming.net/introdu...
Intermediate Python (w/ OOP): pythonprogramming.net/introdu...
Mug link for fellow mug aficionados: amzn.to/2KFwsWn
Channel membership: / @sentdex
Discord: / discord
Support the content: pythonprogramming.net/support...
Twitter: / sentdex
Instagram: / sentdex
Facebook: / pythonprogramming.net
Twitch: / sentdex
#nnfs #python #neuralnetworks

[@Yguy]

I swear I am addicted to these more than Netflix.

[@MasterofPlay7]

lol you are weird xD

[@usamanadeem7974]

I literally wait for KZbin’s notification telling me Sentdex dropped a new tutorial 😂

[@ossamaganne5851]

@@usamanadeem7974 me too brother ahahahah

[@NicolasKaniak]

same here

[@Blahcub]

cringe

[@AaditDoshi]

I don't even look at my calendar anymore. My week ends when sentdex drops a video.

[@akiratoriyama1320]

My week starts when he drops video!!

[@pushkarajpalnitkar1695]

@@akiratoriyama1320 countdown starts today for the next video

[@josephastrahan6403]

Agreed, I'm waiting patiently also :)

[@shauryapatel8372]

i think you might have to make new days cuz it is taking 2 weeks for his next vid

[@soroushe6394]

I’m glad I’m living at a time that people like you share their knowledge in such quality for free. Thank you 🙏🏻

[@slamsandwich19]

I was going to say the same thing

[@MeinGoogleAccount]

yes, absolutely. i come from a time where programming meant you buy a book that acually was outdated the moment you bought it. thank you 🙂

[@sayanguha5570]

Everytime I see a neural network tutorial they start as "import tensorflow as tf" without giving a shit about basic..but this is a very detailed basic clearing video, truly from scratch...THANK YOU FOR THE GOOD WORK

[@lucygaming9726]

I agree with you, although you can check out Deeplearning.ai on Coursera. It's pretty good.

[@aleksszukovskis2074]

@@lucygaming9726 No thanks . Im too poor for that.

[@janzugic6798]

@@aleksszukovskis2074 its free and by andrew ng, the legend

[@aleksszukovskis2074]

@@janzugic6798 thanks

[@supernova6553]

@@janzugic6798 you need a coursera subscription ($49/mo) after 7 day trial period regardless of the course being free

[@sentdex]

Errata: 16:17: initially this anim was incorrect when I recorded. We fixed the anim, but not the audio, resulting in my reading of the incorrect first row of values incorrectly. We're adding row vectors here, so the anim is correct, the words are not. =]

[@usejasiri]

Please clarify the concept of the Gaussian Distribution that you introduced when talking about np.randn

[@anjali7778]

if i draw a neural network of 12 inputs imaging into 3 output and connect each neurons to the output, there will be 36 lines in total, that means there has to be about 36 weights but the weight you took had only 12 weights in array, how is that possible ?

[@mayaankashok2604]

@@anjali7778 He has only 4 inputs to the output layer... therefore number of weights = 4*3 = 12 If instead , you have 12 inputs ,you will get 12*3 = 36 weights

[@fincrazydragon]

Am I wrong, or is there something missing around the 9:08 point?

[@dragonborn7152]

Question: why did we need to transpose weights 2 since they are both 3x3 matrices, index1 of the would equal index 0 right?

[@Blendersky2]

Just imagine if we have tutorials like these on all the AI and Machine learning topics and also on probability and statistics. .. man, every few minutes in the video I try to scroll the video list up and down with the hope that there will be 700 more videos like these but it shows only 7 videos. Amazing work, I will order your book now. Appreciate your dedication and hard work

[@amogh3275]

Bruh this visualisation... Its unreal🔥

[@Saletroo]

ASMR for eyes, thanks Daniel!

[@amogh3275]

@@Saletroo ikr 😂😂

[@aoof6742]

I really appreciate you doing this mate, I really wanted to learn Neural Networks and you are explaining this soo good.

[@sentdex]

Glad to hear it!

[@ramyosama8088]

Please continue with this playlist This is hands down the best series on youtube right now !!!

[@sentdex]

No plans to stop any time soon!

[@knowit3887]

U r just ... God for teaching programming... I am glad to have u as a teacher... 💪

[@prathamkirank]

This is the online classes we all deserve

[@theoutlet9300]

better than most ivy league schools

[@prathamprasoon2535]

This is awesome! Finally, a series on neural nets I can understand easily.

[@usamanadeem7974]

The thing I love about you is just how beautifully you explain concepts, with immaculate animations and then literally make such complex tasks seem so easy! Gonna make my kids watch your tutorials instead of cartoons one day ♥️😂

[@kaustubhkulkarni]

I’d kind of given up on understanding ML and NN. Then I saw Neural Networks from scratch and Sentdex CANNOT make this easier. Loving this series.

[@7Trident3]

I banged my head on numerous videos too. They assume a level of knowledge that was hard to peice together. This series is filling lots of gaps for me. The concepts are starting to jell, this whole field is fascinating!! Kind of empowering.

[@asdfasdfasdf383]

You have created one of the best series on this topic I have found on the internet. Explanations include everything, yet you still proceed at a fast steady pace.

[@Gorlung]

this is actually the first NN tutorial during which I haven't felt asleep.. ps. thank you for explaining some of the things twice!

[@ambarishkapil8004]

I know I have said this before, but I am going to say it again, and keep on saying it till you continue to make such awesome tutorials. Thank you!

[@kenbinner]

I'm really glad you took the time to break down this concept step by step, will surely reduce the number of headaches in the future! Thank you for your great content looking forward to the next one. 😄

[@carlossegura403]

Back when I was learning the concepts behind building a network, most tutorials went straight into the maths, while that is fine - what I wanted to understand was the different compositions from the input to the output. This video was what I was looking for back then before going deep into the theory and methodology. Great content!

[@codiersklave]

Still one of the best series on KZbin to learn the basics of neural networks... fast!

[@bas_kar_na_yar]

I wish anyone had ever taught me any concept the way you do..

[@keshavtulsyan7515]

Feels like learning all the day, it never felt so simple before...thanks a lot 🙏🏻

[@chaks2432]

This is my first time learning about Neural Networks, and you're doing a great job at explaining things in an easy to understand way.

[@littlethings-io]

Just ordered the book - can't wait to dive into it. Thanks you, this is good stuff and a priceless contribution to the evolution of this area of science.

[@chaosmaker781]

this is better explained and with more quality than any neural network video where the concept is mostly shown just by the code

[@ginowadakekalam]

This channel is so good that you'll never find any negative comments

[@sentdex]

They are there sometimes :) but yes fairly rare.

[@usejasiri]

-comment

[@jsnadrian]

i can't believe you created this course - absolutely fantastic and wonderfully thoughtful in its layout - thanks so much

[@Alex-ol9dk]

I have never bought a book from KZbin before but you will be the first. You’ve deserved it. Absolutely love this work. Please keep it up

[@rrshier]

At about 14:51, where you present the matrix multiplied by the vector, the proper mathematical notation would be to have the vector as a column vector, as well as the output vector being a column vector. This is truly how the matrix multiplication is able to work, because a vector is truly just a matrix where one of the dimensions is equal to 1. Other than that, I have to admit, these are my FAVORITE AI/ML videos yet!!!

[@pensivist]

I was looking for this comment. Thanks for pointing that out!

[@aav56]

I've never learned linear algebra and I'm astounded how simple you made matrix multiplication out to be!

[@clementsiow176]

Never have I been so exciting for a new KZbin video, you have earned my respect

[@peppep4426]

This reminds me of the best TV series ... You finish one episode and look forward to the next ... Good job!

[@Alfosan2010]

last time I was this early, Corona was just a beer brand...

[@jonathantribble7013]

Friend: "So what do you do in your free, unwind, leisure time?" Me: "Neural Networks From Scratch" Friend: "..."

[@alexgulewich9670]

Sister: "If that's informative, then what's educational" Me: "Glad you asked!" *starts to explain neural networks and basic QP* Sister: "NO! Make it stop!" *Never asks again*

[@stevenrogersfineart4224]

Story of my life 😁

[@hasneetsingh]

Your explanations are so clear, I really appreciate the hard work you've been through to design this series to make such complex topics so much fun to learn :) . Enjoying a lot

[@gurns681]

Mate, this series is unreal! Love your work

[@lemoi6462]

The interesting part will be the backward propagation, im really looking forward to this

[@classicneupane6196]

Understood batch size finally

[@sentdex]

Glad we could help!

[@TNTeon]

Hey just to let you know, this video 3 years later continues to help and encourage new programmers! I'm in my freshmen year of highschool doing all gen ed courses, but I started working on this tutorial in my free time and I'm having a blast and actually understanding everything perfectly Just wanted to say thank you so much for really helping people like me in our learning of Computer Science and machine learning! These are awesome and super enjoyable!

[@DMBalchemy]

Incredible as always. This one struck a few lightbulbs. Thanks again, Eagerly anticipating #5, I'll have to work through the draft to prep

[@JackSanRio]

I pre-ordered the book because this is interesting and I a eager to learn more

[@yabdelm]

This is the best series by far I've ever seen. Just what I was looking for. I wonder if you'll get into explaining the why also. For instance, often times when I'm watching I do wonder "Why do we even have biases? What function do they serve? How do they enhance predictions? What sort of history/science/neuroscience underlies that and where do AI and neuroscience partways if so? Why does all of this work at all?"

[@asongoneal28]

Youssef I really hope @sentdex reads this ;)

[@naseemsha3010]

I think it was explained in a previous video, how biases help making predictions. Check out the last video guys

[@carloslopez7204]

He explained that in previous videos, but no all your questions

[@yabdelm]

@@carloslopez7204 I agree it was explained a bit but I really didn't feel the explanation gave me a deep understanding of the why unfortunately, just a very rough surface level and vague hint of what might be going on.

[@liadinon1134]

I think that now some things, like the biases dont make sense now. But when you get into training(the lerning process) is all start to make sense.

[@Voyagedudimanche]

Hello! I'am following you for more then 2 years and this is the best course for me! With those explanation of math - it is realy cool. Thank you for this work :)

[@yuwankumar]

After many searches I found this playlist! Thank you for making this Gold.

[@patrickvieira9200]

well finally looks like my linear algebra class was not a waste of time at all

[@thenotsogentlecat5847]

Sentdex: we're arriving at the sexy parts... Python: Oh, yes I am ;)

[@kelpdock8913]

x = we're arriving at the sexy parts... print(x)

[@merth17]

I can’t wait to see the implementation of backpropagation with the chain rule, it’s so simple when you teach it. Tysm

[@realbingus]

At the point where I had a question, I had not fully watched the video yet. So I commented my question. Literally five seconds later in the video you answer my question in the video. I love the series, thanks for doing this!

[@shubhamdamani1057]

Can you please provide a visual representation of how the batches pass along. I mean by using animation using bubbles and lines like you did in the initial videos.

[@harikalatheeswaran9206]

For people watching this video... remember this golden rule : Say we have two Matrices A&B..in order to multiply A with B,i.e A.B The number of Columns of Matrix A should be equal to number of Rows of Matrix B. That's why A.B != B.A Amazing video 👍! Thanks a lot ! Keep up the amazing work !

[@jedisenpei855]

Apart from trying to explain neural networks, you just explained the matrix dot product in the most intuitive way I have ever seen. I know how the dot product works by now, but I also remember how much work I had to give in to understand the concept given lectures and texts i had at university. I had to read through some difficult math equations and really think about what the book was trying to tell me, and I also had to go through a lot exercises to really get a grasp of it and remember it, and then you just explained it in 10 minutes and it makes perfect sense, although I had almost forgottes what it was all about. So easy. I wish my teacher had an animation like the one you show at 9:10. Then I wouldn't have had to struggle through the math classes, as much as I did, in my education as an electrical engineer.

[@frederick3524]

I have been looking forward to this all week!

[@franky0226]

Notification => nnfs P4. Me: clicks on the button faster than the speed of light

[@dippy9119]

6:09 what's a fitment line? Google isn't helping me.

[@devinvenable4587]

I watching this as a refresher as I studied this topic a few years ago, and I find the context you provide really useful. Thanks!

[@lonnie776]

You are doing a great job explaining these concepts in a way that is easy to understand. I can't wait for the next part so I am ordering the ebook. Great job.

[@amogh3275]

16:19 you said the other way around by mistake.. shouldn't it be 2.8 +2, 6.8+2, -0.59+2..

[@fl7977]

Yeah, that really confuse me more than it should have

[@bipanbhatta2736]

yes. It is called broadcasting.

[@DRIP-DRIP-DRIP]

Never clicked on a video so quickly

[@bradley1995]

I just want to again say thank you so much for these videos. They are top notch. It truly has helped me get a deep understanding compared to what many other "tutorials" have. Plus all this information being provided free. I feel blessed!

[@time2learn123]

Why does the dot product switch inputs and weights when working with batches. e.g when input is a 1D array the calculation in the code is np.dot(weights, inputs) but for batch it is np.dot(inputs, transposed_weights). Why doesnt it work when we transpose the inputs instead? Im sure Im missing something simple. Thanks for the videos they are amazing!

[@joelgerlach9406]

Because matrix multiplication is not commutative

[@MrGeordiejon]

I think it is the nature of what we are doing - We are taking inputs - applying weights and biases - and delivering outputs. or entering exiting a decision. so we can't use an entrance to a neuron to exit another neuron. I think the demonstrations by Harrison are to cement the concept and awareness of ValueError shape()... and he also showed how multiplication works between array and vector (multiplication) I went back to lesson 3 for 2 things. 1. I like inputs being the first entry so 'My doors' are labelled correctly 2. Use the npArray().T in that example If he had not shown us 3 before 4 - I would have found it harder to appreciate transpose() - I don't think I will ever just reverse the args when I am coding this stuff. import this

[@413blaze]

Something that I find interesting that I think might have to do with this is that specifically in the case of 1 dimensional arrays, the shape is different. I am used to thinking of a matrix as rows by columns. For example, [[2,2],[3,3]] , would be a 2 by 2 matrix. 2 rows and 2 columns; however, lets take the example [1,2,3,4] , I would have expected the shape of this to be 1 by 4 ( 1 row and 4 columns) but it is not. The shape of [1,2,3,4] is (4, 1) . So, the way to think about it is by elements in a list of lists. The first entry x in shape (x , y) is how many lists in a list and the second entry y is how many entries within each element or list. In his example, his first inputs [1,2,3,4] the shape is (4,1) and when he put [[1,2,3,4],[1,2,3,4],[1,2,3,4]] the shape became (3,4) and if you are thinking about this in rows and columns that wouldn't be the case. I hope that made some modicum of sense lol

[@ollie6989]

You could perform the same operation by transposing the inputs however keep in mind the matrix rule (A.B)' == B'.A' , e.g. inputs.(weights.T) == (weights.(inputs.T)).T aka. the output of inputs.weights_transposed will be equal to the transposed output of weights.inputs_transposed, the issue with the values probably comes from adding the biases without first either transposing them or transposing this output matrix back, as they will be added in a completely different order.

0th! Finally!

[@thewild2334]

daniel!

[@dunder9444]

So op

[@user-ns8dl3vm5z]

U R AWESOME

[@NikhilSandella]

This is the best channel with the best content, with amazing animation. Clear explanation. I'm in love with this man. :)

[@asu4908]

Doing gods work, ordered the book a while ago and finally have time to actually dive into this now-thank you so much bro

[@afafssaf925]

You are wayyyyy more buff than it seems by just your face.

[@sentdex]

I'll keep that in mind

[@ahmedyamany5065]

Great explanation and animation, but in 14:47 [1,2,3,2.5] in python is array which is vector or matrix (4,1) but when you write it in paper or animation you should write in vertical form like column, not row, because [1 2 3 2.5] in animation is matrix (1,4), not (4,1), so we can say every element in array [1,2,3,2.5] is row, 1 is 1st row, 2,5 is 4th row.

[@3alabo]

One of the best tutorials I have seen on the topic , Saludos de Argentina!

[@sharanbabu2001]

Loving the effectiveness! The batch size explanation was amazing!

[@sentdex]

Glad you liked it!!

[@thomasnevolianis8616]

import neural_networks_from_scratch as nnfs from nnfs import moments best_moments = moments(channel='Sentdex') print(best_moments[0]) ''The SEXY part of deep learning''

[@hemanthkotagiri8865]

I can't be more thankful for anyone than you and Daniel. Thank you so much!

[@sentdex]

Happy to do it!

[@aamirkhanmaarofi9705]

Watching this playlist is awesome, it made my task very easy. Have been stuck with the implementation of the multilayer perceptron for two days. Thanks

[@tymion2470]

I'm very thankful for this series, I just learn so much new thing, because you're so good in explaining, and there yet 5 videos to watch!

[@minhtuecung5418]

Now that´s what I call real teaching: triggering curiosity ! Thank you so much, sentdex! Math rules!

[@Gazarodd]

I think this tutorial serie will explode. Atm, it's really clear, you're fantastic

[@super7ace]

God level series on Neural Network. Good job and always proud of you buddy!!

[@garymdmd]

I am on lesson 4 now - you are such a great instructor, I love learning this stuff.

[@benjaminsteakley]

Took my five years to find something like your videos in 2022. I dropped put of college from stress and i can finally sit down and try to understand this math. I hope the video which explains linear regression is as good as these four so far

[@rakshitjoshi823]

High quality animations. Much respect!

[@keshan-spec]

I love this series and i always look forward for the next one. Thank you ❤

[@accounttwo5114]

Fantastic, I'm really excited about the following videos!

[@milindprajapat7177]

Best feeling ever...after seeing sentdex p.4 out...Thank you so much...May god keep you safe from corona virus so that we can get awesome content from you! Love from India ♥️

[@saisiddhanthgujjari8954]

Amazing content sentdex, the visualizations are just top notch and aid to a much clearer explanation.

[@ryangao3564]

Hey sentdex, such addictive content in your videos. Couldnt wait for the next release any longer. So I just pre-ordered the e-book.

[@sentdex]

Woo! Hope you enjoy!

[@bartosz13]

This is nuts. Crazy good quality

[@horseman3253]

Wooow, this how all subjects in school should be explained, amazing visualization, very clear!

[@dHue_52]

literally been waiting for this all week! Thanks sentdex!!

[@sentdex]

Enjoy!

[@violinplayer7201]

Best python neural networks video, for sure

[@TrevorTheITGuy]

thanks for breaking this down as much as you have. between the books and the videos, the concept has become more like legos. atleast in my head.

[@dinarakhaydarova4898]

I thought I understood all of these concepts until I watched your tutorials. it's amazing!

[@danielbardsen4101]

Hi Sentex, Thanks for doing my engineering/programming career so much more interesting! You really are the best

[@gardedesombres3254]

Thanks a lot man ! can't wait for the upcoming videos !

[@fanasisangweni8539]

Im glad I found your channel man, i swear to god, your videos are awesome, Im only starting to understand ANNs after watching your videos.

[@sciWithSaj]

Thanks a lot This will be my first object oriented programming. It was kind of daunting for me, but you made it so simple.

[@FagunRaithatha]

This content is really good. Thanks for making this simple. I have been binge-watching your videos.

[@HJ-jr7zd]

Great video Sentdex. Looking forward to read the when it's out.

[@chuckf5540]

Great explanation and very clear. I look forward to all videos. What a learning process!!

[@_inetuser]

Really excited!

[@santoshtamboli]

This is really awesome. There is no chance for anybody to post negative comments. I like to watch more videos from u on ML and DL

[@julianray6802]

Thought I knew what batch size was about.... I was completely wrong....I need to think like a neuron! Thanks again Sentdex for yet another great video!

[@ayanpanja1956]

You just made me cry. The way you portray my god!!! its really awesome. So much effort. I still dont understand the people who disliked the video.

[@ConnellArt]

Can't wait for the book to arrive! This is a great tutorial so far. Thank you Edward Snowden!