Create a Simple Neural Network in Python from Scratch

[@JonasBostoen]

In the next video we’re going to be making a blockchain in JavaScript, so subscribe if you’re interested in that stuff!

[@SoumilShah]

great video so made everything so easy

[@CrypticConsole]

Dow stupid schools blocked pip and zip archives so I can't install numpy

[@itstatanka]

Which compilar did you use?

[@siddhant5697]

in which software r u coding??

[@frankynakamoto2308]

Polycode Can the neurons and inputs be placed together, like neurons with much built in data?? Also I need a very powerful neural network for several different purposes, speech, faceID and math solving problems, do you have something that you made that is open source that you can share with me??

[@johnc3403]

"stay with me, it's gonna be ok"... dude, that's such a lovely sentiment. You were born to teach I think, with that ability to keep pupils onboard. Very good video my man, thank you so much..

[@mohamedsuhailirfankhazi6628]

My friend, your explanation in 15 minutes gave more clarity to me than hours of crash course tutorials online. So simple and well explained. Awesome stuff my man!

[@morphman86]

After watching hyper-advanced tensorflow/keras stock market prediction tutorials for a while, being completely lost, I stumbled on this. I finally, after weeks of trying to learn NN and decades of practical programming experience, understand it. The iterative backpedaling was what confused me with all of those other videos, but taken down to its most simple form, like in this video, I can now see that it's merely looking at what it got, what it was trying to get and make adjustments to the appropriate synapses based on that, then trying again. It's not the maths that confused me, it's how the machine actually learned. And that was perfectly demonstrated in this video. Thank you!

[@RandageJr]

Do you know where I can find these tutorials? It would be very helpful for me, thanks!

[@jacobokomo1880]

kindly feel free to share with us Who was the teacher who took you through the Previous Tutorials. However, This teacher is doing well. Credits 💪

[@Govind_Sisma]

B

[@morphman86]

@Isaiah _ Neural Network

[@KennTollens]

I agree too. So many videos complicate and dance around simple mechanics. Knowing the flow of the engine and the simple concept of what is happening, the other videos might make more sense now that I can put it into context.

[@djjjo6130]

“Stay with me, it’s gonna be okay” that makes me feel like I’m actually learning something and not just being told something

[@MC_MrOreo]

(I know I’m late but) Literally came to the comment section about this 😂

[@hfe1833]

What the?....this is it, finally I found good tutorial

[@Pancake3000]

same lol Ive finally can actually flippin understand thank much +1 sub i can english.

[@scottpatterson9136]

I agree

[@koksem]

ye someone finally explains what it is XD

[@mariomuysensual]

same!

[@arifmeighan3162]

This tutorial is a perfect blend of talking/programming and slides. Its also quick and to the point 8)

[@mattisaderp8929]

"stay with me it's gonna be okay"

[@wirly-]

TypeError: '

[@henilmalaviya]

@@wirly- Loll

[@chinmayhattewar4456]

@@wirly- hahaha

[@Awesomer5696]

What a fantastic way of explaining it. Whilst this is obviously not immediately useful, It's a sort of toy approach that gives you a building block to understand the greater scope.

[@shimuk8]

I joined my university 2 months late, absolutely had no idea how to learn the lost neural network project topic and then I saw your video !!! Thanks a lot dude !!! For saving my semester HAHAHA

[@JonasBostoen]

meaaaww hahaha nice, share it to any of your buddies if you think they need it ;-)

[@shimuk8]

@@JonasBostoen Oh yes already did that,,, right now you have blessings of many helpless students LOL

[@brehontechologies]

Finally, a clear, straightforward tutorial to code along. GREAT JOB!

[@hdluktv3593]

I watched a lot of videos about Machine Learning because I wanted to unterstand how that works. Non of these Videos explained so good like yours how a neuron and the adjustment actually works. Good work, now I finally understood it.

[@paulschmidt8742]

Bro, it was much easier then I thought. Thx for explaining.

[@joesminis]

At the 10 minute mark and I just wanted to say that your explanations are clicking left and right with me thank you!!!!

[@robertdraxel7175]

Most useful video on the internet for a total beginner, for anyone new to AI. Thanks.

[@abdechafineji8782]

The best one who can give you the right explanation of creating of a neural network from scratch.

[@EricCanton]

Just a note on sigmoid_derivative, for myself as much as anyone else. Since you're inputting the output of sigmoid to sigmoid_derivative, he's using that sigmoid satisfyies the differential equation y'(x) = y * (1 - y) so we can compute the derivative sigmoid'(x) by inputing sigmoid(x) into [y --> y(1-y)]. That's very clever!

[@victoryfirst06]

But you should run the outputs through the sigmoid derivative, right? And the outputs are sigmoided by default, so shouldn't you use the sigmoid twice?

[@stevesajeev6477]

Wow... The perfect tutorial.. I have been searching in the internet for a tutorial on how to make neural networks from scratch . now I got it.. this is soo cool... Very detail explanation...

[@calmo15]

Amazing video, too few sources do the absolute basics. however, can you please crank your volume up!

[@povmaster235]

At last... the video that doesn't just explain stuff but, but actually tells you what to do too!

[@Oleg-kk6xv]

Thank you very much. I constantly see these videos about the theory of Machine Learning and AI but I have never found an in-depth start from scratch tutorial with mo libraries, all while explaining everything. Thank you!

[@nocopyrightgameplaystockvi231]

Line no 16 : synaptic_weights=2 * np.random.random((3,1))-1 this line makes an array of 3X1 or a matrix of size 3X1. I did not understand this line before I tried this line separately. This makes an easy grasp of the random concept, but as I learned in Soft Computing in my Btech, you can directly initialize the weights as 1, which will then get adjusted during training. you can also replace the line with it : synaptic weights=np.array([[1,1,1]]).T THANKS TO YOU for making this short and easy tutorial!

[@Retriiiii]

Hey can you tell me why are we multiplying 2 and subtracting 1?

[@nocopyrightgameplaystockvi231]

​@@Retriiiiiwhere??

[@Retriiiii]

@@nocopyrightgameplaystockvi231 2 * np.random.random((3,1)) -1 ^ ^

[@notyourtypicalanime7475]

This is what I'm looking for, on how to train your datasets by adjusting weights. Thank you so much!

[@MsRAJDIP]

So far the best simplest and practical tutorial I got. U cleared all my doubt and little background in python helped me lot.

[@rahulaga]

This is by far the best explanation. I guess by keeping the complexity level of chosen example pretty low, you landed the message perfectly, thanks !!

[@k.chriscaldwell4141]

Superb! Using the seeded weights so that you and the viewer get the same results was a brilliant touch. Helps the viewer know if he miscoded or not. Thanks.

[@novi0]

2 minutes in and I already have a better understanding than 2 semesters worth of lectures

[@JonasBostoen]

Coding starts at 2:30

[@ChillGuyYoutube]

Polycode ping your comment so others will see it!

[@du42bz]

@@ChillGuyKZbin maybe his firewall blocks icmp packets

[@rr.studios]

@@du42bz I read that as "pimp packets"

[@botancitil92]

I have been looking for a toy example of Neural Networks, thanks to your video I get to see one. Your video is very concise. Thank you. Also, thank you for sharing your Python code.

[@akmaleache4735]

I watched lot of Ann videos on KZbin, and all of them missing something which I am not getting But thanks to you I got what I need. Especially explaining the working. Thank u again

[@JonasBostoen]

Akmal Eache thanks man

[@coleboothman1158]

Hey dude just saw this video from your post on /r/programming - This video is awesome! You're great at explaining everything. Neural nets can sometimes be confusing but this makes a lot of sense to me. Thanks so much!!

[@jeffwads]

It helps to have someone who actually knows how to break a "problem" down to its bare essentials. Excellent work.

[@GabrielCarvv]

Excellent picture

[@chessprogramming591]

Man, this was so to the point! Thanks for your efforts. Best NN basics tutorial I've found so far! Very very useful!

[@aaronisinjapan]

Wow, I’ve been looking for a tutorial just like this for a long time! Subscribed! Please keep making videos!!

[@ciencialmente9969]

1:39 "so we need a little meth"

[@Loading-tr7yv]

I think we all do

[@astrainvictum9638]

Adderall is good for that

[@deekshithtirumala6474]

It's math LoL 😛

[@timothec.8216]

Thanks a lot. This is much more comprehensible than all I have watched and read

[@sreedeepsreedeep2260]

Best tutorial on neural networks i have seen till now....thanks buddy😘

[@REVscape95]

waiting for the next video, this type of explanation really helps

[@JonasBostoen]

I've uploaded it!

[@computerguy7451]

Before I slightly understood how neural networks work, now I understand how they work slightly better than before.

[@volador2828]

Nice work! Finally found someone that can teach the way I can understand it.. I subscribed and look forward to watching all your videos!

[@progmaster15]

Dude this video was really helpful! Thank you for explaining the basics of neural networks! :D

[@ankitds1369]

in output after training : you can use this, and this will round off the decimal as a round off value - print(np.round(outputs,1))

[@samayvarjangbhay8987]

finally a properly structured tutorial

[@0siiris]

Nice profile pic 😂

[@StreetArtist360]

Simple, Clear and straight to the point. Great Job!!!

[@blubaylon]

This is such s good tutorial!!! I finally understand how these things are actually coded!

[@deanresin4804]

This was a such a great tutorial. Very clear, concise and well paced.

[@aizej9896]

thx for the totorial gived the neural network my own training data and it worked geat!

[@SureshSingh-en5uj]

FINALLY!!.... I have been looking for such tutorial which teaches from scratch... That's Very good of you to do so... Keep it up bro.. Make more videos like this... BTW I am new to your channel. Just subscribed

[@harlongbitimung4108]

This video has taught me more than anything about ANN.

[@Pancake3000]

This is the thing that finally helped me understand! Never stop doing the grade vids!

[@title601a]

NICE!!!!! Finally, I can understand what is NN and backpropagation. Simple and Easy to understand. Thank a lot to Polycode :)

[@MCLooyverse]

If Φ(x) = 1 / (1 + e^(-x)), then Φ'(x) = e^(-x) / (1 + e^(-x))^2, not x(1 - x). I'm curious about your Atom setup. Are the text overview on the side and the code suggestions hidden in Atom somewhere, or are they plugins?

[@gamescript6449]

huh

[@ShradhanandVerma]

THANKS FOR VERY SIMPLE WAY TO EXPLAIN... FINALLY UNDERSTOOD.

[@elephant1989811]

what a excellent explanation of complex subject! Please keep up the videos.

[@Democracy_Manifest]

This video deserves an award

[@KonradGebura]

Thanks this was so helpful it really cleared up a lot of my questions about the topics other videos said let’s not talk about that yet..., thanks again these videos are super helpful keep up the amazing work

[@travisjol]

Finally a video I can understand! Thank you

[@nukzzz5652]

There is something i'm not understanding, when its time to change the weights, you're supposed to multiply the input with the adjustment and add it to the weights right? doesn't that mean if the input is 0 then the weights wont change at all? i noticed this when i tried different inputs and outputs, your example works fine but when i tried {0,0,0},{0,1,0},{0,1,1},{0,0,1} as inputs and {0,0,0,0} for outputs it was a mess and no matter how many tests i did it couldnt figure out the correct answer

[@sonic597s]

it does, this is a mistake in the code and can be fixed if you add a learning rate variable to multiply by the adjustments, rather than using the training inputs.

[@sonic597s]

@@havoc3135 instead of dotproducting the (transposed) training inputs with the adjustments, multiply the adjustments by some scalar, so you can scale your adjustments manually. hope this helps

[@traeht]

Thank you for very useful insigth into what is behind the neural network. At 10:00: (the derivative of a sigmoid function)=(sigmoid funcion)*(1-sigmoid function) and not x(1-x)

[@chandlerlabs2478]

Completely new to this and you made it very easy to understand. Thank you and good job!

[@MrFrostsonic]

In line 16, why have you multiplied the random weights by 2 and then subtracted 1 ? Great video .. very helpful .. Thank you very much.

[@JonasBostoen]

np.random.random returns floating point values between 0 and 1, but since we need values between -1 and 1, this is the way to do it.

[@nurhaida1983]

@@JonasBostoen thank you for this clarification. i was lost at this line but luckily stumbled to this comment. thank you very much! cheers!

[@BiCool03]

@@JonasBostoen I'm very late to the party, but since we need a random number between -1 and 1, wouldn't it be better to add two random numbers, then substract 1, or does it matter?

[@landaravi]

This is the tutorial actually I'm searching for understanding of Neural network... Thanks a lot...

[@BeSharpInCSharp]

Lots of people can code only few can teach.. well done

[@KomputasiStatistik]

The best neural network hands on

[@fiveoneecho]

Great tutorial, but I might have used a different approximation for d-sigmoid. I'm not sure where you got x(1-x) from as an approximation- it does not share a derivative with d-sigmoid and the vertex is off in space. I'm not sure if it is a standard to use and I'm just misunderstanding (I'm watching this tutorial to learn, after all), but I did a quick Taylor polynomial approximation and got the function: d-sigmoid ~= (2 - x^2) / 8 -------This won't work very well for things not centered at x = 0 This is about the same in terms of typing effort and computer processing, but a little more accurate. It is also based around x = 0 so it won't be biased towards one outcome (unless you built a weight into your function, in which case it makes a lot of sense). You can continue on to the 4th derivative in the series and add a third term which doesn't factor as nice but is extremely accurate (+/- 0.001) on the domain -1

[@karim741]

Thanks for the video, I try to follow this but I see the solution can be other way in binary logic, the first column is multiplied by the sum of the two other columns, not only first column is what decides the output but the others also as bellow. if we take this table at 0:20 Example 1: 0x(0+1)=0 Example 2: 1x(1+1)=1 Example 3: 1x(0+1)=1 Example 4: 0x(1+1)=0 New situation: 1x(0+0)=0

[@Rekefa]

What a great video! Keep up with the good work, thanks for sharing your knowledge

[@SuryaPrakashVIT]

Wonder full video, this will definitely turn upside down of my project. Thank You so much!!! :)

[@mwont]

Just a note: sigmoid_derivative is based on the exact analytical formula for the sigmoid derivative.

[@sonic597s]

thanks so much for this, I was really confused during that bit!

[@pluronic123]

@@sonic597s dont get it. He still uses x(1-x) which has nothing to do with sigmoid, but it is just an approximation to the shape of the curve (signs are opposit)

[@sonic597s]

@@pluronic123 a derivative finds the slope of the line at some given point. the sigmoid derivative being the formula x(1-x) (where x is the sigmoid fn.) means that if you were to plug in some sigmoid function given some value (z) as x, you would get the slope of the sigmoid fn at that value (z)

[@pluronic123]

@@sonic597s thanks precious internet dude

[@warrenkuah4314]

Incredible! I think this is the first video that has helped me understand the formulas behind a neural network! However, I was wondering how you implement the calculation of biases into the actual code and Backpropagation steps and formula?

[@quasistarsupernova]

One of the best coding videos!

[@industrialdonut7681]

15 minute video... takes me 2 hours to get through XD

[@drakemeyers8746]

So i tweeked training outputs to 1,1,1,0 with an interation in range of 100,000 and the computer gave me a perfect answer to the third output of 1. The other outputs where close to true answers but i didn't think the computer could give a 100% true answer. I guess im confused that it didn't take that many training loops to give that answer. Btw great video finally got me to get the computer out and start!

[@ogregolabo]

Thanks for great video! Possible code to find output for [1,0,0] : p_in=np.array([1,0,0]) p_out=sigmoid(np.dot(p_in, synaptic_weights)) print("Predicted Output After Training:") print(np.round(p_out)) => Predicted Output After Training: [1.]

[@seeking9145]

You are my hero! My prof is so bad explaning exact the same things over I guess 4 or 5 lesson of 3 hours each. And you just need some minutes ... haha I subscirbed you immediately. I need more of it!

[@th69100]

There is an error when I put in numpy. Where do you get the numpy pack? I have it on PyCharm 2019.1.3 64 bit.

[@seddaouiyassine7814]

pip install numpy on the command line :)

[@th69100]

@@seddaouiyassine7814 Did that already along with scipy but the error is still there.

[@critterpower]

Great tutorial, better than the usual,"Just use this library...."

[@marcusaureliusregulus2833]

Output = array[1[1]].value Lol just kidding. This was a great video and I understood a ton

[@madanvishal1]

Excellent Explanation making things crisp and clear

[@dridihamza7157]

this is on of the best yet simple explanation. keep up

[@prasanjitrath281]

Your video is a life saver, thanks! Hope you make more such videos!

[@jefersonferri]

You did a great job, you should make more videos. May be explaining how to make a more complex neural network.

[@MikhailBortsov]

Thanks for thinking about the equality of random weight for us.

[@thegoonist]

0:38 the rule could also be that that first and third outputs have to be 1, and not just the 1st output.

[@vikrantrangnekar4678]

Precisely, i thought the same too

[@peregudovoleg]

It took some time to understand your derivative choice, but it is all good now. The question: any idea why if we use "real" derivative instead of pre-calculating the sigmoid(x) and inserting it into x*(1-x) we could get better results sooner with fewer loops? If you just simply change sigmoid_derivative to "real" one, wouldn't you "sigmoid" the results 2 times? 1st you "sigmoid" the "np.dot(input_layer, weights)", then you "sigmoid" the results of previous "sigmoid". I can't say it is better, but the results are somehow better.

[@pluronic123]

I have noticed the same. It really interesting. If you sigmoid the sigmoid, the result is always between 0.25 and somewhat 0.2 (almost at the peak of the derivative curve). So I think if you "double sigmoid" the correction is always "aggressively". You always at the peak. I am not sure if I assume correctly.

[@reddinghiphop1]

This video is 100% gold, thank you !

[@alidakhil3554]

That is best empirical lesson on basic NN

[@sorooshnazem]

The derivative of sigmoid function is: \phi*(1-\phi). x*(1-x) is wrong

[@taravanova]

Lol, spent like 10 min trying to get his result and then eventually googled it to find out I had the correct result the whole time. At least the correct version was used in the code.

[@VoidFame]

yet somehow it gives the incorrect result when using the correct derivative. Something else is missing here.

[@gabrilrh]

i need more, thats awesome

[@tommygun296]

Very clean! VERY NICE! 🙏😍 Great Video! 😊😊😊 thank you

[@diljithpk1615]

Nice presentation. Made it feel very simple

[@Adam-ze3pr]

Hai, thank you, this is very easy to catch for newbie like me. Simple and clear. Keep going 👍

[@trianglesupreme]

At 0:40 , The output depends on both first and last input not only on first. If i label the inputs a,b,c from left to right respectively, then according to the 4 states truth function, the output is O= abc + ab'c =ac(b+b') =ac. So nn output for 100 input should be 0.

[@MW-rb5fs]

Great video! Thanks for creating it! I am totally new to coding neural nets. Question: After training the neural net, when I supply the inputs x1=0, x2=0 and x3=0 the perceptron output is sigmoid(0)=0.5. Can you explain what extra complexity would need to be included in the code to get a correct predicted output of 0 for a 0,0,0 input?

[@skippy1130]

I have the same question. Looking forward to hearing ideas.

[@OMAAKAAKORJOHN]

you are so wonderful , i quite understand by you basic and easy to learn method, thanks

[@twittuks]

Amazing tutorial, keep up the good work

[@xddddddize]

For this simple problem backpropagation is not needed. The gradient formula can be computed analytically and would reduce the training iterations a lot. (I achieved high confidence with 500 iterations only)

[@tomroggen7469]

Love your vids dude! Keep it up! ;)

[@manosbouzetos4132]

if the input is 0,0,0 i always get 0,5 output can someone explain?

[@Alislaboratory]

Thanks so much! After days of looking, found a great tutorial and can expand my knowledge!!!

[@Alislaboratory]

bruh