I Built a Neural Network from Scratch

Don't click this: tinyurl.com/bde5k7d5
💚 Link to Code: / greencode
How I Learned This: nnfs.io/ (by the awesome ‪@sentdex‬ )
I'm not an AI expert by any means, I probably have made some mistakes. So I apologise in advance :)
Also, I only used PyTorch to test the forward pass. Apart from that, everything else is written in pure Python (+ use of Numpy).
⭐ Other Social Media Links:
🔊 Discord: / discord
🐦 Twitter: / thegreencoding
📸 Instagram: / greencodecodes
🎵 Tiktok: / greencodecodes
🔊 Music I Used in this Video: share.epidemicsound.com/7i1d0b
Current Subs: 14,219

[@Green-Code]

I'm not an AI expert by any means, I probably have made some mistakes. So I apologise in advance :) Also, I only used PyTorch to test the forward pass. Apart from that, everything else is written in pure Python (+ use of Numpy).

[@CrushedAsian255]

"umm Numpy is a 'prebuilt framework'" ☝🤓

[@przemysawtomala9304]

5:44 gradient descent won't ever jump like that with your implementation (without momentums), cause function in that place is rising

[@ProgrammerRajaa]

Is that code available in GitHub if so can you share the link plse

[@genildademeloescrituraviva]

Building a neural network from scratch? what about IN scratch?

[@Daniel_Massicotte]

The only reason I can see for this, is learning the exact math involved in it.. but your solution is useless in a real world application, due to the simple fact that using Python in the mathematical part of the neuron (weight and bias calculations), will be slow as hell, and almost impossible to train something for real, at least in an acceptable period of time, in comparison (in my case) with calls made to Tensorflow that itself uses cuda library to TRAIN the network using all my gpu`s cores at their maximum speeds and in parallel processing, at thousands of times faster than PYTHON shitty scripting interpreted pseudo language for DUMB... at least use c++ or something similar for a bit of performance... Python, despite the fact that I hate it, have it`s place in some very specific applications, like writing a fast and easy solution for something that will be executed one, like creating a neural network , that itself will be running in cuda/tensor cores, but NEVER use Python for CPU-intensive tasks... it will uses 10x to 100x the resources (memory needed and cpu load) compare to c++ compiled for the same task, or if you prefer, it will takes 10x to 100x more time to execute... or last way of seeing it: it will takes 10x to 100x more energy to do the same thing... And it is true that Numpy will accelerate a lot of calculations, but as long as python is used, you will never reach the maximum speed a c++, compiled application can deliver... All that just to make sure everyone understands that Python has a specific area where it is very useful, but never use it for all tasks, it was not created for it. Simple rule: ask yourself `will the code be performed many times`, enough to slow down the task, so better use c++, that will take longer to code, but will be way faster in a cpu demanding task, saving time in the long run (no so long !!!) And if you are learning Python, because it is simplier than C++... Let me tell you that the difference between them are not as huge as you think... and for the ones that want to make a living out of programming.. Python will bring you jobs that are paid less than 100k per year, c++ can easily gets you jobs paid at 250k per year...

[@fragly]

Ngl using that chef hypothetical is such a neat way of explaining how a neural network functions

[@akshikaakalanka]

Yeah loved it!

[@burrdid]

now do it in scratch

[@zennihilator3724]

What does scratch mean? Does he have to make his own programming language too? Does he have to make his own computer? Does he have to design his own pcbs inside the computer? Does he have to put his own layer of silicon in a resin case and dope it? Does he have to generate his own electricity to power his house?

[@KA-kf4ke]

Scratch. 'Website' Scratch. Y'know... the coding language for babies...

[@陸]

​@@zennihilator3724scratch the programming language

[@insaanonline]

​@@zennihilator3724that means explain all the code one by one

[@mesh_devo]

@@zennihilator3724 it has 2 meanings first one is from zero or step by step exp : i built my web from scratch ( means manually from zero) second is SCRATCH a web and app that help you to create simple games

[@SomethingSmellsMichy]

3:12 the equation states that the loss of a network that returns probabilities with values from 0 to 1 is the expected output × the negative log of the actual output. The reason this works is because -log from 0 to 1 gives a higher loss as the probability approaches 0 and almost no loss as it approaches 1. Multiplying that by the expectes probability makes it so that the network only adjust the values for the outputs you want to approach 1.

[@victor3btn598]

Binary Cross entropy

[@Exkolix]

this is why love nerds🤩

[@ShabJimJets]

very nice mr green code very nice. You deserve a lot more subs for how good these videos are. Cant wait to see what the future holds

[@Ari-pq4db]

Subscribed, can't wait for such more informative videos ❤🔥

[@DK-ox7ze]

It's pretty cool to implement all this from scratch. I had studied all this a few months ago but forgot most of it because I never practiced it. But this served as a refresher.

[@NihadBadalov]

hi, where did you study it?

[@Hangglide]

thanks! really cool! especially I just learned neural network and watching your video reinforced what I just learned from the class.

[@Podcast.Motivator]

Awesome bro. Waiting for more videos like this.

[@chrisx2342]

ur expalination is absolutely fantastic!!

[@secondlive1]

This is a great video, keep making them!

[@yassinechritt8816]

Great explanation! keep on postig great stuff 😎😎

[@虚]

Amazing video!

[@adebayokehinde1580]

Making a tutorial is one thing and adding animations is 🔥Great!!

[@baonguyen4278]

Great work bro! u got another subscriber !!

[@santiagogonzalez-hc1vp]

What a great vid, new sub You summarize accurately two weeks of class of a ML Master where I didn't slept Great job doing that and understanding the fundamentals Ignore bad comments Keep the pace

[@LebaneseJesus]

Instant sub, brilliant video

[@sakchais]

This is incredibly fun to watch!

[@nesquickyt]

Your videos are really good and interesting 🔥

[@gabrielrock]

awesome video bro, congrats! I’d like to see a video on how to go from that to a generative AI or a RAG, u know? Are u planning some video like that or do u got some reference?

[@glumpfi]

I went through the same adventure :D I wrote a neural net from scratch in C++ just to get a deep understanding. The backpropagation part took me a while to figure out. I just got to an accuracy of 94% with MNIST, maybe because I still didn't implement optimizers and batches. Thanks for sharing :)

[@weslycosta348]

so underratted channel man, keep goin!

[@Avion_Animz]

Hey man I just discovered your channel and I love it and I've been wondering how long have you been do this programming I mean

[@suvojitsengupt]

cool just subscribed , hope u will continue to post this kind of stuff, loved your work ...

[@familytharun3924]

the video is too gud bro , u have taken lots of efforts..

[@shashanklk7867]

pretty sick, bro 🔥

[@TimStellar]

So good bro, so good!

[@avi_9243]

Ayo great stuff, tysm 😋

[@cen-t]

Nice video, btw do you do live coding session ? Like on twitch or here on yt

[@sirosala]

Excelente el video !!, muy bien explicado, brillante aporte de conocimiento. Saludos desde Rosario - Argentina.

[@sqec]

Nice Video!

[@lelomator]

Super amazing Video!

[@akhildonthula6160]

Please do more videos , i luv the way you do💥

[@kamaljeetsahoo4752]

Quality of content 🔥💯

[@Egon3k]

cool video :) keep on going

[@medakshchoudhary]

loved the way how you explained all of this can you pleaseeeee make a beginner guide tutorial on how to get on to things like this exactly this type of things you got a sub

[@kartikgdrbd]

+1

[@Sumii4242]

Someone know what the muisc at 6:35 is? ah and also, this is seriousely such a cool video, great job mate!

[@gauravmandall]

Amazing video

[@johanlarsson9805]

I did the same thing 15 years ago when I learned about ANNs... but in c++ and no numpy equivalent. Also, since I studier molecular biology at the time and was deep into evolution ofcourse i used a genetic alforithm instead of backpropegation. It all worked well in the end.

[@rnts08]

Your code bullet is better than codebullet. You are what everyone hoped that he would be with his enigma video. Keep it up! 😂

[@AlexJNef]

trying to explain machine learning without too much maths is like trying to explain a steak without knowing what meat is. amazing job dude, incredible simple explanation

[@kshitij7791]

Hey @Green-Code do you have any github repo for your work?

[@Yuzuru_Yamazaki]

"let's think of every neuron as a chef... Now , Let 'em cook 🗿" ahh explanation 😭

[@codeinvasion]

Here before you get viral 🔥💯

[@hariharan-wx9oq]

just amazed!!!

[@user-pb2fp8el9q]

Amazing video bro. Nice explanation, truly 👍

[@commissariomontanaro2931]

why does my acoustic pattern recognition match your avatar poses to Code Bullet? Nice video tho, now do it in C to assert dominance

[@ericdanfunk3966]

This content creator is a clone of Code Bullets style 🥱

[@Smurdy1]

I think it's funny that I'm reading this while waiting for my C++ neural network to train. My code isn't even that optimized and my program only took about 5 minutes to go through the process of training a neuron 49 billion times. It's insane how much faster C++ is at machine learning (and everything) than python.

[@veifingtps4697]

I remember my first timw doing this went crazy because didnt want to do the math but wasnt so bad lol crazy good video and visualization

[@SlimeFroster]

Nice, 17k sub

[@enerz9135]

Your Exaplation Is Very Simple It's Very Easy To Undertand

[@dhruva167]

Love the video

[@EchoPrograms]

I just did the same thing a few days ago lol! (Also from scratch). I did it in JavaScript so I didn't even have numpy. My matrix class is like 150 lines long lol

[@sakethsaketh750]

Cool video

[@wolfgangsell3233]

W video! Can you show off your code for the beginners here?

[@Bigleyp]

Now do if but where there are regions, a neuron has multiple connections and it is either on or off based on how many recent signals go into it.

[@CatBoxOfficial]

Underrrated youtuber ngl

[@afrateam6241]

Only a genius could understand how genius you are . Wow 🎉

[@pentasquare]

"Wait it's all maths?" "Always has been"

[@TheFuture36520]

Bruh is implementing mathematical equations like bernoullis theorems and second order differential equations 😂. You're the best brother ❤️ 💙

[@amunif_]

could you give a link to your code? github or colab notebook? I’ve learned the theoretical stuff for backpropagation and all the parital derivatives during my Masters but never coded them from scratch. Would like to learn how to apply all that math in code.

[@helved807]

This was super interesting! Mainly because this is something I've wanted to do myself, but I've had some troubles implementing backpropagation. Any tips on how to implement it?

[@SomethingSmellsMichy]

*Disclaimer: After writing an explanation, I assumed you understand calculus, but all the steps are broken down and the equations are solved fully. In this video, you were introduced to the loss function. This function does as it states: Quantifying the difference or incorrectness of your Neural Network. Ideally, you want your loss function to be 0 or close to 0. Assuming your loss function is either cross-categorical cross entropy (like in this video) or a more common approach: Mean Squared Error (Which has the formula 1/2(y-o)^2 where y is your expected output and o is your actual output). *In case you want to paste this into an appropriate calculator/document and because I've been practicing; here is the LaTex version of that equation \frac{1}{2}\left(y-o ight)^{2}. Since these functions are never negative, you can assume that your loss function reaches 0 at its absolute minimum (usually you'll hit a local minimum for complicated problems). For Instance, we can tell if a function "f" hit a minimum or a maximum by checking to see if f'(x)=0 (same as 0=d/dx f(x)) or if its slope is equal to 0. Since we know that if the slope is positive at a point as x goes towards infinity then it's rising. If the slope is negative at a point as x goes towards infinity then it's falling we can find the x position of the local minimum by iteratively changing x by x->x - f '(x) * lr (lr being the learning rate). X is usually represented as the input to the function, but since we want to change the weights and not the input, we can assume that the input to your network is constant and that the weights are the input. So since you've already done the forward pass, you likely already know that a network can be structured like this: o = f (W * x). Where o is the output, f is your activation function, W is the weights matrix, and x is your vector of inputs. Assuming you're using the Mean Squared Error, let's try to find the function's derivative with respect to your weights. The entire function is 1/2(y - f (W * x))^2. The chain rule tells us that we can find the derivative dE/dW (E being the cost function) by solving dE/do * do/dW (see how do cancels). *For the sake of visuals this is the LaTex equation: \frac{d \cdot E}{d \cdot o} \cdot \frac{d \cdot o}{d \cdot W} dE/do would be represented as (y - o) or (y - f(W * x)) and do/dW would be f '(W * x) * x. Meaning that dE/dW would be (y - f(W * x)) * f '(W * x) * x. So now you can update your weights with W -> W - dE/do. And for multiple layers, you need to pass dE/dx to the next layer which in this case is (y - f(W * x)) * f '(W * x) * W (Notice how you multiply by the weights instead of the input).

[@SomethingSmellsMichy]

I'd recommend making a class that deals with a single layer of the network. The class should have a method called backward (for backward pass or backpropagation, but you can call it whatever you want) that takes in the error and the inputs for that layer. In the method, multiply the error by the derivative of your activation function and the inputs and add that result to your weights. The method should also return the error multiplied by the derivative of your activation function BUT then multiplied by your weights (preferably before you update them). If you want, I can give you some code to reference.

[@UrFada]

@@SomethingSmellsMichy I love your explaination Understood 70% of it even with little calculus and linear algebra background as am In grade 0 but started practicing calculus and linear algebra for making my own nerual network only part I don't really get is the end, and I find it hard to code something except I can fully visualize how it works so it has been troublesome trying to fully understand it

[@SomethingSmellsMichy]

@@UrFada I can imagine that the end starts to become more complicated as it divulges more into symbols. I was kinda trying to wrap it up because of the character limit on replies.

[@UrFada]

@@SomethingSmellsMichy Ahh yes I will try to better visualize the comment later and maybe I will able to understand but overall thank you for the explaination

[@hemanthkumarvaddipalli5]

bro could u please give a complete road map to learn this stuff for beginners 🥺 . even from learning python

[@johnyohan7022]

i have a physics end sem exam tmr, this has no connection to that yet here i am watching this

[@papierbndc]

4:00 let him cook, Let Him Cook

[@jakeboots]

damn boy you made it look so easy [it isn't]

[@user-pk8wi3of6t]

Hello @Green code how did you did that... I am beginner in machine learning studying numpy, pandas and matplotlib How did you did that. I am also interested in data science but i am not finding exact path.. Please guide bro

[@sarimshaikh5224]

Sir, u drop this sir 👑 , please wear it from next video

[@mkgamingdev]

Although I didnt understand any, but Editing is awsm

[@peashooterman3]

have you ever credited carykh for making lazykh since you seem to use it for basically every video

[@21boomin]

Does someone know a mathematical course for this? i want to learn it in detail

[@Gordy-io8sb]

The song that plays near the beginning is a poppy version of Blue Monday. Lol.

[@rblxcoden]

Nice Vid

[@CrushedAsian255]

google thinks this isn't english for some reason? maybe they need new neural network

[@MarshyMcOfficial]

"its getting 40-50% accuracy it sucks" I know this seems bad greencode but you just taught a computer how to recognize things that we previously thought were only recognizable by humans. thats not bad. good job.

[@abadishaikh]

I would watch a 2h video of this. I dont want cuts and entertainment.

[@Green-Code]

I know, but 2h of this is a whole as movie. It takes me like 50 hours or more just to make one of these videos :)

[@init_yeah]

So that's how you look without the Grey tv

[@stayhappy-forever]

Can you open source the code if you don't mind? I worked on the same project but there are some improvements I feel like I can make. Edit: I implemented SGD and got 94%-95% accuracy, with 16 hidden neurons and only a singular hidden layer (10 epochs). is it possible if you can share your model architecture? Thanks!

[@yds6268]

Can you share yours? I know links in comments are impossible, but the repo name would be amazing

[@stayhappy-forever]

@@yds6268 Do you want the whole code? or do you just want an understanding/run through of what i made

[@motheomkhwanazi9529]

Guys please help me out ,I have an understanding of programming but as far as maths ,I didn’t do so well in high school,where do I need to start with regards to maths to understand this

[@anshtanwar1813]

Wow amazing 💚💚💚💚💚💚💚💚💚💚💚💚💚💚

[@barionlp]

try giving your MNIST model your own hand drawn digits (then tell me again how good it is)

[@roleben3009]

is there a notebook for this video?

[@eazypeazy8559]

> from scratch > using bunch of libraries

[@Superdeeep]

I wanna see you do it without pytorch, tensorflow or a machine learning library

[@eazypeazy8559]

@@Superdeeep literally did it on C and finishing an article. good point here from u, now im thinking about a video

[@hodayfa000h]

How long did it take you? I wanna try it but seems like a bad idea​@@eazypeazy8559

[@StaniG420]

@@Superdeeep Every Data Science student who passes at least fourth semester is able to code NN from scratch, its not a crazy feat.

[@PenguinCatHybridDev]

bro oversimplified one of the most complicated concepts.... your education is insane i hope i get to that level soon

[@RWX0_Extra]

You should create your own programming language and do a neural network with Thats what i call scratch

[@atomnorbi8271]

I NEED you to change how you animate, dont just flip your character, the logo on their shirt flips also, please make separate models for the flipped version of your character please please please

[@PythonIsGod]

Wow man, Python is love, love is Python.

[@Daniel_Massicotte]

Python is one of the worst thing that was created in the industry since long time, it is using a lot of resources ( memory and cpu ) and it is very slow.... Of course it is very useful for a very fast script, that will generate something from data, ex.. making a thumbnail, from a 3d image and adding it to the gcode file... (the Python script is in cura) or things like that, but using Python to make large application, should be considered a crime !!! because all the processing power, memory usage and heat dissipation, used to give the same results compared with some c++ compiles, is ridiculous... Of course the development is faster, but the result is shitty.... Sometimes, in my system, Photoshop, maya, VS code, visual studio, even Unreal engine 5, with many small other apps, are running at the same time... (with 50 to 100 pages opened in web browsers) And it is a fucking good thing that most of the stuff running, is compiled (and well optimised), because if all those applications were made with Python, the GPU's and CPU cores will be red hot, and the whole thing would have crashed at 20% of the maximum I ask the station... If you are all too lazy to learn the real stuff, and want to make professional applications that runs fast, than can be done in no time, and from a complete solution (gui and code) and has a lot of libraries... use XOJO... it is more than 20 years of development, from REAL BASIC, to what it is now.. once your application is compiles (for the os you want, even android and ios, mac, linux, rpi, win).. it runs almost like c++ compiled, but the ease of developpement is 10x Python, and the speed of the execution is 100x Python... documentation.xojo.com/getting_started/introduction/what_is_xojo.html

[@VidsMiracle]

I am seeing potential please don't forget me brother

[@HamsterianDevour]

nice :D

[@suryak3040]

@Green-Code where is the source code for this, and this is a nice tutorial keep continuing and little 👍 for your work.

[@mr.hackerstech2575]

Best video on the internet

[@mr.hackerstech2575]

First time I subscribed and liked a video

[@mind6861]

Can we have the code to go through

[@Fatement]

i thought you were going to do it in Scratch, the app with a fox thing 💀

[@comosaycomosah]

Who let this man cook up a neural network

[@Ash-pw9hy]

nice

[@Aifilmsmaker]

How to create talking animation and hand shaking like your animation

[@markwilson3326]

Anime code bullet that acts smart

[@CrushedAsian255]

7:13 hopefully your dataset doesn't contain any babies or Vietnamese emblems this time

[@charlotte80389]

my mind hurts so damn much

[@brawldude2656]

Backpropagation couldn't be explained simpler

[@lwangacaleb2729]

Now do it in assembly

[@AbdouMadjidi_.]

7:49 I thought it was 2 !