I Built a Neural Network from Scratch

[@Green-Code]

I'm not an AI expert by any means, I probably have made some mistakes. So I apologise in advance :) Also, I only used PyTorch to test the forward pass. Apart from that, everything else is written in pure Python (+ use of Numpy).

[@CrushedAsian255]

"umm Numpy is a 'prebuilt framework'" ☝🤓

[@przemysawtomala9304]

5:44 gradient descent won't ever jump like that with your implementation (without momentums), cause function in that place is rising

[@ProgrammerRajaa]

Is that code available in GitHub if so can you share the link plse

[@genildademeloescrituraviva]

Building a neural network from scratch? what about IN scratch?

[@emmanuelikhide8998]

Hey nice stuff @Green-Code I did enjoy the video. It's nice that the KZbin algorithm recommended it because I'm also building a nn from scratch although I'm currently debugging the hell out of it and trying to optimimze 😂, Thanks for the content amazing stuff

[@fragly]

Ngl using that chef hypothetical is such a neat way of explaining how a neural network functions

[@akshikaakalanka]

Yeah loved it!

[@raynlaze1339]

lol.. "neat" way

[@Jamdoe]

As aways, any example or concept can be explained by using food, or anything that is food related.

[@ByteBringer]

Yea useful for explaining neural nets for 5 year olds...

[@Yuzuru_Yamazaki]

"let's think of every neuron as a chef... Now , Let 'em cook 🗿" ahh explanation 😭

[@habibiibnnabeel]

ok underrated af

[@burrdid]

now do it in scratch

[@zennihilator3724]

What does scratch mean? Does he have to make his own programming language too? Does he have to make his own computer? Does he have to design his own pcbs inside the computer? Does he have to put his own layer of silicon in a resin case and dope it? Does he have to generate his own electricity to power his house?

[@KA-kf4ke]

Scratch. 'Website' Scratch. Y'know... the coding language for babies...

[@ipigtaiwan]

​@@zennihilator3724scratch the programming language

[@insaanonline]

​@@zennihilator3724that means explain all the code one by one

[@mesh_devo]

@@zennihilator3724 it has 2 meanings first one is from zero or step by step exp : i built my web from scratch ( means manually from zero) second is SCRATCH a web and app that help you to create simple games

[@SomethingSmellsMichy]

3:12 the equation states that the loss of a network that returns probabilities with values from 0 to 1 is the expected output × the negative log of the actual output. The reason this works is because -log from 0 to 1 gives a higher loss as the probability approaches 0 and almost no loss as it approaches 1. Multiplying that by the expectes probability makes it so that the network only adjust the values for the outputs you want to approach 1.

[@victor3btn598]

Binary Cross entropy

[@Exkolix]

this is why love nerds🤩

[@quantumHumans]

@@Exkolix great put, same

[@gym_spur]

that all went over my head bro, wish I was that smart too

[@joshcannon6704]

I have a few different neural networks I made from scratch in excel, they run slower but really helped me understand all the math that goes into them

[@AKG58Z]

What how ?

[@junacik9967]

I guess this is the really hard core way. I hope you're not heating your toasts on a sun :D

[@joshcannon6704]

@@AKG58Z ah, maybe I should make a video on how to. I basically have all the formulas in there and the data set and make it run with a macro

[@AKG58Z]

@@joshcannon6704 yeah sure don't forget to tag me

[@ik6071]

@@joshcannon6704 link the video when u do make it

[@DK-ox7ze]

It's pretty cool to implement all this from scratch. I had studied all this a few months ago but forgot most of it because I never practiced it. But this served as a refresher.

[@NihadBadalov]

hi, where did you study it?

[@ammglitch03]

Same I had a summer class but we instead used Tensor flow and OpenCV libraries rather than coding from scratch. The concepts are very abstract but videos like these help me remember and realize what I was doing exactly.

[@rebwar_me]

kzbin.info/www/bejne/boizeHRjhsmDr68si=clXJqgjOiFKEqMQm

[@glumpfi]

I went through the same adventure :D I wrote a neural net from scratch in C++ just to get a deep understanding. The backpropagation part took me a while to figure out. I just got to an accuracy of 94% with MNIST, maybe because I still didn't implement optimizers and batches. Thanks for sharing :)

[@pentasquare]

"Wait it's all maths?" "Always has been"

[@adityavardhanjain]

I was really shocked in my second year when I too realised that it in fact is all math.

[@jonathanalpart7812]

That’s computing in general buddy

[@sebahattinsaral]

now do it in assembly

[@StupdiGo]

Now do it in binary

[@NathanWoyessa]

@@StupdiGo now do it in your head

[@StupdiGo]

@@NathanWoyessa Now do it using a single neuron

[@Fred-lf4qb]

​@@StupdiGo u mean machine code binary language😂 doesn't exist

[@henryyylight]

​@@Fred-lf4qbWhat do you mean??? Binary does exist. Its just instructions for the CPU.

[@Ari-pq4db]

Subscribed, can't wait for such more informative videos ❤🔥

[@exor6100]

Having also made a neural network from scratch, I would recommend anyone else undertaking such a project to buy Neural Networks from Scratch by Kinsley and Kukiela. I don't know if this guy used that book, but its a wonderful reference.

[@ShabJimJets]

very nice mr green code very nice. You deserve a lot more subs for how good these videos are. Cant wait to see what the future holds

[@Alex-ns6hj]

I know nothing about this. Just a noob here in sophomore year learning maths and STEM exploring. I really really find this fascinating, although I can’t really understand it yet because the maths are beyond me right now and I suck at coding, I really want to learn this. I can just tell there’s a beauty to this I can’t yet see but I want to uncover that no matter how long it takes me.

[@wealth-wise2day]

"its getting 40-50% accuracy it sucks" I know this seems bad greencode but you just taught a computer how to recognize things that we previously thought were only recognizable by humans. thats not bad. good job.

[@santiagogonzalez-hc1vp]

What a great vid, new sub You summarize accurately two weeks of class of a ML Master where I didn't slept Great job doing that and understanding the fundamentals Ignore bad comments Keep the pace

[@Hangglide]

thanks! really cool! especially I just learned neural network and watching your video reinforced what I just learned from the class.

[@benedictbrophy5651]

Honestly would love a video(series) that explains and derives the math

[@rebwar_me]

kzbin.info/www/bejne/boizeHRjhsmDr68si=clXJqgjOiFKEqMQm

[@chrisx2342]

ur expalination is absolutely fantastic!!

[@abdulhadiaa3372]

You are the best man, you managed to turn a boring topic into a movie. I think you are going places in the content creation industry. Keep going man 🙌🥇

[@mohamedyacinehamiham6019]

keep going mate , We believe in you

[@golgiguy456]

That was cool. Now do it "in" scratch

[@adebayokehinde1580]

Making a tutorial is one thing and adding animations is 🔥Great!!

[@jonathandyer6385]

Description: "do not click on this: " Me: *Clicks it* Me: *Sees a youtube page that says: "are you sure you want to leave youtube?"*

[@lukemaj_]

this was super cool! Keep up with these great vids! Nicely explained, convinced me to try on my own implementing a custom neural net ^^

[@commissariomontanaro2931]

why does my acoustic pattern recognition match your avatar poses to Code Bullet? Nice video tho, now do it in C to assert dominance

[@ericdanfunk3966]

This content creator is a clone of Code Bullets style 🥱

[@Smurdy1]

I think it's funny that I'm reading this while waiting for my C++ neural network to train. My code isn't even that optimized and my program only took about 5 minutes to go through the process of training a neuron 49 billion times. It's insane how much faster C++ is at machine learning (and everything) than python.

[@ggsap]

@@Smurdy1 pytorch is written in C++. numpy is written in C.

[@Smurdy1]

@@ggsap But still, those things only make up a portion of the code in most Python AIs. The rest goes as fast as Python does.

[@ggsap]

@@Smurdy1 did you do a benchmark?

[@stayhappy-forever]

Can you open source the code if you don't mind? I worked on the same project but there are some improvements I feel like I can make. Edit: I implemented SGD and got 94%-95% accuracy, with 16 hidden neurons and only a singular hidden layer (10 epochs). is it possible if you can share your model architecture? Thanks!

[@yds6268]

Can you share yours? I know links in comments are impossible, but the repo name would be amazing

[@stayhappy-forever]

@@yds6268 Do you want the whole code? or do you just want an understanding/run through of what i made

[@mattkarten3196]

I've tried to make a neural network from scratch multiple times. Although, no matter how much I try, I can't get back propagation to work. With my most recent attempt, I double, triple, and even quadruple-checked my code to see what was wrong. After hours of searching I STILL couldn't find anything. If anyone has any ideas or advice, please let me know. Thanks!!

[@lxhub]

Thank you brother. you're a LEGEND!!!

[@helved807]

This was super interesting! Mainly because this is something I've wanted to do myself, but I've had some troubles implementing backpropagation. Any tips on how to implement it?

[@SomethingSmellsMichy]

*Disclaimer: After writing an explanation, I assumed you understand calculus, but all the steps are broken down and the equations are solved fully. In this video, you were introduced to the loss function. This function does as it states: Quantifying the difference or incorrectness of your Neural Network. Ideally, you want your loss function to be 0 or close to 0. Assuming your loss function is either cross-categorical cross entropy (like in this video) or a more common approach: Mean Squared Error (Which has the formula 1/2(y-o)^2 where y is your expected output and o is your actual output). *In case you want to paste this into an appropriate calculator/document and because I've been practicing; here is the LaTex version of that equation \frac{1}{2}\left(y-o ight)^{2}. Since these functions are never negative, you can assume that your loss function reaches 0 at its absolute minimum (usually you'll hit a local minimum for complicated problems). For Instance, we can tell if a function "f" hit a minimum or a maximum by checking to see if f'(x)=0 (same as 0=d/dx f(x)) or if its slope is equal to 0. Since we know that if the slope is positive at a point as x goes towards infinity then it's rising. If the slope is negative at a point as x goes towards infinity then it's falling we can find the x position of the local minimum by iteratively changing x by x->x - f '(x) * lr (lr being the learning rate). X is usually represented as the input to the function, but since we want to change the weights and not the input, we can assume that the input to your network is constant and that the weights are the input. So since you've already done the forward pass, you likely already know that a network can be structured like this: o = f (W * x). Where o is the output, f is your activation function, W is the weights matrix, and x is your vector of inputs. Assuming you're using the Mean Squared Error, let's try to find the function's derivative with respect to your weights. The entire function is 1/2(y - f (W * x))^2. The chain rule tells us that we can find the derivative dE/dW (E being the cost function) by solving dE/do * do/dW (see how do cancels). *For the sake of visuals this is the LaTex equation: \frac{d \cdot E}{d \cdot o} \cdot \frac{d \cdot o}{d \cdot W} dE/do would be represented as (y - o) or (y - f(W * x)) and do/dW would be f '(W * x) * x. Meaning that dE/dW would be (y - f(W * x)) * f '(W * x) * x. So now you can update your weights with W -> W - dE/do. And for multiple layers, you need to pass dE/dx to the next layer which in this case is (y - f(W * x)) * f '(W * x) * W (Notice how you multiply by the weights instead of the input).

[@SomethingSmellsMichy]

I'd recommend making a class that deals with a single layer of the network. The class should have a method called backward (for backward pass or backpropagation, but you can call it whatever you want) that takes in the error and the inputs for that layer. In the method, multiply the error by the derivative of your activation function and the inputs and add that result to your weights. The method should also return the error multiplied by the derivative of your activation function BUT then multiplied by your weights (preferably before you update them). If you want, I can give you some code to reference.

[@UrFada]

@@SomethingSmellsMichy I love your explaination Understood 70% of it even with little calculus and linear algebra background as am In grade 0 but started practicing calculus and linear algebra for making my own nerual network only part I don't really get is the end, and I find it hard to code something except I can fully visualize how it works so it has been troublesome trying to fully understand it

[@SomethingSmellsMichy]

@@UrFada I can imagine that the end starts to become more complicated as it divulges more into symbols. I was kinda trying to wrap it up because of the character limit on replies.

[@UrFada]

@@SomethingSmellsMichy Ahh yes I will try to better visualize the comment later and maybe I will able to understand but overall thank you for the explaination

[@alienwhitewalker7284]

Man mastered, If you can't explain it simply, you don't understand it well enough.

[@medakshchoudhary]

loved the way how you explained all of this can you pleaseeeee make a beginner guide tutorial on how to get on to things like this exactly this type of things you got a sub

[@kartikgdrbd]

+1

[@rnts08]

Your code bullet is better than codebullet. You are what everyone hoped that he would be with his enigma video. Keep it up! 😂

[@shivanta2]

You explained it so well, I loved it even though I understand the math behind it. ❤

[@dhiraj6727]

The background music makes it look like this topic is just really really cool. Or maybe it's just because I have watched gaming videos with such bg music that similar bg music videos make me feel interested in the topic.

[@enerz9135]

Your Exaplation Is Very Simple It's Very Easy To Undertand

[@احمدعاطف-ت7ذ9ص]

This is my first time watching you. Bro you are awesome, keep going, I love your explanation.

[@LudieMasu]

your creativity and passion shine through every project!

[@softwareengineer8923]

It was extremely useful, thanks a lot 👍

[@abdelrahmanmahmoud-fc5yh]

Cleverly and simply explained, great video

[@reliablespoon]

1:05 It’s sad that I didn’t even flinch at that math problem… that’s what my math classes have gotten to 😭😭

[@Bigleyp]

Now do if but where there are regions, a neuron has multiple connections and it is either on or off based on how many recent signals go into it.

[@divandebruin5767]

Hey man just wanted to find out where you made your avatar! Thanks ahead 🙏🏻

[@familytharun3924]

the video is too gud bro , u have taken lots of efforts..

[@afrateam6241]

Only a genius could understand how genius you are . Wow 🎉

[@Badaboombadaboombaby]

Mad respect for animating your every syllable! Amazing lipsyncing animation ❤❤❤

[@jollyehiabhi360]

I love your explanation. Which parameter(Hyperparameter) did you tweak to move from 50% accuracy to 97% accuracy?

[@sakchais]

This is incredibly fun to watch!

[@trickyfox9518]

This guy has been calling us dumb for 9 minutes

[@Podcast.Motivator]

Awesome bro. Waiting for more videos like this.

[@weslycosta348]

so underratted channel man, keep goin!

[@nicolasdelphino6434]

Dude, i'm actually in the master degree's classes, not understanding shit. I just watch your vid, and it looks bright as the sun

[@LebaneseJesus]

Instant sub, brilliant video

[@-PeterAndrewNamoraMarpaung]

make it from scratch, on scratch, with a scratch, and top it with a neural network title made from SCRATCH

[@EchoPrograms]

I just did the same thing a few days ago lol! (Also from scratch). I did it in JavaScript so I didn't even have numpy. My matrix class is like 150 lines long lol

[@CastyInit]

relu is just math.max(0,x), or just a fancy way of saying "if the output of a neuron is

[@TandaiAnxle]

You should upload more def one of new fav youtubers in general

[@ghostflex642]

Question: what is numpy? Can I use pycharm? Or is it not a ide

[@victormanuelrangel9932]

Hello, I have a question: How can I make a neural network to detect formulas, characters and mathematical numbers? 😅

[@BlackKiller-j7m]

Bro it awesome explanation and this visualization is very helpful to understand. Keep it up and I hate the maths but you skip and explain in simple it good for many people. I think you have to make video on maths for ai and machine learning need to solve

[@Concreteblockmachineug]

When you ask the network if it's a 7 or not..does it initially have a stored reference parameter set to compare with in order to know if the given photo is of a 7?

[@vedant_stone]

I am not an expert, but I humbly ask, why is numpy considered as something which is stock to python?

[@KeithReactsTECH]

YOUR EDITING SKILLS ARE MONEY! YOU SHOULD MAKE VIDEOS IN A DIFFERNT GENRE THAN CS VIDEOS, BETTER RPM AND MORE VIEWRERS

[@nogabs4422]

how did you create your character? was it written in code XD btw i like the character and simple movements, how you do that? im wanting to learn how to youtube but i'm not a fan of being in camera

[@marlonochoaj]

Awesome video man. 🎉❤

[@rebwar_me]

kzbin.info/www/bejne/boizeHRjhsmDr68si=clXJqgjOiFKEqMQm

[@nesquickyt]

Your videos are really good and interesting 🔥

[@cen-t]

Nice video, btw do you do live coding session ? Like on twitch or here on yt

[@kshitij7791]

Hey @Green-Code do you have any github repo for your work?

[@KSBallvardhan-n3v]

Bro can u pls provide a roadmap to learn to do like you with resources, it will be a great help, thank you

[@samarthbansal3410]

College

[@rinorajeti4]

I love the way this video is made. - "Some of this guys invented the dot product" I can't stop laughing on this one.

[@docteurdocteur2743]

Could you do it without any external library ? I mean no numpy, no pytorch No pip install You go and just use standard librairies ? It would be nice and interesting to see

[@baonguyen4278]

Great work bro! u got another subscriber !!

[@bl4ck289]

Code bullet?

[@Siddhi_99]

You are so wierd, yet Funny and Interesting. Loved the way u explained

[@progerua]

yooo, very cute video, but from who u learned all that stuff?

[@AKG58Z]

How and where did you learn to code since im stuck at learning c++.

[@yassinechritt8816]

Great explanation! keep on postig great stuff 😎😎

[@nithinmahesh1]

I've been wanting to learn how to build one from scratch with no libraries since 2019 and never knew where to start. Do you have any suggestions on where to begin as a complete beginner? I know how to code and have been a developer for 5 years at an enterprise level

[@CatBoxOfficial]

Underrrated youtuber ngl

[@cubix02]

I’ve got to say-this video is like a shot of espresso for my brain cells! ☕

[@gabrielrock]

awesome video bro, congrats! I’d like to see a video on how to go from that to a generative AI or a RAG, u know? Are u planning some video like that or do u got some reference?

[@Superduperboy29]

imma be honest i only clicked because i thought it said "in scratch"

[@gray5881]

Like 😢

[@BabyYoda5555]

Cool. This explains forward back propagation. Now explain forward forward propagation.

[@sirosala]

Excelente el video !!, muy bien explicado, brillante aporte de conocimiento. Saludos desde Rosario - Argentina.

[@sarimshaikh5224]

Sir, u drop this sir 👑 , please wear it from next video

[@deadlykgaming4270]

hey man ! I don't know if you're gonna read this but I always wanted to code an AI for myself and see the actual core from where all of this technology came from..can you suggest on where exactly I can learn this hardcore coding that you do because I cannot find it on youtube

[@PythonIsGod]

Wow man, Python is love, love is Python.

[@atomnorbi8271]

I NEED you to change how you animate, dont just flip your character, the logo on their shirt flips also, please make separate models for the flipped version of your character please please please

[@Sumii4242]

Someone know what the muisc at 6:35 is? ah and also, this is seriousely such a cool video, great job mate!

[@GauravGiri-i6r]

Bro love the video.... Can you go in-depth in the coding part. Like every step by step code you used.

[@callmekelvin]

Why don't you share the code with us.

[@crazzzyork8359]

It's a secret sush 🤫

[@Antowan]

Its on his patreon the link is in the description

[@TheFuture36520]

Bruh is implementing mathematical equations like bernoullis theorems and second order differential equations 😂. You're the best brother ❤️ 💙

[@iichoco9626]

i thought this said "i built a neural network on scratch"

[@brawldude2656]

Backpropagation couldn't be explained simpler

[@Monirulislam-c1t]

9:00 the f*cking biritish moment😭😭

[@SamIsPoggers]

good video, i have a question however (might be a silly question but) would making and training a chatbot be similar to this or that would be a different thing?

[@Green-Code]

I'm not an expert on that, but that has to do with transformers (which is a different neural network architecture). So it's related, but not exactly the same. If you're interested search up Andrej Karpathy. He's the goat

[@SamIsPoggers]

@@Green-Code ohh okay thanks, I really am interested in learning about this ai stuff, I recently just started trying to find good ai videos

[@Evanarrations]

How did you make that talking cartoon

[@edwardferrari7342]

Awesome video!

[@Hussain_eng]

Ay man i used you video as a source for my report i hope you don’t mind Im in computer engineering and the report is about (Neuromorphic Computing Using FPGAs)