10.13: Neural Networks: Feedforward Algorithm Part 2 - The Nature of Code

[@RodrigoCastroAngelo]

I've been trying to study neural networks for the last couple of months, and this video series is being surprisingly helpful. Most other more 'formal' courses are much harder to keep up. Your videos are indeed somewhat more loose, but I could finally understand how to code a neural network, so thank you!

[@TheCodingTrain]

This is so nice to hear! It's really hard, I'm trying my best!

[@nilsonsls]

I'm creating a neural network, but I got stuck at some point of the backpropagation implementation and I ended up here watching your videos. You're so great at explaining! I'm glad everything looks so similar to my code so far. Hope my answer is in the next video, cause I've been watching those for 3 and a half years so far lol

[@OLApplin]

Doing this in Vanilla Java while reading the book you mention ... so satisfying when it starts working!!!

[@finninnes2973]

How'd you get on with the java version?

[@dalegriffiths3628]

Dear Dan, I worked through Tariq Rashid's book based on you recommendation(using Jupytor notebooks and python), now I'm working through this in Javascript. You are a great teacher!

[@priyankarsinha685]

0:33 😂, you are the ideal teacher that we require but we do not deserve. Keep going sir, understanding the subject is so easy when theres teaching with a happy, excited face.

[@tiresink1978]

Will you plz tell me language name of algorithm and software in which it is written

[@mandeepthebest]

You are just the best. I have been taking courses on ML and DL, as I am trying to transition from ChemEng to Data Science, but didnt understand them fundamentally. Omg, you are making me feel part of computer science community, now that I understand it.

[@Aditya-hk7vn]

I was late on my AI class yesterday because of work - Your video is the best video that explained about neural network, and the coding part actually made me understand it even better. Thank you! I am sure there are thousands of people that find your video helpful and very informative!

[@alixhurtado]

Side note: convinced my school department to let take an independent study as a tech elective and just follow your videos and code a neural network so thank you

[@TheCodingTrain]

Cool! Where do you go to school?

[@alixhurtado]

The Coding Train, Northern Illinois University! Studying IE on my last semester!!!! Still needed a tech elective and my curriculum isn’t the most updated, (really manufacturing focused) so I wrote up this proposal saying I’m more interested in big data and machine learning so now I’m doing research with my professor and learning together!

[@polowesh]

Merci Beaucoup !!!!!!! You're doing great. Everything is so simple with you aside. So much energy and will to teach. I can perfectly understand what you're explaining althougth I'm french and I don't even know what JavaScript is (I'm a python beginner). It's now 1 am and I've been watching you're neural network lessons since like 10 am (yes from the previous day). Thank you so much coding train !!!

[@TheCodingTrain]

Merci!

[@demian2435]

How do you make everything so simple? you are great!

[@amitbansode]

Presented in so interesting and simplified way ... God bless you

[@junior75980]

This is the theme in my research in fermentation's reactions. That is awasome that my a ideia is not so away of the road by applying this tecnic in de excel/vba code. I hope that you keep going this nice work. Spread the knowledge is so amazing thing to do. Thanks for sharing!

[@alixhurtado]

You’re the best lecturer ever

[@letterborneVods]

I love this channel so much ♥️ I finally feel like I understand Math! I might actually watch all of this channel! (But with my pace it’ll probably take me years lol)

[@clbgrmn]

This was a great tutorial! It was a great combination of your quirky teaching style and being quick and effective! Keep it up! :) Also, you should call the 'train' function 'codingTrain'

[@TheCodingTrain]

Hah!

[@vinitsunita]

Man you are doing amazing work. I love your teaching style.

[@bogdyvyp]

7:50 isn't that going to add the bias matrix to the weights matrix in a element by element fashion resulting in adding the bias to every element so later when you do the sum it will have the bias added in 3 times instead of 1?

[@fr3fou]

i was gonna ask the same

[@omicron296]

Cara, você é muito bom! Sigo seu canal em sempre aprendo mais! Obrigado!

[@MultiLeandrini]

seriously I'm so sad I can only like this once!!

[@raigiku7272]

Amazing! After spending like 9 hours researching this stuff, your videos are the most helpful ones!

[@lewisrichardson4671]

Hi, Firstly this is an awesome channel and it is seriously useful. I am struggling with one thing at the moment, how does the matrix product function work when there is only one column for the inputs? As I see it, to get h1 it loops over the columns of the weights and the rows of the inputs....but then when it moves to h2, it moves to the next row of weights and loops over its columns....but there is no second column to loop over the rows for the inputs??

[@lirongsun5848]

I watch through all the series

[@Pablo-pk2qi]

This "hello" in first second looks like "I'm sorry this series is so long already" :D

[@zet723]

as always perfect, keep on going Prof.

[@AneeshAbbineni]

You can make an array of weights and biases, and use that array system in the algorithms.

[@MatichekYoutube]

This series is dope. Thank you for sharing.

[@neelpatel122]

this guy is crazy and i like it :)

[@10xGarden]

I love you Dan I love you Man...You are Awesome!! ]

[@themirlabs]

Im trying to build a nural network and im a little confused. Dose each layer need a seperate bias input value? because i thought a bias input would be enough? Also why are you making entierly seperate bias matrix's? how do they fit in? maybe i need to watch this again...

[@lirongsun5848]

Excellent explanation

[@theRECONN]

Take in account easy changeability of the activation function (maybe even for each layer). After you are done you should try other (like ELU) activation function and compare Cost Plots :)

[@pallabchakraborty3821]

Thanks for Excellent Tutorials!.I want to know how many neuron or nodes should be inside of hidden layer?? waiting for your ans... Thanks

[@kuhlde1337]

As a primarily C-based language developer, the lack of datatype declarations for variables in JavaScript always makes me a little anxious.

[@samuelgunter]

I have the opposite problem. i'm mainly a javascript programmer, so when I try to use processing or another language, i get somewhat confused of why i have to declare what the variable type is and what the function will return, and what type of variable is the input , because i want to be able to either import a instanceof the neuralnetwork or just the input nodes which is easy in javascript.

[@Nicolas-pd3vy]

16:39 why output isn’t a matrix ? When did it become just a number

[@samuelgunter]

he wrote a function, toArray() { let arr = []; for (let i = 0; i < this.rows; i++) { for (let j = 0; j < this.cols; j++) { arr.push(this.data[i][j]); } } return arr; } that converted the output to an array. Also return output.toArray(); means that it is does the function toArray() and returns the output, and then print(output) in setup is the same as console.log(output);

[@airxperimentboom]

I just want to let you know that I am one of your biggest rainbowed fan :D it's so interesting and I always wanted to learn such a thing. For this purpose, I watched the 3blue1brown videos and then TADA I saw you uploaded a video about neuralnetworks

[@TheCodingTrain]

Thank you!

[@SourabhBhat]

Using a sigmoid function for output 'Y' will restrict the output to range [0, 1]. Will it be better not to use the sigmoid for calculation of Y (i.e. the last layer)?

[@magsudhajiyev7810]

Can you or someone else please show me where you code the nn.js file?

[@samuelgunter]

in this video? i dont understand your question

[@shakhawathossainshihab7998]

@@samuelgunter where is the source code of this video?

[@hullopes]

I can imagine that you'll treat about decision criterion of the class with the output value (threshold?) in the following videos. But now, I'm asking myself: how we adjust of W in feedforward NN??

[@xot9897]

does anyone know why you multiply weight* inputs instead of inputs* weights?

[@samator1236]

So when multiplying 2 matrix's, i thought the collumns of a had to be the same as the rows for b? So why does multiplying the input with the weights work? Because to my understanding, the weights are 2x2 and the inputs are 2x1 in the example you went through. I've been following along and get errors at this stage. Could you explain how the multiply function works? Im struggling to grasp it. thank you :)

[@TheCodingTrain]

I recommend you join our Discord where you can ask code related questions to the community : discord.gg/hPuGy2g - The Coding Train Team

[@narahamashi]

i made a version of matrix library in ES5. everything works fine except when i call the feedforward it returns NaN . i cross checked and found if i call the matrix multiplication from the nn library it results a matrix of NaN. could you guide?

[@greatdouble8966]

Why did you randomize the whole bias_o (and bias_h) vector, when the bias for each weight matrix between two layers should be only with only one particular value?

[@juangreen8194]

Hello, could you post a known output for a given input including all the intermediate weights and biases so I can verify my code is producing the same results, thank you.

[@TheCodingTrain]

This is an excellent idea. I'm not able to do this right now but you can see my code in progress here for the time being (drive.google.com/open?id=1BFim9qORvxFxUqiLrS-g-spbautXtADd). Github link forthcoming.

[@MarneyRS]

does the bias weight have to be random too?

[@no_more_free_nicks]

I would appreciate it if you started with a test. Then in the test you can specify your expectations (like the XOR function) and then backfill the code.

[@zurym4962]

Could you make a video where you create a neural network using R programming?

[@nagesh007]

Amazing , Thanks a lot

[@sofianbenhamman2038]

So the Perceptron class, that you did before in one of your previous videos, you will not use it here?

[@TheCodingTrain]

I'm starting over in order to use the matrix math, but the same ideas should map?

[@ssslava]

Amazing series!

[@crisiberny]

I'm a bit confussed about the Map() function. Isn't there already another function called the same that maps one interval to another? Won't this cause problems by having the same name? Sorry for my english, not my mother tongue.

[@tiresink1978]

Will you plz tell me language of this algorithm

[@olufemitosin3526]

I love your work and jesture keep it up man

[@olufemitosin3526]

How do you get paid for your work?

[@unchayndspersonalaccount7690]

16:15 I don't know if you're still reading comments on this video, but you should make an iteration song for your soundboard for when you're typing "for" loops XD

[@AdityaFingerstyle]

Cannot thank you enough Dan.

[@MarneyRS]

im having some issues, my output is always between 0.71... and 0.98... its never one and never less than 0.71.... i have tested with thousands of inputs

[@tiresink1978]

Will you plz tell me language using in algorithm and software name in which algorithm written

[@samuelmarques2074]

I have doubts about bias, its value is 1, or is it random? Great video!

[@views-hh3el]

Bias -1 is an input too of that neuron, so is affected by a weight too, W*X-Bias*(weight)=0, that weight change when the neuron is learnig

[@luca-lewin]

I've converted the code from Github into Java, but the 'dot' function does not work

[@HeyItsSahilSoni]

This video is unlisted. You might want to publish it and put it in the playlist..

[@PhilBoswell]

I think he normally uploads a whole live-session's-worth of episodes in one go, but only publishes them one-per-day?

[@HeyItsSahilSoni]

no, i dont think so, I think he forgot to publish this one.

[@PhilBoswell]

Is that better now?

[@HeyItsSahilSoni]

Phil Boswell hahaha yeah

[@AugusDogus]

How did you find this video 3 days ago (at the moment of this comment's writing) when it was unlisted?

[@amitkumar-eb3se]

YOU ARE GREAT.....FANTASTIC

[@unnikked]

I might have an idea for future videos. Train weights using Genetic algorithms since you've done some work about GA in the past. Wha do you think?

[@TheCodingTrain]

Yes I am planning to do this for sure!!

[@Hydrozoa]

unnikked I have two videos where I explore this exact concept

[@unnikked]

NEAT 😂

[@Hydrozoa]

unnikked The authors of the paper came up with that 😌

[@unnikked]

I know, pun intended :)

[@nandkishorenangre3541]

Thank you , you owe me

[@derekbaker4299]

Is the complete code for this video available anywhere?

[@derekbaker4299]

I found this, which appears to be close enough: github.com/CodingTrain/website/tree/master/Courses/natureofcode/10.18-toy_neural_network/lib

[@8eck]

Did i miss something? Shouldn't inputs be summed after multiplied? Where is the sum step?

[@8eck]

Oh, forgot that he is doing dot product inside multiply method.

[@vkquads]

You can also normalize your inputs for better results.

[@views-hh3el]

So the inputs are [1,0] the output is [0.59310]? I'm sorry but what's the idea of feedforward if you don't have a tarjet? the neural network reacts to that inputs and gives you a "whatever" number? it's unidireccional, there's not learning nothing

[@samuelgunter]

this is just doing the math for the outputs, back propagation is where it trains to get the desired output

[@tiresink1978]

In which language this program is written... And software.?

[@Pablo-pk2qi]

Java script so far. (ES6 version)

[@evanperrygiblin]

glad to see you got your shit together

[@emilie1977]

I love this video!

[@gabrielkwiecinskiantunes8950]

Great video, bro. Hold back on the Adderall tho. Still, I learned a lot. Thanks.

[@mehmetrzaoz9016]

Thank you for that great work. But i can't find the source code.

[@TheCodingTrain]

It's all in these two places: github.com/CodingTrain/website/tree/master/Courses/natureofcode/10.18-toy_neural_network github.com/CodingTrain/Toy-Neural-Network-JS/

[@mvdlele29]

it would be nice that fromarray function be a constructor

[@thewarlord269]

Excuse me ! I noticed that p5 is needs you to pay to get it, i used to work with p5 earlier and it was free to download So we can't get it for free anymore !? Please i'm waiting for your answer, Thank you!!

[@TheCodingTrain]

p5 is a free, open-source JavaScript library supported by donations. See: p5js.org. For more processingfoundation.org

[@thewarlord269]

The Coding Train but jt asks me every time to make a donation befor I could install it ,! Please can you help !?

[@RodrigoCastroAngelo]

Hi TheWarLord, I could download it here without making a donation. After the file is downloaded, the p5js website indeed redirects me to a page with instructions as to how you can support the project, but the download is working here. Maybe there's something happening with your browser?

[@thewarlord269]

Hey ,Rodrigo Ângelo; yes yes i found the problem , i had an add blocker, it was blocking the installation as well, i just disabled it and it worked fine, thank you anyway !

[@yubrajsharmadhakal6446]

I mean seriously, who the hell disliked???

[@joshstafford9353]

16:07 Just kind of sums up Dan doesn't it :D

[@adityahirwani1672]

You are amazing

[@QG1996]

I thing we need weights for biases too, don't we?

[@syawadhilahpradipta828]

What language are you using?

[@TheCodingTrain]

This is all JavaScript.

[@rusifbudaqov2712]

thanks for videos.It is verry super)

[@mahalakshmil1]

Awesome!!

[@geoffwagner4935]

This is very convenient when the author also points out the kind of lunatic we deal with when we hit pause on a regular basis. this is the real Dan, this picture right here. one should not be fooled by this calm, collect, well trimmed facade before us here. he is showing us the real Dan here, the one trying to escape, all the time, bleeps and bloops occasionally. i'm sure it would be quite a spectacle to let him loose in a data center while he thinks he's alone and it all comes out. Unintelligible bleeps and bloops, a roller chair, spinning in circles, madness, and no English . probably this face the entire time

[@hichemsetif1741]

Awesome.

[@FlareGunDebate]

After all that I feel like I should get a 0.5931076632680795 tattoo haha what the heck.

[@sparknews2218]

how to do this in python??

[@tiresink1978]

In which language this code is written?

[@geoffwagner4935]

wow, this is shockingly more obvious than brown blue. this i very interesting with dot products from a matrix with a couple nodes. freakishly obvious lol hopefully i'm not speaking to soon. the way that just "clicked", felt a needful impulse say something. super wow.

[@webarify]

Adaboost Ensemble Learning pls sir.. :)

18:07 loooooooool

[@ritikkhatri]

16:14

[@toastyPredicament]

When do I get to be with someone ?

[@seabiscuitthechallenger6899]

👍

[@ayaayari5666]

thank youu!!!!!

[@felipedejesus6696]

friendly! 😄

[@Gentleman217]

[@alixhurtado]

I love you

[@Stehocke]

Once, there was also this video - Neural Network + XOR operator: kzbin.info/www/bejne/gZzaiWp8oLGZq8k I really liked it, will you make it public again?

[@professordrabhijitsayamber2299]

Om shanti k

[@KemiksPL]

You really hate simplicity... toArray() { return this.data.slice() }

[@PeterCollingridge]

Slice returns a shallow copy, so if you have a 2D array like here, you'll end up with references to the internal arrays, so changing them will alter the matrix data. Which I think you want to avoid.

[@KemiksPL]

Thank you, I didn't know that

[@adailtonmoraes5981]

Depois falam que professor brasileiro que parece meio louco olha esse do usa ai brincadeira kķkkkkkkkkkkkk.

[@vasyapupkin9338]

doo doo doo

[@maxtwin5380]

Hi Goofy

[@alixhurtado]

Please marry me

[@toastyPredicament]

When do I get to be with someone ?

[@toastyPredicament]

Else