🖥️ POSTMORTEM FOR MY FIRST MACHINE LEARNING GAME! (4/4)

[@_dr_ake]

I'm still impressed at your ability to speak without moving your lips

[@woobear01]

Retired developer here and after watching the 4 videos you make me want to program again. Thanks for the juice.

[@xamael1989]

James Frohliger dont retire fellow keep the fire burning✌✌✌

[@Rostol]

i downloaded visual studio and unity after these 4 videos, so yeah i get it

[@woobear01]

I love being retired but I also love exploring the coding landscape. Always will. Currently chasing golang + graph databases.

[@jabahpureza]

Obrigado pelo curso! Um salve de São Félix, Bahia, Brasil.

[@kenchang3456]

Former programmer here too and this was so approachable and the topic is so amazingly interesting I am grateful for the work presented here.

[@emperorpicard6474]

The reason why Forrest failed is because you didn't incorporate Jenny into the reward algorithm.

[@deepfakestudio7776]

Nice 1

[@b7e286]

hahahaha!!

[@Crisosphinx]

He also didn't have a box of chocolates.

[@emperorpicard6474]

@@Crisosphinx My ma always said, a neural net is like a box of chocolates, you'll never know what you're gonna get.

[@seanld444]

1st video: 'I AM LAZY" 4th video: "Don't be lazy, bro."

[@danvartan]

That was one of the best video/series of machine learning that I have watch. Machine Learning Games! Wow, that would be awesome!

[@TheMarkFeet]

Forrest has evolved already if you tell me: he blamed lag.

[@voxorox]

We'll know he's really getting advanced when he starts accusing the courses of cheating.

[@MrAj198]

Hahahahahhahahahaha!

[@malamhari_]

lmao

[@Alfie275]

How to stop lag affecting the results: Use FixedUpdate and fixedDeltaTime instead of Update and deltaTime. Update gets called every frame, so the actual time between varies. FixedUpdate will get called with a constant average frequency (note actual period in real time may vary, but thats why you use fixedDeltaTime). The way it exactly works is that the time since last frame will be added to some accumulator, eg the accumulator is currently 0 and then you get a frame after 0.023 seconds, it will now be 0.023. Then the first thing (before physics) is that while the accumulator is above the fixedDeltaTime (say 0.02), it will do a fixedUpdate and subtract the 0.02 from the accumulator. Then there will be 0.003 left in this example. This is useful as the "leftover" time accumulates, until you get multiple fixedUpdates per frame (of course, you may be getting many anyway if framerate is bad). Also useful is that each fixedUpdate is called before each physics step, so anything affecting the physics (or relying on it) should be done here as well.

[@filmgbg]

What this guy said. Sure hope he sees this comment.

[@MaddieM4]

On top of this, I'd like to say, the most likely REASON for this unpredictable and seemingly nonsensical lag, is garbage collection. Memory optimization, especially lifetimes and explicit collection timing, are probably the most critical improvements that the game could have used.

[@filmgbg]

Philip Horger You can't do stuff like that manually in C# though, and I'm pretty sure you can't in javascript either.

[@Alfie275]

filmgbg you kind of can, using techniques like object pooling, where instead of creating/deleting objects, you take from the pool (and create if needed) and return them when not needed. Since the pool has references to unused objects the GC doesn't kick in untill the pool itself isn't needed

[@filmgbg]

Alfie275 Yeah of course you can object pool, but that wouldn't apply to this project, except possibly when respawning the AI player.

[@NaudVanDalen]

The hex shapes constantly increase and decrease the distance to the walls next to him if running horizontally and even more confusing when running vertically because the distance to one wall gets large while the distance to another wall becomes small and the other way around.

[@jakobkreft7797]

Yoo you are SO AMAZING! Watched just a few of your videos but damn you are original! I'm proud to be your sub this early because you have the sauce! Hahaha much love from slovenia 😁🔥

[@SSampaleanu]

When Forrest failed on level 4, it looked like he ran directly into the wall at a 90 degree angle. In every other level, he was moving around corners, so walls would always approach on one side and Forrest would know to turn the OTHER direction. If the wall approaches from a head-on angle, Forrest would not know which way to turn since the wall would activate his front-left and front-right equally. I believe including a similar scenario in your training would have corrected this, as you alluded to in the video, but I'm still pretty new to this topic. Love this content, you inject a lot of fun and personality into this topic! thanks for your work

[@DariushMJ]

Good job! here is my analysis: #1: Put the the code in Unity's fixedUpdate function rather than the update function to avoid the lag influencing the AI. #2: Either give him a longer sight range such that he will be able to spot the dead-ends earlier, or equip him with A* path-planning, such that he will only have to follow way-points. #3: Avoid overfitting by testing him on all the maps at the same time, or by giving him a new randomized map in each iteration. #4: Add one more hidden layer to the neural network: 1 layer with 100 neurons perform equal to 2 layers with 2-3 neurons each!

[@alxjones]

All good ideas. For #4, it may be worthwhile (given sufficient training time) to use some kind of optimization algorithm to determine the best NN architecture for the application. And on the subject of time, a quicker method of training, like a version of the game with no or simple graphics, would help significantly.

[@ZackAngelSmith]

does he use unity and python? plz anwser

[@SourabhDesaiBrief]

Yes, I also, Visualised #2...

[@SheeplessNW6]

I loved this series! I think you did a great job of conveying the fun of experimenting with software, and I suspect quite a few of the people who watch this series will be encouraged to try their hand at machine learning as a result. Great job!

[@govinm6973]

Just found this channel, so much information and yet you leave the viewer wanting more to sink their teeth into. Great job! Also, looking forward to trying out some of the machine learning games!

[@WindBendsSteel]

One thing that would help Forrest, is increasing the turning speed with lower distance values, so he has time to turn around before hitting a wall. and also feed him with a backwards very long linetrace as well, so he knows what is behind him, and how he is progressing.

[@RichardVaught]

Your observation @3:26 is essentially a damning critique of our western education system: So deep in the basics that the learner never achieves the general understanding that would allow them to navigate challenges they never trained to face.

[@zitaoqiu]

I am taking machine learning class this semester, your project is very interesting! One possible problem I can think of was the input. I don't think there was enough input to train your agent well. You can play this game well because the environment was fully observable to you. However, your agent didn't have full access to the entire map. Just imagine how hard it would be if you are playing this game in first-person view. Even the top player may not be able to do well if there is some dead end trap.

[@ForsmanTomas]

I spent DAYS trying to figure out how that demo worked when I was just a kid, not even a teen. It's been with me ever since and when machine learning blew up I kinda felt I understood it straight away, though...only in a very very general sort of way. Thanks for this fun demo.

[@busytellingjokes1650]

Its up to you bro you gotta make the breakthrough in machine learning programs and forever be noted in history as the man who changed the world

[@rtesdahl]

Really fun and helpful video series. I have enrolled in an introduction to machine learning course after being inspired by your Forrest project. I'm not changing my career or anything, but I really want to understand the field better. Thank you for your accessible and informative way of talking about your project.

[@maxedwill]

Wow sick series, really enjoyed it, I feel like I understand machine learning much better after watching this than other videos which can get caught up in jargon. You keep it funny and interesting while getting into complex subject matter. I also really appreciate how you analyse and learn from yourself, it's really cool to see your process. Will definitely be watching more of your videos, can't wait for your channel to properly blow up!

[@Not_Pulsus]

Hella impressed with the fact that you powered through this project. Looks exhausting! Such good learning, though!

[@pjbsgalmbakakgwns2817]

The most fun game I have ever played , feels like co-operating with my pc to beat the stage keep being awesome !!

[@dyntaos]

I'm a new viewer and I just wanted to say, I really enjoyed the first 4 videos of yours. Keep up the good work!

[@steventeglman]

First time I've seen you talk while moving your lips. Impressed. But real talk, your channel is amazing, and has inspired me to really start looking into machine learning. Keep up the great work!

[@dfghj241]

i love this topic of machine learning so fucking much because it touches on so many fundamental topics of humanity. You have to understand biological evolution, genetics, education and learning, psychology, statistics, data analysis, programming, and probably more, but i can't think of any right now.

[@BlueTJLP]

Use raycasts next time instead of these static lines, so Forrest could adapt to situations where the lines would be too short in open areas.

[@Photosynthesisbeing]

Watching these 4 vids has really inspired me to learn more about machine learning. Great job keep this up 100k subs very soon.

[@ixon4330]

Hey man, maybe adding a “rewind to safe position” feature could not only increase Forrest learning skills but also, give you the exact moment when your Robot succeed to master any course. And with a good use of probabilities, he could adapt results to any course in real time without (too many) errors(or none).

[@darth_dan8886]

For the first time in a while I actually learned something in this machine learning series. Gotta go check out your links on video 2. Thanks for doing this project!

[@lizardy2867]

In many different NN setups; specifically more linear ones, a common theme with regards to learning is the fact that optimizations are made for general solutions to generally similar situations. Meaning: trying to make a NN learn how to deal with polar situations usually ends with less than lackluster results. Training for one thing can make you a master at specifically that thing. Training for another thing with a similarly structured dataset can make you above average at completing both. This pattern of a downward curve of percentage success over the number of things you are training for, is probably one of the main reasons animals are designed to be able to remember things, and create schemas based on those memories of different situations. This should ultimately achieve similar results to combining multiple simple NNs designed for only one situation in order to act as one NN.

[@JunkiMoturi]

You've taken a very different approach to machine learning that has gotten me fired up and ready to dive in... Time to feed that curiosity.. :D Just subscribed to learn more from you.

[@mdtapilatu]

It took me 4 parts of your video only to understand that forest is no more than what you WILL ALWAYS teach him to be and do. I hope we as human are not mere AIs to the Gods. Nice work btw👍

[@hammadshaikhha]

Awesome video, really good reflection of this ambitious project. I look forward to your machine learning game on steam.

[@JimGiant]

Something which might be worth trying is playing the game through Forrest's eyes. Try the courses yourself but without rendering the course and just showing the inputs Forrest receives. If you struggle try adding more inputs to give a better picture of the world.

[@Rostol]

Dude i cant thank you enough, both for the code and for the line-by-line commenting ... sooo easy. gamejolt here i come ;)

[@MULREADY97]

Student here and after siphoning through loads of crap monotone boring af content I stumbled on your channel, best programming vids I've seen, like a breath of fresh air. Thank you p.s. Subbed

[@gravitonschannel6944]

You are great Jabril!! Good job on the project, and better luck next time :)

[@dustyapples1394]

I don't know much about coding, but if you want him to learn quickly I think one way to speed up the process would be to give him a distance that he can't cross. ie, if any values are closer than .1 to a wall, change direction. Similarly to avoid running circles create a "cannot be in the same spot twice within one lap" rule. Also If lap 1 = success! Then lap 2 = lap 1. The last thing I can think of off the cuff would be to add a "distance from start" calculator and tell Forrest every time distance is improved, run the course that same way up until the improved "checkpoint", and then improvise from there. Similarly, if after getting to a checkpoint distance is degraded (Forrest turned around a circle and ran backwards etc) never repeat that route. Just some thoughts, if any of those just don't make sense/aren't possible weeeeeeeelllllll like I said I don't know much about coding. Cheers! *Edit: I got so caught up in ideas that came to me I forgot to add my appreciation to the comment. I just found you and so far I'm loving every video! This one decided my subscribe. Keep up the good work bud.

[@johnbeck6996]

Good series! The first issue I would suggest attacking, is running into a wall. That simply check if he travels further than the track is, it is immediately a failure. ie: choose a direction, but not if that direction hits a wall. That would stop a LOT of training failures, and aborted runs.

[@robertf9063]

Hey just found this video. My last name is Forrest. I'm going to start my Machine Learning journey now thanks to your videos. I'll do my best to make the master Forrest.

[@omgacat520]

AAAA MACHINE LEARNING IS SO FASCINATING

[@TheGamingPerspective]

Just wanted to commend you on a good series. It does a lot to give a general understanding of neural networks and you use humor well. My only critique would be to show more code. I will probably stick to watching the coding train for most of my watch time simply because he offers the attitude of love for code that you also show, but he spends his time showing the code which is the best way I learn.

[@arianh6553]

amusing and informative, man. thank you for putting in all the work to produce this

[@totally_not_a_bot]

I think a good idea might be to set a lap trigger, like you did, but make it so crossing the trigger backwards gives a heavy penalty, or even kills Forrest outright. Using a pathfinding algorithm to help generate fitness scores could be useful as well. Clearing a greater length of the path gives a higher score, and hitting the lap trigger in more time gives a smaller multiplier, with a simple function of Forrest's speed and path length being used to calculate the max bonus. Total distance travelled could factor in as well, with a bigger number relative to percentage cleared being worse. Maybe if I work hard enough, I'll know how to implement these ideas myself in a few months.

[@kevinowenburress2435]

You need to add a gradient for each of forrests "feelers" so that it becomes more negative to be too close to the wall in any one direction. Maybe use a logarithm. It's also better to use a predictive time series so you would actually have forrest running a frame or number of frames behind where he would run, and then anticipate where to go based on values and use a second, deeper network, in order to make decision trees based on which training sets to use in the next steps. And there are multiple ways to do this so you might want to look at making a neural network that tries them all randomly until the best combinations are found based on the same recursion. Then just add a lap marker at the starting point and make forrest drop markers for where he has already been and train him only on not hitting the walls and covering the least amount of distance to complete the tracks, and causing the first lap markers to be ignored at a certain point before the second lap or erase them up to a distance behind him, before they are in his field of vision, which is probably better except for in a few cases, which is a harder problem to case for randomly generating courses, which is another algorithm that is inversely co-variant with the difficulty in solving them, in some aspects. Being able to calculate that in the same means is of value as well, and you can see why now things are boring, and why it is of no value to explore things that don't make my possible as an independent conservative. Of course doing this in ways that affects people is a whole other domain and is in the realm of managing probability and risk. Since you wanted to get political and this is a matter of science involving politics and what we're not the ones doing..

[@GameCarpenter]

A couple of concerns I had that weren't mentioned, is that this is a harder problem for forest than it is for us, because forest is essentially playing the game in first person mode, and since there is no way for forest to track his previous location during a race (trail markers, memory effects), he has no way to (for example) loop around a hexagon, realize he has made a loop, and continue in a different direction. Everything else, you hit on. Especially the idea that the hexagonal tracks were a particularly treacherous terrain choice.

[@oasishurl]

This is awesome! For more complicated levels, it'd be awesome if you added a way to run into unexplored areas. Like a reward for moving away from everywhere you've already been. Which would require a real time memory of that as the agents explore. That way you could keep agents from getting turned around and going around backwards.

[@arsnakehert]

Great series, man, I'll make sure to follow your next projects!

[@martiddy]

I watched all the Forrest machine learning series and I have to say they are really amazing. Also, I like how you find new obstacle in every video and give your best to find the problem. I'd like to give my opinion about why I think Forrest doesn't play as good as a human. My guess is that since we can observe the whole track and find which path lead to the finish line while Forrest has sensors around him so he only can see what is in front of him, that's why it can't know which path is the right one when he runs a new track for the first time.

[@nammick]

I both found this both extremely painful to watch and amazing to watch your thought process and presentation and closely follows a few of my failures in NN. From the offset it was clear that without any memory, a feed forward state would need more info on it's environments. Ironically I think if you had the NN setup with just 2 more input angles this may have worked but the problem was basic geometry for fitness on your tight hex courses you were essentially getting aliasing so 2 more inputs would have removed aliasing. Very good series and well explained. I have found that if you can think of what you would need to solve a problem and going further in deep learning the number of functions you may need to use to solve a problem + wether it needs to remember some state this can give you good clues on how many inputs you need, what layers you need and what activation functions you need on those layers.

[@maroawesome3068]

Great series man .. I really love the semi technical way of explaining things and I really like your video editing Best of Luck

[@maleck25]

Very well edited and funny. You have a creative mind bro !

[@lespiedsdansleau_]

I love this series. Although I just binge watched it haha I'm a cognitive science student in Germany in his first semester. Unfortunately it'll take two semesters until we start programming with python, but I'm still interested in an in depth versions of this series. Also I'm sure many other viewers are sick of the typical superficial explanations in the many videos on neural networks etc. you can already find on youtube. Keep up the good work! I like the direction you're going with this channel

[@agu160579]

Thanks for your work, time and share!

[@KnThSelf2ThSelfBTrue]

Hey there Jabrils! I just watched all four videos in the series consecutively. This channel is so exciting, and I really hope you keep at it! I'm currently trying to teach myself how to build machine learning algorithms on KZbin, and so far I've got my first, tiny, three-neuron-big network. I'm really interested in a videogame that pits a human against a machine that learns to respond to the player's actions... like some kind of dueling or defense game. If the algorithm was good enough, the machine would always beat the human eventually, but I think players would get a lot out of seeing how long they could outwit the machine in a mechanically balanced game. Any game where player can fulfill a teacher and student role seems exciting to me. A player-versus-player game might have a bit more of a... niche... audience. But it would be fun for us data scientists to put our algorithms to the test in a "microbe army vs. microbe army" sort of game! The excitement might wear off if the global minimum of the game were found.

[@nononono3421]

You probably figured what was wrong by then, but if not here's my view (I'm not a programmer). Having made Forrest human-like in appearance may have mislead you from realizing that he was programmed like a car more than a runner:) He can't stop, he doesn't do head-sweeps, and I'm not sure if he can actually slow down. As soon as I saw that you drew those distance lines as his input, I knew it wouldn't work. When you try to reproduce intelligence, any shortcut is likely to bite you in the butt later. We can't replicate everything really, but you have to be really careful about what you will simulate. In this case, he is missing key runner-like abilities, especially considering the tracks you gave him are sometimes maze-like. It was mission impossible from day one. You gave him a car-like behavior, but maze-like tracks. In some situations, a distance will appear favorable, but then lead to the equivalent of a dead end, where there is always a path that is longer than another, but Forrest won't be able to course correct fast enough before hitting the wall. This limits his ability to spend more time learning what to do in cases where there is actually a chance to survive, because you are losing training time on cases where there is no success potential. For example, imagine there are three doors, and from that alone you cannot learn which can lead to the finish line; you go in one, it's a dead end. First, you don't have the ability to stop, then turn, to find a longer path than the dead end's walls. Even if you see a longer path in in the periphery of your vision, you lack the ability to course correct enough to take it. This wouldn't be as much an issue if the track was not maze-like, but it is, so the need to do more significant course-correction should be key. Second, when you die hitting the wall, what did you learn about the first choice you had to make, which was which door to go through? Nothing. So you wasted some training time making decisions about choices that couldn't be solved. Ideally, you would have learned nothing about which door to use itself, but at least would have survived out of the dead end and continued, learning course-correction all the way, losing no training time. So if you want Forrest to be a "maze-runner", you have to modify him. He cannot predict what won't lead to a dead end, so he has to be able to correct his course when options narrow too much. Maybe add the ability to slow down and look for longer paths. Otherwise, you have to alter the design of your tracks to fit with Forrest' abilities and make them less maze-like, and more car-race like. Cool stuff though, subscribed!

[@giancarlo9866]

I really like your comment mister :)

[@DigitalAwareness]

Awesome series. The main issue is that Machine Learning/Neural Networks are not AI. They only work to regurgitate previously trained data. So you would have to train Forest on one course until he mastered it then he would replicate that. If put into a new course he would undoubtedly fail as his training would be as good as random for the new course. Just because he is trained on a few different courses does not mean he has the ability to make artificial judgement in a new situation. This also depends on how you have setup the game and training. My suggestion would be to gather distances from the walls around him, then it will be like he has sight and can learn which way to go based on that sight. when properly trained with "sight" he should be able to tackle any new course and eventually find his way around it without fail. I presume he is currently just randomly running in a direction and memorizing a path which he improves upon every iteration. With that approach he is just memorizing a specific course when trained over and over, no real cognition is happening. Great job though and loved your content.

[@kl8036]

Great content! Thank you for sharing and explaining everything you learned. It expanded my scope on reinforced learning a lot.

[@puliukonpronssi3429]

This series was really interesting keep up the good quality work!!

[@imnotselma3305]

Sounds like this will be a pretty sweet year (also I don't think I'll ever quite get over just how creepy Forrest looked when he was waving)

[@iamundergrace]

Second process that monitors just his forward. When the forward gets very low or close them then calculate average 2 on left then right. Then go the way of higher.

[@jeffersontbraga]

Good job Jabrils! :)

[@Lefty93]

Wow. That was really, really intersting ! Thanks for theses 4 videos. Like you said, it's up to us ! That's iinspiring !

[@prozacchiwawa]

Really informative series. Thanks for making this. Feeling inspired.

[@Tinkerer_Red]

The issue is that Forest was given 5 inputs, say he is running exactly at a wall, he has an equal choice to either turn left or right, causing them to fight each other. try giving him SLIGHTLY randomized benefits. Also front, fleft, and fright should all be double lines, just slightly off from each other, that way if it's a thin pillar, he can detect that it is ever so slightly to one side.

[@RPAChampion]

Bro, I swear I thought to myself once I was finished watching this I’d see your subscribe count and it was going to be in the millions.. haha Great content and super informative I’ll be coming back for sure.

[@VeNoM0619]

Ever since I watched the first video, I always felt there was an issue with limited input. Like the roundabouts: how does he know to go left, or go right, when he has no idea which direction is the goal. If you are given a T intersection while driving, how do you make your choice when you are told only that you can go left or right? Nothing can prepare you for that decision, you are given limited input, therefor it is an algorithm to "predict" a "coin flip", which doesn't work. Was he well trained for a first person perspective runner? Yes, of course, since its a maze to him, but he won't know he's in a dead end (and can't recover from entering it, once a random bad choice leads him in it).

[@calebmcnevin]

I definitely think using distance from the walls as fitness was a main problem. I would use length of survival and number of laps, which is what the actual objective is

[@Krommandant]

HUGE PROPS FOR THE SERIES! I admire your very thorough analysis in this episode, it gives so much insight on machine learning! I have subscribed to your channel since your videos are great! Note: on the campain video (3/4) the use of split screen was a bit overkill, maybe next time split the screen on only the AI hat have succeeded the task and have a small graph/table/image in a corner of the screen to keep count of which ones are dead.

[@baritoneboy66]

I thoroughly enjoyed this series. I am a CS major so I might be biased, but it wasn't too technical and decently engaging

[@RafaelSValerio]

Thx for the series, really interesting and fun to watch!

[@Aman-to1nj]

The middle axis (one of the 5 axis) you are using to get the distance in front of Forrest must have an algorithm that if the wall comes too close , He should take a U-turn.

[@kamiljanpozezynski7040]

What about creating an algoritm that will create map`s based on forest skills? so for example if forest is starting it will create straight without many curves, and going harder and harder?

[@eaglenebula2172]

Good idea imo

[@yosephmetal]

loved your analysis of the subject of ai/machine learning. I want to learn about this myself and your tactics are extremely approachable! TY for the community at large. this is a subject that is hard to get into with little/no knowledge but the way it is presented.... hats off! it's great! and I hope to learn more soon , as tis a subject that I'm deeply interested in! That is all! ty for your vids!

[@joubertlacerda9535]

Just discovered you bruh! Loving the vids! Loving the energy and spin you putting on the vids! Cant wait for more... prety much just binged on all your vids! Keep it up man! Definitely hitting the bell button on this channel!!

[@jdsr4c]

Great video series. Well done!

[@johnvonhorn2942]

Jabril is the coolest cat in code.

[@imlxh7126]

8:16 hot DAMN! that's eco virtual!! you, sir, are TRULY a man of class

[@someone5781]

I love your videos! The way you explain things makes so much sense, especially after seeing countless other youtube videos that just didn't give me enough of a reason to care about the minutia of ML. I would also love to help out on your video game using whatever skills I have! :)

[@josephjames2767]

Good series! Enjoyed it a lot

[@PaulBrunt]

One notable machine learning game since then....Creatures (Steve Grand 1996)! I think I've still got a super norm in that attic on a floppy disk ;-) Look him up he pioneered genetic algos and neural nets before they where trendy, his robot monkey was cool yet kind of scary.

[@hikester2010]

Maybe you should make a way for him to see around corners like put a T shape sight pattern. It's like when you use your mind to figure out a maze or find your way through a new part of a city or town you have never been to. It will be like giving forest an imagination.

[@GonzoTBA]

Hi! I really enjoyed your videos. Informative, educative, very funny. You have a talent for communication. Thanks for sharing!

[@VRwithAndrew]

Great as always!

[@ewliang]

Looking forward to your next machine learning video!

[@nitish18tayal]

I think that the corner issue can be solved if you add the probability of going back... It may take more time but eventually it will be able to tackle both circles and corners...

[@veritablepixel6904]

So I guess having forest adjust his vector in the direction with the longest ray-trace defeats the purpose of this exercise? Great series. Thank you for this!

[@mattjhann]

I think one of the problems your forrest faced is that he couldn't see around the corners like you could. Maybe weighting the average so that the straight ahead reward value was 2x that of the 45 degree value it might have helped. At some points he had to just guess because he couldn't see around the corner as well. Bear in mind this is coming from somebody that has no actual experience in machine learning, it's just some observations. I'd love to get into it, but I really have no idea where to start. Anyway, incredibly interesting and thanks for the video!

[@redeamed19]

It is interesting that the sharp turns and the round about sections seems to be...opposites? Mastering one almost seemed to mean failing the other. Given the 5 ray input I can kind of imagine why...I think....The sharp turn means maintaining a low average distance on one side, getting quickly around the turn. but in a round about section hugging that turn increases the chance of getting stuck in a loop. Could be that a wider turn means they are more likely to break free of the round about loop, but let likely to succeed at the sharp turns. I'm going to have to take a look at the code. I wonder it we may need to re-examine the input layer in in some way.

[@tyelork]

Awesome series! Just watched the first through to this one :)

[@teacamelot]

freaking amazing videos bro, subbed

[@bellidrael7457]

But what if, you added like, just one more bit of logic... make each hex its own area. Stepping into a hex gives Forest a boost on his score. However it's a one time deal, and when he leaves that hex, it becomes a negative zone, where entering it again gives him a decrease in score, rather than just seeing how long he can stay away from the walls. This way you basically make it so he's learning many things at once. He's learning that progressing through the course is good, going backwards is bad, and going in circles is bad, all because the path behind him becomes a negative impact on his score, and the path ahead of him AKA the "Untapped hex spaces" are a positive to his score.

[@thesmokinggnome]

Very awesome series

[@iffn]

Amazing video series! Looking at the reason why Forrest failed, I would argue that the 5 lines as inputs might not be enough information to determine a solution to every course. This would basically mean that the global minimum for the system is not low enough to solve every course. Thinking about this and looking at the graph at 3:30, I would assume that the closer the lowest point to is to the minimum required to solve the problem, the more difficult it is to determine, weither the problem has a solution. Meanwhile, a narrow valley would mean that the probability of finding it is lower and therefore requires more training time. A practical example of a narrow valley could be Forrest wiggling left and right to get more information from the environment, similar to the way some birds move their head while walking to increase spacial awareness.

[@wangarangmiao8888]

Awesome! Keep up the good work

[@Schrodinger_]

Overfitting was the first thing that came to my mind. When you trained, would you train on one course for a long time, and then move him on to the next one? Maybe it would be better to change courses after each weight update (multiple batches for an epoch). That way, weight updates that help him learn general rules would still be rewarded, but weight updates for memorizing the specific training course would be punished.

[@Krentiles]

Great series thank you

[@NathanHaaren]

you should add a 'distance to finish' reward, which forest will use to choose which way he goes

[@mikhaillobo9489]

Ahhhhhhhhhh he the words came out of his mouth at the end. That threw me off so much holy

[@Indygoflow1]

That's some good channel right there