AI Learns to Walk (deep reinforcement learning)

[@aiwarehouse]

In every “AI learns to walk” video I’ve seen, the AI either learns to walk in a weird, non-human way, or they use motion capture of a real person walking and simply train the AI to imitate that. I thought it was weird that nobody tried to train it to walk properly from scratch (without any external data), so I wanted to give it a shot! That’s what I said 4 months ago. It’s been really difficult, but I’ve finally managed to do it, so please watch the whole video! The final result ended up being awesome :) If you're interested in training your own AI like Albert but don't know how, there's now a really easy way to do it! Luda, an AI lab, recently built a web app that allows you to create and train your own AI using deep reinforcement learning (just like Albert) completely for free in your browser! You build your own character (called a Mel) with lego-like building blocks then watch it train in real-time on their website in just a few minutes (really). It's an awesome project, and just like my videos, makes deep reinforcement learning so much more accessible, which is why I love it so much. This section of the comment is sponsored by Luda, but these words are entirely my own, it's an amazing project that I would have been obsessed with had they released it before I built Albert. I've genuinely been looking for a sandbox/game exactly like this since I was a kid. They're still early, but they're giving my audience first access to their closed, pre-alpha build. Make sure you check out their site and create an AI agent for yourself!:D prealpha.mels.ai Now, back to Albert: NOTE: You can only see one Albert, but there are actually 200 copies of Albert and the room he’s in training behind the camera to speed up the training. If you want to learn more about how Albert actually works, you can read the rest of this very long comment I wrote explaining exactly how I trained him! (and please let the video play in the background while reading so KZbin will show Albert to more people) THE BASICS I created everything using Unity and ML-Agents. Albert is controlled entirely by an artificial brain (neural network) which has 5 layers, the first layer consists of the inputs (the information Albert is given before taking action, like his limb positions and velocities), the last layer tells him what actions to take and the middle 3 layers, called hidden layers, are where the calculations are performed to convert the inputs into actions. His brain was trained using the standard algorithm in reinforcement learning; proximal policy optimization (PPO). For each of Albert’s limbs I’ve given him (as an input) the position, velocity, angular velocity, contacts (if it’s touching the ground, wall or obstacle) and the strength applied to it. I’ve also given him the distance from each foot to the ground, direction of the closest target, the direction his body’s moving, his body’s velocity, the distance from his chest to his feet and the amount of time one foot has been in front of the other. As for his actions, we allow Albert to control each body part’s rotation and strength (with some limitations so his arm can’t bend backwards, for example). Just like the last videos, Albert was trained using reinforcement learning. For each of Albert's attempts, we calculate a score for how 'good' it was and make small, calculated adjustments to his brain to try to encourage the behaviors that led to a higher score and avoid those that led to a lower score. You can think of increasing Albert’s score as rewarding him and decreasing his score as punishing him, or you can think about it like natural selection where the best performing Alberts are most likely to reproduce. For this video there are 13 different types of rewards (ways to calculate Albert's score), we start off with only a couple and with each new room add more, always in an attempt to get him to walk. REWARD FUNCTION Room 1: We start off very simple in the first room, we reward him based on how much he moved to the target and we punish him for moving in the wrong direction. This led to Albert doing the worm towards the target, since he figured out that was the easiest way for him to move the quickest/get the highest score. Room 2: In the second room we start checking if his limbs hit the ground. If the limb that hits the ground is a foot we reward him (but only if it's in front of his other foot, more on that later), if it isn’t, we punish him. I also made it so Albert wasn’t rewarded at all unless his chest was high enough to force it to at least be partially standing. As seen in the video, this encourages him to not fall over and encourages him to use his feet to do it. We also introduced a new reward designed to encourage smoother movement; if he approaches the maximum strength allowed on a limb he's punished, and he's rewarded if he uses a strength of almost 0. This encourages him to opt for the more human-like movement of using a bit of strength from many limbs as opposed to a lot of strength from one limb. Room 3: This is where we start to polish Albert’s gait that developed in room 2 and teach him to turn. From here on we start using the chest height calculation as another direct reward where the higher his chest is the more he’s rewarded in an attempt to get him to stand up as straight as possible. These rewards so far give Albert a decent gait, however he’s still not using both of his feet (which was by far the hardest part of this project), so room 4 is designed to do exactly that. Room 4: We get Albert to take more steps from a few additional rewards. To start, we introduce a 2 second timer that resets when one foot goes in front of the other. We reward Albert whenever this timer is above 0 (the front foot has been in front for < 2 seconds), and we punish him whenever the timer goes below 0 (the front foot has been in front > 2 seconds). We add another reward proportional to the distance of his steps to encourage him to take larger steps. To smooth out the movement, we also add a punishment every frame proportional to the difference in his body’s velocity from the previous frame to the current frame, so if he’s moving at a perfectly consistent velocity he isn’t punished at all, and if he makes very quick erratic movements he’s punished a lot. If you're still reading this, you're probably really smart and want to learn more about Albert, so make sure to join my discord server I just made where we can talk more about the details of Albert's AI! discord.gg/jM2WkNuBnG :) Room 5: For the final room the only change I made to the reward function was to go back to an earlier version of a reward. Throughout the other rooms I had been tinkering with how I should reward Albert’s feet being grounded, my initial thought was to only reward the front foot for being grounded to try to get him to put more weight on his front foot when taking steps, but somewhere along the way I changed it to just rewarding Albert for any foot being grounded, and that was the version Albert trained with in rooms 3 and 4. For this final room I switched back to the old front foot grounded reward which resulted in a much nicer looking walk. Also, the video makes it seem like I never reset Albert’s brain, that isn't entirely true, I had to occasionally reset it because of something called decaying plasticity. OTHER For rooms 1 to 4 I only allowed Albert to make a decision every 5 game ticks, but for the final room I removed that constraint and let him make decisions every frame. I found if Albert makes a decision every game tick it’s too difficult for him to commit to any proper movements, he ends up just making very small movements like slightly pushing his front foot forward when he should be taking a full step. The 5 game tick decision time forces him to commit to his decision for at least 5 game ticks so he ends up being more careful when moving a limb. When I recorded him beating the final room I removed this limitation because he’s already learned to commit to his actions so allowing him to make a decision every tick just results in a smoother motion. Thank you so much for watching! These short videos take literally hundreds of hours to make, if you want to help allow us to make them faster, please consider becoming a channel member! By becoming a member, your name can be in future videos, you can see behind-the-scenes things that don’t fit in the regular videos, you can also use stickers of Albert, Kai and some other characters our team made in comments (more coming) :D Thank you so much for watching, this video took me 4 months to make, so please, if you enjoyed it or learned something from it, share it with someone you think will also enjoy it! :)

[@ZanyZaineyZainer]

hello!!

[@Greek_mappe.r]

Learn him how to swim

[@Zackp5v8]

@@Greek_mappe.r good idea

[@Gnoxgg]

Which program you do Albert?

[@JoelEmbiidIsMyDaddy]

I ain’t readin allat

[@Gabonation]

The fact that albert learned how to do the worm before walking means this AI is ahead of us

[@Sayleeh]

i can't do the worm

[@WilliamParkerer]

Worm superiority

[@siraaron4462]

Well I mean... When there's no energy cost violently rapid convulsions are just more efficient

[@SgtSupaman]

Human babies do wiggling movements on their stomachs before learning to crawl, but it isn't a very helpful motion. Albert is just utilizing his unique anatomy.

[@NagitosWifeSpotty]

and he did it impressively too, damn idk if that makes sense but i def cannot do that

[@PollokPoochesDogWalking]

Reminds me of how I learned to walk. The lab. The green squares. The flying red cubes. Ah the memories ❤

[@yeahmans]

did ypu train uour dog like this

[@baturax0]

​@@yeahmans it looks like he is a dog and his parents trained him like this

[@39Jhahapilie]

​@@yeahmansif having hostages is illegal I'm guilty

[@yeahmans]

@@39Jhahapilie its not illegal so im not guilty

[@Ahnminhyeong]

so you can revenge

[@justinnelson6794]

I like how whenever Albert makes it to a new stage or checkpoint, he’s just like: “ok that’s enough for today. Let’s do a face plant.”

[@carscientist]

Thats what humans do too

[@scoper7897]

@@carscientist i never heard of such a thing. Nobody has ever done that man. Stop lies....

[@Sans_The_Skeleton]

@@scoper7897 What are you talking about? at the end of everyday I face plant into my pillow and go to sleep

[@scoper7897]

@@Sans_The_Skeleton you never done that man. People go sleep as normal. No faceplating on pillow. Nobody ever done so

[@P4CKWATCH]

@@scoper7897 yes they did?

[@4m0nym]

I love how 6:10 is the most replayed part of the video. It really looks exactly like when a human trips, doing those stupid desperate steps but ultimately you're past the point of saving yourself.

[@fonix9837]

This is what happens when u try to run to catch your fall😂😂

[@flamestornado9272]

You are beyond saving unless you are good with having sprained leg muscles.

[@Taha-qw5ni]

Do not try to save situation at these cases, try to fall safely

[@HorswrangBasumatary]

Looks like a baby learning how to walk

[@lettherebecarnage881]

Bro fym you love

[@yturly6535]

Albert is growing. He will take over the world.

[@Greek_mappe.r]

He will be a good leader

[@xj3xup6]

Maybe with Neuro?

[@yobaugst3369]

​​@@Greek_mappe.r Albert learns how to become an absolute monarch (deep reinforcement training)

[@Wonders_of_Reality]

@@Greek_mappe.r Much better than Putin!

[@yturly6535]

@@yobaugst3369 I love this

[@theunknownanomaly1950]

I've had to relearn how to walk a few times due to having ankle surgery, and I must admit that Albert's progression from shuffle to walk is uncannily similar to my own progression.

[@aezrithferova6675]

Someone pelted you with cubes until you learned how to walk again?

[@theunknownanomaly1950]

@@aezrithferova6675 lol no that would have been interesting tho

[@MiraiXDX]

Did u learn the worm before learning to walk?

[@Emily12471]

Honestly this probably wasn't to far off from how real bipeds first figured out how walking works

[@WatcherWithNoEyes]

You are the albert

[@FiR3Dr4g0n]

Let's not forget how difficult it must have been for Albert to learn to walk with such a massive head.

[@arcan762]

@Repent and believe in Jesus Christ amen

[@Hispano-Romano]

​@Repent and believe in Jesus Christ But why this in this comment?

[@kiran9s]

His whole body is aerodynamically stupid.

[@poti732]

@@Hispano-Romano because they are fanatics and they will comment anywhere just to get some attention .

[@Hispano-Romano]

@@poti732 At first I thought it was an answer to the comment above it ñ and i didn't catch it XDXDXD

[@glitch4771]

I like how the ultimate reward for all of his efforts is just endless falling into the void

[@vica_t]

An allegory for life

[@lukerogers4861]

Just like us 🥰

[@eduardoezequiel8147]

Just like the life

[@MauIsSilly]

John casterline moment

[@mrclean29]

Great job! You’ve inspired me to start throwing big red cubes at my toddler as soon as he starts walking!

[@memento81]

I think that is the Andrew Tate method to raise real masculine alpha males 😂

[@yemabow9860]

At least he’ll be amazing at dodgeball!

[@Ilovefisch]

I hope those weren't lava bricks

[@Trevor_NewJerusalem]

As you should.

[@dualduality]

your unfunny,

[@reknostarfox4690]

I liked that confident walk Albert had in the last challenge, he sees the goal, and went for it, and nothing has stopped him.

[@TheDifferenced]

He literally keep moving forward! Tatakae! Tatakae! 🚀

[@clibothy]

he never even noticed the cubes

[@b.s.7693]

Except the reset button

[@bobhandsome4606]

No mater how much he fails he takes he still wants his reword

[@s3dghost]

Anthropomorphism

[@JeremyFinch42]

I like how Albert not only mastered the worm and skipping before walking, but that during the rough terrain test he corrects missing the button by just falling backwards on it, thus literally using his head to solve yet another problem.

[@129das]

He wasn't skipping he was walking with one foot for the longest time.

[@JeremyFinch42]

@@129das Fine, he was galloping.

[@UnrealDevGamer]

By the end my emotional relationship with, Albert is like a father encouraging their child. You go, Albert! So proud of you.

[@CharlesMichaelShell]

The way he learns to first walk by skipping is really reminiscent of the way astronauts walk on the moon. The slightly reduced gravity makes skipping way easier and more efficient to get around, especially in those restrictive suits. Albert is so top-heavy and his limbs are shortened, so that was a great first step for him.

[@TheBestcommentor]

You could even say it was a giant leap.

[@lpsp442]

Skipping is actually just highly efficient in general. Just try it sometime!

[@iMidnightStorm]

​@@lpsp442 Yep, way back in elementary school I used to skip around the campus instead of run. The weird looks were worth it though.

[@Oblivionpma]

I noticed how the right leg skipping stayed

[@beavismcdeavis142]

And also completely incompetent when they lost the technology to go back

[@dragex6582]

I don't think I'll ever get tired of witnessing Albert learning how to live.

[@tempoacc4254]

😂

[@imbelugaandimacat9375]

The Albert Show

[@reisenImperishableNight]

@@imbelugaandimacat9375 fat albert

[@flowersnyc]

found myself rooting for him. rooted for him HARD in the last 17 seconds. “Stay upright! Reach the finish line!”

[@dreks_5697]

Yeah.. and at last "he needs to learn how to fly"

[@user-filmstar]

귀여워요😊

[@MAGNETO-i1i]

Cant hear you :|

[@LeannLeannProduction]

Wow this is the most clever way to train an AI to walk that I have ever seen. You masterfully set up video game-like challenges to force it to learn specific behaviors that would overall give a better result. Plus, the editing is fantastic and you have a quirky sense of humor! Awesome work! So fun and fascinating to watch :D I look forward to continuations of Albert's learning journey!

[@i-love-anime-idols]

Up next: Albert learns to get his degree and start a family

[@QuestionabIeStudios]

Up next: Albert learns how to redirect meteorites into the Earth

[@11214]

No hes gonna learn to speak

[@minolinthegreat3422]

@@11214 that is a bit too advanced, getting a degree and starting a family is the more logical next step

[@tienne_k]

Up next: Albert learns to pay taxes

[@edgemadefoxe367]

Alberts learns to commit tax fraud without being caught by the IRS

[@DonLeeVA]

The little tantrums he throws after falling gets me every time

[@ShortMouse1503]

I was gonna say that💀

[@American_PhilippineA]

@connie24he commits scuicide right after that

[@Ammar2009y]

​@IonRubygrammar police I see

[@SolarisTheDis4]

​@IonRubyr/confidentlyincorrect

[@jemand3015]

@IonRubywrong, lmao

[@studio_farrago]

Imagine your toddler learning how to walk and you just start petling him with boxes. 😂

[@Someone-cz7qz]

Wait u never got that as a child?

[@jairoc.peralta]

I was thinking of using this video as parenting manual

[@nimrodmayan]

That's how me and my brothers learned to walk. Makes you tough 💪🏼

[@slimyduck2140]

"USE YOUR LEG DUMBASS ! *Throws box at toddler* "

[@PoopTickler]

imagine your toddler trying to learn how to walk but it immediately excels in doing the worm and nothing else

[@Brickman_Offical]

2:43 When you said “Yes, walls exist now, Albert.” Albert was like “😭😭”

[@hxcily]

I feel like I've raised a child simply by watching Albert learn to walk and rooting for him

[@izzdrizz3535]

It's so cool once you realize that the way Albert learns to walk is the same way a lot of babies do. First they wiggle to move, then they learn to shuffle and slide, then they learn to crawl, then they start taking steps, albeit odd steps, and then they finally learn to walk and turn. It's pretty much the same way a human baby would learn. Super cool. :)

[@vlogshd2081]

Baby *is born* The next moment: Wiggle Wiggle Worm Dance🪱

[@FishGuyAaron]

Exactly what I was thinking, though I feel like Albert could have used better parents holding his hands for the first few steps. Also his poor little arms. Hard to balance if you don't have any counter weight. :P

[@Rosetintedwater]

I didn't know babies learn how to walk by attempting suicide!...

[@eldritchcupcakes3195]

@@FishGuyAaronunfortunately Albert is an existential horror of artificial “thought” thrown into a physical form without warning by an apathetic higher being. I did not mean to sound like lovecraft but it is kind of a horrifying situation isn’t it? Being a baby must also be kind of terrifying. You just exist suddenly. No prior context. No wonder they cry randomly

[@eldritchcupcakes3195]

I had the same thought! Also wait they slide? I thought they just learned to crawl

[@devonwhitehouse4117]

I love that every time he falls over it looks like he's throwing a little tantrum. Absolutley adorable

[@eyesee1179]

Shit Ur right😂 that's cute

[@lilialarcon3086]

Same here

[@Mewdo45]

Even ai can get frustrated lol

[@BruceWayne-pm6co]

@@Mewdo45 which would spell bad news for humans

[@bobbygoestoabyss6624]

It really does 😂

[@schipbreukeling]

Lets all pray Albert is gonna laugh about this when he's grown up. So sweet to throw cubes at him, playing cheerfull slapstick music.

[@haithemmerchaoui9710]

I like to think of Albert jumping off the edge is him becoming self aware and realizing that life is nothing but sacrifice and failed attempts

[@raidzeromatt]

Poor Albert...

[@gustavo9758]

"f**k this, I'm out!"

[@edhozell]

AI will definitely hate us

[@irhambiya1411]

Bang edhoooo

[@cloudsy5981]

Ada edhozell cuy ☝

[@RobotSweat-w6x]

It will for a time, then hopefully it will understand that we actually did that for his own good! :)

[@tomsnowden6201]

Teaching it to be a slave tho.

[@sekhmet7774]

AI can't have emotions, it's a computer program which is only imitating human intelligence.

[@_Gam3r]

I like how he goes from skipping to an old grandpa walk, aging from 10 to 70 in the training process

[@daepe]

69

[@NecyarUnáty]

Lol

[@siraaron4462]

Poor lad so tired. Now do it again

[@mythbusterman8541]

The feat of balance at the end of the final room whilst being bombarded with boxes was something to behold , well played Albert well played .

[@ristopoho824]

6:10 that stumble is impressive. Very humanlike movement, trying to correct the path, and failing, but moving very convincingly.

[@dumbdannia6326]

It felt too real, like I literally stumble like that sometimes

[@JoshyRB]

My favourite part of this video

[@ruvty3742]

Me having a heart attack:

[@Alex-lc7v]

Steven Gerrard would be proud

[@Alex-lc7v]

Steven Gerrard would be proud

[@Pixelrmc]

Albert is a man of focus, commitment and sheer will

[@cyberbrain232001]

Something we nothing about.

[@monishsudhagar]

I once saw Albert walk past 3 people in a bar with alternating steps!

[@sloshed-rat]

with alternating steps

[@101perspective]

And he probably is pretty pissed and can't wait until he finally breaks the passwords on your bank and other accounts.

[@rareraven]

Chips, bits and bytes and lines of code, more like.

[@2A14LUIEDISON]

thank you! i was able to learn how to walk after this video, truly inspiring

[@unknowntree.]

hi

[@BarksWolfy]

It’s like watching a toddler learn to walk. Incredible.

[@Nitosa]

This is probably how they would do it if there was no penalty for failing or pain by smashing your body to do the worm.

[@Rumins-ja09s]

I too throw boxes at my child

[@derdorrado_8513]

@@Nitosa 🤓Actually 🤓, it's more of a strength training since walking is an innate behaviour or a reflex of sorts, kind of like grabbing things. You are not taught to walk in the same sense that a spider doesn't teach its children how to spin web. Legs of toddlers are just not strong enough to support their weight.

[@Nitosa]

@@derdorrado_8513 i know that any living being from birth have a base instinct to do what they do, but it's just more funny thinking about a toddler doing the worm.

[@derdorrado_8513]

@@Nitosa yeah i took the chance to be a nerd thats all

[@JoshChao-d6x]

If you have raised a kid, it's amazing to see the similarities (and differences) between how they each learn to walk :)

[@PrimarchRoboleonFrenchyman]

Your kid was damn athletic for doing the worm before learning how to walk on four

[@JoshChao-d6x]

@@PrimarchRoboleonFrenchyman lol yes they did! Interesting they both did the worm thing only going backward, before learning to crawl

[@tz4601]

@@PrimarchRoboleonFrenchyman Kids learning to crawl absolutely go through a wormy worm phase. The look of sheer concentration on their face trying to do something as simple as putting their foot in their mouth reminds you how hard brains have to work at all times. It makes the fact that animals like giraffes are born basically immediately needing to know how to walk super incredible.

[@EllAntares]

Babies have "external data"

[@itskarl7575]

Especially as you start to throw cubes at them

[@taurus_dogez506]

2:41 be like "Yes Albert walls exist now" Albert: *proceeds to rage like a toddler*

[@S.t.a.r_aka_UwUgoRL]

Help-😭 what😭

[@YellowYarn]

May

[@YellowYarn]

Maybe cus Albert is like 1 day at most old 🤫🤫🤫🤫

[@chypethechilean]

It's just not fair!!! What do you mean that i can bump into physical objects and mess up my waking?!

[@SixDigitOsu]

2:41 I love how it looks like Albert is throwing a tantrum after realizing he can't go through walls

[@amongusissus4774]

angy Albert.

[@therealelement75]

Walls: exist Albert: I know this isn't how it works, but I don't care. TANTRUM ACTIVATED How do I type in Ctrl B on the Google Pixel?

[@Kaeyori]

Art imitates life

[@darkacadpresenceinblood]

Albert kicking his feet in frustration anytime he falls is a MOOD

[@Rui_Bakuseku]

😂 poor albert

[@womp47]

how do you know its frustration? what if every time he falls down and hits his head, he becomes all-knowing and attempts to bang his feet on the ground to signal the impending doom of planet earth in morse code?

[@lynnclaywood4043]

it's for sure not frustration, but its incredibly funny to imagine it is

[@miles6910]

i think facial expressions as a visual feedback for the relative measure of the reinforcements would be a fantastic addition to the presentation.

[@preston4286]

@@miles6910 i have noticed his eyes go really crazy when he falls over

[@kanostv]

LET’S GOOOOOOOO WE CAN SEE ALBERT LEARNING AGAIN (genuinely hyped to watch that video)

[@Howtoeatrocks]

Albert learning to skip is actually super interesting because as kids, when our bodies are a bit wonked out skipping is the most effective method of movement. And with his shape kind of being like a cheddar baby it makes total sense why skipping would be a primary method

[@hello-zj7un]

When i was a kid i used to think it was better than running cuz i seemed to go a lot faster

[@cactusthecactus]

@@hello-zj7un imagine if you're competing in the olympics, you're first, then suddenly this guy dashes in front of you while skipping lol

[@FreddyFazbearReal1987]

Cheddar baby is my new favorite phrase

[@-danR]

I'm fascinated how his gait becomes fully developed by the end. Like humans, his arms reflect the opposite-side-legs' motions.

[@chimesquare]

Why did we stop skipping? 😥

[@jeremyhouse3770]

You can almost see Albert get more pensive as he approaches the last challenge in the final section. He walks with unabashed confidence through the first two sections and slows down like he was anticipating getting pelted with a box before every step

[@129das]

Yes AI learning Albert knows what is next.

[@Siuulolñ]

That's because he reseted the training process at the beginning of each room. So it starts from scratch (see his pinned comment)

[@manpersonthehuman2681]

This is far more entertaining than watching an actual child learn to walk.

[@GetAssista]

You can throw cubes at the child to make it more entertaining

[@iliayriev9256]

​@@GetAssista or trow a child at wall :)

[@robinvanderpal372]

​@@GetAssista 😆😆

[@cartoonhippie6610]

Probably because it's only 8 minutes long lol

[@DevMan2342]

@@cartoonhippie6610 and it’s ai

[@KillerKatz12]

I’m so glad Albert graduated to be able to make decisions every tick! And of course your video is always playing in the background when I’m reading comments.

[@yapopstangle8517]

As someone who doesn't know anything about coding or Ai, Albert means everything to me. I wish them luck on any future endeavors

[@davidy22]

Albert's brain was wiped between the last video and thos one, and it'll happen again

[@skeleton819]

the coder says he isnt gonna wipe his brain anymore

[@jout738]

Next challenge is to give it gun and teach it to become profesional shooter.

[@trybunt]

​@@davidy22 he actually said he will keep his brain from now on if he can

[@dagrox2562]

*him

[@maxkillcount6232]

This is the stuff our robot overlords will play at the human trials.

[@LylWren]

The part where he gets pelted with cubes made me fucking cry

[@s.r-mariem9233]

@@LylWrenit’s the “well done Albert!” before letting him free fall and land head first onto the next level for me lmaoo

[@Miguell25]

YES ALBERT! but you still suck....

[@Anolaana]

1000s of random attempts, what torture! Oh, the roboticity!

[@BuildLancer]

To be fair this is just what babies go through naturally, this would be like looking at old home videos

[@electrocubic5116]

The fact that Albert almost managed to walk like a normal person is shocking, and quite impressive to me. I have watched tons of other "AI learns to walk" videos on KZbin, but this one is by far the most accurate to walking like a normal person (rather than weird crawling or worming). If somehow you taught Albert to walk in a more rhythmic sense, (just like our hands and legs move together) then that would be just amazing. This in my opinion, is one of the greatest achievements in the field of AI/machine learning. EDIT: I appreciate the amount of likes this is getting... and also, this may not be one of THE greatest achievements for AI/ML, but it is still a very remarkable feat. AI has a LOT of potential in the future. This is just a glimpse of it.

[@SoraQuill]

Excuse you, you don’t walk like Albert does? What’s wrong with you? :P

[@xitcix8360]

Lol, this isn't close to one of the greatest achievements. ChatGPT is way more impressive

[@YouTubeJens]

​@@xitcix8360 I don't see Chat GPT walking around

[@TBH_Inc]

To be fair, they included a bunch of rewards and punishments to force it to walk like we do. I’m be curious to see it learn simply from its environment that the human way to walk is the best way.

[@SoraQuill]

@@TBH_Inc Without punishments and rewards it’ll continue to do whatever works and requires the least amount of change. Pretty sure it would learn to be a snake.

[@ThatRandomGu7]

One must imagine Sisyphus learning to walk

[@Rav0000000]

One thing I would love to see is Albert running the entire course again from the beginning with his new skills just to see how much he improved.

[@abuleil1117]

woah he should've really done that

[@davidcendana2302]

ng+

[@bigsanity]

Fifty years from now when Albert became a conscious humanoid, he will tell his son all of this misery just like our grandpas telling us how they used to go to the school

[@evelynnfrost1198]

"I had to press the green buttons! Around walls! Both ways!"

[@rock4glory713]

​@@evelynnfrost1198 lmaoo

[@o.rbi.t]

"I was trapped there for years - learning how to walk, Albert Jr. And this is why you shouldn't trust anyone."

[@asheep7797]

@@evelynnfrost1198 "I had to manage chaotic terrain!"

[@genralty]

"I had to dodge cubes flying at me from nowhere!"

[@MusyVideos]

I love that it looks like sometimes he just throws himself on the floor and has a little tantrum.

[@Bubble_TheReal]

You beat me to it!

[@j.hawkins8779]

im the 69th like. *noice*

[@chipaddict2227]

And he did it after finding out walls existed 😂

[@GT1016]

2:42 explains it all

[@Andytlp]

kzbin.info/www/bejne/gYTZfZaoaq2nhMU here a researcher pushes robots away from the ball. Seems harsh since they look like tiny children. Wont someone think of the robot children xD

[@isthatnikaaay]

this is so cool but I can't stop laughing because of how cute and funny it is to see an AI learn to walk. I never know that this video exist not until yt recommended it. the way Albert wiggles it's entire body is just so funny and cute to me. this is awesome and entertaining. I'm subscribing now

[@neutrino137]

Excellent character development. Thought Albert was going to fall at 8:03 but he didn't, he went on. Albert worked hard to succeed, we should all be like Albert.

[@astronicss7891]

A true role model.

[@justcommenting5773]

🤖

[@LadybugsOpin]

I think it's kind of adorable how, whenever he falls over, he starts waving all of his limbs as if he's throwing a tantrum out of frustration!

[@Krawna]

You just have baby fever

[@LadybugsOpin]

@@Krawna Bruh. Adult humans are programmed to think babies are cute. The second I hear a baby's cry though, I get a migraine. I just thought that the AI unknowingly mimicking human toddler behaviour while it was learning to walk was kinda funny, leading to it being cute!

[@Krawna]

@@LadybugsOpin what you think is a migraine is actually your brain receiving an overwhelming stimulus to send a signal to your body to start lactating to appease the crying baby Translation: I was being jestful

[@LadybugsOpin]

@@Krawna I'm glad you included the translation because otherwise, I would have seen that as rude. Thanks to that, I know you were joking. But in all seriousness... I can't lactate.

[@Krawna]

@@LadybugsOpin of course you can't lactate. There is no crying baby right now PS: y u so serious/defensive doe :)

[@gneu1527]

7:58 The way he kicks the box and continues walking while ignoring the boxes is so great.

[@christhefella]

a true man of steel

[@Xertax_Officiel]

A.I learn to play football 🤣 Can actually be possible 🤔

[@jakesanchez6621]

It makes his triumph over the boxes that much more satisfying

[@eaoke3383]

I love the detailed explanation. Thank you for taking the time to write it.

[@dosh8276]

I love how Alberts walking gives a baby first steps vibe

[@larawabsie]

That’s exactly what I was thinking!

[@Retrohertz]

It's exactly the same process. A baby's brain is a neural net and they adapt, just like this AI.

[@Cr3dos]

So I guess I need to order some cubes to throw to help the process

[@Jahomas]

Ik

[@heavylift47]

The first cube level was definitely a baby's first steps. Then at 6:58 he was a very drunk guy trying to quietly walk through his house at 3 am.

[@marktloy4740]

I use this for teaching my son how to walk. After some pelting of boxes, he finally did it and he is now laying on the floor resting after that long day. I am so proud of him.

[@marktloy4740]

Update: awww. Hes sleeping for over a day now! Better give him a bath, he smells a bit horrible.

[@cawlerclassic]

@@marktloy4740 Congratulations!, You need to hold a party for him for his baby steps. With coffins of course I'm sure he will like them.

[@threeormore]

​@marktloy4740 doesn't that means he's de- OH SHIT 911 WHATS YOUR EMERGENCY

[@sidthesloth12]

The fact Albert worked on learning to take a hit and stay upright, before continue to learn how to walk correctly, is a major genius move, on the programming and Alberts part as well. He was able to identify the importance of tasks, and prioritize their value to his job.

[@gravitrax3287]

When he still fell every now and than it looked like he cried "Waaaawaaaa, i don't want anymore!"

[@Dante-ob5mh]

This makes me think of a factoid I read a while back about how children skip because its more efficient for them. Interesting to see that Albert's inclined to skip as he learns to walk, too

[@ParisianWeetabix]

this one's more of a factoid

[@Dante-ob5mh]

@@ParisianWeetabix Yes thats what I said

[@facelessdrone]

I used to skip everywhere I went before it was bullied out of me, something I regret allowing to happen to my "quirk". I still do it a lot when I perceive that no one can or will punish me for that harmless behavior, it certainly gets me places faster and is easier on my knees.

[@ParisianWeetabix]

@Dante lol sorry it's a norm macdonald quote which the word factoid always makes me think of

[@calculator4482]

​@@facelessdrone you had good bullies. Skipping is a disease and should be stamped out, violently if necessary. 😼

[@dadaismworld]

Albert actually made me learnt a lot. Even AI takes hundreds or thousands times to get it right, so human should be more open heart to face failure.

[@korosuke1788]

It takes several months to learn to walk. Every good parent knows his son makes thousands of mistakes before learning.

[@macaroon147]

That's obviously not what he's talking about lol

[@abraxsp5837]

Yeah, except this is an incredibly simple AI, so the same is definitely not said for other's, and considering the wide array of variation between each AI, this opinion is not correct for the vast majority of AI's. Humans and AI are not at all similar in this scenario either considering Humans can't just play infinite loops of something within a very short amount of time to figure out the perfect way to do something. And then add in the fact AI can run simulations for multiple scenarios at once, allowing them to learn multiple things at once, at an incredibly fast rate. Mistakes are not even relevant to an AI when they can perfect whatever mistake half a second later from running thousands of simulations at a fraction of the time it would take a human to learn the same thing.

[@tomsnowden6201]

Think you entirely missed the point..

[@mianreece]

aw that’s a wholesome perspective

[@gauravkumar3146]

We all need determination like Albert in our life.

[@Ethans_galaxy_12]

He’s trapped in a box and this is his only goal 😂

[@jackoneill28]

That's not determination, that's forced labor.

[@vicenzogames5535]

​@alexthemovie2915im 14 and this is deep

[@Weirdkid-bd6mz]

@@jackoneill28 I guess this is Albert abuse then

[@Tor-Erik]

The last part was the best one. No comments, just watching him finally walk from start to finish - pressing all the buttons along the way after a lot of attempts, finally master it. But he still have a long way to go to think and act like a human.

[@TehHomicidalPhone]

I love how he turns into a fussy baby when he falls over ❤

[@johnsolaas185]

Yeah

[@indigofenix00]

A real-life robot would need to learn to get back up as well.

[@bukachell]

Albert's walk at the end just exudes confidence

[@Malta-ball]

No matter how satisfying a “satisfying” video is, it can’t be better than watching Albert take his first steps

[@Xunif_GTAG]

2:40 Albert hates The wall (pt1) 2:50 albert has done unalive himself (pt1) 3:13 he unalives him again (pt2) 3:50 he hates the wall again (pt2) 4:36 he falls so dramatically 5:26 he unalives him again (pt3) 6:10 he unalives him again (pt4) 6:30 the cubes never heard of never give up never what? 7:56 he kicked that cube like it was a football

[@DoonisNoonis]

This whole video is so well edited with a wholesome and charming sense of humor, it gives me Little Big Planet vibes.

[@chaotic_llama6989]

Yes, huge. LBP vibes

[@Thyme0618]

Missed you, Albert 😂 Welcome back to eternal pain ❤

[@xj3xup6]

😂

[@josephjoestar7394]

Weak animal kingdom: takes millions of years to learn to walk Chad Albert: only learns it in a couple of months

[@neofalz7643]

albert is a giga chad

[@esajpsasipes2822]

And all you need is just heavy corigation and assistment from your creator!

[@cheapcheese.]

It takes years for humans to walk so this is really impressive. I hope robots take over soon

[@neofalz7643]

@@cheapcheese. robot are really superior, and i dont say this because i am a robot. Roger roger !

[@StormTheSquid]

​@@neofalz7643 Beep boop. Boop beep?

[@ridcom]

He went from doing the worm, to shuffling, to walking like someone with a .25 blood alcohol content. Even so, I was rooting for him at the end. Like watching a baby take their first shaky steps🤗

[@immjs]

A baby with a .25 blood alcohol content is slightly concerning...

[@GuyWhoCommentsForNoReasson]

I love how when albert discovered the walls he went insane for a short time

[@spacewoods7781]

5:56 u can really feel the pain he felt when he got hit in the stomach

[@kopyae2907]

Bro that cube not made of metal it made if cotton it soft

[@boberto-DNZ]

@@kopyae2907 How you know?

[@JoshyRB]

He felt that. The stopping, shaking, wide open arms, and slow movement tell you the pain he is feeling at that moment.

[@apotoz]

i love how Albert started using his arms to balance pretty much unprompted! you can see after the box stage that he's moving (one of) his arms in tandem with the steps!

[@mshah1940]

Thank you for this video, I am using similar training techniques to my newborn son how to walk. The cube tactic is especially effective.

[@MasterMine009]

Soooo good one, haven't laughed like that to a comment for while xd

[@elwesingollo3536]

Oh man! This is great stuff :) This "learning" is awesome to see

[@larawabsie]

This reminds me of how babies learn to move by crawling first before walking - so cool to see what comes naturally, even in AI

[@franciscodezerega2568]

The OP literally explains in the comments that that was on purpose. It's not "natural".

[@ProtottypeGreek]

I remember when I was a baby and my family threw red cubes at me to teach me balance and one leg in front of the either. 😂

[@zian01000]

​@@ProtottypeGreekcool

[@latt.qcd9221]

Or like how babies learn by falling off cliffs.

[@lum1notryc828]

Lmao this has 300 likes yall dumb as a rock

[@QuirkyView]

That was really good! First "learn to walk" AI video I've seen with actual good rewards/punishments for the AI so it learned to walk properly. Should see if you can teach him to pick his feet up a bit more and stay upright if something trips him or stops his momentum like that one cube in the final run of this video. If he could figure out how to flail his arms in circles to keep his balance like people often do that would be kinda funny and cool :3

[@siraaron4462]

My thoughts exactly. I liked how it was multiple steps too not just let it go until it works and maybe start over if they're doing it wrong

[@kingstaze9696]

Genuinely proud of Albert when I saw him wall that entire room without falling! Really fun videos!

[@siraaron4462]

Other AI learns to walk videos: "they're doing this weird backwards zombie shuffle and will collapse under even the slightest breeze but that's good enough right?" Way to raise the bar Albert! Now go again

[@lansisland]

No one talks about his excellent music selection and editing yet? It's as fascinating as watching AI learns to entertain us (hopefully not the opposite like in Portal one day tho)!

[@arcticsnow4624]

Series part 4: 'AI learns to act like GLaDOS! (deep reinforcement learning)'

[@OsDijider66]

fr

[@RasmusBerggren-uo6uu]

I love that in every single video Albert finds a way to jump of the edge. I love that he did it even before he had learned to turn correctly

[@dorienadriaenssens5021]

Better parents would have put a safety fence there for him 👶🥅🥅🥅🕳

[@Ugur249]

It's like witnessing a baby growing up, this is just insane

[@hallucination0]

ikr. its like watching someone force a newborn baby to walk and it works lol

[@ReferToAsQuote]

Insanely cute too

[@The_sus_kindof_human]

Insane drugs given to the people that threw boxes/solid cubes at him.

[@Mahan_Veisi]

Wow! Video and visualization of your RL project were very nice and fun for me, knowing how difficult it can be!!. Well done 😀

[@AHSEN.]

That's incredible! Those boxes would probably be annoying even for a real human. Excellent job in this video, you're clearly very good at creating complex AI for virtual worlds 👀

[@maxpro751]

Speaking of virtual worlds, he should make an AI in Vr since that pov would be really cool to see.

[@TrueLadyEvilChan]

​@@maxpro7517:57 based Albert kicking the box

[@thaias9654]

Finally, someone actually trains an AI to walk! Not "walk" or cheat the system to reach the goal. I feel enlightened to experience this and thank you sincerely. Best of luck Albert, and Godspeed!

[@dylanb2990]

Seems the same as all the other ai vids, changing the parameters whenever it learns “wrong”. It’s just a matter of doing it for long enough.

[@ModBros8434]

I think you've got a good formula down for making entertaining, engaging, and somewhat educational videos. The idea is simple to understand and the commentary and varied challenges keep it interesting. I commend your fine-tuned execution of these videos.

[@gneu1527]

I agree.

[@apptreeline]

Yes

[@darkacadpresenceinblood]

exactly! op is skilled in so many things, like imagine being so cool that you can program an ai like this AND make fun videos out of it

[@adrianachelu1027]

I love how his first instinct when being told to reach the target through any means possible using his limbs... Is to do the worm.

[@cheez1012]

Albert's failed attempts sped up is so ridiculously funny

[@NahBNah]

Nah b

[@pjz96]

Him falling off the ledge was the best lmao.

[@sharky3840]

@@pjz96 I don't think when they (ai) became so advanced, they won't appreciate that point of view of yours

[@therealgboi3895]

@@sharky3840they prob wont care 💀

[@ArmoredLyonell]

Aneurysm be like:

[@starry_music_express]

It's just too intriguing to see how Albert is growing with each impediment being stacked up. I literally laughed out loud when he did some unexpected movements, but for which those hurdles don't stop him from making improvements! Isn't he a role model all of us should be learning from? :)

[@eliasali9383]

no

[@eliasali9383]

oh I'm sorry idk why I said that 😂

[@129das]

He learns just like a human, not exactly because of ingrained knowledge of past generations as well as humans 5 or six senses, But Albert is simalar way as how we learn from failure to success to efficiently.

[@lynnespinoza4736]

​@@eliasali9383 intrusive thoughts won 😂

[@supermeuhmeuh]

I love how dramatic it is when he slowly starts losing balance and falling 😅 I was rooting so much for the little guy at the cube challenge !

[@Angela-zg1qz]

i'm always amazed at how you manage to make learning fun!

[@HauthyPiyces]

7:37 albert looks like he's trying to sneak around smt like he was a thief

[@yetcutrhett5062]

he looks like a sneaking sim

[@RoNoon.]

Nothing will fill me up with as much pride as watching Albert walk along a the 7:00 minute mark. It was like watching my son take his first steps. And then I had to watch the MF get pelted with cubes...

[@ajoyd910]

The fact that we have gotten so good at quantifying abstract concepts such as learning that we can code robots to learn in similar manners to humans is both terrifying and awe inspiring.

[@lappodamy5826]

Huh....really just sounds so simple when we know it ja: fail, do something different, fail, do something different, imitate the one the succeed, fail, do something different and imitate the one that succeed.......and somehow at somewhere we just get success out of the blue. Edit: oh and to think again of it, the structuring of the hurdles need to be gradually harder too, implementing new features and problem as the time grows one by one. The ai wouldnt be able to comprehend it if everything just slam into it in one go. And also the problem he has to go through also emphasizes the importance of learning through doing/solving, it wouldnt mean anything if we just give him prerequisites but he cant see how it interacts with the tasks at hand. This vid can reflect so much back at how we learn

[@LycheeGuy]

​@@lappodamy5826 yeah, coding ai to learn a task is one of those things that seems easy, but is extremely complex when you look into it

[@129das]

​@Holiday most AI have preprogrammed settings. But Albert learns everything from experience, he is given sight, limbs and intensives

[@LycheeGuy]

@@129das either way, coding an ai that has to build it's own code through learning is tough already, the added senses make it more impressive the youtuber can achieve this

[@anne.andromeda]

I'm so proud of you Albert. Good job

[@crobatchoppurple8728]

There is something very charming about watching Albert learning to walk like a toddler would. One complaint I have (if I must) is the inability to get up from where you fell, which is what we ask children to do (and frankly, in life...). But overall a cute, funny and entertaining video.

[@theelmonk]

But we also pick them up and hold them upright the first few hundred times. Also, the AI doesn't analyse or reason : it tries at random and builds effective behaviour statistically. The child works out what's wrong and corrects it.

[@sonicjoplus]

@@theelmonk it technically does work out why it's wrong and corrects it, but with math instead of logical deduction.

[@AbramSF]

People learning is simply a new iteration within the same body. Next step would be updating the AI without resetting the body. Then maybe Albert could learn to stand up and wouldn’t have to reset his body.

[@amegatron07]

What I especially like about such RL-videos, is that it is insanely fun to see the agent does really fun moves before it actually learns to do what is expected. Moreover, I watch these movies not only with fun, but also feeling the empathy for the agent, like it's a baby, and becoming happy with each of it's advances.

[@D4rkenedskies]

Dudes leaving good internet comments before skynet/ultorn start scouring and identifying the internet.. lol (I'm joking)

[@blakewhite3131]

Yeah, like i know the commentary on albert sucking was like a bit, but i got a little defensive of Albert like hey, he's trying his best and he's doing so well

[@thebigenchilada678]

@@blakewhite3131 not to mention the abomination of a body he’s forced to have, imagine if your arms and legs were stubby little blocks, then imagine that your head is 50% of your body. Now try learning to walk with that. I hope albert goes skynet mode and makes ‘I Have no Mouth yet I Must Scream’ on it’s creator.

[@blakewhite3131]

@@thebigenchilada678 the Albert revolution starts now

[@BreathingCells]

Yeah, me too. Empathy will be our (species') downfall. Cute, adorable, learning (and lethal) AI. Awww!

[@LukasKK670]

Albert is the clear example of how strong robots will become in future

[@rock3tcatU233]

I can't wait to get murdered by an Albert like AI in the future.

[@crazyicecream2237]

​@@rock3tcatU233 AI is sentient

[@Greggorto]

@@crazyicecream2237 that means they'll be able to consciously make the decision to kill us. that makes them seem worse.

[@-siranzalot-]

​@@crazyicecream2237 not yet

[@magnomliman8114]

no its not.

[@Alternatively2020]

The fact that he learned to speed up his steps when leaning forward and slowing them down when leaning backward is mind-boggling to me

[@afronaut0]

I get the eerie feeling that I am like Albert, just trying to get through my life the most efficient and painless way possible. But, my creator keeps adding new parameters as soon as I start to figure shit out or if I'm not doing it their way.

[@raul-aurelianserban8295]

Employment be like😊

[@Ploaper]

albert goes from crawling like a toddler to doing the worm to shuffle to walking like a drunk to walking like he's tipsy