Reinforcement Learning: Crash Course AI #9

Reinforcement learning is particularly useful in situations where we want to train AIs to have certain skills we don’t fully understand ourselves. Unlike some of the techniques we’ve discussed so far, reinforcement learning generally only looks at how an AI performs a task AFTER it has completed it. And when an AI completes that task figuring out when and how to reward an AI, called credit assignment, is one of the hardest parts of reinforcement learning. So today, we’re going to explore these ideas, introduce a ton of new terms like value, policy, agent, environment, actions, and states and we’ll show you how we can use strategies like exploration and exploitation to train John Green Bot to find things more efficiently next time.
Crash Course AI is produced in association with PBS Digital Studios:
/ pbsdigitalstudios
Crash Course is on Patreon! You can support us directly by signing up at / crashcourse
Thanks to the following patrons for their generous monthly contributions that help keep Crash Course free for everyone forever:
Eric Prestemon, Sam Buck, Mark Brouwer, Indika Siriwardena, Avi Yashchin, Timothy J Kwist, Brian Thomas Gossett, Haixiang N/A Liu, Jonathan Zbikowski, Siobhan Sabino, Zach Van Stanley, Jennifer Killen, Nathan Catchings, Brandon Westmoreland, dorsey, Kenneth F Penttinen, Trevin Beattie, Erika & Alexa Saur, Justin Zingsheim, Jessica Wode, Tom Trval, Jason Saslow, Nathan Taylor, Khaled El Shalakany, SR Foxley, Sam Ferguson, Yasenia Cruz, Eric Koslow, Caleb Weeks, Tim Curwick, David Noe, Shawn Arnold, William McGraw, Andrei Krishkevich, Rachel Bright, Jirat, Ian Dundore
--
Want to find Crash Course elsewhere on the internet?
Facebook - / youtubecrashcourse
Twitter - / thecrashcourse
Tumblr - / thecrashcourse
Support Crash Course on Patreon: / crashcourse
CC Kids: / crashcoursekids
#CrashCourse #ArtificialIntelligence #MachineLearning

[@hjaiswal768]

"A trade off between exploration and exploitation" - Thats life

[@peka2478]

Well, no, (Conquistador) Life can be both. Usually is.

[@shashanksams]

yay, this was actually better than most of the explanatory videos i have seen. thanks for providing us always with informative content, crash course. looking forward for more of these videos

[@TheCheck999]

8:38 John Green bot from the past

[@cesardachimp8172]

oh man, i didnt notice that

[@Pravaification]

*waves robo hand* Mr. Green Bot! Mr. Green Bot! What if I took TEN actions?

[@thebrianpaige]

Loving this series. Thank you so much! There's so much info in every episode; just fantastic.

[@pedro_a_martins]

I'm really liking this series of videos Keep ip the good work :)

[@StevieRZ]

now,after watching multiple episodes in a row, I really want donuts :P also really enjoying this series :)

[@LearnandGrowKidsTV]

Such a great video! We most definitely do whatever it takes to get more cookies 🍪 😉

[@_Baleful]

This video is fire. Insta-subbed!

[@janyjj1969]

I WOULD LOVEEEE IT IF CRASH COURSE HAD AN ACCOUNTING COURSE!!❤️️.

[@Logan_Explores]

Jany JJ try Kahn Academy. I’m pretty sure they have an accounting section.

[@janyjj1969]

@@Logan_Explores Alright, thank you so much😻.

[@randomalpaca]

US or Canada or other?

[@TheBlizzardfan7]

Thanks for your Awsome Course, I got interested in Machine learning and I am planning to study that for my M.A.

[@geoffreywinn4031]

Cool video!

[@gitoshrisen7687]

This reminds me of Pavlov from my psychology class.

[@TreeNap]

In the john green bot example, is the objective to find the shortest path or get the most points? What would getting more points even do, I feel like in that case exploration is best so that you can find the shortest path, exploiting only when racing another bot

[@rajcivils]

Are you going to use openai for rl and keras when we come to deep reinforcement learning When will this playlist be finished.

[@chaneldelarosa9786]

Thank you!

[@jimmybutler4076]

Yo as a black guy with dreadlocks who likes coding, it’s really cool to listen to another black guy with dreadlocks who likes coding

[@jacquelyncuturic8072]

jabril got a new hat did u notice?

[@cesardachimp8172]

Can we do Crash Course Law? Like if you agree

[@masternobody1896]

Crash course doesn't make sense it makes jibril look bad

[@nealkelly9757]

How about crash course Current Events. It's so hard to keep up with all the developments in US politics and such

[@Muhammad_Syakir]

PBS can try invite Meghan Markle and her team to do all the law educational stuff, she is very good at it. :)

[@thethirdjegs]

@@nealkelly9757 crash course current events would have indefinite number of episodes.

[@cesardachimp8172]

@@masternobody1896 ??? what do you mean?

[@krellend20]

Jabril really has it out for bagels.

[@cornellwaters9089]

🏃 Thank You!

[@MarkyMark1221]

Can we get crash course geography

[@shendosmani4509]

5:12 open all no risk high reward

[@billniko9310]

Can we get computation theory lesson ,CC ?

[@pinkponyofprey1965]

0:17 That cookie looked completely ... edible hahaha! What brand is that? :D Screw Reinforcement Learning .... I'm now officially hungry! [back from the kitchen] Reinforcement Learning is actually extremely interesting! :D

[@billniko9310]

Love you love you love you love ❤️

[@anttibjorklund1869]

Why would JohnGreenBot in that battery example only go in straight lines? Would it not be better to go in a diagonal path?

[@EclecticFruit]

because that is how he was taught to see the room and navigate it. If you want diagonals, maybe we'd have to arrange the room in hexagons instead.

[@nahhale9088]

Can we get crash course music theory?

[@lincolnpepper816]

that's the main thing i've been wanting for years and many people have asked for it. So you know, i remember on one other comment asking for it crash course replied that they've been thinking about it as well. Not sure what the chances are now, but that gives me hope

[@sarujanrupan4831]

It's Jabril!!!

[@void2509]

5:11 I'll just take all three items

[@ianrbuck]

Not sure the kitchen metaphor works for me. Why is the bag more likely to contain donuts than the box? It sure looked like the kind of box that donuts come in to me.

[@castro_hassler]

Nice ,

[@johnopalko5223]

This fellow and his donut obsession. I don't know... 😊

[@dvklaveren]

Black & White and Black & White: Creature Island used reinforcement learning. The creature you commanded could learn incredibly complex routines, such as planting a sapling, water the tree with the water miracle, then pick it up and throw it into the resource center and repeat. With enough training. I really hope that we'll see more games exploring that kind of relationship with a computer character. Imagine a game where you're personally teaching a group of monsters how to hunt and then guiding their instincts by reinforcing or punishing a particular set of circumstances, until they conquer their world.

[@kurtphilly]

I don't agree with the bagel/donut choice example. Why choose the option of two bagels or donuts vs. the greater risk of more donuts (6) or a guaranteed single donut?

[@amitzahirr]

is this related to dijkstra's or a*?

[@dkecskes2199]

9:40 is my hometown. Ok AI, good reinforcement for me.

[@slikh]

Yeah, I recognized Boston St too :)

[@AB-vw8wz]

Markov decision process and Q learning, fcking tedious

[@Graghma]

Is there a better reason than consolidating the total amount of stored data the reason we only store a single value per square? Why not store 4 values per square so you can store a value per direction you could go from the current spot. That way you could find/exploit the near black hole shortcut that the current algorithm is too scared to find.

[@TheSentinelAI]

🤘🤘🤘

[@senthu07]

12th comment is mine...

[@alexixeno4223]

I feel there are people in place of power/rich who need to watch this video... >.>

[@JustaReadingguy]

Outline what they should take way from the video and why.

[@COOPSTOP]

Agent? like..... Agent Smith???!!!

[@argonair4838]

Open Ai and Alpha Go

[@hamzadbz1]

Who drives a car looking at side ways?

[@user-zu6ts5fb6g]

When jabrils is talking his mouth is moving. That is illegal.

[@jimmyshrimbe9361]

That was a robot playing Don't Wake Daddy.

[@rintume8631]

Like if AI beats slavery

[@fabio2234]

First second!

[@masternobody1896]

Knowledge but man I don't understand can you make it easy

[@thethirdjegs]

This episode is hard because it's oversimplified

[@liliabel3160]

You're good but You are going too fast

[@clvrswine]

Jabril? Jabril? Laughing too much to type. Hey, I'm Jabril. Unreal, these people.

[@sefp]

Rare too see a black guy talking about this subject but glad I did.