Reinforcement Learning 10: Classic Games Case Study

[@LuisYu]

amazing high quality lectures. especially enjoyed attention, memory, alpha zero talks.

[@Kingstanding23]

A Nash equilibrium sounds like what happens on roads where traffic evens itself out amongst all the roads towards some destination. When a new road is built, nothing really changes because the traffic just redistributes itself to an new equilibrium.

[@LucyRockprincess]

great real life analogy

[@TheGreatBlackBird]

Shouldn't there also be a reward present in TD error at 42:30 and 50:25 ? edit: ok, it's explained a bit more in the 2015 lecure that this version assumes no intermediate reward

[@johangras3522]

It is possible to access to the course slides ?

[@TuhinChattopadhyay]

@@Sigmav0 Link not working

[@Sigmav0]

@@TuhinChattopadhyay The slide has been moved to www.davidsilver.uk/wp-content/uploads/2020/03/games.pdf Hope this helps !

[@TuhinChattopadhyay]

@@Sigmav0 Got it... many thanks

[@Sigmav0]

@@TuhinChattopadhyay No problem ! 👍

[@dojutsu6861]

@@Sigmav0 these slides are from an older UCLxDeepMind lecture series lead primarily by David Silver. They do not include content on the newer AlphaZero models. Do you by any chance know if these updated slides are available online

[@samagrasharma7755]

Two lectures (CNN and RNN) are missing from this series. Can anyone tell if they are available online?

[@helinw]

Did David do another RL course in 2018? Or just one lecture?

[@ShortVine]

i was thinking the same & searched a lot, but i think he did just one lecture in 2018

[@domino14]

The level of computer play in Scrabble is not superhuman. Quackle beats Maven, and the best humans can 50-50 Quackle in a long series.

[@jakubbielan4784]

Anyone know what was the exact hardware used to train Alpha Go Zero?

[@luisbanegassaybe6685]

deepmind.com/blog/alphago-zero-learning-scratch/

[@stevecarson7031]

Thankyou so much for this series of lectures!

[@alexanderyau6347]

I can comment now. See you again David.

[@yidingyu2739]

Why so many empty seats?

[@yoloswaggins2161]

This stuff not on the exam

[@matveyshishov]

The number of people is lower with later lectures for some reason.

[@markdonald4538]

@@matveyshishov stupid ppl

[@mohammadkhan5430]

I love him, how sad the room is empty

[@KayzeeFPS]

Here's a link to the same video but with slides visible kzbin.info/www/bejne/hGKvfH-Za9qZfbs

[@Dina_tankar_mina_ord]

I would love to see how deepmind would build a city on its own in Cityskyline. See how its optimization would create the best and most efficient layout in real time. Maybe we could learn alot from that.

[@julioandresgomez3201]

Despite the success of A 0 nets in several games, I feel that is better starting point playing (random number) games with humans. Only then, when it has grasped some basic basics (by itself, not forcibly inserted by hand), let it play against itself. This way it could accomplish in thousands of self-play games what from scratch it´d take millions of self-play games, due to the total randomness and clueless of the first games. It´s not the absolute zero approach, but it has no "artificial" parameters handcrafted either. It learns from its own games all the way.

[@Avandale0]

Playing with humans takes considerably more time than running simulations - so actually, playing millions of games by itself is still faster than playing 100 games from playing humans. Knowing that a game of go takes around 1h, you'd have finished 3 games with a human in the time that it took AlphaZero to reach human level play. Same for chess, when you realise it took Alpha Zero 4 hours reach a level higher than Stockfish... It should be clear from these examples that one of the particularities of AlphaZero is the speed at which it learns. Playing humans here both defeats the purpose of self-learning and actually wastes time.

[@omarcusmafait7202]

why does nobody take notes?

[@yoloswaggins2161]

Not on the exam

[@Sigmav0]

@William Davis Sure... In primary school...

[@vijayabhaskar-j]

because slides are available online and lectures are available online, I would listen carefully first in the class