From AlphaGo to MuZero - Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model

3/31/2021 Colloquium
Speaker: Thore Graepel (DeepMind/UCL)
Title: From AlphaGo to MuZero - Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model
Abstract: Constructing agents with planning capabilities has long been one of the main challenges in the pursuit of artificial intelligence. Tree-based planning methods have enjoyed huge success in challenging domains, such as chess and Go, where a perfect simulator is available. However, in real-world problems the dynamics governing the environment are often complex and unknown. In this work we present the MuZero algorithm which, by combining a tree-based search with a learned model, achieves superhuman performance in a range of challenging and visually complex domains, without any knowledge of their underlying dynamics. MuZero learns a model that, when applied iteratively, predicts the quantities most directly relevant to planning: the reward, the action-selection policy, and the value function. When evaluated on 57 different Atari games - the canonical video game environment for testing AI techniques, in which model-based planning approaches have historically struggled - our new algorithm achieved a new state of the art. When evaluated on Go, chess and shogi, without any knowledge of the game rules, MuZero matched the superhuman performance of the AlphaZero algorithm that was supplied with the game rules.

[@edupignatelli]

Thore's explations are always as clear as he is explaining to a 5 years old child, and as respectful as he is speaking in a UN meeting at the same.

[@JimSlattery]

26:10 this part really stuck with me. We see all the handcrafted expert logic in the Stockfish engine, and yet machine learning can achieve all of that and more in an automated way. This is amazing technology!

[@DrTune]

Great vid, very approachable. Thanks!

[@martinsosmucnieks8515]

But very great vid!!!

[@martinsosmucnieks8515]

Why Mu?

[@jeshweedleon3960]

pretty sure “mu” is japanese for “emptiness” or “absence”, referring to the fact that muzero isn’t given hardcoded rules.

[@themanwhoknewtoomuch6667]

@@jeshweedleon3960 Hoftstadterish. Computer Scientist sure has a siccc sense of humor and cool artistic taste!