David Silver: AlphaGo, AlphaZero, and Deep Reinforcement Learning | Lex Fridman Podcast #86

David Silver leads the reinforcement learning research group at DeepMind and was lead researcher on AlphaGo, AlphaZero and co-lead on AlphaStar, and MuZero and lot of important work in reinforcement learning.
Support this podcast by signing up with these sponsors:
- MasterClass: masterclass.com/lex
- Cash App - use code "LexPodcast" and download:
- Cash App (App Store): apple.co/2sPrUHe
- Cash App (Google Play): bit.ly/2MlvP5w
EPISODE LINKS:
Reinforcement learning (book): amzn.to/2Jwp5zG
PODCAST INFO:
Podcast website:
lexfridman.com/podcast
Apple Podcasts:
apple.co/2lwqZIr
Spotify:
spoti.fi/2nEwCF8
RSS:
lexfridman.com/feed/podcast/
Full episodes playlist:
• Lex Fridman Podcast
Clips playlist:
• Lex Fridman Podcast Clips
OUTLINE:
0:00 - Introduction
4:09 - First program
11:11 - AlphaGo
21:42 - Rule of the game of Go
25:37 - Reinforcement learning: personal journey
30:15 - What is reinforcement learning?
43:51 - AlphaGo (continued)
53:40 - Supervised learning and self play in AlphaGo
1:06:12 - Lee Sedol retirement from Go play
1:08:57 - Garry Kasparov
1:14:10 - Alpha Zero and self play
1:31:29 - Creativity in AlphaZero
1:35:21 - AlphaZero applications
1:37:59 - Reward functions
1:40:51 - Meaning of life
CONNECT:
- Subscribe to this KZbin channel
- Twitter: / lexfridman
- LinkedIn: / lexfridman
- Facebook: / lexfridmanpage
- Instagram: / lexfridman
- Medium: / lexfridman
- Support on Patreon: / lexfridman

[@lexfridman]

I really enjoyed this conversation with David. Here's the outline: 0:00 - Introduction 4:09 - First program 11:11 - AlphaGo 21:42 - Rule of the game of Go 25:37 - Reinforcement learning: personal journey 30:15 - What is reinforcement learning? 43:51 - AlphaGo (continued) 53:40 - Supervised learning and self play in AlphaGo 1:06:12 - Lee Sedol retirement from Go play 1:08:57 - Garry Kasparov 1:14:10 - Alpha Zero and self play 1:31:29 - Creativity in AlphaZero 1:35:21 - AlphaZero applications 1:37:59 - Reward functions 1:40:51 - Meaning of life

[@abogaziah]

OMG THANK YOU

[@riccardomereu1813]

Thank you very much Lex 🙏

[@pyshine_official]

Thanks

[@franj4139]

Please invite Humberto Maturana: He had develop theories on human intelligence, consciousness and understanding. He is in his 90s, we could lose his takes on artificial intelligence

[@ivannogolica364]

Bring David Deutsch please! :)

[@vedgupta1686]

"He'll be remembered as the last person to beat AlphaGo" man!!

[@joelkavanagh1464]

,,, kudos n respect on that comment! ... greetINX from s.lem jr ... .. . ...............

[@chanleystow]

Seeing this after the AlphaGo doc!

[@rdcalderon]

Watching the documentary before watching this interview definitely adds value. kzbin.info/www/bejne/jYnYfGmdmtCIZ7s

[@ecavero1]

As have I! I was searching of an Alpha Zero doc. This is where I got so far. Not disappointed at all!

[@maplegoose6364]

Yes came here directly after the Doc as well. Had never heard of GO! prior to 3hrs a go. Indelibly registered and imprinted now :D

[@khall187]

Same

[@schwajj]

maap no need to capitalize and exclaim, any more than you’d write CHESS!

[@oncedidactic]

THIS IS THE ONE I'VE BEEN WAITING FOR!

[@oncedidactic]

@@mikhailfranco dude, thanks 🙌

[@AakarshNair]

His answers are so articulate!

[@SamuelRodriguez10]

Amazing, this conversations are so meaningful to the future of humanity that they should be broadcasted on national television. That way children would more easily find meaningful role models and access to the type of insightful ideas that give birth to passions and eventually discoveries.

[@ehsanmamakani]

I totally agree with you. These are the role models that our children must be familiarized with not some attention addicts on the social media who act as a catalyst to remove the brain from the anatomy of human beings.

[@UnpluggedPerformance]

I also totally agree.. So beautifully phrased!!

[@jamesjenkins9480]

Lol who watches national television though? More people will watch it on youtube.

[@hamentaschen]

Again, Mr. Fridman, THANK YOU for keeping this going, especially now. When I need to get my mind off the current world situation I come here. Your talks always take me to a better place. Thank you. Be safe. Stay healthy.

[@TrappedinaBrain]

This is a banger of an interview. AlphaZero is a harbinger of the future

[@litvinenkoalexander5331]

I am very happy to see that 3.22M people are watching this channel.

[@TheTessatje123]

Thanks for making this podcast. David Silver chooses his words very well, his stories are very clear and inspiring! I could have listened much longer ;-)

[@joaodesouza4649]

I can't describe or express how valuable this interview is for understanding what's going to happen in the future

[@supersnowva6717]

I watched Alpha Go vs. Lee sedol tournament documentary Deepmind recently uploaded, and I cried. It was so inspiring, touching and beautiful. Thanks very much Lex for this podcast.

[@TheRealStructurer]

3 years later I am here... Latest AI developments makes me ask for a second round with David Silver. Thanks for sharing 👍🏼

[@bruceturner4858]

Discovery is a joy. Discovering the existence of David Silver and his amazing way of thinking is pure gold. Thank you Lex.

[@camillorohe6996]

you just gotta love David Silver and his ideas, thoughts and accent

[@ufozencom]

Mind teased, tantalized, and finally thrown into a tizzy. Love every one of your interviews Lex. All I want to do is watch them to get inspired to think in new ways. THANKS MAN!

[@JackSPk]

Oh man! That meaning of life interpretation! I think I'm gonna click this 1:41:20 every night before sleep from now on. Thank you Lex for making this possible! ❤️

[@sabelch]

I initially cringed a little when Lex decided to "go there" with the meaning of life question but pshew! Silver gave a great answer.

[@Jannikheu]

sabelch yes that answer was very impressive and I think demonstrated his capacity of deep thinking

[@iwanjones7334]

I was laughing to myself and thinking: "All he needs to do now is ask him the meaning of life question". And then he did!

[@decidrophob]

Indeed, probably David's comment regarding the meaning of life was by far the most philosophically meaningful I have ever come across.

[@Mikey-lj2kq]

there's a book called 'the fabrics of reality'

[@UnpluggedPerformance]

This interview is LEGENDARY!... watching it for the second time. Definitely in the top 3 on youtube!

[@L.someone]

Wow! This was an incredibly insightful and inspiring conversation. Thank you Lex, David, and your teams for this.

[@jung8935]

Man, David Silver is so incredibly humble...

[@r1s1112]

Awesome conversation, David is incredibly interesting and humble also amazing questions from Lex. Thanks to both of you for making it.

[@geraldsierveldphotographyi1406]

Outlining the episode is the MOST awesome and thoughtful thing foru2have done...

[@samuelec]

Thank you both! It was, again, an awesome conversation.

[@asdf_600]

Incredible podcast, probably my favourite! It would be incredible to have a second part!

[@minerwilly]

This is a really great interview and very enlightening. Thanks for all of your hard work bringing this stuff to us. Keep up the good work.

[@hariomt348]

1:40:51 : One of the best answers for the purpose and meaning of life I have heard so far. Incredible!

[@vladimirgetselevich4704]

Thank you for Lex and David! Very interesting and inspiring conversation about first principles of Artificial Intelligence.

[@user-jx8gv1rd8e]

Lex, It is very clear that you love what you do. It totally shows. You are always super prepared and well engaged with your guests. Yours has become my absolutely favorite podcast. Listening to a 2 hr podcast of yours is as intellectually fulfilling as reading a 400 page incredible book.

[@gallerksee]

I love the content you put out man! It's always interesting, always paradigm challenging, calm, informed, you! Thanks!

[@darylallen2485]

Many academics are terrible at explaining their domain of expertise. David is a quality academic and has remained grounded enough to explain himself to normal folk like me. Well done.

[@andrewg2355]

I love your guests and the way you carry the conversation brother! Great job, love your channel.

[@perfumedsea]

I can ignore everyone else but David Silver talking about AI. His lectures and courses taught me RL.

[@JT-xb6zs]

Thanks for putting the ads in the beginning !! It's way better than getting your concentration broke mid interview

[@einemailadressenbesitzerei8816]

its beautiful to see a man that lives his passion. a man that is what he is creating.

[@Lagruell]

Many thanks for sharing this amazing interview!

[@ottolehto]

Thank you for another enlightening, exploratory, and meaningful conversation that pushes us towards self-questioning and, one hopes, self-understanding.

[@saulocerqueiradealmeida9700]

Thank you so much LF! Great job.

[@kennethcrandall8131]

This interview was so good it brought a tear to my eye!

[@sathvikudupa1668]

Thank you!! Been looking forward to this.

[@duderadley2383]

Thanks for Boss content empowering people, many young people enjoying this content and in my opinion, such a treasure it is, the exponential tune to your tone.

[@garyswift9347]

I love how the wall and window are decorated to resemble a go board

[@isakrathestre6748]

Awesome interview. I start jumping around with excitement. Get so eager to learn more!

[@englishiguana4304]

thank you again lex, another phenomenal interview, i cannot get enough of this wonderful channel!

[@JousefM]

My Saturday blockbuster, thanks Lex. David is a cool dude, have to get Demis in now :)

[@people93]

David Silver is a real legend

[@karlisstigis]

Thank you, one of the most interesting talks in a long time!

[@michaeltheunissen609]

Brilliant interview. Articulate and like yourself, I believe AlphaGo was a tipping point for the progress of humanity.

[@chiefrabbi6735]

Love David Silver's lectures on RL

[@shuhu1234]

Thank you for this amazing discussion!

[@JohnHAdams-vo2pk]

Very proud of my old university - University of Alberta. Dr. Silver got his PhD there under Richard Sutton. Great interview. Was looking forward to this one.

[@bernardvantonder7291]

David is an amazing being.

[@roseleelauper1193]

Excellent podcast, thank you

[@alexcherfan7762]

Crazy Lex.. I just went down the alpha learning machine rabbit hole this week. I watched the documentary on alphago, which was fascinating. I also watched the matches between the pro starcraft players and alphastar, which was even more fascinating (partially because I'm familiar with the game). I wonder in this sphere, how far a deep learning machine like this can go. This podcast was the icing on the cake at the bottom of the rabbithole, thanks brother!

[@adeep_jain]

Fantastic one!! So many cool ideas in there!! Thanks Lex 🤘🏽

[@shawnchen6338]

Trying to reproduce the MCTS results on some other tasks. After several weeks of struggling, I learned that David Silver is really great in a sense that he foresee the future of deep learning research -- computational power really matters.

[@jingtao1181]

Thank you Lex, Great convo.

[@peacock8730]

The great conversation! Now I finally understand how alphaGo and alpha Zero were created.

[@Kyle-oe2vs]

Wow, very insightful, nice to get our minds off of the pandemic and look to a bright future. Incredible potential behind DRL!

[@msulemanf]

This was the AI interview I've been waiting for - it did deliver. It could have been a bit longer and included the protein folding work, though. Perhaps that's ongoing and still a competitive area. There is a certain clarity of articulation from the guests I enjoy most - reminds me of Jeff Hawkins. Also a sense of practical application.

[@palakrishna9921]

Pala

[@Jacob-sb3su]

They figured it out

[@andrewtoebbe3885]

@@Jacob-sb3su they?

[@josephsantarcangelo9310]

his course on youtube is amazing

[@muharremuguryavas9183]

Such an inspiring conversation, as a phd candidate who works on deep RL, I am quite motivated to try even harder! Thanks for your efforts Lex!

[@smegmaprince314]

such an annoying comment, as someone who hates humble bragger, I am quite motivated to downvote your comment! Thanks mr poo on road!

[@DaDankStrafe]

@@smegmaprince314??? He just said he's inspired because he's working toward entering the same field as the podcast guest. Don't be dumb and weird.

[@shivamkushwaha9730]

This is the best of all episodes and I know I am biased. Thanks Lex.

[@brixtoncruddy]

Get Demis on here please!

[@fatayas9463]

Amen

[@Brad_Jacob]

Yes!

[@amandamoore9183]

Yes please Lex Demi’s would be awesome 😎

[@emmanuelboakye1124]

This interview is eye opening👍👍

[@oudarjyasensarma4199]

Thanks Lex! Even bigger greatness is coming your way!! Cheers! Stay safe!

[@egorpanfilov]

This is an instant like from me :)! Many thanks Lex!

[@devonk298]

David is adorable, I have watched his RL Course 3-4o times. Brilliant guy and funny too

[@Voke]

Great stuff, guys! Keep up the hustle

[@bobwelham8792]

Good to hear the logic based programming language PROLOG mentioned.

[@rahulsagarpv]

Hey man, awesome interviews! You seems to be a really good person. Thank you for what you are doing.

[@davida3922]

6 months ago I didn’t even know who Lex was, now I can’t get enough of his podcasts. The powers of the internet. I hope he does become a billionaire.

[@johnsharkey5255]

Hey lex, really interesting episode. A guest I think you should have on your podcast is Leo Gura. His work is more particularly focused on the nature of consciousness and he is for me one of the most insightful people I have ever listened to.

[@corkkyle]

What a fantastic conversation!!!

[@Francisco-qh3qh]

You, Sir, are a gentleman and a scholar.

[@james.arambam]

I must say, one of the best podcasts. Thanks, Lex and David

[@typo44]

Well done. Its great how you went into the deep background at the end there/

[@iwanjones7334]

I am struck by how small the audience is for this astonishing talk. It is so important that it should number in the millions, even billions.

[@tristonedwards7094]

Mate thank you for your videos. your channel is great.

[@jordanjennnings9864]

Thank you lex David you seem like a real gamer very competitive. Great podcast

[@hazemahmed8333]

Man i can’t thank you enough ❤️

[@willikey]

This talk is so inspiring.

[@mohamedarif604]

I've been taking his rl lectures currently.Thanks

[@PedroContipelli2]

Absolutely amazing.

[@antoniolau8762]

Man, David Silver is such a genius! I've enjoyed the interview so much. I wouldn't say Lex interview policy can be considerd as optimal yet, but the story you create through your questions, the way you try to go to the essence when you close your eyes and just the way you are make it be really close. If you read this, thank you

[@olijones9953]

Really enjoyed this one

[@duderadley2383]

Those who don’t have sophisticated backgrounds in Programming can really appreciate the way you relate what the computers are doing and capable of doing to the romantic human narratives

[@robertocarloscaruso6840]

David and demis, hope you get nobel prize someday soon.

[@samvargas2868]

YES DEEPMIND!!! (I had decided to write in all caps when I saw the thumbnail)

[@jonaspiva41]

Haven't watched yet, just settling in for it but I really wanted to say something. Yay!

[@jonaspiva41]

Greatly enjoyed it, and I have a feeling there are more interviews with Deepmind team and I am sooooo stoked. Be safe & have fun.

[@ishtar0077]

It's funny I got chance to watch it today again. Now this interview.

[@Parksukmin]

Thanks!

[@Chess_Intelligence]

Good interview this one.

[@viraatchandra8498]

changing the world, by bringing us the people who are changing them :) Thanks Lex! you rule :)

[@kartikeydetha5582]

I learnt about New dimension of thinking and understanding things.

[@vigneshpadmanabhan]

amazing episode

[@csswithmalikbedarbakht]

Great interview.

[@dizbeefpvdizbeliefdizzy3612]

Very enlightening thanks.

[@yviruss1]

Thank you.

[@pyshine_official]

We are in this together we will win!

[@sabofx]

Anyone else get excited by Deepmind's latest "muzero" algorithm that David discussed, starting from about 1:28:00 into the video? Supposedly a new algorithm that is able to figure-out the rules and constraints of the environment by itself. I'd love to hear more in depth discussions about Muzero's capabilities in future talks with Deepmind's finest 😎!