Michael Littman: Reinforcement Learning and the Future of AI | Lex Fridman Podcast

[@lexfridman]

Here are the timestamps. Please check out our sponsors to support this podcast. 0:00 - Introduction & sponsor mentions: - SimpliSafe: simplisafe.com/lex and use code LEX to get a free security camera - ExpressVPN: expressvpn.com/lexpod and use code LexPod to get 3 months free - MasterClass: masterclass.com/lex to get 2 for price of 1 - BetterHelp: betterhelp.com/lex to get 10% off 2:30 - Robot and Frank 4:50 - Music 8:01 - Starring in a TurboTax commercial 18:14 - Existential risks of AI 36:36 - Reinforcement learning 1:02:24 - AlphaGo and David Silver 1:12:03 - Will neural networks achieve AGI? 1:24:30 - Bitter Lesson 1:37:20 - Does driving require a theory of mind? 1:46:46 - Book Recommendations 1:52:08 - Meaning of life

[@ValtDewller]

You're a big inspiration in my life Lex, thank you

[@RAESGRACEFULWAVES777]

At 29:05 ...such a privilege, this current moment. Immediately wanted to catch a flight to Mass. There are many relevant questions that can be summed up illogically.

[@testosteronecivilization8819]

Hi, I would like to share with you my new article: The term hormone means impetus: a motivator for action. One might wonder what hormones-the biological mechanisms for intercellular signaling-have to do with artificial intelligence (AI). Well, first and foremost, it is human intelligence that creates this machine intelligence. AI is actually modelled after the neural networks that enable the cognitive ability of the human brain. However, our brain has evolved different layers in the hundreds of millions of years of evolution, and the neocortex, which is associated with our rational faculty of cognition, is a more recent development of our large mammalian brain. The neocortex is situated on top of previous layers of the limbic system that is responsible for stimulating and regulating our behavior. The motivation for our behavior is linked to survival through our ability to attain territory and resources, as well as sexual reproduction, which drives the organization of our society into a complex, adaptive social system. Best Regards, Roy Barzilai, testosteronecivilization.com/the-testosterone-hypothesis/

[@brandwas12]

Hey Lex, I'd love to hear a show about the vast influence of small and cabinet level Government institutions (Like the U.S. EEOC) influance on everyday Americans lives through legal precident and policy change. I'm a current Federal EEO Investigator in Los Angeles and would love to discuss it!

[@peterfirth7]

Lex, love your work. I am completely addicted to listening to you podcasts. I have a question, has anyone or is anyone working on the idea of merging HI and AI. Using AI to enhance the human and create a super intelligent human. If we struggle to replicate the human mind can we focus on an enhancement that might lead to replication.

[@LILTANTILT]

Lex is the new Rogan on KZbin now. Welcome Lex to the big league.

[@MaxDeckard]

Yup. Lex for intellect, YMH for comedy.

[@MaxDeckard]

@@Automatic-Diaphragm your mom's house podcast with tom segura, bert kreischer, christina P., Dr. Drew, and guests. They have 4 seperate shows per week uploaded early in the morning.

[@commetking9746]

Don’t that pic look like Joe R with hair! 🤣🤣😂😂

[@CE-vd2px]

@@Automatic-Diaphragm get owned with that yo momma joke

[@corykeane]

Hell yeah! Lex "New Rogan on KZbin Now" Fridman

[@ayceeonethirty6141]

This has quickly become my favorite podcast. Always excited to see a new episode. EDIT: 15:25 is precisely why I said what I said. Too many podcasts try to force conversation to move at a certain pace, or get specific soundbites for youtube clips, and I just never see you do that with anyone. I'd rather you spend 20-30 minutes going into the abyss of an interesting topic than have that quickly sidelined to get to the "named segment we force in here every week".

[@davosholdos1253]

Agreed. This is the new joe rogan for KZbin.

[@ninadesianti9587]

Agreed!

[@eggmendy]

Also, very excited for the solo episodes, those all sound like interesting topics.

[@MaxDeckard]

If theo von and bill burr can do it, lex sure can.

[@MaxDeckard]

@@MrSomarw check out lex's cyberpunk playthrough hahaha... ibwas just screaming at my tv "AIM DOWN THE SIGHTS!!"

[@samberg3864]

@@MaxDeckard Me too man. KZbin thinks "too" is misspelled for some reason even though it's not... But yea, when he was talking about how long it's been since he's played video games, and then I saw him look away from the screen the entire time the message telling him to ADS was on screen, I was like god damnit. He will probably go the rest of his life thinking that you can't aim guns in video games lol.

[@commetking9746]

I’d rather see him interviewing mr Putin Elon again to be honest with you -or how about Alex Jones!🤯😳

[@JaketheBakedSsnake]

Hey Lex, I appreciate that you continue to engage with the commentary on your videos. Your work is gold. Pave the way to normalizing discussion about the opportunities we have all been gifted with. Lets build the future we want.

[@advaithbala3087]

I am used to watching Lex's interviews with popular science figures like Richard Dawkins, Leonard Suskind and such, but it is only after watching him interview someone in the field of AI that I can really appreciate how well versed he is in his own field. Not only are you one of the best interviewers I have seen in a while, but you also seem so knowledgeable in AI.

[@Jen_Panpsychism]

i think we both know why the term you end up going with cannot be "self-play"

[@DonCat-sc3qo]

1:23:25 „It‘s subtle sounding but quite profound“ is the most Lexish quote Lex has ever pulled.

[@MaxDeckard]

Lex, I'm currently working 12 hour nightshifts as a covid screener at a pipeline camp in Canada's rockies and won't be home til next month. Its so refreshing that you have a variety of people on that arent on a constant tangent of covid discussions. Really helps pass the time. Thank you from the bottom of my heart 🤖🤘

[@brianwestphal]

Thank you Dr Littman for your Georgia Tech ML and RL courses!

[@chrisford7033]

I’d love to see you discuss inventory management solutions using machine learning, such as better forecasting, safety stock, optimization problems that feel like systems of equations waiting to be solved. I see companies impressed by a simple moving average inventory model that pre-stages raw materials to produce closer to the target window of the forecast based on known lead times and demand average to calculate cycle stock necessary and then taking a z-score times (in my situation) a static lead time standard deviation (7 weeks = .7) times the demand average to decide safety stock. There has to be an amazing way to do this via machine learning...

[@hardik114433]

We actually worked on this exact problem. Have a look at this papers from our lab 1. arxiv.org/abs/2006.04037 "Reinforcement Learning for Multi-Product Multi-Node Inventory Management in Supply Chains" 2. arxiv.org/abs/1910.00211 "Reinforcement Learning for Multi-Objective Optimization of Online Decisions in High-Dimensional Systems" 3. www.ifaamas.org/Proceedings/aamas2019/pdfs/p1802.pdf "Actor Based Simulation for Closed Loop Control of SupplyChain using Reinforcement Learning" 4. link.springer.com/chapter/10.1007/978-3-030-60843-9_3 "Reinforcement Learning of Supply Chain Control Policy Using Closed Loop Multi-agent Simulation" However, these kind of works are very difficult to publish and get through reviewers as most of them either underestimate the scale of problems or would play down algorithmic novelty.

[@expressionoffreedom7165]

I don't know what you precisely said, but it seems to me what we need to do is defragment society. I'm completely convinced the planet is a giant hard drive and nobody has taken the time to run a defragmentation on it. And now we have these wildly inefficient logistics and people competing in nonsensical ways.

[@expressionoffreedom7165]

Also the problem is your trying to forecast, and I think there needs to be an amazing way to preplan everything so there's no need to forecast. If your customers and their customers and so forth had a much better and more accurate plan, forecasts would have to be perfect.

[@chrisford7033]

@@hardik114433 thank you so much I will dig in deep with these resources.

[@commetking9746]

I’d rather see him interviewing Putin to be honest with you -or how about Alex Jones!

[@martinmuldoon8166]

Lex.... You are killing it. Keep up the fantastic work!

[@lnc-to4ku]

What a great guy, and great conversation! Thank you, Lex!

[@Laayon19]

You have easily become my favourite person on the webs. Your calm presence provides me with hope that this world can be subtly guided into a future where intellectual conversations about our world become more popular than the superbowl.

[@roboangst]

A BIG thumbs up to Lex for having this level of content with NO ads. Thank you Lex. Heading over to Patreon now...

[@tiadiad]

Excellent guest. Also enjoyed the lightheartedness of the episode. Was fun.

[@ninadesianti9587]

Thank you, Lex. I enjoy this conversation.

[@josephhayes6071]

You do you Lex - I think we will all tune in for quality conversations, or a storytime format. Like a radio show in the great depression. Thanks dude, you're keeping me sane. I suggest War of the Worlds as a topic for your project.

[@8adeight]

I'd love to hear Lex's views (and others) on historical turning points and events that shape our ideas of the world today. More clear discussion is vital to our social health these days. And we should be having as many of those conversation as we can--while we still can openly have them. Someone mediating those conversations with a deliberate mind driven towards love, acceptance, and understanding is so incredibly valuable as well. So thank you.

[@michaelheffner2853]

A nice comment for u to enjoy...great job..thank you for sharing your conversations!! Keep up the excellent work!!

[@austinroger4233]

Beautiful, Brilliant podcast.

[@benniegirl1533]

I am officially in love with this pod cast ❤️

[@mxclwolf]

Now trying to watch the whole back log of episodes I have missed

[@benniegirl1533]

@@mxclwolf Right!!! ❤️❤️

[@CISMD]

Interview Dimitri P. Bertsekas, author of Reinforcement Learning and Optimal Control

[@Deb.L.]

Mic - "I do believe in the power of social media to screw us up royally" Lex - "I don't shut up about love and I'm all about kindness, I now have many little mini armies of trolls" There, social media amplify, enable and accelerate ideas and things, we have the bad and the good in these platforms. Both come together, can't have one with out the other

[@Deb.L.]

@Iahi Ghusak Thanks to you, I took a look at the tfatk comments, many were very funny! Burst out laughing at a few of them. Lex is as sophisticated as he is guileless, there is this streak of stubborn naivety about him as a scientist - who is well versed in abstract thinking and capable of complex systems thinking - that is odd. I think many people can see this oddity. It's this contrast of traits that makes him interesting, oddly endearing to some and oddly off-putting to others!

[@eggmendy]

Another interesting guest, can't wait to hear it!

[@brendanfoster5913]

A member of the army of trolls here. A loyal member taking orders from the general and spreading peace and love!

[@TheRealStructurer]

Nice to listen to easy going conversation on such an important topic. Thanks for sharing 👍🏼

[@VileHint]

Yeah, solo episods sounds intrtesting. I would also like to hear them on mentioned topics. Could you also consider something related to 1984 or Zamyatin' "We"?

[@centurionstrengthfitness3671]

"Evian is naive backwards." Brilliant!!!

[@dewildeworld6766]

Let your creativity flow! I for one,look forward to all of your content.

[@burkebaby]

Absolute 🔥 episode Lex, thanks!

[@koko969w]

My favorite episode as of late. Great personality and really fun to listen to!

[@ramihertz]

im a software enginner. I listen to your podcast at work and feel like i can solve my problems better and faster

[@drad565]

Loved him at neurips!

[@cooper7138]

Inspiring conversation

[@mistiehandy2214]

Hey Lex, I saw you on Rogan, now I'm all yours since he sold out to go to spotify! Bad move! So far I love what I see! Thanks! 😘

[@gren287]

Hey Lex, you are not a slow thinker, you are just the bigger model :D

[@sunflowersandrainbows2656]

Awesome guest with great view points. 29 and have always stayed away from (SM) platforms. I just wrote a essay on this tonight i wish i was able to cite this guest before i turned it in! I mostly used travis and jaron. My generation is very confused right now and watching younger generations with technology in their hand at the age of 3 is insane. Hope we can turn this around

[@texasrhino2058]

This is the best A I podcast

[@zygiwong]

Lex, I implore you to read Pluto. A japanese manga written by. the great Naoki Urusawa. Without spoiling too much it handles the social and moral issues of AI and how theyre being integrated with society. Its an amazing story and I believe it is right up your alley.

[@Ollychamberlain]

Apartment tour video!

[@mb345]

I enjoyed this episode. I did find that Michael's humor felt like a dodge in places where the conversation could have gone deeper. And, I bought both the books he recommended. Complete novice questions: Is Alexa an NLP training mechanism that centers on human interaction? If it is, are we training Alexa with every sound we make?

[@tunnelsloth5948]

>bell notification Saturday/Sunday night >"oh shit I bet it's Lex" >*click* >"hell yeah"

[@shak535]

Lets here Lex talk movies ! This show is great .

[@amoya3642]

Lex, I really enjoy your podcast. I’ve been a huge fan of JRE for years and I see a lot of similarities with your show, but I appreciate the slight differences. Have you ever considered bringing someone onto your show that isn’t famous or well known? I think it would be really cool to hear you have a conversation with someone from the general public about the world, random topics, etc. Just a suggestion. I would do it, but if not me, I would like to see someone else. In any case, I love what you do and the message you’re spreading to the world. Love wins.

[@davosholdos1253]

The King of monotone! I love it!

[@samgee7999]

As for future conversation suggestions - I'd love to hear and watch a conversation of you with Dr. Fiona Hill.

[@johanneshagemann3669]

Mentioned book recommendations: - Program or Be Programmed (amzn.to/3gJ5f45) - The Alignment Problem (amzn.to/2K3pT2V) - Exhalation (amzn.to/383uHgZ) If you are interested in the book recommendations of all guests of the Lex Fridman Podcast, check out the: lexfridmanlibrary.com

[@zerolullabiez]

Man, 2 hours can go *real* fast. 😸

[@mlittman]

In case it's not obvious, he's very disarming to talk to. I talked to him for a few hours THEN Charles Isbell came by and we talked some more. Between me and Charles, he talked for something like 7 hours in two days and he was still energized and ready to talk more. Remarkable!

[@SD-ik2xu]

Firstly, I want to express a deep gratitude for all the amazing content you have thus created. Secondly I'd like to offer up two potential guest suggestions... Jim Simons and Matthew McConaughey.... Both ambitious bookings, but ones which I suspect could broaden your audience, and offer great insights into their respective fields. Finally, allow me to finish by saying, keep up the terrific job, signed a fellow RL researcher

[@keithschaub7863]

Hi Lex, if you haven't seen the series Altered Carbon, much of what you and your guests philosophize around are highlighted in this movie. I would be interested to hear your guests take on it as a possible future. It seems at least that nearly all the technology aspects are certainly achievable within the next 100-200 years.

[@arthurwallen7342]

It's awesome how this is turning into everyone's favorite new podcast. But it wont be long until people find a way to call him a sellout since I notice that's the cool new thing to do.

[@stefanagha3248]

Lol it’s always been a thing to call people who get fame sellouts?

[@Jacob-sb3su]

I enjoy listening to these shows, It brings me back to the passion for education. I lost my way for a bit, and it feels like I've closed that door for myself. at 26 I feel like I cant make an impact in that world like I once dreamed of. Your show makes me feel that passion again. Thankyou lex!

[@Laayon19]

He gives you hope hey. Pretty amazing human he is.

[@GuillermoValleCosmos]

man the first minute of the podcast. About making technologies that people can make my own is something Ive just been thinking about for my vr+ai project; it's cool to see instant resonance like that~

[@GuillermoValleCosmos]

and then dance, which is what im actually currently doing for my project. Baader Meinhof be spooky XD

[@puddspudds5102]

Would love some solo episodes..would 100% watch

[@aliakbar-jy7ol]

I recommend some episodes in the research labs to observe working and developing in the real world. Like MIT Computer Science and Artificial Intelligence Laboratory, Alan Turing Institute, or others.

[@nikodraganic]

I very much like the notion of you doing the solo episodes, I'd like to hear about your top SciFi novels and about your thought process while reading them. As far as teaching ML, I will be taking MITx Machine Learning in a few months, so I am looking forward to watching your videos on that. If you're taking suggestions, I propose LSTM, GANs and A3C.

[@omkulkarni637]

I just saw the Assasin's Creed Odyssey figurine beside's Lex. Very Cool

[@ahumandoing6813]

Born too late to explore the planet. Born too early to explore space. Born at just the right time to listen to Lex Fridman podcast.

[@devonk298]

I'd love to see you do an episode on your obvious love for fashion.

[@ka9dgx]

Lex, please read up on Multilevel Security / Capability Based Security, it offers a way to have general purpose computing that is secure by default. Application programmers wouldn't have to worry about their bugs allowing a system to be corrupted, for example. Genode.org is working on a version that is moving towards the mainstream. Eros/KeyKos are historic systems that implemented it on mainframes. It would be a good topic for an interview if you can find the right guest.

[@matthewkeaton656]

Lex, if you do a solo episode on AGI using Ex Machina as a basis, that would be the epitome of podcasting on KZbin in my humble opinion.

[@purbidmusic]

lex please review "can't hurt me" and talk about stuff you learned from David Goggins

[@eddieromero952]

The solo episodes sound great

[@_traphic]

@Lex yo, i just wanted to say that i really appreciate the fact read the comments and make it known. alot of times when i watch or read things on the internet & want to share my input, i dont, because most people of a certain status dont interact with their audience and i feel my input isnt heard or cared about. its a really cool thing when a content creator actually participates in their community, rather than just being someone to watch. it gives us, the viewers, a sense of belonging. we feel you actually care about us as people. we dont feel like another number, another view, another like, or a piece of your paycheck. i can confirm your idea, that there ARE alot of thoughtful, intelligent, kind people on these social platforms that dont speak because they dont feel included. i look forward to a day where these type of people are able to participate in healthy online discussion, even if its conflicting. thank you for being passionate and optimistic. if i feel this way, i know there are other people that feel the same, which means you are making an influence in a positive way. keep it up, bud. thanks again.

[@expressionoffreedom7165]

I think your only option is to make your own videos. In reality the comments are mostly a one-way street. They are after all called comments and not dialog. You are right it's nice to have said dialog, but I just don't think the KZbin comments are where it's at overall. I think that if you have unique input it probably is worth making your videos. I think the algorithm is pretty good at sorting the more unique stuff to the top. For instance, most of the rip-offs are usually ranked lower than the original. I'm not criticizing your comment. I think it's correct. I just don't think KZbin comments is always the healthiest location to express yourself.

[@_traphic]

@@expressionoffreedom7165 What i was saying is mostly a response to things said in the latest Joe Rogan interview w/ Lex which brought me here, to give you some context. I have all the equipment to have my own channel/podcast already, that i purchased for filming live music performances on my channel, Moshpit TV. Some of the stuff i got with plans for commentary & create from a personal platform, but there is some type of creative/motivational roadblock happening with me. im trying to figure myself out & spark my creative energy again. i definitely agree with you & recognize that my satisfaction for self expression could be better served in a different place. But i was more or less responding to the joe rogan piece specifically about conversation in comment sections & on twitter. you may ask why wouldnt i leave my comment on that video. the answer being that joe has said 1000x over again, he doesnt read the comments. so here, im speaking aimed at the other party in that convo, knowing there is a likelihood he will read it & that in itself makes me feel good.

[@molunz4598]

The first “I’m not high on my on supply” interview - thank you.

[@yorkwestenhaver8680]

You should spend some time talking about neurosymbolic reasoning for your upcoming AI class at MIT

[@ALEXSTELSITUL]

Please upload your podcast on Spotify with video too :D

[@soseboogs]

cool talk. really like the topics overlayed on the play timeline.

[@oed572]

Cool talk

[@OfferoC]

I really enjoyed his RL class on udacity.

[@patrickmederitsch1380]

Hey does anyone know why Lex is mentioning that he is russian when asking philosophical questions like "meaning of life"? I noticed this in several podcasts of him now so I was wondering what's the connection. Thank you and happy holidays!

[@MrJack8782]

Can't play an instrument... lmao. Lex taking it like a champ

[@robbyr9286]

13:36-- Showing how it's done.

[@nataq]

@13:34 I was in the other room and heard Michael say: "I'm like you, I can't really play an instrument." I ran to rewind and see Lex's face because he just kept talking.

[@mlittman]

No, no, I said "UNlike you!" Before we started recording, he told me about the various instruments he plays.

[@nataq]

@@mlittman Thank you for your reply. Sorry, I miss-heard.

[@vfpNadrius]

I'd love to see some videos centered around movies like 2001: A Space Odyssey or Ex Machina! Or the Star Trek: The Net Generation episode "The Measure of a Man." Looking forward to seeing what you come up with.

[@KnightMirkoYo]

I wonder if Michael also read Egan's sci-fi. I feel like his works also do the "take a scientific idea and build a world by extrapolating it into the future" thing as he described Chiang's books.

[@johnjordan3851]

love these man

[@Michael-hs6ii]

Great content, nice to get insight and not pay tuition

[@adeline7980]

The idea of a solo episode about the rise and fall of Hitler, and a Stalin one, sounds super interesting. I wanna hear your thoughts on the Rise and Fall of the Third Reich.

[@MegaTyde]

Lex, you're fantastic! Keep this momentum going. Excited for your potential Putin podcast!

[@BILLY-px3hw]

three minutes after post! Hey Lex

[@stefanagha3248]

1 hour after comment, hey Billy!

[@drq3098]

Thanks for this podcast! Can you please think of a podcast about AI completeness? S. C. Shapiro wrote about AI complete tasks in his 1992 paper titled Artificial Intelligence.

[@prof.m.ottozeeejcdecs9998]

Since you seem to like Graucho Marx... here one from George Burns: Look to the future, because there you spend the rest of your life!

[@rickharold7884]

Cool. Thx as always for the awesome video.

[@tomiesz]

The myriad of people with very singular tasks that Michael talked about might also be partially motivated by the various acting/movie production (idk the proper name lol) unions.

[@akhiljohnson6019]

Enjoyed it!!

[@darahamidi]

"Film Canister" LOL...

[@indamountains7953]

Here for the intellectual stimulation

[@m3tamargaret228]

Can you go over how to get into AI later on in life? Where should one start?

[@mlittman]

Depends what you mean by "get into". There are tons of resources online these for diving into the ideas and doing your own experimentation.

[@Sebastian_S_Azar]

It's a useful tool to prefer optimistic thought-patterns, but to play the game of being overly optimistic about everything is a bit annoying, and that's pretty much a valid description of the guest in this episode.

[@ezchx]

Lex: "Our collective intelligence is already being controlled by algorithms." Whoa.

[@paultan5419]

wow im early, love your videos lex keep it up

[@davidfetter728]

I Like Lex's Hedgehog.

[@stalie6098]

It’s not a „found, directly clicked and viewed situation“ it’s a comment without need to watch situation 😬😅🙈doesn’t matter I will like it or not after watching^^ thx Lex !!!

[@delatroy]

1:44:44 forces that rise to the top are not universal

[@ycyang2698]

But when you talk about Stalin and Hitler, pls be sure to talk about Winston Churchill (1920) as well.

[@bautistabaiocchi-lora1339]

Waiting for part 2

[@MG-lg3qy]

Love your works... we once wished the lab rat could give verbal feedback. Now that humans integrate with machine learning...... I give feedback almost too much but changes needed for the specific human program to create the same vision.. Is not used Use our feedback if you want us to cooperate

[@llothsedai3989]

Sounds like from a human perspective, superhuman machine intelligence is like having a genius in the room. The hard part is taking those nuggets that they have learned and taking it back out. Perhaps comparing self play and the best of human knowledge and make a diff, to advance our poor brains. For the go or chess example, we'd want to know the new strategies that it finds, and unless you are an expert you don't necessarily known if a strategy is new or good and why.

[@marisolmendoza9908]

noooo I have a math final tmr I have to study for, but I can't resist such a great podcast. Couldn't you wait until finals were over??? Just kidding, so thankful for all your deep conversations!!!