Sergey Levine: Robotics and Machine Learning | Lex Fridman Podcast

Sergey Levine is a professor at Berkeley and a world-class researcher in deep learning, reinforcement learning, robotics, and computer vision, including the development of algorithms for end-to-end training of neural network policies that combine perception and control, scalable algorithms for inverse reinforcement learning, and deep RL algorithms.
Support this podcast by signing up with these sponsors:
- ExpressVPN at www.expressvpn.com/lexpod
- Cash App - use code "LexPodcast" and download:
- Cash App (App Store): apple.co/2sPrUHe
- Cash App (Google Play): bit.ly/2MlvP5w
EPISODE LINKS:
Sergey's Twitter: / svlevine
Sergey's Website: rail.eecs.berkeley.edu/
Sergey's Papers: scholar.google.com/citations?...
PODCAST INFO:
Podcast website:
lexfridman.com/podcast
Apple Podcasts:
apple.co/2lwqZIr
Spotify:
spoti.fi/2nEwCF8
RSS:
lexfridman.com/feed/podcast/
Full episodes playlist:
• Lex Fridman Podcast
Clips playlist:
• Lex Fridman Podcast Clips
OUTLINE:
0:00 - Introduction
3:05 - State-of-the-art robots vs humans
16:13 - Robotics may help us understand intelligence
22:49 - End-to-end learning in robotics
27:01 - Canonical problem in robotics
31:44 - Commonsense reasoning in robotics
34:41 - Can we solve robotics through learning?
44:55 - What is reinforcement learning?
1:06:36 - Tesla Autopilot
1:08:15 - Simulation in reinforcement learning
1:13:46 - Can we learn gravity from data?
1:16:03 - Self-play
1:17:39 - Reward functions
1:27:01 - Bitter lesson by Rich Sutton
1:32:13 - Advice for students interesting in AI
1:33:55 - Meaning of life
CONNECT:
- Subscribe to this KZbin channel
- Twitter: / lexfridman
- LinkedIn: / lexfridman
- Facebook: / lexfridmanpage
- Instagram: / lexfridman
- Medium: / lexfridman
- Support on Patreon: / lexfridman

[@lexfridman]

I really enjoyed this conversation with Sergey. Here's the outline: 0:00 - Introduction 3:05 - State-of-the-art robots vs humans 16:13 - Robotics may help us understand intelligence 22:49 - End-to-end learning in robotics 27:01 - Canonical problem in robotics 31:44 - Commonsense reasoning in robotics 34:41 - Can we solve robotics through learning? 44:55 - What is reinforcement learning? 1:06:36 - Tesla Autopilot 1:08:15 - Simulation in reinforcement learning 1:13:46 - Can we learn gravity from data? 1:16:03 - Self-play 1:17:39 - Reward functions 1:27:01 - Bitter lesson by Rich Sutton 1:32:13 - Advice for students interesting in AI 1:33:55 - Meaning of life

[@spacenodus7959]

Lol you were about to laugh when you said it's the best vpn 😁

[@Pmc07AyeUrDa]

This podcast is a gold mine. Thank you Lex for bringing these people on!

[@rockefellersavage4122]

@13:00

[@LiveAndLetLive2024]

What a thorough list of contents! I wish every video of this type had such a thing - super helpful. Thanks for taking the time to do that!

[@Karadjordje2]

i

[@ArmandoVieira]

Sergey is one of the few Deep Learning researchers I have a huge appreciation. He is brilliant, prolific and an incredible communicator in one of the hardest areas of AI.

[@BrianLKL222]

This is a podcast that makes the audience smarter, amazing job Lex!

[@jamesgerardmccarthy2897]

Sergey is such a hero in this field, I'm glad you had him on. His Stanford course on RL is excellent

[@rohlay00]

Thanks for the recommendation

[@Stwinky]

His Berkley Deep RL course is great too

[@akarshrastogi3682]

what course in stanford? cs234??

[@zmanx88]

My understanding and appreciation for this field has increased tremendously since listening to lex podcast

[@TheAllboutwin]

I freaking LOVE Sergey!! I'm doing his course from Berkeley and it's beautifully explained. Thank you Sergey!!!

[@mirunahodo4573]

I was so excited for a new episode now that i have finished with my med school exams for this summer. Thank you, Lex!

[@yashkarbhari721]

Lex, bring Geoffrey Hinton, I am very eagerly waiting for this.

[@markusgross2502]

I still can't comprehend why this didn't even happen yet! Without Geoffrey Hinton in Lex' Podcast, the AI picture isn't complete.

[@Stwinky]

I’m sure lex has requested one, but I too am eager for that one

[@autohmae]

@@Stwinky Maybe Lex wants to interview a lot more people to make sure he can come up with some good questions for Geoffrey. :-)

[@gmork5051]

Lex "I apologize for the romanticized question" Fridman

[@readinginterviews2431]

Great conversation. Thank you Lex for conducting these fascinating interviews. Hope to get to interview you one day!

[@boyscoutalchemist]

I don't understand much of this but I can't stop watching. Absolutely fascinating, honestly.

[@jackiekutty]

Best Comment

[@autohmae]

Do you have any questions ?

[@caravantransportllc8864]

Once you start watching, you can’t stop , it’s kinda addictive .

[@GeorgeHennegar]

I love Lex's podcasts because if I feel desperate for meaning, I can pick any given podcast, fastforward to the end and find a potential answer.

[@randpaul9863]

I find that many of my favorite guests on this show are very humble

[@vasileioslambrosflorostson2976]

wow, all top science fields offered by this great channel... AMAZING STUFF

[@insheeption23]

Great conversation as always Lex! Keep it up!

[@gaeb-hd4lf]

This is a very nice length for podcasts in my opinion, between 90 and 120 min. Awesome content!

[@alexandrahopkinson1748]

More = more

[@Stwinky]

Sergey is not only a great researcher, but an amazing instructor. He can take complex aspects of RL and explain them like I’m five.

[@lucaswood7602]

Thank you for such a provoking conversation Lex.

[@markoshivapavlovic4976]

Finally good interview from the area of Robotics and Reinforcement Learning. :)

[@LockeLeon]

Loved the way he answered the question at 17:45

[@xSNYPSx]

all this stuff you doin, Lex, is Amazing

[@pamirghim]

Can you invite Joshua Tenenbaum for a podcast Lex. I think it might be a nice follow up to this episode. Thanks for your beautiful work.

[@Yuri-bl4ec]

Such a lovely guy, loved the podcast, thanks!

[@PussyDestruckter]

Great Podcasts

[@judgeomega]

9:47 i think there is a deeper point to be made about our knowledge base; in humans it is directed learning. we dont memorize the position of every pebble in our driveway. we dont have robust internal models of pebble dispersion over time. even when the pebbles are in sight, we dont consciously think about them. one of the keys to human intelligence is ironically knowing what NOT to think about.

[@denniswigand8066]

Really great! Sergey is a huge inspiration!

[@vivekmittal1454]

I was waiting for this for soo long.

[@henrikjohnsson7403]

This was great! Thank you!

[@vaibai5869]

@Lex Fridman, have you considered inviting Eliezer Yudkowski yet?

[@AlanW]

When Lex asks 'How large is the gap' I really wanted Sergey to ask "in what units?"

[@rohlay00]

Thank you for this!! It has changed my life. I'm currently studying robotic engineering (second year) and my degree focuses a lot more on the hardware side of it. There is a lot of electronics and systems engineering. I have realized that what I find most fascinating is the software and the algorithms that run it. I have recently heard about artificial intelligence and machine learning and I think that is what I want to pursue (applied to robotics). I wonder if maybe I studied the wrong degree and should have studied computer science, but I think that an engineer can still learn the skills to go into that field. This video has really inspired me. Massively. I am seriously thinking about doing a masters and then maybe a phd and do research in AI applied to robotics. But I have no idea what it's about, what the practical real life problems in this area looks like, etc. Could anyone recommend me any courses? (I will do Sergeys!) Any books on AI and machine learning applied to robotics? This video has made me very excited about the future and about my career. Thank you!

[@supavit]

You can learn ML (CS182) and RL (CS285) in his lab channal. kzbin.info/door/4e_-TvgALrwE1dUPvF_UTQ

[@Zhiel01]

I'm happy for you man

[@TheJonas1014]

Lex, I hope you will bring George Church to the podcast :)

[@cristobalossa3871]

From Chile, I congratulate you on your interviews ... someday Compile them in a series explaining the Fundamentals of our science, T0,7 and the challenges that come ... I imagine you have many messages on Linkedin? // Desde Chile, Te felicito por tus entrevistas... algún día Recopiladas en una serie explicando lo Fundamental de nuestra ciencia, T0,7 y los desafíos que vienen... Imagino que tienes muchos mensajes en Linkedin ?

[@florisas.7557]

thanks! more robotics please :)

[@georgesnomicos8943]

great podcast!

[@Virsconte]

12:32 A random walk wouldn't be IID at all though. The samples in a random walk are very correlated, just like the samples from adjacent time steps of a human life. And I think that's what makes those series of related points so much more useful!

[@ConstantineSad]

Отличное интервью!

[@anandbalivada7461]

This was a really cool one! Lex, could you do a podcast with William Bialek (if you haven't already :D )?

[@christianleininger2954]

so great thanks

[@wesleyolis]

I like the idea of NLP for mathematics that can transform mathematical sentences for integration and differentiation.

[@ahmad_serendipity]

Professor Lex ... All your interviews and guests , are certainly great and genuinely interesting ! May I suggest to have Andreas Antonopoulos on the show please ?

[@TreyMK12]

i truly appreciate your work & youtube channel. just a question, but is there any way you could possibly get an editor that breaks down these talks and puts them into small snippets? i enjoy when you post videos shorter than 10 minutes because some times i am in a rush and don’t have time to stop and listen through the entire thing. just a thought, once again i truly appreciate everything you do & bring to our collective consciousness

[@jaredathey1476]

He already spends a huge amount of time editing, breaking down, time stamping, and labeling the sections, not to mention other the pre and post-production work to make these incredible videos. Use those stamps/links or just pause the video for later. If that doesn't work for you then I highly suggest you support him directly through Patreon, for example, so he can afford to hire an editor.

[@marohs5606]

he usually does that after few days from the full interview

[@archier589]

Yay, thank you

[@shoubhikdasguptadg9911]

Lex "I will insert philosophy unnecessarily in to every thing " Fridman

[@TalkThisOut]

so fun! lets go fam

[@benbakhshi]

What is the best way to learn to develop a reinforcement program?

[@rajitchandra8767]

hey I am looking for some machine learning research topics, anyone tell some topic which can help me, also if it's application related to medical then it will be even better.

[@TheAIEpiphany]

I find it beautiful that Isaac Asimov had a profound impact on Sergey's passion for the field. It's funny how people have been influencing each other since the dawn of civilization.

[@JTMoustache]

Legend

[@mj2068]

the man is like the definition of a sharp man... in every way. i bet he's even tall😊

[@DamianReloaded]

1:09:00 I've the feeling that simulations will tend to migrate into being generated by neural networks entirely. If this was the case then the neural network running/generating the simulation could have the ability to improve the simulations it generates from real world data. Our brains have this ability of being able to visualize hypothetical (never seen before) scenarios to solve problems and form new models of the world. There's been a recent (nvidia?) paper about running/generating Pacman entirely from a neural network and others about video prediction (dreaming) so it seems like a possibility.

I bet you both could speak russian

[@pauloabelha]

42:38 Jacob Andreas NL for RL (handle on problem solving through the NL string) arxiv.org/pdf/1906.03926.pdf

[@TheKeysMan100]

@ Lex Fridman, could reinforcement learning be some form or imagination ? from wiki "imagination is an experimental partition of the mind used to develop theories and ideas based on functions. Taking objects from real perceptions, the imagination uses complex If-functions that involve both Semantic and Episodic memory to develop new or revised ideas.[15] This part of the mind is vital to developing better and easier ways to accomplish old and new tasks" , what do you think ?

[@heyrmi]

Finally

[@dipamchakraborty]

We want to learn from each other, and hence also want to learn from AI systems by having them explain their work to us.

[@danielx1912]

Looking back from 2023, this is exactly chatGPT

[@ganeshkumarchiramshetti5587]

Your interviews are really good and much info, But need some humor and comic to run

[@SahakSahakian]

Lex was so sleepy that he even forgot to say the usual "Iam Russian" line this time

[@jonschmid5383]

lol He always sounds like he hasn't slept in 3 days... strange lol

[@judgeomega]

"its easier to create a universe than a brain. thats kinda weird"

[@ka9dgx]

In reality, the situation is reversed... almost anyone can have offspring, creating a brain in the process... Nerfing the universe for them just the right amount, and decreasing the nerf level as the mature... is a lot harder!

[@vast634]

I think the idea of using a human operator to steer the robot - only using the robots sensors and actuator commands - makes sense, to get a sense of the available feedback and control that the robot offers. And potential problems if those are not sufficient. Also to determine the concrete problem space an AI has to operate it. if a human operator cannot solve the problem, its unlikely that the robot could (apart from problems that require very quick reactions).

[@ka9dgx]

1:10:40 - OMG - Parents create a simulation of reality for their children to explore and learn inside of... lots of people don't want to grow up. Parenting is a meta-answer for me tonight.

[@TheKeysMan100]

I'm not sure I'll ever be the same after Bach!

[@emrazum]

Have Sabine Hossenfelder on

[@junweidong2448]

The most SHARPE question of people from classical control theory is:"what can you ganrantee from pure learning?"--- what is the answer.

[@yuvalperry6688]

Common sense is a property of experience

[@marohs5606]

I know I am a Ph.D. scholar and I should be professional but I have always had a crush on Sergey I find him verrry attractive ... OMG.. I will definitely watch all this interview for DRL and Sergy also >__

[@lain11644]

you're creeping me out

[@marohs5606]

@@lain11644 what!! he has a fascinating personality 😍 .. All respect for him😊

[@ai-ur5uv]

I don't know why but I find him attractive too

[@ka9dgx]

1:05:37 - Simple solution, nerf the dishes... just like you do for children. (There is a whole aisle of child usable food containers at your local Target) Punishing contact with the floor, and rewarding the fastest time would fix the dropped dish problem without blowing a budget. Also... once you're a parent, you discover a parallel universe... in between the stores are other ones you didn't notice, where they buy and sell clothing and toys, saving parents a ton of money. (A good place to get things to train robots with too) You could take an old robot, make sure it's water and goo proof... and the use actual water (with food coloring for the vision system), a supply of ice for the solids, and actual nerf balls for any other training. Watch a few kids, replace actual food inputs with something reusable, and fake the reward signal with some well written code. You could train a robot to "eat" nerf balls, for example. This solution brought to you by the parenting experience iceberg. ;-)

[@skipperkongen]

how legal would it be to build a robot, big or tiny, and let it roam freely outside where it could gather experience and have to survive on its own, e.g. keep power on its batteries? How legal would that be? Under what conditions would it be legal/illegal let alone ethical?

[@dr.markbowler-smith1662]

It strikes me that the robot wouldn't break so many dishes if it attributed value to physical objects. Maybe even the idea that dish one is worth x but dish two is worth x^2.

[@dorinfernand7458]

We want Donald hoffman

[@rustyspottedcat8885]

Where is Seth Lloyd ?

[@teamspeak9374]

I feel like you are getting more meta physical with each podcast. Not a criticism, just an interesting observation, i feel like on your first podcasts you were much more focused on discussing the actual topics while now you extrapolate the topics to human nature and meaning of life and so on. I wonder if this is because you are getting more confortable doing podcasts and are letting more of your natural self come forward or if you actually changed during this time and are focusing more on these types of topics in your personal life. Anyway great conversation, thanks for the podcast.

[@sohamdats]

This guy had the maximum number of papers in NIPS 2019 which is 12. That says something.

[@skipperkongen]

Is simulation learning a thing? We start by learning in a simulator, then we use the policy in the real world and learn that it behaves different there compared to the simulator. That means there is an error in the simulator that we then make a step to fix and repeat?

[@frankiethefish73]

Has anyone else noticed that Sergey speaks a little bit like Lorne Michaels, (aka Dr. Evil)?

[@evanwillenson1398]

ISAAC ASIMOV IS AMAZING WHY ARE YOU EMBARRASSED BY IT MR. SERGEY!!! Robot series changed my world. And Foundation obviously. Then you got Arthur C. Clark. Those the GOATS

[@ka9dgx]

1:10:10 - Building up on previous experience in a nerfed world is what children do.... nerfing the real world gets around your simulation problem at the expense of training speed. Of course you can have the systems train 24x7... until somehow it turns out sleep us a useful metalearning mechanism.

[@rogerab1792]

1:21:40

[@divyanshukumar4084]

All the time, He looks downwards. Doesn't look at Lex in the eyes and answer. Classic textbook nerd and genius, Levine is.

[@ka9dgx]

1:22:40 - You align the AI goals the way you do it as a parent... reward good behavior and punish the bad behavior and build trust while it's learning. Many people are smarter than their parents, but still respect them. I'm surprised Elon hasn't thought of that one... he's a Dad.

[@Jannikheu]

I think "parenting" could instill "interests" in an AI learning around and within specific topics and that this could be a very useful way forward to an AGI that in turn is useful to humans and not completely lost in the vast complexity of the world. An AGI agent would then have the plasticity of basically learning anything but still be specialized on some areas and a population of specialized AGI agents could represent something very close to a true AGI.

[@Eltopshottah]

*intensely whistles *

[@carrito1981]

I am a nobody but I will answer your simulation question. You are already in it, just not exactly how you might think of it as a virtual reality, computer program. Your mind is 'rendering' reality through it's own human bio-hardware limitations, your body is the controller. The 'player' is consciousness. Our sliver of reality is what is simulated in our minds, yet it's only a portion of the totality of 'true reality'.

[@ka9dgx]

​@@wyattlightning6681 It is apparent to me that they haven't... almost every time they come to a full stop with an insolvable problem... I find myself screaming some analogy to parenting as the answer...a well understood answer, too. (for parents)

[@bitsurface5654]

You are 100% right! Linux is the best. :)))

[@PierreH1968]

I think XAI is a very important subject. We might be tempted to use AI to explain itself.. Then the AI will realize quickly that lying about, how it came to a wrong decision is more rewarding than explaining the whole truth to a cognitively limited human. XAI might lead to lying AI.

[@TheRohr]

My wild guess would be that it is indeed actually easier to built the universe than to build a brain (1:12:00).

[@TheArthurAbbott]

Bro asks the meaning of life. Smirks. Life is full of meaning. How dare we be so reductionist to pick just one and hold onto it like the gospel truth. Is it to live? To love? To make an (positive) impact? To be happy? It doesn't boil down. But he smirks and asks the question, knowing the chance for a good response is well worth the ask.

[@artking7883]

@theartistbk check out canvas paintings of divine consciousness

[@mobsnitchanonymous213]

Nice outfit.

[@ebeebbeeebbb]

Go Bears!

[@cogoid]

Well put: 33:08 ...the systems that we have now simply inhabit a *different universe.* [of pixels and sentences] And if we build AI systems that are forced to deal with ... *our* universe maybe they will have to acquire our common sense...

[@Torterra_ghahhyhiHd]

actually, i love to build an intelligence without any propose and ambition. because every filosofy stuff etc we thin is right may is really not. could be so much of prejudice inflated by ego or fear of wasted effort, o fear to die, etc. we have so limited real deep understanding. that we assume die i not good a life is good. and if it is wrong the go to the opposite directions. our mind is still mindless compared to real deep meaning in the universe. we say we are aware, we are not aware of the real values even the moral value, etc is full of bugs. we just intent to approach. so every wrong conception on filosofical stuff that we put in the ia it will be exponentially, reflected in the derived way and calculus an integrals problem magnify in the world. keep it pragmatic, keep it simple. do not create firearms. do not realize any complex task like save the world, eliminate poverty or hunger, suppose to get a "mind", be "kind", "don't be kind" (this base is not solid and part of this construction is prejudiced itself in our filosofical conception), etc. "love", "hate".

[@xSNYPSx]

Я писал такое Эссе на тему ИИ, называется Путь к Естественному Интеллекту, если интересно почитай, есть и на английском но там машинный перевод xsnypsx.livejournal.com/265.html xsnypsx.livejournal.com/523.html

[@0ooTheMAXXoo0]

How are you dong this AI wrk without being a Biology expert? Biology, Chemistry and Physics are the base for the understanding you are reaching for... each cell has a lot of choices and decisions to make in its normal life... no atom or molecule is ever alone... there are connections the size of galaxies and the whole universe that look like the networks in living things that seem to make intelligent choices happen. Plant intelligence is a real subject because of similar networks inside plants and how they do react to variations in their environment, seemingly making intelligent choices, albeit with slower results that in animals. Similar networks are found in fungi. Where a single celled organism can be acres in size with many nuclei that travel through the network sharing important information about how to eat various things and fend off various things... Basically, a jumping spider seems to have the intelligence of a large predator like a wolf... Even tiny brains, even individual cells, show an amazing variety of choices and behavior in intelligent ways...

[@schelsullivan]

8:00 "Kinda muddle through it". That there is the key to General AI. The ability of iterative bummfuckery is what makes us successful.

[@hesbonkiptoo1849]

Is it just me or Sergey Levine looks like The Riddled

[@skipperkongen]

Do humans learn to wash the dishes by starting with an off-policy method of imitating our parents and then take it from there? Is it because breaking the dishes is associated with such a strong anti-reward, i.e. getting yelled at, that we burn that behaviour path to the ground and never use it again?

[@thecodingmentor7701]

First

[@charitoncharitonidis2511]

Congratulations 🎉

[@grahamjoss4643]

new topic idea? Emotional Intelligence ? Dare to talk about the US' culture war ?

[@hsrboys9993]

go bears

[@MaksymCzech]

Почему ведущий такой душный :(

[@vadimosm]

та норм ведущий. гораздо лучше дудика. не спрашивает про еблю и лавэ.