Sergey Levine on the bottlenecks to generalization in RL and picking good research problems

Sergey Levine, an assistant professor of EECS at UC Berkeley, is one of the pioneers of modern deep reinforcement learning. His research focuses on developing general-purpose algorithms for autonomous agents to learn how to solve any task. In this episode, we talked about the bottlenecks to generalization in reinforcement learning, why simulation is doomed to succeed, and how to pick good research problems.
Blog: generallyintelligent.com/podcast/2023-03-01-podcast-episode-28-sergey-levine/
Spotify: open.spotify.com/episode/25aWr3OsE3fVNEw6cEhoB4
RSS: anchor.fm/s/42cab330/podcast/rss
About Generally Intelligent
We started Generally Intelligent because we believe that software with human-level intelligence will have a transformative impact on the world. We’re dedicated to ensuring that that impact is a positive one.
We have enough funding to freely pursue our research goals over the next decade, and our backers include Y Combinator, researchers from OpenAI, Astera Institute, and a number of private individuals who care about effective altruism and scientific research.
Our research is focused on agents for digital environments (ex: browser, desktop, documents), using RL, large language models, and self supervised learning. We’re excited about opportunities to use simulated data, network architecture search, and good theoretical understanding of deep learning to make progress on these problems. We take a focused, engineering-driven approach to research.
LinkedIn: linkedin.com/company/generallyintelligent/
Twitter: @genintelligent
Spotify: Generally Intelligent

[@francisrussell3966]

Thanks for the great podcast. I think it will be better if we could see participants in motion instead of a still image here on KZbin.🤗

[@gulllars4620]

This was a great podcast. I really appreciate how good and concise Sergey is at formulating distilled important concepts in an easy to follow way that gives real insights. I think i had like 5-6 aha moments in this podcast, which is an hourly rate that is not matched by much other content. Some notable exception examples have been when Sam Altman recently talked to a manager at the Norwegian bank investment management which had both a fantastic interviewer and subject, and some legends on Lex Fridman podcasts like Jim Keller, Stephen Wolfram, Cris Latner, Max Tegmark, Demis Hassabis, and again Sam Altman. One key thing i took away here is the view of transformers learning world models and generation/simulation from them being orthogonal to RL rather than a competing technology, in a much cleaner way than just RL being possible to use with transformers after they are pre-trained. There's also some deep insights into the strengths of each and where they can give good gains or have pitfalls. I also like how he goes for the extreme formulation first principle research, and why that's important rather than just highest probability return on investment projects the private industry usually does.

[@AnasBELFADIL]

Hi thanks for the great content, please if you can lose those audio artifacts the quality of the interview would be a lot better. Maybe once they're fixed it would be interesting to change the interviewer's voice, but for now it's kind of counter productive, given the amazing efforts put to have these nice discussions.

[@QinyangLiu-gq8gr]

From 1:33:34, the audio track seems overlapped(i.e. Sergey Levine's voice is parallel with the host). Please fix it. Thanks.