Season 1 Ep. 22 OpenAI's Ilya Sutskever: The man who made AI work

On the last episode of Season One, our guest is Ilya Sutskever. Ilya is the Co-Founder and Chief Scientist of OpenAI. As a PhD student at Toronto, Ilya was one of the authors on the 2012 AlexNet paper that completely changed the field of AI, resulting in the widespread adoption of deep learning and the avalanche of AI breakthroughs we’ve seen the past 10 years.
After the AlexNet breakthrough in computer vision, at Google, among many other breakthroughs, Ilya showed that neural networks are unexpectedly great at machine translation, at least at the time it was unexpected, now it’s long become the norm to use neural nets for machine translation. Late 2015 Ilya left Google to co-found OpenAI, where he is Chief Scientist. Some of his breakthroughs include GPT, CLIP, DallE, Codex. Ilya’s academic work, less than 10 years out of his PhD, has ben cited over 250,000 times, reflecting his absolutely mind-blowing influence on the field.
What's in this episode:
00:00:00 Introductions
00:03:00 Why take a closer look at neural net works originally?
00:08:25 What was going through Ilya's mind during the AlexNet discovery?
00:18:25 Ilya's early years
00:21:19 How Ilya stayed motivated
00:29:07 Sam Altman and the beginning of OpenAI
00:36:22 LSTM models and reinforcement learning
00:56:06 How will our productivity change?
01:00:22 Instruction-following models
01:12:13 Ilya's vision of the future of work
01:16:14 Ilya's advice to be productive
| SUBSCRIBE TO THE ROBOT BRAINS PODCAST TODAY |
Website: therobotbrains.ai
Twitter: / therobotbrains
LinkedIn: / the-robot-brains-podcast
Host: Pieter Abbeel
Executive Producers: Ricardo Reyes & Henry Tobias Jones
Associate Producer: Alice Patel
Audio: Kieron Matthew Banerji
Video: Bo Obradovic
Music: Alejandro Del Pozo

[@RickardHallerback]

He is so pedagogical in the way he speaks. He makes it easy to follow and understand. Only someone who knows what they are talking about can make these types of clarifications on these different complicated topics.

[@bhaskarbhachu315]

0

[@bhaskarbhachu315]

P

[@joeremus9039]

Yes, sometimes the best in their fields are also the best teachers in their fields, e.g. Richard Feynman. They have the brains to see the forest from the trees, and will explain the keys concepts so that you can see where the details fit into the general framework.

[@ELECTR0HERMIT]

This is what TRUE GENIUS looks and sounds like folks. Not sitting around theorizing, but actually solving and making a next level invention. I'm so glad these guys are so young, we need a lifetime of young innovators perfecting AI, which will in turn perfect AI which in turn will perfect AI and create a self-realization system that improves exponentially. We will need this to save the planet and develop medicine and power supplies and other future challenges. I'm very very grateful and excited because things are taking off right this very minute. Hang on tight.

[@MrAshu-mx4dh]

absolutely

[@AlecsStan]

These people are historical figures. These are the giants that give their shoulders for future generations to stand on.

[@AlecsStan]

@Intel Bro You are aware of his bio and the fact he cofounded Open AI and developed GPT? What have you done?

[@foreverseethe]

​@Intel Bro paid by who? I was going to make a joke about robots standing on our slumped shoulders, but your lack of awareness is more ridiculous.

[@fanyang1968]

THE best interview about LLM, GPT, OpenAI and Illya's great work! Thanks a million!

[@zeweichu550]

Ilya is a genius we should protect.

[@JousefLITE]

Definitely one of my favorite episodes in this season! Thanks Pieter!

[@TheRobotBrainsPodcast]

Great to hear!

[@prabhavkaula9697]

Thank you so much Pieter sir. This podcast is a gem.

[@balapillai]

Ilya Sutskever. He who inflected AI. The magnitude genius. What great inspiration and company to my toil on steps to infinity & the source of pessimism. The first village in human evolution would not be possible with the amount of pessimism we have now. Let this sink in!

[@LoisSharbel]

Thank you for this fascinating interview. It is dense information for me, with NO study in these fields at all, yet with your charming enthusiasm, interesting questions and Ilya's meticulous answers, I am able to glimpse the processes that have led to such exciting discoveries. I am in awe of individuals like this man, you, Geoff Hinton and Elon Musk, who are openiing our minds to horizons unimagined even recently. THANK YOU!

[@TheYashakami]

This is one of the better interviews I've seen. Well done

[@osalicsomonic6069]

Absolutely great talk! Thank you. Must Rewatch, I guess.

[@emiliomorales2843]

Thank you for this episode. Also It would be great listen to Alex Graves and Aäron van den Oord

[@dik9091]

great to hear this now!

[@Appleloucious]

One Love! Always forward, never ever backward!! ☀☀☀ 💚💛❤ 🙏🏿🙏🙏🏼

[@carvalhoribeiro]

Great conversation. Thanks for sharing this

[@aslanyureky]

WOW ! Thanks for the interview with the legend coder

[@rrr00bb1]

Jeff Hawkins was going on about prediction being the key to everything; in an old video from maybe a few years ago. That always stuck with me. If you have senses and actuators, they are equivalent to current input, and current output. If your brain predicts how something will feel before it is touched, or predict what you will see in the next frame; then you have a generalized basis for your senses to supervise the learning. It is definitely intuitive when it's a matter of predicting the next word.

[@JoePiotti]

So when AI makes a Time Machine, this is the guy we will have to go back in time to stop.

[@TheAIEpiphany]

Cool episode Pieter! It'd be cool to hear a bit more details about Ilya's life and his way of working and thinking about the world. It felt slightly (khm khm :)) ) terse from his side: either because of his cultural background, personality or simply because that's his and OpenAI's secret sauce. :P

[@Kage1128]

cant believe im only discovering ur podcast now so underrated! amazing questions! wow!

[@TheRobotBrainsPodcast]

Welcome aboard!

[@dreamphoenix]

Thank you.

[@rui569]

Well and simple explained. Almost looks a easy matter.

[@Alexander-ym7zx]

WOW! Amazing

[@blek1987]

I'll be listening to this until I understand it all.

[@johnpenner5182]

great interview, with some good probing into the insights (deeper vs wider nets). lolz at his dig about using a convNet (convolutional neural network) could provide good intution for GO compared to the 'non-neural network goblin systems at the time'. 🤣

[@oneflow1934]

Hi, Pieter, your podcast is very inspiring to me. I wonder whether we can translate the closed captions of some podcast into Chinese and publish the content as blogs. We will keep the original link of video in the blog. I believe many others like me will find the content inspiring but didn't know your podcast before.

[@TheRobotBrainsPodcast]

Certainly! We support the sharing of information with the proper citations. Thank you for back linking to us!

[@oneflow1934]

@@TheRobotBrainsPodcast thanks very much.

[@joeremus9039]

He has great insight and is very good at explaining the core concepts. Neural nets are the way to go. Dr Chomsky believes that the are important structures embedded in our minds in addition to the fluid neural nets. Anyone care to create a Chomsky bot that would fool any interviewer into thinking they are speaking with the real Noam Chomsky?

[@SingularityLabsAI]

57:00 To clarify the word medium and Long . medium term < 10 years . Long term < 20 years.

[@prabhavkaula9697]

I wanted to ask you about the Geoffrey Hinton podcast, when will it be shared? (You hinted sometime in june)🤔

[@TheRobotBrainsPodcast]

Stay tuned 🤫 ... we are getting very close to our season finale!

[@billyblackburn864]

i have over 6000 hours in dota 2, i remember when openai showed out at the International. thats when i really saw AI do something amazing for the first time

[@Wyvernnnn]

Can you imagine a machine code to source files translator? My god. We could even use open source stuff as an oracle

[@kemalware4912]

great

[@AxiomaticArt]

Is there a transcript of this podcast?

[@RaviAnnaswamy]

I dont know about official transcript but you can get a usable one, in case you did not know, from this page. If you click on the three dots next to 'SAVE' and choose Open Transcript, the ttimestamped transcript appears on the right of the video. If you want a downloadable copy, you can toggle timestamps (remove) and then copy paste into a document. If you already knew this, sorry for repeating.

[@raghavendrakaushik4871]

At 22:14, what is the name of book and its author?

[@johnsagan7456]

"Genius Makers: The Mavericks Who Brought AI to Google, Facebook, and the World" by Cade Metz

[@barbiefit3282]

Este es un video interesante.

[@fallbro]

Link to the paper by James Martin?

[@alberttakaruza5612]

*Chibaba ichii* He a GOAT!!!

[@lukestrap9949]

mpff, GPT-4 is taking FOREVER to be released

[@Kami84]

Are you happy now 😂

[@lukestrap9949]

@@Kami84 yeah 🤣😅

[@danielthen4415]

Wow very very good north japanese and west vietnamese and south thailand and east taiwanese and central korean and china city in the world cup champions shipping by since 1921......

[@eggsaladsandwich9788]

im pretty sure the host is ai generated

[@dotnet364]

12:36 in a modest way? just give full credit to the real contributors, please.

[@swissheartydogs]

Human thoughts…

[@emcastanea62]

Fascinating. Leaves me wondering about why biological evolution invented neural networks. Doubtful that the primary driver was enjoyment though.

[@jonathanlatouche7013]

1-:55

[@do.what.you.think10101]

Вот у кого Дудь должен взять интервью!

[@csanadtemesvari9251]

LOL I thought Ilya was a girls name

[@dotnet364]

04:01 again with the "every child knows that" insult by Ilya. Well sht, now you are gone after Nov 17 2023!

[@jamescullins2709]

looks and talks like an AI unit to me..... I do not mean this to be disrespectful just an observation.

[@dotnet364]

01:16:14 so much time now, after the scandal of nov 17 2023. ilya is nowhere to be found.

[@I_dont_want_an_at]

Report this video for dangerous acts. Enthusiastic promotion of machine superintelligence is extremely dangerous