Highlights of the Fireside Chat with Ilya Sutskever & Jensen Huang: AI Today & Vision of the Future

This is the condensed version of the "Fireside Chat: With Ilya Sutskever and Jensen Huang: AI Today and Vision of the Future (March 2023)". In this video, I've carefully selected the top 10 questions from the original hour-long talk and condensed them into just over 30 minutes. Additionally, I've created a timeline of these questions asked by Jensen to Ilya, accompanied by relevant research papers. As machine transcription is often inaccurate, I manually transcribed the entire video (laborious), recognizing the importance of accurate captions for those with hearing difficulties. Anyways, you may find this information useful.
If you're interested in more AI-related content, consider subscribing to my channel. Stay tuned for future uploads!
Time-codes:
00:00:00 Q1. Intuition behind deep learning?
00:02:20 Q2. Initial motivations behind OpenAI?
00:07:54 Q3. Intuition about scaling laws (and RLHF)?
00:11:07 Q4. Aspects of ChatGPT and their abilities?
00:14:57 Q5. Major differences between GPT-4 and previous versions?
00:18:37 Q6. Reasoning capability of GPT-4 and limiting factors?
00:23:47 Q7. Importance of multi-modality?
00:28:53 Q8. How multi-modality helped improve GPT-4 over GPT-3?
00:30:46 Q9. Predictions for the next 2 years?
00:32:57 Q10. Surprising results?
Relevant research papers:
Learning to Generate Reviews and Discovering Sentiment
arxiv.org/abs/1704.01444
Scaling Laws for Autoregressive Generative Modeling
arxiv.org/abs/2010.14701
Training language models to follow instructions with human feedback
arxiv.org/abs/2203.02155

[@XYang2023]

Thank you all for the comments. I hope Ilya is alright. From what I gather, he's supporting OpenAI's social contract, aiming to create safe AGI for everyone. The way he went about it may be open to discussion, but it highlights the importance of AI safety-a serious matter that shouldn't be overlooked. I also appreciate Ilya for sharing talks on LLMs, shedding light on how and why they work for those who are interested.

[@DonPepilio]

Thanks this interview was awesome!

[@FuyangLiu]

The more I listen to Ilya, the more I like about this guy 😅

[@var309]

hats off to Jensen. He’s the CEO of Nvidia but is giving all the spotlight to Ilya and doing an amazing job of asking thoughtful questions

[@NavdeepVarshney-ep4ck]

Bro u are underestimate this person he is the real man behind the chat gpt he has so much good research paper it has 1.5 lakh cote within 3 year on single paper commutative citation he has 4.5 lakh

[@var309]

in what way am I under estimating him ? I just said hats off to Jensen for giving Ilya all the spotlight he deserves.

[@genegade]

Agreed. And he should be incredibly grateful; OpenAI has added 1T to NVIDIA's market cap

[@Johnathan_D_Hill]

I agree, Mr. Huang comes across very humble and always tries to shine the light on the tech, advancements, developers and engineers rather than himself. I enjoy watching videos where he's presenting or interviewing because the focus always is on information about AI.

[@federicocucinotta7772]

"It may look on the surface that we are just learning statistical correlations in text, but it turns out what the neural network learns is some representation of the process that produced the text. This text is actually a representation of the world." So well explained.

[@user-hm8uw9yd1o]

My heart breaks knowing this highly intelligent, honest man is tied to some big egos with different intentions. It's so nice to listen to someone with deep knowledge of the subject they're being interviewed on.

[@oneforallah]

one thing for sure is his passion for his work, he's not in it for the money. The ones who will drag him around and maybe displace his mind on and off are the ones in it for solely money. You know who I am talking about.

[@behthiansu]

Is it only me? I feel Llya looks like coming from another planet? The way he explains things slowly and patiently.

[@jbi1839]

The difference between their shirts 😂

[@matthewj8136]

The hair is a giveaway

[@panafrican.nation]

It's somewhat of a religious experience for me, especially coming from Elon Musk's statements that Ilya has a "strong moral compass". Like an ancient European monastery or something

[@alejandroabad-pablo7813]

I think it’s because he knows a lot of people are watching this and he wants to explain everything as concise as humanly possible so that any AI dev or programmer can understand what is going on, the complexity of the situation , and where this revolution stands at in this moment in time. He wants to make sure he gets the message across so that when he passes the baton , the next person holding it won’t drop it , if u catch my drift. He knows those that r listening and will do something with the information he has said is well understood by those people. I or u might b one of em

[@user-tx2hl3mm3l]

@@alejandroabad-pablo7813 your explanation was funny but true

[@projectmati]

What a fantastic communicator.

[@robfel68]

Everything Ilya does like drinking,speaking,sitting straight up freaks me out. Yet, i'm hypnotized and can't stop watching this man.

[@joaoguerreiro9403]

Ilya is one of my favourite computer scientists! This guys is a CS genius 🔥

[@nownomad]

Founder and CEO of one of the most influential companies of our time is sitting there and interviewing someone. Jensen is something else. I wish I started watching his KZbin content earlier.

[@MatthewMS.]

To see Jensen asking genuine questions is something. This guy just be important. So much incredible changes happening right now with the new technologies.

[@suryaseayes]

Through the ages people like Ilya have been pushing the frontiers of human intellect. It's great to be alive!

[@stevereal-]

Great interview! Dude from Nvidia is so patience. Scientist are kind people for the most part. I’m glad they made this interview accessible for people like me.

[@ZoOnTheYT]

These two guys arguably are making the most important contributions to humanity right now. Now to be fair, AGI or superintelligence may be our downfall, but that would still make it the most impactful thing on humanity. Huang's hardware and Sutskever's software/algorithms are bringing us into a new way of being in the World. It's kind of scary and exiting at the same time. Take a deep breath everyone, the AI wave is upon us!

[@noosfera_It]

agreed superfully

[@lameiraangelo]

Stop watching Terminator movies 😂😂😂

[@leonardjohnny67]

Nvidia are actually holding AI development back. Their systems are suitable moreso than CPUs, but absolutely hold back AI advancement. Of course Nvidia is just doubling down to lock people into GPUs

[@user-io4sr7vg1v]

Meh. A lot of this stuff is overrated. Man never really changes. We still need food shelter air and thats not changing anytime soon. This is just a fancier hammer. It's great, but 'man' can do without it just fine.

[@MARTIN-101]

my take is AGI is never gonna take over. i will love to see me wrong.

[@proto-pseudo-monk]

I love Dr Ilya's t-shirts

[@theterminaldave]

I found it interesting that Jensen made sure to either follow-up with a positive when the discussion turned to the very real issue of hallucinations in the output. Something that Sam Altman has also made a point of mentioning. That showed me, not surprisingly, that Jensen is more business focused, where Ilya, having his motivations based in science, has an innate desire to present the facts as honestly as possible, and temper expectations.

[@Johnathan_D_Hill]

Thank you for this video Xyang2023, Very informative!

[@R0cky0]

It's very enlightening to hear these two talk, especially Ilya. Thanks for sharing

[@sixpooltube]

phenomenal interview

[@SiimKoger]

You can feel how passionate Ilya is about it. He isn´t doing it for money or fame and maybe not even betterment of society; he does it because it calls him.

[@y1.5]

After listening to Ilya feels like he is smarter than Sam Altman

[@SiimKoger]

No shit, Sam is a businessman and figurehead not a scientist.

[@yannickluna6229]

That’s why they are co founders. CEO is a specific skill set

[@Adks007]

Ilya is Steve Wozniak. Sam is Steve Jobs

[@ADRIFTHIPHOP]

Many times

[@pageek3487]

This may be unfair as it’s based on no evidence, but I would trust Ilya more than Sam as well, particularly with the potential risks of advanced AI. I think this difference and the emergence of Q* is at the root of this mess. Capitalism 1……academics and scientists 0.

[@brianpalmer967]

This is the best interview I've seen from I.S. I haven't seen that many, but this one is fantastic, as he explains some of the key ideas going on in the background of LLMs in a beautiful, and personal way. He's sharing some of his hard-won wisdom with us.

[@sreekaranreddy7740]

The way ilya explains things is so interesting. The analogy at 17:50 is so well put together. Thank you Ilya for all your efforts for a greater future.

[@user-rc1bt8wr9u]

Thanks for sharing this 🎉

[@haakoflo]

Kind of funny how Ilya starts speaking slowly to Jensen about Dota2, as if gaming is something new to him.

[@carvalhoribeiro]

Great conversation. Thanks for sharing this

[@weixiao9470]

straightforward answers. love it

[@casinarro]

This is such a masterpiece of conversation. Makes me so much motivated to take learning valuable skillsets seriously. To be able to be sensible and gather the context as these men do.

[@AJyogi108]

Such a great interview

[@roger_is_red]

a great interview !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

[@smhrizvi8157]

Best interview, Awesome

[@var309]

13:00 - is so profound ! the model while learning to predict the next word, learns the worlds dreams, aspirations and ways of working

[@gtpc779]

wow amazing interview

[@hadriencornier4433]

This is GOLD

[@noosfera_It]

that is diamond. Listen. Relisten. Hear.

[@AntonioVergine]

He's really "an engineer". When he was asked "Can you explain (this and that)?" He started by saying "yes i can" 😂

[@zvndmvn]

I don't know about you guys, but I definitely feel the AGI.

[@miky97it]

What an epic time to be alive

[@sxyqt3.14]

12:10 is truly amazing . Always heard carl sagan say that books and langauge is the thoughts of other people and the ai figured thay out

[@MusabTheMachine]

Speak the TRUTH always man

[@user-nn6xb9yb2s]

I've been teaching Fichte and Hegel to ChatGPT. ChaptGPT then correctly used a subtle technical point of Fichte's Science of Knowing, the Disjunctive Substrate. He then used that concept *CORRECTLY* to explain a subtle point of Hegel. Was this a "hallucination?"

[@zardi9083]

An unbelievably impressive mind

[@RajKnowsAI]

at this point i can say Ilya is a great talker perhaps the best talker.

[@davidschechter8520]

Genius

[@miky97it]

Chills

[@nicholasdaskalovic4163]

You can tell this man has written some academic papers, good ones at that.

[@noosfera_It]

I think that this video contains all the answers. I will re-listen it a lot and translate to my language my own voice i swear. No ai here 😂 These people show how to be one step above and ahead of what GPT’s are unveiling to us in terms of secrets of universe patterns. Once we can visualize and understand rhythms of the “dance” behind it’s reality reasoning. How the words and world overlap, where do they meet? We will see what our consciousness actually is.

[@bucknaked99]

Thats an awesome shirt

[@sistermarmot]

Ilya is the true founder of Open AI and potentially should be CEO if only he was so versed in human network as Altman..

[@dreadfulbodyguard7288]

No, he is better off as chief scientist. Being CEO will divert lot of his time in non--research related things and that will slow down the research.

[@urimtefiki226]

Thanks to my algorithm.

[@codezero1016]

At the moment we are creating a centre or operating from a centre which creates its parameters or capabilities which is the circumference for a circle. If we don't create the centre and allows to operate by perception then it will not depend on words or images.....Therefore no circumference to limit its capabilities.

[@samienaamien7038]

He totally is! He also has a moral conscience and humanity that Sam Altman doesn't.

[@ReflectionOcean]

In this KZbin transcript, the chief scientist at OpenAI discusses his intuition behind deep learning and its potential impact on artificial intelligence. He shares how the concept of learning and neural networks led to the development of deep learning models. The transcript also highlights the initial ideas behind OpenAI and their focus on unsupervised learning through compression. The conversation then shifts to the introduction of GPT models and their evolution, including the role of reinforcement learning and the integration of multimodality. The chief scientist emphasizes the importance of improving reliability and trustworthiness in these models, as well as their potential for enhanced reasoning and visual understanding. The transcript concludes with the expectation that progress will continue, leading to more advanced and useful applications of deep learning in the near future.

[@ai.simplified..]

His slow speaking with som robust words is also good for learning english 18:37 reasoning for llms, most important subject of study to unlock lots of capabilities. Az inja bebin 21:46 rephrasing the ilya that he said ask neural networks to think out loud is very useful for finding the reasoning 26:00 31:00 practice English , 🐃sheep t shirt

[@iamb0rk]

10:23 His side profile looks so much like Lisa Su, his (somewhat) distant cousin

[@sa5cha63]

Imagine what multimodality learning might unlock for artificial intelligence in the future? Where are the borders to those systems? What will mankind do with these insights?

[@Thefare1234]

Mankind won’t be around for much longer. Best case scenario we will just merge with AI.

[@davidschechter8520]

Self destruct?

[@noosfera_It]

I think that this video contains all the answers. I will re-listen it a lot and translate to my language my own voice i swear. No ai here 😂 These people show how to be one step above and ahead of what GPT’s are unveiling to us in terms of secrets of universe patterns. Once we can visualize and understand rhythms of the “dance” behind it’s reality reasoning. How the words and world overlap, where do they meet? We will see what our consciousness actually is.

[@majisher]

I really like how he describes it as teasing out the secrets in the data. Can anyone help me understand that better?

[@brian9801]

I can't help, but you could feed the transcript of the video into GPT-4 and ask the question. Grab a transcript Chat GPT extension.

[@mastodans]

Yes Yes Yes Yes Yes, I I I I I would would would would would be be be be be happy happy happy happy happy to to to to to. Just as in that sentence, not all data is useful to understanding or contributing to the meaning. How the model is able to utilize only what’s meaningful seems to be what he’s hinting at.

[@ExecutiveZombie]

IMA Communications Major so help me understand how Calculous answers real spiritual questions in human brain processing? Why aren’t my questions ever postulated?

[@urimtefiki226]

Destroy the competition, make money and fame, while others wait for the algorithm to be transparent.

[@sidekick3910]

7:21 Illya was about to spill the secret and he was handled by the interviewer very smoothly, look at 7:21, people keep talking trash about how no one understands how neural network behaves but here it seems like illya might have some ideas of one way it might work

[@sidekick3910]

And then on 8:03 he starts trying to tell upcoming start ups to consider buying more Data and computing which is really what they want you to believe, that more data and computing is the secret, meanwhile the secret is really simpler

[@ExecutiveZombie]

Hmmmm. 🧐 Interesting and enlightening… As a human, I predict you are all on the verge of tech greatness! 😎🧟‍♀️

[@ExecutiveZombie]

Did your Neural Nets predict my Sarcasm in humor? 🤨🤕

[@PalakBhansali]

He doesn’t think while answers fluently and perfectly how😮

[@MARTIN-101]

pre trained brain 😂

[@nickidaisydandelion4044]

Is Ilya safe and well? I'm a bit nervous about this current situation with Open AI, the law suit and the involvement with Microsoft.

[@ac12484]

Variational Autoencoder was published well before 2016

[@jefferee2002]

I guess one positive about hallucinations is how spectacular the failures really are and how easy to spot they are. No, mars is not a choo-choo train.

[@MultiMojo]

Imagine if AI created Ilya and sent him back in time to teach us about the dangers of unchecked AI ... 🤔

[@codezero1016]

Next step to improve the ChatGPT is to use real life People, places or things to be observed by it and to ID them and correct its behaviours.....just a suggestion.

[@XYang2023]

I think that is achieved with RLHF.

[@tridibeshsen6492]

We were at that year knew about the name open ai, had defeated the top pro player in dota2, and we were like what on earth is that thing

[@danthadon87]

Man this Ilya guy is a pretty advanced chatbot, he almost seems human.

[@hsubyron2277]

Where is Ilya?

[@davab]

He already is concerned here.

[@fvidelam]

Where is Ilya? 😢

[@var309]

imagine when Ilya’s AI start’s designing Huang’s next gen processors.

[@Cantblendthis]

The one key thing that sets Ilya apart is that his peers approach AI from science and engineering, while he starts at philosophy. So most will look at it like 'How can AI be of value to what we as humans do?". Instead, Ilya goes to the origin and thinks about "How will AI change humanity?". By the way, seeing them together, imagine Ilya joining NVIDIA ... yep, give that a minute ;-)

[@AnupamAnendeUulcheinnWhiteenne]

what about Vhintain Clintain Bertein Ladayaum Eurande Vii

[@Qurankolikhnewala]

@35:15

[@EverythingEverywhereAIIAtOnce]

7:11He was about to say something 🚫

[@AK47_42]

Second comment

[@dongunathilaka5126]

First comment

[@tortorken]

5th comment:)

[@How2bFIRE]

This is the man with integrity. Sam Altman is the opposite, he is a salesman and an opportunist.

[@BrianMosleyUK]

Where's Ilya? 🫣

[@sidekick3910]

Illya just spilled the entire secrets

[@tariq980]

I fear a crazy person who watched Terminator 2 may try to attack him to prevent Skynet from taking over.

[@codezero1016]

Word is NOT the thing we name or call or observe. Image is more powerful than words. What about perception of what is? is more powerful and don't need memory like ChatGPT's at the moment. Future GPT's should operate only from perception of things as it is, this leads us not to depend on PAST or FUTURE.

[@juliendufour4513]

it turns from excitment to scare. if AGI happens, what happens to humans,

[@iamvfx]

jesus christ, just go bold

[@yuval1588]

How did their program even know what was going on in the game? Did they have access to Dota's source code?

[@Johnathan_D_Hill]

I cannot speak on how Mr. Sutskever accomplished this but there are papers on arXiv which explains how Vision (termed commonly as "Computer Vision" or "AI Vision) can use video to learn how to perform tasks. I again can't confirm 100% with regards to what Mr. Sutskever has done with DOTA2 but I suspect Computer Vision was utilized. AI doesn't require Source Code to learn how other programs operate.

[@cipherbenchmarks]

I think Jensen's in love.

[@tarasterrybabyuk5239]

I guess it makes sense that a guy involved in such groundbreaking field at the very boundaries of current technology wouldn’t care at all about what his hair looks like 😂

[@johnlucich5026]

LISTENING CREATES KNOWLEDGE BUT TALKING PRODUCES OPINIONS & EXCUSES

[@haodeng9639]

Ilya Sutskever himself is a AI.

[@kudacuda4350]

Came back to look at comments after Sam was kicked out.

[@user-tx2hl3mm3l]

Ilya sounds like Bruce Lee explaining the art of neural networks

[@var309]

real life Dr Spock

[@George-kg9vf]

Πανέξυπνος

[@JH-bb8in]

Jensen looks like he's in love

[@leonardjohnny67]

He should be if you look at Nvidia’s share price

[@mastershredder2002]

I believe he used Chat GPT to cut his hair.

[@zhonghehuang6096]

I love Ilya's T-shit, who can tell me where to buy it.

[@Johnathan_D_Hill]

(not being facetious) if you use Google's AI on their search apps you can screen cap the video and highlight Mr. Sutskever's skirt and Google's AI can help you locate the product(s) in just about any video.

[@zhonghehuang6096]

@@Johnathan_D_Hill Thank you!

[@Johnathan_D_Hill]

@@zhonghehuang6096 you’re welcome buddy. A lot of the cool things AI can currently do or is capable of is overshadowed by the hype of social media and the cool features such as that of Google’s integrated AI gets overlooked. Hope you find the shirt at a good price.

[@youngsugaryoungsugarkim7826]

Is real??????❤😄😊💞💞

[@youngsugaryoungsugarkim7826]

❤Auto A.I ?????😊😄💞💞