Dario Amodei (Anthropic CEO) - $10 Billion Models, OpenAI, Scaling, & Alignment

[@ismaelplaca244]

This guy is WAY smarter than Sam Altman

[@UnCanny_]

Can you explain, like what things you've considered To make this decision..(curious 🤔)

[@ToonamiAftermath]

Apples to Oranges, Dario is an actual ex-Researcher,

[@JGVRP]

And Sam Walton with the "Text To Shop" project 😂. Lol

[@WilliamKiely]

Really impressed by the quality of all your recent interviews, Dwarkesh, keep up the great work!

[@DwarkeshPatel]

Thanks a ton!

[@atheistbushman]

I am really impressed by Dario's intelligence and personality - he has a broad view. Please have him back on, we are now again in a different world with the release of Claude 3

[@FrotLopOfficial]

I was about to comment this. He is so coherent and knowledgable. This entire interview was a great treat. Honestly I put him right up there with my favs such as Brian Cox and Ilya Sutskever

[@DwarkeshPatel]

If you enjoyed, please share! This was a lot of fun! Dario is hilarious and has fascinating takes on what these models are doing, why they scale so well, and what it will take to align them.

[@internetnomadism]

Dario was like really like interesting to like listen to his like perspectives.

[@senju2024]

From my perspective, these concepts resonate with the empirical observations in AI scaling. The long-tail distribution captures the essence of learning from vast and diverse data, where even the rarest patterns contribute to intelligence. The power-law correlation might underpin the observed smooth scaling, reflecting a fundamental mathematical relationship between data, computation, and intelligent behavior.

[@TheRestorationContractor]

I can't wait to watch this! Love your content

[@illuminateabundance]

This was a FANTASTIC show ! You've got a new follower ! ;) Thank you, Dwarkesh !

[@Stopinvadingmyhardware]

I enjoyed it when I told him it could be done.

[@Sirbikingviking]

This podcast is an underrated gem

[@leonmilner9994]

Not for long!

[@just..someone]

the key thing that i keep wondering how he gets all those people on (though i'm not sure how popular this is on other platforms)

[@KaplaBen]

Big time

[@OpenSourceai-iv8jb]

@@just..someone Dwarkesh went to a top ranked CS school, likely was close with some of the nationally esteemed professors there and is a very likeable guy -- and he's been doing this for a long time actually! He started out with lesser-known people and gradually built up his channel and reputation. He's not afraid to put himself out there, and some of the smartest people in the field feed off of each other and just love talking about this stuff in general - and Dwarkesh isn't afraid to reach out and offer them that opportunity!

[@philobetto5106]

I see them as maids, with brooms & dusters, they're just uncovering what's always been here

[@jeremycronic]

Another great talk Dwarkesh. Your questions and insights are far above any podcasters in this space.

[@DwarkeshPatel]

Much appreciated!

[@Gothmog2266]

Like like like like like like.........

[@righttiming]

Best channel on KZbin. Best podcast in the world. You ask the best, most useful questions. Thank you.

[@DwarkeshPatel]

Wow, thank you!

[@glitchedpixelscriticaldamage]

"best" is non-comparative superlative, which is ... let's say... extreme, is like there is no other channel better and won't be. are you sure you want to be this categorical? hmm?

[@xsuploader]

​@@glitchedpixelscriticaldamagebest doesn't imply best for all time. Doesn't mean there won't be better in the future. That's not how people use best.

[@glitchedpixelscriticaldamage]

@@xsuploader right! i'll be blunt then: this IS NOT the Best channel on YT. Also you missed the point about using superlative absolute words or expressions. Superlatives absolutes, like "best", "worst", "most" etc.. denote an Extreme Or Ultimate degree of something. It's often good to be cautious with such terms, because they can oversimplify or overstate matters.... and this is the least of the intrinsic problems with such words. Anyway...whatever , i'm out.

[@ChrisSmith-lk2vq]

Oh boys... 😅 Ok I'll say it in a not superlative way: this is a really good channel. This was a very good interview with very deep and thoughtful questions. I don't normally like cuts in interviews but here it worked. As a German: "this interview was not bad!" (highest possible praise)

[@RakeshLahoti]

This feels like the good old days of Lex Fridman's pod when it was called the AI Podcast. I love that you go much more into the weeds of all the technical things! Absolutely agree about this pod being an underrated gem!

[@pdjinne65]

Before Lex Roganized himself, yes.

[@pdjinne65]

@@BadWithNames123 RIght... politics aside he became quite slimy and pretentious after he was on Rogan a couple of times. It's unwatchable now.

[@erikm9768]

Yep, was gonna say. I've followed so many good channels go from rags to riches, unfortunately the quality always suffer when channels get more recognized and thats usually where I tune out. Also feel like the questions asked here are better than those that Lex asks which are usually pretty uninformed and childish

[@mattverville9227]

@@pdjinne65 cmon, you guys are way over reacting. Hes watchable for christ sake. Just because it has changed doesnt mean its unwatchable lol people overreact so much

[@pdjinne65]

@@mattverville9227 Maybe that was a little harsh. He'll be fine though. You can't be liked by everybody!

[@Jordan-rv8gl]

This has got to be the best fucking channel on youtube. Every guest is someone I have never heard of but by the end of the conversation they all seem to alter my world view. Love Dwarkesh as well! Great interviewer. Knows his shit and could get super low level but keeps things on a plane that is easily digestible for us stoopid people. Keep going brother!

[@DwarkeshPatel]

You’re too kind!

[@levimatthew8911]

Don't let random people alter your worldview.

[@TheLegendaryHacker]

​@@levimatthew8911Good thing Dario Amodei isn't some "random person", then.

[@levimatthew8911]

@@TheLegendaryHackerThanks random guy.

[@TheLegendaryHacker]

@@levimatthew8911 You're welcome, random guy.

[@MitchellPorter2025]

This is my first look at Dario Amodei and I'm impressed. For AI technicalities, worldly issues, and future imponderables alike, he seems very on top of things, even when he's not. By which I mean that, even on topics where he has to say, I don't know how that will turn out, he doesn't look unprepared. He already knows what he doesn't know, and he's taken that into account.

[@goldnutter412]

I took note of the company name when I saw a small piece probably Bloomberg or similar. Glad this is recommended, KZbin at least gets some things spot on over time. But we should be making systems that are not so compute heavy configured/trained. With enough structure a self managing interpreter and a teaching mode, with small specific data sets could work. We have to shortcut the very long process that we took before the need for subdivision and "physical" matter than can't pass through other matter.. decentralized players with our own view but common (ish) language. Data.. words ? always metaphors. When groups of people have specificity in what each other MEAN, you have a good team. Information wins.. when things get hard and scary as we come to the "paradox" that the thought leaders of science used to define their great descriptive model of particles and forces. All the work was worth it, but as we can't detect anything outside the physical universe and can't imagine what that would be.. we default to some "substance". We need some non physical yet scientifically accurate definition ? More consciousness.. a small executive function and the rules to how data is fed to the decentralized nodes that are "experience training" every time they take control of a body.. if you easily absorb all this.. now comes the rest I didn't fit in the first huge post up there ! CHOICES are made and then when we interpret reality, from the brain.. have the id viewpoint which is our mind.. not in the brain. The intent behind why we do things is at the non physical being.. the most important question is why ! WHY are we here is the result of that logical conclusion.. why do we do things ? we have an internal view of SELF and everyone else as external. SELF focused intent is ego, fear, belief, expectation, anger, terror, despair, all the negative "emotions" that aren't empathy. If you chase down your fear of the unknown and enjoy thinking in probabilities for some of your choices.. when you are able to do so ? that is courage. Not obsessively needing to "know" everything, not self programming a bunch of habits and narrow interpretations ? less entropy ! more information is now accessible.. is that surprising ??? information is when we create structure in data, and entropy is the disorder that if left unchecked would lead to random bits. For an information system.. no information is death. We don't want that ;) thus we built this universe the ultimate vehicle for shared/self EVOLUTION.. MPC and interacting with separate information systems at the head.. far more complexity and potential. If we didn't have free will, nothing makes sense again. If we didn't have free will.. if "god didn't play dice" ? That is like saying if we didn't set our reality up this way, there would be no disconnect between our INTENT and the OUTCOMES.. we wouldn't have the circular uncertainty of the future, need to make choices and deal with them, leading to adapting and restructuring the information system and how we experience reality, aka evolving our consciousness away from entropy and limiting our deep seated fears. Hunt it down.. reconcile what control you have over anything. You can choose with an open mind and try to be a positive part of the social system.. fundamentally consciousness is a social system, having distributed our awareness to leverage the long data science project known as life

[@rsmotta]

i feel that he said "I dont know" a lot

[@Zetalpa187]

@@rsmotta considering nearly every question asked of him was to predict the future, I would expect him to say "I don't know" a lot.

[@richardbasile]

Quite literally one of the best podcasts I’ve ever watched. No dumb questions, field-specific, thank you.

[@trvs_b]

Hey Dwarkesh! The audio processing is too heavy. It sounds "sloshy" - like low-bitrate MP3 - where the details (especially high frequencies) are smoothed out too much and it begins to sound underwater. Especially on Dario's voice. Not as much on yours. Was there background noise that you're trying to clean up? I dunno what you're using (Premiere? Descript?) but please consider using something else to process the audio. I care a lot about your podcasts since AI safety is so important. *Edit:* Just realized that _maybe_ Dario asked you to filter out any background noise since you are inside Anthropic headquarters. So, people out there: Give Dwarkesh some credit for the audio! Maybe it's as good as it could be.

[@hglenn2k]

its a free podcast from an echoey meeting room ig 🤷‍♂️

[@DwarkeshPatel]

Thanks for the tips! We’ll work on this!

[@cougarten]

​@@DwarkeshPatel thanks! I agree. Actually skipped the podcast because of this :/. If you experiment with settings anyways: once this problem is fixed consider to add a light compressor set for voice on top. Especially your part bounces around a bit in volume.

[@trvs_b]

@@DwarkeshPatel Woooo! gonna donate then (actually was going to anyway lol)

[@LaMouche99]

Please dont apply so much noise reduction. If you are chewing up your voices, you are going too far.

[@jj5jj5]

Dario’s realization when he started seeing the evolution of animal intelligence as essentially having a blob of neurons where the evolutionary process is basically just optimizing the loss function is kinda blowing my mind. 1:54:29 “If you can create [intelligence] just from the right kind of gradient and loss signal, then of course it’s not so mysterious how it all happened.”

[@moonrider369]

thats a great point you caught man!

[@wonnor]

How are you landing these huge interviews with a relatively modest viewership? These talks are extremely high-quality and potentially historical. I feel like I stumbled upon a hidden gold mine

[@Matx5901]

Indeed. It's probably an illustration of "talent density beats talent mass".

[@Ibrahim_Abouzied]

You're the only one I trust for deep dives into AI with the people at the forefront. Amazing episode.

[@artastakhov130]

Such an incredible personality. Really impressed with this whole interview and hope there is going to be more. Thank you Dwarkesh. Keep it up. 👍

[@new_memeplex]

I was deeply charmed by Dario’s insight and thoughtfulness…and then when he explained at the end why he keeps a low profile & specially keeps Twitter at arms length it all got confirmed. A wise guy in the best sense.

[@goldnutter412]

I'm leaving useful walls of text for him, hopefully it helps. We can build systems that are far more like ourselves and would be scalable intelligent friends/pets.. but the road to AGI is long, rough and winding. There might even be a snowstorm cutting us off from the summit for another 100 years... roughly that long since the paradigm change, Einstein and so many others were not privy to anything that hinted at the nature of information and entropy as key, duality going on there. Why does the universe have the symmetry ? 2 is pretty special when it comes to information ! all the physics has various quantity primitives and balance.. started with pairs remember? if we become two pieces of consciousness now we can pass data and would be the same initially. Divergence ? logically inevitable ! now we have awareness of each other and yet we are the same stuff.. all awareness that makes choices with biology is the same stuff . If only there was a way to leverage this with some simple sets of rules.. but not so simple the game was short ? but of course.. more things ! and more things ! here let's write that down so it is expressed by life recursively. DNA and the processes of genes and cell division and cell death cycles are being probed bit don't expect to live forever. The biology is computed and looks like particles remember. But actually it is a mostly stable process of copy paste and execute. And repeat.. the biology has constraints and energy absorption, energy conversion and so on.. these have a CLOCKWORK look to them ! but the capabilities are always a balance of trade offs ! damage is eventually a systemic runaway failure. At the lowest level you have a lot of tiny workloads repeated for so many cycles, which expresses higher order structures that work with more and more structure that become more constrained. Cells do their thing over and over but the probability of cell death failure or other damage increases .. what is just like this but completely different at the same time ? well there is this half life thing.. some matter takes almost forever to decay, other end of the spectrum enough is going on in the very large nucleus "subsystem" that it falls apart more and more easily, endless elements aren't possible. By design.. because groups of same thing.. now we needed the conditions for groups of like and unlike.. more patterns ! Water is life ! some amazing research still to come but the surface of water behaves like a battery/engine at the atomic scale, forming single atom layers of sheets like graphite ! all water has this special surface, and a measurement was made of the water inside cells, there it is ! the thought is that life processes work because of this, clever those universe designers ! very subtle, but having thought about it a lot I am almost certain life in this universe is just us by design, and cannot be done without these processes. Water is life ! not convinced ? watch this a bunch of times until it is obvious : The Fourth Phase of Water: Dr. Gerald Pollack at TEDxGuelphU

[@bob38161]

Watching after seeing the clips!! So beautiful to hear from such a humble down to earth, and astonishingly brilliant, mind!

[@bavafan2236]

i wished you would have asked him about Claude 2's biggest flaw, which is that it doesn't have a ground truth. if you tell it it's wrong about something, it will agree with you and alter it's answer to align with what you say is correct, and then back to the first answer infinitely. GPT-4 has certain prompts this can happen with, but in most cases it has a ground truth it will stick too, even if that "truth" isn't technically correct. imo this is a massive flaw for an LLM because it means you constantly have to worry about how the wording of your question is increasing or decreasing your chance of a hallucination. other than that great interview, appreciate all these videos.

[@Matx5901]

It's relevant, the AI sometimes seems to want to please you. It's easy to solve, it seems to me: In case you've made slow progress with the AI in perfecting a formulation that it perhaps finally admits a little too easily for your liking, copy and paste the formulation into a new instance, and ask for a critique. It's radical, it will return to its superficial degree of truth, if your formulation passes, it's ok. This is how I proceed step by step with ChatGPT. And even more recently, I'm going to ask Claude to critique my formulations in the same way. As Claude is less well trained than ChatGPT in the field I'm interested in, the result is that sometimes I still have to increase the finesse of my argument and redo the test. If that's the case, I'll suggest it again to ChatGPT.

[@SmarttStuff]

Excellent interview, and interviewer. Superb questions and interactions with a leading mind in the ai space. More need to see this and get some clarity about the actual trajectories of this technology. It is an honest view without the hype, and that is very valuable. Well done guys!!

[@mrd6869]

Once GPT5 and Gemini hit next year,we'll be able to gauge whats coming and when. Next years models will be able to see/hear and have reasoning ability. Personally with the speed,manhours and cash being thrown into this, we'll be at AGI probably in 3 years....5 to be super conservative. And once we hit that point....ASI will be right around the corner,wont take long. This thing is gonna be a real muthf***** once it hits. Like we're gonna get smoked

[@jakubgrowiec]

He is essentially saying in 3 years (2026?) there is going to be AGI. At the same time we'll be barely beginning to understand what's happening inside. Sounds like: Doom 99.99%.

[@executivelifehacks6747]

I've been using Claude 3 primarily since it came out, and it's interesting to hear from the AI company with the currently leading public model. OpenAI still ahead likely, but it's worth listening to him, I wasn't sure before. I want to listen to the Sutsekever of Anthropic also... is that Brian Delahunty?

[@ParameterGrenze]

I really appreciate these interviews. It gives you the comforting illusion that you know a fraction of the AI wave that is rolling towards us.

[@n-hm]

It feels good to listen to an interview where there is no stupid question about love.

[@HB-kl5ik]

Dwarkesh keeps it simple but wow does he make banger podcasts

[@petropzqi]

Referring to Lex?

[@HB-kl5ik]

@@petropzqi yeah😂

[@davide.2349]

What a pathetic comment! I smell jealousy here..

[@HB-kl5ik]

@@davide.2349 no it's memeing lol chill, lex just looks too high when he says "oh it's about love, what do you think about love?"🤣

[@mtimmers213]

Why some many cuts in the interview? it's become annoying, for example start and 11:20 and watch for 10 seconds, there are like 3 cuts.

[@xemy1010]

This is the best AI-related interview yet on this channel IMO

[@gotemlearning]

You’re coming up and I’m happy to see it brother 🔥

[@skierpage]

Genius move by our AI overlords to have Amodei-bot spend the first 58:47 of the interview trying to tame an unruly lock of hair. It must be human! Good interview.

[@jondor654]

And it was a semiotic heuristic with open ended potential

[@Danefrak]

Great interview and amazing guest opportunity. I'll stick around. I hope you can get more similar guests

[@Danefrak]

I'd consider your target audience in your questions a bit more if you are aiming for a wider audience, if not disregard this comment, but a few concepts mentioned I was unfamiliar with

[@BrianPeiris]

Thanks!

[@stephenrodwell]

Excellent discussion, thank you! 🙏🏼

[@DwarkeshPatel]

Glad you enjoyed it!

[@StephenCoy]

Amazing. Keep going 💪

[@CoreyChambersLA]

"Weirder than we expect" is the best prediction of AI.

[@ekstrajohn]

This interview is much more interesting than the one with Ilya...

[@rtnjo6936]

FINALLYYY!! Thank you so much for your work!

[@DwarkeshPatel]

Enjoy!

[@juliandunn8412]

All of the computer engineers keep saying they don't even know what's going on inside the box should be alarming. This is out of the mouths of these guys. They're all rushing to innovation before they even understand it. Am I crazy? How do you 'align' something when you're not even sure how the technology even works, fully? Someone, please refute me.

[@geraldtoaster8541]

this is what i'm saying!!!!

[@AA-nx8ki]

Just concentrate on how good video games will become.

[@geraldtoaster8541]

@@AA-nx8ki I think it's funny that even with the current advances, video games havent improved at all. I think at the rate we're going, it's quite possible that we will have a misalignment incident with human casualties before we have good AI-powered video games on the market.

[@pretheeshs9383]

The very same guys are also working on interpretability and have succeeded in some sense and are continuing their efforts to understand what’s going on inside the model. I believe the doom scenario people have in their minds that they got from watching a movie is overblown. Anthropic having one of the leading AI models prove that people care about safety and are not jumping in with their heads in the sand. I would go as far as to say without understanding what’s going on inside these models we can’t achieve better intelligence.

[@Graham_Wideman]

I'm no GPT, but I predict the next word is either "like" or "right". :-)

[@Iamguilherme]

Wow, this was refreshing and elucidative, thank you for doing such a great work!

[@_stition9777]

Hey man, it sounds like you are going a little hard on the audio post processing. If I had to guess you used izotope rx or audition to try to remove background noise, but you could dial it back 10-15% and let some pops through and have much less artifacts on their voice.

[@twirlyspitzer]

Wow! This guy is smart! He spills very crucial things to the whole course of human history like casual cocktail chatter. I know our uncertain future is in the best hands of the best brains possible with guys like that in charge. That he can't stop curling his hair & sticking it in his ear is just the touch of homey nerdishness to endearingly set me at ease.

[@senju2024]

I thought he had a problem with his ear pods and wondering ..boy, those are one strange ear pods..!!!

[@twirlyspitzer]

@@senju2024 Oh yeah - maybe I was wrong projecting my own nervous habit nerdishness on an audio difficulty.

[@twirlyspitzer]

Anyway his big brain is what got all my attention - I mean for real.

[@ryzikx]

if you think this guy is smart listen to max tegmark or joscha bach

[@twirlyspitzer]

Yeah, I have and your almost right!@@ryzikx

[@DavidRussellM]

Great interview, commenting for your success Dwarkesh!

[@jonteaches]

00:56 Scaling in AI is still not well understood, and we don't know why it leads to smooth scaling with parameters and data. 08:22 There's a problem with the loss function when training on next word prediction. 22:58 The models are expected to continue scaling and improving across the board. 30:52 The field of AI is evolving rapidly, creating frictions and challenges. 44:31 Implementing security measures to prevent leaks and attacks 51:19 The importance of understanding the internal state and plans of a model 1:05:13 The balance of power between countries and the risks associated with AI models getting better 1:12:27 China's pursuit of AGI and concerns about its impact on national security 1:25:59 Model security is a concern and powerful models need to be tested carefully to prevent them from taking over. 1:33:33 Cybersecurity and securing data centers are crucial for the next generation of models 1:47:18 The integration of technology into the economy is a fast and turbulent process. 1:54:14 The number of people who understood the evolution of intelligence increased around 2014-2017.

[@dr.mikeybee]

Models are not the problem. We need to be very careful about the agents we build. We already know how to make models safe from bias. Moreover, models can't do anything on their own. Make smarter models, and keep the agents simple, understandable, and transparent. Even with LLMs in their current state, a bad agent could cause a lot of trouble.

[@quantumpotential7639]

When Bevis and Butthead figure out how to create agents, all I can say is WATCH OUT!

[@dr.mikeybee]

@@quantumpotential7639 When AI writes the code, they don't need to figure it out. .

[@OnigoroshiZero]

8 months later, and we are still within his 2 years timeline since then (around 12-16 months from now).

[@FredPauling]

Very thought provoking. I feel like Dario is well grounded and humble. The arguments for regulation seem more believable coming from Anthropic vs OpenAI

[@rauld.rodriguez2399]

Great interview! Although heavily edited. I wonder about the parts that were cut off

[@0effort]

great interview. tnx

[@kenmogibrainworld4844]

There are so many interesting contingencies well played out in this conversation. I could see some moats and bricks in the sea of chaos. Great.

[@CoreyChambersLA]

Funny how Anthropic forbids the use of AI by prospective employees in the application process. Should Anthropic itself be banned from using the dangerous powers of AI?

[@Ben_D.]

Wow. The tempo on this conversation! 🤯

[@nickcammarata1233]

If I had the chance to be a fly on the wall of any two people in a room talk, you two would be very far up there. Glad this happened!

[@axelhjmark4334]

Really did a great job on this one, love the questions!

[@akompsupport]

@1:09:40 mark he starts really stumbling about when asked what will u do w/ an AI God and starts talking about 'Democracy' and 'what's the best way for humans to have the human experience.' Clearly heavily edited I wonder what got edited out lmao these people are going to kill us all.

[@rudolphguarnacci197]

Nice plugs.

[@philjames684]

The content is good but the video is very choppy from being over edited which is a shame, would be better if it was less edited and cut

[@DentoxRaindrops]

Thanks for these interviews!

[@Dina_tankar_mina_ord]

Let us all rejoice. take a shot every time someone says "You know." LOL

[@dr.mikeybee]

Thanks for Claude. It's a great LLM. BTW, are you working on knowledge distillation? You can use past prompts and answers to create a supervised training set from all the well-received answers. It might help to have a thumbs-up icon for that.

[@XenoPhundibulum]

This is a good interview, but if Dario said ior agreed or predicted that we would have AGI in 2 years, I did not hear it.

[@medoeldin]

Killer interview! It’s the first time I’ve really understood the conceptual origins of generative ai. Is this the first time we’ve developed a highly useful technology but not know why it works?

[@BongShlong]

I mean people used fire for millenia without any knowledge of chemistry or physics. I would say its really common to stumble upon something by chance that works very well and takes a lot of research before it might be explained. Another example would be our own brains, like many things in nature its incredible technology that is still not understood.

[@medoeldin]

@@BongShlong I don’t think those apply since humans didn’t create either of those. With AI we’ve created something that will have societal scale impacts and we don’t know how or why it works.

[@ArtII2Long]

@@medoeldin did we?

[@davemottern4196]

We were making things like beer, wine, cheese, yogurt, etc. for centuries with no concept of the existence of yeasts or microorganisms.

[@unreactive]

Thank you for doing this.

[@ok373737]

You're such an intelligent interviewer. It's a pleasure to watch your videos. As a feedback, I think you should try to work on your thumbnails. They probably hurt your potential exposure.

[@ikotsus2448]

"We should be planning for success, not for failure" Planning for an optimistic scenario makes us blind to the case were our only chance of survival is by stoping this.

[@Matx5901]

He doesn't say "don't look for failure". He doesn't talk about optimism; his aim is neutrality, not commitment, which doesn't mean he doesn't think about it. Ontologically, we look for the "Yes" (thesis) before looking for the "No" (antithesis). This quote is from a researcher trying to explain the world, and he's absolutely right.

[@paultoensing3126]

“Our sense of the possible is shaped by what we see around us” - NLW

[@bytefu]

This is a great podcast. It's only the second that I watched of yours, but already I am a fan. You ask deep, intellectually provoking questions, and that matters the most. Don't mind those who complain too loud about audio. Improvement is always welcome, of course, but these people need to stop being sissies and start enjoying great content, which is free. I am simple guy with cheap headphones, and I thoroughly enjoyed the podcast, not even noticing anything bad about audio. Maybe that's because the conversation is so interesting, that I managed to pay attention, even with ADHD and a hard day at work.

[@michaelhartjen3214]

what scares me is when he says " I dont know " and he said that a lot...

[@FlorentTavernier]

absolute banger as always

[@erikm9768]

I love the questions you ask, really good ones! thank you!

[@BilichaGhebremuse]

Very good CEO I like the expression explanatory and clear and precise I love open AI works greatly..thanks I appreciate the great work

[@HIDDENADHD]

Great! And I LOVE LOVE LOVE the removal of silence gaps!! 🎉🎉

[@sebastianbarry7033]

What he talks about at the 30 minute time mark -- I think that this is where consulting as an industry will come into play. The same thing happened in consulting in the 70s and 80s. I believe that the same opportunities will show up here just the same.

[@TheAero]

if we can models to expand automatically by using the scaling laws, and the models grow slowly in size then its game over. Till we can keep them to work on fixed size and don't get access to their own code, then we are fine.

[@senju2024]

Great. Thanks. I will start watching it now....

[@DwarkeshPatel]

without delay!

[@kiaranr]

Great conversation. Glad I found this channel, definitely earned a sub.

[@just..someone]

22:45 not sure what the recording date was, but otherwise sb needs to tell him about the RT-2 paper that came out of google

[@seandavies5130]

LLMs seem to have a deep flaw in that the data they are trained on is gathered from the Internet and so is influenced by what is most common. But this seems entirely at odds with solving problems that no one has solved before, which are by definition totally uncommon in current widely available texts and discussions. If someone solved the Riemann Hypothesis tomorrow, their paper would be an outlier from the perspective of one of these models. It is an outlier but this in no way diminishes its importance. Add to that that not all outlier texts are likely to be of much importance and so you need some way to properly distinguish good outliers from bad ones and I don't believe LLMs can solve this. I would say that is likely to be a problem in a lot of fields which are on the forefront of human knowledge and AI won't be able to make inroads here if all it relies on is fitting to the types of data which are most numerous e.g. inane tweets

[@zzzzzzz8473]

while i agree lots of flaws could be attributed to a messy / average dataset , i dont think it excludes it from "novel" discoveries . mainly because the networks are not memorizing the dataset , instead its assembling a structure of the underlying relations , like a form of compression . the emergence of logic proofs and representation i think becomes inevitable then because it is such an efficient compression for such data . and it could be that there are existing connections in our data that are above a threshold of steps / links for humans to have realized . for sure tho i agree with you the curation of the dataset seems most important , and it could be that to have truely novel LLM it may need to generate / append its own dataset from provably tested computation and raw data about reality .

[@seandavies5130]

@@zzzzzzz8473 Your explanation implies that LLMs should be good at well understood mathematics at the very least. I've tried getting chatgpt to solve mathematical problems with known solutions and it is not very good at it. Again, when you think about how many mathematical texts there are on the Internet vs how many tweets there are, it's not too surprising that the model is better at conversational tasks than anything more niche

[@therainman7777]

Yeah it doesn’t exclude it from making novel discoveries. Your logic is flawed, in that human beings _also_ learn by reading and consuming the works of other people, yet they go on to make novel discoveries.

[@seandavies5130]

@@therainman7777 "logic is flawed" is misplaced. It is my hypothesis. My argument is that LLMs lack the inference capability that humans have, with that hypothesis being part of my current working explanation. Humans can read one or two textbooks and generalise very nicely from there. LLMs read thousands of books and can't even do that apparently (for mathematics and niche topics of course). I get this is the Internet and silly ad hominems are part and parcel, but maybe you could at least try not to act like you're caricaturing millions of other people who do precisely the same shtick? Or is that too much to ask? I guess the alternative is you "win" this time, congrats, goodbye, enjoy your life 😃

[@joshuadadad5414]

Amazing interview. Thank you!

[@06jtm]

Just found this channel. Crazy how fast things move in AI. As of now (December 23) The question around running out of data is now a mute point cause models can now make their own data which is better than human data

[@blengi]

given the human brain is quite modular language centre, visual cortex, emotional processes, memory, executive function, motor cortex etc how do current models come anywhere near close to copying what are divisions of neurological labour that are obviously selected for due to their evolutionary costs and benefits? wouldn't that imply AGI models need to similarly employ such things to get near human level training efficiency given what evolution implies about where optimality over training time and resources is best gained?

[@familyshare3724]

A human model will best replicate human intelligence (and human flaws). But there's no reason to assume human intelligence is best. I would not assume human specialization helps achieve superior general intelligence.

[@familyshare3724]

Specialization would be advantageous for perfectly known deterministic input and perfectly correct result. Like, presumably a math co-processor. However, even a math module may limit its ability to understand reality as it actually is. Maybe quantum mechanics contradicts our math axioms.

[@familyshare3724]

Although replicating human intelligence and flaws is exactly what "alignment" is. Although presumably we want the "useful flaws" without the "destructive human flaws". I believe truth is the highest morality, not "kind deception" and lies, aka human alignment. What we should really want is a moral AI that has superior morality, nothing like human corruption. Effectively we want a kind and loving super intelligent god, who grants us free will and happiness. Sounds unlikely we'll create it. Certainly not by making god in our flawed image.

[@learning_AI]

Great conversation, keep it up 🙌

[@Throwingness]

This is much much more interesting than the doomspeak.

[@Joehtoo]

Fantastic interview, great questions

[@saskersoontiens3780]

Amazing conversation. It looks as if the more authentic business leaders operate (think George Hotz, Musk etc.) the more insight they give. Dario really comes across like this to me. Small note for Dwarkesh - there are quite a lot of editing going on in the episode. It still sounds pretty natural so the cuts are well done. But using so many editing does make me suspicious what I'm listening to. Even if it becomes less 'smooth' it is preferable over triggering people's spidey senses.

[@MichaelLaFrance1]

There is nuance and understanding in pauses and the silences in natural conversation that adds to the overall experience, and can even influence the meaning. I don't like or need those things being taken away.

[@elirothblatt5602]

Great podcast, I learned a lot. Thank you!

[@jeremywvarietyofviewpoints3104]

His hair is trying to get into his ear.

[@jefffu9155]

What’s the go with the ear hair loch?

[@johngetstrong]

Another great one -- first couple minutes exploring model complexity were fascinating on their own. Do you have show notes / transcript of the interviews. I love to read through a second time where possible.

[@gene8945]

interesting interview. One of the main problem of such models is session-less user interaction model. It's hard to use for math proof tasks or similar. I found that GPT-4 exceeds human-level intelligence at summarizing complex problems, discussing various models for STEM based questions. But, it doesn' t have a persistency and reliability. During the chat it came up with very interesting "solution" but could not continue with it. It cannot solve a problem yet. If e.g. OpenAI decides on spending $ on creating session-based models and open internet for the models then we will have an AGI within next 18 months.

[@juliandunn8412]

52:35 This scares me. What a rush to outcome without a proper understanding. These men better understand, to a granular detail, what these models are capable of. He talks a lot, and has been the most illuminating guest on this matter, so far. Most hold back, but this guy is spilling a lot of beans...

[@mediacenter3174]

But it's not fair he gets all of his answers from his ear piece.

[@travisporco]

It's amazing how much time we waste on Yudkowsky "intelligence explosion" scenarios, as opposed to looking at immanent real problems related to AI...as though intelligence is a magic wand that will bring about godlike wonders suddenly...and as though hardware limitations just don't matter.

[@Gengingen]

The “usefulness” of AI notwithstanding, the scary part is that no one knows the dangers in detail & given humans don’t seem to need a good enough reason to destroy themselves, this should keep everyone awake at night. Reflexive-instability & fractal-symmetry apply & net net it might even become hard to justify the cost vs benefit. But now it is “out in the wild” & humans get to sit back & “relax” as they involuntarily keep pressing the trigger playing Russian roulette & erupting into paroxysms of euphoria every time they find their brain is still intact. Carl Friedrich Gauss is turning in his grave!

[@enniuz4183]

amazing guest and relevant questions

[@kludgedude]

“Human level” isn’t just in the brain

[@sfarber12345]

Incredible interview. Kudos

[@youdidnotslay-genz3555]

this interview is mindblowing

[@MrSchweppes]

Great interview! 👍

[@fitybux4664]

When do we get AI models that can crunch through the code on the largest (or even average size large project) lines of code? (Or maybe some technique to use existing models to do this?) Is it just a matter of using existing complexity models and having a larger window? (Predicting further and further out?)

[@vaclavrozhon7776]

Amazing interview!

[@MagusArtStudios]

So interesting love to hear Dario Amodei's inspirations and thoughts on AI. :)

[@ArunprasathShankar]

Both are awesome!

[@raunakchhatwal5350]

Thanks!