Francois Chollet - LLMs won’t lead to AGI - $1,000,000 Prize to find true solution

[@SiphoNgwenya]

Thank goodness for Francois' infinite patience

[@therobotocracy]

What I learned from this: Be the guy who speaks slower under pressure, not the guy who talks faster!

[@kjoseph7766]

what pressure

[@therobotocracy]

@@kjoseph7766 They were clearly putting their own opinions forward, which didn’t align. I would say that put pressure on each other to defend their beliefs. The one dude got super panicked as the French guy kept putting forward clearly articulated ideas and rebuttals.

[@ahpacific]

After listening to this interview and reflecting on it, I was actually thinking that (not Dwarkesh because he's legitimately smart in my opinion) but "people" like Destiny talk extremely fast to compensate for shallow ill-formed thoughts.

[@Gome.o]

@@ahpacific Dwarkesh could help himself out a lot if he slowed down. Simple things like rephrasing what 'french guy' was saying in a way that Francois would agree with would also help tremendously. There is a fundamental difference in epistemology between these two, Francois is emphasising true understanding, and Dwarkesh seems to imply that large gross memorisation leads to understanding- which I don't think Francois would agree with

[@wiczus6102]

​@@ahpacific I am pretty sure Destiny is very transparent on when his thoughts are shallow or not. Notice when he formulates things as open questions and or the adjectives and intonations he uses to suggest the difference between perspective and fact. You can make a false statement and still create a good converstation like that. People like that are fun to talk to as opposed to people who will only say something if they are fully certain.

[@MM-uv3vb]

Need more interviews with legit AI/AGI skeptics to balance out the channel

[@mbican]

There is money in that

[@benbork9835]

Skeptics? You mean realists

[@danypell2517]

nah fck the doomers

[@ModernCentrist]

The issue is, highly intelligent AI skeptics are in short supply.

[@EdFormer]

​@@ModernCentristthat's the impression you get from content focused on hype merchants, alarmists, and those with a vested interest in overinflating the value of current methods. Academia is full of skeptics/realists.

[@perer005]

Dawarkech does not appear to have the ability to adapt on the fly in this interview! 😂

[@vsanden]

He showed by example that some humans are not AGI

[@slm6873]

One of the worst and least informed interviewers, the only way I can get through his videos is fast forwarding through his incessant low information babbling. Yet he keeps getting very smart people on to interview!

[@mm100latests5]

@@slm6873 dudes like this are good for pumping up the stock price

[@skoto8219]

@@slm6873can you tell us what the dumbest question was and why it was dumb?

[@IndiaEdition]

I was gonna like this comment, until I watched the whole talk. Guys watch the whole talk before forming an opinion! really interesting. @dwarkesh - great going as usual

[@cj-ip3zh]

Dwarkesh is LLM, Francois is AGI

[@ClaudioMartella]

perfectly said

[@cj-ip3zh]

@@ClaudioMartella I may have commented too quickly, it seems thar later on in the video I got the impression Dwarkesh was playing devils advocate. Not sure...

[@wiczus6102]

@@cj-ip3zh By impression you mean that he literally said he is playing the devils advocate?

[@cj-ip3zh]

@@wiczus6102 Right but I couldn't tell if he meant just in that shorter exchange or if the whole interview was him taking the opposite site for the debate.

[@XShollaj]

Damn - perfectly put!

[@cmfrtblynmb02]

One advice to host: You need to give your guest space. You are not a salesman. Or a missionary. Challenging them does not mean repeating the same argument over and over again. It was suffocating to listen to your challenges. If it was not for the call and patient demeanor of Chollet, it would be impossible to watch. We were not able to listen to Chollet expanding upon his ideas because host just reversed the clock to zero by repeating the same "but memorization is intelligence" argument. It should be about your host, not showing the supremacy of your ideology or beliefs. If your host is wrong, you can prove them wrong by showing their arguments and ask questions as they expand upon them and then show if they are inconsistent. Not repeating the same thing over and over and over again

[@mmelanoma]

this. on the other hand, it gave me tons of reaction images hahaha, some of francois' sighs and nods are just gold

[@dontwannabefound]

💯

[@Limitless1717]

One advice to you: You can make this point better without insulting Dwarhesh, who is a young man that is still learning. Perhaps you should try hosting a podcast and see if you do better. Want my guess? You would do much worse than you think.

[@aga1nstall0dds]

​@@Limitless1717ya ur right lol... he just a lil bit slow and no offence to him it just demonstrate how francois is a genius

[@cmfrtblynmb02]

@@Limitless1717 this is the most mundane criticism of a criticism. Dwarhesh himself is doing a podcast on topics he is not a specialist on and he is openly criticizing and challenging the views of a specialist on a topic here. So maybe he should work on AGI before challenging François here if he were to take your advice seriously (though he should try to educate himself on topics in any case) And I am not doing podcasts but I have taught many many classes with a lot of teaching awards. Not the same but similar when it comes to expanding on topics. When I teach a concept I don't just attack it on the first sentence. I explain it, allow it to ripe. Put different light on different aspects of the topic. I don't try to destroy the whole concept on the first sentence. So my advice doesn't come out of nowhere. And if he puts himself to public spotlight, my criticism is actually one of the most innocent stuff that he is thrown in his direction. If he takes into account he can improve upon. I am mostly criticizing how he is doing some stuff and even provide what he can do better. It is weird that you take offense to that. Anyways it is up to him to do what he wants but I won't watch him anytime sooner again. As it is now, this is really bad way of arguing with anyone even in private, let alone on a podcast. When someone interrupt me like he does and don't know how to argue, in general I just don't bother

[@driziiD]

Francois doesn't even sound like a skeptic, just an informed educator

[@CortezBumf]

he invented Keras- he knows his stuff

[@randomuser5237]

@@CortezBumf Now I see why there are so many here simping for a fraud like Chollet. Basically these people read "Deep learning with python" and thought Chollet was the frontier person in AI. It's hilarious and ironic. Chollet has made no contribution to frontier AI. He's nowhere near Ilya, Schulman, Hassabis and others who've been interviewed by Dwarkesh. He's just parroting LeCun's viewpoint mixing in his own ideas about general intelligence that are completely unverified.

[@sigmapiepsilon]

Now I see why there are so many here simping for a fraud like Chollet. Basically these people read "Deep learning with python" and thought Chollet was the frontier person in AI. It's hilarious and ironic. Chollet has made no contribution to frontier AI. He's nowhere near Ilya, Schulman, Hassabis and others who've been interviewed by Dwarkesh. He's just parroting LeCun's viewpoint mixing in his own ideas about general intelligence that are completely unverified.

[@randomuser5237]

@@CortezBumf You do know what Keras is, right? It's a frontend to the libraries like PyTorch, Theano or Tensorflow that do the actual heavy lifting. It's basically some syntactic sugar for the masses who couldn't use the somewhat more complex libraries. Now that their interface is simplified Keras is redundant.

[@CortezBumf]

@@randomuser5237 lmao always appreciate a redditor-ass response 🤓☝️

[@ayandas8299]

Francois Chollet is an amazing guy. The best thing is he, like all the LLM guys, also wants to work toward AGI! He just doesn't think the current LLM paradigm will get us there. I'm really excited to see where this goes because he's challenging the current option space in exactly the right way

[@mythiq_]

This interview actually got me believing LLMs might indeed get us there. The guest seems to believe in a form of intelligence that he idolizes but we haven't really seen. Dwarkesh was spot on that no scientist zero-shots their ideas.

[@JimStanfield-zo2pz]

Chollet is actually wrong though. The LLM guys are right. Experience is enough

[@KP-fy5bf]

Him and the others trying to achieve AGI is exactly what will get us eradicated. I agree with Chollet but at some point brute force memorization is all you need to solve any task, you just need to refit the training data.

[@cmfrtblynmb02]

​@@JimStanfield-zo2pz If experience and memorization is enough for AGI, how did people create things they have not seen before? How did Mozart create his music? How did we create skyscrapers? How did we go to moon? How did we discover relativity and quantum physics? Only someone who lived his life like parrot and not create or even attempt to create anything will say this

[@cmfrtblynmb02]

@@KP-fy5bf That's a fair point actually. If one claims LLMs and that approach is sufficient for solving our issues, they just need more development, I might agree. But once people seriously think LLMs are intelligence, it is a different story.

[@TheMrPopper69]

Dwarkesh dismissed a few completely valid answers to try and steer the answer in his preconceived idea of LLMs, I didn’t like that, dude is smart, let him finish and actually take onboard his answer before asking another question

[@therainman7777]

He said he was playing devil’s advocate, calm down. He does that with most of his guests. It generally makes for a more informative and engaging interview than simply taking everything your interview subject says at face value.

[@BrianPeiris]

@@therainman7777 Dwarkesh said himself that they were going in circles. I think this was mostly due to Dwarkesh not really thinking about Chollet's responses in the moment. LLM hype causes brain rot in smart people too.

[@VACatholic]

​@@BrianPeiriswas like an llm was interviewing chollet

[@davidlepold]

It was like llm vs analytical intelligence

[@diracspinors]

It is pretty clear Dwarkesh has a lot of influence from the local SF AI folks. Watch his interview with Leopold. His command of the subject is admirable, and he smartly relies on the researchers in his circle to inform his understanding. Many notable people maintain quite strongly it's only a matter of scaling, and I thought he thoroughly went through these types of arguments. It was a valuable thing to do. What is an example of something Francois wasn't able to eventually effectively articulate because he was cut off?

[@aidanshaw3611]

This is by far the best interview until now. We need to hear the skeptics too, not only the super optimists. I really like the french guy.

[@antonystringfellow5152]

He's not a skeptic, he's a realist. A human, with lower than average intelligence, can learn to safely drive a car in a few hours. No-one's created an AI that can safely drive a car on all roads even when trained on all the data available to mankind. See the problem? Bigger AI models require more training data in order to improve their performance. In other words, it's the greater volume of data that's improving their performance, not their intelligence (memory, not intelligence). An increase in intelligence would enable the model to improve performance without requiring more training data.

[@snarkyboojum]

So good to see someone let some air out of the LLM bubble. Dwarkesh might be a little challenged by this, but it’s great to get out of the echo chamber regularly.

[@ajithboralugoda8906]

yeah truly!!

[@jasonk125]

This strikes me as moving the goal posts. Chollet doesn't seem to understand how dumb the average person is. Can LLMs replace programmers? Can the average human replace a programmer? Go watch Jerry Springer and then tell me how LLMs won't reach AGI. To the average human, these thing are already AGI. Everybody in this video is in the top 1%. They are so far up the intelligence scale that they can't even imagine what average intelligence looks like.

[@mythiq_]

Wanted to agree with him, but Francois Chollet is way off. From the moment he mentioned not "copying" from stack overflow as some sort of example of humans handling novelty in the wild, it was clear he was idealizing some belief that he holds. He refuses to believe that creativity is mostly interpolation.

[@mythiq_]

Edit: I was wrong. This AI cycle is so dead.

[@oranges557]

​@@mythiq_what made you change your mind so abruptly?

[@rossmccannell8689]

"But if we gave LLMs as many Adderalls as I popped before this interview, would then get AGI?" "Ok, that may work." "That was a trick question. I snorted the Adderall."

[@mukutmukherjee309]

🤣🤣

[@juanperezmondragon8076]

Bro's onto something

[@maloxi1472]

I could almost smell the Adderrall through my screen. Insufferable 😫

[@Yossarianhoenikker]

😂

[@BrianPeiris]

Really glad to see people like Chollet are willing to say the emperor has no clothes. I'd recommend Subbarao Kambhampati's talks as well. He goes into some theories about _why_ people are being fooled into thinking that LLMs can reason.

[@jimgsewell]

Thanks for the additional resource

[@benprytherch9202]

Subbarao is wonderful! Great recommendation.

[@diracspinors]

It is pretty glaring how different Francois's interview is from Illya's. Maybe part of it is the result of Dwarkesh polish as in interviewer where for Illya it was a series of questions Illya minimally answered and here Francois openly expanded upon the problem. But, also from the start the two seemed different. Where Francois maintains all the features of researcher, who values an open exchange of ideas, Illya values secrecy and the advantages of being a first mover. I definitely enjoyed this interview much more.

[@MetaverseAdventures]

Interesting observation that I had not noticed, but perhaps felt to some degree. Smart people share information freely as for them it is abundant whereas people who are less smart hold onto knowledge as it is scarce in their world. I think this is the easiest way to really identify if the person you are talking to is smart or just socially smart.

[@diracspinors]

@@MetaverseAdventures No doubt Francois is generous with his understanding because for researchers understanding functions as a public good, ie it does not decrease in value with greater supply. More so, I think it demonstrates the differences of their respective positions. Ilya has a product he needs to deploy and make profitable, and he needs to secure advantage over the competition. It is intrinsically a much more narrow and concentrated effort. This can lead to good result in the short term, but long term, it’s an approach that tends to become stifling. This is also why Francois laments the shift brought about by OpenAI (which is rather ironic).

[@Matis_747]

I don’t think Ilya wants to be first simply for its own sake. He is worried about the future and would like to steer the ship in the right direction. I don’t know exactly what Ilya saw at OpenAI but after watching what Leopold Ashenbrenner had to say, the departure of Jan Leike over the 20% compute that was promised but never delivered and hearing that Ilya is now starting his own AI company called Safe Super Intelligence I suspect he has a good reason to be worried.

[@MetaverseAdventures]

​ @Matis_747 I do appreciate and understand the position you describe that Ilya is in and thus it colours all his words, but I am still not convinced he is as brilliant as made out. We will see as he started a new company that has no profit motive and thus I anticipate more flowing knowledge from him, especially around safety as that is his focus.He is a smart person no doubt, but there is just something that does not add up for me. Probably just me and my lack of understanding as I am just a user of AI and not a creator of AI. I look forward to seeing where Ilya takes things as maybe he is going to be the most important person in AI history, or fade into obscurity.

[@jameshuddle4712]

So... You're saying that Ilya is... Playing his card close to the vest, only responding to direct questions, almost a black box that you have to creatively prompt to get the right answer... Could he be a... LLM? :)

[@josteveadekanbi9636]

Train an LLM to solve math but don't include anything related to calculus in the training data. Then ask the LLM to solve a calculus problem and you'll see it fail. That's essentially what Francois Chollet was saying. Isaac Newton was able to introduce calculus based on his FOUNDATION OF MATH (Memory) and actual INTELLIGENCE (Ability to adapt to change)

[@falklumo]

Francois said the opposite, he actually said that a lot of human math skills rely on memorization too. But actual intelligence to discover/invent new math goes beyond this. This is why even winning a math olympiad would be as meaningless as winning chess, it's old math. Actual intelligence wouldn't win chess but invent the game - without being told so!

[@josteveadekanbi9636]

@@falklumo That's what I meant with the Isaac Newton sentence.

[@matterhart]

Making an AI Isaac Newton vs an AI Calculus Student is a nice and simple way to capture what they're trying to do. Making a great AI Calculus Student is an awesome accomplishment, but we really want a Newton.

[@bossgd100]

Newton AI is more ASI than AGI Most people cant discover calculus, some dont know how to apply it..

[@mennovanlavieren3885]

Many math teachers I've known say that if you memorize and apply the tricks, at least you'll pass the exams. You won'be great at math, but good enough. Up to some level math is about memory, like chess.

[@GabrielMatusevich]

There was 20mins of: "Can LLMs replace programmers?" "No" "But can they? "No" "But can they? "No" "But can they? "No" "But can they? "No" "But can they? "No" "But can they? XD ... it simply becomes clear that LLMs can't replace programmers when you start using them everyday on your programmer job and realize how Bad they perform when you start to do just slightly complex logic

[@wenhanzhou5826]

They were inventing an ARC puzzle on the fly 😂

[@jasonk125]

This strikes me as moving the goal posts. Chollet doesn't seem to understand how dumb the average person is. Can LLMs replace programmers? Can the average human replace a programmer? Go watch Jerry Springer and then tell me how LLMs won't reach AGI. To the average human, these things already are AGI.

[@GabrielMatusevich]

@jasonk125 they are not AGI because they can't perform every task an average human can. Also he was trying to explain they probably can't learn new novel tasks. The things at which LLMs excels are problems that have been solved numerous times and there is a lot of data around those. Even so. The world and its operations is not interconnected and digitized enough for LLMs to take over.

[@jasonk125]

@@GabrielMatusevich Well if Chollet wants to redefine AGI, and then say LLMs aren't AGI (which is what he does) then I guess there is no point arguing with him. From his website: Consensus definition of AGI, "a system that can automate the majority of economically valuable work," Chollet's definition: "The intelligence of a system is a measure of its skill-acquisition efficiency over a scope of tasks, with respect to priors, experience, and generalization difficulty." So they should have first come to an agreed upon definition of AGI (which they did not), before arguing about whether LLMs could meet that definition. Your statement: "they are not AGI because they can't perform every task an average human can" is not arguing within Chollet's definitional framework. It is closer to the consensus framework.

[@GabrielMatusevich]

@jasonk125 yea, is a good point. That reminds that I don't there is even actual consensus on a definition of just "intelligence" .. which makes it even harder 😆

[@jsn355]

I think Francois really got his point across in the end there. I was leaning somewhat to the scaling hypothisis side, he made me question that more. In any case you have to give him credit him for actually coming up with interesting stuff to support his arguments, unlike many other critics.

[@Ailandscapes]

“Coming up with” you mean like on the fly? 😂 but seriously though this guys a verified genius in his field, he didn’t just “com up with it” one morning

[@stri8ted]

What timestamp?

[@jonatan01i]

literally tortures him with "okay but isn't this AGI why is that not AGI?" questions and after not having any positive feedback asks "okay but let's suppose you lost your job to AGI"

[@maximumwal]

To get chollet to concede, you must synthesize new programs. Sampling the database of conversations with LLM hype bros doesn't generalize.

[@h.c4898]

I think these are two separate conversations. Host goes technical, guest goes philosophical on how these LLMs currently lack processes that helps them keep up with conversation the natural way. I kinda know what chollet i's talking about. In case of Gemini, it's unable to answer from an existing thread. It goes into a cycle of vérification from its database automatically. It could simply mined from past conversations to respond. It'll be more efficient in my view, use less tokens, less resources and will be more efficient than the current architecture its in. Also gemini can't process "on the fly' response either. For example for breaking news, it won't be notified but hours later.

[@snarkyboojum]

@@maximumwalunderrated comment :)

[@Gome.o]

@@h.c4898 No I don't think so

[@sloth_in_socks]

And in turn Francois keeps redefining what LLMs are doing and what intelligence is. He starts with LLMs just memorize, then they memorize templates but can't generate new templates, then they generalize but only locally, then they generalize enough to learn new languages that they haven't been exposed to but thats not intelligence either.... sure Francois. Call us when you've discovered AGI

[@dafunkyzee]

There is so much great information here.... I'm at 24:32 and Francois was saying "Generality isn't specificity scaled up..." He seems very aware that the current approach to LLM is bigger, better, more data, and he is right in noting that is not how human intelligence works. We don't need to absorb the whole of human information to be considered intelligent.

[@mythiq_]

Dwarkesh brings up a solid point that no scientist ever "zero shots" their ideas. Francois is partly correct, but he's holding onto some tight beliefs there about creativity, novelty and interpolation.

[@afterthesmash]

28:55 When the point in your day where you first need to synthesize an entirely new template is while interviewing Francois Chollet.

[@kiranramnath2846]

This one line from Francois summed up his argument - generality is not specificity at scale.

[@ZandreAiken]

Great Interview. I've personally done about 50 of the online version of the ARC challenge and the gist of solving them is simply to recognize the basic rules that are used to solve the examples and apply that same rule to get the answer. While some are challenging, most are using basic rules such as symmetry, contained or not contained in, change in color or rotation; or a combo of more than on rules. I'm sure that current large LLMs like GPT4 have internalized these basic rules in order to answer questions. so proficiently. What is perplexing to me is why can't LLM extract those rules and apply them to get more than 90% on any ARC challenge. I think that is the crux of the matter that Francois is getting at. If to solve any ARC challenge basically requires one to identify the simple rules in an example then apply those rules, why are LLMs not crushing it?

[@Martinit0]

Because LLMs - once trained - don't extract rules from input data and do another step of applying those rules. That would be precisely the "synthesizing" step that Chollet talked about. LLMs just ingest the input and vomit out the most likely output. The human equivalent is a gut-feel reaction (what we call intuition) without attempt of reasoning.

[@TheNewPossibility]

Because they can't generalize from 2 examples the rules like humans do.

[@telotawa]

1 million for this is as ridiculous as 1 million for P vs NP, it's a multi trillion dollar problem, it's like offering $1 for someone to find your lost supercar or something

[@Sickbirdy911]

lol absolutely, had the same thought. You could even expand it to "1 million for a discovery that will render capitalism and your reward useless :D"

[@Roboss_Is_Alive]

even worse arguably, since it'd make money obselete

[@TheReferrer72]

What? LLM's should be able to do this. P vs NP would instantly change everything.

[@rincerta]

Are you implying that we’re nowhere near AGI?

[@mbican]

Both of them can be solved by a kid in African on $100 android smart phone. You say trillion only because you assume you need power plants and datacenters maybe one more time over the current total infrastructure, but that's just zero imagination. It's like if you asked in 1500s the answer would be you need 100 million horses and millions of km2 of woods to burn What if the answer is that you need million times the power of Sun like advanced Kardashew 2 civilization?

[@jaydenkelly9086]

I wish this went for longer than an hour, it's refreshing to hear LLM skeptics to balance my view of AI. Yann next?

[@QuantPhilosopher89]

Not sure they would have really gotten any further with more time. I'm 40 minutes in and the conversation basically seems to go in circles. Dwarkesh: "but this and this behavior by LLMs could be interpreted as intelligence, couldn't it?", Francois: "if that were true, then they would be able to perform well on ARC".

[@VACatholic]

@@QuantPhilosopher89 I think it's good because honestly I know a lot of people like Dwarkesh. I obviously have very different metaphysical presuppositions than most people, so being able to find someone who is able to push back against LLM hype in a way that's understandable and reasonable is nice.

[@TheRealUsername]

​@@QuantPhilosopher89 well, if you stopped there then you missed some good insights

[@falklumo]

@@QuantPhilosopher89 Francois would have had a lot to say about program synthesis I would have liked to hear though ...

[@falklumo]

Francois is no LLM sceptics. He sees value of LLMs scale. He's only saying the obvious, LLMs cannot become AGI, only be a part of it.

[@twistedneck]

This has quickly become my goto podcast.. thanks Dwarkesh!

[@Ishabaal]

Nice to get a little return to earth with this one, pie in the sky talk about AGI is fun but the difference between that and LLMs still seems pretty huge.

[@therainman7777]

The thing about exponential curves is that great distances can be crossed in surprisingly little time. Which is most likely what is going happen. We’re not as far as you think, or as Chollet is making it seem.

[@10ahm01]

@@therainman7777 You know that if AI systems were to increase in performance every year by only 1% of each previous year, that would still be considered exponential growth.

[@therainman7777]

@@10ahm01 Yes, and after a sufficient number of years each 1% increment would represent a huge increase in capabilities, just as a 1% return on a trillion dollars is 10 billion dollars. Also, on an empirical level, we can actually estimate the % increase per year in technological progress using a number of different metrics, and it is nowhere near 1%. It is far, far larger than that. Moore’s law, to give just one example, equates to roughly a 40% increase per year. And many metrics relevant to AI development, such as GPU compute, are increasing even faster than that. So your point about 1% per year increases is also irrelevant for this reason. Lastly, this is not an exponential trajectory that start two or three years ago; it started decades ago. Which means the absolute increments of progress per annum at this point is quite large.

[@hydrohasspoken6227]

source?

[@QuantPhilosopher89]

​@@therainman7777a sigmoid function looks like an exponential in the beginning.

[@sjkba]

I love how Francois explains his answers with great patience and a subtle smile in his eyes. What a dude.

[@JumpDiffusion]

Not subtle. More like a stubborn smile…

[@studywithmaike]

I must say I don’t really like the way he keeps interrupting François during the interview

[@rennisdodman8739]

exactly. like whats the point of asking questions if you dont wanna hear the answer. dwarkesh got that journalist mindest: "i only want to hear a certain answer, not hear what they want to say"

[@merryn9000]

Thanks for this! Most interesting conversation on LLMs I've heard for a long time. I think programme memorisation vs novel programme creation is an important distinction. I can personally buy the idea that we mostly rely on programme memorisation in daily life, but we clearly rely on novel programme creation happening at some point! But unsure on the degree to which that happens within individual brains vs happening out of collective processes e.g. cultural evolution etc

[@luisluiscunha]

That was very well thought. And, if it is a collective process LLM can now be an intrinsic part of it. So well done. Vygotsky would agree.

[@proximo08]

This is the most constructive debate I have watched on AGI to be honest. Bravo Patel for asking the right question to Francois. Definitely makes me think more deeper about all of it

[@jasonabc]

Thanks Francois for the reminder that we can't just scale our way to mastering intelligence you can't memorize everything. I took this approach in college and it ultimately fails.

[@danypell2517]

yep. need true UNDERSTANDING to solve NEW PROBLEMS

[@JumpDiffusion]

LLMs are not just about memorization

[@Hohohohoho-vo1pq]

​@@JumpDiffusionand GPTs are not just LLMs

[@michaelthomson5125]

@@JumpDiffusion .. That is literally what they are about. They just have seen a LOT.

[@mythiq_]

@@JumpDiffusion Exactly. Why do they keep parroting the memorization bit. François knows better than to say that there's some copy of the code that the LLMs memorized.

[@anthonydellimuti4962]

TL;DR the host was combative in a way that made him come off as a salesman for AI rather than having a conversation about what the guest thinks for the first half of the conversation. also, the host refused to budge on his *belief* that current LLMs are capable of real understanding despite the guest's points to the contrary. first time seeing this podcast so i don't have a frame of reference for the normal vibes of the host but he seemed extremely defensive. the guest seemed to keep calmly stating that memorization and understanding are two completely different things while the host just kept referring to anecdotes and examples of things that he thinks displays understanding. the example that set my radar off to this was the obscure language dictionary example. After being shot down the first time by the guest by claiming that the ARC puzzles are a set of tests that it would be very hard to make training data for and if LLM's developed true understanding/adaptive capabilities then they should be able to pass the ARC puzzles easily. the host then tries to bring up the example of the chess models which the guest points out is almost exclusively pattern recognition and memorization and instead of wrestling with that point he moves back to the obscure language point. i think that evasion of the chess point is actually extremely telling. if he truly believed that was a good point, he might have pushed back on it or tried to justify why he brought it up but instead he says "sure we can leave that aside" immediately. maybe I'm being a little cynical. maybe he realized that was actually a bad point for the argument he was trying to make. regardless, he went back to the obscure language point which may have been impressive if it was not for the rest of this conversation to this point. earlier, the host tried to give an example of a simple word problem that had to do with counting. the guest countered that with all of its training data, it probably was just referencing a word problem that it had before which, from my understanding of how these things work, is probably accurate. the host clearly did not understand this earlier point because the thing about language models that the guest has to point out AGAIN is that the training data probably contains similar information. not necessarily data on that language but to my imagination, the data probably contains a lot of different dictionaries in a lot of different languages. dictionaries on top of having similar formats across most languages also typically have complete sentences, verb conjugations and word classes. i can see how the guest's point about memorization and pattern recognition would apply to LLM's in this aspect. as i continue watching i am realizing that this has turned into a debate on whether or not LLM's have the capability to understand and process information as well as synthesize new information which i was not expecting nor did i want. i think it is intuitively understood that current models are not capable of these things. this is why they require so much training data to be useful. there were genuinely good parts of this podcast, but the host insisting that LLMs understand things in the way that humans do were not it. this is a little nitpicky but there was a point when the host said something like 'lets say in one year a model can solve ARC, do we have AGI?'. to me this comes of as extremely desperate because the most obnoxious part of that question is also the most useless. the timeframe in which this may happen is completely irrelevant to the question. the guest at no point argued anything about timeframes of when he thinks AGI might happen. in fact when the guest answered in the affirmative the conversation took a turn for the better. finally if you haven't gone and taken the ARC test, i would encourage you to do so because neither the host nor the guest did a very good job explaining what it was. but on the second or third puzzle, i intuitively understood why it would be hard to get our current generation of models to preform well on those tests. they require too much deliberate thought about what you are looking at for the current models to pass. it almost reminded me of the video game "the witness" in its simplicity with the only clues as to how to solve the puzzles in both games is with context of earlier puzzles.

[@kman_34]

You summed up my feelings completely

[@young9534]

I agree with you. But I think you should still check out some of his other podcast episodes

[@anthonydellimuti4962]

@@young9534 i probably wont. the host did not really make me want to see more of him. i am kind of tired of being evangelized to about this tech in its current state. i will likely still follow the space and continue to learn more and seek more information. i hope this host does the same honestly. seems like the space is very full with people who want me to either believe that AGI is impossible or AGI is coming next year. i personally dont appreciate either side's dogmatism and will continue to try and find people with a more measured view on this stuff.

[@zenji2759]

You are overly judgmental. He pushed back because the answers felt too abstract. If they felt too abstract to him, they would ALSO feel too abstract to others. There is literally no personal attachment involved. Too many hosts don’t push back nearly enough due to stuff like this.

[@dokiee2]

Thank you for sharing your thoughts. Really helped me distill the conversation.

[@BoosterShot1010]

Dwarkesh way of talking reminds me of LLMs. Hearing François reminds me of what humans are.

[@DivinesLegacy]

Makes sense because it seems like dwarkesh is much more knowledgeable

[@mythiq_]

Quite the opposite effect on me. François felt like a calm android repeating "arc puzzle" and his beliefs about "novelty", like he has all the answers. Dwarkesh captures the frenzy of the puzzling human experience.

[@JimStanfield-zo2pz]

Dwarkesh is right though. Experience is enough. Chollet is just wrong, even in what he thinks LLMs are doing. LLMs do generalize. Patel didn't make the correct arguments.

[@andykjm]

I can't believe DP was arguing that the ceaser cipher of any N is not enough to close the case on this issue.

[@Rensoku611]

Literally, dude. I was neutral on the matter but that convinced me

[@StrandedKnight84]

Damn, I haven't even finished Leopold yet and you're hitting me with Francois Chollet!? Not complaining, though.

[@euclid2718]

❤ same here :)

[@leegaul8250]

Also same. Leopold is 4+ hours though...

[@wwkk4964]

Same. I couldn't take leopolds effective altruism based ccp fearmongering.

[@sdmarlow3926]

ARC is a great milestone for REAL research, in part because it's immune to brute force efforts where progress can be faked (all results are not equal). The prize money might get people to try things out, but at it's core, a "working solution" signals the existence of new technology (and potential to leap over $100M models). Posting "winning numbers" is less about prize money and more about $xxM in VC funding.

[@PhilippLenssen]

We shouldn't be surprised though that whatever AI solves this will still get accusations of having brute forced it... moving of AGI goalposts and all.

[@VACatholic]

@@PhilippLenssen Why would such accusations necessarily be wrong? This topic was mentioned during the podcast, and the fact that the test isn't perfect was conceded. Why do you think it is a perfect test, when the creator doesn't?

[@sdmarlow3926]

@@PhilippLenssen That will be a problem with any test, but a system that can form new behaviors to understand/solve problems will be able to, as a single system, take-on all challenges, even new ones, without having to go thru a costly training process. Even when we get something that is human-level, people will still question how much is AI, and how much is hidden offshore humans. Only "the real thing" will have staying power.

[@sdmarlow3926]

@@VACatholic A lot of benchmarks are good, as long as you are going at them honestly. Long before adding language abilitites, a system should be human-level on all games (that don't involve language).

[@sdmarlow3926]

Here is the correct way to push-back on the LLM scale still gets there thing: Having a set of all "solution patterns" stored doesn't do anything; it's the human, doing the prompting, that connects the stored pattern with what it needs to be aplied on. With ARC, no one gets to see the test data, so any system has to operate at a level where what it can do is searched based on what it can see. And the key aspect of ARC/AGI, is a system that creates it's own internal solution based on unseen challenges (ie, discovers novel solutions and saves them).

[@vermeerasia]

The perfect ever-changing ARC puzzle set already exists in the dynamic environment of driving a vehicle. This is a test that can't be solved with yet more examples because there is always something unique happening that throws the self-driving program into disarray with the resultant ripping off a Tesla driver's head or the idiotic smashing into the rear of a stationary emergency vehicle. If AI could become AGI through bigger data sets and more memory, then we'd already have flawless self-driving cars, robotaxis and AGI. We don't. Not even close. I think Chollet has highlighted the missing piece of the AI puzzle, pun intended.

[@vishnum3690]

Would love to see a conversation between Ilya and Francois

[@netscrooge]

I think Ilya's understanding is deeper.

[@jameshuddle4712]

@@netscrooge because?

[@netscrooge]

@@jameshuddle4712 His conceptual framework isn't merely sophisticated; it has closer ties to reality.

[@jameshuddle4712]

@@netscrooge Thank you for that insight. I assumed, considering his roots, that he was simply part of the LLM, crowd. Now I will listen with fresh ears.

[@netscrooge]

@@jameshuddle4712 I'm mostly going by what comes out of his own mouth. But if you can find the right interview, we can also hear what Hinton says about working with Ilya when he was his student, the startling way his mind could leap ahead.

[@Mkoivuka]

This is the first AI talk by Dwarkesh I actually enjoy.

[@BlakeEdwards333]

I knew you would bring Francois on the show one of these days. Thanks for making it be today! 🎉❤

[@joannot6706]

In some respects some part of these discussions sounded more like arguing as opposed to interviewing

[@Steven-xf1zy]

I appreciate and very much enjoy these podcasts. I also fully understand the need to play devils advocate. However, to me this felt a lot more biased than most of the other episodes. It's clear which position Dwarkesh has chosen. That's fine, but it really shines through when someone who is not an LLM maximalist is on the podcast. Devils advocate? Yes, always do that. Extreme bias where it becomes waiting for your turn to speak over a discussion? Not ideal in my opinion. I hope if he sees this he doesn't take it personally. Obviously he's very excited about this tech. Most tech folks are either excited or at the very least quite impressed with the advances that have been made over the last few years. I just hope the quality of discussion remains consistent regardless of who is the guest.

[@soltrinox1]

Dwarfish seems to be drinking the omnipotent LLM cool aid , saying that LLMs can do everything a human can do. Even Ilya admits the limitations

[@PseudoSarcasm]

Yep! I think he just can't get past his preconceived notions and keeps banging on about something that was explained to him in the first few minutes.

[@Rais_Latif]

I learned a lot from this. This was so cool! Thanks Dwarkesh

[@julkiewitz]

People are actually much smarter on average than one tends to give them credit for. It's just that we are very very reluctant to use System II. We'll do literally everything else before deploying the full power. But if one's life depended on it or there was sufficient incentive, we can be extremely fast learners. We just naturally try not to get challenged this way in everyday life.

[@117Industries]

Even though I agree with what you're saying- one of the things researchers found that exists as a general difference between persons widely separated along the I.Q. spectrum was that the glucose uptake & thermal output in brains of lower I.Q. people were much greater than those on the higher. This indicates that a more generally intelligent mind is both thermally and resource efficient: expending less fuel and generating less waste per unit of output. What this points to is that some people can activate system 2 with considerably less cognitive burden. Since most of us are pleasure-maximising, instinctually and natively, and since it's distinctly unpleasurable to be in the uncomfortable state of mental strain/discomfort associated with glucose starvation or one's brain overheating, one might expect that behavioural inclination follows from ability. In the same way that a natural weakling doesn't enjoy lifting weights, and so avoids it, an intellectual weakling doesn't enjoy activating system II, and so avoids it. The fundamental reason is the same in both cases: we avoid that for which we lack rewarding feedback (relative to peers) and which relatively strains us (relative to peers). The fact that anyone _can_ activate system II means simply that everyone has and utilises a general form of intelligence. However, the fact that people _don't_ do this suggests that they have a relative deficit in system II (or rather in its utilisation) which explains this avoidant tendency, while simultaneously pointing to the degrees of difference in the general intelligence of people.

[@Lazy-AI]

Dude, with all due respect, ease up and let the guest speak. You keep cutting him off with the same question while he's trying to finish his thoughts. Do you even code? If you used different LLM coding assistants for a while, you'd understand what he means. It's essentially glorified Stack Overflow. And the story won't be any different with any other transformer neural networks.

[@daveqr]

Listened to another podcast where he and his guest spent an hour chasing their tails about nationalisation of ai research. It was a very frustrating listen, and I finally gave up, just like I'm giving up on this one.

[@ZelosDomingo]

It seems like the ARC thing maybe would be difficult for LLMs because they are reliant on visual symmetry that wouldn't be preserved through tokenization? I mean, I'm sure it's not that simple, because then natively visual models would probably be solving them easily. But still, there should be a version of this test that has complete parity between what the human is working with and what the LLM is working with, I.E. already tokenized text data.

[@falklumo]

An LLM can easily transform the JSON test files to ASCII art and still doesn't solve it.

[@benprytherch9202]

Chollet addressed this objection in the video by pointing out that LLMs actually do quite well on these kinds of simple visual puzzles, when the puzzles are very similar to puzzles they've been trained on. So this can't be the answer. We'll find out soon enough how well the multi-modal ones do.

[@allanc3945]

It might not be fair to say an LLM needs millions of training examples but a young child doesn't. By the time a child is old enough to solve these puzzles, they have 'trained' far more than that through interaction with the world. A more accurate comparison would be an untrained LLM vs. a baby.

[@allanc3945]

LOL, I should have watched the rest of the video before posting!

[@Bencee116]

@@allanc3945 And even evolutionary upbringing encodes priors in the brain, like symmetries and basic geometry, hence, human « training » starts far prior to being born

[@jimbojimbo6873]

Thank you someone called it out, an LLM will not achieve AGI, it’s like building faster cars to achieve flying cars.

[@jimgsewell]

Plenty of cars have flown. Just not controllably 🤣

[@francisco444]

Who's building LLM anyways? They're old by now

[@jimbojimbo6873]

@@mint-o5497 it fundamentally cannot come up with something someone hasn’t already. That’s the issue. They are two different problem sets.

[@everettvitols5690]

@@jimbojimbo6873 Please define "come up with something someone hasn't already" and tell me, have you actually ever done this - and what did you come up with?

[@jimbojimbo6873]

@@everettvitols5690 i’ve made a dish I’ve never seen in a recipe book or online before. If we entertain hyperbole then something as grand as self driving cars that meets regulatory requirements is something we don’t haven’t conceived, if it can do something that’s truly innovative then I’d consider it AGI. i don’t see how LLMs will ever achieve that.

[@AI-HOMELAB]

Hey you got yourself an actual expert about the subject. Thanks! 🙏

[@hunghuynh1946]

Finally Dwarkesh got hold of a guest who talks sense out of this LLM madness. LLM will get nowhere near AGI, multimodal or not.

[@eprd313]

But DNNs will, and LLMs are built on them

[@maloxi1472]

@@eprd313 It's not just an architectural issue. The whole epistemological foundation of the prevailing approach to AGI is shaky as hell ( kzbin.info/www/bejne/f5a8aYSXgtiMp7s )

[@eprd313]

@@maloxi1472 that video aged like spilled milk. All the progress made in the last year contradicts its claims and the distance from AGI is now more of a matter of hardware than software, a distance that AI itself is helping us cover as it's designing microchips more efficiently than any human could.

[@nitap109]

Great job Dwarkesh. Always v interesting videos.

[@2394098234509]

I'm so glad Dwarkesh is doing these interviews. He asks all the key questions. Unlike another science/tech podcaster who shall remain unnamed.

[@afterthesmash]

Voldemort asks all the right questions, but rarely more than two in a row, before he rests his throwing arm while firing off a bunch of nerf balls. His best probes are episodic rather than sustained. This stuff is simply too hard to command on an episodic basis.

[@onuryes]

Your podcasts are absolutely fantastic! I always eagerly anticipate each new episode and have learned so much from your content. Thank you for your hard work. This episode especially was inspiring and it gave me so many ideas to try and think about.

[@thewisetemplar]

Francois was very patient in facing the barrage of questions from the LLM champion. I think a conversation with Jeff Hawkins would be very beneficial to understand, in a more general way, why some aspects of intelligence are missing in current deep learning.

[@rufex2001]

EXCELLENT EPISODE! These types of counter arguments against the LLM hype are SUPER IMPORTANT in this public debate, and both Francois and Dwarkesh made great points for both sides of the debate! The pushback from Dwarkesh was excellent, but we need that type of pushback against the proponents of scale = AGI as well.

[@SuperFinGuy]

The thing is that LLM's can only do correlation, not cognitive abstraction. Remember that they are probabilistic models.

[@mythiq_]

How is the brain not interpolating or probabilistic? The only addition that the brain has is qualia and input from the cns, how significant they are for generalized agi is unclear yet. For reference: Oxfords Shamil Chandaria's lectures on the Bayesian brain.

[@calvinjames8357]

@@mythiq_ It goes beyond interpolation and probabilistic solutions. Our brains are fundamentally able to abstract concepts with very few data points, proving that we are very sample efficient even when exposed to an entirely new set of data. LLMs are just really fancy estimators, capable of understanding the semantics of a given problem and generating an outcome based on similar problems its faced before. The semantic understanding of a problem enables interpolation. It does not abstract the given problem and then deal with it with it's own sense of understanding.

[@WismutHansen]

I love the stark contrast to the last episode! Very interesting guests!

[@Azoz195]

Always worth pointing out the LLMs require a server farm that has the energy requirements of a small state, whereas the human brain runs pretty effectively on a bowl of cheerios. I think more people should think about this!

[@CyberKyle]

While this is true, I think it misses the point of the eventual advantage of deep learning systems. Human brains are fixed in size right now, mostly due to the evolutionary pressure of the size of the birth canal. Even if deep learning is multiple orders of magnitude less data and compute efficient than human brains (excluding the horrible compute efficiency of evolution to get us where we are), we can still scale the models to run on ever more power hungry data centers to surpass human brains. At the same time we can do this, our algorithmic and data sample efficiency gets better too, improving the ceiling that we can achieve.

[@bmark6971]

​@@CyberKyleall of that advancement will lead to great things but at its core foundation a LLM cannot achieve AGI. Also keep in mind these models are not even scratching the surface of the brains capabilities to apply intuition, rational, morality, and many other things that contribute to decision making beyond just simply data processing.

[@Martinit0]

Not for inference. Inference can be done on a single (sufficiently large) GPU. It's only the training of LLMs that requires massive server farms.

[@CyberKyle]

@@Martinit0 that’s not true, the operations needed for inference can be sharded across many nodes just like training. It’s just that training requires a ton of forward passes to see what the model outputs before you backpropagate the errors, so it requires large clusters to complete training in a reasonable timeframe. It is conceivable that you could make a ginormous model with many trillions of parameters that you’d shard across many GPUs.

[@eyeofthetiger7]

@@CyberKyle. Although the capabilities will possibly reach AGI even without radical efficiency improvements, AI will always be greatly limited in it's impact until the energy efficiency problem is solved. Most likely there needs to be a fundamental change to what type of hardware architecture is created for AI to run on that can be sparse computationally and reduce memory transfer costs by possibly combining memory and processing into a unified architecture "processing-in-memory" (PIM) like neuromorphic computing.

[@YuraL88]

It's such a fascinating discussion! It definitely deserves more views! I can disagree with Francois Chollet about LLM potential in general, but I must admit that his approach is extremely refreshing and novel. We need more people with nontrivial ideas to build True AGI because just scaling LLM is a risky approach: if we fail the new great AI winter is waiting for us.

[@Grahfx]

LLms are arguably the most significant tech bubble in human history. The gap between public expectations and their actual capabilities is insane.

[@ZachMeador]

but, people use them? every day? this is very different compared to shitcoins, which was much frothier, cumulatively

[@TheSpartan3669]

​@@ZachMeadorPeople's expectations far surpass their uses. ChatGPT is nice but it isn't going to solve the Riemann Hypothesis.

[@eprd313]

When you don't understand the power of DNNs and/or don't know how to use them...

[@Hagaren333]

@@ZachMeador We have to see who the uses are, we see that the majority were from students who wanted to falsify their work, but really the numbers of users are not significantly high, we must also take into account that it is possible that they even inflate the user base thanks to the mandatory implementation of LLM in internet browsers and operating systems

[@Hagaren333]

@@eprd313 We can know that DNN's are probabilistic adjusters, they can help us find the most likely possible answers, but they are not intelligent, nor is a search engine, in fact their implementation in search engines has been catastrophic, especially for Google where they prioritize the answers from Reddit, whether or not these are true does not matter

[@fernandoayon876]

I don't get all these comments saying that there was a lot of repeated questions. The way I see it, the subject was interesting and "tricky" enough to talk about it in depth like you guys did here, yes it would seem that you repeat the same question everytime, but the answers and explanations from Chollet were super interesting and every time we had a new way to look at it, nice interview.

[@ChrisOelerich]

Dwarkesh sure hates code monkeys.

[@wookiee_1]

Yep, he has a bias that was glaringly obvious in this video. The fact alone that he referred to software engineers as code monkeys shows how little he knows about the novelty and complexity present in the day-to-day life of a real world SE. Having a Comp Sci degree doesn't equal to an understanding of software engineering.

[@yuriy5376]

Maybe he got bitten by one as a kid

[@mythiq_]

@@wookiee_1 He is right though. Why else would the majority of US coders hired in the last 5 years come out of 12 week bootcamps.

[@wookiee_1]

@@mythiq_ I really doubt that the majority of coders in the last 5 years came out of boot camps, especially when employers have a bias toward 4 year degrees. Where did you find that info? Coding != Software Engineering. There's a big difference between recent grads and engineers with even 1-2 yrs of experience in a real world production environment. You need to make long-term implementation decisions, architecture planning, solve real-time production issues, creating/maintaining CI/CD pipelines, hardware or cloud resourcing, scaling issues, cost issues, train/mentor junior employees, code reviews, investigate/fix bugs, implement features, groom/point user stories, weigh in on expectations from stakeholders, and bridge the gap between what they say they want and what they really want. Coding is just a small portion of what engineers do. There's a LOT that boot camps don't teach, and most learn on the job or on their own time to catch up. TLDR: just because you're a graduate from a college or boot camp does not make you a software engineer.

[@mythiq_]

@@wookiee_1 CS degrees from the US in 2021 about 100,000 vs bootcamp grads about 200,000. Either way, I agree with you that coding != software engineering. But most "coders" claim to be engineers. My point was that there aren't that many software engineers needed per company, on average most "developers" do work that wouldn't be considered "knowledge work".

[@andrewj22]

I would totally invest in Chollet's research. He has a tons of insight and clarity. I have ideas, but I don't have the background to do the work myself - my background is philosophy. I'd love to participate in this challenge, but it would take me years to embed myself in the academic institutions.

[@sepptrutsch]

I am not so sure that humans do much more then using pattern recognitions to solve the ARC-problems. When I look at them I very quickly recognitions patterns I saw somewhere else. Our brain has been trained on millions of image patterns by evolution.

[@dp2120]

Exactly - and that’s the point Dwarkesh was challenging Chollet to address, but Chollet refused.

[@que_93]

The section of the video from "Skill and Intelligence" to the end of "Future of AI Progress" made for such a compelling and engrossing watch. I must convey huge thanks to Dwarkesh for pressing on the matter-- with such vast grey-area (between "memorization" and "intelligence" (AGI))-- given the present state of research-- that allowed Francois to give out so many priceless points which can rarely be found elsewhere. Francois's insight and experience coupled with Dwarkesh's active back-and-forth questioning gave some insights which are extremely valuable, at least for someone like myself. And I have to commend his impeccable presence of mind, as at 47:58. If it not had been this active, it would have been a bland conversation (podcast). The questions assisted in giving out so many brilliant viewpoints by Francois. Those commenting on Dwarkesh's fast speaking style and Francois's slow paced speaking, it is their individual style. He (Francois) spoke in the same pace with Lex, and Lex himself is a slow-spoken man. It is one's individuality. With his same fast-paced speaking, he made the conversation with Sholto and Trenton very engaging and worthwhile, and so enlightening, as this one. And if I am not wrong, Francois wouldn't be this slow in uttering if he was rather speaking French. In the very beginning of this interview, Dwarkesh told that he had so many guests who were strong proponents of LLMs and their scalability-- which he found a bit "contrived." So, his attempt here is clearly to use those same arguments, some of his own viewpoints, and present the questions to Francois for a better and clearer insights for us, the listeners and the learners, as to what are the "bottlenecks" of LLMs which he perceives-- given there massive "accomplishments" in these last 4-5 years, and what different mechanisms or tests are their to achieve steps closer to "true AGI." This was the backdrop of his questioning. And he created a brilliant premise for such a learned and esteemed guest of the field. Had he not presented a case for LLM while Francois presented a case "against" LLM, it would have been a one-sided talk-- just asking questions on ARC. How else would we have come to know as to why LLMs haven't fared as well on ARC these past 5 years while they have done tremendously well on other benchmarks? How else we would have gotten so many precious inputs from Francois? I bow with respect to Francois for understanding the reasoning behind the premise set for this interview, and to Dwarkesh for his brilliant questions, "hard-presses" on so many important points, and for his presence of mind. I got to learn SO MUCH from this.

[@VoloBuilds]

New ideas are really just combinations of existing ideas. As such, LLMs can indeed create new things that are not in training data. Check my channel for the video "Can AI Create New Ideas?" for more details and examples of this. That aside, ARC is a fascinating benchmark that tests something entirely different: advanced few-shot pattern recognition. This feels like a powerful and important architectural component for future AGI systems, but I would not label it as "AGI" on its own.

[@JordanMillsTracks]

This completely makes sense and has fully brought me back down to earth regarding LLMs 😂

[@jeroenbauwens1986]

More interviews about the importance of system 2 thinking would be awesome, for instance John Carmack (of Doom and MetaAI fame) is also working on this... Your channel is becoming so popular it could easily lead to a technical breakthrough at this point

[@falklumo]

The importance of System-2 thinking is now a trivial fact for everybody working on AGI. But this channel helps to popularize this.

[@jeroenbauwens1986]

@@falklumo if that's true, why did Dwarkesh' ML friends who are working on AGI not know about the ARC benchmark and why were they surprised that the frontier models failed?

[@bazstraight8797]

@@jeroenbauwens1986 The German guy (interviewed in immediately previous podcast) is young (inexperienced) and not a ML researcher. He stated that he was extrapolating straight lines derived from existing data. DP also recently interviewed a pair of guys about LLMs who had only been in ML for a year ie inexperienced.

[@jeroenbauwens1986]

@@bazstraight8797 so your point being that people like John Schulman and Ilya Sutskever are more knowledgeable.. I wouldn't be too sure they know about ARC though, Ilya has said in the past that scaling's essentially all you need. It sounds like this might be a blind spot in all of these companies. I guess Dwarkesh missed the opportunity to ask them

[@jamesperez6964]

One of the best episodes, love the challenging conversation

[@sloth_in_socks]

Prediction: Arc will be solved within a year. Dwarkesh will claim he was right because it was a scaled up model. Francois will claim he was right because it wasn't purely an LLM. In the end it doesn't matter because we got closer to AGI.

[@eprd313]

This, and people will still be stuck in their reactionary prejudices or desperation to see change

[@basbuller7577]

Appreciate the honest and explorative discussion. Good to see both willing to hold opposite views and hence explore the discussion

[@seanfuller3095]

What is the name of the researcher they are talking about with test-time finetuning (mentioned by Dwarkesh at 14min mark)? It sounds like “Jack Cole”?

[@anthonymercuri7020]

"I don't understand" seems to be the key disposition of the interviewer 😅

[@benmaxinm]

He is young and excited with all the knowledge, give him time. Amazing conversation.

[@alexraphaelm]

There’s a reason why our biological neural nets are so efficient. The fact that you need to scale an LLM model so much for it to get a grasp of the world model is telling of its inadequacies - it doesn’t diminish the value of scale or the methods themselves, it just signals their flaws. Intelligence may be a mix of a set of skills that include the memorization LLMs can run, and abstract reasoning from little context. System 1 and System 2 together, not one without the other. I’d expect a model that can actually reason to require much less compute, just like our brain does.

[@TheBobiaan]

Your brain uses serious compute actually.

[@alexraphaelm]

@@TheBobiaan No doubt it does! Several orders of magnitude less of course. I argue it's likely more energy efficient.

[@netscrooge]

Our biological neural nets are not more efficient in terms of complexity. Just the opposite is true. Our largest AI systems are terribly simple compared to the complexity of our brains, way, way less than 1% the complexity.

[@solnassant1291]

Lol I screenshotted the problem at 7:24 and asked ChatGPT. While the image it generated was completely off, its answer was almost there. I sent it the puzzle and asked "what would the 8th image look like?" It replied "[...] Considering the established pattern, the 8th image should show a continuation of the diagonal line, turning green at the intersection points with the red border. Therefore, the next step would involve the following arrangement: The blue diagonal line continues towards the right border. Any part of the diagonal line intersecting the red boundary turns green. So, the 8th image should depict a continuation of the diagonal line, transforming blue cells into green ones at the boundary intersection. [...]" So OK, it didn't get the perfect answer because our line becomes green even before meeting the wall. But it's pretty damn close. GPT 5 should smash these puzzles.

[@TheRealUsername]

Over the past few months I've tried multiple tests similar to IQ tests on Gemini 1.5, Claude 3 Opus and recently GPT-4o, I've noticed there's some of the exercises related to "sequential selection" where it should guess the logic of the sequence order and select among multiple other elements to complete it, it seems very inconsistent, there's one test I've extracted from a logic test with geometric logic rules where each step gradually increase the complexity, GPT-4o succeeded 4/10 but it got some right in a incoherent order, as if the model wasn't actually reasoning, for 6/10 it failed with hallucination at the end of the rationales, there was some that was more complex that it got right while some where simpler it failed, similarly to Claude 3 Opus and Gemini 1.5. My conclusion is that these models don't logically reason despite Visual CoT prompting and high-resolution images for the tests, they generalize over multiple similar training samples, they can't logically reflect as we use to.

[@victormustin2547]

gpt5 is gonna blow past his expectations, coming back to this video when it does will be so fun

[@TheRealUsername]

@@victormustin2547 He's still right, no AGI will come out from LLMs though. Just read some research papers about "Grokking" and "platonic representation" to understand the insurmontable plateau of LLMs

[@solnassant1291]

@@TheRealUsername In the face of what's going on right now (LLMs get better and better) I'm not going to believe in a hypothetical plateau until I see it!

[@wowsa0]

@@victormustin2547 He was very happy to respond to lots of the points with "that's an empirical question, we'll see very soon", and has been very explicit about what would prove him wrong (a pure LLM which beats ARC without massive ARC-specific fine tuning), and has even offered a cash prize to incentivize people solving this problem. This is all great behaviour and should be encouraged. It's exactly how disagreements should be handled! If he does get proven wrong and graciously concedes, then he will deserve to be commended for that. It's weird that you'd take pleasure from someone getting proven wrong when they were so open to having their mind changed in the first place. That's not great behaviour. It's that kind of attitude that makes people defensive, so they become entrenched in their positions and refuse to listen to other points of view. You should celebrate when people admit they're wrong and change their mind, but you shouldn't gloat about it.

[@paulofalca0]

One of the best AI talks I've seen in the last months,thumbs up for Francois Chollet and Mike Knoop. Also thanks for Dwarkesh Patel bringing them :)

[@sammy45654565]

he talks about the importance of synthesising programs on the fly, using pieces of other programs and reassembling them for the task at hand. isn't that process just another form of a program, which could be incorporated into an LLM just like the various existing programs he claims don't represent intelligence?

[@perer005]

Having a program capable of creating programs to solve specific tasks, and having a large number of availible specific programs is not the same thing!

[@sammy45654565]

@@perer005 i'm not saying they're the same thing. just that they're both technically programs potentially able to be embedded into an AI. it's not infeasible that there are patterns within the logic used to piece together a new program which the AI could learn from and improve with.

[@shyama5612]

This is really insightful from francois - great articulation of memorization/benchmarks gaming, true generalization, intelligence and a good working definition of intelligence from Piaget - I recall the other frenchman yann quote him once too. I'd love to see the day when google/Francois ' team creates a new System 2 based Combinatorial search (discrete search) based engine that can do program synthesis on the fly with LLM embedded in them in the future!

[@MIKAEL212345]

this video was basically a philosophical debate on "what is intelligence?"

[@falklumo]

Not philosophical, scientific. A nice example where philosophical discourse stalls and scientists just move ahead :)

[@MIKAEL212345]

@@falklumo this is 100% philosophy. Science cannot answer what is intelligence. Only when philosophy gives a definition of intelligence, can science go out and measure whether and how much beings are intelligent.

[@AlessandroAI85]

No it's not, it's about the difference between acquire new skills and reasoning on new situations on the fly. Only the interviewer doesn't seem to grasp the difference!

[@MIKAEL212345]

@@AlessandroAI85 yes it is. They are both trying to define intelligence and find the boundaries of what is and isn't intelligence as compared to just memorization. What else to call that but philosophy? Philosophy is not a special thing only done in universities.

[@YuraL88]

​@@AlessandroAI85 they both had hard time to understand each other. The main issue is emergent properties. You can't predict if a large enough model have some "true" intelligence or llms are saturaing.

[@dafunkyzee]

Francois is so right on with this idea that LLM is not the same as memory. If it were then LLMs should be able to do ARC tests which humans find easy to do. As humans can do it, and LLMs are not, it gives a lot of strength to his argument that there is something else that is needed before agi happens.

[@Gauchland]

They are still generalising. There is no way it's memorised every single possible prompt. It's just that they're generalising significantly less than humans.

[@netscrooge]

That's right. Significantly less, yes, probably largely because the needed signals are weak in the training data. Considering how weak those signals are and how simple these nets are compared to the complexity of the human brain, it's truly amazing they generalize as much as they do.

[@TheGagman2000]

one very interesting neuroscience question arises regarding this merger between program search and deep nets: If the brain is analog (neurons, synapses, electrical potentials) then in essence it is continuous and not discrete. Even an action potential from a neuron is a smooth function. So somehow a continuous and deep, parametric function, the brain, is representing discrete structures (like the operators) needed by the researchers who create algorithms that synthesise programs, that eventually solve ARC. I am hopeful that people will fund more neuroscience, along with AI and ML.

[@slm6873]

The initiaton of an action potential is all-or-nothing, the action potential generation threshold is what establishes the discontinuity, not the shape of the electrical potential itself. But you can certainly describe the system as smooth functions if you go up to the level of continuously varying firing rates.

[@ericbeck6390]

What a great initiative! Im so grateful to Dwarkesh for giving this publicity!

[@telotawa]

interesting thought experiment: if you had all the data of everything that happened in medieval economic systems, before the industrial revolution: every conversation spoken, and lots of multimodal data, trained the biggest LLM ever on all that, and then jumped forward to today: how well would it do?

[@benmaxinm]

I like this!

[@d96002]

Very interesting conversation !. I'm not an espert but it definitely feels to me like as soon as you step out of the path paved by memorization the current Ais give incorrect answers, look for example at the responses of Gemini about pizza and glue etc. Unless they want to make us believe that it is only memorization...

[@pianoforte611]

Great interview. Good to a have a different well thought out perspective on AI. I appreciated that Dwarkesh was willing to press Chollet on his claims, in particular trying to nail him down on what exactly counts as generalizing beyond the data set. It seems that he didn't really have a good answer apart from "doing well on ARC". I still think he overestimates the extent to which average humans are able to do this, and underestimates the extent to which transformers are able to do this. Also, going from 0% on ARC to 35% in a few years seems like a lot of progress to me, so I'm really surprised he didn't think so. I would bet that they next generation of multimodal models get to 50-60% and that we get to 90% by 2030.

[@netscrooge]

Chollet's claim about human intelligence being unique is weak. Even ants have demonstrated the ability to respond to novelty. In the end, we're all neural networks. Stop deifying human intelligence. Bottom line: Chollet is expressing a form of religion, not science.

[@dextersjab]

Thanks Dwarkesh for giving Francois some more reach. I'd recommend watching Francois' 2nd Lex interview for more context on his thinking.

[@carlt.8266]

First I was glad, that I could solve the Arc puzzles myself and apparently I am not a bot, but then I thought about, how would one do, who has not seen these kind of puzzles? Is the difference from humans to LLMs just the we are able to generalize with less samples in a broader field?

[@perer005]

The unique part is thst humans can use their generalizations to predict/understand the world. It is not about data storage.

[@InfiniteCuriousity]

Good deep dive discussion, I learned quite a few things. The debate itself was a bit extended, but it was useful to hear Francois rebut each possible objection, and that may have been Dwarkesh's goal here, as part of the announcement: pre-empt all the objections and get the answers on the record. Having said all that, the simple answer that would have rebutted all the arguments is the mere fact that Arc-AGI is still not claimed, despite the incredible scale achieved by the frontier models. Reflecting on why that is would be the answer that the "devil's advocate" position would have to contend with.

[@hinton4214]

Dwarkesh, don't listen to the comments, you did extremely well in this interview, much better than Lex when he challenges his guests. Well done and continue this line!

[@YuraL88]

The channel definitely deserves more subscribers) Extremely interesting discussion, and I'm looking forward to new interviews.

[@Analyse_US]

This is a great discussion...

[@blahblahsaurus2458]

Dwarkesh just didn't get the point about what Francois says general intelligence is and gets stuck there. LLM training data is truly vast. It's packed with test questions, example test questions, text books from every country and decade, online courses, homework help requests, brain teasers, etc. It is extremely hard to come up with an actually new problem. That is - something that isn't similar to anything in the training data. For the common questions, the training LLM wouldn't make it far by storing the solution for every version of every problem. It keeps encountering new versions of those problems in training. So it evolves an algorithm to solve that kind of problem. What Francois is saying is that when it solves benchmark problems it can rely on a huge Bank of algorithms which each work on only one class of problem. The benchmarks don't contain many problems that are completely different from every other problem in the training data. Only those problems actual one shot creative problem solving.

[@blahblahsaurus2458]

What he said about IQ tests is also very important. In an official test, the only kind psychologists would accept as an accurate score, you have to wait two years before you can take it again. This is because familiarity with the test will artificially increase the score, making you think you're smarter than you actually are. If an hour long test can boost your intelligence for two years, what does a lifetime of problem solving do? Kids spend most of their waking hours playing games, solving puzzles, being taught by adults. Any of those things could end up being similar to an IQ test question. Imagine you had to make up a test for strength. The trouble is strength doesn't exist. What exists is hundreds of muscles, and each of those muscles has "strength", but it has a different strength depending on its extension, and it has different sustained strength and explosive strength, and so on and so on. So what do you do? You make a weight lifting competition. But weightlifting is only a very narrow type of motion and it uses different muscles in different ways from any other sport. So... what? Climbing? Pole vaulting? MMA? Strength can only be measured by asking people to perform some kind of sport, and different people do a lot of different sports. There has to be one test that everyone takes, otherwise how do you compare and rank people by their strength score? You could test every muscle in different positions and different conditions. But the test also has to be doable in less than 5 hours otherwise only the people who care about strength will do it, and the strength scientists need to test a lot of people from a lot of demographics to get a large enough sample size to publish their paper. And people certainly wouldn't watch it on TV. IQ is the same as strength. Someone decided there must be a "general intelligence" factor called G, and set out to measure it. But first of all, G doesn't exist. What exists is millions of neurons, and the anatomy of a single neuron is almost as complex, in its way, as the anatomy of the human body. But second of all, if G did exist, IQ would not measure it, because IQ tests are standardized questions crafted by formulas that could be similar to a bunch of things a human could be exposed to, and the test is short too. So all an IQ test does is measure the ability to take an IQ test. How many challenges and tasks in your minute to minute existence are anything like an IQ test? Not to mention the lives of your ancestors?

[@gunnerandersen4634]

I think it's not hard to understand his point: Ask GPT to create a novel architecture for AGI, it can't because it can't comeup with an actual novel thing, but a mix of existing ones that looks "novel" but it really is not.

[@falklumo]

That's not the issue. The issue is that GPT can't think about the problem. Even humans come up with new ideas by recombining existing bits and pieces. Look at Jules Verne's "rocket" to the moon which was a big big cannon ball ...

Not actually true.

[@gunnerandersen4634]

@@falklumo okay but it's not the same to combine things to make a new thing than actually proposing a completely novel idea right? So what I understood from his claim was that actually models currently use what they already know and combine that, but they can't take a broader look and redefine, let's say quantum mechanics with a truly novel approach, isn't like that? Maybe I got that wrong IDK 🤔

[@gunnerandersen4634]

I might have missed something then, I just thought that's what this test was all about, perform over completely unseen problems that require to actually abstract more from the details and into the bigger picture to try and find the underlying logic of it. I am not an expert on any of these things, I just thought that was it.

[@HanqiXiao-x1u]

It’s a very engaging conversation, clearly the host is very passionate about the topic and excited to converse

[@lra]

This dude actually threw out the line "I studied Computer Science" to Francois Chollet. We get it, you're an LLM fanboy, but you're speaking to someone that actually knows what they're talking about. Let the man speak!

[@pratiknaikedu]

Dwarkesh is on a role with these interviews.

[@sampatel90069]

I am beyond fully convinced that this is the best podcast on the whole of the global internet.