Jay McClelland: Neural Networks and the Emergence of Cognition | Lex Fridman Podcast

Jay McClelland is a cognitive scientist at Stanford. Please support this podcast by checking out our sponsors:
- Paperspace: gradient.run/lex to get $15 credit
- Skiff: skiff.org/lex to get early access
- Uprising Food: uprisingfood.com/lex to get $10 off 1st starter bundle
- Four Sigmatic: foursigmatic.com/lex and use code LexPod to get up to 60% off
- Onnit: lexfridman.com/onnit to get up to 10% off
EPISODE LINKS:
Jay's Website: stanford.edu/~jlmcc/
PODCAST INFO:
Podcast website: lexfridman.com/podcast
Apple Podcasts: apple.co/2lwqZIr
Spotify: spoti.fi/2nEwCF8
RSS: lexfridman.com/feed/podcast/
Full episodes playlist: • Lex Fridman Podcast
Clips playlist: • Lex Fridman Podcast Clips
OUTLINE:
0:00 - Introduction
0:43 - Beauty in neural networks
5:02 - Darwin and evolution
10:47 - The origin of intelligence
17:29 - Explorations in cognition
23:33 - Learning representations by back-propagating errors
29:58 - Dave Rumelhart and cognitive modeling
43:01 - Connectionism
1:05:54 - Geoffrey Hinton
1:07:49 - Learning in a neural network
1:24:42 - Mathematics & reality
1:31:50 - Modeling intelligence
1:42:28 - Noam Chomsky and linguistic cognition
1:56:49 - Advice for young people
2:07:56 - Psychiatry and exploring the mind
2:20:35 - Legacy
2:26:24 - Meaning of life
SOCIAL:
- Twitter: / lexfridman
- LinkedIn: / lexfridman
- Facebook: / lexfridman
- Instagram: / lexfridman
- Medium: / lexfridman
- Reddit: / lexfridman
- Support on Patreon: / lexfridman

[@lexfridman]

Here are the timestamps. Please check out our sponsors to support this podcast. 0:00 - Introduction & sponsor mentions: - Paperspace: gradient.run/lex to get $15 credit - Skiff: skiff.org/lex to get early access - Uprising Food: uprisingfood.com/lex to get $10 off 1st starter bundle - Four Sigmatic: foursigmatic.com/lex and use code LexPod to get up to 60% off - Onnit: lexfridman.com/onnit to get up to 10% off 0:43 - Beauty in neural networks 5:02 - Darwin and evolution 10:47 - The origin of intelligence 17:29 - Explorations in cognition 23:33 - Learning representations by back-propagating errors 29:58 - Dave Rumelhart and cognitive modeling 43:01 - Connectionism 1:05:54 - Geoffrey Hinton 1:07:49 - Learning in a neural network 1:24:42 - Mathematics & reality 1:31:50 - Modeling intelligence 1:42:28 - Noam Chomsky and linguistic cognition 1:56:49 - Advice for young people 2:07:56 - Psychiatry and exploring the mind 2:20:35 - Legacy 2:26:24 - Meaning of life

[@Mike-ig7cb]

Thank you for putting the time in creating this podcast.

[@ohFoou]

Stay humble my friends

[@JosePerez-pj4cn]

So he said there's no great overarching meaning just what each individual makes for himself, as bill Cosby once asked yes but what if you're asshole. Just kind of kidding, I like the intelligent design , simulation, theory better ,it explains the double slit, quantum entanglement observable experiments much better than the mathematical impossibility of rolling the dice and getting snake eyes billions of times in a row for evolution to actually have started , evolution for all it's hype is still just an unproveable old idea than can not predict any outcome, sorry lex just spilling ,hope you get Mrs Pasulka back she's one of the best interviews you have done ,so much to think about after she talks, candy for the mind

[@DoctorJaguarChannel]

Appreciate the timestamps Lex. Consider me a forever fan.

[@adamdiallo7142]

12:00 origin of intelligence: God (Allah)said to the angels he will create humans and place them on earth and then he taught them language: Remember, when your Lord said to the angels, “I am going to place a successive human authority on earth.” They asked Allah, “Will You place in it someone who will spread corruption there and shed blood while we glorify Your praises and proclaim Your holiness?” Allah responded, “I know what you do not know.” [al-Baqarah 2:30] Then Allah taught Adam the name of all things(isn’t that LANGUAGE📝?) Allah then asked the angels to name the things and they said Glory to you(Allah), we do not know, we only know that which you taught us(interpretation of the meaning of the verse) The angels realised that man is gifted and has ability to know the difference between good and evil, unlike other creations on earth like animals. That is why we are different from all other animals because we have better cognition. Allaah created Adam, and from him He created his wife, and from their progeny He created men and women, as He says (interpretation of the meaning): “O mankind! Be dutiful to your Lord, Who created you from a single person (Adam), and from him (Adam) He created his wife [Hawwa (Eve)], and from them both He created many men and women” [al-Nisaa’ 4:1] As for evolution, I think all the other human like creations were placed by Allah at different times in the world( they are not modern humans even though we have similar genes with other apes).Adam is the first Homo Sapien✅ All other human like creatures were not as smart as Homo sapiens. This is just my humble opinion, Allah knows best.

[@TRKTKO]

I hate to keep saying this but I feel it shows his growth. Lex is the new era Rogan, Joe's getting caught up in bs while Lex is still hungry for knowledge. He does it with respect, kindness and a passion that's intoxicating. I wish everyone had his manners too lol. Keep doing your thing Lex it's been a great journey so far and thanks for taking us along for the ride!

[@mardyross2838]

And it's not erratic, JR can be great and other days it's like being in a bar in the old days when smoking was allowed.

[@ricardoremar1922]

Why do you have to compare?

[@TRKTKO]

@@ricardoremar1922 I found Lex through Joe he talks about how Joe inspired him it's a compliment nothing more.

[@devilsolution9781]

I dont watch rogan anymore just lex

[@indianpapiii7214]

I have my Cognitive Neuropsychology exam on Tuesday and have been studying all weekend. This is perfect timing! Thank you Lex

[@indianpapiii7214]

@Bubb Thanks dude!

[@indianpapiii7214]

@@AndrewHibberd Shrooms? Gotta stay in this dimension for the exam unfortunately

[@peterm8527]

Hope it went well!

[@trevorsucksboatin8920]

Good luck buds!!!

[@dewboy13]

How'd the test go?

[@psychastheneia7]

I love how you went from "What's PDP" on your interview with Matt to speaking with Jay. Jay was one of the people that got it all right 40 years ago. Working with him was one of the biggest privileges of my life. He is by far the brightest person I've interacted with but also the most humble and giving

[@noahl.1003]

Thank you for your comment.

[@ravsuri7249]

Thank you for this fascinating conversation. Jay McClelland is one of the most thoughtful, most generous, and most genuine people I've ever met in my life. His thinking is generative and inspiring. His passion for ideas is infectious and his clarity of thought, in my experience, is unparalleled.

[@Narynan1]

Love getting in on new cognitive guests early. Looking forward to what this conversation is going to unlock!

[@themuslimview]

I love how this man is able to explain neural network and convince me how a neutral network of dumb neurons can create understanding. Gives me this inspiration to try to learn more

[@CAFCmatt55]

So good - Lex is always so prepared from start to finish and his care for producing a great conversation really shows. Truly great doing this 2-4 times a week. Keep it up man

[@mingwayh1532]

This episode is a real gem. I’ve come back to it again and again. Every time I found new meanings in it, intellectually and spiritually.

[@woodandwandco]

37:01 I don't think I've ever felt this excited as when this man relays this tiny tale of massive inspiration.

[@AlexCulturesThings]

I’m struck with this feeling that future me will reflect upon listening to this particular episode of the Lex Fridman Podcast, and finding it to be a favorable wind that steers the charted path.

[@DisasterxUs]

Best host, best guests, best topics, best questions... thanks for all you do!

[@raymundomartinez7469]

The random midnight uploads are the best!!

[@LikeAndFavBF3]

Hello Lex, could you maybe share a synthesis of all the information you gathered from your guests on the topic of consciousness over the years?

[@indianpapiii7214]

Love when he talks about his thoughts on consciousness haha. I could listen to it every podcast (as he does in most)

[@cogoid]

This was amazing. Jay McLelland has been a huge name in Cognitive Science since decades ago, and here he comes across as a very thoughtful and yet humble person. He takes time to think and formulate his thoughts beautifully. For example: 1:32:00 _"The insights that human mathematicians have had is a combination of the kind of the intuitive, kind of connectionist-like knowledge, that makes it so that something is just *obviously true*, so that you don't have to think about why it's true, that then makes it possible to take the next step and ponder and reason and figure out something that you previously didn't have that intuition about."_ _"It then ultimately becomes a part of the intuition that the next generation of mathematical thinkers have to ground their own thinking on, so that they can extend the ideas even further."_ It is also curious that Geoff Hinton apparently did not believe that backpropagation was going to work (1:11:08 _"Hinton was thinking that this wasn't going to work very well"_), even though he himself prompted Rumelhart to develop it (1:07:50 _"[Geoff said] you define your problem and then you figure out how the adjustment of the connection weights will solve the problem"_)

[@sankamohottala6778]

Thank you for this Lex!!

[@Luzeon]

Ur becoming my new long version podcast favorite…btw, how come your Spotify podcasts are only in audio and not video?

[@leedsdrumacademy]

That's on Spotify. Only 10 podcasts on Spotify have video, all the other... 1 million(?) have only audio.

[@robinampipparampil]

This is awesome. Thank you very much for this discussion.

[@4CarbideGaming]

Always enjoy watching these as i sit down for evening dinner or while im working round the house.

[@amandajrmoore3216]

Another riveting episode. Treasure these each week.

[@susanramonat7524]

One of your best, Lex! The last 1:15 minutes are a treasure.

[@giovannimacias212]

It's 420 somewhere

[@Solution_Seeker]

Douglas Hofstadter would make a nice guest!

[@PhillipRhodes]

That would be awesome! Lex had one of Doug's former students (Melanie Mitchell) on already, but it would be beyond amazing to have Hofstadter on the show.

[@williamjmccartan8879]

Wonderful guest, thank you Lex.

[@franciscocobosrodriguez7395]

Love your pod Lex, I hope you never stop.

[@chinmayjoshi4235]

Simply brilliant!

[@arvisz1871]

Very interesting to sequence an episode with Jay McClelland after Douglas Lenat. Looking forward for the next one 👍

[@guitaristdotcom]

Interesting. Also, you've got the intro down to 43 seconds. That's good. This is KZbin.

[@user-ol9rr3my3p]

Informative, thanks!

[@ObscurelyTitled]

A truly great intellect and interview

[@mimoochodom2684]

Can you have a show featuring Historian, Geopolitics expert and UFO researcher/opinionist Richard Dolan. He's a very articulate, pragmatic and knowledgeable person. Would be a great view.

[@Eta_Carinae__]

On emergentism Lex: I don't think it'd be emergentism per se. We don't exactly speak of knowledge as being 'emergent', even though it is a cognitive process - rather, we say knowledge is 'of something', yeah? I imagine we can speak of connectionism in like terms, as in: "there are connective processes, from which emerge (in a different sense) structures isomorphic to a concept plus error". The catch here is to throw our usual notions of hierarchical reduction into doubt, but that already happened in mid-20th century naturalistic philosophy anyway.

[@treyturner5056]

Thank You Lex

[@michaelknight6387]

I love how Lex just seems to get sleepier as time goes on. Just picking, love ya bud.

[@gesuuuuuu]

Everybody exploiting neural networks ; for profit or science ; is standing on the shoulders of giants ; like this brilliant human being (( you provided many great podcast(s) Lex ; this is one of my favorites besides the one you had with your Father [ ...... ] ) ...... ...... ......

[@drhilm]

Mind-blowing quote: "Formally trained academics ... have a mode of engagement that is intuitively deeply structured to be organized around systematicity, and the ability to conform with the principles of a system than is naturally true of the natural human mind without that immersion."

[@drhilm]

That sounds like something GPT3 would say. :-)

[@iranjackheelson]

@@drhilm What sorts of stuff GPT3 say that resembles this quote?

[@Josephus_vanDenElzen]

1:45:26 on the beginner's mind, and the specialist's blindspot

[@tomaspocius2223]

"for me personally i've always been excited about love and friendship between humans and just like the actual experience of it since i was a child just observing people around me and also been excited about robots and there's something in me that thinks i really would love to explore how those two things combine it doesn't make any sense" (Jay's laugh after says me that He knows more than He reveals)

[@renarspodcast]

Thoughts becomes things. We do think a lot about our things and how to get more of them, however, we do seem to neglect our thoughts.

[@CallmeBigfat]

Lex with that late upload

[@Infinitemoneycoin]

Very intriguing 🤔

[@lubitel666]

impressive listening skills here, dr. Lex....

[@kapilbharadwaj1210]

“From a robotics engineering perspective, it's very hard to build these systems. How can, through an undirected process can a complex thing be designed it seems wrong” - Lex There is an underlying presumption that just because we humans can think and reason, just because we can ask big questions, we should be able to reason complex questions. Why do you think that we humans are the final form of evolution? Dinosaurs had over 200 million years to evolve until they were destroyed, and we humans have just been evolving for a mere 6 million years. On another note, for the past 2 years, I have enjoyed all your ideas and conversations - podcasts, discord server, and Clubhouse. You and Eric Weinstien both have been a personal inspiration and really helped me stay motivated and reorient my goals during my time in grad school. Please keep what you’re doing. Stay Healthy. Stay Hard.

[@jstov]

The best channel on KZbin!

[@skipdonahughuk]

Looking a bit rough there lex 🤣 i like it you lurn alot from the edge cases 🙃👍

[@helendelia3140]

222 lets go

[@plsreleasethekraken]

His comments on beginner's mind and expert's blind spot echo those of Douglas Lenat's comment on more educated candidates being worse at ontological engineering.

[@bakeralalwani7890]

bring Dr. Stuart Hameroff please!

[@Ataman997]

Interesting topic contains movie - Quest for Fire, how fire and love emerges, there it have based in understanding and making fire

[@jeffbertjeffbertson4805]

I wonder if Dave Rumelhart was the professor of logic over at the University of Science that Norm MacDonald was talking about 🤔

[@rileykatipa7474]

Beautiful

[@ttfoley8127]

Jay is definitely an extra terrestrial life!

[@andrewroberthook3310]

You live within an electrical field of progressive and sequential potential That being the case Consider it the media And consider yourself as the influenced result of media action Simple and profoundly complex at once

[@internetguy1332]

woahhhhhh here in the first minute from upload

[@factanonverba7547]

We need to be cognizant of our cognition.

[@carlosgray3107]

Complexity out of simplicity ... you've seen it before

[@illitaret8780]

The fundamental difference in your reference is that we can explain the before examples, while we can not currently explain consciousness to a satisfactory degree.

[@hooray_for_sam]

🙌🏼🙌🏼

[@govindagovindaji4662]

1:00:05 Who ever saw a palm tree near a pyramid~? I've only seen pyramids out all alone in the stark desert, myself. (At least in pictures; I've never seen one in person).

[@luisfernandotorresada1006]

Lex, natural selection is not a random process. That is, evolution takes place through the selection of a phenotype by the environment (the ones that survive), thus reducing the biological variance from which it was generated. That’s why Darwin named it Natural Selection, because it’s not a random process.

[@iranjackheelson]

But given that we don't know how that selection process works exactly, the process is still "random" epistemically speaking, wouldn't you say? in other words, sure there are precise mechanisms going on as a result of interaction of environment and the organism but if we don't fully understand that, that space of unknown is effectively random to us--it speaks to our own ignorance about the world rather than the actual physical process like quantum probability (but even this is debatable as to whether the randomness belongs to our own perception or the external world)

[@ghostlack]

Yes, I know. Go on.

[@danielpaulson8838]

The brain is an amazing thing. The highest complexity in biology that we’re aware of, and still evolving. Who knows what the future holds for its direction.

[@dancingdog2790]

Jay is seriously buff! I hope I age as well as he has.

[@RWHsuzuki44]

Maybe have Paul Smolensky on

[@chidalunwaimo876]

Why has nassim taleb not on lex podcast upvote let's have him on

[@theresameuse8583]

catholic Chritians can believe in evolution (instead of "design")

[@kagazuki]

What does he think about Neuralink?

[@primitivegolfer6692]

Computational "neural networks" shouldn't even be in the same discussion of true neural networks since the term is misleading and unequivical.

[@primitivegolfer6692]

@Lex Fridman▪️ pfff

[@christophercotter166]

is'nt it possible we are devolving into a less superior being we were a 10 and now we are a 4 and going down?

[@publicdomain1103]

Zero's are a billionarires best friend.

[@ivankuljis1780]

Lex, one small question. Are you going to Mars?

[@chrisshupe985]

Caveman Lex

[@lesliewilliams1214]

A riddle within a riddle ohhh how can man still be what he was destined to be even so after regeneration restored him too the almighty a whisper came along and said we can make him in our image so therefore his mirror is no longer him a shell of what he once was wgi can restore him back to himself when he can no longer see the me that was he.The day of the singularity is no rhyme or reason, it is a jealousy

[@kingklank6732]

Cool

[@Dan-mm1yl]

animals have language

[@bmelon18]

2nd!!

[@bmelon18]

@Lex Fridman▪️ Live long & prosper my friend! 🖖

[@Wojak369]

stop dodging Dave Weiss

[@effoffutube]

chronocentrism, the cause of, and solution to..

[@jaygage1555]

I loved this interview BUT don't take this the wrong way not trying to come off the way it'll probably come off lol but alot of it seems like "you want to be happy" come from a family that has some $ and able to nurture your interests as a child but sadly a hole hell of a lot people don't have that luxury...

[@JPJelly]

First

[@zxLoneWolf]

Noice

[@jaysphilosophy1951]

Of course, Lex is dressed in the last suit he'll ever need...

[@w4bbitseezin]

he describes Bitcoin perfectly

[@sawcbee]

Cognitive psychology knew cognition was biological, but considered it a black box, similar to neural networks.

[@tomwright9904]

I'm not sure that's quite true. Cognitive psychology tries to build models of cognition with moving parts.

[@sawcbee]

@@tomwright9904 originally they were black boxers

[@morgangreen2601]

🍑

[@mrcardinal7548]

🍑🍆

[@vincentpadgham8613]

Do

[@hacurass7159]

Why am I such a bisexual creqture

[@billlets5460]

No one seems to understand: The direction of current Neuro Net development, AI and cognitive science all build outwardly--away from the core existential expression. This causes a circumference expansion. You must go the other way to get at the center of the light called mind. Answer: Develop these things from the point of view of why anything at all should exist. Not why we are here but rather why does anything exist at all--science is impotent on this. These discussions go nowhere because they are engaged in blind searching--literally groping in the dark. Think first principles. So the nature of direction should be fundamental--and besides, direction is necessary for NN, AI, CS etc.

[@XMyOpinionX]

69th like 😂👌🏼

[@skynet4496]

Lex, your microphones/mixing suck. Guest sounded clearer but still not too clear, very sss like a zoom call. Some of this explains MASS FORMATION (HYPNOSIS)! Funny you bring up chomsky, he was anti establishment, but somehow trusts the government when it comes to jfk, 911, covid. He recently promoted apartheid of the unvaxxed... Same with Chris hedges... This linguistic mind reminds me of the lecture on KZbin: "alphabet vs the goddess" where written language lead humanity to become more chaotic. It's probably why religions (and "science" today) are ok with doing illogical things, because the meaning is twisted in favor of the group think.

[@eanerickson8915]

This guy has stuff to say. 151k views. What does that say about America?

[@crabofchaos7881]

I don't get why they would let people put their rats THERE right on campus

[@taly4life]

Is Lex getting enough sleep? Looking sloppy with hair, a little slurring of the words. Dude is looking rough.

[@tomwright9904]

He consistently pulls all nighters according to his comments

[@mbhinkle]

I will stick with The Lord God of Israel's explanation of how we were created. This guy's answers are hollow.

[@hatchlingone]

I’ve been listening to Bernardo Kastrup. When I listen to this, it reminds me of listening to absolutely nothing. It’s like you are rambling on and on without saying anything meaningful or believable. There is no hard problem of consciousness because of what.. magic. It’s time for a scientific revolution. These lines of thinking are antiquated, boring, depressing and repulsive. Without consciousness, there is no universe. Emergence is nothing more than an appeal to magic, and even this scientist knows for a fact that the process of emergence can never explain consciousness. Yet he clings to it.

[@ZenShaman]

i would like to know why you guys keep researching and enacting such amazing technology that will only be used to torture and enslave human beings . what excuse can you possibly come up with that will excuse your complicity in blatant crimes against humanity. i await your answer.

[@electrotech2253]

So he talks about how everything is connected for 2 hours and 29 minutes then concludes at the end the meaning of life is individual. ok got it.

[@d.lav.2198]

That's the Human Condition my friend.

[@jgiomaxjgio7263]

So annoying These pauses and stops he make in between words You can fit in one more podcast here Couldn't listen for more than 10 minutes

[@hyperbolicandivote]

Really? This is the best podcast around.

[@idobbs737]

Derpderp

[@junehodsdon8037]

Oh brother...these type of comments make me sad for all the folks that miss some really great talks because they're impatient and/or unable to focus.

[@tomwright9904]

Double speed...

[@tomwright9904]

I realized before I couldn't listen to zizek at normal speed, but could at high speed

[@burkebaby]

I have my Cognitive Neuropsychology exam on Tuesday and have been studying all weekend. This is perfect timing! Thank you Lex

[@renarspodcast]

Thoughts becomes things. We do think a lot about our things and how to get more of them, however, we do seem to neglect our thoughts.