John Hopfield: Physics View of the Mind and Neurobiology | Lex Fridman Podcast #76

John Hopfield is professor at Princeton, whose life's work weaved beautifully through biology, chemistry, neuroscience, and physics. Most crucially, he saw the messy world of biology through the piercing eyes of a physicist. He is perhaps best known for his work on associate neural networks, now known as Hopfield networks that were one of the early ideas that catalyzed the development of the modern field of deep learning.
EPISODE LINKS:
Now What? article: bit.ly/3843LeU
John wikipedia: en.wikipedia.org/wiki/John_Ho...
Books mentioned:
- Einstein's Dreams: amzn.to/2PBa96X
- Mind is Flat: amzn.to/2I3YB84
This episode is presented by Cash App. Download it & use code "LexPodcast":
Cash App (App Store): apple.co/2sPrUHe
Cash App (Google Play): bit.ly/2MlvP5w
PODCAST INFO:
Podcast website:
lexfridman.com/podcast
Apple Podcasts:
apple.co/2lwqZIr
Spotify:
spoti.fi/2nEwCF8
RSS:
lexfridman.com/feed/podcast/
Full episodes playlist:
• Lex Fridman Podcast
Clips playlist:
• Lex Fridman Podcast Clips
OUTLINE:
0:00 - Introduction
2:35 - Difference between biological and artificial neural networks
8:49 - Adaptation
13:45 - Physics view of the mind
23:03 - Hopfield networks and associative memory
35:22 - Boltzmann machines
37:29 - Learning
39:53 - Consciousness
48:45 - Attractor networks and dynamical systems
53:14 - How do we build intelligent systems?
57:11 - Deep thinking as the way to arrive at breakthroughs
59:12 - Brain-computer interfaces
1:06:10 - Mortality
1:08:12 - Meaning of life
CONNECT:
- Subscribe to this KZbin channel
- Twitter: / lexfridman
- LinkedIn: / lexfridman
- Facebook: / lexfridmanpage
- Instagram: / lexfridman
- Medium: / lexfridman
- Support on Patreon: / lexfridman

[@lexfridman]

I really enjoyed this conversation with John. Here's the outline: 0:00 - Introduction 2:35 - Difference between biological and artificial neural networks 8:49 - Adaptation 13:45 - Physics view of the mind 23:03 - Hopfield networks and associative memory 35:22 - Boltzmann machines 37:29 - Learning 39:53 - Consciousness 48:45 - Attractor networks and dynamical systems 53:14 - How do we build intelligent systems? 57:11 - Deep thinking as the way to arrive at breakthroughs 59:12 - Brain-computer interfaces 1:06:10 - Mortality 1:08:12 - Meaning of life

[@wizardstein3153]

is there a place to get a transcript of the conversation?

[@maxlieberman578]

Bring David Deutsch please! :)

[@robertodiaz7645]

Lex, could you add subtitles for the hearing impaired people?

[@lsfhieber]

Lex Fridman to Interview this man is a great honor. I have never seen someone describe the loneliest place so perfect!

[@alfonsoreyes911]

@@wizardstein3153 www.linkedin.com/pulse/transcript-interview-john-hopfield-lex-fridman-alfonso-r-reyes/?published=t github.com/f0nzie/transcript_interview_john_hopfield_by_lex_fridman

[@RavenJack23]

I once called Dr. Hopfield uninvited as an undergrad at Oklahoma State University. Dr. Hopfield was at CalTech at the time I think. Dr. Hopfied was very gracious and talked to me for a good 30 minutes explaining his work carefully, focusing on an aspect of his papers that was not making sense to me at the time. I am a life long fan.

[@Stwinky]

Lex’s laugh when Dr. Hopfield responded with “there’s a question of what do you mean by understand” :’)

[@edwardstrinden]

Listening to these conversations gives me a deep ache in my chest, and brings tears to my eyes. The purity of yearning for the knowledge of the yearning for knowledge is such a beautiful thing.

[@sortof3337]

True. People around me think I am weird for listening to these videos.

[@Adam-st8ys]

Nerd

[@edwardstrinden]

@@Adam-st8ys haha, yeah

[@s.chaubey8098]

@@eddieharding6788 do you know him/her personally?

[@eddieharding6788]

@@s.chaubey8098 nah, idk what I was talking about haha. This happens every so often, I get a reply to an old comment I made and I have no clue why tf I made it lol

[@NadjaNiemann]

.... you are such a lucky man to meet all these people and have such long, deep conversations with them. (Yes, I know, you are working super hard to get 'so lucky'...!) - You are so young and when you keep on doing this for the next decades you will turn into a legend yourself... (Silly me thought 'Lex-Icon of AI' ...)

[@s.chaubey8098]

Indeed.

[@abyteuser6297]

17.37 "but the rhythms what??" Damn Hopfield goes #savage on Lex... what amazing insights ... one of the best podcasts. Thanks!

[@EtienneCharlier]

Wonderful to listen to John Hopfield. I had implemented his Hopfield network on silicon in 1988 🙂. I moved to other things but kept a deep interest in brain biology and artificial intelligence.

[@robinj12345]

Took me a while to get into this one, but again very good indeed. I like the quote from Marvin Minski that "consciousness is basically overrated.... all the hard computations are done non-consciously" and "consciousness is your effort to explain to yourself what you have already done" This is very topical because I just was listening to an analyst saying Tesla would not get full autonomy because AI cannot do the sort reasoning we do - I think this is completely wrong - we drive our cars unconsciously most of the time.

[@Zeke-Z]

Lex, I can't thank you enough nor tell you how amazingly educational and useful your podcast has become in my life. Thank you for everything you do, have done and will do. This is a field in which I'm truly passionate about and the information in this conversation is priceless.

[@ErnieRMC]

Thank you so much Lex. I had a lot of these thoughts as a high school seniors. Now I’m 69. Just got me all excited. And I’m on lesson two of deep learning Ai Andrew Ng class thanks to you.

[@MiroslawHorbal]

99% of my brain: a great conversation, great to hear from the man himself and grateful to lex for the content 1% of my brain: this dude's tongue has a mind of its own

[@travisglazebrook3654]

This was a lovely conversation. JH's analogy about the adaptive speedometer was so simple and so profound.

[@afroman91794]

I really love how you bring these diverse brilliant conversation to everyday people!

[@johnnypeck]

This is very much amongst my favorite discussions you've had. So much honest wisdom.

[@kylegushue]

Another killer podcast! Thanks.

[@74LS_NE555]

Absolutely amazing interview, thank you

[@jonascampos1895]

Lex! Watched you a couple of times at Roagans channel! After you expressed your intent behind building your business I got interested in your work. I am now mesmerised by the quality of questions you directed to John. Inspiring, intriguing and humbling. I will be back for sure. Keep it up brother

[@Alistair_Spence]

Thanks for making these kinds of videos. Fascinating stuff.

[@SKARTHIKSELVAN]

You ask good questions. Thanks for your podcast.

[@Pretzik]

I love your podcast Lex. I’m a huge fan. Your conversations are just amazing. Keep up the great work.

[@hanselpedia]

I really liked this deep slo-mo interview! Gives thoughts room for wandering... Thanks Lex!

[@agt5jx87]

Mr. Hopfield has a beautiful and elegant mind. This was such a pleasure to watch. Thank you Lex.

[@DejanBasic]

What a podcast, thank you Lex!

[@vpdabholkar]

Enjoyed this interview. The question "What does it mean to understand something?" which Hopfield asks multiple times during this interview is my key takeaway. Thanks, Lex.

[@forfreedomssake4315]

Thank you man

[@os2171]

Beautiful interview. I’m a neuroscientist neuroethologist and yesterday I was teaching Hopfield landscapes to my psychology and medicine students… next semester I’ll include some of his appreciations from your conversations… for me personally, it gives me motivation to pursue that Postdoc; I’ll be defending my PhD thesis within a month on multimodal integration and neuromodulation in bees

[@nigelstafford635]

The best of his interviews I’ve seen

[@FromFame]

I dont ever feel like commenting as I enjoy your podcasts and that's all I'm here for. But your podcasts are special. We've built an economy of content but not enough context. It's important that these interviews have highlighted not only the search for purpose but their ability to be actionable about it. I hope these podcasts inspire the general public one day.

[@samuelec]

Thank you both. I've great respect for Hopfield. Lex you are doing very good job you can be proud of it and also very lucky. Keep doing!

[@newenglandbarbell4647]

Lex you are on fire 🔥 with these podcasts👌 Thank you so much 🙏

[@ForrestNeal]

I hope I still have my whits about me when I'm 87 like this guy!

[@simonstrandgaard5503]

Great guy. Please interview John Hopfield again.

[@devrim-oguz]

Thank you Hopfield, it was all a coherent sense to me throughout the video.

[@lorin-gabriellauran2559]

Lex "Let's linger a bit on that" Friedman Lex "Beautiful" Friedman

[@NicoA47]

*Fridman

[@timothykalamaros2954]

Great work Lex, thank you 🙏🏻

[@THash-qs5qg]

This was a wonderful interview - Hopfield is a great thinker/scientist and Lex, you asked some really good questions!

[@rodrigo_siqueira]

Incredible insights on neural dynamics and how understanding of dynamical systems can give new ideas to improve artificial neural networks. It's good to learn a bit about "neural coding" and "chaotic attractors" before listening.

[@88chinchin88]

such an inspiring talk .. kudos to both of you.

[@imagreatguy1250]

Single best channel on yt I'd recommend to everyone in the world right here. Your work is amazing Lex.

[@Axcellaful]

Maybe it's just me but I'm enormously jealous Lex. These guests are incredible.

[@RalphDratman]

The "mystery of consciousness" was discussed, albeit indirectly, by Ludwig Wittgenstein in his posthumously published book Philosophical Investigations. I spent many hours, night after night, when I was 17 years old, reading that book line by line. The book is very difficult to understand, partly because Wittgenstein's language was German, which I do not speak or read, and also because Wittgenstein's manner of expressing his ideas was odd and seemingly archaic. Wittgenstein did not discuss consciousness directly in anything I have read, but the content of his work bears strongly on the present discussion. His attitude toward the question, "What is consciousness?" might have been that there is no point discussing something we cannot even define. If you describe a world In which human beings do everything of which we are capable, without mentioning consciousness, what would be lost or incorrect as a result? And the only answer would seem to be that nothing was lost.

[@muhokutan4772]

I'm a simple man, I see John I click like.

[@androidsdream9349]

Excellent, thank you Lex for interviewing Hopfield. Many insights on NLD and its omissions in the ANN world. I wonder if Hopfield has read “Beyond Boundaries” and the research work of Miguel Nicolelis. It sounded like he hadn’t and that you haven’t. Nicolelis has done some beautiful work on measuring collective phase locking behavior of actual networks of various sizes. I realize his research is controversial, since the probes are invasive and the test subjects are laboratory animals. There is no doubt in my mind that the neuroscience world would benefit from improved diagnostic test methods, non-invasive if possible. FMRI is too qualitative and disconnected with the actual NLD.

[@freasy123]

My favorite episode so far

[@tiadiad]

Loved this!

[@veltzerdoron]

Amazed at this Hopfield interview, If Hinton isn't returning your calls you should ask Mike Spivey.

[@carlt2023]

As someone else wrote, wonderful indeed to listen to J. Hopfield. But I was most impressed with Lex's talent as an "interviewer". While J.H. is definately still "all there" in terms of his intellect, etc (my opinion) ... many "mature" scientists provide explanations that leave a fair amount of "white space", perhaps b/c the nuances of their ideas & discoveries start to seem trivial (to them) over time. Lex brilliantly keeps the flow in this conversation with a great scientist/human by steering the conversation in the subtle manner that befits speaking with a brilliant mind that prefers to take longer "strides" with respect to sharing his insights & knowledge. Simply put, Lex prompts JH to connect certain dots but allows for a certain amount of "drift" ... perhaps out of respect, deference, love. That talent (on the part of Lex) is uplifting (and effective). Bottom line ... Lex has orchestrated a wonderful journey into a great mind & person ... yet again. Thx Lex

[@crassflam8830]

The rhythms... Look at central pattern generator circuits in the spine and how they solve the motor control problem...

[@spencerreppe7558]

New favorite KZbinr.

[@mtumasz]

Well done both, thnx

[@marktomasetti8642]

There seemed to be a grinding of gears going on for these two not far under the surface. Lex seemed impatient at times. Not like I’ve seen him in other interviews. Perhaps the difficulties JH had getting some of the words out made the pace difficult for Lex. The analogies that JH was drawing were well worth the wait - utterly brilliant guy. He seemed able to express the gist of a lifetime of insights on how to think clearly about nature and science into short pithy comparisons. JH joked at the end that Lex might have difficulty getting 5 minutes of coherent conversation out of the recording. It all seemed very sensical. Abstract at times but deeply insightful.

[@fixedoddsbetting]

Some good analogies used in this discussion. Interesting to hear how models to approximate how the brain works are evolving.

[@jjurksztowicz]

Great episode! Please have Alan Kay on!

[@damianmurray5290]

Brilliant interview. You should have Professor Leroy Hulsey of UAF on at some point.

[@puya6039]

Very nice work. Could you please also interview synthetic biologists?

[@rickharold7884]

So excellent! Thanks lex

[@abhalla]

An amazing man! I wonder what he thinks of hierarchical temporal networks

[@rabusmccaleb7944]

Thank You

[@lanzadelas2572]

Thanks for it

[@maxlee3838]

I still have quite a bit of hope for approaches like Numenta’s HTM theory.

[@absolute___zero]

well, this HTM theory didn't pick up. Same as it didn't pick up for Imagination Engines with its STANNOs developed by Steven Thaler. Makes me think all these folks who "reinvent the wheel" are just marketing gradient descent or similar another well known techniques under their own name, and get banned by AI community. One vivid example of such a ban is "Extreme Machine Learning" which is nothing more than random projection and a linear regression on top of it.

[@maxlee3838]

Absolute Zero I don’t think you understand how HTM works. No gradient descent at all. Go learn about instead of armchair criticizing.

[@breehogg2878]

Please, please interview Barbara Arrowsmith-Young who wrote ‘the woman who changed her brain’. Also a conversation between Barbara Arrowsmith -Young and John Hopfield would be fascinating and enlightening. They could perhaps lead you to your starting point in your new AI project…

[@tsraikage]

loved this podcast. its not everyday Vision from MCU comes to explain how human brain works

[@alfonsoreyes911]

I added a transcript to the interview in LinkedIn and GitHub: LinkedIn: www.linkedin.com/posts/alfonsorreyes_artificialintelligence-physics-neuralnetworks-activity-6655541541651734528-ScRj GitHub: github.com/f0nzie/transcript_interview_john_hopfield_by_lex_fridman

[@273degreeskelvin]

What? No white shirt? I barely recognized you. I'm looking forward to an experimenter's explanation of how to implement emergent properties in computer simulations of neurobiology. Excellent podcast as always.

[@lastpodcastbeforetheworlde4140]

Very nice

[@claudiabl5787]

Now that I have been listening to your podcasts, the chats are serving as a brain mechanism, the propeller of my discourse, a response mechanism of these words, the other day, last year!, just came to my mind that people are like rocks. In the sense that brains, neurology people act always the same, they have to be tought, trained, and seems they never learn well, we came back everyday to do the same, even though it seems really different for all the days, years in our lives. Analyzing this past sentence: this has some of Descartes rationalism, blank slate, maybe that's where the idea comes from: people are like rocks/slates, as I am introspecting, past knowledge, to understand where I would like to go. Knowing my human experience, existensialism or meta-existence has limits. My learned knowledge in phisics is null, which is impossible, but true is that I have no conscience of phisics in my life, I cook, I go shopping, on the bus, train, subway, but phisics never crosses my mind. I have to get some fundamental book in phisics, I forgot everything from high school phisics! Thank You both 😀

[@truthseeker2275]

Wish I could upvote 10x

[@ozachar]

That was very interesting

[@ElMahdiELMHAMDI]

one of the most overdue Nobel prizes in physics (the other overdue one being Alain Aspect's)

[@afriedli]

That's a smart guy!

[@ewncilo]

Can you interview linus toldvard.?

[@janko6637]

My mind was kinda blow when he said consciousness was just for observing thoughts you all ready had Makes it seem even more useless like is just a hub to connect shit together? Zoom in zoom out Zoom in zoom out

[@sgerdt]

Of various intelligence, what's it like? For me I feel at times as a passenger mediating thoughts. So that leads me to feel this consciousness thing is fleetingly thin, but still enough to obviously affect the outcome by sheer focusing of will or repetition. Are we receptacles in perhapd a biological simulation with determined randomness or do we have a spark of something else, a "curious recursion" as Ramachandran postulated long ago..?

[@Japakak]

If you would like to know more, try Marvin Minsky - The Emotion Machine

[@juanaq]

@@Japakak i'm fascinated this days with "The Deep History of Ourselves: The Four-Billion-Year Story of How We Got Conscious Brains" by Joseph LeDoux, on how consciousness may have evolved, and on what consciousness is from a completely rational point of view. i'm a complete layman on the subject, but the book is amazing.

[@Japakak]

@@juanaq It looks really interesting, thanks.

[@hanselpedia]

Not useless at all: consciousness might be the top-level feedback loop of the brain?

[@giuseppe1926]

I don't know if my memory is caught in a loop (and I'm weaving my own narrative) or the John Dean example had already been used by another guest in the podcast

[@ganeshvaidyanathan8355]

Very profound discussion. Hopfields observation that he cannot yet find a smoking gun that links physical systems to consciousness rings true to me - how do electronic signals in the form of action potentials make a leap into a sense of “I”?

[@tsuchiman]

I want to know why the watermark for "superfluous imagination" is in the subject of narrative making? What purpose would a complex biological system have for narrative, given that nearly every species known does not require this sense/ability?

[@adammobile7149]

1.5 playback speed is the best for this video

[@joeo5533]

55:45 The population of the training set is linked to things outside itself, in the human mind, but not in NNs?

[@jakobfriedl3098]

Im a 5th grade high school studet from austria... as final thesis i wrote an EKF SLAM algorithm with LiDAR and RANSAC... dont know what to do with that knowledge tho xD (Study/Work) HELP

[@Snezana81]

What about Deinococcus Radiodurans?

[@kobiromano6115]

18:12 ... The rhythms are utterly absent from anything that goes on in Google... - Yeah but the rhythms... - But the rhythms WHAT? ------------------------------- When you know you've been SCHOOLED

[@why_do_you_care]

*set life goal to* -- Navier Stokes equations for the brain!!

[@universalbasicoutcome941]

I infer that if a living system is a seemingly messy coalescence of physics, biology, chemistry, computation and what not, any semblance of true artificial general intelligence may well require an unified application of natural general intelligence.

[@alastairbowie]

I feel as though before strong AI comes to be (if ever) we will get increasingly more intelligently designed and implemented software that could perhaps eventually lead to the emergence of AI related technologies that open up potential possibilities towards strong AI. Just my 2 cents xP

[@TheFacelesssoul]

The name in the book. Neural Network students will know...

[@golagaz]

Modern Chips are 3D as well. en.wikipedia.org/wiki/Three-dimensional_integrated_circuit

[@derrickk773]

The more you know.

[@anastasiajj9399]

It's the first time I've seen you, Lex, obviously bored (and I've watched most of your videos, I'm a regular watcher).

[@irvpaton8626]

Lex,there is no way to contact you anywhere. Big tech has blocked you from humninty.I enjoy your work, bbuywant to get mesaage to you.Any suggestions?

[@jackiekunovska8405]

So the alternative is to link the electronics with the brain....aiming to increase efficiency more and more

[@BEATmyguest31]

WOW ANOTHER LEX FRIDMAN. I never once looked him up and every time its recommend i say to stop & click dislike. WTF is themtube doing? I mean, besides steering me away from info i find important from genuine people that care & replace it with nonsense from a machine sounding lame.

[@jackiekunovska8405]

One can see the brain as billions of continuously reconfigurating circuits,...the base is doable electronically...I believe...but the dynamics probably impossible to mimic......

[@GiullarediDio]

These scientists look a bit bemused when you throw around words like 'beautiful' (which sounds a bit capital-R Romantic): notice how Hopfield toned it down to the more neutral 'interesting'.

[@angelatakano6072]

A robot will never be able to evoke the scent of the perfume I was wearing when I thought I was falling in love ( I was 18 ys old) and then memories play like a movie on my mental screen and that automatically brings a smile to my face and a warm feeling in my heart ....... and all appears very smoothly......... Do you think a robot could achieve this just by activating neural networks? No, there is something definitely inherent to human consciousness that machine consciousness does not have, at least not now, but I do t think it will ever achieve: the advantage of being human !

[@sahil5151]

13:23 Sounds like Dropout lol

[@zeytelaloi]

Regarding the example with the ball rolling onto street and knowing that there's a child coming behind it, if I were an alien new to Earth and had no idea that "balls" were "toys" played by "human children" I too would not be able to know that that a child could be rushing out behind it. It's only because I am human, grew up playing balls, grew up seeing balls, grew up knowing that kids play in residential areas and was even taught during my driving lessons to always "keep a distance of safety" that I would be able to figure it out and stop the car. So maybe I as a human am not that smart either?

[@ephraimtheanglo2929]

How come neither of you have considered that the mind may be immaterial? You're talking as if you are Thales or Anaximander, and have yet to see Plato and Aristotle.

[@jackiekunovska8405]

The necessity is magnified when the human is about to venture into hostile environments beyond earth

[@bilzebor8457]

I usually enjoy your podcast, but despite Lex asking good question and being patient, the guest seemed borderline senile, eventhough he would have been an excelent guest had he been 30 years younger... most of what he said felt empty, and kinda "trolly" like Feynman but without the wit

[@breehogg2878]

What is it to understand something? Lex is going to be a busy man.

[@clray123]

You can see that the interviewee is embarrassingly smarter than the interviewer.