Neural networks are an alien artifact | Andrej Karpathy and Lex Fridman

Lex Fridman Podcast full episode: • Andrej Karpathy: Tesla...
Please support this podcast by checking out our sponsors:
- Eight Sleep: www.eightsleep.com/lex to get special savings
- BetterHelp: betterhelp.com/lex to get 10% off
- Fundrise: fundrise.com/lex
- Athletic Greens: athleticgreens.com/lex to get 1 month of fish oil
GUEST BIO:
Andrej Karpathy is a legendary AI researcher, engineer, and educator. He's the former director of AI at Tesla, a founding member of OpenAI, and an educator at Stanford.
PODCAST INFO:
Podcast website: lexfridman.com/podcast
Apple Podcasts: apple.co/2lwqZIr
Spotify: spoti.fi/2nEwCF8
RSS: lexfridman.com/feed/podcast/
Full episodes playlist: • Lex Fridman Podcast
Clips playlist: • Lex Fridman Podcast Clips
SOCIAL:
- Twitter: / lexfridman
- LinkedIn: / lexfridman
- Facebook: / lexfridman
- Instagram: / lexfridman
- Medium: / lexfridman
- Reddit: / lexfridman
- Support on Patreon: / lexfridman

[@LexClips]

Full podcast episode: kzbin.info/www/bejne/mZXMdWBvgrKjmJI Lex Fridman podcast channel: kzbin.info Guest bio: Andrej Karpathy is a legendary AI researcher, engineer, and educator. He's the former director of AI at Tesla, a founding member of OpenAI, and an educator at Stanford.

[@alflud]

I'd like to talk to this guy about deja vu, Lex too. There are aspects to 'memory' that both of them are neglecting to consider in all of this and in these non-physical realms we find the most pertinent answers with regards to the origins of life. I don't need to write a paper, I can sit down with anyone and the mere mention of the words 'deja vu' are understood immediately. _Nobody_ can deny it and yet - nobody can prove it either. Is it a memory? We know what it is - this sense of 'I've seen this before' but it's not that we saw it, we _experienced_ it. Twice. Explain that. In fact, skip that. The main thing about memory isn't that _all_ memories throughout _time_ already exist it's that we're not the only things that experience the universe and remember it, we're not the only things making memories. The pre-Cambrian explosion of life on this planet was an explosion of memory - patterns the universe had built up through evolution up to that point. When _through the process of evolution_ a working pattern is found it's remembered - patterns of life are _stored_ and updated as they adapt to their environments. This _is_ the process of evolution at it's core. The patterns Mike Levin spoke about, they're not stored in the dna, it's somewhere else in the fabric of the universe, somewhere _outside_ of the life-form. Somewhere outside of time even. All patterns for life everywhere are stored and as soon as it's viable, as soon as a planet - or some other body for all we know - as soon as it's possible and wherever it's possible for that most basic of life-form to exist the patterns the universe has 'stored' up to that point are 'seeded' somehow and from that point on whatever makes it makes it - so to speak. From that point evolution takes hold ... again. Evolution couldn't happen without _memory_ - that's a given, but we fail to consider that these memories exist independent of their creator(s). These 'explosions' of life/memory happen on every planet everywhere. Let's face it, if there's life out there _anywhere_ then it's everywhere. Why would it not be? If it _were_ the case then the important thing to recognize is a mechanism by which the universe does this 'seeding' of life. How exactly do these explosions of life/patterns manifest? I feel that if we can get to the root of that then we'll be much closer to understanding the origins of life. We have to take memory into consideration here. Memory is an essential aspect of the universe yet it's often overlooked. They aren't fleeting things. Even if one is struck down with dementia and can't remember their own name, everything they ever experienced, every memory they ever made still exists. Such is the nature of a memory. Experiences don't go away and aren't dependent upon biology in any way, shape or form. They're completely _independent_ from biology and at the moment utterly untouchable by anything in our science toolbag. There's no way to 'prove' any of this. All we have is 'conjecture' so to speak. But ... is that true? We all _know_ what deja vu is - no conjecturing needed. We can't explain it but we all _know_ what it is. It's been part of everyone's experience. Not many can deny it, not many do deny it - it widely accepted, nobody calls you a nutcase for suggesting you saw the future somehow, do they? Everyone nods and says it happened them too.

[@johnanon658]

Yo lex, do you go to the same temple as that sam bankmann friedmann?

[@drewarnold6741]

I honestly don't think the code is in the DNA... per se. I think the DNA is more analogous to a config file and there is more happening that we don't understand.

[@sgttomas]

Nice

[@heresjohnny602]

What's the evidence for your belief.

[@Cyborous]

Check out Chris lehto and his video collective intelligence in cells and galaxies

[@heresjohnny602]

@@Cyborous Who's he ?

[@TheMack]

@@Cyborous Exactly my thoughts. That is a very interesting video indeed.

[@family26ful]

Hey Lex you all ready a Legend and you barely look old to drink! Ty for sharing your journey w all of us on your path seeking knowledge Ive got a grip though. I’ve noticed when you an jr post artwork you don’t give a Link to the artist I think it would be hugely beneficial for the artist but also your fans Home decor and psychedelic explorations Bless

[@archieforde3324]

Hey Lex. Great show. It would be nice if we could incorporate "agents" in our models. Do you think that ,may be, possibly, sometime in the future this would be possible. Is there anyone working towards this multi-agent approach? What would it look like in detail?

[@larsnystrom6698]

Aren't there already Agents in the model? Each car or person the Tesla FSD sees is an Agent with its own predicted performance.

[@Andrew_JTD]

Every podcast gets better and better in terms of material and conversations. Lex is stepping up his game and it’s great.

[@splooshy4014]

Do you include Kanye in that?

[@TheGavalanche]

You mean this guy is more intelligible than YE?!? Get outta here...

[@GodofStories]

This subject is what Lex is working on. So, it's more in-depth as it's a talk between 2 machine learning engineers

[@steveshank9674]

I completely agree. I hope he surpasses Joe Rogan

[@carrillospeed]

The invention of semi conductor transistors was truly remarkable

[@Butcherbg]

Uhh... Yo... I kind of liked the talk you had there... Especially the AI guy said something that made me happy about optimization of information handling or "compression" Because erm... it`s very hard to tell which expands into what and which compresses into what and that if seen trough different "angles" sometimes expansion does not as a practical effect work as compression and vice verca and if so, how does it all really work and can it be "explained" to like whatever cascade of CPU, data storage units are necessary/possible. How does it "Judge" when it reaches "useful" and when it strays from it... The rulesets and the goals themselves could be massive archives ~ or not so large at all.. and ultimately how it will be something different than just one more ~person. But that is not what I actually cared about, I just wanted to type it. I wanna pounder on the "how"... It`s not "that much of a big deal" what is called "natural selection" is like actually super, super, super strained >competition< for everything... Every bit of change that happened to something living somewhere trough the generations was necessity for improvement towards the goal of exist and spread (like intelligence can spread life as a thing outside of where it began, eventually)... But "the leap"... well... outside of everything I've said... Wasn`t there this named female monkey that due to natural radiation background was born with augmented larger cranial and out of her offspring everything up to date intelligent tier began? I think that is what they teach in Medical fields... And the counter example of this is Bees which Hive`s wax coating act like natural background radiation shields so they are some of the most 1:1 as when they began species that exists... or something. Peace, bye.

[@kgill99]

Q 'Most interesting invention?' Think it has to be the slightly prosaic 'farming'. Why? All creatures are alive (we all get here) Most creatures think (most got here) Some creatures are smart (quite a few if you think about it) We are also -omnivore (common) -that lives in many environments (still common) -earthbound (cf can use fire (which aided gut shrinkage and brain development) eg outrules aquatic creatures) (also common) -social but competitive species (aids intelligence development as need to out-compete peers, not just environment) (not uncommon) -have excellent tool-making capabilities (less common eg not dolphins, whales etc) -use fire (very rare, as before aided gut shrinkage and brain development) (note use of farming a booster but not a clincher as I've actually had fire for ?300,000 years prior to our recent tech acceleration) -developed farming (not common but done by some eg ants (nb note how widespread ants are too!) Of all these farming fits best with development of civilisation-->as farming creates main biological need (food) surplus, freeing up man to do other things eg chill, art, invent. Then we just built on these inventions in a compound interest way-things initially slow eg 1st tech worth 0.0001 cents and at 5% interest for a thousand years only equals $2 then later fast eg later tech worth $1 at 7% interest for last 500 years equals a million dollars! Thats how you go from 1522 tech (books rare) to smartphones (a.k.a. a device which holds more information than the greatest libraries in existence) everywhere!

[@beans1979]

Have we had evolution events on par with extinction events.?

[@jaclacscatpack]

Who’s art did he use on the thumbnail?!?

[@larsnystrom6698]

A neural network performing something is like when we do things automatically, without thinking. Akin to very complex reflexes. Like we do when driving without thinking about it. Or when running long distance while in a close to meditative state. That's why neural networks never will reach even close to AGI level. They are simply not structured with the complexity of a natural brain. That doesn't mean that AI can't reach AGI. It just have to be structured differntly, and not just as todays one directional neural networks.

[@jimbaker5110]

Correct. The motive for evolution of complexity in neural networks is the same as for our brains.

[@jimbaker5110]

Is NOT THE SAME I meant to write

[@tritun5154]

I feel there’s life in and out of the quantum realm. Mixed between the two. Past life memories. Life after death. Rebirth maybe.

[@morbidcorpse5954]

I always believed De Javu was an example of living the same life again.

[@MrJdicecc]

I have been watching the Andrej Karpathy conversations - Amazing. And on the other hand you interview Ye (which I didn't watch by the way). Question.... you have a limited number of hours in your productive life - why would you waste time on Ye when you don't have to?

[@StephenLawson]

That picture if from the HR Geiger videogame, Dark Seed. I literally just finished

[@gardeningandlife137]

I don't think people realized how big of a deal it was to discover the atom and DNA, to play with these things?

[@sorincaladera936]

DNA is and always was being played with. Every time something new is born or germinated, it rolled the dice to see what genes it gets and what ones are active. Putting our fingers in that ain't going to change anything significant for a long, long time.

[@Steven-lb4bl]

I played with my thing twice today

[@hermes_logios]

AI should use a multi-agent approach. The point of multi-agency is not so that beta males can collaborate to kill alpha males. The purpose is to rapidly increase the capacity to search through all possible permutations for success/survival (at the current level of technology or consciousness). When the search of the state space is exhausted, that’s when evolutionary leaps are possible and necessary. Unitary AI systems are a mistake.

[@dariozoric7181]

How i love AI terms being used in biology context :D

[@somebodywhocares9293]

What's the point to figure out the brain when we can't explain our soul. Without the soul there is no brain.

[@hrgwea]

4:14 "Invention of fire". Sounds like a bad choice of word.

[@DRKSTRN]

I wonder if you are starting to figure out what the default is.

[@luckyno888]

1.Unless you can make it or duplicate you don't truly understand it. 2. if you only use hammer every solution looks like a nail.

[@sexybrucecampbell]

You mean every problem looks like a nail. The hammer is the solution, right?

[@spaceowl5957]

Yo what happened to his Putin interview?

[@anewperspective247]

It is a rare ocassion when lex ask's an answerable question.... that he doesn't immediately provide the answer to.

[@rkalla]

LOL he’s absolutely one of those “think out loud, in stream of thought” which is obnoxious for an interviewer.

[@Koryogden]

Is it? Or is it just a pet peeves by some? Cuz I don't mind it

[@anewperspective247]

@@Koryogden you probably have 3 jabs and let others think for you.

[@SlickBlackCadillac]

@@anewperspective247 I like how the jabs sorted people out for us

[@peterpetrov6522]

It's also weird that animals have learned not to mess with humans even though we could make such an easy meal. We are even smooshier and slower than the time when we were cavemen. We can't even win a fight against a squirrel. Grab one by the tail and see what happens--it will be pretty intense!

[@j4ckpot1994]

dont know what kind of human u are but i probably kill a squirrel with 1 hit if i had to fight to death with it, but squirrels are no predators so... also many animals will mess with humans unless they have fireweapons with them to scare them of, so ur argument is totally invalid. what do you think is the reason we killed so many wild animals? in my country people even freak out if they see a wild boar if it's not deep in the forest.

[@invzmk]

Animals usually have a good instinct to detect evil and avoid it. Besides they are much more well behaved then us humans are.

[@DRKSTRN]

The giraffes laryngeal nerve directly contradicts this section in that there is no way for organic life to arise by itself. As that implies direct intelligent design, but all mamals and giraffes being the prime example still have a laryngeal nerve that made sense during the transition from fish to terrestrial. The reason why it is dumb, the idiotic design is that the nerve that controls your voice box is the same nerve that controlled gils still loops down into the chest as if we had no neck or torso. If it was an intelligent design the same nerve would in mammals go straight from the brain to the voice box. And yet when we look inside the giraffe that nerve goes all the way down their neck and loops straight back up. You can argue for the transfer of DNA from extraterrestrial origins. But as far as conjecture goes it becomes moot when you realize the survival bias at play. All life alive has survived long enough to reproduce. It doesn't mean the strongest, but that the organisms were just fit enough to divide or find a mate in competition with all other lifeforms. As if you do look at what is at play now and the idea that DNA, all of it and not just the viruses and bacteria that too have been integrated or served as ancestors. Would be the same accidental experiment that even this line of thought is being put forth. What is more interesting and now will go watch the rest of the talk. Is perhaps consciousness as a concept exists as a fundamental part of the universe as a scalar value of complex quantum machines. And so too does each element of the universe have an element of emergence derived from a simple rule set on which we have yet to unify. A giant gap in our understanding, only because we wish to believe something greater and yet contradictory run into a lesser more mundane view of the universe for the sake that there must be some magic or intelligent hand at play. You live in a simulation within your own mind, privy to a small amount of information in which our senses are able to detect and construct.

[@DRKSTRN]

Starting watching and I have to say I hate how well trained my inference has become. Without skipping a beat he is talking about emergence and magical properties about it. This would be the sacred(immutable) vs mundane(mutable) aspect of current argumentation that needs to be pointed out. There is no magic, but sufficiently advanced technology would appear as such, but to those using them would be mundane mutable. This isn't to discredit his achievements, but to point out a logical inconsistency that is preventing understanding and a line of analysis that would lead us to a unified theory.

[@DRKSTRN]

It's always so strange in the autistic experience where you find yourself point out the peculiarities of a persons thought process. As in many ways he is on point and see why he is so successful. But then run into the funny contradiction with his desire for a deterministic universe and yet the loss of local realism pushes that view into more of rationalization and even stranger for the magical quality of emergence. In this discussion and by being so particular it should be clear to point out the contradiction that is preventing steps towards unification as a logical jumping point. Random and yet magic and alien form of intelligence? More food for thought as I hope others find mind candy as alluring as myself. It's not about right or wrong about gradients and nigh infinite specificity.

[@zilatheartist]

well everything isn’t original lol we all just copy off each other throughout history or add on to things that existed, i believe technology wiring/ motherboards was inspired by ancient artifacts and structures that mimic hardware technology

[@JayLeePoe]

VR in a world that's hyper-real and there will become code that has a play cycle like humans stuck in a karma loop. What people forget is that the average human citizen wouldn't even give a damn about computer nerd hobbyists unless their gadget played them a fun game, then it was interaction, then illicit interaction (which was the purpose developed by the government anyway, clandestine effective and quick communique, and oddly does originate in the most real game theory...)

[@steveshank9674]

What about panspermia bro

[@OnePieceWonPeace]

This guy's speaks and moves like the video play speed is at 1.75x.

[@Hatrackman]

Its an equation that passes through big bangs. There is nothing alien, magical or mysterious from the perspective of the universe... and there is only one physical thing that comprises the whole of all.

[@josephs.7496]

Neural Networks/AI/Machine learning is literally linear regression.. the most basic form of algebra. There’s nothing alien about it LOL.

[@xedasxedas]

I think you fail to understand the meaning of literally. Still, i can "see" your point.

[@Juddersbaby1]

A sane person!

[@AtomkeySinclair]

Thank you for noting that... I was just going to. Math... computational brute force in modetn machines makes it implementable today. Adaptive Reasonance Theory came out a long time ago. And take a baby from 5,000 years ago and raise it today and it will learn just the same as a modern child given the bioprogram so to speak is still human DNA....

[@instapowah]

Please invite Graham Hancock to the pod!

[@TFBx]

Lol

[@instapowah]

@@TFBx ❤

[@Zombied77]

oh he's cool, love that guy

[@GaryPalmerJr]

Yes!!! 🎉❤😅

[@jeffcordes3680]

God already told you, let us make man in our image. Father, son, holy spirit.

[@jesusdisciple2614]

" I'm hesitant to say rare" 5:10 . Again, shows a mindset that is limited by empiricism. It's limited by looking to that which is " unknown" in the father reached of the universe for meaning rather than looking closer; looking more within.

[@jesusdisciple2614]

4:57

[@cytroyd]

Why does he end his sentences making them sound like questions?

[@trillj1237]

Yahweh is king!!! All glory to the most high!!🙏🙏

[@trillj1237]

hhc

[@suchen6845]

lol

[@neithanm]

And people reading the bible, koran, talmud etc in 2022... Whate a waste! Open your eyes to the true beauty of nature and the universe! If there is a god or gods, you get closer by studying biology, AI, physics, cosmology... and listen to all these smart people.

[@zelda6378]

lex convinced kanye to talk more holocaust dumb stuff then talks to a google engineers laughs about it . obviously kanye doesnt know quantum computing l0l

[@bryanedwards187]

Wait does Lex see himself as Alpha

[@madisoncarter3970]

ai is just a bunch of if then statements meh

[@parasocialbondsmetaswvoits9078]

Is Lex microdosing here? He kinda looks like he is.

[@Noobnotnoob12]

First

[@shadowdawg04]

He had to say it: punctuated equilibrium... unproven (not provable) & and kills his attempt at cohesion - Capt. BuzzKill