No Priors Ep. 80 | With Andrej Karpathy from OpenAI and Tesla

[@Nkunzi808]

I generally listen to podcasts at 1.5 speed. Andrej is one of the few people whom I have to play at normal speed to keep up with.

[@PolitikCPR]

That Adderall is kicking good! 😂

[@T0mmyAngel0]

I was thinking about slowing it down

[@Sofiazinha-j2w]

Good luck then trying to listn do Marc Andreessen

[@Nkunzi808]

@@Sofiazinha-j2w it's not just about their talking speed, it's more about the depth of their knowledge and how well they articulate it. (*Never had the need to slow it down for Marc)

[@Asaceplays]

They already speed up the playback before uploading the video. So ur good

[@motobanite]

Karpathy is a truly great human being.

[@psi4j]

I concur.

[@iakobkv271]

do YOU guys know HIM

[@janh-r8h]

❤

[@jean-pierrecoffe6666]

he's a very good engineer how does that qualify him to be a "great human being" wtf is he known for charity or something

[@webgpu]

genius first, then altruist, benevolent, (a lot) empathetic, etc.

[@niksapraljak7605]

IMO... Listening to Andrej Karpathy talk about innovation, work on deep learning research (e.g. assembling LayerNorms, Skip connections, and other parts to create a trainable "tissue" called transformers) and current/future problems is a breath of fresh air for young researchers like myself.

[@thelazyanalyst3723]

I want more talk about stuff not related to first principles, get this karpathy guy out of here!

[@hilbertkem528]

@@thelazyanalyst3723he isn’t a researcher anymore just to be clear bud

[@LeonidKotelnikov-jg9fi]

I had to check my playback speed settings, because I thought I’m on x1.5

[@FrunkensteinVonZipperneck]

Glad we didn’t have to follow at the speed Dr. Karpathy’s brain races!❤

[@OceanLlamaMedia]

They are probably trying to game the algorithm. It's about 1.25x speed so I assume they are trying to make it seem like you watched the video longer than it's actual time. I shoot and edit for a living....no one talks that fast.

[@andybaldman]

Same! It’s annoying. They’ve totally sped this up.

[@zhenglyu6956]

@@OceanLlamaMediaActually I had a colleague who was at Stanford as well. His talking is like 2x… so such a speed does exist. Won’t believe it either until I saw one example in real life

[@armin3057]

@@FrunkensteinVonZipperneck God is so unfair...some people are so smart and some are so dumb. and kaparthy is also tall and gl

[@ZacheryGlass]

This guy is an actual genius. You can tell just by the expresses his ideas and is so forward thinking

[@janh-r8h]

❤

[@danklebanov7590]

Andrej is amazing. On the topic of education, he is proposing that current technology would allow for AI to be the front-end to the "perfect" course, where a human designs the course, and uses AI to scale themselves. One critical piece, especially for younger learners, is the human connection. Some kids (like the young Andrej's of this world) can be entirely self-driven, but many need to be inspired and guided by another person. I wonder how AI can be used to "super-charge" a teacher, rather than replacing the human at the front.

[@raymobula]

About this. I saw a startup out of Stanford that leverages gen AI to create course content. The teacher designs the course, topics, etc, & the machine fills in the material and tests. Allows to update courses quickly and potentially dynamically tailor the material to the student’s needs.

[@MathGPT]

One way would be for the AI to take on different characters and pretend to be SpongeBob or whatever teaching long division. Imagine you could pick anyone in the world to be your teacher

Well given the modern state of the humans doing it and their obvious failures from even older methods.. nah let ai work it.

[@GaryMillyz]

"but many need to be inspired and guided by another person." This, as with literally everything, will change.

[@creatvsdd99]

Andrej is truly a visionary:) I hope with AI technology, we can eliminate mental health disorders and eliminate neurological disorders in humans. Consciousness I believe is in the brain and fixing certain things in the brain like mental health disorders and neurlogical disorders would enhance the mind and enhance intelligence and enhance one’s quality of life. I’d love to have a mind like a Sadhguru, and a mind like a spiritual guru badly in this life:)!

[@hsiaohu2959]

thank you Andrej for explaining technical things that a layperson such as myself can understand, or at least get a sense of😃

[@gazmendqose8732]

Just discovered this podcast. What a gem . Now I need to catch up with 80+ episodes 😆

[@LoisSharbel]

Joy! By accident, I stumbled onto this podcast and it's brilliant! Andre Karpathy is so likeable, one wants to concentrate to catch the meaning as he speeds through his ideas. He impresses me as a good person and good for the world. Thank you for this!!! I'm a follower, not with understanding of physics but in getting a peek at the edges.

[@kunstrikerasochi2103]

As always with Andrej, awesome insights

[@SLAM2977]

great stuff! Glad to see Andrej very happy doing his own things rather than being limited at OpenAI

[@CrazyFoxMovies]

That was such an amazing episode.

[@muhannadobeidat]

Andrej is a legend! He answers normal questions by saying “this will take you down to a local minimum”. This should now be an expression in the English language. Great interview, thanks for putting this together and letting Andrej share more gems with the rest of us.

[@righttiming]

legendary guest most underrated new pod in the world imo. get ilya, elon, geohot, friend founder, sama next 💙

[@XiallaLife]

Best interview ever relating to AI, FSD and Optimus.

[@ThysRoes]

Yes we need technical people, we also still need storytellers, educators, performers and philosophers to give meaning to what we can create with technology.

[@TheMrPopper69]

I think Andrej will be written into history books for years to come, he’s taught so many and brought open source to the forefront of technological innovation

[@TheJayLenoFly]

what an inspiring episode

[@willbogusz6839]

The ABSOLUTE last person I expected to find on an AI technical interview video. Love your Craig Ferguson compilation work

[@TiagoChilanti]

Andrej seems to speak in 1.75X I usually whatch videos on 2X or 1.75X but this one I had to watch at 1X

[@babatundeadewole2488]

I had to check my playback speed to be sure it was in normal 😂. I am sure he’s mouth was trying to catchup with his brain

[@TiagoChilanti]

@@babatundeadewole2488 exactly. I did check it twice

[@OceanLlamaMedia]

They are speeding the video up. He doesn't actually talk this fast in his own videos. kzbin.info/www/bejne/omnTg4arip1khrc

[@holdingW0]

This man is one of the most important technologists of our time. Thanks for the interview!

[@palimondo]

That comment belongs to the No Priors podcast channel. You meant to say “Thanks for highlighting and discussing the interview.” Credit where credit is due!

[@Arcticwhir]

shes a great interviewer and Andrej is all around a great speaker and has great ideas

[@michaelm358]

Well done guys! You are the coolest!

[@techiheed1845]

Loved that. Andrej is such a gentleman with sublime knowledge.

[@christianmantseris4365]

much insight into areas you normally don't hear much about especially autonomous driving and robotics (although they are closely related). There are many sources for AI but most of them of low quality, so it's very important to focus on excellent sources to get deeper insights, Andrej is one of them.

[@edkim962]

Thank you for doing this pod!

[@ssumargroup7382]

AI being the Front End for a backend heavily technical course. That´s it. I am home now. Thanks GOAT.

[@Ben_D.]

I love the teaching models idea that caters to the specific person, their age, demographic, skills, languages, handicaps, and what not. I would love to be taught the things that are important to me by a perfect teacher.

[@SeanAbraham]

Excellent interview as always with Andrej. Just hearing him describe things is inspiring and makes a lot of the optimistic viewpoints on the future come into sharper focus

[@PhilipAndreGjone]

Finally Andrej on Tesla . Been waiting for this a LONG time 😊🙏

[@GET2222]

Finally… someone giving Andre the opportunity to reinforce the idea that TESLA is leading in this space with the right approach. Funny how neither one of these interviewers want to go there… silence… the bias is so thick and filled with disgust. This discredits their own knowledge because they clearly can’t get out of their own way. They don’t want Tesla to win… so dumb.

[@developit1152]

He joined long ago

[@michaelholmes8848]

@@developit1152he joined what?

[@developit1152]

@@michaelholmes8848 Tesla

[@deeplearningpartnership]

what?

[@carvalhoribeiro]

In every conversation with Andrej Karpathy, I learn a lot of things. In 27:10 I think it is very interesting hypothesis and I believe it would be interesting to test it. Thanks for sharing this.

[@clray123]

The woman agreeing to pay to "rent her brain" is the actually frightening part about this "progress". Reminds me a very much of soldiers willing to "shut their conscience" in favor of following someone else's orders (and getting their pay). To make such statements without stopping to imagine the consequences you have to be either dumb or evil, or maybe both.

[@Clammer999]

Andrej is one of my favourite AI legends. His love for AI and teaching is what makes sets him apart. The way he spews out Python code is like Python flows in his blood.

[@kyung-hoonkim5963]

Thank you for the great talk. Thanks to Andrej’s lecture, I learned so much about how tokenizer works and built gpt2 from scratch. Regarding the distillation, I think it kinda resembles how proverbs works when transferring the knowledge from experiences in a short compressed manner. I do hope the cognitive part will reach really small parameters and embedded into our handy devices, and democratize AI for all.

[@erniea5843]

So good, always fun to listen to Andrej

[@carsond67]

Such intelligent questions asked by the interviewers. So much better than the usual surface level stuff you see.

[@Warley.Araujo]

Great talk with Andrej!!

[@abhiramreddy1598]

This video is so good. I want everything karpathy says to turn true.

[@nickmerrill9178]

42:33 I think Karpathy’s points about what learners should focus on are excellent. I especially appreciate his openness to 20% non-“symbol manipulation” subjects. The only thing I would add is that subjects like writing and history can also strengthen your reasoning abilities. I agree that memorizing facts is not a good use of a learner’s time, but reasoning about cause and effect is. Such reasoning can be practiced outside of “STEM”. I think a lot of folks with STEM background (including myself) have a slight bias against non-STEM subjects, so I feel it’s important to add that perspective. Finally, there are some important “human skills” such as empathy that can arise from subjects like literature, so I appreciate Karpathy’s openness to being well rounded. Thanks for the solid interview!

[@xianzhangdong3430]

I like Andrej Karpathy❤

[@EnigmaCodeCrusher]

Great show

[@ahmedtremo]

Great episode!

[@Hopyboby]

this Karpathy guy is a king. like if there ever was or ever will be a king. it's that guy.

[@mikestaub]

Andrej is our generation's Richard Feynman.

[@wwkk4964]

Thank you, great show!

[@RichardBonn]

Andrej Karpathy is a wonderful human being!

[@ScienceMediaGaming]

I want andrej’s courses, but I want him on podcasts everyday 😂🖤

[@ilkoderez601]

Accelerando! Love that book! So great! OMG I love this channel you people are so freaking cool!

[@quez]

Andrej is a rockstar. Love everything he says and does

[@DistortedV12]

Wow Eureka is a fascinating company

[@SamsonMathew]

🎯 Key points for quick navigation: 00:02:00 *🚗 Andrej Karpathy discusses his work in self-driving technology and its parallels with AGI, emphasizing the impressive strides made over the past decade.* 00:03:21 *🤖 Karpathy believes Tesla is ahead of Waymo in self-driving capabilities, attributing Tesla's strength to its scalable deployment of vehicles.* 00:04:03 *🌐 Tesla's approach to sensors entails extensive use of expensive equipment during training rather than in operational deployment, making it a cost-effective strategy.* 00:06:23 *🏭 Karpathy argues Tesla should be viewed as a robotics company rather than just a car manufacturer due to its innovative approach in robotics at scale.* 00:08:45 *🔄 He suggests that the best early applications for humanoid robots will likely emerge in businesses, focusing on tasks like material handling to minimize legal liabilities.* 00:10:37 *🌱 The promise of humanoid robots includes niche applications like leaf blowing, showcasing the practicality of their development.* 00:11:36 *🧠 Karpathy emphasizes the importance of transfer learning between tasks, advocating for a single humanoid platform to optimize efficiency.* 00:15:22 *✨ He highlights the transformative impact of Transformers in neural network architecture, marking a significant leap in research capabilities and scalability.* 00:16:33 *🔍 Karpathy concludes that the bottlenecks in AI research have shifted from architecture to loss functions and datasets.* 17:15 *📊 Data innovation focus is shifting towards synthetic data and enhanced data collection methods due to limitations in available internet data.* 17:57 *🧠 Understanding the brain's reasoning processes could advance AI development, as current models help refine future designs.* 19:10 *⚠️ Ensuring diversity and richness in synthetic data is crucial to prevent model collapse and maintain data quality.* 21:16 *🧬 Transformers may eventually outperform human cognitive abilities in specific tasks but are currently constrained by data limitations.* 22:40 *🤖 Human-AI augmentation is likely to evolve, blurring the lines between technology and human cognition.* 25:30 *🔄 The ecosystem of large AI labs may create a reliance on closed platforms, raising concerns about ownership and control of cognitive enhancements.* 27:09 *📏 Smaller, efficient AI models could be viable, focusing on cognitive core functionalities rather than unnecessary data retention.* 29:55 *🌍 AI education tools could democratize learning, empowering individuals with personalized and scalable access to knowledge.* 32:06 *🎓 AI has the potential to revolutionize education by serving as a personalized interface that adapts to students' needs, enhancing their learning experience.* 34:26 *🏆 Better educational tools and curricula could significantly enhance individual performance, similar to advancements in sports training methods.* 34:54 *🌍 Language adaptability in AI translations is effective, serving as low-hanging fruit for improving education.* 35:50 *🔍 Personal learning context varies; AI should adapt to individual backgrounds and learning rates for effectiveness.* 36:44 *🏛️ Lineage in the AI research community influences knowledge propagation, but it's crucial to minimize its gatekeeping effect.* 37:12 *🤝 Community clustering in AI facilitates knowledge sharing and education, shaping personal motivations and cultural values.* 39:06 *🎓 Education should become more accessible, helping people learn for practical reasons while also shifting towards personal enjoyment.* 40:04 *🧠 Learning should be viewed as mental exercise; the challenge of education can parallel the effort needed in physical fitness.* 41:29 *👩‍🎓 The upcoming course targets undergraduates in technical fields, emphasizing continuous learning as technology evolves.* 42:26 *📚 Young learners should focus on math, physics, and computer science as foundational skills for effective problem solving and adaptability in the future.*

[@palimondo]

Nice map of the interview, thanks! Would you mind sharing the process you followed to prepare it?

[@edgelord6560]

Awesome interview and nice to see knowledgeable hosts for once

[@dotnet364]

u didnt see the Ilya episode?

[@meow-oj2ml]

very insightful and the hosts are great.

[@SteffenProbst-qt5wq]

Love how he just tells the optimal training recipes under the analogy of human learning and education

[@salepros]

Great show as always!

[@cybernoir]

awesome interview!

[@goodtothinkwith]

Great discussion. I was surprised at his comment at the end. If education is for you rather than for money, I’m not sure why you would choose math and CS first. I would think that curricula that catalyze understanding of yourself and the world would be far more important.

[@FrunkensteinVonZipperneck]

AI is a subset of psychology - the science of knowing oneself.

[@ineumeyer]

Amazing interview. Thank you!

[@perwis9893]

Great interview. Subscribed!

[@RK-fr4qf]

When folks at Tesla are sending their children to school in self driving cars that will be a sign of confidence.

[@FrunkensteinVonZipperneck]

Agreed. That authentic assessment will define the critical inflection point.

[@juliahello6673]

Confidence from the regulators

[@nickfosterxx]

When everyone sends their kids to school in self driving cars America will realise the mistake it made in being so car dependent. Congestion will be insane. The new equilibrium of inconvenience with people having breakfast or working/sleeping on the commute, will mean that the wealthiest might prefer to live in a walkable city centre free of cars.

[@wiktorm9858]

Andrey is shockingly wise. This field has a lot bright minds, but this guy is just watching otgers from above.

[@soylentpink7845]

Nice!!! Waiting for his course!!!

[@FrunkensteinVonZipperneck]

Dr. Karpathy developed Stanford’s first AI curriculum❤

[@GoldenDumpling]

12:03 An arm on a platform, on wheels, sounds like a great way to minimize cost. It would still work great for many simple and repetitive tasks, that might not warrant the expense of a full bipedal bot. Make the base heavy and allow it raise and lower the arm. The sophisticated hands that Optimus brings aren’t always necessary either. For example, at a BBQ restaurant, towards the last hours of a cook it is better to add smaller quantities of wood repeatedly, so as to target higher temp and airflow. Airflow helps evaporation, which helps the exterior parts from getting overdone. Very fussy work that could be automated.

[@HurricaneChise]

From a consumer perspective, I’d be more comfortable with arms on wheels in my home rather than a humanoid. I know it’s irrational, but I have this fear of waking up in the middle of the night and my humanoid robot standing in my doorway looking at me. Just creeps me out lol.

[@jondavies5885]

Great guy to listen to, articulates complex subject well... At 0.75 speed for me.

[@redyican5341]

Very insightful 🎉

[@adamoreilly6546]

Andrej, if you're reading this: You mentioned using the older hardware heavy lidar approach to use as training data for the pixels. Why not just train on video from 99th percentile human drivers?

[@FrunkensteinVonZipperneck]

That is Tesla’s strategy. As videos were collected, human raters categorized driving on a 5-point scale. Training occurs on the “5” videos. Separate training occurs on crash videos - system practices crash avoidance…

[@iclick4122]

Love his take on Toronto too lol

[@philtrubey7480]

Sarah, you just casually drop that you have a Unitree G1 robot and leave it there??? How is it programmable? You've got to do an unboxing video or at least explain what it's all about!

[@similarvideo]

Pretty good content👍 thank for sharing!

[@Chadpritai]

Sensei Karpathy🙇🏻‍♂

[@helloryantanaka]

Cool interview.

[@abhiramreddy1598]

Ohh man, Is the videos is on 2x?

[@mraarone]

Transfer learning on simulated data does really great on mathematically based models, then the entropy or somewhat clean real world data whether it is sensor data or encoded data works. I’ve trained models 100% translated data that transferred to real world data with over 90% accuracy and balanced error. It was really impressive.

[@ilkoderez601]

Great channel! New subscriber.

[@janh-r8h]

Me too

[@spectralvalkyrie]

Go Team (Augmented) Human! 😁

[@akhilkatpally]

my brain hurts, its too fast, dense, always on point, no BS

[@developit1152]

U don't have to understand everything Just skim through it

[@HansSchulze]

Watch it on slow playback?

[@janh-r8h]

❤

[@0nelight19-SaveSoil]

Regarding the cognition question - 20:55 - Can recommend: "Maturana - From being to doing - The Origins of the Biology of Cognition". Memory, memory, memory :-)

[@kurtvega3053]

thanks for mentioning The Diamond Age!

[@lppoqql]

University of Toronto is a very under rated school. Their grads are mostly humble low key but highly skilled.

[@rickfearn3663]

Inspiring.

[@claudioagmfilho]

🇧🇷🇧🇷🇧🇷🇧🇷👏🏻, Great interview guys, thanks for sharing!

[@moderncontemplative]

Key prediction: in a post AGI society education will be entertainment. I concur. We already see people like me educating themselves for fun, especially autodidacts! Anyone who understands AI realizes the possibility of a Utopia, if governments and AI companies align to solve ethical and cybersecurity problems.

[@DailyProg]

This guy is very smart, he will go places

[@deeplearningpartnership]

go places?

[@matthewr759]

Already has! And he's helping everyone go further with his education efforts!!

[@cosmosuncle]

Great episode! 1. Who is Nate Friedman (sp?) and what is the "Leaf Blower Challenge"? 2. What IS the real price of a G1? Website says $16k.

[@Grahfx]

The only man on earth I have to slow down to 0.75x just to keep up.

[@LisaSamaritan]

Quietly remove leaves? Use a "rake" if they are on grass/sand/soil, or a "broom", if they are on other types of ground. Picking up leaves individually would be cost (time and energy/electricity spent on the task) prohibitive.

[@SydneyApplebaum]

It's a fun talk from Andrej, but sounds very speculative.

[@patruff]

His name is Car Pathy and he designs AI so the car goes on the right path, I can't make this up people!

[@kazioo2]

10:05 - I thought the same thing the moment I saw that "Leaf Blower Challenge". Just tell a humanoid robot to do it - done. And then something else occurred to me: why would you need a dishwasher or a washing machine if you have a humanoid robot? It won't substitute a fridge, but some automation we take for granted wouldn't be necessary when you have a universal robot. A lawnmower? Just get some grass shears :D

[@raymond_luxury_yacht]

bruh. mad respekt

[@JoelSapp]

34:32 - This might be a little recency bias. The guardian did a nice article on 100 yard dash for the last 100 years and surprisingly the numbers were very similar. This analogy may be off between physical limits and the brain's limits however.

[@TuringTestFiction]

"The demo is near but the product is far." Ha ha! Shots fired! Take cover Sama!

[@souprememc]

Am I high or is the playback speed hiked up on this 😂

[@FrunkensteinVonZipperneck]

A.K. thinks so fast, his lips always race!

[@umabugde4978]

He talks so fast😂

[@rverm1000]

now is the time to do a national or world wide questionnaire on problem solving to get to agi. at 62 im facinated by all this machine learning, llms. can i learn enough about this to create my own useful model.

[@marncore7048]

The humanoid form is not optimal for picking up the leaves from the street @Andrej. This problem is much better suited for a Unitree quadruped robot. You just stick a robotic arm on the top and ta-da. It will be much quieter and easier because it has fewer degrees of freedom and the humanoid would have to constantly bend down. Also cheaper and faster lol. The quadruped form is actually the way to go here! You can put it on wheels too so it's more stable.

[@QueenLover-j5i]

Insights By "YouSum Live" 00:00:05 AI's impact on education and self-driving technology 00:01:04 Self-driving cars have reached significant capabilities 00:02:30 Tesla's software problem is easier than Waymo's 00:04:15 Tesla's approach uses cameras for cost efficiency 00:07:12 Humanoid robots share technology with self-driving cars 00:07:55 Robotics at scale is Tesla's true identity 00:09:01 Humanoid robots will start in B2B applications 00:31:04 Education should empower people, not replace them 00:35:00 AI can provide personalized learning experiences 00:35:50 Future education will require adaptability and accessibility 00:43:09 Math and physics are essential for critical thinking Insights By "YouSum Live"

[@wburris2007]

I have been trying to learn math, physics, and computing for 50 years and haven't made much progress. I need an AI to guide me and help me stay focused. Current LLMs are most helpful for the computer programming, but not so much for the math and physics.

[@rebeccamiller8772]

Is the video sped up for anyone else? The time goes faster but my playback speed is set to normal.

[@jeffgorchynski]

"we're speaking with karpathy who needs no introduction" ....*proceeds to introduce him*

[@anameofauser6866]

I had to double check the speed of the video wasnt 2x because karpathy speaks so fast 😂

[@manasprinceton]

Maths Physics CS, courses to develop thinking, so rightly put!