George Hotz: How tinygrad works | Lex Fridman Podcast Clips

Lex Fridman Podcast full episode: • George Hotz: Tiny Corp...
Please support this podcast by checking out our sponsors:
- Numerai: numer.ai/lex
- Babbel: babbel.com/lexpod and use code Lexpod to get 55% off
- NetSuite: netsuite.com/lex to get free product tour
- InsideTracker: insidetracker.com/lex to get 20% off
- AG1: drinkag1.com/lex to get 1 year of Vitamin D and 5 free travel packs
GUEST BIO:
George Hotz is a programmer, hacker, and the founder of comma-ai and tiny corp.
PODCAST INFO:
Podcast website: lexfridman.com/podcast
Apple Podcasts: apple.co/2lwqZIr
Spotify: spoti.fi/2nEwCF8
RSS: lexfridman.com/feed/podcast/
Full episodes playlist: • Lex Fridman Podcast
Clips playlist: • Lex Fridman Podcast Clips
SOCIAL:
- Twitter: / lexfridman
- LinkedIn: / lexfridman
- Facebook: / lexfridman
- Instagram: / lexfridman
- Medium: / lexfridman
- Reddit: / lexfridman
- Support on Patreon: / lexfridman

[@LexClips]

Full podcast episode: kzbin.info/www/bejne/mn_VhaWuaZd3fbM Lex Fridman podcast channel: kzbin.info Guest bio: George Hotz is a programmer, hacker, and the founder of comma-ai and tiny corp.

[@511tt]

I understand 10% of the video and I work in tech.

[@TheGavalanche]

I used to work in tech and am about 3.5% understanding 😅

[@visualEduration]

Really depends in which niche you work I think. Have you seen him on web development 😂?

[@WazapianArmy]

Take a class in neural networks and you will understand like 30% this guy is just so next level even Lex's brain was hurting lol and he is a expert in the field

[@michalbotor]

i studied math and physics. nothing top notch, but i had a linear and abstract algebra, so i understood it when he talked about tensors and operations on them, and laziness too. i also did some programming on gpus with cuda back when pytorch wasn't yet a thing, so i understood it when he talked about kernels and flops. but everything from and especially after "porting new ml accelerators quickly" sound like an alien language to me.

[@Haannibal777]

@@michalbotorI have a physics PhD background but not a practising physicist. But I still feel like a little kid listening on a grownup conversation.

[@snarkyboojum]

My guess is that tinygrad effectively started as a clone of micrograd (that Karpathy had built and streamed live for educational purposes). George was likely inspired by that and wanted to build a little toy library for himself. That was when Tinygrad was "always going to be" < 200 lines of Python or something. George was dissing all the larger libraries for being too large and complex. He tends to do this, and then gradually builds towards the same complexity as he gets real features in his projects. I agree with his relentless focus on simplicity and quality though - we need more of that idealism in the industry. Comma took a similar path too. George started off effectively betting Elon that he could beat Tesla FSD computer vision and since then has gradually realised that probably isn't going to be possible. George seems to take inspiration from other people's ideas, then starts small and simple and gradually builds to the same kind of feature set and complexity. He's a bit naive and brash (see his latest interactions with AMD on tinybox), but who hasn't been in their youth? :) He's entertaining, and inspiring a bunch of younger folks to get into tech. More power to him.

[@user-kn4wt]

still don't understand what this library is solving. is it just a simpler pytorch?

[@mrmrmrkevin]

pytorch's design means it's a lot of work to make it efficient or even possible to use a non-Nvidia gpu or a new type of processor, and like a lot of big frameworks, it works very well until you do something less ordinary/mainstream. That's problematic for lots of reasons: - running on hardware that's not a big desktop or giant server somewhere might be impractical or near impossible - it slows/impedes hardware innovation - it hinders competition/consumer choice in the gpu/compute market - it limits computational efficiency and performance. Nvidia gpus are great hardware, but many people think they're used because nothing else has the power - in reality, it's just software design problems that makes it difficult to use different (potentially faster) hardware today.

[@mzubuki]

Luke 15:3-7 (KJV) 3 Then Jesus told them this parable: 4 “Suppose one of you has a hundred sheep and loses one of them. Doesn’t he leave the ninety-nine in the open country and go after the lost sheep until he finds it? 5 And when he finds it, he joyfully puts it on his shoulders 6 and goes home. Then he calls his friends and neighbors together and says, ‘Rejoice with me; I have found my lost sheep.’ 7 I tell you that in the same way there will be more rejoicing in heaven over one sinner who repents than over ninety-nine righteous persons who do not need to repent.

[@priusa8113]

Lex, I pray Allah give you hydayah. Greetings and Salams from Dallas, same area of Dr Omar Suleyman :):)❤❤

[@KcKeegan]

Damn fanatic.

[@fntr]

alhamdulillah dog got rekt

[@4144758]

Lex is the most boring person I've ever seen in KZbin bar none.

[@Godfather-qr6ej]

And also very stupid too.

[@MrCryingDragon]

Trying to get internet cool points are we.

[@GamerBoyRobby]

I find him to be one of the most interesting with the most interesting guests