Ep 18: Petaflops to the People - with George Hotz of tinycorp

How tinygrad is taking on Nvidia, Google, and PyTorch with a tiny team, building in public with AMD, hot takes on ggml, Mojo, and GPT-4, and why AI Girlfriend is next.
Writeup and show notes: www.latent.space/p/geohot
Hosts' Twitter: @swyx and @fanahova
Timestamps:
00:00:00 - Introducing George
00:02:59 - Tinycorp's 3 Theses
00:11:12 - Tinygrad's creation
00:15:58 - Operation fusing in Tinygrad
00:19:11 - Tinygrad debugging
00:21:14 - Tiny Competitiveness on QCOMM vs NVDA
00:23:21 - geohot vs AMD
00:28:21 - Tinygrad vs ggml
00:30:01 - Importance of Good CI
00:30:37 - Mojo and Compatibility
00:32:43 - ggml quantization is made up
00:35:18 - tinygrad: benchmark int8 vs fp16
00:37:39 - Why you can't build tinybox
00:40:28 - The personal compute cluster
00:43:08 - Compute Optimal to Inference optimal
00:45:06 - Announcing FLOPcoin
00:46:23 - Why Federated AI won't work
00:47:38 - 5x faster than Nvidia
00:48:53 - A Person of Compute
00:49:49 - GPT-4's real architecture
00:51:07 - BatchNorm, FlashAttention
00:52:34 - The Bitter Lesson
00:55:31 - Hiring in the Age of AI
01:00:02 - Why AI doesn't replace developers & artists
01:03:02 - Comma Body
01:07:34 - AI Girlfriend
01:11:00 - The Goddess of Everything Else
01:13:43 - John Carmack Insights
01:17:41 - on Elon
01:18:47 - on e/acc
01:20:24 - Avatar 2

[@TheLakeJake3]

George knows how to make me feel like I need to give my life to working on AI

[@realhet]

That's a closed circuit: You make him feel he can give his life working on whatever he wants. :D

[@androwaydie4081]

That's marketing for you.

[@ahpacific]

I always learn so much from Hotz. Thanks guys.

[@LatentSpaceTV]

we did too. super grateful for him taking the time.

[@mrdbourke]

Epic show! I love how direct George’s responses are, almost all from pure experience “I don’t like x because I tried it for 16 hours straight and it didn’t work”

[@thsstphok7937]

Exactly.

[@akj3344]

Hey big man. Love your both courses!

[@bogslurp3532]

10 minutes in and I generally have no idea what George is saying but I could listen all day.

[@prpl__5115]

same 😆

[@JohnCooganPlus]

what a great episode! this show is awesome

[@vladusa]

Coogan's a Hotz fan?!

[@SasskiaLudin]

This man is such a genius. Pure delight to hear him. Thank you so much for this enlightening video!

[@DikTrom]

you think that is a compliment? It is actually a curse to be a real genius

[@JalaleddinElFirqi]

thank you for the episode guys, great guest and sharp hosts

[@GThomas-qq6mp]

This is actually great to dissect the progress. Along the streams it's harder to understand what George is doing. I've learned a few things from this.

[@shridharangp4865]

Democratising the big tech for everyone, your efforts are really appreciated 🙏

[@MWileY-nj1yb]

I appreciate you guys and your fresh approach to dense subject matter. Always love listening to George.

[@larry_the]

Wonderful episode. Always love hearing more from George.

[@NotAnonymousNo80014]

Yes , they should raise the gain on his mic.

[@HassanElMghari]

What a great interview! Great job y'all

[@chickenwings273]

Thank you for making this!

[@deersakamoto2167]

25:18 GeoHot's email to AMD's Lisa Su I like that email starts with "I started a company" - no "hello", no pleasantries, just straight to the point

[@420_gunna]

I really like the banner that you have across the top of the video that describes the topic -- great idea on that.

[@Shan224]

Thanks guys. Subbed ❤

[@LatentSpaceTV]

Writeup and show notes: www.latent.space/p/geohot

[@d33w]

great episode, seems I found a good podcast in the ML field. Kudos and keep going!

[@LatentSpaceTV]

Thanks for listening! We have ~17 episodes you can catchup on as well :) latent.space/

[@AshutoshSharma-ml6sv]

need more podcasts like this which actually talk about tech

[@meteorinc4259]

How do they have only 287 subs! Best guest, animations to explain concepts (more of this please), and current topic of discussion on top. Great editing!! Thanks Latent Space and keep it up!

[@LatentSpaceTV]

It's our first day on KZbin! All of our past episodes are here: latent.space/. Glad you enjoyed it!

[@recursion.]

@@LatentSpaceTVyo I hope you'll only focus in the tech space unlike lex fridman who interviews anyone with clout.

[@jonzo_]

Great talk!

[@AnthonyZboralski]

Subscribed! I'm annoyed I only discovered your channel today :)

[@LatentSpaceTV]

We only created it a couple days ago, but you can find all of our old episodes on latent.space! More to come!

[@AnthonyZboralski]

@@LatentSpaceTV yeah I noticed and subscribed on Apple Podcast. Keep up the good work

[@RonanMcGovern]

Congrats, great content. Wonder what geohot thinks of the llama license being non commercial

[@Nour-le8jn]

great episode, very knowledgeable hosts absolutely better than Lex. George never disappoint. keep it up guys

[@hd-2]

Listening to what George wants to do, and then getting a recommendation to watch Tenstorrent discussions from Jim Keller really makes me wonder how the two have not collaborated already.

[@princemredlana1198]

I'm in awe

[@SamirPatnaik]

Towards a end of it… George is a god-tier soul. om namo shivay

[@oryxchannel]

Soooo freakin coool. It's like a Pat Metheny concert. You just pick up on the jazz.

[@Bill-cy2cy]

Yessssss… thank you.

[@DD-ws6cu]

I have no idea wtf he’s talking about for 90% of this but I still find it fascinating lol

[@ujjwalshrestha5720]

awesome

[@vev]

Great

[@math3068]

That's great! you guys should try to bring David Holz from midjourney next

[@dwolrdcojp]

George is the best

[@andrewferguson6901]

Ep 18, your only vid on this channel. Nice

[@andrewferguson6901]

not to like. be a hater. cause this is legit a great interview

[@nxnu2119]

Are you guys on nostr ?

[@jaya8352]

Good stuff, but please boost volume, if obs add filter in mic settings. Bt speaker on Mac volume still don't hear it all

[@jaya8352]

To clarify, hosts are plenty loud, but Geo is far quieter multiplied by he talks very quickly, he needs more volume, can barely hear him and any ambient noise he's unhearable, I'm missing at least 30% of what he's saying, mentally fatiguing.

[@jaya8352]

I subscribed and looking forward to more, but please do more thorough sound checks and monitoring the levels.

[@jaya8352]

Oh now I see he is far back from the mic often, maybe just police that more, I mean the whole point is to share audio, while if the key speaker is talking not into the mic most of the time then well....

[@herashak]

Good vid, side note that the audio is quieter than most other youtube videos.

[@LatentSpaceTV]

Thanks for the feedback! We mix these for podcasts separately, we'll amp it up a bit next.

[@vankram1552]

Amazing show

[@torsteinsrnes4872]

The volume is a bit too low and it is too hard to hear what you're saying sometimes. Otherwise great podcast!

[@trhashthings8045]

ok, min 8 and Hotz started to see the matrix 😂

[@Johnmoe_]

Will you guys upload anymore episodes????

[@semtex6412]

great idea bringing in GeoHot! subbing to this channel. hope u bring in more awesome people

[@LatentSpaceTV]

Thank you, we had a bunch on our podcast already from OpenAI, MosaicML, Databricks, etc but no full videos: latent.space/

[@j031t8]

Incredible.

[@kompila]

I am geohotz best fan from africa.

[@Abhishekkumar-qj6hb]

Aren't the compute intensive models also inference intensive models ?

[@abemore]

Can you use AlphaDev to make fast tiny code?

[@peter.g6]

Why I always feel if I could do 1/10th of what George can do my startup would be worth billions?

[@johnhammer8668]

14:54 whats the differnce ? tensor dot and tensor dot??

[@LatentSpaceTV]

If you have a tensor in variable `a`, you have to do `Tensor.add(a, x)` instead of `a.add(x)`

[@noninvasive_rectal_probe8990]

George is the reason I don't kms :3

[@jr8209]

audio levels low. You can tell by clicking on other videos and listening to how loud they are compared to this.

[@LatentSpaceTV]

Thanks for the feedback! We mix these for podcasts separately, we'll amp it up a bit next.

[@eirickbuckley9998]

jeez. not gonna lie, a bunch of this is wayyyy over my head.

[@codycarse]

GeoHOT

[@stevenbaron358]

Here at 1k

[@psibarpsi]

Where are the rest of the 17 episodes?

[@LatentSpaceTV]

latent.space/ and on all podcast providers!

[@recursion.]

Boys unarchive the previous episodes and looking forward to next one.

[@recursion.]

Great job on time stamps btw.😊

[@recursion.]

57:26 😂😂😂

[@recursion.]

Ai girlfriend is crazy

[@LatentSpaceTV]

You can find the audio + writeup here: www.latent.space/, more to come soon on here!

[@vev]

😊🙃

[@vev]

50 subscriber

[@LatentSpaceTV]

new channel! pls like n subscribe haha

[@dan2250]

George is the programmer’s programmer

[@ScottGrunwald]

I just realized listening to this how much Georgie sounds like Jason Calacanis

[@sandworm9528]

Except George has skills

[@markovchainy]

Great show but you need to process your audio better; remove noise, compress, normalise and high-pass filter to avoid the low thumps

[@LatentSpaceTV]

This was our old studio, we've setup a new one since then!

[@gauthamsai7506]

Addressing the turing complete issue blew my mind, ml workloads are not trying to find something exact. So hardware need not be tuned to be exact, it could be a finite piece of simple operation which moves over data, and that data movement in itself would give outcome

[@s243a]

Why not water cooling?

[@lukaspfitscher8737]

arent FPGAs clockless?

[@LatentSpaceTV]

In theory they can be clockless; in practice, no one really builds clockless ones.

[@losthighway4840]

A generalized framework is never going to match a behemoth like PyTorch on Nvidia. Reinventing the wheel here.

[@edh615]

I can't even search the episode

[@LatentSpaceTV]

www.latent.space/p/geohot#details

[@dogecoinx3093]

Raspberry pie version of apple silicone system on board chips

[@kenw8875]

gorge is Silicon Intelligence

[@larryabecid2819]

What? Is his comment on GPT-4 true?

[@Yuber898]

Flip a coin, heads is yes and tails is no

[@NikhilKumar-fo5on]

volume is pretty low, but good podcast otherwise!

[@MikeTrieu]

Yeah, it was so low that it fell under NewPipe's silence threshold and caused it to basically speedrun through the whole interview, making Hotz's speech barely liminal 😂

[@hooxenv9692]

what is this goddess of everything else thing and how did George know what that was?

[@YuriAchermann]

Maybe for next guests try to expand more the topic they are talking about, instead of just throwing new questions to bring new topic cutting the last one, it's more engaging...

[@gaussdog]

Rap Link, please..

[@geohotarchive]

kzbin.info/www/bejne/b5q4p6iXeM13e8k

[@LatentSpaceTV]

kzbin.info/www/bejne/b5q4p6iXeM13e8k You can always find all the links in the show notes: www.latent.space/p/geohot#details

[@wolfram77]

Why are the three of them wearing hedphones?

[@saint8082]

I can’t hear hotz

[@sunny_disposition]

26:35 "iTs NoT OpeN SoUrCE iF yOU duMP the sOurcE OnTo a giThUB rEpo anD thEN fORGet AbOUt iT uNTiL tHE nExT reEleSe... REEEEEEEEEE"

[@benjaminkaarst]

Why does this feel like they are giving him a technical interview?

[@arnaudjean1159]

I don't want only a copy of my memories in the stream I don't care a copy I just want me the original I just want keep on experimenting my own feelings keep my soul Wtf making a thing of a copy exept if your narcissic devious I don't care that bs

[@Muaahaa]

From their website: "We’re hiring software engineers to work on tinygrad. _In person_ in San Diego..." Why? Isn't this largely about contributing to OSS? Isn't the model for OSS the god father of remote work?

[@shephusted2714]

his ideas are already old - time marches on - if it was such a great idea he would already be shipping but it is just cots hw - anybody can make one, why pay him for the fun?

[@cosmicray007]

beware of the fast-talking man. attention is all you need !! 1:16

[@Pawsandthenclaws]

I'm trying to understand you bro. You are not a very good teacher.

[@ankursinghprime]