George Hotz | Researching | Chatting | rewriting the linearizer (tinygrad) | TVM | People | Part 1

Date of the stream 27 Jan 2024.
from $1250 buy comma.ai/shop/comma-3x & best ADAS system in the world openpilot.comma.ai
Live-stream chat added as Subtitles/CC - English (Twitch Chat) - at the bottom - Show Transcript
Follow for notifications:
- twitch.tv/georgehotz
Support George:
- twitch.tv/subs/georgehotz
Pre-order tinybox:
- buy.stripe.com/5kAaGL6lk9uX9nW144 (tinygrad.org/)
Chapters:
00:00:00 intro
00:00:10 rack, donuts, full time android
00:02:50 a brief history of intelligence
00:04:25 skibidi toilet
00:06:20 hipstreamwritevalue32
00:10:40 linearizer
00:15:30 shapetrackers and group for reduce
00:16:16 graph
00:17:10 ADD and LOAD, topological sort
00:20:30 Alex
00:23:15 linearizer great code
00:26:00 most garbage code in tinygrad
00:27:30 difference, liniar list of code
00:29:20 Alex
00:31:00 new linearizer
00:31:45 kernels that can store more than one thing
00:32:50 food
00:34:20 buffer loads, iteration, shape
00:36:40 how do compilers work
00:38:00 chris lattner
00:39:00 color dimensions
00:40:00 tvm, optimism work, thinking what you want to write
00:41:40 installing tvm
00:46:00 jonathan blow precedence compiler
00:48:20 binding axis to gpu variables
00:50:30 cse_var_2, common subexpression elimination
00:51:10 reading documentation from others
00:52:30 not thinking
00:53:55 tvm_gemm.py
00:55:20 what is tvm
00:55:40 schedule API in tinygrad
00:56:10 linearizer
00:57:10 jira in companies, project management
00:59:40 jira board, complexity, work signaling
01:00:40 tiny corp board
01:01:35 better expression of ideas
01:02:00 Proofs from THE BOOK
01:02:25 scrum, sprints at comma, agile
01:04:00 above the API line
01:04:40 comma is going to beat tesla in FSD, adding value tomorrow, V12
01:07:00 if anyone want's to write tinygrad docs, tutorials like tvm
01:07:40 blame, working with elon, companies structure
01:09:00 non technical project manager
01:10:10 better programmer than people you manage
01:11:20 senior vs junior
01:14:49 type inference
01:15:45 junior engineers, learn lessons the hard way
01:17:10 not getting more intelligent
01:18:50 competition programming, high schoolers
01:20:20 spam programming vs slow high quality programmer
01:21:55 codegen and scheduler
01:23:20 halide tvm
01:25:10 complex, chris lattner about tinygrad, compilers
01:28:20 model optimization tvm
01:30:00 building tvm, tvm output verbose code
01:33:00 tvm same ideas as tinygrad, tensor accelerator
01:35:00 group_for_reduce, kernels
01:37:00 searching for the right name
01:40:00 graph rewriting
01:40:50 break
01:48:56 hhkb professional hybrid type-s
01:50:15 blueprint name
01:51:10 ajusting new keyboard
01:57:50 vscode spaces
02:01:50 Alex
02:04:30 typing on new keyboard
02:06:30 new linearizer concepts
02:16:50 group
02:20:30 comments overrated, wrong documentation
02:26:00 ecromata.com
02:27:00 hammer = perplexity
02:28:00 land, managing people, plans
02:31:40 starting your own company, selling land
02:33:30 no capital constraints, how do you deploy $500M
02:34:20 delusional founders, what limits the growth, 10 new comma people, integration to the company
02:35:00 working together superlinear effects, scale book
02:36:00 practical factorization of the problem, limiting factor of growth
02:36:15 complex problem, breaking into appropriate parts
02:37:50 Tesla, Elon average decision better
02:39:20 management coefficient, people
02:41:10 cult followers, dedicate your life, hitler, burning man
02:43:20 social media, dating apps, quality
02:44:10 low and high buyin
02:45:30 hippie cults 60s failing, machines, energy
02:46:30 robots, people who want to take way more than taking back
02:48:40 bounty systems, burning man works, people, social dynamics
02:51:00 what are people, how do they work together, power in system, market dynamics
02:55:30 inadequate equilibria
02:56:00 the bible stories, stoicism, modern scientific people dynamics, racial differences in intelligence
02:57:40 what can, can't you change, how change happens
02:59:20 clones scaling
03:02:30 power on the land, solar panels
03:08:35 projected revenue numbers, fake wineries
03:09:50 horse boarding, we live in capitalism = good
03:11:30 aravind srinivas perplexity, tinyboxes on land
03:15:10 degree value, poker
03:18:15 land location, road to land
03:21:40 market already priced it in, beat the market
03:22:10 linearizer 2 week project, more inteligence vs clones, piano
03:25:00 expensive housing, how to make money, fraud, ben shapiro rap
Official George Hotz communication channels:
- geohot.com
- realGeorgeHotz
- georgehotz
- tinygrad.org
- geohot.github.io/blog
- github.com/geohot
We archive George Hotz and comma.ai videos for fun.
Follow for notifications:
- geohotarchive
Thank you for reading and using the SHOW MORE button.
We hope you enjoy watching George's videos as much as we do.
See you at the next video.

[@geohotarchive]

ecromata.com | Bounties for tiny corp / tinygrad -> docs.google.com/spreadsheets/d/1WKHbT-7KOgjEawq5h5Ic1qUWzpfAzuD_J06N1JwOCGs/ kzbin.info/www/bejne/op-5gqaAf6uWmsk Hiring entire stack for tiny corp join if you are interested | kzbin.info/www/bejne/op-5gqaAf6uWmsk work major source of value in your life Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | github.com/tinygrad/tinygrad comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 intro 00:00:10 rack gone, donuts, full time android 00:02:50 a brief history of intelligence 00:04:25 skibidi toilet 00:06:20 hipstreamwritevalue32, hipstreamwaitvalue32 00:10:40 linearizer 00:15:30 optimizations for shapetrackers and group for reduce 00:16:16 graph 00:17:10 ADD and LOAD, topological sort 00:20:30 Alex 00:23:15 linearizer great code 00:26:00 most garbage code in tinygrad 00:27:30 difference, liniar list of code 00:29:20 Alex 00:31:00 the idea of new linearizer 00:31:45 kernels that can store more than one thing 00:32:50 food 00:34:20 buffer loads, shape, buffer iteration 00:36:40 how do compilers work? 00:38:00 chris lattner 00:39:00 color dimensions 00:40:00 tvm, optimism work, thinking about what you want to write 00:41:40 installing tvm 00:46:00 jonathan blow precedence compiler 00:48:20 binding axis to gpu variables 00:50:30 cse_var_2, common subexpression elimination 00:51:10 reading documentation from others 00:52:30 not thinking 00:53:55 tvm_gemm.py 00:55:20 what is tvm 00:55:40 schedule API in tinygrad 00:56:10 linearizer 00:57:10 jira in companies, project management 00:59:40 jira board, complexity, work signaling 01:00:40 tiny corp board 01:01:35 better expression of ideas 01:02:00 Proofs from THE BOOK 01:02:25 scrum, sprints at comma, agile 01:04:00 above the API line 01:04:40 comma is going to beat tesla in FSD, adding value tomorrow, V12 01:07:00 if anyone want's to write tinygrad docs, tutorials like tvm 01:07:40 blame, working with elon, elon's companies structure 01:09:00 the concept of non technical project manager 01:10:10 better programmer than people you manage 01:11:20 senior vs junior engineers 01:14:49 type inference 01:15:45 junior engineers, learn lessons the hard way 01:17:10 not getting more intelligent 01:18:50 competition programming, high schoolers 01:20:20 spam programming vs slow high quality programmer 01:21:55 codegen and scheduler 01:23:20 halide tvm 01:25:10 complex, chris lattner about tinygrad, compilers 01:28:20 model optimization tvm 01:30:00 building tvm, tvm output verbose code 01:33:00 tvm same ideas as tinygrad, tensor accelerator 01:35:00 group_for_reduce, kernels 01:37:00 searching for the right name 01:40:00 graph rewriting 01:40:50 break 01:48:56 hhkb professional hybrid type-s 01:50:15 blueprint name 01:51:10 ajusting new keyboard 01:57:50 vscode spaces 02:01:50 Alex 02:04:30 typing on new keyboard 02:06:30 new linearizer concepts 02:16:50 group 02:20:30 comments overrated, wrong documentation 02:26:00 ecromata.com 02:27:00 hammer = perplexity 02:28:00 land, managing people, plans 02:31:40 starting your own company, selling land 02:33:30 no capital constraints, how do you deploy $500M 02:34:20 delusional founders, what limits the growth, 10 new comma people, integration to the company 02:35:00 working together superlinear effects, scale book 02:36:00 practical factorization of the problem, limiting factor of growth 02:36:15 complex problem, breaking into appropriate parts 02:37:50 Tesla, Elon average decision better 02:39:20 management coefficient, people 02:41:10 cult followers, dedicate your life, hitler, burning man 02:43:20 social media, dating apps, quality 02:44:10 low and high buyin 02:45:30 hippie cults 60s failing, machines, energy 02:46:30 robots, people who want to take way more than taking back 02:48:40 bounty systems, burning man works, people dynamics, social dynamics 02:51:00 what are people, how do they work together, power in system, market dynamics 02:55:30 inadequate equilibria 02:56:00 the bible stories, stoicism, modern scientific people dynamics, racial differences in intelligence 02:57:40 what can you change, what you can't change, how change happens 02:59:20 clones scaling 03:02:30 power on the land, solar panels 03:08:35 projected revenue numbers, fake wineries 03:09:50 horse boarding, we live in capitalism = good 03:11:30 aravind srinivas perplexity, swimming upstream, tinyboxes on land 03:15:10 degree value, poker 03:18:15 land location, road to land 03:21:40 market already priced it in, hard to beat the market 03:22:10 linearizer 2 week project, 5 times more inteligence vs 10 clones, piano 03:25:00 housing expensive, how to make money, fraud, ben shapiro rap

[@defini7]

Dude, you're so smart that i haven't even understood the "skibidi toilet" part

[@micuzzu]

got the hhkb hybrid type s too and love it

[@kenqz2623]

We are getting George promoting skibidi toilet before GTA 6

[@lynix48]

no timestamps? :(

[@geohotarchive]

@lynix48 timestamps in progress. will post them tomorrow if not finished today. EDIT: finally, timestamps finished and posted. Enjoy.

[@lynix48]

@@geohotarchive ❤

[@gaganghotra]

​@@geohotarchive thanks for your efforts 🙏🏻🙏🏻

[@k_otey]

​@@geohotarchivehow much time have you spent on this channel for timestamps alone?

[@TLOZ1986]

​@@k_oteyi think 1-2 hours 🤣

[@KALLAN8]

there is a big difference between lying and being hardcore capitalist. it is totally possible that the kid with the starlink idea was just a rooky at non-disclosure and Elon cut him out while not needing to actually lie. bedrock of the Christian ethos is truth 🙏🏼

[@Sapientiaa]

I also got the same keyboard several years ago. I love it.

[@BritishBungler]

Does anyone know whether HIP/ROCm are going to support AMD's desktop APUs (processor + iGPU + AI Engine)? Like the 8700G or 8600G (desktop SKUs). I saw recent news that they'll support Strix Halo APUs, but that's a laptop part coming out in late 2024 or 2025...

[@dave4347]

George, if you want to make a quick $100k, put your old HHKB up for public auction.

[@viacheslavprokopev8192]

When are we getting AOC to join the American accelerationist party?

[@notyourbruh]

soon

[@oggan97]

1:04:28 What does he mean with "the api line"?

[@geohotarchive]

You either tell robots what to do, or are told by robots what to do. www.ribbonfarm.com/2017/08/17/the-premium-mediocre-life-of-maya-millennial/ rein.pk/replacing-middle-management-with-apis www.forbes.com/sites/anthonykosner/2015/02/04/google-cabs-and-uber-bots-will-challenge-jobs-below-the-api/

[@oggan97]

@@geohotarchive Thanks! Interesting, learned new stuff

[@ultrasound1459]

We doing Linear RIZZ today😂hell yeah 😎

[@aleksay2142]

@geohotarchive dude, if u use hhkb with wire most of the time, just remove those batteries… it is written in their manual as well… helps with natural capacity decrease. I just put AAs when i need it as mostly using with just wire… Yeah and hhkb should have sponsored you. I know couple of people including me who have started with hhkb because of you.

[@MasamuneX]

its all about writing good rock solid code that is robust and can do many things without the need for 100,000 lines of code

[@sid-prod]

Mr. Hotz after noticing your doughnut choice I've decided to work for your company (not a sarcasm)

[@daniellanciotti8908]

He is just amazing

[@shawnali7121]

what color scheme is that hes using for vs code and his terminal?

[@kevinmccallister7647]

Default one

[@mrethantheb]

It seems to me the managers aren't technical because the technical people don't genuinely want to manage.

[@mikaackermann4072]

No timestamps..?

[@geohotarchive]

@mikaackermann4072 timestamps in progress. will post them tomorrow if not finished today. EDIT: finally, timestamps finished and posted. Enjoy.

[@pulisichhh]

I literally just got the same keyboard but Snow version, it's pretty good but I never had a non-type-s so I can't tell the difference. Though 320$ USD hurt like a bitch, that's a month salary from where I'm from lol.

[@5kr0dy16]

My 6 yr old watched skibidi toilet constantly. I feel you two would have similar interests in videos.

[@davedei3955]

Did he call Z fold 5 is the "greatest phone ever"? Or was it a different one?

[@RyanMoreau]

@3:05 this is what the Chinese do

[@LetsPlay30497]

so what keyboard is he using?

[@ayayawei2007]

Hhkb type s, holy grail/end game of keyboard

[@pseudosquilla]

6:56

[@hamzaomari7052]

The font is so small dawg

[@davidgardener1152]

i just skipped to the bit where your food arrived, i only follow your channel for the eating ASMR content. this AI stuff will never catch on.

[@frydegz]

no shot in hell he opened the stream with skibidi toilet

[@joseVte]

You will always be my god, from the moment you hacked the PS3

[@SignatureScentReviews]

Hotz....have you given up on making beats? And another question...Do you prefer Python over C?

[@j4s0n67]

I mean he is making an ml project sooo im guessing python

[@SignatureScentReviews]

@@j4s0n67 I know. But Python is the goto for ML. But he’s a very good C coder. I used some of his C code many years ago. He was very good for a kid at the time. That’s why I’m asking. I learned C in college so I was very impressed at his code.

[@somerandomidk7091]

George not sure even if you are the admin behind the channel but I must ask. I know that you support self taught education and believe that Universities are a scam e.t.c and that you've actually dropped out from CMU. But how did you learn all that ? I am not talking about the coding specifically but the theory and math behind it, especially the math. Did you learn through your projects ? Did you learn from books ? I generally believe that even though it might be a bit hard, you can learn anything with discipline and motivation, but what is the correct way to do it, or how did you do it ?

[@ojaypopeyee179]

Brother you answered your own question geroge doesn't have the answer to that you do just jump right in deep and willing into any topic and within month you will be decently knowledgeable on it

[@Siroitin]

I recommend to read at least his Wikipedia page. He got recognized first time 2004 and 2005 when he competed in "a science competition for high school students". I think that might have helped his career

[@moviesynopsis001]

Just start learning whatever interest you. George has deep passion for this, he doesnt need an A from some old professor to let him he know he is good.

[@Siroitin]

@@moviesynopsis001 learning "what ever" isn't so easy.

[@moviesynopsis001]

@@Siroitin It is if it interest you, doing things that dont is near impossible to sustain

[@Gff-n1e]

George George George

[@keebmat]

one of us! one of us! (hhkb) 😅

[@Towersfam43232]

next thing you know he creates evil corp XD

[@faceofdead]

no mute start and no timestamps. what are you doing with your life, mr. Hotz?

[@geohotarchive]

@faceofdead timestamps in progress. will post them tomorrow if not finished today. EDIT: finally, timestamps finished and posted. Enjoy.

[@haxurn-h4l]

bro you amazing

[@zoltanhorvath2238]

tipic american guy, lets order some dunkin dnts :D

[@TheChipMcDonald]

29:23 (wakes up) WTF WAS THAT?

[@aga1nstall0dds]

Jira and agile r for people who doesnt know how to play chess

[@gianni2729]

0 clue what's going on this entire video haah

[@okay7262]

LOL,HHKB~~~~

[@maximmelnik7998]

Second!

[@HimanshuKumar-zx1ll]

😅3rd :p

[@dichiara]

First :D !!

[@hrmf32]

Mr Hotz don't take me wrong but i think you can try to use AI on code editor (bard, chatgpt, etc) and it enhances some steps you do on code also things like moving blocks code, fixing errors, autocomplete and many many more, give it a try. At some point can be useful in vscode or neovim can be added.

[@NodirjonMuhammadaliyev]

4 th😂

[@GraczPierwszy]

when gpt 5?