George Hotz | Just Chatting | how to actually win? | tinygrad.org

[@geohotarchive]

As per George request previous low quality (deleted) stream will not be uploaded. Read -> github.com/tinygrad/toonygrad/blob/master/PLAN.md github.com/tinygrad/tinygrad | tinygrad.org/#tinybox | Bounties for tiny corp / tinygrad -> docs.google.com/spreadsheets/d/1WKHbT-7KOgjEawq5h5Ic1qUWzpfAzuD_J06N1JwOCGs/ Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org/#tinybox from $1250 buy -> comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters and twitch chat: TBD

[@peterslaboratory6070]

Gratz on 200k subs! We all like to watch George

[@RegiJatekokMagazin]

Ye so we dont need evolution, if there is no competition and everyting is free, reproduction was the base of life. Its still easy to my 3 IQ. Also I solved M theory, the Dad universe fucked the Mom universe and thats how our universe got created. XD. Ok forgot it , it was a joke. Also I know, if there is more knowledge internalized within peoples brain, and peoples getting more intelligent and smart, they will harden each others work, instead simplifying it, and they lose a lot of time with each other, same as with not knowing something. Also they can invent new things to make money with, new sickness, new smart phone, new house building techs. etc. it just has to be infected to them, so theyr influenced by. Also having different memories, different interactions with the world we used to live when we were young, it also interferes with our thinking process, thats what environment we are in, so we have to control what informations we want to internalize and what we dont want to get involved with. Also I think intelligence can be learned, for some its harder, fo some its easy or already they have higher IQ. Also I believe when we bind feeling to something, as mathematics, or programming, or cooking, playing guitar etc. we tend to have a big drive to reach those goal and get into those areas more deeply, its more at young ages its more like to influence you. I think, if we can make a basic value set, and goals, we can control our self, if we dont have that, others more likely to control us, and our environment and memories. So basically everything manipulates us around, the environment and the peoples, and things. If we can make own personality, own goals, and values, we can control ourself even there are other peoples around trying to make us sick. Peoples even at higher IQ are so simple, they just reverse things psychodynamically, the best weapon is critical informations and attacking back versus an attack. If someone would know how everything works at the universe, how he was able to get that information, and doesnt he became the universe itself? Doesnt the universe drived to him that road? If I pick up the phone and I start to do something on it, is the phone controls me or me the phone? If I buy an apple, and I eat it, the apple wanted it or me to eat it? Doesnt the apple has conciousness? Do we have conciousness? Or the apple has partial conciousness for partial time, if it becames one with us? Is the human the observer or the apple? Is information delayed to make action an observer? What is temperature and gravity? How does physics make sure its functioning at 100% integrity? What is time?

[@LampeAlcoholic]

Love the just chatting sessions - thanks for uploading

[@saykharng]

These are the best

[@duke618]

George went from a hoddie hacker in his house to 40 years old teacher in a class

[@faustozampa]

this is top tier. Please do more free thinking/ reading streams George

[@chickenwings273]

george uttering his thoughts and philosophy is the best kind of stream

[@Gome.o]

@1:26:32 is the proverbial money shot: "Once we've built an abstraction layer that makes the training job independent of the GPU, we can absolutely compete in this market. We can do even better by analyzing the workflows and the types of jobs people are actually submitting. This will allow us to determine which chip is best optimized to run those workflows. We can then build that chip and seamlessly switch it out for our customers."

[@Gome.o]

Edit* there's even more mind-blasting stuff after this point, I take back what I said about this being the money shot. "Build tech that's intextricable from its narrative" So many geohot bangers. Possibly my favourite vid to date.

[@snarkyboojum]

Like the larger hyperscalers have been doing for 10+ years with standard compute? That's a lot of catching up to do.

[@bio_stack]

Two things that make open source work: - it prevents rentseeking - it guarantees some future progress if you do it yourself Comma only provides the first one cause its a closed dataset and training code.

[@houkensjtu]

George is drinking Japanese tea instead of coffee nowadays. Legend.

[@-mwolf]

In a society of super-abundance, cooperation beats competition. Life wouldn't exist without cooperation. The degree of cooperation is a sign for the development of a society.

[@danielfaller5617]

Isnt it the other way around, when you are struggling to meat basic needs you will cooperate other people, you have to, this is partly which pushed people to build civilizations, but when you have abundance than you can afford to be destructive, when you have100 years worth of food in youre storage, you can shit on other people.

[@-mwolf]

@@danielfaller5617 Good point! I left out the most important bit: For primitive classless societies (hunter gatherers) this was the case as you described. With the development of classes, i.e. those that work and those that just own and profit off of other's work, this changed. We progressed in certain aspects, but this didn't change the fact that competition dominates. Through this progress, we now have the means to move back to cooperation "from the realm of necessity to the realm of freedom", if we abolish classes, i.e. private property of mean if production, which allows us to plan the economy rationally (as opposed to the irrational planning that is happening now, with (unhealthy) competition between companies and nation states (wars, patents, unfree software, pollution, misleading or poisoning/addicting customers, ...) or lack thereof (monopolies)). I see this analogous to Michael Levin's work... like cells coming together, developing higher goals than any individual could, improving the overall prosperity. Under capitalism, this is weak. Cancers dominate. We need to scale the lightcone of humanity. I don't have time to go into details here, but for the historical background, I can recommend reading "the origins of family, private property and the state" by engels.

[@peternguyen9995]

I’m not a quant dev or prepping for an interview, but I subscribed to you after this video because I love Math and understanding low-level systems. Good stuff.

[@opioman2429]

When I can't concentrate on my code, I listen to this in the background. I'm more productive afterwards.

[@cgnomazoid]

Hey George if you see this next stream could you discuss how you think we will go about merging training and inference? I’m curious why this isn’t the biggest open problem being worked on in AI, it seems fairly obvious that we have to figure that out.

[@tejasahluwalia8713]

What's most profitable is not always obvious. It changes depending on the timeframe you are looking at. And if you try to look at longer timeframes, it gets very hard because you are basically trying to predict the future.

[@pytheas222]

compute will never be distributed because energy usage isn't

[@pytheas222]

whoever has the most energy expenditure available to them will also have the most energy allocated towards computation. I can imagine it being a couple of American companies, the us gov, and maybe the government of China could be players, but the average person wont be able to compete in terms of energy or unified objective.

[@DavideZamblera]

@@pytheas222I agree, in the competition between social groups an individual will hardly ever matter. Competition between individuals is a whole other story. I think it is not impossible to imagine a society where some people have their personal computing infrastructure powered by local solar plants, giving them a professional edge over other individuals.

[@pytheas222]

@@DavideZamblera i think energy and money should be viewed almost interchangeably, the dedication of large amounts of energy towards specific compute tasks requires resources that are distributed throughout society in a pareto distribution.

[@DavideZamblera]

@@pytheas222 there is no reason for the compute power to be distributed equally throughout society. I think it is too soon to speak about numbers but a fair amount of distribution, in a highly developed nation, is not an impossibility in my opinion. Of course the compute tasks will always remain limited with respect to the ones pursued by nations or other organizations.

[@kinvert]

tinygrad please help get rid of the aphids in my homestead. Switch to gardening captchas instead of all these stop lights. Obeying stoplights is blue pilled. Massacring aphids is one step toward becoming ungovernable. Homeschool your kids and raise your own chickens my people.

[@doooofus]

Help this n1gga tinygrad fr @georgehotz

[@souvikbhattacharyya2480]

I miss the old time stamps :(

[@hanchi8710]

i miss the old kanye

[@edosrotogati]

Books mentioned (description by AI). There may be others but I haven't seen the whole thing... Richard Cantillon, An Essay on Economic Theory - published posthumously in 1755, is the first comprehensive work on economics, introducing key concepts like entrepreneurship, supply and demand, and the circular flow of income. It laid the foundation for classical economic thought. Greg Egan, Permutation City - is a science fiction novel exploring virtual realities, consciousness, and the nature of existence, where characters upload their minds to simulated environments. It explores philosophical questions about identity, free will, and the possibility of infinite simulated worlds.

[@JasonKT13]

Atlas Shrugged by Ayn Rand was also mentioned

[@tylerhanson133]

Always like listening to you. Great motivation, great thinking.

[@bilalazhar4495]

WE are gettng streams almost everyday now ??? are we blessed ??

[@Brymcon]

Log in to perplexity and pick the advanced reasoning to use the pro feature.(Only o1 mini). The synthetic data producer. Still for 50billion parameters mini is unreal. O1 preview is the teacher with 500billion. parameters. Thats high quality data.

[@nakunam1256]

I randomly like or comment on videos, and this channel gives me a pleasant vibe, George. Keep doing what you're doing and keep up with great content!

[@felicityspyder]

Google Fiber and Google Edge Network came to mind when thinking about infrastructure

[@Radium3D]

i switched to oled too, set brightness to 50-60% max

[@bollvigblack]

I was saying this many years ago on this channel, telling George to ditch that autopilot idea and build an AI similar to ChatGPT or Claude. The reason I suggested this was because George was one of the people who could actually make it happen. If he had done it, he would be rich by now. But of course, he thought my comment was stupid

[@HeshamEldawy-ng9hf]

And he still thinks lol

[@bollvigblack]

@HeshamEldawy-ng9hf that's why hes still broke somewhere in china. classical dev mindset

[@JasonKT13]

@@bollvigblack He would still be programming somewhere in china even he got rich tbf. The problem is how little leverage he now has to implement the ideas he mentioned, although I still think it's a great move looking on a 5-10 year time span. I feel like he picked the wrong problems to solve so far, maybe because he wanted to beat Elon so bad. He even admitted here that he can't build data centers just like Elon can (leverage), he seems more grounded now and opting for a complementary vertical, rather than a straight up disruptive one, might be it for him this time.

[@ilovepainting1996]

I'm confused what the interface is for CLOUD=1; clearly it's rest api saw the /buffer route etc. Is this some open cloud compute api spec I haven't heard of? Or is it just tinycorp's new API for the tinyboxes that geo building.

[@oodyyftnnctewiomc]

I like these kind of streams better because I don't have to follow some context on your screen and can listen to your thoughts while working.

[@aryanwinningson7955]

No market is "free" Each of them is fine tuned by the government.

[@diggerkropkapl]

That's actually an underrated comment.

[@alienkishorekumar]

Became a saint at the end.

[@nikhilreddy9112]

Where was yesterdays stream? I missed most of it, but remember a different background. ARE YOU GATEKEEPING???

[@zednik8956]

hes in china he talked about it in a prior stream

[@eyobbekele3231]

The Tinygrad project is to machine learning what Nix is to package management??.

[@zion4688]

Did you ever use obsidian or anki ?

[@wedding_photography]

Please install a VSCode extension that does grammar/spellchecking.

[@vinith3773]

Oh there's a market. I dont want to deal with customs or even have to find out if my power bill is going to be reasonable with tinyboxes. Too much of a commitment. Would definitely buy compute at a different abstraction level that can be metered and proved. I think in AGI is about faster and more experiments and this tries to make the market be about that (it already kind of is). You don't need perfect centralisation or decentralisation there is a whole gradient of marketplaces between.

[@MEMOKSAxD]

this stream is gold

[@oscbit]

Why isn't George picking up MLIR? Seems like the perfect fit for his plan.

[@robloxminecraft9055]

what did he talk about in the previous stream?

[@More_Row]

Why do you need someone to tell you. Go check

[@abrokenmailbox]

@@More_Row it's not uploaded

[@More_Row]

@@abrokenmailbox ? It is uploaded

[@abrokenmailbox]

@@More_Rowthe cloud=1 one isn’t the latest stream

[@geohotarchive]

@More_Row twitter.com/geohotarchive/status/1845941422647918727

[@Dave_0mo]

What vscode theme is he using?

[@Fibra_GB]

Why despair because of incomprehensibility? If there's anything the 20th century has to teach us, is that this is part of the world perceived by a limited system. We can of course strive for better abstractions of the world, but our view is always going to be incomplete. Embrace the "control" slipping though your fingers, and you'll understand that we never had it. The future, ironically enough, will "belong" to those that have the best abstractions to understand it, not the best technology. In this respect, metaphysics was, is, and will always be the biggest frontier. It allows you to morph your entire knowledge body, by shifting your perspective and underlying assumptions you've about the world.

[@kv6257]

are you 16-20 years old, and have russian/ukrainian/bielorussian/polish ancestry ?

[@benslic]

@@kv6257 Are people with russian/ukrainian/bielorussian/polish ancestry more into metaphysics? What was your intuition behind the comment?

[@mistervanderveer]

Does improvement have an E?

[@the_yugandharr]

thanks for uploading!

[@Arcticwhir]

Perplexity is decent, i'd still be weary of hallucinations - if you really look deep into the claims sometimes - its a reason why i dont like this recent trend of people on twitter screen-shotting perplexity searches and using it as a "source" in replies.

[@Aedonius]

congratulations your intelligence is working

[@Arcticwhir]

its not only a question of hallucinations, but also the source of that info. that 7.7M figure was a rumor, the actual values can be found on nvidia site, for Llama 3.1 405B - its 30.84M GPU hours. So if your going through all this time anyways to double check the sources - how much time is it really saving you. pros and cons..

[@bradm8529]

Agreed although Perplexity Pro seems to provide better results across more sources

[@JohnboyCollins]

Ha, how about a PM on performance of different jobs specs after training?

[@PrabodhGyawali]

Please tell George Hotz about Complexity Extension of Perplexity it is really great

[@timjenkinson26]

He should somehow convince Justine Tunney to work with him together they would make something magical.

[@-mwolf]

Planning doesn't need to be central... Scale free structures, small wordlness. Aka democratic centralism. Works for the brain, will work for the economy.

[@ArkaSaura]

Why can't you use Nix to really define all sorts of things that would allow it to scale infinitely? Shouldn't it be possible to implement all the abstract server side processes through Nix and have it declaratively deploy anywhere, defining infra requirements, dbs, etc. and have it be a singular source of an entire system?

[@vslow4795]

The reason Nvidia is so successful is that they build general purpose compute. I agree with a lot what George says, but specialized hardware will not win.

[@-mwolf]

The best we can hope for is a denocratically planned economy... What the marked does with our societal resources we can clearly see... What we need is true democracy. All compute divided by all humans. Allocate it to your will.

[@pytheas222]

will mycelium networks generate a random number???

[@Zeroduckies]

I like the way you think

[@caien729]

try keeping twich chat in captions

[@marcosramirez8056]

32:23 this tends to happens to me also, i am not alone haha

[@mrwenr0347]

I think the problem is gpu owners want more revenue, whether it's tinygrad or other platforms.

[@egor.okhterov]

Why do you go this same hype route with GPT that big companies pursue at the moment? Let them spend billions and get 1% extra on their benchmarks. Focus on ARC challenge. Think of type II reasoning and planning. Think of the ways how to do a combinatorial search through discrete space. GPT narrows down the search space and then we do the reasoning phase. It's the same as with chess. First you get intuition about the board position and then you calculate the possibilities withing the reasonable space. And there has to be back and forth bouncing between interpolation (GPT) and combinatorial search. Everyone is deeply invested with deep NNs, so there's no value in herding this route. The most value for us as individuals lies in exploring how to perform the reasoning phase. ARC challenge is a nice playground to test ideas.

[@egor.okhterov]

kzbin.info/www/bejne/qWjCf5-hd9yXoJo

[@-mwolf]

"AI infra must be taken granted for free like linux" ... so should music, games, pictures, patents, ... so should all of this stuff that *is* basically free when we divide the costs of production by the amount of consumers forever after. What's holding it back is the private property to these things, upkept by the state.

[@-mwolf]

Btw, if you follow this to its logical conclusion, of course infra like food, water, housing, internet and healthcare should be granted for free as well. We have the means to do so. It's an organizational problem of society, we've long passed the technological hurdle.

[@timjenkinson26]

So is he trying to build something like Edge Impulse? It sounds very similar.

[@iantimmis651]

He changes the goal every few weeks

[@timjenkinson26]

@@iantimmis651 I think he just got a little distracted by politics and went a bit doomer. He seems to be pulling out of that now

[@bio_stack]

Expressing entire machine learning systems in a simple, non turing complete language seems difficult. A bit like inventing C in the 1800s. Or like some linus writing an operating system that surpasses microsoft. The marketplace for training runs could be interesting though, maybe the math is a few kb, but what about the (stolen) dataset?

[@Paul-iv8st]

>cries about rent seeking >rent seeks at 46:45 Edit 54:05 again rent seeking hypocrite

[@simonk2036]

Whats your Keyboard?

[@hanchi8710]

hhkb pro. this one sounds like type s

[@simonk2036]

@@hanchi8710 Is this a mechanical keyboard

[@ChubaiosBoomers]

​@@simonk2036no hydraulic

[@ThaLiquidEdit]

I would really appreciate python typing in this project. How can you know what variable is what.

[@hoeszyslak6989]

Brain power

[@ThaLiquidEdit]

​@@hoeszyslak6989 This gets more and more unfeasible in my experience.

[@Zeroduckies]

Variables are for the weak. Direct memory access runs better 😂

[@shouganaitekitou13]

I love your brain.

[@mikestaub]

What worries me is that eventually AI will get so good ALL software will be commodified so there will be no way to rent seek anymore and thus the investments will dwindle which will cause stagnation in innovation as everyone fights to hoard more Bitcoin, as it will be the only thing the AI's can't deflate.

[@pavelhegler]

Are you okay George? Last video I saw you, you were on comma AI and it appeared like you were doing a lot better :/ I am a little worried about you since you were the one who inspired me to get into vim and propelled me to one of these super nerds doing stuff no normal human being around me understands...

[@minimalist_trd]

Really interesting that this guy never got into markets and trading algorithms...

[@leon2385]

This is awesome

[@TonySmith-tz7tp]

Is driving Chill enough for you??????

[@micha74774]

1:13:00

[@mementomori8991]

Why did he move to china?

[@fanpeter-z3c]

cos he's chinese

[@mike-yii]

he is visiting his grandma

[@ahmadshabaz2724]

Big fans sir .

[@NoobaLV]

Goodbye! See you next year

[@Iknowwereyousleep289]

Models do develop the same representations objective reality exists the market is going to end

[@Iknowwereyousleep289]

Libertarians live in kind of a subjectivist world Platonic representation hypothesis is a great anti-thesis to that and a great paper

[@commonlogic3646]

Do meditation on stream again

[@Dermotten]

I believe you're confusing profit-seeking with rent-seeking. According to wikipedia, rent-seeking provides no economic value. PerplexityAi in their attempt to stay alive and pay for their infrastructure, is trying to charge their users a fee for use. This is profit-seeking, not rent-seeking, because they are providing their users value. Even if we may not like the UX of how they ask for money, that doesn't make it rent-seeking.

[@Kunal-k-u-m-a-r]

how to you a new language from scracth

[@tear728]

Tinygrad is no longer tiny 😢

[@abdullahziker]

Cheap ETH is giving me scam ideas ......is that normal??😅😅😅😅😅😅

[@arthur-zhuk]

Total misread/play moving out of the states...

[@natpainter8185]

netflix circle jerk is the market right now. you were dopping nuggets the whole time. would you want options in tesla/musk or meta/zuck if you were not a tier0 programmer? I understand but dont. get it? anywho, good luck and Gspeed Hotz

[@-mwolf]

Communism == maximum agency for the individual. Giving people control over their own lives, workplaces, society. Capitalism is a democracy for the owners of the means of production. It's agency for the wealthy. But "If we ask the right questions, it is much more likely to find the meaning of human existence. As a heuristic in the search space, we should try to increase the diversity of people (or conscious beings) asking such questions". We don't want to optimize the proxy profit. It is irrational if our goal is to maximize progress and prosperity

[@-mwolf]

There is no diverse competition under late-stage capitalism.

[@LeMonke3]

George zoom in at least one more level on your code wtf.

[@Zeroduckies]

Anyone wants to buy a kidney? I need me a tiny box pro 😊

[@sadmanhossain4079]

Hi George i am a nonsubscriber 😄

[@dimanft6160]

i eat smegma

[@Aedonius]

fuck you for typing the most disgusting combination of characters possible

[@vercolgo]

NO.

[@jonathanrandrianarivony7639]

Jailbreak iPhone xs max please😭

[@Ne-lm7dy]

free market is a fallacy bud

[@whome1319]

More like you need to understand Lump of labour fallacy

[@dembert4666]

You are a retarded communist

[@tear728]

​​@@whome1319 a free market is non-existent, just like capitalism and communism are non-existent. These are just "platonic" abstractions

[@literailly]

Y tho

[@JoaoBatista-yq4ml]

​@@literailly When China opened its markets and many companies invested in factories there, a lot of people criticized it for "slave labor", exploitation, etc. Despite all of those criticisms, China is now incredibly wealthy and tons of Chinese were lifted out of poverty. It still has its problems but it improved a lot by having freer markets. People usually dislike free markets because it's easy to take to take wealth for granted. Conversely it's hard to realize that human beings are born in poverty and there's a whole process to get out of it.

[@interests3279]

Sam Altman rent seeking? Cool it with the antisemitism.

[@בעלזבוב-ס6ע]

bros a genius