The 10 Trillion Parameter AI Model With 300 IQ

[@chapterme]

Chapters (Powered by ChapterMe) - 0:00 Coming Up 0:54 What models get unlocked with the biggest venture round ever? 5:35 Some discoveries take a long time to actually be felt by regular people 9:53 Distillation may be how most of us benefit 14:26 o1 making previously impossible things possible 21:17 The new Googles 23:47 o1 makes the GPU needs even bigger 25:44 Voice apps are fast growing 27:05 Incumbents aren’t taking these innovations seriously 31:52 Ten trillion parameters 33:15 Outro

[@thehari75]

Lol did sam altman tell yall to avoid talking about claude and comouter use update

[@sakchais]

They talked about developers in their batch switching from OpenAI's models to Claude's models at around the 12:50 mark.

[@hanchisun6164]

One order of magnitude higher is referring to compute, not parameter size. The current frontier model (4o/pro/sonnet) is all much smaller than their previous sota (4/ultra/opus). GPT4-1.8T MoE might continue to be the largest model people train for a while.

[@amirsync]

I'm a VoIP (Voice Over Internet Protocol) developer at a small company, I want to start a startup in the comming 5 months and now I'm looking for co-founders, my previous startup (in generative AI space) got accepted into Microsoft for founders hub, if you are a technical, innovative or market person reply me here to discuss more. 😊

[@fil4dworldcomo623]

Winning AI company capturing all the value is speculative, therefore an answer can also be speculative that the more any AI becomes powerful, the more complex society becomes, hence the value of being human to cope with new contexts to feed the "hungry AI" proportionately increases and hopefully it's fun.

[@oiuhwoechwe]

but can it code crysis?

[@NoCodeRussia]

The 10 Trillion Parameter AI Model With 300 IQ @ Starting speak about this: 31:50 good work, guyz!

[@Archer-hg9rw]

Is o1 a significant upgrade over o1 preview

[@runvnc208]

It's really frustrating to me that no one mentions new memory-centric computing paradigms that already have (low-scale) prototypes in labs.

[@yesyes-om1po]

so in other words, cache? VRAM and Sys memory does that already no?

[@MicahBratt]

Even if AI doesn't improve from here companies like Microsoft, palantir etc. are building out the AI eco systems that you will start seeing big companies using for all their backend systems. The average Joe will see the changes pretty fast I would think.

[@artefactxinnovation]

Thanks for the inspiring sharing! Let's leverage the power of AI to improve human lives and building a more sustainable future!

[@ayakitokugawa]

Fact Check: Terence Tao's parents are first-generation immigrants from Hong Kong to Australia. Which makes him Australian-American after he naturalised for duo citizenship.

[@ycombinator]

Oops. Sorry about this. -Garry

[@colbr6733]

Perhaps as the world becomes more corporatized and people get fed up with Tech overreach there will be great opportunities for entirely new OS that don't support AI?

[@miraculixxs]

There is literally nothing in this podcast that make it worth spending the time to listen to it

[@Zeegoner]

Was looking for this comment. It's like a kindergarten class.

[@lucface]

Need a 3x button on yt

[@beofonemind]

I'm hoping we hit a local maximum and hang out there a while.

[@tyc00n]

I love the runtime paradigm of o1 but I sure do struggle to find a use case where its better than a solid prompt template and 4o, quite the opposite infact o1 gives me worse results most of the time

[@mattie3875]

I do like it better for coding tougher problems and debugging problems.

[@Escape_The_Mundane]

How do you solve quantum coherence multibody physics problems. I think they said a practically useful quantum computer has 200,000 to 1,000,000 qubits. They will probably need AI's help to solve such hard problems.

[@dannyisrael]

What are we expecting to gain from more parameters? I’m not feeling a lack for parameters.

[@Brain4Brain]

Scale. The bigger the model, the smarter and the more well it generalize

[@ZevUhuru]

"Feel" - There is nothing to "feel", mathematically they require more parameters to enhance the model output.

[@yesyes-om1po]

more parameters means more better-er, people often confuse parameters with neurons, but its actually more like synapses, and humans have 100 trillion synapses, GPT-3.5 is only 305b parameters or so, and GPT-4 is just an MoE model which runs 11 GPT-3.5s in parallel with one master model choosing which model to use based on the prompt, GPT-4o is likely smaller since its cheaper.

[@UNCLESPARK]

This has me thinking when we’ll be able to run these large models on a usb drive size computer ….

[@Gpt6master]

Good thinking but not likely until “AI” helps human invent the sub atomic circuits to replace today’s semiconductor technology.

[@yesyes-om1po]

never, unless you think we can fit a couple tbs of ram on a USB stick, we'd have to figure out an entirely different method of computing that doesn't rely on silicon, or even semi-conductors...

[@miraculixxs]

Omg why do they taaaalk like thIIIIs?

[@Brahvim]

0:35 I heard that as "then Windows will just be whoever builds the best...". _Well..._

[@sanesanyo]

I really get bugged by the name OpenAI now when we know its so closed that they dont even disclose any model related information in their technical reports.

[@tracy419]

I feel bad for you. It seems like such a pointless thing to obsess over. It takes money to do this stuff, and unless you are ready to fund their requirements, you should probably just get over it and focus on something that actually matters. One of their goals was to provide AI to everyone, and considering anyone with an Internet connection and smartphone or computer can use it for free, they are following through. But they said!!! Blah blah blah.... Move on and be thankful they put this technology out into the world so that it's not being hoarded by the elite.

[@ZevUhuru]

​@@tracy419 Seriously. It's very frustrating when these free loaders want to use the state of the art AI but refuse to acknowledge the insane cost required to get it. Another reason I never understood the ridiculous criticisms of Sam Altman.

[@AdamDahdah.]

A great Notification

[@zerge69]

It's quite obvious that in a very few years there will be no software industry. Your AI will be your only application.

[@kangaroomax8198]

The only people who say this build neither software nor AI.

[@ikleveland]

Yes, but how you interface with that ai will be important. So its important to build a platform or network or processing company now. People will need platforms & how they interface will be different in the future. Device startups realize this, as Steve Jobs said, the best UI is no UI .

[@zerge69]

@ikleveland Temporarily. Soon the AI will build the required interface on the fly, and will manage all networking tasks. The era of software is coming to an end.

[@zerge69]

@@kangaroomax8198 but I do, my friend. That's why I KNOW.

[@SomeHobo]

It's quite obvious that you are not in the software industry.

[@saang]

Ant Financial's fundraise was larger?

[@GoNoGoDIY]

Open AI 4.0+ cannot work effectively at the same time successfully in multiple windows. It cannot retain a topic if you come back later, it's like talking to a new person. I'm ADHD this feels like my failure when any request cannot be met. It frequently cuts me off even though I pay for plus to come back in two hours that's some bs. I'm creating gold, shot down by you used too much come back in two hours 🤣lame

[@nedyalkokarabadzhakov5405]

LLama 70B makes a lot mistakes. Nope not true.

[@funmif]

Excellent comparison with Fourier transform!

[@miraculixxs]

the CFO telling us how their tech works is hillarious

[@Rudzani]

Hardly, but whatever.

[@PSpace-j4r]

What does it mean for our healthcare system? Perhaps an AI doctor assistant to every human one?

[@Digi4DD]

AI agent can listen to you describing the symptoms, run through every combination of symptom and all diseases know to humanity in seconds and suggest the doctor which direction he should go next or propose most accurate diagnosis.

[@yesyes-om1po]

@@Digi4DD doctors aren't glorified look-up tables though, you don't need AI to do that.

[@boonkiathan]

problem is that while compute poring over data can progressively build more and better the 'routine' apps and systems we know and use today you will be stuck with the 60pt system with no inhouse ability to reach 61points and you have to wait for AI to be able to generate code that can get to 61 and nobody of even average talent would care to, understand or work on that pile of code that was conveniently generated in minutes, even bother to push it to 61 - and that is a latent threat to the system development ecosystem

[@yesyes-om1po]

sounds like a load of baloney, the only reason nobody would understand the code is because AI generally writes hard to read code, even if they put copious amounts of "documentation" with it. Sounds like the solution to this is just to write code with minimal AI intervention, unless you plan to completely replace the programmer.

[@michaelmeram6156]

Wattupp Garry

[@wakenupforbreakfasttoday]

Can you have a ten trillion parameter model? Can you extract that many parameters from the dataset/language? I honestly don't know, but at some point you end up with a larger portion of parameters that have very few (if any?) instances, no? (ie most columns are all NA).

[@John4343sh]

@@WearyTimeTraveler That is total bullshit.

[@Vu8uy]

Laplace Transform :V

[@jasperstoj]

It can't do 90% of knowledge work today. It cant even do basic second year accounting journals right.

[@ycombinator]

Skill issue

[@Brain4Brain]

It can, you just don’t know how

[@drawnyuun]

Yeah this whole thing was odd. We didn’t have o1 then maybe, but we do have it now and it’s not even as good as claude. Certainly not the leap they seem to claim it is.

[@AaronBlox-h2t]

@@Brain4Brain What do you mean?

[@Brain4Brain]

@@AaronBlox-h2t RAG

[@perbhatkumar7033]

Is this mono audio 😭

[@Sketching4Sanity]

LOVE ✊🏿

[@BlueBirdgg]

Btw, a little thing. I think you guys may (or not) want to hide your feet. They sometimes dangling take a little of the attention of the content... And thanks for the content.

[@Enedee007]

Take your eyes off them feet and focus on the content. I don’t even know they were there. I would have never seen them, focus!!!

[@aglitchinthematrix157]

Hahahahahaha!!!

[@BlueBirdgg]

@@Enedee007 I love to know the brand of their shoes, don't you?

[@Enedee007]

Sure, I do! 😝

[@ElfProduct]

@@Enedee007no I completely agree with him, now he mentions it, the feet are incredibly distracting.

[@pauldannelachica2388]

❤❤❤❤❤

[@BT-xm7ti]

Incredible video and discussion. High quality debates. Congrats for the work and study

[@angloland4539]

❤️☺️🍓

[@petersuvara]

Scaling like this doesn’t work.

[@Brain4Brain]

Says random guy on internet…

[@petersuvara]

@ says random bot on the internet. Do your research, everything points to scaling as a logarithmic curve, particularly for CNN and Large language model networks.

[@kucchbecho]

what's the use of your AI, when you accept Arab funding, add policies that hide the any islamic things from GPT, AI. Literally, GPT dont talk directly but give excuses like Mullah of ISLAM to legtitimise the atrocities by founder

[@kemoro3018]

What you are on