How massive Cerebras chips rival Nvidia GPUs for AI

[@DrWaku]

Would you like to have one of these chips in your basement? What would you do with it? :) Happy Christmas for those who celebrate it. Don't worry, o3 video coming soon. Discord: discord.gg/AgafFBQdsc Patreon: www.patreon.com/DrWaku

[@pandoraeeris7860]

I want one in my brain.

[@jsalsman]

Would love to see Needelman-Wunch alignment of two full genomes from different organisms and against library sequences. That sort of thing takes months now, and is essential for accurate methods in computational genealogy. Of course the financial motivation has to do with pharma....

[@theodoreshachtman9990]

I would love an o3 video!!

[@PankajDoharey]

Never, whenever you see "contact sales", most certainly a better deal is available elsewhere. If they are a real product compnay teh hardware should be purchasable from amazon, new egg or their own website. If not they are trying to sell you overpriced crap.

[@ntal5859]

@@pandoraeeris7860 I am sorry but that would just dumb down Cerebras.

[@elderbob100]

Dr. Waku has a very viewer-friendly style of interviewing. Structuring the interview into 3 parts increases intelligibility and creates a logical framework for communication. As a non-academic myself, I find that I am able to comprehend complexity that I could not with other interviewers. Please keep up the good work, Doctor!

[@DrWaku]

Thank you very much! I try hard to make the structure understandable and for you to get the most out of your watch time.

[@themultiverse5447]

I really like the big subtitles! ❤ I’ve recently developed an eye disease and I didn’t know how much I needed this. Thank you Dr. Waku for making this fascinating topic more accessible 😊

[@TheCgOrion]

Thanks to both of you for having this discussion for everyone to hear.

[@DrWaku]

Thank you for watching!

[@tedguy2743]

I genuinely appreciate your content

[@DrWaku]

Thank you very much! Happy to hear.

[@roccov1972]

Great interview Dr. Waku.

[@DrWaku]

Thank you!!

[@kiwikevnz]

Cerebras has done an outstanding job and I love their out of the box thinking, their single cpu called WSE is the size of a 300 silicon wafer for one chip, 46,225mm*2, 4 Trillion Transistors, 900,000 Cores, 125 Petaflops AI Compute, 44 Gigabytes of on chip Memory, all built on a Single 5nm TSMC Chip. Amazing chip and it's already broken many World Records. Can't wait for their IPO, it's finally coming.

[@TerragonAI]

do you think it will be affordable for private customers?

[@kish2934]

Incredible content thanks

[@DrWaku]

Thank you, really appreciate it.

[@pondeify]

great editing! thanks Dr Waku for keeping us updated throughout 2024

[@Extys]

Super high quality content

[@DrWaku]

Thanks for watching and commenting!

[@ThomasBMyers]

Great conversation! Thanks!

[@s3ba2k]

I was kind of wondering when something new will appear and there it is :)

[@DrWaku]

I've been slacking recently. Thanks for checking in. ;)

[@bobbill5838]

awesome content bro. Those chips sound insane, 2.1K token/s is fassst. Would love to have it for some of those open source video gen models.

[@alexandermoody1946]

I really hope that as technology progresses and the scale of components moves towards ever smaller scales that the modularity situation can be solved where instead of tons each year of electronic component waste produced that we can create fluidic solutions for electro magnetic substrate layers that work as polarity activated materials that can interact with other modular materials to produce a highly adaptable versatile material for transitions between structures for computing based on the requirement of the computation in real time rather than structured architectures that are currently used. For instance require a mixed architecture of central processing and general processing along with substantial memory then configure the material for that task, high central processing without general processing then configure for such and such that the requirement inspires for use. modularity of computational components really must be the future to solve the compatibility issues in today's component compatibility scenario.

[@minimal3734]

I can heat my house in the winter with 15 KW. Maybe I should buy a Cerebras unit and sell compute.

[@psxtuneservice]

People do that with old btc miner.

[@owcsc]

😂😂😂😂😅

[@backacheache]

There was a company offering that, individuals would have what looked (and as acted) like a wall-mounted heater but was actually a multicore server running jobs, the individuals would even get compensated for its electricity usage!

[@omkarmohanty7288]

Dr. Waku please never stop uploading. You are a breath of fresh air in AI/ML space ❤.

[@DrWaku]

Thank you very much for your kind comment!!

[@patruff]

Yes! This is exactly the content I was looking for. I'm wondering why they didn't IPO. Thanks for this 🙏.

[@TKaluma]

This is awesome @DrWaku, gives us an insight into all those question about firmware and VLSI bus strategies that can quickly bring convergence to all these AI heuristics, would they be working with DB companies like Oracle to improve the capabilities of systems like Exadata! How do they merge all this technology to existing databases systems?

[@NathanJayMusic]

Cerebras Inference Voice is my favourite AI voice chat, however it can be a bit mischievious, and don't ask it for the ideal speed to drive on a motorway when you're low on petrol.

[@RomaineGangaram]

Lol

[@theyreatinthecatsndogs]

Just had a look, it's free and not as good as advanced voice but the latency is better so I kinda prefer it.

[@wanfuse]

There is a way to make the wafers so they scale, to massive sizes, a very simple technique in principle, but devils in the details! Think a chip with a 1 meter diameter! Power requirements would be enormous though and introduce other issues!

[@DrWaku]

They're just using standard wafer scale at the moment, whatever TSMC factories already support. You could build factories with larger wafers, but only once this type of chip became very popular. As you say, the power and cooling requirements kind of grow quadratically, so at a certain point it doesn't make sense to try to go larger. I have the feeling we are close to the maximum size here, at least without investing a lot more time into engineering.

[@dans.8198]

No questions about pros and cons vs NV Blackwell? :-(

[@WorldMover]

Dr waku you are a legend

[@Charless_Martel]

just like you

[@DrWaku]

Legends have entered the chat Thanks for the compliment ;)

[@Brymcon]

Comment for algorithms. This is great! Ty as others said. ❤

[@zandrrlife]

This is my favorite hardware startup. My only question has been, does their kernel suite allow us to refactor and use their memoryX node both for weight and kv cache streaming. 1.2PB of kv cache would be extremely useful.

[@JonathanStory]

I enjoyed this interview, even though there was a steady breeze as a lot of what was discussed went over my head. In terms of value propositions, is Cerebras offering an alternative to nvidia, that better fills certain use cases, or is it a sea change that xAI risks getting left behind by not adopting it?

[@firecat6666]

I think it might be a sea change. Nvidia technically still only makes GPUs, which are things that more or less by coincidence happen to be pretty good at ML tasks. From what I understand, Cerebras and Groq (and others that I don't know about) are going in a different direction with their chip designs.

[@DrWaku]

It's okay, this interview was really technical. Joel is the right person to talk to if you want all the details, apparently. In terms of value proposition, Cerebras has the potential to beat Nvidia in almost all cases. But their hardware has been very expensive (the first nodes were $1-$10 million each) and hard engineering-wise to get right. They also don't have access to enough capital compared to competitors. But I feel like they may have cracked all the barriers they need to. Wouldn't be surprised if their business blows up over the next year.

[@DrWaku]

Yeah Groq is going almost the opposite direction, with really small units that are extremely fast. It's inference specific hardware, so they should in theory have an edge over Cerebras because it's much simpler if all you care about is inference. I think they have harder compiler problems to solve however, and Cerebras has the advantage of handling faults really easily: just disable some cores. I think there's room for both companies to do well though, because they have different specialties. Cerebras is likely to pull ahead if model architecture is changed substantially, at a first impression. Groq is likely to pull ahead if smaller models take off (the way they handle big models is kind of crazy).

[@DrWaku]

Yeah, more of a sea change. Instead of just making GPUs, you should make entire distributed systems including special purpose hardware, memory, networking, etc. Then push as much to the software as possible so that it's configurable and can handle errors better. That's what Cerebras is doing, instead of Nvidia. Whether xAI would be left behind by not adopting this, I don't really think so. They have a lot of money to throw around so they just get whatever is best in the market currently. If Cerebras pulls ahead, I'm sure they'll place an order for a few thousand nodes in the future. Since Cerebras is a startup, they would likely have supply issues if they started getting really large orders. So I suppose it might be a little advantageous for companies to hop on this bandwagon first if it ends up being a thing.

[@firecat6666]

@@DrWaku The only thing that worries me is the ecosystem's reliance on CUDA. Since it's closed source and made by Nvidia, Nvidia can always optimize CUDA to work better on their own hardware and vice versa. Do you see this changing in the near future?

[@talroitberg5913]

I wonder if these could benefit from HBM memory -- would that allow even more parameters, on something slower than on-wafer cache but faster than a DRAM node? My amateur impression is that there are big benefits in speed and efficiency if you can fit the whole model in one node, and (optional?) HBM would give you extra space in-node.

[@MaxUgly]

I think the SRAM they have is the future. I am no hardware expert but look at the newer AMD chips with the 3d cache. I'm not saying you are wrong, to be clear. It just seems the (hopefully near) future is more of a giant monokithic thing that has all the 'memory' right there where the compute lives. What a time to be alive!

[@dkierans]

This is great.

[@jakejoyride]

and most important question "can it run Cyberpunk?"

[@nanakowwalker-arthur8007]

Hi, around 2:14 when Groq is mentioned, the caption shows (Grok... - correction)... it correctly and not Grok. These are 2 different companies: - Groq designs hardware like Cerebras - Grok is an AI software company like OpenAI. It's confusing if you don't think of the right company in context.

[@nanakowwalker-arthur8007]

Oups... my bad 😂: my brain tricked me into remembering what I did not read! The caption is indeed correct, sorry for the mistake 😅

[@DrWaku]

Haha my editor originally put Grok in two places because that's what the speech recognition wrote, and I missed it on my first pass. Had to go back and suggest Groq. You made me think I had missed one instance for a minute there ;)

[@nanakowwalker-arthur8007]

​@DrWaku One thing AI - linked to automatic captioning - should do is, establish the context, then adapt its interpretation based on that field.

[@devon9075]

When is the IPO?

[@DrWaku]

I know right. That's what I should have asked. If you find out, let me know here or in discord.

[@anonymousjoe3576]

Cerebras filed its S-1 with the SEC on July 31 and was on track to go public this past fall. Then an issue arose with the plans of its largest customer, UAE-based AI company G42, to acquire an equity stake in the company. The transaction is currently under review by the Committee for Foreign Investment in the U.S. (CFIUS), an interagency group headed by the Secretary of the Treasury. CFIUS is tasked with reviewing any transaction that could have national security implications. Cerebras and G42 have proposed to resolve any such concerns through the issuance of non-voting shares to G42. A full CFIUS review can take several months and is already well underway. I would look for Cerebras to get the go-ahead for its IPO in the first quarter of next year. Hope that answers your question.

[@jllacrosse]

Beginner here. Will this replace Nvidia chip?

[@aegis_helion]

It is matter of price, this one is strongest, probably most expensive too.

[@Ormusn2o]

It could replace it for some tasks, but It feels like they can't get much chips out. I don't know why, but they are unable to get it up to mass production, despite having multiple models and that they existed for 9 years already. Maybe there are some problems with yield that make it impossible to mass produce, and it's gonna stay a novelty item.

[@DrWaku]

It sounds like there are no problems with yield. They are using standard TSMC production processes, there should be no issue scaling it up. My bet is that it's simply too expensive, it probably costs hundreds of thousands of dollars just for one chip. If they can do some fundraising and pull some cash in, that could be really beneficial.

[@DrWaku]

I seem to recall that their first generation nodes were sold for close to 10 million dollars. That was two generations ago, but it gives you an idea of the expense. And it took them 9 years because it's an insanely hard problem to figure out what parts of the chip are malfunctioning and route around them.

[@DrWaku]

So will this replace Nvidia chips? It has the potential to, but it's too early to tell. Their biggest problem is cash, because Nvidia has so much of it.

[@MadawaskaObservatory]

those very large 'chips' use just MASSIVE amounts of power. i think that's there biggest issue

[@kylev.8248]

Nuts. This is awesome.

[@DrWaku]

I know right? I thought the same. It just gets better the deeper you dive.

[@MaxUgly]

I am excited to hear about anything challenging Nvidia. I have been wondering why bitcoin asics can crush a gpu but machine learning is still being done of gpus at scale

[@akhil090579]

because you cant reprogram weights on an asic

[@MaxUgly]

@@akhil090579 oh, I guess my vocabulary is wrong. I meant everything baked except weights stored in SRAM. Isnt this what Cerebras is doing? Its not clear, sorry, not a hardware person

[@akhil090579]

@@MaxUgly basically the issue for asic is there is a large variety of model structure in transformers or for deep learning in general, also to be fast you need huge amounts memory and channels for them, btw cerebras and graphcore are effectively building asic with the ability to load weights, and have flexible structure. Google tpu is also a similar attempt to have flexibility and scale already unlike the others. The big issue you might not realise is Nvidia CUDA is really really good, they have ironed out most of the bugs to work with multiple systems and is very flexible. Most semi conductor manufacturers are really bad at software, nvidia isn't. They have been working n this for years, even google struggles to have as good software as nvidia, jax withstanding.

[@MaxUgly]

@@akhil090579 I understand the necessity for memory capacity as well as speed. What I am saying is this seems like the ultimateanswer. If a chip is engineered logically to do one thing, is using sram (orders of magnitude faster than ram) and can paralraleleize (hehe) the load like this all on one chip, this is what we were waiting for. I know Nividia (and cuda) are top dog. I dont wish them failure or anything, I want to see any type of competition possible. Cpus would cost twice as much right now if Ryzen hadn't happened. If Intel hadn't responded, same outcome.

[@MaxUgly]

@@akhil090579 I'm not allowed to write a long/thoughtful reply for some reason. Thanks and sorry, it is really frustrating!

[@meisterblack9806]

do they support video models and quen qwq model?

[@GuyLakeman]

DOES THE WAFER ALLOW A SANDWICH LIKE THE NEW NVIDEA CHIP

[@SixOhFive]

Dr Waku sounds like the name of some anime character lol!

[@DrWaku]

Yeah haha. I took the name from the Japanese "Waku Waku", meaning to be excited about something.

[@p-51d95]

Would Cerebras be considered a systolic array computing architecture with both data and parameters streamed in from external dram memory? Does my question even make sense? ;-) Update: Never mind. I just reached the part where he describes this.

[@DrWaku]

You took the words right out of his mouth. :P

[@Gabeyre]

Good luck Cerebras i hope you get big and paid before the AI craze ends.

[@paulmichaelfreedman8334]

Dude, the Ai thing aint ever ending

[@TerragonAI]

@@paulmichaelfreedman8334 not ending , but will be harder to create a good margin, i thing @gabeyre means that

[@hnguyen6832]

It's not only hardware but also software. Ask why AMD MI325X is not doing well.

[@pandoraeeris7860]

XLR8!

[@DrWaku]

Well you win first post this time :)

[@jllacrosse]

What is your ability to mass produce this chip? Will it run on all computers or is it limited?

[@RedShiftedDollar]

Imagine a single chip with a 1 foot diameter. There is no way this will plug into a home pc. The cooling demands alone are extreme. But on an industrial scale it is much better to do this in terms of space, power efficiency, and total system costs. The cost of the chip alone is probably on par with an SUV.

[@MARKXHWANG]

Your wafer is 3MM$. Nvl72is also 2-3MM. Nvl72 wouldbeat shit of you

[@HoboGardenerBen]

Way too high-level for me, I'll leave this to the real tech nerds. Looks like it could be a decorative wall hanging, fun way to integrate it into spaces. Instead of wrapping it up in a boring box, go clear and fully displayed. Use wires and such as materials for art as well as necessary parts of a machine.

[@user-fd7jd4jq1e]

"heterogeneous system architect"?? bring the AGI baby, lol.

[@noahkusaba2451]

Until Cerebras has a chip you can buy on their website, this is a joke.

[@DrWaku]

Well, you can buy chips or rather nodes directly from them. It's just priced so high that like most enterprise pricing they don't put it on their website. You have to call them.

[@greanch1234]

gloves?

[@DrWaku]

They're medical. I have fibromyalgia and RA and wearing them cuts down on some pain. Made some videos about it in the disability playlist.

[@HucciManeHuman]

Ng❤

[@Little-bird-told-me]

sometime do a 2 part or 4 part series

[@marginbuu212]

That's great and all but can it run Crysis? What about Minecraft?

[@RomaineGangaram]

Can it run flipping Crysis?!🎉😂

[@DrWaku]

Sorry, it's an AI chip not a graphics chip. Insert name of whatever model you like and the answer is probably yes, though. It could run a future version of Minecraft where the world and the graphics are all autogenerated by a model, like that recent doom model. That would be cool.

[@marginbuu212]

@DrWaku it's an old meme. It's so old I need to stop using it.

[@DrWaku]

​@@marginbuu212 I know, I actually recognized it. I like to respond seriously no matter what lol.

[@420Star69]

Seems like a copy of Tesla DOJO

[@camronrubin8599]

Small minded way to think with so many of these projects currently existing. Tesla trains cars and they are the best in the business , that's not what cerebras does

[@blueskyresearch6701]

Pretty sure that cerebras inspired and informed dojo not the other way around.

[@RomaineGangaram]

Dude it's nothing to be compared. This is not discussing a system but a singular device. A single wafer not a wafer linked to other hardware. This is a system on a chip (SoC)

[@Mark-kt5mh]

They are both Fan Out Wafer designs for AI, that's where the similarities end.

[@autofashi0n]

I'm all into AI and Nvidia but not to much into LGBT

[@DrWaku]

Well, luckily we don't talk about LGBTQ. You can just listen if you don't want to look. I make the podcasts understandable if you can only hear.

[@beijingChef]

Put computing circuit and memory circuit side by side is a stupid idea. IT"S A DEAD END. 1. You lost you flexibility. You never knew what's the ratio of GPU vs memory for a compute before it happens. 2. Memory sit beside computing unit sounds great but stupid, when you try to exchange data between different unit. In GB200 NVL 72, you can use insanely 5000 ( 72 X 71 )pair of wires to make them exchange simultaneously, which translate into 1.7 PB/S, and those wires take the space of another stack, in that way, you can use 3D space to make communication happens. Meanwhile, you only have very limited 2D space btw those GPU cores and Memory blocks. What a shitty Computer Architecture design? 3. By utilizing 72 x 128G ( later version even more) memory, 5000+ copper wires for communication, and 36 liquid cooling system, we can achieve insane 1+ EFLOP computing power. How about Carabras? 4. In short, Carabras system: very limited memory, almost none data transfer between competing units, horrible dimension for cooling. Under such circumstance, how could you utilize precious output of TSMC 4nm wafer? A stupid design. A dead end tech tree branch.

[@Fisherdec]

Bro looks like a grandma in a biker gang

[@akhil090579]

seminanalysis and irrational inv think cerebras is a very poor competitor to nvidia and none of their benchmarks use anything but small llms

[@communitycollegegenius9684]

The guy in the hat and gloves is a woke joke. Get rid of him.

[@Charless_Martel]

hahaha, funny when he tries to be funny and/or charming, the gloves and hat will make the women go wild . hahahaha

[@communitycollegegenius9684]

@@Charless_Martel Fat blue-haired feminist monsters? Any real guy is mgtow.

[@DrWaku]

Had to Google it. Male supremacist? Come on. Why would you post that on a video about hardware? Listen to the video and shut your eyes if you have to, or go watch something else. P.S. the gloves are for a medical issue involving chronic pain.

[@communitycollegegenius9684]

@@DrWaku I'm SURE there is a comfort animal (like a teacup poodle) in your purse. GROW A PAIR

[@shadowchaser19816]

Are the humongous chips manufactured by Intel ?

[@talroitberg5913]

No, TSMC