How Amazon Is Making Custom Chips To Catch Up In Generative A.I. Race

Despite its firm footing as the world’s biggest cloud provider, Amazon Web Services got a slow start to the generative AI race. AWS released its large language model, Titan, months after Microsoft’s reported $13 billion investment in ChatGPT-maker OpenAI and Google’s release of Bard. But AWS is also designing its own custom AI microchips, shown to CNBC in an exclusive tour of its Austin chip lab. Now analysts say AWS may gain a long term advantage in AI by offering an alternative to Nvidia GPUs.
Chapters:
00:00 - Introduction
03:00 - Designing custom silicon
07:06 - Generative A.I. race
09:39 - Full-stack approach
11:31 - Leveraging cloud dominance
Produced and Shot by: Katie Tarasov
Edited by: Evan Lee Miller
Additional Reporting: Jordan Novet
Supervising Producer: Jeniece Pettitt
Animation: Midnight Snacks
Additional Camera: Katie Brigham, Sydney Boyo, Andrew Evers, Joseph Huerta
Additional Footage: Amazon, Airbus, AMD, Apple, Autobus, Getty Images, Google, Intel, Microsoft, Nvidia, TSMC
» Subscribe to CNBC: cnb.cx/SubscribeCNBC
» Subscribe to CNBC TV: cnb.cx/SubscribeCNBCtelevision
About CNBC: From 'Wall Street' to 'Main Street' to award winning original documentaries and Reality TV series, CNBC has you covered. Experience special sneak peeks of your favorite shows, exclusive video and more.
Connect with CNBC News Online
Get the latest news: www.cnbc.com/
Follow CNBC on LinkedIn: cnb.cx/LinkedInCNBC
Follow CNBC News on Facebook: cnb.cx/LikeCNBC
Follow CNBC News on Twitter: cnb.cx/FollowCNBC
Follow CNBC News on Instagram: cnb.cx/InstagramCNBC
#CNBC
How Amazon Is Making Custom Chips To Catch Up In Generative A.I. Race

[@netizencapet]

CNBC: these tech industry deep dives are GOLD. Addictive and we're all rapt with curiosity. Keep it up, expand, and promote the writers and design/production staff.

[@davidanalyst671]

this isn't a deep dive. they are telling you this because AI is going to put every company out of business in like 5 years.

[@Aubue]

It's fascinating to see Amazon making its move into the generative AI space with custom chips. AWS' dominant position in the cloud market could indeed give them an edge, enabling existing customers to explore generative AI easily. The competition is heating up, and it's exciting to see how this will shape the future of AI development.

[@wyntoncolter1067]

I agree Amazon definitely wants to continue to raise the bar from a competitive and also from innovation standpoint. Which is expected to be a branch of an already established billion dollar industry. chip manufacturing.

[@sr4087]

😂😂😂amazon corpo propo

[@timothymcglynn1935]

🎉

[@luisojedaguzman7624]

Microsoft Azure is catching up and fast. It's likely to overtake AWS in the near future. In addition, Microsoft has plans to incorporate ChatGPT into most of its PaaS, IsaaS, and SaaS offerings.

[@webbsurfer]

Amazing. I just realized I've watched a 15 minute ad.

[@ambition112]

0:10: 💻 Amazon is designing its own custom microchips to power data centers and AI models, positioning itself to supply the growing demand for chips in the industry. 3:17: 💻 AWS is making strides in generative AI and custom chips to catch up with Google and Microsoft. 5:36: ⚡ Trainium provides a significant improvement in price-performance for training machine learning models on AWS, but Nvidia's GPUs are still the preferred choice. 8:38: ⏰ Amazon is expanding its AI offerings, including large language models, and focusing on tools rather than building a ChatGPT competitor. 11:15: 🚀 Amazon is leveraging its existing customer base to drive adoption of generative AI solutions. 13:55: 🏁 The development of generative AI applications and the race to advance AI technology are ongoing, with concerns around security, regulation, and national security. Recap by Tammy AI

[@johndawson6057]

Thank you

[@DrRichardMarques]

Amazing! 👏

[@looseygoosey1349]

wow what a report. That was well done.

[@josephheald1887]

Using his analogy, since the racers get exponentially faster with every step they take, it is reasonable to consider who is winning within the first few steps.

[@cg7524]

But different players are going to get bigger and faster at different speeds and some who are moving quickly now may slow down.

[@marshallmcluhan33]

How are their 8-bit Float (FP8) tensor cores? I have a lot of matmul to do.

[@johncgibson4720]

The hostess is lovely. She also covered GPU market, Nvidia, etc.

[@user-if1ly5sn5f]

Should look at the nvidia clips of the chip designer tools. Gonna be a revolutionary future. Lets make some T100s baby!

[@antoniocarlossouza6086]

Micro chip é a evolução. Futuro!

[@lokesh303101]

Better get SoCs included all with Generative AI and NPUs.

[@tajinderjsingh826]

Thank u.

[@pauldannelachica2388]

Thanks info

[@nicolasdujarrier]

I am a bit surprised that Amazon is not using the hardware from start-up Cerebras (CS-2), Wafer Scale Engine which seems to have some impressive proporties. Also, I have a hunch that some yet missing (not mature) building blocks like Non-Volatile Memory (NVM) like MRAM (VG-SOT-MRAM) would dramatically increase AI efficiency : therefore, big tech companies should actually collectively invest in scaling up manufacturing of this technology to remove some bottlenecks to enable new opportunities !!!

[@Larry21924]

This is an absolute delight. I had the privilege of reading something similar, and it was an absolute delight. "Mastering AWS: A Software Engineers Guide" by Nathan Vale

[@typebeats8448]

Amazon bASICs?

[@Sumukharao12]

1000 IQ comment

[@inferno3080]

😂😂😂

[@henrikbergman4055]

'How Amazon fails also on second attempt to deliver my new freezer, even after talking to customer service after they messed up the first time.'

[@edcarlosmarciano8182]

Uma tecnologia fascinante

[@Party50361]

Cool

[@gargamelandrudmila8078]

With this next gen' AI, everyone will have to become an entrepreneur. Work is over. They will need to own or access all the factors of production: -Land -labour - capital machinery - entrepreneurship Given that machines can also be labour being a worker in the strictest sense will become more and more unlikely because more and more of the workforce will be machines (of software agents). As long as society enables workers to become owners and or managers of the factors of production then I see all being well . If not, then society will become even more polarised between the haves and the have nots. In this case I see workers not voting for Christmas and Governments having to ban these next gen AI technologies all over the world.

[@nexus2384]

Nope, Generative AI fails to do way too many physical tasks or write truly original papers itself

[@DhirajPatra]

But nobody able to make general-purpose microprocessors like Intel and AMD does. Even Apple's M1 and M2 chips also do not support many AI libraries and ecosystems. Different AI chips or just GPU would go haywire and divide the market and eventually the whole development system collapse.

[@HS2ND]

At the end of the day these guys want to sell products (whether most need them or not) and shove it down most people’s throats for good old-fashioned profits!All the wide-eyed jargon-dropping,awe-inducing interviewees are doing that,their bonuses depend on it.

[@pareshpaul001]

Best wishes...

[@cupq]

what's the music in the end of the video ?

[@cardboardpig]

Just what we need; tech giants taking over even more of our lives...

[@WorkSmarter__]

it’s fascinating but at same time it feels its bubble which ought to burst

[@thedeepend2949]

Here comes skynet !

[@darkashes9953]

Amazon could use IBM optical circuit technology

[@daspicsman]

If you’ve been watching the Terminator movies over the last several decades, then you know where all this is really headed.

[@zmitch88]

FIRED.

[@sirus312]

Hold up we about to have a crazy chip glut coming???

[@nadtz]

Right now Nvidia is not meeting capacity making something like 400k H100's a year. There is quite a bit of room for other players but where they will really need to catch up is with software, AMD finally figured this out and at this point I'd guess everyone else is working on it as well. So long story short assuming there is no 'AI crash" it will probably be at the very least a few years before the market evens out.

[@McCallHall1]

It’s like we’re really watching the future unfold.

[@DragonKingGaav]

Katie is beyond beautiful!!!

[@rlonzoraddison6921]

tweezers, soldering irons, and a freaking pencil!

[@ApexGuardian]

Its crazy that amazon is trying to bust into the extremely hard area of computer chips. But with 5.2B in revenue, they sure have a huge head start but it's going to be impossible for them to compete with intel any time soon. Since intel sports a nice 80B in revenue and already has the state of the art manufacturing facilities and decades of experience. Im sure if amazon wanted they could do it. But dont expect Intel to give up any market share anytime soon.

[@cp1011986]

There is no way they manufacture silicone chips, I bet they are using intel or tsmc, and they won't be using the latest 3nm process.

[@arthurswanson3285]

Agree, since silicone is for breast implants.

[@cdtape]

are those the same technologies that are going to take our jobs at some point?

[@picasospace1828]

"Our jobs"? I don't understand why you feel entitled to a job? Also, it never was your job if someone/something can do it better and for cheaper.

[@bushidomayfield5784]

@@picasospace1828speak on it

[@DistrustHumanz]

Your job maybe. Not mine.

[@josephlalock8378]

@@DistrustHumanz nor mine

[@brandont5859]

en.m.wikipedia.org/wiki/The_War_on_Normal_People kzbin.info/www/bejne/bYHUXoZrapyLh7c

[@smithhoowe]

"Hey, we are building next generation AI chips" "Oh right on, where can I drop an application?" "In Texas..." "Never-mind"

[@timothymcglynn1935]

🎉

[@EricPham-ui6bt]

Electroplating is easiest and cheapest to custom chips by drawing circuit on the plastic of other pcb then soak in electroplating aqueous layer by layer then any micro controller for sensor or anything control customize chip need but crystal processing light is much better using progressive lense and process light interference then very good if include transient lense on top then CPU easier

[@wokeymcwokeface1974]

LLMs in the current form won’t be the state of art an year from now.

[@whoneedssalt]

This feels like an ad ngl

[@TheWebstaff]

2:05 thats aging like fine cheese although they have sniped the CEO.

[@crawkn]

The only thing surprising about this is how late AWS is moving into custom chip design, if that is in fact the case. Perhaps the only thing new here is concerning AI custom chips, which every big player is surely doing by now. Tesla has been designing their own AI-focused processors for years already. It's just the obvious next step. The advantage of GPU's is their speed and throughput, they aren't specialized for AI. AI has unique needs which aren't served well by standard hardware.

[@Atipat12]

AMAZING 🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏

[@PsyQoBoy]

Imagine making a 3nm chip by hand.

[@tazztone]

yea this sounds so ridiculous. comparing TSMC multi billion dollar machine parks with an amazon office.

[@AN-bs7nl]

Rooting for Amazon

[@flex19112]

Pandoras Box.

[@Emc2Eggs]

More like my garage "clean room" than tsmc's fabulous fabs of $20 billion

[@ditoiuc]

Yeah so in 2 small rooms,a handfull of employees design chips. Fried chips

[@xiaomidubai8423]

Nice Amazon has the Stability Ai its game over...

[@depth386]

We have a housing crisis and we are about to have food shortages, but at least Alexa will talk to us while we starve. I am amazed such a massive capital misallocation can occur after the Fed has hiked so many times. Rates are going to have to go to 20% to fix this nonsense.

[@fabiodeoliveiraribeiro1602]

China invented gunpowder, but fell victim to efficient European cannons. Americans are racing to develop and profit from AI, but the US could end up being sunk by this very technology. More AI will not mean increased happiness, but the deepening of political instability caused by unemployment. With the rise, success and hegemony of AI there will be a disincentive to laborious university studies and a consequent sustainable cognitive decline of the majority of the population. In the long term, the decrease in corporate profits caused by the reduction in the consumption of goods and services caused by the impoverishment of ever-increasing sections of the American population is predictable.

[@garyp3472]

So amazon is making chips in the garage, by hand? What?

[@wokeymcwokeface1974]

Even potato chip making is more complicated than that!

[@patricks8364]

Design, not make. Watch the video again.

[@mehmetozkaya]

Amazon hq at 1999

[@youtubesucks8024]

Tony Stark made one in a cave. From a box of scraps.

[@garyp3472]

@@youtubesucks8024 lol damn I forgot about that

[@oeao2841]

Red Forman was right Sodering is the future

[@Atipat12]

COOL 😎😎😎😎😎😎😎😎😎😎😎

[@JacksonPGomezRose]

OOOUUUUUU AMAZON SCARY

[@ghostman3329]

This makes me want to work for Amazon even more now ..

[@mr.anderson5077]

And get piped for no reason😢

[@sintayehumulisa3294]

❤from ethiopan

[@thelmasoares8693]

Evolucao

[@Incurious_]

I for one welcome our new AI overlords.

[@solonepeon5805]

Nice but Nvidia is too big for Amazon to catch up but good luck to how far they go.

[@superfluous5162]

Chip designing is one part and manufacturing is another .

[@grimgoreironhide9985]

@@superfluous5162That’s what TSMC is for. Amazon just has to design a good product and contract TSMC to manufacture.

[@trainsplanesandotherthings5187]

Just call them what it is SKYNET..

[@muaidk8511]

How much did Amazon pay you for this video?

[@Vitan91]

Why does this feel like a commercial?

[@srinikoganti]

Hard to compete with AMD. Even Nvidia seem to be lagging behind AMD which is having advantage with Chiplets architecture.

[@tikingrk3639]

Annapurna ❤

[@techcafe0]

Amazon needs to be broken up.

[@alexjose7356]

Annapurna - is it indian😮

[@mainstreet3023]

Amazon is incredibly ambitious when it comes to AI. When the fuss has died down with ChatGPT, they will be riding bigger wave. Their AWS employees must be being incentivised to a very high level. Whether all this will pan out to make them the AI leader, who knows.

[@Hunter-go4bv]

When someone claims they are the best be suspicious.

[@safirot3263]

Tesz robię czipy jak oni ale z gliny nie z papieru....

[@safirot3263]

Jak kvrwa make... Jak oni je robią lurownica kvrwa w 5 nm... Ja pierdole... I to ma być legitne...

[@Atipat12]

BUY #KUIPER 🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥

[@safirot3263]

Co to kvrwa za karp...

[@harishs9003]

Amazon ready to conquer chip market

[@PremiumToyCollect]

😂

[@Mocktailmetal]

What chips? Potato chips? People are using very normal clothing, no dust control environment? No wonder Amazon A.I. stuff fails badly, Nvidia built full dust controlled environment before starting chip R&D.

[@diomedesrodriguez52]

Same observation here...Not a clean room environment whether for chip making or design?

[@GamerbyDesign]

Ah yes Skynet is coming to help us with our human problem.

[@rifleman42051]

Those Amazon chips look like AMD chips!!! I smell a lawsuit brewing... ROFL

[@paulm6074]

No mention of Tesla's custom chip AI training chip or inference chip?

[@user-tv2ei8gr1f]

Um

[@mitkram99]

Microsoft OpenAI ChatGPT is way far ahead against their competitors in the AI world. No doubt about that.

[@mimi21746]

AI Software, not AI Hardware.

[@TheOriginalCameron]

In which ways

[@tringuyen7519]

@@mimi21746Agreed. Dependency on Nvidia is a weakness. Nvidia announced GH200 3 months after H100 bc of H100’s memory limitation!

[@luisojedaguzman7624]

Not only that but Microsoft Azure has doubled its market share of cloud computing in the last 4 years.

[@archmad]

Databricks and Snowflakes eating big part of the cake

[@gargamelandrudmila8078]

I think we all need to go back to business 101. What is the business case for my company using AI or in this case generative AI and LLM's. We are currently stuck in the tech and not the business. We always need to start from the business need since LLMs must have been used to try to solve some initial business need. If not then we are all trying to give a ROI to lab rat's AI discovery and make his pet project viable because they sunk in $1bln. This is not my problem or risk. This could be another Blockchain /Smart Contracts debacle. I hope you realise NO ONE is talking about Blockchain 😕😒😂😂😂😂😂

[@exirokia1331]

So.. when are they going to start talking about the big-big savings the average consumer will get

[@oliviao2238]

The market is geared toward the upper middle class; go figure.

[@sr4087]

😂

[@oliviao2238]

@@sr4087 It's one of those things we have to deal with it

[@AA-il9pc]

What does this have to do with the “average consumer”? This is for AWS which isn’t used by any consumers. It’s business to business sales.

[@jawedmanowar657]

😂😂 all these database and Digitized leading more and more power need which leading huge requirements for Coal fired power plants leading to catastrophic leavel of global warming and US is facing such in large scale Devastatingly Wild Forest Fire and drought

[@astr0nox]

People like Swami Sivasubramanian need to stop vocal frying when giving interviews because it is very hard to make out what they are saying 😠

[@sigmahacker1]

Annapurna is a Hindu name. Pronunciation is wrong. I think that Israeli startup has Hindu connection.

[@TheWebstaff]

Why do half the comments in here seem like ai generated comments? 😂

[@brandodooferman9378]

It's cool technology but there is no way anything with advanced AI will end up being good

[@Atipat12]

LOOK #ASEAN MARKET NOW !!!!

[@ivanhvg]

Yeah nice Amazon AD, nothing else. In reality MS is way ahead with Azure

[@MrGlitch888]

What the guy said at 6:07 is true. He knows. We don’t need more custom hardware. All these people at aws look dead 💀

[@yomajo]

Amazon trying hard to stay relevant. Cute ad.

[@temech-eng]

Chat GPS is kinda gimmick to show what AI is able to do. AWS has many other priorities to do.

[@zmitch88]

Why are you selling alpha. Where is my money.

[@abhishektenneti1604]

Why does the narration seem AI generated???

[@NP1066]

Everyone is Indian here