What runs ChatGPT? Inside Microsoft's AI supercomputer | Featuring Mark Russinovich

[@Pazfora]

This was a great Microsoft ad.

[@acdsp]

Definitely they’ve earned this right.

[@wilsonbotlero2363]

Well, duh! What else did you expect?

[@Papers40]

😂

[@vylbird8014]

@Pathipati Sai Well, I wouldn't go that far. I'd say they are /less/ evil than they used to be, but that's really not a difficult achievement.

[@chrisrogers1092]

Great Nvidia ad as well

[@Ionizap]

A fascinating look at the big picture, I've always wondered about that. This vid shows it . I'm an Apple Guy at home but I am so impressed with what MS has done in the last few years. I run MS Edge and Bing Ai on my desktop at home and at work. The brilliance of these people is just beyond belief.

[@romlfox]

ربزؤؤمظميمؤمؤمؤخ

[@SirIsaacMewtonIII]

microsoft is just killing it in this area. seriously impressive engineering. i wonder if they'll ever release an ai agent for pc that will fix os-level issues (eventlog issues, give human-readable advice based on eventlog errors, etc)

[@Nightspyz1]

Microsoft Copilot?

[@Ozymandias1]

No more low level techs.

[@professional7583]

NEVER 😂

[@animaze86]

It's called 'Clippy'

[@absbi0000]

Windows Copilot is already a thing. Things are moving fast.

[@flwi]

Impressive to see how Microsoft has transformed.

[@kitebeachinnbeachinn2888]

The new CEO is a vision-er much better than Steve Developer, develope, developer... MS should come back with windows phone.

[@jamesjonnes]

Much better than Meta, at least.

[@VadimBolshakov]

how?

[@4.0.4]

It's impressive how GPUs have become such a powerhouse. CUDA was a bright idea way back when. Wish AMD competed better here.

[@theskeletonboi]

Don't forget Tensor Cores

[@Speedboycentral]

george hotz is going to change that - AMD will be a powerhouse soon

[@soraaoixxthebluesky]

@@SpeedboycentralI personally admire George but this is not a simple problems. Even he himself gave up on AMD hardware after trying to run the 7900XTX to utilize his own AI framework, Tinygrad. But in AMD has already build a translation layer for ROCm to run CUDA on AMD hardware. How much of a performance penalty? We don’t know.

[@harriehausenman8623]

Finally some actual insight into the system! Thank you so much for clarifying LoRA usage, clustering and your "AIOps" apporach 🙏 Good to see someone so knowledgeable talk about it!

[@AlexanderMorou]

Holy crap. The number of iterations in the technology stack that they've done in the background is massive. To make these tools usable, scalable, distributed, and likely things I don't even comprehend since it's out of my domain, is massive. I can't imagine how much it costs to use this stack, probably an enterprise level offering.

[@glitchedpixelscriticaldamage]

after listening to these guys.... i feel that my work is useless and only a few people over the world get to do really impressive stuff.

[@aeromotive2]

what do you do

[@HelloThere-xs8ss]

Kinda yeah. People benefit in all industries from a small group of capable and driven people

[@andrewmayorga6649]

Comparison is the killer of joy!

[@sbrunner69]

Same. Chin up, your work is good too.

[@therealb888]

​@@monad_tcp This is unfortunately true because everyone wants the easiest path to make the most money.

[@TomWhi]

I love it when Mark is on the show, he's an absolute tech heart throb!

[@TheB1nary]

Just said that to my wife. The look I got! 🤣😂

[@TomWhi]

@@TheB1nary rookie mistake, I quite quickly found out my wife is very judgmental about my tech-celeb crushes. For example I definitely don't talk about John Savill in front of her any more! 😂

[@TheB1nary]

@@TomWhi Duly noted -- I often watch the man-beast John Savill and brag about him! 🤣

[@TomWhi]

@@TheB1nary it's hard not to! The brain on that guy...!

[@ArnaudMEURET]

He is legend ! 😊 #FluffYouSony

[@naipaulojar7761]

A very good session showing what is under the covers with Microsoft's AI computing. Microsoft is going a good job with the confidential compute which is ultra important for business who want to use AI for both the business & their customers.

[@ankushalgudkar1646]

Absolutely fascinating look into the BTS of these incredibly powerful AI tools now. I was always guessing that the easier something looks and feels for the consumer, ALOT of manpower, cost, time, resources would have gone into making that a reality. Massive respect to Microsoft for the transparency and releasing these videos on how they're building their AI capabilities. Incredible time to be alive!

[@ITube4RealFun]

The legendary Mark Russinovich.

[@maithriashokan]

It was fun and informative listening to the CTO talk about Azure infrastructure. His vocabulary is amazing and the jargons he uses is so smooth. It shows his tenure!

[@shabsZA]

Love hearing about the hardware that exists behind the scenes - thanks for sharing

[@udirt]

holy... I had no idea Mark would be quite young still - he was already god-level windows guru when friends and I started our careers in the late 90s - he must've broken through the technological ceiling at 25 or something!

[@staffanlundberg]

I am a fitness freak and my first thought when I saw him was like...ok, this somewhat elderly guy is EFFICIENT due to his physical condition. He actually looks like another fitness freak ....which makes me wonder...I always thought efficient nerds were pale and skinny as they work too much, but this guy must be semi-retired to be in that shape ? If not, then I don´t understand how he does it !

[@brownianmotion6319]

Rumour has it that he keeps a self portrait in the loft.

[@harriehausenman8623]

@@brownianmotion6319 🤣That would be quite "gray" !

[@raffriff42]

@@staffanlundberg He only works part time (sold Winternals to MS long ago) and can afford a good gym with pro trainer(s). Good for him, I say.

[@tombranson9341]

Mark is the Master of the Microsoft Universe!!!! He has got more genius in his pinkie that I got in my entire body.

[@cowboyuniverse7258]

I’ve never seen comp specs as this sexy. It’s freakin insane.

[@KazenoniKakuremi]

I don't think people how insane MSFT execution of their Ai strategy is - I have never ever seen a company execute a strategy so cohesively as a unit - not even small business - let alone a behemoth like MSFT To get every single department of MSFT to work collaboratively to embed Ai into every product is unheard of ---

[@harriehausenman8623]

BTW, there's your "moat" right there: MS is in the perfect position in the middle of the triangle between hardware (nvidia), foundation systems (OpenAI) and the customer (Azure)!

[@p.c.336]

I didn't imagine Mark Russinovich like this, when I used his tiny but very powerful tools several decades ago 😁 Bginfo, filemon etc..

[@dibu28]

Same😂

[@harriehausenman8623]

@@dibu28 procexp64.exe FTW! 🥳

[@aldosansan2335]

Russinovich Is one of my biggest inspiration in IT!

[@kaizen9554]

At this point Microsoft is killing it. On services: Azure, productivity suite, search engine, and also with edge browser being the best browser on the market; I do everything on edge- reading pdf, it reads text now, fast browsing. Microsoft is making a big comeback

[@emaayan]

1:37 did they actually called it MEGATRON?

[@You_Name_It]

Yup they did

[@MSFTMechanics]

Indeed. From the blog, "the largest and the most powerful monolithic transformer language model trained to date, with 530 billion parameters" www.microsoft.com/en-us/research/blog/using-deepspeed-and-megatron-to-train-megatron-turing-nlg-530b-the-worlds-largest-and-most-powerful-generative-language-model/

[@canalgeek42]

LOL :D

[@MasayaShida]

Uh ohh we need Optimus.

[@MSFTMechanics]

​@@MasayaShida...more than meets the eye

[@CarletonTorpin]

15:00 - Confidential GPU seems like a great idea; individual models can retain their IP while simultaneously contributing to a larger brain.

[@ndolson]

chat gpt와 대화할때, 대화 한줄에 얼마의 전력과 냉각수를 사용할까요? 물 50ml정도를 냉각수로 사용하나요? 궁금하네요 How much power and coolant do you use for a line of conversation when you talk to a chat gpt? Do you use about 50ml of water as cooling water? I'm wondering.

[@letsworksimple]

Less than a bitcoin lol

[@fmcmarques]

I remember Mark Russinovich from the old days, from sysinternals tools

[@aliveandwellinisrael2507]

lol yep

[@Shahawir]

The fat tree topology still used in every data center, brilliant and simple ideas live very long

[@huytranvan2754]

Can't believe I spent nearly 16 minutes watching a MS Azure advert... :) So interesting

[@ivan_i_am]

Mark is the Azure, fascinating amount of knowledge came from this guy in short period of time. Go systernals! 😊

[@88heiling]

And plebs believe Microsoft is just using OpenAI’s AI when GPT-4 solely exists because of Microsoft’s supercomputer.

[@DanKaschel]

Does everything built on azure belong to Microsoft?

[@CMDRScotty]

Its intersting to see how supercomputers have changed over time. Supercomputer before the Cray 1 in 1976 were the size of warehouses. Cray built small supercomputers then starting in the 90s they started getting bigger again and now they are the size of warehouses again.

[@vylbird8014]

The Crays were from that brief time when supercomputers were custom-engineered architecture. Every supercomputer in recent decades is made by connecting off-the-shelf hardware together. The engineering challenge is in making all that hardware operate coherently without excessive overhead and able to tolerate the inevitable hardware faults. You take a pile of hardware accelerators and stuff them in a server, a stack of those servers in a rack, a row of those in a room, wire it all together with the fastest ethernet or infiniband you can afford, then hire some crazy-good computer science specialists to make them all work together, and some pretty-good HVAC and power engineers to stop the thing from melting itsself or the local substation.

[@TerragonDE]

Very interesting insight, thank you! Microsoft is back :-)

[@Mr.Denmark]

Keep it up, thanks microsoft.

[@bright5801]

he knows his stuff.

[@nickspang]

If I saw this without knowing that it’s successfully in production, I’d be think to myself “yeah it’s a lot of hype, but does it really work?” My perception of Microsoft has now changed.

[@meepk633]

What is in production?

[@markarca6360]

From the man who brought us Sysinternals!

[@Lou44000]

At 2:00 the question was « it is super ressource intensive to run and expensive » But you did not answer the question. In a context of global warming and in an energy constrained world it would have been interesting to have an answer.

[@coopmanfamily]

Do you build power plants by these servers?

[@AndersKeisHansen]

Epic video, thank you :)

[@ShareReachCommunity]

This was so nice. Cant wait to learn more about Azure.

[@seebradrun]

Excellent presentation

[@tuapuikia]

9:45 I'm glad criu is now used in mainstream

[@KlausRosenberg-et2xv]

Why have I never heard of a Microsoft's Megatron Turing? It is three times bigger than gpt-3 and can do a variety of stuff, even use natural language...

[@mantrax314]

Just came to my youtube this video... it is very impressive, the infrastructure.

[@mpp9796]

Does it requires a reboot?

[@devrim-oguz]

10:08 it is a bad idea to name it as LoRa since the name is used for radio communication modules. I guess the guy invented the name never "googled" it.

[@ruturajpathak1684]

Microsoft Azure is indeed leading the DC infrastructure innovation!

[@danypell2517]

So cool. Keep it up!

[@wildfire6866]

Very impressive!

[@8eck]

Great and skillful people are pushing big companies. Microsoft understood its weakness and now self-healing step by step.

[@cricketer1987]

Very impressive. Makes me regret selling Microsoft stock last year - they are just so ahead of the game when it comes to AI. Fortunately though it is still driving my S&P 500 and NASDAQ 100 ETF gains for this year so still benefitting from the excellent innovations from Microsoft and OpenAI.

[@timburtell8718]

I like to imagine a time traveling 1700s pilgrim watching this video and trying to make sense of anything said here.

[@Maisonier]

So ... tell us ... are you using megatron-turing nlg 530b for your own enterprise decisions?

[@zy2870]

it's great to see how MS and NVIDIA are working together to build the AI infra. Though wondering how good is it compared to Amazon and GCP?

[@MSFTMechanics]

Azure's support for NVIDIA Quantum InfiniBand and NVLink is a differentiator here and it's something Azure has had a head start on vs. other vendors who currently offer it.

[@ItsWesSmithYo]

Watching the free market at work is an amazing thing 🤙🏽😎🖤🐓

[@monad_tcp]

8:59 so this is why GPU access is so expensive in all the other products, you just hog the GPU, damn, I want this.

[@grey8_]

13:42 "... and that's where we excel." Me: "Word."

[@TonyFarley-gi2cv]

So how come your linear and at the options of Wi-Fi won't align together

[@NotSure416]

What is the reliance of ChatGPT on phase detractors and magneto reluctance?

[@ApostolisKourtis]

Yea but can it run Crysis?

[@randomsam83]

Thanks for Sysinternals

[@jeffzhang769]

Great video!

[@WilliamAshleyOnline]

What I am wondering is if they have built in industrial process to use the heat produced for some industrial process as well as thermocoupling to capture energy from the differential? Is the heat built into the model to process like dehydration, or chemical process?

[@christerry1773]

So is chatGPT essentially located in one location? Where he described the server stacks and gpu’s.

[@MSFTMechanics]

There are multiple instances of GPT-4 running concurrently in multiple locations to run ChatGPT and other GPT-based services.

[@VulcanOnWheels]

12:20 That keyboard looks as if the video is mirrored, but the text does not look mirrored. What's going on here?

[@marie-chantalcote3157]

She have a left hand keyboard

[@jeremiahmullikin]

Is there any reason why GPT-4 can't handle calculating simple problems like power factors or the height of the meniscus in a capillary tube without making mistakes?

[@bendito999]

when combined with Wolfram to help it do the math, it can do better. Gpt itself is bad at being exact with numbers, being good with the numbers isn't one of the things it is trained in, that's not the 'game' GPT itself is playing. GPT itself is playing more of a 'if this is written so far, what do you think should be written next' game, that it is pretty good at

[@kevinmcfarlane2752]

In my playing around with the Chat AIs I’ve found that they are both better than you expect and worse than you expect. There are some surprisingly simple things they still can’t do. But there is also an art to formulating your prompts too. So you can sometimes correct their answers with a little prodding.

[@markmcdowell3105]

Is there a slow down button for his speaking? I can't keep up

[@nangld]

Will MS finish the Project Milo now?

[@SereneDiaries]

Impressive!

[@tombyrne6433]

Awesome stuff appricaite all the effor Microsoft puts in to build the future!

[@vibdib]

How do you input 530 billion parameters

[@javerikr]

How can I prove that I am the only one using this application, and it is a real person? But my question is, why is it a real person?

[@portlyoldman]

Sounds wonderfully like a talk about the Turbo Encabulator 🤣

[@eyesoffloraandfauna8728]

3:40 why they use GPUs and without gpu is it possible ? Apple like technology used

[@oldtwinsna8347]

Running exclusively on CPUs would be exponentially slower for the same amount of money spent on hardware. GPUs are purposely built for parallel and vector processing, which is what a lot of AI needs. CPUs, meanwhile, are built for general instruction processing.

[@AtYourService-fh5fu]

Great insight into a great technology!

[@tobycortes]

what about STABILITY AI , weredo they fit into all of this ??

[@senju2024]

That Mark guy seems to know what he is talking about. All the smart people are in IT. All the dumb people are in politics. We need better distribution of smart people in other domains. But probably need AIG to help out on other domains where we lack smart people.

[@DanKaschel]

Yeah. Politics and tech require the same disposition and type of intelligence. Totally interchangeable.

[@brownianmotion6319]

Don't worry. The smart people in IT will replace the politicians with AI for the better soon.

[@ComplexKangaroo]

Vivek anyone?

[@Lewis01Brown]

Why would Microsoft say that Wayve are the leaders in autonomous driving instead of Tesla? Is there a conflict of interest perhaps?

[@AndreasHopfgartner]

Marc said they are a leader and not the leader. Moreover, there are ppl that think they are ahead of Tesla.

[@Lewis01Brown]

@@AndreasHopfgartner Microsoft are investors so they're financially inclined to make such claims

[@ThaddeusFortenberry]

Also, Tesla uses their internal cluster/cloud resources, not Azure.

[@DanKaschel]

They just are. Tesla does very little work with truly autonomous driving.

[@ChnesRep中華民國OfTaiwan]

without the advanced chip manufacturing skills in Taiwan, non of these would be possible.

[@lassmirandadennsiewillja3943]

Its now the ENTRA CTO:-)

[@sergeymatpoc]

that's an amazing story. Now I feel jealous =). Why can't I work on the most sophisticated hardware for the most cutting edge technologies, and just providing services to our Biz customers (whoever they may be) =). (P.S. technically, I work in Microsoft though, but still - our sub-org is way far behind)

[@therealb888]

Are we going to see the latest gen nvidia H100s being used here? How much of a difference would it make if nvidia open sourced their drivers?

[@SlCKB0Y-sb1kg]

Lol. Never going to happen!

[@therealb888]

@@SlCKB0Y-sb1kg I haven't followed the news lately, but why? Last time I checked the out cry was to halt LLM research before these H100s are deployed. Why say, never gonna happen?

[@SlCKB0Y-sb1kg]

@@therealb888 Sorry, I should have given context. Various individuals and organisations related to Linux have been extremely vocal with trying to get Nvidia to open their drivers for literally decades. Nvidia has always been extremely hostile to open source and consider their drivers to be an integral part of their intellectual property.

[@therealb888]

@@SlCKB0Y-sb1kg I know right, even Linus flipped nvidia 😂. I hope there would be leak of all their firmware, drivers, etc A change in senior management?, even better. But now with AI the demand for open sourcing should be from deep pockets. Even microsoft sold windows source code to governments. It's an unfortunate misery that AI research relies on nvidia cuda & it's closed source stack while AMD is going out of it's way to opensource. Time has proven that in software, opensource wins.

[@SlCKB0Y-sb1kg]

@@therealb888 Yea, I love it! kzbin.info/www/bejne/n4q6q4Csoc9liLM . Realistically though, I think the fact that the whole AI industry is so dependent on Nvidia will become really problematic in the near future.

[@supersmart671]

Microsoft Research is being synonymously with OpenAI

[@berbudy]

Damn so hardware is not the bottle neck for AI huh, it only needs 96 GPU 😮

[@letsworksimple]

The network cards seemed to have been the bottle neck, 8 GPUs later and 8k per second video streaming possible

[@keegang.justice1457]

With their checkpoint/save type mechanism it was cut down to less GPU's than that and requiring less memory. Really quite brilliant

[@alwanexus]

That was for fine tuning to a specific domain. 24 for LoRA, low rank adaptive fine tuning.

[@eglintonflats]

aren't we clueless.

[@kevinverbo]

I think the winner is NVIDIA company

[@Popdad]

Loved it

[@Krishnendulaha]

If Microsoft now launches Surface devices with its own in house AI chip with Windows 12 full with AI features its going to change the personal computers... I hope

[@NK-iw6rq]

I love Chat GPT!

[@optimusctts]

Quite the work

[@chrisw8105]

Here from MisoDope

[@SaurabhOKumar]

What are the energy optimization techniques you use to save the power for all these forms information processing?🤠🤔What does your electric meter display read now-a-days? 🧐🤫🤑🤯 Can you portray energy consumption in modules/sections and represent it in a graphical workflow for the whole picture? Not trying to be sarcastic, just asking. Okay🤗👍

[@scosminv]

Chat GPT is the new Crypto :D

[@thuokagiri5550]

Brilliant

[@maximodakila2873]

Can somebody tell me what's the name of the VSC plugin that creates separate blocks for each python method? It looks neat

[@SaurierDNA]

I wonder, are these two guys real or are they simulated by AI? Anybody knows?

[@moneyall]

Nvidia is making bank of this ai boom.

[@marshallmcluhan33]

Heavily subsidized by the Americans. Can't have the Chinese have too many GPUs better fix the market lol

[@moneyall]

@@marshallmcluhan33 just don't expect gpu prices to go down anytime soon.

[@marshallmcluhan33]

@@moneyall The used market for the 3000 series is great because of mining. You can get a 3090 for way cheaper than retail. It's all about the vram.

[@sirloin869]

Microsoft Azure CTO claims distribution of AI training is needed as AI datacenters approach power grid limits

[@clarkflavor]

where can I start learning stuff to understand more than 1% of what these fine gentlemen are talking about - I feel absolutely clueless, lol

[@Larimuss]

What does it take to run it… computers.. who would have thought.

[@onetruekeeper]

It would be great if it could make a customized videogame just for me at no charge.

[@davidlee50]

How many people have token miners?