Building a Portable PC for AI! 2x RTX 3090, 20-cores, 256GB RAM

[@WSS_the_OG]

I'm only a minute in and had to comment something; very clever doing this in the kitchen so that you can use the waste-heat to do some cooking after it's built! Haha, I jest of course. Really excited to see how this turns out with the two GPUs installed! Edit: made it to the end. Total success, congrats! Very surprised at thermals on such a compact build (especially a dual 3090 build). Really amazed.

[@OTechnology]

Thanks for watching to the end haha! I was surprised by how cool the 3090s ran too lol considering they pull 400W each overclocked. I will have more videos on this build and I already made some new changes to it to improve the cooling.

[@WSS_the_OG]

@@OTechnology I am really looking forward to that. I just got into building a deep learning rig myself, so your video really hit home. Yours is considerably better than mine already, haha, with twice the RAM, and the dual GPUs. Mine's 128 GB system RAM, and half the VRAM (it's a single 4090). Anyway, again, great job. Really impressed and amazed by your build's compactness and impressive thermals. It really surprised me.

[@mgaudurfirdaus]

will u do a 4070 super, 4070 ti super, 4080 super buying guide? please

[@VioFax]

buying guide for 4070: Don't buy the 4070... Its e-waste.

[@FrostDagger]

Step 1: have lots of money to buy the best gpu with tons of. Vram.

[@fabriglas]

Sir buy the 4070ti super. If your after a 4070!

[@MercuriMedici]

I wish you linked the parts you used

[@darkages9507]

The cameraman is Tony Soprano.

[@CrypticEnigma23]

NVLINK improves performance for AI training significantly

[@jesusleguiza77]

And for text generation?

[@KirillPodcast]

100%

[@potatohead-tv9fp]

Hey, i'm trying to build a similar setup from scratch, if u can provide the models of the parts especially the case used that would be of great help to me, i'm a begineer when it comes to building pcs so yk

[@CaptainScorpio24]

so many months has been passed dude where were yuh .. i actually searched your channel now on 12.55am in india.

[@OttoBissVomBismarck]

The ROG RAMPAGE V EDITION 10 only supports 128GB of RAM and has only PCIe 3.0.

[@boringtaskai]

Hi in the final assembly I didn’t see you use nvlink 4 slot? So what do you think, it is faster Using nvlink or not?

[@mrbabyhugh]

28:47 i been WAITING to see if you were going to mention that you switched out the PSU.

[@andre-le-bone-aparte]

Just found your channel. Excellent Content. Another sub for you sir!

[@Beauty.and.FashionPhotographer]

are 2x 3090 faster than one single 4090? how about 4x 3090s ? what case would you use to run 4x 3090 ?

[@xenos321]

Why did you not connect your 3090s with nvlink again? Also out of curiosity, what are use cases of a "portable" ML PC? Why not just leave your server back home and just ssh into it from wherever you are?!

[@Deviantsoundz]

16:35

[@ChrisBraddock]

Not only does NVLINK improve performance but some problematic projects (at least in terms of using both GPUs) suddenly work with NVLINK

[@xenos321]

@@ChrisBraddock Agreed. Again, I've been a ML Engineer by day and AI researcher by night for over 6 years and I'm yet to come across a use-case for a "portable" ML machine.

[@ChrisBraddock]

@@xenos321 honestly I’d like it. Mostly because I don’t plan on living in the same home. Right my rigs are open air. I’d love to (correctly) build one in a tactical suitcase and see how it went. 😁 Maybe convert both.

[@xenos321]

@@ChrisBraddock Well that's an idea!

[@mikeworld2020]

Can this CPU/Motherboard combo run both 3090’s at PCI’s 4.0 at 16X? Thanks

[@notsoinvolution]

Can you post a new video show us how you use this server? How to install the softwares and what is the performance. And the performance to run llama3.1 .Thanks. Since I am planing to build a similar PC. But the mother board and CPU can only buy second hand.

[@mrbabyhugh]

16:00 next i am noting is, did you work out how you are going to route your cables, especially to the GPUs, in particular the 2nd one.......

[@luqmanhaqim97]

This will be semi portable, the problems with using a normal layout like this is that either the Pcie or the GPU will crack if moved too much. That's the reason why itx build is better for portability since most of the case have the GPU in a separate compartment connected with risers.

[@ray-charc3131]

he needs carefully hand carrying

[@clwoni1414]

Could you please put the build & parts list in the discription?

[@tsizzle]

No liquid cooling? Does the motherboard have a built in bmc?

[@detwog1677]

Hi, Cool Video… Can you please send me the exact Specification or a Link for the DDR4 - B- Die 256GB Ram you chose for your build? 😃 Thank you!!!

[@detwog1677]

Are you using Non ECC or ECC Ram since the motherboard is a Non ECC Board as far as I can read on the webpage

[@julianfiacconi709]

@@detwog1677 Most, if not all brand name X99 motherboards can use both ECC and non-ECC memory when paired with a Xeon e5-1600v3/4 series or e5-2600v3/v4 CPU.

[@k1tajfar714]

Awesome awesome awesome! Thank you for this Valuable video!!! Just Please if you you, have the NVLINK RTX TITAN build as well and compare these two t/s in 70B LLMs or training them. It'll be Awesomest.

[@iemandopaarde2406]

hi, can you please do a 3060 and 3060 TI buying guide? i dont know what verson off the cards to buy. thx

[@dholzric1]

3060 is better than the 3060ti for AI. VRam is important.

[@Cuplex1]

You didn't install any storage, so how do you boot it up? What OS are you using?

[@boukeelsinghorst4848]

He installed an nvme for storage

[@mchl_mchl]

How’s this running? This seems like a perfect sweet spot for llama 3 70b

[@shshankbhealay21]

Great assembly at present time's, please change the direction of the cpu fan. The CPU fan should be in-way and not toward the motherboard out devices.

[@Animeinfinity365]

i have i9-1250HX in my workstation laptop, and i am not currently doing anything huge with my laptop. my daily tasks on laptop are 3-4 chrome tabs like youtube,online courses and netflix. i have my maximum cpu performance on 30%. WILL THIS AFFECT MY LAPTOP IN FUTURE?????

[@tejiriamrasa3258]

no

[@treniotajuodvarnis]

hot air from gpus will be sucked in back to the case onto cpu and psu, psu sucks hot air from inside the case, should be installed upside down. not to mention that two gpus close by the one on top sucks already hot air from the bottom one, at least keep the case open, but people do not do this at home :) well the system will work but will be heavily throttling

[@glendekoker3682]

Can we have are own Ai feed it info to help us to live life?

[@sithounetsith9877]

😂😂

[@sithounetsith9877]

AI will feed humans and hunt humans like deer 🦌 because it can 😂

[@WSS_the_OG]

@@sithounetsith9877 Or, it will feed deer, and hunt humans ....

[@Miuitutorial]

Lama sdh tdk mmpir k sini lg. Seperti sdh full bahas Inggris smua kontennya 😅

[@Beauty.and.FashionPhotographer]

Instead of LLMs , did you ever try Large Image Models, LIMs?

[@androidplus.1]

Great stuff 🎉

[@minedustry]

Are those spots for 60mm fans ?

[@JoeBaloney]

Where's the nvlink?

[@publicsectordirect982]

Would be great to see some fine-tuning on this setup!

[@YP-zh5vq]

how much noise do this setup makes on full power usage?

[@julianfiacconi709]

Very nice build, and you articulate it very well. However, this is the wrong platform for leveraging those RTX 3090’s I would have gone for the Asus WS C422 PRO/SE motherboard with TPM onboard (for Win 11) hosting the Socket 2066 with a Xeon W 2200 series CPU. For AI and Machine Learning it is helpful to have the AVX512 instruction set, not available on the X99 platform. As well, to run Windows 11 with stability you need to purchase TPM module with no guarantee that aftermarket module will work. Prices for Socket 2066 motherboards and a W 2200 series cpu, are only a few hundred more dollars than the X99 platform. AMD’s Threadripper is also an alternative, but the AVX512 instruction set was not available till the TR 7900 series processors, and those are prohibitively expensive to the enthusiast consumer. Even the 5900 Threadrippers are still overpriced, although both those AMD platforms utilize PCie 4.0 slots for your 3090’s. Choices and trade offs, oh we live in the best of times. Cheers.

[@KirillPodcast]

100% 👏🏻🔥🤝🏻

[@boomerlife44]

Q: You used EVGA 1600 PSU for most of the video then changed to Corsair 1200, any specific reason?

[@toonnut1]

29:58 i hope your not powering that through that multi plug 😂

[@antoniob.1508]

Which motherboard did you use? I didn’t get the full name

[@xavengarcia851]

Very Nice! This build is awesome and looks incredible!

[@ahsansalahuddin5067]

will it run crysis?

[@jesusleguiza77]

Apart from the motherboard you used in the video, what other motherboard on eBay do you recommend? Regards

[@KirillPodcast]

What type of CPU do you have? 🙄

[@qtly5333]

wich rtx 4060 buy now pls

[@nikextracool]

Hey what were the memory temperature while training?

[@publicsectordirect982]

Key point

[@Otakugima]

Has having the extra 3090 come in handy yet?

[@Christopher-lb6rf]

Why so much ram? Are you using cpu for some layers?

what case?

[@KirillPodcast]

I want to built system on two 3090/3090Ti in NvLink mode on AM4 socket x570 chipset PCIe 4.0 😁

[@georgiareload6599]

Will it game?

[@At0max]

is there any whay to make them work together on the same project? Like Stable Dif and enable them both for more VRAM?

[@CoffiBot2dotoh]

Yep

[@BlackEditionMMD]

Out of curiosity with that not NVlink does windows see the gpus as having 48gb vram on task manager, or is it similar to just sli where regardless of data-throughput it sees only one gpu worth of vram 24gb? Thanks for the video!

[@KirillPodcast]

NvLink doesn't support by Win 10 😢

[@BlackEditionMMD]

@KirillPodcast thank you

[@SpreadingKnowledgeVlogs]

how come you put a pic in the thumbnail with the two gpus connected via a sli or nvlink connection but in the video you didnt go over that aspect any? that was the whole reason i clicked on the video i was hoping you were testing sli and gaming on a newer card since not many people have tested it and those that have say its sort of useless.

[@OTechnology]

Its completely useless for gaming. There is no games that can run with multi GPU anymore. I am using it for training AI models where it needs to communicate between GPUs. I only got the NVLink after making the video so unfortunately it's not in the video lol.

[@SpreadingKnowledgeVlogs]

@OTechnology that's not true all the old games that supported sli still support it as well as a small handful of more current titles although it's not well optimized nor does it help much since each gpu has to render individually it makes very little sense performance wise but in some cases I does work well. It's just there are only 3 or 4 games where it works well and I've only seen it done with gtx gpus that's why I'd love to see a rtx try it. How does the nvlink help with ai calculations? Sorry but I know very little about ai but it fascinates me. Wouldn't a ai specific gpu be a better option here due to the cores being more centered around ai tasks vs the rtx which has a mixture of cores for different tasks?

[@thomaschao5882]

How much does it weigh ?

[@KirillPodcast]

It's very heavy 😅

[@mrbabyhugh]

5:00 if i could get a 3090 FE for $750 even used but in that good condition, heck yeah would buy it!

[@garyc5245]

Great video, I am in the middle of my first HEDT build with Gigabytes AERO TRX50 with a 7980x and trying to watch as many videos and hoping to find a video for connecting the two GPU's with a nvlink bridge so it will have 48GB Vram

[@OTechnology]

It won't get connected into a 48GB pool. It just makes memory transfers between cards faster.

[@SpreadingKnowledgeVlogs]

Yea even when running two or three cards in sli configuration for gaming or anything else for that matter each gpu still needs that vram and it doesn't really combine them in a way that your describing like otech explained. I didn't even realize it made memory transfers faster but then again that totally makes sense. I kinda wish they made a way to daisy chain gpus in a way that was effective and efficient so that you would be doubling your gpu performance but unfortunately it doesn't work that way from my understanding. If it did though it would be awesome cause imagine getting two super cheap 1080tis and getting almost 3090 like performance from them but unfortunately I don't think it's possible unless someone developed a new way of using the sli and even then I don't know if it would be possible.

[@garyc5245]

@@SpreadingKnowledgeVlogs Yeah, that is what I was shooting for. Two 3090's to perform like a A6000 and save a bunch of money. I just finished the PC this morning for the most part and going to have a dual boot OS with Windows 11 and Linux (Mint or Ubuntu...haven't decided yet). A friend of mine is into the computer vision program learning and thought I would make a machine geared towards that and deep learning. Bit of a learning curve vs just building gaming PC's for customers

[@SpreadingKnowledgeVlogs]

@garyc5245 I don't think you will see much benefit from the two gpus vs just one I've watched many videos on it and it does work that well. What would be interesting is if someone made a program where one gpu does all the rendering while the other gpu acts as a accelerator or upscaler or some sort to make the final i,age better quality and smoother running. Proble, is you would have to do all the r&d yourself likely and it's likely not easy work. If your using this computer for gaming mostly I would urge u to look into steam os it's basically Linux with steams skin on it and runs ga,es way better then windows is what I hear or I should say runs games more efficiently which results is better everything just about.

[@garyc5245]

@@SpreadingKnowledgeVlogs Thanks, for the food for thought, it will not be a gaming PC...this is just a very expensive experiment and would be great if this was the foundation PC for someone in that line of work; however I am signed up for some classes to get a little smarter on the topic.

[@fragtrap0083]

love your build

[@REMISCStv]

Good video! No comment on cables 😊

[@OTechnology]

Thanks Anthony! Haha no cable management space in there...

[@REMISCStv]

@@OTechnology Yeah, don't blame you :) Hope you are well.

[@U13SMILECRY]

Either the camera person has asthma or smokes cigarettes because I can hear them breathing even when the builder is speaking.

[@HiveGod-k2d]

Damn, you really spent it all on this machine Where'd you go

[@gileneusz]

I consider building 8x4090 build for AI on GENOAD8X-2T/BCM motherboard with pcie extenders inside Thermaltake W200. Please tell me it's a bad idea before I'll do this mistake 🤣

[@Zeroduckies]

Do it

[@LawrenceTimme]

You should run 3dmark with the 3090's in sli when you get bridge 😂😂 i bet they are faster than 4090.

[@Uruz7Laevatein]

Hmm a single 4090 would outperform it in AI/ML workloads unless NVlink scales with VRAM while being vastly more energy efficient and lower latency (with all the compute units on one die)

[@OTechnology]

I need more VRAM than anything. Which is why 2x 3090 vs a single 4090.

[@OTechnology]

Lol yea I will do a 3DMark run for sure. Although the CPU single core will really bottleneck it.

[@CaptainScorpio24]

​@@OTechnology but 3090s aren't AI gpus.

[@VioFax]

​@@CaptainScorpio24 NOPE! It's for games and has been kneecapped accordingly. Infact a pascal p40 can come close to its inference speed. The p40 is 8 years old. but the difference is it was BUILT for early AI training. Buy an older server GPU that's built for AI. like an H100 or an A1000... These 30-4000 series gaming class cards arent the best crystal first of all, they are overworked and impure silicon... Good for prof of concept or some playing around, or just driving into the ground playing games, knowing you will replace it in a year or two (disposable). But they aren't REAL AI gear.

[@lllongreen]

And what exactly if AI ML software and models are you running ? See a lot of this and the workload is not worth running (bad and outdated models) and what the person thay built them "learns" about AI is almost not teaching them anything useful

[@dtrcServerinfo]

i think everyone is wondering why u pulled it out of the oven lol

[@shephusted2714]

pretty decent 2500 llm ws!

[@____________________________.x]

Why would portability be a consideration? Just use a VPN if you need to do anything from the top of a mountain

[@raen2167]

Buying guide for the 4070super line? :D

[@ThorQi]

Semangat bang

[@ImportRace]

Nice, do a blender test render

[@mrbabyhugh]

15:13 so u did note all of that prior to getting parts and STILL went ahead to try to install GPUs before. SMH.

[@ihsanersen5565]

You can donate the parts you remove to an intern who is willing to work with llm's (like me 😂😂🙌🙌💖)

[@mrbabyhugh]

11:32 u keep praising this case but man, IT IS SO WEIRD TO ME. haha

[@harbinjar]

Where cound you possibly get a rtx 3090 for under $800? Oh wait could it be they fell off the back of a truck? 🤣

[@serg1ou]

Will you do a 4070 super buying guide? 🥵

[@simonlngaihte]

Let's have morning breakfast 😂.. curry will be high end CPU build, ingredients are GTX 3090, motherboard, ram etc etc 😅

[@marianb.5960]

Hello! Are you making a tier list of 4070 super?

[@droddel123]

please do a 4070 ti super buying guide :)

[@keksekonrad]

Pls make a 4070 super buying guide

[@mrbabyhugh]

14:34 smh, you dont seem like a person who has done a lot of builds like you've said. why in the world would you think to install both GPUs and you knew how many NVME you are going to use, and should have taken note where the slot(s) are that you are going to use. Your level of excitement has either affected your judgement or exposed your lack of experience or both.

[@lkline000]

NEED 7800XT BUYING AVOID VIDEO

[@mrbabyhugh]

19:08 WHAT ARE YOU DOING AND SHOWING TO PEOPLE AND ACTING LIKE IT'S OK TO BE PULLING AIR FROM INSIDE THE CASE TO COOL THE PSU!!! You should be banned from KZbin for this!!! And it looks like cables may be an issue with those fans, those they are set to exhaust, BUT YOU NEED TO BE PUSHING HOT AIR OUT WITHOUT ISSUES!! 19:15 THEN, you proceed to tell people it is fine because the PSU fan barely runs!! IT DOESN'T MATTER!!! If the PSU needs it's fan to operate for any reason it should be able to do so at optimal level!!

[@Ro7770]

waiting for your 4070 super, 4070 ti super, 4080 super buying guides bossman :)

[@zeneyep6590]

i clicked this because i know this is just a ridiculous build. Most games are unstable with dual GPUs. i sure hope this guy is using this to CREATE video games rather than playing them with this build. EDIT: ok cool its his work station. no longer shaking my head in disappointment.

[@MichaelBTryn]

What an idiot