7995WX Threadripper Build - World's Fastest PC?

[@jonathan.sullivan]

Guy just casually adds 2 White 🦄 GPU's in a non white build for S's & G's. Combined with the current world's fastest nvme. NBD.

[@DigitalSpaceport]

It was all intended to go into a white Lian Li Evo O11 XL but it doesnt support the motherboards last set of screws and this mobo is very heavy. I should have mentioned the case issue more explicitly, was pretty unhappy about it. Im not sure a eeb compatible white variant case exists.

[@empedance1933]

@@DigitalSpaceport id recommend the Lian li v3000+, it's kind of tailored towards custom water-cooling like the old Caselabs cases, but it's monstrous and should support that. Usually goes for like 500ish USD but Microcenter has the white edition for sale for $300 currently if you have one near you

[@DigitalSpaceport]

Hmmm interesting idea, but unfortunatly a 600 mile roundtrip to nearest MC. I need to decide if Im putting this in a open frame rack so I can access all the pcie slots or not. The premium of this all is strongly tied to that and no conventional case seems to account for risers being needed.

[@JohnKhenius]

The CPU alone is 10,000. Insane.

[@inflatablemicrowave8187]

@@DigitalSpaceport how about the fractal torrent xl?

[@HaydonRyan]

Love those crucial SSDs.. I think I just found my next drive!

[@MikeKirkReloaded]

That's a very nice setup! I continue to be amazed at how capable 'consumer' combos are these days: like an AM4/AM5 w/128GB RAM and a couple Gen4 NVMe's can be a little monster of a system: with vitrualization+containers they can be a homelab-in-a-box! But I will always lust after a high-memory EPYC... ;)

[@DigitalSpaceport]

Yeah ram BW and pcie lanes is all its got over consumer, and that gap is closing some recently.

[@Thex-W.I.T.C.H.-xMaster]

You are going to send this great workstation to me immediately............... 😊.

[@FSK1138]

this is so good !!! in the 90's when i got a 486DX someone asked me why i would ever need such a powerful machine 😅 i think i was running a dual Pentium Pro on a tyan motherboard a short while after VR and 3d now A.I 🤓

[@DigitalSpaceport]

Im impressed with SWE's ability to always throw us some new workload type that demands insane processing capabilities!

[@ChairmanMeow1]

Literal $10K CPU. I wouldnt be able to even install it I'd be so nervous

[@DigitalSpaceport]

Yes it was hard to cut the seal and open it for me. Using the torque tighten it down really cemented t.

[@martin777xyz]

Just like people can upload performance benchmarks for cpus, I think it would be good if ollama added the option to upload tps to a central server. After all, it knows what what the hardware is already & what the performance it gets. Would be powerful because it casts a far wider net that the great work you are doing here.

[@DigitalSpaceport]

Yeah if it was opt-in only that would be awesome

[@HaydonRyan]

Ran the same howdy + cat story prompts against llama3.1:8b-instruct-fp16 (2x the precision) on a 5950x + 3090 with 32gb ram and got 42.76tokens /s

[@DigitalSpaceport]

Nice! I still like my 5950x also. Great chip.

[@arigornstrider]

That looks like so much fun!

[@DigitalSpaceport]

Its a pretty decent 8k gaming rig also lol

[@ItsUforia]

You have any luck finding 4090D's? I will be going to China in January, so going to look around for them. I've found one or two, but I want eight. If you run across any in the US, let me know.

[@DigitalSpaceport]

check your email

[@droknron]

With regards to your ollama testing, I'm seeing on Linux a significant uplift in performance. I have an RTX A4000 which is slower than an RTX 3070 and getting 68 tokens per second on LLama 3.1 8b vs the 40-ish tokens you're seeing on the 4090 in Windows. And then when I run the llama 3.1 70b it runs on my EPYC CPU with 256GB of RAM without issue, it's of course very slow, but it works without crashing. I know you mentioned in the video to check out your Linux videos so just thought I'd chime in with a comment to what I'm seeing. I'm using the same WebUI front end as you to run llama3.1 models in ollama.

[@DigitalSpaceport]

Yeah i was expecting faster on that. This rig is heading to linux asap.

[@oliviertableau1703]

Have you tested Boinc and Folding@home on your machine ?.

[@martin777xyz]

To get more power in, is it possible to have 2 power supplies, or does that create issues? Ah, later in the video you say can have 2 psu 😊

[@DigitalSpaceport]

Okay so I tried with a 2psu power addon unit. It did work, but also kept a phantom draw of ~125w when the system was powered off. For sure that is not okay so I am checking more on other options that wont lead to dangerous system conditions.

[@adisaksukul8088]

Do you think there will be a big difference between Linux and Windows? I'm running Ollama on Windows. CPU TRP 5995WX with 256GB DDR4 is the same situation when the model is larger than Vram of 4090, and it is super slow. I have not start trying Linux yet.

[@DigitalSpaceport]

Yes there is a difference win to linux in ollama/llamacpp cpu processing, but it doesn't make it usable really. 2t/s isnt much better vs 1t/s imho. Ive ran the 405b off my 7995wx and 256gb ddr5 6400. Just not usable really. Its really vram or bust from my perspective.

[@adisaksukul8088]

@@DigitalSpaceport Thank you for this reply. It is really helpful!

[@Kamology_]

@@DigitalSpaceportwhat quant are you typically running for these LLMs? A system of that spec, even running off CPU/RAM is faster than most people’s GPUs. It should be pretty damn usable, frankly I’ve never even seen anyone run models on a system that fast so I wouldn’t even know what to expect. In this video you mentioned needing 4 24GB GPUs to run Llama3.1 70b which really shouldn’t be the case unless you’re running waaaay bigger quants than you really should. You can comfortably run 70b with 2 24 gig cards at a 3-5 quant. And I mean hell, people happily run 70b even at a 2 quant.

[@bestof467]

@@DigitalSpaceport Would Apple Ultra Studio be able to run 405b Llama with 192GB+ RAM? Would be nice if you could do a video on Apple Ultra vs Threadripper 4090 for AI LLMs and Stable diffusion vids etc.

[@CaioCesar-nc6zh]

Do you mind sharing the build price?

[@MehmetSahTellek]

Everything is great, but the PC is too underpowered. I don't think it's worth shooting videos for such a powerless PC.

[@DigitalSpaceport]

Id kinda agree for most use cases, it is not ideal.

[@Flllyi2]

Do you plug in pci_cpu12v(1) and (2) when using single psu? I'm having issues with this board. Can't get anything to display on monitor when booting

[@DigitalSpaceport]

No I think there are 4 CPU power plugs iirc. I'm in bed and cant check but I am pretty sure that is right.

[@Flllyi2]

​@@DigitalSpaceportI can't figure out why my build isn't displaying anything on monitor. I have the 20 pin in. Then two 8 pins above and below the 20 pin. Then two 8 pins along the top. There are two empty 8 pins on my board that are pci_cpu_12v.

[@DigitalSpaceport]

@Flllyi2 did you try using the bmc as2500 video out?

[@Flllyi2]

@@DigitalSpaceport is that the vga port? I did try the vga port

[@ronwatkins5775]

What workloads prefer high core count and low core speed vs. low core count and high core speed? How to choose between the two?

[@DigitalSpaceport]

Rendering lightmass and several GIS tasks related to GDAL. That's my main use thats highly multithreaded. Lightmass is happy to eat an entire datacenter for rendering on really large projects. Ray Tracing is not clean and gpu lightmass calcs likely are stable in the next 2 years. A 7975WX is a very good balance chip I considered also. I mainly went big to retain earnings as much as possible. Topend chip holds value pretty well until its replaced. Ill sell this before then

[@hatemadness]

After effects. What can use this much power and does it go faster?

[@DigitalSpaceport]

3d and GIS software, which is what it lives its life doing most of the time... with 3-5 tabs open max lol.

[@MaPooNDaSpOOn]

"gtx 4090" thats crazy.

[@DigitalSpaceport]

*RTX 😅

[@Catge]

How do you justify the price of this?

[@DigitalSpaceport]

There is only 1 way that one could, for working purposes that generate cashflow that is greater as a result of the machines capabilities. I will disclose that my professional work is in the rendering field for a certain gis industry and the software I often use is optimized for massive cpu bound workload processing. If you see the 4 R930's behind me, well that was like 2 generations ago SOTA processing. I didnt buy those new, but someone paid retail for them to exist. Retail was ~80K each lol.

[@Catge]

@@DigitalSpaceport Extremely interesting insight. Thanks

[@cameronfaul1440]

Can you test verus coin hashrates?

[@DigitalSpaceport]

Actually sure. That should be funny lol.

[@cameronfaul1440]

All them threads should be doing work 💪

[@cameronfaul1440]

My guess is 240mh

[@cameronfaul1440]

Updates?

[@win7best]

next time when you say 8 pin maybe say what kind of 8 pin. two 4090s is crazy and that threadripper has as many cores as my entire bladesystem💀.

[@DigitalSpaceport]

Oh the 8 pin setup for the 4090s? I have the 4 standard 8 pin to 1 12vph plug type (whatever that new one is called) and also the straight shot 2 eight pin to 1 12vph. Im not sure which is better, if either. Im not a fan of the 12vph plugs though, stinks the best gpus are forcing this. Core density is indeed insane!

[@ALL-THE-THINGSSS]

@

[@jeremytine]

all that money spent and you went with gimped cpu cooling

[@TekJumble]

5-10% more performance ain't worth it for water cooling trust me I KNOW. Now if you want to do it not just for performance but also looks and space saving on them huge GPU's sure but if only for performance i would skip it.

[@DigitalSpaceport]

Yeah space is a major issue, but an open air frame seems like a good solution for that. Then I can use all the slots. Im also really hitting thermal bottlenecks on this during any heavy workloads, I've not seen a single core hit 5ghz still. Disabling smt helped but thinking of a heatkiller block to external 420x420 rad.

[@TekJumble]

@@DigitalSpaceport Yeah in that case water is the only option are you running PBO ?