Why Every Server Is Not a 4-Socket Server

[@PatrickKennedy2]

This is the last one using the older production setup. After a month of sitting in the queue, it finally went live!

[@jli8225]

subbed to the warowl lol, do you play CSGO?

[@hipantcii]

One think you forgot to mention as disadvantage for the 4S servers is - node fail over protection. With 4S server you need to overprovision more capacity for case of node failure.

[@ServeTheHomeVideo]

Good point. We got into that a bit more on the main site article. It is a bigger concern in smaller installations.

[@aninditabasak7694]

@@ServeTheHomeVideo 4 socket servers are overkill, overpriced and unnecessary. Dual socket servers with multiple GPUs are a far better option as GPU computing has become mainstream these days.

[@acquacow]

For I/O Intensive workloads (eg, databases) 2-socket systems will run circles around 4-socket boxes. Gotta love NUMA overhead =P

[@steelwolf411]

Up until using 4s servers give you more memory capacity. Socket interconnects are faster than infiniband or whatever the newest interconnect. There is a reason 128 socket x86 servers exist.

[@circletech7745]

Where are you seeing 128 socket servers?

[@someguy4915]

@@steelwolf411 128 socket servers for x86 do not exist, IBM has made a few quad-socket servers which can be linked together by external cabling to hook 4 of those for a 16 socket system but they didn't really catch on, nor were they particularly scalable anyway. You're confusing sockets with cores. Sockets are the physical places where an entire CPU can be placed, that CPU can have multiple cores. So while there are 20 core Xeons, there are not 20 socket servers for those Xeons.

[@steelwolf411]

@@circletech7745 SGI UV2000 had 256 sockets under a single memory space.

[@steelwolf411]

@@someguy4915 Look up the SGI UV2000, KZbin it giving me trouble with linking it.

[@virtualinfinity6280]

We will see less 4S servers for simple space reasons. Xeons up to Broadwell has 4chan memory with 2DPC (Dimms per channel). With Skylake, this went up to 6chan with 2DPC and EPYC has 8chan with 2DPC. Putting two EPYCs side by side with memory in between them just barely fits in width in a 19" chassis. Imagine, EPYC would do 4S, you would have two rows of CPUs and memory. How would you route the PCI-e lanes from the first row to the back of the chassis? You would need an insane amount of layers in the PCB (you would most likely need ground layers between the PCI-e layer and other signal layers as well). Then you have the sockets with already 4000+ contacts (EPYC). You'll need much more contacts for the additional interconnects (see EPYC here again, which is most likely the main reason, why EPYC is 2S only). Getting mechanically stable connections between socket and CPU is tricky enough already with EPYC. Imagine, that socket would have 4200+ contacts... Putting memory in cartriges does not work very well either, if you consider that all modern CPUs do at least DDR4-2666, or in case of EPYC DDR4-2933. Same applies for putting CPU and memory on (giant) daugther-boards. Again, with EPYC, that would mean routing PCI-e lanes across two connectors: CPU-card->Mainboard->PCI-e slot. Something, that does not work well with PCI-e 4.0. Which is why systems like the HP DL385-gen10 with PCI-e cages do not support PCI-e 4.0. Only mainboards with PCI-e slots directly on the board support 4.0. And that will get much worse with PCI-e 5.0. In the above, I was mainly talking EPYC, although EPYC clearly is 2S only. But it is the latest in server-CPU development and it is 2S only for exactly the reasons shown above. I bet upcoming Intel Xeon generations will either be 2S only as well, or will have lower memory channel count or PCI-e 3.0 - both of which is unlikely. Let's face it: 4S is dead already, EPYC will never be 4S and next-gen Xeon will follow that. That same applies to the blade-formfactor as well. You cannot fit 2S+memort channels in the current blade formfactor. Which is why you see no EPYC blade systems anywhere. Likewise, 2S is going to die pretty soon too. See current gen of EPYC mainboards. The 2S supermicro board is a joke. And at E-ATX or SSI-CEB (whatever you want to call the formfactor...) it is about as big as it gets. the SP3 sockets have 1DPC, which is not economical at current DDR4 price levels. There are some 1S E-ATX EPYC boards with 2DPC, severely limiting PCI-e add-in card length (Gigabyte has one). Now, imagine how this will go, when we see sockets with 12chan memory... You may welcome new mainboard-formfactors in Tower-systems, but there is absolutely no easy way to overcome the current 19" infrastructure, which is where the servers are deployed. 1S is the mid-term future for CPUs with 12+chan memory. Beyond that, I fail to see viable socket options (you *seriously* do not want a 6000+ contact socket), nor an ecosystem holding >1S setups. We are in the need of an entirely new DRAM architecture, where a DRAM tier is attached via fast serial links instead of 64bit parallel access, backed up by a beefy amount of HBM on the CPU itself.

[@mistakenotou7681]

Yeah with even bigger epic you wont have room to fit for of them

[@caprature]

Might be able to do a stacked board akin to the r820

[@MrAtomUniverse]

Could you talk about pcie lanes , especially on how it limits number of storage drive you can have ?

[@ServeTheHomeVideo]

That is probably a different topic than the 4-socket server discussion, but I agree it may make a great topic for another video.

[@jcl410]

Take a look at the IBM z15 mainframe configuration..... Its smallest config is a four "socket" drawer. A single z15 can have 5 drawers which will give you up to 190 customer usable cores plus 25 more cores for support!

[@johnmadsen37]

5 second answer. CPUs have on chip memory cache. CPUs get data from RAM. RAM is connected to the CPU via a bus (set of lanes like freeway). Two or more CPUs have to transfer from another CPUs RAM and cache. This transfer between CPUs is tremendously slow. This is where NUMA comes into play. NUMA aware operating systems try to keep the data within the same cpu/cache/ram set. This still takes cpu cycles to manage it. And trivially, yes, more power consumption. This is why single chip CPUs with many many cores (64 cores) are more efficient. Some loads multi cpu is slower. But this is rare. So overall, with NUMA aware, it’s fine to get more cpu socket chips. Whatever. This type of performance is not relevant until you get into data processing as a business to generate revenue.

[@ServeTheHomeVideo]

A bit to add here is that 4S servers are actually more efficient when looking at performance per watt than two 2S servers with the same CPU and memory for each socket. 4S is also more efficient than 4x 1S servers with the same amount of CPU and memory per socket. Some folks think 4S are actually less efficient because they use more power than a 2S server. You are totally right that scaling up in each socket is more efficient than scaling the same cores in multiple sockets.

[@prashanthb6521]

@@ServeTheHomeVideo They are efficient w.r.t performance per watt. But they are inefficient w.r.t performance per compute bandwidth. What I mean is resource contention increases in 4S and hence there is a loss of performance.

[@Mtaalas]

That thing with number 4 sounding like the word "death" is also in Japan, because they take a lot from their language from ancient China. Shi = four but it's a homonym with Shi = death (they're written differently though). So number 4 is considered unlucky as well... they don't even use that word for the number 4 in Japan, they usually use word "Yon" to avoid saying "chi"... a bit like there's a lots of buildings in USA that don't have 13th floor... because people are idiots no matter the country...

[@Saturn2888]

What a neat topic I never cared about before now :). I had no clue 4-socket servers had so many benefits and never understood all the drawbacks of multi-hop scenarios. This is why InfinityFabric is so important; it provides a centralized networking layer between cores. I see now how that's different from Intel's ring-approach to processors.

[@ZoruaZorroark]

and here i am, wondering how much data can be processed for things like folding@home along with a couple rtx quadro 8000 cards

[@Spacefish007]

Why buy a 4 socket Intel Server if you can get 3-4 AMD two socket servers with higher performance for the same price?

[@johnmadsen37]

Why?

[@mikkelbreiler3846]

Network cabling, cooling, cost of running 3-4 server vs the one you need, licensing per node vs licensing per core, interconnect bandwidth, complexity, and when was the last time you changed Intel to AMD in your servers - for performance ?

[@siddheshthakur6079]

Omg, Why is this channel not more popular?

[@andljoy]

EPYC does not need to do 4 node :). Also cool how the links on a 4 socket server look exactly the same as a 1st gen epyc.

[@blkspade23]

Already being over twice as dense per socket, while using less power and costing less give Epyc a huge advantage. At that point the only thing you could be after is stuffing more RAM into a single unit, but you could probably afford to run a whole other 2S server with the savings.

[@wildmanjeff42]

very informative, thanks for the video

[@victorbart]

Last year I bought a Sun X4600 M2 for cheap! It is an 4U 8way Dual core opteron with 128gb ddr2 (64 modules) :D It is super intresting how it is build up with all the modules and paths :)

[@p3chv0gel22]

128GB ddr2... That this is possible blows my mind

[@andrewphi4958]

Sun servers are really awesome! Too bad Orakill killed everything that was good in Sun. (

[@09fk0qzitaugk5]

I think the concept of missing one row or a floor is more frightening.

[@kyozon8925]

Hi Patrick. Very interesting video. I have been watching videos and reading about Servers and I believe they are the next logical step to upgrade to from my current workstations for heavy VFX production. Some of the critical Software that I work with has a rather extreme licensing cost to enable networking functionality. For long-term usage it seems like 4S would actually be more cost-effective than 2S+2S in this scenario. But I am having a hard time to find a 4S Motherboard for the 2nd Gen Xeon Scalable. Would you have any recommendations on this front? Thanks!

[@ServeTheHomeVideo]

Generally with these, you want pre-built servers. The motherboards are too big for standard ATX/ EATX form factors. Since they are custom sizes, that is why you are having a hard time finding a motherboard.

[@semosesam]

What about a dual socket Epyc? Not sure if AMD's cores play nice with your workload, but 128 cores is going to come close to what you'd get with a 4S Intel box.

[@ServeTheHomeVideo]

@@semosesam We talk about this in the video and in the main site article. Not really a huge factor today. Perhaps in the future, it may be more of a factor.

[@semosesam]

@@ServeTheHomeVideo By not a big factor, I'm guessing you mean not very common? I'd agree with you there, but I think if Cristiano wants to do a scratch build, it will be much easier to find a standard size motherboard with dual Epyc sockets vs. a quad Intel board. Easy to find EATX dual Epyc Supermicro motherboards, for example. I think the real question comes down to whether or not his workload is optimized for AMD cores. Unless, as you suggest, he goes with a pre-built workstation. In which case, you are again correct. I don't see any pre-built dual Epyc workstations from the major players.

[@xojo]

something to being drunk while listening to Patrick explain how four is unlucky in china while explaining 4-socketed servers

[@kian8382]

As a Chinese, 4 really isn't a big ideal, I don't think we relate the number 4 to death just because they may sound remotely similar. But in some areas with strong dialects that might be true.

[@ServeTheHomeVideo]

Yea, it is not a strong indicator. Just something interesting I have heard several times from vendors and on a trip to China last year.

[@noahlistgarten7832]

And in the States not everyone is superstitious about the number 13. But even if it’s only 1% of people, why exclude them from your market? Companies attempt to appease everyone in their market. Except Apple, who seems to be ok just appealing to their design teams.

[@kian8382]

@@noahlistgarten7832 It may sound right to appease everyone in the market, but why hasn't anyone made a 3-socket server, since it's better than both 2 and "death"? My point is, when it comes to servers or anywhere serious work needs to be done, I highly doubt the number thing matters at all. The ultimate reason for 2-socket in my opinion is how Intel designed their UPI.

[@jasonlisonbee]

Still trying to figure why there's a load that can't be optimized to not care about internode performance when each has so many cores. Optimize to minimize transactions between CCXs and between sockets. Make those transactions preemptive and happen on spare cycles.

[@tacticalcenter8658]

I see alot less 4 socket servers being made these days. Maybe 4 years ago we built alot of them (not a lot in comparison to dual socket).

[@mdd1963]

With 2 socket Epyc mainboards having 128 cores (64 cores x 2), it would be hard to argue 'I need more cores per 2U or 4U!'

[@sinoperture]

With cloud is any of this even relevant anymore in systems architecture? Isn't the monolithic server and app era long gone?

[@gerald4027]

That might help to launch solitaire and help me locate the 10 million saved solitaire games plus maximize raid 1000.

[@uzefulvideos3440]

Why are there no 3-socket servers?

[@kasra7907]

If you have a server prosesing systems with fpfas for example a system whith stratix 10 yo can have a sysrem with 8 fpgas each talking to the other using 6 30 gigabit persecond line annd 6 57 gigabit per second lines and it might beat most other 4 to tow socket sistems in the world

[@morgorth3242]

i would say that a 4 socket server is cheaper than a 1 or 2 socket server cus it got more compute power per racket

[@dr.oliebol]

Even Gartner advices to use single socket servers these days. The NUMA penalty with 2 socket servers is quite ok, but using 4 sockets ones it is getting bad with a lot of calculations

[@green.holden]

4 sockets in one board is so economical if you are buying old sub $60 xeons.

[@blasterman789]

Because with virtualization nobody cares about sockets except for VMwares licensing lawyers. Everything is p vs v core. Quad socket only makes sense in very high density scenarios or when you can only refresh vertically because your CIO will only allow server refreshes every two presidential terms. Dual and even single socket architecture is cheaper, and if partially refreshed every 3-4 years gives you better bang per buck. Per core efficacy still trumps all other considerations (throwing more cores at logical VMs doesn't make them faster) and single or dual core platforms do this cheaper. Last, because you have a big server rack doesn't mean you can only buy specific for factor servers that fit.

[@HORNOMINATOR]

me looking at my recent four socket opteron build: 😒

[@fbifido2]

Lic for Management port ??? - how about not buying those servers and buy servers that has it built in like supermicro. 4x cpu per node - super overprice and expensive (mainly because they are all intel) VMWare & Hyper-V(non-free ver) - per CPU/Core, windows/SQL server - better wait for a single CPU with 128/265-cores from AMD. ps: intel can also make 128-core single cpu, but guest what, why would they, when they can make half-ass-cpu and people will still buy them.

[@fabianstuckmann3611]

If you just want PCI lanes, why do you go Intel? AMD has PCI4.0 and more lanes...

[@vorlock7149]

The title "Why Every Server Is Not a 4-Socket Server" implies that 4-Socket Servers don't exist. Because "...every...is not a..." is the same as "...no...is a...".

[@mdd1963]

NOt sure how you get that it implies that 4 sockets don't exist. I don't jump to that thought train AT ALL from reading it.

[@Daniel_WR_Hart]

Yea the title should have been "Why not all servers are 4-socket servers", but I knew what he meant since I already know that 4-socket servers exist

[@mdd1963]

Same reason every server is not 8 sockets? :)

[@Ang3lUki]

dry cracked lips

[@MarloMitchell]

ikr god dam