20 Gigabytes per SECOND of Performance?!

Jordan got out to the Innovation Lab with Tyler from KIOXIA and took a look at the performance of the KIOXIA 15TB CD8 drive in the context of a Supermicro 8-way A100 DGX type AI server.
Testing Notes and Info:
-GDSIO from CUDA 11.2
-500M file size, 1M and 4M chunks, sequential and random read/writes.
-7500M file size, 1M chunks, sequential and random read/writes.
Performance ranged from about 14GB/sec to 22GB/s depending on the workload. Important to consider here that there was no software RAID optimization done, and the peaks were selected for discussion in the video, since in a production environment you would tune workloads for the drives.
KIOXIA Spec Sheet: americas.kioxia.com/en-us/bus...
StorageReview Article: www.storagereview.com/news/ki...
#storage #ssd #ai #supermicro #servers #techreview #datacenter #storagereview
AI Performance of the CD8 TLC SSD In the Kioxia Innovation Lab

[@relaxingnature2617]

all those numbers mean very little without including the price

[@toadbroz30]

Besides "AI" what other workloads are being run with these systems? I understand running the tests to show how these are performing (which is absolutely bonkers), but I personally would be interested to hear the benefit of running, say, (me making up an idea) some real-time analysis of a synthetic antibody fighting a virus.

[@StorageReview]

NVIDIA A100 DGX systems are incredibly versatile, powering a wide range of tasks beyond AI. They excel in high-performance computing (HPC) for scientific research, accelerate big data analytics for real-time insights, and are pivotal in genomics for faster disease research and personalized medicine development. Additionally, they boost rendering and simulations for digital content, support financial modeling for quicker market analysis, enhance edge computing for real-time processing in IoT and autonomous vehicles, and even simulate quantum computing.

[@abartlett7350]

But Can It Run Crysis?

[@paulwais9219]

so the GPUs lock up every hour, hence fast checkpointing is worth $$, and saving a 640GB model will take only 32 sec? or was that 22GB/s benchmark just some tiny throwaway file instead of something real? cmon blackwell and the new anti-crash engine isn’t here yet seriously tho how do these hold up when the dram is saturated? they’re not as linear as optane are they?

[@StorageReview]

Tiny? depends on your definition, we ran 500MB and 7.5GB files on the test. We have done models that need all 640GB for training, but the checkpoints are only 20Mb. As we addressed toward the end, realistically in a production environment you would tune the whole system. It's not about avoiding a "lock up", but rather getting a model that is just right. For instance over-fitting could be an issue, but if you go back a few epochs in the checkpoints, you might find a well balanced model. -J

[@mejestic124]

First