From Tom’s Hardware The MI300 3D design allows for incredible data throughput between the CPU, GPU and memory dies while also allowing the CPU and GPU to work on the same data in memory simultaneously (zero-copy), which saves power, boosts performance, and simplifies programming.
There are a couple of reasons why MI300 is not selling 1. It is not ready - it won't be ready until next year 2024. 2. The older versions (MI250, MI100) are not selling well because the supporting software is not mature. But this will change because large software houses Microsoft, Pytorch, and Hugging Face are helping AMD to optimize its software. Large tech companies want to have a second source.
SOC, system on chip, 系統一直在被集成一個單一晶片. 所以根本無所謂 系統重要或晶片重要 , 因為今天的系統可能就是明天的晶片.
@johnnytshi5 ай бұрын
Actually, in LLM, higher memory is way more important. If you have to sync intermediate values, it's a lot slower. So if the chip can hold the entire model, that would be the fastest, with data sharding only.
NVLink is a connection between the CPUs and GPUs , so between sockets. The Infinity Fabric is many things more as it's build within the CPU/GPU, provides a link between dies and across sockets. Infinity Fabric isn't a kind of hypertransport but a superset of it. NVLink 是 CPU 和 GPU之间的连接,也是插槽之间的连接。 Infinity Fabric (AMD) 具有更多功能,因为它构建在 CPU/GPU 内,提供芯片之间和跨插槽的链接。 Infinity Fabric 不是一种超传输,而是它的超集