The implementation of NVLink-C2C’s 900GB/s bandwidth should also be based on the time required for the NVIDIA Grace Hopper Superchip to read 96GB of HBM3 memory and write it to GH200’s 141GB of HBM3e memory, shouldn’t it?

The 900GB/s bandwidth between the Grace CPU and NVIDIA Hopper GPU is made possible by NVIDIA’s NVLink-C2C technology. This technology connects the CPU, GPU, and memory in a memory-coherent, high-bandwidth, low-latency manner. It provides a bandwidth that is 7 times faster than PCIe Gen5. NVLink-C2C combines the Grace CPU and Hopper GPU into a single superchip, providing a CPU+GPU coherent memory model for accelerated AI and HPC applications. This optimizes data transfer between the CPU and GPU, enhancing processing for large-scale AI and HPC applications. In addition, the Grace CPU Superchip supports up to 128 PCIe Gen 5 lanes, providing I/O connectivity. Each PCIe Gen 5 x16 link supports a maximum bandwidth of 128GB/s in both directions. This can be split into 2x8 for additional connectivity and supports various PCIe slot formats. Therefore, the combination of NVLink-C2C technology and PCIe Gen 5 lanes enables high bandwidth between the Grace CPU and NVIDIA Hopper GPU. The 900GB/s bandwidth between the Grace CPU and NVIDIA Hopper GPU represents the data transfer speed, which indicates the time required to exchange data between the two chips. This plays a crucial role in large-scale AI and high-performance computing (HPC) applications. This bandwidth represents the time required for the Grace CPU and NVIDIA Hopper GPU to exchange data with each other, which is separate from the internal memory capacity of the two chips. In fact, the NVIDIA Grace Hopper Superchip uses 96GB of HBM3 memory, and the GH200 uses 141GB of HBM3e memory technology, providing more than 3 times the bandwidth of the A100. Therefore, the 900GB/s bandwidth of NVLink-C2C represents the data exchange speed between the Grace CPU and NVIDIA Hopper GPU, which is separate from the internal memory capacity of the two chips. The 900GB/s bandwidth of NVLink-C2C and the data processing speed of HBM3E represent different aspects. The 900GB/s bandwidth of NVLink-C2C represents the data exchange speed between the Grace CPU and NVIDIA Hopper GPU. This is separate from the internal memory capacity of the two chips. NVLink-C2C combines the Grace CPU and Hopper GPU into a single superchip, providing a CPU+GPU coherent memory model for accelerated AI and HPC applications. This optimizes data transfer between the CPU and GPU, enhancing processing for large-scale AI and HPC applications. On the other hand, the data processing speed of HBM3E represents the speed at which the memory chip itself reads and writes data. This plays an important role in data processing within the GPU. Therefore, while the 900GB/s bandwidth of NVLink-C2C technology and the data processing speed of HBM3E represent different aspects, these two performance indicators are related, and both play a crucial role in the overall performance of the system. And yes, a PCIe Gen 5 x16 link does support a maximum bandwidth of 128GB/s in both directions. This is according to the standard specifications of PCIe Gen 5. However, this is separate from the 900GB/s bandwidth of NVLink-C2C. NVLink-C2C is NVIDIA’s proprietary technology, designed for data exchange between the Grace CPU and NVIDIA Hopper GPU. This is a separate performance indicator from the bandwidth of PCIe Gen 5. The 900GB/s bandwidth of NVLink-C2C allows for fast data transfer, but the actual time required to read and write data heavily depends on the performance of the memory. Therefore, the time required for the NVIDIA Grace Hopper Superchip to read data from 96GB of HBM3 memory and for the GH200 to write data to 141GB of HBM3e memory will vary depending on the memory performance of each. Also, the same memory performance is required even if the direction of reading and writing data is reversed. That is, the same memory performance is required even if data is read from HBM3e memory and written to HBM3 memory. Therefore, the data processing bandwidth of HBM3e memory is an important factor. This is essential for quickly processing data and performing high-performance computing tasks. This memory performance plays an important role in high-performance computing, artificial intelligence, machine learning, and other tasks.

 

Deep Network, a one-person startup specializing in consulting for super-large language models  

E-mail : sayhi7@daum.net    

Representative of a one-person startup /  SeokWeon Jang