NVIDIA H100 (SXM5, 96 GB)

The NVIDIA H100 (SXM5, 96 GB) is a workstation graphics card that launched in Q1 2022. It is built on the Hopper GPU microarchitecture (codename GH100) and is manufactured on a 4 nm process.

Memory

The H100 has 96 GB of HBM3 memory, with a 1,313 MHz memory clock and a 5,120 bit interface. This gives it a memory bandwidth of 1,681 Gb/s, which affects how fast it can transfer data to and from memory. GPU memory stores temporary data that helps the GPU with complex math and graphics operations. More memory is generally better, as not having enough can cause performance bottlenecks.

Cores and Clock Speeds

The H100 includes 16,896 CUDA cores, the processing units for handling parallel computing tasks. The GPU operates at a core clock speed of 1,665 MHz and can dynamically boost its clock speed up to 1,837 MHz. Complementing the processing units are 528 texture mapping units (TMUs) for efficient texture filtering and 24 render output units (ROPs) for pixel processing. Additionally, the GPU features 528 tensor cores optimized for AI-accelerated workloads.

Compatibility & Power Consumption

NVIDIA recommends a power supply of at least 1,100 W to handle the GPU's thermal design power (TDP) of 700 W.