NVIDIA Ampere GA102 GPU Architecture: Second-Generation RTX

Key insights

6 takeaways

01
The GA10x SM architecture doubles the peak processing rate for FP32 operations compared to the Turing generation by enabling FP32 processing on both primary datapaths.See it on page 7
02
Second-generation RT cores in GA10x GPUs provide a 2x performance improvement in ray/triangle intersection testing compared to the previous Turing architecture.See it on page 7
03
Third-generation Tensor cores introduce a sparsity feature that allows for a 2x increase in throughput for matrix multiplication operations.See it on page 24
04
GDDR6X memory utilizes PAM4 signaling to transfer two bits of data per clock edge, enabling memory bandwidth exceeding 900 GB/sec.See it on page 30
05
RTX IO technology offloads decompression tasks from the CPU to the GPU, potentially improving I/O performance by up to 100x compared to traditional storage APIs.See it on page 33
06
The GA10x architecture includes the first GPUs to provide AV1 hardware decode support, which can play up to 8K60 HDR content with very low CPU usage.See it on page 40

Summary

The NVIDIA Ampere GA102 and GA104 GPU architecture whitepaper details the technical specifications and performance capabilities of the GA10x class of graphics processors. The architecture is fabricated on Samsung’s 8nm 8N process and incorporates programmable CUDA cores, second-generation RT cores, and third-generation Tensor cores. The document outlines improvements in FP32 throughput, memory bandwidth via GDDR6X, and the introduction of RTX IO for GPU-based asset decompression.

The architecture supports concurrent execution of ray tracing and shading or compute workloads. A full GA102 GPU incorporates 10,752 CUDA Cores, 84 second-generation RT Cores, and 336 third-generation Tensor Cores; GA102 is used in the GeForce RTX 3090, GeForce RTX 3080, NVIDIA RTX A6000, and NVIDIA A40. The document notes that Ampere architecture GPUs are up to 1.7x faster in traditional raster graphics and up to 2x faster in ray tracing compared to the Turing architecture.

Additional features include support for PCIe Gen 4, HDMI 2.1, and AV1 hardware decoding. The architecture introduces fine-grained structured sparsity, which can double the throughput of Tensor core operations. The document also describes the RTX A6000 and A40 GPUs, which utilize 48GB of GDDR6 memory, scalable to 96GB via third-generation NVLink. The document also describes Error Detection and Replay technology for the GDDR6X memory interface in Appendix B.