Mysterious "GPU-N" in research paper could be GH100 NVIDIA Hopper GPU with 100GB of HBM2 VRAM, 8576 CUDA Cores, and 779 TFLOPs of FP16 compute

Twitter user @Redfire75369 recently discovered a research paper talking about using low-precision GPU compute for deep learning workloads, that references and describes a mysterious "GPU-N." Based on the specifications and performance mentioned in the paper, and aligned with existing leaks, it is highly possible that GPU-N is a single-die variant of NVIDIA's upcoming GH100 "Hopper" GPU for the HPC market.

GPU N features 8576 CUDA cores, and 100GB of HBM2 VRAM delivering up to 6.3 TB/s of memory bandwidth. GPU-N delivers 24.2 TFLOPs of FP32 compute. More relevant for deep learning use cases, however, is the fact that in FP16 workloads, it can go up to 779 TFLOPs. This is nearly 2.5 times the processing power of the Ampere-based A100. 

Previous leaks indicated that Hopper cards would deliver 2-3 times the compute performance of Ampere. While GPU-N offers only a 25 percent boost to FP32 capabilities relative to GA100, the 250 percent increase in FP16 performance is in line with expected gains. 

GPU-N features "just" 8576 CUDA cores. At first glance, this puts into question the idea that we're looking at a GH100 Hopper part. Hopper's MCM (multi-chip module) architecture allows for tens of thousands of shader cores. @Redfire75369, however, concludes that GPU-N might be a special single-die variant. 

If the Twitter user proves correct, this could be our very first glimpse at NVIDIA's next-generation HPC graphics architecture. 

Check out the GeForce RTX 3090 here on Amazon.