New JEDEC standard increases HBM2 memory limit from 8GB to 24GB
NVIDIA's Tesla V100 and AMD's anticipated Instinct MI60 are high-end high-bandwidth memory- (or HBM2) based cards for applications such as deep learning and ultra-powerful computing. Currently, their capacities are limited to 8GB per stack, or to 32GB per full card, with between 900GB and 1TB of bandwidth. This is perfectly sufficient for the demands of right now, but may fall behind relatively quickly in the future. Fortunately, JEDEC has come out with some new standards that will expand these limits considerably.
This new standard for HBM2, known as JESD235B, allows for HBM at densities of up to 24 GB per stack, with a bandwidth of up to 307 GB per second (Gbps) for each stack. These new max specs are to be distributed across a 1024 bit-wide device interface. Unlike its predecessor, which only permitted 4- or 8-high stacks, JESD235B is also rated for 2-high and 12-high stacks. It also supports the novel 16Gb layer (which requires a new hardware footprint) and a 2.4Gbps per pin bandwidth.
This effectively extends the theoretical capacity of the V100, MI60 or its successors to a maximum of 96GB, at a possible bandwidth of as much as 1.2TBps. This should make for some particularly extreme machine-learning (if also extremely expensive) processing in the future.