Cerebras CS-2 processor scales up to 163 million cores to beat the complexity of the human brain

Cerebras Systems is known for the world’s largest processors that integrate hundreds of thousands of cores per single chip. These cores are not the usual X86 ones we see in computer processors these days. Instead, they are AI-optimized cores similar to Nvidia’s Tensor cores, so the chips are designed to process complicated compute models that rival the complexity of the human brain. Now, the Cerebras CS-2 processor that currently holds the record for the largest and fastest single-chip processor in the world was introduced a couple of years ago, but the company has been working to improve its throughput and is now able to offer a CS-2-based system that can process 120 trillion parameter AI models, theoretically beating the human brain that can fire 100 trillion synapses. For reference, current supercomputer GPU clusters top out at 1 trillion parameters.

Apparently, a single CS-2 processor with 850,000 cores can now beat the human brain, but Cerebras also offers a scaling solution that allows up to 192 CS-2 systems to work in tandem and juggle with even more trillion parameters through as much as 162 million AI-optimized cores. The CS-2 chip is the size of a large tablet and is produced on TSMC’s 7 nm nodes with 40 GB of on-chip SRAM. This amount of memory is not enough for the brain-scale models, so Cerebras came up with external MemoryX cabinets that provide more memory and store the model parameters off-chip, freeing up the on-chip memory for the model itself. This approach combined with the new SwarmX Fabric allows Cerebras to scale performance near-linearly across 192 CS-2 systems.

The SwarmX Fabric integrates Ethernet at the PHY level and compresses the data to transfer it faster across existing nodes of up to 32 CS-2 systems with ~1 Tb bandwidth per node. All the SwarmX Fabric switches connect directly to the MemoryX box that can support up to 2.4 PB of memory in a mix of flash and DRAM. With full memory capacity, the box is able to store up to 120 trillion parameters.

These systems most likely cost so much that the list of Cerebras customers only includes military, intelligence and scientific entities like the Argonne National Laboratory.

Buy the NVIDIA Tesla V100 Volta GPU on Amazon