Notebookcheck Logo

Cerebras CS-2 processor scales up to 163 million cores to beat the complexity of the human brain

The Cerebras CS-2 processor is as large as an 11-inch tablet. (Image Source: Cerebras)
The Cerebras CS-2 processor is as large as an 11-inch tablet. (Image Source: Cerebras)
A single CS-2 chip with 850,000 cores can now process 120 trillion parameter AI models that are more complex than the human brain with its 100 trillion synapses. This is made possible through the introduction of the MemoryX cabinets that come with up to 2.4 PB of flash+DRAM memory, and Cerebras also offers a scaling solution that allows up to 192 CS-2-based systems to interconnect through the SwarmX fabric for a total of 163.2 million core processing power.

Cerebras Systems is known for the world’s largest processors that integrate hundreds of thousands of cores per single chip. These cores are not the usual X86 ones we see in computer processors these days. Instead, they are AI-optimized cores similar to Nvidia’s Tensor cores, so the chips are designed to process complicated compute models that rival the complexity of the human brain. Now, the Cerebras CS-2 processor that currently holds the record for the largest and fastest single-chip processor in the world was introduced a couple of years ago, but the company has been working to improve its throughput and is now able to offer a CS-2-based system that can process 120 trillion parameter AI models, theoretically beating the human brain that can fire 100 trillion synapses. For reference, current supercomputer GPU clusters top out at 1 trillion parameters.

Apparently, a single CS-2 processor with 850,000 cores can now beat the human brain, but Cerebras also offers a scaling solution that allows up to 192 CS-2 systems to work in tandem and juggle with even more trillion parameters through as much as 162 million AI-optimized cores. The CS-2 chip is the size of a large tablet and is produced on TSMC’s 7 nm nodes with 40 GB of on-chip SRAM. This amount of memory is not enough for the brain-scale models, so Cerebras came up with external MemoryX cabinets that provide more memory and store the model parameters off-chip, freeing up the on-chip memory for the model itself. This approach combined with the new SwarmX Fabric allows Cerebras to scale performance near-linearly across 192 CS-2 systems.

The SwarmX Fabric integrates Ethernet at the PHY level and compresses the data to transfer it faster across existing nodes of up to 32 CS-2 systems with ~1 Tb bandwidth per node. All the SwarmX Fabric switches connect directly to the MemoryX box that can support up to 2.4 PB of memory in a mix of flash and DRAM. With full memory capacity, the box is able to store up to 120 trillion parameters.

These systems most likely cost so much that the list of Cerebras customers only includes military, intelligence and scientific entities like the Argonne National Laboratory.


Buy the NVIDIA Tesla V100 Volta GPU on Amazon

Read all 2 comments / answer
static version load dynamic
Loading Comments
Comment on this article
Please share our article, every link counts!
> Notebook / Laptop Reviews and News > News > News Archive > Newsarchive 2021 08 > Cerebras CS-2 processor scales up to 163 million cores to beat the complexity of the human brain
Bogdan Solca, 2021-08-25 (Update: 2021-08-25)