d-Matrix presents Corsair C8 card 9x faster than Nvidia's H100 GPU in generative AI workloads

While TSMC warns that Nvidia’s H100 AI GPUs will continue to see short supplies in the next 18 months due to insufficient chip-on-wafer-on-substrate (CoWoS) packaging capacity, other companies are seeing an opportunity to step in and fulfill the overwhelming demand for generative AI hardware. One such company is a new startup called d-Matrix, which recently managed to secure US$110 million from investors like Microsoft by demonstrating the unparalleled speeds of its Corsair C8 compute card that is said to have 9x the throughput of an Nvidia H100 GPU in generative AI workloads.

The impressive speeds are achieved with d-Matrix’s digital-in-memory computer (DIMC) technology. Unlike Nvidia’s A100 and H100 GPUs that are designed for a wider variety of high-performance computing tasks, d-Matrix’s Corsair C8 cards are specifically created for generative AI and inference workloads. A single Corsair C8 card integrates 2,048 DIMC cores grouped in 8 chiplets with a total of 130 billion transistors, and also features 256 GB of LPDDR5 RAM to easily fit entire AI models in memory. All the chiplets are interconnected via PCIe 5.0 pathways with 1 TB/s bandwidth leading to a total of 8 TB/s die-to-die bandwidth. d-Matrix includes 2 GB of SRAM with 150 TB/s bandwidth, as well. Depending on the AI workload, the Corsair C8 provides between 2400 and 9600 TFLOPS.

Thanks to the optimized scaling capacity and the frictionless nature of the open source software stack, d-Matrix claims that a cluster of Corsair C8 cards could lead to 20x better power efficiency, 20x lower latency and up to 30x lower costs over similar solutions from Nvidia.

Buy the Lumina 4K webcam powered by AI on Amazon