Gamble pays off: Developer scores $80,000 worth of hardware with 960 GB DDR5 memory and Nvidia GH200 superchip at a tenth of the price
Less than a week ago, we covered a story originating from Reddit where a gamer managed to get their hands on a powerful and well-equipped gaming PC worth around $3,000 new at just $600. Hilariously, the 64 GB of DDR5 memory in the system alone is worth $880 as of this writing, courtesy of the ongoing DRAM supply squeeze driving an unprecedented surge in memory prices.
$80k worth of hardware at $8k: AI dev hits the jackpot
Now, an AI developer, also through Reddit, managed to hunt down a mouth-watering deal - a used AI server with dual Nvidia GH200 superchips being sold for just $8,000. If bought brand-new, the system would cost somewhere in the ballpark of $80,000, most likely even more since the Nvidia H100 is alone worth around $40,000, as noted by the author. Oh, and also, the system packs a whopping 960 GB of LPDDR5X memory. The exact details of the system have been detailed below, courtesy of the buyer, David Noel Ng:
- Dual Nvidia Grace-Hopper GH200 Superchip - each with a 72-core ARM CPU with 480 GB LPDDR5X memory, Hopper H100 GPU with 96 GB of HBM3 memory
- NVlink C2C with 900 GB/s bandwidth
- 3 kW PSU
- Dual PCIe Gen4 M.2 22110/2280 slots
- Four FHFL PCIe Gen 5 x16
- RJ45, Mini DisplayPort
Nvidia GH200 superchip at a bargain? There's a catch
The seller, in a different reply, revealed that the system was being sold at a bargain because of its poor condition, owing to having been converted from liquid cooled to air cooled. Moreover, they also mentioned that it could not be mounted on a typical server rack, which is generally a pre-requisite.
However, David was unfazed by these drawbacks, and drove to the seller himself to have the system checked out. Luckily, he discovered that the seller was of good repute, and happened to be situated roughly 2 hours away from where he lives.
Once the system was brought home, David arranged a vacuum cleaner, and a few liters of cleaning solution before getting to work. Once the server was cleaned, a new water-cooling setup was installed, and a few repairs took place. Perhaps the most challenging was figuring out a faulty temperature sensor which was repeatedly causing the server to crash, but it was all sorted out in the end.
Massive LLMs right at home: 235B parameter models running smoothly
David appears quite happy with his purchase - and his subsequent ability to run massive LLMs (235B) right at home. Of course, the whole process required a ton of work, most of which is beyond the scope for those completely new to enterprise-grade hardware. But considering that the 960 GB of DDR5 memory alone is now worth more than what he paid for the entire system, it was definitely a happy ending.
For those who want to run smaller AI models at home without having to shell out $8,000 dollars and several more hours in setup, the Strix Halo-powered GMKtec Evo-X2 (currently $1,529.99 on Amazon) might be a better choice.
Source(s)
David Noel Ng, Reddit, spotted by Tom's Hardware
