Launched on January 20, 2025, DeepSeek-R1 is a 671B parameter Mixture-of-Experts (MoE) model with 37B active parameters per token. Designed for advanced reasoning, it supports 128K token inputs and generates up to 32K tokens. Thanks to its MoE architecture, it delivers top-tier performance while using fewer resources than traditional dense models.
Independent testing suggests that the R1 language model achieves performance comparable to OpenAI’s O1, positioning it as a competitive alternative in high-stakes AI applications. Let`s find out what we need to run it locally.
The hardware
This build centers around dual AMD Epyc CPUs and 768GB of DDR5 RAM—no expensive GPUs needed.
- Case: Enthoo Pro 2 Server
- Motherboard: Gigabyte MZ73-LM0 or MZ73-LM1 (has two CPU sockets & 24 RAM slots)
- CPU: 2x AMD Epyc 9004/9005 (9115 or 9015 work as more budget-friendly options)
- Cooling: Arctic Freezer 4U-SP5
- RAM: 24x 32GB DDR5 RDIMM (768GB total)
- Storage: 1TB+ NVMe SSD (to quickly load 700GB of model weights)
- Power Supply: Corsair HX1000i (1000W, plenty for dual CPUs)
Software & Setup
Once assembled, Linux and llama.cpp need be installed in order to run the model. A crucial BIOS tweak, setting NUMA groups to 0, doubles RAM efficiency for better performance. The full 700GB of DeepSeek-R1 weights can be downloaded from Hugging Face.
Performance
This setup generates 6-8 tokens per second—not bad for a fully local high-end AI model. It skips GPU entirely, but that’s intentional. Running Q8 quantization (for high quality) on GPUs would require 700GB+ of VRAM, costing over $100K. Despite its raw power, the entire system consumes under 400W, making it surprisingly efficient.
For those who want full control over frontier AI, no cloud, no restrictions, this is a game changer. It proves that high-end AI can be run locally, in a fully open-source fashion, while prioritizing data privacy, minimizing vulnerabilities to breaches, and eliminating reliance on external systems.
Source(s)
Matthew Carrigan on X, Docsbot, DeepSeek, teaser image: Pixabay