DeepSeek V4 Pro: Self-Hosting Estimate
DeepSeek V4 Pro (DeepSeek) on different GPUs: configuration, estimated performance and use cases.
Data updated: 2026-09-16B300 288GB
288 GB · 8,000 GB/s · Top-tier Blackwell card (no official channels to China)- Suggested precision
- INT8
- GPUs needed
- 8priced as 1 × 8-GPU HGX node
- Est. decode speed
- 1,306 tok/ssingle-stream theoretical ceiling
- Est. batched throughput
- out 2,220 tok/sin 23,878 tok/s · many concurrent requests
- Hardware list price
- $400,084–$440,093Cloud rental (live) ×8: $79.52–$100.01/hr (2026-11-20)
- Hardware market price
- $2,545,253–$2,994,415
- Typical use
- Enterprise-grade ultra-large-scale training and high-concurrency inference clusters
GB200 192GB (single card equivalent)
192 GB · 8,000 GB/s · Top-tier Blackwell card (no official channels to China)- Suggested precision
- INT4 (AWQ/GPTQ)
- GPUs needed
- 8
- Est. decode speed
- 2,612 tok/ssingle-stream theoretical ceiling
- Est. batched throughput
- out 4,441 tok/sin 26,531 tok/s · many concurrent requests
- Hardware list price
- $239,745–$359,617
- Hardware market price
- $718,660–$1,437,319
- Typical use
- Ultra-large-scale trillion-parameter model training and inference clusters (networked with Grace CPU)
B200 180GB
180 GB · 8,000 GB/s · Top-tier Blackwell card (no official channels to China)- Suggested precision
- INT4 (AWQ/GPTQ)
- GPUs needed
- 8priced as 1 × 8-GPU HGX node
- Est. decode speed
- 2,612 tok/ssingle-stream theoretical ceiling
- Est. batched throughput
- out 4,441 tok/sin 23,878 tok/s · many concurrent requests
- Hardware list price
- $360,075–$440,093Cloud rental (live) ×8: $66.22–$70.01/hr (2026-11-20)
- Hardware market price
- $449,162–$524,023
- Typical use
- Enterprise-grade large-scale online inference and training
H200 141GB
141 GB · 4,800 GB/s · Top-tier export-controlled card (channels to China are volatile)- Suggested precision
- INT4 (AWQ/GPTQ)
- GPUs needed
- 8priced as 1 × 8-GPU HGX node
- Est. decode speed
- 1,567 tok/ssingle-stream theoretical ceiling
- Est. batched throughput
- out 2,664 tok/sin 10,496 tok/s · many concurrent requests
- Hardware list price
- $248,053–$320,067Cloud rental (live) ×8: $29.90–$57.06/hr (2026-11-20)
- Hardware market price
- $1,003,129–$1,033,073
- Typical use
- Enterprise-grade production environment high-concurrency inference
H100 SXM5 80GB
80 GB · 3,350 GB/s · Top-tier export-controlled card (no official channels to China)- Suggested precision
- INT4 (AWQ/GPTQ)
- GPUs needed
- 16 (exceeds one node: multi-node needed)priced as 2 × 8-GPU HGX node
- Est. decode speed
- 2,188 tok/ssingle-stream theoretical ceiling
- Est. batched throughput
- out 3,719 tok/sin 20,991 tok/s · many concurrent requests
- Hardware list price
- $592,123–$618,129Cloud rental (live) ×16: $27.54–$69.73/hr (2026-11-20)
- Hardware market price
- $1,003,129
- Typical use
- Enterprise-grade production environment high-concurrency inference/training
H800 SXM 80GB
80 GB · 3,350 GB/s · Top-tier export-controlled card (discontinued)- Suggested precision
- INT4 (AWQ/GPTQ)
- GPUs needed
- 16 (exceeds one node: multi-node needed)priced as 2 × 8-GPU HGX node
- Est. decode speed
- 2,188 tok/ssingle-stream theoretical ceiling
- Est. batched throughput
- out 3,719 tok/sin 20,991 tok/s · many concurrent requests
- Hardware list price
- $586,905–$622,838
- Hardware market price
- $808,492–$838,436
- Typical use
- Enterprise-grade production deployment (original China-compliant model, now discontinued)
H20 96GB
96 GB · 4,000 GB/s · Export-compliant special supply card (significantly reduced compute power)- Suggested precision
- INT4 (AWQ/GPTQ)
- GPUs needed
- 16 (exceeds one node: multi-node needed)
- Est. decode speed
- 2,612 tok/ssingle-stream theoretical ceiling
- Est. batched throughput
- out 4,441 tok/sin 3,141 tok/s · many concurrent requests
- Hardware list price
- n/a
- Hardware market price
- $419,218
- Typical use
- Suboptimal choice in export-compliant environments, medium-to-large scale inference deployment (FP16 compute power is only about 15% of H100, but higher memory bandwidth offers an advantage in decode-intensive scenarios)
A800 PCIe 80GB
80 GB · 1,935 GB/s · Compliant special supply card (export compliance status adjusted multiple times)- Suggested precision
- INT4 (AWQ/GPTQ)
- GPUs needed
- 16 (exceeds one node: multi-node needed)
- Est. decode speed
- 1,264 tok/ssingle-stream theoretical ceiling
- Est. batched throughput
- out 1,289 tok/sin 3,973 tok/s · many concurrent requests · PCIe multi-GPU penalty
- Hardware list price
- $239,553–$249,135
- Hardware market price
- $316,210
- Typical use
- Medium-scale inference/training deployment in export-compliant environments
Ascend 950DT 144GB
144 GB · 4,000 GB/s · Huawei's new-generation SuperNode card (domestic, parameters and prices for reference only)- Suggested precision
- INT4 (AWQ/GPTQ)
- GPUs needed
- 8
- Est. decode speed
- 1,306 tok/ssingle-stream theoretical ceiling
- Est. batched throughput
- out 2,220 tok/sin 5,306 tok/s · many concurrent requests
- Hardware list price
- n/a
- Hardware market price
- $929,467
- Typical use
- Domestic large model inference SuperNode (64-card SuperPoD), report states inference performance is approx. 0.86x that of H200 at the same latency
Ascend 910C 128GB
128 GB · 3,200 GB/s · Huawei dual-chip data center card (domestically produced)- Suggested precision
- INT4 (AWQ/GPTQ)
- GPUs needed
- 8
- Est. decode speed
- 1,045 tok/ssingle-stream theoretical ceiling
- Est. batched throughput
- out 1,776 tok/sin 7,980 tok/s · many concurrent requests
- Hardware list price
- n/a
- Hardware market price
- $95,821–$239,553
- Typical use
- Domestically produced large model training and inference clusters, CloudMatrix and other super-node solutions
Ascend 910B 64GB
64 GB · 400 GB/s · Domestic alternative (not subject to US export controls)- Suggested precision
- INT4 (AWQ/GPTQ)
- GPUs needed
- 16 (exceeds one node: multi-node needed)
- Est. decode speed
- 261 tok/ssingle-stream theoretical ceiling
- Est. batched throughput
- out 444 tok/sin 6,792 tok/s · many concurrent requests
- Hardware list price
- n/a
- Hardware market price
- $263,509–$287,464
- Typical use
- Domestic alternative, not subject to US export controls, medium-to-large scale inference/training deployment, memory bandwidth significantly lower than NVIDIA cards of similar VRAM capacity
RTX PRO 6000 Blackwell 96GB
96 GB · 1,792 GB/s · Professional workstation/server card (China-specific version is 6000D 84GB)- Suggested precision
- INT4 (AWQ/GPTQ)
- GPUs needed
- 16 (exceeds one node: multi-node needed)
- Est. decode speed
- 1,170 tok/ssingle-stream theoretical ceiling
- Est. batched throughput
- out 1,194 tok/sin 6,367 tok/s · many concurrent requests · PCIe multi-GPU penalty
- Hardware list price
- $256,614Cloud rental (live) ×16: $21.15–$32.11/hr (2026-11-20)
- Hardware market price
- $407,240
- Typical use
- Single-card large VRAM inference, local deployment of medium MoE models, private all-in-one machine
RTX PRO 5500 Blackwell 84GB
84 GB · 1,398 GB/s · Professional workstation card (specs are NVIDIA preliminary data)- Suggested precision
- INT4 (AWQ/GPTQ)
- GPUs needed
- 16 (exceeds one node: multi-node needed)
- Est. decode speed
- 913 tok/ssingle-stream theoretical ceiling
- Est. batched throughput
- out 931 tok/sin 5,756 tok/s · many concurrent requests · PCIe multi-GPU penalty
- Hardware list price
- $182,446
- Hardware market price
- $203,620–$215,598
- Typical use
- Workstation large VRAM inference, departmental private deployment
RTX PRO 5000 Blackwell 48GB
48 GB · 1,344 GB/s · Professional workstation card- Suggested precision
- INT4 (AWQ/GPTQ)
- GPUs needed
- 32 (exceeds one node: multi-node needed)
- Est. decode speed
- 1,755 tok/ssingle-stream theoretical ceiling
- Est. batched throughput
- out 1,791 tok/sin 6,571 tok/s · many concurrent requests · PCIe multi-GPU penalty
- Hardware list price
- n/aCloud rental (live) ×32: $19.26–$36.35/hr (2026-11-20)
- Hardware market price
- $137,868
- Typical use
- Small to medium model inference, development testing, single-machine multi-card private deployment
L40S 48GB
48 GB · 864 GB/s · Compliant inference card- Suggested precision
- INT4 (AWQ/GPTQ)
- GPUs needed
- 32 (exceeds one node: multi-node needed)
- Est. decode speed
- 1,128 tok/ssingle-stream theoretical ceiling
- Est. batched throughput
- out 1,151 tok/sin 9,220 tok/s · many concurrent requests · PCIe multi-GPU penalty
- Hardware list price
- $215,598–$311,419Cloud rental (live) ×32: $14.94–$25.66/hr (2026-11-20)
- Hardware market price
- $215,598–$311,419
- Typical use
- SME teams building inference services, medium-low concurrency
L20 48GB
48 GB · 864 GB/s · Compliant inference card- Suggested precision
- INT4 (AWQ/GPTQ)
- GPUs needed
- 32 (exceeds one node: multi-node needed)
- Est. decode speed
- 1,128 tok/ssingle-stream theoretical ceiling
- Est. batched throughput
- out 1,151 tok/sin 3,044 tok/s · many concurrent requests · PCIe multi-GPU penalty
- Hardware list price
- $111,857–$112,499
- Hardware market price
- $167,687
- Typical use
- SME teams building inference services, medium-low concurrency
RTX 5090 32GB
32 GB · 1,792 GB/s · Consumer flagship (not sold in China, 5090D for domestic market)- Suggested precision
- INT4 (AWQ/GPTQ)
- GPUs needed
- 32 (exceeds one node: multi-node needed)
- Est. decode speed
- 2,341 tok/ssingle-stream theoretical ceiling
- Est. batched throughput
- out 2,387 tok/sin 10,672 tok/s · many concurrent requests · PCIe multi-GPU penalty
- Hardware list price
- $64,009Cloud rental (live) ×32: $14.59–$19.23/hr (2026-11-20)
- Hardware market price
- $229,971–$249,135
- Typical use
- Local inference and development for individuals/small teams, multi-card VRAM pooling for quantized MoE
RTX 5090D 24GB
24 GB · 1,344 GB/s · Consumer-grade/small-scale self-built- Suggested precision
- INT4 (AWQ/GPTQ)
- GPUs needed
- 64 (exceeds one node: multi-node needed)
- Est. decode speed
- 3,511 tok/ssingle-stream theoretical ceiling
- Est. batched throughput
- out 3,581 tok/sin 15,129 tok/s · many concurrent requests · PCIe multi-GPU penalty
- Hardware list price
- $158,096
- Hardware market price
- $268,300–$383,285
- Typical use
- Personal development/small-scale verification/POC, not recommended for production-level high concurrency
RTX 4090D 24GB
24 GB · 1,008 GB/s · Consumer-grade/small-scale self-built- Suggested precision
- INT4 (AWQ/GPTQ)
- GPUs needed
- 64 (exceeds one node: multi-node needed)
- Est. decode speed
- 2,633 tok/ssingle-stream theoretical ceiling
- Est. batched throughput
- out 2,686 tok/sin 14,976 tok/s · many concurrent requests · PCIe multi-GPU penalty
- Hardware list price
- $124,558
- Hardware market price
- $167,678
- Typical use
- Personal development/small-scale verification/POC, not recommended for production-level high concurrency
FAQ
How is the estimated decode speed calculated?
Estimated speed = (memory bandwidth × GPU count) ÷ (active parameters × bytes per parameter): the memory-bound upper limit for one request (roofline). It scales linearly with GPU count and ignores interconnect overhead, batching and framework losses, so real results are usually well below it.
Can I trust the speed numbers for consumer GPUs?
The linear assumption is reasonable for NVLink data-center cards (H/B series) but too optimistic for consumer PCIe cards without NVLink (RTX 5090D/4090D); multi-GPU consumer setups lose far more to communication, so equal-looking numbers do not mean equal performance.
How is “Suggested precision” chosen?
It is the highest precision that fits within 8 GPUs; beyond that it is flagged as multi-node.
Where do the prices come from?
USD converted at ¥6.68 per USD. Cloud rental is the international vast.ai market rate, not a China purchase price.