<!-- Markdown version for AI agents. Canonical HTML: https://www.tandiu.cn/en/local-deploy . Data is updated daily; see https://www.tandiu.cn/llms.txt -->
SELF-HOSTING

# Self-Hosting LLMs: VRAM & GPU Sizing

VRAM, GPU count and speed for running open-weights models yourself, next to the vendor's API price and measured speed.
Data updated: 2026-11-05
SortPopularity = Hugging Face downloads of the official repo, last 30 days (2026-11-05)

Alibaba Qwen**3 models · 11.8M downloads (30d) · latest 2026-08-24**

## [Qwen3.8 27B](https://www.tandiu.cn/en/local-deploy-dashscope-qwen3-8-27b)

28B total · 28B active · apache-2.0

Popularity
**8.0M**HF downloads (30d) · 19,154 likes
Released
**2026-08-05**HF repo created
Public benchmarks
**AA 20.1 · LiveBench 60.5**
Vendor API
**$0.50 / $3.00**input / output per 1M tokens
FP16 VRAM
**≈67 GB**INT4 ≈17 GB

**VRAM and reference setups (FP16 / INT8 / INT4)**

FP16/BF16 (full precision)
**≈67 GB VRAML20 48GB×2**List price $6,985–$7,025Market price $10,472
**Ascend 910B 64GB×2**List price n/aMarket price $32,911–$35,903
**RTX PRO 6000 Blackwell 96GB×1**List price $16,025Market price $17,586Cloud rental (live): $1.20–$2.27/hr (2026-11-05)
**RTX 5090 32GB×4**List price n/aMarket price $26,927Cloud rental (live) ×4: $1.77–$2.13/hr (2026-11-05)
**RTX 5090D 24GB×4**List price $9,873Market price $16,755–$23,935Cloud rental (live) ×4: $1.77–$2.12/hr (2026-11-05)
INT8
**≈33 GB VRAML20 48GB×1**List price $3,493–$3,513Market price $5,236
**Ascend 910B 64GB×1**List price n/aMarket price $16,455–$17,951
**RTX PRO 6000 Blackwell 96GB×1**List price $16,025Market price $17,586Cloud rental (live): $1.20–$2.27/hr (2026-11-05)
**RTX 5090 32GB×2**List price n/aMarket price $13,463Cloud rental (live) ×2: $0.88–$1.06/hr (2026-11-05)
**RTX 5090D 24GB×2**List price $4,936Market price $8,377–$11,968Cloud rental (live) ×2: $0.88–$1.06/hr (2026-11-05)
INT4 (AWQ/GPTQ)
**≈17 GB VRAML20 48GB×1**List price $3,493–$3,513Market price $5,236
**Ascend 910B 64GB×1**List price n/aMarket price $16,455–$17,951
**RTX PRO 6000 Blackwell 96GB×1**List price $16,025Market price $17,586Cloud rental (live): $1.20–$2.27/hr (2026-11-05)
**RTX 5090 32GB×1**List price n/aMarket price $6,732Cloud rental (live): $0.44–$0.53/hr (2026-11-05)
**RTX 5090D 24GB×1**List price $2,468Market price $4,189–$5,984Cloud rental (live): $0.44–$0.53/hr (2026-11-05)
[Full setup across 18 GPUs →](https://www.tandiu.cn/en/local-deploy-dashscope-qwen3-8-27b) [Official repo →](https://huggingface.co/Qwen/Qwen3.8-27B)

## [Qwen3.8-Flash-Next](https://www.tandiu.cn/en/local-deploy-hf-qwen-qwen3-8-flash-next)

180B total · 6B active · other

Popularity
**3.8M**HF downloads (30d) · 6,785 likes
Released
**2026-08-24**HF repo created
Public benchmarks
**No public benchmark yet**
Vendor API
**—**input / output per 1M tokens
FP16 VRAM
**≈432 GB**INT4 ≈108 GB

**VRAM and reference setups (FP16 / INT8 / INT4)**

FP16/BF16 (full precision)
**≈432 GB VRAML20 48GB×16**List price $55,882–$56,202Market price $83,773
**Ascend 910B 64GB×8**List price n/aMarket price $131,644–$143,612
**RTX PRO 6000 Blackwell 96GB×8**List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
**RTX 5090 32GB×16**List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
**RTX 5090D 24GB×32**List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
INT8
**≈216 GB VRAML20 48GB×8**List price $27,941–$28,101Market price $41,887
**Ascend 910B 64GB×4**List price n/aMarket price $65,822–$71,806
**RTX PRO 6000 Blackwell 96GB×4**List price $64,100Market price $70,346Cloud rental (live) ×4: $4.81–$9.08/hr (2026-11-05)
**RTX 5090 32GB×8**List price n/aMarket price $53,853Cloud rental (live) ×8: $3.54–$4.26/hr (2026-11-05)
**RTX 5090D 24GB×16**List price $39,491Market price $67,019–$95,741Cloud rental (live) ×16: $7.07–$8.48/hr (2026-11-05)
INT4 (AWQ/GPTQ)
**≈108 GB VRAML20 48GB×4**List price $13,970–$14,051Market price $20,943
**Ascend 910B 64GB×2**List price n/aMarket price $32,911–$35,903
**RTX PRO 6000 Blackwell 96GB×2**List price $32,050Market price $35,173Cloud rental (live) ×2: $2.41–$4.54/hr (2026-11-05)
**RTX 5090 32GB×4**List price n/aMarket price $26,927Cloud rental (live) ×4: $1.77–$2.13/hr (2026-11-05)
**RTX 5090D 24GB×8**List price $19,745Market price $33,509–$47,871Cloud rental (live) ×8: $3.54–$4.24/hr (2026-11-05)
[Full setup across 18 GPUs →](https://www.tandiu.cn/en/local-deploy-hf-qwen-qwen3-8-flash-next) [Official repo →](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)

## [Qwen3.8 2.4T A95B](https://www.tandiu.cn/en/local-deploy-dashscope-qwen3-8-2-4t-a95b)

2,446B total · 95B active · Qwen custom open-source license agreement (HF labeled license:other, non-standard OSI protocol, specific terms must be reviewed before use)

Popularity
**26.3K**HF downloads (30d) · 1,364 likes
Released
**2026-08-08**HF repo created
Public benchmarks
**No public benchmark yet**
Vendor API
**$2.00 / $6.00**input / output per 1M tokens
FP16 VRAM
**≈5,871 GB**INT4 ≈1,468 GB

**VRAM and reference setups (FP16 / INT8 / INT4)**

FP16/BF16 (full precision)
**≈5,871 GB VRAML20 48GB×123**List price $429,590–$432,056Market price $644,008
**Ascend 910B 64GB×92**List price n/aMarket price $1,513,905–$1,651,533
**RTX PRO 6000 Blackwell 96GB×64**List price $1,025,597Market price $1,125,531Cloud rental (live) ×64: $76.99–$145.22/hr (2026-11-05)
**RTX 5090 32GB×184**List price n/aMarket price $1,238,622Cloud rental (live) ×184: $81.33–$97.89/hr (2026-11-05)
**RTX 5090D 24GB×245**List price $604,703Market price $1,026,224–$1,466,034Cloud rental (live) ×245: $108.29–$129.85/hr (2026-11-05)
INT8
**≈2,935 GB VRAML20 48GB×64**List price $223,527–$224,809Market price $335,094
**Ascend 910B 64GB×64**List price n/aMarket price $1,053,151–$1,148,892
**RTX PRO 6000 Blackwell 96GB×32**List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
**RTX 5090 32GB×92**List price n/aMarket price $619,311Cloud rental (live) ×92: $40.66–$48.94/hr (2026-11-05)
**RTX 5090D 24GB×123**List price $303,585Market price $515,206–$736,009Cloud rental (live) ×123: $54.37–$65.19/hr (2026-11-05)
INT4 (AWQ/GPTQ)
**≈1,468 GB VRAML20 48GB×32**List price $111,763–$112,405Market price $167,547
**Ascend 910B 64GB×32**List price n/aMarket price $526,576–$574,446
**RTX PRO 6000 Blackwell 96GB×16**List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
**RTX 5090 32GB×64**List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
**RTX 5090D 24GB×64**List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
[Full setup across 18 GPUs →](https://www.tandiu.cn/en/local-deploy-dashscope-qwen3-8-2-4t-a95b) [Official repo →](https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B)

Zhipu AI (GLM)**5 models · 11.3M downloads (30d) · latest 2026-08-25**

## [GLM-5.3-Flash](https://www.tandiu.cn/en/local-deploy-zai-glm-5-3-flash)

321B total · 18B active · mit

Popularity
**6.3M**HF downloads (30d) · 3,050 likes
Released
**2026-08-25**HF repo created
Public benchmarks
**AA 27.8 · LiveBench 65.8**
Vendor API
**$0.15 / $0.50**input / output per 1M tokens
FP16 VRAM
**≈771 GB**INT4 ≈193 GB

**VRAM and reference setups (FP16 / INT8 / INT4)**

FP16/BF16 (full precision)
**≈771 GB VRAML20 48GB×32**List price $111,763–$112,405Market price $167,547
**Ascend 910B 64GB×16**List price n/aMarket price $263,288–$287,223
**RTX PRO 6000 Blackwell 96GB×16**List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
**RTX 5090 32GB×32**List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
**RTX 5090D 24GB×64**List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT8
**≈386 GB VRAML20 48GB×16**List price $55,882–$56,202Market price $83,773
**Ascend 910B 64GB×8**List price n/aMarket price $131,644–$143,612
**RTX PRO 6000 Blackwell 96GB×8**List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
**RTX 5090 32GB×16**List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
**RTX 5090D 24GB×32**List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
INT4 (AWQ/GPTQ)
**≈193 GB VRAML20 48GB×8**List price $27,941–$28,101Market price $41,887
**Ascend 910B 64GB×4**List price n/aMarket price $65,822–$71,806
**RTX PRO 6000 Blackwell 96GB×4**List price $64,100Market price $70,346Cloud rental (live) ×4: $4.81–$9.08/hr (2026-11-05)
**RTX 5090 32GB×8**List price n/aMarket price $53,853Cloud rental (live) ×8: $3.54–$4.26/hr (2026-11-05)
**RTX 5090D 24GB×16**List price $39,491Market price $67,019–$95,741Cloud rental (live) ×16: $7.07–$8.48/hr (2026-11-05)
[Full setup across 18 GPUs →](https://www.tandiu.cn/en/local-deploy-zai-glm-5-3-flash) [Official repo →](https://huggingface.co/zai-org/GLM-5.3-Flash)

## [GLM-5.3](https://www.tandiu.cn/en/local-deploy-zai-glm-5-3)

753B total · 40B active (est.) · MIT License

Popularity
**2.3M**HF downloads (30d) · 2,381 likes
Released
**2026-08-25**HF repo created
Public benchmarks
**AA 31.7 · LiveBench 0.0**
Vendor API
**$1.40 / $4.40**input / output per 1M tokens
FP16 VRAM
**≈1,808 GB**INT4 ≈452 GB

**VRAM and reference setups (FP16 / INT8 / INT4)**

FP16/BF16 (full precision)
**≈1,808 GB VRAML20 48GB×64**List price $223,527–$224,809Market price $335,094
**Ascend 910B 64GB×32**List price n/aMarket price $526,576–$574,446
**RTX PRO 6000 Blackwell 96GB×32**List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
**RTX 5090 32GB×64**List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
**RTX 5090D 24GB×76**List price $187,581Market price $318,339–$454,770Cloud rental (live) ×76: $33.59–$40.28/hr (2026-11-05)
INT8
**≈904 GB VRAML20 48GB×32**List price $111,763–$112,405Market price $167,547
**Ascend 910B 64GB×16**List price n/aMarket price $263,288–$287,223
**RTX PRO 6000 Blackwell 96GB×16**List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
**RTX 5090 32GB×32**List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
**RTX 5090D 24GB×64**List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT4 (AWQ/GPTQ)
**≈452 GB VRAML20 48GB×16**List price $55,882–$56,202Market price $83,773
**Ascend 910B 64GB×8**List price n/aMarket price $131,644–$143,612
**RTX PRO 6000 Blackwell 96GB×8**List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
**RTX 5090 32GB×16**List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
**RTX 5090D 24GB×32**List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
[Full setup across 18 GPUs →](https://www.tandiu.cn/en/local-deploy-zai-glm-5-3) [Official repo →](https://huggingface.co/zai-org/GLM-5.3)

## [GLM-5](https://www.tandiu.cn/en/local-deploy-zai-glm-5)

754B total · 40B active · mit

Popularity
**1.5M**HF downloads (30d) · 2,141 likes
Released
**2026-02-11**HF repo created
Public benchmarks
**AA — · LiveBench 0.0**
Vendor API
**$1.00 / $3.20**input / output per 1M tokens
FP16 VRAM
**≈1,809 GB**INT4 ≈452 GB

**VRAM and reference setups (FP16 / INT8 / INT4)**

FP16/BF16 (full precision)
**≈1,809 GB VRAML20 48GB×64**List price $223,527–$224,809Market price $335,094
**Ascend 910B 64GB×32**List price n/aMarket price $526,576–$574,446
**RTX PRO 6000 Blackwell 96GB×32**List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
**RTX 5090 32GB×64**List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
**RTX 5090D 24GB×76**List price $187,581Market price $318,339–$454,770Cloud rental (live) ×76: $33.59–$40.28/hr (2026-11-05)
INT8
**≈905 GB VRAML20 48GB×32**List price $111,763–$112,405Market price $167,547
**Ascend 910B 64GB×16**List price n/aMarket price $263,288–$287,223
**RTX PRO 6000 Blackwell 96GB×16**List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
**RTX 5090 32GB×32**List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
**RTX 5090D 24GB×64**List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT4 (AWQ/GPTQ)
**≈452 GB VRAML20 48GB×16**List price $55,882–$56,202Market price $83,773
**Ascend 910B 64GB×8**List price n/aMarket price $131,644–$143,612
**RTX PRO 6000 Blackwell 96GB×8**List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
**RTX 5090 32GB×16**List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
**RTX 5090D 24GB×32**List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
[Full setup across 18 GPUs →](https://www.tandiu.cn/en/local-deploy-zai-glm-5) [Official repo →](https://huggingface.co/zai-org/GLM-5)

## [GLM-5.1](https://www.tandiu.cn/en/local-deploy-zai-glm-5-1)

754B total · 40B active · MIT License

Popularity
**922.6K**HF downloads (30d) · 1,850 likes
Released
**2026-04-03**HF repo created
Public benchmarks
**AA — · LiveBench 0.0**
Vendor API
**$1.40 / $4.40**input / output per 1M tokens
FP16 VRAM
**≈1,809 GB**INT4 ≈452 GB

**VRAM and reference setups (FP16 / INT8 / INT4)**

FP16/BF16 (full precision)
**≈1,809 GB VRAML20 48GB×64**List price $223,527–$224,809Market price $335,094
**Ascend 910B 64GB×32**List price n/aMarket price $526,576–$574,446
**RTX PRO 6000 Blackwell 96GB×32**List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
**RTX 5090 32GB×64**List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
**RTX 5090D 24GB×76**List price $187,581Market price $318,339–$454,770Cloud rental (live) ×76: $33.59–$40.28/hr (2026-11-05)
INT8
**≈905 GB VRAML20 48GB×32**List price $111,763–$112,405Market price $167,547
**Ascend 910B 64GB×16**List price n/aMarket price $263,288–$287,223
**RTX PRO 6000 Blackwell 96GB×16**List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
**RTX 5090 32GB×32**List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
**RTX 5090D 24GB×64**List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT4 (AWQ/GPTQ)
**≈452 GB VRAML20 48GB×16**List price $55,882–$56,202Market price $83,773
**Ascend 910B 64GB×8**List price n/aMarket price $131,644–$143,612
**RTX PRO 6000 Blackwell 96GB×8**List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
**RTX 5090 32GB×16**List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
**RTX 5090D 24GB×32**List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
[Full setup across 18 GPUs →](https://www.tandiu.cn/en/local-deploy-zai-glm-5-1) [Official repo →](https://huggingface.co/zai-org/GLM-5.1)

## [GLM-5.2](https://www.tandiu.cn/en/local-deploy-zai-glm-5-2)

753B total · 39B active · MIT License

Popularity
**332.2K**HF downloads (30d) · 5,202 likes
Released
**2026-06-16**HF repo created
Public benchmarks
**AA — · LiveBench 0.0**
Vendor API
**$1.40 / $4.40**input / output per 1M tokens
FP16 VRAM
**≈1,808 GB**INT4 ≈452 GB

**VRAM and reference setups (FP16 / INT8 / INT4)**

FP16/BF16 (full precision)
**≈1,808 GB VRAML20 48GB×64**List price $223,527–$224,809Market price $335,094
**Ascend 910B 64GB×32**List price n/aMarket price $526,576–$574,446
**RTX PRO 6000 Blackwell 96GB×32**List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
**RTX 5090 32GB×64**List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
**RTX 5090D 24GB×76**List price $187,581Market price $318,339–$454,770Cloud rental (live) ×76: $33.59–$40.28/hr (2026-11-05)
INT8
**≈904 GB VRAML20 48GB×32**List price $111,763–$112,405Market price $167,547
**Ascend 910B 64GB×16**List price n/aMarket price $263,288–$287,223
**RTX PRO 6000 Blackwell 96GB×16**List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
**RTX 5090 32GB×32**List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
**RTX 5090D 24GB×64**List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT4 (AWQ/GPTQ)
**≈452 GB VRAML20 48GB×16**List price $55,882–$56,202Market price $83,773
**Ascend 910B 64GB×8**List price n/aMarket price $131,644–$143,612
**RTX PRO 6000 Blackwell 96GB×8**List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
**RTX 5090 32GB×16**List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
**RTX 5090D 24GB×32**List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
[Full setup across 18 GPUs →](https://www.tandiu.cn/en/local-deploy-zai-glm-5-2) [Official repo →](https://huggingface.co/zai-org/GLM-5.2)

DeepSeek**2 models · 2.3M downloads (30d) · latest 2026-09-10**

## [DeepSeek V4.1 Flash](https://www.tandiu.cn/en/local-deploy-deepseek-v4-flash)

763B total · 16B active · MIT

Popularity
**2.0M**HF downloads (30d) · 4,863 likes
Released
**2026-09-10**HF repo created
Public benchmarks
**AA 32.6 · LiveBench 75.8**
Vendor API
**$0.30 / $1.20**input / output per 1M tokens
FP16 VRAM
**≈1,832 GB**INT4 ≈458 GB

**VRAM and reference setups (FP16 / INT8 / INT4)**

FP16/BF16 (full precision)
**≈1,832 GB VRAML20 48GB×64**List price $223,527–$224,809Market price $335,094
**Ascend 910B 64GB×32**List price n/aMarket price $526,576–$574,446
**RTX PRO 6000 Blackwell 96GB×32**List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
**RTX 5090 32GB×64**List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
**RTX 5090D 24GB×77**List price $190,049Market price $322,528–$460,754Cloud rental (live) ×77: $34.03–$40.81/hr (2026-11-05)
INT8
**≈916 GB VRAML20 48GB×32**List price $111,763–$112,405Market price $167,547
**Ascend 910B 64GB×16**List price n/aMarket price $263,288–$287,223
**RTX PRO 6000 Blackwell 96GB×16**List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
**RTX 5090 32GB×32**List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
**RTX 5090D 24GB×64**List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT4 (AWQ/GPTQ)
**≈458 GB VRAML20 48GB×16**List price $55,882–$56,202Market price $83,773
**Ascend 910B 64GB×8**List price n/aMarket price $131,644–$143,612
**RTX PRO 6000 Blackwell 96GB×8**List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
**RTX 5090 32GB×16**List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
**RTX 5090D 24GB×32**List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
[Full setup across 18 GPUs →](https://www.tandiu.cn/en/local-deploy-deepseek-v4-flash) [Official repo →](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash)

## [DeepSeek V4 Pro](https://www.tandiu.cn/en/local-deploy-deepseek-v4-pro)

1,599B total · 49B active · DeepSeek open-source license agreement

Popularity
**336.9K**HF downloads (30d) · 5,696 likes
Released
**2026-04-22**HF repo created
Public benchmarks
**AA 23.1 · LiveBench 0.0**
Vendor API
**$1.32 / $3.96**input / output per 1M tokens
FP16 VRAM
**≈3,837 GB**INT4 ≈959 GB

**VRAM and reference setups (FP16 / INT8 / INT4)**

FP16/BF16 (full precision)
**≈3,837 GB VRAML20 48GB×80**List price $279,408–$281,012Market price $418,867
**Ascend 910B 64GB×64**List price n/aMarket price $1,053,151–$1,148,892
**RTX PRO 6000 Blackwell 96GB×64**List price $1,025,597Market price $1,125,531Cloud rental (live) ×64: $76.99–$145.22/hr (2026-11-05)
**RTX 5090 32GB×120**List price n/aMarket price $807,797Cloud rental (live) ×120: $53.04–$63.84/hr (2026-11-05)
**RTX 5090D 24GB×160**List price $394,908Market price $670,187–$957,410Cloud rental (live) ×160: $70.72–$84.80/hr (2026-11-05)
INT8
**≈1,919 GB VRAML20 48GB×64**List price $223,527–$224,809Market price $335,094
**Ascend 910B 64GB×32**List price n/aMarket price $526,576–$574,446
**RTX PRO 6000 Blackwell 96GB×32**List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
**RTX 5090 32GB×64**List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
**RTX 5090D 24GB×80**List price $197,454Market price $335,094–$478,705Cloud rental (live) ×80: $35.36–$42.40/hr (2026-11-05)
INT4 (AWQ/GPTQ)
**≈959 GB VRAML20 48GB×32**List price $111,763–$112,405Market price $167,547
**Ascend 910B 64GB×16**List price n/aMarket price $263,288–$287,223
**RTX PRO 6000 Blackwell 96GB×16**List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
**RTX 5090 32GB×32**List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
**RTX 5090D 24GB×64**List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
[Full setup across 18 GPUs →](https://www.tandiu.cn/en/local-deploy-deepseek-v4-pro) [Official repo →](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)

Moonshot AI (Kimi)**5 models · 1.5M downloads (30d) · latest 2026-11-02**

## [Kimi K3](https://www.tandiu.cn/en/local-deploy-moonshot-kimi-k3)

2,780B total · 104B active · Modified MIT License

Popularity
**768.7K**HF downloads (30d) · 11,939 likes
Released
**2026-06-13**HF repo created
Public benchmarks
**AA — · LiveBench 69.5**
Vendor API
**$3.00 / $15.00**input / output per 1M tokens
FP16 VRAM
**≈6,672 GB**INT4 ≈1,668 GB

**VRAM and reference setups (FP16 / INT8 / INT4)**

FP16/BF16 (full precision)
**≈6,672 GB VRAML20 48GB×139**List price $485,472–$488,258Market price $727,781
**Ascend 910B 64GB×105**List price n/aMarket price $1,727,826–$1,884,901
**RTX PRO 6000 Blackwell 96GB×70**List price $1,121,747Market price $1,231,050Cloud rental (live) ×70: $84.21–$158.83/hr (2026-11-05)
**RTX 5090 32GB×209**List price n/aMarket price $1,406,913Cloud rental (live) ×209: $92.38–$111.19/hr (2026-11-05)
**RTX 5090D 24GB×278**List price $686,152Market price $1,164,450–$1,663,500Cloud rental (live) ×278: $122.88–$147.34/hr (2026-11-05)
INT8
**≈3,336 GB VRAML20 48GB×70**List price $244,482–$245,885Market price $366,509
**Ascend 910B 64GB×64**List price n/aMarket price $1,053,151–$1,148,892
**RTX PRO 6000 Blackwell 96GB×64**List price $1,025,597Market price $1,125,531Cloud rental (live) ×64: $76.99–$145.22/hr (2026-11-05)
**RTX 5090 32GB×105**List price n/aMarket price $706,822Cloud rental (live) ×105: $46.41–$55.86/hr (2026-11-05)
**RTX 5090D 24GB×139**List price $343,076Market price $582,225–$831,750Cloud rental (live) ×139: $61.44–$73.67/hr (2026-11-05)
INT4 (AWQ/GPTQ)
**≈1,668 GB VRAML20 48GB×64**List price $223,527–$224,809Market price $335,094
**Ascend 910B 64GB×32**List price n/aMarket price $526,576–$574,446
**RTX PRO 6000 Blackwell 96GB×32**List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
**RTX 5090 32GB×64**List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
**RTX 5090D 24GB×70**List price $172,772Market price $293,207–$418,867Cloud rental (live) ×70: $30.94–$37.10/hr (2026-11-05)
[Full setup across 18 GPUs →](https://www.tandiu.cn/en/local-deploy-moonshot-kimi-k3) [Official repo →](https://huggingface.co/moonshotai/Kimi-K3)

## [Kimi K2.5](https://www.tandiu.cn/en/local-deploy-moonshot-kimi-k2-5)

1,027B total · 32B active · Open-source weight license

Popularity
**395.4K**HF downloads (30d) · 2,898 likes
Released
**2026-01-01**HF repo created
Public benchmarks
**AA — · LiveBench 0.0**
Vendor API
**$0.60 / $3.00**input / output per 1M tokens
FP16 VRAM
**≈2,465 GB**INT4 ≈616 GB

**VRAM and reference setups (FP16 / INT8 / INT4)**

FP16/BF16 (full precision)
**≈2,465 GB VRAML20 48GB×64**List price $223,527–$224,809Market price $335,094
**Ascend 910B 64GB×64**List price n/aMarket price $1,053,151–$1,148,892
**RTX PRO 6000 Blackwell 96GB×32**List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
**RTX 5090 32GB×78**List price n/aMarket price $525,068Cloud rental (live) ×78: $34.48–$41.50/hr (2026-11-05)
**RTX 5090D 24GB×103**List price $254,222Market price $431,433–$616,333Cloud rental (live) ×103: $45.53–$54.59/hr (2026-11-05)
INT8
**≈1,232 GB VRAML20 48GB×32**List price $111,763–$112,405Market price $167,547
**Ascend 910B 64GB×32**List price n/aMarket price $526,576–$574,446
**RTX PRO 6000 Blackwell 96GB×16**List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
**RTX 5090 32GB×64**List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
**RTX 5090D 24GB×64**List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT4 (AWQ/GPTQ)
**≈616 GB VRAML20 48GB×16**List price $55,882–$56,202Market price $83,773
**Ascend 910B 64GB×16**List price n/aMarket price $263,288–$287,223
**RTX PRO 6000 Blackwell 96GB×8**List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
**RTX 5090 32GB×32**List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
**RTX 5090D 24GB×32**List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
[Full setup across 18 GPUs →](https://www.tandiu.cn/en/local-deploy-moonshot-kimi-k2-5) [Official repo →](https://huggingface.co/moonshotai/Kimi-K2.5)

## [Kimi K2.6](https://www.tandiu.cn/en/local-deploy-moonshot-kimi-k2-6)

1,027B total · 32B active · Modified MIT License

Popularity
**244.2K**HF downloads (30d) · 1,640 likes
Released
**2026-04-14**HF repo created
Public benchmarks
**AA — · LiveBench 0.0**
Vendor API
**$0.95 / $4.00**input / output per 1M tokens
FP16 VRAM
**≈2,465 GB**INT4 ≈616 GB

**VRAM and reference setups (FP16 / INT8 / INT4)**

FP16/BF16 (full precision)
**≈2,465 GB VRAML20 48GB×64**List price $223,527–$224,809Market price $335,094
**Ascend 910B 64GB×64**List price n/aMarket price $1,053,151–$1,148,892
**RTX PRO 6000 Blackwell 96GB×32**List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
**RTX 5090 32GB×78**List price n/aMarket price $525,068Cloud rental (live) ×78: $34.48–$41.50/hr (2026-11-05)
**RTX 5090D 24GB×103**List price $254,222Market price $431,433–$616,333Cloud rental (live) ×103: $45.53–$54.59/hr (2026-11-05)
INT8
**≈1,232 GB VRAML20 48GB×32**List price $111,763–$112,405Market price $167,547
**Ascend 910B 64GB×32**List price n/aMarket price $526,576–$574,446
**RTX PRO 6000 Blackwell 96GB×16**List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
**RTX 5090 32GB×64**List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
**RTX 5090D 24GB×64**List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT4 (AWQ/GPTQ)
**≈616 GB VRAML20 48GB×16**List price $55,882–$56,202Market price $83,773
**Ascend 910B 64GB×16**List price n/aMarket price $263,288–$287,223
**RTX PRO 6000 Blackwell 96GB×8**List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
**RTX 5090 32GB×32**List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
**RTX 5090D 24GB×32**List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
[Full setup across 18 GPUs →](https://www.tandiu.cn/en/local-deploy-moonshot-kimi-k2-6) [Official repo →](https://huggingface.co/moonshotai/Kimi-K2.6)

## [Kimi K2.7 Code](https://www.tandiu.cn/en/local-deploy-moonshot-kimi-k2-7-code)

1,027B total · 32B active · Modified MIT License

Popularity
**81.8K**HF downloads (30d) · 1,429 likes
Released
**2026-06-11**HF repo created
Public benchmarks
**AA — · LiveBench 0.0**
Vendor API
**$0.95 / $4.00**input / output per 1M tokens
FP16 VRAM
**≈2,465 GB**INT4 ≈616 GB

**VRAM and reference setups (FP16 / INT8 / INT4)**

FP16/BF16 (full precision)
**≈2,465 GB VRAML20 48GB×64**List price $223,527–$224,809Market price $335,094
**Ascend 910B 64GB×64**List price n/aMarket price $1,053,151–$1,148,892
**RTX PRO 6000 Blackwell 96GB×32**List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
**RTX 5090 32GB×78**List price n/aMarket price $525,068Cloud rental (live) ×78: $34.48–$41.50/hr (2026-11-05)
**RTX 5090D 24GB×103**List price $254,222Market price $431,433–$616,333Cloud rental (live) ×103: $45.53–$54.59/hr (2026-11-05)
INT8
**≈1,232 GB VRAML20 48GB×32**List price $111,763–$112,405Market price $167,547
**Ascend 910B 64GB×32**List price n/aMarket price $526,576–$574,446
**RTX PRO 6000 Blackwell 96GB×16**List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
**RTX 5090 32GB×64**List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
**RTX 5090D 24GB×64**List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT4 (AWQ/GPTQ)
**≈616 GB VRAML20 48GB×16**List price $55,882–$56,202Market price $83,773
**Ascend 910B 64GB×16**List price n/aMarket price $263,288–$287,223
**RTX PRO 6000 Blackwell 96GB×8**List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
**RTX 5090 32GB×32**List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
**RTX 5090D 24GB×32**List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
[Full setup across 18 GPUs →](https://www.tandiu.cn/en/local-deploy-moonshot-kimi-k2-7-code) [Official repo →](https://huggingface.co/moonshotai/Kimi-K2.7-Code)

## [Kimi K3.1 Pro](https://www.tandiu.cn/en/local-deploy-auto-moonshotai-kimi-k3-1-pro)

2,780B total · 104B active · other

Popularity
**797**HF downloads (30d) · 510 likes
Released
**2026-11-02**HF repo created
Public benchmarks
**AA 46.5 · LiveBench 74.7**
Vendor API
**$2.00 / $10.00**input / output per 1M tokens
FP16 VRAM
**≈6,672 GB**INT4 ≈1,668 GB

**VRAM and reference setups (FP16 / INT8 / INT4)**

FP16/BF16 (full precision)
**≈6,672 GB VRAML20 48GB×140**List price $488,964–$491,771Market price $733,017
**Ascend 910B 64GB×105**List price n/aMarket price $1,727,826–$1,884,901
**RTX PRO 6000 Blackwell 96GB×70**List price $1,121,747Market price $1,231,050Cloud rental (live) ×70: $84.21–$158.83/hr (2026-11-05)
**RTX 5090 32GB×209**List price n/aMarket price $1,406,913Cloud rental (live) ×209: $92.38–$111.19/hr (2026-11-05)
**RTX 5090D 24GB×279**List price $688,620Market price $1,168,639–$1,669,484Cloud rental (live) ×279: $123.32–$147.87/hr (2026-11-05)
INT8
**≈3,336 GB VRAML20 48GB×70**List price $244,482–$245,885Market price $366,509
**Ascend 910B 64GB×64**List price n/aMarket price $1,053,151–$1,148,892
**RTX PRO 6000 Blackwell 96GB×64**List price $1,025,597Market price $1,125,531Cloud rental (live) ×64: $76.99–$145.22/hr (2026-11-05)
**RTX 5090 32GB×105**List price n/aMarket price $706,822Cloud rental (live) ×105: $46.41–$55.86/hr (2026-11-05)
**RTX 5090D 24GB×140**List price $345,544Market price $586,414–$837,734Cloud rental (live) ×140: $61.88–$74.20/hr (2026-11-05)
INT4 (AWQ/GPTQ)
**≈1,668 GB VRAML20 48GB×64**List price $223,527–$224,809Market price $335,094
**Ascend 910B 64GB×32**List price n/aMarket price $526,576–$574,446
**RTX PRO 6000 Blackwell 96GB×32**List price $512,798Market price $562,766Cloud rental (live) ×32: $38.50–$72.61/hr (2026-11-05)
**RTX 5090 32GB×64**List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
**RTX 5090D 24GB×70**List price $172,772Market price $293,207–$418,867Cloud rental (live) ×70: $30.94–$37.10/hr (2026-11-05)
[Full setup across 18 GPUs →](https://www.tandiu.cn/en/local-deploy-auto-moonshotai-kimi-k3-1-pro) [Official repo →](https://huggingface.co/moonshotai/Kimi-K3.1-Pro)

MiniMax**2 models · 206.6K downloads (30d) · latest 2026-10-17**

## [MiniMax M3](https://www.tandiu.cn/en/local-deploy-minimax-minimax-m3)

427B total · 23B active · MIT License

Popularity
**176.4K**HF downloads (30d) · 1,601 likes
Released
**2026-06-02**HF repo created
Public benchmarks
**AA 18.9 · LiveBench 0.0**
Vendor API
**$0.30 / $1.20**input / output per 1M tokens
FP16 VRAM
**≈1,025 GB**INT4 ≈256 GB

**VRAM and reference setups (FP16 / INT8 / INT4)**

FP16/BF16 (full precision)
**≈1,025 GB VRAML20 48GB×32**List price $111,763–$112,405Market price $167,547
**Ascend 910B 64GB×32**List price n/aMarket price $526,576–$574,446
**RTX PRO 6000 Blackwell 96GB×16**List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
**RTX 5090 32GB×64**List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
**RTX 5090D 24GB×64**List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT8
**≈512 GB VRAML20 48GB×16**List price $55,882–$56,202Market price $83,773
**Ascend 910B 64GB×16**List price n/aMarket price $263,288–$287,223
**RTX PRO 6000 Blackwell 96GB×8**List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
**RTX 5090 32GB×32**List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
**RTX 5090D 24GB×32**List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
INT4 (AWQ/GPTQ)
**≈256 GB VRAML20 48GB×8**List price $27,941–$28,101Market price $41,887
**Ascend 910B 64GB×8**List price n/aMarket price $131,644–$143,612
**RTX PRO 6000 Blackwell 96GB×4**List price $64,100Market price $70,346Cloud rental (live) ×4: $4.81–$9.08/hr (2026-11-05)
**RTX 5090 32GB×16**List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
**RTX 5090D 24GB×16**List price $39,491Market price $67,019–$95,741Cloud rental (live) ×16: $7.07–$8.48/hr (2026-11-05)
[Full setup across 18 GPUs →](https://www.tandiu.cn/en/local-deploy-minimax-minimax-m3) [Official repo →](https://huggingface.co/MiniMaxAI/MiniMax-M3)

## [MiniMax M3.1 Flash](https://www.tandiu.cn/en/local-deploy-minimax-minimax-m3-1-flash)

220B total · 23B active · other

Popularity
**30.2K**HF downloads (30d) · 548 likes
Released
**2026-10-17**HF repo created
Public benchmarks
**AA 29.1 · LiveBench 70.6**
Vendor API
**$0.15 / $0.60**input / output per 1M tokens
FP16 VRAM
**≈529 GB**INT4 ≈132 GB

**VRAM and reference setups (FP16 / INT8 / INT4)**

FP16/BF16 (full precision)
**≈529 GB VRAML20 48GB×16**List price $55,882–$56,202Market price $83,773
**Ascend 910B 64GB×16**List price n/aMarket price $263,288–$287,223
**RTX PRO 6000 Blackwell 96GB×8**List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
**RTX 5090 32GB×32**List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
**RTX 5090D 24GB×32**List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
INT8
**≈265 GB VRAML20 48GB×8**List price $27,941–$28,101Market price $41,887
**Ascend 910B 64GB×8**List price n/aMarket price $131,644–$143,612
**RTX PRO 6000 Blackwell 96GB×4**List price $64,100Market price $70,346Cloud rental (live) ×4: $4.81–$9.08/hr (2026-11-05)
**RTX 5090 32GB×16**List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
**RTX 5090D 24GB×16**List price $39,491Market price $67,019–$95,741Cloud rental (live) ×16: $7.07–$8.48/hr (2026-11-05)
INT4 (AWQ/GPTQ)
**≈132 GB VRAML20 48GB×4**List price $13,970–$14,051Market price $20,943
**Ascend 910B 64GB×4**List price n/aMarket price $65,822–$71,806
**RTX PRO 6000 Blackwell 96GB×2**List price $32,050Market price $35,173Cloud rental (live) ×2: $2.41–$4.54/hr (2026-11-05)
**RTX 5090 32GB×8**List price n/aMarket price $53,853Cloud rental (live) ×8: $3.54–$4.26/hr (2026-11-05)
**RTX 5090D 24GB×8**List price $19,745Market price $33,509–$47,871Cloud rental (live) ×8: $3.54–$4.24/hr (2026-11-05)
[Full setup across 18 GPUs →](https://www.tandiu.cn/en/local-deploy-minimax-minimax-m3-1-flash) [Official repo →](https://huggingface.co/MiniMaxAI/MiniMax-M3.1-Flash)

Tencent Hunyuan**1 model · 67.5K downloads (30d) · latest 2025-06-25**

## [Hunyuan A13B](https://www.tandiu.cn/en/local-deploy-manual-hunyuan-a13b)

80B total · 13B active · Tencent Hunyuan Open Source License Agreement

Popularity
**67.5K**HF downloads (30d) · 795 likes
Released
**2025-06-25**HF repo created
Public benchmarks
**No public benchmark yet**
Vendor API
**$0.07 / $0.30**input / output per 1M tokens
FP16 VRAM
**≈192 GB**INT4 ≈48 GB

**VRAM and reference setups (FP16 / INT8 / INT4)**

FP16/BF16 (full precision)
**≈192 GB VRAML20 48GB×4**List price $13,970–$14,051Market price $20,943
**Ascend 910B 64GB×4**List price n/aMarket price $65,822–$71,806
**RTX PRO 6000 Blackwell 96GB×2**List price $32,050Market price $35,173Cloud rental (live) ×2: $2.41–$4.54/hr (2026-11-05)
**RTX 5090 32GB×8**List price n/aMarket price $53,853Cloud rental (live) ×8: $3.54–$4.26/hr (2026-11-05)
**RTX 5090D 24GB×8**List price $19,745Market price $33,509–$47,871Cloud rental (live) ×8: $3.54–$4.24/hr (2026-11-05)
INT8
**≈96 GB VRAML20 48GB×2**List price $6,985–$7,025Market price $10,472
**Ascend 910B 64GB×2**List price n/aMarket price $32,911–$35,903
**RTX PRO 6000 Blackwell 96GB×1**List price $16,025Market price $17,586Cloud rental (live): $1.20–$2.27/hr (2026-11-05)
**RTX 5090 32GB×4**List price n/aMarket price $26,927Cloud rental (live) ×4: $1.77–$2.13/hr (2026-11-05)
**RTX 5090D 24GB×4**List price $9,873Market price $16,755–$23,935Cloud rental (live) ×4: $1.77–$2.12/hr (2026-11-05)
INT4 (AWQ/GPTQ)
**≈48 GB VRAML20 48GB×1**List price $3,493–$3,513Market price $5,236
**Ascend 910B 64GB×1**List price n/aMarket price $16,455–$17,951
**RTX PRO 6000 Blackwell 96GB×1**List price $16,025Market price $17,586Cloud rental (live): $1.20–$2.27/hr (2026-11-05)
**RTX 5090 32GB×2**List price n/aMarket price $13,463Cloud rental (live) ×2: $0.88–$1.06/hr (2026-11-05)
**RTX 5090D 24GB×2**List price $4,936Market price $8,377–$11,968Cloud rental (live) ×2: $0.88–$1.06/hr (2026-11-05)
[Full setup across 18 GPUs →](https://www.tandiu.cn/en/local-deploy-manual-hunyuan-a13b) [Official repo →](https://github.com/Tencent-Hunyuan/Hunyuan-A13B)

IFM(MBZUAI)**1 model · 9.8K downloads (30d) · latest 2026-09-01**

## [K2 Horizon 375B A23B](https://www.tandiu.cn/en/local-deploy-hf-ifm-k2-horizon-375b-a23b)

379B total · 23B active · apache-2.0

Popularity
**9.8K**HF downloads (30d) · 118 likes
Released
**2026-09-01**HF repo created
Public benchmarks
**No public benchmark yet**
Vendor API
**—**input / output per 1M tokens
FP16 VRAM
**≈910 GB**INT4 ≈228 GB

**VRAM and reference setups (FP16 / INT8 / INT4)**

FP16/BF16 (full precision)
**≈910 GB VRAML20 48GB×32**List price $111,763–$112,405Market price $167,547
**Ascend 910B 64GB×16**List price n/aMarket price $263,288–$287,223
**RTX PRO 6000 Blackwell 96GB×16**List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
**RTX 5090 32GB×32**List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
**RTX 5090D 24GB×64**List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT8
**≈455 GB VRAML20 48GB×16**List price $55,882–$56,202Market price $83,773
**Ascend 910B 64GB×8**List price n/aMarket price $131,644–$143,612
**RTX PRO 6000 Blackwell 96GB×8**List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
**RTX 5090 32GB×16**List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
**RTX 5090D 24GB×32**List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
INT4 (AWQ/GPTQ)
**≈228 GB VRAML20 48GB×8**List price $27,941–$28,101Market price $41,887
**Ascend 910B 64GB×4**List price n/aMarket price $65,822–$71,806
**RTX PRO 6000 Blackwell 96GB×4**List price $64,100Market price $70,346Cloud rental (live) ×4: $4.81–$9.08/hr (2026-11-05)
**RTX 5090 32GB×8**List price n/aMarket price $53,853Cloud rental (live) ×8: $3.54–$4.26/hr (2026-11-05)
**RTX 5090D 24GB×16**List price $39,491Market price $67,019–$95,741Cloud rental (live) ×16: $7.07–$8.48/hr (2026-11-05)
[Full setup across 18 GPUs →](https://www.tandiu.cn/en/local-deploy-hf-ifm-k2-horizon-375b-a23b) [Official repo →](https://huggingface.co/IFM/K2-Horizon-375B-A23B)

阶跃星辰**1 model · 6.0K downloads (30d) · latest 2026-09-20**

## [Step 5 Preview](https://www.tandiu.cn/en/local-deploy-stepfun-step-5-preview)

604B total · 27B active · other

Popularity
**6.0K**HF downloads (30d) · 137 likes
Released
**2026-09-20**HF repo created
Public benchmarks
**AA 28.5 · LiveBench 61.0**
Vendor API
**$1.05 / $2.99**input / output per 1M tokens
FP16 VRAM
**≈1,450 GB**INT4 ≈363 GB

**VRAM and reference setups (FP16 / INT8 / INT4)**

FP16/BF16 (full precision)
**≈1,450 GB VRAML20 48GB×32**List price $111,763–$112,405Market price $167,547
**Ascend 910B 64GB×32**List price n/aMarket price $526,576–$574,446
**RTX PRO 6000 Blackwell 96GB×16**List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
**RTX 5090 32GB×64**List price n/aMarket price $430,825Cloud rental (live) ×64: $28.29–$34.05/hr (2026-11-05)
**RTX 5090D 24GB×64**List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT8
**≈725 GB VRAML20 48GB×16**List price $55,882–$56,202Market price $83,773
**Ascend 910B 64GB×16**List price n/aMarket price $263,288–$287,223
**RTX PRO 6000 Blackwell 96GB×8**List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
**RTX 5090 32GB×32**List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
**RTX 5090D 24GB×32**List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
INT4 (AWQ/GPTQ)
**≈363 GB VRAML20 48GB×8**List price $27,941–$28,101Market price $41,887
**Ascend 910B 64GB×8**List price n/aMarket price $131,644–$143,612
**RTX PRO 6000 Blackwell 96GB×4**List price $64,100Market price $70,346Cloud rental (live) ×4: $4.81–$9.08/hr (2026-11-05)
**RTX 5090 32GB×16**List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
**RTX 5090D 24GB×16**List price $39,491Market price $67,019–$95,741Cloud rental (live) ×16: $7.07–$8.48/hr (2026-11-05)
[Full setup across 18 GPUs →](https://www.tandiu.cn/en/local-deploy-stepfun-step-5-preview) [Official repo →](https://huggingface.co/stepfun-ai/Step-5-Preview)

Motif Technologies**1 model · 1.8K downloads (30d) · latest 2026-08-07**

## [Motif 3](https://www.tandiu.cn/en/local-deploy-hf-motif-technologies-motif-3)

315B total · 13B active · mit

Popularity
**1.8K**HF downloads (30d) · 154 likes
Released
**2026-08-07**HF repo created
Public benchmarks
**No public benchmark yet**
Vendor API
**—**input / output per 1M tokens
FP16 VRAM
**≈756 GB**INT4 ≈189 GB

**VRAM and reference setups (FP16 / INT8 / INT4)**

FP16/BF16 (full precision)
**≈756 GB VRAML20 48GB×16**List price $55,882–$56,202Market price $83,773
**Ascend 910B 64GB×16**List price n/aMarket price $263,288–$287,223
**RTX PRO 6000 Blackwell 96GB×8**List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
**RTX 5090 32GB×32**List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
**RTX 5090D 24GB×32**List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
INT8
**≈378 GB VRAML20 48GB×8**List price $27,941–$28,101Market price $41,887
**Ascend 910B 64GB×8**List price n/aMarket price $131,644–$143,612
**RTX PRO 6000 Blackwell 96GB×4**List price $64,100Market price $70,346Cloud rental (live) ×4: $4.81–$9.08/hr (2026-11-05)
**RTX 5090 32GB×16**List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
**RTX 5090D 24GB×16**List price $39,491Market price $67,019–$95,741Cloud rental (live) ×16: $7.07–$8.48/hr (2026-11-05)
INT4 (AWQ/GPTQ)
**≈189 GB VRAML20 48GB×4**List price $13,970–$14,051Market price $20,943
**Ascend 910B 64GB×4**List price n/aMarket price $65,822–$71,806
**RTX PRO 6000 Blackwell 96GB×2**List price $32,050Market price $35,173Cloud rental (live) ×2: $2.41–$4.54/hr (2026-11-05)
**RTX 5090 32GB×8**List price n/aMarket price $53,853Cloud rental (live) ×8: $3.54–$4.26/hr (2026-11-05)
**RTX 5090D 24GB×8**List price $19,745Market price $33,509–$47,871Cloud rental (live) ×8: $3.54–$4.24/hr (2026-11-05)
[Full setup across 18 GPUs →](https://www.tandiu.cn/en/local-deploy-hf-motif-technologies-motif-3) [Official repo →](https://huggingface.co/Motif-Technologies/Motif-3)

Nex AGI**1 model · 880 downloads (30d) · latest 2026-06-03**

## [Nex-N2-Pro](https://www.tandiu.cn/en/local-deploy-hf-nex-agi-nex-n2-pro)

397B total · 18B active (est.) · apache-2.0

Popularity
**880**HF downloads (30d) · 376 likes
Released
**2026-06-03**HF repo created
Public benchmarks
**No public benchmark yet**
Vendor API
**—**input / output per 1M tokens
FP16 VRAM
**≈952 GB**INT4 ≈238 GB

**VRAM and reference setups (FP16 / INT8 / INT4)**

FP16/BF16 (full precision)
**≈952 GB VRAML20 48GB×32**List price $111,763–$112,405Market price $167,547
**Ascend 910B 64GB×16**List price n/aMarket price $263,288–$287,223
**RTX PRO 6000 Blackwell 96GB×16**List price $256,399Market price $281,383Cloud rental (live) ×16: $19.25–$36.30/hr (2026-11-05)
**RTX 5090 32GB×32**List price n/aMarket price $215,413Cloud rental (live) ×32: $14.14–$17.02/hr (2026-11-05)
**RTX 5090D 24GB×64**List price $157,963Market price $268,075–$382,964Cloud rental (live) ×64: $28.29–$33.92/hr (2026-11-05)
INT8
**≈476 GB VRAML20 48GB×16**List price $55,882–$56,202Market price $83,773
**Ascend 910B 64GB×8**List price n/aMarket price $131,644–$143,612
**RTX PRO 6000 Blackwell 96GB×8**List price $128,200Market price $140,691Cloud rental (live) ×8: $9.62–$18.15/hr (2026-11-05)
**RTX 5090 32GB×16**List price n/aMarket price $107,706Cloud rental (live) ×16: $7.07–$8.51/hr (2026-11-05)
**RTX 5090D 24GB×32**List price $78,982Market price $134,037–$191,482Cloud rental (live) ×32: $14.14–$16.96/hr (2026-11-05)
INT4 (AWQ/GPTQ)
**≈238 GB VRAML20 48GB×8**List price $27,941–$28,101Market price $41,887
**Ascend 910B 64GB×4**List price n/aMarket price $65,822–$71,806
**RTX PRO 6000 Blackwell 96GB×4**List price $64,100Market price $70,346Cloud rental (live) ×4: $4.81–$9.08/hr (2026-11-05)
**RTX 5090 32GB×8**List price n/aMarket price $53,853Cloud rental (live) ×8: $3.54–$4.26/hr (2026-11-05)
**RTX 5090D 24GB×16**List price $39,491Market price $67,019–$95,741Cloud rental (live) ×16: $7.07–$8.48/hr (2026-11-05)
[Full setup across 18 GPUs →](https://www.tandiu.cn/en/local-deploy-hf-nex-agi-nex-n2-pro) [Official repo →](https://huggingface.co/nex-agi/Nex-N2-Pro)

INFERENCE ENGINES

## Open-source inference enginesWhich software runs the model when you self-host · from each engine's official repo/docs, checked 2026-10-24

| Engine | By | Models | Hardware | Online serving | License | Notes |
|---|---|---|---|---|---|---|
| vLLM | Community (UC Berkeley origin) | Most open-weights models | NVIDIA / AMD / TPU / Intel etc.; Ascend via vllm-ascend plugin | Yes (OpenAI-compatible API) | Apache-2.0 | PagedAttention; de-facto default, new models usually supported on day one |
| SGLang | Community (LMSYS origin) | Most open-weights models | NVIDIA / AMD / Ascend / TPU etc. | Yes (OpenAI-compatible API) | Apache-2.0 | RadixAttention prefix cache; common for large MoE deployments |
| TensorRT-LLM | NVIDIA | Major open-weights models (per official support list) | NVIDIA only | Yes (trtllm-serve / Triton) | Apache-2.0 | NVIDIA-optimized; first to use new GPU features |
| MindIE | Huawei | Major open-weights models (per Ascend support list) | Ascend only | Yes (MindIE Service) | 华为软件许可 | Huawei's official Ascend inference suite |
| llama.cpp | Community | GGUF quantized models | CPU / NVIDIA / AMD / Apple etc., mostly consumer hardware | Yes (llama-server, single-node) | MIT | Low-bit quantization for PCs and single GPUs; Ollama and others build on it |
| DeepInfer · released 2026-10-20 | DeepSeek | DeepSeek V4.1 Flash only | NVIDIA SM100 (B200/B300) / Ascend 950 | No (offline inference only in this release) | Apache-2.0 | Powers DeepSeek's own production; KV cache system, prefill/decode disaggregation and load balancing not included |

**GPU prices**

List price, market price (SXM GPUs per 8-GPU node) and cloud rental for 18 GPUs: [GPU prices & cloud rental →](https://www.tandiu.cn/en/compute)

FAQ

## FAQ

**How is the VRAM requirement calculated?**

VRAM = total parameters (B) × bytes per parameter × 1.2 runtime headroom. It covers model weights only, not concurrency or long-context KV cache; leave another 20–30% and load-test before buying.

**How are GPU counts derived?**

GPU counts use reference cards (L20 48 GB, Ascend 910B 64 GB, RTX PRO 6000 96 GB, RTX 5090 32 GB, RTX 5090D 24 GB) as examples; any GPU of similar VRAM works. Every model page covers all GPUs.

**What do “active parameters” mean for MoE models?**

Active parameters set compute and speed but do not reduce weight VRAM, so sizing uses total parameters; “est.” marks active-parameter counts inferred from sibling releases.

**Where do the GPU prices come from?**

SXM data-center GPUs (B300/B200/H200/H100/H800) are quoted per 8-GPU HGX node, as the industry does; per-GPU equivalents are shown and sizing bills whole nodes. USD converted from CNY at ¥6.68, taken from public reports and re-searched weekly; they swing widely and are indicative only.

**More questions (1)What is “Cloud rental (live)”?**

It comes from the public vast.ai market API, refreshed daily, and does not reflect China-market purchase prices. TANDIU does not sell or resell hardware.