钽TANDIU钽铥
SELF-HOSTING

Self-Hosting LLMs: VRAM & GPU Sizing

VRAM, GPU count and speed for running open-weights models yourself, next to the vendor's API price and measured speed.

Data updated: 2026-11-20
SortPopularity = Hugging Face downloads of the official repo, last 30 days (2026-11-20)
Zhipu AI (GLM)5 models · 10.1M downloads (30d) · latest 2026-08-25

GLM-5.3-Flash

321B total · 18B active · mit
Popularity
6.4MHF downloads (30d) · 3,292 likes
Released
2026-08-25HF repo created
Public benchmarks
AA 28.0 · LiveBench 64.7
Vendor API
$0.15 / $0.50input / output per 1M tokens
FP16 VRAM
≈771 GBINT4 ≈193 GB
VRAM and reference setups (FP16 / INT8 / INT4)
FP16/BF16 (full precision)
≈771 GB VRAM
L20 48GB×32List price $111,857–$112,499Market price $167,687
Ascend 910B 64GB×16List price n/aMarket price $263,509–$287,464
RTX PRO 6000 Blackwell 96GB×16List price $256,614Market price $407,240Cloud rental (live) ×16: $21.15–$32.11/hr (2026-11-20)
RTX 5090 32GB×32List price $64,009Market price $229,971–$249,135Cloud rental (live) ×32: $14.59–$19.23/hr (2026-11-20)
RTX 5090D 24GB×64List price $158,096Market price $268,300–$383,285
INT8
≈386 GB VRAM
L20 48GB×16List price $55,928–$56,249Market price $83,844
Ascend 910B 64GB×8List price n/aMarket price $131,754–$143,732
RTX PRO 6000 Blackwell 96GB×8List price $128,307Market price $203,620Cloud rental (live) ×8: $10.58–$16.06/hr (2026-11-20)
RTX 5090 32GB×16List price $32,004Market price $114,986–$124,568Cloud rental (live) ×16: $7.30–$9.62/hr (2026-11-20)
RTX 5090D 24GB×32List price $79,048Market price $134,150–$191,643
INT4 (AWQ/GPTQ)
≈193 GB VRAM
L20 48GB×8List price $27,964–$28,125Market price $41,922
Ascend 910B 64GB×4List price n/aMarket price $65,877–$71,866
RTX PRO 6000 Blackwell 96GB×4List price $64,154Market price $101,810Cloud rental (live) ×4: $5.29–$8.03/hr (2026-11-20)
RTX 5090 32GB×8List price $16,002Market price $57,493–$62,284Cloud rental (live) ×8: $3.65–$4.81/hr (2026-11-20)
RTX 5090D 24GB×16List price $39,524Market price $67,075–$95,821
Full setup across 19 GPUs → Official repo →

GLM-5.3

753B total · 40B active (est.) · MIT License
Popularity
1.6MHF downloads (30d) · 2,499 likes
Released
2026-08-25HF repo created
Public benchmarks
AA 32.0 · LiveBench 0.0
Vendor API
$1.40 / $4.40input / output per 1M tokens
FP16 VRAM
≈1,808 GBINT4 ≈452 GB
VRAM and reference setups (FP16 / INT8 / INT4)
FP16/BF16 (full precision)
≈1,808 GB VRAM
L20 48GB×64List price $223,714–$224,998Market price $335,375
Ascend 910B 64GB×32List price n/aMarket price $527,017–$574,928
RTX PRO 6000 Blackwell 96GB×32List price $513,228Market price $814,481Cloud rental (live) ×32: $42.30–$64.22/hr (2026-11-20)
RTX 5090 32GB×64List price $128,017Market price $459,942–$498,271Cloud rental (live) ×64: $29.18–$38.46/hr (2026-11-20)
RTX 5090D 24GB×76List price $187,738Market price $318,606–$455,151
INT8
≈904 GB VRAM
L20 48GB×32List price $111,857–$112,499Market price $167,687
Ascend 910B 64GB×16List price n/aMarket price $263,509–$287,464
RTX PRO 6000 Blackwell 96GB×16List price $256,614Market price $407,240Cloud rental (live) ×16: $21.15–$32.11/hr (2026-11-20)
RTX 5090 32GB×32List price $64,009Market price $229,971–$249,135Cloud rental (live) ×32: $14.59–$19.23/hr (2026-11-20)
RTX 5090D 24GB×64List price $158,096Market price $268,300–$383,285
INT4 (AWQ/GPTQ)
≈452 GB VRAM
L20 48GB×16List price $55,928–$56,249Market price $83,844
Ascend 910B 64GB×8List price n/aMarket price $131,754–$143,732
RTX PRO 6000 Blackwell 96GB×8List price $128,307Market price $203,620Cloud rental (live) ×8: $10.58–$16.06/hr (2026-11-20)
RTX 5090 32GB×16List price $32,004Market price $114,986–$124,568Cloud rental (live) ×16: $7.30–$9.62/hr (2026-11-20)
RTX 5090D 24GB×32List price $79,048Market price $134,150–$191,643
Full setup across 19 GPUs → Official repo →

GLM-5

754B total · 40B active · mit
Popularity
1.1MHF downloads (30d) · 2,143 likes
Released
2026-02-11HF repo created
Public benchmarks
AA — · LiveBench 0.0
Vendor API
$1.00 / $3.20input / output per 1M tokens
FP16 VRAM
≈1,809 GBINT4 ≈452 GB
VRAM and reference setups (FP16 / INT8 / INT4)
FP16/BF16 (full precision)
≈1,809 GB VRAM
L20 48GB×64List price $223,714–$224,998Market price $335,375
Ascend 910B 64GB×32List price n/aMarket price $527,017–$574,928
RTX PRO 6000 Blackwell 96GB×32List price $513,228Market price $814,481Cloud rental (live) ×32: $42.30–$64.22/hr (2026-11-20)
RTX 5090 32GB×64List price $128,017Market price $459,942–$498,271Cloud rental (live) ×64: $29.18–$38.46/hr (2026-11-20)
RTX 5090D 24GB×76List price $187,738Market price $318,606–$455,151
INT8
≈905 GB VRAM
L20 48GB×32List price $111,857–$112,499Market price $167,687
Ascend 910B 64GB×16List price n/aMarket price $263,509–$287,464
RTX PRO 6000 Blackwell 96GB×16List price $256,614Market price $407,240Cloud rental (live) ×16: $21.15–$32.11/hr (2026-11-20)
RTX 5090 32GB×32List price $64,009Market price $229,971–$249,135Cloud rental (live) ×32: $14.59–$19.23/hr (2026-11-20)
RTX 5090D 24GB×64List price $158,096Market price $268,300–$383,285
INT4 (AWQ/GPTQ)
≈452 GB VRAM
L20 48GB×16List price $55,928–$56,249Market price $83,844
Ascend 910B 64GB×8List price n/aMarket price $131,754–$143,732
RTX PRO 6000 Blackwell 96GB×8List price $128,307Market price $203,620Cloud rental (live) ×8: $10.58–$16.06/hr (2026-11-20)
RTX 5090 32GB×16List price $32,004Market price $114,986–$124,568Cloud rental (live) ×16: $7.30–$9.62/hr (2026-11-20)
RTX 5090D 24GB×32List price $79,048Market price $134,150–$191,643
Full setup across 19 GPUs → Official repo →

GLM-5.1

754B total · 40B active · MIT License
Popularity
708.8KHF downloads (30d) · 1,850 likes
Released
2026-04-03HF repo created
Public benchmarks
AA — · LiveBench 0.0
Vendor API
$1.40 / $4.40input / output per 1M tokens
FP16 VRAM
≈1,809 GBINT4 ≈452 GB
VRAM and reference setups (FP16 / INT8 / INT4)
FP16/BF16 (full precision)
≈1,809 GB VRAM
L20 48GB×64List price $223,714–$224,998Market price $335,375
Ascend 910B 64GB×32List price n/aMarket price $527,017–$574,928
RTX PRO 6000 Blackwell 96GB×32List price $513,228Market price $814,481Cloud rental (live) ×32: $42.30–$64.22/hr (2026-11-20)
RTX 5090 32GB×64List price $128,017Market price $459,942–$498,271Cloud rental (live) ×64: $29.18–$38.46/hr (2026-11-20)
RTX 5090D 24GB×76List price $187,738Market price $318,606–$455,151
INT8
≈905 GB VRAM
L20 48GB×32List price $111,857–$112,499Market price $167,687
Ascend 910B 64GB×16List price n/aMarket price $263,509–$287,464
RTX PRO 6000 Blackwell 96GB×16List price $256,614Market price $407,240Cloud rental (live) ×16: $21.15–$32.11/hr (2026-11-20)
RTX 5090 32GB×32List price $64,009Market price $229,971–$249,135Cloud rental (live) ×32: $14.59–$19.23/hr (2026-11-20)
RTX 5090D 24GB×64List price $158,096Market price $268,300–$383,285
INT4 (AWQ/GPTQ)
≈452 GB VRAM
L20 48GB×16List price $55,928–$56,249Market price $83,844
Ascend 910B 64GB×8List price n/aMarket price $131,754–$143,732
RTX PRO 6000 Blackwell 96GB×8List price $128,307Market price $203,620Cloud rental (live) ×8: $10.58–$16.06/hr (2026-11-20)
RTX 5090 32GB×16List price $32,004Market price $114,986–$124,568Cloud rental (live) ×16: $7.30–$9.62/hr (2026-11-20)
RTX 5090D 24GB×32List price $79,048Market price $134,150–$191,643
Full setup across 19 GPUs → Official repo →

GLM-5.2

753B total · 39B active · MIT License
Popularity
310.9KHF downloads (30d) · 5,210 likes
Released
2026-06-16HF repo created
Public benchmarks
AA — · LiveBench 0.0
Vendor API
$1.40 / $4.40input / output per 1M tokens
FP16 VRAM
≈1,808 GBINT4 ≈452 GB
VRAM and reference setups (FP16 / INT8 / INT4)
FP16/BF16 (full precision)
≈1,808 GB VRAM
L20 48GB×64List price $223,714–$224,998Market price $335,375
Ascend 910B 64GB×32List price n/aMarket price $527,017–$574,928
RTX PRO 6000 Blackwell 96GB×32List price $513,228Market price $814,481Cloud rental (live) ×32: $42.30–$64.22/hr (2026-11-20)
RTX 5090 32GB×64List price $128,017Market price $459,942–$498,271Cloud rental (live) ×64: $29.18–$38.46/hr (2026-11-20)
RTX 5090D 24GB×76List price $187,738Market price $318,606–$455,151
INT8
≈904 GB VRAM
L20 48GB×32List price $111,857–$112,499Market price $167,687
Ascend 910B 64GB×16List price n/aMarket price $263,509–$287,464
RTX PRO 6000 Blackwell 96GB×16List price $256,614Market price $407,240Cloud rental (live) ×16: $21.15–$32.11/hr (2026-11-20)
RTX 5090 32GB×32List price $64,009Market price $229,971–$249,135Cloud rental (live) ×32: $14.59–$19.23/hr (2026-11-20)
RTX 5090D 24GB×64List price $158,096Market price $268,300–$383,285
INT4 (AWQ/GPTQ)
≈452 GB VRAM
L20 48GB×16List price $55,928–$56,249Market price $83,844
Ascend 910B 64GB×8List price n/aMarket price $131,754–$143,732
RTX PRO 6000 Blackwell 96GB×8List price $128,307Market price $203,620Cloud rental (live) ×8: $10.58–$16.06/hr (2026-11-20)
RTX 5090 32GB×16List price $32,004Market price $114,986–$124,568Cloud rental (live) ×16: $7.30–$9.62/hr (2026-11-20)
RTX 5090D 24GB×32List price $79,048Market price $134,150–$191,643
Full setup across 19 GPUs → Official repo →
Alibaba Qwen3 models · 9.5M downloads (30d) · latest 2026-08-24

Qwen3.8 27B

28B total · 28B active · apache-2.0
Popularity
6.6MHF downloads (30d) · 20,682 likes
Released
2026-08-05HF repo created
Public benchmarks
AA 20.0 · LiveBench 57.1
Vendor API
$0.50 / $3.00input / output per 1M tokens
FP16 VRAM
≈67 GBINT4 ≈17 GB
VRAM and reference setups (FP16 / INT8 / INT4)
FP16/BF16 (full precision)
≈67 GB VRAM
L20 48GB×2List price $6,991–$7,031Market price $10,480
Ascend 910B 64GB×2List price n/aMarket price $32,939–$35,933
RTX PRO 6000 Blackwell 96GB×1List price $16,038Market price $25,453Cloud rental (live): $1.32–$2.01/hr (2026-11-20)
RTX 5090 32GB×4List price $8,001Market price $28,746–$31,142Cloud rental (live) ×4: $1.82–$2.40/hr (2026-11-20)
RTX 5090D 24GB×4List price $9,881Market price $16,769–$23,955
INT8
≈33 GB VRAM
L20 48GB×1List price $3,496–$3,516Market price $5,240
Ascend 910B 64GB×1List price n/aMarket price $16,469–$17,966
RTX PRO 6000 Blackwell 96GB×1List price $16,038Market price $25,453Cloud rental (live): $1.32–$2.01/hr (2026-11-20)
RTX 5090 32GB×2List price $4,001Market price $14,373–$15,571Cloud rental (live) ×2: $0.91–$1.20/hr (2026-11-20)
RTX 5090D 24GB×2List price $4,940Market price $8,384–$11,978
INT4 (AWQ/GPTQ)
≈17 GB VRAM
L20 48GB×1List price $3,496–$3,516Market price $5,240
Ascend 910B 64GB×1List price n/aMarket price $16,469–$17,966
RTX PRO 6000 Blackwell 96GB×1List price $16,038Market price $25,453Cloud rental (live): $1.32–$2.01/hr (2026-11-20)
RTX 5090 32GB×1List price $2,000Market price $7,187–$7,785Cloud rental (live): $0.46–$0.60/hr (2026-11-20)
RTX 5090D 24GB×1List price $2,470Market price $4,192–$5,989
Full setup across 19 GPUs → Official repo →

Qwen3.8-Flash-Next

180B total · 6B active · other
Popularity
2.9MHF downloads (30d) · 7,401 likes
Released
2026-08-24HF repo created
Public benchmarks
No public benchmark yet
Vendor API
—input / output per 1M tokens
FP16 VRAM
≈432 GBINT4 ≈108 GB
VRAM and reference setups (FP16 / INT8 / INT4)
FP16/BF16 (full precision)
≈432 GB VRAM
L20 48GB×16List price $55,928–$56,249Market price $83,844
Ascend 910B 64GB×8List price n/aMarket price $131,754–$143,732
RTX PRO 6000 Blackwell 96GB×8List price $128,307Market price $203,620Cloud rental (live) ×8: $10.58–$16.06/hr (2026-11-20)
RTX 5090 32GB×16List price $32,004Market price $114,986–$124,568Cloud rental (live) ×16: $7.30–$9.62/hr (2026-11-20)
RTX 5090D 24GB×32List price $79,048Market price $134,150–$191,643
INT8
≈216 GB VRAM
L20 48GB×8List price $27,964–$28,125Market price $41,922
Ascend 910B 64GB×4List price n/aMarket price $65,877–$71,866
RTX PRO 6000 Blackwell 96GB×4List price $64,154Market price $101,810Cloud rental (live) ×4: $5.29–$8.03/hr (2026-11-20)
RTX 5090 32GB×8List price $16,002Market price $57,493–$62,284Cloud rental (live) ×8: $3.65–$4.81/hr (2026-11-20)
RTX 5090D 24GB×16List price $39,524Market price $67,075–$95,821
INT4 (AWQ/GPTQ)
≈108 GB VRAM
L20 48GB×4List price $13,982–$14,062Market price $20,961
Ascend 910B 64GB×2List price n/aMarket price $32,939–$35,933
RTX PRO 6000 Blackwell 96GB×2List price $32,077Market price $50,905Cloud rental (live) ×2: $2.64–$4.01/hr (2026-11-20)
RTX 5090 32GB×4List price $8,001Market price $28,746–$31,142Cloud rental (live) ×4: $1.82–$2.40/hr (2026-11-20)
RTX 5090D 24GB×8List price $19,762Market price $33,537–$47,911
Full setup across 19 GPUs → Official repo →

Qwen3.8 2.4T A95B

2,446B total · 95B active · Qwen custom open-source license agreement (HF labeled license:other, non-standard OSI protocol, specific terms must be reviewed before use)
Popularity
29.6KHF downloads (30d) · 1,404 likes
Released
2026-08-08HF repo created
Public benchmarks
No public benchmark yet
Vendor API
$2.00 / $6.00input / output per 1M tokens
FP16 VRAM
≈5,871 GBINT4 ≈1,468 GB
VRAM and reference setups (FP16 / INT8 / INT4)
FP16/BF16 (full precision)
≈5,871 GB VRAM
L20 48GB×123List price $429,950–$432,418Market price $644,548
Ascend 910B 64GB×92List price n/aMarket price $1,515,174–$1,652,917
RTX PRO 6000 Blackwell 96GB×64List price $1,026,457Market price $1,628,962Cloud rental (live) ×64: $84.61–$128.45/hr (2026-11-20)
RTX 5090 32GB×184List price $368,050Market price $1,322,334–$1,432,528Cloud rental (live) ×184: $83.90–$110.58/hr (2026-11-20)
RTX 5090D 24GB×245List price $605,210Market price $1,027,084–$1,467,264
INT8
≈2,935 GB VRAM
L20 48GB×64List price $223,714–$224,998Market price $335,375
Ascend 910B 64GB×64List price n/aMarket price $1,054,034–$1,149,856
RTX PRO 6000 Blackwell 96GB×32List price $513,228Market price $814,481Cloud rental (live) ×32: $42.30–$64.22/hr (2026-11-20)
RTX 5090 32GB×92List price $184,025Market price $661,167–$716,264Cloud rental (live) ×92: $41.95–$55.29/hr (2026-11-20)
RTX 5090D 24GB×123List price $303,840Market price $515,638–$736,626
INT4 (AWQ/GPTQ)
≈1,468 GB VRAM
L20 48GB×32List price $111,857–$112,499Market price $167,687
Ascend 910B 64GB×32List price n/aMarket price $527,017–$574,928
RTX PRO 6000 Blackwell 96GB×16List price $256,614Market price $407,240Cloud rental (live) ×16: $21.15–$32.11/hr (2026-11-20)
RTX 5090 32GB×64List price $128,017Market price $459,942–$498,271Cloud rental (live) ×64: $29.18–$38.46/hr (2026-11-20)
RTX 5090D 24GB×64List price $158,096Market price $268,300–$383,285
Full setup across 19 GPUs → Official repo →
DeepSeek2 models · 1.8M downloads (30d) · latest 2026-09-10

DeepSeek V4.1 Flash

763B total · 16B active · MIT
Popularity
1.5MHF downloads (30d) · 5,038 likes
Released
2026-09-10HF repo created
Public benchmarks
AA 33.0 · LiveBench 73.3
Vendor API
$0.30 / $1.20input / output per 1M tokens
FP16 VRAM
≈1,832 GBINT4 ≈458 GB
VRAM and reference setups (FP16 / INT8 / INT4)
FP16/BF16 (full precision)
≈1,832 GB VRAM
L20 48GB×64List price $223,714–$224,998Market price $335,375
Ascend 910B 64GB×32List price n/aMarket price $527,017–$574,928
RTX PRO 6000 Blackwell 96GB×32List price $513,228Market price $814,481Cloud rental (live) ×32: $42.30–$64.22/hr (2026-11-20)
RTX 5090 32GB×64List price $128,017Market price $459,942–$498,271Cloud rental (live) ×64: $29.18–$38.46/hr (2026-11-20)
RTX 5090D 24GB×77List price $190,209Market price $322,798–$461,140
INT8
≈916 GB VRAM
L20 48GB×32List price $111,857–$112,499Market price $167,687
Ascend 910B 64GB×16List price n/aMarket price $263,509–$287,464
RTX PRO 6000 Blackwell 96GB×16List price $256,614Market price $407,240Cloud rental (live) ×16: $21.15–$32.11/hr (2026-11-20)
RTX 5090 32GB×32List price $64,009Market price $229,971–$249,135Cloud rental (live) ×32: $14.59–$19.23/hr (2026-11-20)
RTX 5090D 24GB×64List price $158,096Market price $268,300–$383,285
INT4 (AWQ/GPTQ)
≈458 GB VRAM
L20 48GB×16List price $55,928–$56,249Market price $83,844
Ascend 910B 64GB×8List price n/aMarket price $131,754–$143,732
RTX PRO 6000 Blackwell 96GB×8List price $128,307Market price $203,620Cloud rental (live) ×8: $10.58–$16.06/hr (2026-11-20)
RTX 5090 32GB×16List price $32,004Market price $114,986–$124,568Cloud rental (live) ×16: $7.30–$9.62/hr (2026-11-20)
RTX 5090D 24GB×32List price $79,048Market price $134,150–$191,643
Full setup across 19 GPUs → Official repo →

DeepSeek V4 Pro

1,599B total · 49B active · DeepSeek open-source license agreement
Popularity
321.4KHF downloads (30d) · 5,723 likes
Released
2026-04-22HF repo created
Public benchmarks
AA 23.0 · LiveBench 0.0
Vendor API
$1.32 / $3.96input / output per 1M tokens
FP16 VRAM
≈3,837 GBINT4 ≈959 GB
VRAM and reference setups (FP16 / INT8 / INT4)
FP16/BF16 (full precision)
≈3,837 GB VRAM
L20 48GB×80List price $279,642–$281,247Market price $419,218
Ascend 910B 64GB×64List price n/aMarket price $1,054,034–$1,149,856
RTX PRO 6000 Blackwell 96GB×64List price $1,026,457Market price $1,628,962Cloud rental (live) ×64: $84.61–$128.45/hr (2026-11-20)
RTX 5090 32GB×120List price $240,032Market price $862,392–$934,258Cloud rental (live) ×120: $54.72–$72.12/hr (2026-11-20)
RTX 5090D 24GB×160List price $395,239Market price $670,749–$958,213
INT8
≈1,919 GB VRAM
L20 48GB×64List price $223,714–$224,998Market price $335,375
Ascend 910B 64GB×32List price n/aMarket price $527,017–$574,928
RTX PRO 6000 Blackwell 96GB×32List price $513,228Market price $814,481Cloud rental (live) ×32: $42.30–$64.22/hr (2026-11-20)
RTX 5090 32GB×64List price $128,017Market price $459,942–$498,271Cloud rental (live) ×64: $29.18–$38.46/hr (2026-11-20)
RTX 5090D 24GB×80List price $197,619Market price $335,375–$479,106
INT4 (AWQ/GPTQ)
≈959 GB VRAM
L20 48GB×32List price $111,857–$112,499Market price $167,687
Ascend 910B 64GB×16List price n/aMarket price $263,509–$287,464
RTX PRO 6000 Blackwell 96GB×16List price $256,614Market price $407,240Cloud rental (live) ×16: $21.15–$32.11/hr (2026-11-20)
RTX 5090 32GB×32List price $64,009Market price $229,971–$249,135Cloud rental (live) ×32: $14.59–$19.23/hr (2026-11-20)
RTX 5090D 24GB×64List price $158,096Market price $268,300–$383,285
Full setup across 19 GPUs → Official repo →
Moonshot AI (Kimi)5 models · 1.7M downloads (30d) · latest 2026-11-02

Kimi K3

2,780B total · 104B active · Modified MIT License
Popularity
734.2KHF downloads (30d) · 12,006 likes
Released
2026-06-13HF repo created
Public benchmarks
AA — · LiveBench 69.4
Vendor API
$3.00 / $15.00input / output per 1M tokens
FP16 VRAM
≈6,672 GBINT4 ≈1,668 GB
VRAM and reference setups (FP16 / INT8 / INT4)
FP16/BF16 (full precision)
≈6,672 GB VRAM
L20 48GB×139List price $485,879–$488,667Market price $728,392
Ascend 910B 64GB×105List price n/aMarket price $1,729,275–$1,886,482
RTX PRO 6000 Blackwell 96GB×70List price $1,122,687Market price $1,781,677Cloud rental (live) ×70: $92.54–$140.49/hr (2026-11-20)
RTX 5090 32GB×209List price $418,056Market price $1,501,999–$1,627,165Cloud rental (live) ×209: $95.30–$125.61/hr (2026-11-20)
RTX 5090D 24GB×278List price $686,728Market price $1,165,426–$1,664,895
INT8
≈3,336 GB VRAM
L20 48GB×70List price $244,687–$246,092Market price $366,816
Ascend 910B 64GB×64List price n/aMarket price $1,054,034–$1,149,856
RTX PRO 6000 Blackwell 96GB×64List price $1,026,457Market price $1,628,962Cloud rental (live) ×64: $84.61–$128.45/hr (2026-11-20)
RTX 5090 32GB×105List price $210,028Market price $754,593–$817,475Cloud rental (live) ×105: $47.88–$63.10/hr (2026-11-20)
RTX 5090D 24GB×139List price $343,364Market price $582,713–$832,447
INT4 (AWQ/GPTQ)
≈1,668 GB VRAM
L20 48GB×64List price $223,714–$224,998Market price $335,375
Ascend 910B 64GB×32List price n/aMarket price $527,017–$574,928
RTX PRO 6000 Blackwell 96GB×32List price $513,228Market price $814,481Cloud rental (live) ×32: $42.30–$64.22/hr (2026-11-20)
RTX 5090 32GB×64List price $128,017Market price $459,942–$498,271Cloud rental (live) ×64: $29.18–$38.46/hr (2026-11-20)
RTX 5090D 24GB×70List price $172,917Market price $293,453–$419,218
Full setup across 19 GPUs → Official repo →

Kimi K2.5

1,027B total · 32B active · Open-source weight license
Popularity
391.3KHF downloads (30d) · 2,902 likes
Released
2026-01-01HF repo created
Public benchmarks
AA — · LiveBench 0.0
Vendor API
$0.60 / $3.00input / output per 1M tokens
FP16 VRAM
≈2,465 GBINT4 ≈616 GB
VRAM and reference setups (FP16 / INT8 / INT4)
FP16/BF16 (full precision)
≈2,465 GB VRAM
L20 48GB×64List price $223,714–$224,998Market price $335,375
Ascend 910B 64GB×64List price n/aMarket price $1,054,034–$1,149,856
RTX PRO 6000 Blackwell 96GB×32List price $513,228Market price $814,481Cloud rental (live) ×32: $42.30–$64.22/hr (2026-11-20)
RTX 5090 32GB×78List price $156,021Market price $560,555–$607,267Cloud rental (live) ×78: $35.57–$46.88/hr (2026-11-20)
RTX 5090D 24GB×103List price $254,435Market price $431,795–$616,850
INT8
≈1,232 GB VRAM
L20 48GB×32List price $111,857–$112,499Market price $167,687
Ascend 910B 64GB×32List price n/aMarket price $527,017–$574,928
RTX PRO 6000 Blackwell 96GB×16List price $256,614Market price $407,240Cloud rental (live) ×16: $21.15–$32.11/hr (2026-11-20)
RTX 5090 32GB×64List price $128,017Market price $459,942–$498,271Cloud rental (live) ×64: $29.18–$38.46/hr (2026-11-20)
RTX 5090D 24GB×64List price $158,096Market price $268,300–$383,285
INT4 (AWQ/GPTQ)
≈616 GB VRAM
L20 48GB×16List price $55,928–$56,249Market price $83,844
Ascend 910B 64GB×16List price n/aMarket price $263,509–$287,464
RTX PRO 6000 Blackwell 96GB×8List price $128,307Market price $203,620Cloud rental (live) ×8: $10.58–$16.06/hr (2026-11-20)
RTX 5090 32GB×32List price $64,009Market price $229,971–$249,135Cloud rental (live) ×32: $14.59–$19.23/hr (2026-11-20)
RTX 5090D 24GB×32List price $79,048Market price $134,150–$191,643
Full setup across 19 GPUs → Official repo →

Kimi K2.7 Code

1,027B total · 32B active · Modified MIT License
Popularity
322.7KHF downloads (30d) · 1,431 likes
Released
2026-06-11HF repo created
Public benchmarks
AA — · LiveBench 0.0
Vendor API
$0.95 / $4.00input / output per 1M tokens
FP16 VRAM
≈2,465 GBINT4 ≈616 GB
VRAM and reference setups (FP16 / INT8 / INT4)
FP16/BF16 (full precision)
≈2,465 GB VRAM
L20 48GB×64List price $223,714–$224,998Market price $335,375
Ascend 910B 64GB×64List price n/aMarket price $1,054,034–$1,149,856
RTX PRO 6000 Blackwell 96GB×32List price $513,228Market price $814,481Cloud rental (live) ×32: $42.30–$64.22/hr (2026-11-20)
RTX 5090 32GB×78List price $156,021Market price $560,555–$607,267Cloud rental (live) ×78: $35.57–$46.88/hr (2026-11-20)
RTX 5090D 24GB×103List price $254,435Market price $431,795–$616,850
INT8
≈1,232 GB VRAM
L20 48GB×32List price $111,857–$112,499Market price $167,687
Ascend 910B 64GB×32List price n/aMarket price $527,017–$574,928
RTX PRO 6000 Blackwell 96GB×16List price $256,614Market price $407,240Cloud rental (live) ×16: $21.15–$32.11/hr (2026-11-20)
RTX 5090 32GB×64List price $128,017Market price $459,942–$498,271Cloud rental (live) ×64: $29.18–$38.46/hr (2026-11-20)
RTX 5090D 24GB×64List price $158,096Market price $268,300–$383,285
INT4 (AWQ/GPTQ)
≈616 GB VRAM
L20 48GB×16List price $55,928–$56,249Market price $83,844
Ascend 910B 64GB×16List price n/aMarket price $263,509–$287,464
RTX PRO 6000 Blackwell 96GB×8List price $128,307Market price $203,620Cloud rental (live) ×8: $10.58–$16.06/hr (2026-11-20)
RTX 5090 32GB×32List price $64,009Market price $229,971–$249,135Cloud rental (live) ×32: $14.59–$19.23/hr (2026-11-20)
RTX 5090D 24GB×32List price $79,048Market price $134,150–$191,643
Full setup across 19 GPUs → Official repo →

Kimi K2.6

1,027B total · 32B active · Modified MIT License
Popularity
199.0KHF downloads (30d) · 1,646 likes
Released
2026-04-14HF repo created
Public benchmarks
AA — · LiveBench 0.0
Vendor API
$0.95 / $4.00input / output per 1M tokens
FP16 VRAM
≈2,465 GBINT4 ≈616 GB
VRAM and reference setups (FP16 / INT8 / INT4)
FP16/BF16 (full precision)
≈2,465 GB VRAM
L20 48GB×64List price $223,714–$224,998Market price $335,375
Ascend 910B 64GB×64List price n/aMarket price $1,054,034–$1,149,856
RTX PRO 6000 Blackwell 96GB×32List price $513,228Market price $814,481Cloud rental (live) ×32: $42.30–$64.22/hr (2026-11-20)
RTX 5090 32GB×78List price $156,021Market price $560,555–$607,267Cloud rental (live) ×78: $35.57–$46.88/hr (2026-11-20)
RTX 5090D 24GB×103List price $254,435Market price $431,795–$616,850
INT8
≈1,232 GB VRAM
L20 48GB×32List price $111,857–$112,499Market price $167,687
Ascend 910B 64GB×32List price n/aMarket price $527,017–$574,928
RTX PRO 6000 Blackwell 96GB×16List price $256,614Market price $407,240Cloud rental (live) ×16: $21.15–$32.11/hr (2026-11-20)
RTX 5090 32GB×64List price $128,017Market price $459,942–$498,271Cloud rental (live) ×64: $29.18–$38.46/hr (2026-11-20)
RTX 5090D 24GB×64List price $158,096Market price $268,300–$383,285
INT4 (AWQ/GPTQ)
≈616 GB VRAM
L20 48GB×16List price $55,928–$56,249Market price $83,844
Ascend 910B 64GB×16List price n/aMarket price $263,509–$287,464
RTX PRO 6000 Blackwell 96GB×8List price $128,307Market price $203,620Cloud rental (live) ×8: $10.58–$16.06/hr (2026-11-20)
RTX 5090 32GB×32List price $64,009Market price $229,971–$249,135Cloud rental (live) ×32: $14.59–$19.23/hr (2026-11-20)
RTX 5090D 24GB×32List price $79,048Market price $134,150–$191,643
Full setup across 19 GPUs → Official repo →

Kimi K3.1 Pro

2,780B total · 104B active · other
Popularity
60.9KHF downloads (30d) · 2,334 likes
Released
2026-11-02HF repo created
Public benchmarks
AA 46.0 · LiveBench 72.6
Vendor API
$2.00 / $10.00input / output per 1M tokens
FP16 VRAM
≈6,672 GBINT4 ≈1,668 GB
VRAM and reference setups (FP16 / INT8 / INT4)
FP16/BF16 (full precision)
≈6,672 GB VRAM
L20 48GB×140List price $489,374–$492,183Market price $733,632
Ascend 910B 64GB×105List price n/aMarket price $1,729,275–$1,886,482
RTX PRO 6000 Blackwell 96GB×70List price $1,122,687Market price $1,781,677Cloud rental (live) ×70: $92.54–$140.49/hr (2026-11-20)
RTX 5090 32GB×209List price $418,056Market price $1,501,999–$1,627,165Cloud rental (live) ×209: $95.30–$125.61/hr (2026-11-20)
RTX 5090D 24GB×279List price $689,198Market price $1,169,619–$1,670,884
INT8
≈3,336 GB VRAM
L20 48GB×70List price $244,687–$246,092Market price $366,816
Ascend 910B 64GB×64List price n/aMarket price $1,054,034–$1,149,856
RTX PRO 6000 Blackwell 96GB×64List price $1,026,457Market price $1,628,962Cloud rental (live) ×64: $84.61–$128.45/hr (2026-11-20)
RTX 5090 32GB×105List price $210,028Market price $754,593–$817,475Cloud rental (live) ×105: $47.88–$63.10/hr (2026-11-20)
RTX 5090D 24GB×140List price $345,834Market price $586,905–$838,436
INT4 (AWQ/GPTQ)
≈1,668 GB VRAM
L20 48GB×64List price $223,714–$224,998Market price $335,375
Ascend 910B 64GB×32List price n/aMarket price $527,017–$574,928
RTX PRO 6000 Blackwell 96GB×32List price $513,228Market price $814,481Cloud rental (live) ×32: $42.30–$64.22/hr (2026-11-20)
RTX 5090 32GB×64List price $128,017Market price $459,942–$498,271Cloud rental (live) ×64: $29.18–$38.46/hr (2026-11-20)
RTX 5090D 24GB×70List price $172,917Market price $293,453–$419,218
Full setup across 19 GPUs → Official repo →
MiniMax2 models · 214.5K downloads (30d) · latest 2026-10-17

MiniMax M3.1 Flash

220B total · 23B active · other
Popularity
110.4KHF downloads (30d) · 608 likes
Released
2026-10-17HF repo created
Public benchmarks
AA 29.0 · LiveBench 68.2
Vendor API
$0.15 / $0.60input / output per 1M tokens
FP16 VRAM
≈529 GBINT4 ≈132 GB
VRAM and reference setups (FP16 / INT8 / INT4)
FP16/BF16 (full precision)
≈529 GB VRAM
L20 48GB×16List price $55,928–$56,249Market price $83,844
Ascend 910B 64GB×16List price n/aMarket price $263,509–$287,464
RTX PRO 6000 Blackwell 96GB×8List price $128,307Market price $203,620Cloud rental (live) ×8: $10.58–$16.06/hr (2026-11-20)
RTX 5090 32GB×32List price $64,009Market price $229,971–$249,135Cloud rental (live) ×32: $14.59–$19.23/hr (2026-11-20)
RTX 5090D 24GB×32List price $79,048Market price $134,150–$191,643
INT8
≈265 GB VRAM
L20 48GB×8List price $27,964–$28,125Market price $41,922
Ascend 910B 64GB×8List price n/aMarket price $131,754–$143,732
RTX PRO 6000 Blackwell 96GB×4List price $64,154Market price $101,810Cloud rental (live) ×4: $5.29–$8.03/hr (2026-11-20)
RTX 5090 32GB×16List price $32,004Market price $114,986–$124,568Cloud rental (live) ×16: $7.30–$9.62/hr (2026-11-20)
RTX 5090D 24GB×16List price $39,524Market price $67,075–$95,821
INT4 (AWQ/GPTQ)
≈132 GB VRAM
L20 48GB×4List price $13,982–$14,062Market price $20,961
Ascend 910B 64GB×4List price n/aMarket price $65,877–$71,866
RTX PRO 6000 Blackwell 96GB×2List price $32,077Market price $50,905Cloud rental (live) ×2: $2.64–$4.01/hr (2026-11-20)
RTX 5090 32GB×8List price $16,002Market price $57,493–$62,284Cloud rental (live) ×8: $3.65–$4.81/hr (2026-11-20)
RTX 5090D 24GB×8List price $19,762Market price $33,537–$47,911
Full setup across 19 GPUs → Official repo →

MiniMax M3

427B total · 23B active · MIT License
Popularity
104.2KHF downloads (30d) · 1,607 likes
Released
2026-06-02HF repo created
Public benchmarks
AA 19.0 · LiveBench 0.0
Vendor API
$0.30 / $1.20input / output per 1M tokens
FP16 VRAM
≈1,025 GBINT4 ≈256 GB
VRAM and reference setups (FP16 / INT8 / INT4)
FP16/BF16 (full precision)
≈1,025 GB VRAM
L20 48GB×32List price $111,857–$112,499Market price $167,687
Ascend 910B 64GB×32List price n/aMarket price $527,017–$574,928
RTX PRO 6000 Blackwell 96GB×16List price $256,614Market price $407,240Cloud rental (live) ×16: $21.15–$32.11/hr (2026-11-20)
RTX 5090 32GB×64List price $128,017Market price $459,942–$498,271Cloud rental (live) ×64: $29.18–$38.46/hr (2026-11-20)
RTX 5090D 24GB×64List price $158,096Market price $268,300–$383,285
INT8
≈512 GB VRAM
L20 48GB×16List price $55,928–$56,249Market price $83,844
Ascend 910B 64GB×16List price n/aMarket price $263,509–$287,464
RTX PRO 6000 Blackwell 96GB×8List price $128,307Market price $203,620Cloud rental (live) ×8: $10.58–$16.06/hr (2026-11-20)
RTX 5090 32GB×32List price $64,009Market price $229,971–$249,135Cloud rental (live) ×32: $14.59–$19.23/hr (2026-11-20)
RTX 5090D 24GB×32List price $79,048Market price $134,150–$191,643
INT4 (AWQ/GPTQ)
≈256 GB VRAM
L20 48GB×8List price $27,964–$28,125Market price $41,922
Ascend 910B 64GB×8List price n/aMarket price $131,754–$143,732
RTX PRO 6000 Blackwell 96GB×4List price $64,154Market price $101,810Cloud rental (live) ×4: $5.29–$8.03/hr (2026-11-20)
RTX 5090 32GB×16List price $32,004Market price $114,986–$124,568Cloud rental (live) ×16: $7.30–$9.62/hr (2026-11-20)
RTX 5090D 24GB×16List price $39,524Market price $67,075–$95,821
Full setup across 19 GPUs → Official repo →
Tencent Hunyuan1 model · 76.1K downloads (30d) · latest 2025-06-25

Hunyuan A13B

80B total · 13B active · Tencent Hunyuan Open Source License Agreement
Popularity
76.1KHF downloads (30d) · 794 likes
Released
2025-06-25HF repo created
Public benchmarks
No public benchmark yet
Vendor API
$0.07 / $0.30input / output per 1M tokens
FP16 VRAM
≈192 GBINT4 ≈48 GB
VRAM and reference setups (FP16 / INT8 / INT4)
FP16/BF16 (full precision)
≈192 GB VRAM
L20 48GB×4List price $13,982–$14,062Market price $20,961
Ascend 910B 64GB×4List price n/aMarket price $65,877–$71,866
RTX PRO 6000 Blackwell 96GB×2List price $32,077Market price $50,905Cloud rental (live) ×2: $2.64–$4.01/hr (2026-11-20)
RTX 5090 32GB×8List price $16,002Market price $57,493–$62,284Cloud rental (live) ×8: $3.65–$4.81/hr (2026-11-20)
RTX 5090D 24GB×8List price $19,762Market price $33,537–$47,911
INT8
≈96 GB VRAM
L20 48GB×2List price $6,991–$7,031Market price $10,480
Ascend 910B 64GB×2List price n/aMarket price $32,939–$35,933
RTX PRO 6000 Blackwell 96GB×1List price $16,038Market price $25,453Cloud rental (live): $1.32–$2.01/hr (2026-11-20)
RTX 5090 32GB×4List price $8,001Market price $28,746–$31,142Cloud rental (live) ×4: $1.82–$2.40/hr (2026-11-20)
RTX 5090D 24GB×4List price $9,881Market price $16,769–$23,955
INT4 (AWQ/GPTQ)
≈48 GB VRAM
L20 48GB×1List price $3,496–$3,516Market price $5,240
Ascend 910B 64GB×1List price n/aMarket price $16,469–$17,966
RTX PRO 6000 Blackwell 96GB×1List price $16,038Market price $25,453Cloud rental (live): $1.32–$2.01/hr (2026-11-20)
RTX 5090 32GB×2List price $4,001Market price $14,373–$15,571Cloud rental (live) ×2: $0.91–$1.20/hr (2026-11-20)
RTX 5090D 24GB×2List price $4,940Market price $8,384–$11,978
Full setup across 19 GPUs → Official repo →
IFM(MBZUAI)1 model · 10.9K downloads (30d) · latest 2026-09-01

K2 Horizon 375B A23B

379B total · 23B active · apache-2.0
Popularity
10.9KHF downloads (30d) · 122 likes
Released
2026-09-01HF repo created
Public benchmarks
No public benchmark yet
Vendor API
—input / output per 1M tokens
FP16 VRAM
≈910 GBINT4 ≈228 GB
VRAM and reference setups (FP16 / INT8 / INT4)
FP16/BF16 (full precision)
≈910 GB VRAM
L20 48GB×32List price $111,857–$112,499Market price $167,687
Ascend 910B 64GB×16List price n/aMarket price $263,509–$287,464
RTX PRO 6000 Blackwell 96GB×16List price $256,614Market price $407,240Cloud rental (live) ×16: $21.15–$32.11/hr (2026-11-20)
RTX 5090 32GB×32List price $64,009Market price $229,971–$249,135Cloud rental (live) ×32: $14.59–$19.23/hr (2026-11-20)
RTX 5090D 24GB×64List price $158,096Market price $268,300–$383,285
INT8
≈455 GB VRAM
L20 48GB×16List price $55,928–$56,249Market price $83,844
Ascend 910B 64GB×8List price n/aMarket price $131,754–$143,732
RTX PRO 6000 Blackwell 96GB×8List price $128,307Market price $203,620Cloud rental (live) ×8: $10.58–$16.06/hr (2026-11-20)
RTX 5090 32GB×16List price $32,004Market price $114,986–$124,568Cloud rental (live) ×16: $7.30–$9.62/hr (2026-11-20)
RTX 5090D 24GB×32List price $79,048Market price $134,150–$191,643
INT4 (AWQ/GPTQ)
≈228 GB VRAM
L20 48GB×8List price $27,964–$28,125Market price $41,922
Ascend 910B 64GB×4List price n/aMarket price $65,877–$71,866
RTX PRO 6000 Blackwell 96GB×4List price $64,154Market price $101,810Cloud rental (live) ×4: $5.29–$8.03/hr (2026-11-20)
RTX 5090 32GB×8List price $16,002Market price $57,493–$62,284Cloud rental (live) ×8: $3.65–$4.81/hr (2026-11-20)
RTX 5090D 24GB×16List price $39,524Market price $67,075–$95,821
Full setup across 19 GPUs → Official repo →
阶跃星辰1 model · 9.4K downloads (30d) · latest 2026-09-20

Step 5 Preview

604B total · 27B active · other
Popularity
9.4KHF downloads (30d) · 146 likes
Released
2026-09-20HF repo created
Public benchmarks
AA 29.0 · LiveBench 57.9
Vendor API
$1.05 / $2.99input / output per 1M tokens
FP16 VRAM
≈1,450 GBINT4 ≈363 GB
VRAM and reference setups (FP16 / INT8 / INT4)
FP16/BF16 (full precision)
≈1,450 GB VRAM
L20 48GB×32List price $111,857–$112,499Market price $167,687
Ascend 910B 64GB×32List price n/aMarket price $527,017–$574,928
RTX PRO 6000 Blackwell 96GB×16List price $256,614Market price $407,240Cloud rental (live) ×16: $21.15–$32.11/hr (2026-11-20)
RTX 5090 32GB×64List price $128,017Market price $459,942–$498,271Cloud rental (live) ×64: $29.18–$38.46/hr (2026-11-20)
RTX 5090D 24GB×64List price $158,096Market price $268,300–$383,285
INT8
≈725 GB VRAM
L20 48GB×16List price $55,928–$56,249Market price $83,844
Ascend 910B 64GB×16List price n/aMarket price $263,509–$287,464
RTX PRO 6000 Blackwell 96GB×8List price $128,307Market price $203,620Cloud rental (live) ×8: $10.58–$16.06/hr (2026-11-20)
RTX 5090 32GB×32List price $64,009Market price $229,971–$249,135Cloud rental (live) ×32: $14.59–$19.23/hr (2026-11-20)
RTX 5090D 24GB×32List price $79,048Market price $134,150–$191,643
INT4 (AWQ/GPTQ)
≈363 GB VRAM
L20 48GB×8List price $27,964–$28,125Market price $41,922
Ascend 910B 64GB×8List price n/aMarket price $131,754–$143,732
RTX PRO 6000 Blackwell 96GB×4List price $64,154Market price $101,810Cloud rental (live) ×4: $5.29–$8.03/hr (2026-11-20)
RTX 5090 32GB×16List price $32,004Market price $114,986–$124,568Cloud rental (live) ×16: $7.30–$9.62/hr (2026-11-20)
RTX 5090D 24GB×16List price $39,524Market price $67,075–$95,821
Full setup across 19 GPUs → Official repo →
Motif Technologies1 model · 2.7K downloads (30d) · latest 2026-08-07

Motif 3

315B total · 13B active · mit
Popularity
2.7KHF downloads (30d) · 156 likes
Released
2026-08-07HF repo created
Public benchmarks
No public benchmark yet
Vendor API
—input / output per 1M tokens
FP16 VRAM
≈756 GBINT4 ≈189 GB
VRAM and reference setups (FP16 / INT8 / INT4)
FP16/BF16 (full precision)
≈756 GB VRAM
L20 48GB×16List price $55,928–$56,249Market price $83,844
Ascend 910B 64GB×16List price n/aMarket price $263,509–$287,464
RTX PRO 6000 Blackwell 96GB×8List price $128,307Market price $203,620Cloud rental (live) ×8: $10.58–$16.06/hr (2026-11-20)
RTX 5090 32GB×32List price $64,009Market price $229,971–$249,135Cloud rental (live) ×32: $14.59–$19.23/hr (2026-11-20)
RTX 5090D 24GB×32List price $79,048Market price $134,150–$191,643
INT8
≈378 GB VRAM
L20 48GB×8List price $27,964–$28,125Market price $41,922
Ascend 910B 64GB×8List price n/aMarket price $131,754–$143,732
RTX PRO 6000 Blackwell 96GB×4List price $64,154Market price $101,810Cloud rental (live) ×4: $5.29–$8.03/hr (2026-11-20)
RTX 5090 32GB×16List price $32,004Market price $114,986–$124,568Cloud rental (live) ×16: $7.30–$9.62/hr (2026-11-20)
RTX 5090D 24GB×16List price $39,524Market price $67,075–$95,821
INT4 (AWQ/GPTQ)
≈189 GB VRAM
L20 48GB×4List price $13,982–$14,062Market price $20,961
Ascend 910B 64GB×4List price n/aMarket price $65,877–$71,866
RTX PRO 6000 Blackwell 96GB×2List price $32,077Market price $50,905Cloud rental (live) ×2: $2.64–$4.01/hr (2026-11-20)
RTX 5090 32GB×8List price $16,002Market price $57,493–$62,284Cloud rental (live) ×8: $3.65–$4.81/hr (2026-11-20)
RTX 5090D 24GB×8List price $19,762Market price $33,537–$47,911
Full setup across 19 GPUs → Official repo →
Nex AGI1 model · 747 downloads (30d) · latest 2026-06-03

Nex-N2-Pro

397B total · 18B active (est.) · apache-2.0
Popularity
747HF downloads (30d) · 377 likes
Released
2026-06-03HF repo created
Public benchmarks
No public benchmark yet
Vendor API
—input / output per 1M tokens
FP16 VRAM
≈952 GBINT4 ≈238 GB
VRAM and reference setups (FP16 / INT8 / INT4)
FP16/BF16 (full precision)
≈952 GB VRAM
L20 48GB×32List price $111,857–$112,499Market price $167,687
Ascend 910B 64GB×16List price n/aMarket price $263,509–$287,464
RTX PRO 6000 Blackwell 96GB×16List price $256,614Market price $407,240Cloud rental (live) ×16: $21.15–$32.11/hr (2026-11-20)
RTX 5090 32GB×32List price $64,009Market price $229,971–$249,135Cloud rental (live) ×32: $14.59–$19.23/hr (2026-11-20)
RTX 5090D 24GB×64List price $158,096Market price $268,300–$383,285
INT8
≈476 GB VRAM
L20 48GB×16List price $55,928–$56,249Market price $83,844
Ascend 910B 64GB×8List price n/aMarket price $131,754–$143,732
RTX PRO 6000 Blackwell 96GB×8List price $128,307Market price $203,620Cloud rental (live) ×8: $10.58–$16.06/hr (2026-11-20)
RTX 5090 32GB×16List price $32,004Market price $114,986–$124,568Cloud rental (live) ×16: $7.30–$9.62/hr (2026-11-20)
RTX 5090D 24GB×32List price $79,048Market price $134,150–$191,643
INT4 (AWQ/GPTQ)
≈238 GB VRAM
L20 48GB×8List price $27,964–$28,125Market price $41,922
Ascend 910B 64GB×4List price n/aMarket price $65,877–$71,866
RTX PRO 6000 Blackwell 96GB×4List price $64,154Market price $101,810Cloud rental (live) ×4: $5.29–$8.03/hr (2026-11-20)
RTX 5090 32GB×8List price $16,002Market price $57,493–$62,284Cloud rental (live) ×8: $3.65–$4.81/hr (2026-11-20)
RTX 5090D 24GB×16List price $39,524Market price $67,075–$95,821
Full setup across 19 GPUs → Official repo →
INFERENCE ENGINES

Open-source inference enginesWhich software runs the model when you self-host · from each engine's official repo/docs, checked 2026-10-24

EngineByModelsHardwareOnline servingLicenseNotes
vLLMCommunity (UC Berkeley origin)Most open-weights modelsNVIDIA / AMD / TPU / Intel etc.; Ascend via vllm-ascend pluginYes (OpenAI-compatible API)Apache-2.0PagedAttention; de-facto default, new models usually supported on day one
SGLangCommunity (LMSYS origin)Most open-weights modelsNVIDIA / AMD / Ascend / TPU etc.Yes (OpenAI-compatible API)Apache-2.0RadixAttention prefix cache; common for large MoE deployments
TensorRT-LLMNVIDIAMajor open-weights models (per official support list)NVIDIA onlyYes (trtllm-serve / Triton)Apache-2.0NVIDIA-optimized; first to use new GPU features
MindIEHuaweiMajor open-weights models (per Ascend support list)Ascend onlyYes (MindIE Service)华为软件许可Huawei's official Ascend inference suite
llama.cppCommunityGGUF quantized modelsCPU / NVIDIA / AMD / Apple etc., mostly consumer hardwareYes (llama-server, single-node)MITLow-bit quantization for PCs and single GPUs; Ollama and others build on it
DeepInferreleased 2026-10-20DeepSeekDeepSeek V4.1 Flash onlyNVIDIA SM100 (B200/B300) / Ascend 950No (offline inference only in this release)Apache-2.0Powers DeepSeek's own production; KV cache system, prefill/decode disaggregation and load balancing not included
GPU prices

Pricing for 19 GPUs: GPU Hardware Prices and GPU Cloud Rental

FAQ

FAQ

How is the VRAM requirement calculated?

VRAM = total parameters (B) × bytes per parameter × 1.2 runtime headroom. It covers model weights only, not concurrency or long-context KV cache; leave another 20–30% and load-test before buying.

How are GPU counts derived?

GPU counts use reference cards (L20 48 GB, Ascend 910B 64 GB, RTX PRO 6000 96 GB, RTX 5090 32 GB, RTX 5090D 24 GB) as examples; any GPU of similar VRAM works. Every model page covers all GPUs.

What do “active parameters” mean for MoE models?

Active parameters set compute and speed but do not reduce weight VRAM, so sizing uses total parameters; “est.” marks active-parameter counts inferred from sibling releases.

Where do the GPU prices come from?

SXM data-center GPUs (B300/B200/H200/H100/H800) are quoted per 8-GPU HGX node, as the industry does; per-GPU equivalents are shown and sizing bills whole nodes. USD converted from CNY at ¥6.68, taken from public reports and re-searched weekly; they swing widely and are indicative only.

More questions (1)
What is “Cloud rental (live)”?

It comes from the public vast.ai market API, refreshed daily, and does not reflect China-market purchase prices. TANDIU does not sell or resell hardware.