How much GPU memory does DeepSeek V3.2 need?
DeepSeek V3.2 by DeepSeek has 685B parameters, 37B of them active for each token. Its weights take 1,371 GB at 16-bit, 685 GB at 8-bit and 386 GB at 4-bit; released in FP8, its files are 689 GB. Each token of a conversation adds 85.9 KB of KV cache, so a 32K conversation holds 2.81 GB.
DeepSeek · released in FP8
DeepSeek V3.2 at a glance
- Parameters
- 685B
- 37B active for each token
- As released
- 689 GB
- Its FP8 files
- At 16-bit
- 1,371 GB
- At 4-bit
- 386 GB
- NVFP4
- KV cache a token
- 85.9 KB
- At 16-bit
- Longest context
- 163,840
- Tokens
GPUs needed at each number format
For 1 conversation of 32,768 tokens with a 16-bit KV cache, counting 92% of each GPU's memory as vLLM does. Under each count, the most tokens a second memory allows.
| GPU | Memory | 16-bit | 8-bit | 4-bit |
|---|---|---|---|---|
| L4 | 24 GB | 64 GPUs31 tokens/s | 32 GPUs60 tokens/s | 24 GPUs102 tokens/s |
| GeForce RTX 5090 | 32 GB | 48 GPUs187 tokens/s | 24 GPUs360 tokens/s | 16 GPUs607 tokens/s |
| L40S | 48 GB | 32 GPUs90 tokens/s | 16 GPUs174 tokens/s | 16 GPUs293 tokens/s |
| A100 80GB | 80 GB | 24 GPUs212 tokens/s | 16 GPUs410 tokens/s | 8 GPUs690 tokens/s |
| H100 | 80 GB | 24 GPUs349 tokens/s | 16 GPUs673 tokens/s | 8 GPUs1,134 tokens/s |
| RTX PRO 6000 Blackwell | 96 GB | 16 GPUs187 tokens/s | 8 GPUs360 tokens/s | 8 GPUs607 tokens/s |
| DGX Spark | 128 GB | 16 GPUs28 tokens/s | 8 GPUs55 tokens/s | 4 GPUs46 tokens/s |
| H200 | 141 GB | 16 GPUs500 tokens/s | 8 GPUs964 tokens/s | 4 GPUs813 tokens/s |
| B200 | 180 GB | 16 GPUs833 tokens/s | 8 GPUs1,607 tokens/s | 4 GPUs1,354 tokens/s |
| Instinct MI300X | 192 GB | 8 GPUs552 tokens/s | 4 GPUs532 tokens/s | 4 GPUs897 tokens/s |
| Instinct MI325X | 256 GB | 8 GPUs625 tokens/s | 4 GPUs603 tokens/s | 2 GPUs508 tokens/s |
| Instinct MI355X | 288 GB | 8 GPUs833 tokens/s | 4 GPUs804 tokens/s | 2 GPUs677 tokens/s |
Models of a similar size
- Qwen3.5 122B A10B125B parameters, 70.4 GB at 4-bit
- Mistral Medium 3.5 128B128B parameters, 71.8 GB at 4-bit
- Qwen3.5 397B A17B403B parameters, 227 GB at 4-bit
- GLM-5.3753B parameters, 424 GB at 4-bit
- Kimi K2.61T parameters, 578 GB at 4-bit
- Qwen3.8 2.4T A95B2.4T parameters, 1,376 GB at 4-bit
Questions
- How much VRAM does DeepSeek V3.2 need?
- 1,371 GB for its weights at 16-bit, 685 GB at 8-bit and 386 GB at 4-bit, plus the KV cache: 85.9 KB for each token, 2.81 GB for a conversation of 32K tokens and 14.1 GB at its longest context of 163,840 tokens. The server also keeps about 8% of each GPU free.
- Can DeepSeek V3.2 run on 1 GPU?
- Not on any single GPU here: at 4-bit with a 32K conversation it needs 8 H100s.
- How many H100s does DeepSeek V3.2 need?
- With 1 conversation of 32K tokens, 24 H100s (3 servers of 8) at 16-bit and 8 H100s at 4-bit. Each H100 has 80 GB, of which vLLM uses 92% by default.
- Can I run DeepSeek V3.2 on an RTX 5090?
- The RTX 5090 has 32 GB. With a 32K conversation, DeepSeek V3.2 needs 16 GeForce RTX 5090s (2 servers of 8) at 4-bit and 48 GeForce RTX 5090s (6 servers of 8) at 16-bit.
- How fast does DeepSeek V3.2 run?
- At 4-bit on 8 H100s, memory lets 1 conversation get at most 1,134 tokens a second. Real servers reach part of that ceiling, and each conversation gets less when many run at once.
Sources
Where each fact comes from
- 1Hugging Face: deepseek-ai/DeepSeek-V3.2
- 2vLLM: Engine arguments (gpu_memory_utilization)
- 3vLLM: Hybrid KV cache manager
- 4vLLM: Parallelism and scaling
- 5NVIDIA: Introducing NVFP4
- 6NVIDIA: L4 specifications
- 7NVIDIA: GeForce RTX 5090 specifications
- 8NVIDIA: L40S specifications
- 9NVIDIA: A100 80GB specifications
- 10NVIDIA: H100 specifications
- 11NVIDIA: RTX PRO 6000 Blackwell specifications
- 12NVIDIA: DGX Spark specifications
- 13NVIDIA: H200 specifications
- 14NVIDIA: B200 specifications
- 15AMD: Instinct MI300X specifications
- 16AMD: Instinct MI325X specifications
- 17AMD: Instinct MI355X specifications