How much VRAM does Qwen3.8 27B need?

55.6 GB for its weights at 16-bit, 27.8 GB at 8-bit and 15.6 GB at 4-bit, plus the KV cache: 65.5 KB for each token, 2.3 GB for a conversation of 32K tokens and 17.3 GB at its longest context of 262,144 tokens. The server also keeps about 8% of each GPU free.

Can Qwen3.8 27B run on 1 GPU?

Yes, at 4-bit: the smallest single GPU here that holds it with a 32K conversation is the L4.

How many H100s does Qwen3.8 27B need?

With 1 conversation of 32K tokens, 1 H100 at 16-bit and 1 H100 at 4-bit. Each H100 has 80 GB, of which vLLM uses 92% by default.

Can I run Qwen3.8 27B on an RTX 5090?

The RTX 5090 has 32 GB. With a 32K conversation, Qwen3.8 27B needs 1 GeForce RTX 5090 at 4-bit and 2 GeForce RTX 5090s at 16-bit.

How fast does Qwen3.8 27B run?

At 4-bit on 1 H100, memory lets 1 conversation get at most 187 tokens a second. Real servers reach part of that ceiling, and each conversation gets less when many run at once.

How much GPU memory does Qwen3.8 27B need?

Qwen3.8 27B by Qwen has 27.8B parameters. Its weights take 55.6 GB at 16-bit, 27.8 GB at 8-bit and 15.6 GB at 4-bit; released in BF16, its files are 55.6 GB. Each token of a conversation adds 65.5 KB of KV cache, so a 32K conversation holds 2.3 GB.

Size Qwen3.8 27B for your setup Questions

Qwen · released in BF16

Qwen3.8 27B at a glance

Parameters: 27.8B
As released: 55.6 GB; Its BF16 files
At 16-bit: 55.6 GB
At 4-bit: 15.6 GB; NVFP4
KV cache a token: 65.5 KB; At 16-bit
Longest context: 262,144; Tokens

GPUs needed at each number format

For 1 conversation of 32,768 tokens with a 16-bit KV cache, counting 92% of each GPU's memory as vLLM does. Under each count, the most tokens a second memory allows.

GPU	Memory	16-bit	8-bit	4-bit
L4	24 GB	4 GPUs21 tokens/s	2 GPUs20 tokens/s	1 GPU17 tokens/s
GeForce RTX 5090	32 GB	2 GPUs62 tokens/s	2 GPUs119 tokens/s	1 GPU100 tokens/s
L40S	48 GB	2 GPUs30 tokens/s	1 GPU29 tokens/s	1 GPU48 tokens/s
A100 80GB	80 GB	1 GPU35 tokens/s	1 GPU68 tokens/s	1 GPU114 tokens/s
H100	80 GB	1 GPU58 tokens/s	1 GPU111 tokens/s	1 GPU187 tokens/s
RTX PRO 6000 Blackwell	96 GB	1 GPU31 tokens/s	1 GPU60 tokens/s	1 GPU100 tokens/s
DGX Spark	128 GB	1 GPU4.7 tokens/s	1 GPU9.1 tokens/s	1 GPU15 tokens/s
H200	141 GB	1 GPU83 tokens/s	1 GPU160 tokens/s	1 GPU268 tokens/s
B200	180 GB	1 GPU138 tokens/s	1 GPU266 tokens/s	1 GPU446 tokens/s
Instinct MI300X	192 GB	1 GPU92 tokens/s	1 GPU176 tokens/s	1 GPU296 tokens/s
Instinct MI325X	256 GB	1 GPU104 tokens/s	1 GPU199 tokens/s	1 GPU335 tokens/s
Instinct MI355X	288 GB	1 GPU138 tokens/s	1 GPU266 tokens/s	1 GPU446 tokens/s

Models of a similar size

Every model

Questions

How much VRAM does Qwen3.8 27B need?: 55.6 GB for its weights at 16-bit, 27.8 GB at 8-bit and 15.6 GB at 4-bit, plus the KV cache: 65.5 KB for each token, 2.3 GB for a conversation of 32K tokens and 17.3 GB at its longest context of 262,144 tokens. The server also keeps about 8% of each GPU free.
Can Qwen3.8 27B run on 1 GPU?: Yes, at 4-bit: the smallest single GPU here that holds it with a 32K conversation is the L4.
How many H100s does Qwen3.8 27B need?: With 1 conversation of 32K tokens, 1 H100 at 16-bit and 1 H100 at 4-bit. Each H100 has 80 GB, of which vLLM uses 92% by default.
Can I run Qwen3.8 27B on an RTX 5090?: The RTX 5090 has 32 GB. With a 32K conversation, Qwen3.8 27B needs 1 GeForce RTX 5090 at 4-bit and 2 GeForce RTX 5090s at 16-bit.
How fast does Qwen3.8 27B run?: At 4-bit on 1 H100, memory lets 1 conversation get at most 187 tokens a second. Real servers reach part of that ceiling, and each conversation gets less when many run at once.

Sources

Qwen3.8 27B at a glance

GPUs needed at each number format

Models of a similar size

Questions

Where each fact comes from