How much VRAM does GLM-5.3 need?

1,507 GB for its weights at 16-bit, 753 GB at 8-bit and 424 GB at 4-bit, plus the KV cache: 110 KB for each token, 3.6 GB for a conversation of 32K tokens and 115 GB at its longest context of 1,048,576 tokens. The server also keeps about 8% of each GPU free.

Can GLM-5.3 run on 1 GPU?

Not on any single GPU here: at 4-bit with a 32K conversation it needs 8 H100s.

How many H100s does GLM-5.3 need?

With 1 conversation of 32K tokens, 24 H100s (3 servers of 8) at 16-bit and 8 H100s at 4-bit. Each H100 has 80 GB, of which vLLM uses 92% by default.

Can I run GLM-5.3 on an RTX 5090?

The RTX 5090 has 32 GB. With a 32K conversation, GLM-5.3 needs 16 GeForce RTX 5090s (2 servers of 8) at 4-bit and 56 GeForce RTX 5090s (7 servers of 8) at 16-bit.

How fast does GLM-5.3 run?

At 4-bit on 8 H100s, memory lets 1 conversation get at most 1,027 tokens a second. Real servers reach part of that ceiling, and each conversation gets less when many run at once.

How much GPU memory does GLM-5.3 need?

GLM-5.3 by Z.ai has 753B parameters, 40B of them active for each token. Its weights take 1,507 GB at 16-bit, 753 GB at 8-bit and 424 GB at 4-bit; released in FP8, its files are 756 GB. Each token of a conversation adds 110 KB of KV cache, so a 32K conversation holds 3.6 GB.

Size GLM-5.3 for your setup Questions

Z.ai · released in FP8

GLM-5.3 at a glance

Parameters: 753B; 40B active for each token
As released: 756 GB; Its FP8 files
At 16-bit: 1,507 GB
At 4-bit: 424 GB; NVFP4
KV cache a token: 110 KB; At 16-bit
Longest context: 1,048,576; Tokens

GPUs needed at each number format

For 1 conversation of 32,768 tokens with a 16-bit KV cache, counting 92% of each GPU's memory as vLLM does. Under each count, the most tokens a second memory allows.

GPU	Memory	16-bit	8-bit	4-bit
L4	24 GB	72 GPUs29 tokens/s	40 GPUs55 tokens/s	24 GPUs92 tokens/s
GeForce RTX 5090	32 GB	56 GPUs171 tokens/s	32 GPUs329 tokens/s	16 GPUs549 tokens/s
L40S	48 GB	40 GPUs83 tokens/s	24 GPUs159 tokens/s	16 GPUs265 tokens/s
A100 80GB	80 GB	24 GPUs195 tokens/s	16 GPUs374 tokens/s	8 GPUs625 tokens/s
H100	80 GB	24 GPUs321 tokens/s	16 GPUs615 tokens/s	8 GPUs1,027 tokens/s
RTX PRO 6000 Blackwell	96 GB	24 GPUs171 tokens/s	16 GPUs329 tokens/s	8 GPUs549 tokens/s
DGX Spark	128 GB	16 GPUs26 tokens/s	8 GPUs50 tokens/s	4 GPUs42 tokens/s
H200	141 GB	16 GPUs459 tokens/s	8 GPUs881 tokens/s	4 GPUs736 tokens/s
B200	180 GB	16 GPUs766 tokens/s	8 GPUs1,468 tokens/s	4 GPUs1,226 tokens/s
Instinct MI300X	192 GB	16 GPUs507 tokens/s	8 GPUs973 tokens/s	4 GPUs812 tokens/s
Instinct MI325X	256 GB	8 GPUs574 tokens/s	4 GPUs550 tokens/s	2 GPUs460 tokens/s
Instinct MI355X	288 GB	8 GPUs766 tokens/s	4 GPUs734 tokens/s	2 GPUs613 tokens/s

Models of a similar size

Every model

Questions

How much VRAM does GLM-5.3 need?: 1,507 GB for its weights at 16-bit, 753 GB at 8-bit and 424 GB at 4-bit, plus the KV cache: 110 KB for each token, 3.6 GB for a conversation of 32K tokens and 115 GB at its longest context of 1,048,576 tokens. The server also keeps about 8% of each GPU free.
Can GLM-5.3 run on 1 GPU?: Not on any single GPU here: at 4-bit with a 32K conversation it needs 8 H100s.
How many H100s does GLM-5.3 need?: With 1 conversation of 32K tokens, 24 H100s (3 servers of 8) at 16-bit and 8 H100s at 4-bit. Each H100 has 80 GB, of which vLLM uses 92% by default.
Can I run GLM-5.3 on an RTX 5090?: The RTX 5090 has 32 GB. With a 32K conversation, GLM-5.3 needs 16 GeForce RTX 5090s (2 servers of 8) at 4-bit and 56 GeForce RTX 5090s (7 servers of 8) at 16-bit.
How fast does GLM-5.3 run?: At 4-bit on 8 H100s, memory lets 1 conversation get at most 1,027 tokens a second. Real servers reach part of that ceiling, and each conversation gets less when many run at once.

Sources

GLM-5.3 at a glance

GPUs needed at each number format

Models of a similar size

Questions

Where each fact comes from