What is quantization in AI?

Storing a model's numbers with less precision, such as 4 bits instead of 16, so it needs far less memory and can run on smaller hardware, at some cost in accuracy.

intermediate · Also called Quantisation

A model is a very large set of numbers. Each one has to be stored, and the question is how precisely. Quantization means storing them less precisely, to save space.

The arithmetic

Meta's Llama 3 8B has 8 billion numbers, published at 16 bits each. 16 bits is 2 bytes, so the model needs about 16 GB just to hold its weights.

Store each number in 8 bits and it needs about 8 GB. Store each in 4 bits and it needs about 4 GB, a quarter of the original. That is the difference between needing a large graphics card and fitting on a much smaller one.

The sum is the same at any size: parameters times bytes per parameter. It is the first sum to do before you try to run an open model yourself.

What you give up

Fewer bits means each number is rounded more coarsely. Picture recording every price in a shop to the nearest pound instead of the nearest penny. Most totals come out close; a few do not. A quantized model behaves the same way: close to the original on much of its work, a little worse on some of it.

The better methods are careful about where to round. Hugging Face describes them as lowering memory while trying to keep as much accuracy as possible, and some need a calibration pass on sample data to choose the rounding well. Squeezing all the way to 1 or 2 bits is where those careful methods are needed most.

Where you see it

If you download an open model to run at home, with tools built on llama.cpp for example, you will usually be offered the same model at several bit sizes. The smaller file loads on smaller hardware; the larger one is closer to the original.

Hugging Face lists more than 20 quantization methods in its own library, some going as low as 1 bit, each suited to different hardware.

How to choose

Start from the memory you have. Pick the largest version that fits with some room to spare, since a long conversation needs memory for the KV cache on top of the weights. Then test it on your own tasks, because the averages in a model card may not match your use.

Sources

Where each fact comes from

This page was last checked against these sources on 14 October 2026.

1
Quantization overview
Hugging Face, Transformers documentation · 14 October 2026
Quantization lowers the memory needed to load and use a model by storing the weights in a lower precision while trying to keep as much accuracy as possible. Weights are typically stored in full precision (fp32), and half precision (fp16 or bf16) is increasingly popular. Some methods reduce precision further to integers such as int8 or int4. Some methods need calibration for greater accuracy and extreme compression of 1 to 2 bits. GGUF for llama.cpp is one of the formats listed, among more than 20 methods, some going as low as 1 bit, with different hardware support. Hugging Face's quantization picker checks whether a model fits in memory at each precision, including the KV cache.
2
Meta-Llama-3-8B model card
Meta, on Hugging Face · 25 September 2026
The 8 billion parameter model is published as 16-bit numbers (tensor type BF16).
3
How ChatGPT and our foundation models are developed
OpenAI Help Center · 25 September 2026
A model is a large set of numbers called weights or parameters.

Inference
Using a trained model to produce an answer. Training builds the model once; inference is every time someone asks it something, and it is what you pay for per token.
Parameter
A learned weight or bias value in a neural network; large language models may have billions of parameters.
4‑bit Quantization
Representing weights with 4‑bit integers.
Half‑Precision
A 16‑bit floating‑point format used to accelerate training and inference while reducing memory.
BF16
Bfloat16 — a 16‑bit floating‑point format with wider exponent range than FP16, common on TPUs.
QLoRA
Quantized LoRA that fine‑tunes 4‑bit quantized base models.
KV Cache
Work a model saves from the text it has already read, so it is not redone for every new token. Prompt caching keeps that work between requests and charges less for it.