What are parameters in an AI model?

When a model is called "8B" or "70B", the number is its parameter count: about 8 or 70 billion numbers that training has tuned. Everything the model can do is stored in those numbers. There is no separate list of facts and no rule book.

Weights and biases

A neural networkNeural NetworkA computing system inspired by biological neural networks that learn patterns from data is built from layers that multiply their input by a grid of numbers, add a few more, and pass the result on. The numbers in the grid are weightsWeightA trainable parameter in a neural network that scales input values and is updated during training.; the extra added ones are biasesBiasA trainable parameter added to the weighted sum of inputs in a neural network layer, allowing the activation function to be shifted.. Together they are the parameters.

The count comes from the shapes. A layer that turns 4,096 numbers into 4,096 others has a grid of 4,096 times 4,096 weights, about 16.8 million, plus 4,096 biases. A large language model stacks dozens of such layers, and the billions add up quickly.

Training sets every one of them, by small steps of gradient descentGradient DescentThe way a neural network learns. Measure how wrong it is, work out which way each weight should move to be less wrong, take a small step that way, repeat.. Before training they are random; afterwards their pattern holds grammar, facts, style and skills, spread across all of them rather than stored in any one place.

What the count tells you

Memory. Each parameter is stored as a number of a fixed size. At 16 bits that is 2 bytes, so a 70B model needs about 140 GB just to hold its weights. QuantizationQuantizationStoring a model's numbers with less precision, such as 4 bits instead of 16, so it needs far less memory and can run on smaller hardware, at some cost in accuracy. to 4 bits brings that to about 35 GB. This is the first sum to do before running a model yourself, and the GPU memory calculator does it for any open model.

Rough capability. Within one family, the larger model is usually better. Across families the count says much less. Training data matters as much: DeepMind's Chinchilla, at 70B, beat models up to 530B that were trained on less data.

Not the speed or cost, directly. A mixture-of-expertsMixture of ExpertsA neural architecture that routes tokens through different expert subnetworks. model may have hundreds of billions of parameters in total but use only a fraction of them for each token. Its "active" count is what sets the work per token; its total count is what sets the memory.

Parameters and settings

People also call the settings you send with a request (temperature, max tokens) "parameters". Those are settings, sometimes called hyperparameters when they shape training. They are chosen by a person. The model's parameters are learned, and nobody sets them by hand.