# What are parameters in an AI model? A number inside a model that training sets, a weight or a bias. "70B" in a model's name means about 70 billion of them. - Page: https://tecyfy.com/concepts/parameter - Subject: How AI works (https://tecyfy.com/concepts) - Level: beginner - Also called: Model Parameters, Parameter Count When a model is called "8B" or "70B", the number is its parameter count: about 8 or 70 billion numbers that training has tuned. Everything the model can do is stored in those numbers. There is no separate list of facts and no rule book. ## Weights and biases A [neural network](https://tecyfy.com/concepts#neural-network) is built from layers that multiply their input by a grid of numbers, add a few more, and pass the result on. The numbers in the grid are [weights](https://tecyfy.com/concepts#weight); the extra added ones are [biases](https://tecyfy.com/concepts#bias). Together they are the parameters. The count comes from the shapes. A layer that turns 4,096 numbers into 4,096 others has a grid of 4,096 times 4,096 weights, about 16.8 million, plus 4,096 biases. A large language model stacks dozens of such layers, and the billions add up quickly. Training sets every one of them, by small steps of [gradient descent](https://tecyfy.com/concepts/gradient-descent). Before training they are random; afterwards their pattern holds grammar, facts, style and skills, spread across all of them rather than stored in any one place. ## What the count tells you **Memory.** Each parameter is stored as a number of a fixed size. At 16 bits that is 2 bytes, so a 70B model needs about 140 GB just to hold its weights. [Quantization](https://tecyfy.com/concepts/quantization) to 4 bits brings that to about 35 GB. This is the first sum to do before running a model yourself, and the [GPU memory calculator](/ai-agents/gpu-memory-calculator) does it for any open model. **Rough capability.** Within one family, the larger model is usually better. Across families the count says much less. Training data matters as much: DeepMind's Chinchilla, at 70B, beat models up to 530B that were trained on less data. **Not the speed or cost, directly.** A [mixture-of-experts](https://tecyfy.com/concepts#mixture-of-experts) model may have hundreds of billions of parameters in total but use only a fraction of them for each token. Its "active" count is what sets the work per token; its total count is what sets the memory. ## Parameters and settings People also call the settings you send with a request (temperature, max tokens) "parameters". Those are settings, sometimes called hyperparameters when they shape training. They are chosen by a person. The model's parameters are learned, and nobody sets them by hand. ## Related - [Weight](https://tecyfy.com/concepts#weight): A trainable parameter in a neural network that scales input values and is updated during training. - [Bias](https://tecyfy.com/concepts#bias): A trainable parameter added to the weighted sum of inputs in a neural network layer, allowing the activation function to be shifted. - [Large Language Model](https://tecyfy.com/concepts/large-language-model): A model trained on a huge amount of text to predict the next token. Further training turns it into an assistant that follows instructions. - [Neural Network](https://tecyfy.com/concepts#neural-network): A computing system inspired by biological neural networks that learn patterns from data - [Quantization](https://tecyfy.com/concepts/quantization): Storing a model's numbers with less precision, such as 4 bits instead of 16, so it needs far less memory and can run on smaller hardware, at some cost in accuracy. - [Mixture of Experts](https://tecyfy.com/concepts#mixture-of-experts): A neural architecture that routes tokens through different expert subnetworks. ## Sources - [torch.nn.Linear](https://docs.pytorch.org/docs/stable/generated/torch.nn.Linear.html) - [Model training anatomy](https://huggingface.co/docs/transformers/model_memory_anatomy) - [Training compute-optimal large language models](https://arxiv.org/abs/2203.15556)