# What is parameter-efficient fine-tuning (PEFT), and when should you use it instead of full fine-tuning?

Fine-tuning that freezes a model's own weights and trains a small number of new ones, such as a LoRA adapter, so a task costs a fraction of the memory and storage of full fine-tuning.

- Page: https://tecyfy.com/concepts/parameter-efficient-fine-tuning
- Subject: How AI works (https://tecyfy.com/concepts)
- Level: advanced
- Also called: PEFT, Parameter Efficient Fine Tuning

Full [Fine-Tuning](https://tecyfy.com/concepts/fine-tuning) updates every weight in a model. That needs memory for the weights, their gradients and the optimizer's state for each one, and it produces a full new copy of the model for every task. Parameter-efficient fine-tuning, PEFT for short, freezes the original weights and trains only a small set of new ones. The name is also that of Hugging Face's library for doing it.

## Why it took over

The cost of full fine-tuning grows with the model. The LoRA paper's example is GPT-3 175B: each fine-tuned copy has 175 billion parameters. Training only a small addition changes the arithmetic:

- **Memory to train.** No gradients or optimizer state for frozen weights. LoRA cut GPU memory 3 times on GPT-3 175B, and [QLoRA](https://tecyfy.com/concepts/qlora), by keeping the frozen weights in 4 bits, fine-tuned a 65B model in under 48 GB where 16-bit fine-tuning needed more than 780 GB.
- **Storage.** 100 tasks on 1 base model take about 354 GB as LoRA adapters, against 35 TB as full copies, in the LoRA paper's sums.
- **Serving.** 1 copy of the base model in memory can serve many tasks, each request choosing its adapter.

And on the models the paper tested, quality matched full fine-tuning or beat it.

## The main kinds

- **[LoRA](https://tecyfy.com/concepts/lora)** adds 2 small matrices beside chosen layers and learns a low-rank correction. It can be merged into the weights, so it adds no delay when the model runs. It is the usual choice today.
- **QLoRA** is LoRA trained on a 4-bit copy of the base model, for when memory is the limit.
- **[Adapters](https://tecyfy.com/concepts#adapter)** insert small new layers into the network. They work, but the LoRA paper notes they add inference latency, since the extra layers run in sequence.
- **Prompt and prefix tuning** learn a few vectors placed in front of the input instead of changing any layer.

## When to use it

Use PEFT when you fine-tune an open model at all: it is cheaper, the result is a small file you can version and roll back, and it rarely costs quality. Full fine-tuning is for when an adapter measurably falls short on your [eval set](https://tecyfy.com/concepts/eval-set) and you have the hardware.

Training settings differ a little from full fine-tuning. Hugging Face's TRL notes that adapters usually take a higher learning rate, around 1e-4, because only the new weights are learning. The bigger question comes before any of this: whether the job needs fine-tuning at all, or a better prompt, a few examples or retrieval.

## Related

- [Fine-Tuning](https://tecyfy.com/concepts/fine-tuning): Training an existing model a little further on your own examples so it picks up a task, a format or a style. Often a good prompt does the job first.
- [LoRA](https://tecyfy.com/concepts/lora): A way to fine-tune a model by freezing its weights and training 2 small matrices beside each chosen layer, so a task needs a tiny file instead of a full copy of the model.
- [QLoRA](https://tecyfy.com/concepts/qlora): LoRA trained on a copy of the base model stored in 4 bits, so fine-tuning a large model needs a fraction of the GPU memory, with 3 tricks that keep the quality of 16-bit fine-tuning.
- [Adapter](https://tecyfy.com/concepts#adapter): A small bottleneck module inserted in each transformer layer to adapt a model to new tasks without updating full weights.
- [Knowledge Distillation](https://tecyfy.com/concepts/knowledge-distillation): Training a small "student" model to copy the outputs of a large "teacher" model, so the small one keeps much of the large one's skill at a fraction of the cost to run.

## Sources

- [LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://arxiv.org/abs/2305.14314)
- [SFT Trainer](https://huggingface.co/docs/trl/main/en/sft_trainer)