In Azure OpenAI, what distinguishes fine-tuning from prompt engineering as ways to adapt a pre-trained model?

Prompt engineering changes what you send to the model; fine-tuning changes the model. With prompt engineering, instructions, examples and context go into each request and the deployed model stays the same. With fine-tuning, you train a copy of a base model on your own examples and deploy that copy, which then behaves differently even with a short prompt.

Prompt engineering

• Instructions in the system message, a few examples of the wanted output, and the data the answer should use (often retrieved from your documents, which is retrieval-augmented generation).
• Changes take effect at once: edit the prompt, re-run your tests.
• Costs nothing extra up front, but long prompts are paid for on every request.
• Works with any model you can call, and keeps working when you move to a newer model, usually with small adjustments.

Fine-tuning

• You prepare training data (in Azure OpenAI, JSONL files of example conversations), run a training job and deploy the resulting model.
• Microsoft Foundry offers supervised fine-tuning (prompts and the answers you want), preference fine-tuning with direct preference optimization (a preferred and a rejected answer to the same input) and reinforcement fine-tuning (a grader scores answers) for models that support each.
• Teaches a consistent style, format or domain behaviour that is hard to describe in words, and can let a smaller, cheaper model match a larger one on a narrow task.
• Adds training and hosting costs, needs good data and held-out evaluation, and has to be repeated when you move to a new base model.

What fine-tuning doesn't do It is a poor way to add facts. Knowledge that changes, or that the answer must cite, belongs in retrieval, and fine-tuning doesn't replace the application's safety controls.

How to choose Start with prompt engineering and retrieval, and measure the results on a test set. Consider fine-tuning only when the prompt is already good, the remaining failures are about behaviour or format rather than missing knowledge, and you have hundreds of high-quality examples. Teams often combine the two: a fine-tuned model for consistent behaviour, with retrieval and a prompt for the facts of each request.