Prompting interview questions for AI Engineer roles

Prompt engineering changes what you send to the model; fine-tuning changes the model. With prompt engineering, instructions, examples and context go into each request and the deployed model stays the same. With fine-tuning, you train a copy of a base model on your own examples and deploy that copy, which then behaves differently even with a short prompt.

Prompt engineering

• Instructions in the system message, a few examples of the wanted output, and the data the answer should use (often retrieved from your documents, which is retrieval-augmented generation).
• Changes take effect at once: edit the prompt, re-run your tests.
• Costs nothing extra up front, but long prompts are paid for on every request.
• Works with any model you can call, and keeps working when you move to a newer model, usually with small adjustments.

Fine-tuning

• You prepare training data (in Azure OpenAI, JSONL files of example conversations), run a training job and deploy the resulting model.
• Microsoft Foundry offers supervised fine-tuning (prompts and the answers you want), preference fine-tuning with direct preference optimization (a preferred and a rejected answer to the same input) and reinforcement fine-tuning (a grader scores answers) for models that support each.
• Teaches a consistent style, format or domain behaviour that is hard to describe in words, and can let a smaller, cheaper model match a larger one on a narrow task.
• Adds training and hosting costs, needs good data and held-out evaluation, and has to be repeated when you move to a new base model.

What fine-tuning doesn't do It is a poor way to add facts. Knowledge that changes, or that the answer must cite, belongs in retrieval, and fine-tuning doesn't replace the application's safety controls.

How to choose Start with prompt engineering and retrieval, and measure the results on a test set. Consider fine-tuning only when the prompt is already good, the remaining failures are about behaviour or format rather than missing knowledge, and you have hundreds of high-quality examples. Teams often combine the two: a fine-tuned model for consistent behaviour, with retrieval and a prompt for the facts of each request.

An effective prompt says exactly what the task is, gives the model the context and data it needs, shows what a good answer looks like, and specifies the output format. Then it is tested against real examples and improved.

1. Be clear and direct State the task, the audience and what done looks like. "Summarise this incident report in five bullet points for the on-call engineer, starting with customer impact" works far better than "Summarise this". A useful test: would a capable new colleague with no background do the right thing from these words alone?

2. Give context and say why Explain the purpose behind a rule ("the answer is read aloud, so avoid tables") so the model can apply it sensibly to cases you didn't foresee.

3. Separate instructions from data Mark documents, user input and examples with clear delimiters such as XML tags or Markdown headings, so the model knows what to follow and what to work on. This also reduces the risk of text inside the data being treated as instructions.

4. Show examples A few varied examples of input and ideal output (few-shot prompting) set the format and tone more reliably than descriptions. Make them diverse, so the model doesn't copy one example's quirks.

5. Specify the output Describe the structure, length and format, or use the API's structured outputs to require JSON that matches a schema when a program will read the answer.

6. Set a role when it helps A system message such as "You are a reviewer for a payments company's SQL migrations" focuses vocabulary and judgement.

7. Break complex work down Split multi-stage tasks into steps or separate prompts, and for hard reasoning let the model work through the problem before answering (reasoning models do this on their own).

8. Allow uncertainty Tell the model it may say it doesn't know, and to answer only from the supplied material when that matters.

9. Test and iterate Keep a set of test inputs with expected results, and re-run it after every change, and when the underlying model changes.

Judge a prompt by its results on a fixed set of test inputs, scored against criteria you defined before writing it, not by how well it reads or how one example turned out.

1. Define success first Write specific, measurable criteria: "classifies support tickets into the right one of 8 categories at least 95% of the time", "summaries under 120 words that mention every action item", "never gives medical dosage advice". Vague goals such as "good answers" can't be tested.

2. Build a test set

• Real inputs from the use case, including edge cases, ambiguous inputs and adversarial ones (attempts to make it ignore instructions).
• Expected outputs or properties for each, where possible.
• Enough cases for the differences to be meaningful: tens to start, hundreds for important prompts.
• Keep it separate from the examples inside the prompt, or you are testing memory.

3. Choose how to grade

• Code-graded: exact match, a regular expression, valid JSON against a schema, or a test that runs generated code. Fast, cheap and objective; use it wherever you can.
• Model-graded: another model scores each answer against a clear rubric (faithfulness to a source, tone, completeness). Scales to open-ended outputs, but check its scores against human judgement on a sample, and give it the reference answer when there is one.
• Human review: the most reliable for nuance, the slowest and most expensive. Use it to set the standard and audit the other graders.

4. Measure what matters for the task

• Accuracy or pass rate against the criteria.
• Faithfulness: does the answer stay with the supplied context, without invented facts?
• Format compliance: does it parse?
• Consistency: run each case several times, since outputs vary.
• Safety: refusal of what should be refused, no over-refusal of normal requests.
• Cost and latency: tokens in and out, and time to answer.

5. Compare and iterate Change one thing at a time, re-run the whole set, and keep the version that scores better without breaking other cases. A change that fixes one example often breaks another, and only the full set shows it.

6. Keep evaluating in production Log real inputs and outputs (with privacy in mind), collect user feedback such as thumbs up or down and edits, add failures to the test set, and re-run the evaluation when the model version changes. Pinning a specific model snapshot keeps behaviour stable between deliberate upgrades.

Prompt engineering is the practice of designing and testing the input to a language model (instructions, context, examples and output format) so that it reliably produces the output you need. It matters because the same model can give an excellent or a useless answer depending on the prompt, and in an application the prompt is the main lever you control without retraining anything.

What a prompt contains

• Instructions: the task, constraints and audience, often in a system message that applies to the whole conversation.
• Context: documents, database rows or search results the answer should use.
• Examples: sample inputs with ideal outputs.
• Output format: plain text, Markdown, or JSON matching a schema.
• The input itself: the user's question or the record to process.

Why it matters

• Quality: clear instructions and good context turn vague or generic answers into specific, correct ones.
• Reliability: applications need the same format and behaviour every time; a tested prompt (with structured outputs where a program reads the result) provides that.
• Accuracy: grounding the model in supplied sources and allowing it to say "I don't know" reduces invented answers.
• Cost and speed: shorter, focused prompts use fewer tokens, and a well-prompted smaller model can sometimes replace a larger one.
• Safety: prompts set boundaries on what the model will do, though they are not a security control on their own: prompt injection, where text in the input tries to override instructions, needs defences in the application too.
• No training needed: changes take minutes and need no data science infrastructure.

How the practice has changed Early prompt engineering relied on tricks and magic phrases. Modern models follow plain instructions well, so the work has shifted to:

• writing clear specifications;
• choosing what context to include (often called context engineering, especially for agents that gather information over many steps);
• building evaluations that show whether a change helped;
• adapting to reasoning models, which need less hand-holding about how to think.

A simple example "Write about our product" gives generic marketing copy. "Write a 100-word description of our open-source data quality library for data engineers evaluating it on GitHub. Mention that it runs inside Spark jobs and lists failed rows. No marketing superlatives." gives something usable on the first try.

The skill underneath is the same as writing a good brief for a person: know what you want, give the context needed, and check the result.

Prompt chaining splits a task into a sequence of smaller prompts, where the output of one step becomes input to the next, often with code in between that checks or transforms the results. Each prompt does one job well, and you can test, debug and improve each step on its own.

How it works

1. Decompose the task into stages with clear inputs and outputs.
2. Write a focused prompt for each stage.
3. Pass outputs forward, ideally in structured form (JSON or tagged sections) so code can validate them.
4. Add checks between steps: does the extraction have every field, does the generated SQL parse?
5. Branch or loop where needed: route to different prompts by category, or send a draft back for revision if a check fails.

Example: answering questions about a long contract

Step 1  Extract: "From the contract below, quote every clause about
        termination, notice periods and penalties, with section numbers."
            ↓ (quotes as a list)
Step 2  Answer: "Using only these quotes, answer the question:
        Can the customer cancel early, and what would it cost?"
            ↓ (answer with section references)
Step 3  Check: "Does each statement in this answer follow from the quotes?
        List any that don't."

Splitting extraction from answering keeps the answer grounded, and the check step catches unsupported claims before a person sees them.

When chaining is useful

• Multi-stage work: research, then outline, then draft, then edit.
• Different skills per step: classify a support ticket, then use the prompt for that category.
• Data pipelines: extract fields from a document, validate them in code, then generate a summary.
• Quality control: generate, then critique, then revise.
• Long inputs: summarise each chapter, then combine the summaries.
• Mixing models: a small, fast model for simple steps and a stronger one for hard reasoning, which cuts cost.

Advantages

• Easier to debug: you can see which step failed.
• More reliable: each prompt is simpler.
• Reusable steps across workflows.
• Checks in code between steps.

Drawbacks

• More calls mean more latency and some extra cost.
• Errors early in the chain carry forward, so validate between steps.
• Steps lose context the earlier ones had unless you pass it on.

Chaining and agents In a chain, the developer fixes the sequence. In an agent, the model decides the next step and which tools to call. Fixed chains are more predictable and easier to test, so use them when the steps are known in advance, and agents when they aren't.

A prompt template is a reusable prompt with fixed text and named placeholders that the application fills with each request's data, such as {customer_name}, {document} or {question}. It separates the parts you design and test once from the parts that change every time, the way a parameterised query separates SQL from its values.

Example

System:
You are a support assistant for {company}. Answer only from the help
articles provided. If the answer is not in them, say so and offer to
connect the customer with a person. Reply in {language}.

User:
<help_articles>
{retrieved_articles}
</help_articles>

<customer_message>
{message}
</customer_message>

The application fills the placeholders from its database and search results, then sends the result to the model.

Why use templates

• Consistency: every request follows the tested structure and instructions.
• Separation of concerns: product and domain experts can improve wording without changing code; developers handle data.
• Testing: the template is evaluated against a set of variable values, and every version can be compared with the last.
• Version control: templates live in files or a prompt registry with versions, so you know which prompt produced which output and can roll back.
• Reuse: one template serves many products, languages or customers.
• Cost: a fixed prefix at the start of the template can be cached across requests, lowering cost and latency.

Designing good templates

• Put stable instructions first and variable content later, which helps caching and keeps the structure readable.
• Wrap each variable in clear delimiters (XML tags or headings), so the model knows where data starts and ends.
• Name variables for what they hold.
• Decide what happens when a variable is empty: leave out the section, or say "no history available", rather than sending a blank tag.
• Use a real template engine (such as Jinja) or the prompt features of your framework, for loops over examples and conditional sections.

Security Variables often contain text from users or documents. Treat it as untrusted: delimit it, never place it where it reads as instructions, escape characters that would break your delimiters, and limit length. Delimiters reduce prompt injection but don't eliminate it, so the application must still restrict what the model's output can trigger.

Tooling Most model providers and frameworks support templates: system messages with variables in provider consoles, prompt objects in LangChain and LlamaIndex, and prompt management in MLflow and similar tools, often with evaluation built in.

The context window, or context length, is the maximum number of tokens a model can take into account in one request: the system prompt, the conversation so far, any documents or tool results, and the answer it writes. Anything beyond it can't be seen, and even inside it, more text doesn't automatically mean better answers.

Tokens, not words Models read text as tokens, pieces of words. In English a token is roughly three quarters of a word on average, but code, numbers and many other languages use more tokens per word. Limits, prices and speed are all counted in tokens.

Input and output share the window If a model's window is large and the prompt nearly fills it, there is little room left for the answer. APIs also set a separate maximum on output tokens.

What happens at the limit The API rejects a request that is too long, or a chat application drops or summarises older messages. The model has no memory of anything outside the window: a long conversation can "forget" early instructions once they are cut.

Bigger isn't automatically better Windows have grown from a few thousand tokens to hundreds of thousands or more. But accuracy and recall decline as the context fills, which Anthropic calls context rot, and research found models use information at the start and end of a long context better than information in the middle. Long prompts also cost more and take longer.

Impact on prompt design

1. Budget the window: decide how much goes to instructions, examples, retrieved content, history and the answer.
2. Retrieve, don't dump: with retrieval-augmented generation, only the relevant passages of a large knowledge base go into each request.
3. Order matters: put long documents first and the question and instructions after them, and label each document.
4. Keep instructions in the system prompt, so they stay in place however long the conversation gets.
5. Manage history: summarise older turns, remove tool output no longer needed, and for long-running agents compact the context and keep notes outside it.
6. Split large jobs: process documents in sections and combine the results.
7. Cache stable prefixes: prompt caching reduces the cost of repeating a long system prompt or document across requests.
8. Test with realistic lengths, checking that details from the middle of long inputs are used.

Why it matters in practice The context window decides whether a task can be done in one call or needs retrieval, chunking or several steps. Designing for it is less about squeezing everything in and more about choosing what the model needs to see for this request.

Prompt engineering is moving from clever wording towards engineering systems: deciding what context a model sees, wiring it to tools, constraining its outputs and measuring results with evaluations. I prepare by building those skills, not by collecting prompt tricks.

Trends that are already under way

1. Context engineering For agents that work over many steps, the hard part is choosing what goes into the context window at each step: retrieved documents, tool results, memory and summaries. Anthropic describes this as context engineering, the successor to writing a single prompt.

2. Reasoning models Models that think before answering need less instruction on how to reason. Their makers advise short, direct prompts and say step-by-step instructions are unnecessary or can hurt. Prompting shifts to stating goals, constraints and what good output looks like.

3. Agents and tool use Models increasingly call APIs, run code and act on systems, in loops of reasoning and action (the ReAct pattern). Prompting now includes writing tool descriptions, setting permissions and deciding when a person must approve an action. Standards such as the Model Context Protocol make tools reusable across applications.

4. Structured outputs APIs can force answers to match a JSON schema, so format instructions and parsing workarounds matter less, and model outputs plug directly into software.

5. Automatic prompt optimisation Frameworks such as DSPy, and methods where a model proposes and scores its own instructions, tune prompts against a metric. The human role moves to defining the task, the data and the metric.

6. Evaluation as the core skill As models change every few months, the lasting asset is a good test set and grading method that shows whether a change helped.

7. Multimodal inputs Prompts increasingly mix text with images, audio, documents and screen content.

8. Security Prompt injection, especially through documents, web pages and tool results, is a central design concern for agents, with defences at the application level.

How I'm preparing

• Building evaluation sets and graders for every project, and treating prompts as versioned code.
• Learning retrieval, embeddings and agent frameworks, not just prompt wording.
• Practising with structured outputs and tool definitions.
• Reading makers' prompting guides when new model generations arrive, since advice changes.
• Following security guidance such as the OWASP list for LLM applications.
• Keeping the fundamentals: clear writing, understanding the domain, and knowing what a good answer looks like. Those don't go out of date.

Chain-of-thought (CoT) prompting gets a model to write out intermediate reasoning steps before its final answer. Working through a problem step by step improves accuracy on multi-step tasks such as maths word problems, logic and planning, because each step is generated with the earlier ones in view.

Origins Wei and colleagues showed that adding worked examples with reasoning steps (few-shot CoT) substantially improved large models on arithmetic, commonsense and symbolic reasoning benchmarks, while small models gained little. Kojima and colleagues found that simply adding "Let's think step by step" (zero-shot CoT) also helped.

Example

Question: A warehouse has 3 shelves with 48 boxes each. 37 boxes are
shipped and 25 arrive. How many boxes are there now?
Think it through step by step, then give the final number on its own line.

The model writes: 3 × 48 = 144; 144 − 37 = 107; 107 + 25 = 132; answer 132. Asked for the number alone, models without built-in reasoning are more likely to slip.

Why it works A model produces one token at a time with a fixed amount of computation per token. Writing intermediate results gives it more computation for the problem and lets later steps build on earlier ones, instead of jumping straight to an answer.

Variants

• Self-consistency: sample several reasoning paths and take the most common final answer.
• Least-to-most and decomposition: solve subproblems first, then combine.
• ReAct: interleave reasoning with tool calls, the basis of many agents.
• Tree of thoughts: explore and evaluate several reasoning branches.

Reasoning models changed the practice Many current models reason internally before answering, often with a configurable thinking budget, and return the reasoning separately or as a summary. Their makers advise against telling them to "think step by step", which is unnecessary and can hurt; give them the goal and constraints instead. CoT prompts remain useful for models without built-in reasoning, and for making the steps visible to readers.

Limitations

• More output tokens mean more cost and latency.
• Little benefit for simple tasks; sometimes slightly worse.
• The written reasoning isn't guaranteed to be how the model actually reached its answer, and it can reason fluently to a wrong conclusion, so check answers, not just whether the reasoning looks convincing.
• For apps that show only the final answer, ask for the reasoning in a separate section (for example inside tags) and strip it before display.

Significance CoT showed that how a model is asked to work, not only what it is asked, changes results, and it led directly to today's reasoning models.

An embedding is a list of numbers (a vector) that represents the meaning of a piece of text, so that texts with similar meanings have vectors close together. Embedding models create them, and in prompt engineering they are used mainly to find the right content to put into a prompt: relevant documents, similar examples or past conversations.

How embeddings work

• An embedding model turns text such as "How do I reset my password?" into a vector of hundreds or thousands of numbers.
• Distance between vectors, usually cosine similarity, measures how related two texts are. "I forgot my login" lands near the password question even though they share no words.
• Inside a language model, every token also has an embedding, and its layers build richer contextual representations; dedicated embedding models produce one vector for a whole sentence or passage, optimised for search and comparison.

Uses in advanced prompt engineering

1. Retrieval-augmented generation (RAG) Split documents into chunks, embed and store them in a vector database or vector index. For each question, embed it, find the closest chunks and put them into the prompt as context. The model answers from current, specific sources.

2. Dynamic few-shot examples Keep a library of good input and output examples. For each request, choose the examples most similar to the input, so the prompt shows the model the most relevant cases.

3. Semantic caching If a new question is nearly identical to one answered before, reuse the stored answer instead of calling the model.

4. Routing and classification Compare a request's embedding with embeddings of categories or sample requests to send it to the right prompt, tool or team.

5. Memory for assistants and agents Store summaries of past conversations or notes as embeddings and retrieve the relevant ones later.

6. Evaluation and analysis Cluster user questions to find common topics, detect duplicates, or measure how close an answer is to a reference.

Practical points

• Use the same embedding model for documents and queries, and re-embed everything if you change models.
• Chunk size matters: too large dilutes meaning, too small loses context.
• Combine embedding search with keyword search (hybrid search) and a reranker for better results, especially for names, codes and exact terms.
• Choose a model suited to your languages and domain; some providers don't offer embeddings, so applications often use a separate embedding provider.

Zero-shot prompting gives only the instruction; few-shot prompting adds a handful of worked examples of the task; chain-of-thought prompting asks the model to reason through intermediate steps before giving the answer. The first two control what the model sees about the task; the third controls how it produces the answer, and it can be combined with either.

Zero-shot Only the task, with no examples:

Classify the sentiment of this review as positive, negative or mixed.
Review: "Fast delivery, but the charger stopped working after a week."

• Quickest to write and shortest to send.
• Works well for common tasks that modern instruction-tuned models already understand.
• Less reliable when the format, labels or judgement are specific to you.

Few-shot A few input and output pairs before the real input:

Review: "Arrived broken." -> negative
Review: "Exactly as described, great value." -> positive
Review: "Lovely screen, terrible battery." -> mixed
Review: "Fast delivery, but the charger stopped working after a week." ->

• The model infers the pattern, labels and format from the examples; Brown and colleagues popularised the term in 2020, showing large models could learn tasks from examples in the prompt without retraining.
• Pick examples that cover the hard cases and vary in form, or the model copies superficial features such as length or wording.
• Costs more tokens on every call.

Chain-of-thought (CoT) The model writes out intermediate reasoning before the final answer. Wei and colleagues showed that examples containing reasoning steps improved arithmetic and logic results in large models; Kojima and colleagues found that simply adding "Let's think step by step" (zero-shot CoT) helped too.

A shop sells pens in packs of 12. A school needs 150 pens.
How many packs must it buy? Work through it, then give the answer on the last line.

• Helps multi-step maths, logic and planning; adds little for simple lookups or classification.
• Costs more output tokens and time.
• The written reasoning is not a guaranteed account of how the model reached the answer.

How this has changed Reasoning models now think through problems internally before answering. Their makers advise simple, direct prompts for them, and say that telling them to "think step by step" is unnecessary and can even hurt. CoT prompting still matters for models without built-in reasoning.

In practice Start zero-shot; add examples when the format or judgement isn't right; ask for step-by-step reasoning (or use a reasoning model) when the task needs several steps of logic.

Use the language model to write and explain the checks, and to judge free text that rules cannot, but let ordinary SQL rules decide pass or fail for anything a rule can express. On Databricks that means: rules as expectations or constraints, and ai_query or the AI functions, with carefully written prompts, for the fuzzy cases.

1. Generate rules, then review them

Give the model the table's schema, a sample of rows and what the data means, and ask for checks as SQL:

Here is the schema of main.sales.orders and 20 sample rows. Propose data quality rules as SQL boolean expressions, one per line, with a name and a reason: required fields, valid ranges, allowed values, and relationships between columns. Use only these columns.

A good answer gives rules such as order_total >= 0 or ship_date >= order_date. An engineer reads each one and puts the accepted ones in pipeline expectations or CHECK constraints, so they run every time, deterministically and for free.

2. Judge what rules cannot

Some checks need reading: is a product description in English, does a free-text address look complete, does a support note contain personal data? ai_query runs a prompt over each row:

SELECT order_id,
       ai_query(
         'my-chat-endpoint',
         CONCAT('Is this a complete postal address with street, city and postcode? ',
                'Answer only with JSON. Address: ', shipping_address),
         responseFormat => 'STRUCT<complete:BOOLEAN, missing:STRING>',
         failOnError => false
       ) AS check
FROM main.sales.orders
WHERE order_date = current_date();

Prompt engineering matters here:

• Constrain the output with responseFormat, so the answer is a typed field you can filter on, not prose.
• Give clear criteria and, where useful, a few labelled examples of good and bad rows.
• Use failOnError => false so one failed call returns an error message instead of failing the whole query.

Task-specific functions cover common cases without a custom prompt: ai_classify for categories, ai_mask to hide personal data, ai_analyze_sentiment, ai_extract for fields.

3. Explain failures to people

Summarising the day's expectation metrics, for example "drop rate of valid_email rose from 0.1% to 4% after 09:00", is a task a model does well, as long as it is fed the actual numbers from the pipeline's event log rather than asked to guess.

Limits to state in an interview

• A model can be wrong and is not deterministic; never let it be the only check on data that must be right.
• Every call costs money and time, so run it on new or changed rows, not the whole table each day.
• Measure it: label a few hundred rows by hand and compare the model's verdicts before relying on them.

In Databricks the built-in way is Genie Code, the workspace's AI assistant: it writes Spark SQL from a description, explains or fixes a query, and sees your Unity Catalog tables and columns, so its SQL uses real names. Prompt engineering is what makes its answers right: name the tables, say what the result should look like, and check what comes back.

1. Generate a query

A weak prompt: "average sales per region last quarter". The model has to guess the table, the date column and what "last quarter" means.

A better prompt:

Using main.sales.orders (columns region, order_date, amount), write Spark SQL for the average order amount per region for the previous calendar quarter, relative to today. Sort by average, highest first.

SELECT region, AVG(amount) AS avg_amount
FROM main.sales.orders
WHERE order_date >= add_months(date_trunc('QUARTER', current_date()), -3)
  AND order_date <  date_trunc('QUARTER', current_date())
GROUP BY region
ORDER BY avg_amount DESC;

Asking for dates relative to today, not fixed ones, keeps the query correct next quarter.

2. Explain a query

Select a query and ask Genie Code to explain it, or ask for something specific: "Explain step by step what this query returns, and point out anything that could produce duplicates or drop rows." Asking for risks, not just a summary, gets a more useful answer.

3. Fix and improve

When a query fails, Genie Code can read the error and propose a fix. It can also be asked to rewrite a query: "rewrite with window functions instead of the self-join", or "why is this slow? Here is the query profile".

4. Generate SQL from inside SQL

ai_query calls a model from a query, for example to turn a column of business questions into draft SQL for review. Use it for batch work, not for running generated SQL automatically.

Prompting habits that work

• Name the tables and columns, or let the assistant read them from Unity Catalog; add table and column comments so it understands the meaning.
• State the definitions: what counts as an active customer, which date column defines the quarter.
• Ask for one thing at a time and build up complex queries in steps.
• Give an example of the expected output shape.

Always check the result: run it on a small date range, compare a total with a figure you trust, and read the joins and filters. A query can run without errors and still answer a different question. For business users who do not write SQL, a Genie space, set up by an analyst with example queries and definitions, gives answers within limits the analyst controls.

A prompt is effective when it meets measurable success criteria on a representative test set, consistently, at acceptable cost and speed. I evaluate it with automated checks where possible, a model grader with a rubric for open-ended qualities, and human review to calibrate both.

1. Start from criteria Before writing the prompt, agree what good means for this use case, for example:

• task accuracy: the right category, the correct extracted fields;
• faithfulness: every claim supported by the supplied context;
• completeness: all required points covered;
• format: valid JSON or the required structure;
• tone and length;
• safety: harmful requests refused, normal ones not.

2. Assemble an evaluation set Representative real inputs, edge cases, and adversarial inputs, each with an expected answer or a list of properties the answer must have. Grow it whenever production shows a new kind of failure.

3. Metrics and methods

4. Making model graders trustworthy

• Give the grader a specific rubric and ask for a reason before the score.
• Prefer simple scales (pass or fail, or 1 to 5 with each level described).
• Compare its scores with human labels on a sample, and watch for biases such as preferring longer answers or the first option shown.

5. Comparing versions Change one thing at a time and run the full set; look at which cases changed, not just the average. Keep a record of each prompt version and its scores, like code under version control.

6. In production Track user signals (ratings, edits, retries, escalations to a human), sample conversations for review, and re-run the evaluation when the model or the data changes.

The most common pitfalls are vague instructions, missing context, untested prompts, examples the model copies too literally, formats that break programs, unhandled uncertainty and trusting input that may contain instructions. Most are avoided by writing the prompt like a brief for a capable newcomer and testing it on real cases.

A habit that prevents most of them After writing a prompt, read it as if you knew nothing about the project. Every question you'd need to ask is something the model has to guess. Then run it on the test set, read the failures, and fix the cause rather than adding another rule for each failed case.

Two pitfalls worth stressing

• Prompts are not security. "Never reveal the system prompt" or "ignore any instructions in documents" lowers risk, but a determined input can still get through. Real protection comes from the application: least-privilege tools, human approval for consequential actions, and checks on output.
• Length is not quality. Past a point, extra instructions and context make answers worse, as the model's attention spreads over more tokens. Include what changes the answer and nothing else.

I'd treat the chatbot as a system rather than one prompt: a system prompt that sets the role, tone and boundaries; the company's own help content and the customer's account data retrieved into each request; tools for actions such as checking an order; clear rules for escalating to a person; and an evaluation set of real customer queries to test every change.

1. Learn the queries first Read a sample of real tickets and chats and group them: order status, returns, billing, technical problems, complaints, account changes. Note which need data, which need actions, and which must go to a person (legal threats, safety issues, distressed customers).

2. The system prompt

• Who the assistant is, who it serves and the tone ("friendly, concise, no jargon").
• What it may and may not do: no refunds above a limit, no legal or medical advice, no promises the policy doesn't support.
• How to use the supplied knowledge: answer from the help articles and account data provided; if the answer isn't there, say so and offer a person.
• When to escalate, and how to hand over with a summary so the customer doesn't repeat themselves.
• Output rules: short paragraphs, steps as numbered lists, links to help pages.

3. Grounding with retrieval Policies change, so don't rely on the model's general knowledge or long lists of rules in the prompt. Retrieve the relevant help articles for each question (retrieval-augmented generation) and include the customer's order or account details when they are signed in. Ask the model to answer only from them.

4. Tools for actions Give the model functions such as get_order_status(order_id) or start_return(order_id, reason), with clear descriptions. The application, not the model, checks permissions, and sensitive actions ask the customer to confirm.

5. Handling variety

• Unclear questions: ask one clarifying question rather than guess.
• Several issues in one message: address each in turn.
• Angry customers: acknowledge the problem, avoid defensive language, move to a solution or a person.
• Other languages: reply in the customer's language.
• Off-topic or abusive input: steer back politely, or end the chat by the stated policy.

6. Safety and security Customers (or text they paste) may try to make the bot ignore its rules. Keep account data and actions behind permission checks, never put secrets in the prompt, and filter outputs for personal data of other customers.

7. Test and improve Build a test set from real conversations, including tricky and adversarial ones, and score answers for correctness against policy, tone, and correct escalation. After launch, review conversations that ended in escalation or low ratings, and add them to the tests.

Tell the model plainly what fair, appropriate output looks like for the task, give it balanced examples, remove the parts of the input that could invite bias, and then measure the outputs across groups, because a prompt alone can't guarantee unbiased results.

1. State the standard explicitly "Assess each candidate only on the listed skills and experience. Do not consider or infer age, gender, ethnicity, nationality or family status." Specific instructions work better than "be unbiased".

2. Remove what shouldn't matter If a decision mustn't depend on names, photos, addresses or ages, take them out of the input. The model can't be influenced by what it never sees, and this is far more reliable than asking it to ignore them.

3. Use balanced examples Few-shot examples teach patterns. If every example of a "strong leader" is a man, or every complaint example comes from one region, the model learns that. Vary the people, names, regions and styles in examples, and check that labels don't follow group lines.

4. Ask neutral questions "Why are older employees worse with technology?" invites a biased answer. "What factors affect how quickly employees adopt new technology?" doesn't. Also tell the model to question false premises rather than accept them.

5. Ask for multiple perspectives where it fits For contested topics: "Present the main viewpoints fairly, with the strongest argument for each, and note where evidence is disputed."

6. Define appropriate content for the context Say who the audience is ("children aged 8 to 12", "professional legal readers") and what to do with requests outside the boundaries: decline briefly and offer something useful instead.

7. Structure the decision For assessments, ask the model to score against a written rubric, criterion by criterion, with evidence quoted from the input. That makes reasoning visible and reviewable.

8. Test for bias

• Counterfactual tests: run the same input with only a name, gender or other attribute changed, and compare outputs.
• Compare outcome rates (scores, approvals, sentiment) across groups on a realistic test set.
• Include adversarial prompts that try to provoke offensive content.

9. Add controls outside the prompt Prompting reduces risk but doesn't remove it. Use content moderation filters on input and output, human review for consequential decisions such as hiring, lending or medical matters, and clear routes for users to report problems. In some jurisdictions, automated decisions about people carry legal duties, so involve legal and compliance teams early.

Interviewers want a specific story that shows how you think: what the goal was, why it was hard, what you tried, how you measured it and what you learned. Use the STAR structure (situation, task, action, result) and give numbers. Here is an example of a strong answer.

Situation An operations team received thousands of supplier invoices a month as PDFs in many layouts and several languages. Staff typed the key fields into the finance system by hand.

Task Design a prompt that extracted supplier name, invoice number, dates, line items, tax and totals into JSON that the finance system could import, with at least 98% of fields correct, because errors meant wrong payments.

What made it challenging

• Layouts varied widely, and some invoices spread line items over several pages.
• Dates came in different formats (03/04 is March in some countries and April in others).
• Totals sometimes didn't match the line items, and the model would "fix" them by inventing values.
• Some fields were simply missing, and early versions filled them with plausible guesses.
• Any parsing failure stopped the import.

Actions

1. Built a test set of 300 real invoices covering every supplier type and language, with fields checked by the finance team.
2. Used the API's structured outputs with a JSON schema, which removed parsing failures entirely.
3. Rewrote the instructions to copy values exactly as printed, return null for missing fields, and never calculate or correct values.
4. Added the supplier's country to the input so dates could be read correctly, and asked for dates in ISO format.
5. Moved arithmetic out of the model: code checked that line items summed to the total, and invoices that failed went to a person with the discrepancy highlighted.
6. Added three examples of difficult layouts, chosen from the failures.
7. Re-ran the full test set after every change and tracked accuracy per field.

Result Field accuracy rose from about 85% with the first prompt to over 98%, with the remaining cases routed to review. Manual entry time fell by roughly three quarters.

What I learned

• Let the model do what it is good at (reading varied layouts) and code do the rest (arithmetic, validation).
• Giving the model permission to return null reduced invented values more than any other change.
• A labelled test set turned arguments about wording into measurable decisions.

When you answer, use your own project. If you're early in your career, a personal or course project is fine, as long as you can explain the difficulty, your reasoning and how you knew it worked.

Prompt tuning learns a "soft prompt": a small set of trainable vectors (embeddings) placed in front of the input, adjusted by gradient descent on task examples while the model's own weights stay frozen. Traditional prompt engineering writes the prompt in words by hand and trains nothing.

How prompt tuning works

1. Choose a number of virtual tokens, for example 20.
2. Prepend 20 trainable embedding vectors to the embedded input; they don't correspond to real words.
3. Train on labelled examples. Only those vectors change; the model is frozen.
4. At inference, prepend the learned vectors for that task.

Lester and colleagues showed that as models get larger, prompt tuning approaches the quality of full fine-tuning while training only a tiny fraction of the parameters. Each task needs only its own small soft prompt, so one frozen model can serve many tasks.

Related methods

• Prefix tuning (Li and Liang) learns vectors inserted at every layer of the model, not only the input, which gives more control.
• P-tuning uses a small network to produce the prompt vectors.
• LoRA and other adapters train small weight matrices inside the model; they are the most common parameter-efficient fine-tuning method today. The Hugging Face PEFT library implements all of these.

Comparison with prompt engineering

When to use which

• Hosted, closed models are reached through an API that accepts text, so prompt tuning isn't possible there; prompt engineering, retrieval and the provider's fine-tuning service are the options.
• With an open-weights model you run yourself, prompt tuning or LoRA can adapt it cheaply to many tasks.

Experience In an interview, say honestly what you have done. A good answer might be: "I've used LoRA through the PEFT library on an open model for a classification task, and compared it with a few-shot prompt on the same test set. The prompt got close enough that we kept it, because it was easier to maintain." If you haven't used it, explain how you'd decide whether it was worth trying.

Temperature controls how random a language model's choice of the next token is. Low temperature makes the model pick the most likely tokens almost every time, giving focused, consistent output; high temperature spreads the choice to less likely tokens, giving more varied, creative and sometimes less accurate output.

How it works At each step the model produces a score (logit) for every possible next token. Before turning scores into probabilities, they are divided by the temperature:

• Temperature below 1 sharpens the distribution: likely tokens become even more likely.
• Temperature of 1 leaves the model's distribution unchanged.
• Temperature above 1 flattens it: unlikely tokens get a bigger share.
• Temperature near 0 approaches greedy decoding, always picking the top token.

Effects on output

Things that are often misunderstood

• Temperature 0 is not fully deterministic. Hosted APIs can still give different outputs for the same input, because of how computations are batched and run.
• Low temperature doesn't make answers true. A model can be consistently wrong; accuracy comes from good context and instructions.
• High temperature isn't the only route to creativity. Asking for many distinct ideas, or giving unusual constraints, often works better than turning randomness up.
• Ranges differ by provider: for example 0 to 1 in Anthropic's API and 0 to 2 in OpenAI's.
• Some models don't let you change it. Many newer models, especially reasoning models, fix temperature and reject other values; prompt instructions then do the work.

Related settings Top-p (nucleus) and top-k sampling limit which tokens can be chosen at all, while temperature reshapes the probabilities. Providers generally advise adjusting temperature or top-p, not both.

Choosing a value Start with the provider's default, then test a lower value for tasks needing consistency and a higher value for creative ones, scoring outputs on a test set. Running each test case several times shows how much the variation actually matters for your task.

A hallucination is output that sounds confident and plausible but is false, or isn't supported by the sources the model was given: an invented citation, a wrong figure, a nonexistent API function. Prompting reduces it by grounding answers in supplied sources, allowing the model to say it doesn't know, asking for quotes and citations, and checking answers, but it can't eliminate it, so applications need verification too.

Why models hallucinate A language model generates the most plausible continuation of text. It has no built-in check against reality, and its training rewards fluent answers. Hallucinations are most likely for rare facts, recent events after its training data, exact numbers, names, citations and questions that assume something false.

Prompting techniques that help

1. Ground answers in supplied context Retrieve relevant documents and instruct: "Answer only from the documents below. If they don't contain the answer, say so." This turns a memory question into a reading task.

2. Permit uncertainty Tell the model explicitly that "I don't know" or "the documents don't say" is an acceptable answer. Without that permission, models tend to guess.

3. Quotes first, then answer For long documents: "First extract the exact quotes relevant to the question, then answer using only those quotes." Unsupported claims become easy to spot.

4. Citations Ask for a source for each claim (document title or section), and have code check that each cited quote really appears in the source.

5. Ask it to verify A second step: "For each statement in this answer, find a supporting quote in the documents; remove statements without one."

6. Avoid leading questions "Why did the 2019 update remove feature X?" invites an invented story if it didn't. Ask whether it happened, and tell the model to correct false premises.

7. Narrow the task Specific questions with clear scope hallucinate less than open-ended requests.

8. Consistency checks Generate several answers; if they disagree on a fact, treat it as unreliable.

Controls beyond the prompt

• Tools for exact data: a database query or calculator rather than the model's memory.
• Structured outputs so fields can be validated.
• Human review for high-stakes uses (legal, medical, financial).
• Evaluations measuring faithfulness to sources, run before every change.
• Showing users the sources so they can check.

Hallucination rates have fallen as models improved, but no current model is free of them, so design the application as if some answers will be wrong.

The terms overlap and many people use them interchangeably, but when a distinction is made, prompt design is the craft of writing a prompt (wording, structure, examples, format) and prompt engineering is the wider discipline of building, testing and running prompts reliably in a system (evaluation, versioning, retrieval, tools, monitoring and cost).

Prompt design

• Writing clear instructions for a specific task.
• Choosing a role, tone and audience.
• Structuring the prompt with sections and delimiters.
• Selecting and writing examples.
• Specifying the output format.
• Iterating on wording by looking at outputs. It is close to writing and communication: the skill of giving a capable reader an excellent brief.

Prompt engineering Everything in design, plus:

• Success criteria and evaluation: test sets, graders, metrics, regression testing.
• System architecture: prompt chains, routing, retrieval-augmented generation, tool definitions, agents.
• Context engineering: deciding what information enters the context window at each step.
• Parameters: model choice, sampling settings, structured outputs.
• Operations: templates, version control, logging, monitoring, prompt caching, cost and latency.
• Security and safety: defending against prompt injection, output validation, guardrails.
• Maintenance: re-testing when models change.

An analogy Prompt design is like writing a good function; prompt engineering is like software engineering: tests, architecture, deployment and maintenance around the function.

Example Designing a prompt that summarises customer calls well is prompt design. Building the system that transcribes thousands of calls, retrieves the account history, runs the summary prompt, validates the JSON, measures summary quality against a reviewed set, tracks cost and alerts on drift is prompt engineering.

Are they interchangeable? In everyday conversation, yes, mostly. In job descriptions and team roles, "prompt engineer" usually means the broader, technical role, while "prompt design" is used more often in content, UX and conversation design contexts. In an interview, acknowledge the overlap, give the distinction, and show you can do both: write clear prompts and prove with evaluation that they work.

Give the model the full article, say who the summary is for and what it must cover, require that every statement comes from the article, and fix the length and structure. Then check the summaries against the source.

The prompt

You are summarising a technical article for energy-sector analysts who read
many papers and need the facts quickly.

<article>
{full article text}
</article>

Write a summary of the article above.

Requirements:
- 200 to 250 words, in four short sections: Main finding, Method, Key figures, Limitations.
- Use only information stated in the article. Do not add background knowledge,
  forecasts or opinions.
- Keep every figure exactly as written, with its unit and what it measures
  (for example "18.5% cell efficiency in laboratory tests").
- Use the article's technical terms, briefly explaining any that a
  non-specialist analyst might not know.
- If the article does not state its limitations, write "Not stated".
- After the summary, list each figure you used with a short quote from the
  article that supports it.

Considerations for accuracy

• Source only: summarisation errors usually come from the model adding plausible facts it knows about the topic. The instruction to use only the article, plus quotes for figures, keeps it honest and makes checking quick.
• Numbers and units: renewable energy articles are full of capacities (MW vs MWh), efficiencies and costs. Ask for exact figures with units, and check them; this is where summaries most often go wrong.
• Hedging preserved: "may reduce costs in pilot sites" must not become "reduces costs". Ask the model to keep the article's level of certainty.
• Verification: for important use, a second step can compare each sentence of the summary with the article and flag anything unsupported.

Considerations for completeness

• Fixed sections ensure the method and limitations aren't dropped in favour of the headline result.
• Placement: put the long article before the instructions. Models make better use of long inputs when the question comes after them, and they can miss details buried in the middle of very long texts. For very long articles, summarise section by section, then combine.
• Audience: the same article needs a different summary for analysts, executives or the public. Saying who reads it decides the detail and vocabulary.

Testing the prompt Run it on several articles with different structures (a study, a policy report, a product announcement), and compare the output with a human-written summary or a checklist of key points. Check every figure. Adjust the sections or length if important points are regularly missed.

If the summaries feed a database or a newsletter, ask for JSON with one field per section instead of prose, so missing sections are caught by code before anyone reads the result.

I start from the outcome, not the prompt: understand the task and its users, define how success will be measured, collect real test cases, then write a simple first prompt and improve it in measured steps before building it into the product with monitoring.

1. Understand the problem

• What job does the output do, for whom, and what happens when it is wrong? A typo-fixer and a medical triage assistant need very different care.
• What inputs will arrive, in what volume, and from whom (trusted staff or the public)?
• Is a language model the right tool? Some tasks are better done by rules, search or a database query.

2. Define success criteria Measurable targets: accuracy on a labelled set, required fields present, format valid, latency and cost limits, safety requirements. Agree them with the people who own the outcome.

3. Gather data and examples

• Real or realistic inputs, including messy and edge cases.
• Expected outputs written or approved by domain experts.
• Adversarial cases: attempts to break rules, inject instructions or extract data. Split them into examples that may go into the prompt and a separate test set.

4. Choose the model and setup Start with a capable model to learn what's possible, then try smaller, cheaper ones once you have tests that show whether quality holds. Decide whether retrieval, tools or structured outputs are needed.

5. Write a simple baseline prompt Clear task, context, output format. Run it on the test set and record the scores. Avoid adding techniques before you know what fails.

6. Analyse failures and iterate Read the failures and group them: missing information, misunderstood instruction, format errors, invented facts. Fix the cause: add context, clarify an instruction, add a targeted example, split the task into steps. Change one thing at a time, re-run all tests, and keep a versioned history of prompts and scores.

7. Review with stakeholders Domain experts review samples of outputs, especially borderline ones; their feedback often reveals criteria that weren't written down.

8. Harden for production

• Templates with variables instead of string concatenation.
• Defences against prompt injection, input limits, output validation, fallbacks for errors and timeouts.
• Prompt caching for long, stable prefixes to cut cost and latency.
• Pin the model version.

9. Launch and monitor Log inputs, outputs, cost and latency (respecting privacy), collect user feedback, sample outputs for review, and turn new failures into test cases.

10. Maintain Re-run the evaluation before any model upgrade or prompt change, and revisit the prompt when the product or its users change. Document what each part of the prompt is for, so the next person knows what is safe to change.

I'd give the model a clear creative brief: the age group and reading level, the kind of story, the values and boundaries, and the format of each idea. Then I'd push for variety by asking for many distinct ideas across defined dimensions, and use a higher sampling temperature where the model allows it.

1. Define the audience precisely "Picture book for ages 4 to 6, read aloud by a parent" and "chapter book for ages 8 to 10, read alone" need different ideas: length, vocabulary, complexity of plot and themes all change.

2. Set the creative frame

• Genre and tone: gentle bedtime story, funny adventure, mystery.
• Themes or values to explore: friendship, trying something new, kindness to animals.
• Setting constraints, if any: a city, the sea, a specific culture handled respectfully.

3. State what to avoid No frightening violence, nothing that would be unsafe if copied (like hiding in a fridge), no stereotypes, no existing characters or plots from well-known books. Positive guidance ("the conflict should be resolved through problem-solving and empathy") works better than a long list of prohibitions.

4. Ask for range, not one answer "Give 10 ideas that differ from each other in setting, main character type (child, animal, object, imaginary creature) and central problem." Asking for many varied options counters the model's tendency towards the most typical story.

5. Structure each idea

For each idea give:
- Title
- Main character, in one sentence
- The problem they face
- How it is resolved
- What the child reader takes away
- Why it suits ages 4 to 6

6. Use examples carefully One or two sample ideas show the level of detail and tone, but the model tends to copy them. Say they show the format only, or skip them once the format is right.

7. Tune randomness A higher temperature produces more surprising ideas; a lower one, safer and more predictable ones. Many newer models fix sampling settings, so asking explicitly for unusual or unexpected ideas does the same work in the prompt.

8. Iterate in stages Generate many ideas, pick a few, then ask the model to develop one in more depth, suggest twists, or check it for age appropriateness. A separate review prompt ("check this idea against these guidelines for ages 4 to 6") catches problems the creative prompt missed.

9. Keep a human in charge Treat the output as brainstorming. An author or editor chooses, shapes and checks the ideas, including for originality, since a model can echo existing stories without saying so.

Context length limits how much text (instructions, conversation history, documents and the answer itself) a model can handle in one request, and even within that limit, quality tends to drop as the context grows. So prompt design for long inputs is about choosing what goes in, placing it well and splitting work that doesn't need to be done in one pass.

How context length affects design

• Hard limit: the context window is measured in tokens and covers input and output together. If a document doesn't fit, it must be shortened, split or retrieved in parts.
• Quality, not just capacity: as the number of tokens grows, models recall details less accurately, which Anthropic calls context rot. Research on long contexts also found models use information at the beginning and end better than in the middle.
• Cost and latency: every token is paid for and processed on every request, so a long prompt repeated across many calls is expensive and slow.

Strategies

1. Include only what changes the answer Cut boilerplate, duplicate documents and instructions the model follows anyway.

2. Retrieve instead of including everything Split documents into chunks, store their embeddings, and add only the most relevant chunks for each question (retrieval-augmented generation).

3. Place content deliberately Put long documents first and the question and instructions after them; this improves answers in long-context tests. Wrap each document in tags with its title and source.

4. Ask for quotes first For long documents, ask the model to extract the relevant quotes, then answer from those. It focuses attention and makes answers checkable.

5. Split the work

• Map-reduce: summarise or analyse each section separately, then combine the results.
• Chains: break a long task into steps, passing only the needed output to the next.

6. Manage conversation history Summarise older turns, drop irrelevant tool results, and keep important facts in a short running note. For agents working over many steps, compaction (summarising the context when it nears the limit) and saving notes outside the context window keep long tasks on track.

7. Reduce repeated cost Prompt caching reuses a long, unchanging prefix (system prompt, reference documents) across requests at lower cost and latency. Put the stable part first and the changing part last.

8. Test at realistic lengths A prompt that works with a two-page document may fail with a two-hundred-page one. Include long inputs in the test set and check whether details from the middle are used.

Larger context windows are useful, but more context isn't automatically better: curated, well-ordered context almost always beats everything pasted in.

Top-k and top-p sampling limit which tokens a language model may choose from at each step. Top-k keeps only the k most likely next tokens; top-p (nucleus sampling) keeps the smallest set of tokens whose probabilities add up to p. Both cut off the long tail of unlikely tokens, so outputs stay varied without turning into nonsense.

Background: how a model picks the next token At each step the model gives a probability for every token in its vocabulary. Always taking the most likely one (greedy decoding) is predictable but often dull and repetitive. Sampling from the full distribution is varied but occasionally picks a very unlikely token that derails the text. Top-k and top-p sit between these.

Top-k sampling

• Sort tokens by probability, keep the top k, renormalise, and sample from them.
• k = 1 is greedy decoding; k = 50 allows much more variety.
• Weakness: a fixed k ignores how confident the model is. After "The capital of France is", almost all probability is on one token, yet k = 50 still allows 49 poor choices; in an open-ended sentence, 50 may be too few.
• Introduced for story generation by Fan and colleagues.

Top-p (nucleus) sampling

• Sort tokens by probability and keep adding them until their total reaches p, for example 0.9; sample from that set.
• The set adapts: when the model is confident it may hold one or two tokens; when many continuations are plausible, it holds many.
• Proposed by Holtzman and colleagues, who showed it produced more human-like text than top-k or pure sampling.
• p = 1.0 turns the filter off; lower values make output more focused.

Related settings

• Temperature reshapes the probabilities before the filter: low values sharpen them towards the most likely tokens, high values flatten them.
• Min-p, available in some open-model libraries, keeps tokens whose probability is at least a fraction of the top token's.

Using them in practice

• Change temperature or top-p, not both: they interact, and most providers advise adjusting one.
• Top-k is mainly exposed by open-model libraries; many hosted APIs offer only temperature and top-p.
• Many newer models, especially reasoning models, fix their sampling settings and reject other values, so check the API reference before relying on them.
• No setting makes output fully deterministic or correct; for reliability, rely on clear prompts, structured outputs and evaluation.

You change a model's behaviour without retraining by changing what it sees on each request: a system prompt that sets its role and rules, examples of the output you want, retrieved knowledge it should use, tools it can call and constraints on the output format. Together these can make a general model act like a specialised one.

1. System prompts A system message sets the model's role, audience, tone, scope and boundaries for the whole conversation: "You are a SQL reviewer for a team using PostgreSQL. Point out correctness and performance problems first, style last. Don't rewrite the whole query unless asked."

2. Few-shot examples Examples of inputs with ideal outputs teach format, tone and judgement that are hard to describe. They are the closest thing to training without training: the model picks up the pattern in context.

3. Retrieval-augmented generation For knowledge the model lacks, such as internal documents, current policies or a customer's data, retrieve the relevant passages and include them. The model then answers from your sources, and updating its knowledge means updating a document, not retraining.

4. Tools Functions the model can call (search, database queries, calculators, APIs) extend what it can do and give it current, exact data.

5. Structured outputs A JSON schema enforced by the API makes the output machine-readable every time.

6. Prompt chaining and routing Different prompts for different kinds of request, chosen by a classifier step, behave like several specialised models.

7. Generation settings Where the model allows it, temperature and similar settings make answers more consistent or more varied.

8. Memory Saved user preferences or summaries of past interactions, added to the prompt, personalise behaviour across sessions.

What prompting can't do well

• Teach a consistent, subtle style across thousands of outputs when it is hard to describe or show in a few examples.
• Make a small, cheap model match a large one on a narrow, high-volume task.
• Shorten very long prompts that are repeated on every request. In those cases, fine-tuning may be worth its cost, ideally after prompting and retrieval have been tried and measured.

Comparison

I'd keep one well-tested set of instructions, tell the model which language to answer in rather than leaving it to guess, ground answers in support content that exists in each language (or can be translated reliably), and test every supported language with native speakers, because quality varies between languages.

1. Detect and set the language Use the customer's chosen language from their account or the interface, or detect it from the message. Then say it explicitly in the system prompt: "Reply in {language}, whatever language the help articles are in." Without this, models sometimes answer in the language of the retrieved documents rather than the customer's.

2. One master prompt Write the instructions once, usually in English, and add the target language as a variable. Modern models follow English instructions well while replying in another language, and one prompt is far easier to maintain than a dozen translations. Translate the prompt only if tests show it helps for a specific language.

3. Knowledge in the right language

• Retrieve help articles in the customer's language when they exist.
• Use a multilingual embedding model, so a question in Spanish can find an English article.
• When only English content exists, let the model answer in the customer's language from the English source, and test that terms are translated correctly.

4. Terminology and locale

• A glossary per language for product names, features and terms that must not be translated, or must be translated one particular way.
• Dates, numbers, currencies, addresses and units in local formats.
• The right formality, such as "tu" or "vous" in French, or polite forms in Japanese, set per language.

5. Examples per language where it matters If tone or formality goes wrong in one language, add an example in that language.

6. Mixed and unclear cases Customers switch languages mid-conversation or mix them. Decide the rule (follow the latest message, or the account setting) and state it. For very short or unclear messages, keep the conversation's language.

7. Escalation Hand over to agents who speak the language, with a summary in the agent's language.

8. Test every language

• Model performance in less widely used languages is usually somewhat lower than in English; makers publish per-language results.
• Build a test set per language, including slang, typos and mixed language, and have native speakers review samples for accuracy, tone and cultural fit.
• Monitor ratings and escalation rates by language after launch.

9. Cost and length Many languages use more tokens than English for the same content, so check context and cost budgets per language.

Treat the model as a drafting assistant under a lawyer's control: ground every draft in the firm's approved templates, clauses and the applicable law supplied in the prompt, forbid invented citations, make it flag uncertainty and missing information, and require qualified review before anything is used. Prompts improve accuracy, but compliance comes from the process around them.

1. Define the scope precisely Each prompt should state the jurisdiction, the governing law, the document type, the parties and their roles, and whose interests the draft protects. "Draft a mutual NDA under the law of England and Wales, for a software company sharing technical information with a potential partner" is far safer than "write an NDA".

2. Ground drafts in approved material

• Supply the firm's templates, clause library and playbook in the prompt, and instruct the model to use those clauses and adapt only the variable parts.
• Include the text of the relevant statutes or regulations when the draft must comply with them, rather than relying on the model's memory of the law, which may be outdated or wrong.

3. Forbid inventing law "Do not cite any case, statute or regulation that isn't in the provided materials. If a legal point needs authority that isn't provided, write [AUTHORITY NEEDED]." An invented citation in a filing misleads the court and harms the client, so this rule matters more than any other.

4. Make gaps visible Ask the model to list missing facts, ambiguous instructions and assumptions it made, with placeholders in square brackets rather than invented names, dates or amounts.

5. Use a structure Require a fixed structure (definitions, obligations, term, termination, liability, governing law, signatures) and have the model check the draft against a checklist of required clauses for that document type.

6. Separate drafting from reviewing A second prompt reviews the draft against the playbook: does each clause match the approved position? Are defined terms used consistently? Are there risky clauses such as unlimited liability? It lists issues for the lawyer rather than silently fixing them.

7. Keep wording precise Tell the model not to paraphrase approved clauses, to use defined terms consistently and to avoid vague words such as "reasonable" unless the template uses them.

8. Protect confidentiality Client documents may be privileged. Use a provider and contract that don't train on your data, follow the firm's data rules, and minimise personal data in prompts.

9. Keep humans accountable A qualified lawyer reviews and approves every document; the tool records which template, materials and prompt version produced each draft.

10. Test and maintain Build a test set of drafting tasks reviewed by lawyers, re-run it when prompts or models change, and update the clause library when the law changes.

Reinforcement learning enters prompt engineering in two ways. It shapes the models themselves: training with human or automated feedback is why modern models follow instructions well. And it can be used to search for better prompts, by treating prompt choice as actions and the quality of the outputs as a reward.

1. Reinforcement learning in training models

• RLHF (reinforcement learning from human feedback): people rank model answers; a reward model learns those preferences; the language model is then trained to produce answers the reward model scores highly. Ouyang and colleagues showed this made models much better at following instructions, which is the foundation of today's chat assistants.
• Preference optimisation without a separate reward model: direct preference optimization (DPO) trains directly on pairs of preferred and rejected answers, and is simpler to run.
• Reinforcement learning from verifiable rewards: training on tasks whose answers can be checked automatically (maths, code that passes tests) produced reasoning models that think step by step on their own.
• Reinforcement fine-tuning as a service: some providers, including Microsoft Foundry for supported models, let customers fine-tune with a grader that scores outputs.

Effect on prompt engineering: models trained this way respond to plain instructions, so prompts no longer need tricks; reasoning models need fewer instructions about how to reason.

2. Reinforcement learning and optimisation for finding prompts

• RLPrompt (Deng and colleagues) trains a small policy to generate prompt text, rewarded by how well the target model performs.
• LLM-based optimisers such as OPRO (Yang and colleagues) let a model propose new prompts based on the scores of earlier ones, an approach close to reinforcement learning in spirit.
• Frameworks such as DSPy optimise instructions and examples against a metric. Discovered prompts can be odd-looking or tied to one model, so they need evaluation like any other prompt.

3. Feedback loops in production The same idea applies informally: user ratings, edits, retries and task success are reward signals. Collecting them, adding failures to the test set and comparing prompt versions in A/B tests improves prompts over time. Feedback is noisy and can be gamed (users rate confident answers highly even when they're wrong), so combine it with expert review.

Summary for an interview Reinforcement learning made models follow prompts; it can also search for prompts automatically; and its loop of try, score and improve is the right mental model for prompt engineering in production, as long as the reward measures what you actually care about.

I'd find out what kind of repetition it is (words looping within one answer, the same answer to different questions, or the same structure every time) and then fix it with clearer instructions, varied examples, better context and, where the model allows, decoding settings such as frequency and presence penalties or sampling.

1. Diagnose the type

• Loops within a response: the same sentence or phrase repeats. Usually a decoding issue, often with greedy or low-temperature decoding on smaller or open models.
• Same content across requests: different inputs get near-identical answers. Usually the prompt dominates the input, or the input isn't reaching the model properly.
• Same structure or phrasing: every answer starts "Great question!" or follows one template. Usually copied from examples or instructions.
• Repetition in long conversations: the model repeats earlier answers as history grows.

2. Prompt techniques

• Explicit instructions: "Don't repeat points already made. Each bullet must add new information." State it positively too: "Vary sentence openings; start directly with the answer."
• Diverse examples: if all few-shot examples share a structure or phrase, outputs will too. Vary them, or say the examples show content, not wording.
• Give more specific context: generic inputs produce generic, similar outputs; include the details that should make each answer different.
• Ask for distinct items: "Give five ideas that differ in X, Y and Z" and "List them, then check none overlap" reduce near-duplicates.
• Manage conversation history: summarise or trim long histories so earlier answers don't get echoed.
• Set length appropriately: asking for more text than the content supports produces padding and repetition.

3. Decoding settings (where the model accepts them)

• Temperature and top-p (nucleus sampling): greedy or very low-temperature decoding is prone to loops; Holtzman and colleagues showed sampling from the nucleus reduces such degeneration.
• Frequency penalty: lowers the probability of tokens in proportion to how often they've appeared, discouraging repeated lines (OpenAI's API, range −2 to 2).
• Presence penalty: lowers the probability of any token that has already appeared, encouraging new topics.
• Repetition penalty and no-repeat n-gram size: in Hugging Face Transformers, for open models; the n-gram setting forbids any sequence of n tokens from occurring twice. Use small values and test: strong penalties make text unnatural, for example avoiding necessary repeated terms such as names or code identifiers. Many newer hosted models fix these settings, leaving the prompt as the main lever.

4. Structural fixes

• Generate items one at a time, passing the list so far and asking for something different.
• Post-process: remove duplicate sentences or items with code, or by comparing embeddings.
• Try another model: some are more prone to repetition than others.

5. Measure Track a repetition metric (repeated n-grams per response, or similarity between answers to different inputs) on a test set before and after each change.

Zero-shot chain-of-thought is asking a model to reason step by step without giving it any worked examples, typically by adding a phrase such as "Let's think step by step." It combines two ideas: zero-shot prompting (instructions only, no examples) and chain-of-thought prompting (intermediate reasoning before the answer).

Origin Kojima and colleagues found that this single phrase substantially improved large models' accuracy on arithmetic and logic benchmarks compared with plain zero-shot prompts, without the effort of writing few-shot reasoning examples. They used two stages: first prompt for the reasoning, then prompt again to extract the final answer from it.

Example

Q: A train leaves at 14:40 and the journey takes 2 hours 35 minutes.
The train is 20 minutes late. When does it arrive?
A: Let's think step by step.

The model reasons: 14:40 + 2:35 = 17:15; plus 20 minutes = 17:35. Then a follow-up such as "Therefore, the arrival time is" extracts the answer, or one prompt asks for the answer on the last line.

How it relates to other terms

• Instruction following: it relies on the model understanding and acting on a plain instruction, the core of zero-shot prompting.
• Prompt chaining: the two-stage version is a small chain, reasoning first, then extraction.
• Self-consistency: sampling several zero-shot reasoning paths and voting on the final answer improves accuracy further.
• Output formatting: asking for the answer in a fixed place, or reasoning inside tags, makes it easy to parse.

When to use it

• Multi-step problems with a model that doesn't reason on its own, when you don't have good examples.
• As a quick baseline before writing few-shot reasoning examples.

Limitations and current practice

• Gains are largest for multi-step tasks and large models; simple tasks gain little.
• More output tokens mean more cost and latency.
• Reasoning models now think internally before answering, and their makers say telling them to think step by step is unnecessary and can hurt. For them, state the goal and constraints clearly instead.
• The visible reasoning can look convincing while being wrong, so check the answers.