The words of AI, in plain English
Every term you meet in AI, agents and data engineering, each explained in one sentence, with what else it is called and what sits next to it. Look a word up, or test yourself and keep track of the words you know.
Look it up
How AI works · beginner
Large Language Model
A model trained on a huge amount of text to predict the next token. Further training turns it into an assistant that follows instructions.
Also called LLM
Next to it
Explore by subject
Model CardKV CacheCarbon FootprintHugging Face TransformersGuardrailVision‑Language ModelTokenizerDALL·EBiasOptimizerParameterWhisperGroundingPrompt InjectionSpeech‑to‑TextFine-TuningNegative PromptGradientSystem PromptModel DeprecationContext WindowKnowledge CutoffAdapterChain of ThoughtHallucinationLarge Language ModelTemperatureToken
AI agents
Models that plan, use tools and act on their own.
LLamaIndexAgentContext EngineeringLarge Action ModelAgent MemoryHaystackHuman in the LoopSubagentCompactionFunction SchemaFunction CallingOAuthLangChainPlanner-Executor PatternModel Context ProtocolReAct PatternObservationAgentic WorkflowMemory ToolTool Use
Data engineering in the AI era
The data a model is only as good as, and the systems that store, move and search it.
Shuffle and Data SkewQuery RewritingSort-Merge JoinUnity CatalogCosine SimilarityVector DatabaseELTZillizVector SearchData QualityLakehouseDelta LakeEmbedding DimensionRAGSentenceTransformersPineconeEmbeddingStar SchemaData PreprocessingSentence EmbeddingData WarehouseIVF‑PQChromaData PipelineWord EmbeddingShardingDataset CardData Lake
Every word, A to Z
S
Safety FilterScaling LawSchema DriftScore‑Based ModelSelf-AttentionSelf‑ConsistencySemantic LayerSentence EmbeddingSentencePieceSentenceTransformersShardingShuffle and Data SkewSlowly Changing DimensionsSmall Language ModelSoraSort-Merge JoinSpark Executor MemorySpark ShuffleSpark SpillSparse Expert ModelSpeculative DecodingSpeech‑to‑TextStable DiffusionStar SchemaState Space ModelStreamingStructured OutputSubagentSubword TokenSupervised Fine‑TuningSystem Prompt
How AI worksintermediate
3D Generation
Creating three‑dimensional content using generative models.
Techniques include NeRF, Gaussian Splatting, and 3D‑aware diffusion.
Next to it
How AI worksadvanced
4‑bit Quantization
Representing weights with 4‑bit integers.
Enables running billion‑parameter models on consumer GPUs; often paired with groupwise scale/zero‑points.
Next to it
Data engineering in the AI eraintermediate
ACID Transactions
Also called ACID, Atomicity, consistency, isolation, durability, Multi-table transactions
The 4 promises a database makes about a change: it happens completely or not at all, leaves valid data, does not see other changes half-done, and survives a crash once committed.
How AI worksintermediate
Adam Optimizer
Also called Adam, AdamW
The most used way to train neural networks. A version of gradient descent that gives every weight its own step size, from a running memory of its past gradients.
How AI worksbeginner
Adapter
A small bottleneck module inserted in each transformer layer to adapt a model to new tasks without updating full weights.
Next to it
Data engineering in the AI eraadvanced
Adaptive Query Execution
Also called AQE, spark.sql.adaptive.enabled, AQEShuffleRead, Skew join optimization, Coalesce shuffle partitions
Spark's way of re-planning a query while it runs: after each shuffle it looks at the real sizes, then merges small partitions, splits skewed ones and can switch a join to a broadcast.
How AI worksadvanced
Adversarial Attack
Input crafted to force a model to err.
Perturbations are often imperceptible to humans but fool models.
Next to it
AI agentsbeginner
Agent
Also called AI Agent
A model that uses tools in a loop, deciding each next step from what the last one returned, until the task is done.
AI agentsintermediate
Agent Memory
Also called AI Memory
The ways an agent keeps what it needs beyond a single context window, such as summaries, notes in files, and saved facts it reads back later. The model itself remembers nothing.
Next to it
AI agentsintermediate
Agentic Workflow
Also called AI Workflow, LLM Workflow, Prompt Chaining, Routing, Evaluator-Optimizer
A fixed sequence of model calls and tools that your code controls, such as classify, then look up, then draft. An agent decides its own next steps; a workflow follows the path you wrote.
How AI worksbeginner
AI watermark
Also called Text watermark, AI text watermark, SynthID Text, textGrain
A hidden pattern a maker puts in what its model generates, so a detector holding the right key can later tell that the model was likely involved.
Next to it
How AI worksadvanced
Alignment
Ensuring model objectives match human values and intent
Alignment research seeks techniques (RLHF, constitutional AI, interpretability) that make model outputs safe, truthful, and beneficial, reducing harmful or biased generations.
Next to it
How AI worksbeginner
Amazon Bedrock
Also called AWS Bedrock, Bedrock
AWS's managed service for calling AI models from many makers, Anthropic, OpenAI, Amazon, DeepSeek and others, through one API inside your AWS account, with AWS permissions, logs and billing.
Software and toolsbeginner
API
Also called Application Programming Interface
Application Programming Interface - a set of protocols and tools for building software applications
An API (Application Programming Interface) is a set of rules, protocols, and tools that allows different software applications to communicate with each other. It defines the methods of communication between various components and specifies how software components should interact.
Next to it
Data engineering in the AI eraintermediate
Approximate Nearest Neighbor Search
Also called ANN search, Approximate nearest neighbour search
Finding the stored vectors closest to a query without comparing it with every one, by using an index that may now and then miss the true closest in exchange for far faster searches.
How AI worksintermediate
Attention Head
One parallel attention mechanism within a multi‑head attention layer.
Each head learns different relational patterns; combining them lets Transformers model diverse dependencies.
Next to it
How AI worksbeginner
Azure OpenAI
Also called Azure OpenAI Service, Azure OpenAI in Foundry Models
OpenAI's models run by Microsoft inside Azure, now part of Microsoft Foundry. Same models and SDK as OpenAI's API, but your data stays with Microsoft, under your Azure agreement.
How AI worksintermediate
Backpropagation
Also called Backprop
An algorithm for computing gradients in deep networks via the chain rule.
Backpropagation propagates error derivatives backward through layers, enabling efficient parameter updates and is foundational to deep learning.
How AI worksbeginner
Batch Processing
Also called Batch API, Message Batches, Batch Inference
Sending many model requests together to be answered within hours instead of seconds, at a lower price; the right choice for any work no one is waiting on.
How AI worksadvanced
Beam Search
An approximate search algorithm that keeps the top N hypotheses at each decoding step
Beam search explores multiple candidate sequences in parallel, pruning to the best‑scoring beams and producing more deterministic outputs than pure sampling, though sometimes less diverse.
Next to it
How AI worksintermediate
BF16
Also called Bfloat16, Brain Floating Point
A 16-bit number format with the same range as 32-bit floats but less precision. Half the memory of FP32, and far less likely than FP16 to overflow, so most large models are trained and shipped in it.
How AI worksbeginner
Bias
A trainable parameter added to the weighted sum of inputs in a neural network layer, allowing the activation function to be shifted.
How AI worksbeginner
Billion‑Parameter Model
An LLM with at least one billion trainable parameters.
Next to it
How AI worksintermediate
Byte Pair Encoding
Also called BPE
A sub‑word tokenization algorithm based on merging frequent symbol pairs
BPE iteratively merges the most common pair of bytes or characters, creating a vocabulary that balances token count and coverage. Widely used in GPT and other Transformer models.
Next to it
How AI worksbeginner
Carbon Footprint
The total greenhouse‑gas emissions produced by training or using an AI model.
How AI worksintermediate
Chain of Thought
Also called Chain-of-Thought Prompting, CoT, Thinking, Reasoning
Having a model write out the steps of its reasoning before it gives an answer. Once a prompting trick, now built into "thinking" models.
Data engineering in the AI eraintermediate
Change Data Capture
Also called CDC
Copying only the rows that were added, changed or deleted in a source database, read from its change log, instead of copying the whole table again.
Next to it
Data engineering in the AI erabeginner
Chroma
An open‑source vector database focused on developer productivity and RAG workflows.
Next to it
Data engineering in the AI eraintermediate
Chunking
Also called Text splitting
Cutting documents into smaller passages before they are embedded for retrieval, so a search returns the part that answers a question rather than a whole file.
Next to it
Software and toolsintermediate
CI/CD
Also called Continuous Integration/Continuous Deployment
Continuous Integration and Continuous Deployment - practices for automating software development workflows
CI/CD stands for Continuous Integration and Continuous Deployment (or Delivery). CI involves automatically integrating code changes from multiple contributors into a shared repository, running automated tests to catch issues early. CD extends this by automatically deploying tested code to production environments, enabling faster and more reliable software releases.
Next to it
How AI worksadvanced
Classifier‑Free Guidance
Also called CFG, Classifier-Free Diffusion Guidance
The trick that makes image generators follow the prompt. At each step the model predicts with and without the prompt, and the difference is pushed further, by an amount called the guidance scale.
AI agentsintermediate
Compaction
Also called Context Compaction, Conversation Summarisation, Context Compression
Replacing the older part of a long conversation with a summary, so an agent can keep working when its context fills up.
How AI worksbeginner
Compute Budget
The total amount of compute resources allocated for training or inference.
Next to it
How AI worksbeginner
Content Moderation
Automated detection of toxic, unsafe, or disallowed content.
LLMs and classifiers flag or block content that violates policy guidelines.
Next to it
AI agentsintermediate
Context Engineering
Choosing what goes into a model's context at each step of a task, what to fetch only when needed, and what to clear, note down or summarise as the context grows.
How AI worksbeginner
Context Window
Also called Context Length
Everything a model can see at once while it writes an answer, counted in tokens. When the window is full, something has to be left out.
Next to it
How AI worksadvanced
ControlNet
Extension for diffusion models that adds condition‑control via additional networks.
Enables fine‑grained control such as pose, edge, or depth maps, while reusing pretrained weights.
Next to it
Data engineering in the AI erabeginner
Cosine Similarity
Also called Cosine distance
A measure of how closely 2 vectors point the same way, from 1 for the same direction to −1 for opposite; with embeddings, a higher score means closer in meaning.
How AI worksadvanced
Cross‑Attention
An attention mechanism that relates one sequence to another
In encoder‑decoder Transformers or multimodal models, cross‑attention lets the decoder focus on encoder outputs (e.g., text attending to image patches), enabling conditional generation.
Next to it
How AI worksbeginner
DALL·E
A model that generates images from natural‑language descriptions
Developed by OpenAI, DALL·E combines a Transformer with a discrete VAE to render imaginative images that correspond to textual prompts, demonstrating the power of multimodal generative models.
Next to it
Data engineering in the AI eraintermediate
Data Contract
An agreement between the team that produces data and the teams that use it, written down and checked, that fixes its columns, meaning and freshness.
Next to it
Data engineering in the AI erabeginner
Data Lake
Cheap storage that holds data of any kind in its raw form, usually as files in cloud object storage, to be cleaned and structured later.
Next to it
Data engineering in the AI eraintermediate
Data Lineage
The record of where a piece of data came from and everything it passed through, from source tables to the dashboards, models and AI answers built on it.
Data engineering in the AI eraintermediate
Data Parallelism
Also called DDP, Distributed Data Parallel, FSDP, ZeRO
Training on several GPUs by giving each a full copy of the model and a different slice of each batch, then averaging their gradients so every copy stays the same.
Data engineering in the AI eraintermediate
Data Partitioning and Clustering
Also called Table partitioning, Partition pruning, Liquid clustering, Clustering key, Z-ordering
Two ways to lay out a large table so a query reads only the files it needs: partitioning splits it by a column such as the date, clustering keeps similar values in the same files.
Data engineering in the AI erabeginner
Data Pipeline
A set of steps that moves data from where it is made to where it is used, cleaning and reshaping it on the way, run on a schedule or as data arrives.
Data engineering in the AI eraadvanced
Data Poisoning
Also called Training Data Poisoning, Backdoor Attack, RAG Poisoning
Planting text in the data a model learns from, or in the documents it searches, so it behaves the way an attacker wants.
Data engineering in the AI erabeginner
Data Preprocessing
The process of cleaning, transforming, and organizing raw data before analysis.
Data preprocessing is a crucial step in any data science project that involves cleaning messy data, handling missing values, encoding categorical variables, and scaling features.
Data engineering in the AI erabeginner
Data Quality
How far data can be trusted: complete, correct, on time and in the expected shape. Pipelines check it with rules and stop or flag rows that fail.
Data engineering in the AI erabeginner
Data Warehouse
A database built for analysis: cleaned, structured data from many systems, kept in one place and shaped for fast reports and dashboards.
Next to it
Data engineering in the AI erabeginner
Dataset Card
Documentation describing a dataset’s composition, collection process, and recommended uses.
Data engineering in the AI erabeginner
Delta Lake
Also called Delta table, Delta tables, Delta Lake table format
An open table format that turns Parquet files in cloud storage into a table with transactions, a version history, schema checks, and updates and deletes, so a data lake behaves like a database.
Data engineering in the AI eraadvanced
Dense Passage Retrieval
Also called DPR, Dense Retrieval, Dual Encoder Retrieval
Finding passages by comparing embeddings, from one encoder for questions and one for passages, instead of matching words. The 2020 DPR paper showed it beating BM25 keyword search on open-domain question answering.
How AI worksadvanced
Diffusion Model
A generative model that iteratively denoises data starting from random noise
Diffusion models learn the reverse of a gradual noising process. By sampling noise and applying the learned denoising steps, they generate high‑quality images and audio. Notable examples include Imagen and Stable Diffusion.
Software and toolsintermediate
Docker
A platform for developing, shipping, and running applications using containerization technology
Docker is a containerization platform that allows developers to package applications and their dependencies into lightweight, portable containers. These containers can run consistently across different environments, from development to production, solving the "it works on my machine" problem.
Next to it
How AI worksadvanced
DreamBooth
Technique to personalize a diffusion model with a few example images.
Fine‑tunes a model to render a specific subject or style in new scenes.
Next to it
Data engineering in the AI erabeginner
ELT
Also called ETL, Extract, load, transform
Loading raw data into the warehouse or lakehouse first and transforming it there, the reverse of the older ETL order, which transformed data before loading it.
Data engineering in the AI erabeginner
Embedding
Also called Vector Embedding, Vector Representation
A list of numbers that stands for the meaning of a piece of text, an image or other data. Things with similar meaning get similar numbers, so a computer can find them.
Data engineering in the AI erabeginner
Embedding Dimension
Number of elements in an embedding vector.
Higher dimensions capture more information but increase storage and compute cost.
Next to it
Data engineering in the AI eraadvanced
Embedding Pipeline
Also called Vector ingestion
The pipeline that turns source documents into searchable vectors: it reads new and changed content, chunks it, embeds each chunk and keeps the vector index in step with the source.
How AI worksbeginner
Energy Efficiency
The ratio of model performance to energy consumed, an important sustainability metric.
How AI worksintermediate
Eval Set
Also called Evaluation Set, Eval Dataset, Golden Dataset, Test Cases for LLMs
The list of test cases an LLM app is graded on after every change. Each case is an input and what a good answer must do, so a score going up or down shows whether the change helped.
Data engineering in the AI eraadvanced
FAISS
Facebook AI Similarity Search library for fast vector similarity.
Open‑source C++/Python library that powers many vector search back‑ends and supports efficient indexes like IVF, HNSW.
Next to it
How AI worksintermediate
Few-shot Learning
Also called Few-shot Prompting, Multishot Prompting
Putting a few worked examples of the task in the prompt, so the model copies their format, labels and tone without any training.
How AI worksintermediate
Fine-Tuning
Also called Fine Tuning, Finetuning
Training an existing model a little further on your own examples so it picks up a task, a format or a style. Often a good prompt does the job first.
How AI worksadvanced
Flash Attention
An algorithm that computes attention using tiling to GPU shared memory.
Provides up to 3× speed‑up and lower memory by avoiding redundant reads/writes.
How AI worksadvanced
FP8
Also called 8-bit floating point, E4M3, E5M2
An 8-bit floating-point number format, in 2 kinds, E4M3 and E5M2. It halves a model's memory against BF16, and in tests it matched 16-bit results.
AI agentsintermediate
Function Calling
Also called Tool Calling
The developer's name for tool use. The model replies with the name of a function and its arguments in a fixed format, and your code runs it.
Next to it
AI agentsintermediate
Function Schema
Also called Tool Schema, Input Schema, Tool Definition
The description of a tool that a model reads before calling it, its name, what it does and a JSON Schema of its arguments. The model fills in the arguments; your code checks them and runs the function.
How AI worksintermediate
Generative Adversarial Network
Also called GAN
A deep learning architecture where two neural networks compete to generate realistic data
A Generative Adversarial Network (GAN) consists of a generator that produces synthetic data and a discriminator that evaluates it against real data. Through this adversarial process, the generator learns to create outputs—images, audio, or text—that are indistinguishable from genuine samples.
How AI worksbeginner
GPT
Also called Generative Pre‑trained Transformer
A Generative Pre‑trained Transformer language model
GPT models are first pre‑trained on massive text corpora with a next‑token objective, then optionally fine‑tuned. They can generate coherent paragraphs, answer questions, and perform diverse language tasks with in‑context learning.
How AI worksbeginner
Gradient
The vector of partial derivatives of the loss function with respect to model parameters, used to update weights during training.
Next to it
How AI worksintermediate
Gradient Checkpointing
Also called Activation Checkpointing, Activation Recomputation
A training trick that saves GPU memory by throwing away most in-between results on the forward pass and recomputing them when the backward pass needs them.
How AI worksbeginner
Gradient Descent
Also called Stochastic Gradient Descent, SGD
The way a neural network learns. Measure how wrong it is, work out which way each weight should move to be less wrong, take a small step that way, repeat.
How AI worksbeginner
Grounding
Also called Grounded Generation, Groundedness, Faithfulness
Making a model answer from sources you give it, such as retrieved documents, and only from them, so every statement can be traced to a passage and a missing answer is admitted, not invented.
How AI worksbeginner
Guardrail
Also called Guardrails
Checks placed around a model that stop or change what goes in and what comes out, such as a filter on user messages or an approval step before an agent acts.
How AI worksbeginner
Guidance Scale
A hyperparameter controlling the strength of conditional guidance in diffusion sampling.
How AI worksbeginner
Half‑Precision
Also called FP16
A 16‑bit floating‑point format used to accelerate training and inference while reducing memory.
Next to it
How AI worksbeginner
Hallucination
Also called Fabrication, Confabulation
When a model states something false as if it were true, such as a wrong date, a made-up quote or a source that does not exist.
AI agentsbeginner
Haystack
An open‑source framework for building end‑to‑end question‑answering and RAG systems.
Data engineering in the AI eraadvanced
HNSW
Also called Hierarchical Navigable Small World, HNSW index
Hierarchical Navigable Small World: a vector index that links each vector to its near neighbours in layers of graphs, so a search hops from coarse to fine and checks only a small share of the vectors.
How AI worksbeginner
Hugging Face Transformers
Also called Transformers Library
A popular Python library providing pretrained models, tokenizer tooling, and training utilities.
Next to it
AI agentsbeginner
Human in the Loop
Also called Human-in-the-loop, HITL
A person approves, corrects or stops an AI system at set points, such as before an agent sends, pays or deletes something.
Next to it
Data engineering in the AI eraintermediate
Hybrid Search
Also called Hybrid Retrieval, BM25 Plus Vector Search, Keyword and Semantic Search
Running a keyword search (usually BM25) and a vector search for the same query and merging the 2 ranked lists, so exact terms like a product code and loose wording like "money back" are both found.
Data engineering in the AI eraintermediate
Idempotent Pipelines and Exactly-Once
Also called Idempotency, Idempotent pipeline, Exactly-once processing, Exactly-once semantics, At-least-once delivery
A pipeline step is idempotent when running it twice gives the same result as running it once. That is what makes retries safe, and it is how systems that deliver a message more than once still count it exactly once.
How AI worksintermediate
In-Context Learning
Also called ICL
A model doing a new task from instructions and examples in its prompt alone, with no change to its weights; it forgets the task when the call ends.
How AI worksbeginner
Inference
Also called Serving
Using a trained model to produce an answer. Training builds the model once; inference is every time someone asks it something, and it is what you pay for per token.
How AI worksintermediate
Inference Latency
Also called Time Per Output Token, TPOT, Prefill and Decode
The time a model takes to produce an answer, split into reading the prompt (prefill, which sets the time to first token) and writing the answer one token at a time (decode).
How AI worksbeginner
Instruction Tuning
Also called Instruction Fine-Tuning
Fine-tuning a pre-trained model on many tasks written as instructions with good answers, so it follows a request instead of just continuing the text.
Data engineering in the AI erabeginner
IVF‑PQ
Also called Inverted File with Product Quantization
A vector‑indexing technique combining clustering (IVF) with compressed sub‑vector codes (PQ).
How AI worksintermediate
Jailbreak
Also called Jailbreaking
A prompt written to talk a model out of its safety rules, so it produces something it was trained or instructed to refuse.
Software and toolsbeginner
JSON
JavaScript Object Notation - a lightweight data interchange format
JSON (JavaScript Object Notation) is a lightweight, text-based data interchange format that is easy for humans to read and write, and easy for machines to parse and generate. Despite its name suggesting a connection to JavaScript, JSON is language-independent and widely used across different programming languages.
Next to it
How AI worksbeginner
Knowledge Cutoff
Also called Training Cutoff, Training Data Cutoff
The date a model's training text ends. The model knows nothing after it unless it is given new text, for example by a web search.
Next to it
How AI worksintermediate
Knowledge Distillation
Also called Distillation, Model distillation, Teacher-student training
Training a small "student" model to copy the outputs of a large "teacher" model, so the small one keeps much of the large one's skill at a fraction of the cost to run.
Software and toolsadvanced
Kubernetes
An open-source container orchestration platform for automating deployment, scaling, and management of containerized applications
Kubernetes (often abbreviated as K8s) is an open-source container orchestration platform that automates the deployment, scaling, and management of containerized applications. It provides a robust framework for running distributed systems resiliently, handling scaling and failover for your applications.
How AI worksadvanced
KV Cache
Also called Key-Value Cache
Work a model saves from the text it has already read, so it is not redone for every new token. Prompt caching keeps that work between requests and charges less for it.
Data engineering in the AI erabeginner
Lakehouse
Also called Data lakehouse
One store for all of an organisation's data: cheap files in cloud storage, as in a data lake, with the tables, transactions and fast queries of a data warehouse on top.
AI agentsintermediate
LangChain
Also called LangGraph, create_agent
An open-source framework for building AI agents: create_agent runs a model that calls tools in a loop, with integrations for most model providers. It runs on LangGraph, the lower-level runtime from the same company.
AI agentsbeginner
Large Action Model
An agentic AI system capable of planning and executing sequences of actions in the physical or digital world.
How AI worksbeginner
Large Language Model
Also called LLM
A model trained on a huge amount of text to predict the next token. Further training turns it into an assistant that follows instructions.
How AI worksbeginner
Latency
Also called LLM Latency, Response Time
How long a user waits for a model's answer, measured as the time to the first token and the time to the last, and judged by the slowest requests, not the average.
How AI worksadvanced
Latent Diffusion
Also called Latent Diffusion Model, LDM
A diffusion process applied in compressed latent space.
Reduces computational cost by operating on lower‑dimensional representations instead of pixel space.
Latent Diffusion Models (LDMs) first encode images into a lower‑dimensional latent space using an autoencoder. The diffusion process then runs in this space, greatly reducing compute while preserving quality, as popularized by Stable Diffusion.
Next to it
How AI worksbeginner
Latent Space
The compressed vector space learned by a model to represent data
GANs, VAEs, and diffusion models encode inputs into a latent space where arithmetic operations correspond to meaningful transformations (e.g., changing image style).
Next to it
How AI worksbeginner
Latent Vector
A vector representing a point in latent space, often sampled to generate content.
Next to it
How AI worksbeginner
Learning Rate
Also called LR, Step Size
The size of the step a model's weights take on each training update. Too high and training blows up, too low and it barely moves.
How AI worksintermediate
LLM as a Judge
Also called LLM-as-a-Judge, LLM Judge, Model-Graded Eval, LLM Grader
Using a language model to grade another model's output against a rubric, for qualities code cannot check. It scales to thousands of answers, but must be checked against a person's verdicts before it is trusted.
How AI worksintermediate
Logit Bias
Also called logit_bias
An API setting that adds a fixed amount to chosen tokens' scores before the model picks one, to make them more likely, less likely or banned outright.
How AI worksadvanced
LoRA
Also called Low-Rank Adaptation, LoRA adapter
A way to fine-tune a model by freezing its weights and training 2 small matrices beside each chosen layer, so a task needs a tiny file instead of a full copy of the model.
How AI worksbeginner
Loss Function
Also called Loss, Cost Function, Objective Function
The formula that scores how wrong a model's predictions are during training, as one number to push down.
How AI worksintermediate
Machine Learning
Also called ML
A subset of artificial intelligence that enables computers to learn and make decisions from data without being explicitly programmed.
Machine Learning is a powerful computational method that allows systems to automatically improve their performance on a specific task through experience. It involves algorithms that can identify patterns in data and make predictions or decisions based on those patterns.
Next to it
Data engineering in the AI eraintermediate
Medallion Architecture
Also called Bronze, silver, gold, Multi-hop architecture
Organising a lakehouse in 3 layers of rising quality: bronze for raw data, silver for cleaned and joined data, and gold for tables shaped for reports and AI.
Next to it
AI agentsbeginner
Memory Tool
A component that stores and retrieves conversational or task‑relevant information for an agent.
Next to it
How AI worksbeginner
Microsoft Foundry
Also called Azure AI Foundry, Azure AI Studio, Foundry
Microsoft's Azure platform for AI apps and agents, formerly Azure AI Foundry and Azure AI Studio: one resource to deploy models from many makers, build agents and govern both.
Data engineering in the AI eraintermediate
Milvus
An open‑source cloud‑native vector database.
Provides distributed indexing, HNSW/IVF, replication, and supports SQL‑like queries via MilvusQL.
Next to it
How AI worksadvanced
Mixture of Experts
Also called MoE
A neural architecture that routes tokens through different expert subnetworks.
Increases parameter count while keeping compute cost manageable; examples include Switch Transformer and Mixtral.
Next to it
How AI worksbeginner
Model Card
Also called Model Cards
The documentation that comes with a trained model, saying what it is for, how it was trained and evaluated, and where it should not be used.
AI agentsbeginner
Model Context Protocol
Also called MCP
An open standard for connecting AI applications to tools and data. A system offered once as an MCP server works in any application that speaks MCP.
How AI worksbeginner
Model Deprecation
Also called Model retirement, Deprecated model, Model lifecycle, End of life (EOL)
The notice that a model will be switched off on a set date. Until then it still answers; after its retirement date, every request to it fails, so code must move to a newer model first.
Next to it
How AI worksintermediate
Model Routing
Also called LLM Routing, Model Cascade, LLM Router
Sending each request to the cheapest model that handles it well, by rules, a classifier or a cascade, and checking with an eval that quality holds.
How AI worksadvanced
Multi‑Head Attention
Also called MHA
Parallel attention layers allowing a model to focus on different subspaces
Instead of a single attention computation, multiple heads independently attend to information, whose outputs are concatenated and projected, enriching the representation.
Next to it
How AI worksbeginner
Multimodal Model
Also called Multi-Modal Model, Multimodal AI
A model that works with more than one kind of input or output, such as text and images, or text, sound and video, instead of text alone.
How AI worksbeginner
Negative Prompt
Text describing what the model should avoid generating, used in diffusion or text‑to‑image pipelines.
How AI worksadvanced
NeRF
Neural Radiance Field; a model that represents 3D scenes from 2D images.
Optimizes a continuous field that outputs color and density for any 3D coordinate and viewing angle.
How AI worksadvanced
Neural Network
Also called Neural Networks
A computing system inspired by biological neural networks that learn patterns from data
Neural networks are computing systems inspired by the biological neural networks found in animal brains. They consist of interconnected nodes (neurons) organized in layers that process information by passing signals between them. Neural networks can learn complex patterns from data through training, making them fundamental to modern machine learning and artificial intelligence applications.
Neural networks are the foundation of deep learning, consisting of layers of interconnected nodes (neurons) that process information. Each connection has a weight that adjusts as the network learns.
Next to it
AI agentsintermediate
OAuth
Also called OAuth 2.0, OAuth 2.1, OAuth client ID
The standard way to let one app act for you at another service without giving it your password; the app gets a limited, revocable access token instead. MCP servers use it to let AI agents sign in.
Next to it
AI agentsbeginner
Observation
A piece of information an agent receives after executing an action.
Agents iterate over Thought, Action, Observation cycles.
Next to it
How AI worksintermediate
ONNX
Also called Open Neural Network Exchange, ONNX Runtime
An open file format for trained models, so a model built in one framework (such as PyTorch) can run in another runtime, on another device or language, without the original code.
Data engineering in the AI eraintermediate
Open Table Format
Also called Apache Iceberg, Table format
A published way of storing a table as plain data files plus a log of changes, so many engines can read and safely write it. Delta Lake and Apache Iceberg are the best known.
How AI worksbeginner
Open-Weight Model
Also called Open weights, Open-weights model, Open model
A model whose trained weights anyone can download and run on their own hardware, under a licence that sets what you may do with them. Open weights alone do not make a model open source.
How AI worksbeginner
Optimizer
An algorithm that adjusts model parameters to minimize the loss function, e.g., Adam or SGD.
Next to it
Data engineering in the AI eraintermediate
Orchestration
Also called Workflow orchestration
Running the steps of data pipelines in the right order and at the right time, retrying failures and alerting someone when a step breaks.
Next to it
How AI worksadvanced
PagedAttention
Also called Paged Attention
A way of storing each request's KV cache in small fixed-size blocks, like pages of virtual memory, so a serving engine wastes almost none of it and fits more requests in a batch.
How AI worksbeginner
Parameter
Also called Model Parameters, Parameter Count
A number inside a model that training sets, a weight or a bias. "70B" in a model's name means about 70 billion of them.
How AI worksadvanced
Parameter-Efficient Fine-Tuning
Also called PEFT, Parameter Efficient Fine Tuning
Fine-tuning that freezes a model's own weights and trains a small number of new ones, such as a LoRA adapter, so a task costs a fraction of the memory and storage of full fine-tuning.
Data engineering in the AI eraadvanced
pgvector
A PostgreSQL extension that adds vector similarity search.
Stores embeddings in a 'vector' column type and indexes them with approximate nearest neighbour algorithms like HNSW.
Next to it
Data engineering in the AI erabeginner
Pinecone
A managed vector database service.
Offers hosted vector storage with hybrid search, metadata filtering, and horizontal scaling.
Next to it
How AI worksadvanced
Pipeline Parallelism
Also called PP, Inter-Layer Model Parallelism
Putting different layers of a model on different GPUs and passing work from one to the next like an assembly line, with small batches keeping every stage busy.
AI agentsintermediate
Planner-Executor Pattern
Also called Plan and Execute, Plan-and-Execute Agent, Orchestrator-Workers, ReWOO
An agent design that splits the work in 2. A planner writes the whole plan first, and an executor, which can be cheaper code or smaller models, carries out each step. The plan is changed only when a step fails.
How AI worksadvanced
Positional Encoding
A method to inject position information into Transformers
Because self‑attention is position‑agnostic, positional encodings—sinusoidal functions or learned vectors—allow the model to consider token order in sequences.
Next to it
How AI worksadvanced
Presence Penalty
A penalty applied to tokens already present in the context to encourage new topics.
Presence penalty pushes the model to explore less‑covered ideas, balancing coherence and novelty.
Next to it
How AI worksbeginner
Prompt
The input text given to a generative model.
Prompts can include instructions, examples, and context. Crafting effective prompts is key to leveraging LLM capabilities.
Next to it
How AI worksintermediate
Prompt Caching
Also called Context Caching, Cache Control
Letting the API keep the processed start of a prompt for a few minutes, so later requests that begin the same way are cheaper and quicker.
How AI worksbeginner
Prompt Engineering
Also called Prompt Design, Prompting
Writing and testing the instructions, context and examples you give a model so it does a job reliably, and measuring each change instead of guessing.
How AI worksintermediate
Prompt Injection
An attack that hides instructions in text a model reads, such as a web page, an email or a file, so the model follows the attacker instead of you.
How AI worksbeginner
Prompt Template
Also called Prompt Versioning, Versioned Prompt
A prompt kept as a file with placeholders filled in at run time, so it can be versioned, reviewed and tested like the rest of your code.
How AI worksadvanced
Proximal Policy Optimization
Also called PPO
A reinforcement learning method that improves a model in small, capped steps. It was the algorithm behind the RLHF stage of the first ChatGPT-style assistants.
Software and toolsbeginner
Python
A high-level, interpreted programming language known for its simple syntax and versatility.
Python is a popular programming language that emphasizes code readability and simplicity. It's widely used in data science, web development, automation, and artificial intelligence.
Next to it
How AI worksadvanced
QLoRA
Also called Quantized LoRA, Quantized Low-Rank Adaptation
LoRA trained on a copy of the base model stored in 4 bits, so fine-tuning a large model needs a fraction of the GPU memory, with 3 tricks that keep the quality of 16-bit fine-tuning.
How AI worksintermediate
Quantization
Also called Quantisation
Storing a model's numbers with less precision, such as 4 bits instead of 16, so it needs far less memory and can run on smaller hardware, at some cost in accuracy.
Data engineering in the AI eraintermediate
Query Rewriting
Also called Query Reformulation, Query Transformation, Standalone Question, HyDE, Hypothetical Document Embeddings
Having a model turn what the user typed into a better search query before retrieval, most often turning a follow-up like "and the bigger one?" into a full question the search can match.
Data engineering in the AI eraintermediate
RAG
Also called Retrieval-Augmented Generation
A way to answer from your own documents. A search finds the passages that match the question, and they are added to the prompt before the model writes.
How AI worksintermediate
Rate Limit
Also called Rate Limits, 429 Too Many Requests, RPM and TPM, Retries and Backoff
The most an API lets you send in a period, counted in requests and tokens per minute. Past it, requests fail with a 429 error until capacity comes back, so code waits and retries.
Next to it
AI agentsintermediate
ReAct Pattern
Also called ReAct, Reason and Act, Thought, Action, Observation, Agent Loop
An agent design where the model alternates between reasoning about what to do next and taking an action, such as a tool call, then reads the result before deciding again. Most agent loops today follow it.
How AI worksintermediate
Red Teaming
Also called AI Red Teaming, LLM Red Teaming, Adversarial Testing
Attacking your own AI system on purpose, the way a real attacker or a careless user would, to find where it fails before they do.
How AI worksadvanced
Reinforcement Learning from Human Feedback
Also called RLHF
A training step where people rank a model's answers, a reward model learns those preferences, and the language model is tuned to score well on it.
How AI worksadvanced
Repetition Penalty
A logit scaling that discourages previously generated tokens to reduce loops.
Penalties help prevent the model from repeating phrases or sentences verbatim.
Next to it
Data engineering in the AI eraintermediate
Reranking
Also called Re-ranking, Reranker, Cross-Encoder Reranking, Retrieve and Rerank
A second search step that reads each candidate passage together with the question and scores how well it answers, so the best few of a fast first search go to the model in the right order.
Software and toolsintermediate
REST
Representational State Transfer - an architectural style for designing networked applications
REST (Representational State Transfer) is an architectural style for designing web services. It uses standard HTTP methods (GET, POST, PUT, DELETE) and follows principles like statelessness, uniform interface, and resource-based URLs to create scalable and maintainable APIs.
Data engineering in the AI eraintermediate
Retrieval Evaluation
Also called Recall at k, Recall@k, Precision at k, Mean Reciprocal Rank, MRR, RAG Evaluation
Testing the search step of a RAG system on its own, with questions whose answering passages you know, and scoring it with recall at k, the share of those passages found in the top k results.
How AI worksadvanced
RLHF Reward Model
A model trained to score outputs according to human preferences.
Acts as a learned objective for policy optimisation in RLHF pipelines.
How AI worksadvanced
RoPE
Also called Rotary Positional Embedding, Rotary Position Embeddings
Rotary Positional Embedding; a positional encoding that rotates query and key vectors by relative angles.
RoPE enables extrapolation to longer context lengths with minimal overhead and is used in models like Llama 3.
RoPE enables extrapolation to longer sequences and is used in GPT‑NeoX, Llama, and others.
Alias term.
How AI worksbeginner
Safety Filter
A model or rule set that blocks disallowed content.
Often run after generation to remove harmful outputs before display.
Next to it
How AI worksintermediate
Scaling Law
Also called Scaling Laws, Chinchilla Scaling, Compute-Optimal Training
The measured pattern that a language model's loss falls predictably as its parameters, training data and compute grow, used to plan how big to make a model and how long to train it.
Data engineering in the AI eraintermediate
Schema Drift
Also called Schema change
When the shape of incoming data changes without warning, such as a renamed or new column, and breaks or silently corrupts the pipelines that read it.
How AI worksbeginner
Score‑Based Model
A generative model trained to predict the gradient (score) of the data distribution’s log‑density.
Next to it
How AI worksintermediate
Self-Attention
Also called Intra-Attention, Scaled Dot-Product Attention
The step in a transformer where every token looks at the other tokens in the same text and takes in what is relevant to it, so meaning depends on context.
How AI worksintermediate
Self‑Consistency
Also called Self-Consistency Prompting, Majority Voting
Asking a model the same question several times with some randomness, letting it reason each time, and taking the answer it reaches most often.
Data engineering in the AI eraintermediate
Semantic Layer
Also called Metrics layer
One shared definition of business terms and metrics, such as revenue or active customer, that reports, dashboards and AI assistants all read instead of writing their own.
Next to it
Data engineering in the AI erabeginner
Sentence Embedding
Also called Text Embedding, Sentence Vector, Bi-Encoder, Sentence-BERT
One list of numbers for the meaning of a whole sentence or passage, made so that texts which mean the same thing get lists close together. It is what semantic search compares.
How AI worksintermediate
SentencePiece
A language‑independent tokenizer based on sub‑set Unigram/BPE models.
Developed by Google, SentencePiece treats input text as a raw byte sequence so it supports any language without preprocessing.
Next to it
Data engineering in the AI erabeginner
SentenceTransformers
Also called Sentence Transformers, SBERT
The open-source Python library most people use to turn text into embeddings on their own machine, with thousands of ready models and a few lines of code.
Data engineering in the AI eraintermediate
Shuffle and Data Skew
Also called Data skew, Skew join, Straggler task
A shuffle moves rows between machines so rows with the same key meet, for a join or group by; skew is when one key has far more rows, so one task does most of the work.
Data engineering in the AI eraintermediate
Slowly Changing Dimensions
Also called SCD, SCD type 1, SCD type 2, History tracking
The ways a warehouse handles a change to a descriptive record, such as a customer's address: type 1 overwrites it, type 2 adds a new dated version, and type 3 keeps the previous value in an extra column.
How AI worksbeginner
Small Language Model
Also called SLM
A language model small enough to run on a laptop, phone, robot or single GPU, usually a few billion parameters, trading some broad knowledge for speed, low memory and privacy.
How AI worksadvanced
Sora
OpenAI's video‑generation model announced in 2025.
Uses diffusion‑transformer hybrid architecture to create high‑resolution, realistic videos up to 60 seconds.
Next to it
Data engineering in the AI eraintermediate
Sort-Merge Join
Also called SortMergeJoin, Shuffle sort merge join, MERGE join hint, Broadcast hash join vs sort-merge join
Spark's default join for 2 large tables: it shuffles both by the join key, sorts each partition, then walks the 2 sorted sides together to find matching rows.
Data engineering in the AI eraadvanced
Spark Executor Memory
Also called Executor memory, spark.executor.memory, Memory overhead, Spark memory management
The memory each Spark worker process gets: a Java heap, split into a shared pool for shuffles, joins and cached data and a pool for your own objects, plus overhead outside the heap.
Data engineering in the AI eraintermediate
Spark Shuffle
Also called Shuffle, Exchange, Shuffle read, Shuffle write, Shuffle partitions
The step that moves rows between executors so rows with the same key end up together, for a join, group by or repartition; it splits a job into stages.
Data engineering in the AI eraintermediate
Spark Spill
Also called Spill to disk, Shuffle spill, Spill (memory), Spill (disk)
When a task's sort, join or aggregation no longer fits in execution memory, Spark writes part of it to local disk and reads it back later, so the task finishes, only slower.
How AI worksadvanced
Sparse Expert Model
A model where only a fraction of parameters (experts) are active per token.
Cuts inference cost compared to dense models while scaling capacity.
Next to it
How AI worksadvanced
Speculative Decoding
Also called Speculative sampling, Draft model, Multi-token prediction, MTP
A small, fast drafter guesses the next few tokens and the large model checks them all in one pass, keeping the ones it agrees with. Answers come faster and stay exactly the same.
How AI worksbeginner
Speech‑to‑Text
The task or model that converts spoken language audio into written text.
How AI worksbeginner
Stable Diffusion
An open‑source latent diffusion model for text‑to‑image generation
Stable Diffusion, released by Stability AI in 2022, converts text prompts into high‑resolution images. Its open weights and permissive license sparked a wave of creative and commercial applications.
Next to it
Data engineering in the AI erabeginner
Star Schema
Also called Dimensional modelling, Dimensional modeling, Fact table, Dimension table, Kimball model
A way to arrange warehouse tables for reporting: a fact table of events and their numbers, joined by keys to dimension tables that describe them, such as customer, product and date.
How AI worksadvanced
State Space Model
Also called SSM, Mamba, Mamba-2, Hybrid Mamba-Transformer
A layer that reads text in order and carries a fixed-size summary forward instead of keeping every past token, so its memory stays the same however long the conversation gets. Mamba is the best-known kind.
How AI worksintermediate
Streaming
Also called Streaming Responses, Token Streaming, Server-Sent Events
Getting a model's answer in small pieces as it is written, instead of all at once at the end, so people see text start sooner and long answers do not hit a timeout.
Next to it
How AI worksintermediate
Structured Output
Also called Structured Outputs, JSON Mode, JSON Schema Output
A setting that makes a model's answer follow a JSON schema you give it, so your code can read fields such as a category or an order number without parsing free text.
AI agentsintermediate
Subagent
Also called Sub-agent, Multi-Agent System, Worker Agent
An agent started by another agent for one part of a task. It works in its own fresh context and returns only its findings.
How AI worksbeginner
Subword Token
A token representing a fragment of a word, enabling open‑vocabulary modeling.
Next to it
How AI worksintermediate
Supervised Fine‑Tuning
Also called SFT
Standard fine‑tuning on labeled input‑output pairs.
Often the first step before RLHF to teach task‑specific format.
Next to it
How AI worksbeginner
System Prompt
Also called System Message, Developer Message, Instructions
Instructions the builder of an AI app gives the model before you type anything. It sets the role, the tone and the rules, and it outranks what users say.
How AI worksbeginner
Temperature
Also called Temperature (Sampling), Sampling Temperature
A setting that controls how much chance goes into each word a model picks. Low gives focused, predictable answers; high gives more varied ones.
How AI worksadvanced
Tensor Parallelism
Also called TP, Intra-Layer Model Parallelism
Splitting each layer's weight matrices across several GPUs, so every GPU does part of every layer and they combine their results after each one.
How AI worksintermediate
Text Generation Inference
Also called TGI
Hugging Face’s optimized server for LLM deployment.
TGI supports continuous batching, quantization, and streaming responses.
Next to it
Data engineering in the AI eraintermediate
Text-to-SQL
Also called Natural language to SQL, NL2SQL
A model turning a question in plain words into a database query, so people can ask the data directly. It is only as right as its knowledge of the tables and their meaning.
How AI worksadvanced
Textual Inversion
Method that learns a new embedding token to represent a concept in diffusion models.
Allows teaching a model a new style or object without altering core weights.
How AI worksbeginner
Throughput
Also called Tokens Per Second, Requests Per Minute
How much work a model system gets through in a given time, counted in requests or tokens per minute; on an API it is capped by rate limits, on your own hardware by batching and memory.
How AI worksbeginner
Token
The small piece of text a language model reads and writes, often part of a word. Prices, limits and the context window are all counted in tokens.
How AI worksbeginner
Tokenization
The process of breaking text into tokens for model input
Tokenization converts raw text into discrete units—words, sub‑words, or characters—that a language model can process. Choice of tokenizer affects vocabulary size and handling of rare words.
Next to it
How AI worksbeginner
Tokenizer
Also called Tokeniser
The part that cuts text into the tokens a model reads, and turns the model's tokens back into text. Each model family has its own.
Next to it
AI agentsbeginner
Tool Use
Also called Tools
Giving a model a list of things it can ask to have done, such as search the web or read a file. The model asks; the program around it does the work and hands back the result.
How AI worksintermediate
Top-p Sampling
Also called Nucleus Sampling, Top-p
Picking the next token at random from only the smallest set of likely tokens whose chances add up to p, so the model never picks from the long tail of unlikely words.
How AI worksadvanced
Top‑A Sampling
Filtering method that selects tokens above a probability threshold A.
Variant proposed by OpenAI to balance quality and diversity during decoding.
Next to it
How AI worksintermediate
Top‑k Sampling
A decoding strategy that restricts choices to the k most probable tokens
During generation, the probability distribution is truncated to the top k options before sampling, reducing unlikely words and promoting diversity without completely random output.
How AI worksintermediate
Tracing
Also called LLM Tracing, Agent Tracing, Trace, LLM Observability
Recording every step an LLM app or agent takes for one request, each model call, tool call and retrieval with its input, output, tokens and time, so you can see why an answer went wrong.
How AI worksintermediate
Transformer
The design behind almost every modern language model. It uses attention to let each word in a passage look at every other word, all at once.
Data engineering in the AI erabeginner
Unity Catalog
Also called Databricks Unity Catalog, UC
Databricks' governance layer: one place that names every table, file volume, model and function as catalog.schema.object, decides who may use each, and records lineage and access.
How AI worksintermediate
Variational Autoencoder
Also called VAE
A generative model that learns a probabilistic latent space representation of data
A Variational Autoencoder (VAE) encodes input data into a latent distribution rather than a single point. By sampling from this distribution during decoding, VAEs can generate new, coherent samples. They balance reconstruction accuracy with latent-space regularization through the Kullback–Leibler divergence.
Next to it
Data engineering in the AI eraintermediate
Vector Database
Also called Vector Store
A database built to store embeddings and quickly find the ones closest to a query, which is how AI apps search by meaning.
Data engineering in the AI erabeginner
Vector Search
Also called Similarity Search, Nearest Neighbour Search, Semantic Search
Finding the stored items whose embeddings are closest to a query's embedding, so a search matches meaning rather than exact words.
How AI worksadvanced
Video Generation
Producing video sequences from text or images with generative models.
Approaches use latent diffusion, temporal U‑Nets, or autoregressive transformers.
How AI worksbeginner
Vision‑Language Model
Also called VLM, Vision Language Model
A model that takes images as well as text and answers in text, so it can describe a photo, read a chart or a screenshot, or answer questions about a document.
How AI worksadvanced
vLLM
Also called vLLM server, vllm serve
An open-source engine for serving language models fast, built on PagedAttention, that answers requests through the same kinds of API the big model makers offer.
Data engineering in the AI eraintermediate
Watermark
Also called Event-time watermark, Late data threshold, withWatermark
In stream processing, a marker that says how far event time has got: data older than the latest event seen, less a set delay, is treated as too late, so windows can close and their state be freed.
Data engineering in the AI eraintermediate
Weaviate
An OSS vector search engine with GraphQL API.
Supports modules for OpenAI embeddings, classification, and hybrid BM25‑vector search.
Next to it
How AI worksbeginner
Weight
A trainable parameter in a neural network that scales input values and is updated during training.
How AI worksbeginner
Whisper
Also called OpenAI Whisper
OpenAI's open-source speech recognition model. It turns audio in many languages into text, or translates it into English, and runs on your own machine.
Data engineering in the AI erabeginner
Word Embedding
An embedding specifically representing individual words, capturing semantic relationships.
Next to it
How AI worksintermediate
Zero‑shot Learning
Solving a task without any task‑specific examples
Zero‑shot generation relies on massive pre‑training so the model can infer the task solely from the prompt wording, without supplied examples.
Next to it
Data engineering in the AI erabeginner
Zilliz
The company behind Milvus, offering managed vector database services.
Next to it
Try it
Some words are easier to play with than to read
Each demo sits on the page that explains its word, runs in your browser, and can be put on your own site with the code at the end of that page.
- Turn the temperature up and downTemperature
- See what a word pays attention toSelf-Attention
- Fill a context windowContext Window
- Write an answer with and without the KV cacheKV Cache
- Squeeze a model into less memoryQuantization
- Cut a policy into chunksChunking
- Search an index by meaningEmbedding Pipeline
- Follow 7 orders from bronze to goldMedallion Architecture
- Keep a copy of a table in stepChange Data Capture
- Read a table as it wasOpen Table Format
- Run a nightly jobOrchestration
- Walk the lineage of an answerData Lineage
- Play a day of ordersData Pipeline
- Search a graph index layer by layerHNSW
- Turn and stretch two vectorsCosine Similarity
- Wait for late paymentsWatermark
- Move a customer and keep her historySlowly Changing Dimensions
The cat sat on the …
- mat50%
- sofa22%
- floor17%
- roof7%
- keyboard4%
- moon1%
Low temperature: the likeliest word nearly every time. High: the unlikely words get a real chance.
The scores are made up to show the effect, not taken from a real model.