The words of AI, in plain English
Every term you meet in AI, agents and data engineering, each explained in one sentence, with what else it is called and what sits next to it. Look a word up, or test yourself and keep track of the words you know.
Look it up
How AI works · beginner
Large Language Model
A model trained on a huge amount of text to predict the next token. Further training turns it into an assistant that follows instructions.
Also called LLM
Next to it
Explore by subject
Model CardKV CacheONNXCarbon FootprintHugging Face TransformersGuardrailVision‑Language ModelBF16TokenizerDALL·EBiasTensor ParallelismOptimizerParameterWhisperGroundingPrompt InjectionSpeech‑to‑TextFine-TuningSelf‑ConsistencyNegative PromptContext WindowKnowledge CutoffChain of ThoughtHallucinationLarge Language ModelTemperatureToken
Data engineering in the AI era
The data a model is only as good as, and the systems that store, move and search it.
Every word, A to ZDense Passage RetrievalHNSWCosine SimilarityVector DatabaseZillizVector SearchData PoisoningData ParallelismFAISSEmbedding DimensionRAGSentenceTransformersWeaviatePineconeEmbeddingData PreprocessingSentence EmbeddingMilvusIVF‑PQChromaWord EmbeddingApproximate Nearest Neighbor SearchShardingpgvectorDataset Card
Every word, A to Z
How AI worksintermediate
3D Generation
Creating three‑dimensional content using generative models.
Techniques include NeRF, Gaussian Splatting, and 3D‑aware diffusion.
Next to it
How AI worksadvanced
4‑bit Quantization
Representing weights with 4‑bit integers.
Enables running billion‑parameter models on consumer GPUs; often paired with groupwise scale/zero‑points.
Next to it
How AI worksbeginner
Adam Optimizer
Also called Adam
A popular adaptive learning-rate optimization algorithm combining momentum and RMSProp techniques.
A popular adaptive learning-rate optimization algorithm combining momentum and RMSProp techniques.
How AI worksbeginner
Adapter
A small bottleneck module inserted in each transformer layer to adapt a model to new tasks without updating full weights.
A small bottleneck module inserted in each transformer layer to adapt a model to new tasks without updating full weights.
Next to it
How AI worksadvanced
Adversarial Attack
Input crafted to force a model to err.
Perturbations are often imperceptible to humans but fool models.
Next to it
AI agentsbeginner
Agent
Also called AI Agent
A model that uses tools in a loop, deciding each next step from what the last one returned, until the task is done.
AI agentsintermediate
Agent Memory
Also called AI Memory
The ways an agent keeps what it needs beyond a single context window, such as summaries, notes in files, and saved facts it reads back later. The model itself remembers nothing.
Next to it
AI agentsadvanced
Agentic Workflow
A multi‑step process orchestrated by autonomous AI agents.
Combines planning, tool use, and memory to solve complex tasks without constant human supervision.
Next to it
How AI worksadvanced
Alignment
Ensuring model objectives match human values and intent
Alignment research seeks techniques (RLHF, constitutional AI, interpretability) that make model outputs safe, truthful, and beneficial, reducing harmful or biased generations.
Next to it
Software and toolsbeginner
API
Also called Application Programming Interface
Application Programming Interface - a set of protocols and tools for building software applications
An API (Application Programming Interface) is a set of rules, protocols, and tools that allows different software applications to communicate with each other. It defines the methods of communication between various components and specifies how software components should interact.
Data engineering in the AI erabeginner
Approximate Nearest Neighbor Search
Also called ANN Search
Algorithms that return “close enough” nearest neighbors in high‑dimensional spaces to trade accuracy for speed.
Algorithms that return “close enough” nearest neighbors in high‑dimensional spaces to trade accuracy for speed.
How AI worksintermediate
Attention Head
One parallel attention mechanism within a multi‑head attention layer.
Each head learns different relational patterns; combining them lets Transformers model diverse dependencies.
Next to it
How AI worksintermediate
Backpropagation
Also called Backprop
An algorithm for computing gradients in deep networks via the chain rule.
Backpropagation propagates error derivatives backward through layers, enabling efficient parameter updates and is foundational to deep learning.
How AI worksadvanced
Beam Search
An approximate search algorithm that keeps the top N hypotheses at each decoding step
Beam search explores multiple candidate sequences in parallel, pruning to the best‑scoring beams and producing more deterministic outputs than pure sampling, though sometimes less diverse.
Next to it
How AI worksbeginner
BF16
Bfloat16 — a 16‑bit floating‑point format with wider exponent range than FP16, common on TPUs.
Bfloat16 — a 16‑bit floating‑point format with wider exponent range than FP16, common on TPUs.
How AI worksbeginner
Bias
A trainable parameter added to the weighted sum of inputs in a neural network layer, allowing the activation function to be shifted.
A trainable parameter added to the weighted sum of inputs in a neural network layer, allowing the activation function to be shifted.
How AI worksbeginner
Billion‑Parameter Model
An LLM with at least one billion trainable parameters.
An LLM with at least one billion trainable parameters.
Next to it
How AI worksintermediate
Byte Pair Encoding
Also called BPE
A sub‑word tokenization algorithm based on merging frequent symbol pairs
BPE iteratively merges the most common pair of bytes or characters, creating a vocabulary that balances token count and coverage. Widely used in GPT and other Transformer models.
Next to it
How AI worksbeginner
Carbon Footprint
The total greenhouse‑gas emissions produced by training or using an AI model.
The total greenhouse‑gas emissions produced by training or using an AI model.
How AI worksintermediate
Chain of Thought
Also called Chain-of-Thought Prompting, CoT, Thinking, Reasoning
Having a model write out the steps of its reasoning before it gives an answer. Once a prompting trick, now built into "thinking" models.
Data engineering in the AI erabeginner
Chroma
An open‑source vector database focused on developer productivity and RAG workflows.
An open‑source vector database focused on developer productivity and RAG workflows.
Next to it
Software and toolsintermediate
CI/CD
Also called Continuous Integration/Continuous Deployment
Continuous Integration and Continuous Deployment - practices for automating software development workflows
CI/CD stands for Continuous Integration and Continuous Deployment (or Delivery). CI involves automatically integrating code changes from multiple contributors into a shared repository, running automated tests to catch issues early. CD extends this by automatically deploying tested code to production environments, enabling faster and more reliable software releases.
Next to it
How AI worksbeginner
Classifier‑Free Guidance
A sampling technique for diffusion models that mixes conditional and unconditional scores to steer outputs.
A sampling technique for diffusion models that mixes conditional and unconditional scores to steer outputs.
Next to it
Software and toolsintermediate
Clean Code
Code that is readable, maintainable, and follows best practices.
Clean code is code that is easy to read, understand, and modify. It follows consistent naming conventions, has clear structure, and is well-documented.
Next to it
How AI worksbeginner
Compute Budget
The total amount of compute resources allocated for training or inference.
The total amount of compute resources allocated for training or inference.
Next to it
How AI worksbeginner
Content Moderation
Automated detection of toxic, unsafe, or disallowed content.
LLMs and classifiers flag or block content that violates policy guidelines.
Next to it
How AI worksbeginner
Context Window
Also called Context Length
Everything a model can see at once while it writes an answer, counted in tokens. When the window is full, something has to be left out.
Next to it
How AI worksadvanced
ControlNet
Extension for diffusion models that adds condition‑control via additional networks.
Enables fine‑grained control such as pose, edge, or depth maps, while reusing pretrained weights.
Next to it
Data engineering in the AI erabeginner
Cosine Similarity
A metric that measures the cosine of the angle between two vectors.
Values range from −1 to 1; widely used for text embedding similarity.
Next to it
How AI worksadvanced
Cross‑Attention
An attention mechanism that relates one sequence to another
In encoder‑decoder Transformers or multimodal models, cross‑attention lets the decoder focus on encoder outputs (e.g., text attending to image patches), enabling conditional generation.
Next to it
Software and toolsbeginner
CSS
Also called Cascading Style Sheets
Cascading Style Sheets - a stylesheet language used for describing the presentation of web documents
CSS (Cascading Style Sheets) is a stylesheet language used to describe the presentation and formatting of HTML documents. It controls the layout, colors, fonts, spacing, and overall visual appearance of web pages, enabling the separation of content from presentation.
How AI worksbeginner
DALL·E
A model that generates images from natural‑language descriptions
Developed by OpenAI, DALL·E combines a Transformer with a discrete VAE to render imaginative images that correspond to textual prompts, demonstrating the power of multimodal generative models.
Next to it
Data engineering in the AI erabeginner
Data Parallelism
Replicating model weights across devices and splitting batches to train in parallel.
Replicating model weights across devices and splitting batches to train in parallel.
Next to it
Data engineering in the AI eraadvanced
Data Poisoning
Malicious manipulation of training data to subvert model outputs.
Can cause backdoors or reduce performance; mitigations include data audits and robust training.
Data engineering in the AI erabeginner
Data Preprocessing
The process of cleaning, transforming, and organizing raw data before analysis.
Data preprocessing is a crucial step in any data science project that involves cleaning messy data, handling missing values, encoding categorical variables, and scaling features.
Data engineering in the AI erabeginner
Dataset Card
Documentation describing a dataset’s composition, collection process, and recommended uses.
Documentation describing a dataset’s composition, collection process, and recommended uses.
Data engineering in the AI eraadvanced
Dense Passage Retrieval
Also called DPR
A retrieval technique using dual‑encoder embeddings for query and passage.
Embeddings are precomputed; at query time approximate nearest neighbour search retrieves relevant passages.
Next to it
Software and toolsadvanced
Design Patterns
Reusable solutions to commonly occurring problems in software design.
Design patterns are proven solutions to recurring design problems in software development. They provide a template for how to solve problems that can be used in many different situations.
Next to it
How AI worksadvanced
Diffusion Model
A generative model that iteratively denoises data starting from random noise
Diffusion models learn the reverse of a gradual noising process. By sampling noise and applying the learned denoising steps, they generate high‑quality images and audio. Notable examples include Imagen and Stable Diffusion.
Software and toolsintermediate
Docker
A platform for developing, shipping, and running applications using containerization technology
Docker is a containerization platform that allows developers to package applications and their dependencies into lightweight, portable containers. These containers can run consistently across different environments, from development to production, solving the "it works on my machine" problem.
Next to it
How AI worksadvanced
DreamBooth
Technique to personalize a diffusion model with a few example images.
Fine‑tunes a model to render a specific subject or style in new scenes.
Next to it
Data engineering in the AI erabeginner
Embedding
Also called Vector Embedding, Vector Representation
A list of numbers that stands for the meaning of a piece of text, an image or other data. Things with similar meaning get similar numbers, so a computer can find them.
Data engineering in the AI erabeginner
Embedding Dimension
Number of elements in an embedding vector.
Higher dimensions capture more information but increase storage and compute cost.
Next to it
How AI worksbeginner
Energy Efficiency
The ratio of model performance to energy consumed, an important sustainability metric.
The ratio of model performance to energy consumed, an important sustainability metric.
Data engineering in the AI eraadvanced
FAISS
Facebook AI Similarity Search library for fast vector similarity.
Open‑source C++/Python library that powers many vector search back‑ends and supports efficient indexes like IVF, HNSW.
Next to it
How AI worksintermediate
Few‑shot Learning
The ability of a model to generalize from a small number of examples
In prompt‑based few‑shot learning, the model sees only a handful of demonstrations within the prompt yet performs the desired task thanks to broad pre‑training.
How AI worksintermediate
Fine-Tuning
Also called Fine Tuning, Finetuning
Training an existing model a little further on your own examples so it picks up a task, a format or a style. Often a good prompt does the job first.
How AI worksadvanced
Flash Attention
An algorithm that computes attention using tiling to GPU shared memory.
Provides up to 3× speed‑up and lower memory by avoiding redundant reads/writes.
AI agentsintermediate
Function Calling
Also called Tool Calling
The developer's name for tool use. The model replies with the name of a function and its arguments in a fixed format, and your code runs it.
Next to it
AI agentsintermediate
Function Schema
JSON schema that describes tool arguments for function calling.
The schema constrains model output and allows validation before execution.
Next to it
How AI worksintermediate
Generative Adversarial Network
Also called GAN
A deep learning architecture where two neural networks compete to generate realistic data
A Generative Adversarial Network (GAN) consists of a generator that produces synthetic data and a discriminator that evaluates it against real data. Through this adversarial process, the generator learns to create outputs—images, audio, or text—that are indistinguishable from genuine samples.
How AI worksbeginner
GPT
Also called Generative Pre‑trained Transformer
A Generative Pre‑trained Transformer language model
GPT models are first pre‑trained on massive text corpora with a next‑token objective, then optionally fine‑tuned. They can generate coherent paragraphs, answer questions, and perform diverse language tasks with in‑context learning.
How AI worksbeginner
Gradient
The vector of partial derivatives of the loss function with respect to model parameters, used to update weights during training.
The vector of partial derivatives of the loss function with respect to model parameters, used to update weights during training.
Next to it
How AI worksbeginner
Gradient Checkpointing
A memory‑saving technique that recomputes activations during backprop to trade compute for RAM.
A memory‑saving technique that recomputes activations during backprop to trade compute for RAM.
How AI worksbeginner
Gradient Descent
An optimization algorithm that iteratively updates model parameters in the direction that minimizes the loss.
An optimization algorithm that iteratively updates model parameters in the direction that minimizes the loss.
Software and toolsintermediate
GraphQL
A query language and runtime for APIs that provides a flexible and efficient alternative to REST
GraphQL is a query language for APIs and a runtime for executing those queries with your existing data. Unlike REST APIs that expose multiple endpoints, GraphQL uses a single endpoint and allows clients to request exactly the data they need. This reduces over-fetching and under-fetching of data, making applications more efficient.
How AI worksbeginner
Grounding
Ensuring that generated content is verifiably linked to factual sources or retrieved context.
Ensuring that generated content is verifiably linked to factual sources or retrieved context.
How AI worksbeginner
Guardrail
Also called Guardrails
Checks placed around a model that stop or change what goes in and what comes out, such as a filter on user messages or an approval step before an agent acts.
How AI worksbeginner
Guidance Scale
A hyperparameter controlling the strength of conditional guidance in diffusion sampling.
A hyperparameter controlling the strength of conditional guidance in diffusion sampling.
How AI worksbeginner
Half‑Precision
Also called FP16
A 16‑bit floating‑point format used to accelerate training and inference while reducing memory.
A 16‑bit floating‑point format used to accelerate training and inference while reducing memory.
Next to it
How AI worksbeginner
Hallucination
Also called Fabrication, Confabulation
When a model states something false as if it were true, such as a wrong date, a made-up quote or a source that does not exist.
AI agentsbeginner
Haystack
An open‑source framework for building end‑to‑end question‑answering and RAG systems.
An open‑source framework for building end‑to‑end question‑answering and RAG systems.
Data engineering in the AI eraadvanced
HNSW
Also called HNSW Algorithm
Hierarchical Navigable Small World; a graph‑based ANN index.
Provides logarithmic search complexity for high‑dimensional vectors; used in FAISS, Milvus, pgvector.
The algorithm underlying HNSW index for efficient ANN search.
Next to it
Software and toolsbeginner
HTML
HyperText Markup Language - the standard markup language for creating web pages.
HTML provides the structure and content of web pages using elements and tags. It defines the semantic meaning of content and how it should be displayed.
How AI worksbeginner
Hugging Face Transformers
Also called Transformers Library
A popular Python library providing pretrained models, tokenizer tooling, and training utilities.
A popular Python library providing pretrained models, tokenizer tooling, and training utilities.
Next to it
How AI worksintermediate
In‑Context Learning
A model’s ability to learn from examples provided in the prompt context
Rather than updating weights, the model internalizes patterns during inference, making predictions consistent with the prompt examples—key to large language model versatility.
Next to it
How AI worksbeginner
Inference
Also called Serving
Using a trained model to produce an answer. Training builds the model once; inference is every time someone asks it something, and it is what you pay for per token.
How AI worksbeginner
Inference Latency
The time elapsed between sending a request to a model and receiving its output.
The time elapsed between sending a request to a model and receiving its output.
How AI worksbeginner
Instruction Tuning
Fine‑tuning a model on datasets of (instruction, response) pairs to make it follow user instructions better.
Fine‑tuning a model on datasets of (instruction, response) pairs to make it follow user instructions better.
Next to it
Data engineering in the AI erabeginner
IVF‑PQ
Also called Inverted File with Product Quantization
A vector‑indexing technique combining clustering (IVF) with compressed sub‑vector codes (PQ).
A vector‑indexing technique combining clustering (IVF) with compressed sub‑vector codes (PQ).
How AI worksintermediate
Jailbreak
Also called Jailbreaking
A prompt written to talk a model out of its safety rules, so it produces something it was trained or instructed to refuse.
Software and toolsintermediate
JavaScript
A high-level, dynamic programming language primarily used for web development.
JavaScript is a versatile programming language that runs in web browsers and servers. It enables interactive web pages and is an essential part of web applications.
Software and toolsbeginner
JSON
JavaScript Object Notation - a lightweight data interchange format
JSON (JavaScript Object Notation) is a lightweight, text-based data interchange format that is easy for humans to read and write, and easy for machines to parse and generate. Despite its name suggesting a connection to JavaScript, JSON is language-independent and widely used across different programming languages.
Next to it
How AI worksbeginner
Knowledge Cutoff
Also called Training Cutoff, Training Data Cutoff
The date a model's training text ends. The model knows nothing after it unless it is given new text, for example by a web search.
Next to it
How AI worksbeginner
Knowledge Distillation
A technique where a smaller “student” model learns to mimic a larger “teacher” model.
A technique where a smaller “student” model learns to mimic a larger “teacher” model.
Software and toolsadvanced
Kubernetes
An open-source container orchestration platform for automating deployment, scaling, and management of containerized applications
Kubernetes (often abbreviated as K8s) is an open-source container orchestration platform that automates the deployment, scaling, and management of containerized applications. It provides a robust framework for running distributed systems resiliently, handling scaling and failover for your applications.
How AI worksadvanced
KV Cache
Also called Key-Value Cache, Prompt Caching
Work a model saves from the text it has already read, so it is not redone for every new token. Prompt caching keeps that work between requests and charges less for it.
AI agentsintermediate
LangChain
A modular framework for building LLM applications with chains, agents, and tools.
Provides abstractions for prompt templates, memory, vector stores, and integrations with OpenAI, HuggingFace, etc.
Next to it
AI agentsbeginner
Large Action Model
An agentic AI system capable of planning and executing sequences of actions in the physical or digital world.
An agentic AI system capable of planning and executing sequences of actions in the physical or digital world.
How AI worksbeginner
Large Language Model
Also called LLM
A model trained on a huge amount of text to predict the next token. Further training turns it into an assistant that follows instructions.
How AI worksbeginner
Latency
The time between sending a request to a model and receiving the first token of its response.
Low latency is crucial for interactive applications such as chatbots; techniques like model quantization and KV‑cache help reduce it.
Next to it
How AI worksadvanced
Latent Diffusion
Also called Latent Diffusion Model, LDM
A diffusion process applied in compressed latent space.
Reduces computational cost by operating on lower‑dimensional representations instead of pixel space.
Latent Diffusion Models (LDMs) first encode images into a lower‑dimensional latent space using an autoencoder. The diffusion process then runs in this space, greatly reducing compute while preserving quality, as popularized by Stable Diffusion.
Next to it
How AI worksbeginner
Latent Space
The compressed vector space learned by a model to represent data
GANs, VAEs, and diffusion models encode inputs into a latent space where arithmetic operations correspond to meaningful transformations (e.g., changing image style).
Next to it
How AI worksbeginner
Latent Vector
A vector representing a point in latent space, often sampled to generate content.
A vector representing a point in latent space, often sampled to generate content.
Next to it
How AI worksbeginner
Learning Rate
A hyperparameter that controls the step size during gradient descent updates.
A hyperparameter that controls the step size during gradient descent updates.
Next to it
How AI worksbeginner
Logit Bias
A mechanism that applies positive or negative biases to specific tokens’ logits to encourage or discourage their generation.
A mechanism that applies positive or negative biases to specific tokens’ logits to encourage or discourage their generation.
Next to it
How AI worksadvanced
LoRA
Also called Low‑Rank Adaptation
Low‑Rank Adaptation, a parameter‑efficient fine‑tuning method
LoRA freezes the original model weights and trains small, low‑rank matrices inserted into each layer, achieving task adaptation with a tiny parameter footprint.
Next to it
How AI worksbeginner
Loss Function
A metric that quantifies the difference between a model’s predictions and the ground truth during training.
A metric that quantifies the difference between a model’s predictions and the ground truth during training.
How AI worksintermediate
Machine Learning
Also called ML
A subset of artificial intelligence that enables computers to learn and make decisions from data without being explicitly programmed.
Machine Learning is a powerful computational method that allows systems to automatically improve their performance on a specific task through experience. It involves algorithms that can identify patterns in data and make predictions or decisions based on those patterns.
Next to it
AI agentsbeginner
Memory Tool
A component that stores and retrieves conversational or task‑relevant information for an agent.
A component that stores and retrieves conversational or task‑relevant information for an agent.
Next to it
Software and toolsadvanced
Microservices
An architectural approach where applications are built as a collection of small, independent services
Microservices architecture breaks down applications into small, independent services that communicate over well-defined APIs. Each service owns its data and business logic, can be developed and deployed independently, and is typically organized around business capabilities. This approach enables better scalability, technology diversity, and team autonomy.
Data engineering in the AI eraintermediate
Milvus
An open‑source cloud‑native vector database.
Provides distributed indexing, HNSW/IVF, replication, and supports SQL‑like queries via MilvusQL.
Next to it
How AI worksadvanced
Mixture of Experts
Also called MoE
A neural architecture that routes tokens through different expert subnetworks.
Increases parameter count while keeping compute cost manageable; examples include Switch Transformer and Mixtral.
Next to it
How AI worksbeginner
Model Card
A standardized documentation of a machine‑learning model’s intended use, limitations, and ethical considerations.
A standardized documentation of a machine‑learning model’s intended use, limitations, and ethical considerations.
How AI worksadvanced
Multi‑Head Attention
Also called MHA
Parallel attention layers allowing a model to focus on different subspaces
Instead of a single attention computation, multiple heads independently attend to information, whose outputs are concatenated and projected, enriching the representation.
Next to it
How AI worksbeginner
Multimodal Model
Also called Multi-Modal Model, Multimodal AI
A model that works with more than one kind of input or output, such as text and images, or text, sound and video, instead of text alone.
How AI worksbeginner
Negative Prompt
Text describing what the model should avoid generating, used in diffusion or text‑to‑image pipelines.
Text describing what the model should avoid generating, used in diffusion or text‑to‑image pipelines.
How AI worksadvanced
NeRF
Neural Radiance Field; a model that represents 3D scenes from 2D images.
Optimizes a continuous field that outputs color and density for any 3D coordinate and viewing angle.
How AI worksadvanced
Neural Network
Also called Neural Networks
A computing system inspired by biological neural networks that learn patterns from data
Neural networks are computing systems inspired by the biological neural networks found in animal brains. They consist of interconnected nodes (neurons) organized in layers that process information by passing signals between them. Neural networks can learn complex patterns from data through training, making them fundamental to modern machine learning and artificial intelligence applications.
Neural networks are the foundation of deep learning, consisting of layers of interconnected nodes (neurons) that process information. Each connection has a weight that adjusts as the network learns.
Next to it
AI agentsbeginner
Observation
A piece of information an agent receives after executing an action.
Agents iterate over Thought, Action, Observation cycles.
Next to it
How AI worksbeginner
ONNX
Also called Open Neural Network Exchange
An open format to represent machine‑learning models for portable inference across runtimes.
An open format to represent machine‑learning models for portable inference across runtimes.
Next to it
How AI worksbeginner
Optimizer
An algorithm that adjusts model parameters to minimize the loss function, e.g., Adam or SGD.
An algorithm that adjusts model parameters to minimize the loss function, e.g., Adam or SGD.
Next to it
How AI worksadvanced
PagedAttention
Attention algorithm that manages KV cache in GPU pages.
Enables very long sequences without out‑of‑memory errors.
Next to it
How AI worksbeginner
Parameter
A learned weight or bias value in a neural network; large language models may have billions of parameters.
A learned weight or bias value in a neural network; large language models may have billions of parameters.
Next to it
How AI worksadvanced
Parameter‑Efficient Fine‑Tuning
Also called PEFT
Adapting a model by updating only a small subset of parameters
Techniques like LoRA, Adapters, and Prefix‑Tuning enable customization with reduced compute and storage by freezing core weights and training lightweight modules.
A family of methods (e.g., LoRA, Adapters) that fine‑tune large models with a small number of additional parameters.
Next to it
Data engineering in the AI eraadvanced
pgvector
A PostgreSQL extension that adds vector similarity search.
Stores embeddings in a 'vector' column type and indexes them with approximate nearest neighbour algorithms like HNSW.
Next to it
Data engineering in the AI erabeginner
Pinecone
A managed vector database service.
Offers hosted vector storage with hybrid search, metadata filtering, and horizontal scaling.
Next to it
How AI worksbeginner
Pipeline Parallelism
Splitting model layers across GPUs and passing activations forward like an assembly line.
Splitting model layers across GPUs and passing activations forward like an assembly line.
AI agentsbeginner
Planner‑Executor Pattern
An agent architecture that separates high‑level planning from low‑level execution.
An agent architecture that separates high‑level planning from low‑level execution.
Next to it
How AI worksadvanced
Positional Encoding
A method to inject position information into Transformers
Because self‑attention is position‑agnostic, positional encodings—sinusoidal functions or learned vectors—allow the model to consider token order in sequences.
Next to it
How AI worksadvanced
Presence Penalty
A penalty applied to tokens already present in the context to encourage new topics.
Presence penalty pushes the model to explore less‑covered ideas, balancing coherence and novelty.
Next to it
How AI worksbeginner
Prompt
The input text given to a generative model.
Prompts can include instructions, examples, and context. Crafting effective prompts is key to leveraging LLM capabilities.
Next to it
How AI worksbeginner
Prompt Engineering
Crafting input prompts to steer generative models toward desired outputs
Prompt engineering involves choosing wording, structure, and examples that elicit specific behaviors from large language models. Techniques include zero‑shot, few‑shot, and chain‑of‑thought prompts.
Next to it
How AI worksintermediate
Prompt Injection
An attack that hides instructions in text a model reads, such as a web page, an email or a file, so the model follows the attacker instead of you.
How AI worksbeginner
Prompt Template
A reusable prompt with placeholders that are filled dynamically at runtime.
A reusable prompt with placeholders that are filled dynamically at runtime.
Next to it
How AI worksbeginner
Proximal Policy Optimization
Also called PPO
A popular reinforcement‑learning algorithm often used in RLHF pipelines.
A popular reinforcement‑learning algorithm often used in RLHF pipelines.
Software and toolsbeginner
Python
A high-level, interpreted programming language known for its simple syntax and versatility.
Python is a popular programming language that emphasizes code readability and simplicity. It's widely used in data science, web development, automation, and artificial intelligence.
Next to it
How AI worksadvanced
QLoRA
Quantized LoRA that fine‑tunes 4‑bit quantized base models.
Reduces memory footprint so 70‑B models fit on a single GPU for supervised fine‑tuning.
Next to it
How AI worksintermediate
Quantization
Also called Quantisation
Storing a model's numbers with less precision, such as 4 bits instead of 16, so it needs far less memory and can run on smaller hardware, at some cost in accuracy.
Data engineering in the AI eraintermediate
RAG
Also called Retrieval-Augmented Generation
A way to answer from your own documents. A search finds the passages that match the question, and they are added to the prompt before the model writes.
How AI worksintermediate
React
A JavaScript library for building user interfaces, particularly single-page applications.
React is a declarative, efficient, and flexible JavaScript library for building user interfaces. It lets you compose complex UIs from small and isolated pieces of code called components.
Next to it
AI agentsadvanced
ReAct Pattern
Reason + Action prompting framework for agents.
The LLM alternates between thought and action steps, deciding which tool to call and reflecting on observations.
Next to it
How AI worksintermediate
Red Teaming
Process of stress‑testing AI systems to find failure modes.
Teams intentionally try to induce unsafe or biased behaviour to improve safety.
Next to it
How AI worksadvanced
Reinforcement Learning from Human Feedback
Also called RLHF
A training technique that aligns model outputs with human preferences
RLHF uses human preference data to train a reward model. The base language model is then fine‑tuned via reinforcement learning to maximize that reward, producing more helpful and aligned responses—as in ChatGPT.
Next to it
How AI worksadvanced
Repetition Penalty
A logit scaling that discourages previously generated tokens to reduce loops.
Penalties help prevent the model from repeating phrases or sentences verbatim.
Next to it
Software and toolsintermediate
REST
Representational State Transfer - an architectural style for designing networked applications
REST (Representational State Transfer) is an architectural style for designing web services. It uses standard HTTP methods (GET, POST, PUT, DELETE) and follows principles like statelessness, uniform interface, and resource-based URLs to create scalable and maintainable APIs.
How AI worksadvanced
RLHF Reward Model
A model trained to score outputs according to human preferences.
Acts as a learned objective for policy optimisation in RLHF pipelines.
How AI worksadvanced
RoPE
Also called Rotary Positional Embedding, Rotary Position Embeddings
Rotary Positional Embedding; a positional encoding that rotates query and key vectors by relative angles.
RoPE enables extrapolation to longer context lengths with minimal overhead and is used in models like Llama 3.
RoPE enables extrapolation to longer sequences and is used in GPT‑NeoX, Llama, and others.
Alias term.
How AI worksbeginner
Safety Filter
A model or rule set that blocks disallowed content.
Often run after generation to remove harmful outputs before display.
Next to it
How AI worksbeginner
Scaling Law
An empirical relationship showing how model performance scales with data, compute, and parameters.
An empirical relationship showing how model performance scales with data, compute, and parameters.
Next to it
How AI worksbeginner
Score‑Based Model
A generative model trained to predict the gradient (score) of the data distribution’s log‑density.
A generative model trained to predict the gradient (score) of the data distribution’s log‑density.
Next to it
How AI worksintermediate
Self‑Attention
Also called Intra‑Attention
A mechanism that lets a model weigh the importance of different positions in a single sequence
Self‑attention computes pairwise interactions within a sequence, allowing the network to capture long‑range dependencies without recurrence. It is the core building block of the Transformer and many modern large language models.
Next to it
How AI worksbeginner
Self‑Consistency
An inference‑time strategy that samples multiple reasoning paths and selects the answer occurring most often.
An inference‑time strategy that samples multiple reasoning paths and selects the answer occurring most often.
Next to it
Data engineering in the AI erabeginner
Sentence Embedding
A vector representation capturing the meaning of an entire sentence.
A vector representation capturing the meaning of an entire sentence.
How AI worksintermediate
SentencePiece
A language‑independent tokenizer based on sub‑set Unigram/BPE models.
Developed by Google, SentencePiece treats input text as a raw byte sequence so it supports any language without preprocessing.
Next to it
Data engineering in the AI erabeginner
SentenceTransformers
A library for producing sentence and text embeddings based on transformer models.
A library for producing sentence and text embeddings based on transformer models.
Next to it
How AI worksadvanced
Sora
OpenAI's video‑generation model announced in 2025.
Uses diffusion‑transformer hybrid architecture to create high‑resolution, realistic videos up to 60 seconds.
Next to it
How AI worksadvanced
Sparse Expert Model
A model where only a fraction of parameters (experts) are active per token.
Cuts inference cost compared to dense models while scaling capacity.
Next to it
How AI worksbeginner
Speech‑to‑Text
The task or model that converts spoken language audio into written text.
The task or model that converts spoken language audio into written text.
How AI worksbeginner
Stable Diffusion
An open‑source latent diffusion model for text‑to‑image generation
Stable Diffusion, released by Stability AI in 2022, converts text prompts into high‑resolution images. Its open weights and permissive license sparked a wave of creative and commercial applications.
Next to it
How AI worksbeginner
Subword Token
A token representing a fragment of a word, enabling open‑vocabulary modeling.
A token representing a fragment of a word, enabling open‑vocabulary modeling.
Next to it
How AI worksintermediate
Supervised Fine‑Tuning
Also called SFT
Standard fine‑tuning on labeled input‑output pairs.
Often the first step before RLHF to teach task‑specific format.
Next to it
How AI worksbeginner
System Prompt
Also called System Message, Developer Message, Instructions
Instructions the builder of an AI app gives the model before you type anything. It sets the role, the tone and the rules, and it outranks what users say.
How AI worksbeginner
Temperature
Also called Temperature (Sampling), Sampling Temperature
A setting that controls how much chance goes into each word a model picks. Low gives focused, predictable answers; high gives more varied ones.
How AI worksbeginner
Tensor Parallelism
Distributing different tensor slices of a model layer across multiple GPUs.
Distributing different tensor slices of a model layer across multiple GPUs.
How AI worksintermediate
Text Generation Inference
Also called TGI
Hugging Face’s optimized server for LLM deployment.
TGI supports continuous batching, quantization, and streaming responses.
Next to it
How AI worksadvanced
Textual Inversion
Method that learns a new embedding token to represent a concept in diffusion models.
Allows teaching a model a new style or object without altering core weights.
How AI worksbeginner
Throughput
The number of tokens or examples a model can process per unit time.
The number of tokens or examples a model can process per unit time.
Next to it
How AI worksbeginner
Token
The small piece of text a language model reads and writes, often part of a word. Prices, limits and the context window are all counted in tokens.
How AI worksbeginner
Tokenization
The process of breaking text into tokens for model input
Tokenization converts raw text into discrete units—words, sub‑words, or characters—that a language model can process. Choice of tokenizer affects vocabulary size and handling of rare words.
Next to it
How AI worksbeginner
Tokenizer
Also called Tokeniser
The part that cuts text into the tokens a model reads, and turns the model's tokens back into text. Each model family has its own.
Next to it
AI agentsbeginner
Tool Use
Also called Tools
Giving a model a list of things it can ask to have done, such as search the web or read a file. The model asks; the program around it does the work and hands back the result.
How AI worksadvanced
Top‑A Sampling
Filtering method that selects tokens above a probability threshold A.
Variant proposed by OpenAI to balance quality and diversity during decoding.
Next to it
How AI worksintermediate
Top‑k Sampling
A decoding strategy that restricts choices to the k most probable tokens
During generation, the probability distribution is truncated to the top k options before sampling, reducing unlikely words and promoting diversity without completely random output.
How AI worksintermediate
Top‑p Sampling
Also called Nucleus Sampling
A decoding strategy that samples from the smallest set of tokens whose cumulative probability exceeds p
Also called nucleus sampling, top‑p adapts the candidate set size dynamically based on probability mass, balancing diversity and coherence better than fixed‑k methods.
Next to it
How AI worksintermediate
Transformer
The design behind almost every modern language model. It uses attention to let each word in a passage look at every other word, all at once.
Software and toolsintermediate
TypeScript
A strongly typed programming language that builds on JavaScript by adding static type definitions.
TypeScript extends JavaScript by adding static type definitions, making it easier to catch errors early and build more robust applications.
Next to it
How AI worksintermediate
Variational Autoencoder
Also called VAE
A generative model that learns a probabilistic latent space representation of data
A Variational Autoencoder (VAE) encodes input data into a latent distribution rather than a single point. By sampling from this distribution during decoding, VAEs can generate new, coherent samples. They balance reconstruction accuracy with latent-space regularization through the Kullback–Leibler divergence.
Next to it
Data engineering in the AI eraintermediate
Vector Database
Also called Vector Store
A database built to store embeddings and quickly find the ones closest to a query, which is how AI apps search by meaning.
Data engineering in the AI erabeginner
Vector Search
Finding items whose embeddings are most similar to a query vector.
Finding items whose embeddings are most similar to a query vector.
Next to it
How AI worksadvanced
Video Generation
Producing video sequences from text or images with generative models.
Approaches use latent diffusion, temporal U‑Nets, or autoregressive transformers.
How AI worksbeginner
Vision‑Language Model
A multi‑modal model jointly trained on image and text data to link visual and textual understanding.
A multi‑modal model jointly trained on image and text data to link visual and textual understanding.
Next to it
Data engineering in the AI eraintermediate
Weaviate
An OSS vector search engine with GraphQL API.
Supports modules for OpenAI embeddings, classification, and hybrid BM25‑vector search.
Next to it
Software and toolsintermediate
Webpack
A static module bundler for modern JavaScript applications
Webpack is a module bundler that takes modules with dependencies and generates static assets representing those modules. It processes your application, builds a dependency graph that maps every module your project needs, and bundles them into one or more optimized files for the browser. Webpack supports loaders and plugins to transform and optimize various file types.
How AI worksbeginner
Weight
A trainable parameter in a neural network that scales input values and is updated during training.
A trainable parameter in a neural network that scales input values and is updated during training.
How AI worksbeginner
Whisper
An open‑source multi‑lingual speech‑to‑text model released by OpenAI.
An open‑source multi‑lingual speech‑to‑text model released by OpenAI.
Next to it
Data engineering in the AI erabeginner
Word Embedding
An embedding specifically representing individual words, capturing semantic relationships.
An embedding specifically representing individual words, capturing semantic relationships.
Next to it
How AI worksintermediate
Zero‑shot Learning
Solving a task without any task‑specific examples
Zero‑shot generation relies on massive pre‑training so the model can infer the task solely from the prompt wording, without supplied examples.
Next to it
Data engineering in the AI erabeginner
Zilliz
The company behind Milvus, offering managed vector database services.
The company behind Milvus, offering managed vector database services.
Next to it
Try it
Some words are easier to play with than to read
A setting that controls how much chance goes into each word a model picks. Low gives focused, predictable answers; high gives more varied ones.
The cat sat on the …
- mat50%
- sofa22%
- floor17%
- roof7%
- keyboard4%
- moon1%
Low temperature: the likeliest word nearly every time. High: the unlikely words get a real chance. The scores are made up to show the effect.