# What is a transformer in AI?

The design behind almost every modern language model. It uses attention to let each word in a passage look at every other word, all at once.

- Page: https://tecyfy.com/concepts/transformer
- Subject: How AI works (https://tecyfy.com/concepts)
- Level: intermediate
- Last checked: 2026-10-14

Most of the language models you have heard of are built on one design, the transformer. It comes from a 2017 paper by 8 researchers, with a title that became famous: "Attention Is All You Need".

## What came before

Earlier language systems read text the way you read a line of print, one word after another, carrying a running summary forward. That made them slow to train, because each step had to wait for the last.

## What the transformer changed

The transformer drops the word-by-word reading. It looks at the whole passage at once and uses a mechanism called [attention](https://tecyfy.com/concepts#self-attention) to decide, for each word, which other words matter to it.

Take "The trophy did not fit in the case because it was too big." To understand "it", the model has to connect it to "trophy", not "case". Attention is how it makes that connection. Each word asks a question of every other word, and the answers are weighted by how relevant they are. A transformer does this many times over, in many layers, and with several "heads" each looking for a different kind of link.

Because every word is handled at the same time, the work can be split across many chips. That is the practical reason the design won: in the paper it beat the older systems at translation while being far easier to run in parallel. Training on far more text, on far more chips, is what made models large.

## The part that still runs one at a time

Reading is parallel. Writing is not. A language model built on a transformer still produces its answer one [Token](https://tecyfy.com/concepts/token) after another, and each new token can look back at everything before it but not ahead. The [KV cache](https://tecyfy.com/concepts/kv-cache) exists to make that step cheaper.

## Why you might care

You never set up a transformer to use ChatGPT. But the design explains 2 things you do notice. A model can connect a detail on page 1 with a question on page 40. And the work of attention grows with the length of the text, faster than the text itself, which is part of why a long [context](https://tecyfy.com/concepts/context-window) is slow and costly.

## Related

- [Large Language Model](https://tecyfy.com/concepts/large-language-model): A model trained on a huge amount of text to predict the next token. Further training turns it into an assistant that follows instructions.
- [Self‑Attention](https://tecyfy.com/concepts#self-attention): A mechanism that lets a model weigh the importance of different positions in a single sequence
- [Multi‑Head Attention](https://tecyfy.com/concepts#multi-head-attention): Parallel attention layers allowing a model to focus on different subspaces
- [Positional Encoding](https://tecyfy.com/concepts#positional-encoding): A method to inject position information into Transformers
- [KV Cache](https://tecyfy.com/concepts/kv-cache): Work a model saves from the text it has already read, so it is not redone for every new token. Prompt caching keeps that work between requests and charges less for it.
- [Token](https://tecyfy.com/concepts/token): The small piece of text a language model reads and writes, often part of a word. Prices, limits and the context window are all counted in tokens.

## Sources

- [Attention Is All You Need](https://arxiv.org/abs/1706.03762), Ashish Vaswani and others, arXiv 1706.03762, 2017-06-12. The dominant sequence models were based on recurrent or convolutional networks. The paper proposes the Transformer, based solely on attention mechanisms. On 2 machine translation tasks it was better in quality while being more parallelizable. It reached 28.4 BLEU on English-German and 41.8 BLEU on English-French. The 8 authors include Ashish Vaswani and Noam Shazeer. Version 1 was posted on 12 June 2017.
- [How caching works](https://huggingface.co/docs/transformers/main/en/cache_explanation), Hugging Face, Transformers documentation, 2026-10-14. Attention uses query, key and value matrices projected from the token representations. In causal attention a mask stops the model from attending to future tokens. Attention is calculated independently in each layer of the model. Without a cache, the attention cost of each new token is quadratic with the length of the sequence.
- [Glossary](https://platform.claude.com/docs/en/about-claude/glossary), Anthropic, Claude API documentation, 2026-10-14. Autoregressive language models such as Claude's underlying model are pretrained to predict the next word, given the previous context.