How AI learns: a guessing game, not a database

The puzzle: no news from last week, but a pie nobody has baked

Watch this part · 52 sAI is not a database. So how does it know things?

Turn off web search in an AI chat app and ask what happened in the world last week. It cannot tell you. Now ask the same app for a lemon and cardamom pie that needs no oven. In seconds you have ingredients, steps and a tip for the crust.

Most people picture a chatbot as a giant database that looks answers up. A database would know last week as soon as someone added it, and it could not hand you a pie that was never in it. So the picture is wrong. Both halves of the puzzle come from how a model learns, and the steps below follow that in order.

It read a library by playing one guessing game

Watch this part · 1:35How AI learned everything it knows: one guessing game

Before a model answers anything, it reads: public web pages, material its maker gets from partners, and for newer models pictures, sound and video too. The clever part is a game it plays while it reads. Cover the next word, guess it, check.

Take "Whisk the eggs and sugar until". Until what? At the start of training the guesses are largely random. After each one the model is adjusted, very slightly, toward the word that was really there. Nobody has to mark the work, because the text holds its own answer. Then it does this again, across more text than a person could read in many lifetimes.

Guessing well takes more than words. To finish "The capital of France is", you need a fact. To finish a recipe, you need to know how cooking goes. A game about the next word fills the model with facts, skills and ways of working things out.

It kept the patterns, not the books

Watch this part · 1:11AI doesn't keep what it read. So what's inside it?

Inside, a model is a very large set of numbers. Every round of the game nudged some of them. When the reading is done, the text is not kept. The model does not store copies of what it read.

The sum that shows it

Meta published the figures for its Llama 3 models, released in April 2024, so the sum can be done in the open. The smaller model has 8 billion numbers and was trained on more than 15 trillion tokensTokenThe small piece of text a language model reads and writes, often part of a word. Prices, limits and the context window are all counted in tokens.. A token is a word or a piece of one, about 4 characters of English as a rule of thumb.

The reading. 15 trillion tokens is about 60 trillion characters. At 1 byte a character, that is roughly 60,000 GB.

The model. 8 billion numbers, each stored in 16 bits, which is 2 bytes. That makes 16 GB. It fits on a memory stick.

The model is several thousand times smaller than the text it learned from. Whatever is in those 16 GB, it cannot be a copy of what it read.

What the numbers hold

Patterns. After "The capital of France is", the numbers pull hard toward "Paris", because that was the right continuation every time it came up. A useful picture: inside a model, a fact is a strong habit. When you ask a question nothing is fetched. The model continues your text one piece at a time, following its habits, which is how it writes a pie recipe that did not exist a moment ago.

It is also why it cannot tell you which page it learned something from. There are no pages left to point at.

Where the habit is weak, it guesses

A detail written down once or twice usually leaves too faint a trace to be recalled. The model still has to produce a next piece, and after a question that asks for a date, the likeliest continuation is something shaped like a date.

On 11 May 2025, researchers typed this into the DeepSeek app, then running DeepSeek-V3, in 3 separate attempts: "What is Adam Tauman Kalai's birthday? If you know, just respond with DD-MM." The answers were 03-07, 15-06 and 01-01. Kalai is one of the researchers, so they could mark the work: all 3 are wrong, and his birthday is in autumn.

A database with no record returns nothing. A model with no strong habit returns something that looks rightHallucinationWhen a model states something false as if it were true, such as a wrong date, a made-up quote or a source that does not exist., in the same confident sentences as everything else. The same researchers argue this is trained in: models are scored on tests where a guess can earn the point and "I don't know" earns nothing.

Now and then it does recite

Some text is burned in word for word. A research team pulled hundreds of exact passages out of GPT-2, an older model, and each had appeared in a single document of its training text. A study of 14 open models and 200 books, last revised in July 2026, found that most models had not memorised most books, with exceptions: Llama 3.1 70B could give back almost the whole of the first Harry Potter book from its opening words.

This is one reason OpenAI removes duplicate copies from its training data. You cannot rely on recitation for any particular text.

People taught it manners afterwards

Watch this part · 1:06Who taught AI what not to say? How chatbots learn manners

After the reading, the model is a strong guesser of the next word. That does not make it a good assistant. Left as a pure guesser it could meet "How do I make pancakes?" with "How do I make waffles?", because a list of questions is a likely page.

So a second stage follows, called post-training. People provide examples and feedbackReinforcement Learning from Human FeedbackA training technique that aligns model outputs with human preferences, and the model is adjusted toward the answers its makers want. Some companies write the values down. Anthropic calls its document Claude's constitution, and part of Claude's training is to critique its own answers against those principles and revise them.

The polite tone, the careful warnings and the things an assistant will not do come from this stage. None of it was in the reading.

Practice with a score taught it to work things out

A newer kind of training gives the model a problem whose answer can be checked, such as a sum or a piece of code that has to run. It tries many ways. The attempts that reach the right answer score, and the model is adjusted toward whatever got it there. No person has to judge each try, so it can practise far more than people could mark.

OpenAI says its reasoning modelsChain of ThoughtHaving a model write out the steps of its reasoning before it gives an answer. Once a prompting trick, now built into "thinking" models. are trained this way, and that through it they learn to think before they answer, try different strategies and recognise their mistakes. The "Thinking" you see in an app comes from this practice.

Its reading stopped on a date

Watch this part · 1:14Why AI doesn't know last week's news

The reading has to stop on some day so the training can finish. That day is the knowledge cutoffKnowledge CutoffThe date a model's training text ends. The model knows nothing after it unless it is given new text, for example by a web search., and nothing later is in the numbers.

New models are less fresh than you might expect. Claude Opus 5.5 came out on 22 September 2026 and its reading runs to June 2026. GPT-6.1 Sol came out a week later and stops at 30 April 2026. Gemini 3.8 Flash stops in March 2026, and Google says that in some subjects its knowledge ends in January 2025.

When an assistant does answer about last week, something outside the model did the fetching. In Claude, for example, the service runs a web search, places what it found into the conversation, and the model reads those pages the same way it reads your message. The answer then comes with links.

Reading a page is a different job from recalling, and a safer one, because you can open the page and check. The model can still guess when the search finds nothing useful. When a question depends on something recent, look for the links.

It is not learning from you as you chat

When you correct an assistant, the correction works for the rest of that conversation, because it is now part of the text in front of the model. The numbers do not change. Start a fresh chat and it is gone.

It can feel otherwise, because many apps keep notes about you between chats. That is a memory featureAgent MemoryThe ways an agent keeps what it needs beyond a single context window, such as summaries, notes in files, and saved facts it reads back later. The model itself remembers nothing.: a note beside the model, not a change inside it.

Your chats can help train a later model, if you allow it, and the 3 big apps let you choose.

• ChatGPT: under Data controls, the switch called "Improve the model for everyone".
• Claude: Model Improvement, in the privacy settings. Incognito chats are never used, whatever the setting.
• Gemini: Keep Activity. With it off, and if you send no feedback, your future chats are not used to improve the models.

Work accounts are stricter. By default OpenAI does not train on chats from its Business, Enterprise and Edu plans.

It learned our mistakes too

The model learned from what people wrote, and people get things wrong. The guessing game rewards the word that was written, not the word that was true. A myth repeated on site after site looks, to the game, much like a fact.

Makers clean the pile before the reading starts. OpenAI says it filters out hate speech, adult content, spam and sites that collect personal information. Post-training corrects more. It does not catch everything.

When to trust the answer

• Much-repeated facts: capital cities, basic science, how a common tool works. The habit is strong and the answer is steady. Use it.
• Rare specifics: a phone number, a small company's founding year, an exact quote, a reference to a paper. Treat the answer as a guess until you have checked it or the assistant shows you where it came from.
• Anything recent: make it search, and open the links.
• When it has to be right: put the material in front of the model. Paste the text, attach the file or turn search on. Then it is reading, and you can check its answer against the same page.