What is a data contract?

Most broken pipelines are broken by someone who did nothing wrong. An app team renames a column in their own database, as they are entitled to, and the next morning 3 dashboards and a model are reading empty values. Nobody told the data team, because nobody knew the data team depended on that column.

A data contract is an agreement, written down and checked, between the team that produces a dataset and the teams that use it.

What goes in one

A contract states what the producer promises. The details vary, but the common parts are:

• Schema: the columns, their types, which are required, and what each one means.
• Quality: rules the data will meet, such as "every order has a positive total" or "no duplicate order ids".
• Service level: how fresh the data will be, and how often it updates.
• Ownership: which team owns it and how to reach them when something is wrong.

There is an open standard for writing these down. The Open Data Contract Standard, kept by the Bitol project under the LF AI & Data Foundation, sets out the keys of a contract as a YAML file, with sections for schema, data quality, service levels, team, roles and support channels. A shared format means tools can read a contract and check data against it, rather than leaving it in a wiki.

A contract is only as good as its checks

A contract nobody checks is documentation, and documentation drifts. The value comes from turning each promise into a test that runs:

• The quality rules become pipeline checks. On Databricks they map naturally to expectations that drop a bad row or stop the update.
• The schema becomes a hard rule at the point of loading. Auto Loader, for example, can be set to fail when a new column appears instead of absorbing it, so a change the contract does not allow stops at the door.

And the most important check runs before the change ships: the producer's own tests fail when they change something a contract promises. That turns a broken dashboard into a conversation.

How it fits with the rest

Data qualityData QualityHow far data can be trusted: complete, correct, on time and in the expected shape. Pipelines check it with rules and stop or flag rows that fail. rules say what good data looks like. Schema driftSchema DriftWhen the shape of incoming data changes without warning, such as a renamed or new column, and breaks or silently corrupts the pipelines that read it. is what happens when the shape changes without warning. A contract is the agreement that makes both someone's responsibility. LineageData LineageThe record of where a piece of data came from and everything it passed through, from source tables to the dashboards, models and AI answers built on it. shows who would be hurt by a change, which is how you know who needs a contract with whom.

Why it matters for AI

An AI assistant built on company data is one more consumer, and a fragile one. A dashboard that breaks shows an error. An assistant whose table quietly lost a column may keep answering, just with less information, and nobody sees the gap. The tables an assistant reads are the ones most worth putting under contract, because the reader cannot tell anyone when something changed.