What is a data lake?
Cheap storage that holds data of any kind in its raw form, usually as files in cloud object storage, to be cleaned and structured later.
beginner
In a 2010 blog post about Pentaho and Hadoop, James Dixon needed a name for the raw data a company keeps before anyone has decided what it is for. He compared a data mart to bottled water, "cleansed and packaged and structured for easy consumption", and the data lake to "a large body of water in a more natural state". The name stuck.
Store first, decide later
A data lake is cheap storage that holds data of any kind in its raw form: database exports, log files, CSVs, JSON from an app, PDFs, images, recordings. The files are usually in open formats such as Parquet, so many different tools can read them.
The difference from a warehouseData WarehouseA database built for analysis: cleaned, structured data from many systems, kept in one place and shaped for fast reports and dashboards. is when structure arrives. A warehouse is schema-on-write: data must fit a designed table before it is stored. A lake is schema-on-read: you store the file as it came, and decide what the columns mean when you read it. That makes a lake fast to fill and able to keep data no one has a use for yet.
From Hadoop to cloud storage
The first lakes ran on Apache Hadoop and its file system, HDFS, on a company's own servers. From around 2015 cloud object storage (Amazon S3, Azure Data Lake Storage, Google Cloud Storage) replaced HDFS for most lakes. It is very cheap, keeps data safe across places, and can move old data to even cheaper archive tiers.
The cost of no rules
What a lake gains in freedom it gives up in guarantees. Nothing stops a malformed file from landing next to good ones. A folder of files has no idea which ones make up the current version of a table, so a reader can pick up half of a write. Quality and governance are left for whoever reads the data later.
That is why lakes were usually paired with a warehouse. The lake kept everything; a cleaned subset was copied into the warehouse for the reports people trusted.
The fix, in the end, was to bring the guarantees to the lake. An open table formatOpen Table FormatA published way of storing a table as plain data files plus a log of changes, so many engines can read and safely write it. Delta Lake and Apache Iceberg are the best known. adds a log that makes a folder of files behave like a table, with whole writes and versions. A lake with that layer and a catalog on top is what is now called a LakehouseLakehouseOne store for all of an organisation's data: cheap files in cloud storage, as in a data lake, with the tables, transactions and fast queries of a data warehouse on top..
Why it matters for AI
The data a modern AI system wants most is the data warehouses never held: support tickets, contracts, product manuals, call recordings. It sits in the lake. An assistant that answers from company documents is reading lake files through a chunkingChunkingCutting documents into smaller passages before they are embedded for retrieval, so a search returns the part that answers a question rather than a whole file. and embedding pipelineEmbedding PipelineThe pipeline that turns source documents into searchable vectors: it reads new and changed content, chunks it, embeds each chunk and keeps the vector index in step with the source..
So the old lake problem comes back in a new form. If nobody knows which PDF is the current policy, the assistant will not know either. Raw storage is cheap; knowing what is in it is the expensive part.
Sources