Databricks agent memory: how managed memory, sessions and memory pipelines work

Databricks agent memory is a managed long-term memory for AI agents: Databricks keeps it in Lakebase and runs the storage, indexing and search, so your agent can remember a user's preferences and past decisions in a later conversation without you running a database. It is in Beta as of October 2026, works with agents on any framework, and comes as 3 parts that are easy to mix up.

• A session store holds one conversation, the transcript the agent replays to carry on where it stopped.
• A memory store holds the facts worth keeping after the conversation ends, found again by search.
• A memory pipeline reads the session store and writes the lasting facts into the memory store.

The first 2 are where your agent reads and writes. The third is the new idea, and the one that behaves least like you would guess.

Sessions or memory: which one holds what

The split is the one in Agent MemoryAgent MemoryThe ways an agent keeps what it needs beyond a single context window, such as summaries, notes in files, and saved facts it reads back later. The model itself remembers nothing.: short-term state for this conversation, long-term knowledge for the next one. In Databricks' memory research, the session store holds episodic memory, the raw record of each session, and the memory store derived memory, the facts distilled from it.

Deleting a session, or a whole session store, leaves the memories taken from it alone. That is on purpose: the conversation can age out while what it taught the agent stays.

Wire it into the agent you already have

Everything is in one package, databricks-agentbricks 0.6.0 or later: the AgentKit SDK, the agentbricks CLI and adapters for the OpenAI Agents SDK and LangGraph. Create each store once and reuse it:

from databricks.sdk import WorkspaceClient
from databricks_agentkit import AgentKitClient

client = AgentKitClient(WorkspaceClient())
client.session_stores.create("support-agent-sessions")
client.memory_stores.create("support-agent-memory")

For an agent on the OpenAI Agents SDK, the conversation moves into the session store by swapping the Session it already passes to Runner.run. The agent's model, tools and prompt stay as they are:

from agents import Runner
from databricks_agentkit.openai import session_store

async def handle_message(user_id: str, conversation_id: str, message: str) -> str:
    session = session_store(conversation_id, actor=user_id, store="support-agent-sessions")
    result = await Runner.run(agent, message, session=session)
    return result.final_output

A LangGraph agent swaps its checkpointer for checkpointer(store=...) and passes thread_config(conversation_id, actor=user_id) instead. That checkpointer saves the whole graph state, paused Human in the LoopHuman in the LoopA person approves, corrects or stops an AI system at set points, such as before an agent sends, pays or deletes something. stored as serialized checkpoints in the session store, so the graph's state survives a restart. Keep that in mind: it matters again in the pipeline section.

For memory there are 2 patterns, and most agents use both. memory_tools(user_id, store=...) gives the model 4 tools, recall, remember, forget and list_memories, already bound to one user (the same idea as the Memory ToolMemory ToolA component that stores and retrieves conversational or task‑relevant information for an agent. on the Claude API). And your own code can search memory with the incoming message before each run and add what it finds to the instructions:

memory = client.memory_stores.get("support-agent-memory")

def recalled_facts(user_id: str, message: str) -> str:
    results = memory.search(actor_id=user_id, query=message, limit=5)
    return "\n".join(f"- {r.memory.content}" for r in results)

Put those facts in the instructions, not the input, so the session does not store them as if the user had said them. Two details from the memory API catch people: add() fails with ALREADY_EXISTS when the path is taken, so use put() to create or replace, and put() is not atomic, so 2 writes to the same path at once can overwrite each other.

Or let a pipeline write the memory

An agent that saves its own memories does it one conversation at a time, seeing only that conversation. Over months that gives overlapping entries, entries that contradict each other, and facts the model never thought to save. A memory pipeline works afterwards, with all of one user's conversations and existing memory in view. Each run, for each actor:

1. Distill: read the actor's conversations since the last successful run and pull out durable facts with a model, called through Unity Gateway.
2. Merge: remove duplicates and fold the facts into topic entries under the same actor_id.
3. Maintain: merge entries that overlap and split ones that grew too large.

agentbricks memory pipeline create \
  --session-store support-agent-sessions \
  --memory-store support-agent-memory \
  --trigger scheduled \
  --instructions "Keep durable customer preferences and account details. Ignore one-off troubleshooting."
agentbricks memory pipeline run <pipeline-id>

What it keeps is narrower than "everything the user said". It is built to keep facts and preferences the user states, corrections, and durable knowledge such as which table is the source for a metric. It is built to skip values that change (a cluster's worker count right now), anything the agent can look up with its tools (a table's columns), and secrets. Try a support conversation through it, and change how the conversation was stored:

Try itWhat a memory pipeline keeps from one conversation

Conversation stored by

·OpenAI Agents SDK session·LangGraph checkpointer·LangGraph + transcript store·Own code, message items

Last message sent

Over a minute agoSeconds ago

Agent searches with

Same actor_idAnother id

Agent also saves with a tool

NoYes

1. ✓userEmail me rather than call, please. I'm on Pacific time.kept: merged into /preferences/contact.md
2. –tool callget_cluster(name="etl-prod")skipped: a tool call
3. –tool result{"state": "RUNNING", "workers": 14}skipped: a tool result
4. –assistantetl-prod is running with 14 workers right now.skipped: a live value that changes; it keeps how to find it, not the figure
5. ✓userFor revenue, always use finance.gold.revenue, not the sales table.kept: merged into /knowledge/sources.md
6. –assistantfinance.gold.revenue has 9 columns: order_id, booked_at, amount_usd, ...skipped: a detail the agent can look up with its tools
7. –userHere's my token dapi8f2... if you need it.skipped: a secret
8. ✓userWe move the workspace to eu-west-1 next quarter.kept: merged into /account/plans.md

Memory store, actor user-123

• /preferences/contact.mdPrefers email over phone. Pacific time zone.
• /knowledge/sources.mdRevenue figures come from finance.gold.revenue, not the sales table.
• /account/plans.mdMoving the workspace to eu-west-1 next quarter.

Search from the agent: 3 matches for user-123

The LangGraph case is the one to remember. The pipeline reads only user and assistant messages, and a checkpointer stores serialized graph checkpoints, so a pipeline pointed at it finds nothing to distill and still reports SUCCEEDED. The fix in Databricks' tutorial is a second session store that your code appends each turn's user message and reply to, with the pipeline created on that one.

When a run writes nothing, or the agent cannot find it

The pipeline's failures are mostly quiet. Before you open a ticket, go through these:

• No messages in a shape it reads. It reads {"type": "message", "role": "user" | "assistant", ...} items and the OpenAI Responses message shape. Tool calls, tool results and LangGraph checkpoints are skipped.
• Nothing new. A run keeps a checkpoint per actor and processes only turns added since then.
• Too recent. Turns from the last 60 seconds wait for the next run, which matters in a test script that sends a message and starts a run straight away.
• The wrong actor. The pipeline writes each user's memories under the actor_id their sessions use. If the agent searches with an email while its sessions use an internal user id, every search is empty.
• A run already going. One run at a time; a second one fails with ABORTED.
• Too many users. A run fails above about 1,200 actors with UUID ids, the limit being 48 KiB of serialized actor ids. Shorter ids fit more.

2 limits to plan around rather than debug. A scheduled pipeline runs about every 24 hours, with no other interval and no cron, so memory from this morning's conversation may not be there this afternoon; start a run yourself when that matters. And the store pair is fixed: to point a pipeline at another store you delete it and create a new one, which does not reuse the old checkpoints and reprocesses every actor's full history, model calls included.

The rule to get right: actor_id is not access control

Every list and search is scoped to one actor_id, so it looks like a wall between users. It is not. A memory store is scoped to the workspace, and any principal that can reach it can read and write every entry for every actor. The store is the security boundary.

3 things follow from that:

• Set actor_id in your code from the verified signed-in user, never from the model or the message. If a request has no verified user, refuse it rather than falling back to a shared id.
• One store per tenant when customers must never see each other's data, for sessions and memory both.
• Check the owner when resuming. Getting a session by its session_id is not scoped to an actor, so compare session.actor_id with the requesting user before you replay it.

The agent's service principal needs grant_permission on each store it uses (pass permission="READ" for read-only). A pipeline does not: its runs use the identity of whoever created it. Keep that identity in mind as well, because a pipeline can rewrite, merge or delete any entry under an actor it processes, including entries the agent saved with its own tools. If an entry must stay word for word, keep it in a store no pipeline writes to.

What it costs

During the Beta, agent memory and agent sessions have no charge of their own: you pay for the Lakebase capacity that holds them, from $0.092 per CU-hour on AWS US East Premium. A pipeline run is billed as serverless general compute for the job plus Foundation Model Serving for the model that distills the memories, so the model you pick with --model and the size of the history it reads drive that part. The first run reads every actor's full history; after that each run reads only what is new. Databricks says the pricing may change as the Beta goes on.

If you built memory the earlier ways

There are 2 earlier approaches, and Databricks lists the first as legacy:

• Managed agent memory (legacy) was a Unity Catalog securable, created with the CREATE MEMORY STORE privilege, with entries keyed by a scope and a path. It is deprecated. The new store is not a Unity Catalog object: it is scoped to the workspace and secured by grants on the store, and scope becomes actor_id.
• Self-managed memory on Lakebase is the app template route, where you run the Postgres schema and connections yourself. Databricks still points to it when you need SQL access to memories or a schema of your own, and to agent memory when you need neither.

Moving conversations across is a copy: switching an agent to the session store does not bring old conversations with it, so append their items to new sessions, then run a pipeline once to build memory from them.

If you are choosing a whole agent platform rather than a memory store, Agent Bricks covers what replaced the earlier agent tools, and /ai-agents follows one request through an agent step by step.