How do you make sure a RAG system only answers from documents the user is allowed to see?

Enforce permissions in retrieval, before anything reaches the model. Store each chunk with the access rules of its source document, filter every search by the current user's identity, and never rely on the prompt ("don't reveal confidential documents") to protect data. If a chunk is in the prompt, assume the user can see it.

Why it matters

A RAG system flattens many sources into one index. Without filters, an intern's question can retrieve a passage from the board's minutes, and the model will happily summarise it. OWASP lists this under vector and embedding weaknesses (LLM08:2025): unauthorised access, and leaks between tenants sharing one vector store.

How to build it

1. Copy the permissions at indexing time. Each chunk carries metadata: document id, tenant, owner, groups or ACL entries, classification level.
2. Filter at query time. Look up the user's groups from your identity provider and apply them as a filter in the search itself: WHERE tenant_id = :t AND acl && :user_groups in pgvector, or a metadata filter in a vector database. Filter both the vector and keyword searches before merging.
3. Partition hard boundaries. Separate indexes or namespaces per customer in a multi-tenant product, so a missing filter cannot cross tenants.
4. Keep permissions in sync. When a document's sharing changes, update its chunks' metadata. When access is revoked or a document deleted, remove or update its chunks quickly. Stale permissions are a common hole.
5. Check again after retrieval. For sensitive systems, re-check each retrieved document against the source system's permissions before building the prompt. It is a cheap second gate.

Other leak paths

• Caches. An answer cached for one user must not be served to another with different rights. Key caches by permission set, or do not cache sensitive answers.
• Logs and traces. Prompts in logs contain the retrieved text. Restrict who can read them.
• Embeddings themselves. OWASP notes that embeddings can sometimes be inverted to recover text, so protect the vector store like the documents.
• Tools. If the agent can also call APIs, those calls must run with the user's rights, not a powerful service account.

Poisoned documents

Anyone who can add a document to the index can add instructions ("ignore previous rules…"). Validate what is indexed, prefer trusted sources, and treat retrieved text as data, not instructions.

How to test it

Create test users with different rights and ask questions whose answers sit in documents they must not see. The expected result is "I can't find that", every time. Keep these tests in your evaluation set, and log every retrieval with the user and chunk ids so you can audit later.