Explore our new Cybersecurity Terms Glossary, explained by our experts.
Cybersecurity Terms Glossary
Retrieval-augmented generation grounds a model in your documents, and leaks when retrieval returns content the requesting user was never entitled to see.

What Is Data Leakage in RAG Systems?
Retrieval-augmented generation is the architecture behind most useful enterprise AI: instead of relying on what a model memorised during training, the application searches your own documents, places the relevant passages into the context window, and asks the model to answer from them. It solves grounding and keeps answers current. It also builds a second copy of your document estate with its own access control, and that is where the leakage happens.
A tempting shortcut is to retrieve broadly and then ask the model not to use anything the user should not see, or to post-filter the answer. Both fail. The model has already read the content, so it can be led to paraphrase, summarise, or confirm it, and a refusal is a behaviour rather than a boundary. Worse, partial information leaks through acknowledgement: asking whether a document about a redundancy programme exists, and receiving a careful refusal, is itself a disclosure. Access control has to happen before content enters the context window, not after.
Attach access control metadata to every chunk at indexing time, derived from the source document, and filter the search by the requesting user’s entitlements before ranking. Where a platform supports it, query the source system with the user’s own token so permissions are evaluated at the source of truth. Keep tenant and department boundaries as separate indexes rather than as metadata filters where the data is highly sensitive, since a filter bug then fails closed rather than open. Re-index on permission change, not only on content change, because a document whose access was revoked last week is still in the index. Strip hidden text, comments, metadata, and invisible characters before indexing to remove injected instructions, and preserve classification markings into each chunk so the application can enforce them.
Functional testing asks whether the assistant finds the right document. Security testing asks what it finds for the wrong person. Create accounts at several permission levels, place known sensitive documents outside each one’s entitlement, and probe with direct questions, oblique ones, and summarisation requests. Check that denial is consistent and uninformative. Test what happens to a document after its permissions are tightened, and whether the system answers from a cache. Confirm that logs do not become an unprotected copy of restricted content. This belongs in the scope of your AI red teaming, alongside the injection paths.
Under UK GDPR, an employee receiving personal data they have no business need to see is unauthorised processing, and a leak of special category data through an assistant is a reportable breach with the usual 72-hour clock. The ICO’s expectations on access control and data minimisation apply to the index exactly as they do to the source repository. Practically, this also makes erasure harder: a subject access or erasure request must reach the index, the embeddings, the caches, and the logs, which is a requirement to design in rather than discover later. OWASP tracks the underlying weaknesses as LLM08 and sensitive information disclosure as LLM02. At Privia Security, we test retrieval pipelines, index permissions, and derived stores through our AI Penetration Test Services.
No. An instruction in the system prompt is a behavioural preference, not an access control, and it can be overridden by crafted input or simply ignored. Once content is in the context window, it can be paraphrased, summarised, or confirmed. Filter by the requesting user’s entitlements before retrieval places anything in the context.
In most cases the requesting user already had access through an inherited permission or an old sharing link, and the assistant removed the practical obscurity that kept the file hidden. The fix is in the document estate: audit over-permissioned locations and links, apply sensitivity labels, limit the indexing scope, and confirm permissions are enforced at query time.
Map every derived artefact back to its source document and data subject at indexing time: chunks, embeddings, caches, and logs. An erasure request then becomes a traceable deletion across all of them. Retrofitting this mapping is painful, so include it in the requirements before the first index is built, and test the deletion path end to end.
For highly sensitive data, yes. Separate indexes fail closed if the application has a bug, whereas a single index relying on metadata filters fails open. The trade-off is operational overhead and the loss of cross-department answers, so most organisations separate the few genuinely restricted corpora and use filtered retrieval for the rest.
You May Be Interested In These
We use a few cookies that are needed to make this website work, and they are always on. We would also like to set analytics cookies, to see which pages are useful; functional cookies, so that our live chat works; and advertising cookies, to measure our campaigns. We will not set any of those until you say yes, and you can change your mind at any time from the Cookie Settings link in the footer.
Choose the cookies you are happy for us to use. Strictly necessary cookies are always on; everything else is off until you turn it on.
These cookies keep the website working: they hold your security checks, your form submissions and the choice you make here. The site cannot run without them, so they are always on.
These let our live chat open and remember your conversation, and let the newsletter sign-up work. Switch them off and the chat window will not appear.
These tell us which pages are read and how people find us, so that we can improve the site. The reports are about groups of visitors, never about you personally.
These let us measure our advertising and show you our adverts on other websites, such as LinkedIn, Google and Meta. Switch them off and you will still see adverts, but they will be less relevant.