Explore our new Cybersecurity Terms Glossary, explained by our experts.
Cybersecurity Terms Glossary
A membership inference attack determines whether a specific record was in a model’s training data, which can disclose something sensitive even when no data is extracted.

What Is a Membership Inference Attack?
A membership inference attack answers one question: was this particular record part of the data used to train this model. It sounds modest compared with stealing a dataset, and in most contexts it is. But if the model was trained on the medical records of people with a specific condition, on the customers of a debt advice service, or on the case files of a domestic abuse charity, then membership is itself the sensitive fact. The attacker learns something about an individual without ever seeing a row of data.
Models behave differently on data they have seen. Confidence tends to be higher, loss lower, and the output distribution sharper for training examples than for new ones—a side effect of memorisation that increases with overfitting. Reza Shokri and colleagues formalised the attack in 2017 using shadow models: train several models on data drawn from a similar distribution, label their outputs as member or non-member, and train a classifier to tell the difference. That classifier is then applied to the target model’s responses. Later work simplified the approach to thresholding loss values and refined it with per-example difficulty calibration, which made the attacks far more reliable at low false positive rates.
With large language models the question shifts from a statistical inference to outright extraction. Research published in 2021 showed that training data could be recovered verbatim from a production language model, including names, phone numbers, and addresses that appeared only a handful of times in the corpus. Later studies showed that simple techniques such as repeating a token could trigger a model to emit memorised passages. Rare, unusual records memorise most readily, which inverts the comforting intuition that an individual is safe in a crowd: the more distinctive your data, the more likely it is to be recoverable. Fine-tuning on a small internal dataset is particularly exposed, because repetition over a limited corpus drives memorisation hard.
The ICO’s position on identifiability is the relevant test. If a model can be queried to reveal that a named individual was in a training set, that processing concerns personal data, and the model itself may need to be treated as containing personal data for the purposes of UK GDPR. Three consequences follow. Article 32 security obligations apply to the model and its inference API. Erasure requests become genuinely difficult, since deleting a row from a dataset does not remove its influence from trained weights, and retraining may be the only complete remedy. And a model published or shared with a partner is a disclosure, which needs a lawful basis and a transfer analysis in the same way a dataset would. These questions arrive alongside embedding inversion, where derived artefacts turn out to carry the source data with them.
Membership inference is the weakest of a family and often the first step. Attribute inference predicts a missing sensitive field for a known individual. Property inference discloses characteristics of the training population, such as the demographic composition of a hospital’s dataset. Training data extraction recovers records outright. Model extraction copies the model, after which every other attack can be run offline without rate limits. NIST AI 100-2 groups these under privacy attacks and is a useful reference when documenting your threat model. At Privia Security, our Cyber Risk Analysis Services assess what a trained model discloses before it is shared or published.
It can be. If individuals can be identified from the model—through membership inference, attribute inference, or verbatim extraction—then the identifiability test the ICO applies is met, and the model falls within scope. The answer depends on the data, the training method, and the access an attacker has, so it needs assessing per model rather than settling as a general rule.
Remove the record from the source dataset and from any derived stores such as embeddings, then decide whether the trained model still carries its influence. For high-risk data, scheduled retraining is the only complete answer. Machine unlearning techniques exist but offer weaker assurances. Document your approach and the reasoning, because this is a question the ICO expects you to have considered.
Not reliably. Removing direct identifiers does not stop a model memorising a distinctive combination of attributes, and unusual records are exactly the ones most likely to be memorised. Pseudonymised data remains personal data under UK GDPR. Differential privacy during training provides a measurable bound; removing names does not.
Classic shadow-model approaches need thousands to build a reliable classifier, while simple loss-threshold attacks can work with very few per target once calibrated. For generative models, extraction may need only a handful of well-chosen prompts. Rate limiting and authentication raise the cost of the high-volume variants but do not address the low-query ones.
You May Be Interested In These
We use a few cookies that are needed to make this website work, and they are always on. We would also like to set analytics cookies, to see which pages are useful; functional cookies, so that our live chat works; and advertising cookies, to measure our campaigns. We will not set any of those until you say yes, and you can change your mind at any time from the Cookie Settings link in the footer.
Choose the cookies you are happy for us to use. Strictly necessary cookies are always on; everything else is off until you turn it on.
These cookies keep the website working: they hold your security checks, your form submissions and the choice you make here. The site cannot run without them, so they are always on.
These let our live chat open and remember your conversation, and let the newsletter sign-up work. Switch them off and the chat window will not appear.
These tell us which pages are read and how people find us, so that we can improve the site. The reports are about groups of visitors, never about you personally.
These let us measure our advertising and show you our adverts on other websites, such as LinkedIn, Google and Meta. Switch them off and you will still see adverts, but they will be less relevant.