
Pseudonymisation or anonymisation: what IT does so test data is not personal data
Pseudonymisation and anonymisation are often used as if they mean the same, but the difference decides whether data protection law still applies to your data. A dataset with the names removed is usually not anonymous but pseudonymised. This article is about the technique: what each method does, where the key belongs and how to test and analyse without real personal data.
Pseudonymised data is still personal data: with additional information kept separately, it can be traced back. Anonymous data no longer can, and so falls outside the GDPR and the UK GDPR.
Removing names is almost never anonymisation. A combination of postcode, date of birth and sex, a rare value or a free-text field often still points to one person.
Pseudonymisation stands or falls with the key: it belongs in a different environment, with different administrators. Whether a dataset is anonymous enough is the controller’s judgement; we are not lawyers.
Pseudonymisation or anonymisation: the difference
The GDPR, and the UK GDPR with it, defines pseudonymisation as processing personal data so that it can no longer be attributed to a person without additional information, provided that information is kept separately and protected. A customer number instead of a name, with a table elsewhere linking number and name, is the textbook example. The data remains personal data, and the law applies in full.
Anonymisation goes further: the data can no longer be linked to a person by any means reasonably likely to be used, by anyone, including you. Then it is no longer personal data, and the processing falls outside data protection law. The ICO’s guidance draws the same line.
That makes the choice matter. Pseudonymisation is a security measure the law itself names: it reduces the impact of a breach and the number of people who can recognise someone. Anonymisation is a way of using data that is no longer personal data. Whoever does the first and believes it is the second is processing personal data without the protection that goes with it.
When a dataset is truly anonymous
European regulators have set out three questions for testing whether a dataset is anonymous. Can someone still:
- single out one person, because their record is the only one with a certain combination of values;
- link the data to another source, such as a public register or a leaked file, and find the person that way;
- infer something about a person with high probability, even without knowing their name.
In practice the risk rarely sits in the name and almost always in the rest. Postcode, date of birth and sex together often point to a single person. A rare job title in a small office does the same. Free-text fields contain names, phone numbers and diagnoses. And metadata, timestamps and location data are often more precise than the record itself.
Anonymity is also not a property a dataset keeps forever. What cannot be linked today may be linkable tomorrow, once a new source appears. So record which test you applied, with which assumptions, and repeat it when the dataset or its use changes.
The techniques, and what they do and do not do
No technique anonymises on its own. Each one reduces a particular risk, and you usually need more than one.
- Tokenisation. An identifying value is replaced by a random token, with a lookup table kept separately. Pseudonymisation, and easily reversible for whoever holds the table.
- Hashing. A plain hash of an email address or a National Insurance number can be reversed by hashing every possible value and comparing. With a secret key added (a keyed hash) it is pseudonymisation; without one it is often not even that.
- Masking. Hiding parts of a value, such as the middle digits of an account number. Useful on screens and in reports, but the rest of the record can still be recognisable.
- Generalisation. Date of birth becomes year of birth or an age band, a full postcode becomes only its outward code. Reduces the chance of singling out.
- Aggregation. Only totals and averages, with a minimum number of people per cell. Close to anonymous, but you lose the individual records.
- Synthetic data. Artificial records with the same structure and distribution as the original. Often the best choice for testing, as long as the generator does not reproduce real records.
The key kept apart: where pseudonymisation stands or falls
Pseudonymisation is exactly as strong as the separation between the data and the information that links it back. A lookup table in the same database, with the same administrators, is not a separation.
- A different environment. The lookup table or key sits in another system or a separate vault, not next to the dataset.
- Different people. Those who work with the pseudonymised data cannot re-identify it themselves. Re-identification is a separate action with its own permission.
- A record of who re-identifies. Every time a pseudonym is linked back to a person, it is logged with a reason.
- Rotating and destroying keys. A key that lasts for years becomes a risk in itself. And destroying the key does not automatically make the data anonymous: the rest of the record may still be recognisable.
How to manage keys in practice, and where they can live, is covered under key management and encryption.
Test environments without real personal data
The most common place where personal data sits needlessly is a test or acceptance environment filled with a copy of live data. Developers, testers and sometimes outside suppliers have access, security is lighter, and nobody cleans up the copy.
An approach that works:
- Make masking a fixed step in the copy procedure, not something someone does afterwards. The copy only enters the test environment once names, contact details, numbers and free text have been replaced.
- Keep relationships intact. Use consistent replacements, so the same customer gets the same pseudonym in every table and the tests keep working.
- Consider synthetic data for new features and for suppliers, so no real records leave the building at all.
- Do not forget the by-products. Backups, log files and exports of the test environment contain the same data as the environment itself.
If real data must be used for testing after all, for example to reproduce a fault, record who, why, for how long and when the copy was deleted.
Reporting, analysis and sharing with others
For a reporting or BI environment, pseudonymisation is often enough: analysts see trends per customer or employee without knowing who it is, and a limited number of people can re-identify when needed. Remove fields the analysis does not need, and generalise where you can.
When sharing with another organisation, such as a researcher, a consultant or an analytics provider, the question is sharper. If the dataset leaves pseudonymised, it is still personal data and needs an agreement on what the recipient may do with it. If you want it to be anonymous, the test from the previous sections has to hold for the recipient, taking into account the data that recipient already has.
If the processing carries a high risk, the choice between pseudonymisation and anonymisation belongs in the DPIA, with the reasoning.
Where our work stops
What we do. In the environments we supply, configure or manage, we set up the technique: masking as a fixed step when copying to test, tokenisation and keyed hashes, the separation between data and key, access rights and logging around re-identification. We describe which technique was applied to which fields, so that your privacy lead or data protection officer can assess that choice. We work under ISO/IEC 27001; that describes how we work, not a judgement on your processing.
What we do not do. We are not lawyers and not a data protection officer. We do not declare that a dataset is anonymous in the legal sense, we do not assess whether processing of pseudonymised data is lawful, and we do not decide whether data may be shared with a third party. That judgement belongs to the controller, with advice from a privacy lawyer or data protection officer.
Questions we get about this
The five that come up most often.
What is the difference between pseudonymisation and anonymisation?
With pseudonymisation you replace identifying data with a pseudonym, and additional information kept separately lets you link it back to the person. With anonymisation that is no longer possible by reasonable means, for anyone. Pseudonymised data is personal data; anonymous data is not.
Is pseudonymised data still covered by the GDPR?
Yes. Pseudonymised data remains personal data, because it can be traced back with the separately held information. The GDPR and the UK GDPR therefore still apply in full. Pseudonymisation is, however, a security measure the law itself names, because it reduces the impact of a breach.
Is hashing the same as anonymisation?
No. A plain hash of a known type of value, such as an email address or a reference number, can be reversed by hashing every possible value and comparing. With a secret key added it is pseudonymisation. A dataset only becomes anonymous when the remaining data no longer points to a person either.
How do you create test data without personal data?
Make masking a fixed step when copying live data to test, with consistent replacements so that relationships between tables stay intact. For new features and outside suppliers, synthetic data is often better, because no real records are used at all. Do not forget the backups and log files of the test environment.
Can you confirm that our dataset is anonymous?
No. Whether a dataset is anonymous in the legal sense is a judgement for the controller, with advice from a privacy lawyer or data protection officer. We set up the techniques and describe exactly what was done to which fields, so that judgement can rest on facts.
Further reading
What pseudonymisation connects to.
Is there real personal data in your test environment?
We map where copies of live data sit and set up masking as a fixed step. Whether the result is anonymous enough is for your privacy lead to judge.
Practical IT knowledge in your inbox
New guides on management, security and the workplace, written by the people doing the work. No sales talk, and you can unsubscribe in one click.