Differential privacy

What is differential privacy?

Differential privacy is a mathematical guarantee that the result of an analysis changes only a little when one person's data is added or removed. That is the point: someone looking at the published result should not be able to tell whether a specific person was in the dataset.

The technique protects people by adding carefully controlled random noise to the answer. Ask for the average salary of a group and the system returns a number that is deliberately slightly off. Close enough to be useful for analysis, noisy enough that one person's value is not exposed.

You can picture it as a room where everyone answers the same question while a low murmur is added over the top. You can still hear the pattern of the room, but you cannot pick out one voice.

The privacy budget

The amount of noise is not chosen by instinct. It is controlled by a number called epsilon, often described as the privacy budget. Epsilon expresses the trade-off between privacy and accuracy.

A small epsilon means more noise: stronger privacy, but less precise results. A large epsilon means less noise: more accurate figures, but weaker privacy. There is no universally correct value. It is a policy choice that should match the sensitivity of the data, the size of the groups, and the risk of harm if individuals are inferred.

The budget also gets spent. Each query on the same underlying data consumes part of the privacy budget. If a system answers unlimited questions, someone can combine those answers and average away the noise. A serious differential privacy system therefore limits the number and shape of the allowed queries. When the budget is gone, the tap closes.

A salary example

Suppose you publish the average salary of a team of ten. Averages sound safe because no names appear. But if someone also knows the average for the same team without Ana, they can use the difference to calculate Ana's salary.

With differential privacy, both answers receive their own controlled noise. The numbers still say something useful about the team's pay level, but the difference between the two answers no longer points reliably to Ana's exact salary. The guarantee is designed to hold even when the attacker has other background knowledge.

That last point is what makes the technique powerful. Ordinary masking often depends on guessing what outside information an attacker might have. Differential privacy gives a bound on what the analysis itself can reveal.

Differential privacy versus anonymisation

Classic anonymisation and pseudonymisation work on the records: remove names, blur dates of birth, replace a customer number with a code, aggregate small groups. Those techniques are useful, but they can fail when the remaining fields are linked with another source.

Differential privacy turns the question around. Instead of trying to make every record safe to release, it controls what the output of an analysis can reveal about any one person. You usually do not publish the raw rows at all. You publish counts, averages, histograms, model statistics, or synthetic outputs produced under a defined privacy budget.

That does not make ordinary anonymisation obsolete. In practice, privacy engineering often combines deletion, minimisation, aggregation, access control, and differential privacy. The key is to be clear about what each measure guarantees.

Where is differential privacy used?

Official statistics
The US Census Bureau used a differential privacy-based disclosure avoidance system for the 2020 Census redistricting data. The goal was to publish useful population tables while reducing the risk that individual households could be reconstructed from detailed tabulations.

Device analytics
Apple has described local differential privacy systems where noise is added on the device before data is sent. That lets Apple learn aggregate patterns, such as popular emoji or certain usage trends, without receiving a clean record of one user's behaviour.

Browser and product telemetry
Google's RAPPOR work used randomised reporting to collect aggregate statistics while limiting what any one client report disclosed. The broader idea is the same: learn population-level patterns without trusting the server with exact individual answers.

Business reporting and benchmarks
For a company, the realistic use case is usually aggregated sharing. You might want to publish benchmark figures, supplier performance, or regional sales without exposing a tiny customer segment. Differential privacy can help when ordinary aggregation leaves small cells that are still too revealing.

What to watch out for

Privacy costs accuracy
The stronger the privacy guarantee, the more the numbers move. Large groups usually tolerate that well. Small groups can become useless because the noise is larger than the signal.

Epsilon is governance, not decoration
If a tool says it uses differential privacy but does not make the privacy budget clear, ask harder questions. Epsilon decides how much privacy people actually receive.

Repeated queries leak
Noise only works if the system controls composition: how many answers are released from the same data and how the budgets add up. Exporting twenty slightly different versions of the same report weakens the protection.

It does not automatically remove GDPR duties
If you start from personal data, the processing still needs a lawful basis and appropriate safeguards. A differentially private output may be anonymous in a practical case, but that conclusion needs a documented risk assessment. The technique lowers re-identification risk; it is not a legal exemption stamp.

Implementation details matter
Differential privacy is easy to invoke and hard to implement well. Bad clipping, tiny groups, poorly chosen budgets, or unlimited analyst access can undo the intended protection. Treat it as a statistical system that needs design, testing, and monitoring.