Entity resolution

What is entity resolution?

Entity resolution is the process of deciding which records refer to the same real-world entity: the same person, company, product, supplier, address, or asset. Records that belong together are linked. Records that merely look similar are kept apart.

The problem appears because business systems rarely share a clean identifier. The same customer might appear as Jan Peeters in the CRM, J. Peeters in the webshop, and Peeters BV in accounting. There is no universal customer number, but there are clues: email address, phone number, company number, VAT number, address, domain, name, and order history.

Think of a receptionist comparing two forms. The names are slightly different and one address is old, but the phone number and date of birth match. A person can make that judgement by feel. Entity resolution does it at scale, with rules, scores, thresholds, and review queues.

Entity resolution versus deduplication

Deduplication removes duplicates inside one list or system. The same customer entered three times in the CRM becomes one CRM record.

Entity resolution is broader. It links identities across systems and datasets: CRM, ERP, webshop, support desk, marketing platform, billing system, acquired company, reference list. Deduplication is one special case of entity resolution where all records happen to live in the same table.

This distinction matters in data projects. Removing duplicates from one source is useful, but it does not give you a Customer 360 view. For that, you need to decide which records across sources represent the same person or company.

How an entity resolution pipeline works

1. Standardise the data
First, make fields comparable. Lowercase email addresses, normalise phone numbers, standardise company suffixes, parse addresses, split names where useful, and remove obvious formatting differences. Better data quality at the source reduces the work here, but it never removes it completely.

2. Block candidate pairs
Comparing every record with every other record does not scale. One million records produce roughly half a trillion possible pairs. Blocking narrows the search to plausible candidates: same postcode, same email domain, same first letters of a name, same VAT number country, or another coarse key.

3. Compare pairs
Within each block, records are compared field by field. Some matches are deterministic: the same VAT number means the same company in many contexts. Others need probabilistic scoring. The classic Fellegi-Sunter record linkage model gives a formal way to score evidence by looking at how likely an agreement is among true matches versus random non-matches.

4. Choose thresholds
A high score becomes a match, a low score becomes a non-match, and the grey zone goes to review. The threshold is a business decision. In marketing, a missed match may be worse than a false merge. In compliance or healthcare, a false merge can be much more dangerous.

5. Cluster linked records
Matches are pairs, but entities are groups. If A matches B, and B matches C, those records may form one entity even if A and C were not compared directly. Clustering turns the pair decisions into entity groups.

6. Build a canonical record
Finally, the system chooses the best values for the golden record: the most recent address, the verified email, the legal company name, the preferred phone number. That canonical record can feed reporting, CRM clean-up, MDM, or operational systems.

A CRM and webshop example

Suppose you want to know how many webshop buyers also exist as CRM contacts.

The CRM record says Jan Peeters, Acme NV, jan.peeters@acme.be, office address in Antwerp. The webshop order says J. Peeters, the same email, and a delivery address in Berchem.

After standardisation, the email addresses match exactly. Blocking puts the two records together because the email domain and surname are compatible. Pairwise comparison gives heavy weight to the email match, some weight to the name similarity, and a small penalty for the different address. The score passes the threshold, so the records are linked.

The canonical customer profile can now keep both the office and delivery address, while reporting knows that the CRM contact and the webshop buyer are one customer.

What entity resolution is used for

Customer 360
Sales, support, ecommerce, billing, and marketing records are joined into one view of the customer. Entity resolution supplies the links. Master data management then governs the golden record and how changes flow back into systems.

Mergers and migrations
When two companies merge, their customer and supplier lists rarely line up. Entity resolution finds overlaps so the combined business does not invoice, contact, or count the same relationship twice.

Fraud and risk
Banks, insurers, and marketplaces use entity resolution to find hidden connections: many accounts sharing an address, one person using several identities, or companies linked through directors and phone numbers.

Knowledge graphs
Resolved entities are the nodes that make a graph useful. Once people, companies, addresses, and assets are linked reliably, you can analyse the relationships between them.

What to watch out for

False merges are expensive
Joining two different people into one record can break privacy, reporting, customer service, and compliance. Tune thresholds based on the cost of a wrong merge, not just on average accuracy.

False splits hide the truth
Missing a match leaves the same customer scattered across systems. That weakens customer lifetime value, churn analysis, credit exposure, and support history.

Human review still matters
Scores near the threshold are judgement calls. A data steward should review uncertain cases and feed those decisions back into the rules or model.

Privacy rules apply
Entity resolution can make data more identifying by joining fragments that were previously separate. Treat matching projects as personal-data processing when people are involved, with a clear purpose, access control, retention, and audit trail.

Source systems need ownership
A golden record is only useful if someone decides which system owns which field. If CRM and ERP disagree on the billing address, the matching algorithm cannot decide the business rule for you.