Named entity recognition (NER)

What is named entity recognition (NER)?

Named entity recognition is the step where software reads a piece of text and marks the named things inside it. A person, a company, a place, a date, an amount, a product code, an invoice number, a national register number. Every hit gets a label saying what kind of thing it is.

What comes back is a list of spans. A span is one stretch of text with four things attached: the exact words, where they start, where they stop, and the label. spaCy, the open source library most Python teams reach for, describes its recogniser as a system that "assigns labels to contiguous spans of tokens". Managed services normally add a confidence score per span.

Those positions matter more than people expect, because of what happens next. The output fills a column in a database and an invoice becomes a row with a supplier, a number and a total. It drives a redaction step, where you need to know exactly which characters to replace before the text travels on. Or it feeds a search index, so a document becomes findable by the companies and the people mentioned inside it.

NER and classification get mixed up, and the difference is where the label lands. Classification puts one label on the whole document: this email is a complaint, this invoice is a credit note. NER puts labels on pieces inside the document and tells you where each piece sits. Most pipelines run both, in that order: classify the ticket as a complaint, then pull the customer name and the order number out of it. NER is also not entity resolution, which is the separate job of deciding that the company name you just found and a record in your ERP are the same company.

Three ways to find entities

Rules and dictionaries. Anything with a fixed shape is better handled by arithmetic than by a model. A Belgian national register number is eleven digits with a check digit you can recompute, so a regular expression finds the candidates and the check throws out the typos. The same goes for IBANs, VAT numbers and any code your own systems issue, while a dictionary covers your six hundred suppliers. spaCy has a component for this, the EntityRuler, and its documentation gives the reason to bother: you can "combine both approaches and improve a statistical model with rules to handle very specific cases and boost accuracy".

A trained model. For the types with no fixed shape, a model that has seen thousands of labelled examples does what rules cannot, because a person is a person by the words around it. spaCy's Dutch pipelines recognise eighteen labels out of the box, among them PERSON, ORG, DATE and MONEY. Microsoft's Azure AI Language offers the same as a service, with a preset list running from Person and Organization through Address and PhoneNumber, plus a custom option where you train it on types of your own.

A language model with a prompt. The third route skips training. You describe the type in a sentence, hand the model the text, and get the spans back. "Find every reference to a maintenance contract, with its contract number and its end date" is a type nobody sells a model for and one you have no labelled data for. To keep the output usable, constrain it: Anthropic's structured outputs return "valid JSON matching your schema", and the documentation names extracting data from text as one of the things the feature is for.

A trained model or an LLM prompt

The two general-purpose options pull against each other on one dimension: cost per row against flexibility.

A trained model is a download that runs on your own CPU. The cost per row is machine time, so a hundred thousand tickets overnight barely moves the server bill. What you give up is reach, because the labels were fixed when the model was trained and adding "contract reference" means collecting examples, labelling them and training again. An LLM prompt inverts both. Every document is a model call, so those same hundred thousand tickets become a real invoice and a much longer run, but a new entity type costs one sentence and the model copes with text a trained pipeline has never seen the shape of.

In practice we often settle it in the middle. Run the LLM once over a few hundred documents, have a person correct the output, then train the cheap model on that. You pay the expensive route once instead of on every row, and you end up with the labelled set you needed anyway.

How you know it works

Precision is the share of the spans the system flagged that were genuinely right, and recall is the share of the spans that were actually there that it found. Published scores tell you little about either on your own text. spaCy reports its numbers per pipeline, and the large Dutch model in version 3.7.0 lands at 78.51 precision and 75.03 recall on spaCy's own evaluation set. Your purchase invoices and your Dutch support tickets are not that text, so you label a sample of your own: a few hundred documents, marked up by someone who knows the domain, is enough for an honest reading and doubles as the set you re-run after every change.

Which number leads depends on what happens after the span. A national register number missed in a redaction pipeline is a leak, so recall is what you are buying and you accept flagging the odd order reference by mistake. A product code missed in a search index costs one document a few places in the ranking, while a false hit that redacts an ordinary word makes the text unreadable, so precision leads there. Set the threshold per label, not once for the whole pipeline.

Dutch text and Belgian identifiers

Flemish surnames carry particles that are ordinary Dutch words. "De Clercq", "Van den Broeck", "Van der Auwera": a model trained mostly on published Dutch prose tends to cut the particle off and return "Clercq". In our experience that is the Dutch NER error most worth patching with a dictionary of the names you actually deal with. Legal forms work in your favour. A name ending in BV, NV, VZW or CV is an organisation by rule, not by prediction.

The identifiers are where rules earn their place. The enterprise number issued by the Kruispuntbank van Ondernemingen is ten digits starting with a 0 or a 1, and once the tax administration activates it, that same number with BE in front is your VAT number. The 0 series ran out and the first number starting with a 1 was issued on 19 September 2023, so a rule written before that date and never revisited now rejects valid numbers. An establishment unit number is also ten digits but starts with a digit from 2 to 8, so a pattern that matches "ten digits" quietly mixes the two.

The national register number is the strictest of the lot and therefore the easiest. Eleven digits: the birth date as year, month and day, then a three-digit sequence that is odd for men and even for women, then two check digits. The check is the nine-digit number formed by the date and the sequence, divided by 97, with the remainder subtracted from 97, and people born from 2000 onwards get a 2 written in front of those nine digits before the division. Recompute that and almost every false hit disappears.

Then there is the bilingual reality. One Belgian file holds a French quotation, Dutch terms and conditions and an English mail thread, while most services want a single language code per document. Azure's text-based personal data detection covers Dutch and French and ships Belgium National Number and Belgium Value Added Tax Number as built-in types, but its conversation feature for chat and call transcripts is generally available in English, French, German and Spanish only, so Dutch transcripts fall outside it. Check the language table of the exact feature you call, not of the product.

Two worked examples

Purchase invoices into the accounting system

A shared mailbox collects supplier invoices as PDFs, OCR turns each page into text with positions, and the wider pipeline around that is Intelligent Document Processing. NER is the part that decides what the recognised text means. The VAT number printed on the invoice does more work than the company name at the top, because it matches your supplier table exactly while the name arrives in three spellings and two legal forms. Invoice numbers have no shared format, so you learn a pattern per supplier and fall back on the model for new ones.

Amounts are the awkward part. The model finds four monetary spans on the page and cannot tell you which is the total, so a rule takes the amount closest to the words "Totaal" or "Te betalen" and a calculation confirms the lines add up to it. Anything under threshold, or an invoice whose totals do not reconcile, goes to a person with the original next to the proposed values.

Redacting support tickets before they reach an AI tool

You want an assistant to draft replies to support tickets, and the personal data has to come out first. A detection pass runs over the ticket body for names, email addresses, phone numbers, addresses, IBANs and national register numbers, and replaces each span with a placeholder such as [KLANT_1]. The mapping stays in a table you control, so the draft can be filled back in with the real name before anyone sends it. Microsoft's personal data feature does the finding and the replacing in one call and hands back the entity categories, the confidence scores and the redacted text.

Where the detection runs decides whether any of this helps. If you send the raw ticket to a hosted LLM to find the personal data, you have already sent the personal data to the hosted LLM. The detector belongs on your own infrastructure or in a service you have a processing agreement with, and only the masked text travels on. Recall decides the rest, because one national register number the detector misses lands in the AI tool anyway, which is why this pipeline pairs a model for the names with a hard rule and a check digit for the numbers.

Under the GDPR a name and an identification number are personal data (Article 4(1)), and replacing them with placeholders while you keep the key somewhere else is pseudonymisation as defined in Article 4(5). Worth doing, and still not the finish line: the masked text remains personal data and the rules keep applying to it.

What to watch out for with named entity recognition

Ask an LLM for the exact substring, never for character offsets. Models are poor at counting characters and will hand you a plausible start position that is off by six. Take the returned string and find it in your own text yourself, and if it does not appear there verbatim, fail loudly instead of redacting the wrong part of the sentence.

Every label needs a written definition. Is the accountancy firm named in the footer an organisation you care about? Is the bank a supplier? Two colleagues will answer differently and so will two runs of the same model. Write the definition down before you measure anything.

Detection is not access control. Redaction reduces exposure, it does not guarantee removal. Where the requirement is that certain data never leaves the building, keep it in a system that never sends it, rather than trusting a detector to catch every mention.

The numbers move. A new supplier layout, a vendor swapping the model behind an endpoint, a retrain on fresh examples: each can shift recall without anyone noticing. Keep the labelled sample, re-run it after every change, and treat a drop as a release blocker rather than a curiosity.