Conformance checking

What is conformance checking?

Conformance checking compares how a process in your organisation actually ran against how it should run according to an agreed model. You take the event log from your ERP, CRM or ticketing system, place a reference model next to it (a BPMN diagram, an internal procedure or an approved SOP), and the tool shows step by step where reality deviates.

Together with discovery and enhancement, conformance checking is one of the three pillars of process mining. Discovery draws the map of what happens. Conformance checking compares that map against the agreed route. Enhancement enriches the model with extra information such as throughput times or costs.

When do you use conformance checking?

Conformance checking is useful whenever there is a clear agreement on how a process should run, and deviations carry enough risk or cost to be worth seeing:

• Audits: instead of manually walking through a sample of a hundred cases, you show in a few minutes which of the tens of thousands of cases strayed from the book and exactly where.

• Four-eyes control: check whether a purchase or payment was truly approved by two different people, rather than signed off by one person through a workaround.

• SLA monitoring: find which customer service tickets exceeded the agreed response time and at which step the delay started.

How does conformance checking work?

A conformance checking tool places each case next to the model and computes two numbers: fitness and precision.

Fitness measures how well reality fits the model. A fitness of 1.0 means every case flows perfectly through the model. At around 0.85 there is a lot of behaviour in the data that the model does not anticipate: skipped steps, steps in the wrong order, or activities the model does not contain.

Precision measures the opposite: how much behaviour does the model allow that never happens in practice? A permissive model scores high on fitness (almost everything fits) but low on precision (it predicts nothing). A good model scores well on both.

Under the hood, most tools use token-based replay (first described by Rozinat and van der Aalst in 2008) or alignments (later developed by Adriansyah, van Dongen and van der Aalst). Replay plays each case on a Petri net of the model and records where it gets stuck. Alignments search for the cheapest way to match data and model, which is heavier to compute but gives a sharper diagnosis. Tools such as Celonis, Fluxicon Disco and Microsoft Process Advisor handle that for you.

Conformance checking versus process discovery

The two techniques belong together but answer different questions.

Process discovery draws the process from the data without a prior model. Blank slate, the tool builds the process map from whatever actually sits in the event log. Ideal for exploration: you do not yet know how the process runs, or you suspect the official model is out of date.

Conformance checking starts from an existing model and looks for the deviations. Ideal for control: you know how it should run and you want to see where it goes wrong.

Many projects start with discovery, build a cleaned-up model from it, and then switch to conformance checking to keep that model under continuous watch.

Worked example: approval flow for purchase invoices

A mid-sized manufacturer runs a procurement policy: every invoice above 5,000 euro must be approved by two different people before it is paid. One for the accounting check, one for management sign-off. The ERP has steps in place for both.

The event log over a year contains around 18,000 invoices above that threshold. A conformance check against the reference model shows three patterns:

• The bulk flows as intended. Fitness on that subset is 1.0.

• In a small percentage the management approval is simply skipped, often on invoices that just crossed the threshold.

• In the stubborn group, both approvals were carried out by the same employee. Two steps at two moments, so the model looks respected, but in reality the control was signed off by one person. Over a year that can quickly add up to hundreds of thousands of euro of uncontrolled spend.

Without conformance checking, a pattern like that only surfaces during a deep audit or after an incident. With a monthly check you see it within half a day and you can intervene where it matters: adjust ERP rights or wire in an alert for the at-risk group.