Model card

What is a model card?

A model card is a short document that goes with a machine learning model, in the way a datasheet goes with an electronic component. It says what the model is meant to do, how well it does it, on which groups of people or cases it was measured, and where it should not be used.

The idea comes from a 2018 paper by a team at Google, with Margaret Mitchell as lead author and Timnit Gebru among the co-authors. Their argument was straightforward. Models were being published with a headline accuracy figure and nothing else, which told you almost nothing about whether the model was suitable for your situation. Their proposal was to publish evaluation results broken down across different groups, alongside a clear statement of what the model was intended for.

That breakdown is the heart of it. A model that is 95 percent accurate overall can be 98 percent accurate on one group and 70 percent on another. A single average hides exactly the information you need in order to decide whether to use it.

What is in one

Model cards vary between providers, and the sections in the original proposal are still the ones worth looking for.

• Intended use. What the model was built for, and just as importantly, what it was not built for. That section tells you whether your plan sits inside or outside what the model was designed for.

• Factors. The groups, conditions and circumstances that affect performance. Demographic groups, but also things like image quality, language, or recording conditions.

• Metrics. Which measures were used and why. If a model was measured on the wrong metric for your use case, the number tells you little.

• Evaluation data. What the model was tested on. If that data does not look like yours, the reported numbers do not transfer.

• Training data. What it learned from, to whatever extent the provider is willing or able to disclose.

• Ethical considerations. Known risks, sensitive uses, and the trade-offs the builders were aware of.

• Caveats and recommendations. What the builders themselves say you should be careful about.

Read in the right order, those sections answer one question: is this model appropriate for what I want to do with it. That is a more useful question than how good the model is.

Model card, system card, and the AI Act

Two neighbouring terms are worth separating.

A model card describes the model. A system card describes a whole system built around a model, including the safety layers, the filters and the testing that was done on the assembled product. As models increasingly ship as part of a product rather than as a bare artefact, system cards have become the more common format from the large providers.

Both connect to the AI Act, though neither is named in it as such. The Act requires providers of general-purpose AI models to prepare technical documentation and to make information available to companies building on the model, so those companies understand its capabilities and limitations. For high-risk systems, technical documentation is a formal obligation with its own requirements. Neither of those obligations is automatically satisfied by a model card. It is the recognisable, readable form that much of that information takes.

How to use one when you are buying

Most companies will never write a model card. Almost all of them should be reading them, and there are four questions worth taking into a supplier conversation.

Is my use case in the intended use section? If your plan is not mentioned, that is not permission by omission. Ask.

Was it evaluated on anything resembling my situation? Something evaluated on English customer service transcripts tells you little about Dutch technical support.

What do the caveats say? This section is written by the people who know the model best and is routinely skipped by the people who most need it.

Is there a card at all? A supplier who cannot produce documentation about the model inside their product is telling you something. Not necessarily that the product is bad, but certainly that you will not be able to answer questions about it later.

What to watch out for with model cards

It is written by the provider. A model card is self-reported. It is a considered statement from the people who built the thing, not an independent assessment, and the incentive to present the model favourably is exactly what you would expect.

It ages. Models get updated, sometimes without a version bump you can see. A card written for a version you are no longer running describes something else.

It says nothing about your data. Every reported metric comes from the provider's evaluation set. How the model performs on your documents, in your language, on your process is a question only your own testing answers.

A card is not compliance. Having documentation from your supplier does not discharge your own obligations if your use case is high-risk. It is an input to your file, not a substitute for it.

The absence of a section is information. A card with no training data section and no caveats is not a card that has nothing to disclose. It is a card that chose not to.