Metadata

What is metadata?

Metadata is data about data. It explains what a dataset is, where it came from, how fresh it is, who owns it, and how it may be used. The rows and columns are the data. The names, descriptions, lineage, refresh time, sensitivity label, and owner are metadata.

A photo on your phone is a simple example. The image is the data. The date, location, device type, file size, and camera settings are metadata. That is why you can search for photos from last summer without opening every image.

For business data, metadata turns a table from a mystery into something people can use. A table called sales_fct_v3 is hard to trust. Add field descriptions, source, owner, refresh schedule, and business definitions, and a colleague can decide whether it fits the report they are building.

The main types of metadata

Technical metadata
Schema details such as table names, columns, data types, keys, partitions, file formats, row counts, and storage locations. Systems can usually capture much of this automatically.

Business metadata
The meaning people need: definitions, owners, glossary terms, calculation rules, approved use, classifications, and context. A scanner can see that a column is numeric. It cannot know by itself whether revenue includes returns or VAT.

Operational metadata
Runtime information such as refresh time, pipeline status, duration, failures, usage, query history, and freshness. This tells you whether a dataset is alive, stale, popular, or ignored.

Data catalog tools bring these layers together. Technical metadata comes from scans and system APIs, operational metadata from pipelines and logs, and business metadata from owners and stewards.

What good metadata gives you

Findability. People can discover existing datasets instead of rebuilding the same extract in a spreadsheet or a one-off report.

Trust. A dashboard number is easier to believe when you can see the source, owner, refresh time, and definition.

Impact analysis. If a column is renamed or a source system changes, data lineage tells you which reports, models, and processes may break.

Governance. Ownership, sensitivity labels, retention rules, glossary terms, and access rules all rely on metadata.

AI readiness. AI agents and copilots use names, descriptions, relationships, and field meanings to decide which data to query. A model with clear metadata gives them a much better chance of choosing the right table and interpreting the result correctly.

Where metadata lives

Some metadata lives inside the systems that create or store data. Databases know their schemas. Cloud storage knows file properties. Power BI semantic models carry relationships, measures, descriptions, and security rules. Pipelines know when they ran and whether they failed.

That local metadata is useful, but it becomes scattered as the data estate grows. A central data catalog collects metadata from many systems and makes it searchable. Microsoft Purview, AWS Glue Data Catalog, OpenMetadata, DataHub, Alation, Atlan, and Collibra all work in this space, with different strengths.

A catalog usually stores metadata, not the underlying business data. It tells you that a customer table exists, who owns it, and which reports depend on it. Access to the actual rows still depends on the source system.

Metadata and Microsoft Purview

Microsoft Purview is the Microsoft platform where metadata, governance, security, and compliance meet. Its Data Map scans sources and collects technical metadata. The Unified Catalog makes assets searchable and adds business context such as descriptions, glossary terms, owners, data products, quality rules, and lineage.

In a Fabric and Power BI environment, that matters because reports, lakehouses, warehouses, semantic models, and OneLake items can all become part of the same governance view. Purview is broader than a simple catalog because it also connects to sensitivity labels, DLP, audit, retention, and compliance workflows.

Keeping metadata useful

Metadata goes stale when it is treated as a documentation project. Three months of cleanup can look good on launch day and decay quietly after the next schema change.

Use a smaller, steadier routine instead. Give important datasets named owners. Make metadata updates part of the same change request as a pipeline or model change. Automate technical and operational capture where the tool supports it. Keep human effort for meaning, definitions, ownership, and exceptions.

Usage metadata helps with scope. A dataset used by the board, customers, finance, or regulatory reporting deserves more care than a one-off analysis table that nobody has opened in six months.

What to watch out for with metadata

More fields do not mean better metadata. A catalog with twenty mandatory fields usually becomes a wall of empty values. Start with owner, description, source, freshness, sensitivity, and definition for the important datasets.

One reference place matters. If definitions live in a wiki, a spreadsheet, and a model description, they will drift. Pick the place that owns the definition and link from the others.

Wrong metadata is worse than missing metadata. A stale description creates false confidence. Make it easy for users to report outdated definitions and for owners to fix them.