Medallion architecture

What is medallion architecture?

Medallion architecture organises the data in a lakehouse into layers with different responsibilities. The familiar names are Bronze for source-faithful data, Silver for reusable cleaned data, and Gold for data products made for a defined audience.

The colours describe the state and intended use of the data. They do not prescribe one technology, one storage account, or exactly three physical copies. A layer is useful when it creates a meaningful contract about quality, ownership, access, or performance.

Databricks uses the Bronze, Silver, and Gold names for this lakehouse pattern, while Microsoft Fabric recommends medallion as one way to organise data in OneLake. The underlying idea is older than either platform. Data warehouses have long separated landing data, integrated data, and presentation models.

Think of a kitchen. Bronze records what arrived at the back door. Silver is the shared preparation area where ingredients are checked and made consistent. Gold is a dish prepared for a particular menu. The kitchen only needs the preparation step when that work is worth sharing.

What belongs in each layer?

Bronze: what did the source deliver?

Bronze preserves data as close to its source form as practical. Files, database changes, API responses, and events land with enough ingestion metadata to identify their source, arrival time, and processing batch. That makes Bronze a restart point and part of the audit trail.

Bronze does not mean that absolutely no technical processing is allowed. A platform might wrap relational data in Delta tables, decompress a file, or attach load metadata. The defining rule is that ingestion must not silently replace the source meaning with business interpretation.

Silver: what happened in a consistent form?

Silver turns source-shaped records into dependable, reusable data. It validates types, standardises units and timestamps, handles duplicates, isolates invalid records, and connects identifiers across systems. It normally retains enough row-level detail for exploration and new downstream uses.

The exact purpose of Silver is to stop every Gold product from repeating the same source integration work. Silver absorbs source-specific complexity and presents stable entities or events to downstream teams. A customer can have one usable identifier even when the CRM, invoicing system, and support platform all use different keys.

Silver should avoid definitions that exist for only one report. Cleaning a malformed date belongs in Silver. Deciding that a customer counts as active after buying within the last 90 days may belong in Gold because that rule expresses a particular business use.

Gold: what does a consumer need?

Gold publishes data for a named purpose and audience. It can contain dimensional models, certified metrics, aggregations, machine-learning feature sets, or detailed tables with agreed business meaning. Gold is a contract, not a synonym for a summary table.

One Silver sales model can feed several Gold products. Finance may need recognised revenue by accounting period, Sales may need pipeline conversion, and an AI model may need historical customer features. Each Gold product can have its own grain, freshness, owner, and access policy.

Diagram: each medallion layer answers a different question. Silver is the reusable contract between source ingestion and consumer products.

Do you always need all three layers?

No. Medallion is a design pattern, not a compulsory route through three boxes. Going directly from Bronze to Gold is reasonable when one simple source feeds one consumer, the transformation is easy to test, and there is no reusable integration logic.

A small daily report might load a clean application table into Bronze and publish one Gold model directly. Adding Silver would only rename an intermediate query. The extra layer would create another dataset to operate without creating a useful contract.

Silver becomes worthwhile when several products need the same cleaning, when sources must be joined, or when source schemas change independently of consumers. It also gives analysts and data scientists a dependable detailed dataset before business-specific calculations remove information.

Use this test: if one Gold product disappeared, would its cleaning and integration logic still be useful elsewhere? If yes, that logic probably belongs in Silver. If no, a direct Bronze-to-Gold path may be clearer.

Data does not have to move only forward. Engineers may return to Bronze to rebuild a table after correcting a rule. Analysts may use Silver to investigate a Gold result. The production dependency should still be explicit, so a report does not accidentally rely on an undocumented intermediate table.

Which layer should you consume?

Use the highest layer that still contains the detail and freshness required by the task. Gold is the normal entry point for shared consumption, but Silver and Bronze have legitimate specialist uses.

1. Consume Gold for shared decisions and applications. Recurring dashboards, certified semantic models, operational reports, and downstream applications should normally use Gold. Definitions and expected performance belong with the published data product.

2. Consume Silver for trustworthy detail. Analysts use Silver when they need clean row-level records before those records are shaped for one business question. Data scientists often use it for exploration and feature engineering because it retains detail while removing ingestion problems.

3. Consume Silver for deliberate real-time drill-down. A monitoring screen may need individual events that a Gold model has summarised. Fabric Real-Time Intelligence supports this pattern. Give the Silver dataset a documented schema and access policy instead of allowing every report to bypass Gold.

4. Consume Bronze to inspect or reproduce the source. Data engineers and operations teams use it to diagnose ingestion failures, replay processing, or reconcile a result with the original record. Audit work can also require source-faithful data. Bronze should not be the routine source for KPIs because its records have not passed downstream quality rules.

Repeated direct consumption is a design signal. If several teams keep using the same Silver table, consider publishing a governed Gold product for them. If many users need Bronze, the expected source-level use may deserve its own controlled interface.

Diagram: choose the highest layer that still contains the detail and freshness your task requires.

A worked example

Imagine an online retailer with orders from its webshop, invoices from an ERP system, customers from a CRM, and click events from its website.

Bronze stores the source extracts and events with load metadata. The webshop may supply JSON, the ERP may export relational tables, and the CRM may identify the same customer with another key. Nothing in Bronze claims that these records already agree.

Silver parses the events, converts timestamps to one standard, removes duplicate deliveries, and maps source identifiers to shared customer and product identifiers. Invalid records are quarantined rather than silently discarded. The resulting order, invoice, customer, and interaction data can be reused.

Gold publishes several products. A finance model defines recognised revenue. A sales model describes orders and returns. A customer model provides approved features for churn prediction, while a monitoring product keeps detailed recent events for operational alerts.

A missing invoice can be traced backwards. The analyst sees the gap in Gold, checks whether the invoice survived the Silver quality rules, and then compares it with the ERP record in Bronze. Each layer answers a different question without making the report responsible for source diagnosis.

Alternatives and complementary architectures

Medallion organises data by refinement and intended use. Other architectures organise it around business models, history, processing latency, or ownership. Some replace the three-layer pipeline; others can sit inside or around it.

Layered data warehouse

A traditional data warehouse often moves from staging to an integrated enterprise model and then to data marts. The responsibilities resemble Bronze, Silver, and Gold, but the warehouse usually expects structured relational data and schema-on-write processing.

Dimensional modelling

Dimensional modelling starts with business processes and represents them as facts and dimensions. It is particularly useful for BI. A Gold layer often contains these star schemas, so dimensional modelling and medallion commonly work together.

Staging, intermediate models, and marts

SQL transformation projects often use staging models for light source cleanup, intermediate models for reusable transformations, and marts for consumer-facing data. This is close to medallion with names that describe the job rather than the quality level. The intermediate layer should earn its place in the same way as Silver.

Data Vault

Data Vault separates auditable source history in a Raw Vault from derived rules in a Business Vault and consumer models in information marts. Consider it when many changing sources must be integrated without losing their history. Its extra modelling and loading machinery is harder to justify for a small analytical estate.

Lambda and Kappa architectures

Lambda uses separate batch and real-time processing paths that meet at the serving layer. Kappa processes data through one event stream and rebuilds state by replaying retained events. These patterns answer a latency question rather than a data-quality question, so a streaming solution can still label its data Bronze, Silver, and Gold.

Data mesh

Data mesh distributes ownership to business domains that publish data products. It answers who owns the data, while medallion describes how data changes state. A Sales domain and a Finance domain can each use their own medallion pipeline.

How medallion supports an AI-ready organisation

Medallion can supply governed data for Artificial Intelligence, but it is only the data foundation. Production AI also needs model operations, retrieval or feature serving, evaluation, monitoring, and controls over what a model or agent may access.

Predictive machine learning

Bronze preserves historical observations, Silver creates consistent events and entities, and Gold can publish training datasets or approved feature sets. A feature store may sit beside the Gold products when the same calculated input must be reproduced for training and live predictions.

MLOps manages the lifecycle beyond the data layers: experiments, code, dataset versions, model approval, deployment, monitoring, and retraining. A medallion pipeline without that lifecycle can prepare training data but cannot operate a model safely.

Generative AI and RAG

For RAG, Bronze can retain original documents and their source metadata. Silver can hold extracted text, normalised metadata, permissions, and chunks. Gold can publish a use-case-specific retrieval collection or approved structured tools for an assistant.

A vector database is one possible serving component, not a substitute for the data pipeline. The organisation still needs to handle document updates, deletions, permissions, source citations, and retrieval evaluation.

Real-time AI

Fraud detection, recommendations, and operational alerts may need an event-driven or Kappa-style path. Historical Silver or Gold data can train the model, while a stream supplies current features and inference requests. The online serving path must use the same definitions as training to avoid a mismatch between development and production.

Data products and governance

AI applications should depend on owned data, feature, and knowledge products rather than arbitrary tables. Each product should state its meaning, freshness, quality checks, permitted uses, and change policy. Governance must cover datasets, models, prompts, retrieval indexes, evaluations, and agent permissions.

Diagram: medallion prepares governed data, while serving patterns and MLOps or LLMOps operate the AI applications built on top.

Implementing medallion architecture in Fabric

Fabric can implement the layers as separate lakehouses or warehouses. Teams can also separate them with schemas when the required access boundaries permit it. Separate workspaces provide stronger layer-level control but add administration.

1. Land Bronze in its source format where practical. Data Factory pipelines, mirroring, shortcuts, and streaming tools can bring source data into OneLake. Relational sources can also land in Delta Lake tables when that suits the ingestion design.

2. Build Silver with notebooks, dataflows, or materialised lake views. These transformations apply validation, standardisation, deduplication, and joins, then write reusable detailed tables.

3. Publish Gold in a lakehouse or warehouse. A warehouse suits teams that want a relational serving layer and SQL-first development. A lakehouse can serve Gold through its SQL analytics endpoint.

4. Connect consumption to the intended contract. Most shared Power BI models should use Gold. Silver can be appropriate for deliberately high-granularity or real-time analysis.

The layers can all live within OneLake, Fabric's logical data lake. Transformed Bronze, Silver, and Gold tables are still distinct datasets. Shortcuts can avoid copying data into Bronze when the source location and governance model support them.

What to watch out for with medallion architecture

Layers without contracts
A folder called Silver says little on its own. Document what checks have run, who owns the dataset, how fresh it should be, and what changes consumers must expect.

Gold as a dumping ground
If every team publishes tables without an owner or agreed meaning, consumers cannot tell which figures to trust. Keep Gold curated and document data lineage for important products.

Silver as compulsory ceremony
Do not persist an intermediate dataset merely to complete the colour sequence. Add Silver when reuse, quality control, recovery, or access separation makes it useful.

Gold logic copied between products
If several Gold products repeat the same source joins and corrections, move that stable logic into Silver. Business definitions that genuinely differ should remain in their named products.

Weak Bronze retention
Overwriting Bronze removes the reliable restart point. Define whether ingestion is append-only or versioned, capture enough metadata to trace a load, and set retention according to recovery and compliance needs.

Permissions copied across layers
Bronze and Silver can contain personal, malformed, or quarantined records that should not be available to every Gold consumer. Grant access per layer and product, then document every exception.