Data downtime

What is data downtime?

Data downtime is any period where data is not reliable enough to use. The table may be missing, stale, half-loaded, duplicated, wrongly transformed, or silently out of sync with the source. The dashboard still opens, but the numbers should not be trusted.

The term was popularised by Monte Carlo in the data observability space. It borrows the idea of downtime from software operations. When an application is down, everyone sees it. When data is down, a report can keep showing neat charts while the underlying table has not updated for days.

That is what makes data downtime dangerous. The failure is often quiet. Decisions continue, meetings continue, automated models continue, and nobody realises the input is broken until someone notices a strange number.

Data downtime versus data quality

Data quality asks whether the data is correct, complete, valid, unique, and fit for purpose.

Data downtime asks how long unreliable data stayed in circulation. A dataset can have a quality issue for ten minutes or for three weeks. Those two cases have very different business impact.

So data downtime is a reliability metric. It pushes teams to treat bad data as an incident with detection, ownership, resolution, and follow-up.

How do you calculate data downtime?

A practical way to think about it is:

number of data incidents x (time to detect + time to resolve)

The three pieces matter in different ways.

Number of incidents
How often data breaks, arrives late, changes shape, or violates expectations.

Time to detect
How long it takes before the team knows there is a problem. Without monitoring, this can be days or weeks.

Time to resolve
How long it takes to fix the issue after detection: repair the pipeline, reload the table, correct the transformation, or communicate the impact.

The lesson is useful: you reduce data downtime by preventing incidents, detecting them earlier, and resolving them faster. A bad load caught in fifteen minutes is very different from the same bad load discovered after month-end reporting.

Where data downtime comes from

A pipeline fails halfway
A nightly job stops after loading some tables but before loading others. The orchestrator may show a failure, or worse, a green run with empty output.

A source schema changes
A SaaS system renames a field, changes a data type, or adds a new event shape. The pipeline still runs, but downstream joins or measures break.

A source sends late or wrong data
A partner, API, or operational system sends a file late, sends an empty file, or sends a file with different meaning than yesterday.

A transformation bug ships
A dbt model, notebook, SQL view, or Python script changes and accidentally duplicates rows, drops a filter, or changes the grain of a table.

A dashboard refreshes old data
The report refresh succeeds technically, but the source table has not received new rows. Users see yesterday's or last week's numbers as if they were current.

A concrete example

Monte Carlo has shared a Kargo example where an external partner sent incorrect information, a main pipeline failed, and the issue lasted three days before being resolved. Kargo estimated the impact at about 500,000 dollars.

The useful part of the story is not the size of the company. It is the pattern: a small upstream data issue can become expensive when the downstream team detects it too late.

The SME version is familiar. A nightly load quietly stops updating the sales table. The revenue dashboard still opens every morning. A week later, someone notices the numbers feel wrong, and the team spends hours reconstructing which decisions used stale data.

How to reduce data downtime

Monitor freshness
Know whether important tables have received new data on time. A table that should update hourly should alert before users discover it is six hours old.

Monitor volume
Unexpected row-count drops or spikes often reveal broken loads, duplicate data, or missing source files.

Monitor schema
Detect new, removed, renamed, or type-changed columns before they break downstream models.

Use data tests
Write explicit rules for known expectations: order IDs are unique, amounts are non-negative, required fields are not null, dates are within a sensible range.

Use lineage
When a table breaks, lineage shows which reports, models, and teams are affected. That reduces recovery time and avoids blind communication.

Treat it as an incident
Give every critical dataset an owner, alert the right channel, log the incident, and run a short review after major failures. Data reliability improves when it has the same operating discipline as software reliability.