Idempotence

What is idempotence?

Idempotence means that running the same operation once or many times leaves the system in the same final state. If you send the same set this customer's status to active command five times, the customer still ends up active. The repeated attempts do not create extra effects.

The idea comes from mathematics: a function is idempotent when applying it twice gives the same result as applying it once. In software, the useful version is practical: can we safely retry this operation when the network, queue, or API response is unreliable?

Idempotence is why retries can be safe. Without it, a timeout leaves you guessing. Did the request fail before it reached the server, or did the server process it and only the response got lost?

A simple API example

A PUT request that replaces user 42 with the same body is idempotent. Send it once, and user 42 has that body. Send it again, and user 42 still has that body.

A POST request that creates a new payment is usually not idempotent by default. Send it once, and one payment is created. Send it again, and a second payment may be created.

HTTP formalises this distinction. RFC 9110 treats GET, HEAD, PUT, DELETE, OPTIONS, and TRACE as idempotent methods. POST is not idempotent by method definition, although an API can make a POST operation safe by supporting idempotency keys.

Why idempotence matters in distributed systems

Distributed systems fail in awkward ways. A client sends a request. The server processes it. The confirmation never arrives. The client cannot know whether the request was lost or only the confirmation was lost.

If the client does not retry, it may lose work. If it retries a non-idempotent operation, it may duplicate work. That is the tension behind at-most-once and at-least-once delivery.

Idempotent processing lets you choose the safer retry path. A webhook can be delivered twice, a queue message can be redelivered, or a pipeline batch can restart, while the business result still happens once.

That is often called effectively once: the message may arrive more than once, but the final effect is recorded once.

Common implementation patterns

Idempotency keys
The client sends a unique key for one logical operation, often a UUID. The server stores the first result for that key. If the same key appears again, the server returns the stored result instead of repeating the operation. Stripe is the well-known payment example, and many cloud APIs use the same idea under names such as client request IDs.

Natural key plus upsert
If a record already has a stable business key, such as order number, invoice ID, customer ID, or external event ID, write with an upsert instead of a blind insert. Replaying the same sync updates the same row rather than creating another one.

Processed-event table
For event streams and webhooks, store the event IDs you have already processed. When the same event arrives again, skip it or return the earlier result.

Declarative state instead of deltas
Write set balance status to reconciled rather than increment reconciliation_count. Setting a state is often idempotent. Incrementing a counter is not.

Version checks
Use ETags, version numbers, or optimistic locking so a retry does not silently overwrite a newer change.

Where idempotence shows up

• Payment APIs. A retry must not charge the customer twice.

• Webhooks. Providers often retry until your endpoint returns success, so your receiver must handle duplicates.

• Data pipelines. Backfills, CDC jobs, and batch restarts should not double-count orders or invoices.

• Message queues. At-least-once delivery protects against loss but can redeliver messages.

• Reverse ETL. Syncing customers into a CRM should update the same contact, not create a duplicate on every retry.

• Infrastructure APIs. Cloud services often use client request IDs to make create or update calls safe to retry.

What to watch out for

Keys have retention windows
An API may keep idempotency keys for a limited time. Stripe documents that keys can be removed after at least 24 hours. If you retry after the retention window, the operation may run again.

Parameters must match
An idempotency key should belong to one logical operation. Reusing the same key with different parameters is a bug. Good APIs detect that and reject the request.

Side effects need their own protection
Writing the order row idempotently does not automatically make the email, SMS, payment, or CRM update idempotent. Each side effect needs a key, outbox, or deduplication rule.

Deduplication tables need retention too
If duplicate events can arrive days later, do not expire processed-event IDs after one hour.

Exactly-once claims have boundaries
Vendors may provide exactly-once behaviour inside a broker, stream processor, or connector pair. The moment your code calls an external API or database, check whether that destination participates in the guarantee. If it does not, you still need idempotence.