Idempotency

Networks fail after the server has done the work: a client sends POST /payments, the connection drops, and it can't know whether the card was charged. Retry, and the customer may pay twice. Don't, and the order may never be paid. Money makes the stakes obvious, but every write with a side effect has the same problem: an email sent twice, an order shipped twice, a job queued twice.

An idempotency key removes the dilemma. The client attaches a key to the operation, and the server promises that the same key always produces the same result, with the side effect happening at most once. Retrying becomes always safe, so clients can retry freely, which is exactly what you want them to do.

The Header#

Every POST accepts an Idempotency-Key header, as defined by the IETF Idempotency-Key draft.

POST /payments
Idempotency-Key: f47ac10b-58cc-4372-a567-0e02b2c3d479
Content-Type: application/json

{ "amount": 1999, "currency": "USD", "source": "tok_visa" }

• The client generates the key, because only the client knows when two requests are the same operation. A UUID v4 is ideal: random, with no information to leak. Up to 255 characters.
• One key per logical operation, not per HTTP attempt. A key generated fresh on every retry protects nothing. Generate it when the user clicks "Pay", store it with the order, and reuse it until the operation resolves.
• Never put sensitive data in a key. Keys end up in logs and storage. An email or card number in a key is a leak waiting to happen.
• Only POST needs one. GET, PUT and DELETE are idempotent by definition in RFC 9110: repeating them leaves the same state. PATCH isn't by definition, but a JSON Merge Patch that sets absolute values is idempotent in practice. A key sent with any of these is ignored.

What the Server Stores#

The first request with a key records:

Field	Purpose
Key, scoped to the API account	Two accounts' keys can't collide, and one account can't probe another's
Fingerprint of the request	Method, path and a hash of the body, to detect a key reused for something else
Status	in_progress or completed
Response	Status code, headers and body, once completed
Created at	For expiry

The key record and the side effect are written in the same database transaction wherever possible. That's the whole trick: if the payment row commits, so does the key, and if either fails, neither exists. Storing the key in a separate system (a cache, a different database) reopens the gap the key was meant to close, because a crash between the two writes leaves a charge with no record of it.

When the side effect lives somewhere else, like a card network, a single transaction isn't possible. Record the key as in_progress first, call the processor with its own idempotency key derived from the client's, then complete the record. A crash in the middle leaves an in_progress key that a recovery job reconciles against the processor.

Claiming a key is a single atomic operation: an insert against a unique constraint, or a set-if-absent in a key store. A read followed by a write lets two simultaneous first requests both see "new" and both run.

If the key store is unreachable, fail closed: reject the request with a 503 and do nothing. A rate limiter can reasonably keep serving when its store is down (Rate Limiting), but a write without its idempotency protection is exactly what the key exists to prevent.

Behavior#

Request	Result
New key	Processed normally, response stored
Same key, same request, completed	Stored response replayed, nothing re-executed
Same key, same request, still in progress	409 Conflict: retry shortly with the same key
Same key, different request	400 Bad Request: the key is already bound to another operation
Key older than the retention window	Treated as new

A replayed response is byte-for-byte the original and adds one header, so the client can tell it's a replay:

HTTP/1.1 201 Created
Idempotent-Replayed: true

The mismatch case is a client bug: the same key was attached to two different operations. The IETF draft suggests 422 here, and I return 400 instead, as Stripe does. In these docs 422 means field-level validation failed and comes with an errors object pointing at the fields (Error Responses). A key mismatch has no field to point at. The body itself may be perfectly valid; it's the pairing with the key that's wrong.

What Gets Stored, and Why Errors Count

A result is stored once the endpoint begins executing, whatever the outcome. That includes declines and even 500 errors, which surprises people.

A 500 from a payment endpoint doesn't mean nothing happened; it means the server doesn't know what happened. The charge may have gone through before the error, and re-executing on retry could charge the card again. Replaying the stored 500 is the honest answer: "this outcome is uncertain, go and check". The client reconciles by fetching the payment or watching webhooks instead of gambling with a second charge.

Requests rejected before execution aren't stored: failed validation, bad authentication, rate limiting (429), or a 409 for a concurrent request. Nothing happened, so the client fixes the problem and retries with the same key.

Retention#

Keys are kept for at least 24 hours, then pruned. Reusing a pruned key starts a new operation.

A retry window has to outlast realistic failures: seconds cover network blips, minutes cover a deploy, and a day covers a client that crashed overnight and drains its queue in the morning. Past that, a "retry" is really a new decision someone should make deliberately, and keeping keys forever means a table that grows with every write the API has ever served. Stripe uses the same 24-hour minimum. Past the window, clients stop retrying with any key and reconcile: fetch the object, or check events.

Idempotency Beyond the API#

The header protects the API boundary. The same rule applies at every hop inside the system, because every hop can fail halfway:

• Calls to processors carry their own idempotency key, derived from the client's, so a retry to the card network can't double-charge either.
• Queue consumers dedupe by message ID, because every queue worth using delivers at least once.
• Webhook receivers dedupe by event ID. See Webhooks.

Exactly-once delivery doesn't exist across a network. At-least-once delivery plus idempotent handling is how you get exactly-once effects, and that's the only kind that matters.

Client Expectations#

1. Generate a key before the first attempt, and persist it with the operation so it survives process restarts.
2. Reuse that key on every retry, with the exact same body, backing off as in Retries & Timeouts.
3. On 409, retry the same key. A new key could start a second operation while the first is still running.
4. On 400 for a key mismatch, you've reused a key for a different operation. Fix the client.
5. To change the parameters, first confirm the original didn't succeed and isn't still running, then use a new key.

References#

Specifications

• IETF: The Idempotency-Key HTTP Header Field (draft), the header, 409 for concurrent requests, and published expiry
• RFC 9110: HTTP Semantics, which methods are idempotent by definition

Reference

• MDN: Idempotency-Key, header syntax
• MDN: 409 Conflict, concurrent requests with the same key

In practice

• Stripe: Idempotent requests, storing results once execution begins, including 500s, and a 24-hour minimum
• Stripe: Advanced error handling, retrying within the window, then reconciling
• Stripe: Designing robust and predictable APIs with idempotency, why keys exist
• Brandur Leach: Implementing Stripe-like Idempotency Keys in Postgres, storing keys in the same transaction as the side effect
• AWS Builders' Library: Making retries safe with idempotent APIs, idempotency across internal hops