Idempotency Keys: The API Pattern That Prevents Duplicate...

Network requests fail. Timeouts happen. Clients retry. And without idempotency keys, a single payment request can become three charges.

Idempotency keys are one of those API patterns that's easy to overlook until something goes wrong in production. Here's how they work, when to use them, and how to implement them correctly.

What Is Idempotency?

An operation is idempotent if running it multiple times produces the same result as running it once. GET requests are naturally idempotent — fetching a resource 10 times changes nothing. But POST requests (creating a payment, sending an email, provisioning a server) are not. Each call creates a new thing.

Idempotency keys give POST requests the safety of GET requests. You attach a unique key to the request, and the server uses it to deduplicate repeated calls.

How It Works

The client generates a unique key (typically a UUID) and sends it in a header:

POST /payments
Idempotency-Key: a8098c1a-f86e-11da-bd1a-00112444be1e
Content-Type: application/json

{
  "amount": 5000,
  "currency": "usd",
  "source": "tok_visa"
}

The server stores the key alongside the result. If the same key arrives again, it returns the cached response instead of re-executing the operation:

First request  → Process payment → Store result → Return 201 + result
Retry request  → Look up key    → Return cached result (no charge)

Implementing Idempotency Keys Server-Side

Here's a minimal implementation in Node.js with Express and Redis:

const express = require('express');
const redis = require('redis');
const app = express();
const redisClient = redis.createClient();
app.use(express.json());

app.post('/payments', async (req, res) => {
  const idempotencyKey = req.headers['idempotency-key'];

  if (!idempotencyKey) {
    return res.status(400).json({ error: 'Idempotency-Key header required' });
  }

  // Check cache first
  const cached = await redisClient.get(`idempotency:${idempotencyKey}`);
  if (cached) {
    return res.status(200).json(JSON.parse(cached));
  }

  // Acquire lock to prevent concurrent duplicate execution
  const lock = await redisClient.set(
    `lock:${idempotencyKey}`, '1', { NX: true, EX: 30 }
  );
  if (!lock) {
    return res.status(409).json({ error: 'Request in progress, retry shortly' });
  }

  try {
    const result = await processPayment(req.body);

    // Cache the result for 24 hours
    await redisClient.setEx(
      `idempotency:${idempotencyKey}`,
      86400,
      JSON.stringify(result)
    );

    return res.status(201).json(result);
  } finally {
    await redisClient.del(`lock:${idempotencyKey}`);
  }
});

Three details matter here:

Cache the full response, not just a flag. The retry needs to receive exactly what the original call returned — same status, same body.

Set a TTL. Keys don't need to live forever. 24–48 hours covers any realistic retry window.

Lock before processing. If two retries arrive simultaneously before either has stored a result, you need a distributed lock (Redis SET NX) to prevent both threads from executing the payment concurrently.

Sending Idempotency Keys as a Client

On the client side, generate a UUID per logical operation — not per HTTP call. If you're retrying, reuse the same key:

import httpx
import uuid

def create_payment(amount: int, currency: str, retries: int = 3) -> dict:
    # Generate ONCE before the retry loop
    idempotency_key = str(uuid.uuid4())

    for attempt in range(retries):
        try:
            response = httpx.post(
                "https://api.example.com/payments",
                json={"amount": amount, "currency": currency},
                headers={"Idempotency-Key": idempotency_key},
                timeout=10.0
            )
            response.raise_for_status()
            return response.json()
        except (httpx.TimeoutException, httpx.HTTPStatusError) as e:
            if attempt == retries - 1:
                raise
            continue  # Retry with the SAME idempotency_key

The critical mistake to avoid: generating a new UUID inside the retry loop. That defeats the entire purpose.

When to Use Idempotency Keys

Use them on any non-idempotent endpoint where duplicate execution causes real harm:

Payment processing — the canonical case
Email and notification sending
Resource provisioning (servers, databases, subscriptions)
Inventory deductions
Financial transfers

Skip them on read-heavy endpoints or operations already naturally idempotent (like PUT with a full resource replacement).

Edge Cases Worth Knowing

Different payload, same key — decide whether to return an error or the cached result. Stripe returns a 422 if the payload doesn't match the original. That's the safer choice and makes bugs visible.

Failure between charging and caching — if your payment processor charges the card but the Redis write fails, a retry will charge again. Use a transactional outbox or write-ahead log to keep charging and caching atomic.

Key namespace by user — prefix keys with a user or account ID to prevent one user's key from colliding with another's in multi-tenant systems.

Testing the Flow

Testing idempotency manually means crafting the right headers, sending the same request twice, and verifying the second call returns a cached response without re-executing. This is where having a proper API client matters. APIKumo lets you save idempotency key headers directly in your request collections, replay requests with the same headers, and inspect response diffs — so you can verify deduplication behavior without juggling curl flags or writing throwaway scripts. If you're building or integrating payment-grade APIs, it's the kind of workflow support that pays for itself the first time you catch a double-charge bug before it ships.

推荐订阅源

DEV Community