Designing Idempotent APIs for Financial Ledgers

The Problem

In a multi-currency financial ledger, the most dangerous class of bugs are silent duplicates. A user retries a failed network request, your load balancer forwards the request twice, or a consumer processes a Kafka message after a pod restart — in each case, you risk crediting or debiting an account more than once. This is a double-spend vulnerability, and in production, it silently destroys trust.

What Idempotency Means in Practice

An idempotent operation is one where performing it N times has the same effect as performing it once. For an API, this means: given the same input and the same Idempotency-Key header, return the same result without side effects.

The most common implementation uses a client-generated UUID as an Idempotency-Key header. The server stores it in Redis with a TTL:

func HandlePayment(ctx context.Context, req PaymentRequest) (*PaymentResult, error) {
    key := "idempotency:" + req.IdempotencyKey
    
    // Check if request was already processed
    cached, err := redis.Get(ctx, key).Result()
    if err == nil {
        // Return the cached result
        var result PaymentResult
        json.Unmarshal([]byte(cached), &result)
        return &result, nil
    }

    // Process the payment
    result, err := processPayment(ctx, req)
    if err != nil {
        return nil, err
    }

    // Cache the result for 24 hours
    data, _ := json.Marshal(result)
    redis.Set(ctx, key, data, 24*time.Hour)
    
    return result, nil
}

Distributed Locking to Prevent Races

The Redis check above has a race condition: two concurrent requests with the same key can both miss the cache, both process, and both write back. The fix is a distributed lock.

func HandlePayment(ctx context.Context, req PaymentRequest) (*PaymentResult, error) {
    lockKey := "lock:payment:" + req.IdempotencyKey
    
    // Acquire distributed lock (SET NX with expiry)
    acquired, err := redis.SetNX(ctx, lockKey, "1", 10*time.Second).Result()
    if !acquired || err != nil {
        return nil, errors.New("concurrent request in flight, retry")
    }
    defer redis.Del(ctx, lockKey)

    // ... rest of idempotency logic
}

PostgreSQL as the Source of Truth

Redis is fast but ephemeral. For production financial systems, also persist the idempotency record in PostgreSQL with a unique constraint:

CREATE TABLE idempotency_records (
    key         TEXT PRIMARY KEY,
    response    JSONB NOT NULL,
    created_at  TIMESTAMPTZ DEFAULT NOW()
);

The unique constraint on the primary key means the database will reject a second INSERT for the same key at the storage level — a final safety net even if Redis fails.

Key Takeaways