Intro

Idempotency means applying the same logical operation multiple times produces the same final state as applying it once. In distributed systems this is not optional because retries, timeouts, dropped acknowledgments, and at least once delivery are normal operating conditions, not edge cases. You reach for idempotency on any retriable operation such as API writes, message consumers, payment captures, and order creation. Without it, retries become dangerous and can double charge customers, create duplicate orders, or drift state across services.

Mechanism

Idempotency is implemented at the boundary where duplicates can enter the system, then reinforced at persistence boundaries so duplicates cannot corrupt state.

Natural idempotency

• SET balance = 100 is idempotent because repeating it keeps the same result.
• INCREMENT balance by 100 is not idempotent because each retry changes state again.
• HTTP PUT and DELETE are idempotent by semantics.
• HTTP POST is not idempotent by default because each call usually creates a new server side effect.

Idempotency keys

For non-idempotent operations, the client sends a unique key per logical operation in Idempotency-Key.

Server flow:

1. Read Idempotency-Key.
2. Lookup key in a durable store.
3. If key exists and request fingerprint matches, return stored response.
4. If key does not exist, process exactly once, persist response, and bind response to key.

This turns ambiguous network outcomes into deterministic behavior for retries.

Database level techniques

• UPSERT: INSERT ... ON CONFLICT DO UPDATE protects read model and projection handlers from duplicate writes.
• Unique constraints: enforce one row per business identity such as merchant_id + client_operation_id.
• Conditional updates: optimistic concurrency like UPDATE ... WHERE version = @expectedVersion prevents repeated stale updates from silently overwriting new state.

CREATE TABLE payments (
    payment_id UUID PRIMARY KEY,
    merchant_id UUID NOT NULL,
    client_operation_id TEXT NOT NULL,
    amount_cents BIGINT NOT NULL,
    status TEXT NOT NULL,
    version INT NOT NULL,
    created_utc TIMESTAMPTZ NOT NULL,
    UNIQUE (merchant_id, client_operation_id)
);

sequenceDiagram
    participant Client
    participant Api
    participant KeyStore
    participant Domain
    Client->>Api: Send request with key
    Api->>KeyStore: Check key
    KeyStore-->>Api: Key missing
    Api->>Domain: Process operation
    Domain-->>Api: Result ready
    Api->>KeyStore: Save key and result
    Api-->>Client: Return result
    Client->>Api: Retry with same key
    Api->>KeyStore: Check key
    KeyStore-->>Api: Key found with result
    Api-->>Client: Return cached result

Idempotency is especially important for Message Queues consumers and for multi step workflows like Distributed Transactions where partial failures are expected.

HTTP Methods and Idempotency

• Idempotent by spec: GET, PUT, DELETE, HEAD, OPTIONS.
• Not idempotent by default: POST, PATCH.

Interview critical distinction:

• PUT is idempotent because it replaces the representation with a full target state.
• PATCH applies a delta, and applying the same delta repeatedly can compound side effects unless the patch document itself is designed to be idempotent.

Even with idempotent methods, distributed replicas can still show temporary divergence depending on Consistency Models, so method semantics and system consistency level are separate concerns.

.NET Implementation Example

Example below uses ASP.NET Core .NET 8 with PostgreSQL and an external payment gateway. It handles duplicate concurrent requests by inserting an idempotency row first under a unique key and reusing a cached response on retries.

CREATE TABLE api_idempotency (
    idempotency_key TEXT PRIMARY KEY,
    request_hash TEXT NOT NULL,
    status_code INT,
    response_json JSONB,
    state TEXT NOT NULL,
    created_utc TIMESTAMPTZ NOT NULL,
    expires_utc TIMESTAMPTZ NOT NULL
);

using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using Dapper;
using Npgsql;

var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();

app.MapPost("/payments", async (HttpContext http, NpgsqlDataSource dataSource, CancellationToken ct) =>
{
    var key = http.Request.Headers["Idempotency-Key"].ToString();
    if (string.IsNullOrWhiteSpace(key))
        return Results.BadRequest(new { error = "Idempotency-Key header is required" });

    var request = await JsonSerializer.DeserializeAsync<ChargeRequest>(http.Request.Body, cancellationToken: ct);
    if (request is null || request.AmountCents <= 0)
        return Results.BadRequest(new { error = "Invalid request body" });

    var requestHash = ComputeRequestHash(request);
    await using var conn = await dataSource.OpenConnectionAsync(ct);
    await using var tx = await conn.BeginTransactionAsync(ct);

    const string insertPending = """
        INSERT INTO api_idempotency (idempotency_key, request_hash, state, created_utc, expires_utc)
        VALUES (@Key, @Hash, 'Pending', now(), now() + interval '24 hours')
        ON CONFLICT DO NOTHING;
        """;

    var inserted = await conn.ExecuteAsync(new CommandDefinition(
        insertPending,
        new { Key = key, Hash = requestHash },
        tx,
        cancellationToken: ct));

    if (inserted == 0)
    {
        const string readExisting = """
            SELECT request_hash, status_code, response_json, state
            FROM api_idempotency
            WHERE idempotency_key = @Key;
            """;

        var existing = await conn.QuerySingleAsync<IdempotencyRecord>(new CommandDefinition(
            readExisting,
            new { Key = key },
            tx,
            cancellationToken: ct));

        if (!string.Equals(existing.RequestHash, requestHash, StringComparison.Ordinal))
            return Results.Conflict(new { error = "Key reused with different payload" });

        if (existing.State == "Completed" && existing.StatusCode is not null && existing.ResponseJson is not null)
            return Results.Json(existing.ResponseJson, statusCode: existing.StatusCode.Value);

        return Results.StatusCode(StatusCodes.Status409Conflict);
    }

    ChargeResult gatewayResult;
    try
    {
        gatewayResult = await ChargeProvider.ChargeAsync(request, key, ct);
    }
    catch (Exception ex)
    {
        await tx.RollbackAsync(ct);
        return Results.Problem($"Payment provider error {ex.Message}", statusCode: StatusCodes.Status502BadGateway);
    }

    var apiResponse = new
    {
        paymentId = gatewayResult.PaymentId,
        status = gatewayResult.Status,
        amountCents = request.AmountCents,
        currency = request.Currency
    };

    const string complete = """
        UPDATE api_idempotency
        SET state = 'Completed',
            status_code = @StatusCode,
            response_json = @Response::jsonb
        WHERE idempotency_key = @Key;
        """;

    await conn.ExecuteAsync(new CommandDefinition(
        complete,
        new
        {
            Key = key,
            StatusCode = StatusCodes.Status200OK,
            Response = JsonSerializer.Serialize(apiResponse)
        },
        tx,
        cancellationToken: ct));

    await tx.CommitAsync(ct);
    return Results.Json(apiResponse, statusCode: StatusCodes.Status200OK);
});

app.Run();

static string ComputeRequestHash(ChargeRequest request)
{
    var json = JsonSerializer.Serialize(request);
    var bytes = SHA256.HashData(Encoding.UTF8.GetBytes(json));
    return Convert.ToHexString(bytes);
}

public sealed record ChargeRequest(Guid CustomerId, long AmountCents, string Currency, string CardToken);

public sealed record ChargeResult(string PaymentId, string Status);

public sealed class IdempotencyRecord
{
    public string RequestHash { get; init; } = string.Empty;
    public int? StatusCode { get; init; }
    public JsonElement? ResponseJson { get; init; }
    public string State { get; init; } = string.Empty;
}

public static class ChargeProvider
{
    public static Task<ChargeResult> ChargeAsync(ChargeRequest request, string idempotencyKey, CancellationToken ct)
    {
        var result = new ChargeResult($"pay_{Guid.NewGuid():N}", "Succeeded");
        return Task.FromResult(result);
    }
}

Why this avoids race conditions:

• The unique key insert is the concurrency gate.
• Exactly one request wins INSERT and proceeds to process the charge.
• Concurrent duplicates lose the insert and fetch existing status or cached response.
• Request hash prevents accidental key reuse across different payloads.

Pitfalls

Message handlers not idempotent under at least once delivery

• What goes wrong: duplicate events apply side effects multiple times, such as multiple shipment records.
• Why it happens: brokers redeliver when ack is lost or consumer crashes after processing.
• How to avoid it: persist processed message ids, use upserts, and make handlers safe to replay.

Idempotency key scope set incorrectly

• What goes wrong: broad scope blocks legitimate operations or narrow scope fails to deduplicate retries.
• Why it happens: key design ignores tenant and operation boundaries.
• How to avoid it: scope keys by business boundary like tenant plus operation id and enforce payload hash checks.

Check then process race window

• What goes wrong: two concurrent duplicates both pass pre check and both charge.
• Why it happens: non atomic check then execute logic without a uniqueness barrier.
• How to avoid it: move gate into atomic insert or unique constraint with transaction and conflict handling.

Idempotent is not safe

• What goes wrong: teams assume idempotent operations have no side effects.
• Why it happens: confusion between HTTP safe and HTTP idempotent semantics.
• How to avoid it: remember DELETE is idempotent but still mutates state and can require authorization and auditing.

Tradeoffs

Decision rule: start with database uniqueness for core entities, add idempotency keys for externally visible POST operations and third party side effects, then use optimistic concurrency for high contention aggregate updates.

Questions

References

• Stripe API docs Idempotent requests - Anchor source with concrete semantics for key reuse, result caching, TTL, and retry behavior.
• Stripe engineering blog Designing robust and predictable APIs with idempotency - Practitioner writeup on failure modes and retry design tradeoffs from production payment APIs.
• Microsoft Learn Recommendations for handling transient faults - Reliability guidance that explicitly calls out idempotency as prerequisite for safe retries.
• Microsoft Learn Data platform for mission critical workloads on Azure idempotent message processing - Practical cloud architecture guidance for dedup in at least once messaging.
• Chris Richardson Idempotent Consumer pattern - Practitioner pattern for implementing duplicate safe message handlers.
• IETF RFC 9110 HTTP Semantics - Protocol level definition of idempotent methods and their intended behavior.