Retrying a send without duplicates: the idempotency key

Your application sends an order confirmation. The request goes out, then nothing: the timeout expires with no response. Was the message accepted? You can't know. If you send it again, you risk writing to the customer twice. If you don't, you risk that they receive nothing.

The idempotency key settles this dilemma: you can retry as many times as needed, and Duva accepts the message only once. This article explains how it works, how to choose it and the mistakes to avoid.

The problem: a lost response

A network request can fail in two very different ways:

• Before arriving. The connection is refused, DNS doesn't answer. Nothing was processed: retrying is safe.
• After being processed. The server accepted the message, then the response got lost on the way, or your timeout expired just before it arrived. The outcome of the first attempt is unknown.

In the second case, a plain "retry" creates a duplicate. Idempotency makes the operation safe: sending the same request with the same key has the same result as sending it once.

How it works with Duva

Add the Idempotency-Key header to POST /v1/{domain}/messages:

curl -X POST https://api.duva.ca/v1/example.com/messages \
  -H "Authorization: Bearer $DUVA_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: order-4821-confirmation" \
  -d '{
    "from": "Example <[email protected]>",
    "to": ["[email protected]"],
    "subject": "Your order",
    "text": "Thank you for your order."
  }'

The rules, as the API reference describes them:

• The key is 1 to 255 printable ASCII characters, with no spaces, and it is unique per domain.
• Same key and same request: the response is a 202 with the same message id and the Idempotent-Replayed: true header. No quota is consumed again, even if it is exhausted.
• Same key, different request: 409 with the idempotency_conflict code.
• Two identical requests at the same time with the same key create only one message.
• Keys don't expire as long as the message exists.

In the lost-response case, the second attempt therefore receives the 202 the first one should have received, with the same msg_... identifier.

Choosing the key

The main trap: a key generated randomly on each attempt protects nothing, since every attempt would have a different key. The key must identify the logical send, not the attempt.

• Good: order-4821-confirmation. It is derived from what you are sending (the confirmation of order 4821) and stays identical on every attempt, even after your application restarts.
• Good: a job identifier from your queue (job-98213), stored before the first try.
• Bad: uuid4() computed inside the retry loop.
• Bad: the recipient's address alone. The second email you send them tomorrow would be taken for a replay of the first, or refused with a 409.

If you use a random identifier, generate it once, before the first try, and keep it with the job.

One key per reason for sending also helps: order-4821-confirmation and order-4821-shipped are two different messages for the same order.

What not to change between two attempts

For Duva, "the same request" means the same content: recipients, subject, body. If you retry with a modified subject, or one more recipient, the response is a 409 idempotency_conflict: the key has already been used for a different request. That is deliberate. It keeps you from believing a corrected version of the message went out when the first one was accepted.

If you really want to send a corrected version, use a new key: it will be a new message.

A retry example

The official Python library does the work for you. To deduplicate a logical send, give it a stable key; it also exposes replayed, which is True when the response comes from a replay:

import os

from duva import Duva

duva = Duva(api_key=os.environ["DUVA_API_KEY"], domain="example.com")

message = duva.messages.send(
    from_="Example <[email protected]>",
    to=["[email protected]"],
    subject="Your order",
    text="Thank you for your order.",
    idempotency_key="order-4821-confirmation",
)
print(message.id, message.replayed)

The library retries transient failures (up to twice by default) and, when you don't give a key, generates one for the call. That generated key protects retries within one call, but not two separate calls: to avoid a duplicate between two runs of your code, supply your own.

Without a library, here is the same idea using Python's standard library:

import json
import time
import urllib.error
import urllib.request

API = "https://api.duva.ca/v1/example.com/messages"


def send_with_retries(api_key: str, payload: dict, key: str, attempts: int = 3) -> dict:
    body = json.dumps(payload).encode()
    for attempt in range(attempts):
        request = urllib.request.Request(
            API,
            data=body,
            method="POST",
            headers={
                "Authorization": f"Bearer {api_key}",
                "Content-Type": "application/json",
                "Idempotency-Key": key,  # the same key on every attempt
            },
        )
        try:
            with urllib.request.urlopen(request, timeout=10) as response:
                return json.load(response)
        except urllib.error.HTTPError as error:
            if error.code < 500:
                raise  # 4xx: retrying the same request will not change the answer
        except OSError:
            pass  # timeout or network error: the outcome is unknown, so retry
        time.sleep(0.5 * 2**attempt)
    raise RuntimeError("send failed after retries")

We ran it against a local fake server that accepts the first send and then "loses" the response: the second attempt receives the same message identifier, and only one message exists. With the same key and a different recipient, the server answers 409, as Duva does.

When to retry, when to stop

Retrying isn't always the right idea:

For a rate_limited 429, don't fire off a burst of new attempts: a 429 of that kind doesn't consume your sending quota, but insisting makes the problem worse.

Over SMTP

If you send over SMTP (smtp.duva.ca), the principle is the same. You can set the X-Idempotency-Key header, which Duva reads and then removes before sending: the recipient never sees it. Without that header, Duva computes a key from the Message-ID and the envelope recipients: software that resends the same message after an interruption doesn't create a duplicate, and gets the same response. The same Message-ID for other recipients is a different message.

A 4xx response from the server invites a retry; a 5xx response is final for that message.

Idempotency on the other side too

Duva sends its webhooks at least once: the same event can arrive more than once. It is up to your receiver to deduplicate on webhook-id, as detailed in Verifying a webhook signature. The two protections complement each other: the key prevents duplicates on the way in, deduplication prevents acting twice on the way out.

And once the message is accepted?

A 202, even a replayed one, means the message is accepted and queued, not that it is delivered. To follow what happens next, keep the msg_... identifier and read its status: see An email didn't arrive: how to find out why.

A checklist

1. Each logical send has a stable key, derived from what it represents.
2. The key is chosen before the first try and kept with the job.
3. Every attempt reuses the same key and the same request.
4. A corrected version of the message uses a new key.
5. 5xx responses, timeouts and network errors are retried with increasing waits and a bounded number of tries.
6. A quota_exceeded is not retried in a loop.
7. 4xx responses are not retried without a fix.
8. You keep the msg_... identifier returned, replay included.