Skip to content
Shipworthy

Idempotent writes in DynamoDB with a guard item

How to make a DynamoDB write safe to retry: a guard item, a conditional TransactWrite, and why that beats checking first and writing second under load.

By · · 2 min read

A write is idempotent when doing it twice has the same effect as doing it once. In a serverless app that matters more than it sounds: Lambda retries, clients retry on timeouts, and users double-click. If the write is "add 20 points" or "record this payment", running it twice is a bug. In DynamoDB the clean fix is a guard item written in the same transaction as the real change, with a condition that it must not already exist.

Why check-then-write fails

The obvious approach is to look first and write second:

const seen = await get({ pk: `IDEM#${key}` });
if (!seen) {
  await awardPoints(userId, 20);
  await put({ pk: `IDEM#${key}` });
}

Two requests with the same key can both run the get before either runs the put. Both see nothing, and both award the points. Under load that is not rare; a retry after a slow response triggers it by design. The check and the write have to be a single atomic operation.

The guard item pattern

Put the guard item and the real changes in one TransactWriteItems call. The guard item's put carries a condition: attribute_not_exists(pk). If the key was used before, the condition fails and DynamoDB cancels the whole transaction, so none of the other writes happen either.

import {
  DynamoDBClient,
  TransactionCanceledException,
} from "@aws-sdk/client-dynamodb";
import {
  DynamoDBDocumentClient,
  TransactWriteCommand,
} from "@aws-sdk/lib-dynamodb";

const client = DynamoDBDocumentClient.from(new DynamoDBClient({}));
const table = process.env.TABLE_NAME;

export async function awardOnce(
  userId: string,
  key: string,
  amount: number,
): Promise<"awarded" | "duplicate"> {
  const now = Math.floor(Date.now() / 1000);
  try {
    await client.send(
      new TransactWriteCommand({
        TransactItems: [
          {
            Put: {
              TableName: table,
              Item: { pk: `IDEM#${key}`, sk: "IDEM", expiresAt: now + 90 * 86400 },
              ConditionExpression: "attribute_not_exists(pk)",
            },
          },
          {
            Update: {
              TableName: table,
              Key: { pk: `USER#${userId}`, sk: "PROFILE" },
              UpdateExpression: "ADD xp :amount",
              ExpressionAttributeValues: { ":amount": amount },
            },
          },
        ],
      }),
    );
    return "awarded";
  } catch (error) {
    const guardFailed =
      error instanceof TransactionCanceledException &&
      error.CancellationReasons?.[0]?.Code === "ConditionalCheckFailed";
    if (guardFailed) return "duplicate";
    throw error;
  }
}

Three details make this work:

  1. The guard is first in the list, so CancellationReasons[0] is the one to inspect. The reasons come back in the same order as the items.
  2. A duplicate is a success. The caller asked for the award to have happened, and it has. Return normally instead of surfacing an error.
  3. Anything else is a real failure. A cancellation for another reason, or throttling, should propagate so the caller can retry with the same key.

Choosing the key

The key must describe the intent, not the request. For "award XP for finishing this article" a good key is the user, the article and the event type together, such as user123#post-slug#article.completed. Two different requests that mean the same thing then collide, which is the point. A random request ID would not: every retry would get a new one.

Expiring guard items

Guard items accumulate. Give them a TTL attribute (expiresAt above, in epoch seconds) and enable TTL on the table, and DynamoDB deletes them for free. The TTL only has to outlast any realistic retry or replay of the same intent. For events that must never repeat, such as a one-time signup bonus, skip the TTL and keep the guard forever.

Costs and limits

  • Transactional writes use twice the write capacity of the same writes done individually.
  • A transaction can include up to 100 items, and they cannot target the same item twice.
  • TransactWriteItems also accepts a ClientRequestToken, which makes an identical retry of the same call idempotent for 10 minutes. It is a useful extra layer for network retries, but it does not catch two different requests with the same intent, so it does not replace the guard item.

This is the pattern Shipworthy's reading rewards are designed around: every award has an idempotency key, so a replayed request can never pay out twice. The first build log explains why the site has rewards at all.

Frequently asked questions

Isn't ClientRequestToken enough for idempotent transactions?

It helps, but only within a 10-minute window and only for identical retries of the same request. A guard item keyed by your own idempotency key lasts as long as you keep it and works across different requests that mean the same thing.

What does a duplicate look like with this pattern?

The transaction is cancelled with TransactionCanceledException, and the cancellation reason for the guard item is ConditionalCheckFailed. Treat that as success: the work was already done.

Does a transaction cost more than separate writes?

Yes. Transactional writes consume twice the write capacity of the same writes done individually, so keep the transaction to the items that must change together.

Keep reading