Exactly Once Is a Business Requirement: Making AI Agents Safe to Retry

Every distributed system eventually delivers something twice. Payment teams learned this long ago and built idempotency keys, ledgers, and reconciliation jobs around it. Agent teams are relearning it now, often from a customer who received two refunds, two cancellation notices, or two tickets for one problem.

Four ways an agent does the same thing twice

Duplicate actions in agent systems come from four places. Only one of them involves the model, and each familiar fix covers only some of them.

The usual fixes are partial. Deduplicating on a message ID handles the third case but not the fourth. A provider’s idempotency header handles the first, provided you send the same key twice — which in the second case you will not, if the key came from the call. A single mechanism that covers all four has to work at the level of business intent, the only level at which all four duplicates are the same thing.

Key the intent, not the request

An idempotency key answers one question: is this the same action as one we have already taken? The answer depends on what you hash. Three candidates are tempting, and all three are wrong.

• The tool call ID. Each tool_use block carries an ID that its tool_result echoes back.¹ That ID belongs to one message from the model, so a regenerated call gets a new one. It identifies a message, not an action.
• A hash of the raw arguments. Model retries rephrase. A free-text reason field or a reformatted amount produces a different hash for the same intent.
• A fresh UUID per attempt. This works for a client retrying one HTTP request with the key it already generated. It fails for anything that regenerates the request. Temporal’s guidance follows the same logic: an activity’s key should come from identifiers that stay constant across its retries, not from anything created inside the attempt.²

What works is the business identity of the action: which tenant, which operation, which object, and — critically — the business’s own discriminator for legitimate repeats. A customer can be owed two refunds on one order. They are not owed two refunds on one return authorization. If the business has no such discriminator, raise that before the agent ships, because it means nobody can say whether two refunds were intended.

def canonical(obj):
    return json.dumps(obj, sort_keys=True,
                      separators=(",", ":")).encode()

def intent_key(action, **identity):
    # Who, what, which instance. Never the model's raw
    # arguments, a tool call ID, or a timestamp.
    return sha256(canonical({"action": action, **identity})).hexdigest()

def fingerprint(**details):
    # Normalize first, so "40" and "40.00" agree.
    return sha256(canonical(details)).hexdigest()

key = intent_key("refund.issue", tenant="acme",
                 order="ORD-88412", return_auth="RA-20931")
fp  = fingerprint(amount_cents=4000, currency="USD")

The split matters when a repeat arrives with different details. Same key and same fingerprint is a duplicate: return the original result. Same key with a different fingerprint is a conflict — the agent is trying to refund $40 against a return authorization that was already refunded for $35 — and the right response is to stop and escalate, not to pick one. Stripe applies the same rule at its own boundary: as of October 2026, its idempotency layer compares incoming parameters with those of the original request and returns an error if they differ.³

Hashing also keeps raw identifiers out of a value you are about to send to other companies. Stripe’s documentation asks callers not to put sensitive data such as email addresses in idempotency keys, and suggests random V4 UUIDs.³ A random UUID is the right advice for a client retrying its own request. For a system that regenerates its requests, a hash of the intent meets the same goals: it has plenty of entropy and carries no personal data.

Claim before you act

The key needs somewhere to live that outlasts the worker, the session, and the model’s context. That place is the side-effect ledger: one durable row per intent, written before the effect happens and updated after it.

The order of those two writes is the whole design. Write the row after the effect, and a crash in between loses the record of an action that happened. Write it before, as a conditional insert, and exactly one caller can claim a given intent; every other caller learns that someone already has.

def claim(key, fp, owner, lease_s=300):
    now = int(time.time())
    try:
        ledger.put_item(
            Item={"pk": key, "state": "IN_PROGRESS", "fp": fp,
                  "owner": owner, "lease_until": now + lease_s},
            ConditionExpression="attribute_not_exists(pk)")
        return Claimed()
    except ledger.meta.client.exceptions.ConditionalCheckFailedException:
        row = ledger.get_item(Key={"pk": key},
                              ConsistentRead=True)["Item"]

    if row["fp"] != fp:
        return Conflict(row)           # same intent, new details
    if row["state"] == "COMPLETED":
        return AlreadyDone(row["result"])
    if row["lease_until"] > now:
        return InFlight(row)           # another worker holds it
    return Reconcile(row)              # lease lapsed: outcome unknown

The last line is the one most implementations get wrong. An expired lease looks like an abandoned claim, and the tempting response is to take it over and run the action. But a lease expires precisely when a worker died mid-action — exactly when the effect may already have happened. An expired lease means unknown, not not done, and the way to resolve an unknown is to ask the downstream system, not to act again.

The conditional write is a documented DynamoDB primitive: with attribute_not_exists on the key, the write proceeds only when no item with that key exists, and is rejected otherwise.⁴ Any store with compare-and-set or a unique constraint can play the same role.

Powertools for AWS Lambda packages much of this for function handlers: an in-progress record, a stored response, optional validation that the payload has not changed under the same key, and an error for a concurrent duplicate. Its defaults deserve a careful read. As of October 2026, records expire after one hour by default. The utility also documents that if a function times out mid-invocation and a retry arrives after the in-progress window, it runs the invocation again as if the record had expired.⁵ That is the right behavior when the downstream call is itself idempotent by key. When it is not, it is the ambiguous window above, handled by acting again. Use the utility where it fits, and decide case by case whether a lapsed claim should re-run or reconcile.

Carry the key across the boundary

The ledger makes your side of the boundary safe. The effect itself happens in someone else’s system, and the ambiguous window — effect committed, acknowledgment lost — sits exactly on that boundary. How you close it depends on what the downstream API offers.

A timeout is not a failure

When a call times out, you do not know whether the effect happened. Marking the intent failed invites a retry that may duplicate it. Marking it complete records an action that may not exist. The only accurate state is the third one.

try:
    result = payments.create_refund(
        order=order, amount_cents=4000,
        idempotency_key=f"{key}:refund")
except (Timeout, ConnectionReset):
    # Not a failure. The refund may exist.
    ledger.mark(key, "UNKNOWN")
    return reconcile_later(key)
ledger.complete(key, result)

A reconciler resolves UNKNOWN rows by asking the downstream system about the key or the natural reference, then marks each row completed or releases it for another attempt. If the downstream API honors idempotency keys, reconciling can be as simple as retrying with the same key inside its retention window — a strong argument for choosing systems that support them.

Tell the agent what already happened

The ledger stops a duplicate from executing. It does not stop the agent from trying, and an agent that keeps trying burns tokens and latency — and, wherever a tool lacks a guard, creates risk. The fourth source of duplicates is a context problem, and the fix is to put what the ledger knows where the model can see it.

Two places matter. First, when a guarded tool refuses a repeat, its result should say so in terms the model can act on. A bare error invites a workaround: the model retries with different arguments, or reaches for another tool that has the same effect. A result that reports the action as already completed, with the original outcome, gives the model a true statement about the world to plan from.

status:        already_completed
intent:        refund.issue ORD-88412 / RA-20931
result:
  refund_id:   re_3Q8xK2
  amount:      40.00 USD
completed_at:  2026-10-20T14:02:11Z
action_taken:  none

Second, when a session resumes or a long run is compacted to fit the context window, carry the completed intents forward explicitly rather than trusting a summary to mention them. A planner cannot check anything against a compacted history that says “handled the customer’s refund request.” It can check against a list of completed intent keys and their results. Agent memory has to be designed rather than accumulated, and this list is the part of it with money attached.⁶

Sort the tools by what a repeat costs

Not every tool needs the same machinery. Treating them all alike either over-builds the harmless ones or under-builds the dangerous ones. A useful first pass sorts every side-effecting tool into four classes before any of the code above is written.

The classes also order the plan. When a workflow mixes them, irreversible steps belong last, after everything that can fail has had its chance to. A workflow that sends the confirmation email before booking the appointment has made its only irreversible step depend on everything after it succeeding. Notifications in particular are better written to an outbox, keyed on the intent, by the step that makes the business change, and sent by a separate dispatcher that marks each one delivered. A retried step then rewrites the same outbox row instead of sending a second message.

This is the same sorting that sets autonomy levels — matching autonomy to consequence⁷ — applied one layer down, to what happens when the action repeats.

Approvals arrive twice too

Human approval is usually presented as the safety mechanism for risky actions. It is also a new source of duplicates, with failure modes of its own. A reviewer double-clicks. A notification goes to two channels and is approved in both. An approval arrives after another path has already handled the request. Or the agent re-plans while it waits, and the action it eventually executes is not quite the one that was approved.

Your platform’s guarantee is shorter than your agent’s memory

Every queue, runtime, and workflow engine documents a delivery guarantee, and the differences matter. As of October 2026, the vendors’ own documentation says the following. Read the table as a snapshot of current behavior, not a fixed rule, and check it against your own configuration.

Two patterns run through that table. First, every deduplication feature has a window, and the windows are short compared with an agent’s timescale: five minutes for FIFO deduplication, an hour by default for a Powertools record,⁵ as little as a day for Stripe’s v1 key retention.³ An agent that waits on a human approval, or a session that resumes the next morning, outlives all of them. Set the ledger’s retention from the longest legitimate gap between an intent and its last possible retry, human waits included, not from a framework default.

Second, orchestrator identifiers deduplicate orchestrator repeats. Temporal recommends building a key from the workflow run ID and the activity ID, a value that stays constant across retries of one activity but is unique across workflow executions — exactly what it is for.² When an agent re-plans and schedules the same business action as a new activity, that key is new, and only an intent key catches the repeat. Use both: the platform’s guarantee for the infrastructure, the intent key for the business.

Prove it by crashing the worker

Idempotency is a claim about how a system behaves under failure. That behavior never shows up in a happy-path demo or in an evaluation set of model outputs, so the only way to test it is to cause the failures on purpose. The suite that earns trust is small and specific.

def test_crash_between_effect_and_record(fake_payments, ledger, clock):
    with crash_after(fake_payments.create_refund):
        with pytest.raises(WorkerCrash):
            run_step(refund_intent)

    clock.advance(seconds=301)          # the lease lapses
    run_step(refund_intent)              # the redelivery

    assert fake_payments.refunds_for("ORD-88412") == 1
    assert ledger.state(refund_intent.key) == "COMPLETED"

Run these in CI against fakes that count effects, and again in staging against the real downstream sandboxes, because the real systems’ key retention and error behavior are part of what you are testing. It is the same discipline as asserting on properties rather than strings when models change:¹² the assertion is about the world — one refund exists — not about what any component reported.

What to take from this

None of these techniques is new. Idempotency keys, conditional writes, leases, outboxes, and reversing actions are standard tools of payment and distributed-systems engineering. What is new is how many teams now put a component that generates its own requests — and can forget it made them — in front of systems that move money and send messages.

That component makes the problem worse in one specific way: it erases the boundary most deduplication relies on. Infrastructure deduplicates repeated requests. An agent produces new requests that mean the same thing. Only at the level of business intent are those recognizably the same, so that is where the key has to come from and where the ledger has to live.