Find the first missing durable record.
Pick one affected event. In your own database, look for its receipt, queued work, processing attempt and final state change. Carry the provider event ID between those records. A request log alone cannot show that a transaction committed.
- No receipt: verify endpoint, environment and filtering.
- Receipt but no queued work: inspect the acceptance transaction.
- Queued but not claimed: inspect worker health, routing and delay.
- Processed but wrong result: check customer mapping and the final write.
Use JSON diff to compare the stored result with the intended state. Remove personal data before putting examples in tickets.
Acknowledge durable acceptance.
Do not start an unawaited promise and then return success from a short-lived handler. The process may stop after the response. Make acceptance durable first, then let a worker perform expensive work. The database and queue handoff must not leave an acknowledged event with nowhere to run.
verifySignature(rawBody, signature);
await database.transaction(async tx => {
await tx.insertReceiptOnce(providerEventId);
await tx.insertOutboxOnce(providerEventId, work);
});
return new Response(null, { status: 200 });This is an outbox pattern sketch, not a complete queue implementation. A dispatcher must deliver committed outbox rows, with retries, and consumers must make repeated work safe.
Separate a duplicate from unfinished work.
A receipt marked “seen” is not necessarily finished. If the first attempt failed after creating that receipt, skipping every repeat can preserve the failure forever. Keep processing state and a retry policy. Make entitlement writes or other effects idempotent with a unique business-operation key.
Stripe documents duplicate delivery and event ordering. Do not let an older event blindly overwrite a newer entitlement. Consult the current source of truth where the event protocol requires it.
Verify the customer-facing result.
Replay a signed development delivery, interrupt the worker after acceptance, and restart it. Confirm the work resumes, the intended account changes once, and an authorised request sees the new state. Then send the same event again.
The local failure lab demonstrates why a receipt is not completion. It uses authored in-memory cases; it is not a production queue test.
References: Stripe webhooks and OWASP webhook security.