Webhook retries that stop when they should

Retries are the difference between a demo webhook and production infrastructure. The hard part is not retrying forever. It is knowing when to stop, what to retry, and how to explain that to a human on-call.

Backoff with a ceiling

Start with short delays (seconds), grow exponentially, and cap both interval and attempt count. Outemit uses exponential backoff and records every attempt on the delivery so you can see latency and response bodies without grepping worker logs.

Permanent 4xx should not loop

A 401, 403, or 404 from the receiver usually means the URL or auth is wrong. Retrying for hours burns quota and hides the real bug. Treat non-429 4xx as terminal for automatic retries. Keep 408/429 and 5xx as retryable. Timeouts and connection errors are also retryable.

Retry is not Replay

Retry re-attempts the same delivery to the same endpoint (new attempt row). Replay re-runs the message through the pipeline and may create new deliveries for current subscribers. Mixing them in support runbooks causes duplicate side effects or missed endpoints. Outemit keeps the verbs distinct in API and UI. See Retry vs Replay.

Auto-disable after N failures

After repeated consecutive failures, disable the endpoint and raise an Issue. That protects both sides: you stop hammering a broken URL, and the customer gets a clear signal to rotate secrets or fix their handler.

Related: debug deliveries, signature verifier.