Delivery guarantees

At-least-once delivery, best-effort ordering and a retry window that ends — and what each one means for a number somebody will argue about.

Three properties of delivery decide how you have to write a reporting handler. All three are the platform contract, documented in Delivery; this page is what they cost you if you ignore them.

At-least-once

The platform retries a failed delivery with a doubling gap between attempts, 11 attempts over about 8.5 hours. A short outage on your side does not lose an event, and the same event can reach you more than once.

For reporting that is not a rounding error, it is a wrong total. An INSERT per delivery double-counts every retried event, and the double count is invisible: nothing in your store says which row was a duplicate.

Two safe shapes:

  • Upsert keyed on the event id from X-Revenexx-Event-Id. The second delivery of the same event overwrites the first row instead of adding one.
  • A processed-ids table. Record the id before you act, and drop anything already there. Slightly more work, and it survives a handler that writes to several places.

What does not work is deduplicating on the entity id, or on a timestamp, or on "it looks like one we already had".

Best-effort ordering

Order is not guaranteed. Retries and backoff mean a retried event can arrive after a later one — order.acknowledged can land before order.placed.

Three ways to be immune:

  1. Use the entity's own time, not arrival. Sort and window on the timestamp the entity carries, from data or a re-read, and your series is right whatever order the deliveries came in. A delivery does not carry the event's time.
  2. Make handling commutative. If the end state is the same regardless of order, ordering stops being a correctness problem. Upserting a row keyed on the entity, with a last-write-wins on its modification time, gets you there.
  3. Re-read the entity when the order matters. If deriving state from a sequence of events is unavoidable, let the event be the trigger and the API be the source of truth — see Incremental sync.

Beware the handler that assumes a creation event exists before an update event. It works in testing, where events arrive one at a time in order, and fails on the first busy afternoon.

When the retries run out

An event whose 11 attempts all failed is not sent again, and there is no manual redelivery, from Cockpit or the API. The endpoint's delivery attempts in Cockpit show which attempts failed and what your receiver returned, but not the payloads.

For a reporting pipeline that makes the API the recovery tool:

  • Your endpoint was down for longer than the retry window. Re-read the window from the API rather than waiting for events that will not come.
  • You shipped a bug in the handler. Fix it, then re-read the affected entities; the events already acknowledged are not sent again.
  • You are standing up a new store. Backfill from the API; the stream has nothing to replay.

Anything you will need in a year, store yourself, which is the argument for recording the raw payload as it arrives. An endpoint is not switched off automatically, but a person can switch it off in Cockpit, and for a metrics pipeline a switched-off endpoint reads exactly like a quiet week.

The reconciliation habit

Guarantees describe the transport, not your correctness. The habit that makes reporting trustworthy:

Daily
1. Handle events as they arrive → your store
2. Once a day, re-read the affected entities from the API for the window
3. Compare, and fix your store where they disagree
4. Alert if the disagreement is not zero

Step 4 is the one people skip, and it is the one that tells you the pipeline broke before a customer does.

Where to go next

Was this page helpful?