There was nothing unusual about building login yourself. A users table, password hashes, a login endpoint and a reset-password flow were simply part of the application. Over time, that became a much bigger responsibility: MFA, OAuth, SSO, session management, token rotation, audit logs and abuse protection all had to be dealt with.

Today, teams can still own all of this themselves, but it's usually a deliberate architectural decision rather than something that happens because somebody needed a login form.

Event delivery hasn't quite made the same transition.

It often starts with an HTTP request and a retry when the request fails. That's perfectly adequate until the receiving system starts returning 429s or 500s, accepts a request without the acknowledgement making it back, or disappears for three hours and comes back to 80,000 waiting events.

The application gradually picks up everything needed to deal with those situations: somewhere to persist failed events, retry workers and backoff policies, idempotency, dead-letter handling, replay, monitoring, rate limits and eventually some way of recovering without sending the entire backlog at a system that has only just come back online.

None of these things is particularly unusual on its own. That's partly why this happens so easily. The infrastructure arrives one retry job at a time, and eventually an application whose actual job might be accounting, healthcare or logistics contains a fairly substantial delivery system as well.

The HTTP call was never the expensive part. It's the responsibility that accumulates around it.

Handing an event over to Queuey takes about 20 lines of application code. Those lines aren't replacing an HTTP call. They're handing off everything that tends to grow around it: durable buffering, retries, backoff, idempotency, dead-letter handling, replay, destination protection and recovery.

A recent example from Inngest

On September 18, Inngest ran into a particularly nasty version of this.

The failure wasn't an event-delivery problem, but the mechanism is familiar.

A large account deletion triggered an expensive database transaction. While it was still running, the same deletion was attempted again, and those attempts waited behind the original transaction while holding database connections of their own. Client connections eventually reached around 14 times normal levels, PgBouncer became saturated, and services unrelated to the original deletion started losing database access.

In its postmortem, Inngest wrote that "the retries are what turned a slow transaction into an outage."

Simply changing the retry interval might have reduced the pressure, but the changes Inngest is making go further. Hard deletion is being moved out of the request path and represented internally as events with deterministic IDs. The deletion work will be serialized, batched, rate-limited and deduplicated.

Why this still lives in application code

There's nothing inherently wrong with implementing delivery yourself. There are systems where that will be the right decision, just as there are good reasons to own your authentication stack.

What's surprising is how often the decision is never really made.

A queue gets added to solve one problem. A scheduled retry job solves another. Failed deliveries get a database table. Someone adds an admin function for replay. Each addition makes sense locally, but together they amount to a delivery system that now has to be maintained, monitored and operated.

We already use dedicated infrastructure for email, payments and identity because operating these things reliably turned out to be a different job from using them.

That's how we think about Queuey. The application hands over an event, and Queuey takes responsibility for getting it to the destination. If the destination is unavailable, slow or recovering from an outage, dealing with that shouldn't require more delivery machinery in the application.

When everything works, the distinction is easy to dismiss. An HTTP POST and a retry are trivial to implement. The architecture only becomes obvious later, when the destination has been unavailable for three hours, thousands of events are waiting and somebody has to decide how to get them moving again without causing another problem.

Login followed a similar path. It started as ordinary application code and gradually accumulated enough responsibility that treating it as infrastructure made more sense.

Event delivery is following the same path.

Sources