carlos / platform Public

Ask

A per-client rate limit on public HTTP traffic at the edge, so an app instance never sees one client's flood, while legitimate traffic still reaches it at full speed.

• Keyed by client IP. IPv6 should be keyed by prefix (a /64), so one host can't rotate addresses within its own allocation.
• Over the limit, the edge answers 429 with Retry-After itself, without waking or forwarding to the instance.
• Generous defaults. It exists to stop one client from monopolising an instance, not to cap a real on-sale rush, where thousands of buyers each fetch a page and a few assets. A burst allowance on top of a sustained rate fits that shape.
• Configurable per app, ideally per route/host. Tito would want its own numbers, and a carlos verb or app config, not an env var on the box.
• Observable. Rejections should be counted per host somewhere we can read (logs at minimum), so we can tell when a real customer is the one being limited.

Why now

Tito is working to raise per-instance throughput on event pages (target 160–320 req/s per instance on the fleet box; it is about 46 today). As the instance gets faster, the gap between "what one client can push" and "what the instance can serve" is the thing protecting everyone else on that instance. That protection belongs at the edge, not in each app. Per our platform-first rule, we are asking for it rather than building a limiter into titogo.

What exists today (checked on origin/main d18e619e)

• internal/edge/edge.go:2765 — CARLOS_BACKHAUL_MAX_INFLIGHT (default 64): a sender-side concurrency cap on cross-region backhaul. It bounds egress bandwidth, not per-client load, and only applies to backhauled requests.
• internal/gate/gate.go:153 — the password gate's 429 on too many password attempts. Not general traffic.
• Nothing per client IP, per host or per instance on the public request path. No 429 for ordinary traffic.

Titogo has only endpoint-specific limiters of its own (sign-in links, SAML, promotion-code guesses, admin-token guesses, address lookup). Those stay; they guard credentials, not capacity.

Open questions for the platform

1. Where does the client IP come from at each hop? It should be the same address the edge already trusts for X-Forwarded-For, and never a header a client can set.
2. Is the limit per edge box, or shared across a region's edges? Per box is probably fine to start, if the defaults allow for it.
3. Should the activator count limited requests as wake signals? Our suggestion: no, a rejected request should never wake a sleeping instance.