carlos / platform Public

docs: rate limits per instance, modes and plan ceilings (#358)

New docs/site/rate-limits.md (approved copy), a plans.md section on the
three ceiling fields, and a nav entry after plans. The carlos ratelimit
usage fence in cli.md gets the text language the fence gate requires.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

claude 14 hours ago 21dc5a182ecd1f5313fdc2a7dbde96312564c9d5 parent 38dc29f

4 files changed, +67 −1

docs/site/cli.md +1 −1
docs/site/nav.json +5 −0
docs/site/plans.md +4 −0
docs/site/rate-limits.md +57 −0

diff --git a/docs/site/cli.md b/docs/site/cli.md

index da7115d7c..7fea3b6c6 100644

--- a/docs/site/cli.md

+++ b/docs/site/cli.md

@@ -1193,7 +1193,7 @@ users; an app with real sign-in keeps doing that itself.

### carlos ratelimit

-```

+```text

usage: carlos ratelimit <show|set|clear> [flags]

show -app <a> [-host <h>]

diff --git a/docs/site/nav.json b/docs/site/nav.json

index 3ead9a5b3..ad5e9cdac 100644

--- a/docs/site/nav.json

+++ b/docs/site/nav.json

@@ -17,6 +17,11 @@

"slug": "plans",

"label": "Plans",

"blurb": "Give accounts limits, packs and sponsorship. Plans stay off until you configure a catalog."

+ },

+ {

+ "slug": "rate-limits",

+ "label": "Rate limits",

+ "blurb": "Stop one visitor taking over an instance, set limits per instance, and roll them out safely."

}

]

}

diff --git a/docs/site/plans.md b/docs/site/plans.md

index 006392789..d1dbd328f 100644

--- a/docs/site/plans.md

+++ b/docs/site/plans.md

@@ -53,3 +53,7 @@ carlos accounts plan --account bac --set free --sponsored limits.json --reason "

```

`limits.json` holds any of the limit names from the catalog. A sponsored limit can go above a ceiling: that is a decision you made for that account.

+## Rate limit ceilings

+A plan can cap how far an account may raise an instance's rate limits, with `rate_client_share`, `rate_per_second` and `rate_burst` in its limits or in `ceilings`. Leave one out and that number has no ceiling. Packs cannot raise them. A `rate_per_second` below 300 or a `rate_burst` below 1000 is refused when the catalog loads, because a plan may never make limits stricter than the platform's defaults. See [Rate limits](rate-limits.md).

diff --git a/docs/site/rate-limits.md b/docs/site/rate-limits.md

new file mode 100644

index 000000000..c09773dc7

--- /dev/null

+++ b/docs/site/rate-limits.md

@@ -0,0 +1,57 @@

+# Rate limits

+The platform stops one visitor from taking over an instance. It does this at the edge, so a refused request never reaches the app and never wakes a sleeping one.

+A quiet instance is never limited. Limits only start when the instance is busy, and then the visitors with the most requests in flight are turned away first. Everyone else carries on as normal, including many people sharing one address, like an office or a phone network.

+## What a refused visitor sees

+A `429 Too Many Requests` answer with a `Retry-After` header and a one-line plain-text body. Browsers and scripts both handle it.

+## The four numbers

+Each instance has four numbers. You only set the ones you want to change; the rest keep the platform's defaults.

+| Setting | Default | What it does |

+| --- | --- | --- |

+| busy at | 64 | The instance counts as busy when this many requests are in flight at once. |

+| share | 16 | While the instance is busy, one address may have this many requests in flight. A request over its share is refused. |

+| rate | 300 | The backstop: requests a second from one address, busy or not. Only a flood reaches it. |

+| burst | 1000 | How far over the rate one address can go in a short burst. |

+An address is one IPv4 address, or one IPv6 /64.

+## Different numbers for one path

+An API is usually called harder than a page is browsed. You can give a path prefix, such as `/api/`, its own share, rate and burst. Requests under that prefix count against those numbers only, so heavy API use does not use up a visitor's allowance for pages. An instance can have up to 8 of these.

+## Setting them

+```sh

+carlos ratelimit show -app shop

+carlos ratelimit set -app shop -host shop.example.com -rate 500 -burst 2000

+carlos ratelimit set -app shop -path /api/:share=32,rate=1000

+carlos ratelimit clear -app shop -host shop.example.com

+```

+Leave out `-host` to change every instance of the app. The box picks up a change within seconds. The four main numbers can also be set on the app's Settings page in the console.

+`show` also prints how many requests each instance turned away in the last 24 hours, split into those turned away while busy and those over the backstop. The first is the limit doing its job. If the second shows up on real customers' traffic, the backstop is too low for them.

+## Plans

+On a deployment with a plan catalog, a plan can set the most an account may raise the share, rate and burst to. See [Plans](plans.md). A self-hosted deployment with no catalog has no such ceiling.

+## For operators: turning it on

+Each box reads `CARLOS_RATE_LIMIT`:

+| Value | What happens |

+| --- | --- |

+| `observe` (the default) | Count and log what would be refused, but refuse nothing. |

+| `enforce` | Refuse. |

+| `off` | Do nothing. |

+Start in `observe`, read the counts for a few days, then switch to `enforce`, dev boxes first.

+Instances on a fleet behind a tunnel are limited on the hub that faces the internet, not on the fleet box.