amadan
spec: fleet status page — public commitments, operator role, bucket-backed serve
Brainstormed with Paul 2026-08-01: fleet/ops scope only, opt-in tenant naming, console-private with an operator toggle to public, published by a lease-holding box and served bucket-backed so the page outlives the deployment it reports on.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 file changed,
+185
−0
docs/superpowers/specs/2026-08-01-fleet-status-page-design.md+185 −0
diff --git a/docs/superpowers/specs/2026-08-01-fleet-status-page-design.md b/docs/superpowers/specs/2026-08-01-fleet-status-page-design.md| new file mode 100644 |
| index 000000000..2f61eaf3a |
| --- /dev/null |
| +++ b/docs/superpowers/specs/2026-08-01-fleet-status-page-design.md |
| @@ -0,0 +1,185 @@ |
| +# Fleet status page — design |
| + |
| +Every CARLOS deployment computes a fleet status view; the operator chooses |
| +whether it is public. The flagship's is public, at |
| +`status.carlosframework.com`, holding the same public commitments as |
| +Eleven's status pages (`status.elevenmessenger.com`): aggregates only, |
| +band words instead of needles, loud unknowns, and nothing published that |
| +wasn't deliberately chosen. |
| + |
| +Scope decision: **fleet/ops only** — the CARLOS analogue of Eleven's |
| +`fleet.json`/`index.html` pair. Eleven's other dashboards (books, carbon, |
| +prompts, contributors, security, SSH ledger, …) are that project's ledgers, |
| +not platform features; any of them can layer on later as separate specs. |
| + |
| +## 1. Access & audiences |
| + |
| +Three audiences, two of them existing: |
| + |
| +- **Operators** — a new console role. Identity is the console's existing |
| + Keymail OAuth (an operator is a person with a keymail address; no second |
| + identity system). Authorization is an `operator` flag on the user, |
| + granted **only** via the box-side CLI: `console grant -operator |
| + <address>`. The root of trust stays what it really is — possession of |
| + the box / the AWS account — and the console role is a projection of it |
| + into the UI, exactly like owner grants today. There is no self-service |
| + path to operator. The role gates: the private fleet-status view, the |
| + all-apps/all-routes view, and future operator surfaces. It deliberately |
| + does **not** replace the deeper credentials: shipping still requires the |
| + release signing key, infra still requires AWS — the role can never do |
| + more than the console process itself can. |
| +- **App owners** — unchanged; they see their own apps' pages only. |
| +- **The public** — only when the deployment opts in. `STATUS_PUBLIC=1` |
| + publishes the static page at `STATUS_HOST`; private is real access |
| + control (console auth + operator role), never an unlisted URL. |
| + |
| +A self-hosted deployment authenticating its console against keymail.dev is |
| +trusting a third-party identity provider; that's each operator's choice to |
| +make, stated plainly in the docs (Keymail itself is self-hostable, which |
| +keeps the choice sovereign). |
| + |
| +## 2. Data model & the privacy boundary |
| + |
| +Two bucket artifacts, both written by machinery that never lets a needle |
| +out. Every commitment below is enforced in the *writer* — ratios, caps, |
| +hostnames, and machine ids are read internally and die in the process that |
| +banded or counted them. |
| + |
| +### Per-box snapshots — `status/boxes/<box>.json` |
| + |
| +Written by each box's agent on a 5-minute timer: |
| + |
| +```json |
| +{ |
| + "generated_at": "2026-08-01T18:00:00Z", |
| + "label": "ada-stargazing", |
| + "region": "EU · Ireland", |
| + "routes": { "static": 2, "resident": 3, "hibernating": 4, |
| + "awake": 1, "asleep": 3 }, |
| + "load": "comfortable", |
| + "capacity": "plenty", |
| + "replication": { "healthy": true, "lag_seconds": 42 } |
| +} |
| +``` |
| + |
| +- **Label**: whimsical name-action pair hashed from the machine id — |
| + stable per box; never the hostname, IP, or instance id. |
| +- **Region**: operator-set, coarse by design — continent + country from |
| + the deployment's region registry, never a city or datacenter; unset is |
| + omitted, never guessed. |
| +- **`awake`/`asleep`** split the hibernating set by whether each tenant's |
| + socket is bound and listening, read side-effect-free from the kernel |
| + socket table (`ss -xlH` equivalent) — dialing would wake the tenant. |
| + Unreadable → both `null`. Unknowns publish as null, never as guesses. |
| +- **`load` / `capacity`**: band words only — `comfortable`/`busy`/ |
| + `critical` and `plenty`/`getting-full`/`at-capacity`, banded at 70% and |
| + 90%, computed as the worst of disk-floor proximity and memory |
| + utilization. The ratios are never serialized. |
| +- **`replication`**: seconds since litestream last progressed; `null` |
| + when unknown. |
| + |
| +### The merged feed — `status/fleet.json` + `status/index.html` |
| + |
| +Rendered from the snapshots by whichever box holds the status lease |
| +(reusing `internal/lease` on `status/lock.json` — no new election |
| +machinery; any box can render, exactly one does): |
| + |
| +```json |
| +{ |
| + "generated_at": "2026-08-01T18:05:00Z", |
| + "boxes": [ { "label": "ada-stargazing", "region": "EU · Ireland", |
| + "load": "comfortable", "stale": false, |
| + "routes": { "static": 2, "resident": 3, "hibernating": 4, |
| + "awake": 1, "asleep": 3 } } ], |
| + "instances": { "total": 9, "static": 2, "resident": 3, |
| + "hibernating": 4, "awake": 1, "asleep": 3 }, |
| + "capacity": { "level": "plenty" }, |
| + "apps": [ { "name": "kass.training", "instances": 1, "healthy": true, |
| + "channel": "stable", "version": "aedcae3" } ], |
| + "apps_unlisted": 5, |
| + "services": [ { "name": "svc/pdf", "up": true } ], |
| + "replication": { "healthy": true, "lag_seconds": 42 } |
| +} |
| +``` |
| + |
| +- **Apps are opt-in by name.** Default is Eleven-strict: no app names, no |
| + hostnames. The opt-in is an additive `status_named` column on the route |
| + (set via `carlos add -status-named`, and later from the manifest when |
| + manifests land) — the app's own choice, recorded where the publisher |
| + already reads. An opted-in app appears as name + instance count + |
| + health + channel + version-short-sha. Everyone else folds into `apps_unlisted` — one |
| + honest count, so the aggregate never lies by omission. |
| +- **`capacity.level`** is the worst box's band. |
| +- **`services`**: each `svc/*` route's up/down from its health check. |
| +- **Staleness is loud**: a box whose snapshot has missed 3 consecutive |
| + periods renders as stale, never silently dropped. |
| + |
| +### The rule for new fields |
| + |
| +A field must be a fleet-wide aggregate, a band word, or behind the app's |
| +own opt-in — or it doesn't ship. The dry-run self-check and the contract |
| +test both fail if a fixture hostname or a digit inside the |
| +capacity/load subtrees reaches the output. |
| + |
| +## 3. Publish & serve |
| + |
| +The viewer rides the binary, the data rides the bucket, the edge joins |
| +them: |
| + |
| +- **Page assets** (`index.html`, one stylesheet; no JS frameworks, no |
| + external requests — same-origin only) are embedded in the carlos binary |
| + like the systemd units are. The lease-holding publisher writes them to |
| + `status/` beside `fleet.json` each pass. Page changes deploy with the |
| + binary; data changes deploy with the timer. |
| +- **No release churn.** Publishing every 5 minutes must not mint CAS |
| + releases. The status route is a small `kind=status` route the edge |
| + serves **bucket-backed** from the `status/` prefix with a short (60s) |
| + cache, reusing the static handler's plumbing — not channel-pointer |
| + tars. |
| +- **Config**: `STATUS_HOST` + `STATUS_PUBLIC` in the deployment's host |
| + config. Public = the route exists (normal ACME via the route table) |
| + *and* the `status/` prefix is public-read. Private = neither; the |
| + console renders the same `fleet.json` for operators, and that is the |
| + only window. A box with no status config publishes nothing (the |
| + litestream convention: install everywhere, inert until configured). |
| +- **Incident fallback** — the reason this architecture won: on a public |
| + deployment the raw bucket URL serves the identical page even when every |
| + box is down; the page footer links it as "if this page is unreachable". |
| + The status page outlives the deployment it reports on. |
| +- **Honest limit**: a fully-private deployment whose console is down has |
| + no status page — the operator falls back to `aws s3 cp` of |
| + `fleet.json`; documented in the runbook, not papered over. |
| +- **Flagship**: `status.carlosframework.com`, public. One A record in the |
| + carlosframework.com zone (Paul Campbell DNSimple account), bucket |
| + policy for the public prefix in platform-infrastructure, and |
| + `console grant -operator paul@keymail.dev`. |
| + |
| +## 4. Testing & guard rails |
| + |
| +All build-failing, ported from Eleven's pattern: |
| + |
| +- **Allowlisted schemas**: a contract test pins every key `fleet.json` |
| + and the snapshots may contain — an extra field fails; the |
| + capacity/load subtrees additionally fail on any digit. |
| +- **Leak tests**: the writer runs against a fixture registry full of |
| + hostnames and machine ids; none may survive into output except apps |
| + explicitly opted in; unknowns must surface as `null` (an `ss` failure |
| + must never render as `awake: 0`). |
| +- **Banding tests** at the 70/90 boundaries; worst-of-boxes capacity; |
| + stale-after-3; `apps_unlisted` always equals total minus named. |
| +- **`carlos status -dry-run`**: builds the whole page from fixtures with |
| + no box state, network, or upload — the test surface and the |
| + self-hoster's "what would I publish?" preview in one verb. |
| +- **e2e**: the edge serves a `kind=status` route bucket-backed; the |
| + console shows the fleet view to an operator and 404s a mere owner. |
| + |
| +## 5. Rollout |
| + |
| +1. Platform repo: `internal/status` (writer + publisher), the agent |
| + timer, edge `kind=status`, console operator role + fleet view. |
| + Code-only, green before any deploy. |
| +2. Ship/adopt the binary on both deployments — inert until configured. |
| +3. Flagship: public config, bucket-policy tofu, DNS A record, operator |
| + grant, verify page + fallback URL. |
| +4. Tito: private by default, with the console-less fallback noted; the |
| + operator can flip it public or deploy a console later. |