| index 787043a2d..ef3ec7294 100644 |
| --- a/docs/superpowers/plans/2026-09-28-instance-moves-p7-edge-door.md |
| +++ b/docs/superpowers/plans/2026-09-28-instance-moves-p7-edge-door.md |
| @@ -1,4 +1,4 @@ |
| -# Instance moves P7: the edge-local door — Implementation Plan (rev 4) |
| +# Instance moves P7: the edge-local door — Implementation Plan (rev 5) |
| |
| > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. |
| |
| @@ -6,14 +6,23 @@ |
| |
| **Architecture:** A signed-in person mints a door key at the console. The console's own database is the authority for keys. The bucket holds a signed projection of them, under `control/door-keys/<account>/<app>.json`, signed with the deployment's root release key, carrying a revision and a 72-hour expiry, re-signed every 12 hours, and written by one serialized, compare-and-swap publisher that always rebuilds from the database. Every edge mirrors the projections on their own 30-second loop and verifies them against `CARLOS_RELEASE_PUBKEY`. A second listener on `/run/carlos/edge-local.sock` (0660, group `carlos-sockets` through the directory's setgid) authenticates `X-Carlos-Door-Key`, strips it, removes every forwarded-address header, marks the request as a door caller and hands it to `Edge.ServeHTTP`. For a host this box serves, `ServeHTTP` refuses reserved handlers to door callers and checks ownership against the final upstream it is about to serve. For a host served elsewhere, the door checks the owner the home box published, which a box publishes only for a host whose every route has one owner, and then backhauls it as an ordinary hop. |
| |
| -## Security model (rev 4; Paul's ruling, 2026-09-28) |
| +## Security model (rev 5; Paul's ruling, 2026-09-28) |
| |
| **The door grants no privilege over the public door.** Paul ruled (2026-09-28, after Astra's round 3 showed that a believed address changes the password gate's guess limits and any app that authorizes by IP) that the door believes no caller-supplied visitor address. So: |
| |
| -- The door deletes `Forwarded`, `X-Forwarded-For`, `X-Forwarded-Host` and `X-Forwarded-Proto` from every request (`stripForwardedHeaders`, `homemap.go:806`). |
| -- It sets `RemoteAddr` to `door/<account>/<app>`. That value has no port, so `net.SplitHostPort` fails on it. The gate's rate-limit key (`gate/gate.go:230-235`) and the analytics visitor address (`analytics.go:86-90`) then fall back to the raw string. Gate guesses through the door share **one limit per app per box**, and door traffic counts as **one visitor** per app per box. No gate or analytics code changes. |
| -- `setForwarded` (`homemap.go:912`) gives a door request `X-Forwarded-For: unknown` and `X-Forwarded-Proto: https`, before its hop check. `unknown` is the value RFC 7239 reserves for an undisclosed client. A backhauled door request therefore arrives at the far edge carrying `unknown`. The far edge believes that chain as it believes any verified hop's, so the app sees `unknown` wherever the instance lives. The far edge's TLS makes the scheme `https` there too, so the app sees one visitor address and one scheme whichever box serves it (round 3, finding 3). |
| -- No loopback, private or other real address is ever used. Some apps treat loopback as a local administrator, and the door must not be the way to become one. |
| +- The door deletes every header that claims a client address: `Forwarded`, `X-Forwarded-For`, `X-Forwarded-Host` and `X-Forwarded-Proto` (`stripForwardedHeaders`, `homemap.go:806`), plus `doorAddressHeaders`: `Client-IP`, `X-Real-IP`, `True-Client-IP`, `CF-Connecting-IP`, `Fastly-Client-IP`, `X-Client-IP`, `X-Cluster-Client-IP` and `X-Originating-IP`. Frameworks consult these when the forwarded chain is unusable (Astra round 4: Rails 8's `RemoteIp`). |
| +- It sets `RemoteAddr` to `door/<account>/<app>`. That value has no port, so `net.SplitHostPort` fails on it, and the gate's key (`gate/gate.go:230-235`) and the analytics address (`analytics.go:86-90`) fall back to the raw string. The partitions are what the code does, not a new promise: |
| + - **Served on this box:** the gate's limiter is keyed by (route host, `door/<account>/<app>`) (`gate.go:151`). Analytics hashes (host, User-Agent, `door/<account>/<app>`) (`internal/analytics/meter.go:178`). |
| + - **Backhauled:** the far gate keys on the sending edge's address, as every backhauled request does today. Far analytics hashes (host, User-Agent, `192.0.0.8`). |
| + |
| + No gate or analytics code changes. |
| +- `setForwarded` (`homemap.go:912`) gives a door request `X-Forwarded-For: 192.0.0.8` and `X-Forwarded-Proto: https`, before its hop check. **192.0.0.8 is the IPv4 dummy address (RFC 7600)**, reserved for exactly this: a source that has no real address to give. |
| + - It is a valid IP, so no framework discards the chain and falls back to an alternate header or to the TCP peer. For apps behind the edge's loopback TCP proxy (`edge.go:2182`), that peer would be loopback, which Rails treats as `local?` (Astra round 4, finding 1). |
| + - It is not loopback, not private (`net.IP.IsPrivate`), and not in Rails' default trusted-proxy set. A framework takes it as the client and runs its usual spoofing checks against it. |
| + - A backhauled door request reaches the far edge carrying `192.0.0.8`. The far edge believes that chain as it believes any verified hop's, so the app sees `192.0.0.8` wherever the instance lives. The far edge's TLS makes the scheme `https` there too (round 3, finding 3). |
| +- **Degraded mode, stated:** if the far edge cannot verify the sender's mesh key (the rotation window), it strips the forwarded headers and serves the request as a client at the sending edge's address (`homemap.go:793`), exactly as it does for a public backhaul in the same failure. That address is a platform box's public EIP, never loopback or private, so it confers no locality. The app sees an infrastructure address instead of `192.0.0.8` until the key mirror catches up (Astra round 4, finding 3). |
| +- **Door requests bypass the response cache:** no lookup, no store. The cache keys `Vary` on the inbound request (`edge.go:1583`, `:1702`), which the door has rewritten, so a door response could otherwise be served to a public request or the reverse (Astra round 4, finding 2). The same inbound-versus-outbound mismatch exists for public traffic on `Vary: X-Forwarded-For`, and is filed separately rather than fixed here. |
| +- No loopback or private address is ever used. Some apps treat loopback as a local administrator, and the door must not be the way to become one. |
| - An app that needs the buyer's address has its own router send it in the app's own signed header, as Tito already does for identity (`X-Tito-Auth-*`). Headers the app sets pass through untouched. The platform reads none of them. |
| |
| **What the door key is for, then.** It says which app's processes are calling, and confines them to that app's hosts (ruling Q3). With no privilege behind the door, a gap in that confinement lets a caller reach a host through the door that it could already reach from the internet, with less than an internet request carries. That is why the residuals below are accepted. They are hygiene, not security boundaries: |
| @@ -46,6 +55,15 @@ The credential machinery (signed, revisioned, expiring records; one serialized p |
| | 9 | wrong file and fixture references | **Accepted:** `runInstances` is in `cli.go:284`; fixture is `fx.ts`/`fx.pub`, helpers in `apiinstances_test.go`; catchall publisher `edgesfile.go:363` and the alias-versus-route fold covered; docs go to `docs/site/cli.md` (the only reference page). | |
| | 10 | root signing acceptable; add a domain prefix | **Accepted:** signatures cover `"carlos door-keys v1\n" + payload`; cross-kind rejection tests (Task 1). | |
| |
| +## Revision 5: Astra round 4 dispositions |
| + |
| +| # | finding | disposition | |
| +|---|---|---| |
| +| R4-1 | `unknown` lets frameworks fall back to `Client-IP` or loopback | **Accepted:** the address is `192.0.0.8` (RFC 7600), and the door strips every alternate client-address header (Security model; Task 6 tests with a Rails-rule resolver). | |
| +| R4-2 | the cache varies on inbound headers the door rewrote | **Accepted:** door requests bypass the cache (Task 5); the public-traffic mismatch is filed separately. | |
| +| R4-3 | failed hop verification substitutes an infrastructure address | **Accepted as a stated degraded mode** (Security model): it is the same outcome public backhaul has in that failure, and a box's public EIP confers nothing. | |
| +| R4-4 | gate and visitor partitions misstated | **Accepted:** the Security model states the actual keys. | |
| + |
| ## Revision 4: Astra round 3 dispositions |
| |
| | # | finding | disposition | |
| @@ -97,7 +115,7 @@ The credential machinery (signed, revisioned, expiring records; one serialized p |
| 2. **A path-prefix hostname whose prefixes belong to different apps**: locally, app A's key never reaches app B's prefix; remotely, the host publishes no owner and the door refuses it (Tasks 4 and 5 tests). |
| 3. **A gate login POST whose `next` names another app's prefix** is refused before the gate judges the password (Task 5 test). |
| 4. **A revoke racing a renewal** never leaves the older record in the bucket (Task 2 test). |
| -5. **Any `Forwarded` or `X-Forwarded-*` header a caller sends** never reaches the app, the gate or analytics. The app sees `X-Forwarded-For: unknown` and `X-Forwarded-Proto: https`, served on this box or across a hop (Tasks 5 and 6 tests). |
| +5. **Any `Forwarded` or `X-Forwarded-*` header a caller sends** never reaches the app, the gate or analytics. The app sees `X-Forwarded-For: 192.0.0.8` and `X-Forwarded-Proto: https`, served on this box or across a hop (Tasks 5 and 6 tests). |
| |
| --- |
| |
| @@ -640,16 +658,19 @@ Mirror rules, each with its own test: |
| - `func (e *Edge) backhaulTo(w http.ResponseWriter, r *http.Request, host, eip, box string) bool`: the dial half of `backhaul`, split out so the door dials exactly the target `Owner` returned. `backhaul` becomes `Home` plus `backhaulTo`, and its behaviour is unchanged. |
| - `func doorOwnsRoute(c doorCaller, rt registry.Route) bool` → `release.RouteAccount(rt) == c.Account && rt.App == c.App` |
| - `func (e *Edge) doorReservedHost(host string) bool` — true for the tunnel hostname and the steering probe hostname, compared after lowercasing and stripping one trailing dot (the normalization `steeringProbe` and `canonicalizeHopHost` use) |
| +- `const doorVisitorAddr = "192.0.0.8"` (RFC 7600, the IPv4 dummy address) |
| +- `var doorAddressHeaders = []string{"Client-IP", "X-Real-IP", "True-Client-IP", "CF-Connecting-IP", "Fastly-Client-IP", "X-Client-IP", "X-Cluster-Client-IP", "X-Originating-IP"}` |
| - `setForwarded` gains a first branch: |
| |
| ```go |
| func setForwarded(pr *httputil.ProxyRequest) { |
| pr.SetXForwarded() |
| - // A door request discloses no visitor (Security model): one fixed |
| - // address and the scheme the far side of a hop would give it, so the |
| - // app sees the same thing whichever box serves it. |
| + // A door request discloses no visitor (Security model): RFC 7600's |
| + // dummy address, a valid IP no framework treats as local, and the |
| + // scheme the far side of a hop would give it, so the app sees the same |
| + // thing whichever box serves it. |
| if _, ok := doorCallerFrom(pr.In.Context()); ok { |
| - pr.Out.Header.Set("X-Forwarded-For", "unknown") |
| + pr.Out.Header.Set("X-Forwarded-For", doorVisitorAddr) |
| pr.Out.Header.Set("X-Forwarded-Proto", "https") |
| return |
| } |
| @@ -664,8 +685,9 @@ func setForwarded(pr *httputil.ProxyRequest) { |
| |
| Enforcement, in `ServeHTTP` order: |
| 1. **First statement after the port strip:** `door, isDoor := doorCallerFrom(r.Context())`. If `isDoor && e.doorReservedHost(host)`: `e.unknownHost(w, r); return`. If `isDoor`, skip the steering probe and tunnel branches entirely (wrap them in `if !isDoor { ... }`). The mark is only ever set by `serveDoor` on this box. Nothing carries it across a hop, and `acceptHop` is unchanged. |
| -2. **Miss branch:** if `isDoor`, call `acct, app, eip, box, ok := e.homes.Load().Owner(host)`. Unless `ok && acct == door.Account && app == door.App`, answer `unknownHost`. Otherwise `backhaulTo(w, r, host, eip, box)`. The receiving edge sees an ordinary verified hop, as it does for public backhaul today. It believes the forwarded chain, which is `unknown`, so the app there sees exactly what a local serve shows it (Security model). |
| -3. **After the final `up` is chosen**, that is, after the login POST re-resolution block (~`edge.go:1416-1446`) and before the gate (`:1516`), the cache and dispatch: `if isDoor && !doorOwnsRoute(door, up.route) { e.unknownHost(w, r); return }`. The same `up` value is then used for everything after it, so authorization and dispatch cannot diverge. |
| +2. **Miss branch:** if `isDoor`, call `acct, app, eip, box, ok := e.homes.Load().Owner(host)`. Unless `ok && acct == door.Account && app == door.App`, answer `unknownHost`. Otherwise `backhaulTo(w, r, host, eip, box)`. The receiving edge sees an ordinary verified hop, as it does for public backhaul today. It believes the forwarded chain, which is `192.0.0.8`, so the app there sees exactly what a local serve shows it (Security model). |
| +3. **Cache:** wherever `ServeHTTP` consults or fills the response cache (`edge.go:1583` lookup, `:1702` store), add `!isDoor` to the condition, so a door request neither reads nor writes it. |
| +4. **After the final `up` is chosen**, that is, after the login POST re-resolution block (~`edge.go:1416-1446`) and before the gate (`:1516`), the cache and dispatch: `if isDoor && !doorOwnsRoute(door, up.route) { e.unknownHost(w, r); return }`. The same `up` value is then used for everything after it, so authorization and dispatch cannot diverge. |
| |
| - [ ] **Step 1: Failing tests** (an in-process `Edge` built the way existing `edge_test.go` tests build one with local routes and a stubbed home map; `grep -n "func newTestEdge\|func testEdge" internal/edge/*_test.go`): |
| 1. door caller, local route of its app → served; |
| @@ -709,6 +731,9 @@ func (e *Edge) serveDoor(w http.ResponseWriter, r *http.Request) { |
| // header goes, and RemoteAddr names the calling app without a port, so |
| // the gate's limiter and the visitor hash key on it as one client. |
| stripForwardedHeaders(r) |
| + for _, h := range doorAddressHeaders { |
| + r.Header.Del(h) |
| + } |
| r.RemoteAddr = "door/" + tok.Account + "/" + tok.App |
| e.ServeHTTP(w, r.WithContext(context.WithValue(r.Context(), doorCallerKey{}, doorCaller{Account: tok.Account, App: tok.App}))) |
| } |
| @@ -725,9 +750,12 @@ func doorRefused(w http.ResponseWriter) { |
| 1. no key, malformed key, revoked key → 401 with `Cache-Control: no-store`, upstream never called; |
| 2. valid key → served; the upstream sees no `X-Carlos-Door-Key` and sees `X-Tito-Auth-User` untouched; |
| 3. inbound `X-Carlos-Hop`, `X-Carlos-Hop-Auth` → neither reaches the upstream, and a miss goes to the door's owner check, never `staleHopRefusal`; |
| - 4. forwarded headers: a request with `X-Forwarded-For: 203.0.113.9`, `Forwarded: for=198.51.100.7` and `X-Forwarded-Proto: http` reaches the upstream with exactly `X-Forwarded-For: unknown` and `X-Forwarded-Proto: https` and no `Forwarded`. The same holds with none of them sent. `clientIP` returns `door/bab/a`. The gate stub's `clientKey` sees `door/bab/a` for two requests that claimed different addresses; |
| + 4. forwarded headers: a request with `X-Forwarded-For: 203.0.113.9`, `Forwarded: for=198.51.100.7` and `X-Forwarded-Proto: http` reaches the upstream with exactly `X-Forwarded-For: 192.0.0.8` and `X-Forwarded-Proto: https` and no `Forwarded`. The same holds with none of them sent. `clientIP` returns `door/bab/a`. The gate stub's `clientKey` sees `door/bab/a` for two requests that claimed different addresses; |
| + 4b. alternate address headers: a request carrying every name in `doorAddressHeaders` reaches the upstream with none of them; |
| + 4c. **a Rails-rule resolver in the upstream**, written in the test to Rails 8.0.2's `RemoteIp#calculate_ip` rules (drop non-IP XFF entries; strip trusted proxies: loopback, `fc00::/7`, 10/8, 172.16/12, 192.168/16; if nothing is left, prefer `Client-IP`, else the peer), fed the upstream's `X-Forwarded-For`, `Client-IP` and a loopback peer, resolves `192.0.0.8` for a door request, with and without an inbound `Client-IP`, served locally and backhauled. It never resolves loopback or a caller-chosen address; |
| 5. a gated route still answers with the gate's page; |
| - 5b. a backhauled door request: using two in-process edges joined as the existing backhaul tests join them (`grep -n "func.*TwoEdges\|backhaul.*httptest" internal/edge/*_test.go`), the far upstream sees `X-Forwarded-For: unknown` and `X-Forwarded-Proto: https`, the same as case 4; |
| + 5a. cache: with a cacheable upstream response, a door request neither populates the cache (a following public request goes to the upstream) nor is served from it (a public request's cached response is not returned to a following door request); |
| + 5b. a backhauled door request: using two in-process edges joined as the existing backhaul tests join them (`grep -n "func.*TwoEdges\|backhaul.*httptest" internal/edge/*_test.go`), the far upstream sees `X-Forwarded-For: 192.0.0.8` and `X-Forwarded-Proto: https`, the same as case 4; |
| 6. `openDoor` on a temp path makes a 0660 socket; a real unix-socket client round-trips case 2; `closeDoor` removes the socket; `openDoor` with a nil mirror returns an error and creates nothing. |
| - [ ] **Step 2–4:** run, implement, run. |
| - [ ] **Step 5: Commit** `edge: the edge-local door (#175 P7)`. |
amadan