| 1 | # Census: console invite delivery |
| 2 | |
| 3 | Status: RULED as census-ruling-console-invite-delivery (platform-1e, R1–R5), |
| 4 | on 5aef01db. The ruling is |
| 5 | `.superpowers/sdd/2026-10-02-console-invite-delivery/census-ruling.md`. |
| 6 | The overlay is loaded last in `TestPlan14RootReadCensus`, its rows carry |
| 7 | `rev3=census-ruling-console-invite-delivery`, and the seal is moved to |
| 8 | `fcdf9de0…d7a3`. platform-1e rules as the Plan 14 census owner. The |
| 9 | proposal below is kept as it was put, with the PROPOSED wording brought up |
| 10 | to date where the ruling settled it. |
| 11 | |
| 12 | ## The ruling |
| 13 | |
| 14 | - **R1. Accepted.** The seven units (1 to 7 below) are accepted legacy under |
| 15 | the identities named: `serve`, `NewServer`, `Keymail.do`, |
| 16 | `Keymail.sealEnvelope` and `Store.CountPendingInvites` run in the console |
| 17 | (cmd/console serve under console.service, user carlos-console); the two |
| 18 | region-host files are run by tofu on an operator's machine. The basis is |
| 19 | that the root `carlos` binary links neither `internal/console` nor |
| 20 | `internal/mail`. No check pinned that before this ruling, so one is added: |
| 21 | `TestRootBinaryLinksNoConsoleOrMail` |
| 22 | (`internal/provision/root_binary_deps_test.go:18`) runs |
| 23 | `go list -deps .` for linux, darwin and windows and fails if either |
| 24 | package, or anything under it, appears. It skips when `go` is not on |
| 25 | PATH. This ruling is the separate scope ruling the legacy freeze needs, |
| 26 | so the seal moves. |
| 27 | - **R2. Accepted.** `Keymail.SendApp`'s read row is retired by an H3 |
| 28 | census-ruling removal, in the form census ruling carlos-run used for |
| 29 | `apiSession.getJSON` (`plan14-root-reads-carlos-run.tsv:126-127`). Its |
| 30 | one read, `crypto/rand.Read` for the envelope id, moved unchanged into |
| 31 | `sealEnvelope`, the same class. |
| 32 | - **R3. Out of census scope.** The `ses:SendEmail` grant goes on the shared |
| 33 | host role, which every process holding the box's instance credentials can |
| 34 | use. Merging is fine, because the policy has count 0 by default. It needs |
| 35 | Paul's own security ruling before anyone sets `console_mail_from`. The |
| 36 | rollout gate, verbatim: "Paul approves the host-role SES grant (or a |
| 37 | console-only role) before console_mail_from is set". Whether tenant code |
| 38 | can reach the instance credentials is answered under "R3: who can reach |
| 39 | the instance credentials" below. |
| 40 | - **R4. Rewritten.** The `PROPOSED-console-invite-delivery` marker is now |
| 41 | `census-ruling-console-invite-delivery` in every row, as box-publish-record |
| 42 | and the step 6 overlays did, and the overlay header names the ruling. The |
| 43 | `@legacy` fingerprints were recomputed for the new basis text. The seal, |
| 44 | recomputed with the overlay loaded after box-publish-record, is |
| 45 | `fcdf9de0ba39db40d6bc7d1d2137d4f40d8248df8d286a380b2eae4e8b31d7a3`, the |
| 46 | value the drafter computed for this form. |
| 47 | - **R5. Not a merge blocker** (HostClaim is off, and ruling 6 holds). It is a |
| 48 | claim-activation blocker for any console that sends mail, filed on #250. |
| 49 | With `CARLOS_HOST_CLAIM` on, the console mailer's `EnsureDomain` and |
| 50 | `VerifyDomain` must go through the claim mail gate: either console-mail is |
| 51 | registered as a platform identity the admitter knows, or the mailer refuses |
| 52 | to start when HostClaim is on. Until then a console under claims must not |
| 53 | have console mail enabled. |
| 54 | - Re-digest after 0da9b965 (HostClaim guard passes hostClaim into |
| 55 | consoleMailer; no new read site): serve and cmd/console envelope |
| 56 | re-digested, seal fcdf9de0 → 53b3cbb6, awaiting platform-1e's |
| 57 | confirmation. |
| 58 | |
| 59 | ## R3: who can reach the instance credentials |
| 60 | |
| 61 | The question is whether tenant code on a box can reach the EC2 instance |
| 62 | metadata service (IMDS, 169.254.169.254) and so the host role's |
| 63 | credentials. If it can, it can send as `console_mail_from` once the grant is |
| 64 | on. |
| 65 | |
| 66 | What holds today, from this repo: |
| 67 | |
| 68 | - **Exec-backed instances in the flat layout can reach IMDS.** The flat |
| 69 | layout is the default: `ExecBackend.Start` runs every row that is not |
| 70 | nested, fenced or in conflict as an exec child of the agent |
| 71 | (`internal/activator/backend_exec.go:1829-1837`; flat is "today's exec |
| 72 | child", `internal/activator/layout.go:15-17`). Those children run as |
| 73 | uid carlos inside `carlos-edge.service`'s delegated cgroup |
| 74 | (`internal/host/units/carlos-edge.service:28` `User=carlos`, `:64` "The |
| 75 | agent and its flat children", `:95-101` `Delegate=`). That unit has no |
| 76 | `IPAddressDeny=` line, because the agent itself reads IMDS |
| 77 | (`internal/host/agent.go:2964`, `internal/host/memwatch.go:186-189`). The |
| 78 | only other fence is the nftables skuid gate, and it lets uid carlos |
| 79 | through by construction (`memwatch.go:189`). |
| 80 | - **Exec-backed instances in the nested layout (#126) cannot.** They are |
| 81 | still uid carlos, not a separate uid |
| 82 | (`internal/host/units/carlos-instance@.service:4` and `:28`), but each |
| 83 | template carries a unit-level `IPAddressDeny=` that covers |
| 84 | 169.254.169.254/32 and fd00:ec2::254/128 |
| 85 | (`carlos-instance@.service:142`, `carlos-instance-sidecar@.service:142`, |
| 86 | `carlos-instance-cmd@.service:133`, `carlos-instance-gen@.service:115`, |
| 87 | `carlos-instance-seed@.service:143`). systemd applies it as a cgroup BPF |
| 88 | firewall, so it holds whatever the nftables rules say. |
| 89 | - **Unit-backed instances (DynamicUser) cannot, unless they hold |
| 90 | `host_role`.** `carlos-app@.service:40` is `DynamicUser=yes` and `:69` is |
| 91 | `IPAddressDeny=169.254.169.254/32 fd00:ec2::254/128`. On a hardened box |
| 92 | the skuid gate also refuses dynamic uids (`carlos-app@.service:9-15`). A |
| 93 | tenant granted `host_role` gets a drop-in with |
| 94 | `IPAddressAllow=169.254.169.254/32 fd00:ec2::254/128`, which wins over |
| 95 | the deny (`internal/caps/render.go:252-259`), and then only the skuid gate |
| 96 | stands between it and the role. |
| 97 | |
| 98 | What depends on settings this repo does not hold: |
| 99 | |
| 100 | - **The skuid gate.** It is written by the infra repo's |
| 101 | `scripts/harden-host.sh`, not by anything here |
| 102 | (`internal/host/adopt_advice.go:204-207`). A box without it has no |
| 103 | nftables fence at all (`adopt_advice.go:156`; tito, issue #351, per |
| 104 | `carlos-app@.service:10-11`). On such a box a `host_role` unit tenant |
| 105 | reaches IMDS too. |
| 106 | - **IMDS options.** `aws_instance.host` |
| 107 | (`infra/modules/region-host/main.tf:482-511`) sets no `metadata_options`, |
| 108 | so `http_tokens` and `http_put_response_hop_limit` take the AMI's and the |
| 109 | account's defaults (the scanner host sets `http_tokens = "required"` and a |
| 110 | hop limit of 1, `infra/modules/scanner-host/main.tf:126-129`; the region |
| 111 | host does not). Neither setting would fence these tenants anyway. They |
| 112 | share the host's network namespace, so a hop limit of 1 reaches them, and |
| 113 | any local process can do the IMDSv2 token PUT. |
| 114 | - **Which layout a box's rows are in.** A row is nested only after plan 4's |
| 115 | migration moves it. Until then, on any box that runs flat exec-backed |
| 116 | tenants, including the console's box if it hosts them, tenant code can |
| 117 | fetch the host role's credentials and, with the grant on, send as |
| 118 | `console_mail_from`. A console-only role, or a skuid gate that separates |
| 119 | the console's uid from carlos, would close that. |
| 120 | |
| 121 | Branch `design-console-invite-delivery`, head 05cfbec0, with origin/main |
| 122 | merged in at c80e2f3b. Design: |
| 123 | `docs/superpowers/specs/2026-09-28-console-invite-delivery-design.md`. |
| 124 | |
| 125 | ## What changed |
| 126 | |
| 127 | An owner's invite goes out by Keymail federation first (into the invitee's |
| 128 | main inbox, sent as the console, no grant needed). If Keymail can't deliver, |
| 129 | it goes by the console's own email. If that fails too, the owner gets the link |
| 130 | to pass on. Two limits refuse an invite before its row exists: 20 waiting per |
| 131 | account, and 20 an hour. |
| 132 | |
| 133 | What the root-read census sees: |
| 134 | |
| 135 | - `internal/console/keymail.go`: `Keymail.do` returns a typed |
| 136 | `keymailError` for a non-200 answer. SendApp's envelope sealing moves, |
| 137 | unchanged, into a new `Keymail.sealEnvelope`, which `SendApp` and the new |
| 138 | `SendFederated` (keymailfed.go) share. |
| 139 | - `internal/console/server.go`: `NewServer` sets the invite delivery fields. |
| 140 | `handleInvite` is rewritten, but it has no read site of its own. |
| 141 | - `internal/console/store.go`: new `Store.CountPendingInvites`. |
| 142 | - `cmd/console/main.go`: `serve()` builds the console mailer |
| 143 | (`consoleMailer`) and starts its identity ensure in a goroutine. |
| 144 | - `infra/modules/region-host`: an optional `ses:SendEmail` grant on the host |
| 145 | role (`main.tf`), and the two variables that switch it on (`variables.tf`). |
| 146 | - New files with no read site: `internal/console/consolemail.go`, |
| 147 | `keymailfed.go`, `keymailpow.go`, `internal/mail/consolemail.go`, |
| 148 | `internal/mail/sessend.go`. `internal/mail/walk.go` gains the console-mail |
| 149 | teardown guards. |
| 150 | |
| 151 | None of this runs as root. The root `carlos` binary links neither |
| 152 | `internal/console` nor `internal/mail`: `go list -deps .` names neither, and |
| 153 | the only importers of `internal/mail` are `cmd/console` and |
| 154 | `internal/console`. `TestRootBinaryLinksNoConsoleOrMail` pins that (R1). The `.tf` files are run by tofu on an operator's machine. |
| 155 | No binary embeds them and no box reads them. |
| 156 | |
| 157 | ## Root-read census (TestPlan14RootReadCensus) |
| 158 | |
| 159 | Overlay `internal/provision/testdata/plan14-root-reads-console-invite-delivery.tsv`, |
| 160 | to load last, after `plan14-root-reads-box-publish-record.tsv`. |
| 161 | |
| 162 | After the merge the test reports 18 discrepancies. With the overlay loaded, |
| 163 | 17 clear and one is left, "changed legacy freeze; separate scope ruling |
| 164 | required", which is the seal. With the ruled seal set as well, every |
| 165 | `TestPlan14*` test passes. |
| 166 | |
| 167 | Every row is accepted-legacy and tagged `rev3=census-ruling-console-invite-delivery` (R4). |
| 168 | Each basis names the executing identity, in the form census rulings carlos-run |
| 169 | and seeded-instances use for console rows ("the console (cmd/console serve |
| 170 | under console.service, user carlos-console): ..."). |
| 171 | |
| 172 | ### Changed units, re-frozen with their read sites unchanged |
| 173 | |
| 174 | Each of these was a "changed root read" and a "changed legacy declaration". |
| 175 | |
| 176 | 1. **`cmd/console/main.go:serve`**: data-literal@3446,4547,4784, digest |
| 177 | `a9ecdd3b…9825`. Main froze it under seeded-instances at |
| 178 | @3331,4432,4669. The same three literals moved by 115 bytes, which is the |
| 179 | one inserted line at main.go:297 (`consoleMail, ensureConsoleMail := |
| 180 | consoleMailer(...)`). The literals are the `CARLOS_REGISTRY` default |
| 181 | `/data/registry.db` (main.go:333) and the `CONSOLE_DB` default |
| 182 | `/data/console/console.db`, used twice (main.go:352 and 355). The other |
| 183 | changes are `ConsoleMail: consoleMail` in Config, and a goroutine bounded at |
| 184 | two minutes that runs `ensureConsoleMail` (main.go:457). The mailer is built |
| 185 | over the release store, provisioner and DNS writer that serve already holds. |
| 186 | `consoleMailer` itself (main.go:995) has no read site. |
| 187 | **Proposed:** accepted-legacy, the console. |
| 188 | |
| 189 | 2. **`internal/console/server.go:NewServer`**: data-literal@9108, digest |
| 190 | `4d82a686…b032`. Main had it at @8926. The literal is the route pattern |
| 191 | `"GET /a/{sqid}/apps/{app}/logs/data"` (server.go:1416), not a path. The |
| 192 | change sets `consoleMail`, `inviteTimeout` and `inviteKeymailShare` |
| 193 | (server.go:1309), and binds `sendFederated` beside `sendApp` |
| 194 | (server.go:1329). No goroutine is started. The record-write census |
| 195 | re-digested the same unit in 05cfbec0. |
| 196 | **Proposed:** accepted-legacy, the console. |
| 197 | |
| 198 | 3. **`internal/console/keymail.go:Keymail.do`**: file:ReadAll@594, which is |
| 199 | the same site as on main. Only the digest moved, to `e2f193e4…e7be`. The |
| 200 | read is `io.ReadAll(io.LimitReader(res.Body, 1<<20))` (keymail.go:344), |
| 201 | which reads Keymail's HTTP response body. The change is the error value |
| 202 | (keymail.go:356). Its Error() text is the same as before, and it adds |
| 203 | Status and Msg so that `IsKeymailBlocked` can classify the answer. |
| 204 | **Proposed:** accepted-legacy, the console. |
| 205 | |
| 206 | 4. **`infra/modules/region-host/main.tf:external`**: digest |
| 207 | `6ae47ec3…9c5b`. Adds `aws_iam_role_policy.host_console_mail` |
| 208 | (main.tf:702). It has count 0 unless `console_mail_from` is set. It grants |
| 209 | `ses:SendEmail` on the one sending identity, the `carlos-bab` configuration |
| 210 | set and its tenants, on condition that `ses:FromAddress` equals |
| 211 | `console_mail_from` (main.tf:717). |
| 212 | **Proposed:** accepted-legacy, tofu on an operator's machine. No read is |
| 213 | added. See question 3: this is an authority grant on the box's role, not a |
| 214 | read. |
| 215 | |
| 216 | 5. **`infra/modules/region-host/variables.tf:external`**: digest |
| 217 | `ffdad585…250e`. Adds `console_mail_from` (variables.tf:221, validated as |
| 218 | one lowercase address) and `console_mail_region` (variables.tf:231). Both |
| 219 | are empty by default. |
| 220 | **Proposed:** accepted-legacy, tofu on an operator's machine. |
| 221 | |
| 222 | ### New units |
| 223 | |
| 224 | 6. **`internal/console/keymail.go:Keymail.sealEnvelope`**: |
| 225 | "unclassified root read … file:Read@387", digest `b140fde1…7658`. |
| 226 | `file:Read@387` is `rand.Read(idb)` at keymail.go:258, and `rand` is |
| 227 | `crypto/rand` (keymail.go:21). It reads the 16-byte envelope id from the |
| 228 | kernel's random source. It reads no file, store or box state. It is the |
| 229 | site main froze in `SendApp` as file:Read@949 (accepted-legacy, J4 |
| 230 | root-input), moved over unchanged with the rest of the sealing. |
| 231 | **Proposed:** accepted-legacy, the console, the same class as the site it |
| 232 | came from. |
| 233 | |
| 234 | 7. **`internal/console/store.go:Store.CountPendingInvites`**: |
| 235 | "unclassified root read … file:QueryRow@102", digest `1733078d…18e5`. One |
| 236 | SQLite `QueryRow` (store.go:1400): `SELECT COUNT(*) FROM invites WHERE |
| 237 | account_id = ? AND accepted_at = 0 AND expires_at >= ?`. It runs on the |
| 238 | console's own database (`CONSOLE_DB`). Its one caller is handleInvite's |
| 239 | pending cap (server.go:2446), and it can only refuse an invite. Its |
| 240 | siblings `Store.PendingInvites`, `CreateInvite` and `RevokeInvite` are |
| 241 | accepted-legacy (J4 root-input) on main, and so are all 84 `store.go` |
| 242 | rows. |
| 243 | **Proposed:** accepted-legacy, the console. |
| 244 | |
| 245 | ### Retired unit |
| 246 | |
| 247 | 8. **`internal/console/keymail.go:Keymail.SendApp`**: "stale root read" and |
| 248 | "changed legacy declaration". It is still declared (keymail.go:279) and |
| 249 | reads nothing now. Its one read moved into `sealEnvelope`, which it calls |
| 250 | at keymail.go:293. A frozen legacy row cannot have empty kinds (the |
| 251 | compare reports it stale), so it has to be retired, not re-frozen. |
| 252 | **Proposed:** `@remove` and `@removed … H3`, the form census ruling |
| 253 | carlos-run used for `apiSession.getJSON` and `ensureExe`. The loader |
| 254 | accepts an H3 removal only with a `census-ruling-` basis, so the row reads |
| 255 | `census-ruling-console-invite-delivery: …` (R2). A removal is not part of |
| 256 | the seal. |
| 257 | |
| 258 | ### Source envelopes |
| 259 | |
| 260 | 9. The four "changed root source envelope" rows are re-pinned: |
| 261 | - `cmd/console` to `2af70021…c0c6` |
| 262 | - `infra/modules/region-host` to `82e90b10…56e7` |
| 263 | - `internal/console` to `18fc22e0…5756` |
| 264 | - `internal/mail` to `c45dd340…e317` |
| 265 | |
| 266 | The scan finds no read site in any new file. In the changed files, the only |
| 267 | read-site changes are units 1 to 8. One thing the scanner does not see: |
| 268 | `mail.NewSESEmailSender` (sessend.go:22) calls |
| 269 | `awsconfig.LoadDefaultConfig`. That is the console's own credential |
| 270 | chain, which on a box is the host role. It is already listed in |
| 271 | `TestClaimAuthoritySourceCensus` as a send-only SES client (05cfbec0). |
| 272 | |
| 273 | ### Seal |
| 274 | |
| 275 | The overlay adds seven `@legacy` rows, which takes the frozen set from 2147 |
| 276 | to 2154. Computed with the overlay loaded after box-publish-record: |
| 277 | |
| 278 | - As drafted (the marker kept on ruling, as census ruling seed did): |
| 279 | `5ebadd0439ec247784a13739c74ab7363e22a3a9fb8da3d7ef03a76f1ab75468` |
| 280 | - With the tag rewritten to `rev3=census-ruling-console-invite-delivery`, |
| 281 | as R4 ruled, and the seal the test now uses: |
| 282 | **`fcdf9de0ba39db40d6bc7d1d2137d4f40d8248df8d286a380b2eae4e8b31d7a3`** |
| 283 | |
| 284 | The test edit as proposed is below. As applied, it sits after the |
| 285 | box-publish-record load in `TestPlan14RootReadCensus` with R4's seal: |
| 286 | |
| 287 | ```go |
| 288 | // Census overlay console-invite-delivery (console invite delivery), |
| 289 | // loaded last, under census-ruling-console-invite-delivery (platform-1e): |
| 290 | // ... the ruling's lines ... |
| 291 | if err = loadRootReadInventory(filepath.Join(root, "internal/provision/testdata/plan14-root-reads-console-invite-delivery.tsv"), &inv, true); err != nil { |
| 292 | t.Fatal(err) |
| 293 | } |
| 294 | seal = "5ebadd0439ec247784a13739c74ab7363e22a3a9fb8da3d7ef03a76f1ab75468" |
| 295 | ``` |
| 296 | |
| 297 | ## Questions for platform-1e |
| 298 | |
| 299 | 1. Do you accept the seven units (1 to 7) as accepted legacy, under the |
| 300 | executing identities named (the console as carlos-console, and tofu on an |
| 301 | operator's machine), with seal `5ebadd04…5468`? |
| 302 | 2. Should `Keymail.SendApp` be retired by an H3 census-ruling removal (the |
| 303 | carlos-run getJSON form), now that its only read lives in `sealEnvelope`? |
| 304 | 3. The `ses:SendEmail` grant is on the shared host role. Any process that |
| 305 | holds the box's role credentials (root, the agent or the console) can send |
| 306 | as `console_mail_from`, and only as that address. That is a grant, not a |
| 307 | read. Is it inside this census's scope, or does it need its own ruling |
| 308 | elsewhere? |
| 309 | 4. Keep the `PROPOSED-console-invite-delivery` marker in the frozen bases |
| 310 | (seal `5ebadd04…`), or rewrite it to a `census-ruling-` tag (seal |
| 311 | `fcdf9de0…`)? |
| 312 | 5. This is not a census item, but you own host claims, so it is flagged for |
| 313 | you. The console mailer is built over the raw release store, not |
| 314 | `claimMailStore()`. Its `EnsureDomain` and `VerifyDomain` on |
| 315 | `ops/console-mail` would skip the claim mail gate when HostClaim is on. |
| 316 | Switching to the gated store would fail today, because `console-mail` is |
| 317 | not a registered app. HostClaim is off by default, so nothing changes now. |
| 318 | Should this block the merge, or wait for a HostClaim design decision? |
| 319 | |
amadan