amadan
specs: instance-socket consumers design (issue #103)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QytpMG8wVhBmWwz3uohtc7
1 file changed,
+131
−0
docs/superpowers/specs/2026-08-08-instance-socket-consumers-design.md+131 −0
diff --git a/docs/superpowers/specs/2026-08-08-instance-socket-consumers-design.md b/docs/superpowers/specs/2026-08-08-instance-socket-consumers-design.md| new file mode 100644 |
| index 000000000..33aab1a96 |
| --- /dev/null |
| +++ b/docs/superpowers/specs/2026-08-08-instance-socket-consumers-design.md |
| @@ -0,0 +1,131 @@ |
| +# Instance-socket consumers (issue #103) — design |
| + |
| +Date: 2026-08-08. From issue #103: on the tito box, titogo's router (running |
| +as user `titogo`) proxies root-domain surfaces onward to platform-provisioned |
| +instances' unix sockets (`/run/carlos/<host>.sock`) and got |
| +`connect: permission denied` the moment a reconciler-provisioned instance |
| +woke. Two layers blocked the cross-user dial: `/run/carlos` is |
| +`0750 carlos:carlos` with no story for enrolling a consumer, and the socket's |
| +mode is whatever the app's umask/chmod habits produce (titogo bound 0600). |
| +The interim (`usermod -aG carlos titogo` + hand-chmod per wake) works but |
| +doesn't survive a rebind — and the failure is a buyer-facing 502 on an |
| +otherwise healthy instance. Autonomous-session design: the issue offered two |
| +directions and left the choice to the platform; both halves below were chosen |
| +inline, with the trade-offs recorded. |
| + |
| +## Decision 1: a dedicated `carlos-sockets` group, never group `carlos` |
| + |
| +The tito interim enrolled the consumer into group `carlos`. That works for |
| +traversal, but group `carlos` is also the group on `/data` (`0750 |
| +carlos:carlos`) — membership grants group-read of every instance database and |
| +every `<host>.env`, including each instance's `CARLOS_ADMIN_TOKEN`. A grant |
| +whose meaning is "may dial instance sockets" must not silently carry "may |
| +read every instance's secrets". |
| + |
| +So the blessed consumers group is a dedicated system group, |
| +**`carlos-sockets`**: |
| + |
| +- `carlos bootstrap` creates it (`groupadd --system`, idempotent, live root |
| + only — a staging root prints the command, same as `useradd`). |
| +- `/run/carlos` becomes **`2750 carlos carlos-sockets`** — group changed to |
| + the consumers group (members get `r-x`: traversal to the sockets), setgid |
| + bit added so **every socket bound inside inherits the group**. That |
| + inheritance is what makes the mode contract (decision 2) sufficient: the |
| + agent never needs to chgrp anything, and no app needs to know the group |
| + exists. Both the tmpfiles.d entry and bootstrap's own dir handling change |
| + together (they are the same truth in two places). |
| +- `/run/carlos/apps` stays **`0750 carlos carlos`**. Unit tenants get their |
| + listener fd passed by systemd (`SocketUser=carlos`, 0600) and never need |
| + path access — and consumer enrollment deliberately does not reach them. |
| + A `carlos-sockets` member can traverse `/run/carlos` but not `apps/`. |
| + |
| +**Enrollment** is bootstrap's job, per the issue's ask: a repeatable |
| +`-socket-consumer <user>` flag runs `usermod -aG carlos-sockets <user>` on a |
| +live root (printed under a staging root). Idempotent — `usermod -aG` on an |
| +existing member is a no-op, so re-running bootstrap with the same flags |
| +converges. A missing user fails loudly via usermod's own error. |
| + |
| +**Scope of the grant, stated plainly:** membership in `carlos-sockets` is |
| +box-wide — a member may dial *every* exec-backed instance socket on the box. |
| +That is the intended semantic (the consumer is the deployment's own trusted |
| +host app, e.g. titogo's router), not a per-instance ACL; a deployment that |
| +needs per-instance isolation between consumers needs a different mechanism |
| +and should say so in an issue. |
| + |
| +## Decision 2: the agent normalizes the socket mode — the app contract is a |
| +guarantee, not a chore |
| + |
| +The issue's two options were "write 0660 into the app contract" or "the |
| +agent normalizes the mode after health". Docs-only loses to enforcement: |
| +the failure case that motivated this issue *was* an app's own chmod habit, |
| +and the exec'd child inherits the agent's umask either way — a contract no |
| +mechanism enforces is rediscovered per deployment, which is exactly the |
| +state #103 asks us out of. |
| + |
| +So: **`ExecBackend.Start` chmods the instance socket to `0660` immediately |
| +after the health wait succeeds.** Properties that make this the right spot: |
| + |
| +- It runs only for children the activator itself spawned — it can never |
| + touch a socket an external process is serving (the #17 external-socket |
| + advisory's territory). |
| +- The agent and the child run as the same user, so the chmod is on a socket |
| + we own; failure is essentially impossible, and is logged loudly rather |
| + than failing the wake (a box with no consumers must not lose a healthy |
| + wake to a normalization it doesn't need). |
| +- After health means after the app has bound (and possibly chmodded) the |
| + socket — the normalization wins against any app habit. The window between |
| + bind and normalization is the health-probe interval (milliseconds); a |
| + consumer's dial in that window fails once and its retry succeeds. |
| + |
| +With decision 1's setgid dir, the normalized socket is |
| +`0660 carlos:carlos-sockets`: group members dial, everyone else is refused. |
| +On a box where bootstrap has never been re-run (no setgid, group absent), |
| +the socket's group is `carlos` and the chmod grants nothing new — the |
| +normalization is inert until the box converges, never a widening. |
| + |
| +The seed path binds no sockets; sidecars bind no sockets; the agent's own |
| +`.control` purge socket is bound with default umask (no group write) and |
| +stays effectively agent-only — none of them normalize. |
| + |
| +**The contract, stated for app authors** (alongside the seed contract): bind |
| +your instance socket at the path the platform passes via `--socket`, with |
| +whatever mode your runtime produces — the platform normalizes it to `0660` |
| +after your first health response. Do not tighten the mode after startup; a |
| +post-health chmod strands the deployment's sibling consumers. |
| + |
| +## What changes where |
| + |
| +- `internal/host/bootstrap.go` — `socketsGroup` const; `tmpfilesConf` line |
| + for `/run/carlos` → `2750 carlos carlos-sockets`; live-run: create group, |
| + `chgrp` + setgid `/run/carlos` (after the recursive `chown` to the service |
| + user, which would otherwise reset the group); `-socket-consumer` |
| + repeatable flag → `usermod -aG`; staging prints all of it. |
| +- `internal/activator/backend_exec.go` — the post-health `os.Chmod(rt.Socket, |
| + 0o660)` in `Start`, log-on-error. |
| +- Tests: bootstrap staging asserts the new tmpfiles line and printed plan; |
| + an ExecBackend test spawns a real child (test-binary re-exec) that binds |
| + its socket, chmods it 0600 (the titogo habit), and serves — asserting the |
| + socket is 0660 after `Start` returns. |
| + |
| +No registry, wire, or storage shape changes. No console surface. |
| + |
| +## Deployment notes |
| + |
| +Per box, after adopting a binary with this change: |
| + |
| +1. Re-run `carlos bootstrap -socket-consumer <user> ...` (live root). On |
| + **tito**: enroll the titogo router/relay users (the ones titogo#535 put |
| + in group `carlos`); afterwards remove them from group `carlos` |
| + (`gpasswd -d <user> carlos`) — the interim grant carries /data read and |
| + the new group replaces it. On **flagship**: no consumers today; plain |
| + re-run just converges the dir. |
| +2. `systemctl restart` the consumer processes (group membership is read at |
| + process start). |
| +3. Already-live instance sockets keep their old group/mode until their next |
| + wake; converge immediately with one `chgrp carlos-sockets /run/carlos/*.sock |
| + && chmod 0660 /run/carlos/*.sock`, or just let the next hibernate/wake |
| + cycle do it. |
| + |
| +Follow-ups owed elsewhere: website docs batch (standing rule) — the |
| +app-author contract line belongs on the public app-contract page when that |
| +batch runs. |