| 1 | # Isolated Exec, Plan 1 of 4: Foundation Implementation Plan |
| 2 | |
| 3 | > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. |
| 4 | |
| 5 | **Goal:** Land every building block of the isolated-exec design that changes nothing on a flat fleet: the path helpers and instance identifier, the symlink-refusing file helper, the unix-dial checks, the slice and mount units, and the bootstrap wiring for them. |
| 6 | |
| 7 | **Architecture:** Four self-contained pieces that later plans consume by name: `provision.InstanceID` and the nested path helpers (§3.1), the `instancefs` package built on `openat2` (§3.2), `unixdial.Checked` used by every unix dial the edge and agent make (§5.4, §6.1), and the `carlos.slice` / `carlos-instances.slice` / three `nosymfollow` mount units with their tmpfiles lines and bootstrap arming (§3.2, §4.3). The edge unit moves into `carlos.slice` and loses its own memory lines. No spawn path changes in this plan. |
| 8 | |
| 9 | **Tech Stack:** Go 1.26, `golang.org/x/sys/unix` (already in go.mod), systemd 257 unit files embedded via `embed.FS`, the repo's existing test style (`testing`, no assertion library). |
| 10 | |
| 11 | **Spec:** `docs/superpowers/specs/2026-09-11-isolated-exec-design.md` (revision 12). Read §2, §3, §4.3, §5.4, §6.1 and §9 before starting. §13 records every codex ruling; none may be reopened. |
| 12 | |
| 13 | ## Global Constraints |
| 14 | |
| 15 | - `go build ./...`, `go vet ./...`, `go test ./... -count=1`, `gofmt -l .` clean before every commit (AGENTS.md). Use `GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off`; socket-binding tests need `TMPDIR=/tmp/g` (create it) and may need the sandbox off. |
| 16 | - CI cross-compiles `darwin`, `windows` and `linux-arm64`: every Linux-only syscall lives in a `_linux.go` file with a `//go:build linux` tag and has a `!linux` stub that returns `errors.ErrUnsupported`. |
| 17 | - `InstanceID` is at most 40 bytes; a hashed identifier starts with `_`; the longest socket address the helpers produce is under 108 bytes (spec §3.1). |
| 18 | - Unit files are the source of truth in `internal/host/units/` and are embedded; nothing is applied by drop-in. |
| 19 | - `carlos-edge.service` keeps every hardening line it has today; only `Slice=`, `Requires=`/`After=` for the mounts, and the three memory lines change (spec §4.3). |
| 20 | - Copy rules: no em dashes in any user-facing string; log lines use the existing `verb: subject: detail` shape. |
| 21 | - Commit after every task with the attribution trailer the session gives you. Never push to `main`. The branch is `worktree-isolated-exec-spec` (worktree `.claude/worktrees/isolated-exec-spec`). |
| 22 | |
| 23 | --- |
| 24 | |
| 25 | ## File Structure |
| 26 | |
| 27 | | File | Responsibility | |
| 28 | |---|---| |
| 29 | | `internal/provision/provision.go` (modify, after line 1012) | `InstanceID`, `InstanceDir`, `NestedDBPath`, `InstanceRunDir`, `NestedSocket`, `LogSocket`, `ControlSocketDir`, `ControlSocketPath`, `IsControlEndpoint`, `MaxUnixPath`, `LongestSocketAddress` | |
| 30 | | `internal/provision/provision_test.go` (modify) | tests for the above | |
| 31 | | `internal/instancefs/instancefs.go` (create) | `OpenBeneath`, `ReadBeneath`, `WriteAtomic`, `MountNoSymfollow`; documentation of the contract | |
| 32 | | `internal/instancefs/instancefs_linux.go` (create) | the `openat2` and `renameat` implementations | |
| 33 | | `internal/instancefs/instancefs_other.go` (create) | `!linux` stubs | |
| 34 | | `internal/instancefs/instancefs_linux_test.go` (create) | tests | |
| 35 | | `internal/unixdial/unixdial.go` (create) | `Checked(ctx, dialer, path)`: refuse the control endpoint, dial, verify `getpeername` | |
| 36 | | `internal/unixdial/unixdial_test.go` (create) | real socket-pair tests | |
| 37 | | `internal/edge/edge.go:2647` (modify) | `dialUnixWithRetry` uses `unixdial.Checked` | |
| 38 | | `internal/activator/backend_exec.go:916` (modify) | `socketClient` uses `unixdial.Checked` | |
| 39 | | `internal/release/versions.go:364` (modify) | `unixHTTPClient` uses `unixdial.Checked` | |
| 40 | | `internal/host/socket_advice.go:48` (modify) | `socketServing` uses `unixdial.Checked` | |
| 41 | | `internal/host/units/carlos.slice` (create) | outer memory wall | |
| 42 | | `internal/host/units/carlos-instances.slice` (create) | parent of per-host slices | |
| 43 | | `internal/host/units/data-instances.mount` (create) | `/data/.instances` → `/data/instances`, nosymfollow | |
| 44 | | `internal/host/units/run-carlos-instances.mount` (create) | `/run/carlos/.instances` → `/run/carlos/instances` | |
| 45 | | `internal/host/units/run-carlos-logs.mount` (create) | `/run/carlos/.logs` → `/run/carlos/logs` | |
| 46 | | `internal/host/units/carlos-edge.service` (modify) | `Slice=carlos.slice`; `Requires=`/`After=` the mounts; drop `MemoryHigh=`/`MemoryMax=` | |
| 47 | | `internal/host/units/carlos-litestream.service` (modify) | `After=data-instances.mount` | |
| 48 | | `internal/host/bootstrap.go` (modify) | embed glob, tmpfiles lines, dirs, `enabledUnits`, `systemctlPlan` | |
| 49 | | `internal/host/units_isolation_test.go` (create) | unit-file assertions for this plan | |
| 50 | | `internal/host/host_test.go:1195` (modify) | the plan test learns about sockets and mounts | |
| 51 | |
| 52 | --- |
| 53 | |
| 54 | ### Task 1: Instance identifier and nested path helpers |
| 55 | |
| 56 | **Files:** |
| 57 | - Modify: `internal/provision/provision.go` (append after `DefaultDBPath`, line 1012) |
| 58 | - Test: `internal/provision/provision_test.go` (append) |
| 59 | |
| 60 | **Interfaces:** |
| 61 | - Consumes: `SocketDir` (`"/run/carlos"`), `DefaultSocket`, `DefaultDBPath` (unchanged: they keep their flat meaning in this release, spec §3.1). |
| 62 | - Produces: |
| 63 | ```go |
| 64 | const MaxInstanceID = 40 |
| 65 | const MaxUnixPath = 107 // sockaddr_un sun_path minus the NUL |
| 66 | const ControlSocketDir = "/run/carlos-control" |
| 67 | const ControlSocketPath = "/run/carlos-control/control.sock" |
| 68 | func InstanceID(host string) string |
| 69 | func InstanceDir(dataDir, host string) string |
| 70 | func NestedDBPath(dataDir, host string) string |
| 71 | func InstanceRunDir(host string) string |
| 72 | func NestedSocket(host string) string |
| 73 | func LogSocket(host, role string) string |
| 74 | func IsControlEndpoint(path string) bool // true for anything under ControlSocketDir |
| 75 | func LongestSocketAddress() int // the worst case over every helper, for the pinned test |
| 76 | ``` |
| 77 | |
| 78 | - [ ] **Step 1: Write the failing tests** |
| 79 | |
| 80 | Append to `internal/provision/provision_test.go`: |
| 81 | |
| 82 | ```go |
| 83 | func TestInstanceIDShortHostsAreThemselves(t *testing.T) { |
| 84 | for _, h := range []string{"h", "bill.go.tito.io", strings.Repeat("a", 40)} { |
| 85 | if got := InstanceID(h); got != h { |
| 86 | t.Errorf("InstanceID(%q) = %q, want the host itself", h, got) |
| 87 | } |
| 88 | } |
| 89 | } |
| 90 | |
| 91 | func TestInstanceIDLongHostsAreBoundedAndPrefixed(t *testing.T) { |
| 92 | long := strings.Repeat("a", 60) + ".example.com" |
| 93 | got := InstanceID(long) |
| 94 | if len(got) != MaxInstanceID { |
| 95 | t.Fatalf("InstanceID(long) = %q (%d bytes), want %d", got, len(got), MaxInstanceID) |
| 96 | } |
| 97 | if got[0] != '_' { |
| 98 | t.Errorf("InstanceID(long) = %q, want a leading underscore: no valid host has one", got) |
| 99 | } |
| 100 | if got[1:23] != long[:22] { |
| 101 | t.Errorf("InstanceID(long) = %q, want the host's first 22 bytes after the underscore", got) |
| 102 | } |
| 103 | if got[23] != '-' { |
| 104 | t.Errorf("InstanceID(long) = %q, want a dash before the hash", got) |
| 105 | } |
| 106 | if got != InstanceID(long) { |
| 107 | t.Error("InstanceID is not stable") |
| 108 | } |
| 109 | // Two long hosts sharing a 22-byte prefix get different identifiers. |
| 110 | other := strings.Repeat("a", 60) + ".example.org" |
| 111 | if InstanceID(other) == got { |
| 112 | t.Errorf("InstanceID(%q) == InstanceID(%q); the hash must separate them", long, other) |
| 113 | } |
| 114 | } |
| 115 | |
| 116 | func TestInstanceIDDomainsNeverMeet(t *testing.T) { |
| 117 | // codex R6-1: the hashed form of one host must never equal any valid |
| 118 | // short host. A leading underscore is not a valid host label. |
| 119 | long := strings.Repeat("a", 31) + ".example.com" |
| 120 | if ValidHostShape(InstanceID(long)) { |
| 121 | t.Errorf("InstanceID(%q) = %q is a valid host shape; the domains overlap", long, InstanceID(long)) |
| 122 | } |
| 123 | } |
| 124 | |
| 125 | func TestNestedHelpersAreKeyedByInstanceID(t *testing.T) { |
| 126 | long := strings.Repeat("b", 50) + ".example.com" |
| 127 | iid := InstanceID(long) |
| 128 | if got, want := InstanceDir("/data", long), "/data/instances/"+iid; got != want { |
| 129 | t.Errorf("InstanceDir = %q, want %q", got, want) |
| 130 | } |
| 131 | if got, want := NestedDBPath("/data/", long), "/data/instances/"+iid+"/"+long+".db"; got != want { |
| 132 | t.Errorf("NestedDBPath = %q, want %q (file names keep the full host)", got, want) |
| 133 | } |
| 134 | if got, want := InstanceRunDir(long), "/run/carlos/instances/"+iid; got != want { |
| 135 | t.Errorf("InstanceRunDir = %q, want %q", got, want) |
| 136 | } |
| 137 | if got, want := NestedSocket(long), "/run/carlos/instances/"+iid+"/app.sock"; got != want { |
| 138 | t.Errorf("NestedSocket = %q, want %q", got, want) |
| 139 | } |
| 140 | if got, want := LogSocket(long, "sidecar"), "/run/carlos/logs/"+iid+".sidecar.sock"; got != want { |
| 141 | t.Errorf("LogSocket = %q, want %q", got, want) |
| 142 | } |
| 143 | } |
| 144 | |
| 145 | func TestFlatDefaultsUnchangedThisRelease(t *testing.T) { |
| 146 | // Spec §8.1: nothing mints nested in this release. The flat helpers |
| 147 | // are what routeFor and `carlos add` still use. |
| 148 | if got := DefaultSocket("h"); got != "/run/carlos/h.sock" { |
| 149 | t.Errorf("DefaultSocket = %q", got) |
| 150 | } |
| 151 | if got := DefaultDBPath("/data", "h"); got != "/data/h.db" { |
| 152 | t.Errorf("DefaultDBPath = %q", got) |
| 153 | } |
| 154 | } |
| 155 | |
| 156 | func TestLongestSocketAddressFitsSockaddrUn(t *testing.T) { |
| 157 | // codex R5-3: an 80-byte host overflowed the first draft's paths. The |
| 158 | // bound is computed, not assumed, over a 253-byte host. |
| 159 | if got := LongestSocketAddress(); got > MaxUnixPath { |
| 160 | t.Fatalf("longest socket address is %d bytes, over the %d-byte limit", got, MaxUnixPath) |
| 161 | } |
| 162 | host := strings.Repeat("c", 253) |
| 163 | for _, p := range []string{NestedSocket(host), NestedSocket(host) + ".control", LogSocket(host, "sidecar")} { |
| 164 | if len(p) > MaxUnixPath { |
| 165 | t.Errorf("%q is %d bytes", p, len(p)) |
| 166 | } |
| 167 | } |
| 168 | } |
| 169 | |
| 170 | func TestIsControlEndpoint(t *testing.T) { |
| 171 | for path, want := range map[string]bool{ |
| 172 | ControlSocketPath: true, |
| 173 | "/run/carlos-control/anything": true, |
| 174 | "/run/carlos-control": true, |
| 175 | "/run/carlos/h.sock": false, |
| 176 | "/run/carlos/instances/h/app.sock": false, |
| 177 | "/run/carlos-controlx/control.sock": false, |
| 178 | "/run/titogo/example.sock": false, |
| 179 | } { |
| 180 | if got := IsControlEndpoint(path); got != want { |
| 181 | t.Errorf("IsControlEndpoint(%q) = %v, want %v", path, got, want) |
| 182 | } |
| 183 | } |
| 184 | } |
| 185 | ``` |
| 186 | |
| 187 | Add `"strings"` to the test file's imports if it is not already there (it is used by other tests in that file; check with `grep -n '"strings"' internal/provision/provision_test.go`). |
| 188 | |
| 189 | - [ ] **Step 2: Run the tests to verify they fail** |
| 190 | |
| 191 | Run: `GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/provision -run 'TestInstanceID|TestNestedHelpers|TestFlatDefaults|TestLongestSocket|TestIsControlEndpoint' -count=1` |
| 192 | Expected: FAIL with `undefined: InstanceID` (and the others). |
| 193 | |
| 194 | - [ ] **Step 3: Implement the helpers** |
| 195 | |
| 196 | Append to `internal/provision/provision.go` after `DefaultDBPath` (line 1012). Add `"crypto/sha256"` and `"encoding/hex"` to the import block. |
| 197 | |
| 198 | ```go |
| 199 | // MaxInstanceID bounds InstanceID. A unix socket address holds MaxUnixPath |
| 200 | // bytes of path and a host may be 253 bytes, so the directories a socket |
| 201 | // lives under are keyed by a bounded identifier, never by the host itself |
| 202 | // (isolated-exec design §3.1, codex R5-3). |
| 203 | const MaxInstanceID = 40 |
| 204 | |
| 205 | // MaxUnixPath is sockaddr_un's sun_path (108 bytes) minus the NUL Go |
| 206 | // insists on room for. |
| 207 | const MaxUnixPath = 107 |
| 208 | |
| 209 | // ControlSocketDir and ControlSocketPath are the root instance-control |
| 210 | // channel (design §5). Deliberately NOT under SocketDir: an instance's |
| 211 | // namespace replaces /run/carlos with an empty tmpfs, and nothing an |
| 212 | // instance can reach must resolve to this path. |
| 213 | const ( |
| 214 | ControlSocketDir = "/run/carlos-control" |
| 215 | ControlSocketPath = ControlSocketDir + "/control.sock" |
| 216 | ) |
| 217 | |
| 218 | // InstanceID is the bounded per-host identifier every isolated-exec |
| 219 | // directory, socket, unit instance and slice is keyed by. A host of at |
| 220 | // most MaxInstanceID bytes is its own identifier. A longer host becomes |
| 221 | // "_" + its first 22 bytes + "-" + 16 hex digits of its sha256, also |
| 222 | // MaxInstanceID bytes. |
| 223 | // |
| 224 | // The underscore is the whole reason the two forms can share one |
| 225 | // namespace: no valid host contains one (controlname labels are |
| 226 | // [a-z0-9-]), so a hashed identifier can never equal any short host, and |
| 227 | // two hashed identifiers collide only on a 64-bit hash collision between |
| 228 | // two long hosts with the same 22-byte prefix (codex R6-1). Anything that |
| 229 | // finds two rows with one identifier treats them as a conflict, never as |
| 230 | // the same host. |
| 231 | func InstanceID(host string) string { |
| 232 | if len(host) <= MaxInstanceID { |
| 233 | return host |
| 234 | } |
| 235 | sum := sha256.Sum256([]byte(host)) |
| 236 | return "_" + host[:22] + "-" + hex.EncodeToString(sum[:8]) |
| 237 | } |
| 238 | |
| 239 | // InstanceDir is the per-host data directory of the nested layout: |
| 240 | // <dataDir>/instances/<InstanceID(host)>. File names INSIDE it keep the |
| 241 | // full host (a file name may be 255 bytes; a socket path may not). |
| 242 | func InstanceDir(dataDir, host string) string { |
| 243 | return filepath.Join(dataDir, "instances", InstanceID(host)) |
| 244 | } |
| 245 | |
| 246 | // NestedDBPath is the database under InstanceDir. It is what the layout |
| 247 | // pass writes into a migrated row's DBPath; DefaultDBPath stays flat in |
| 248 | // this release (design §8.1). |
| 249 | func NestedDBPath(dataDir, host string) string { |
| 250 | return filepath.Join(InstanceDir(dataDir, host), host+".db") |
| 251 | } |
| 252 | |
| 253 | // InstanceRunDir is the per-host socket directory of the nested layout. |
| 254 | func InstanceRunDir(host string) string { |
| 255 | return filepath.Join(SocketDir, "instances", InstanceID(host)) |
| 256 | } |
| 257 | |
| 258 | // NestedSocket is the instance's listener under InstanceRunDir; its |
| 259 | // control socket is NestedSocket + ".control", as for every exec route. |
| 260 | func NestedSocket(host string) string { |
| 261 | return filepath.Join(InstanceRunDir(host), "app.sock") |
| 262 | } |
| 263 | |
| 264 | // LogSocket is the agent's per-role log listener for a host (design |
| 265 | // §6.4): outside every tenant mount, keyed by the identifier. |
| 266 | func LogSocket(host, role string) string { |
| 267 | return filepath.Join(SocketDir, "logs", InstanceID(host)+"."+role+".sock") |
| 268 | } |
| 269 | |
| 270 | // IsControlEndpoint reports whether path is the control channel or |
| 271 | // anything else under its directory. The edge and the agent refuse to |
| 272 | // dial such a path as an upstream (design §5.4): a registry row pointing |
| 273 | // at it is refused before connect(). |
| 274 | func IsControlEndpoint(path string) bool { |
| 275 | clean := filepath.Clean(path) |
| 276 | return clean == ControlSocketDir || strings.HasPrefix(clean, ControlSocketDir+"/") |
| 277 | } |
| 278 | |
| 279 | // LongestSocketAddress is the longest unix socket path any helper here |
| 280 | // can produce, over the longest valid host, so a test can pin it under |
| 281 | // MaxUnixPath rather than trusting arithmetic in a comment. |
| 282 | func LongestSocketAddress() int { |
| 283 | host := strings.Repeat("x", 253) |
| 284 | longest := 0 |
| 285 | for _, p := range []string{ |
| 286 | NestedSocket(host), |
| 287 | NestedSocket(host) + ".control", |
| 288 | LogSocket(host, "sidecar"), // the longest role name |
| 289 | } { |
| 290 | if len(p) > longest { |
| 291 | longest = len(p) |
| 292 | } |
| 293 | } |
| 294 | return longest |
| 295 | } |
| 296 | ``` |
| 297 | |
| 298 | - [ ] **Step 4: Run the tests to verify they pass** |
| 299 | |
| 300 | Run: `GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/provision -count=1` |
| 301 | Expected: PASS (the whole package, including `TestDefaults`). |
| 302 | |
| 303 | - [ ] **Step 5: gofmt and commit** |
| 304 | |
| 305 | ```bash |
| 306 | gofmt -l internal/provision |
| 307 | git add internal/provision/provision.go internal/provision/provision_test.go |
| 308 | git commit -m "provision: instance identifier and nested path helpers (isolated exec plan 1, task 1)" |
| 309 | ``` |
| 310 | |
| 311 | --- |
| 312 | |
| 313 | ### Task 2: The `instancefs` package |
| 314 | |
| 315 | **Files:** |
| 316 | - Create: `internal/instancefs/instancefs.go` |
| 317 | - Create: `internal/instancefs/instancefs_linux.go` |
| 318 | - Create: `internal/instancefs/instancefs_other.go` |
| 319 | - Create: `internal/instancefs/instancefs_linux_test.go` |
| 320 | |
| 321 | **Interfaces:** |
| 322 | - Consumes: `golang.org/x/sys/unix` (`Openat2`, `OpenHow`, `RESOLVE_BENEATH`, `RESOLVE_NO_SYMLINKS`, `RESOLVE_NO_MAGICLINKS`, `Renameat`), `crypto/rand`. |
| 323 | - Produces: |
| 324 | ```go |
| 325 | package instancefs |
| 326 | var ErrSymlink = errors.New("instancefs: path contains a symlink") |
| 327 | var ErrEscapes = errors.New("instancefs: path escapes its directory") |
| 328 | // OpenBeneath opens name (a relative path with no "..") under dir, following no symlink at any component. |
| 329 | func OpenBeneath(dir, name string, flag int, perm os.FileMode) (*os.File, error) |
| 330 | // ReadBeneath is OpenBeneath + io.ReadAll. |
| 331 | func ReadBeneath(dir, name string) ([]byte, error) |
| 332 | // WriteAtomic writes data to a random O_EXCL temp file in dir and renames it over name. Never follows a symlink at the destination. |
| 333 | func WriteAtomic(dir, name string, data []byte, perm os.FileMode) error |
| 334 | // MountNoSymfollow reports whether mountpoint is a mount whose options include nosymfollow, per /proc/self/mountinfo. |
| 335 | func MountNoSymfollow(mountpoint string) (bool, error) |
| 336 | ``` |
| 337 | Every function returns `errors.ErrUnsupported` on non-Linux. |
| 338 | |
| 339 | - [ ] **Step 1: Write the failing tests** |
| 340 | |
| 341 | Create `internal/instancefs/instancefs_linux_test.go`: |
| 342 | |
| 343 | ```go |
| 344 | //go:build linux |
| 345 | |
| 346 | package instancefs |
| 347 | |
| 348 | import ( |
| 349 | "errors" |
| 350 | "os" |
| 351 | "path/filepath" |
| 352 | "strings" |
| 353 | "testing" |
| 354 | ) |
| 355 | |
| 356 | func TestOpenBeneathReadsAPlainFile(t *testing.T) { |
| 357 | dir := t.TempDir() |
| 358 | if err := os.WriteFile(filepath.Join(dir, "h.env"), []byte("A=1\n"), 0o600); err != nil { |
| 359 | t.Fatal(err) |
| 360 | } |
| 361 | got, err := ReadBeneath(dir, "h.env") |
| 362 | if err != nil { |
| 363 | t.Fatalf("ReadBeneath: %v", err) |
| 364 | } |
| 365 | if string(got) != "A=1\n" { |
| 366 | t.Errorf("got %q", got) |
| 367 | } |
| 368 | } |
| 369 | |
| 370 | func TestOpenBeneathRefusesASymlinkAtTheLeaf(t *testing.T) { |
| 371 | dir := t.TempDir() |
| 372 | secret := filepath.Join(t.TempDir(), "secret") |
| 373 | if err := os.WriteFile(secret, []byte("token"), 0o600); err != nil { |
| 374 | t.Fatal(err) |
| 375 | } |
| 376 | if err := os.Symlink(secret, filepath.Join(dir, "h.env")); err != nil { |
| 377 | t.Fatal(err) |
| 378 | } |
| 379 | _, err := ReadBeneath(dir, "h.env") |
| 380 | if !errors.Is(err, ErrSymlink) { |
| 381 | t.Fatalf("ReadBeneath through a symlink: err = %v, want ErrSymlink", err) |
| 382 | } |
| 383 | } |
| 384 | |
| 385 | func TestOpenBeneathRefusesASymlinkedDirectory(t *testing.T) { |
| 386 | dir := t.TempDir() |
| 387 | other := t.TempDir() |
| 388 | if err := os.WriteFile(filepath.Join(other, "x"), []byte("x"), 0o600); err != nil { |
| 389 | t.Fatal(err) |
| 390 | } |
| 391 | if err := os.Symlink(other, filepath.Join(dir, "sub")); err != nil { |
| 392 | t.Fatal(err) |
| 393 | } |
| 394 | _, err := ReadBeneath(dir, "sub/x") |
| 395 | if !errors.Is(err, ErrSymlink) { |
| 396 | t.Fatalf("err = %v, want ErrSymlink", err) |
| 397 | } |
| 398 | } |
| 399 | |
| 400 | func TestOpenBeneathRefusesDotDotAndAbsolute(t *testing.T) { |
| 401 | dir := t.TempDir() |
| 402 | for _, name := range []string{"../x", "a/../../x", "/etc/passwd"} { |
| 403 | _, err := ReadBeneath(dir, name) |
| 404 | if !errors.Is(err, ErrEscapes) { |
| 405 | t.Errorf("ReadBeneath(%q): err = %v, want ErrEscapes", name, err) |
| 406 | } |
| 407 | } |
| 408 | } |
| 409 | |
| 410 | func TestWriteAtomicReplacesAndNeverFollows(t *testing.T) { |
| 411 | dir := t.TempDir() |
| 412 | if err := WriteAtomic(dir, "h.config.env", []byte("one\n"), 0o600); err != nil { |
| 413 | t.Fatalf("WriteAtomic: %v", err) |
| 414 | } |
| 415 | if err := WriteAtomic(dir, "h.config.env", []byte("two\n"), 0o600); err != nil { |
| 416 | t.Fatalf("WriteAtomic (replace): %v", err) |
| 417 | } |
| 418 | got, err := os.ReadFile(filepath.Join(dir, "h.config.env")) |
| 419 | if err != nil || string(got) != "two\n" { |
| 420 | t.Fatalf("got %q, %v", got, err) |
| 421 | } |
| 422 | fi, err := os.Stat(filepath.Join(dir, "h.config.env")) |
| 423 | if err != nil || fi.Mode().Perm() != 0o600 { |
| 424 | t.Errorf("mode = %v, %v", fi.Mode(), err) |
| 425 | } |
| 426 | // A symlink planted at the destination is REPLACED as a directory |
| 427 | // entry; its target is untouched. |
| 428 | victim := filepath.Join(t.TempDir(), "victim") |
| 429 | if err := os.WriteFile(victim, []byte("keep"), 0o600); err != nil { |
| 430 | t.Fatal(err) |
| 431 | } |
| 432 | if err := os.Remove(filepath.Join(dir, "h.config.env")); err != nil { |
| 433 | t.Fatal(err) |
| 434 | } |
| 435 | if err := os.Symlink(victim, filepath.Join(dir, "h.config.env")); err != nil { |
| 436 | t.Fatal(err) |
| 437 | } |
| 438 | if err := WriteAtomic(dir, "h.config.env", []byte("three\n"), 0o600); err != nil { |
| 439 | t.Fatalf("WriteAtomic over a symlink: %v", err) |
| 440 | } |
| 441 | if b, _ := os.ReadFile(victim); string(b) != "keep" { |
| 442 | t.Errorf("symlink target was written through: %q", b) |
| 443 | } |
| 444 | if fi, err := os.Lstat(filepath.Join(dir, "h.config.env")); err != nil || fi.Mode()&os.ModeSymlink != 0 { |
| 445 | t.Errorf("destination is still a symlink: %v %v", fi, err) |
| 446 | } |
| 447 | // No temp file is left behind. |
| 448 | entries, _ := os.ReadDir(dir) |
| 449 | for _, e := range entries { |
| 450 | if strings.Contains(e.Name(), ".tmp-") { |
| 451 | t.Errorf("temp file left behind: %s", e.Name()) |
| 452 | } |
| 453 | } |
| 454 | } |
| 455 | |
| 456 | func TestMountNoSymfollowReadsMountinfo(t *testing.T) { |
| 457 | // / is a mount and does not carry nosymfollow on any ordinary box; a |
| 458 | // path that is not a mountpoint is reported as not a nosymfollow mount |
| 459 | // with a nil error (the caller's message names the path). |
| 460 | got, err := MountNoSymfollow("/") |
| 461 | if err != nil { |
| 462 | t.Fatalf("MountNoSymfollow(/): %v", err) |
| 463 | } |
| 464 | if got { |
| 465 | t.Error("/ reported as nosymfollow") |
| 466 | } |
| 467 | got, err = MountNoSymfollow(t.TempDir()) |
| 468 | if err != nil || got { |
| 469 | t.Errorf("temp dir: %v %v", got, err) |
| 470 | } |
| 471 | // The parser, against a fixture line, since a real nosymfollow mount |
| 472 | // needs root to create. |
| 473 | line := "36 35 98:0 /.instances /data/instances rw,nosuid,nodev,noexec,nosymfollow,relatime - ext4 /dev/root rw" |
| 474 | if !mountinfoLineHas(line, "/data/instances", "nosymfollow") { |
| 475 | t.Error("parser missed nosymfollow on a fixture line") |
| 476 | } |
| 477 | if mountinfoLineHas(line, "/data", "nosymfollow") { |
| 478 | t.Error("parser matched the wrong mountpoint") |
| 479 | } |
| 480 | } |
| 481 | ``` |
| 482 | |
| 483 | - [ ] **Step 2: Run the tests to verify they fail** |
| 484 | |
| 485 | Run: `GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/instancefs -count=1` |
| 486 | Expected: FAIL to build (`undefined: ReadBeneath` and so on; or "no Go files"). |
| 487 | |
| 488 | - [ ] **Step 3: Write the package** |
| 489 | |
| 490 | Create `internal/instancefs/instancefs.go`: |
| 491 | |
| 492 | ```go |
| 493 | // Package instancefs is how the agent opens anything inside an instance's |
| 494 | // own directory. Every exec-backed instance runs as the agent's uid, and |
| 495 | // its directory is writable by it, so any path the agent opens there is a |
| 496 | // request the instance makes on the agent's behalf: a symlink at |
| 497 | // <host>.env pointing at /data/litestream.env would hand the instance the |
| 498 | // bucket credentials as its own environment (isolated-exec design §1.3). |
| 499 | // |
| 500 | // The kernel closes that for every reader at once with the nosymfollow |
| 501 | // mount option on the instance trees (design §3.2). This package is the |
| 502 | // belt to that brace: it opens with openat2(RESOLVE_BENEATH | |
| 503 | // RESOLVE_NO_SYMLINKS | RESOLVE_NO_MAGICLINKS), so a symlink is refused |
| 504 | // with a named error even on a box whose mount is missing the option, and |
| 505 | // it writes through O_EXCL temp files and rename, so a planted symlink at |
| 506 | // a destination is replaced as a directory entry rather than written |
| 507 | // through. |
| 508 | // |
| 509 | // Nothing here is Linux-specific in its contract; the implementations |
| 510 | // are, and the !linux build returns errors.ErrUnsupported so the CLI still |
| 511 | // builds on a Mac (AGENTS.md's cross-compile gate). |
| 512 | package instancefs |
| 513 | |
| 514 | import ( |
| 515 | "errors" |
| 516 | "os" |
| 517 | "path/filepath" |
| 518 | "strings" |
| 519 | ) |
| 520 | |
| 521 | // ErrSymlink is returned when any component of the path is a symlink. |
| 522 | var ErrSymlink = errors.New("instancefs: path contains a symlink") |
| 523 | |
| 524 | // ErrEscapes is returned for a name that is absolute or reaches above dir. |
| 525 | var ErrEscapes = errors.New("instancefs: path escapes its directory") |
| 526 | |
| 527 | // cleanRel validates name as a relative path that stays inside dir. |
| 528 | func cleanRel(name string) (string, error) { |
| 529 | if name == "" || filepath.IsAbs(name) { |
| 530 | return "", ErrEscapes |
| 531 | } |
| 532 | c := filepath.Clean(name) |
| 533 | if c == ".." || strings.HasPrefix(c, "../") { |
| 534 | return "", ErrEscapes |
| 535 | } |
| 536 | return c, nil |
| 537 | } |
| 538 | |
| 539 | // ReadBeneath is OpenBeneath followed by reading the whole file. |
| 540 | func ReadBeneath(dir, name string) ([]byte, error) { |
| 541 | f, err := OpenBeneath(dir, name, os.O_RDONLY, 0) |
| 542 | if err != nil { |
| 543 | return nil, err |
| 544 | } |
| 545 | defer f.Close() |
| 546 | return readAll(f) |
| 547 | } |
| 548 | |
| 549 | func readAll(f *os.File) ([]byte, error) { |
| 550 | var out []byte |
| 551 | buf := make([]byte, 32*1024) |
| 552 | for { |
| 553 | n, err := f.Read(buf) |
| 554 | out = append(out, buf[:n]...) |
| 555 | if err != nil { |
| 556 | if err.Error() == "EOF" { |
| 557 | return out, nil |
| 558 | } |
| 559 | return out, err |
| 560 | } |
| 561 | if n == 0 { |
| 562 | return out, nil |
| 563 | } |
| 564 | } |
| 565 | } |
| 566 | ``` |
| 567 | |
| 568 | Replace the hand-rolled `readAll` with `io.ReadAll` (add `"io"` to imports and delete the helper). It is shown expanded only so the file compiles if `io` is forgotten; the committed version must use `io.ReadAll`. |
| 569 | |
| 570 | Create `internal/instancefs/instancefs_linux.go`: |
| 571 | |
| 572 | ```go |
| 573 | //go:build linux |
| 574 | |
| 575 | package instancefs |
| 576 | |
| 577 | import ( |
| 578 | "bufio" |
| 579 | "crypto/rand" |
| 580 | "encoding/hex" |
| 581 | "errors" |
| 582 | "fmt" |
| 583 | "os" |
| 584 | "path/filepath" |
| 585 | "strings" |
| 586 | |
| 587 | "golang.org/x/sys/unix" |
| 588 | ) |
| 589 | |
| 590 | const resolveFlags = unix.RESOLVE_BENEATH | unix.RESOLVE_NO_SYMLINKS | unix.RESOLVE_NO_MAGICLINKS |
| 591 | |
| 592 | // openDir opens dir itself as a directory fd, also refusing symlinks in |
| 593 | // dir's own components: the instance directory's parent is the agent's, |
| 594 | // but the instance directory itself could have been swapped for a |
| 595 | // symlink by a same-uid process before the mount option landed. |
| 596 | func openDir(dir string) (int, error) { |
| 597 | fd, err := unix.Open(dir, unix.O_PATH|unix.O_DIRECTORY|unix.O_NOFOLLOW|unix.O_CLOEXEC, 0) |
| 598 | if err != nil { |
| 599 | if errors.Is(err, unix.ELOOP) { |
| 600 | return -1, fmt.Errorf("%w: %s", ErrSymlink, dir) |
| 601 | } |
| 602 | return -1, &os.PathError{Op: "open", Path: dir, Err: err} |
| 603 | } |
| 604 | return fd, nil |
| 605 | } |
| 606 | |
| 607 | // OpenBeneath opens name under dir with openat2, following no symlink at |
| 608 | // any component and never leaving dir. |
| 609 | func OpenBeneath(dir, name string, flag int, perm os.FileMode) (*os.File, error) { |
| 610 | rel, err := cleanRel(name) |
| 611 | if err != nil { |
| 612 | return nil, err |
| 613 | } |
| 614 | dfd, err := openDir(dir) |
| 615 | if err != nil { |
| 616 | return nil, err |
| 617 | } |
| 618 | defer unix.Close(dfd) |
| 619 | how := unix.OpenHow{ |
| 620 | Flags: uint64(flag) | unix.O_CLOEXEC, |
| 621 | Mode: uint64(perm.Perm()), |
| 622 | Resolve: resolveFlags, |
| 623 | } |
| 624 | fd, err := unix.Openat2(dfd, rel, &how) |
| 625 | if err != nil { |
| 626 | switch { |
| 627 | case errors.Is(err, unix.ELOOP): |
| 628 | return nil, fmt.Errorf("%w: %s", ErrSymlink, filepath.Join(dir, rel)) |
| 629 | case errors.Is(err, unix.EXDEV): |
| 630 | return nil, fmt.Errorf("%w: %s", ErrEscapes, filepath.Join(dir, rel)) |
| 631 | } |
| 632 | return nil, &os.PathError{Op: "openat2", Path: filepath.Join(dir, rel), Err: err} |
| 633 | } |
| 634 | return os.NewFile(uintptr(fd), filepath.Join(dir, rel)), nil |
| 635 | } |
| 636 | |
| 637 | // WriteAtomic writes data to a fresh O_EXCL temp file beside name and |
| 638 | // renames it over name. rename(2) replaces the directory entry and never |
| 639 | // follows a symlink at the destination, so a planted link is displaced, |
| 640 | // not written through. The temp name is random, so no same-uid process |
| 641 | // can pre-plant it (the O_EXCL refuses it if one does). |
| 642 | func WriteAtomic(dir, name string, data []byte, perm os.FileMode) error { |
| 643 | rel, err := cleanRel(name) |
| 644 | if err != nil { |
| 645 | return err |
| 646 | } |
| 647 | if strings.Contains(rel, "/") { |
| 648 | return fmt.Errorf("%w: WriteAtomic takes a bare file name, got %q", ErrEscapes, name) |
| 649 | } |
| 650 | dfd, err := openDir(dir) |
| 651 | if err != nil { |
| 652 | return err |
| 653 | } |
| 654 | defer unix.Close(dfd) |
| 655 | var rnd [8]byte |
| 656 | if _, err := rand.Read(rnd[:]); err != nil { |
| 657 | return err |
| 658 | } |
| 659 | tmp := "." + rel + ".tmp-" + hex.EncodeToString(rnd[:]) |
| 660 | how := unix.OpenHow{ |
| 661 | Flags: unix.O_WRONLY | unix.O_CREAT | unix.O_EXCL | unix.O_CLOEXEC, |
| 662 | Mode: uint64(perm.Perm()), |
| 663 | Resolve: resolveFlags, |
| 664 | } |
| 665 | fd, err := unix.Openat2(dfd, tmp, &how) |
| 666 | if err != nil { |
| 667 | return &os.PathError{Op: "openat2", Path: filepath.Join(dir, tmp), Err: err} |
| 668 | } |
| 669 | f := os.NewFile(uintptr(fd), filepath.Join(dir, tmp)) |
| 670 | cleanup := func() { f.Close(); _ = unix.Unlinkat(dfd, tmp, 0) } |
| 671 | if _, err := f.Write(data); err != nil { |
| 672 | cleanup() |
| 673 | return err |
| 674 | } |
| 675 | if err := f.Sync(); err != nil { |
| 676 | cleanup() |
| 677 | return err |
| 678 | } |
| 679 | if err := f.Close(); err != nil { |
| 680 | _ = unix.Unlinkat(dfd, tmp, 0) |
| 681 | return err |
| 682 | } |
| 683 | if err := unix.Renameat(dfd, tmp, dfd, rel); err != nil { |
| 684 | _ = unix.Unlinkat(dfd, tmp, 0) |
| 685 | return &os.PathError{Op: "renameat", Path: filepath.Join(dir, rel), Err: err} |
| 686 | } |
| 687 | return nil |
| 688 | } |
| 689 | |
| 690 | // MountNoSymfollow reports whether mountpoint is a mountpoint whose |
| 691 | // options include nosymfollow, by reading /proc/self/mountinfo. A path |
| 692 | // that is not a mountpoint is (false, nil). |
| 693 | func MountNoSymfollow(mountpoint string) (bool, error) { |
| 694 | f, err := os.Open("/proc/self/mountinfo") |
| 695 | if err != nil { |
| 696 | return false, err |
| 697 | } |
| 698 | defer f.Close() |
| 699 | want := filepath.Clean(mountpoint) |
| 700 | sc := bufio.NewScanner(f) |
| 701 | for sc.Scan() { |
| 702 | if mountinfoLineHas(sc.Text(), want, "nosymfollow") { |
| 703 | return true, nil |
| 704 | } |
| 705 | } |
| 706 | return false, sc.Err() |
| 707 | } |
| 708 | |
| 709 | // mountinfoLineHas parses one mountinfo line: field 5 is the mountpoint |
| 710 | // (octal-escaped), field 6 the per-mount options. |
| 711 | func mountinfoLineHas(line, mountpoint, option string) bool { |
| 712 | fields := strings.Fields(line) |
| 713 | if len(fields) < 6 { |
| 714 | return false |
| 715 | } |
| 716 | if unescapeMountinfo(fields[4]) != mountpoint { |
| 717 | return false |
| 718 | } |
| 719 | for _, o := range strings.Split(fields[5], ",") { |
| 720 | if o == option { |
| 721 | return true |
| 722 | } |
| 723 | } |
| 724 | return false |
| 725 | } |
| 726 | |
| 727 | // unescapeMountinfo undoes the \040-style escapes mountinfo applies to |
| 728 | // spaces, tabs, newlines and backslashes in paths. |
| 729 | func unescapeMountinfo(s string) string { |
| 730 | if !strings.Contains(s, `\`) { |
| 731 | return s |
| 732 | } |
| 733 | var b strings.Builder |
| 734 | for i := 0; i < len(s); i++ { |
| 735 | if s[i] == '\\' && i+3 < len(s) { |
| 736 | var v int |
| 737 | if _, err := fmt.Sscanf(s[i+1:i+4], "%o", &v); err == nil { |
| 738 | b.WriteByte(byte(v)) |
| 739 | i += 3 |
| 740 | continue |
| 741 | } |
| 742 | } |
| 743 | b.WriteByte(s[i]) |
| 744 | } |
| 745 | return b.String() |
| 746 | } |
| 747 | ``` |
| 748 | |
| 749 | Create `internal/instancefs/instancefs_other.go`: |
| 750 | |
| 751 | ```go |
| 752 | //go:build !linux |
| 753 | |
| 754 | package instancefs |
| 755 | |
| 756 | import ( |
| 757 | "errors" |
| 758 | "os" |
| 759 | ) |
| 760 | |
| 761 | func OpenBeneath(dir, name string, flag int, perm os.FileMode) (*os.File, error) { |
| 762 | if _, err := cleanRel(name); err != nil { |
| 763 | return nil, err |
| 764 | } |
| 765 | return nil, errors.ErrUnsupported |
| 766 | } |
| 767 | |
| 768 | func WriteAtomic(dir, name string, data []byte, perm os.FileMode) error { |
| 769 | if _, err := cleanRel(name); err != nil { |
| 770 | return err |
| 771 | } |
| 772 | return errors.ErrUnsupported |
| 773 | } |
| 774 | |
| 775 | func MountNoSymfollow(mountpoint string) (bool, error) { return false, errors.ErrUnsupported } |
| 776 | |
| 777 | func mountinfoLineHas(line, mountpoint, option string) bool { return false } |
| 778 | ``` |
| 779 | |
| 780 | - [ ] **Step 4: Run the tests to verify they pass, and cross-compile** |
| 781 | |
| 782 | Run: `GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/instancefs -count=1` |
| 783 | Expected: PASS. |
| 784 | |
| 785 | Run: `GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off GOOS=darwin go build ./internal/instancefs && GOOS=windows go build ./internal/instancefs` |
| 786 | Expected: both build. (`RESOLVE_*` and `OpenHow` exist in x/sys v0.46.0 for linux; the other file never references them.) |
| 787 | |
| 788 | - [ ] **Step 5: gofmt and commit** |
| 789 | |
| 790 | ```bash |
| 791 | gofmt -l internal/instancefs |
| 792 | git add internal/instancefs |
| 793 | git commit -m "instancefs: symlink-refusing opens and atomic writes for instance directories (plan 1, task 2)" |
| 794 | ``` |
| 795 | |
| 796 | --- |
| 797 | |
| 798 | ### Task 3: Checked unix dials in the edge and the agent |
| 799 | |
| 800 | **Files:** |
| 801 | - Create: `internal/unixdial/unixdial.go` |
| 802 | - Create: `internal/unixdial/unixdial_test.go` |
| 803 | - Modify: `internal/edge/edge.go:2647-2664` (`dialUnixWithRetry`) |
| 804 | - Modify: `internal/activator/backend_exec.go:916-925` (`socketClient`) |
| 805 | - Modify: `internal/release/versions.go:364-374` (`unixHTTPClient`) |
| 806 | - Modify: `internal/host/socket_advice.go:48-55` (`socketServing`) |
| 807 | |
| 808 | **Interfaces:** |
| 809 | - Consumes: `provision.IsControlEndpoint`. |
| 810 | - Produces: |
| 811 | ```go |
| 812 | package unixdial |
| 813 | var ErrControlEndpoint = errors.New("unixdial: refusing to dial the instance control channel as an upstream") |
| 814 | var ErrPeerMismatch = errors.New("unixdial: listener is bound to a different path than the one dialled") |
| 815 | // Checked dials path as a unix stream socket and returns the connection only if |
| 816 | // (1) path is not under provision.ControlSocketDir and (2) getpeername on the |
| 817 | // connected socket equals filepath.Clean(path). Otherwise the connection is |
| 818 | // closed and the error names which check failed. |
| 819 | func Checked(ctx context.Context, d *net.Dialer, path string) (net.Conn, error) |
| 820 | ``` |
| 821 | |
| 822 | - [ ] **Step 1: Write the failing tests** |
| 823 | |
| 824 | Create `internal/unixdial/unixdial_test.go`: |
| 825 | |
| 826 | ```go |
| 827 | package unixdial |
| 828 | |
| 829 | import ( |
| 830 | "context" |
| 831 | "errors" |
| 832 | "net" |
| 833 | "os" |
| 834 | "path/filepath" |
| 835 | "testing" |
| 836 | ) |
| 837 | |
| 838 | // short returns a socket path short enough for sockaddr_un under the |
| 839 | // session's long TMPDIR (SDD lessons: run with TMPDIR=/tmp/g). |
| 840 | func short(t *testing.T, name string) string { |
| 841 | t.Helper() |
| 842 | dir, err := os.MkdirTemp("", "ud") |
| 843 | if err != nil { |
| 844 | t.Fatal(err) |
| 845 | } |
| 846 | t.Cleanup(func() { os.RemoveAll(dir) }) |
| 847 | return filepath.Join(dir, name) |
| 848 | } |
| 849 | |
| 850 | func listen(t *testing.T, path string) net.Listener { |
| 851 | t.Helper() |
| 852 | ln, err := net.Listen("unix", path) |
| 853 | if err != nil { |
| 854 | t.Skipf("cannot bind a unix socket here (%v); run with TMPDIR=/tmp/g outside the sandbox", err) |
| 855 | } |
| 856 | t.Cleanup(func() { ln.Close() }) |
| 857 | go func() { |
| 858 | for { |
| 859 | c, err := ln.Accept() |
| 860 | if err != nil { |
| 861 | return |
| 862 | } |
| 863 | c.Close() |
| 864 | } |
| 865 | }() |
| 866 | return ln |
| 867 | } |
| 868 | |
| 869 | func TestCheckedAcceptsTheListenerItDialled(t *testing.T) { |
| 870 | path := short(t, "app.sock") |
| 871 | listen(t, path) |
| 872 | c, err := Checked(context.Background(), &net.Dialer{}, path) |
| 873 | if err != nil { |
| 874 | t.Fatalf("Checked: %v", err) |
| 875 | } |
| 876 | c.Close() |
| 877 | } |
| 878 | |
| 879 | func TestCheckedRefusesASymlinkToAnotherListener(t *testing.T) { |
| 880 | real := short(t, "other.sock") |
| 881 | listen(t, real) |
| 882 | link := short(t, "app.sock") |
| 883 | if err := os.Symlink(real, link); err != nil { |
| 884 | t.Fatal(err) |
| 885 | } |
| 886 | c, err := Checked(context.Background(), &net.Dialer{}, link) |
| 887 | if c != nil { |
| 888 | c.Close() |
| 889 | } |
| 890 | if !errors.Is(err, ErrPeerMismatch) { |
| 891 | t.Fatalf("err = %v, want ErrPeerMismatch", err) |
| 892 | } |
| 893 | } |
| 894 | |
| 895 | func TestCheckedRefusesTheControlEndpointBeforeDialling(t *testing.T) { |
| 896 | // No listener exists; the refusal must come from the path check, not |
| 897 | // from a connect error. |
| 898 | _, err := Checked(context.Background(), &net.Dialer{}, "/run/carlos-control/control.sock") |
| 899 | if !errors.Is(err, ErrControlEndpoint) { |
| 900 | t.Fatalf("err = %v, want ErrControlEndpoint", err) |
| 901 | } |
| 902 | } |
| 903 | |
| 904 | func TestCheckedAcceptsAListenerCreatedByAnotherProcessShape(t *testing.T) { |
| 905 | // The carlos-app@.socket shape: the listener's creator is not the |
| 906 | // process that serves it. Credentials are never consulted, only the |
| 907 | // bound path, so this passes as long as the path matches. |
| 908 | path := short(t, "apps.sock") |
| 909 | listen(t, path) |
| 910 | c, err := Checked(context.Background(), &net.Dialer{}, path) |
| 911 | if err != nil { |
| 912 | t.Fatalf("Checked: %v", err) |
| 913 | } |
| 914 | c.Close() |
| 915 | } |
| 916 | |
| 917 | func TestCheckedAcceptsACustomPathOutsideRunCarlos(t *testing.T) { |
| 918 | // codex R5-1: an operator's --socket /run/titogo/example.sock keeps |
| 919 | // working; only the control endpoint is refused by path. |
| 920 | path := short(t, "example.sock") |
| 921 | listen(t, path) |
| 922 | c, err := Checked(context.Background(), &net.Dialer{}, path) |
| 923 | if err != nil { |
| 924 | t.Fatalf("Checked: %v", err) |
| 925 | } |
| 926 | c.Close() |
| 927 | } |
| 928 | |
| 929 | func TestCheckedPassesConnectErrorsThrough(t *testing.T) { |
| 930 | path := short(t, "absent.sock") |
| 931 | _, err := Checked(context.Background(), &net.Dialer{}, path) |
| 932 | if err == nil || errors.Is(err, ErrPeerMismatch) || errors.Is(err, ErrControlEndpoint) { |
| 933 | t.Fatalf("err = %v, want the raw connect error (ENOENT) so retry logic still sees it", err) |
| 934 | } |
| 935 | } |
| 936 | ``` |
| 937 | |
| 938 | - [ ] **Step 2: Run the tests to verify they fail** |
| 939 | |
| 940 | Run: `mkdir -p /tmp/g; TMPDIR=/tmp/g GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/unixdial -count=1` |
| 941 | Expected: build failure, `undefined: Checked`. |
| 942 | |
| 943 | - [ ] **Step 3: Write the package** |
| 944 | |
| 945 | Create `internal/unixdial/unixdial.go`: |
| 946 | |
| 947 | ```go |
| 948 | // Package unixdial is the one way the edge and the agent dial a unix |
| 949 | // socket. Two checks ride on every dial (isolated-exec design §5.4): |
| 950 | // |
| 951 | // - the reserved control endpoint is refused BEFORE connect(): a registry |
| 952 | // row pointing rt.Socket at the root control channel must never be |
| 953 | // proxied to, however it got there; |
| 954 | // - the listener must answer to the path that was dialled: after |
| 955 | // connect() and before a byte is sent, getpeername() names the path |
| 956 | // the LISTENER bound, which is its real path however it was reached. |
| 957 | // A symlink at /run/carlos/<host>.sock pointing at a sibling's socket |
| 958 | // or at the control socket therefore yields a mismatch, and the |
| 959 | // connection is closed unused. |
| 960 | // |
| 961 | // Credentials are deliberately not consulted: a listener systemd created |
| 962 | // and passed to a DynamicUser service reports root in SO_PEERCRED, and |
| 963 | // that shape (carlos-app@.socket) must keep working (codex R4-1). |
| 964 | package unixdial |
| 965 | |
| 966 | import ( |
| 967 | "context" |
| 968 | "errors" |
| 969 | "fmt" |
| 970 | "net" |
| 971 | "path/filepath" |
| 972 | |
| 973 | "github.com/carlosframework/platform/internal/provision" |
| 974 | ) |
| 975 | |
| 976 | // ErrControlEndpoint is returned, before any connect, for a path under |
| 977 | // provision.ControlSocketDir. |
| 978 | var ErrControlEndpoint = errors.New("unixdial: refusing to dial the instance control channel as an upstream") |
| 979 | |
| 980 | // ErrPeerMismatch is returned, after connect and before any byte is sent, |
| 981 | // when the listener's bound path is not the path that was dialled. |
| 982 | var ErrPeerMismatch = errors.New("unixdial: listener is bound to a different path than the one dialled") |
| 983 | |
| 984 | // Checked dials path as a unix stream socket with d and applies both |
| 985 | // checks. Connect errors (ENOENT, ECONNREFUSED) are returned unwrapped so |
| 986 | // callers' retry logic keeps seeing them. |
| 987 | func Checked(ctx context.Context, d *net.Dialer, path string) (net.Conn, error) { |
| 988 | if provision.IsControlEndpoint(path) { |
| 989 | return nil, fmt.Errorf("%w: %s", ErrControlEndpoint, path) |
| 990 | } |
| 991 | if d == nil { |
| 992 | d = &net.Dialer{} |
| 993 | } |
| 994 | conn, err := d.DialContext(ctx, "unix", path) |
| 995 | if err != nil { |
| 996 | return nil, err |
| 997 | } |
| 998 | if err := verifyPeer(conn, path); err != nil { |
| 999 | conn.Close() |
| 1000 | return nil, err |
| 1001 | } |
| 1002 | return conn, nil |
| 1003 | } |
| 1004 | |
| 1005 | // verifyPeer compares the peer address of a connected unix socket with the |
| 1006 | // dialled path. net's UnixConn fills RemoteAddr from getpeername(2) at |
| 1007 | // connect time, which is the listener's bound address. |
| 1008 | func verifyPeer(conn net.Conn, dialled string) error { |
| 1009 | ra := conn.RemoteAddr() |
| 1010 | ua, ok := ra.(*net.UnixAddr) |
| 1011 | if !ok || ua == nil { |
| 1012 | return fmt.Errorf("%w: peer address %v is not a unix address", ErrPeerMismatch, ra) |
| 1013 | } |
| 1014 | want := filepath.Clean(dialled) |
| 1015 | got := ua.Name |
| 1016 | if got == "" || got[0] == '@' || filepath.Clean(got) != want { |
| 1017 | return fmt.Errorf("%w: dialled %s, listener is bound to %q", ErrPeerMismatch, want, got) |
| 1018 | } |
| 1019 | return nil |
| 1020 | } |
| 1021 | ``` |
| 1022 | |
| 1023 | Note for the implementer: Go's `net` package sets a dialed `UnixConn`'s remote address from `getpeername(2)` after `connect(2)` succeeds (`netFD.connect` returns the peer sockaddr and `dial` installs it). If a test shows `RemoteAddr()` echoing the dialled symlink path instead of the bound path, replace `verifyPeer`'s body with an explicit `unix.Getpeername` through `conn.(*net.UnixConn).SyscallConn().Control(...)` in a `_linux.go` file with a `!linux` stub that returns nil, and keep the tests as they are. The tests are the contract. |
| 1024 | |
| 1025 | - [ ] **Step 4: Run the tests to verify they pass** |
| 1026 | |
| 1027 | Run: `TMPDIR=/tmp/g GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/unixdial -count=1 -v` |
| 1028 | Expected: PASS for all six (they skip, not fail, where the sandbox refuses `socket()`; run outside the sandbox to see them pass). |
| 1029 | |
| 1030 | - [ ] **Step 5: Wire the four dial sites** |
| 1031 | |
| 1032 | `internal/edge/edge.go:2647`: replace the body of `dialUnixWithRetry`'s dial line. |
| 1033 | |
| 1034 | ```go |
| 1035 | func dialUnixWithRetry(ctx context.Context, d *net.Dialer, socketPath string) (net.Conn, error) { |
| 1036 | const window = 13 * time.Second |
| 1037 | deadline := time.Now().Add(window) |
| 1038 | for { |
| 1039 | conn, err := unixdial.Checked(ctx, d, socketPath) |
| 1040 | if err == nil { |
| 1041 | return conn, nil |
| 1042 | } |
| 1043 | if ctx.Err() != nil || !isRetryableDialErr(err) || time.Now().After(deadline) { |
| 1044 | return nil, err |
| 1045 | } |
| 1046 | select { |
| 1047 | case <-ctx.Done(): |
| 1048 | return nil, ctx.Err() |
| 1049 | case <-time.After(100 * time.Millisecond): |
| 1050 | } |
| 1051 | } |
| 1052 | } |
| 1053 | ``` |
| 1054 | |
| 1055 | `isRetryableDialErr` is unchanged: `ErrControlEndpoint` and `ErrPeerMismatch` are not `ECONNREFUSED`/`ENOENT`, so they are not retried. Add the import `"github.com/carlosframework/platform/internal/unixdial"`. |
| 1056 | |
| 1057 | `internal/activator/backend_exec.go:916`: |
| 1058 | |
| 1059 | ```go |
| 1060 | func socketClient(socket string) *http.Client { |
| 1061 | return &http.Client{ |
| 1062 | Timeout: 3 * time.Second, |
| 1063 | Transport: &http.Transport{ |
| 1064 | DialContext: func(ctx context.Context, _, _ string) (net.Conn, error) { |
| 1065 | return unixdial.Checked(ctx, &net.Dialer{}, socket) |
| 1066 | }, |
| 1067 | }, |
| 1068 | } |
| 1069 | } |
| 1070 | ``` |
| 1071 | |
| 1072 | `internal/release/versions.go:364`: |
| 1073 | |
| 1074 | ```go |
| 1075 | func unixHTTPClient(socket string) *http.Client { |
| 1076 | return &http.Client{ |
| 1077 | Timeout: 3 * time.Second, |
| 1078 | Transport: &http.Transport{ |
| 1079 | DialContext: func(ctx context.Context, _, _ string) (net.Conn, error) { |
| 1080 | return unixdial.Checked(ctx, &net.Dialer{}, socket) |
| 1081 | }, |
| 1082 | }, |
| 1083 | } |
| 1084 | } |
| 1085 | ``` |
| 1086 | |
| 1087 | `internal/host/socket_advice.go:48`: |
| 1088 | |
| 1089 | ```go |
| 1090 | func socketServing(socket string) bool { |
| 1091 | ctx, cancel := context.WithTimeout(context.Background(), 500*time.Millisecond) |
| 1092 | defer cancel() |
| 1093 | c, err := unixdial.Checked(ctx, &net.Dialer{}, socket) |
| 1094 | if err != nil { |
| 1095 | return false |
| 1096 | } |
| 1097 | c.Close() |
| 1098 | return true |
| 1099 | } |
| 1100 | ``` |
| 1101 | |
| 1102 | Add `"context"` and the `unixdial` import there if missing. Check for an import cycle: `unixdial` imports `provision`; `provision` must not import `edge`, `activator`, `release` or `host` (it does not today: `grep -n '"github.com/carlosframework/platform/internal/' internal/provision/*.go`). If `release` is imported by `provision` (it is, for the store), that is fine: `release` importing `unixdial` importing `provision` importing `release` WOULD be a cycle. **Check `go build ./...` after wiring `release/versions.go`; if it reports an import cycle, move `IsControlEndpoint`, `ControlSocketDir` and `ControlSocketPath` out of `provision` into a new leaf package `internal/controlsock` with no imports, have `provision` re-export them as constants (`const ControlSocketDir = controlsock.Dir`), and have `unixdial` import `controlsock` only.** Update Task 1's tests only if the names move; the values do not. |
| 1103 | |
| 1104 | - [ ] **Step 6: Build, vet, run the touched packages** |
| 1105 | |
| 1106 | Run: `GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go build ./... && go vet ./internal/edge ./internal/activator ./internal/release ./internal/host ./internal/unixdial` |
| 1107 | Expected: clean. |
| 1108 | |
| 1109 | Run: `TMPDIR=/tmp/g GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/edge ./internal/activator ./internal/release ./internal/host -count=1` |
| 1110 | Expected: PASS (these bind sockets; run outside the sandbox with `TMPDIR=/tmp/g`). |
| 1111 | |
| 1112 | - [ ] **Step 7: gofmt and commit** |
| 1113 | |
| 1114 | ```bash |
| 1115 | gofmt -l internal |
| 1116 | git add internal/unixdial internal/edge/edge.go internal/activator/backend_exec.go internal/release/versions.go internal/host/socket_advice.go |
| 1117 | git commit -m "unixdial: refuse the control endpoint and verify the listener's bound path on every unix dial (plan 1, task 3)" |
| 1118 | ``` |
| 1119 | |
| 1120 | --- |
| 1121 | |
| 1122 | ### Task 4: Slice and mount units, and the edge unit's move into `carlos.slice` |
| 1123 | |
| 1124 | **Files:** |
| 1125 | - Create: `internal/host/units/carlos.slice` |
| 1126 | - Create: `internal/host/units/carlos-instances.slice` |
| 1127 | - Create: `internal/host/units/data-instances.mount` |
| 1128 | - Create: `internal/host/units/run-carlos-instances.mount` |
| 1129 | - Create: `internal/host/units/run-carlos-logs.mount` |
| 1130 | - Modify: `internal/host/units/carlos-edge.service` (`[Unit]` lines 6-9; `[Service]` lines 71-79) |
| 1131 | - Modify: `internal/host/units/carlos-litestream.service` (`[Unit]`, line 12) |
| 1132 | - Modify: `internal/host/bootstrap.go:33` (embed glob) |
| 1133 | - Test: `internal/host/units_isolation_test.go` (create) |
| 1134 | |
| 1135 | **Interfaces:** |
| 1136 | - Consumes: nothing from earlier tasks. |
| 1137 | - Produces: the unit names `carlos.slice`, `carlos-instances.slice`, `data-instances.mount`, `run-carlos-instances.mount`, `run-carlos-logs.mount`, which Task 5 enables, and which plan 2's templates `Requires=`. |
| 1138 | |
| 1139 | - [ ] **Step 1: Write the failing tests** |
| 1140 | |
| 1141 | Create `internal/host/units_isolation_test.go`: |
| 1142 | |
| 1143 | ```go |
| 1144 | package host |
| 1145 | |
| 1146 | import ( |
| 1147 | "strings" |
| 1148 | "testing" |
| 1149 | ) |
| 1150 | |
| 1151 | func readUnit(t *testing.T, name string) string { |
| 1152 | t.Helper() |
| 1153 | b, err := unitsFS.ReadFile("units/" + name) |
| 1154 | if err != nil { |
| 1155 | t.Fatalf("%s: %v", name, err) |
| 1156 | } |
| 1157 | return string(b) |
| 1158 | } |
| 1159 | |
| 1160 | // The outer memory wall moves from the edge unit to carlos.slice |
| 1161 | // (isolated-exec design §4.3): it bounds the agent plus every awake |
| 1162 | // instance as before, but instances are now sibling units, not children. |
| 1163 | func TestCarlosSliceCarriesTheOuterWall(t *testing.T) { |
| 1164 | s := readUnit(t, "carlos.slice") |
| 1165 | for _, want := range []string{"[Slice]", "MemoryHigh=70%", "MemoryMax=80%"} { |
| 1166 | if !strings.Contains(s, want) { |
| 1167 | t.Errorf("carlos.slice lacks %q", want) |
| 1168 | } |
| 1169 | } |
| 1170 | e := readUnit(t, "carlos-edge.service") |
| 1171 | if !strings.Contains(e, "Slice=carlos.slice") { |
| 1172 | t.Error("carlos-edge.service is not in carlos.slice") |
| 1173 | } |
| 1174 | for _, never := range []string{"MemoryHigh=", "MemoryMax="} { |
| 1175 | if strings.Contains(e, never) { |
| 1176 | t.Errorf("carlos-edge.service still carries %q; the wall is the slice's now", never) |
| 1177 | } |
| 1178 | } |
| 1179 | // What must NOT change on the edge unit. |
| 1180 | for _, keep := range []string{"OOMPolicy=continue", "Delegate=cpu memory io pids", "User=carlos", |
| 1181 | "AmbientCapabilities=CAP_NET_BIND_SERVICE", "ReadWritePaths=/data /run/carlos", "RestrictNamespaces=yes"} { |
| 1182 | if !strings.Contains(e, keep) { |
| 1183 | t.Errorf("carlos-edge.service lost %q", keep) |
| 1184 | } |
| 1185 | } |
| 1186 | i := readUnit(t, "carlos-instances.slice") |
| 1187 | if !strings.Contains(i, "[Slice]") { |
| 1188 | t.Error("carlos-instances.slice has no [Slice] section") |
| 1189 | } |
| 1190 | for _, never := range []string{"MemoryHigh=", "MemoryMax="} { |
| 1191 | if strings.Contains(i, never) { |
| 1192 | t.Errorf("carlos-instances.slice carries %q; the walls are carlos.slice and the per-host slices", never) |
| 1193 | } |
| 1194 | } |
| 1195 | } |
| 1196 | |
| 1197 | // The three mounts: a bind of a raw directory onto its protected view, |
| 1198 | // nosymfollow, with DefaultDependencies=no and explicit ordering, because |
| 1199 | // the default ordering (before local-fs.target) and tmpfiles (after it) |
| 1200 | // would form a cycle systemd breaks by dropping the mount (codex R2-2). |
| 1201 | func TestInstanceMountsAreShapedForNosymfollow(t *testing.T) { |
| 1202 | cases := map[string][2]string{ |
| 1203 | "data-instances.mount": {"/data/.instances", "/data/instances"}, |
| 1204 | "run-carlos-instances.mount": {"/run/carlos/.instances", "/run/carlos/instances"}, |
| 1205 | "run-carlos-logs.mount": {"/run/carlos/.logs", "/run/carlos/logs"}, |
| 1206 | } |
| 1207 | for name, ww := range cases { |
| 1208 | m := readUnit(t, name) |
| 1209 | for _, want := range []string{ |
| 1210 | "What=" + ww[0], "Where=" + ww[1], "Type=none", |
| 1211 | "Options=bind,nosymfollow,nosuid,nodev,noexec", |
| 1212 | "DefaultDependencies=no", |
| 1213 | "After=", "systemd-tmpfiles-setup.service", |
| 1214 | "Before=", "carlos-edge.service", |
| 1215 | "Conflicts=umount.target", "Before=umount.target", |
| 1216 | "WantedBy=multi-user.target", |
| 1217 | } { |
| 1218 | if !strings.Contains(m, want) { |
| 1219 | t.Errorf("%s lacks %q", name, want) |
| 1220 | } |
| 1221 | } |
| 1222 | if ww[0] == ww[1] { |
| 1223 | t.Errorf("%s binds a directory onto itself; the migration rename would fail with EXDEV (codex R4-2)", name) |
| 1224 | } |
| 1225 | } |
| 1226 | d := readUnit(t, "data-instances.mount") |
| 1227 | for _, want := range []string{"RequiresMountsFor=/data", "local-fs.target", "carlos-litestream.service"} { |
| 1228 | if !strings.Contains(d, want) { |
| 1229 | t.Errorf("data-instances.mount lacks %q", want) |
| 1230 | } |
| 1231 | } |
| 1232 | e := readUnit(t, "carlos-edge.service") |
| 1233 | for _, want := range []string{"Requires=data-instances.mount run-carlos-instances.mount run-carlos-logs.mount", |
| 1234 | "After=network-online.target data-instances.mount run-carlos-instances.mount run-carlos-logs.mount"} { |
| 1235 | if !strings.Contains(e, want) { |
| 1236 | t.Errorf("carlos-edge.service lacks %q: an edge that starts without the mounts must not start", want) |
| 1237 | } |
| 1238 | } |
| 1239 | l := readUnit(t, "carlos-litestream.service") |
| 1240 | if !strings.Contains(l, "After=network-online.target data-instances.mount") { |
| 1241 | t.Error("carlos-litestream.service is not ordered after data-instances.mount") |
| 1242 | } |
| 1243 | } |
| 1244 | |
| 1245 | // Every unit file, whatever its suffix, is embedded: a .slice or .mount |
| 1246 | // left out of the glob would be silently absent from every box. |
| 1247 | func TestEmbeddedUnitsIncludeSlicesAndMounts(t *testing.T) { |
| 1248 | names, err := unitNames() |
| 1249 | if err != nil { |
| 1250 | t.Fatal(err) |
| 1251 | } |
| 1252 | have := map[string]bool{} |
| 1253 | for _, n := range names { |
| 1254 | have[n] = true |
| 1255 | } |
| 1256 | for _, want := range []string{"carlos.slice", "carlos-instances.slice", "data-instances.mount", "run-carlos-instances.mount", "run-carlos-logs.mount"} { |
| 1257 | if !have[want] { |
| 1258 | t.Errorf("%s is not embedded (bootstrap.go's go:embed glob)", want) |
| 1259 | } |
| 1260 | } |
| 1261 | } |
| 1262 | ``` |
| 1263 | |
| 1264 | - [ ] **Step 2: Run the tests to verify they fail** |
| 1265 | |
| 1266 | Run: `GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/host -run 'TestCarlosSlice|TestInstanceMounts|TestEmbeddedUnits' -count=1` |
| 1267 | Expected: FAIL (`carlos.slice: open units/carlos.slice: file does not exist`, and so on). |
| 1268 | |
| 1269 | - [ ] **Step 3: Write the unit files** |
| 1270 | |
| 1271 | `internal/host/units/carlos.slice`: |
| 1272 | |
| 1273 | ```ini |
| 1274 | # The outer memory wall of the memory-pressure containment design |
| 1275 | # (2026-08-28 §2.2), moved here from carlos-edge.service by the isolated-exec |
| 1276 | # design (§4.3): exec-backed instances used to be the edge's children, so a |
| 1277 | # ceiling on the edge unit bounded the agent plus every awake instance at |
| 1278 | # once. They are sibling units now, so the ceiling lives one level up, on |
| 1279 | # a slice that holds the edge AND carlos-instances.slice, and bounds exactly |
| 1280 | # what it bounded before. Borrowing between agent and tenants and the OOM |
| 1281 | # candidate set (a tenant at OOMScoreAdjust=500 is picked before the agent) |
| 1282 | # are unchanged. |
| 1283 | # |
| 1284 | # Percentages so one file fits a t4g.micro and a t4g.medium; 80% leaves what |
| 1285 | # the box's other units (keymail, console, ssm, litestream, journald) use. |
| 1286 | [Unit] |
| 1287 | Description=carlos (edge, activator and every exec-backed instance) |
| 1288 | |
| 1289 | [Slice] |
| 1290 | MemoryHigh=70% |
| 1291 | MemoryMax=80% |
| 1292 | ``` |
| 1293 | |
| 1294 | `internal/host/units/carlos-instances.slice`: |
| 1295 | |
| 1296 | ```ini |
| 1297 | # The parent of every per-host slice (carlos-instances-<iid>.slice, created |
| 1298 | # on demand by the first carlos-instance*@ unit that names it; isolated-exec |
| 1299 | # design §4.3). No cap of its own: the outer wall is carlos.slice, the inner |
| 1300 | # walls are the per-host slices the control service sets the tier on. This |
| 1301 | # level exists so the sampler has one root to scan and so an operator can |
| 1302 | # `systemctl status carlos-instances.slice`. |
| 1303 | [Unit] |
| 1304 | Description=carlos exec-backed instances |
| 1305 | |
| 1306 | [Slice] |
| 1307 | ``` |
| 1308 | |
| 1309 | `internal/host/units/data-instances.mount`: |
| 1310 | |
| 1311 | ```ini |
| 1312 | # The protected view of the instance data tree (isolated-exec design §3.2). |
| 1313 | # |
| 1314 | # /data/.instances is the RAW directory, opened by the layout pass only; |
| 1315 | # this bind of it onto /data/instances is the view everything else uses. |
| 1316 | # nosymfollow is the isolation's belt: path resolution under this mount |
| 1317 | # returns ELOOP at any symlink, for every process on the box, in every |
| 1318 | # namespace that sees the mount, with no check-then-use race. An instance |
| 1319 | # runs as the agent's uid and can write its own directory, so any symlink |
| 1320 | # it plants would otherwise be followed by the agent (reading a sibling's |
| 1321 | # env file as this instance's), by the edge (dialling a sibling's socket |
| 1322 | # as this host) and by PID 1. Here none of them follow it. |
| 1323 | # |
| 1324 | # It is a bind of a DIFFERENT directory, not of itself, because rename(2) |
| 1325 | # refuses to cross a mount boundary (EXDEV) even on one filesystem, and |
| 1326 | # the migration moves /data/<host>.db into this tree by rename (codex R4-2). |
| 1327 | # |
| 1328 | # DefaultDependencies=no, with the ordering written out: a mount unit's |
| 1329 | # default place is before local-fs.target, and systemd-tmpfiles-setup, |
| 1330 | # which creates the two directories, runs after it, so the defaults form a |
| 1331 | # cycle systemd breaks by dropping this mount (codex R2-2). Shutdown |
| 1332 | # ordering is written out for the same reason. |
| 1333 | [Unit] |
| 1334 | Description=carlos instance data (bind, nosymfollow) |
| 1335 | DefaultDependencies=no |
| 1336 | RequiresMountsFor=/data |
| 1337 | After=local-fs.target systemd-tmpfiles-setup.service |
| 1338 | Before=carlos-edge.service carlos-instance-control.service carlos-litestream.service |
| 1339 | Conflicts=umount.target |
| 1340 | Before=umount.target |
| 1341 | |
| 1342 | [Mount] |
| 1343 | What=/data/.instances |
| 1344 | Where=/data/instances |
| 1345 | Type=none |
| 1346 | Options=bind,nosymfollow,nosuid,nodev,noexec |
| 1347 | |
| 1348 | [Install] |
| 1349 | WantedBy=multi-user.target |
| 1350 | ``` |
| 1351 | |
| 1352 | `internal/host/units/run-carlos-instances.mount`: |
| 1353 | |
| 1354 | ```ini |
| 1355 | # The protected view of the instance socket tree (isolated-exec design |
| 1356 | # §3.2): /run/carlos/.instances bound onto /run/carlos/instances with |
| 1357 | # nosymfollow. See data-instances.mount for why a bind, why of a different |
| 1358 | # directory, and why DefaultDependencies=no. Nothing is ever renamed into |
| 1359 | # this tree, so the raw directory has no caller at all. /run is tmpfs, so |
| 1360 | # the two directories come from the tmpfiles line every boot, which is why |
| 1361 | # this is ordered after systemd-tmpfiles-setup.service. |
| 1362 | [Unit] |
| 1363 | Description=carlos instance sockets (bind, nosymfollow) |
| 1364 | DefaultDependencies=no |
| 1365 | After=systemd-tmpfiles-setup.service |
| 1366 | Before=carlos-edge.service carlos-instance-control.service |
| 1367 | Conflicts=umount.target |
| 1368 | Before=umount.target |
| 1369 | |
| 1370 | [Mount] |
| 1371 | What=/run/carlos/.instances |
| 1372 | Where=/run/carlos/instances |
| 1373 | Type=none |
| 1374 | Options=bind,nosymfollow,nosuid,nodev,noexec |
| 1375 | |
| 1376 | [Install] |
| 1377 | WantedBy=multi-user.target |
| 1378 | ``` |
| 1379 | |
| 1380 | `internal/host/units/run-carlos-logs.mount`: |
| 1381 | |
| 1382 | ```ini |
| 1383 | # The protected view of the agent's per-role log sockets (isolated-exec |
| 1384 | # design §3.2, §6.4): /run/carlos/.logs bound onto /run/carlos/logs with |
| 1385 | # nosymfollow. This directory is bound into NO instance namespace; the |
| 1386 | # mount option is for the transition release, in which unisolated flat |
| 1387 | # tenants still share the agent's uid and could otherwise turn a log |
| 1388 | # socket path into a symlink that PID 1 opens as root for some unit's |
| 1389 | # stdout (codex R3-1). See data-instances.mount for the ordering. |
| 1390 | [Unit] |
| 1391 | Description=carlos instance log sockets (bind, nosymfollow) |
| 1392 | DefaultDependencies=no |
| 1393 | After=systemd-tmpfiles-setup.service |
| 1394 | Before=carlos-edge.service carlos-instance-control.service |
| 1395 | Conflicts=umount.target |
| 1396 | Before=umount.target |
| 1397 | |
| 1398 | [Mount] |
| 1399 | What=/run/carlos/.logs |
| 1400 | Where=/run/carlos/logs |
| 1401 | Type=none |
| 1402 | Options=bind,nosymfollow,nosuid,nodev,noexec |
| 1403 | |
| 1404 | [Install] |
| 1405 | WantedBy=multi-user.target |
| 1406 | ``` |
| 1407 | |
| 1408 | - [ ] **Step 4: Edit the edge and litestream units** |
| 1409 | |
| 1410 | `internal/host/units/carlos-edge.service`, `[Unit]` section (lines 6-9) becomes: |
| 1411 | |
| 1412 | ```ini |
| 1413 | [Unit] |
| 1414 | Description=carlos edge (TLS terminator, host router, activator, release adoption) |
| 1415 | # The three nosymfollow mounts of the isolated-exec design (§3.2) are hard |
| 1416 | # requirements: an edge that starts without them would start unprotected, |
| 1417 | # so it does not start at all. |
| 1418 | Requires=data-instances.mount run-carlos-instances.mount run-carlos-logs.mount |
| 1419 | After=network-online.target data-instances.mount run-carlos-instances.mount run-carlos-logs.mount |
| 1420 | Wants=network-online.target |
| 1421 | ``` |
| 1422 | |
| 1423 | In `[Service]`, replace lines 71-72 (`MemoryHigh=70%` and `MemoryMax=80%`) and the comment block above them (the block that begins `# The outer wall (memory-pressure spec 2026-08-28 §2.2)`) with: |
| 1424 | |
| 1425 | ```ini |
| 1426 | # The outer wall (memory-pressure spec 2026-08-28 §2.2) used to be two |
| 1427 | # MemoryHigh/MemoryMax lines here, bounding the agent plus its exec |
| 1428 | # children. Instances are sibling units now (isolated-exec design §4.3), so |
| 1429 | # the wall is carlos.slice's, which holds this unit and every instance and |
| 1430 | # bounds exactly what it bounded before. The oom_score_adj story is |
| 1431 | # unchanged: tenants carry OOMScoreAdjust=500 in their template. |
| 1432 | Slice=carlos.slice |
| 1433 | ``` |
| 1434 | |
| 1435 | Keep `OOMPolicy=continue` and its comment, `TasksMax=1024`, and `Delegate=` exactly as they are. |
| 1436 | |
| 1437 | `internal/host/units/carlos-litestream.service`, line 12: `After=network-online.target` becomes `After=network-online.target data-instances.mount` (the replicator reads nested databases through the protected view; it must see the mount). |
| 1438 | |
| 1439 | - [ ] **Step 5: Widen the embed glob** |
| 1440 | |
| 1441 | `internal/host/bootstrap.go:33`: |
| 1442 | |
| 1443 | ```go |
| 1444 | //go:embed units/*.service units/*.timer units/*.socket units/*.path units/*.slice units/*.mount |
| 1445 | var unitsFS embed.FS |
| 1446 | ``` |
| 1447 | |
| 1448 | - [ ] **Step 6: Run the tests to verify they pass** |
| 1449 | |
| 1450 | Run: `GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/host -run 'TestCarlosSlice|TestInstanceMounts|TestEmbeddedUnits|TestBootstrap|TestSystemUnits|TestAppTemplate|TestVerifyDrills' -count=1` |
| 1451 | Expected: PASS. If an existing test asserts the edge unit's `MemoryMax=80%` (search: `grep -n 'MemoryMax=80' internal/host/*_test.go internal/caps/*_test.go`), move that assertion to `carlos.slice` in that test and keep the rest of it. |
| 1452 | |
| 1453 | - [ ] **Step 7: Commit** |
| 1454 | |
| 1455 | ```bash |
| 1456 | git add internal/host/units/carlos.slice internal/host/units/carlos-instances.slice internal/host/units/data-instances.mount internal/host/units/run-carlos-instances.mount internal/host/units/run-carlos-logs.mount internal/host/units/carlos-edge.service internal/host/units/carlos-litestream.service internal/host/bootstrap.go internal/host/units_isolation_test.go |
| 1457 | git commit -m "units: carlos.slice outer wall and the three nosymfollow instance mounts (plan 1, task 4)" |
| 1458 | ``` |
| 1459 | |
| 1460 | --- |
| 1461 | |
| 1462 | ### Task 5: Bootstrap wiring: tmpfiles lines, directories, arming |
| 1463 | |
| 1464 | **Files:** |
| 1465 | - Modify: `internal/host/bootstrap.go` (`tmpfilesConf` at line 61; `enabledUnits` at line 116; `dirs` at line 296; `systemctlPlan` at line 165) |
| 1466 | - Modify: `internal/host/host_test.go:1195` (`TestBootstrapPlanJSONNamesWhatALiveRunArms`) |
| 1467 | - Test: `internal/host/units_isolation_test.go` (append) |
| 1468 | |
| 1469 | **Interfaces:** |
| 1470 | - Consumes: the unit names from Task 4. |
| 1471 | - Produces: `tmpfilesConf` lines for `/data/.instances`, `/data/instances`, `/data/layout`, `/run/carlos/.instances`, `/run/carlos/instances`, `/run/carlos/.logs`, `/run/carlos/logs`, `/run/carlos-control`, and the probe fixture; `systemctlPlan` starting `.socket` and `.mount` units; the mounts in `enabledUnits`. |
| 1472 | |
| 1473 | - [ ] **Step 1: Write the failing tests** |
| 1474 | |
| 1475 | Append to `internal/host/units_isolation_test.go`: |
| 1476 | |
| 1477 | ```go |
| 1478 | // tmpfiles creates the raw directories and the mountpoints every boot |
| 1479 | // (/run is tmpfs), with the modes the design gives them (§3.1, §3.2): |
| 1480 | // the socket trees keep the carlos-sockets group and setgid (issue #103), |
| 1481 | // the log tree and data trees are the agent's alone, and the control |
| 1482 | // directory is root's so no carlos process can create anything in it. |
| 1483 | func TestTmpfilesCreatesTheInstanceTrees(t *testing.T) { |
| 1484 | for _, want := range []string{ |
| 1485 | "d /data/.instances 0700 carlos carlos -", |
| 1486 | "d /data/instances 0755 carlos carlos -", |
| 1487 | "d /data/layout 0700 carlos carlos -", |
| 1488 | "d /run/carlos/.instances 2750 carlos carlos-sockets -", |
| 1489 | "d /run/carlos/instances 2750 carlos carlos-sockets -", |
| 1490 | "d /run/carlos/.logs 0700 carlos carlos -", |
| 1491 | "d /run/carlos/logs 0700 carlos carlos -", |
| 1492 | "d /run/carlos-control 0750 root carlos -", |
| 1493 | "f /etc/carlos/probe-secret 0640 root carlos - probe", |
| 1494 | } { |
| 1495 | if !strings.Contains(tmpfilesConf, want+"\n") { |
| 1496 | t.Errorf("tmpfilesConf lacks %q", want) |
| 1497 | } |
| 1498 | } |
| 1499 | // The existing lines stay: the flat socket dir and the apps subtree. |
| 1500 | for _, keep := range []string{"d /run/carlos 2750 carlos carlos-sockets -", "d /run/carlos/apps 0750 carlos carlos -"} { |
| 1501 | if !strings.Contains(tmpfilesConf, keep+"\n") { |
| 1502 | t.Errorf("tmpfilesConf lost %q", keep) |
| 1503 | } |
| 1504 | } |
| 1505 | } |
| 1506 | |
| 1507 | // bootstrap -root <dir> creates the same directories in the staging tree |
| 1508 | // (a live run's chown -R covers them), so a fresh box has them before the |
| 1509 | // mounts try to bind them. |
| 1510 | func TestBootstrapCreatesTheInstanceDirectories(t *testing.T) { |
| 1511 | root := t.TempDir() |
| 1512 | if err := bootstrap(root, nil); err != nil { |
| 1513 | t.Fatalf("bootstrap: %v", err) |
| 1514 | } |
| 1515 | for _, d := range []string{"data/.instances", "data/instances", "data/layout", "run/carlos/.instances", "run/carlos/instances", "run/carlos/.logs", "run/carlos/logs", "run/carlos-control"} { |
| 1516 | if fi, err := os.Stat(filepath.Join(root, d)); err != nil || !fi.IsDir() { |
| 1517 | t.Errorf("%s: %v", d, err) |
| 1518 | } |
| 1519 | } |
| 1520 | } |
| 1521 | |
| 1522 | // The mounts are enabled AND started by the plan: `systemctl enable` on a |
| 1523 | // mount unit writes a symlink for the next boot and nothing else, the same |
| 1524 | // trap timers and path units fell into (codex R1-14). A socket unit the |
| 1525 | // same. The relay is still the only service the plan starts. |
| 1526 | func TestSystemctlPlanStartsMountsAndSockets(t *testing.T) { |
| 1527 | plan := systemctlPlan([]string{"carlos-edge.service", "data-instances.mount", "run-carlos-instances.mount", "run-carlos-logs.mount", "carlos-adopt.timer", "carlos-instance-control.socket"}) |
| 1528 | var started []string |
| 1529 | for _, args := range plan { |
| 1530 | if args[0] == "start" { |
| 1531 | started = append(started, args[1:]...) |
| 1532 | } |
| 1533 | } |
| 1534 | got := strings.Join(started, " ") |
| 1535 | for _, want := range []string{"data-instances.mount", "run-carlos-instances.mount", "run-carlos-logs.mount", "carlos-adopt.timer", "carlos-instance-control.socket"} { |
| 1536 | if !strings.Contains(got, want) { |
| 1537 | t.Errorf("plan does not start %s: %v", want, plan) |
| 1538 | } |
| 1539 | } |
| 1540 | if strings.Contains(got, "carlos-edge.service") { |
| 1541 | t.Errorf("plan starts the edge: %v", plan) |
| 1542 | } |
| 1543 | } |
| 1544 | |
| 1545 | func TestEnabledUnitsIncludeTheMounts(t *testing.T) { |
| 1546 | have := map[string]bool{} |
| 1547 | for _, u := range enabledUnits { |
| 1548 | have[u] = true |
| 1549 | } |
| 1550 | for _, want := range []string{"data-instances.mount", "run-carlos-instances.mount", "run-carlos-logs.mount"} { |
| 1551 | if !have[want] { |
| 1552 | t.Errorf("enabledUnits lacks %s", want) |
| 1553 | } |
| 1554 | } |
| 1555 | } |
| 1556 | ``` |
| 1557 | |
| 1558 | Add `"os"` and `"path/filepath"` to that test file's imports. |
| 1559 | |
| 1560 | - [ ] **Step 2: Run the tests to verify they fail** |
| 1561 | |
| 1562 | Run: `GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/host -run 'TestTmpfiles|TestBootstrapCreatesTheInstance|TestSystemctlPlanStartsMounts|TestEnabledUnitsInclude' -count=1` |
| 1563 | Expected: FAIL on every one. |
| 1564 | |
| 1565 | - [ ] **Step 3: Extend `tmpfilesConf`, the directory list, `enabledUnits` and `systemctlPlan`** |
| 1566 | |
| 1567 | `internal/host/bootstrap.go:61`, replace the constant: |
| 1568 | |
| 1569 | ```go |
| 1570 | const tmpfilesConf = "d /run/carlos 2750 " + serviceUser + " " + socketsGroup + " -\n" + |
| 1571 | "d /run/carlos/apps 0750 " + serviceUser + " " + serviceUser + " -\n" + |
| 1572 | // The isolated-exec trees (design §3.1, §3.2). Each protected view is |
| 1573 | // a nosymfollow bind of the dotted raw directory beside it (the mount |
| 1574 | // units); tmpfiles creates both every boot because /run is tmpfs and |
| 1575 | // because the mounts are ordered after it. The socket trees keep the |
| 1576 | // consumers group and setgid, exactly like /run/carlos above, so an |
| 1577 | // enrolled router can still dial a migrated instance (issue #103). |
| 1578 | "d /data/.instances 0700 " + serviceUser + " " + serviceUser + " -\n" + |
| 1579 | "d /data/instances 0755 " + serviceUser + " " + serviceUser + " -\n" + |
| 1580 | "d /data/layout 0700 " + serviceUser + " " + serviceUser + " -\n" + |
| 1581 | "d /run/carlos/.instances 2750 " + serviceUser + " " + socketsGroup + " -\n" + |
| 1582 | "d /run/carlos/instances 2750 " + serviceUser + " " + socketsGroup + " -\n" + |
| 1583 | "d /run/carlos/.logs 0700 " + serviceUser + " " + serviceUser + " -\n" + |
| 1584 | "d /run/carlos/logs 0700 " + serviceUser + " " + serviceUser + " -\n" + |
| 1585 | // The root instance-control channel's directory (design §5.2): root's, |
| 1586 | // so no carlos process can create, rename or replace anything in it; |
| 1587 | // group carlos with search so the agent can connect. |
| 1588 | "d /run/carlos-control 0750 root " + serviceUser + " -\n" + |
| 1589 | // The readiness probe's fixture (design §8.5, codex R11-1): a file |
| 1590 | // root:carlos 0640 that an isolated instance must NOT be able to read. |
| 1591 | // Written here, not by the control service, whose ProtectSystem=strict |
| 1592 | // makes /etc read-only to it. |
| 1593 | "f /etc/carlos/probe-secret 0640 root " + serviceUser + " - probe\n" |
| 1594 | ``` |
| 1595 | |
| 1596 | In `bootstrapWith`'s `dirs` slice (line 296 onward), add after the `{filepath.Join(runDir, "apps"), 0o750},` entry: |
| 1597 | |
| 1598 | ```go |
| 1599 | // The isolated-exec trees (design §3.1). The dotted names are the |
| 1600 | // raw directories the mount units bind onto the undotted views; |
| 1601 | // see tmpfilesConf for the modes and why both exist. |
| 1602 | {filepath.Join(dataDir, ".instances"), 0o700}, |
| 1603 | {filepath.Join(dataDir, "instances"), 0o755}, |
| 1604 | {filepath.Join(dataDir, "layout"), 0o700}, |
| 1605 | {filepath.Join(runDir, ".instances"), 0o750 | os.ModeSetgid}, |
| 1606 | {filepath.Join(runDir, "instances"), 0o750 | os.ModeSetgid}, |
| 1607 | {filepath.Join(runDir, ".logs"), 0o700}, |
| 1608 | {filepath.Join(runDir, "logs"), 0o700}, |
| 1609 | // Root-owned on a live box (privilegedPlan chowns /run/carlos |
| 1610 | // recursively to the service user; this one sits BESIDE it, not |
| 1611 | // under it, precisely so that chown never reaches it). |
| 1612 | {filepath.Join(root, "run", "carlos-control"), 0o750}, |
| 1613 | ``` |
| 1614 | |
| 1615 | `enabledUnits` (line 116): |
| 1616 | |
| 1617 | ```go |
| 1618 | var enabledUnits = []string{"carlos-edge.service", relayUnitName, "carlos-adopt.timer", "carlos-adopt.path", "carlos-stage.path", "carlos-restart.path", |
| 1619 | "carlos-system-update.path", "carlos-system-recover.service", |
| 1620 | // The isolated-exec mounts (design §3.2): the edge Requires= them. |
| 1621 | "data-instances.mount", "run-carlos-instances.mount", "run-carlos-logs.mount"} |
| 1622 | ``` |
| 1623 | |
| 1624 | `systemctlPlan` (line 165): the `armed` predicate becomes |
| 1625 | |
| 1626 | ```go |
| 1627 | for _, u := range units { |
| 1628 | if strings.HasSuffix(u, ".timer") || strings.HasSuffix(u, ".path") || |
| 1629 | strings.HasSuffix(u, ".socket") || strings.HasSuffix(u, ".mount") || u == relayUnitName { |
| 1630 | armed = append(armed, u) |
| 1631 | } |
| 1632 | } |
| 1633 | ``` |
| 1634 | |
| 1635 | and extend the function's doc comment with one paragraph: |
| 1636 | |
| 1637 | ```go |
| 1638 | // Socket and mount units are armed the same way (isolated-exec design |
| 1639 | // §8.6, codex R1-14): an enabled-but-unstarted socket is a control channel |
| 1640 | // nobody is listening on, and an enabled-but-unmounted bind is an instance |
| 1641 | // tree with no protection, until the next reboot. Starting either is |
| 1642 | // idempotent. |
| 1643 | ``` |
| 1644 | |
| 1645 | Update `TestBootstrapPlanJSONNamesWhatALiveRunArms` (`host_test.go:1195`): add the three mounts to its `want` map. Its "starts no service but the relay" loop is unchanged and still passes. |
| 1646 | |
| 1647 | - [ ] **Step 4: Run the tests to verify they pass** |
| 1648 | |
| 1649 | Run: `GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/host -count=1` |
| 1650 | Expected: PASS for the whole package. (`TestBootstrapWritesJournaldCap` and the other bootstrap tests run `bootstrap(root, nil)` against a temp root and are unaffected by the new directories.) |
| 1651 | |
| 1652 | - [ ] **Step 5: Full gate** |
| 1653 | |
| 1654 | Run, outside the sandbox with `TMPDIR=/tmp/g`: |
| 1655 | |
| 1656 | ```bash |
| 1657 | export GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off TMPDIR=/tmp/g |
| 1658 | go build ./... && go vet ./... && gofmt -l . && go test ./... -count=1 |
| 1659 | GOOS=darwin go build ./... && GOOS=windows go build ./... && GOARCH=arm64 go build ./... |
| 1660 | ``` |
| 1661 | |
| 1662 | Expected: every package passes, `gofmt -l .` prints nothing, the three cross-compiles build. |
| 1663 | |
| 1664 | - [ ] **Step 6: Commit** |
| 1665 | |
| 1666 | ```bash |
| 1667 | git add internal/host/bootstrap.go internal/host/host_test.go internal/host/units_isolation_test.go |
| 1668 | git commit -m "bootstrap: create and arm the isolated-exec trees, mounts and control directory (plan 1, task 5)" |
| 1669 | ``` |
| 1670 | |
| 1671 | --- |
| 1672 | |
| 1673 | ## Self-review |
| 1674 | |
| 1675 | **Spec coverage for this plan's scope.** §3.1 helpers: Task 1. §3.1 `InstanceID` bound and the `_` domain rule: Task 1. §3.2 mounts, tmpfiles, `instancefs`, `O_EXCL` writes: Tasks 2, 4, 5. §4.3 `carlos.slice`, `carlos-instances.slice`, edge unit changes: Task 4. §5.4 dial checks on every dial the edge and agent make: Task 3 (edge `unixProxy` via `dialUnixWithRetry`, activator `socketClient`, release `unixHTTPClient`, host `socketServing`; `schedule_wiring.proxyFor` uses `edge.UnixProxy` and is covered; `internal/vet/vet.go` is a local harness dialling a socket it created itself and is deliberately left alone). §8.6 planner starts sockets and mounts: Task 5. §9 units and installers, tmpfiles: Tasks 4, 5. Deferred to later plans, by design: the three templates, the probe, the control units and verbs (plan 2); the activator unit path and every consumer of the nested paths (plan 3); the layout verb, journal, status field, downgrade guards and `docs/site/cli.md` (plan 4). `update-platform-binary.sh` in pinfra (enable the new units by name) is a pinfra change noted for the roll, outside this repo. |
| 1676 | |
| 1677 | **Placeholder scan.** None; every step carries its code. The one conditional in Task 3 step 5 (an import cycle through `release`) names the exact fallback. |
| 1678 | |
| 1679 | **Type consistency.** `InstanceID`, `InstanceDir`, `NestedDBPath`, `InstanceRunDir`, `NestedSocket`, `LogSocket`, `IsControlEndpoint`, `ControlSocketDir`, `ControlSocketPath`, `MaxInstanceID`, `MaxUnixPath`, `LongestSocketAddress` are spelled identically in Tasks 1 and 3. `unixdial.Checked(ctx, d, path)` is spelled identically at all four call sites. `unitNames()` exists (`bootstrap.go:482`) and is used unchanged in Task 4's test. |
| 1680 | |
amadan