carlos / platform Public

Clone
git clone https://amadan.net/carlos/platform

Plain git — no account needed to clone.

Download

Download this file

1# Sidecar Members Implementation Plan
2
3> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
4
5**Goal:** Make a sidecar a first-class platform member — provisioned through
6an API like a web app instance, unreachable from the internet by
7construction, hibernating with timer and command wakes, placeable on a box
8with no inbound path at all.
9
10**Architecture:** Mirror instance provisioning one directory over
11(`sidecars.json` manifest + `sidecars/<name>.json` create-only records +
12a reconcile pass + console verbs + an app-held token), key sidecars by a
13synthetic `<name>.<app>.<sqid>.sidecar.internal` host so every host-keyed
14subsystem works unchanged, and reuse `activator.Service` for the lifecycle
15while keeping sidecars entirely off its request path. A sidecar binds
16nothing; it reports readiness, next-due and idleness by dialing **out** to a
17new agent-side unix socket.
18
19**Tech Stack:** Go (stdlib only — no new dependencies), modernc sqlite
20(`internal/registry`), S3/dir-backed `release.Store`, litestream via
21`internal/lease`.
22
23**Spec:** `docs/superpowers/specs/2026-08-17-sidecar-members-design.md`
24
25## Global Constraints
26
27- **The gate, before every commit:** `go build ./...`, `go vet ./...`,
28 `go test ./... -count=1` all clean, and `gofmt -l` clean for files you
29 touched. Three pre-existing unformatted files elsewhere are left alone.
30- **Sandbox note:** use `GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod
31 GOPROXY=off`. Go commands may need the sandbox disabled ("read-only file
32 system" from the toolchain is the tell).
33- **The gate is not Linux-only.** `internal/host` links into the CLI, which
34 people run on Macs. `GOOS=darwin go build ./...` must pass too.
35- **Wire-shape changes are additive.** Live pointers, manifests and registry
36 rows exist in production buckets and on production boxes; anything written
37 before your change must still parse. Never remove or repurpose a field.
38- **When a storage shape changes, the consumer list is everything that
39 PARSES the shape** — not everything that calls the API.
40- **No new dependencies.** Standard library only.
41- **Never touch a live box.** This branch is build-and-PR only. No deploys,
42 no `ssm`, no `aws` calls, no writes to any real bucket.
43- **Reserved suffix, exact string:** `.sidecar.internal`
44- **Synthetic host format, exact:** `<name>.<app>.<sqid>.sidecar.internal`
45- **Registry Kind value, exact:** `sidecar`
46- **Agent socket path, exact:** `/run/carlos/agent.sock`
47- **Env var names, exact:** `CARLOS_AGENT_SOCKET`, `CARLOS_SIDECAR_TOKEN`,
48 `CARLOS_SIDECAR_NAME`, `CARLOS_SIDECAR_HOST`, `CARLOS_INBOUND`
49- **Bucket keys, exact:** `apps/<sqid>/<app>/sidecars.json`,
50 `apps/<sqid>/<app>/sidecars/<name>.json`,
51 `apps/<sqid>/<app>/sidecar-wake/<name>.json`
52- **Defaults, exact:** idle `5m`, `max_run` `1h`, settle window `10s`,
53 hibernate `true`, placement `any`.
54- **Commit style:** one commit per task, imperative subject, body explaining
55 *why*. End every commit message with:
56 ```
57 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
58 Claude-Session: https://claude.ai/code/session_014WvcQVWggXkE5BAkjRDgWF
59 ```
60
61---
62
63## File Structure
64
65**Created:**
66- `internal/provision/sidecar.go` — sidecar manifest + record wire shapes,
67 key helpers, validation, store operations. Sibling to `provision.go`,
68 same package: it is the same kind of object one directory over, and it
69 reuses `labelRe`, `ValidRegion`, `ValidPool` directly.
70- `internal/provision/sidecar_test.go`
71- `internal/provision/sidecar_reconcile.go` — the records→routes pass.
72- `internal/provision/sidecar_reconcile_test.go`
73- `internal/agentsock/agentsock.go` — the agent-side unix socket: listener,
74 per-wake token registry, the four verbs. Its own package because it is a
75 server with its own lifecycle, and because `internal/activator` must be
76 able to depend on it without a cycle.
77- `internal/agentsock/agentsock_test.go`
78- `internal/release/sidecarwake.go` — the wake generation object.
79- `internal/release/sidecarwake_test.go`
80- `internal/console/apisidecars.go` — the console CLI verbs.
81- `internal/console/apisidecars_test.go`
82- `sidecars_cli.go` (root package) — `carlos sidecars`.
83- `sidecars_cli_test.go`, `sidecars_e2e_test.go` (root package)
84- `infra/no-inbound-box.md` — the private-subnet runbook (Paul applies).
85
86**Modified:**
87- `internal/registry/registry.go` — `args` column (FOUR places), `SetArgs`,
88 `DueInstances` widening.
89- `internal/edge/edge.go` — structural sidecar refusal.
90- `internal/host/fleetbind.go` — never publish a sidecar binding.
91- `internal/host/restart.go` — cycle sidecars.
92- `internal/host/agent.go` — `CARLOS_INBOUND`, the sidecar reconcile leg,
93 the agent socket's lifecycle, the wake pass.
94- `internal/appconfig/syncexec.go` — deliver config to sidecars.
95- `internal/activator/activator.go` — `StartWorker` seam, `WakeWorker`,
96 socketless stop, `max_run`.
97- `internal/activator/backend_exec.go`, `backend_unit.go` — the seam's two
98 implementations.
99- `internal/provision/reconcile.go` — instance removal phase skips sidecars.
100- `internal/console/authcli.go` — route registration.
101- `internal/console/instances.go`, `overview.go` — Kind guards, sidecar words.
102- `internal/status/snapshot.go` — sidecars are not public Residents.
103- `cli.go` (root) — `carlos add` rails off `-kind sidecar` and the suffix.
104- `main.go` (root) — `sidecars` dispatch.
105- `apimode.go` (root) — the client methods.
106- `CLAUDE.md` — the co-spawn/native split.
107
108---
109
110## Task ordering and dependencies
111
112```
1131 registry ──┬── 3 edge/fleetbind ── 4 suffix rails
114 ├── 5 StartWorker ── 6 lifecycle ── 7 agentsock ── 8 wiring
115 ├── 11 config+restart
116 └── 15 console UI
1172 provision shapes ──┬── 9 reconciler ── 10 placement
118 ├── 12 console API ── 14 CLI
119 └── 13 wake object
12016 docs (last)
121```
122
123Tasks 1 and 2 are independent of each other and unblock everything else.
124
125---
126
127### Task 1: Registry — the `sidecar` kind, the `args` column, and the due-time widening
128
129**Files:**
130- Modify: `internal/registry/registry.go`
131- Test: `internal/registry/registry_test.go`
132
133**Interfaces:**
134- Consumes: nothing.
135- Produces: `Route.Args string` (JSON argv, `""` = none);
136 `Route.IdleSeconds int`, `Route.MaxRunSeconds int` (0 = use the platform
137 default); `(*Registry).SetArgs(host, args string) error`;
138 `(*Registry).SetWorkerLimits(host string, idle, maxRun int) error`;
139 `DueInstances` now returns `kind IN ('instance','sidecar')`.
140
141**Why the ceilings live on the row.** The sweeper decides on every tick
142whether a worker has outrun its ceiling, and it must do that from local
143state — a bucket read per wake would put the control plane in the hot path
144of a timer. So `idle_seconds`/`max_run_seconds` are materialized from the
145app's manifest by the reconcile pass exactly as `args` and `seed` are:
146truth stays in the bucket, the row is the box's local copy.
147
148**Why this is first and why it matters:** `DueInstances` is the sweeper's
149only source of timer wakes. Left filtering `kind = 'instance'`, a sidecar's
150`next_due` would be written, stored, and never once acted on — the schedule
151wake source would silently not exist, with no error anywhere.
152
153- [ ] **Step 1: Write the failing tests**
154
155Add to `internal/registry/registry_test.go`:
156
157```go
158func TestDueInstancesIncludesSidecars(t *testing.T) {
159 r := testRegistry(t) // use this file's existing helper for a temp registry
160 if err := r.Upsert(Route{
161 Host: "bot.eleven.bdm.sidecar.internal", Kind: "sidecar",
162 DBPath: "/tmp/bot.db", Hibernate: true, Status: "ready",
163 }); err != nil {
164 t.Fatalf("upsert sidecar: %v", err)
165 }
166 if err := r.SetNextDue("bot.eleven.bdm.sidecar.internal", 100); err != nil {
167 t.Fatalf("SetNextDue: %v", err)
168 }
169
170 due, err := r.DueInstances(200)
171 if err != nil {
172 t.Fatalf("DueInstances: %v", err)
173 }
174 if len(due) != 1 || due[0].Host != "bot.eleven.bdm.sidecar.internal" {
175 t.Fatalf("DueInstances = %+v, want the due sidecar — a sidecar's timer wake is its ONLY schedule wake source", due)
176 }
177}
178
179func TestArgsRoundTrips(t *testing.T) {
180 r := testRegistry(t)
181 if err := r.Upsert(Route{Host: "bot.x.internal", Kind: "sidecar", Args: `["sidecar","run"]`}); err != nil {
182 t.Fatalf("upsert: %v", err)
183 }
184 got, err := r.Get("bot.x.internal")
185 if err != nil {
186 t.Fatalf("get: %v", err)
187 }
188 if got.Args != `["sidecar","run"]` {
189 t.Fatalf("Args = %q, want the argv back", got.Args)
190 }
191
192 if err := r.SetArgs("bot.x.internal", `["worker","run"]`); err != nil {
193 t.Fatalf("SetArgs: %v", err)
194 }
195 got, _ = r.Get("bot.x.internal")
196 if got.Args != `["worker","run"]` {
197 t.Fatalf("Args after SetArgs = %q", got.Args)
198 }
199}
200
201func TestWorkerLimitsRoundTrip(t *testing.T) {
202 r := testRegistry(t)
203 if err := r.Upsert(Route{Host: "bot.x.internal", Kind: "sidecar", IdleSeconds: 300, MaxRunSeconds: 3600}); err != nil {
204 t.Fatalf("upsert: %v", err)
205 }
206 got, _ := r.Get("bot.x.internal")
207 if got.IdleSeconds != 300 || got.MaxRunSeconds != 3600 {
208 t.Fatalf("limits = %d/%d, want 300/3600 — the sweeper reads these locally on every tick", got.IdleSeconds, got.MaxRunSeconds)
209 }
210 if err := r.SetWorkerLimits("bot.x.internal", 60, 120); err != nil {
211 t.Fatalf("SetWorkerLimits: %v", err)
212 }
213 got, _ = r.Get("bot.x.internal")
214 if got.IdleSeconds != 60 || got.MaxRunSeconds != 120 {
215 t.Fatalf("limits after set = %d/%d", got.IdleSeconds, got.MaxRunSeconds)
216 }
217}
218
219func TestRouteWrittenBeforeArgsStillParses(t *testing.T) {
220 // The additive rule: a row written by older code has no args value.
221 r := testRegistry(t)
222 if err := r.Upsert(Route{Host: "old.example.com", Kind: "instance"}); err != nil {
223 t.Fatalf("upsert: %v", err)
224 }
225 got, err := r.Get("old.example.com")
226 if err != nil {
227 t.Fatalf("get: %v", err)
228 }
229 if got.Args != "" || got.IdleSeconds != 0 || got.MaxRunSeconds != 0 {
230 t.Fatalf("row = %+v, want zero values for a pre-existing row", got)
231 }
232}
233```
234
235If `testRegistry` is not the helper's name in this file, use whatever the
236file already uses to open a registry in `t.TempDir()` — do not add a new
237helper.
238
239- [ ] **Step 2: Run the tests and watch them fail**
240
241```bash
242GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
243 go test ./internal/registry/ -run 'DueInstancesIncludesSidecars|ArgsRoundTrips|BeforeArgs' -count=1
244```
245
246Expected: compile failure — `Route` has no field `Args`, `SetArgs` undefined.
247
248- [ ] **Step 3: Add the column in all FOUR places**
249
250`registry.go` warns at the `routeCols` comment that a column lives in four
251hand-maintained lists and that a miscount is a **silent column shift**, not
252an error. Make all four edits together:
253
2541. Struct field, beside `Seed` (`registry.go` ~line 127 area):
255 ```go
256 // Args is a sidecar's argv as a JSON array, materialized from the app's
257 // sidecars manifest on every reconcile pass (truth stays in the bucket).
258 // Empty for every other kind of route.
259 Args string
260 ```
2612. Migration — append to the `ALTER TABLE` list in `migrate()`:
262 ```go
263 `ALTER TABLE routes ADD COLUMN args TEXT NOT NULL DEFAULT ''`,
264 ```
2653. `routeCols` — append `args, idle_seconds, max_run_seconds` at the very
266 END of the list (after `created_at`). Position does not matter as long
267 as it is THE SAME position in all four lists; appending at the end is
268 the least error-prone choice.
2694. `scanRoute` — add `&t.Args, &t.IdleSeconds, &t.MaxRunSeconds` at the end
270 of the destination list, in that order.
2715. `Upsert` — add three `?` placeholders at the end of the placeholder row
272 and `t.Args, t.IdleSeconds, t.MaxRunSeconds` at the end of the argument
273 list, in that order.
274
275Then add the setter beside `SetSeed`:
276
277```go
278// SetArgs repoints a sidecar's argv — the narrow UPDATE the sidecar
279// reconcile pass uses when the app's manifest changed.
280func (r *Registry) SetArgs(host, args string) error {
281 _, err := r.sql.Exec(`UPDATE routes SET args = ? WHERE host = ?`, args, host)
282 return err
283}
284
285// SetWorkerLimits repoints a sidecar's idle window and run ceiling, both in
286// seconds, 0 meaning "use the platform default". Same narrow-UPDATE shape
287// and same reason as SetArgs: the manifest is the truth, this is the copy
288// the sweeper reads on every tick.
289func (r *Registry) SetWorkerLimits(host string, idle, maxRun int) error {
290 _, err := r.sql.Exec(
291 `UPDATE routes SET idle_seconds = ?, max_run_seconds = ? WHERE host = ?`,
292 idle, maxRun, host)
293 return err
294}
295```
296
297- [ ] **Step 4: Widen `DueInstances`**
298
299```go
300// DueInstances returns hibernatable instance AND sidecar routes whose
301// next_due has arrived (0 means no schedule and never matches). Sidecars
302// are included because the due-time sweep is their ONLY timer wake source:
303// nothing can dial a sidecar, so no request will ever wake one.
304func (r *Registry) DueInstances(now int64) ([]Route, error) {
305 rows, err := r.sql.Query(`SELECT `+routeCols+` FROM routes
306WHERE kind IN ('instance','sidecar') AND next_due > 0 AND next_due <= ? ORDER BY host`, now)
307```
308
309- [ ] **Step 5: Run the full registry suite**
310
311```bash
312GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
313 go test ./internal/registry/ -count=1
314```
315
316Expected: PASS, including `TestUpsertRoundTripsAllColumns` — that test is
317the canary for a four-list miscount. If it fails with a value in the wrong
318field, you shifted a column: recount all four lists.
319
320- [ ] **Step 6: Full gate, then commit**
321
322```bash
323GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go build ./... && \
324GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go vet ./... && \
325GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./... -count=1
326gofmt -l internal/registry/
327git add internal/registry/
328git commit -m "registry: sidecar kind, args column, and due-time widening
329
330DueInstances filtered kind='instance', which would have stored every
331sidecar's next_due and acted on none of them — the schedule wake source
332would silently not have existed. A sidecar cannot be dialed, so the
333due-time sweep is its only timer wake.
334
335args carries a sidecar's argv to the box, materialized from the app's
336manifest each pass exactly as seed is; truth stays in the bucket.
337[trailers]"
338```
339
340---
341
342### Task 2: Provision — sidecar manifest and record shapes
343
344**Files:**
345- Create: `internal/provision/sidecar.go`
346- Test: `internal/provision/sidecar_test.go`
347
348**Interfaces:**
349- Consumes: `release.Store`, and this package's existing `labelRe`,
350 `ValidRegion`, `ValidPool`, `DefaultDBPath`.
351- Produces:
352 ```go
353 const SidecarSuffix = ".sidecar.internal"
354 type SidecarManifest struct{ Schema int; App, Pool, Channel, Placement string
355 Regions []string; Args []string; Hibernate *bool; IdleSeconds, MaxRunSeconds int }
356 type SidecarRecord struct{ Schema int; Name, Host, App, Account, Pool, Region,
357 Channel, Placement, Environment, CreatedAt, CreatedBy string }
358 func SidecarHost(name, app, sqid string) string
359 func ValidSidecarName(name string) bool
360 func ValidSidecarHost(host, app, sqid string) bool
361 func SidecarManifestKey(acct, app string) string
362 func SidecarsPrefix(acct, app string) string
363 func SidecarRecordKey(acct, app, name string) string
364 func ParseSidecarKey(key string, known map[string]bool) (acct, app, name string, ok bool)
365 func ValidateSidecarShape(rec SidecarRecord) error
366 func GetSidecarManifest(ctx, s, acct, app) (*SidecarManifest, error)
367 func GetSidecarManifestETag(ctx, s, acct, app) (*SidecarManifest, string, error)
368 func PutSidecarManifest(ctx, s, acct, app, m) error
369 func PutSidecarManifestIfMatch(ctx, s, acct, app, m, etag string) error
370 func CreateSidecarRecord(ctx, s, rec) error
371 func GetSidecarRecord(ctx, s, acct, app, name) (*SidecarRecord, error)
372 func ListSidecarRecords(ctx, s, acct, app string, max int) ([]SidecarRecord, bool, error)
373 func DeleteSidecarRecord(ctx, s, acct, app, name string) error
374 func (m SidecarManifest) IdleOrDefault() time.Duration // 5m
375 func (m SidecarManifest) MaxRunOrDefault() time.Duration // 1h
376 func (m SidecarManifest) HibernateOrDefault() bool // true
377 func (m SidecarManifest) PlacementOrDefault() string // "any"
378 ```
379
380**Durations are integer seconds on the wire** (`idle_seconds`,
381`max_run_seconds`), not Go duration strings — the existing records use plain
382JSON scalars and a string would invite `"5 minutes"`.
383
384- [ ] **Step 1: Write the failing tests**
385
386Create `internal/provision/sidecar_test.go`. Mirror the idiom in
387`provision_test.go` and `reconcile_test.go`: a real `release.DirStore` in
388`t.TempDir()`, no mocks.
389
390```go
391func TestSidecarHostAndValidation(t *testing.T) {
392 got := SidecarHost("bot-acme", "eleven", "bdm")
393 if want := "bot-acme.eleven.bdm.sidecar.internal"; got != want {
394 t.Fatalf("SidecarHost = %q, want %q", got, want)
395 }
396 if !ValidSidecarHost(got, "eleven", "bdm") {
397 t.Fatal("its own SidecarHost output must validate")
398 }
399 for _, bad := range []string{
400 "bot-acme.other.bdm.sidecar.internal", // wrong app
401 "bot-acme.eleven.bad.sidecar.internal", // wrong sqid
402 "bot-acme.eleven.bdm.example.com", // wrong suffix
403 "a.b.bot.eleven.bdm.sidecar.internal", // more than one label
404 } {
405 if ValidSidecarHost(bad, "eleven", "bdm") {
406 t.Fatalf("ValidSidecarHost(%q) = true, want false", bad)
407 }
408 }
409 for _, bad := range []string{"", "-lead", "trail-", "UPPER", "has.dot", "has/slash", "..", strings.Repeat("a", 64)} {
410 if ValidSidecarName(bad) {
411 t.Fatalf("ValidSidecarName(%q) = true — the name becomes a bucket key AND a filesystem path segment", bad)
412 }
413 }
414}
415
416func TestParseSidecarKey(t *testing.T) {
417 known := map[string]bool{"bdm": true}
418 acct, app, name, ok := ParseSidecarKey("apps/bdm/eleven/sidecars/bot-acme.json", known)
419 if !ok || acct != "bdm" || app != "eleven" || name != "bot-acme" {
420 t.Fatalf("ParseSidecarKey = %q/%q/%q ok=%v", acct, app, name, ok)
421 }
422 for _, bad := range []string{
423 "apps/bdm/eleven/sidecars.json", // the manifest, not a record
424 "apps/bdm/eleven/sidecars/nested/x.json", // nested
425 "apps/bdm/eleven/sidecars/.json", // empty name
426 "apps/unknown/eleven/sidecars/bot.json", // account not materialized
427 "apps/bdm/channels/sidecars/bot.json", // release-plane collision
428 "apps/bdm/eleven/instances/bot.json", // the other family
429 } {
430 if _, _, _, ok := ParseSidecarKey(bad, known); ok {
431 t.Fatalf("ParseSidecarKey(%q) parsed, want refusal", bad)
432 }
433 }
434}
435
436func TestCreateSidecarRecordIsCreateOnly(t *testing.T) {
437 ctx := context.Background()
438 s := dirStore(t) // the helper reconcile_test.go already uses
439 rec := SidecarRecord{
440 Schema: 1, Name: "bot-acme", Host: SidecarHost("bot-acme", "eleven", "bdm"),
441 App: "eleven", Account: "bdm", Channel: "stable",
442 }
443 if err := CreateSidecarRecord(ctx, s, rec); err != nil {
444 t.Fatalf("first create: %v", err)
445 }
446 err := CreateSidecarRecord(ctx, s, rec)
447 if !errors.Is(err, release.ErrCASConflict) {
448 t.Fatalf("second create err = %v, want ErrCASConflict — a lost race must never silently overwrite", err)
449 }
450}
451
452func TestGetSidecarRecordRefusesRecordThatLiesAboutItsPath(t *testing.T) {
453 ctx := context.Background()
454 s := dirStore(t)
455 body := []byte(`{"schema":1,"name":"bot-acme","app":"OTHER","account":"bdm"}`)
456 if _, err := s.Put(ctx, SidecarRecordKey("bdm", "eleven", "bot-acme"), body, ""); err != nil {
457 t.Fatalf("seed: %v", err)
458 }
459 if _, err := GetSidecarRecord(ctx, s, "bdm", "eleven", "bot-acme"); err == nil {
460 t.Fatal("a record disagreeing with its own key path must be refused, never guessed at")
461 }
462}
463
464func TestSidecarManifestDefaults(t *testing.T) {
465 var m SidecarManifest
466 if m.IdleOrDefault() != 5*time.Minute {
467 t.Fatalf("idle default = %v, want 5m", m.IdleOrDefault())
468 }
469 if m.MaxRunOrDefault() != time.Hour {
470 t.Fatalf("max_run default = %v, want 1h", m.MaxRunOrDefault())
471 }
472 if !m.HibernateOrDefault() {
473 t.Fatal("hibernate must default true")
474 }
475 if m.PlacementOrDefault() != "any" {
476 t.Fatalf("placement default = %q, want any", m.PlacementOrDefault())
477 }
478 no := false
479 m2 := SidecarManifest{Hibernate: &no}
480 if m2.HibernateOrDefault() {
481 t.Fatal("an explicit false must win — tri-state, like instances' Hibernate")
482 }
483}
484
485func TestSidecarManifestSchemaNewerIsRefused(t *testing.T) {
486 ctx := context.Background()
487 s := dirStore(t)
488 if _, err := s.Put(ctx, SidecarManifestKey("bdm", "eleven"), []byte(`{"schema":2}`), ""); err != nil {
489 t.Fatalf("seed: %v", err)
490 }
491 if _, err := GetSidecarManifest(ctx, s, "bdm", "eleven"); err == nil {
492 t.Fatal("a schema newer than this binary understands must be refused, not silently truncated")
493 }
494}
495
496func TestSidecarManifestAbsenceIsNotExist(t *testing.T) {
497 _, err := GetSidecarManifest(context.Background(), dirStore(t), "bdm", "eleven")
498 if !errors.Is(err, os.ErrNotExist) {
499 t.Fatalf("absence err = %v, want os.ErrNotExist — absence means the feature does not exist for that app", err)
500 }
501}
502
503func TestValidateSidecarShape(t *testing.T) {
504 ok := SidecarRecord{Schema: 1, Name: "bot", Host: SidecarHost("bot", "eleven", "bdm"),
505 App: "eleven", Account: "bdm", Channel: "stable", Placement: "no-inbound"}
506 if err := ValidateSidecarShape(ok); err != nil {
507 t.Fatalf("valid record refused: %v", err)
508 }
509 bad := ok
510 bad.Placement = "sideways"
511 if err := ValidateSidecarShape(bad); err == nil {
512 t.Fatal("placement vocabulary must be closed")
513 }
514}
515```
516
517- [ ] **Step 2: Run and watch it fail**
518
519```bash
520GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
521 go test ./internal/provision/ -run Sidecar -count=1
522```
523Expected: compile failure, everything undefined.
524
525- [ ] **Step 3: Implement `internal/provision/sidecar.go`**
526
527Write the file mirroring `provision.go` structurally. The store operations
528are **the same six functions with `Sidecar` in the name and the sidecar key
529helpers substituted** — copy the bodies and their doc comments' reasoning
530from these exact sites:
531
532- `GetSidecarManifest` / `GetSidecarManifestETag` ← `provision.go:508-529`
533 (schema>1 refusal included).
534- `PutSidecarManifest` ← `provision.go:535-557` (CAS upsert, one retry).
535- `PutSidecarManifestIfMatch` ← `provision.go:568-575` (no retry).
536- `CreateSidecarRecord` ← `provision.go:580-587` (`Put` with `ifMatch ""`).
537- `GetSidecarRecord` ← `provision.go:687-709`, **keeping the
538 identity-vs-path agreement check** (`rec.Account != acct || rec.App != app
539 || rec.Name != name` → error).
540- `ListSidecarRecords` ← `provision.go:726-760` (sorted, capped, `truncated`
541 reported on key count, malformed bodies skipped).
542- `DeleteSidecarRecord` ← `provision.go:641-643`.
543
544The new logic is only this:
545
546```go
547// SidecarSuffix is the reserved, never-resolvable suffix every sidecar host
548// carries. It is refused at every door where a name could become routable
549// (see cli.go's addRail, the route door, and domain attach) — reservation
550// is enforced, not documented.
551const SidecarSuffix = ".sidecar.internal"
552
553// SidecarHost builds a sidecar's synthetic host. It is the registry key, the
554// bucket key segment and the on-disk path segment, which is why every
555// component is validated before it is ever joined.
556func SidecarHost(name, app, sqid string) string {
557 return name + "." + app + "." + sqid + SidecarSuffix
558}
559
560// ValidSidecarName pins one DNS label exactly as ValidHost pins an
561// instance's: the name becomes a bucket key segment and the <host>.db /
562// cgroup path segments, so the label regex is what keeps a hostile name an
563// opaque, safe path component.
564func ValidSidecarName(name string) bool { return labelRe.MatchString(name) }
565
566func ValidSidecarHost(host, app, sqid string) bool {
567 rest, ok := strings.CutSuffix(host, SidecarSuffix)
568 if !ok {
569 return false
570 }
571 name, tail, ok := strings.Cut(rest, ".")
572 if !ok || tail != app+"."+sqid {
573 return false
574 }
575 return ValidSidecarName(name)
576}
577```
578
579Key helpers, beside the instance ones (`provision.go:155-159`):
580
581```go
582func SidecarManifestKey(acct, app string) string {
583 return release.AppPrefix(acct, app) + "sidecars.json"
584}
585func SidecarsPrefix(acct, app string) string {
586 return release.AppPrefix(acct, app) + "sidecars/"
587}
588func SidecarRecordKey(acct, app, name string) string {
589 return SidecarsPrefix(acct, app) + name + ".json"
590}
591```
592
593`ParseSidecarKey` is `ParseRecordKey` (`provision.go:171-196`) with
594`"instances/"` → `"sidecars/"`: keep the `known` account guard, the
595`channels`/`releases` collision refusal, and the exactly-one-non-empty-
596segment rule that makes `sidecars.json` and any nested path both refuse.
597
598Defaults:
599
600```go
601func (m SidecarManifest) IdleOrDefault() time.Duration {
602 if m.IdleSeconds > 0 {
603 return time.Duration(m.IdleSeconds) * time.Second
604 }
605 return 5 * time.Minute
606}
607func (m SidecarManifest) MaxRunOrDefault() time.Duration {
608 if m.MaxRunSeconds > 0 {
609 return time.Duration(m.MaxRunSeconds) * time.Second
610 }
611 return time.Hour
612}
613func (m SidecarManifest) HibernateOrDefault() bool {
614 if m.Hibernate != nil {
615 return *m.Hibernate
616 }
617 return true
618}
619func (m SidecarManifest) PlacementOrDefault() string {
620 if m.Placement == "" {
621 return PlacementAny
622 }
623 return m.Placement
624}
625```
626
627with `const ( PlacementAny = "any"; PlacementNoInbound = "no-inbound" )`.
628
629`ValidateSidecarShape` checks: schema ≤ 1; `ValidSidecarName(rec.Name)`;
630`rec.App`/`rec.Account` non-empty; placement in the closed vocabulary;
631`ValidRegion(rec.Region)` when non-empty; `ValidPool(rec.Pool)` when
632non-empty.
633
634- [ ] **Step 4: Run the tests**
635
636```bash
637GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
638 go test ./internal/provision/ -count=1
639```
640Expected: PASS.
641
642- [ ] **Step 5: Full gate, then commit**
643
644```
645provision: sidecar manifest and record shapes
646
647Mirrors the instance record family one directory over, reusing its
648validation rails verbatim — the label regex, the identity-vs-path
649agreement check, create-only writes, and the absence-means-disabled
650convention. The name becomes a bucket key AND a filesystem path segment,
651which is why it is validated before it is ever joined into a host.
652```
653
654---
655
656### Task 3: The edge cannot bind a sidecar, and no box advertises one
657
658**Files:**
659- Modify: `internal/edge/edge.go` (the `Reload` dispatch switch, ~364-445)
660- Modify: `internal/host/fleetbind.go` (`publishResidentFleetBindings`, ~301-316)
661- Test: `internal/edge/edge_test.go`, `internal/host/fleetbind_test.go`
662
663**Interfaces:**
664- Consumes: Task 1's `Kind == "sidecar"`.
665- Produces: nothing other tasks call.
666
667**Why this is not paranoia.** A sidecar row is unroutable today only by
668*omission*: it has no Socket, Addr or Tunnel, so it falls to
669`default: continue` and never enters the routing map. But the dispatch
670switch has a `case isFleetPlaced && e.tunnels != nil` arm with no Kind
671check, and `publishResidentFleetBindings` iterates **every** route with no
672Kind filter. Eleven's sidecars are fleet-placed (`region: fleet/eleven`) —
673so the row that reaches that arm by shape is exactly the row this feature
674creates. Omission is not good enough; refuse explicitly at both ends.
675
676- [ ] **Step 1: Write the failing tests**
677
678In `internal/edge/edge_test.go` (follow the file's existing Reload/route
679fixtures):
680
681```go
682func TestSidecarRouteIsNeverServable(t *testing.T) {
683 // Adversarial on purpose: this row carries a socket, an addr, a tunnel
684 // AND a fleet placement. A sidecar must be refused on KIND, so that no
685 // present or future arm of the dispatch switch can ever bind one.
686 e := newTestEdge(t)
687 rt := registry.Route{
688 Host: "bot.eleven.bdm.sidecar.internal", Kind: "sidecar",
689 Socket: "/tmp/whatever.sock", Addr: "127.0.0.1:9999",
690 Tunnel: "eleven/box1", Region: "fleet/eleven", FleetBox: "box1",
691 Status: "ready", App: "eleven",
692 }
693 if err := e.Reload([]registry.Route{rt}); err != nil {
694 t.Fatalf("Reload: %v", err)
695 }
696 if _, ok := e.current()["bot.eleven.bdm.sidecar.internal"]; ok {
697 t.Fatal("a sidecar row entered the routing map — it is reachable from the internet")
698 }
699 if e.Allowed("bot.eleven.bdm.sidecar.internal") == nil {
700 t.Fatal("a certificate must never be requested for a sidecar host")
701 }
702}
703```
704
705(Match `newTestEdge`/`current`/`Allowed` to whatever the file actually
706provides; if `Allowed` returns an error use `!= nil` as "refused".)
707
708In `internal/host/fleetbind_test.go`:
709
710```go
711func TestSidecarBindingIsNeverPublished(t *testing.T) {
712 routes := []registry.Route{
713 {Host: "app.example.com", Kind: "instance", Socket: "/run/carlos/a.sock"},
714 {Host: "bot.eleven.bdm.sidecar.internal", Kind: "sidecar", Region: "fleet/eleven"},
715 }
716 got := residentBindingHosts(routes) // the pure helper you extract in step 3
717 if len(got) != 1 || got[0] != "app.example.com" {
718 t.Fatalf("published %v — a sidecar must never be advertised as served by a box", got)
719 }
720}
721```
722
723- [ ] **Step 2: Run and watch both fail**
724
725```bash
726GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
727 go test ./internal/edge/ ./internal/host/ -run 'Sidecar' -count=1
728```
729Expected: the edge test FAILS (the row is served via the tunnel/fleet arm);
730the fleetbind test fails to compile (`residentBindingHosts` undefined).
731
732- [ ] **Step 3: Refuse in the edge, before the switch**
733
734In `Reload`'s per-route loop, as the FIRST thing inside the loop body:
735
736```go
737// A sidecar is a member that serves nothing: it binds no socket, and
738// nothing may ever dial it. Refuse it on KIND here, before the dispatch
739// switch, rather than relying on it having no socket/addr/tunnel — a
740// fleet-placed sidecar (which is Eleven's shape) reaches the fleet arm
741// below on shape alone. This refusal is the structural guarantee that
742// survives a future arm, column or placement finding the row.
743if t.Kind == "sidecar" {
744 continue
745}
746```
747
748- [ ] **Step 4: Refuse in the fleet-binding publisher**
749
750Extract the host selection into a pure, testable helper and filter there:
751
752```go
753// residentBindingHosts is the set of hosts this box advertises as served.
754// Sidecars are excluded: nothing routes to one, so a binding claiming a box
755// serves it is a lie that the platform-side reconciler would then act on
756// (SetFleetBox -> the edge's fleet arm -> a live proxy).
757func residentBindingHosts(routes []registry.Route) []string {
758 var out []string
759 for _, rt := range routes {
760 if rt.Host == "" || rt.Kind == "sidecar" {
761 continue
762 }
763 out = append(out, rt.Host)
764 }
765 return out
766}
767```
768
769and call it from `publishResidentFleetBindings` in place of its current
770inline loop.
771
772- [ ] **Step 5: Run both tests, then the full gate, then commit**
773
774```
775edge, host: a sidecar row is refused on kind, at both ends of the fleet chain
776
777A sidecar was unroutable only by omission — no socket, addr or tunnel, so
778it fell to the dispatch switch's default. But the switch's fleet-placed
779arm checks no kind, and the binding publisher filtered no kind, and
780Eleven's sidecars are fleet-placed. That is a live path from a resident
781sidecar row to a working internet-facing proxy, so both ends now refuse
782explicitly and the test asserts it against a row that carries every
783routable column at once.
784```
785
786---
787
788### Task 4: The reserved suffix is enforced at every minting door
789
790**Files:**
791- Modify: `cli.go` (root, `runAdd` ~68-260 and `runRoute`'s rails ~1197-1373)
792- Modify: `internal/domains/reconcile.go` (alias minting)
793- Modify: `internal/console/domainattach.go` (attach door)
794- Test: `cli_test.go`, `internal/domains/reconcile_test.go`,
795 `internal/console/domainattach_test.go`
796
797**Interfaces:**
798- Consumes: `provision.SidecarSuffix`.
799- Produces: nothing.
800
801`carlos add` today accepts `-kind sidecar` with **no validation at all**
802(`cli.go:75,194-199`), and any host string reaches `registry.Upsert`. These
803rails are defence in depth behind Task 3's structural refusal.
804
805- [ ] **Step 1: Write the failing tests**
806
807```go
808func TestAddRefusesSidecarSuffixAndKind(t *testing.T) {
809 if err := addRail("bot.eleven.bdm.sidecar.internal", "instance"); err == nil {
810 t.Fatal("a host under the reserved suffix must be refused")
811 }
812 if err := addRail("app.example.com", "sidecar"); err == nil {
813 t.Fatal("carlos add must not mint a sidecar row — sidecars are provisioned, not added")
814 }
815 if err := addRail("app.example.com", "instance"); err != nil {
816 t.Fatalf("an ordinary add must still work: %v", err)
817 }
818}
819```
820
821Add matching cases asserting `domains` never mints an alias whose source or
822target carries the suffix, and that `domainattach` refuses to attach a
823custom domain to a host under it.
824
825- [ ] **Step 2: Run and watch it fail** (`addRail` undefined).
826
827- [ ] **Step 3: Implement the rail**
828
829In `cli.go`, beside `grantRail`/`backingRail`:
830
831```go
832// addRail refuses the two things `carlos add` must never mint: a host under
833// the reserved sidecar suffix, and a sidecar row itself. Sidecars are
834// provisioned through their own record family (carlos sidecars create), so
835// an operator minting one by hand would produce a row no reconcile pass
836// owns and no record explains.
837func addRail(host, kind string) error {
838 if strings.HasSuffix(host, provision.SidecarSuffix) {
839 return fmt.Errorf("%s is under the reserved suffix %s — sidecar hosts are minted by `carlos sidecars create`, never by hand", host, provision.SidecarSuffix)
840 }
841 if kind == "sidecar" {
842 return errors.New("carlos add does not mint sidecars — use `carlos sidecars create -app <app> <name>`")
843 }
844 return nil
845}
846```
847
848Call it at the top of `runAdd`, before any registry work. Apply the same
849`HasSuffix` refusal in the domains alias minting path and in the console's
850domain-attach handler (return the family's ordinary 400 there, not a panic).
851
852- [ ] **Step 4: Run the tests, full gate, commit.**
853
854---
855
856### Task 5: `StartWorker` — spawning a child that binds nothing
857
858**Files:**
859- Modify: `internal/activator/activator.go` (the `Backend` interface, ~134-167)
860- Modify: `internal/activator/backend_exec.go`
861- Modify: `internal/activator/backend_unit.go`
862- Test: `internal/activator/backend_exec_test.go` (or the existing backend test file)
863
864**Interfaces:**
865- Consumes: Task 1's `Route.Args`.
866- Produces:
867 ```go
868 // on Backend:
869 StartWorker(ctx context.Context, rt registry.Route, env map[string]string) (Proc, string, error)
870 ```
871 Returns the `Proc` and the spawn build version, exactly like `Start`.
872
873**Why a new method rather than a flag on `Start`:** `ExecBackend.Start` ends
874with an unconditional `waitHealthyCtx(ctx, rt.Socket, spawnHealthTimeout)`
875(`backend_exec.go:601`). Handed `Socket ""` it dials `unix://""` every 150ms
876for the full 30s and then fails the wake. A worker has no socket by
877definition, so it needs a spawn path with no probe — which is very nearly
878what `StartSidecar` (`backend_exec.go:746-773`) already is.
879
880- [ ] **Step 1: Write the failing test**
881
882```go
883func TestStartWorkerSpawnsWithoutAnyHealthProbe(t *testing.T) {
884 dir := t.TempDir()
885 b := &ExecBackend{DataDir: dir, OptDir: dir, Cgroups: nil}
886 // A binary that runs until killed and never binds anything.
887 writeFakeLive(t, dir, "bot.x.internal", "#!/bin/sh\nsleep 30\n")
888
889 rt := registry.Route{Host: "bot.x.internal", Kind: "sidecar",
890 Args: `["sidecar","run"]`, DBPath: filepath.Join(dir, "bot.db")}
891
892 start := time.Now()
893 p, _, err := b.StartWorker(context.Background(), rt, map[string]string{"CARLOS_SIDECAR_NAME": "bot"})
894 if err != nil {
895 t.Fatalf("StartWorker: %v", err)
896 }
897 defer p.Stop(context.Background())
898 if d := time.Since(start); d > 5*time.Second {
899 t.Fatalf("StartWorker took %v — it waited for a health probe that can never answer", d)
900 }
901 if p.Exited() {
902 t.Fatal("worker exited immediately")
903 }
904}
905
906func TestStartWorkerRefusesEmptyArgs(t *testing.T) {
907 b := &ExecBackend{DataDir: t.TempDir()}
908 _, _, err := b.StartWorker(context.Background(), registry.Route{Host: "b.x.internal", Kind: "sidecar"}, nil)
909 if err == nil {
910 t.Fatal("a worker with no argv has nothing to run — refuse rather than exec the bare binary")
911 }
912}
913
914func TestUnitBackendRefusesWorkers(t *testing.T) {
915 var b UnitBackend
916 if _, _, err := b.StartWorker(context.Background(), registry.Route{Host: "x"}, nil); err == nil {
917 t.Fatal("unit-backed tenants have no agent-managed workers, exactly as they have no agent-managed sidecars")
918 }
919}
920```
921
922`writeFakeLive` — if the test file has no such helper, write one that
923creates the `live/<host>` symlink target as an executable shell script,
924matching however the existing exec-backend tests fake a binary.
925
926- [ ] **Step 2: Run and watch it fail.**
927
928- [ ] **Step 3: Add the method to the `Backend` interface**
929
930```go
931 // StartWorker spawns a child that binds NOTHING: no socket, no port.
932 // No health probe is possible or wanted — readiness is the caller's
933 // settle window, or the child's own call over the agent socket. env is
934 // merged over the route's own env layers and wins conflicts (it carries
935 // the per-wake sidecar token, which the tenant must never override).
936 StartWorker(ctx context.Context, rt registry.Route, env map[string]string) (Proc, string, error)
937```
938
939- [ ] **Step 4: Implement `ExecBackend.StartWorker`**
940
941Model it on `Start` (`backend_exec.go:476-636`) minus the probe and the
942socket-mode normalization, and on `StartSidecar` for the no-probe framing:
943
944```go
945func (b *ExecBackend) StartWorker(ctx context.Context, rt registry.Route, extra map[string]string) (Proc, string, error) {
946 var argv []string
947 if err := json.Unmarshal([]byte(rt.Args), &argv); err != nil || len(argv) == 0 {
948 return nil, "", fmt.Errorf("worker %s: no runnable argv in args %q", rt.Host, rt.Args)
949 }
950 if err := b.ensureLiveLink(rt.Host); err != nil {
951 return nil, "", err
952 }
953 spawnVer, _ := b.liveVersion(rt.Host) // same call Start uses; "" = unknown
954
955 fileEnv := b.instanceEnv(rt) // the same merged layers Start composes
956 for k, v := range extra {
957 fileEnv = withVar(fileEnv, k, v)
958 }
959
960 cmd, err := b.Cgroups.Start(rt.Host, func() *exec.Cmd {
961 c := exec.Command(b.liveLink(rt.Host), argv...)
962 c.Env = childEnv(fileEnv)
963 b.teeStd(c, rt)
964 return c
965 })
966 if err != nil {
967 return nil, "", err
968 }
969 p := &childProc{cmd: cmd, done: make(chan struct{})}
970 go func() {
971 p.err = cmd.Wait()
972 b.Cgroups.CleanupHost(rt.Host)
973 close(p.done)
974 }()
975 // Deliberately NO waitHealthy: a worker binds nothing, so there is
976 // nothing to probe. Readiness is the activator's settle window.
977 return p, spawnVer, nil
978}
979```
980
981Use the real names this file already has for the live-link, version and env
982helpers (`ensureLiveLink`, `liveLink`, `instanceEnv`, `childEnv`,
983`withVar`, `teeStd`); do not invent new ones. If `b.Cgroups` may be nil in
984tests, follow whatever nil-guard `Start` uses.
985
986`UnitBackend.StartWorker` and `SplitBackend.StartWorker` mirror
987`StartSidecar` (`backend_unit.go:186-188`, `:304-306`): refuse, and
988delegate by `b.pick(rt)` respectively.
989
990- [ ] **Step 5: Run the tests, full gate, commit.**
991
992---
993
994### Task 6: The standalone sidecar lifecycle inside the activator
995
996**Files:**
997- Modify: `internal/activator/activator.go`
998- Test: `internal/activator/activator_test.go`
999
1000**Interfaces:**
1001- Consumes: Task 5's `StartWorker`.
1002- Produces:
1003 ```go
1004 func (a *Service) WakeWorker(rt registry.Route) error // idempotent; nil if already awake
1005 func (a *Service) NoteWorkerContact(host string) // stamps lastActive
1006 func (a *Service) SetWorkerNextDue(host string, at int64) error
1007 func (a *Service) SleepWorker(host string) // /v1/idle
1008 func (a *Service) WorkerToken(host string) string // "" unless awake
1009 func (a *Service) NoteWorkerReady(host, version string)
1010 ```
1011 `WakeWorker` takes the route ALONE: the idle window and run ceiling ride
1012 on the row (Task 1), so the sweeper never needs a bucket read to decide a
1013 worker has outrun its ceiling.
1014
1015**Design constraint:** sidecars stay entirely off the request path.
1016`beginRequest`, `BeginSidecarRequest` and `ServeHTTP` are not modified —
1017`ServeHTTP` already gates on `rt.Kind == "instance" && rt.Hibernate`.
1018
1019- [ ] **Step 1: Write the failing tests**
1020
1021```go
1022func TestWorkerWakeUsesStartWorkerAndNoProbe(t *testing.T) {
1023 a, b, _, _ := testActivator(t, 10*time.Second)
1024 rt := testWorkerRoute("bot.x.internal")
1025 if err := a.WakeWorker(rt); err != nil {
1026 t.Fatalf("WakeWorker: %v", err)
1027 }
1028 if got := b.events(); !contains(got, "startWorker:bot.x.internal") {
1029 t.Fatalf("events %v — a worker must spawn through StartWorker, never Start", got)
1030 }
1031 if b.startCount() != 0 {
1032 t.Fatal("Start (the socket path, with its health probe) must never be called for a worker")
1033 }
1034}
1035
1036func TestWorkerSleepsWhenIdleAndRecordsNextDue(t *testing.T) {
1037 a, b, clk, reg := testActivator(t, 10*time.Second)
1038 rt := testWorkerRoute("bot.x.internal")
1039 if err := a.WakeWorker(rt); err != nil {
1040 t.Fatalf("WakeWorker: %v", err)
1041 }
1042 if err := a.SetWorkerNextDue("bot.x.internal", 4242); err != nil {
1043 t.Fatalf("SetWorkerNextDue: %v", err)
1044 }
1045
1046 clk.advance(11 * time.Second)
1047 a.sweepOnce()
1048 waitFor(t, "worker hibernation", func() bool { return a.entryCount() == 0 })
1049
1050 got, err := reg.Get("bot.x.internal")
1051 if err != nil {
1052 t.Fatalf("get: %v", err)
1053 }
1054 if got.NextDue != 4242 {
1055 t.Fatalf("next_due = %d, want 4242 — a worker's next-due arrives over the agent socket while it is ALIVE, not from a probe of a process we are about to kill", got.NextDue)
1056 }
1057 if b.dueProbes() != 0 {
1058 t.Fatal("a worker must never be probed for next-due: it has no socket to answer on")
1059 }
1060 if !b.lastRep().isClosed() {
1061 t.Fatal("the lease must be released and the DB parked on a worker sleep, exactly as for an instance")
1062 }
1063}
1064
1065func TestWorkerContactDefersSleep(t *testing.T) {
1066 a, _, clk, _ := testActivator(t, 10*time.Second)
1067 if err := a.WakeWorker(testWorkerRoute("bot.x.internal")); err != nil {
1068 t.Fatalf("WakeWorker: %v", err)
1069 }
1070 clk.advance(9 * time.Second)
1071 a.NoteWorkerContact("bot.x.internal")
1072 clk.advance(9 * time.Second)
1073 a.sweepOnce()
1074 if a.entryCount() != 1 {
1075 t.Fatal("agent-socket contact must defer the idle clock — it is a worker's only sign of life")
1076 }
1077}
1078
1079func TestWorkerMaxRunStopsALoopingWorker(t *testing.T) {
1080 a, _, clk, _ := testActivator(t, time.Hour) // idle window never fires
1081 rt := testWorkerRoute("bot.x.internal")
1082 rt.MaxRunSeconds = 60
1083 if err := a.WakeWorker(rt); err != nil {
1084 t.Fatalf("WakeWorker: %v", err)
1085 }
1086 for i := 0; i < 10; i++ { // a worker that keeps talking but never idles
1087 clk.advance(10 * time.Second)
1088 a.NoteWorkerContact("bot.x.internal")
1089 }
1090 a.sweepOnce()
1091 waitFor(t, "max_run stop", func() bool { return a.entryCount() == 0 })
1092}
1093
1094func TestSleepWorkerIsImmediate(t *testing.T) {
1095 a, _, _, _ := testActivator(t, time.Hour)
1096 if err := a.WakeWorker(testWorkerRoute("bot.x.internal")); err != nil {
1097 t.Fatalf("WakeWorker: %v", err)
1098 }
1099 a.SleepWorker("bot.x.internal")
1100 waitFor(t, "idle-requested stop", func() bool { return a.entryCount() == 0 })
1101}
1102
1103func TestWakeWorkerIsIdempotent(t *testing.T) {
1104 a, b, _, _ := testActivator(t, time.Hour)
1105 rt := testWorkerRoute("bot.x.internal")
1106 for i := 0; i < 3; i++ {
1107 if err := a.WakeWorker(rt); err != nil {
1108 t.Fatalf("WakeWorker #%d: %v", i, err)
1109 }
1110 }
1111 if n := b.workerStarts(); n != 1 {
1112 t.Fatalf("workerStarts = %d, want 1 — a wake for an already-awake worker is a no-op, not a second child", n)
1113 }
1114}
1115```
1116
1117Extend `fakeBackend` with `StartWorker` (recording `startWorker:<host>` in
1118its `order` log and incrementing a `workerStarts` counter), and add
1119`testWorkerRoute(host)` returning
1120`registry.Route{Host: host, Kind: "sidecar", Args: `["sidecar","run"]`, DBPath: ..., Hibernate: true, Status: "ready"}`.
1121Add the `dueProbes()` accessor if the fake does not already count `NextDue`
1122calls.
1123
1124- [ ] **Step 2: Run and watch them fail.**
1125
1126- [ ] **Step 3: Implement**
1127
1128Add to `actEntry`:
1129
1130```go
1131 worker bool // this entry is a sidecar: no socket, no requests
1132 workerIdle time.Duration // 0 = fall back to cfg.Idle
1133 workerMaxRun time.Duration // 0 = no ceiling
1134 workerToken string // the per-wake agent-socket token
1135 spawnedAt time.Time
1136```
1137
1138populated from the route's `IdleSeconds`/`MaxRunSeconds` at wake time.
1139
1140```go
1141// WakeWorker brings a sidecar up, or does nothing if it is already awake.
1142// It is the request-less twin of beginRequest: a sidecar has no socket, so
1143// no request can ever wake one — every wake arrives from the due-time
1144// sweep, an operator command, or the agent's boot pass.
1145func (a *Service) WakeWorker(rt registry.Route) error
1146```
1147
1148Implementation:
1149
1150- Under `a.mu`: if an entry exists and is `actAwake` or `actWaking`, return
1151 nil (idempotent). If `actStopping`, wait on `e.stopped` then retry.
1152- Otherwise create the entry with `worker: true`, `workerMaxRun: maxRun`,
1153 mint `workerToken` (32 bytes from `crypto/rand`, hex), launch
1154 `go a.wake(e)` and **do not unlock before falling into the wait** — the
1155 singleflight discipline PR #199 established; then wait on `e.ready`.
1156- In `doWake`, branch step 4:
1157 ```go
1158 var (
1159 proc Proc
1160 spawnVer string
1161 )
1162 if e.worker {
1163 proc, spawnVer, err = a.backend.StartWorker(ctx, rt, a.workerEnv(e))
1164 } else {
1165 proc, spawnVer, err = a.backend.Start(ctx, rt)
1166 }
1167 ```
1168 and skip step 6 (sidecar co-spawn) entirely for a worker — a worker is
1169 nobody's parent.
1170- In `stop`, skip the next-due probe when `e.worker`: its next-due arrived
1171 over the agent socket while it was alive. Everything after (stop the
1172 proc, final checkpoint, lease release) is shared unchanged.
1173- In `sweepOnce`, beside the existing idle check, add:
1174 ```go
1175 if e.worker && e.workerMaxRun > 0 && now.Sub(e.spawnedAt) >= e.workerMaxRun {
1176 // A worker that never reports idle still has a ceiling: a wedged
1177 // reconnect loop must not hold a lease and a box forever.
1178 a.beginStopLocked(e)
1179 continue
1180 }
1181 ```
1182 and make the existing idle predicate use the entry's own window when set
1183 (`e.workerIdle`), falling back to `a.cfg.Idle`. Otherwise it is unchanged:
1184 `inflight` is always 0 for a worker, and `lastActive` is stamped by
1185 agent-socket contact instead of by a released request.
1186
1187The four small public methods:
1188
1189```go
1190func (a *Service) NoteWorkerContact(host string) {
1191 a.mu.Lock()
1192 defer a.mu.Unlock()
1193 if e := a.entries[host]; e != nil && e.worker {
1194 e.lastActive = a.now()
1195 }
1196}
1197
1198func (a *Service) SetWorkerNextDue(host string, at int64) error {
1199 return a.due.SetNextDue(host, at)
1200}
1201
1202func (a *Service) SleepWorker(host string) {
1203 a.mu.Lock()
1204 defer a.mu.Unlock()
1205 if e := a.entries[host]; e != nil && e.worker && e.state == actAwake {
1206 a.beginStopLocked(e)
1207 }
1208}
1209
1210// WorkerToken reports the live per-wake token for host, or "" — the agent
1211// socket's authorization check. A token from a previous wake never matches.
1212func (a *Service) WorkerToken(host string) string {
1213 a.mu.Lock()
1214 defer a.mu.Unlock()
1215 if e := a.entries[host]; e != nil && e.worker && e.state == actAwake {
1216 return e.workerToken
1217 }
1218 return ""
1219}
1220```
1221
1222`a.workerEnv(e)` returns
1223`{"CARLOS_AGENT_SOCKET": a.cfg.AgentSocket, "CARLOS_SIDECAR_TOKEN": e.workerToken, "CARLOS_SIDECAR_HOST": e.host, "CARLOS_SIDECAR_NAME": <first label of e.host>}`.
1224Add `AgentSocket string` to `activator.Config`, defaulting to
1225`/run/carlos/agent.sock`.
1226
1227- [ ] **Step 4: Run the activator suite in full**
1228
1229```bash
1230GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
1231 go test ./internal/activator/ -count=1
1232```
1233Expected: PASS, **including every pre-existing instance test** — especially
1234`TestActivatorFailedWakeIsSingleflight`, `TestActivatorCoStopOrdering` and
1235`TestActivatorIdleSleep`. If any instance test changed behaviour, you put a
1236worker branch on a shared path; move it.
1237
1238- [ ] **Step 5: Full gate, commit.**
1239
1240---
1241
1242### Task 7: The agent socket
1243
1244**Files:**
1245- Create: `internal/agentsock/agentsock.go`, `internal/agentsock/agentsock_test.go`
1246
1247**Interfaces:**
1248- Consumes: an interface it defines and Task 6 satisfies:
1249 ```go
1250 type Workers interface {
1251 WorkerToken(host string) string
1252 NoteWorkerContact(host string)
1253 SetWorkerNextDue(host string, at int64) error
1254 SleepWorker(host string)
1255 NoteWorkerReady(host, version string)
1256 }
1257 ```
1258- Produces:
1259 ```go
1260 func New(w Workers) *Server
1261 func (s *Server) Serve(ctx context.Context, path string) error // listens, cleans up
1262 func (s *Server) Handler() http.Handler // for tests
1263 ```
1264
1265**The direction is the whole point:** the agent listens, children dial. The
1266platform never dials a sidecar.
1267
1268Wire protocol — all `POST`, JSON in, `204` out, `Authorization: Bearer <token>`
1269and `Carlos-Sidecar-Host: <host>`:
1270
1271| Path | Body | Effect |
1272|---|---|---|
1273| `/v1/ready` | `{"version":"<build>"}` | ends the settle window, records the running version |
1274| `/v1/next-due` | `{"at": <unix seconds>}` | `SetWorkerNextDue` |
1275| `/v1/idle` | — | `SleepWorker` |
1276| `/v1/heartbeat` | — | `NoteWorkerContact` |
1277
1278- [ ] **Step 1: Write the failing tests**
1279
1280```go
1281type fakeWorkers struct {
1282 mu sync.Mutex
1283 tokens map[string]string
1284 contacts []string
1285 due map[string]int64
1286 slept []string
1287 ready map[string]string
1288}
1289
1290func (f *fakeWorkers) WorkerToken(host string) string { /* guarded read */ }
1291// ... the rest recording into the slices/maps
1292
1293func TestAgentSockRejectsWrongToken(t *testing.T) {
1294 f := &fakeWorkers{tokens: map[string]string{"bot.x.internal": "good"}}
1295 h := New(f).Handler()
1296
1297 for _, tc := range []struct{ name, host, tok string }{
1298 {"wrong token", "bot.x.internal", "bad"},
1299 {"no token", "bot.x.internal", ""},
1300 {"unknown host", "nope.x.internal", "good"},
1301 {"token for another host", "other.x.internal", "good"},
1302 } {
1303 rec := post(h, "/v1/heartbeat", tc.host, tc.tok, "")
1304 if rec.Code != http.StatusUnauthorized {
1305 t.Fatalf("%s: code = %d, want 401", tc.name, rec.Code)
1306 }
1307 }
1308 if len(f.contacts) != 0 {
1309 t.Fatal("a refused call must have no effect at all")
1310 }
1311}
1312
1313func TestAgentSockTokenIsScopedToOneHost(t *testing.T) {
1314 // The security property: possession of A's token says nothing about B.
1315 f := &fakeWorkers{tokens: map[string]string{"a.x.internal": "tok-a", "b.x.internal": "tok-b"}}
1316 h := New(f).Handler()
1317 if rec := post(h, "/v1/idle", "b.x.internal", "tok-a", ""); rec.Code != http.StatusUnauthorized {
1318 t.Fatalf("A's token spoke about B: code = %d", rec.Code)
1319 }
1320 if len(f.slept) != 0 {
1321 t.Fatal("B must not have been slept by A's token")
1322 }
1323}
1324
1325func TestAgentSockVerbs(t *testing.T) {
1326 f := &fakeWorkers{tokens: map[string]string{"bot.x.internal": "good"}}
1327 h := New(f).Handler()
1328
1329 if rec := post(h, "/v1/ready", "bot.x.internal", "good", `{"version":"abc123"}`); rec.Code != http.StatusNoContent {
1330 t.Fatalf("ready: %d", rec.Code)
1331 }
1332 if f.ready["bot.x.internal"] != "abc123" {
1333 t.Fatal("ready must record the running version — it is a sidecar's only version signal")
1334 }
1335 if rec := post(h, "/v1/next-due", "bot.x.internal", "good", `{"at":1756}`); rec.Code != http.StatusNoContent {
1336 t.Fatalf("next-due: %d", rec.Code)
1337 }
1338 if f.due["bot.x.internal"] != 1756 {
1339 t.Fatalf("due = %d", f.due["bot.x.internal"])
1340 }
1341 if rec := post(h, "/v1/idle", "bot.x.internal", "good", ""); rec.Code != http.StatusNoContent {
1342 t.Fatalf("idle: %d", rec.Code)
1343 }
1344 if len(f.slept) != 1 {
1345 t.Fatal("idle must ask for a stop")
1346 }
1347}
1348
1349func TestAgentSockRefusesGarbage(t *testing.T) {
1350 f := &fakeWorkers{tokens: map[string]string{"bot.x.internal": "good"}}
1351 h := New(f).Handler()
1352 for _, body := range []string{"not json", `{"at":"soon"}`, strings.Repeat("x", 1<<20)} {
1353 rec := post(h, "/v1/next-due", "bot.x.internal", "good", body)
1354 if rec.Code != http.StatusBadRequest {
1355 t.Fatalf("body %.20q: code = %d, want 400", body, rec.Code)
1356 }
1357 }
1358 if rec := post(h, "/v1/nope", "bot.x.internal", "good", ""); rec.Code != http.StatusNotFound {
1359 t.Fatalf("unknown verb: %d", rec.Code)
1360 }
1361 req := httptest.NewRequest("GET", "/v1/idle", nil)
1362 // GET on a POST-only verb
1363 rec := httptest.NewRecorder()
1364 h.ServeHTTP(rec, req)
1365 if rec.Code != http.StatusMethodNotAllowed && rec.Code != http.StatusUnauthorized {
1366 t.Fatalf("GET: %d", rec.Code)
1367 }
1368}
1369
1370func TestAgentSockServeCleansUpItsSocket(t *testing.T) {
1371 dir := t.TempDir()
1372 path := filepath.Join(dir, "agent.sock")
1373 ctx, cancel := context.WithCancel(context.Background())
1374 done := make(chan error, 1)
1375 go func() { done <- New(&fakeWorkers{}).Serve(ctx, path) }()
1376 waitFor(t, "socket to appear", func() bool { _, err := os.Stat(path); return err == nil })
1377 cancel()
1378 <-done
1379 if _, err := os.Stat(path); !os.IsNotExist(err) {
1380 t.Fatal("a stale socket file must not survive shutdown — the next start would fail to bind")
1381 }
1382}
1383```
1384
1385Write `post(h http.Handler, path, host, tok, body string) *httptest.ResponseRecorder`
1386as a small local helper.
1387
1388- [ ] **Step 2: Run and watch it fail.**
1389
1390- [ ] **Step 3: Implement**
1391
1392Package doc first, stating the direction:
1393
1394```go
1395// Package agentsock is the agent's own listening socket — the ONE channel
1396// between a sidecar and the platform, and it points inward: children dial
1397// the agent, the agent never dials a child. That is what lets a sidecar
1398// bind nothing, and what lets the identical artifact run off-platform
1399// behind NAT with no protocol difference.
1400//
1401// Every exec child runs as the agent's user, so the socket's file mode is
1402// not an identity boundary. Identity is the per-wake token the activator
1403// mints and passes in the child's environment (CARLOS_SIDECAR_TOKEN),
1404// scoped to one host for one wake: a token for A cannot speak about B, and
1405// a token from a previous wake matches nothing.
1406package agentsock
1407```
1408
1409Implementation notes that the tests pin:
1410
1411- Compare tokens with `crypto/subtle.ConstantTimeCompare`, and refuse when
1412 `WorkerToken(host)` returns `""` (asleep or unknown) — one 401 for every
1413 failure, no distinguishing message.
1414- Bound the body: `io.LimitReader(r.Body, 4096)`, the `NextDue` decode
1415 precedent in `backend_exec.go:451`.
1416- `Serve` should `os.Remove(path)` before binding (a stale socket from a
1417 hard kill), `net.Listen("unix", path)`, `os.Chmod(path, 0o660)`, serve
1418 with `http.Server`, and on `ctx.Done()` shut down and remove the file.
1419- No timeouts longer than a few seconds; these are local calls.
1420
1421- [ ] **Step 4: Run the tests, full gate, commit.**
1422
1423---
1424
1425### Task 8: Wire the socket, the boot pass and the ceilings into the agent
1426
1427**Files:**
1428- Modify: `internal/host/agent.go`
1429- Modify: `internal/activator/activator.go` (implement `NoteWorkerReady`)
1430- Test: `internal/host/agent_test.go`
1431
1432**Interfaces:**
1433- Consumes: Tasks 6 and 7.
1434- Produces: a running agent socket on every box that runs the activator, and
1435 a boot pass that starts non-hibernating sidecars.
1436
1437- [ ] **Step 1: Write the failing test**
1438
1439```go
1440func TestWorkerBootPassStartsOnlyNonHibernatingSidecars(t *testing.T) {
1441 routes := []registry.Route{
1442 {Host: "a.x.sidecar.internal", Kind: "sidecar", Hibernate: false, Status: "ready"},
1443 {Host: "b.x.sidecar.internal", Kind: "sidecar", Hibernate: true, Status: "ready"},
1444 {Host: "c.example.com", Kind: "instance", Hibernate: false, Status: "ready"},
1445 {Host: "d.x.sidecar.internal", Kind: "sidecar", Hibernate: false, Status: "draining"},
1446 }
1447 got := workersToStart(routes)
1448 if len(got) != 1 || got[0].Host != "a.x.sidecar.internal" {
1449 t.Fatalf("workersToStart = %v — only a ready, non-hibernating sidecar starts at boot; a hibernating one waits for its timer or a command", got)
1450 }
1451}
1452```
1453
1454- [ ] **Step 2: Run and watch it fail.**
1455
1456- [ ] **Step 3: Implement**
1457
1458```go
1459// workersToStart is the boot pass's selection: a sidecar that does not
1460// hibernate must be running whenever the agent is. A hibernating sidecar is
1461// deliberately NOT started here — it waits for its due time, an operator
1462// command, or (later) a delivery signal, which is the whole point of the
1463// class.
1464func workersToStart(routes []registry.Route) []registry.Route {
1465 var out []registry.Route
1466 for _, rt := range routes {
1467 if rt.Kind == "sidecar" && !rt.Hibernate && (rt.Status == "" || rt.Status == "ready") {
1468 out = append(out, rt)
1469 }
1470 }
1471 return out
1472}
1473```
1474
1475Then, in the agent's startup block beside the other loops:
1476
1477- construct `agentsock.New(svc)` and `go srv.Serve(ctx, "/run/carlos/agent.sock")`
1478 **only when the activator is enabled** (the same condition that
1479 constructs the activator — with hibernation off there are no workers).
1480- run the boot pass once after the first registry read, calling
1481 `svc.WakeWorker(rt)` for each result.
1482- The sweeper already handles the due-time wakes; make sure its
1483 `DueInstances` results route to `WakeWorker` for `Kind == "sidecar"`
1484 rather than the instance wake path.
1485
1486Add `NoteWorkerReady(host, version string)` to the activator: stamp
1487`lastActive`, mark the settle window satisfied, and record the version via
1488the registry's existing version setter so §9.2's honesty rule holds (a
1489sidecar that never reports shows unknown, never "current").
1490
1491- [ ] **Step 4: Run the tests, full gate, commit.**
1492
1493---
1494
1495### Task 9: The sidecar reconciler — and the removal-authority collision
1496
1497**Files:**
1498- Create: `internal/provision/sidecar_reconcile.go`, `sidecar_reconcile_test.go`
1499- Modify: `internal/provision/reconcile.go` (removal phase, ~651-673)
1500- Modify: `internal/host/agent.go` (call it through `reconcileSerialized`)
1501
1502**Interfaces:**
1503- Consumes: Task 2's shapes, Task 1's `Args`.
1504- Produces: `func ReconcileSidecars(ctx context.Context, s release.Store, reg *registry.Registry, cfg ReconcileConfig) error`
1505
1506**The collision, stated plainly:** the instance pass's removal phase is
1507scoped by `Provisioned`, region, pool and absence-from-records, with **no
1508Kind check**. Sidecar rows are `Provisioned` and can never appear in the
1509instance record set, so the existing pass would delete every sidecar on
1510every tick — minting and destroying forever, with no error anywhere. Each
1511pass's removal authority must be scoped to its own Kind.
1512
1513- [ ] **Step 1: Write the failing tests**
1514
1515```go
1516func TestInstanceReconcileNeverRemovesSidecarRows(t *testing.T) {
1517 f := setup(t) // reconcile_test.go's existing fixture
1518 must(t, f.reg.Upsert(registry.Route{
1519 Host: "bot.eleven.bdm.sidecar.internal", Kind: "sidecar",
1520 App: "eleven", Account: "bdm", Provisioned: true, Status: "ready",
1521 }))
1522 // No instance records at all: the pass's removal phase runs in full.
1523 if err := Reconcile(context.Background(), f.store, f.reg, ReconcileConfig{DataDir: f.dir}); err != nil {
1524 t.Fatalf("Reconcile: %v", err)
1525 }
1526 if _, err := f.reg.Get("bot.eleven.bdm.sidecar.internal"); err != nil {
1527 t.Fatal("the instance pass deleted a sidecar row — each pass's removal authority is scoped to its OWN kind, or the two passes destroy each other's work forever")
1528 }
1529}
1530
1531func TestSidecarReconcileMints(t *testing.T) {
1532 f := setup(t)
1533 seedAccount(t, f.store, "bdm")
1534 writeSidecarManifest(t, f.store, "bdm", "eleven", SidecarManifest{
1535 Schema: 1, App: "eleven", Args: []string{"sidecar", "run"}, Channel: "stable",
1536 })
1537 writeSidecarRecord(t, f.store, SidecarRecord{
1538 Schema: 1, Name: "bot", Host: SidecarHost("bot", "eleven", "bdm"),
1539 App: "eleven", Account: "bdm", Channel: "stable",
1540 })
1541
1542 if err := ReconcileSidecars(context.Background(), f.store, f.reg, ReconcileConfig{DataDir: f.dir}); err != nil {
1543 t.Fatalf("ReconcileSidecars: %v", err)
1544 }
1545 rt := mustGet(t, f.reg, SidecarHost("bot", "eleven", "bdm"))
1546 if rt.Kind != "sidecar" {
1547 t.Fatalf("Kind = %q", rt.Kind)
1548 }
1549 if rt.Socket != "" || rt.Addr != "" {
1550 t.Fatalf("a sidecar row must carry NO socket and NO addr, got socket=%q addr=%q", rt.Socket, rt.Addr)
1551 }
1552 if rt.DBPath == "" || !rt.Provisioned || rt.Args != `["sidecar","run"]` {
1553 t.Fatalf("route = %+v", rt)
1554 }
1555}
1556
1557func TestSidecarReconcileRemovesOnlyItsOwnKind(t *testing.T) {
1558 f := setup(t)
1559 seedAccount(t, f.store, "bdm")
1560 must(t, f.reg.Upsert(registry.Route{Host: "gone.eleven.bdm.sidecar.internal",
1561 Kind: "sidecar", App: "eleven", Account: "bdm", Provisioned: true}))
1562 must(t, f.reg.Upsert(registry.Route{Host: "app.example.com",
1563 Kind: "instance", App: "eleven", Account: "bdm", Provisioned: true}))
1564
1565 if err := ReconcileSidecars(context.Background(), f.store, f.reg, ReconcileConfig{DataDir: f.dir}); err != nil {
1566 t.Fatalf("ReconcileSidecars: %v", err)
1567 }
1568 if _, err := f.reg.Get("gone.eleven.bdm.sidecar.internal"); err == nil {
1569 t.Fatal("a provisioned sidecar with no record must be removed")
1570 }
1571 if _, err := f.reg.Get("app.example.com"); err != nil {
1572 t.Fatal("the sidecar pass must never touch an instance row")
1573 }
1574}
1575
1576func TestSidecarReconcileSkipsRemovalWhenAReadFailed(t *testing.T) {
1577 // The fail-safe: a transient read failure must never mass-retire a
1578 // tenant's fleet. Minting still proceeds; minting is additive.
1579 f := setup(t)
1580 seedAccount(t, f.store, "bdm")
1581 must(t, f.reg.Upsert(registry.Route{Host: "keep.eleven.bdm.sidecar.internal",
1582 Kind: "sidecar", App: "eleven", Account: "bdm", Provisioned: true}))
1583 f.store.failReadsFor(SidecarsPrefix("bdm", "eleven")) // fixture hook
1584
1585 _ = ReconcileSidecars(context.Background(), f.store, f.reg, ReconcileConfig{DataDir: f.dir})
1586 if _, err := f.reg.Get("keep.eleven.bdm.sidecar.internal"); err != nil {
1587 t.Fatal("a read failure skipped nothing — one bad list must not retire a fleet")
1588 }
1589}
1590
1591func TestSidecarReconcileRefusesHostWithAStagedCoSpawnFile(t *testing.T) {
1592 f := setup(t)
1593 seedAccount(t, f.store, "bdm")
1594 host := SidecarHost("bot", "eleven", "bdm")
1595 must(t, os.WriteFile(filepath.Join(f.dir, host+"-sidecar.env"), []byte("X=1\n"), 0o600))
1596 writeSidecarManifest(t, f.store, "bdm", "eleven", SidecarManifest{Schema: 1, App: "eleven", Args: []string{"sidecar", "run"}})
1597 writeSidecarRecord(t, f.store, SidecarRecord{Schema: 1, Name: "bot", Host: host, App: "eleven", Account: "bdm"})
1598
1599 _ = ReconcileSidecars(context.Background(), f.store, f.reg, ReconcileConfig{DataDir: f.dir})
1600 if _, err := f.reg.Get(host); err == nil {
1601 t.Fatal("a host with a staged legacy co-spawn file must not also get a native sidecar row — one host, one mechanism")
1602 }
1603}
1604
1605func TestSidecarReconcileConvergesArgsFromTheManifest(t *testing.T) {
1606 f := setup(t)
1607 seedAccount(t, f.store, "bdm")
1608 writeSidecarManifest(t, f.store, "bdm", "eleven", SidecarManifest{Schema: 1, App: "eleven", Args: []string{"sidecar", "run"}})
1609 writeSidecarRecord(t, f.store, SidecarRecord{Schema: 1, Name: "bot", Host: SidecarHost("bot", "eleven", "bdm"), App: "eleven", Account: "bdm"})
1610 must(t, ReconcileSidecars(context.Background(), f.store, f.reg, ReconcileConfig{DataDir: f.dir}))
1611
1612 writeSidecarManifest(t, f.store, "bdm", "eleven", SidecarManifest{Schema: 1, App: "eleven", Args: []string{"worker", "run"}})
1613 must(t, ReconcileSidecars(context.Background(), f.store, f.reg, ReconcileConfig{DataDir: f.dir}))
1614
1615 rt := mustGet(t, f.reg, SidecarHost("bot", "eleven", "bdm"))
1616 if rt.Args != `["worker","run"]` {
1617 t.Fatalf("Args = %q — truth lives in the bucket and is re-stamped every pass, exactly as seed is", rt.Args)
1618 }
1619}
1620
1621func TestSidecarReconcileConvergesTheCeilings(t *testing.T) {
1622 f := setup(t)
1623 seedAccount(t, f.store, "bdm")
1624 writeSidecarManifest(t, f.store, "bdm", "eleven", SidecarManifest{
1625 Schema: 1, App: "eleven", Args: []string{"sidecar", "run"}, IdleSeconds: 300, MaxRunSeconds: 3600})
1626 writeSidecarRecord(t, f.store, SidecarRecord{Schema: 1, Name: "bot",
1627 Host: SidecarHost("bot", "eleven", "bdm"), App: "eleven", Account: "bdm"})
1628 must(t, ReconcileSidecars(context.Background(), f.store, f.reg, ReconcileConfig{DataDir: f.dir}))
1629
1630 writeSidecarManifest(t, f.store, "bdm", "eleven", SidecarManifest{
1631 Schema: 1, App: "eleven", Args: []string{"sidecar", "run"}, IdleSeconds: 60, MaxRunSeconds: 120})
1632 must(t, ReconcileSidecars(context.Background(), f.store, f.reg, ReconcileConfig{DataDir: f.dir}))
1633
1634 rt := mustGet(t, f.reg, SidecarHost("bot", "eleven", "bdm"))
1635 if rt.IdleSeconds != 60 || rt.MaxRunSeconds != 120 {
1636 t.Fatalf("ceilings = %d/%d, want 60/120 — the sweeper reads the row, so the row must follow the manifest", rt.IdleSeconds, rt.MaxRunSeconds)
1637 }
1638}
1639```
1640
1641- [ ] **Step 2: Run and watch them fail.**
1642
1643- [ ] **Step 3: Guard the instance pass's removal phase**
1644
1645In `reconcile.go`'s removal loop, in the same condition that already checks
1646`rt.Provisioned` and scope:
1647
1648```go
1649 // Removal authority is scoped to this pass's OWN kind. A sidecar row
1650 // is Provisioned too, and can never appear in the instance record
1651 // set — without this guard the two passes would delete each other's
1652 // rows on every tick, forever, with no error anywhere.
1653 if rt.Kind == "sidecar" {
1654 continue
1655 }
1656```
1657
1658- [ ] **Step 4: Implement `ReconcileSidecars`**
1659
1660Mirror `Reconcile` (`reconcile.go:330-675`) exactly, with these
1661substitutions and additions:
1662
1663- records come from `ParseSidecarKey` over `apps/`, manifests from
1664 `GetSidecarManifest`.
1665- scope is the same `rec.Region != cfg.Region || !poolServed(cfg.Pools, rec.Pool)` skip.
1666- **placement gate**: skip a record whose `PlacementOrDefault()` is
1667 `"no-inbound"` unless `cfg.Inbound == "none"` (Task 10 supplies the field).
1668- mint:
1669 ```go
1670 func sidecarRouteFor(rec SidecarRecord, m SidecarManifest, dataDir string) registry.Route {
1671 argv, _ := json.Marshal(m.Args)
1672 return registry.Route{
1673 Host: rec.Host, Slug: rec.Name, Kind: "sidecar",
1674 App: rec.App, Account: rec.Account, Pool: rec.Pool,
1675 Region: rec.Region, Channel: rec.Channel, Environment: rec.Environment,
1676 DBPath: DefaultDBPath(dataDir, rec.Host),
1677 Hibernate: m.HibernateOrDefault(),
1678 Args: string(argv),
1679 IdleSeconds: int(m.IdleOrDefault() / time.Second),
1680 MaxRunSeconds: int(m.MaxRunOrDefault() / time.Second),
1681 Provisioned: true,
1682 Status: "ready",
1683 // Socket and Addr are deliberately left empty: a sidecar binds
1684 // nothing, and the edge refuses the kind outright.
1685 }
1686 }
1687 ```
1688- converge: re-stamp `Args` (narrow `SetArgs`), the ceilings (narrow
1689 `SetWorkerLimits`), `Channel`, `Environment`, and `Hibernate` when they
1690 differ.
1691- removal: only `rt.Kind == "sidecar" && rt.Provisioned`, in scope, absent
1692 from the record set, and not in `protect`; **skipped entirely when any
1693 read failed**.
1694- the co-spawn conflict: before minting, `os.Stat(filepath.Join(cfg.DataDir, rec.Host+"-sidecar.env"))`
1695 — if it exists, log loudly and skip. One host, one mechanism.
1696
1697Finally, add the leg to `internal/host/agent.go` beside the instance one,
1698**inside `reconcileSerialized`'s mutex** so it cannot race the instance pass
1699(the comment at `agent.go:525-537` explains exactly why an overlapping pass
1700is dangerous), and inside `fullPass` so a console serial bump converges it
1701within ~2s.
1702
1703- [ ] **Step 5: Run the tests, full gate, commit.**
1704
1705---
1706
1707### Task 10: Placement — boxes that accept no-inbound work
1708
1709**Files:**
1710- Modify: `internal/host/agent.go` (`provisionTickConfig`, ~1316-1334)
1711- Modify: `internal/provision/reconcile.go` (mint gate)
1712- Modify: `internal/provision/sidecar_reconcile.go` (placement gate)
1713- Test: `internal/host/agent_test.go`, `internal/provision/reconcile_test.go`
1714
1715**Interfaces:**
1716- Consumes: Task 9.
1717- Produces: `ReconcileConfig.Inbound string` (`""`/`"any"` or `"none"`).
1718
1719- [ ] **Step 1: Write the failing tests**
1720
1721```go
1722func TestProvisionTickConfigReadsInbound(t *testing.T) {
1723 env := map[string]string{"CARLOS_REGION": "ie", "CARLOS_POOLS": "eleven", "CARLOS_INBOUND": "none"}
1724 cfg := provisionTickConfig(func(k string) string { return env[k] })
1725 if cfg.Inbound != "none" {
1726 t.Fatalf("Inbound = %q", cfg.Inbound)
1727 }
1728 cfg = provisionTickConfig(func(string) string { return "" })
1729 if cfg.Inbound != "" {
1730 t.Fatal("an unset CARLOS_INBOUND must be an inert default — today's behaviour exactly")
1731 }
1732}
1733
1734func TestNoInboundBoxRefusesRoutedInstances(t *testing.T) {
1735 f := setup(t)
1736 seedAccount(t, f.store, "bdm")
1737 writeManifest(t, f.store, "bdm", "tito", Manifest{Schema: 1, App: "tito", Domain: "go.tito.io"})
1738 writeRecord(t, f.store, Record{Schema: 1, Host: "acme.go.tito.io", Slug: "acme", App: "tito", Account: "bdm", Channel: "stable"})
1739
1740 err := Reconcile(context.Background(), f.store, f.reg, ReconcileConfig{DataDir: f.dir, Inbound: "none"})
1741 if err != nil {
1742 t.Fatalf("Reconcile: %v", err)
1743 }
1744 if _, err := f.reg.Get("acme.go.tito.io"); err == nil {
1745 t.Fatal("a box that cannot serve must never be handed something to serve — a misfiled record is a log line, not a silent outage")
1746 }
1747}
1748
1749func TestNoInboundPlacementOnlyLandsOnMatchingBoxes(t *testing.T) {
1750 f := setup(t)
1751 seedAccount(t, f.store, "bdm")
1752 writeSidecarManifest(t, f.store, "bdm", "eleven", SidecarManifest{
1753 Schema: 1, App: "eleven", Args: []string{"sidecar", "run"}, Placement: "no-inbound"})
1754 writeSidecarRecord(t, f.store, SidecarRecord{Schema: 1, Name: "bot",
1755 Host: SidecarHost("bot", "eleven", "bdm"), App: "eleven", Account: "bdm", Placement: "no-inbound"})
1756
1757 must(t, ReconcileSidecars(context.Background(), f.store, f.reg, ReconcileConfig{DataDir: f.dir})) // ordinary box
1758 if _, err := f.reg.Get(SidecarHost("bot", "eleven", "bdm")); err == nil {
1759 t.Fatal("a no-inbound workload must not land on a box that accepts inbound work")
1760 }
1761
1762 must(t, ReconcileSidecars(context.Background(), f.store, f.reg, ReconcileConfig{DataDir: f.dir, Inbound: "none"}))
1763 if _, err := f.reg.Get(SidecarHost("bot", "eleven", "bdm")); err != nil {
1764 t.Fatal("it must land on a box that declares CARLOS_INBOUND=none")
1765 }
1766}
1767```
1768
1769- [ ] **Step 2: Run and watch them fail.**
1770- [ ] **Step 3: Implement** — add the field, read the env var, and add the
1771 two gates with the comments the tests' failure messages quote.
1772- [ ] **Step 4: Run the tests, full gate, commit.**
1773
1774---
1775
1776### Task 11: Config delivery and restart reach sidecars
1777
1778**Files:**
1779- Modify: `internal/appconfig/syncexec.go` (`execRouteBase`, ~398-407)
1780- Modify: `internal/host/restart.go` (~77, 192, 545)
1781- Test: `internal/appconfig/syncexec_test.go`, `internal/host/restart_test.go`
1782
1783**`release.Adoptable` is deliberately NOT changed.** It is the most reused
1784predicate in the repo — adopt membership, capability convergence, socket and
1785backing advisories, status pages — and every one of those is correct to
1786exclude a sidecar. Each consumer that genuinely wants sidecars opts in by
1787name. Two deliberate edits instead of one blind one.
1788
1789- [ ] **Step 1: Write the failing tests**
1790
1791```go
1792func TestSidecarGetsItsConfigEnvFile(t *testing.T) {
1793 rt := registry.Route{Host: "bot.eleven.bdm.sidecar.internal", Kind: "sidecar",
1794 App: "eleven", Status: "ready"}
1795 if !execRouteBase(rt) {
1796 t.Fatal("a sidecar must receive config delivery — this is the direct replacement for the hand-staged <host>-sidecar.env, and the whole reason the file path existed")
1797 }
1798}
1799
1800func TestSidecarConfigStillExcludesPlatformAndUnitRoutes(t *testing.T) {
1801 base := registry.Route{Host: "bot.x.sidecar.internal", Kind: "sidecar", App: "eleven", Status: "ready"}
1802 plat := base
1803 plat.Platform = true
1804 if execRouteBase(plat) {
1805 t.Fatal("a Platform alias is still excluded")
1806 }
1807 unit := base
1808 unit.Backing = registry.BackingUnit
1809 if execRouteBase(unit) {
1810 t.Fatal("unit-backed routes are still excluded")
1811 }
1812}
1813
1814func TestRestartPassCyclesSidecars(t *testing.T) {
1815 routes := []registry.Route{
1816 {Host: "bot.x.sidecar.internal", Kind: "sidecar", App: "eleven", Account: "bdm", Status: "ready"},
1817 }
1818 if len(restartCandidates(routes)) != 1 {
1819 t.Fatal("carlos restart -app X must cycle that app's sidecars too")
1820 }
1821}
1822```
1823
1824- [ ] **Step 2: Run and watch them fail.**
1825- [ ] **Step 3: Implement** — widen `execRouteBase`'s kind test to
1826 `(rt.Kind == "instance" || rt.Kind == "sidecar")`, leaving every other
1827 clause exactly as it is; and in `restart.go` accept sidecars alongside
1828 `release.Adoptable` routes, cycling them via the activator's stop/wake
1829 rather than the adopt path. Also confirm the reaper in
1830 `syncexec.go:617` still treats `<host>-sidecar.env` as someone else's file
1831 — the legacy path must stay untouched (Task 16 documents why).
1832- [ ] **Step 4: Run the tests, full gate, commit.**
1833
1834---
1835
1836### Task 12: Console API — the sidecar verbs
1837
1838**Files:**
1839- Create: `internal/console/apisidecars.go`, `apisidecars_test.go`
1840- Modify: `internal/console/authcli.go` (`registerCLIRoutes`, ~273-286)
1841
1842**Interfaces:**
1843- Consumes: Task 2's shapes, Task 13's wake object (build 13 first if you
1844 prefer; the wake handler can land in either order).
1845- Produces the routes:
1846 ```
1847 PUT /api/cli/apps/{account}/{app}/sidecars-manifest
1848 GET /api/cli/apps/{account}/{app}/sidecars
1849 POST /api/cli/apps/{account}/{app}/sidecars
1850 GET /api/cli/apps/{account}/{app}/sidecars/{name}
1851 DELETE /api/cli/apps/{account}/{app}/sidecars/{name}
1852 POST /api/cli/apps/{account}/{app}/sidecars/{name}/wake
1853 ```
1854
1855**The app-held token works unchanged.** `CLIAppToken` binds a token to
1856(account, app); these verbs authorize against the same principal, which is
1857exactly the "apps can provision sidecar instances easily" requirement —
1858Eleven's app process, holding no AWS credentials and no box access, creates
1859a bot per enrolled member.
1860
1861Refusal order, copied from `apiinstances.go:474-478` and its tests:
1862**principal gate → rate limit → releases configured → manifest present
1863(409 naming `carlos sidecars enable`) → name shape (400) → region shape and
1864allow-list (400) → channel (400) → shape validation (400) → create-only
1865write (409 on a lost race) → audit + `bumpSerial` → 201.**
1866
1867- [ ] **Step 1: Write the failing tests**
1868
1869Mirror `apiinstances_test.go` exactly, reusing its fixture helpers
1870(`rawReq`, `canonicalNotFound`, `assertNotFound`, `newConsoleFixture`):
1871
1872```go
1873func TestAPISidecarRefusalMatrix(t *testing.T) {
1874 // Every refusal before the principal is proven must be BYTE-IDENTICAL
1875 // to http.NotFound's own output: a non-member must never learn that an
1876 // account, an app, or a sidecar exists.
1877 // ... one case per: no such account, no such app, not a member,
1878 // app token bound to a different app, user token on an app verb.
1879}
1880
1881func TestAPISidecarCreateRefusalOrder(t *testing.T) {
1882 // Walk the order above with one token, asserting each stage's code and
1883 // that the NEXT stage was not reached.
1884}
1885
1886func TestAPISidecarAppTokenContainment(t *testing.T) {
1887 // An app token creates/lists/deletes/wakes its OWN app's sidecars, and
1888 // 401s on promote and on minting another token.
1889}
1890
1891func TestAPISidecarCreateIsCreateOnly(t *testing.T) {
1892 // Second POST for the same name -> 409, never a silent overwrite.
1893}
1894
1895func TestAPISidecarDeleteIsIdempotent(t *testing.T) {
1896 // DELETE of an absent name -> 200. The caller's goal is "not there".
1897 // And it must work after the manifest is gone: deletion outlives policy.
1898}
1899
1900func TestAPISidecarCreateWithoutManifestNamesTheEnableVerb(t *testing.T) {
1901 // 409 whose body names `carlos sidecars enable` — membership is already
1902 // proven at this point, so this is not a leak.
1903}
1904
1905func TestAPISidecarWakeBumpsTheGeneration(t *testing.T) {
1906 // Two wakes -> generation 1 then 2, and the serial moved both times.
1907}
1908```
1909
1910- [ ] **Step 2: Run and watch them fail.**
1911- [ ] **Step 3: Implement** `apisidecars.go` following `apiinstances.go`'s
1912 structure handler-for-handler, and register the routes. The enable handler
1913 is gated by `apiAccountApp` **directly** (it is a human act), like
1914 `handleAPIInstanceEnable`; the rest go through the shared gate.
1915- [ ] **Step 4: Run the tests, full gate, commit.**
1916
1917---
1918
1919### Task 13: The wake generation object and the box-side pass
1920
1921**Files:**
1922- Create: `internal/release/sidecarwake.go`, `sidecarwake_test.go`
1923- Modify: `internal/host/restart.go` or a sibling in `internal/host`
1924- Test: `internal/host/*_test.go`
1925
1926**Interfaces:**
1927- Produces:
1928 ```go
1929 func SidecarWakeKey(acct, app, name string) string // apps/<acct>/<app>/sidecar-wake/<name>.json
1930 type SidecarWake struct{ Schema, Generation int; At, By, Reason string }
1931 ```
1932
1933Copy `restartgen.go` (`internal/release/restartgen.go`) wholesale: a small
1934CAS-guarded generation object whose **etag moving** is the entire signal.
1935The box compares etags per (account, app, name) against a cache and wakes
1936what moved — the `restartGenPass` shape (`internal/host/restart.go:128`).
1937
1938Nothing is pushed at a box. That is what lets the box live in a private
1939subnet.
1940
1941- [ ] **Step 1: Write the failing tests** — an etag move wakes exactly the
1942 named sidecar; an unchanged etag wakes nothing; a first sighting on an
1943 unseeded cache does **not** replay a wake (the no-replay-on-absence rule
1944 the serial watcher already uses, `agent.go:1032-1042`); a read error for
1945 one app does not suppress the others.
1946- [ ] **Step 2: Run and watch them fail.**
1947- [ ] **Step 3: Implement.**
1948- [ ] **Step 4: Run the tests, full gate, commit.**
1949
1950---
1951
1952### Task 14: `carlos sidecars`
1953
1954**Files:**
1955- Create: `sidecars_cli.go`, `sidecars_cli_test.go`, `sidecars_e2e_test.go`
1956- Modify: `main.go` (dispatch), `apimode.go` (client methods)
1957
1958**Interfaces:**
1959- Consumes: Task 12's routes.
1960- Produces the verbs:
1961 ```
1962 carlos sidecars enable -app <app> -args '<argv>' [-placement no-inbound]
1963 [-regions ie,fleet/eleven] [-pool p] [-channel c]
1964 [-idle 5m] [-max-run 1h] [-no-hibernate]
1965 carlos sidecars create -app <app> <name>
1966 carlos sidecars ls -app <app> [-long]
1967 carlos sidecars wake -app <app> <name>
1968 carlos sidecars rm -app <app> <name>
1969 carlos sidecars logs -app <app> <name>
1970 ```
1971
1972Follow `instances_cli.go` precisely: subcommand words dispatch positionally
1973before flag parsing; `enable` works in **both** API and direct/store mode
1974(operator-only flags refused in signed-in mode); `create`/`ls`/`wake`/`rm`
1975are **API-mode only** via `requireAPISession`, whose refusal must name
1976sidecars' own advice, never another verb's.
1977
1978`ls` prints the short name by default and the synthetic host only under
1979`-long`. It shows **running version against desired channel version**, and a
1980sidecar that has never called `/v1/ready` shows `unknown`, never the desired
1981version — a stamp that overclaims is worse than an absent one.
1982
1983`logs` delegates to the existing `carlos logs` path for the sidecar's host.
1984
1985- [ ] **Step 1: Write the failing tests** — unit tests for flag parsing and
1986 the duration flags (`-idle 5m` → `IdleSeconds: 300`), plus an e2e test in
1987 the `instances_e2e_test.go` mould driving a real `console.Server` over
1988 `httptest` end to end: enable → create → ls → wake → rm.
1989
1990 **Every test that shells out to the carlos binary must build its env
1991 through `carlosTestEnv` and set `cmd.Dir` to a scratch dir.** The binary
1992 walks cwd ancestors for `.carlos/config`; the fallback is the operator's
1993 real login and default console. A logged-in laptop running
1994 `go test ./...` once minted 40+ real accounts on the production console.
1995- [ ] **Step 2: Run and watch them fail.**
1996- [ ] **Step 3: Implement.**
1997- [ ] **Step 4: Run the tests, full gate, commit.**
1998
1999---
2000
2001### Task 15: The console tells the truth about sidecars
2002
2003**Files:**
2004- Modify: `internal/console/instances.go` (~88-97), `internal/console/overview.go`
2005 (~129-144, ~237-264, ~286-306), `internal/status/snapshot.go` (~406-427)
2006- Test: the matching `_test.go` files
2007
2008- [ ] **Step 1: Write the failing tests**
2009
2010```go
2011func TestReportingHostIsNeverASidecar(t *testing.T) {
2012 routes := []registry.Route{
2013 {Host: "bot.eleven.bdm.sidecar.internal", Kind: "sidecar", Status: "ready"},
2014 {Host: "eleven.example.com", Kind: "instance", Status: "ready"},
2015 }
2016 if got := reportingHost(routes); got != "eleven.example.com" {
2017 t.Fatalf("reportingHost = %q — polling an unroutable sidecar makes a HEALTHY app read 'not reporting'", got)
2018 }
2019}
2020
2021func TestSidecarIsNotAPublicResident(t *testing.T) {
2022 counts := CollectBoxCounts([]registry.Route{{Host: "bot.x.sidecar.internal", Kind: "sidecar"}})
2023 if counts.Resident != 0 {
2024 t.Fatal("a sidecar must not be counted as a public resident app instance")
2025 }
2026 if counts.Sidecars != 1 {
2027 t.Fatal("it belongs in its own bucket")
2028 }
2029}
2030
2031func TestSidecarHealthWordIsNotServing(t *testing.T) {
2032 if w := healthWord("sidecar", "ready", false, false, false); w == "Serving" {
2033 t.Fatal("a sidecar never serves a request — 'Serving' is false")
2034 }
2035}
2036```
2037
2038- [ ] **Step 2: Run and watch them fail.**
2039- [ ] **Step 3: Implement** — filter the reporting-host picker by Kind, give
2040 `CollectBox` an explicit sidecar branch and a `Sidecars` counter (additive
2041 field), count sidecars separately in the overview, and give `healthWord`
2042 the sidecar vocabulary: `Running`, `Asleep`, `Waking`, `Crashed`,
2043 `Never started`. Sidecars are **not** on the public status page: their
2044 existence is account-visible, not public.
2045- [ ] **Step 4: Run the tests, full gate, commit.**
2046
2047---
2048
2049### Task 16: Documentation, the runbook, and the legacy split
2050
2051**Files:**
2052- Create: `infra/no-inbound-box.md`
2053- Modify: `AGENTS.md` (was `CLAUDE.md` — see Step 3)
2054- Modify: `internal/activator/activator.go` (package doc), `backend_exec.go`
2055 (the `HasSidecar`/`StartSidecar` comments)
2056
2057- [ ] **Step 1: Write `infra/no-inbound-box.md`** — the private-subnet
2058 runbook, in `infra/eleven-fleet.md`'s voice: preconditions, the tofu
2059 shape (private subnet, no IGW route, egress via NAT, no inbound SG rules,
2060 no SSM), the box env contract (`CARLOS_INBOUND=none` plus the fleet
2061 contract), verification, and an abort path at every step. Mark clearly:
2062 **written for Paul's applies; nothing here has been applied.**
2063
2064 Note the easier posture too: fleet boxes are *already* outbound-only dial,
2065 so Eleven's first sidecar box needs no new network shape at all — the
2066 private-subnet variant is the harder posture, available when wanted.
2067
2068- [ ] **Step 2: Record the legacy split** in the activator's package doc and
2069 beside `HasSidecar`: the file-staged `<host>-sidecar.env` co-spawn is
2070 **legacy but live** (Eleven runs on it today), the record path is native,
2071 a host may not have both, and retiring the file path is a follow-up gated
2072 on Eleven's own migration.
2073
2074- [ ] **Step 3: Add an AGENTS.md bullet** under "Rules this repo has paid
2075 for". (Amended 2026-09-05: this said CLAUDE.md, and it was right when it
2076 was written. Main has since reduced CLAUDE.md to a pointer and moved the
2077 guidance — that section included — into AGENTS.md, which is where the
2078 bullet now lives.):
2079
2080 ```markdown
2081 - **A sidecar is a member that nothing can dial.** `Kind == "sidecar"`
2082 rows carry no socket and no addr, are refused by the edge on kind (not
2083 merely unmatched by it), and are never advertised in a fleet binding —
2084 a fleet-placed row reaches the edge's fleet arm on shape alone. Each
2085 reconcile pass's removal authority is scoped to its OWN kind: the
2086 instance pass and the sidecar pass would otherwise delete each other's
2087 rows on every tick, forever, with no error anywhere.
2088 ```
2089
2090- [ ] **Step 4: Full gate, commit.**
2091
2092---
2093
2094## Final verification before the PR
2095
2096- [ ] `go build ./...`, `go vet ./...`, `go test ./... -count=1` — all clean.
2097- [ ] `GOOS=darwin go build ./...` — clean (`internal/host` links into the
2098 CLI, which people run on Macs).
2099- [ ] `gofmt -l` clean for every file this branch touched.
2100- [ ] `git log --oneline main..HEAD` reads as one coherent story.
2101- [ ] No live box was touched; no real bucket was written.
2102- [ ] Push the branch and open the PR. **Do not merge it** — this is a new
2103 subsystem touching the activator, the edge and the console's auth
2104 surface, which is exactly the diff CLAUDE.md says waits for a person.
2105