carlos / platform Public

sidecar-members

Sidecar members: a serving class that nothing can dial — caught up to main twice, and swept twice.

This is PR #223's branch. Opened 2026-08-18, it has now been brought forward through two catch-ups: 600 commits of main on 2026-09-05, and another 405 on 2026-09-15. Each time it had gone CONFLICTED rather than merely stale, and each time the rerun sweep found a consumer that reaches a sidecar and did not exist when the previous sweep was written.

What the branch is for

A sidecar today is a file on a box: HasSidecar is os.Stat of /data/<host>-sidecar.env, nothing in the platform ever writes it, and the only way to run one is to get root and place it by hand — the exact thing AGENTS.md forbids. And the runtime that exists is a CO-SPAWN: one sidecar bound to one routed instance, alive only while that instance is awake. Eleven's bots must run on a box with no routed app on it, which cannot be expressed that way at all — there is no parent to hang them from.

So: standalone members, own record, own registry row, own lifecycle, on a synthetic host <name>.<app>.<sqid>.sidecar.internal that is refused at every door where a name becomes routable. Plus carlos sidecars enable|create|ls|wake|rm|logs, and the -args field that lets an app name the argv instead of the platform hardcoding <binary> sidecar run.

What the two catch-ups changed

The console CLI API grew a role gate. Every door registers through cliRoute with an svcrole.Set, so the six sidecar doors moved onto it — left on mux.HandleFunc they would have tripped the default-deny test. Five take the Instance role, which IS this design's stated requirement: an app process minting a bot per enrolled member with the CARLOSINSTANCETOKEN it already holds, no AWS credentials, no box access. The MANIFEST door takes the zero Set, for the instances-manifest reason and a sharper one of its own — it carries args, the argv the platform execs, and a credential the app holds must not rewrite what its own boxes will run.

Isolated exec arrived. StartSidecar is a dispatcher over layout now: nested starts a unit through the control channel, fenced refuses, and the old child spawn became startSidecarChild. The legacy co-spawn note moved onto that arm, where it is true. StartWorker was also spawning uncapped through Cgroups.Start — spawnInCgroup is the seam that applies the route's resource tier, and the direct call still compiled.

Fencing and unproven stops (§6.3). doWake gained wakeAllowed checks and a stopOutcome it records through; stop() keeps the lease until termination is proven. The worker settle window now reports through cleanupFailedWake rather than its own stop-and-release, because that helper is what writes the outcome — a wake that tore its own child down silently would leave stopHistory blind to exactly the case it exists for.

Two mis-merges worth naming, because the tempting fix was to delete them: git spliced main's housekeeper stand-down onto the end of serveAgentSocket after an infinite for (unreachable; the compiler only objected because a variable was out of scope), and labelRe had become controlname.ValidLabel, which is where ValidSidecarName belongs anyway.

What the reruns found

Three consumers, one shape, three times:

  1. The edges file published sidecars as role: "home" (§14.1). §6.1's edge refusal lives in Reload, which builds THIS box's map from rows THIS box holds — a REMOTE edge never sees a Kind. It missed its own lookup, read the file, and BACKHAULED: a live proxy into the one member the platform guarantees nothing can dial. The same entry also satisfied deploymentHost, which admits a name to the cert machinery.
  2. Path-based routing minted into a third table (§14.2). Both ends refused now. The TARGET end had nothing else in front of it: a sidecar belongs to the claim's own account, so the cross-tenant check passes it, and a live sidecar row is ready, so the readiness check passes it.
  3. The attach door aliased custom domains onto sidecars (§15.1). readyRouteHost is firstReadyUpstream "minus the upstream copy" — but that clause was doing two jobs, and the second one, "is this row dialable at all", was the only thing keeping a sidecar out. Host order means bot.<app>... is the FIRST candidate, not a straggler. Not a breach (ReconcileAliases refuses the claim box-side) but worse in a quiet way: 200 to the member, and a domain that never serves with nothing saying why.

Every guard was verified by neutralising it and watching the test fail, not by writing a test that passes either way.

The pattern is the real finding (§15.2). None of the three was written carelessly and none is a sidecar bug. Each is a predicate that excluded sidecars as a SIDE EFFECT of asking something else — "has an upstream to copy", "is a row this box serves" — then narrowed, copied or re-derived by someone who correctly read its stated purpose and could not see the unstated one. For the next sweep: grepping registry.Route or Kind is the wrong instinct. Ask of every consumer what in its predicate happens to require a socket or an address, then check whether that clause is load-bearing somewhere nobody documented.

Left open on purpose

Three gaps, all in the safe direction — a sidecar is left alone rather than acted on wrongly — recorded in §14.3 and §15.4 rather than fixed here:

  • Narrowed restarts reach no sidecar. scopedHosts gates on release.Adoptable, which is Kind == "instance" and stays that way for §9.3's reason.
  • A trashed app's sidecar rows are not retired. #340 gave the instance pass a retirement half guarded by the production flag; the sidecar pass keeps the mint-only predicate rather than inheriting an authority its removal phase was not designed around.
  • StartWorker has no layout dispatch. On a nested box a native sidecar would come up outside the isolation every other child gets. Not wrong today — no box runs both — but exactly the kind of agreement that stops being true quietly.

Also unchanged: delivery-wake is designed, not built (§7.5); no infra is applied; and the file-staged co-spawn stays, because Eleven runs on it today. Retiring it is gated on Eleven's own migration, whose phase 1 step A is already serving on the shared fleet.

Gate

go build ./..., go vet ./..., gofmt -l ., the GOOS=darwin and GOOS=windows compiles and go test ./... -count=1 all clean, less two failures that reproduce IDENTICALLY on a clean origin/main worktree and belong to neither this branch nor this merge: the root package's tests want a console login, and TestExecBackendStartPassesControlSocketInEnvAndStartsPurgeListener binds a unix socket whose path exceeds the 108-byte sun_path limit because the test's own name is long. Note for anyone reproducing: TMPDIR must be short AND a real directory — instancefs refuses a symlink anywhere in a path, by design, so a symlinked scratch dir fails a dozen tests for the wrong reason.

Commits 0
Discussions 0 0 open
Tasks 0/0 done

Discussions

  • No discussions on this branch yet.

Tasks

  • No tasks yet.

Activity

  • No activity yet.