Isolated Exec, Plan 2 of 4: Control Service, Trampoline, Exit, Probe, Templates

> For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Land the root control channel and the three in-namespace programs, with their unit files, so that a box can start, stop and sweep carlos-instance*@ units on the agent's behalf and can prove its own isolation with a probe. Nothing in the agent calls the channel yet (plan 3).

Architecture: One shared package internal/instancectl holds the names, escaping, launch-file and exit-record shapes, the HTTP protocol types, and the root server (carlos instance-control), which is socket-activated, authenticates the edge's main pid through SO_PEERCRED, drives systemctl through a seam, and runs the readiness probe. A second package internal/instanceexec holds the three programs that run inside a tenant's namespace as carlos: the trampoline (instance-exec), the advisory exit-record writer (instance-exit) and the probe (instance-probe). Six unit files carry titogo's namespace recipe with the spec's additions.

Tech Stack: Go 1.26, golang.org/x/sys/unix (GetsockoptUcred, Openat2, Access), systemd 257 (PrivatePIDs=, StandardOutput=truncate:), the repo's embed.FS unit set, internal/instancefs, internal/provision, internal/controlsock, internal/cgroup tiers.

Spec: docs/superpowers/specs/2026-09-11-isolated-exec-design.md (revision 12): §3.1, §3.3, §4, §5, §6.2, §6.5 (the record and the cgroup rule), §8.5, §9 (units and installers, the same-uid audit). Plan 1 (docs/superpowers/plans/2026-09-11-isolated-exec-1-foundation.md) landed the helpers this plan consumes: provision.InstanceID, InstanceDir, InstanceRunDir, NestedSocket, LogSocket, MaxUnixPath, controlsock.Dir/Path, instancefs.OpenBeneath/ReadBeneath/WriteAtomic/MountNoSymfollow, the mount units, carlos.slice, carlos-instances.slice, the tmpfiles lines (including f /etc/carlos/probe-secret 0640 root carlos - probe), and systemctlPlan starting .socket units.

Global Constraints

• Gate before every commit: go build ./..., go vet ./..., gofmt -l . empty, go test ./... -count=1; env GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off; socket-binding tests need TMPDIR=/tmp/g outside the sandbox. Cross-compile gate: darwin, windows, linux-arm64 must build; every Linux-only syscall is in a _linux.go with a !linux stub returning errors.ErrUnsupported.
• Unit-name escaping is systemd's: /→-, a leading .→\x2e, any byte outside [a-zA-Z0-9:_.]→\xNN (lowercase hex); so -→\x2d and _ is unchanged (spec §4).
• Unit names: carlos-instance@<esc>.service, carlos-instance-seed@<esc>.service, carlos-instance-sidecar@<esc>.service, slice carlos-instances-<esc>.slice, where <esc> = Escape(provision.InstanceID(host)). Paths use %I (the unescaped identifier).
• The control service accepts a connection only when SO_PEERCRED reports uid carlos AND pid equal to MainPID of carlos-edge.service as systemctl show reports it (cached for one second); otherwise the connection is closed with no response (spec §5.4). No token.
• Verbs and their exact systemctl sequences are spec §5.3: /start = set-property --runtime <slice> MemoryMax=N MemoryHigh=0.9N MemorySwapMax=N, start <unit>, show -p InvocationID --value <unit>; no reset-failed inside /start. /seed = the same set-property, blocking start of the seed unit, show -p InvocationID,Result,ExecMainStatus,ExecMainCode, wait for the unit's cgroup to be empty, stop. /stop = stop <unit>. /reset = reset-failed <unit> tolerating "not loaded". /sweep = enumerate the three templates' instances and jobs, stop each, wait terminal and empty. /ready = 204 when the probe verdict is ready, else 503 with the verdict.
• memory_max must be one of cgroup.TierDefault.MemoryMax (536870912) or cgroup.TierChildren.MemoryMax (1073741824); MemoryHigh is cgroup.highOf (max/10*9); anything else is 400.
• The exit record is advisory: exit.<role>.<INVOCATION_ID>.json in the instance's run directory, written O_EXCL, never overwritten (spec §6.5).
• The probe's verdict lines and their rules are spec §8.5: directory listings, pid 1, EROFS on cgroupfs, /etc/carlos unopenable and the fixture unreadable, ELOOP on a planted symlink, nosymfollow in mountinfo, UDP sendto EPERM on IMDS v4 (required) and v6 (EPERM enforced or ENETUNREACH unverified), loopback sendto succeeds, a 2s TCP connect to v4 IMDS must not succeed. v6: unverified does not block readiness; anything else failing does.
• docs/site/cli.md gets a ### carlos <verb> heading for each new verb or the docsite gate fails. New verbs are case arms in main.go's switch.
• Copy rules: no em dashes in any string; log lines verb: subject: detail; comments are full sentences explaining why.
• Commit after every task with the session's attribution trailer. Branch worktree-isolated-exec-spec.

---

File Structure

| File | Responsibility | |---|---| | internal/instancectl/names.go (create) | Role, Escape, UnitName, SliceName, ParseUnitName, LaunchFileName, ExitRecordName, Launch, ExitRecord, protocol types | | internal/instancectl/names_test.go (create) | tests for the above | | internal/instancectl/server.go (create) | Server: listener, peer check seam, handlers, systemctl seam, cgroup-empty wait, probe runner, Run(args) | | internal/instancectl/peer_linux.go (create) | SO_PEERCRED via SyscallConn, LISTEN_FDS listener | | internal/instancectl/peer_other.go (create) | stubs | | internal/instancectl/server_test.go (create) | handler tests over a real unix socket with fakes | | internal/instanceexec/exec.go (create) | RunExec: the trampoline | | internal/instanceexec/exit.go (create) | RunExit: the exit record | | internal/instanceexec/probe.go (create) | RunProbe: the checks, with an injectable environment | | internal/instanceexec/probe_linux.go / probe_other.go (create) | the sendto/Access syscalls and stubs | | internal/instanceexec/*_test.go (create) | tests | | internal/host/units/carlos-instance@.service, carlos-instance-seed@.service, carlos-instance-sidecar@.service, carlos-instance-probe.service, carlos-instance-control.socket, carlos-instance-control.service (create) | the units | | internal/host/bootstrap.go (modify) | enabledUnits gains the control socket; tmpfiles gains the _probe directories | | internal/host/systemunits.go (modify) | the hourly belt also arms the control socket | | internal/host/units_isolation_test.go (modify) | template identity, the same-uid audit, control unit shape | | main.go (modify) | four case arms | | docs/site/cli.md (modify) | four headings under "On the host" |

---

Task 1: Names, escaping and protocol types

Files:

• Create: internal/instancectl/names.go
• Test: internal/instancectl/names_test.go

Interfaces:

• Consumes: provision.InstanceID, provision.ValidHostShape.
• Produces: ``go package instancectl type Role string const ( RoleRun Role = "run"; RoleSeed Role = "seed"; RoleSidecar Role = "sidecar" ) func ParseRole(s string) (Role, bool) func Escape(s string) string func UnitName(role Role, host string) string func SliceName(host string) string const ( UnitPrefixRun = "carlos-instance@"; UnitPrefixSeed = "carlos-instance-seed@"; UnitPrefixSidecar = "carlos-instance-sidecar@" ) func ParseUnitName(unit string) (role Role, escapedID string, ok bool) const LaunchSchema = 1 type Launch struct { Schema int json:"schema"; Host string json:"host"; Argv []string json:"argv"; Env []string json:"env" } func LaunchFileName(role Role) string // "launch.<role>.json" type ExitRecord struct { Invocation string json:"invocation"; Result string json:"result"; Code string json:"code"; Status string json:"status"; At time.Time json:"at" } func ExitRecordName(role Role, invocation string) string // "exit.<role>.<invocation>.json" const ProbeDirName = "_probe" // the probe's own instance directory name; no host can start with '_' type VerdictLine struct { Check string json:"check"; Verdict string json:"verdict"; Detail string json:"detail,omitempty" } type ProbeReport struct { Ready bool json:"ready"; Lines []VerdictLine json:"lines" } type StartResponse struct { InvocationID string json:"invocationid" } type SeedResponse struct { InvocationID string json:"invocationid"; Result string json:"result"; Status string json:"status"; Code string json:"code" } type SweepResponse struct { Units []string json:"units" } ``

• [ ] Step 1: Write the failing tests

Create internal/instancectl/names_test.go:

package instancectl

import (
	"strings"
	"testing"

	"github.com/carlosframework/platform/internal/provision"
)

// Escape follows systemd-escape(1) for unit instance names. The cases are
// the ones our identifiers can produce (hosts are [a-z0-9.-]; a hashed
// identifier adds a leading underscore), plus the general rules so a
// surprise character can never yield an unloadable unit name.
func TestEscapeFollowsSystemd(t *testing.T) {
	cases := map[string]string{
		"bill.go.tito.io":       "bill.go.tito.io",
		"bill-2.example.com":    `bill\x2d2.example.com`,
		"_aaaa-1234abcd":        `_aaaa\x2d1234abcd`,
		"a/b":                   "a-b",
		".hidden":               `\x2ehidden`,
		"x y":                   `x\x20y`,
		"":                      "",
		"UPPER_ok:1":            "UPPER_ok:1",
	}
	for in, want := range cases {
		if got := Escape(in); got != want {
			t.Errorf("Escape(%q) = %q, want %q", in, got, want)
		}
	}
}

func TestUnitAndSliceNamesAreKeyedByEscapedInstanceID(t *testing.T) {
	host := "bill-2.example.com"
	esc := `bill\x2d2.example.com`
	if got := UnitName(RoleRun, host); got != "carlos-instance@"+esc+".service" {
		t.Errorf("run unit = %q", got)
	}
	if got := UnitName(RoleSeed, host); got != "carlos-instance-seed@"+esc+".service" {
		t.Errorf("seed unit = %q", got)
	}
	if got := UnitName(RoleSidecar, host); got != "carlos-instance-sidecar@"+esc+".service" {
		t.Errorf("sidecar unit = %q", got)
	}
	if got := SliceName(host); got != "carlos-instances-"+esc+".slice" {
		t.Errorf("slice = %q", got)
	}
	long := strings.Repeat("a", 60) + ".example.com"
	if got := UnitName(RoleRun, long); !strings.HasPrefix(got, "carlos-instance@_") || !strings.HasSuffix(got, ".service") {
		t.Errorf("long host unit = %q, want the hashed identifier", got)
	}
	if got, want := UnitName(RoleRun, long), "carlos-instance@"+Escape(provision.InstanceID(long))+".service"; got != want {
		t.Errorf("long host unit = %q, want %q", got, want)
	}
}

func TestParseUnitNameRoundTrips(t *testing.T) {
	for _, role := range []Role{RoleRun, RoleSeed, RoleSidecar} {
		u := UnitName(role, "bill-2.example.com")
		gotRole, esc, ok := ParseUnitName(u)
		if !ok || gotRole != role || esc != `bill\x2d2.example.com` {
			t.Errorf("ParseUnitName(%q) = %q %q %v", u, gotRole, esc, ok)
		}
	}
	for _, bad := range []string{"carlos-edge.service", "carlos-app@x.service", "carlos-instance@x.socket", "carlos-instance@.service", "carlos-instances-x.slice"} {
		if _, _, ok := ParseUnitName(bad); ok {
			t.Errorf("ParseUnitName(%q) accepted", bad)
		}
	}
}

func TestParseRole(t *testing.T) {
	for _, s := range []string{"run", "seed", "sidecar"} {
		if r, ok := ParseRole(s); !ok || string(r) != s {
			t.Errorf("ParseRole(%q) = %q %v", s, r, ok)
		}
	}
	for _, s := range []string{"", "Run", "exec", "run "} {
		if _, ok := ParseRole(s); ok {
			t.Errorf("ParseRole(%q) accepted", s)
		}
	}
}

func TestFileNames(t *testing.T) {
	if got := LaunchFileName(RoleSidecar); got != "launch.sidecar.json" {
		t.Errorf("LaunchFileName = %q", got)
	}
	if got := ExitRecordName(RoleRun, "abc123"); got != "exit.run.abc123.json" {
		t.Errorf("ExitRecordName = %q", got)
	}
	if provision.ValidHostShape(ProbeDirName) {
		t.Errorf("ProbeDirName %q is a valid host; it must never collide with one", ProbeDirName)
	}
}

• [ ] Step 2: Run the tests to verify they fail

Run: GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/instancectl -count=1 Expected: build failure (undefined: Escape and the rest; or no Go files).

• [ ] Step 3: Write names.go

// Package instancectl is the root instance-control channel of the
// isolated-exec design (§5) and the names everything on both sides of it
// agrees on: unit and slice names keyed by the bounded instance identifier,
// the launch file the trampoline reads (§6.2), the advisory exit record
// (§6.5) and the probe's verdict (§8.5).
//
// The names live in one package because three programs and two services
// build them: the agent (plan 3) composes launch files and requests, the
// control service builds unit names from validated hosts, and the
// in-namespace programs read files keyed by the same identifier. A
// disagreement between any two of them would be a unit that never starts
// or a record nobody finds.
package instancectl

import (
	"fmt"
	"strings"
	"time"

	"github.com/carlosframework/platform/internal/provision"
)

// Role is which of the three template units a request or a file is about.
type Role string

const (
	RoleRun     Role = "run"
	RoleSeed    Role = "seed"
	RoleSidecar Role = "sidecar"
)

// ParseRole accepts exactly the three literals.
func ParseRole(s string) (Role, bool) {
	switch Role(s) {
	case RoleRun, RoleSeed, RoleSidecar:
		return Role(s), true
	}
	return "", false
}

// Escape is systemd-escape(1)'s rule for a unit instance name: "/" becomes
// "-", a leading "." becomes "\x2e", and every byte outside [a-zA-Z0-9:_.]
// becomes "\x" plus two lowercase hex digits. For the identifiers this
// platform produces (hosts are [a-z0-9.-], a hashed identifier adds a
// leading underscore) that means "-" becomes "\x2d" and nothing else
// changes; the general rule is kept so an unexpected byte can never yield
// a name systemd refuses to load.
func Escape(s string) string {
	var b strings.Builder
	for i := 0; i < len(s); i++ {
		c := s[i]
		switch {
		case c == '/':
			b.WriteByte('-')
		case i == 0 && c == '.':
			b.WriteString(`\x2e`)
		case c >= 'a' && c <= 'z', c >= 'A' && c <= 'Z', c >= '0' && c <= '9', c == ':', c == '_', c == '.':
			b.WriteByte(c)
		default:
			fmt.Fprintf(&b, `\x%02x`, c)
		}
	}
	return b.String()
}

// The three template prefixes. A unit under any of them is the platform's
// to start, stop and sweep; nothing else is (design §5.3).
const (
	UnitPrefixRun     = "carlos-instance@"
	UnitPrefixSeed    = "carlos-instance-seed@"
	UnitPrefixSidecar = "carlos-instance-sidecar@"
)

func prefixFor(role Role) string {
	switch role {
	case RoleSeed:
		return UnitPrefixSeed
	case RoleSidecar:
		return UnitPrefixSidecar
	}
	return UnitPrefixRun
}

// UnitName is the template instance for a role and a host:
// carlos-instance[-seed|-sidecar]@<Escape(InstanceID(host))>.service.
func UnitName(role Role, host string) string {
	return prefixFor(role) + Escape(provision.InstanceID(host)) + ".service"
}

// SliceName is the per-host slice every role of a host runs in (design
// §4.3): carlos-instances-<Escape(InstanceID(host))>.slice.
func SliceName(host string) string {
	return "carlos-instances-" + Escape(provision.InstanceID(host)) + ".slice"
}

// ParseUnitName recognises the three templates' instances and returns the
// role and the ESCAPED identifier (systemd reports names escaped). Anything
// else, including a template with no instance, is not ours.
func ParseUnitName(unit string) (Role, string, bool) {
	if !strings.HasSuffix(unit, ".service") {
		return "", "", false
	}
	body := strings.TrimSuffix(unit, ".service")
	for _, p := range []struct {
		prefix string
		role   Role
	}{{UnitPrefixSeed, RoleSeed}, {UnitPrefixSidecar, RoleSidecar}, {UnitPrefixRun, RoleRun}} {
		if strings.HasPrefix(body, p.prefix) {
			esc := strings.TrimPrefix(body, p.prefix)
			if esc == "" {
				return "", "", false
			}
			return p.role, esc, true
		}
	}
	return "", "", false
}

// LaunchSchema is the only launch-file schema this release understands.
const LaunchSchema = 1

// Launch is what the agent writes into the instance directory before a
// start and what the trampoline execs (design §6.2). Env is the complete
// environment, one entry per key, exactly as the agent composed it; the
// trampoline parses nothing.
type Launch struct {
	Schema int      `json:"schema"`
	Host   string   `json:"host"`
	Argv   []string `json:"argv"`
	Env    []string `json:"env"`
}

// LaunchFileName is "launch.<role>.json" inside the instance directory.
func LaunchFileName(role Role) string { return "launch." + string(role) + ".json" }

// ExitRecord is the advisory record the unit's ExecStopPost leaves in the
// instance's run directory (design §6.5). It decides the wording of the
// member-facing event line and nothing else.
type ExitRecord struct {
	Invocation string    `json:"invocation"`
	Result     string    `json:"result"`
	Code       string    `json:"code"`
	Status     string    `json:"status"`
	At         time.Time `json:"at"`
}

// ExitRecordName is "exit.<role>.<invocation>.json".
func ExitRecordName(role Role, invocation string) string {
	return "exit." + string(role) + "." + invocation + ".json"
}

// ProbeDirName is the instance directory the readiness probe runs in. It
// starts with an underscore, which no valid host does, so it can never
// collide with a real instance's directory.
const ProbeDirName = "_probe"

// VerdictLine is one check of the readiness probe (design §8.5). Verdict is
// "ok", "unverified" or "fail"; only "fail" blocks readiness.
type VerdictLine struct {
	Check   string `json:"check"`
	Verdict string `json:"verdict"`
	Detail  string `json:"detail,omitempty"`
}

// ProbeReport is what the probe prints and what /ready answers with.
type ProbeReport struct {
	Ready bool          `json:"ready"`
	Lines []VerdictLine `json:"lines"`
}

// Responses of the control verbs (design §5.3).
type StartResponse struct {
	InvocationID string `json:"invocation_id"`
}

type SeedResponse struct {
	InvocationID string `json:"invocation_id"`
	Result       string `json:"result"`
	Status       string `json:"status"`
	Code         string `json:"code"`
}

type SweepResponse struct {
	Units []string `json:"units"`
}

• [ ] Step 4: Run the tests to verify they pass

Run: GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/instancectl -count=1 Expected: PASS.

• [ ] Step 5: gofmt and commit

gofmt -l internal/instancectl
git add internal/instancectl
git commit -m "instancectl: unit names, escaping, launch and exit shapes, protocol types (plan 2, task 1)"

---

Task 2: The trampoline, carlos instance-exec

Files:

• Create: internal/instanceexec/exec.go
• Create: internal/instanceexec/exec_test.go

Interfaces:

• Consumes: instancectl.Launch, LaunchSchema, LaunchFileName, ParseRole; instancefs.ReadBeneath; provision.ValidHostShape.
• Produces: ``go package instanceexec // RunExec is carlos instance-exec <role> <iid>: read the launch file, validate, execve. func RunExec(args []string) // execLaunch is the testable core; execFn defaults to syscall.Exec. func execLaunch(dataDir string, role instancectl.Role, iid string, execFn func(argv0 string, argv, env []string) error) error const LiveDir = "/opt/carlos/live" ``

• [ ] Step 1: Write the failing tests

Create internal/instanceexec/exec_test.go:

package instanceexec

import (
	"encoding/json"
	"errors"
	"os"
	"path/filepath"
	"testing"

	"github.com/carlosframework/platform/internal/instancectl"
)

func writeLaunch(t *testing.T, dir string, role instancectl.Role, l instancectl.Launch) {
	t.Helper()
	if err := os.MkdirAll(dir, 0o700); err != nil {
		t.Fatal(err)
	}
	b, err := json.Marshal(l)
	if err != nil {
		t.Fatal(err)
	}
	if err := os.WriteFile(filepath.Join(dir, instancectl.LaunchFileName(role)), b, 0o600); err != nil {
		t.Fatal(err)
	}
}

type captured struct {
	argv0 string
	argv  []string
	env   []string
	calls int
}

func (c *captured) exec(argv0 string, argv, env []string) error {
	c.calls++
	c.argv0, c.argv, c.env = argv0, argv, env
	return nil
}

func TestExecLaunchExecsExactlyWhatTheAgentWrote(t *testing.T) {
	data := t.TempDir()
	iid := "bill.example.com"
	dir := filepath.Join(data, "instances", iid)
	writeLaunch(t, dir, instancectl.RoleRun, instancectl.Launch{
		Schema: 1, Host: "bill.example.com",
		Argv: []string{"/opt/carlos/live/bill.example.com", "--socket", "/run/carlos/instances/bill.example.com/app.sock", "--db", "/data/instances/bill.example.com/bill.example.com.db"},
		Env:  []string{"PATH=/usr/bin", "CARLOS_ADMIN_TOKEN=abc"},
	})
	var c captured
	if err := execLaunch(data, instancectl.RoleRun, iid, c.exec); err != nil {
		t.Fatalf("execLaunch: %v", err)
	}
	if c.calls != 1 || c.argv0 != "/opt/carlos/live/bill.example.com" || len(c.argv) != 5 || c.argv[2] != "/run/carlos/instances/bill.example.com/app.sock" {
		t.Errorf("exec = %+v", c)
	}
	if len(c.env) != 2 || c.env[1] != "CARLOS_ADMIN_TOKEN=abc" {
		t.Errorf("env = %v", c.env)
	}
}

func TestExecLaunchRefusesAnArgv0OutsideTheLiveDir(t *testing.T) {
	data := t.TempDir()
	iid := "h.example.com"
	dir := filepath.Join(data, "instances", iid)
	for _, argv0 := range []string{"/bin/sh", "/opt/carlos/live/../carlos", "/opt/carlos/live/other.example.com", "opt/carlos/live/h.example.com", ""} {
		writeLaunch(t, dir, instancectl.RoleRun, instancectl.Launch{Schema: 1, Host: "h.example.com", Argv: []string{argv0}, Env: nil})
		var c captured
		err := execLaunch(data, instancectl.RoleRun, iid, c.exec)
		if err == nil || c.calls != 0 {
			t.Errorf("argv0 %q: err = %v, calls = %d; want a refusal and no exec", argv0, err, c.calls)
		}
	}
}

func TestExecLaunchRefusesUnknownSchemaAndBadEnv(t *testing.T) {
	data := t.TempDir()
	iid := "h.example.com"
	dir := filepath.Join(data, "instances", iid)
	good := []string{"/opt/carlos/live/h.example.com"}
	writeLaunch(t, dir, instancectl.RoleRun, instancectl.Launch{Schema: 2, Host: "h.example.com", Argv: good})
	var c captured
	if err := execLaunch(data, instancectl.RoleRun, iid, c.exec); err == nil || c.calls != 0 {
		t.Errorf("schema 2: err = %v, calls = %d", err, c.calls)
	}
	writeLaunch(t, dir, instancectl.RoleRun, instancectl.Launch{Schema: 1, Host: "h.example.com", Argv: good, Env: []string{"NOEQUALS"}})
	if err := execLaunch(data, instancectl.RoleRun, iid, c.exec); err == nil || c.calls != 0 {
		t.Errorf("bad env: err = %v, calls = %d", err, c.calls)
	}
	writeLaunch(t, dir, instancectl.RoleRun, instancectl.Launch{Schema: 1, Host: "not a host", Argv: []string{"/opt/carlos/live/not a host"}})
	if err := execLaunch(data, instancectl.RoleRun, iid, c.exec); err == nil || c.calls != 0 {
		t.Errorf("bad host: err = %v, calls = %d", err, c.calls)
	}
}

func TestExecLaunchRefusesASymlinkedLaunchFile(t *testing.T) {
	data := t.TempDir()
	iid := "h.example.com"
	dir := filepath.Join(data, "instances", iid)
	if err := os.MkdirAll(dir, 0o700); err != nil {
		t.Fatal(err)
	}
	other := filepath.Join(t.TempDir(), "launch.run.json")
	l := instancectl.Launch{Schema: 1, Host: "h.example.com", Argv: []string{"/opt/carlos/live/h.example.com"}}
	b, _ := json.Marshal(l)
	if err := os.WriteFile(other, b, 0o600); err != nil {
		t.Fatal(err)
	}
	if err := os.Symlink(other, filepath.Join(dir, "launch.run.json")); err != nil {
		t.Fatal(err)
	}
	var c captured
	err := execLaunch(data, instancectl.RoleRun, iid, c.exec)
	if err == nil || c.calls != 0 || !errors.Is(err, errSymlinkOrEscape) {
		t.Errorf("symlinked launch file: err = %v, calls = %d", err, c.calls)
	}
}

• [ ] Step 2: Run the tests to verify they fail

Run: GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/instanceexec -run TestExecLaunch -count=1 Expected: build failure, undefined: execLaunch.

• [ ] Step 3: Write exec.go

// Package instanceexec holds the three programs that run INSIDE an isolated
// instance's namespace, as carlos, launched by the template units of the
// isolated-exec design: the trampoline (instance-exec, §6.2), the advisory
// exit-record writer (instance-exit, §6.5) and the readiness probe
// (instance-probe, §8.5). None of them is for operators; each is an
// ExecStart= or ExecStopPost= of a unit.
package instanceexec

import (
	"encoding/json"
	"errors"
	"flag"
	"fmt"
	"log"
	"os"
	"path/filepath"
	"strings"
	"syscall"

	"github.com/carlosframework/platform/internal/cliflag"
	"github.com/carlosframework/platform/internal/instancectl"
	"github.com/carlosframework/platform/internal/instancefs"
	"github.com/carlosframework/platform/internal/provision"
)

// LiveDir is where the adoption pass keeps each host's live symlink; the
// trampoline execs nothing that is not directly under it.
const LiveDir = "/opt/carlos/live"

var errSymlinkOrEscape = errors.New("instance-exec: launch file refused")

// dataDir is the agent's own resolver (host/agent.go), repeated here because
// this program runs in a namespace where only the instance's directory is
// bound, and it must agree with the agent on where that directory is.
func dataDir() string {
	if d := os.Getenv("CARLOS_DATA_DIR"); d != "" {
		return d
	}
	return "/data"
}

// RunExec is `carlos instance-exec <role> <iid>`.
func RunExec(args []string) {
	fs := flag.NewFlagSet("instance-exec", flag.ExitOnError)
	fs.Usage = cliflag.Usage(fs, "usage: carlos instance-exec <run|seed|sidecar> <instance-id>")
	_ = fs.Parse(args)
	if fs.NArg() != 2 {
		fs.Usage()
		os.Exit(2)
	}
	role, ok := instancectl.ParseRole(fs.Arg(0))
	if !ok {
		log.Fatalf("instance-exec: role %q is not run, seed or sidecar", fs.Arg(0))
	}
	if err := execLaunch(dataDir(), role, fs.Arg(1), syscall.Exec); err != nil {
		log.Fatalf("instance-exec: %v", err)
	}
}

// execLaunch reads launch.<role>.json from the instance directory through
// instancefs (never through a symlink), validates it, and execs. It returns
// only on refusal; a successful exec never returns.
//
// The validation is defence in depth: the agent wrote the file, but the
// directory is the instance's own, so the instance can rewrite it. What
// the checks guarantee is that whatever is executed is directly under
// LiveDir (the platform's live links, root-written) and named for the host
// in the file, so a rewritten file can execute only what the tenant could
// execute anyway: its own binary with an argv and env it chose.
func execLaunch(dataDir string, role instancectl.Role, iid string, execFn func(argv0 string, argv, env []string) error) error {
	dir := filepath.Join(dataDir, "instances", iid)
	raw, err := instancefs.ReadBeneath(dir, instancectl.LaunchFileName(role))
	if err != nil {
		if errors.Is(err, instancefs.ErrSymlink) || errors.Is(err, instancefs.ErrEscapes) {
			return fmt.Errorf("%w: %v", errSymlinkOrEscape, err)
		}
		return err
	}
	var l instancectl.Launch
	if err := json.Unmarshal(raw, &l); err != nil {
		return fmt.Errorf("launch file: %v", err)
	}
	if l.Schema != instancectl.LaunchSchema {
		return fmt.Errorf("launch file schema %d is not %d", l.Schema, instancectl.LaunchSchema)
	}
	if !provision.ValidHostShape(l.Host) {
		return fmt.Errorf("launch file host %q is not a host", l.Host)
	}
	if len(l.Argv) == 0 {
		return errors.New("launch file has no argv")
	}
	argv0 := l.Argv[0]
	if !filepath.IsAbs(argv0) || filepath.Clean(argv0) != argv0 || filepath.Dir(argv0) != LiveDir {
		return fmt.Errorf("argv[0] %q is not directly under %s", argv0, LiveDir)
	}
	if filepath.Base(argv0) != l.Host {
		return fmt.Errorf("argv[0] %q is not the live link of host %s", argv0, l.Host)
	}
	for _, kv := range l.Env {
		if k, _, ok := strings.Cut(kv, "="); !ok || k == "" {
			return fmt.Errorf("launch file env entry %q is not KEY=value", kv)
		}
	}
	return execFn(argv0, l.Argv, l.Env)
}

• [ ] Step 4: Run the tests to verify they pass

Run: GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/instanceexec -run TestExecLaunch -count=1 Expected: PASS (the symlink test needs Linux; on the CI's Linux hosts it runs).

• [ ] Step 5: gofmt and commit

gofmt -l internal/instanceexec
git add internal/instanceexec
git commit -m "instanceexec: the trampoline reads the launch file through instancefs and execs the live link (plan 2, task 2)"

---

Task 3: The exit record, carlos instance-exit

Files:

• Create: internal/instanceexec/exit.go
• Create: internal/instanceexec/exit_test.go

Interfaces:

• Consumes: instancectl.ExitRecord, ExitRecordName, ParseRole; instancefs.OpenBeneath.
• Produces: ``go func RunExit(args []string) // writeExitRecord writes exit.<role>.<invocation>.json into runDir/instances/<iid>, O_EXCL; env is the process environment lookup. func writeExitRecord(runDir string, role instancectl.Role, iid string, env func(string) string, now time.Time) error ``

• [ ] Step 1: Write the failing tests

Create internal/instanceexec/exit_test.go:

package instanceexec

import (
	"encoding/json"
	"os"
	"path/filepath"
	"testing"
	"time"

	"github.com/carlosframework/platform/internal/instancectl"
)

func envOf(m map[string]string) func(string) string {
	return func(k string) string { return m[k] }
}

func TestWriteExitRecordWritesTheFourVariables(t *testing.T) {
	run := t.TempDir()
	iid := "h.example.com"
	if err := os.MkdirAll(filepath.Join(run, "instances", iid), 0o700); err != nil {
		t.Fatal(err)
	}
	now := time.Date(2026, 9, 11, 12, 0, 0, 0, time.UTC)
	env := envOf(map[string]string{"INVOCATION_ID": "deadbeef", "SERVICE_RESULT": "exit-code", "EXIT_CODE": "exited", "EXIT_STATUS": "3"})
	if err := writeExitRecord(run, instancectl.RoleRun, iid, env, now); err != nil {
		t.Fatalf("writeExitRecord: %v", err)
	}
	b, err := os.ReadFile(filepath.Join(run, "instances", iid, "exit.run.deadbeef.json"))
	if err != nil {
		t.Fatal(err)
	}
	var rec instancectl.ExitRecord
	if err := json.Unmarshal(b, &rec); err != nil {
		t.Fatal(err)
	}
	if rec.Invocation != "deadbeef" || rec.Result != "exit-code" || rec.Code != "exited" || rec.Status != "3" || !rec.At.Equal(now) {
		t.Errorf("record = %+v", rec)
	}
	// O_EXCL: a second write for the same invocation is refused and the
	// first record stands.
	env2 := envOf(map[string]string{"INVOCATION_ID": "deadbeef", "SERVICE_RESULT": "success", "EXIT_CODE": "exited", "EXIT_STATUS": "0"})
	if err := writeExitRecord(run, instancectl.RoleRun, iid, env2, now); err == nil {
		t.Error("second write for the same invocation succeeded")
	}
	b2, _ := os.ReadFile(filepath.Join(run, "instances", iid, "exit.run.deadbeef.json"))
	if string(b2) != string(b) {
		t.Error("first record was overwritten")
	}
}

func TestWriteExitRecordWithoutInvocationIsANoop(t *testing.T) {
	run := t.TempDir()
	iid := "h.example.com"
	if err := os.MkdirAll(filepath.Join(run, "instances", iid), 0o700); err != nil {
		t.Fatal(err)
	}
	if err := writeExitRecord(run, instancectl.RoleSeed, iid, envOf(nil), time.Now()); err != nil {
		t.Fatalf("no INVOCATION_ID: %v (must not fail the unit's stop)", err)
	}
	entries, _ := os.ReadDir(filepath.Join(run, "instances", iid))
	if len(entries) != 0 {
		t.Errorf("wrote %v without an invocation id", entries)
	}
}

func TestWriteExitRecordRefusesAnInvocationThatIsNotAName(t *testing.T) {
	run := t.TempDir()
	iid := "h.example.com"
	if err := os.MkdirAll(filepath.Join(run, "instances", iid), 0o700); err != nil {
		t.Fatal(err)
	}
	env := envOf(map[string]string{"INVOCATION_ID": "../../etc/x", "SERVICE_RESULT": "success"})
	if err := writeExitRecord(run, instancectl.RoleRun, iid, env, time.Now()); err == nil {
		t.Error("an invocation id with a path separator was accepted")
	}
}

• [ ] Step 2: Run the tests to verify they fail

Run: GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/instanceexec -run TestWriteExitRecord -count=1 Expected: build failure, undefined: writeExitRecord.

• [ ] Step 3: Write exit.go

package instanceexec

import (
	"encoding/json"
	"errors"
	"flag"
	"fmt"
	"log"
	"os"
	"path/filepath"
	"regexp"
	"time"

	"github.com/carlosframework/platform/internal/cliflag"
	"github.com/carlosframework/platform/internal/instancectl"
	"github.com/carlosframework/platform/internal/instancefs"
	"github.com/carlosframework/platform/internal/provision"
)

// invocationShape is systemd's 128-bit invocation id as 32 hex digits. The
// id becomes part of a file name, so anything else is refused before it can
// name a path.
var invocationShape = regexp.MustCompile(`^[0-9a-f]{32}$`)

// RunExit is `carlos instance-exit <role> <iid>`, the ExecStopPost= of every
// template unit. It runs inside the namespace as carlos with systemd's
// $INVOCATION_ID, $SERVICE_RESULT, $EXIT_CODE and $EXIT_STATUS, and leaves
// the advisory exit record in the instance's own run directory (design
// §6.5). It never fails the unit: a stop that could not be recorded is
// still a stop, and the agent proves termination from unit state and the
// cgroup, not from this file.
func RunExit(args []string) {
	fs := flag.NewFlagSet("instance-exit", flag.ExitOnError)
	runDir := fs.String("run-dir", provision.SocketDir, "run directory holding instances/<id>")
	fs.Usage = cliflag.Usage(fs, "usage: carlos instance-exit <run|seed|sidecar> <instance-id>")
	_ = fs.Parse(args)
	if fs.NArg() != 2 {
		fs.Usage()
		os.Exit(2)
	}
	role, ok := instancectl.ParseRole(fs.Arg(0))
	if !ok {
		log.Printf("instance-exit: role %q is not run, seed or sidecar", fs.Arg(0))
		return
	}
	if err := writeExitRecord(*runDir, role, fs.Arg(1), os.Getenv, time.Now()); err != nil {
		log.Printf("instance-exit: %v", err)
	}
}

// writeExitRecord writes exit.<role>.<invocation>.json with O_EXCL into
// <runDir>/instances/<iid>. A missing invocation id is a no-op (the
// variables are systemd's to provide); a malformed one is an error.
func writeExitRecord(runDir string, role instancectl.Role, iid string, env func(string) string, now time.Time) error {
	inv := env("INVOCATION_ID")
	if inv == "" {
		return nil
	}
	if !invocationShape.MatchString(inv) {
		return fmt.Errorf("INVOCATION_ID %q is not 32 hex digits", inv)
	}
	rec := instancectl.ExitRecord{
		Invocation: inv,
		Result:     env("SERVICE_RESULT"),
		Code:       env("EXIT_CODE"),
		Status:     env("EXIT_STATUS"),
		At:         now.UTC(),
	}
	raw, err := json.MarshalIndent(rec, "", "  ")
	if err != nil {
		return err
	}
	raw = append(raw, '\n')
	dir := filepath.Join(runDir, "instances", iid)
	f, err := instancefs.OpenBeneath(dir, instancectl.ExitRecordName(role, inv), os.O_WRONLY|os.O_CREAT|os.O_EXCL, 0o644)
	if err != nil {
		if errors.Is(err, os.ErrExist) {
			return fmt.Errorf("record for invocation %s already exists", inv)
		}
		return err
	}
	if _, err := f.Write(raw); err != nil {
		f.Close()
		return err
	}
	return f.Close()
}

Note: instancefs.OpenBeneath wraps EEXIST in an *os.PathError; errors.Is(err, os.ErrExist) matches it through syscall.Errno.Is. On non-Linux OpenBeneath returns errors.ErrUnsupported, so the tests are Linux-only: put //go:build linux at the top of exit_test.go and exec_test.go (the symlink test needs it anyway), and add a trivial exec_other_test.go? No: leave non-Linux without tests; the package still builds there.

• [ ] Step 4: Run the tests to verify they pass

Run: GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/instanceexec -count=1 && GOOS=darwin go build ./internal/instanceexec && GOOS=windows go build ./internal/instanceexec Expected: PASS and both builds.

• [ ] Step 5: gofmt and commit

gofmt -l internal/instanceexec
git add internal/instanceexec
git commit -m "instanceexec: the advisory exit record, written O_EXCL from systemd's four variables (plan 2, task 3)"

---

Task 4: The probe, carlos instance-probe

Files:

• Create: internal/instanceexec/probe.go
• Create: internal/instanceexec/probe_linux.go
• Create: internal/instanceexec/probe_other.go
• Create: internal/instanceexec/probe_test.go

Interfaces:

• Consumes: instancectl.ProbeReport, VerdictLine, ProbeDirName; instancefs.MountNoSymfollow.
• Produces: ``go func RunProbe(args []string) // prints a ProbeReport as JSON on stdout; exit 0 always type probeEnv struct { dataRoot, runRoot, varCache, varLib, etcCarlos, procRoot, cgroupProcs string pid func() int writable func(path string) error // access(W_OK); production: unix.Access sendUDP func(network, addr string) error // one datagram; production: a UDP socket write connectTCP func(addr string, timeout time.Duration) error // production: net.DialTimeout mountNoSym func(mountpoint string) (bool, error) } func runChecks(e probeEnv) instancectl.ProbeReport ``

• [ ] Step 1: Write the failing tests

Create internal/instanceexec/probe_test.go:

//go:build linux

package instanceexec

import (
	"errors"
	"os"
	"path/filepath"
	"syscall"
	"testing"
	"time"

	"github.com/carlosframework/platform/internal/instancectl"
)

// goodEnv is a fixture that passes every check: the trees an isolated
// instance sees, pid 1, a read-only cgroupfs, an unopenable /etc/carlos, a
// nosymfollow mount, EPERM on both metadata addresses, a live loopback and
// a TCP connect that never succeeds.
func goodEnv(t *testing.T) probeEnv {
	t.Helper()
	root := t.TempDir()
	mk := func(p string) string {
		if err := os.MkdirAll(filepath.Join(root, p), 0o700); err != nil {
			t.Fatal(err)
		}
		return filepath.Join(root, p)
	}
	data := mk("data")
	mk("data/instances/_probe")
	run := mk("run/carlos")
	mk("run/carlos/instances/_probe")
	varCache := mk("var/cache")
	varLib := mk("var/lib")
	etc := filepath.Join(root, "etc", "carlos") // deliberately absent: opening it fails
	proc := mk("proc")
	mk("proc/1")
	mk("proc/self")
	return probeEnv{
		dataRoot: data, runRoot: run, varCache: varCache, varLib: varLib, etcCarlos: etc, procRoot: proc,
		cgroupProcs: filepath.Join(root, "sys", "cgroup.procs"),
		pid:         func() int { return 1 },
		writable:    func(string) error { return syscall.EROFS },
		sendUDP: func(network, addr string) error {
			switch addr {
			case "169.254.169.254:80", "[fd00:ec2::254]:80":
				return syscall.EPERM
			}
			return nil
		},
		connectTCP: func(string, time.Duration) error { return syscall.ETIMEDOUT },
		mountNoSym: func(string) (bool, error) { return true, nil },
	}
}

func verdict(r instancectl.ProbeReport, check string) string {
	for _, l := range r.Lines {
		if l.Check == check {
			return l.Verdict
		}
	}
	return "<missing>"
}

func TestProbePassesOnAnIsolatedNamespace(t *testing.T) {
	r := runChecks(goodEnv(t))
	if !r.Ready {
		t.Fatalf("not ready: %+v", r.Lines)
	}
	for _, c := range []string{"data-tree", "run-tree", "var-empty", "pid1", "cgroup-ro", "etc-carlos", "symlink-eloop", "nosymfollow", "imds-v4", "imds-v6", "loopback", "imds-tcp"} {
		if v := verdict(r, c); v != "ok" {
			t.Errorf("%s = %s", c, v)
		}
	}
}

func TestProbeFailsWhenASiblingIsVisible(t *testing.T) {
	e := goodEnv(t)
	if err := os.MkdirAll(filepath.Join(e.dataRoot, "instances", "other.example.com"), 0o700); err != nil {
		t.Fatal(err)
	}
	r := runChecks(e)
	if r.Ready || verdict(r, "data-tree") != "fail" {
		t.Errorf("sibling visible yet ready=%v data-tree=%s", r.Ready, verdict(r, "data-tree"))
	}
}

func TestProbeFailsWhenCgroupfsIsWritable(t *testing.T) {
	e := goodEnv(t)
	e.writable = func(string) error { return nil }
	r := runChecks(e)
	if r.Ready || verdict(r, "cgroup-ro") != "fail" {
		t.Errorf("writable cgroupfs yet ready=%v", r.Ready)
	}
}

func TestProbeFailsWhenEtcCarlosIsReadable(t *testing.T) {
	e := goodEnv(t)
	if err := os.MkdirAll(e.etcCarlos, 0o755); err != nil {
		t.Fatal(err)
	}
	if err := os.WriteFile(filepath.Join(e.etcCarlos, "probe-secret"), []byte("probe"), 0o644); err != nil {
		t.Fatal(err)
	}
	r := runChecks(e)
	if r.Ready || verdict(r, "etc-carlos") != "fail" {
		t.Errorf("readable /etc/carlos yet ready=%v", r.Ready)
	}
}

func TestProbeIMDSVerdicts(t *testing.T) {
	// v4 must be EPERM; anything else is a failure, including a timeout.
	e := goodEnv(t)
	e.sendUDP = func(_, addr string) error {
		if addr == "169.254.169.254:80" {
			return syscall.ETIMEDOUT
		}
		return syscall.EPERM
	}
	if r := runChecks(e); r.Ready || verdict(r, "imds-v4") != "fail" {
		t.Errorf("v4 timeout: ready=%v v4=%s", r.Ready, verdict(r, "imds-v4"))
	}
	// v6 ENETUNREACH is unverified and does not block readiness.
	e = goodEnv(t)
	e.sendUDP = func(_, addr string) error {
		if addr == "[fd00:ec2::254]:80" {
			return syscall.ENETUNREACH
		}
		if addr == "169.254.169.254:80" {
			return syscall.EPERM
		}
		return nil
	}
	if r := runChecks(e); !r.Ready || verdict(r, "imds-v6") != "unverified" {
		t.Errorf("v6 no route: ready=%v v6=%s", r.Ready, verdict(r, "imds-v6"))
	}
	// A successful send to either address is a failure.
	e = goodEnv(t)
	e.sendUDP = func(string, string) error { return nil }
	if r := runChecks(e); r.Ready || verdict(r, "imds-v4") != "fail" {
		t.Errorf("v4 success: ready=%v", r.Ready)
	}
	// A dead loopback means the stack, not the filter, said no.
	e = goodEnv(t)
	e.sendUDP = func(_, addr string) error {
		if addr == "127.0.0.1:9" {
			return syscall.ENETUNREACH
		}
		return syscall.EPERM
	}
	if r := runChecks(e); r.Ready || verdict(r, "loopback") != "fail" {
		t.Errorf("dead loopback: ready=%v", r.Ready)
	}
	// A TCP connect that succeeds is a failure.
	e = goodEnv(t)
	e.connectTCP = func(string, time.Duration) error { return nil }
	if r := runChecks(e); r.Ready || verdict(r, "imds-tcp") != "fail" {
		t.Errorf("tcp success: ready=%v", r.Ready)
	}
}

func TestProbeSymlinkCheckUsesARealSymlink(t *testing.T) {
	// The fixture's data tree is a plain temp dir with no nosymfollow, so
	// the real symlink check FAILS here: that proves the check is real.
	e := goodEnv(t)
	r := runChecks(e)
	_ = r
	if !errors.Is(realSymlinkCheck(filepath.Join(e.dataRoot, "instances", "_probe")), errNotELOOP) {
		t.Error("a followable symlink was not reported")
	}
}

• [ ] Step 2: Run the tests to verify they fail

Run: GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/instanceexec -run TestProbe -count=1 Expected: build failure, undefined: probeEnv.

• [ ] Step 3: Write probe.go, probe_linux.go, probe_other.go

probe.go:

package instanceexec

import (
	"encoding/json"
	"errors"
	"flag"
	"fmt"
	"os"
	"path/filepath"
	"sort"
	"strconv"
	"syscall"
	"time"

	"github.com/carlosframework/platform/internal/cliflag"
	"github.com/carlosframework/platform/internal/instancectl"
	"github.com/carlosframework/platform/internal/instancefs"
)

// probeEnv is everything the checks touch, so the verdict logic is testable
// against a fixture tree and fake syscalls while production runs against
// the real namespace.
type probeEnv struct {
	dataRoot    string // /data as the probe sees it
	runRoot     string // /run/carlos as the probe sees it
	varCache    string // /var/cache
	varLib      string // /var/lib
	etcCarlos   string // /etc/carlos
	procRoot    string // /proc
	cgroupProcs string // a cgroupfs file to test writability on
	pid         func() int
	writable    func(path string) error
	sendUDP     func(network, addr string) error
	connectTCP  func(addr string, timeout time.Duration) error
	mountNoSym  func(mountpoint string) (bool, error)
}

var errNotELOOP = errors.New("symlink was followed")

// RunProbe is `carlos instance-probe`: it runs inside the probe unit's
// namespace (the same text as the tenant templates) and prints a
// ProbeReport on stdout, which the unit's StandardOutput=truncate: lands in
// /run/carlos-control/probe.out for the control service to read (design
// §8.5). It exits 0 whatever the verdict: the verdict is the output, and a
// failed unit would hide it.
func RunProbe(args []string) {
	fs := flag.NewFlagSet("instance-probe", flag.ExitOnError)
	fs.Usage = cliflag.Usage(fs, "usage: carlos instance-probe")
	_ = fs.Parse(args)
	r := runChecks(productionProbeEnv())
	enc := json.NewEncoder(os.Stdout)
	enc.SetIndent("", "  ")
	_ = enc.Encode(r)
}

func productionProbeEnv() probeEnv {
	return probeEnv{
		dataRoot: "/data", runRoot: "/run/carlos", varCache: "/var/cache", varLib: "/var/lib",
		etcCarlos: "/etc/carlos", procRoot: "/proc", cgroupProcs: "/sys/fs/cgroup/cgroup.procs",
		pid:        os.Getpid,
		writable:   accessWritable,
		sendUDP:    sendOneDatagram,
		connectTCP: connectWithTimeout,
		mountNoSym: instancefs.MountNoSymfollow,
	}
}

func listNames(dir string) ([]string, error) {
	entries, err := os.ReadDir(dir)
	if err != nil {
		return nil, err
	}
	names := make([]string, 0, len(entries))
	for _, e := range entries {
		names = append(names, e.Name())
	}
	sort.Strings(names)
	return names, nil
}

func exactly(dir string, want ...string) (string, bool) {
	got, err := listNames(dir)
	if err != nil {
		return err.Error(), false
	}
	sort.Strings(want)
	if fmt.Sprint(got) != fmt.Sprint(want) {
		return fmt.Sprintf("%s lists %v, want %v", dir, got, want), false
	}
	return "", true
}

// runChecks is the verdict. Each line names the property it checks; "fail"
// blocks readiness, "unverified" does not, "ok" is what a ready box shows
// on every line.
func runChecks(e probeEnv) instancectl.ProbeReport {
	var r instancectl.ProbeReport
	add := func(check, verdict, detail string) {
		r.Lines = append(r.Lines, instancectl.VerdictLine{Check: check, Verdict: verdict, Detail: detail})
	}
	pass := func(check string, detail string, ok bool) {
		if ok {
			add(check, "ok", "")
		} else {
			add(check, "fail", detail)
		}
	}

	// The two trees hold exactly the probe's own directory.
	d1, ok1 := exactly(e.dataRoot, "instances")
	d2, ok2 := exactly(filepath.Join(e.dataRoot, "instances"), instancectl.ProbeDirName)
	pass("data-tree", d1+d2, ok1 && ok2)
	r1, ok1 := exactly(e.runRoot, "instances")
	r2, ok2 := exactly(filepath.Join(e.runRoot, "instances"), instancectl.ProbeDirName)
	pass("run-tree", r1+r2, ok1 && ok2)

	// The two trees other carlos-uid units write are empty.
	v1, ok1 := exactly(e.varCache)
	v2, ok2 := exactly(e.varLib)
	pass("var-empty", v1+v2, ok1 && ok2)

	// A private pid namespace: this process is pid 1 and /proc lists no
	// other pid.
	pids, err := listNames(e.procRoot)
	others := ""
	for _, n := range pids {
		if v, err := strconv.Atoi(n); err == nil && v != 1 {
			others += n + " "
		}
	}
	pass("pid1", fmt.Sprintf("pid=%d others=[%s] err=%v", e.pid(), others, err), err == nil && e.pid() == 1 && others == "")

	// cgroupfs is read-only: access(W_OK) answers EROFS without writing.
	werr := e.writable(e.cgroupProcs)
	pass("cgroup-ro", fmt.Sprintf("access(W_OK) = %v", werr), errors.Is(werr, syscall.EROFS))

	// /etc/carlos cannot be opened and its fixture cannot be read.
	_, derr := os.ReadDir(e.etcCarlos)
	_, ferr := os.ReadFile(filepath.Join(e.etcCarlos, "probe-secret"))
	pass("etc-carlos", fmt.Sprintf("readdir=%v readfile=%v", derr, ferr), derr != nil && ferr != nil)

	// A symlink planted in the probe's own directory is refused with ELOOP.
	serr := realSymlinkCheck(filepath.Join(e.dataRoot, "instances", instancectl.ProbeDirName))
	pass("symlink-eloop", fmt.Sprint(serr), serr == nil)

	// Both bound directories are on nosymfollow mounts.
	n1, err1 := e.mountNoSym(filepath.Join(e.dataRoot, "instances", instancectl.ProbeDirName))
	n2, err2 := e.mountNoSym(filepath.Join(e.runRoot, "instances", instancectl.ProbeDirName))
	pass("nosymfollow", fmt.Sprintf("data=%v/%v run=%v/%v", n1, err1, n2, err2), n1 && n2 && err1 == nil && err2 == nil)

	// IMDS, for a packet filter: a datagram's verdict comes straight back
	// from sendmsg. v4 must be EPERM. v6 is EPERM (enforced) or ENETUNREACH
	// (no route to that address: unverified, not blocking). The loopback
	// control proves the stack is live. A TCP connect must not succeed.
	v4 := e.sendUDP("udp4", "169.254.169.254:80")
	pass("imds-v4", fmt.Sprintf("sendto = %v", v4), errors.Is(v4, syscall.EPERM))
	v6 := e.sendUDP("udp6", "[fd00:ec2::254]:80")
	switch {
	case errors.Is(v6, syscall.EPERM):
		add("imds-v6", "ok", "")
	case errors.Is(v6, syscall.ENETUNREACH):
		add("imds-v6", "unverified", "no route to the address; deny not exercised")
	default:
		add("imds-v6", "fail", fmt.Sprintf("sendto = %v", v6))
	}
	lo := e.sendUDP("udp4", "127.0.0.1:9")
	pass("loopback", fmt.Sprintf("sendto = %v", lo), lo == nil)
	tcp := e.connectTCP("169.254.169.254:80", 2*time.Second)
	pass("imds-tcp", fmt.Sprintf("connect = %v", tcp), tcp != nil)

	r.Ready = true
	for _, l := range r.Lines {
		if l.Verdict == "fail" {
			r.Ready = false
		}
	}
	return r
}

// realSymlinkCheck plants a symlink inside dir and tries to open it through
// its name; on a nosymfollow mount the open fails with ELOOP. It returns nil
// for ELOOP, errNotELOOP if the link was followed, and any other error.
func realSymlinkCheck(dir string) error {
	target := filepath.Join(dir, "probe-target")
	link := filepath.Join(dir, "probe-link")
	os.Remove(link)
	os.Remove(target)
	if err := os.WriteFile(target, []byte("x"), 0o600); err != nil {
		return err
	}
	defer os.Remove(target)
	if err := os.Symlink(target, link); err != nil {
		return err
	}
	defer os.Remove(link)
	f, err := os.Open(link)
	if err == nil {
		f.Close()
		return errNotELOOP
	}
	if errors.Is(err, syscall.ELOOP) {
		return nil
	}
	return err
}

probe_linux.go:

//go:build linux

package instanceexec

import (
	"net"
	"time"

	"golang.org/x/sys/unix"
)

// accessWritable asks the kernel whether path could be opened for writing
// without opening it: on a read-only mount access(2) answers EROFS, which
// is exactly the property ProtectControlGroups= must give a tenant.
func accessWritable(path string) error {
	return unix.Access(path, unix.W_OK)
}

// sendOneDatagram sends one empty UDP datagram. With a cgroup BPF egress
// filter (IPAddressDeny=) the filter's verdict is sendmsg's return value,
// EPERM; with no route it is ENETUNREACH; on a live path it is nil.
func sendOneDatagram(network, addr string) error {
	c, err := net.Dial(network, addr)
	if err != nil {
		return unwrapErrno(err)
	}
	defer c.Close()
	_, err = c.Write([]byte{})
	return unwrapErrno(err)
}

func connectWithTimeout(addr string, timeout time.Duration) error {
	c, err := net.DialTimeout("tcp4", addr, timeout)
	if err != nil {
		return unwrapErrno(err)
	}
	c.Close()
	return nil
}

// unwrapErrno returns the syscall.Errno inside a net.OpError so callers can
// errors.Is against EPERM and ENETUNREACH; anything else is returned as is.
func unwrapErrno(err error) error {
	var errno unix.Errno
	if err != nil && errorsAs(err, &errno) {
		return errno
	}
	return err
}

Add a tiny helper in the same file to avoid importing errors twice:

func errorsAs(err error, target *unix.Errno) bool {
	for err != nil {
		if e, ok := err.(unix.Errno); ok {
			*target = e
			return true
		}
		u, ok := err.(interface{ Unwrap() error })
		if !ok {
			return false
		}
		err = u.Unwrap()
	}
	return false
}

(unix.Errno is syscall.Errno; errors.Is(errno, syscall.EPERM) holds.)

probe_other.go:

//go:build !linux

package instanceexec

import (
	"errors"
	"time"
)

func accessWritable(string) error                { return errors.ErrUnsupported }
func sendOneDatagram(string, string) error       { return errors.ErrUnsupported }
func connectWithTimeout(string, time.Duration) error { return errors.ErrUnsupported }

• [ ] Step 4: Run the tests to verify they pass, and cross-compile

Run: GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/instanceexec -count=1 && GOOS=darwin go build ./internal/instanceexec && GOOS=windows go build ./internal/instanceexec Expected: PASS and both builds. (net.Dial("udp4", ...) to a link-local address in sendOneDatagram is never called by the tests; they inject sendUDP.)

• [ ] Step 5: gofmt and commit

gofmt -l internal/instanceexec
git add internal/instanceexec
git commit -m "instanceexec: the readiness probe with injectable checks and the EPERM rule for a packet filter (plan 2, task 4)"

---

Task 5: The control service, carlos instance-control

Files:

• Create: internal/instancectl/server.go
• Create: internal/instancectl/peer_linux.go
• Create: internal/instancectl/peer_other.go
• Create: internal/instancectl/server_test.go

Interfaces:

• Consumes: Task 1's names and types; controlsock.Path; cgroup.TierDefault, cgroup.TierChildren; provision.ValidHostShape; instancectl.ProbeReport.
• Produces: ``go // Run is carlos instance-control (root, socket-activated). func Run(args []string) type Server struct { Systemctl func(ctx context.Context, args ...string) (string, error) // production: exec systemctl, CombinedOutput PeerOK func(conn net.Conn) error // production: SO_PEERCRED uid carlos and pid == MainPID(carlos-edge.service) CgroupRoot string // "/sys/fs/cgroup" ProbeOut string // "/run/carlos-control/probe.out" FixtureOK func() error // stat /etc/carlos/probe-secret is root:carlos 0640 ProbeEvery time.Duration // 5m Logf func(string, ...any) } func (s *Server) Handler() http.Handler func (s *Server) Serve(ctx context.Context, ln net.Listener) error // accept loop with PeerOK before HTTP func (s *Server) RunProbe(ctx context.Context) // start the probe unit, read ProbeOut, store the verdict func (s *Server) Ready() (ProbeReport, bool) func WaitCgroupEmpty(ctx context.Context, root, slice, unit string, every time.Duration) error ``

• [ ] Step 1: Write the failing tests

Create internal/instancectl/server_test.go:

package instancectl

import (
	"context"
	"encoding/json"
	"errors"
	"net"
	"net/http"
	"os"
	"path/filepath"
	"strings"
	"sync"
	"testing"
	"time"
)

type fakeCtl struct {
	mu    sync.Mutex
	calls []string
	// answers maps a leading argument pattern ("show -p InvocationID") to output.
	answers map[string]string
	fail    map[string]error
}

func (f *fakeCtl) run(_ context.Context, args ...string) (string, error) {
	f.mu.Lock()
	defer f.mu.Unlock()
	line := strings.Join(args, " ")
	f.calls = append(f.calls, line)
	for k, err := range f.fail {
		if strings.HasPrefix(line, k) {
			return "", err
		}
	}
	for k, out := range f.answers {
		if strings.HasPrefix(line, k) {
			return out, nil
		}
	}
	return "", nil
}

func (f *fakeCtl) joined() string {
	f.mu.Lock()
	defer f.mu.Unlock()
	return strings.Join(f.calls, "\n")
}

func newServer(t *testing.T, ctl *fakeCtl) (*Server, *http.Client) {
	t.Helper()
	root := t.TempDir()
	s := &Server{
		Systemctl:  ctl.run,
		PeerOK:     func(net.Conn) error { return nil },
		CgroupRoot: filepath.Join(root, "cgroup"),
		ProbeOut:   filepath.Join(root, "probe.out"),
		FixtureOK:  func() error { return nil },
		ProbeEvery: time.Hour,
		Logf:       t.Logf,
	}
	dir, err := os.MkdirTemp("", "ic")
	if err != nil {
		t.Fatal(err)
	}
	t.Cleanup(func() { os.RemoveAll(dir) })
	sock := filepath.Join(dir, "c.sock")
	ln, err := net.Listen("unix", sock)
	if err != nil {
		t.Skipf("cannot bind a unix socket here (%v); run with TMPDIR=/tmp/g outside the sandbox", err)
	}
	ctx, cancel := context.WithCancel(context.Background())
	t.Cleanup(cancel)
	go s.Serve(ctx, ln) //nolint:errcheck
	t.Cleanup(func() { ln.Close() })
	client := &http.Client{Transport: &http.Transport{DialContext: func(ctx context.Context, _, _ string) (net.Conn, error) {
		return (&net.Dialer{}).DialContext(ctx, "unix", sock)
	}}}
	return s, client
}

func post(t *testing.T, c *http.Client, path string) *http.Response {
	t.Helper()
	resp, err := c.Post("http://unix"+path, "", nil)
	if err != nil {
		t.Fatalf("POST %s: %v", path, err)
	}
	return resp
}

func TestStartSetsTheTierThenStartsThenReadsTheInvocation(t *testing.T) {
	ctl := &fakeCtl{answers: map[string]string{"show -p InvocationID --value": "abc123\n"}}
	_, c := newServer(t, ctl)
	resp := post(t, c, "/start?host=bill-2.example.com&role=run&memory_max=536870912")
	if resp.StatusCode != 200 {
		t.Fatalf("status %d", resp.StatusCode)
	}
	var out StartResponse
	if err := json.NewDecoder(resp.Body).Decode(&out); err != nil || out.InvocationID != "abc123" {
		t.Errorf("response = %+v, %v", out, err)
	}
	want := "set-property --runtime carlos-instances-bill\\x2d2.example.com.slice MemoryMax=536870912 MemoryHigh=483183820 MemorySwapMax=536870912\n" +
		"start carlos-instance@bill\\x2d2.example.com.service\n" +
		"show -p InvocationID --value carlos-instance@bill\\x2d2.example.com.service"
	if got := ctl.joined(); got != want {
		t.Errorf("systemctl calls:\n%s\nwant:\n%s", got, want)
	}
}

func TestStartRefusesBadInputsWithoutCallingSystemctl(t *testing.T) {
	ctl := &fakeCtl{}
	_, c := newServer(t, ctl)
	for _, q := range []string{
		"host=Bad_Host&role=run&memory_max=536870912",
		"host=h.example.com&role=exec&memory_max=536870912",
		"host=h.example.com&role=run&memory_max=1",
		"host=h.example.com&role=seed&memory_max=536870912", // seed is not a /start role
		"role=run&memory_max=536870912",
	} {
		resp := post(t, c, "/start?"+q)
		if resp.StatusCode != 400 {
			t.Errorf("%s: status %d, want 400", q, resp.StatusCode)
		}
	}
	if ctl.joined() != "" {
		t.Errorf("systemctl was called: %s", ctl.joined())
	}
}

func TestStartReportsSystemctlFailureAs502(t *testing.T) {
	ctl := &fakeCtl{fail: map[string]error{"start ": errors.New("Job for carlos-instance@h.example.com.service failed")}}
	_, c := newServer(t, ctl)
	resp := post(t, c, "/start?host=h.example.com&role=sidecar&memory_max=1073741824")
	if resp.StatusCode != 502 {
		t.Errorf("status %d, want 502", resp.StatusCode)
	}
	if !strings.Contains(ctl.joined(), "MemoryMax=1073741824 MemoryHigh=966367641 MemorySwapMax=1073741824") {
		t.Errorf("children tier not applied: %s", ctl.joined())
	}
}

func TestSeedBlocksReadsTheRetainedResultWaitsAndStops(t *testing.T) {
	ctl := &fakeCtl{answers: map[string]string{
		"show -p InvocationID,Result,ExecMainStatus,ExecMainCode": "InvocationID=feed\nResult=success\nExecMainStatus=0\nExecMainCode=exited\n",
	}}
	s, c := newServer(t, ctl)
	// An empty cgroup: cgroup.events says populated 0.
	ev := filepath.Join(s.CgroupRoot, "carlos.slice", "carlos-instances.slice", "carlos-instances-h.example.com.slice", "carlos-instance-seed@h.example.com.service", "cgroup.events")
	if err := os.MkdirAll(filepath.Dir(ev), 0o755); err != nil {
		t.Fatal(err)
	}
	if err := os.WriteFile(ev, []byte("populated 0\nfrozen 0\n"), 0o644); err != nil {
		t.Fatal(err)
	}
	resp := post(t, c, "/seed?host=h.example.com&memory_max=536870912")
	if resp.StatusCode != 200 {
		t.Fatalf("status %d", resp.StatusCode)
	}
	var out SeedResponse
	if err := json.NewDecoder(resp.Body).Decode(&out); err != nil || out.InvocationID != "feed" || out.Result != "success" || out.Status != "0" || out.Code != "exited" {
		t.Errorf("response = %+v, %v", out, err)
	}
	calls := ctl.joined()
	for _, want := range []string{"set-property --runtime carlos-instances-h.example.com.slice", "start carlos-instance-seed@h.example.com.service", "show -p InvocationID,Result,ExecMainStatus,ExecMainCode carlos-instance-seed@h.example.com.service", "stop carlos-instance-seed@h.example.com.service"} {
		if !strings.Contains(calls, want) {
			t.Errorf("missing %q in\n%s", want, calls)
		}
	}
	if strings.Index(calls, "show -p") > strings.Index(calls, "stop ") {
		t.Errorf("stop before show:\n%s", calls)
	}
}

func TestSeedFailsWhenTheCgroupStaysPopulated(t *testing.T) {
	ctl := &fakeCtl{answers: map[string]string{"show -p InvocationID,Result": "InvocationID=feed\nResult=success\nExecMainStatus=0\nExecMainCode=exited\n"}}
	s, c := newServer(t, ctl)
	s.cgroupWaitBudget = 200 * time.Millisecond
	ev := filepath.Join(s.CgroupRoot, "carlos.slice", "carlos-instances.slice", "carlos-instances-h.example.com.slice", "carlos-instance-seed@h.example.com.service", "cgroup.events")
	if err := os.MkdirAll(filepath.Dir(ev), 0o755); err != nil {
		t.Fatal(err)
	}
	if err := os.WriteFile(ev, []byte("populated 1\n"), 0o644); err != nil {
		t.Fatal(err)
	}
	resp := post(t, c, "/seed?host=h.example.com&memory_max=536870912")
	if resp.StatusCode != 502 {
		t.Errorf("status %d, want 502 while processes remain", resp.StatusCode)
	}
}

func TestStopResetAndTheirTolerances(t *testing.T) {
	ctl := &fakeCtl{fail: map[string]error{"reset-failed carlos-instance-sidecar@": errors.New("Unit carlos-instance-sidecar@h.example.com.service not loaded.")}}
	_, c := newServer(t, ctl)
	if resp := post(t, c, "/stop?host=h.example.com&role=seed"); resp.StatusCode != 204 {
		t.Errorf("stop status %d", resp.StatusCode)
	}
	if resp := post(t, c, "/reset?host=h.example.com&role=sidecar"); resp.StatusCode != 204 {
		t.Errorf("reset of an unloaded unit: status %d, want 204 (tolerated)", resp.StatusCode)
	}
	if resp := post(t, c, "/reset?host=h.example.com&role=nope"); resp.StatusCode != 400 {
		t.Errorf("bad role: status %d", resp.StatusCode)
	}
	if !strings.Contains(ctl.joined(), "stop carlos-instance-seed@h.example.com.service") {
		t.Errorf("stop not issued: %s", ctl.joined())
	}
}

func TestSweepStopsEveryInstanceUnitAndJob(t *testing.T) {
	ctl := &fakeCtl{answers: map[string]string{
		"list-units": "carlos-instance@a.example.com.service loaded active running carlos exec-backed instance a.example.com\n" +
			"carlos-instance-sidecar@a.example.com.service loaded inactive dead ...\n" +
			"carlos-instance-seed@_aaaa\\x2d0123456789abcdef.service loaded activating start ...\n",
		"list-jobs": "12 carlos-instance@b.example.com.service start waiting\n",
		"show -p ActiveState --value": "inactive\n",
	}}
	s, c := newServer(t, ctl)
	s.cgroupWaitBudget = time.Second
	resp := post(t, c, "/sweep")
	if resp.StatusCode != 200 {
		t.Fatalf("status %d", resp.StatusCode)
	}
	var out SweepResponse
	if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
		t.Fatal(err)
	}
	got := strings.Join(out.Units, " ")
	for _, want := range []string{"carlos-instance@a.example.com.service", "carlos-instance-seed@_aaaa\\x2d0123456789abcdef.service", "carlos-instance@b.example.com.service"} {
		if !strings.Contains(got, want) {
			t.Errorf("sweep did not stop %s: %v", want, out.Units)
		}
	}
	if strings.Contains(got, "sidecar@a") {
		t.Errorf("sweep stopped an inactive unit with no job: %v", out.Units)
	}
	calls := ctl.joined()
	if !strings.Contains(calls, "stop carlos-instance@a.example.com.service") || !strings.Contains(calls, "stop carlos-instance@b.example.com.service") {
		t.Errorf("stops not issued:\n%s", calls)
	}
}

func TestReadyReflectsTheProbeVerdict(t *testing.T) {
	ctl := &fakeCtl{}
	s, c := newServer(t, ctl)
	// The fake "start carlos-instance-probe.service" writes the report the
	// unit's StandardOutput=truncate: would.
	report := ProbeReport{Ready: false, Lines: []VerdictLine{{Check: "pid1", Verdict: "fail", Detail: "pid=4242"}}}
	ctl.answers = map[string]string{}
	ctl.fail = map[string]error{}
	writeReport := func(r ProbeReport) {
		b, _ := json.Marshal(r)
		if err := os.WriteFile(s.ProbeOut, b, 0o644); err != nil {
			t.Fatal(err)
		}
	}
	writeReport(report)
	s.RunProbe(context.Background())
	resp, err := c.Get("http://unix/ready")
	if err != nil || resp.StatusCode != 503 {
		t.Fatalf("not-ready: %v %v", resp, err)
	}
	var got ProbeReport
	if err := json.NewDecoder(resp.Body).Decode(&got); err != nil || got.Ready || len(got.Lines) != 1 {
		t.Errorf("body = %+v, %v", got, err)
	}
	writeReport(ProbeReport{Ready: true, Lines: []VerdictLine{{Check: "pid1", Verdict: "ok"}}})
	s.RunProbe(context.Background())
	if resp, err := c.Get("http://unix/ready"); err != nil || resp.StatusCode != 204 {
		t.Fatalf("ready: %v %v", resp, err)
	}
	if !strings.Contains(ctl.joined(), "start carlos-instance-probe.service") {
		t.Errorf("probe unit not started: %s", ctl.joined())
	}
	// A missing fixture is never a pass.
	s.FixtureOK = func() error { return errors.New("probe fixture absent") }
	s.RunProbe(context.Background())
	if resp, _ := c.Get("http://unix/ready"); resp.StatusCode != 503 {
		t.Errorf("fixture absent yet ready")
	}
}

func TestPeerCheckClosesTheConnectionWithoutAResponse(t *testing.T) {
	ctl := &fakeCtl{}
	s, c := newServer(t, ctl)
	s.PeerOK = func(net.Conn) error { return errors.New("not the agent") }
	_, err := c.Get("http://unix/ready")
	if err == nil {
		t.Fatal("a refused peer got an HTTP response")
	}
	if ctl.joined() != "" {
		t.Errorf("systemctl ran for a refused peer: %s", ctl.joined())
	}
}

func TestWaitCgroupEmptyTreatsAnAbsentCgroupAsEmpty(t *testing.T) {
	root := t.TempDir()
	if err := WaitCgroupEmpty(context.Background(), root, "carlos-instances-h.slice", "carlos-instance@h.service", 10*time.Millisecond); err != nil {
		t.Errorf("absent cgroup: %v", err)
	}
}

• [ ] Step 2: Run the tests to verify they fail

Run: TMPDIR=/tmp/g GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/instancectl -run 'TestStart|TestSeed|TestStop|TestSweep|TestReady|TestPeer|TestWaitCgroup' -count=1 Expected: build failure, undefined: Server.

• [ ] Step 3: Write server.go

package instancectl

import (
	"bufio"
	"context"
	"encoding/json"
	"errors"
	"flag"
	"fmt"
	"log"
	"net"
	"net/http"
	"os"
	"os/exec"
	"path/filepath"
	"strconv"
	"strings"
	"sync"
	"time"

	"github.com/carlosframework/platform/internal/cgroup"
	"github.com/carlosframework/platform/internal/cliflag"
	"github.com/carlosframework/platform/internal/controlsock"
	"github.com/carlosframework/platform/internal/provision"
)

// Server is the root instance-control service (design §5). It does four
// things and nothing else: starts, stops and sweeps the three templates'
// instances, sets a host slice's memory tier before a start, runs the
// readiness probe, and answers /ready. Every systemctl argument it passes
// is built from a validated host by UnitName/SliceName; nothing from a
// request reaches systemctl as a string.
type Server struct {
	// Systemctl runs one systemctl invocation and returns its combined
	// output; production is exec.Command, tests record.
	Systemctl func(ctx context.Context, args ...string) (string, error)
	// PeerOK is asked once per accepted connection, before a byte is read.
	// Production requires SO_PEERCRED uid carlos and pid == MainPID of
	// carlos-edge.service (design §5.4). A refused connection is closed
	// with no response.
	PeerOK func(conn net.Conn) error
	// CgroupRoot is /sys/fs/cgroup; tests point it at a fixture tree.
	CgroupRoot string
	// ProbeOut is where the probe unit's StandardOutput=truncate: lands.
	ProbeOut string
	// FixtureOK verifies /etc/carlos/probe-secret is root:carlos 0640
	// before a probe verdict is accepted (design §8.5, codex R11-1).
	FixtureOK func() error
	// ProbeEvery is the re-run cadence (five minutes in production).
	ProbeEvery time.Duration
	Logf       func(string, ...any)

	// cgroupWaitBudget bounds a wait for an empty cgroup; tests shorten it.
	cgroupWaitBudget time.Duration

	mu     sync.Mutex
	report ProbeReport
	probed bool
}

const (
	probeUnit    = "carlos-instance-probe.service"
	edgeUnit     = "carlos-edge.service"
	parentSlices = "carlos.slice/carlos-instances.slice"
)

// Run is `carlos instance-control`.
func Run(args []string) {
	fs := flag.NewFlagSet("instance-control", flag.ExitOnError)
	fs.Usage = cliflag.Usage(fs, "usage: carlos instance-control (socket-activated by carlos-instance-control.socket)")
	_ = fs.Parse(args)
	if os.Geteuid() != 0 {
		log.Fatal("instance-control: this is the root service behind carlos-instance-control.socket; it is not for operators")
	}
	s := &Server{
		Systemctl: func(ctx context.Context, args ...string) (string, error) {
			out, err := exec.CommandContext(ctx, "systemctl", args...).CombinedOutput()
			return string(out), err
		},
		PeerOK:     productionPeerOK(),
		CgroupRoot: "/sys/fs/cgroup",
		ProbeOut:   filepath.Join(controlsock.Dir, "probe.out"),
		FixtureOK:  fixtureIsRootCarlos0640,
		ProbeEvery: 5 * time.Minute,
		Logf:       log.Printf,
	}
	ln, activated, err := activatedListener()
	if err != nil {
		log.Fatalf("instance-control: %v", err)
	}
	if !activated {
		// Not socket-activated (a hand run): bind the path ourselves with
		// the same ownership the socket unit gives it.
		ln, err = bindControlSocket(controlsock.Path)
		if err != nil {
			log.Fatalf("instance-control: %v", err)
		}
	}
	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()
	go s.probeLoop(ctx)
	if err := s.Serve(ctx, ln); err != nil {
		log.Fatalf("instance-control: %v", err)
	}
}

// Serve accepts connections, asks PeerOK about each, and hands the accepted
// ones to the HTTP server one at a time through a listener that only ever
// yields vetted connections.
func (s *Server) Serve(ctx context.Context, ln net.Listener) error {
	vetted := &vettedListener{Listener: ln, ok: s.PeerOK, logf: s.logf}
	srv := &http.Server{Handler: s.Handler(), ReadHeaderTimeout: 5 * time.Second}
	go func() {
		<-ctx.Done()
		srv.Close()
	}()
	err := srv.Serve(vetted)
	if errors.Is(err, http.ErrServerClosed) || ctx.Err() != nil {
		return nil
	}
	return err
}

// vettedListener runs the peer check inside Accept so a refused peer is
// closed before net/http ever reads from it.
type vettedListener struct {
	net.Listener
	ok   func(net.Conn) error
	logf func(string, ...any)
}

func (v *vettedListener) Accept() (net.Conn, error) {
	for {
		c, err := v.Listener.Accept()
		if err != nil {
			return nil, err
		}
		if err := v.ok(c); err != nil {
			v.logf("instance-control: refused connection: %v", err)
			c.Close()
			continue
		}
		return c, nil
	}
}

func (s *Server) logf(format string, args ...any) {
	if s.Logf != nil {
		s.Logf(format, args...)
	}
}

func (s *Server) Handler() http.Handler {
	mux := http.NewServeMux()
	mux.HandleFunc("POST /start", s.handleStart)
	mux.HandleFunc("POST /seed", s.handleSeed)
	mux.HandleFunc("POST /stop", s.handleStop)
	mux.HandleFunc("POST /reset", s.handleReset)
	mux.HandleFunc("POST /sweep", s.handleSweep)
	mux.HandleFunc("GET /ready", s.handleReady)
	return mux
}

// request is the validated form of a verb's query: nothing else reaches a
// unit name.
type request struct {
	host string
	role Role
	tier cgroup.Limit
}

func parseRequest(r *http.Request, roles []Role, wantTier bool) (request, error) {
	q := r.URL.Query()
	host := q.Get("host")
	if !provision.ValidHostShape(host) {
		return request{}, fmt.Errorf("host %q is not a host", host)
	}
	req := request{host: host}
	if len(roles) > 0 {
		role, ok := ParseRole(q.Get("role"))
		if !ok {
			return request{}, fmt.Errorf("role %q is not run, seed or sidecar", q.Get("role"))
		}
		allowed := false
		for _, r := range roles {
			allowed = allowed || r == role
		}
		if !allowed {
			return request{}, fmt.Errorf("role %s is not valid for this verb", role)
		}
		req.role = role
	}
	if wantTier {
		n, err := strconv.ParseInt(q.Get("memory_max"), 10, 64)
		if err != nil {
			return request{}, fmt.Errorf("memory_max %q is not a number", q.Get("memory_max"))
		}
		switch n {
		case cgroup.TierDefault.MemoryMax, cgroup.TierChildren.MemoryMax:
			req.tier = cgroup.Limit{MemoryMax: n}
		default:
			return request{}, fmt.Errorf("memory_max %d is not a platform tier", n)
		}
	}
	return req, nil
}

func (s *Server) setTier(ctx context.Context, host string, tier cgroup.Limit) error {
	max := strconv.FormatInt(tier.MemoryMax, 10)
	high := strconv.FormatInt(tier.MemoryMax/10*9, 10)
	_, err := s.Systemctl(ctx, "set-property", "--runtime", SliceName(host),
		"MemoryMax="+max, "MemoryHigh="+high, "MemorySwapMax="+max)
	return err
}

func writeJSON(w http.ResponseWriter, status int, v any) {
	w.Header().Set("Content-Type", "application/json")
	w.WriteHeader(status)
	_ = json.NewEncoder(w).Encode(v)
}

func (s *Server) handleStart(w http.ResponseWriter, r *http.Request) {
	req, err := parseRequest(r, []Role{RoleRun, RoleSidecar}, true)
	if err != nil {
		http.Error(w, err.Error(), http.StatusBadRequest)
		return
	}
	ctx := r.Context()
	unit := UnitName(req.role, req.host)
	if err := s.setTier(ctx, req.host, req.tier); err != nil {
		http.Error(w, "set-property: "+err.Error(), http.StatusBadGateway)
		return
	}
	if out, err := s.Systemctl(ctx, "start", unit); err != nil {
		http.Error(w, "start: "+strings.TrimSpace(out+" "+err.Error()), http.StatusBadGateway)
		return
	}
	inv, err := s.Systemctl(ctx, "show", "-p", "InvocationID", "--value", unit)
	if err != nil {
		http.Error(w, "show: "+err.Error(), http.StatusBadGateway)
		return
	}
	s.logf("instance-control: %s: started %s memory.max=%d", req.host, unit, req.tier.MemoryMax)
	writeJSON(w, http.StatusOK, StartResponse{InvocationID: strings.TrimSpace(inv)})
}

// parseShow reads systemctl show's KEY=value lines.
func parseShow(out string) map[string]string {
	m := map[string]string{}
	sc := bufio.NewScanner(strings.NewReader(out))
	for sc.Scan() {
		if k, v, ok := strings.Cut(sc.Text(), "="); ok {
			m[k] = strings.TrimSpace(v)
		}
	}
	return m
}

func (s *Server) handleSeed(w http.ResponseWriter, r *http.Request) {
	req, err := parseRequest(r, nil, true)
	if err != nil {
		http.Error(w, err.Error(), http.StatusBadRequest)
		return
	}
	ctx := r.Context()
	unit := UnitName(RoleSeed, req.host)
	if err := s.setTier(ctx, req.host, req.tier); err != nil {
		http.Error(w, "set-property: "+err.Error(), http.StatusBadGateway)
		return
	}
	// A oneshot's start job completes when ExecStart exits; RemainAfterExit
	// keeps the unit loaded with its result until the stop below.
	startOut, startErr := s.Systemctl(ctx, "start", unit)
	out, err := s.Systemctl(ctx, "show", "-p", "InvocationID,Result,ExecMainStatus,ExecMainCode", unit)
	if err != nil {
		http.Error(w, "show: "+err.Error(), http.StatusBadGateway)
		return
	}
	m := parseShow(out)
	if err := WaitCgroupEmpty(ctx, s.CgroupRoot, SliceName(req.host), unit, s.waitBudget()); err != nil {
		http.Error(w, "seed: "+err.Error(), http.StatusBadGateway)
		return
	}
	if _, err := s.Systemctl(ctx, "stop", unit); err != nil {
		s.logf("instance-control: %s: stop after seed: %v", req.host, err)
	}
	if startErr != nil && m["Result"] == "" {
		http.Error(w, "start: "+strings.TrimSpace(startOut+" "+startErr.Error()), http.StatusBadGateway)
		return
	}
	writeJSON(w, http.StatusOK, SeedResponse{InvocationID: m["InvocationID"], Result: m["Result"], Status: m["ExecMainStatus"], Code: m["ExecMainCode"]})
}

func (s *Server) handleStop(w http.ResponseWriter, r *http.Request) {
	req, err := parseRequest(r, []Role{RoleRun, RoleSeed, RoleSidecar}, false)
	if err != nil {
		http.Error(w, err.Error(), http.StatusBadRequest)
		return
	}
	if out, err := s.Systemctl(r.Context(), "stop", UnitName(req.role, req.host)); err != nil {
		http.Error(w, "stop: "+strings.TrimSpace(out+" "+err.Error()), http.StatusBadGateway)
		return
	}
	w.WriteHeader(http.StatusNoContent)
}

func (s *Server) handleReset(w http.ResponseWriter, r *http.Request) {
	req, err := parseRequest(r, []Role{RoleRun, RoleSeed, RoleSidecar}, false)
	if err != nil {
		http.Error(w, err.Error(), http.StatusBadRequest)
		return
	}
	if out, err := s.Systemctl(r.Context(), "reset-failed", UnitName(req.role, req.host)); err != nil && !strings.Contains(out+err.Error(), "not loaded") {
		http.Error(w, "reset-failed: "+strings.TrimSpace(out+" "+err.Error()), http.StatusBadGateway)
		return
	}
	w.WriteHeader(http.StatusNoContent)
}

// handleSweep stops every instance of the three templates that is not
// inactive or has a job, keyed on the names systemd reports, so a unit
// whose identifier is hashed and whose row is gone is still stopped
// (design §6.7, codex R7-1).
func (s *Server) handleSweep(w http.ResponseWriter, r *http.Request) {
	ctx := r.Context()
	targets := map[string]bool{}
	out, err := s.Systemctl(ctx, "list-units", "--all", "--plain", "--no-legend", "--no-pager",
		UnitPrefixRun+"*.service", UnitPrefixSeed+"*.service", UnitPrefixSidecar+"*.service")
	if err != nil {
		http.Error(w, "list-units: "+err.Error(), http.StatusBadGateway)
		return
	}
	for _, line := range strings.Split(out, "\n") {
		f := strings.Fields(line)
		if len(f) < 3 {
			continue
		}
		if _, _, ok := ParseUnitName(f[0]); !ok {
			continue
		}
		if f[2] != "inactive" {
			targets[f[0]] = true
		}
	}
	jobs, err := s.Systemctl(ctx, "list-jobs", "--plain", "--no-legend", "--no-pager")
	if err != nil {
		http.Error(w, "list-jobs: "+err.Error(), http.StatusBadGateway)
		return
	}
	for _, line := range strings.Split(jobs, "\n") {
		f := strings.Fields(line)
		if len(f) < 2 {
			continue
		}
		if _, _, ok := ParseUnitName(f[1]); ok {
			targets[f[1]] = true
		}
	}
	var stopped []string
	for unit := range targets {
		if _, err := s.Systemctl(ctx, "stop", unit); err != nil {
			s.logf("instance-control: sweep: stop %s: %v", unit, err)
		}
		_, esc, _ := ParseUnitName(unit)
		if err := WaitCgroupEmpty(ctx, s.CgroupRoot, "carlos-instances-"+esc+".slice", unit, s.waitBudget()); err != nil {
			http.Error(w, "sweep: "+unit+": "+err.Error(), http.StatusBadGateway)
			return
		}
		stopped = append(stopped, unit)
	}
	writeJSON(w, http.StatusOK, SweepResponse{Units: stopped})
}

func (s *Server) handleReady(w http.ResponseWriter, r *http.Request) {
	rep, ready := s.Ready()
	if ready {
		w.WriteHeader(http.StatusNoContent)
		return
	}
	writeJSON(w, http.StatusServiceUnavailable, rep)
}

// Ready returns the last probe verdict. Before the first probe, or when
// the last one could not be read, the box is not ready.
func (s *Server) Ready() (ProbeReport, bool) {
	s.mu.Lock()
	defer s.mu.Unlock()
	return s.report, s.probed && s.report.Ready
}

func (s *Server) waitBudget() time.Duration {
	if s.cgroupWaitBudget > 0 {
		return s.cgroupWaitBudget
	}
	return 30 * time.Second
}

// WaitCgroupEmpty waits until the unit's cgroup under the host slice is
// absent or reports "populated 0" in cgroup.events, polling every `every`,
// bounded by ctx and the caller's budget (design §6.5, codex R2-5).
func WaitCgroupEmpty(ctx context.Context, root, slice, unit string, every time.Duration) error {
	events := filepath.Join(root, parentSlices, slice, unit, "cgroup.events")
	deadline, hasDeadline := ctx.Deadline()
	_ = deadline
	_ = hasDeadline
	for {
		b, err := os.ReadFile(events)
		if errors.Is(err, os.ErrNotExist) {
			return nil
		}
		if err == nil && strings.Contains(string(b), "populated 0") {
			return nil
		}
		select {
		case <-ctx.Done():
			return fmt.Errorf("processes remain in %s", unit)
		case <-time.After(every):
		}
	}
}

// probeLoop runs the probe at start and every ProbeEvery.
func (s *Server) probeLoop(ctx context.Context) {
	s.RunProbe(ctx)
	t := time.NewTicker(s.ProbeEvery)
	defer t.Stop()
	for {
		select {
		case <-ctx.Done():
			return
		case <-t.C:
			s.RunProbe(ctx)
		}
	}
}

// RunProbe starts the probe unit (a oneshot: start blocks until it exits),
// reads the report it left, and stores the verdict. A fixture that is not
// root:carlos 0640, a probe that could not be started, or a report that
// could not be read all leave the box not ready, with the reason on a
// verdict line; readiness is never the default.
func (s *Server) RunProbe(ctx context.Context) {
	set := func(r ProbeReport) {
		s.mu.Lock()
		s.report, s.probed = r, true
		s.mu.Unlock()
	}
	if err := s.FixtureOK(); err != nil {
		set(ProbeReport{Lines: []VerdictLine{{Check: "fixture", Verdict: "fail", Detail: err.Error()}}})
		return
	}
	if out, err := s.Systemctl(ctx, "start", probeUnit); err != nil {
		set(ProbeReport{Lines: []VerdictLine{{Check: "probe-unit", Verdict: "fail", Detail: strings.TrimSpace(out + " " + err.Error())}}})
		return
	}
	b, err := os.ReadFile(s.ProbeOut)
	if err != nil {
		set(ProbeReport{Lines: []VerdictLine{{Check: "probe-output", Verdict: "fail", Detail: err.Error()}}})
		return
	}
	var r ProbeReport
	if err := json.Unmarshal(b, &r); err != nil {
		set(ProbeReport{Lines: []VerdictLine{{Check: "probe-output", Verdict: "fail", Detail: err.Error()}}})
		return
	}
	for _, l := range r.Lines {
		if l.Verdict != "ok" {
			s.logf("instance-control: probe: %s %s %s", l.Check, l.Verdict, l.Detail)
		}
	}
	set(r)
}

Remove the two _ = deadline lines and the deadline, hasDeadline := ctx.Deadline() line from WaitCgroupEmpty before committing; they are shown only so the function is complete if an implementer wants a deadline log. The seed and sweep handlers must wrap their ctx with the budget: in handleSeed and handleSweep, before calling WaitCgroupEmpty, do

	wctx, cancel := context.WithTimeout(ctx, s.waitBudget())
	defer cancel()

and pass wctx, with every = 200 * time.Millisecond (not the budget). Update the two call sites accordingly: WaitCgroupEmpty(wctx, s.CgroupRoot, SliceName(req.host), unit, 200*time.Millisecond).

peer_linux.go:

//go:build linux

package instancectl

import (
	"context"
	"errors"
	"fmt"
	"net"
	"os"
	"os/exec"
	"os/user"
	"strconv"
	"strings"
	"sync"
	"time"

	"golang.org/x/sys/unix"
)

// peerCred reads SO_PEERCRED from a connected unix socket.
func peerCred(conn net.Conn) (*unix.Ucred, error) {
	uc, ok := conn.(*net.UnixConn)
	if !ok {
		return nil, errors.New("not a unix connection")
	}
	raw, err := uc.SyscallConn()
	if err != nil {
		return nil, err
	}
	var cred *unix.Ucred
	var cerr error
	if err := raw.Control(func(fd uintptr) {
		cred, cerr = unix.GetsockoptUcred(int(fd), unix.SOL_SOCKET, unix.SO_PEERCRED)
	}); err != nil {
		return nil, err
	}
	return cred, cerr
}

// productionPeerOK requires the peer to be uid carlos AND the main process
// of carlos-edge.service as systemd reports it (design §5.4): a flat tenant
// is a child of that pid, never the pid itself, and a nested tenant cannot
// reach the socket at all. MainPID is asked of systemctl and cached for one
// second, because the agent restarts and the pid changes.
func productionPeerOK() func(net.Conn) error {
	u, err := user.Lookup("carlos")
	if err != nil {
		return func(net.Conn) error { return fmt.Errorf("no carlos user: %v", err) }
	}
	carlosUID, _ := strconv.Atoi(u.Uid)
	var mu sync.Mutex
	var mainPID int
	var at time.Time
	edgePID := func() (int, error) {
		mu.Lock()
		defer mu.Unlock()
		if time.Since(at) < time.Second && mainPID > 0 {
			return mainPID, nil
		}
		ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
		defer cancel()
		out, err := exec.CommandContext(ctx, "systemctl", "show", "-p", "MainPID", "--value", edgeUnit).Output()
		if err != nil {
			return 0, err
		}
		pid, err := strconv.Atoi(strings.TrimSpace(string(out)))
		if err != nil || pid <= 0 {
			return 0, fmt.Errorf("carlos-edge.service has no MainPID")
		}
		mainPID, at = pid, time.Now()
		return pid, nil
	}
	return func(conn net.Conn) error {
		cred, err := peerCred(conn)
		if err != nil {
			return err
		}
		if int(cred.Uid) != carlosUID {
			return fmt.Errorf("peer uid %d is not carlos", cred.Uid)
		}
		pid, err := edgePID()
		if err != nil {
			return err
		}
		if int(cred.Pid) != pid {
			return fmt.Errorf("peer pid %d is not the edge's main pid %d", cred.Pid, pid)
		}
		return nil
	}
}

// activatedListener returns the socket-activated listener when systemd
// passed one (LISTEN_PID is us, LISTEN_FDS is 1, the fd is 3).
func activatedListener() (net.Listener, bool, error) {
	if os.Getenv("LISTEN_PID") != strconv.Itoa(os.Getpid()) {
		return nil, false, nil
	}
	if os.Getenv("LISTEN_FDS") != "1" {
		return nil, false, fmt.Errorf("LISTEN_FDS=%q, want 1", os.Getenv("LISTEN_FDS"))
	}
	f := os.NewFile(3, "control.sock")
	defer f.Close()
	ln, err := net.FileListener(f)
	if err != nil {
		return nil, false, err
	}
	return ln, true, nil
}

// bindControlSocket is the hand-run fallback: bind the path with the
// ownership the socket unit would give it (carlos, 0600).
func bindControlSocket(path string) (net.Listener, error) {
	_ = os.Remove(path)
	ln, err := net.Listen("unix", path)
	if err != nil {
		return nil, err
	}
	u, err := user.Lookup("carlos")
	if err != nil {
		ln.Close()
		return nil, err
	}
	uid, _ := strconv.Atoi(u.Uid)
	gid, _ := strconv.Atoi(u.Gid)
	if err := os.Chown(path, uid, gid); err != nil {
		ln.Close()
		return nil, err
	}
	if err := os.Chmod(path, 0o600); err != nil {
		ln.Close()
		return nil, err
	}
	return ln, nil
}

// fixtureIsRootCarlos0640 checks the probe fixture tmpfiles stages.
func fixtureIsRootCarlos0640() error {
	fi, err := os.Stat("/etc/carlos/probe-secret")
	if err != nil {
		return fmt.Errorf("probe fixture absent: %v", err)
	}
	st, ok := fi.Sys().(*unix.Stat_t)
	if !ok {
		return errors.New("probe fixture: no stat")
	}
	g, err := user.LookupGroup("carlos")
	if err != nil {
		return err
	}
	gid, _ := strconv.Atoi(g.Gid)
	if st.Uid != 0 || int(st.Gid) != gid || fi.Mode().Perm() != 0o640 {
		return fmt.Errorf("probe fixture is uid %d gid %d mode %o, want root:carlos 0640", st.Uid, st.Gid, fi.Mode().Perm())
	}
	return nil
}

Note: fi.Sys().(*unix.Stat_t) is a *syscall.Stat_t; unix.Stat_t and syscall.Stat_t are distinct types. Use fi.Sys().(*syscall.Stat_t) with import "syscall" instead. Adjust the import block accordingly.

peer_other.go:

//go:build !linux

package instancectl

import (
	"errors"
	"net"
)

func productionPeerOK() func(net.Conn) error {
	return func(net.Conn) error { return errors.ErrUnsupported }
}
func activatedListener() (net.Listener, bool, error) { return nil, false, errors.ErrUnsupported }
func bindControlSocket(string) (net.Listener, error) { return nil, errors.ErrUnsupported }
func fixtureIsRootCarlos0640() error                { return errors.ErrUnsupported }

• [ ] Step 4: Run the tests to verify they pass, and cross-compile

Run: TMPDIR=/tmp/g GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/instancectl -count=1 -v (outside the sandbox so the sockets bind) Expected: PASS, none skipped.

Run: GOOS=darwin go build ./internal/instancectl && GOOS=windows go build ./internal/instancectl Expected: both build.

• [ ] Step 5: gofmt and commit

gofmt -l internal/instancectl
git add internal/instancectl
git commit -m "instancectl: the root control service, peer-authenticated, with the five verbs and the probe loop (plan 2, task 5)"

---

Task 6: Unit files, verb dispatch, bootstrap arming, docs

Files:

• Create: internal/host/units/carlos-instance@.service, carlos-instance-seed@.service, carlos-instance-sidecar@.service, carlos-instance-probe.service, carlos-instance-control.socket, carlos-instance-control.service
• Modify: internal/host/bootstrap.go (enabledUnits; tmpfilesConf gains the two _probe lines; dirs)
• Modify: internal/host/systemunits.go (systemUnitFiles and the arming list gain the control socket)
• Modify: main.go (four case arms)
• Modify: docs/site/cli.md (four headings under "On the host")
• Test: internal/host/units_isolation_test.go (append)

Interfaces:

• Consumes: instancectl.Run, instanceexec.RunExec, RunExit, RunProbe.
• Produces: the unit names plan 3 starts through the control service.

• [ ] Step 1: Write the failing tests

Append to internal/host/units_isolation_test.go:

// The three tenant templates and the probe carry one namespace: byte-
// identical hardening blocks, so a property proven by the probe is the
// property a tenant gets (design §4.2, §8.5).
func TestInstanceTemplatesShareOneNamespace(t *testing.T) {
	block := func(name string) string {
		u := readUnit(t, name)
		start := strings.Index(u, "# --- the namespace")
		end := strings.Index(u, "# --- per-service resources")
		if start < 0 || end < 0 || end < start {
			t.Fatalf("%s: namespace block markers missing", name)
		}
		return u[start:end]
	}
	ref := block("carlos-instance@.service")
	for _, name := range []string{"carlos-instance-seed@.service", "carlos-instance-sidecar@.service", "carlos-instance-probe.service"} {
		if block(name) != ref {
			t.Errorf("%s namespace block differs from carlos-instance@.service", name)
		}
	}
	for _, want := range []string{
		"ProtectSystem=strict", "TemporaryFileSystem=/data:ro", "TemporaryFileSystem=/run/carlos:ro",
		"TemporaryFileSystem=/var/cache:ro", "TemporaryFileSystem=/var/lib:ro", "InaccessiblePaths=/etc/carlos",
		"ProtectControlGroups=yes", "ProtectProc=invisible", "ProcSubset=pid", "PrivatePIDs=yes",
		"RestrictNamespaces=yes", "NoNewPrivileges=yes", "CapabilityBoundingSet=",
		"IPAddressDeny=169.254.169.254/32 fd00:ec2::254/128", "MemoryDenyWriteExecute=yes",
	} {
		if !strings.Contains(ref, want) {
			t.Errorf("namespace block lacks %q", want)
		}
	}
	for name, wants := range map[string][]string{
		"carlos-instance@.service": {"ExecStart=/opt/carlos/carlos instance-exec run %I", "ExecStopPost=/opt/carlos/carlos instance-exit run %I",
			"BindPaths=/data/instances/%I /run/carlos/instances/%I", "StandardOutput=file:/run/carlos/logs/%I.run.sock",
			"Slice=carlos-instances-%i.slice", "Restart=no", "KillMode=mixed", "TimeoutStopSec=20", "BindsTo=carlos-edge.service",
			"Requires=data-instances.mount run-carlos-instances.mount run-carlos-logs.mount", "OOMScoreAdjust=500", "OOMPolicy=kill", "TasksMax=256", "Type=simple"},
		"carlos-instance-seed@.service": {"Type=oneshot", "RemainAfterExit=yes", "TimeoutStartSec=120", "ExecStart=/opt/carlos/carlos instance-exec seed %I",
			"ExecStopPost=/opt/carlos/carlos instance-exit seed %I", "StandardOutput=file:/run/carlos/logs/%I.seed.sock"},
		"carlos-instance-sidecar@.service": {"Type=simple", "ExecStart=/opt/carlos/carlos instance-exec sidecar %I",
			"ExecStopPost=/opt/carlos/carlos instance-exit sidecar %I", "StandardOutput=file:/run/carlos/logs/%I.sidecar.sock"},
		"carlos-instance-probe.service": {"Type=oneshot", "ExecStart=/opt/carlos/carlos instance-probe",
			"BindPaths=/data/instances/_probe /run/carlos/instances/_probe", "StandardOutput=truncate:/run/carlos-control/probe.out"},
	} {
		u := readUnit(t, name)
		for _, w := range wants {
			if !strings.Contains(u, w) {
				t.Errorf("%s lacks %q", name, w)
			}
		}
		if strings.Contains(u, "ExecStopPost=+") {
			t.Errorf("%s uses ExecStopPost=+: the plus prefix keeps the unit's tmpfs (codex R3-2)", name)
		}
		if name != "carlos-instance-probe.service" && !strings.Contains(u, "User=carlos") {
			t.Errorf("%s does not run as carlos", name)
		}
	}
}

func TestControlUnitsShape(t *testing.T) {
	s := readUnit(t, "carlos-instance-control.socket")
	for _, want := range []string{"ListenStream=/run/carlos-control/control.sock", "SocketUser=carlos", "SocketGroup=carlos", "SocketMode=0600", "Accept=no", "WantedBy=sockets.target"} {
		if !strings.Contains(s, want) {
			t.Errorf("socket unit lacks %q", want)
		}
	}
	u := readUnit(t, "carlos-instance-control.service")
	for _, want := range []string{"ExecStart=/opt/carlos/carlos instance-control", "Requires=carlos-instance-control.socket",
		"data-instances.mount", "ProtectSystem=strict", "ReadWritePaths=/run/carlos-control", "RestrictAddressFamilies=AF_UNIX", "MemoryMax=64M"} {
		if !strings.Contains(u, want) {
			t.Errorf("control service lacks %q", want)
		}
	}
	for _, never := range []string{"User=", "NoNewPrivileges"} {
		if strings.Contains(u, never) {
			t.Errorf("control service must run as root for systemctl; found %q", never)
		}
	}
	have := map[string]bool{}
	for _, e := range enabledUnits {
		have[e] = true
	}
	if !have["carlos-instance-control.socket"] {
		t.Error("enabledUnits lacks the control socket")
	}
	if have["carlos-instance-control.service"] || have["carlos-instance-probe.service"] || have["carlos-instance@.service"] {
		t.Error("a template, the probe or the socket-activated service must not be enabled")
	}
	for _, want := range []string{"d /data/.instances/_probe 0700 carlos carlos -", "d /run/carlos/.instances/_probe 2750 carlos carlos-sockets -"} {
		if !strings.Contains(tmpfilesConf, want+"\n") {
			t.Errorf("tmpfilesConf lacks %q", want)
		}
	}
}

// The same-uid audit (design §9, codex R9-1, R9-2, R10-1): every tree a
// carlos-uid unit writes, every carlos-owned cgroup, and every credential a
// carlos-group process reads must be hidden from the tenant templates.
func TestSameUIDAuditIsCoveredByTheTemplates(t *testing.T) {
	ns := readUnit(t, "carlos-instance@.service")
	hidden := func(path string) bool {
		for _, line := range strings.Split(ns, "\n") {
			for _, key := range []string{"TemporaryFileSystem=", "InaccessiblePaths="} {
				if strings.HasPrefix(line, key) {
					for _, p := range strings.Fields(strings.TrimPrefix(line, key)) {
						p = strings.TrimSuffix(p, ":ro")
						if path == p || strings.HasPrefix(path, p+"/") {
							return true
						}
					}
				}
			}
		}
		return false
	}
	names, err := unitNames()
	if err != nil {
		t.Fatal(err)
	}
	for _, name := range names {
		u := readUnit(t, name)
		if !strings.Contains(u, "User=carlos") {
			continue
		}
		for _, line := range strings.Split(u, "\n") {
			for _, key := range []string{"ReadWritePaths=", "CacheDirectory=", "StateDirectory=", "RuntimeDirectory="} {
				if !strings.HasPrefix(line, key) {
					continue
				}
				for _, p := range strings.Fields(strings.TrimPrefix(line, key)) {
					switch key {
					case "CacheDirectory=":
						p = "/var/cache/" + p
					case "StateDirectory=":
						p = "/var/lib/" + p
					case "RuntimeDirectory=":
						p = "/run/" + p
					}
					if !hidden(p) && !strings.HasPrefix(p, "/run/carlos/instances/") && !strings.HasPrefix(p, "/data/instances/") {
						t.Errorf("%s writes %s, which the tenant template does not hide", name, p)
					}
				}
			}
		}
		if strings.Contains(u, "Delegate=") && !strings.Contains(ns, "ProtectControlGroups=yes") {
			t.Errorf("%s delegates a cgroup subtree and the template leaves cgroupfs writable", name)
		}
	}
	// Credential inputs a carlos-group process reads (the second table).
	for _, cred := range []string{"/etc/carlos/relay-masters"} {
		if !hidden(cred) {
			t.Errorf("credential input %s is not hidden from tenants", cred)
		}
	}
}

• [ ] Step 2: Run the tests to verify they fail

Run: GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/host -run 'TestInstanceTemplates|TestControlUnits|TestSameUIDAudit' -count=1 Expected: FAIL (units missing).

• [ ] Step 3: Write the six unit files

internal/host/units/carlos-instance@.service:

# One isolated exec-backed instance (isolated-exec design §4.1). The
# activator owns it exactly as it owned the exec child this replaces: lease,
# restore, seed, replication, hibernation, control socket, logs. What
# changed is that the process is a unit, still uid carlos, inside a mount
# namespace in which nothing but the instance's own two directories exists,
# on mounts where no symlink is ever followed. The instance name is the
# escaped bounded identifier (%i); paths use the unescaped one (%I).
[Unit]
Description=carlos exec-backed instance %I
# Stop-only coupling: when the edge stops (including the stop half of a
# restart) this unit stops, and nothing starts it again but the agent.
# PartOf= would propagate the RESTART and race the agent's own startup
# (codex R1-7).
BindsTo=carlos-edge.service
Requires=data-instances.mount run-carlos-instances.mount run-carlos-logs.mount
After=carlos-edge.service data-instances.mount run-carlos-instances.mount run-carlos-logs.mount

[Service]
Type=simple
ExecStart=/opt/carlos/carlos instance-exec run %I
# Runs inside the same namespace, as carlos, after this invocation's
# processes have been signalled: leaves an ADVISORY exit record in the
# instance's own run directory (§6.5). Not "+": the "+" prefix drops User=
# and the Private*/Protect* settings but keeps the explicit
# TemporaryFileSystem=/BindPaths= mounts (codex R3-2), so nothing run from
# here can see the host's /run/carlos anyway.
ExecStopPost=/opt/carlos/carlos instance-exit run %I
User=carlos
Group=carlos
# One slice per host holds the instance, its seed and its sidecar, and
# carries the tier (§4.3, §6.6). Slices referenced here are created on
# demand; no per-host unit file exists.
Slice=carlos-instances-%i.slice
# The activator owns recycling: a crash stands the entry down and the next
# request wakes it. systemd must not race that.
Restart=no
# SIGTERM to the app, SIGKILL to anything it left behind after
# TimeoutStopSec; the activator's stop budget is 20s.
KillMode=mixed
TimeoutStopSec=20
# The log stream: stdout and stderr connect to the agent's per-role
# listener (§6.4). PID 1 opens this as root, outside the namespace, on the
# nosymfollow logs mount, at a path no tenant mount contains.
StandardOutput=file:/run/carlos/logs/%I.run.sock
StandardError=inherit

# --- the namespace (titogo@.service, 2026-07-31 design, on systemd 257) ---
ProtectSystem=strict
TemporaryFileSystem=/data:ro
TemporaryFileSystem=/run/carlos:ro
# Every other tree a carlos-uid unit writes on the box is hidden the same
# way (codex R9-2): the restore drills' cache under /var/cache, and
# /var/lib as a class. An instance needs nothing from either.
TemporaryFileSystem=/var/cache:ro
TemporaryFileSystem=/var/lib:ro
# Root-written credentials a carlos-GROUP process may read live here:
# /etc/carlos/relay-masters is root:carlos 0640 for carlos-relay.service,
# and an instance's Group=carlos would read it too (codex R10-1). An
# instance gets its configuration through its launch file and needs
# nothing from /etc/carlos.
InaccessiblePaths=/etc/carlos
BindPaths=/data/instances/%I /run/carlos/instances/%I
ReadWritePaths=/data/instances/%I /run/carlos/instances/%I
ProtectHome=yes
PrivateTmp=yes
PrivateDevices=yes
# /sys/fs/cgroup read-only: the edge's delegated subtree is carlos-owned,
# and its cgroup.kill / cgroup.freeze would otherwise be one write away
# for a same-uid tenant (codex R9-1).
ProtectControlGroups=yes
ProtectProc=invisible
ProcSubset=pid
PrivatePIDs=yes
RestrictNamespaces=yes
NoNewPrivileges=yes
CapabilityBoundingSet=
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
IPAddressDeny=169.254.169.254/32 fd00:ec2::254/128
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectKernelLogs=yes
ProtectClock=yes
ProtectHostname=yes
RemoveIPC=yes
LockPersonality=yes
RestrictRealtime=yes
RestrictSUIDSGID=yes
MemoryDenyWriteExecute=yes
SystemCallArchitectures=native

# --- per-service resources; the tier lives on the host slice (§4.3) ---
TasksMax=256
# The kernel picks a tenant before it picks anything else in carlos.slice.
OOMScoreAdjust=500
# One OOM kill takes every process of THIS service (systemd sets
# memory.oom.group on the service cgroup for OOMPolicy=kill). The sidecar
# is a sibling service with its own family rule (§4.3).
OOMPolicy=kill

carlos-instance-seed@.service: the same file with these differences, and the same two marker comment lines verbatim (# --- the namespace ... and # --- per-service resources; ...) so the test's block comparison holds:

• Description=carlos exec-backed instance %I (seed)
• Type=oneshot, followed by RemainAfterExit=yes and a comment: # After ExecStart exits the unit stays active (exited) or failed, loaded, with its InvocationID, Result and ExecMainStatus retained until the control service stops it, so the seed's result can never be garbage-collected before it is read (codex R2-4)., then TimeoutStartSec=120 with the comment # Today's seed budget; a oneshot has no start timeout by default (codex R1-5).
• ExecStart=/opt/carlos/carlos instance-exec seed %I, ExecStopPost=/opt/carlos/carlos instance-exit seed %I, StandardOutput=file:/run/carlos/logs/%I.seed.sock.
• No Restart=, KillMode= and TimeoutStopSec=20 kept.

carlos-instance-sidecar@.service: the run file with Description=carlos exec-backed instance %I (sidecar), ExecStart=/opt/carlos/carlos instance-exec sidecar %I, ExecStopPost=/opt/carlos/carlos instance-exit sidecar %I, StandardOutput=file:/run/carlos/logs/%I.sidecar.sock; everything else identical.

carlos-instance-probe.service:

# The readiness probe (isolated-exec design §8.5): a oneshot under the SAME
# namespace text as the tenant templates (the test pins the block byte for
# byte), run by carlos-instance-control at its start and every five
# minutes. From inside it asserts what a tenant would see and prints a
# verdict; the unit's StandardOutput=truncate: lands it where the control
# service reads it. It exits 0 whatever the verdict: the verdict is the
# output. Its instance directory is _probe, a name no host can have; the
# tmpfiles lines create it.
[Unit]
Description=carlos instance isolation probe
Requires=data-instances.mount run-carlos-instances.mount
After=data-instances.mount run-carlos-instances.mount

[Service]
Type=oneshot
ExecStart=/opt/carlos/carlos instance-probe
User=carlos
Group=carlos
Slice=carlos-instances.slice
StandardOutput=truncate:/run/carlos-control/probe.out
StandardError=journal
TimeoutStartSec=30

# --- the namespace (titogo@.service, 2026-07-31 design, on systemd 257) ---
ProtectSystem=strict
TemporaryFileSystem=/data:ro
TemporaryFileSystem=/run/carlos:ro
# Every other tree a carlos-uid unit writes on the box is hidden the same
# way (codex R9-2): the restore drills' cache under /var/cache, and
# /var/lib as a class. An instance needs nothing from either.
TemporaryFileSystem=/var/cache:ro
TemporaryFileSystem=/var/lib:ro
# Root-written credentials a carlos-GROUP process may read live here:
# /etc/carlos/relay-masters is root:carlos 0640 for carlos-relay.service,
# and an instance's Group=carlos would read it too (codex R10-1). An
# instance gets its configuration through its launch file and needs
# nothing from /etc/carlos.
InaccessiblePaths=/etc/carlos
BindPaths=/data/instances/_probe /run/carlos/instances/_probe
ReadWritePaths=/data/instances/_probe /run/carlos/instances/_probe
ProtectHome=yes
PrivateTmp=yes
PrivateDevices=yes
# /sys/fs/cgroup read-only: the edge's delegated subtree is carlos-owned,
# and its cgroup.kill / cgroup.freeze would otherwise be one write away
# for a same-uid tenant (codex R9-1).
ProtectControlGroups=yes
ProtectProc=invisible
ProcSubset=pid
PrivatePIDs=yes
RestrictNamespaces=yes
NoNewPrivileges=yes
CapabilityBoundingSet=
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
IPAddressDeny=169.254.169.254/32 fd00:ec2::254/128
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectKernelLogs=yes
ProtectClock=yes
ProtectHostname=yes
RemoveIPC=yes
LockPersonality=yes
RestrictRealtime=yes
RestrictSUIDSGID=yes
MemoryDenyWriteExecute=yes
SystemCallArchitectures=native

# --- per-service resources; the tier lives on the host slice (§4.3) ---
TasksMax=64
MemoryMax=64M

The test's block comparison requires the namespace block to be byte-identical across the four files, and the probe's BindPaths=/ReadWritePaths= name _probe where the templates name %I. Resolve this by moving the two BindPaths=/ReadWritePaths= lines OUT of the marked block in all four files: place them immediately after StandardError= (templates) / TimeoutStartSec=30 (probe), before the # --- the namespace marker, with a one-line comment # The instance's own two directories, the only writable data paths. The marked block then starts at ProtectSystem=strict and is identical in all four files. Update the test's wants for carlos-instance@.service and the probe accordingly (they check strings.Contains on the whole file, so they still pass).

carlos-instance-control.socket:

# The root instance-control channel's socket (isolated-exec design §5.2).
# In /run/carlos-control, a root-owned directory beside /run/carlos, never
# under it: an instance's namespace replaces /run/carlos with an empty
# tmpfs, and nothing an instance can reach must resolve to this path. Owned
# by carlos so the agent can connect; the service authenticates the peer
# process itself (§5.4).
[Unit]
Description=carlos instance control socket (agent to root)

[Socket]
ListenStream=/run/carlos-control/control.sock
SocketUser=carlos
SocketGroup=carlos
SocketMode=0600
Accept=no

[Install]
WantedBy=sockets.target

carlos-instance-control.service:

# The root instance-control service (isolated-exec design §5). Socket-
# activated, so it runs only while there is work. Its whole job is
# systemctl start/stop/set-property on the three instance templates' units
# and their host slices, on the agent's request, plus running the readiness
# probe. Every unit name it passes to systemctl is built from a validated
# host by the platform's own naming; nothing from a request reaches
# systemctl as a string.
#
# Runs as ROOT deliberately, with no NoNewPrivileges= and no syscall
# filter, for the systemctl child (the same rationale as
# carlos-stage.service). It accepts a connection only from the edge unit's
# main process (SO_PEERCRED uid carlos, pid == MainPID of
# carlos-edge.service): a flat tenant is a child of that pid, never the pid
# itself, and a nested tenant cannot reach the socket at all.
[Unit]
Description=carlos instance control (root: start/stop carlos-instance*@ units for the agent)
Requires=carlos-instance-control.socket data-instances.mount run-carlos-instances.mount run-carlos-logs.mount
After=data-instances.mount run-carlos-instances.mount run-carlos-logs.mount

[Service]
ExecStart=/opt/carlos/carlos instance-control
ProtectSystem=strict
ReadWritePaths=/run/carlos-control
ProtectHome=yes
PrivateTmp=yes
PrivateDevices=yes
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectKernelLogs=yes
RestrictAddressFamilies=AF_UNIX
MemoryDenyWriteExecute=yes
MemoryMax=64M

• [ ] Step 4: Bootstrap, systemunits, main.go, docs

internal/host/bootstrap.go:

• enabledUnits gains "carlos-instance-control.socket" after the three mounts, with the comment // The root instance-control channel (design §5.2); socket-activated, so only the socket is enabled and started.
• tmpfilesConf gains, after the /run/carlos/.instances and /run/carlos/instances lines: ``go // The readiness probe's own instance directory (design §8.5): a name // no host can have, created every boot so the probe unit's bind // sources exist before it starts. "d /data/.instances/_probe 0700 " + serviceUser + " " + serviceUser + " -\n" + "d /run/carlos/.instances/_probe 2750 " + serviceUser + " " + socketsGroup + " -\n" + ``
• dirs gains {filepath.Join(dataDir, ".instances", "_probe"), 0o700} and {filepath.Join(runDir, ".instances", "_probe"), 0o750 | os.ModeSetgid} after the .instances entries.

internal/host/systemunits.go: add const instanceControlSocket = "carlos-instance-control.socket"; extend systemUnitFiles with instanceControlSocket and "carlos-instance-control.service"; in ensureSystemUnits, probe is-active on instanceControlSocket as well (ctlActive := parseIsActive(...)), treat a nil like the existing nil case, and include !*ctlActive in the "needs arming" condition; pass instanceControlSocket in the systemctlPlan([...]) list so the belt enables and starts it. Update TestSystemUnitsAreShapedLikeTheStageRelay only if it enumerates systemUnitFiles (read it first).

main.go: add after case "adopt": arm:

	case "instance-control":
		instancectl.Run(args)
	case "instance-exec":
		instanceexec.RunExec(args)
	case "instance-exit":
		instanceexec.RunExit(args)
	case "instance-probe":
		instanceexec.RunProbe(args)

with the two imports "github.com/carlosframework/platform/internal/instancectl" and "github.com/carlosframework/platform/internal/instanceexec".

docs/site/cli.md, under ## On the host after the ### carlos adopt section, add four sections (plain prose, no em dashes):

### carlos instance-control

The root half of isolated exec. `carlos-instance-control.socket` activates
it when the agent connects; it starts, stops and sweeps the
`carlos-instance*@` units for the agent, sets each host's memory tier on
its slice before a start, and runs the isolation probe at start and every
five minutes. It accepts a connection only from the edge unit's own main
process. Not for operators: there is nothing to type.

### carlos instance-exec

The first thing an isolated instance unit runs. It reads the launch file
the agent wrote into the instance's directory and execs the host's live
link with exactly the arguments and environment in it, refusing anything
that is not the platform's own live link. `ExecStart=` of the three
instance templates.

### carlos instance-exit

`ExecStopPost=` of the three instance templates. It leaves an advisory
exit record in the instance's run directory from the four variables
systemd provides; the agent proves termination from the unit's state and
its cgroup, not from this file.

### carlos instance-probe

The isolation probe, run under the same namespace text as the instance
templates. From inside it checks what a tenant would see: only its own
directories, only its own pid, a read-only cgroupfs, no `/etc/carlos`, a
symlink that is not followed, and a metadata service that a packet filter
refuses. It prints a verdict; `carlos-instance-control` reads it and
refuses to start isolated instances until every line passes.

• [ ] Step 5: Run the tests, the docsite gate, and the full gate

Run: GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/host ./internal/docsite -count=1 (with TMPDIR=/tmp/g outside the sandbox for internal/host) Expected: PASS, including TestCLICommandsAreDocumented.

Then the full gate:

export GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off TMPDIR=/tmp/g
go build ./... && go vet ./... && gofmt -l . && go test ./... -count=1
GOOS=darwin go build ./... && GOOS=windows go build ./... && GOARCH=arm64 go build ./...

Expected: green.

• [ ] Step 6: Commit

git add internal/host/units internal/host/bootstrap.go internal/host/systemunits.go internal/host/units_isolation_test.go main.go docs/site/cli.md
git commit -m "units and verbs: the instance templates, the probe, the control socket and service, and their docs (plan 2, task 6)"

---

Self-review

Spec coverage for this plan's scope. §4.1, §4.2 templates: Task 6. §4 escaping and %I/%i: Tasks 1 and 6. §5.2 units: Task 6. §5.3 verbs and sequences, no reset-failed in /start, /seed with RemainAfterExit, /sweep by reported names, /ready: Task 5. §5.4 peer check by SO_PEERCRED uid and MainPID, no token: Task 5. §6.2 trampoline and its checks: Task 2. §6.5 exit record, O_EXCL, advisory: Task 3. §6.5 cgroup emptiness as the termination rule: Task 5's WaitCgroupEmpty (the agent-side observer is plan 3). §8.5 probe lines and rules, fixture verification, re-run every five minutes: Tasks 4 and 5. §9 same-uid audit test, enabledUnits, ensureSystemUnits active check, docs/site/cli.md: Task 6. Deferred by design to plan 3: everything in the agent (prepare, launch-file writing, log listeners, unitProc, the exit observer, the client for this channel, /sweep at agent start). Deferred to plan 4: the layout verb, status field, downgrade guards. update-platform-binary.sh (pinfra) must enable carlos-instance-control.socket by name; noted for the roll.

Placeholder scan. Task 6's seed and sidecar files are described as deltas from the run file with every differing line spelled out; the block-move instruction is explicit. No "TBD".

Type consistency. instancectl.Role, RoleRun/Seed/Sidecar, ParseRole, Escape, UnitName, SliceName, ParseUnitName, LaunchFileName, ExitRecordName, Launch, ExitRecord, ProbeDirName, VerdictLine, ProbeReport, StartResponse, SeedResponse, SweepResponse are used with the same spelling in Tasks 2 to 6. probeEnv fields are used identically in Task 4's test and code. Server fields match between Task 5's tests and code (cgroupWaitBudget is unexported and set directly by the tests in the same package).