amadan

carlos / platform Public

plan: carlos system update, task by task

Sixteen tasks from the pointer to the docs, each with its tests, in
dependency order. Names real helpers found in the code: signedWrapper,
systemctlPlan, fastDeps, cliRoute, the console fixture.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015QCUCpfxbCyEAVQ2m51dH5
Paul Campbell pushed by paul@keymail.dev fc904cf04ff3f3850a4f0dd0e9efca2e7f60af62 parent a49b162
1 file changed, +5934 −0
  • docs/superpowers/plans/2026-09-06-carlos-system-update.md +5934 −0

CI passed — run details

diff --git a/docs/superpowers/plans/2026-09-06-carlos-system-update.md b/docs/superpowers/plans/2026-09-06-carlos-system-update.md
new file mode 100644
index 000000000..f904116b7
--- /dev/null
+++ b/docs/superpowers/plans/2026-09-06-carlos-system-update.md
@@ -0,0 +1,5934 @@
+# `carlos system update` Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** A fleet operator runs `carlos system update` and every box on the channel fetches, verifies, installs and health-gates the new platform binary itself, rolling back to the previous one if it does not come up.
+
+**Architecture:** A signed `SystemPointer` at `control/system/router/<channel>.json` names a version and its per-platform artifacts mirrored into the deployment bucket. A sandboxed agent leg notices the pointer on the serial poll, streams and verifies the artifact into `/data`, and touches a marker. A root oneshot (`carlos system apply`) re-verifies everything from root-owned copies, journals each step, swaps binary and units, restarts the edge, gates it, and rolls back to `carlos.prev` on failure; a boot-time unit recovers an interrupted apply. The console does the promote: fetch the public signed release, mirror, sign the pointer, bump the serial.
+
+**Tech Stack:** Go (stdlib, ed25519, net/http), systemd units embedded in the binary, S3/dir/fleet-door stores behind `release.Store`.
+
+**Spec:** `docs/superpowers/specs/2026-09-06-carlos-system-update-design.md` — read §3, §4, §6 before the task you are on. The spec wins where this plan and it disagree; say so in the commit if you find such a place.
+
+## Global Constraints
+
+- Go build cache is read-only in the sandbox: run every `go` command with `dangerouslyDisableSandbox: true`, and `export TMPDIR=/tmp/c` first (long socket paths break `internal/edge`/`internal/activator` tests).
+- Capture test exit codes explicitly: `out=$(go test ./pkg/... 2>&1); rc=$?; echo "$out" | tail -40; echo rc=$rc`. Never pipe `go test` into `grep`/`tail` and read the pipeline's status.
+- `-count=1` on every test run. `gofmt -l .` must print nothing before a commit.
+- Never run full `carlos bootstrap` on a live box path; the apply stages units into a temp root (`bootstrap -root <dir>`) and copies only `etc/`. Never copy a staging root's `data/`.
+- sysusers before tmpfiles, always, in every code path that installs units.
+- Root reads exactly one file under `/data` (`requested.json`), walks to it with `O_DIRECTORY|O_NOFOLLOW` per component, and derives every other path from a validated version string. Root never deletes under `/data`.
+- Apply never `systemctl start`s a `.service`; it calls `systemctlPlan`, which starts only `.timer`/`.path` units.
+- Every root state file is written temp → fsync → rename → directory fsync (`writeFileDurable`, Task 4).
+- The component name is `router`; channels are `canary` and `stable`. Platform strings are `<goos>-<goarch>` matching `^[a-z0-9]+-[a-z0-9]+$`.
+- Commit after every task with the attribution footer:
+ ```
+ Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
+ Claude-Session: https://claude.ai/code/session_015QCUCpfxbCyEAVQ2m51dH5
+ ```
+- Work on branch `carlos-system-update` from forge `main`; land with `amadan branch merge`, never `gh`.
+
+---
+
+## File Structure
+
+| File | Responsibility |
+|---|---|
+| `internal/release/system.go` (+`_test`) | `SystemPointer`, `SystemRelease`, `SystemArtifact`; key builders; validate/sign/verify (root key only); `GetSystemPointer`, `PromoteSystem`, `RollbackSystem` |
+| `internal/release/store.go`, `internal/fleetapi/fleetapi.go` (+tests) | `Opener` side-interface, `Open` on `DirStore`/`S3Store`/`prefixStore`/fleetapi, `OpenObject`, `FetchToFile` |
+| `internal/edge/storedoor.go` (+`_test`) | `aclRead` grants for the two new key shapes |
+| `internal/host/systemstate.go` (+`_test`) | On-box state types (request, result, seen, applied, journal), paths, `writeFileDurable`, outcome constants |
+| `internal/status/snapshot.go`, `internal/host/agent.go` | `Platform`, `SystemChannel`, `System` in `Probes`/`BoxMetrics`; filled in `statusLoop` |
+| `internal/host/units/carlos-system-update.path`, `carlos-system-update.service`, `carlos-system-recover.service`; `internal/host/bootstrap.go`; `internal/host/probes.go` | The three units, `enabledUnits`, `-plan-json`, the path-unit probe |
+| `internal/host/systemleg.go` (+`_test`) | `systemLeg`: `observe`/`tick`, fetch, request, marker re-touch; wired into `fastDeps` |
+| `internal/host/systemapply.go` (+`_test`) | `RunSystemApply`, `systemApplier` with an injectable `sysOps` seam: refuse path, install path, gate, rollback, recovery, lock |
+| `internal/host/agent.go` `RunAdopt` | The hourly belt: install and arm the three units if missing |
+| `internal/publicrelease/feed.go` (+`_test`) | `Feed`: latest tag, verified asset fetch — factored out of `update.go`, shared with the console |
+| `internal/console/apisystem.go` (+`_test`) | `/api/cli/system/{status,update,upload,jobs/{id},rollback}`; ops-owner gate; in-memory update job |
+| `system.go` (+`_test`, package main), `main.go` | `carlos system update|status|rollback|apply`, `carlos system router …` alias |
+| `docs/site/cli.md`, `AGENTS.md` | Operator docs; the env contract |
+
+Task order is dependency order. Tasks 1–3 are independent of each other and can run in parallel; 4 before 5/7/8; 6 before 8; 10 before 11; 11 before 12.
+
+---
+
+### Task 1: The system pointer (`internal/release/system.go`)
+
+**Files:**
+- Create: `internal/release/system.go`
+- Test: `internal/release/system_test.go`
+
+**Interfaces:**
+- Consumes: `signedWrapper`, `Signer` (fields `key`, `scope`, method `usable()`), `Promotion`, `historyMax`, `auditMax`, `prependCapped`, `ValidChannel`, `ValidVersion`, `ParsePublicKey`, `Store`, `ErrCASConflict` — all in package `release` (`release.go`, `signer.go`, `store.go`).
+- Produces (all exported, package `release`):
+ ```go
+ const SystemComponentRouter = "router"
+ func SystemPointerKey(component, channel string) string // control/system/<c>/<ch>.json
+ func SystemArtifactKey(version, platform string) string // bootstrap/router/<v>/carlos-<platform>
+ type SystemArtifact struct{ Key, SHA256 string }
+ type SystemRelease struct{ Version string; Artifacts map[string]SystemArtifact; Source string; ReleasedAt time.Time }
+ type SystemPointer struct{ Schema int; Component, Channel string; Current SystemRelease; History []SystemRelease; Spread string; PromotedAt time.Time; Audit []Promotion }
+ func (p *SystemPointer) SpreadDuration() time.Duration
+ func GetSystemPointer(ctx context.Context, s Store, pub ed25519.PublicKey, component, channel string) (*SystemPointer, string, error)
+ type SystemPromoteOptions struct{ Actor string; Spread time.Duration }
+ func PromoteSystem(ctx context.Context, s Store, sg Signer, component, channel string, rel SystemRelease, o SystemPromoteOptions) (*SystemPointer, error)
+ func RollbackSystem(ctx context.Context, s Store, sg Signer, component, channel, actor string) (*SystemPointer, error)
+ var ErrNoSystemHistory = errors.New("system pointer: nothing to roll back to")
+ ```
+
+- [ ] **Step 1: Write the failing tests**
+
+```go
+package release
+
+import (
+ "context"
+ "encoding/json"
+ "errors"
+ "os"
+ "strings"
+ "testing"
+ "time"
+)
+
+func testSystemRelease(v string) SystemRelease {
+ return SystemRelease{
+ Version: v,
+ Artifacts: map[string]SystemArtifact{
+ "linux-arm64": {Key: SystemArtifactKey(v, "linux-arm64"), SHA256: strings.Repeat("ab", 32)},
+ },
+ Source: "release:" + v,
+ ReleasedAt: time.Date(2026, 9, 6, 8, 0, 0, 0, time.UTC),
+ }
+}
+
+func TestSystemPointerRoundTrip(t *testing.T) {
+ priv, pub := testKeys(t)
+ s := NewDirStore(t.TempDir())
+ ctx := context.Background()
+ if _, _, err := GetSystemPointer(ctx, s, pub, SystemComponentRouter, "stable"); !errors.Is(err, os.ErrNotExist) {
+ t.Fatalf("missing pointer: err = %v, want ErrNotExist", err)
+ }
+ p, err := PromoteSystem(ctx, s, RootSigner(priv), SystemComponentRouter, "stable", testSystemRelease("v0.18.0"),
+ SystemPromoteOptions{Actor: "paul", Spread: 5 * time.Minute})
+ if err != nil {
+ t.Fatal(err)
+ }
+ if p.Current.Version != "v0.18.0" || p.Spread != "5m0s" || p.Schema != 1 || p.PromotedAt.IsZero() {
+ t.Fatalf("promoted pointer = %+v", p)
+ }
+ got, etag, err := GetSystemPointer(ctx, s, pub, SystemComponentRouter, "stable")
+ if err != nil || etag == "" {
+ t.Fatalf("get: %v etag=%q", err, etag)
+ }
+ if got.Current.Artifacts["linux-arm64"].SHA256 != strings.Repeat("ab", 32) {
+ t.Fatalf("artifact lost: %+v", got.Current)
+ }
+ if got.SpreadDuration() != 5*time.Minute {
+ t.Fatalf("spread = %v", got.SpreadDuration())
+ }
+ if len(got.Audit) != 1 || got.Audit[0].Actor != "paul" || got.Audit[0].Version != "v0.18.0" {
+ t.Fatalf("audit = %+v", got.Audit)
+ }
+}
+
+func TestSystemPromoteKeepsWholeReleaseInHistoryAndRollbackRestoresIt(t *testing.T) {
+ priv, pub := testKeys(t)
+ s := NewDirStore(t.TempDir())
+ ctx := context.Background()
+ sg := RootSigner(priv)
+ if _, err := PromoteSystem(ctx, s, sg, "router", "canary", testSystemRelease("v0.18.0"), SystemPromoteOptions{}); err != nil {
+ t.Fatal(err)
+ }
+ first, _, _ := GetSystemPointer(ctx, s, pub, "router", "canary")
+ time.Sleep(1100 * time.Millisecond) // PromotedAt is second-truncated; the rollback must be NEWER
+ if _, err := PromoteSystem(ctx, s, sg, "router", "canary", testSystemRelease("v0.19.0"), SystemPromoteOptions{}); err != nil {
+ t.Fatal(err)
+ }
+ p, _, _ := GetSystemPointer(ctx, s, pub, "router", "canary")
+ if len(p.History) != 1 || p.History[0].Version != "v0.18.0" || p.History[0].Artifacts["linux-arm64"].Key == "" {
+ t.Fatalf("history must carry the whole release: %+v", p.History)
+ }
+ time.Sleep(1100 * time.Millisecond)
+ rb, err := RollbackSystem(ctx, s, sg, "router", "canary", "paul")
+ if err != nil {
+ t.Fatal(err)
+ }
+ if rb.Current.Version != "v0.18.0" || rb.Current.Artifacts["linux-arm64"].Key != SystemArtifactKey("v0.18.0", "linux-arm64") {
+ t.Fatalf("rollback current = %+v", rb.Current)
+ }
+ if !rb.PromotedAt.After(first.PromotedAt) || !rb.PromotedAt.After(p.PromotedAt) {
+ t.Fatalf("a rollback is a NEW promote: promoted_at %v must be newer than %v and %v", rb.PromotedAt, first.PromotedAt, p.PromotedAt)
+ }
+ if len(rb.History) != 0 {
+ t.Fatalf("history after rollback = %+v", rb.History)
+ }
+ if _, err := RollbackSystem(ctx, s, sg, "router", "canary", "paul"); !errors.Is(err, ErrNoSystemHistory) {
+ t.Fatalf("second rollback err = %v", err)
+ }
+}
+
+func TestSystemPromoteSameVersionDoesNotGrowHistory(t *testing.T) {
+ priv, pub := testKeys(t)
+ s := NewDirStore(t.TempDir())
+ ctx := context.Background()
+ sg := RootSigner(priv)
+ for i := 0; i < 2; i++ {
+ if _, err := PromoteSystem(ctx, s, sg, "router", "stable", testSystemRelease("v1.0.0"), SystemPromoteOptions{}); err != nil {
+ t.Fatal(err)
+ }
+ }
+ p, _, _ := GetSystemPointer(ctx, s, pub, "router", "stable")
+ if len(p.History) != 0 {
+ t.Fatalf("re-promoting the running version must not push it into history: %+v", p.History)
+ }
+}
+
+func TestSystemPointerRefusesScopedSigner(t *testing.T) {
+ _, _, sg := scopedFixture(t, []string{RungCanary, RungEdge}, 30*24*time.Hour)
+ s := NewDirStore(t.TempDir())
+ _, err := PromoteSystem(context.Background(), s, sg, "router", "canary", testSystemRelease("v1.0.0"), SystemPromoteOptions{})
+ if err == nil || !strings.Contains(err.Error(), "root key") {
+ t.Fatalf("scoped signer signed a system pointer: err = %v", err)
+ }
+}
+
+func TestSystemPointerRefusesAGrantOnTheWire(t *testing.T) {
+ // A wrapper carrying a grant is refused BEFORE the signature is checked,
+ // like the production flag (production.go) and the pipeline.
+ priv, pub := testKeys(t)
+ s := NewDirStore(t.TempDir())
+ ctx := context.Background()
+ if _, err := PromoteSystem(ctx, s, RootSigner(priv), "router", "stable", testSystemRelease("v1.0.0"), SystemPromoteOptions{}); err != nil {
+ t.Fatal(err)
+ }
+ body, etag, _ := s.Get(ctx, SystemPointerKey("router", "stable"))
+ var sw signedWrapper
+ if err := json.Unmarshal(body, &sw); err != nil {
+ t.Fatal(err)
+ }
+ sw.Grant = &signedGrant{Payload: "x", Sig: "y"}
+ tampered, _ := json.Marshal(sw)
+ if _, err := s.Put(ctx, SystemPointerKey("router", "stable"), tampered, etag); err != nil {
+ t.Fatal(err)
+ }
+ if _, _, err := GetSystemPointer(ctx, s, pub, "router", "stable"); err == nil || !strings.Contains(err.Error(), "root key") {
+ t.Fatalf("grant-bearing wrapper accepted: err = %v", err)
+ }
+}
+
+func TestSystemPointerTamperAndCrossChannelRefused(t *testing.T) {
+ priv, pub := testKeys(t)
+ s := NewDirStore(t.TempDir())
+ ctx := context.Background()
+ if _, err := PromoteSystem(ctx, s, RootSigner(priv), "router", "canary", testSystemRelease("v1.0.0"), SystemPromoteOptions{}); err != nil {
+ t.Fatal(err)
+ }
+ body, _, _ := s.Get(ctx, SystemPointerKey("router", "canary"))
+ // Cross-channel replay: the canary wrapper planted at the stable key.
+ if _, err := s.Put(ctx, SystemPointerKey("router", "stable"), body, ""); err != nil {
+ t.Fatal(err)
+ }
+ if _, _, err := GetSystemPointer(ctx, s, pub, "router", "stable"); err == nil {
+ t.Fatal("a canary pointer verified as stable")
+ }
+ // Tampered signature.
+ var sw signedWrapper
+ _ = json.Unmarshal(body, &sw)
+ sw.Sig = "AA" + sw.Sig[2:]
+ bad, _ := json.Marshal(sw)
+ _, etag, _ := s.Get(ctx, SystemPointerKey("router", "canary"))
+ if _, err := s.Put(ctx, SystemPointerKey("router", "canary"), bad, etag); err != nil {
+ t.Fatal(err)
+ }
+ if _, _, err := GetSystemPointer(ctx, s, pub, "router", "canary"); err == nil {
+ t.Fatal("tampered signature verified")
+ }
+}
+
+func TestSystemPointerValidation(t *testing.T) {
+ priv, _ := testKeys(t)
+ sg := RootSigner(priv)
+ ctx := context.Background()
+ cases := map[string]SystemRelease{
+ "dev version": {Version: "dev", Artifacts: testSystemRelease("x").Artifacts},
+ "no artifacts": {Version: "v1.0.0"},
+ "bad platform": {Version: "v1.0.0", Artifacts: map[string]SystemArtifact{"linux_arm64": {Key: "bootstrap/router/v1.0.0/carlos-linux_arm64", SHA256: strings.Repeat("a", 64)}}},
+ "key not derived": {Version: "v1.0.0", Artifacts: map[string]SystemArtifact{"linux-arm64": {Key: "bootstrap/carlos", SHA256: strings.Repeat("a", 64)}}},
+ "short sha": {Version: "v1.0.0", Artifacts: map[string]SystemArtifact{"linux-arm64": {Key: SystemArtifactKey("v1.0.0", "linux-arm64"), SHA256: "abc"}}},
+ "version traversal": {Version: "../x", Artifacts: testSystemRelease("../x").Artifacts},
+ }
+ for name, rel := range cases {
+ s := NewDirStore(t.TempDir())
+ if _, err := PromoteSystem(ctx, s, sg, "router", "stable", rel, SystemPromoteOptions{}); err == nil {
+ t.Errorf("%s: accepted", name)
+ }
+ }
+ s := NewDirStore(t.TempDir())
+ if _, err := PromoteSystem(ctx, s, sg, "Router", "stable", testSystemRelease("v1.0.0"), SystemPromoteOptions{}); err == nil {
+ t.Error("bad component accepted")
+ }
+ if _, err := PromoteSystem(ctx, s, sg, "router", "stable", testSystemRelease("v1.0.0"), SystemPromoteOptions{Spread: 48 * time.Hour}); err == nil {
+ t.Error("48h spread accepted")
+ }
+}
+
+func TestSystemPromoteCASConflict(t *testing.T) {
+ priv, _ := testKeys(t)
+ ctx := context.Background()
+ inner := NewDirStore(t.TempDir())
+ // A store whose Put always says "someone else moved it".
+ s := conflictingStore{Store: inner}
+ if _, err := PromoteSystem(ctx, s, RootSigner(priv), "router", "stable", testSystemRelease("v1.0.0"), SystemPromoteOptions{}); !errors.Is(err, ErrCASConflict) {
+ t.Fatalf("err = %v, want ErrCASConflict", err)
+ }
+}
+
+type conflictingStore struct{ Store }
+
+func (c conflictingStore) Put(ctx context.Context, key string, body []byte, ifMatch string) (string, error) {
+ return "", ErrCASConflict
+}
+```
+
+- [ ] **Step 2: Run the tests to verify they fail**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/release/ -run 'TestSystem' -count=1 2>&1); rc=$?; echo "$out" | tail -20; echo rc=$rc`
+Expected: build failure naming `SystemPointer`, `PromoteSystem`, etc. as undefined.
+
+- [ ] **Step 3: Implement `internal/release/system.go`**
+
+```go
+package release
+
+import (
+ "context"
+ "crypto/ed25519"
+ "encoding/base64"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "os"
+ "regexp"
+ "time"
+)
+
+// SystemPointer is the platform's own channel pointer: what a ChannelPointer
+// is for an app, for the thing that does the adopting. It lives under
+// control/, not apps/, because the substrate is not an app (the
+// console-as-an-app spec's dividing line), and it is ROOT-KEY-ONLY: a scoped
+// grant's rungs name app channels, and a key that can roll the platform is a
+// bigger key than any app promoter should hold. Spec §3.
+type SystemPointer struct {
+ Schema int `json:"schema"`
+ Component string `json:"component"`
+ Channel string `json:"channel"`
+ Current SystemRelease `json:"current"`
+ History []SystemRelease `json:"history,omitempty"`
+ Spread string `json:"spread"`
+ PromotedAt time.Time `json:"promoted_at"` // of THIS write; the box's replay guard reads only this
+ Audit []Promotion `json:"audit,omitempty"`
+}
+
+// SystemRelease is everything a box needs to install one version. It travels
+// whole into History so a rollback re-installs what was there, not a version
+// string with someone else's artifacts.
+type SystemRelease struct {
+ Version string `json:"version"`
+ Artifacts map[string]SystemArtifact `json:"artifacts"` // keyed by "<goos>-<goarch>"
+ Source string `json:"source,omitempty"`
+ ReleasedAt time.Time `json:"released_at"` // audit only; never compared
+}
+
+// SystemArtifact is one mirrored binary. Its sha in the signed pointer IS
+// the artifact's signature: boxes need no .sig object.
+type SystemArtifact struct {
+ Key string `json:"key"`
+ SHA256 string `json:"sha256"`
+}
+
+// SystemComponentRouter is the one component today: the substrate binary
+// (edge + activator + adoption, one process). A split adds a name here.
+const SystemComponentRouter = "router"
+
+// ErrNoSystemHistory is RollbackSystem's answer when the pointer has never
+// moved: there is nothing older to go back to.
+var ErrNoSystemHistory = errors.New("system pointer: nothing to roll back to")
+
+const (
+ systemSpreadMax = 24 * time.Hour
+ systemSpreadDefault = 5 * time.Minute
+)
+
+var (
+ systemComponentRe = regexp.MustCompile(`^[a-z][a-z0-9-]{0,31}$`)
+ systemPlatformRe = regexp.MustCompile(`^[a-z0-9]+-[a-z0-9]+$`)
+ systemSHA256Re = regexp.MustCompile(`^[0-9a-f]{64}$`)
+)
+
+// SystemPointerKey is control/system/<component>/<channel>.json.
+func SystemPointerKey(component, channel string) string {
+ return "control/system/" + component + "/" + channel + ".json"
+}
+
+// SystemArtifactKey is bootstrap/router/<version>/carlos-<platform>: under
+// bootstrap/ because the store door already means "the platform's own
+// binary" by that name, per-version because canary and stable boxes must be
+// able to run different binaries at once.
+func SystemArtifactKey(version, platform string) string {
+ return "bootstrap/router/" + version + "/carlos-" + platform
+}
+
+// SpreadDuration is Spread parsed, or the default when it is unparseable —
+// validation refuses the unparseable case on the way in, so this only
+// defaults for a hand-edited or pre-field pointer.
+func (p *SystemPointer) SpreadDuration() time.Duration {
+ d, err := time.ParseDuration(p.Spread)
+ if err != nil || d < 0 || d > systemSpreadMax {
+ return systemSpreadDefault
+ }
+ return d
+}
+
+func validateSystemRelease(r SystemRelease) error {
+ if !ValidVersion(r.Version) || r.Version == "dev" {
+ return fmt.Errorf("version %q is not a valid platform version", r.Version)
+ }
+ if len(r.Artifacts) == 0 {
+ return fmt.Errorf("version %s names no artifacts", r.Version)
+ }
+ for plat, a := range r.Artifacts {
+ if !systemPlatformRe.MatchString(plat) {
+ return fmt.Errorf("platform %q is not <goos>-<goarch>", plat)
+ }
+ if a.Key != SystemArtifactKey(r.Version, plat) {
+ return fmt.Errorf("artifact key %q is not %s", a.Key, SystemArtifactKey(r.Version, plat))
+ }
+ if !systemSHA256Re.MatchString(a.SHA256) {
+ return fmt.Errorf("artifact %s: sha256 %q is not 64 hex characters", a.Key, a.SHA256)
+ }
+ }
+ return nil
+}
+
+func validateSystemPointer(p *SystemPointer, wantComponent, wantChannel string) error {
+ if p.Schema != 1 {
+ return fmt.Errorf("system pointer schema %d unsupported", p.Schema)
+ }
+ if p.Component != wantComponent || p.Channel != wantChannel {
+ return fmt.Errorf("system pointer is for %s/%s, not %s/%s", p.Component, p.Channel, wantComponent, wantChannel)
+ }
+ if !systemComponentRe.MatchString(p.Component) || !ValidChannel(p.Channel) {
+ return fmt.Errorf("system pointer %s/%s: bad component or channel", p.Component, p.Channel)
+ }
+ if err := validateSystemRelease(p.Current); err != nil {
+ return fmt.Errorf("system pointer %s/%s: %w", p.Component, p.Channel, err)
+ }
+ if d, err := time.ParseDuration(p.Spread); err != nil || d < 0 || d > systemSpreadMax {
+ return fmt.Errorf("system pointer %s/%s: spread %q must be a duration between 0 and %s", p.Component, p.Channel, p.Spread, systemSpreadMax)
+ }
+ if p.PromotedAt.IsZero() {
+ return fmt.Errorf("system pointer %s/%s: promoted_at unset", p.Component, p.Channel)
+ }
+ if len(p.History) > historyMax {
+ return fmt.Errorf("system pointer %s/%s: history longer than %d", p.Component, p.Channel, historyMax)
+ }
+ for _, h := range p.History {
+ if err := validateSystemRelease(h); err != nil {
+ return fmt.Errorf("system pointer %s/%s: history: %w", p.Component, p.Channel, err)
+ }
+ }
+ if len(p.Audit) > auditMax {
+ return fmt.Errorf("system pointer %s/%s: audit longer than %d", p.Component, p.Channel, auditMax)
+ }
+ return nil
+}
+
+// signSystemPointer mirrors signPointer with two differences: the signer
+// must be the root key, and the wrapper never carries a grant.
+func signSystemPointer(sg Signer, p *SystemPointer) ([]byte, error) {
+ if err := sg.usable(); err != nil {
+ return nil, err
+ }
+ if sg.scope != nil {
+ return nil, errors.New("system pointer: signed by a scoped key — the platform takes the root key only")
+ }
+ p.Schema = 1
+ p.PromotedAt = time.Now().UTC().Truncate(time.Second)
+ if err := validateSystemPointer(p, p.Component, p.Channel); err != nil {
+ return nil, err
+ }
+ payload, err := json.MarshalIndent(p, "", " ")
+ if err != nil {
+ return nil, err
+ }
+ b64 := base64.StdEncoding.EncodeToString(payload)
+ sig := ed25519.Sign(sg.key, []byte(b64))
+ return json.MarshalIndent(signedWrapper{Payload: b64, Sig: base64.StdEncoding.EncodeToString(sig)}, "", " ")
+}
+
+// verifySystemPointer is the single parse site for system pointers. A grant
+// on the wire is refused before the signature is looked at (the production
+// flag's rule, production.go).
+func verifySystemPointer(pub ed25519.PublicKey, wrapper []byte, wantComponent, wantChannel string) (*SystemPointer, error) {
+ var sw signedWrapper
+ if err := json.Unmarshal(wrapper, &sw); err != nil {
+ return nil, fmt.Errorf("system pointer %s/%s: %w", wantComponent, wantChannel, err)
+ }
+ if sw.Grant != nil {
+ return nil, fmt.Errorf("system pointer %s/%s: signed by a scoped key — the platform takes the root key only", wantComponent, wantChannel)
+ }
+ sig, err := base64.StdEncoding.DecodeString(sw.Sig)
+ if err != nil {
+ return nil, fmt.Errorf("system pointer %s/%s: bad signature encoding", wantComponent, wantChannel)
+ }
+ if len(pub) != ed25519.PublicKeySize || !ed25519.Verify(pub, []byte(sw.Payload), sig) {
+ return nil, fmt.Errorf("system pointer %s/%s: signature does not verify", wantComponent, wantChannel)
+ }
+ payload, err := base64.StdEncoding.DecodeString(sw.Payload)
+ if err != nil {
+ return nil, fmt.Errorf("system pointer %s/%s: bad payload encoding", wantComponent, wantChannel)
+ }
+ var p SystemPointer
+ if err := json.Unmarshal(payload, &p); err != nil {
+ return nil, fmt.Errorf("system pointer %s/%s: %w", wantComponent, wantChannel, err)
+ }
+ if err := validateSystemPointer(&p, wantComponent, wantChannel); err != nil {
+ return nil, err
+ }
+ return &p, nil
+}
+
+// GetSystemPointer reads and verifies one channel's pointer. A missing key
+// is os.ErrNotExist, which is the normal state of every deployment before
+// its first `carlos system update`.
+func GetSystemPointer(ctx context.Context, s Store, pub ed25519.PublicKey, component, channel string) (*SystemPointer, string, error) {
+ body, etag, err := s.Get(ctx, SystemPointerKey(component, channel))
+ if err != nil {
+ return nil, "", err
+ }
+ p, err := verifySystemPointer(pub, body, component, channel)
+ if err != nil {
+ return nil, "", err
+ }
+ return p, etag, nil
+}
+
+// SystemPromoteOptions: who did it, and how long boxes spread their starts.
+type SystemPromoteOptions struct {
+ Actor string
+ Spread time.Duration // 0 = hotfix, every box at once; unset in the CLI means the default
+}
+
+// PromoteSystem writes the pointer for one channel, CAS-guarded. The
+// outgoing release goes whole into History unless it is the same version
+// (a re-promote of the running version is a no-op for boxes and must not
+// push that version into its own history).
+func PromoteSystem(ctx context.Context, s Store, sg Signer, component, channel string, rel SystemRelease, o SystemPromoteOptions) (*SystemPointer, error) {
+ key := SystemPointerKey(component, channel)
+ var old *SystemPointer
+ body, etag, err := s.Get(ctx, key)
+ switch {
+ case err == nil:
+ if old, err = verifySystemPointer(sg.verifyPub(), body, component, channel); err != nil {
+ return nil, err
+ }
+ case errors.Is(err, os.ErrNotExist):
+ etag = ""
+ default:
+ return nil, err
+ }
+ next := &SystemPointer{Component: component, Channel: channel, Current: rel, Spread: o.Spread.String()}
+ if rel.ReleasedAt.IsZero() {
+ next.Current.ReleasedAt = time.Now().UTC().Truncate(time.Second)
+ }
+ from := ""
+ if old != nil {
+ next.History, next.Audit = old.History, old.Audit
+ from = old.Current.Version
+ if old.Current.Version != rel.Version {
+ next.History = prependCapped(old.Current, old.History, historyMax)
+ }
+ }
+ next.Audit = prependCapped(Promotion{Version: rel.Version, From: from, To: channel, Actor: o.Actor, At: time.Now().UTC()}, next.Audit, auditMax)
+ wrapper, err := signSystemPointer(sg, next)
+ if err != nil {
+ return nil, err
+ }
+ if _, err := s.Put(ctx, key, wrapper, etag); err != nil {
+ return nil, err
+ }
+ return next, nil
+}
+
+// RollbackSystem swaps Current with History[0] and writes a NEW pointer —
+// new PromotedAt — so every box's replay guard sees a promote, not a replay.
+func RollbackSystem(ctx context.Context, s Store, sg Signer, component, channel, actor string) (*SystemPointer, error) {
+ key := SystemPointerKey(component, channel)
+ body, etag, err := s.Get(ctx, key)
+ if err != nil {
+ return nil, err
+ }
+ old, err := verifySystemPointer(sg.verifyPub(), body, component, channel)
+ if err != nil {
+ return nil, err
+ }
+ if len(old.History) == 0 {
+ return nil, ErrNoSystemHistory
+ }
+ next := &SystemPointer{
+ Component: component, Channel: channel,
+ Current: old.History[0], History: old.History[1:], Spread: old.Spread,
+ Audit: prependCapped(Promotion{Version: old.History[0].Version, From: old.Current.Version, To: channel, Actor: actor, At: time.Now().UTC()}, old.Audit, auditMax),
+ }
+ wrapper, err := signSystemPointer(sg, next)
+ if err != nil {
+ return nil, err
+ }
+ if _, err := s.Put(ctx, key, wrapper, etag); err != nil {
+ return nil, err
+ }
+ return next, nil
+}
+```
+
+Note `sg.verifyPub()` is the existing unexported method on `Signer` (`signer.go:66`); for a root signer it returns the root public key.
+
+- [ ] **Step 4: Run the tests**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/release/ -count=1 2>&1); rc=$?; echo "$out" | tail -20; echo rc=$rc`
+Expected: `ok`, rc=0. If `TestSystemPromoteCASConflict` fails because `conflictingStore` lacks a method, the `Store` interface grew — embed satisfies it; check the error text.
+
+- [ ] **Step 5: gofmt and commit**
+
+```bash
+gofmt -l internal/release/ && git add internal/release/system.go internal/release/system_test.go && git commit -q -F - <<'EOF'
+release: the platform's own channel pointer, root-key-only
+
+control/system/<component>/<channel>.json names a version and its
+per-platform artifacts. History carries whole releases so a rollback
+restores artifacts, and a rollback is a new promote (new PromotedAt).
+A grant on the wire is refused before the signature is read, like the
+production flag.
+
+Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
+Claude-Session: https://claude.ai/code/session_015QCUCpfxbCyEAVQ2m51dH5
+EOF
+```
+
+---
+
+### Task 2: Streaming reads (`Opener`, `FetchToFile`)
+
+**Files:**
+- Modify: `internal/release/store.go` (after the `SizeLister` block, ~line 190)
+- Modify: `internal/fleetapi/fleetapi.go` (after `Get`, ~line 156)
+- Create: `internal/release/fetch.go`
+- Test: `internal/release/fetch_test.go`, `internal/fleetapi/fleetapi_test.go` (add one test)
+
+**Interfaces:**
+- Consumes: `Store`, `DirStore.path`, `S3Store.Client/Bucket`, `s3NotFound`, `prefixStore.full`; fleetapi `store.newRequest`, `objectURL`.
+- Produces:
+ ```go
+ // package release
+ type Opener interface{ Open(ctx context.Context, key string) (io.ReadCloser, string, error) }
+ func OpenObject(ctx context.Context, s Store, key string) (io.ReadCloser, string, error)
+ var ErrTooLarge = errors.New("store: object larger than the caller's limit")
+ func FetchToFile(ctx context.Context, s Store, key, dst string, limit int64, wantSHA256 string) error
+ ```
+ `FetchToFile` writes `dst+".tmp"`, streams with a running sha, refuses more than `limit` bytes (`ErrTooLarge`), refuses a sha mismatch (error contains "sha256"), fsyncs, renames, fsyncs the directory, and removes the temp on any failure. Etag from `Open` may be `""` (DirStore).
+
+- [ ] **Step 1: Write the failing tests**
+
+`internal/release/fetch_test.go`:
+```go
+package release
+
+import (
+ "context"
+ "crypto/sha256"
+ "encoding/hex"
+ "errors"
+ "os"
+ "path/filepath"
+ "strings"
+ "testing"
+)
+
+func shaOf(b []byte) string { s := sha256.Sum256(b); return hex.EncodeToString(s[:]) }
+
+func TestFetchToFileStreamsAndVerifies(t *testing.T) {
+ ctx := context.Background()
+ body := []byte(strings.Repeat("carlos", 10000))
+ for name, s := range map[string]Store{
+ "dir": NewDirStore(t.TempDir()),
+ "prefix": NewPrefixStore(NewDirStore(t.TempDir()), "fleets/pi/"),
+ } {
+ t.Run(name, func(t *testing.T) {
+ if _, err := s.Put(ctx, "bootstrap/router/v1/carlos-linux-arm64", body, ""); err != nil {
+ t.Fatal(err)
+ }
+ dst := filepath.Join(t.TempDir(), "carlos")
+ if err := FetchToFile(ctx, s, "bootstrap/router/v1/carlos-linux-arm64", dst, int64(len(body)), shaOf(body)); err != nil {
+ t.Fatal(err)
+ }
+ got, _ := os.ReadFile(dst)
+ if string(got) != string(body) {
+ t.Fatal("content differs")
+ }
+ if _, err := os.Stat(dst + ".tmp"); !errors.Is(err, os.ErrNotExist) {
+ t.Fatal("temp file left behind")
+ }
+ // One byte over the limit is refused, and nothing is left on disk.
+ dst2 := filepath.Join(t.TempDir(), "carlos")
+ if err := FetchToFile(ctx, s, "bootstrap/router/v1/carlos-linux-arm64", dst2, int64(len(body))-1, shaOf(body)); !errors.Is(err, ErrTooLarge) {
+ t.Fatalf("over-limit err = %v", err)
+ }
+ if _, err := os.Stat(dst2); !errors.Is(err, os.ErrNotExist) {
+ t.Fatal("over-limit fetch left a file")
+ }
+ // A sha mismatch is refused and leaves nothing.
+ dst3 := filepath.Join(t.TempDir(), "carlos")
+ err := FetchToFile(ctx, s, "bootstrap/router/v1/carlos-linux-arm64", dst3, int64(len(body)), strings.Repeat("0", 64))
+ if err == nil || !strings.Contains(err.Error(), "sha256") {
+ t.Fatalf("mismatch err = %v", err)
+ }
+ if _, err := os.Stat(dst3); !errors.Is(err, os.ErrNotExist) {
+ t.Fatal("mismatch fetch left a file")
+ }
+ // Missing key is ErrNotExist.
+ if err := FetchToFile(ctx, s, "bootstrap/router/v9/carlos-linux-arm64", dst3, 10, shaOf(nil)); !errors.Is(err, os.ErrNotExist) {
+ t.Fatalf("missing err = %v", err)
+ }
+ })
+ }
+}
+
+func TestPrefixStoreOpenRebasesTheKey(t *testing.T) {
+ // prefixStore embeds the Store INTERFACE, so a promoted Open would forward
+ // the raw key and silently escape the prefix. It must be written out.
+ ctx := context.Background()
+ inner := NewDirStore(t.TempDir())
+ if _, err := inner.Put(ctx, "fleets/pi/x", []byte("in"), ""); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := inner.Put(ctx, "x", []byte("OUT"), ""); err != nil {
+ t.Fatal(err)
+ }
+ rc, _, err := OpenObject(ctx, NewPrefixStore(inner, "fleets/pi/"), "x")
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer rc.Close()
+ b := make([]byte, 8)
+ n, _ := rc.Read(b)
+ if string(b[:n]) != "in" {
+ t.Fatalf("read %q through the prefix, want \"in\"", b[:n])
+ }
+}
+
+func TestOpenObjectFallsBackToGet(t *testing.T) {
+ // A Store that is not an Opener still works: OpenObject buffers via Get.
+ s := getOnly{NewDirStore(t.TempDir())}
+ if _, err := s.Put(context.Background(), "k", []byte("v"), ""); err != nil {
+ t.Fatal(err)
+ }
+ rc, _, err := OpenObject(context.Background(), s, "k")
+ if err != nil {
+ t.Fatal(err)
+ }
+ b := make([]byte, 4)
+ n, _ := rc.Read(b)
+ if string(b[:n]) != "v" {
+ t.Fatalf("got %q", b[:n])
+ }
+}
+
+type getOnly struct{ *DirStore }
+```
+(`getOnly` embeds `*DirStore`, so it has `Open` promoted — to make it a non-Opener, shadow it:)
+```go
+func (getOnly) Open() {} // different signature: getOnly is NOT an Opener
+```
+
+`internal/fleetapi/fleetapi_test.go`, add:
+```go
+func TestOpenStreamsThroughTheDoor(t *testing.T) {
+ d := newTestDoor(t)
+ body := bytes.Repeat([]byte("x"), 1<<20)
+ d.seed(t, "bootstrap/carlos", body)
+ c := d.box(t)
+ op, ok := c.(release.Opener)
+ if !ok {
+ t.Fatal("fleetapi store is not an Opener")
+ }
+ rc, _, err := op.Open(context.Background(), "bootstrap/carlos")
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer rc.Close()
+ got, _ := io.ReadAll(rc)
+ if !bytes.Equal(got, body) {
+ t.Fatalf("read %d bytes, want %d", len(got), len(body))
+ }
+ if _, _, err := op.Open(context.Background(), "bootstrap/nothing"); !errors.Is(err, os.ErrNotExist) {
+ t.Fatalf("missing: %v", err)
+ }
+}
+```
+(`newTestDoor`, `seed`, `box` exist in that file, `fleetapi_test.go:52-105`; `bootstrap/carlos` is already readable by a box.)
+
+- [ ] **Step 2: Run to verify failure**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/release/ ./internal/fleetapi/ -run 'Fetch|Open' -count=1 2>&1); rc=$?; echo "$out" | tail -20; echo rc=$rc`
+Expected: undefined `Opener`, `OpenObject`, `FetchToFile`, `ErrTooLarge`.
+
+- [ ] **Step 3: Implement**
+
+`internal/release/store.go`, after the `SizeLister` assertions (~line 190):
+```go
+// Opener is the optional streaming read. Store.Get returns the whole object
+// in memory, which is fine for a pointer and wrong for a platform binary on
+// a box under MemoryHigh=70%. Implemented by every production store; a test
+// fake that only has Get still works through OpenObject. The etag may be ""
+// (DirStore cannot know it without reading the file).
+type Opener interface {
+ Open(ctx context.Context, key string) (io.ReadCloser, string, error)
+}
+
+var (
+ _ Opener = (*DirStore)(nil)
+ _ Opener = (*S3Store)(nil)
+ _ Opener = prefixStore{}
+)
+
+// OpenObject streams key from s, buffering through Get when s cannot stream.
+func OpenObject(ctx context.Context, s Store, key string) (io.ReadCloser, string, error) {
+ if o, ok := s.(Opener); ok {
+ return o.Open(ctx, key)
+ }
+ b, etag, err := s.Get(ctx, key)
+ if err != nil {
+ return nil, "", err
+ }
+ return io.NopCloser(bytes.NewReader(b)), etag, nil
+}
+```
+`DirStore`:
+```go
+func (d *DirStore) Open(_ context.Context, key string) (io.ReadCloser, string, error) {
+ p, err := d.path(key)
+ if err != nil {
+ return nil, "", err
+ }
+ f, err := os.Open(p)
+ if err != nil {
+ if errors.Is(err, os.ErrNotExist) {
+ return nil, "", fmt.Errorf("dirstore: %s: %w", key, os.ErrNotExist)
+ }
+ return nil, "", err
+ }
+ return f, "", nil
+}
+```
+`S3Store`:
+```go
+func (s *S3Store) Open(ctx context.Context, key string) (io.ReadCloser, string, error) {
+ out, err := s.Client.GetObject(ctx, &awss3.GetObjectInput{Bucket: &s.Bucket, Key: &key})
+ if err != nil {
+ if s3NotFound(err) {
+ return nil, "", fmt.Errorf("s3store: %s: %w", key, os.ErrNotExist)
+ }
+ return nil, "", err
+ }
+ return out.Body, aws.ToString(out.ETag), nil
+}
+```
+`prefixStore` (explicit, next to its `Get`):
+```go
+// Open is written out, not promoted: the embedded field is the interface,
+// and a promoted Open would forward the UNREBASED key past the prefix.
+func (p prefixStore) Open(ctx context.Context, key string) (io.ReadCloser, string, error) {
+ return OpenObject(ctx, p.Store, p.full(key))
+}
+```
+Add `"bytes"` and `"io"` to the imports if absent.
+
+`internal/fleetapi/fleetapi.go`, after `Get`:
+```go
+// Open streams one object. Unlike Get it does not buffer the body, which
+// is what a 25 MiB platform binary on a Pi needs; the 404 mapping is the
+// same as Get's so callers can errors.Is(err, os.ErrNotExist).
+func (s *store) Open(ctx context.Context, key string) (io.ReadCloser, string, error) {
+ req, err := s.newRequest(ctx, http.MethodGet, s.objectURL(key), nil)
+ if err != nil {
+ return nil, "", err
+ }
+ resp, err := s.client.Do(req)
+ if err != nil {
+ return nil, "", errorf("open %s: %w", key, err)
+ }
+ switch resp.StatusCode {
+ case http.StatusOK:
+ return resp.Body, resp.Header.Get("ETag"), nil
+ case http.StatusNotFound:
+ resp.Body.Close()
+ return nil, "", fmt.Errorf("fleetapi: %s: %w", key, os.ErrNotExist)
+ default:
+ resp.Body.Close()
+ return nil, "", errorf("open %s: unexpected status %d", key, resp.StatusCode)
+ }
+}
+
+var _ release.Opener = (*store)(nil)
+```
+
+`internal/release/fetch.go`:
+```go
+package release
+
+import (
+ "context"
+ "crypto/sha256"
+ "encoding/hex"
+ "errors"
+ "fmt"
+ "io"
+ "os"
+ "path/filepath"
+)
+
+// ErrTooLarge: the object exceeded the caller's limit. The reader is cut at
+// limit+1 bytes, so an object EXACTLY at the limit is told from one over it.
+var ErrTooLarge = errors.New("store: object larger than the caller's limit")
+
+// FetchToFile streams key into dst with a running sha256, never holding the
+// object in memory: temp file, fsync, rename, directory fsync. Any failure
+// removes the temp and leaves dst untouched.
+func FetchToFile(ctx context.Context, s Store, key, dst string, limit int64, wantSHA256 string) (err error) {
+ rc, _, err := OpenObject(ctx, s, key)
+ if err != nil {
+ return err
+ }
+ defer rc.Close()
+ tmp := dst + ".tmp"
+ f, err := os.OpenFile(tmp, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0o600)
+ if err != nil {
+ return err
+ }
+ defer func() {
+ if err != nil {
+ f.Close()
+ os.Remove(tmp)
+ }
+ }()
+ h := sha256.New()
+ n, err := io.Copy(io.MultiWriter(f, h), io.LimitReader(rc, limit+1))
+ if err != nil {
+ return fmt.Errorf("fetch %s: %w", key, err)
+ }
+ if n > limit {
+ return fmt.Errorf("fetch %s: %w (limit %d)", key, ErrTooLarge, limit)
+ }
+ if got := hex.EncodeToString(h.Sum(nil)); got != wantSHA256 {
+ return fmt.Errorf("fetch %s: sha256 %s does not match the signed %s", key, got, wantSHA256)
+ }
+ if err = f.Sync(); err != nil {
+ return err
+ }
+ if err = f.Close(); err != nil {
+ return err
+ }
+ if err = os.Rename(tmp, dst); err != nil {
+ return err
+ }
+ return syncDir(filepath.Dir(dst))
+}
+
+// syncDir makes a rename durable: the directory entry is what fsync(file)
+// does not cover.
+func syncDir(dir string) error {
+ d, err := os.Open(dir)
+ if err != nil {
+ return err
+ }
+ defer d.Close()
+ return d.Sync()
+}
+```
+
+- [ ] **Step 4: Run the tests**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/release/ ./internal/fleetapi/ ./internal/edge/ -count=1 2>&1); rc=$?; echo "$out" | tail -20; echo rc=$rc`
+Expected: all `ok`. (`internal/edge` is included because its test fakes assign to `release.Store`; the interface did not change, so they still compile.)
+
+- [ ] **Step 5: Commit**
+
+```bash
+gofmt -l internal/ && git add internal/release/store.go internal/release/fetch.go internal/release/fetch_test.go internal/fleetapi/fleetapi.go internal/fleetapi/fleetapi_test.go && git commit -q -F - <<'EOF'
+release: stream an object to a file with a running sha
+
+Opener is a side-interface (48 test fakes implement Store; none break),
+implemented by DirStore, S3Store, prefixStore and the fleet door client.
+FetchToFile is what a box uses for a platform binary: limit+1 read, sha
+checked before rename, directory fsync after.
+
+Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
+Claude-Session: https://claude.ai/code/session_015QCUCpfxbCyEAVQ2m51dH5
+EOF
+```
+
+---
+
+### Task 3: Store-door grants for the pointer and artifact shapes
+
+**Files:**
+- Modify: `internal/edge/storedoor.go:383-397` (`aclRead`) and the doc comment above it (`:352-382`)
+- Modify: `internal/edge/storedoor_test.go` (`TestStoreDoorACLMatrix` rows at `:69+`, `TestStoreDoorGetServesOwnEstate` at `:365`, the prose in `TestDoneNeverServesCertMaterial`'s comment at `:519`)
+
+**Interfaces:**
+- Consumes: `aclRead(sqid, fleet, key string) bool`; `canonicalKey` already refuses any `..`/`.`/empty segment before the ACL runs, so traversal never reaches here.
+- Produces: `func systemReadKey(key string) bool` (unexported) — true for exactly `control/system/<c>/<ch>.json` and `bootstrap/router/<v>/<asset>`.
+
+- [ ] **Step 1: Add the failing matrix rows and GET cases**
+
+In `TestStoreDoorACLMatrix`'s table, after the `bootstrap/carlos.sig` row (`:84`):
+```go
+ {"control/system/router/stable.json", opRead, true, "the platform channel pointer (system update spec §3)"},
+ {"control/system/router/canary.json", opRead, true, "…and the canary one"},
+ {"control/system/router/stable.json", opWrite, false, "a box never writes a pointer"},
+ {"control/system/router", opRead, false, "the pointer shape is exactly three segments ending .json"},
+ {"control/system/router/stable.json/x", opRead, false, "…not four"},
+ {"control/system/router/stable.txt", opRead, false, "…and .json"},
+ {"bootstrap/router/v0.18.0/carlos-linux-arm64", opRead, true, "a mirrored platform artifact"},
+ {"bootstrap/router/v0.18.0/carlos-linux-arm64", opWrite, false, "a box must never publish the platform binary"},
+ {"bootstrap/router/v0.18.0", opRead, false, "the artifact shape is exactly three segments"},
+ {"bootstrap/router/v0.18.0/a/b", opRead, false, "…not four"},
+ {"bootstrap/router.sig", opRead, false, "nothing else under bootstrap/"},
+ {"bootstrap/router/../../x", opRead, false, "traversal is refused by canonicalKey before the ACL"},
+ {"control/system/../serial", opRead, false, "…on both prefixes"},
+```
+In `TestStoreDoorGetServesOwnEstate`, seed two more objects with `seedObject` and add their keys to the loop that expects 200:
+```go
+ seedObject(t, store, "control/system/router/stable.json", []byte(`{"payload":"x"}`))
+ seedObject(t, store, "bootstrap/router/v1/carlos-linux-arm64", []byte("ELF"))
+```
+Update the comment at `storedoor_test.go:519` (the allowlist prose) to add "`control/system/<c>/<ch>.json`, `bootstrap/router/<v>/<asset>`".
+
+- [ ] **Step 2: Run to verify failure**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/edge/ -run 'TestStoreDoorACLMatrix|TestStoreDoorGetServesOwnEstate' -count=1 2>&1); rc=$?; echo "$out" | tail -30; echo rc=$rc`
+Expected: FAIL on the four `want true` rows and the two new GET keys (404).
+
+- [ ] **Step 3: Implement**
+
+In `storedoor.go`, replace the `switch key` in `aclRead` with:
+```go
+ switch key {
+ case "control/serial", "bootstrap/carlos":
+ return true
+ }
+ if systemReadKey(key) {
+ return true
+ }
+```
+and add, below `aclRead`:
+```go
+// systemReadKey is the two shapes the system-update spec (§3) grants every
+// fleet box: its platform channel pointer and a mirrored platform artifact.
+// Shapes, not prefixes — exactly three segments each, so nothing else that
+// ever lands under control/ or bootstrap/ is readable by a box. Listing is
+// not granted (aclDir is unchanged), and canonicalKey has already refused
+// any "." or ".." segment before this runs.
+func systemReadKey(key string) bool {
+ if rest, ok := strings.CutPrefix(key, "control/system/"); ok {
+ comp, file, ok := strings.Cut(rest, "/")
+ return ok && comp != "" && file != "" && !strings.Contains(file, "/") &&
+ strings.HasSuffix(file, ".json") && len(file) > len(".json")
+ }
+ if rest, ok := strings.CutPrefix(key, "bootstrap/router/"); ok {
+ ver, asset, ok := strings.Cut(rest, "/")
+ return ok && ver != "" && asset != "" && !strings.Contains(asset, "/")
+ }
+ return false
+}
+```
+Extend the `aclRead` doc comment's allowlist enumeration (`:352-382`) with the two shapes and the sentence: "`bootstrap/carlos` stays until `update-platform-binary.sh` is retired (spec §10)."
+
+- [ ] **Step 4: Run the edge tests**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/edge/ -count=1 2>&1); rc=$?; echo "$out" | tail -20; echo rc=$rc`
+Expected: `ok`.
+
+- [ ] **Step 5: Commit**
+
+```bash
+gofmt -l internal/edge/ && git add internal/edge/storedoor.go internal/edge/storedoor_test.go && git commit -q -F - <<'EOF'
+edge: the store door grants a box its platform pointer and artifact
+
+Two exact shapes under control/system/ and bootstrap/router/, no
+listing, no writes. Without the pointer grant a door-backed fleet box
+would never see a platform promote.
+
+Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
+Claude-Session: https://claude.ai/code/session_015QCUCpfxbCyEAVQ2m51dH5
+EOF
+```
+
+---
+### Task 4: On-box state files (`internal/host/systemstate.go`)
+
+**Files:**
+- Create: `internal/host/systemstate.go`
+- Test: `internal/host/systemstate_test.go`
+
+**Interfaces:**
+- Consumes: nothing new.
+- Produces (package `host`, unexported unless noted):
+ ```go
+ const (
+ systemComponent = "router"
+ systemRequestMarker = "system-update-requested" // touched under dataDir
+ systemRequestEvery = time.Minute
+ systemArtifactLimit = int64(96 << 20)
+ systemOutcomeApplied = "applied"; systemOutcomeRolledBack = "rolled-back"; systemOutcomeRollbackFailed = "rollback-failed"
+ systemOutcomeRefused = "refused"; systemOutcomeDeferred = "deferred"; systemOutcomeNoted = "noted"
+ systemPhaseBackingUp = "backing-up"; systemPhaseInstalling = "installing"; systemPhaseInstalled = "installed"
+ systemPhaseUnits = "units"; systemPhaseRestarted = "restarted"; systemPhaseGated = "gated"
+ )
+ type systemRequest struct{ Version string; PromotedAt time.Time; NoOp bool; SeenAt time.Time }
+ type systemResult struct{ Channel, Version string; PromotedAt time.Time; Outcome, Reason string; At time.Time }
+ type systemApplied struct{ Version, Channel string; At time.Time }
+ type systemSeen map[string]time.Time
+ type systemJournal struct{ Phase, Target, PrevBuild string; UnitsAdded, UnitsRemoved []string; StartedAt time.Time }
+ func systemDataDir(dd string) string // <dd>/system/router
+ func systemRequestPath(dd string) string // <dd>/system/router/requested.json
+ func systemCachePath(dd, version string) string // <dd>/system/router/<version>/carlos
+ func systemRootDir(optDir string) string // <optDir>/system/router
+ func systemPrevPath(optDir string) string // <optDir>/carlos.prev
+ func systemBinaryPath(optDir string) string // <optDir>/carlos
+ func writeFileDurable(path string, data []byte, mode os.FileMode) error
+ func writeJSONDurable(path string, v any, mode os.FileMode) error
+ func readJSONFile(path string, v any) error // os.ErrNotExist passthrough
+ func readSystemResult(optDir string) (*systemResult, error)
+ func readSystemSeen(optDir string) (systemSeen, error) // missing file = empty map, nil error
+ func readSystemApplied(optDir string) (*systemApplied, error)
+ func readSystemJournal(optDir string) (*systemJournal, error)
+ func (r *systemResult) matches(channel, version string, promotedAt time.Time) bool
+ func (r *systemResult) terminal() bool // applied|rolled-back|rollback-failed|refused
+ func validSystemVersion(v string) bool // ^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$
+ ```
+ The root files are `result.json`, `seen.json`, `applied.json`, `journal.json` directly under `systemRootDir`.
+
+- [ ] **Step 1: Write the failing tests**
+
+```go
+package host
+
+import (
+ "errors"
+ "os"
+ "path/filepath"
+ "testing"
+ "time"
+)
+
+func TestSystemPathsAreDerivedNotTrusted(t *testing.T) {
+ if got := systemCachePath("/data", "v1.2.3"); got != "/data/system/router/v1.2.3/carlos" {
+ t.Fatalf("cache path = %s", got)
+ }
+ if got := systemRequestPath("/data"); got != "/data/system/router/requested.json" {
+ t.Fatalf("request path = %s", got)
+ }
+ if got := systemRootDir("/opt/carlos"); got != "/opt/carlos/system/router" {
+ t.Fatalf("root dir = %s", got)
+ }
+ for _, bad := range []string{"", ".", "..", "../x", "a/b", ".hidden", "v1 2", string(make([]byte, 65))} {
+ if validSystemVersion(bad) {
+ t.Errorf("validSystemVersion(%q) = true", bad)
+ }
+ }
+ for _, good := range []string{"v0.18.0", "66587fa", "a", "release-2026.09.06_rc1"} {
+ if !validSystemVersion(good) {
+ t.Errorf("validSystemVersion(%q) = false", good)
+ }
+ }
+}
+
+func TestWriteFileDurableLeavesWholeFileOrNothing(t *testing.T) {
+ dir := t.TempDir()
+ p := filepath.Join(dir, "journal.json")
+ if err := writeFileDurable(p, []byte("one"), 0o644); err != nil {
+ t.Fatal(err)
+ }
+ if err := writeFileDurable(p, []byte("two"), 0o644); err != nil {
+ t.Fatal(err)
+ }
+ b, _ := os.ReadFile(p)
+ if string(b) != "two" {
+ t.Fatalf("content = %q", b)
+ }
+ if _, err := os.Stat(p + ".tmp"); !errors.Is(err, os.ErrNotExist) {
+ t.Fatal("temp left behind")
+ }
+ st, _ := os.Stat(p)
+ if st.Mode().Perm() != 0o644 {
+ t.Fatalf("mode = %v", st.Mode().Perm())
+ }
+ // A directory that does not exist is an error, not a silent no-op.
+ if err := writeFileDurable(filepath.Join(dir, "no", "such", "f"), []byte("x"), 0o644); err == nil {
+ t.Fatal("wrote into a missing directory")
+ }
+}
+
+func TestSystemStateReadersRoundTrip(t *testing.T) {
+ opt := t.TempDir()
+ root := systemRootDir(opt)
+ if err := os.MkdirAll(root, 0o755); err != nil {
+ t.Fatal(err)
+ }
+ if r, err := readSystemResult(opt); err != nil || r != nil {
+ t.Fatalf("missing result: %v %v", r, err)
+ }
+ seen, err := readSystemSeen(opt)
+ if err != nil || len(seen) != 0 {
+ t.Fatalf("missing seen: %v %v", seen, err)
+ }
+ at := time.Date(2026, 9, 6, 9, 0, 0, 0, time.UTC)
+ res := systemResult{Channel: "stable", Version: "v1", PromotedAt: at, Outcome: systemOutcomeRolledBack, Reason: "gate", At: at}
+ if err := writeJSONDurable(filepath.Join(root, "result.json"), res, 0o644); err != nil {
+ t.Fatal(err)
+ }
+ got, err := readSystemResult(opt)
+ if err != nil || !got.matches("stable", "v1", at) || !got.terminal() {
+ t.Fatalf("result = %+v err=%v", got, err)
+ }
+ if got.matches("canary", "v1", at) || got.matches("stable", "v2", at) || got.matches("stable", "v1", at.Add(time.Second)) {
+ t.Fatal("matches ignores part of the tuple")
+ }
+ got.Outcome = systemOutcomeDeferred
+ if got.terminal() {
+ t.Fatal("deferred is not terminal")
+ }
+ got.Outcome = systemOutcomeNoted
+ if got.terminal() {
+ t.Fatal("noted is not terminal for the leg: the version may still differ next time")
+ }
+ seen["stable"] = at
+ if err := writeJSONDurable(filepath.Join(root, "seen.json"), seen, 0o644); err != nil {
+ t.Fatal(err)
+ }
+ seen2, _ := readSystemSeen(opt)
+ if !seen2["stable"].Equal(at) {
+ t.Fatalf("seen = %v", seen2)
+ }
+ j := systemJournal{Phase: systemPhaseInstalled, Target: "v1", PrevBuild: "v0", UnitsAdded: []string{"a.path"}, StartedAt: at}
+ if err := writeJSONDurable(filepath.Join(root, "journal.json"), j, 0o600); err != nil {
+ t.Fatal(err)
+ }
+ j2, err := readSystemJournal(opt)
+ if err != nil || j2.Phase != systemPhaseInstalled || j2.UnitsAdded[0] != "a.path" {
+ t.Fatalf("journal = %+v err=%v", j2, err)
+ }
+}
+```
+
+- [ ] **Step 2: Run to verify failure**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/host/ -run 'TestSystemPaths|TestWriteFileDurable|TestSystemStateReaders' -count=1 2>&1); rc=$?; echo "$out" | tail -20; echo rc=$rc`
+Expected: undefined identifiers.
+
+- [ ] **Step 3: Implement `internal/host/systemstate.go`**
+
+```go
+package host
+
+import (
+ "encoding/json"
+ "errors"
+ "os"
+ "path/filepath"
+ "regexp"
+ "time"
+)
+
+// The on-box state of the platform self-update (spec §4, §6). Two homes:
+//
+// <data>/system/router/ — written by the sandboxed agent: the
+// request and the artifact cache. Root
+// reads exactly one file here and trusts
+// nothing in it beyond a version string.
+// <opt>/system/router/ — written by root: journal, seen (replay
+// watermarks), applied (the .prev
+// authority), result (what the agent
+// publishes). Root-owned, world-readable.
+const (
+ systemComponent = "router"
+ systemRequestMarker = "system-update-requested"
+ systemRequestEvery = time.Minute
+ systemArtifactLimit = int64(96 << 20)
+
+ systemOutcomeApplied = "applied"
+ systemOutcomeRolledBack = "rolled-back"
+ systemOutcomeRollbackFailed = "rollback-failed"
+ systemOutcomeRefused = "refused"
+ systemOutcomeDeferred = "deferred"
+ systemOutcomeNoted = "noted"
+
+ systemPhaseBackingUp = "backing-up"
+ systemPhaseInstalling = "installing"
+ systemPhaseInstalled = "installed"
+ systemPhaseUnits = "units"
+ systemPhaseRestarted = "restarted"
+ systemPhaseGated = "gated"
+)
+
+// systemRequest is the agent's hint to root: which promote it saw. No
+// channel (root's own host.env says), no key, no sha, no path (root derives
+// the cache path from the version and takes key and sha from the pointer).
+type systemRequest struct {
+ Version string `json:"version"`
+ PromotedAt time.Time `json:"promoted_at"`
+ NoOp bool `json:"noop"`
+ SeenAt time.Time `json:"seen_at"`
+}
+
+// systemResult is root's answer to the last request, keyed by the REQUEST's
+// tuple so a refusal of one promote never suppresses the next.
+type systemResult struct {
+ Channel string `json:"channel"`
+ Version string `json:"version"`
+ PromotedAt time.Time `json:"promoted_at"`
+ Outcome string `json:"outcome"`
+ Reason string `json:"reason,omitempty"`
+ At time.Time `json:"at"`
+}
+
+func (r *systemResult) matches(channel, version string, promotedAt time.Time) bool {
+ return r != nil && r.Channel == channel && r.Version == version && r.PromotedAt.Equal(promotedAt)
+}
+
+// terminal: outcomes the leg must not retry for the same pointer.
+func (r *systemResult) terminal() bool {
+ switch r.Outcome {
+ case systemOutcomeApplied, systemOutcomeRolledBack, systemOutcomeRollbackFailed, systemOutcomeRefused:
+ return true
+ }
+ return false
+}
+
+// systemApplied is the last build that PASSED A GATE on this box, forward or
+// rollback. It is the only thing that may authorise refreshing carlos.prev.
+type systemApplied struct {
+ Version string `json:"version"`
+ Channel string `json:"channel"`
+ At time.Time `json:"at"`
+}
+
+// systemSeen is the replay watermark per channel: the newest PromotedAt root
+// has verified. Advanced on every verified pointer, including no-ops.
+type systemSeen map[string]time.Time
+
+// systemJournal is the open transaction. Written before every box-changing
+// step, removed last; a run that finds one recovers before anything else.
+type systemJournal struct {
+ Phase string `json:"phase"`
+ Target string `json:"target"`
+ PrevBuild string `json:"prev_build"`
+ UnitsAdded []string `json:"units_added,omitempty"`
+ UnitsRemoved []string `json:"units_removed,omitempty"`
+ StartedAt time.Time `json:"started_at"`
+}
+
+var systemVersionRe = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$`)
+
+// validSystemVersion is what makes the cache path derivable: no leading dot,
+// no separator, nothing that could be a traversal.
+func validSystemVersion(v string) bool { return systemVersionRe.MatchString(v) }
+
+func systemDataDir(dd string) string { return filepath.Join(dd, "system", systemComponent) }
+func systemRequestPath(dd string) string {
+ return filepath.Join(systemDataDir(dd), "requested.json")
+}
+func systemCachePath(dd, version string) string {
+ return filepath.Join(systemDataDir(dd), version, "carlos")
+}
+func systemRootDir(optDir string) string { return filepath.Join(optDir, "system", systemComponent) }
+func systemPrevPath(optDir string) string { return filepath.Join(optDir, "carlos.prev") }
+func systemBinaryPath(optDir string) string { return filepath.Join(optDir, "carlos") }
+
+// writeFileDurable: temp, fsync, rename, directory fsync. A crash leaves the
+// old file or the new one, never a torn one, and the rename itself is on
+// disk before the caller's next step depends on it.
+func writeFileDurable(path string, data []byte, mode os.FileMode) (err error) {
+ tmp := path + ".tmp"
+ f, err := os.OpenFile(tmp, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, mode)
+ if err != nil {
+ return err
+ }
+ defer func() {
+ if err != nil {
+ f.Close()
+ os.Remove(tmp)
+ }
+ }()
+ if _, err = f.Write(data); err != nil {
+ return err
+ }
+ if err = f.Chmod(mode); err != nil {
+ return err
+ }
+ if err = f.Sync(); err != nil {
+ return err
+ }
+ if err = f.Close(); err != nil {
+ return err
+ }
+ if err = os.Rename(tmp, path); err != nil {
+ return err
+ }
+ d, err := os.Open(filepath.Dir(path))
+ if err != nil {
+ return err
+ }
+ defer d.Close()
+ return d.Sync()
+}
+
+func writeJSONDurable(path string, v any, mode os.FileMode) error {
+ b, err := json.MarshalIndent(v, "", " ")
+ if err != nil {
+ return err
+ }
+ return writeFileDurable(path, append(b, '\n'), mode)
+}
+
+// readJSONFile leaves os.ErrNotExist intact so callers can treat "no file"
+// as a state, not a failure.
+func readJSONFile(path string, v any) error {
+ b, err := os.ReadFile(path)
+ if err != nil {
+ return err
+ }
+ return json.Unmarshal(b, v)
+}
+
+func readSystemResult(optDir string) (*systemResult, error) {
+ var r systemResult
+ err := readJSONFile(filepath.Join(systemRootDir(optDir), "result.json"), &r)
+ if errors.Is(err, os.ErrNotExist) {
+ return nil, nil
+ }
+ return &r, err
+}
+
+func readSystemSeen(optDir string) (systemSeen, error) {
+ s := systemSeen{}
+ err := readJSONFile(filepath.Join(systemRootDir(optDir), "seen.json"), &s)
+ if errors.Is(err, os.ErrNotExist) {
+ return s, nil
+ }
+ return s, err
+}
+
+func readSystemApplied(optDir string) (*systemApplied, error) {
+ var a systemApplied
+ err := readJSONFile(filepath.Join(systemRootDir(optDir), "applied.json"), &a)
+ if errors.Is(err, os.ErrNotExist) {
+ return nil, nil
+ }
+ return &a, err
+}
+
+func readSystemJournal(optDir string) (*systemJournal, error) {
+ var j systemJournal
+ err := readJSONFile(filepath.Join(systemRootDir(optDir), "journal.json"), &j)
+ if errors.Is(err, os.ErrNotExist) {
+ return nil, nil
+ }
+ return &j, err
+}
+```
+
+- [ ] **Step 4: Run the tests**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/host/ -run 'TestSystemPaths|TestWriteFileDurable|TestSystemStateReaders' -count=1 2>&1); rc=$?; echo "$out" | tail -20; echo rc=$rc`
+Expected: PASS.
+
+- [ ] **Step 5: Commit**
+
+```bash
+gofmt -l internal/host/ && git add internal/host/systemstate.go internal/host/systemstate_test.go && git commit -q -F - <<'EOF'
+host: the on-box state of a platform self-update
+
+Request (agent-written hint), result, seen watermarks, applied, and
+the journal, with durable writes: temp, fsync, rename, directory
+fsync. Cache paths are derived from a validated version string.
+
+Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
+Claude-Session: https://claude.ai/code/session_015QCUCpfxbCyEAVQ2m51dH5
+EOF
+```
+
+---
+
+### Task 5: The heartbeat learns platform, channel and last apply outcome
+
+**Files:**
+- Modify: `internal/status/snapshot.go` — `Probes` (~`:98`, next to `Release`), `BoxMetrics` (~`:158`), `CollectBox` (~`:404`, where `Release: p.Release` is copied), the emptiness guard (~`:543`)
+- Modify: `internal/host/agent.go` `statusLoop` (~`:2318`, the `Release: version.Build,` line)
+- Create: `internal/host/systemprobe.go`
+- Test: `internal/status/snapshot_test.go` (add), `internal/host/systemprobe_test.go`
+
+**Interfaces:**
+- Consumes: `readSystemResult` (Task 4), `release.OptDir()`, `status.Probes`, `status.BoxMetrics`.
+- Produces:
+ ```go
+ // package status
+ type SystemState struct{ Channel, Version string; PromotedAt time.Time; Outcome, Reason string; At time.Time }
+ // Probes gains: Platform string; SystemChannel string; System *SystemState
+ // BoxMetrics gains (json): platform, system_channel, system
+ // package host
+ func systemChannel(getenv func(string) string) string // CARLOS_SYSTEM_CHANNEL, default "stable"
+ func systemStateProbe(optDir string) *status.SystemState // nil when no result.json
+ const systemChannelEnv = "CARLOS_SYSTEM_CHANNEL"
+ ```
+
+- [ ] **Step 1: Write the failing tests**
+
+`internal/status/snapshot_test.go`, add:
+```go
+func TestBoxMetricsCarryThePlatformSelfUpdateState(t *testing.T) {
+ at := time.Date(2026, 9, 6, 9, 0, 0, 0, time.UTC)
+ p := Probes{Release: "v1", Platform: "linux-arm64", SystemChannel: "canary",
+ System: &SystemState{Channel: "canary", Version: "v2", PromotedAt: at, Outcome: "rolled-back", Reason: "gate", At: at}}
+ b := CollectBox(nil, p, at)
+ if b.Metrics == nil || b.Metrics.Platform != "linux-arm64" || b.Metrics.SystemChannel != "canary" || b.Metrics.System == nil || b.Metrics.System.Outcome != "rolled-back" {
+ t.Fatalf("metrics = %+v", b.Metrics)
+ }
+ out, _ := json.Marshal(b)
+ for _, want := range []string{`"platform":"linux-arm64"`, `"system_channel":"canary"`, `"outcome":"rolled-back"`} {
+ if !strings.Contains(string(out), want) {
+ t.Errorf("json lacks %s: %s", want, out)
+ }
+ }
+ // A box with only the new fields is not "empty".
+ b2 := CollectBox(nil, Probes{Platform: "linux-arm64"}, at)
+ if b2.Metrics == nil {
+ t.Fatal("platform alone must produce a metrics block")
+ }
+ // The public fleet feed still never carries the metrics block.
+ f := BuildFleet([]Box{b}, at)
+ fb, _ := json.Marshal(f)
+ if strings.Contains(string(fb), "system_channel") || strings.Contains(string(fb), "rolled-back") {
+ t.Fatalf("fleet.json leaked the private metrics: %s", fb)
+ }
+}
+```
+(Check `BuildFleet`'s exact signature at `fleet.go:157` and adjust the call; it takes the box list and a time.)
+
+`internal/host/systemprobe_test.go`:
+```go
+package host
+
+import (
+ "os"
+ "path/filepath"
+ "testing"
+ "time"
+)
+
+func TestSystemChannelDefaultsToStable(t *testing.T) {
+ if got := systemChannel(func(string) string { return "" }); got != "stable" {
+ t.Fatalf("default = %q", got)
+ }
+ if got := systemChannel(func(k string) string {
+ if k == systemChannelEnv {
+ return " canary "
+ }
+ return ""
+ }); got != "canary" {
+ t.Fatalf("env = %q", got)
+ }
+}
+
+func TestSystemStateProbeReadsRootsResult(t *testing.T) {
+ opt := t.TempDir()
+ if got := systemStateProbe(opt); got != nil {
+ t.Fatalf("no result.json → %+v, want nil", got)
+ }
+ root := systemRootDir(opt)
+ os.MkdirAll(root, 0o755)
+ at := time.Date(2026, 9, 6, 9, 0, 0, 0, time.UTC)
+ writeJSONDurable(filepath.Join(root, "result.json"), systemResult{Channel: "stable", Version: "v3", PromotedAt: at, Outcome: "applied", At: at}, 0o644)
+ got := systemStateProbe(opt)
+ if got == nil || got.Version != "v3" || got.Outcome != "applied" || !got.PromotedAt.Equal(at) {
+ t.Fatalf("probe = %+v", got)
+ }
+}
+```
+
+- [ ] **Step 2: Run to verify failure**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/status/ ./internal/host/ -run 'TestBoxMetricsCarry|TestSystemChannel|TestSystemStateProbe' -count=1 2>&1); rc=$?; echo "$out" | tail -20; echo rc=$rc`
+Expected: undefined fields/functions.
+
+- [ ] **Step 3: Implement**
+
+`internal/status/snapshot.go` — in `Probes`, after `Release string` (`:98`):
+```go
+ // Platform is runtime.GOOS-runtime.GOARCH, so the console knows which
+ // artifacts a platform promote must mirror. SystemChannel is the box's
+ // CARLOS_SYSTEM_CHANNEL, so the console knows which boxes a promote
+ // reaches. System is root's last self-update outcome (spec §5, §6).
+ Platform string
+ SystemChannel string
+ System *SystemState
+```
+Add the type near `BoxMetrics`:
+```go
+// SystemState is the last platform self-update outcome root recorded on the
+// box, published so `carlos system status` can show a rolled-back box with
+// its reason within one heartbeat.
+type SystemState struct {
+ Channel string `json:"channel"`
+ Version string `json:"version"`
+ PromotedAt time.Time `json:"promoted_at"`
+ Outcome string `json:"outcome"`
+ Reason string `json:"reason,omitempty"`
+ At time.Time `json:"at"`
+}
+```
+In `BoxMetrics`, after `Release` (`:158`):
+```go
+ Platform string `json:"platform,omitempty"` // runtime.GOOS-runtime.GOARCH
+ SystemChannel string `json:"system_channel,omitempty"` // CARLOS_SYSTEM_CHANNEL
+ System *SystemState `json:"system,omitempty"` // last self-update outcome
+```
+In `CollectBox`, where `Release: p.Release` is copied into the metrics block, add `Platform: p.Platform, SystemChannel: p.SystemChannel, System: p.System`. In the emptiness guard (`:543`), add `|| p.Platform != "" || p.SystemChannel != "" || p.System != nil`.
+
+`internal/host/systemprobe.go`:
+```go
+package host
+
+import (
+ "strings"
+
+ "github.com/carlosframework/platform/internal/status"
+)
+
+// systemChannelEnv is the box's platform channel: canary or stable. Set in
+// host.env, read through the getenv seam like every other CARLOS_* value;
+// the ROOT apply reads it from its own unit environment, never from the
+// agent's request (spec §6 step 2).
+const systemChannelEnv = "CARLOS_SYSTEM_CHANNEL"
+
+func systemChannel(getenv func(string) string) string {
+ if c := strings.TrimSpace(getenv(systemChannelEnv)); c != "" {
+ return c
+ }
+ return "stable"
+}
+
+// systemStateProbe is root's result.json as the heartbeat carries it, or nil
+// when this box has never been asked to update.
+func systemStateProbe(optDir string) *status.SystemState {
+ r, err := readSystemResult(optDir)
+ if err != nil || r == nil {
+ return nil
+ }
+ return &status.SystemState{Channel: r.Channel, Version: r.Version, PromotedAt: r.PromotedAt, Outcome: r.Outcome, Reason: r.Reason, At: r.At}
+}
+```
+`internal/host/agent.go` `statusLoop`, next to `Release: version.Build,` (`:2318`):
+```go
+ Platform: runtime.GOOS + "-" + runtime.GOARCH,
+ SystemChannel: systemChannel(os.Getenv),
+ System: systemStateProbe(release.OptDir()),
+```
+(add `"runtime"` to the imports). Add `CARLOS_SYSTEM_CHANNEL` to the env contract in the package doc comment at the top of `agent.go`, one line: `CARLOS_SYSTEM_CHANNEL — platform channel this box follows for self-update, canary|stable (default stable)`.
+
+- [ ] **Step 4: Run the tests**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/status/ ./internal/host/ -count=1 2>&1); rc=$?; echo "$out" | tail -20; echo rc=$rc`
+Expected: `ok` for both.
+
+- [ ] **Step 5: Commit**
+
+```bash
+gofmt -l internal/ && git add internal/status/snapshot.go internal/status/snapshot_test.go internal/host/agent.go internal/host/systemprobe.go internal/host/systemprobe_test.go && git commit -q -F - <<'EOF'
+status: a box reports its platform, its system channel and its last self-update outcome
+
+Three optional fields on the private metrics block; the public fleet
+feed does not carry them, as before.
+
+Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
+Claude-Session: https://claude.ai/code/session_015QCUCpfxbCyEAVQ2m51dH5
+EOF
+```
+
+---
+
+### Task 6: The three units, `enabledUnits`, `-plan-json`, the path probe
+
+**Files:**
+- Create: `internal/host/units/carlos-system-update.path`, `internal/host/units/carlos-system-update.service`, `internal/host/units/carlos-system-recover.service`
+- Modify: `internal/host/bootstrap.go` (`enabledUnits` `:72`, `RunBootstrap` flags `:115-132`, the staging print block `:327-341`)
+- Modify: `internal/host/probes.go` (add a fourth `*Active()`)
+- Modify: `internal/host/host_test.go` (`expectedUnits` `:29-46`; `TestProbedUnitsAreTheOnesBootstrapArms` `:1129`)
+- Test: `internal/host/host_test.go` (add `TestBootstrapPlanJSON`)
+
+**Interfaces:**
+- Consumes: `unitsFS`, `unitNames`, `systemctlPlan`, `bootstrap(root, consumers)`, `parseIsActive`.
+- Produces:
+ ```go
+ // package host
+ var inertUnits = []string{"carlos-restore-verify.timer", "carlos-hibernate-verify.timer"}
+ type bootstrapPlan struct{ Enabled []string `json:"enabled"`; Inert []string `json:"inert"` }
+ func currentBootstrapPlan() bootstrapPlan
+ // `carlos bootstrap -root <dir> -plan-json` prints ONLY the JSON plan on stdout
+ const systemUpdatePathUnit = "carlos-system-update.path"
+ func systemUpdatePathActive() *bool
+ ```
+
+- [ ] **Step 1: Add the failing tests**
+
+In `host_test.go`, add the three names to `expectedUnits` (keep it alphabetical):
+```go
+ "carlos-system-recover.service",
+ "carlos-system-update.path",
+ "carlos-system-update.service",
+```
+and add:
+```go
+func TestBootstrapPlanJSONNamesWhatALiveRunArms(t *testing.T) {
+ p := currentBootstrapPlan()
+ want := map[string]bool{"carlos-edge.service": true, "carlos-adopt.timer": true, "carlos-adopt.path": true,
+ "carlos-stage.path": true, "carlos-restart.path": true, "carlos-system-update.path": true, "carlos-system-recover.service": true}
+ got := map[string]bool{}
+ for _, u := range p.Enabled {
+ got[u] = true
+ }
+ for u := range want {
+ if !got[u] {
+ t.Errorf("plan lacks %s", u)
+ }
+ }
+ if got["carlos-system-update.service"] {
+ t.Error("the path-triggered oneshot must not be enabled: it is started by its .path")
+ }
+ inert := map[string]bool{}
+ for _, u := range p.Inert {
+ inert[u] = true
+ }
+ if !inert["carlos-restore-verify.timer"] || !inert["carlos-hibernate-verify.timer"] {
+ t.Errorf("inert = %v", p.Inert)
+ }
+ // And systemctlPlan never starts a service, whatever is enabled.
+ for _, args := range systemctlPlan(p.Enabled) {
+ if args[0] == "start" {
+ for _, u := range args[1:] {
+ if strings.HasSuffix(u, ".service") {
+ t.Errorf("plan starts a service: %v", args)
+ }
+ }
+ }
+ }
+}
+
+func TestSystemUnitsAreShapedLikeTheStageRelay(t *testing.T) {
+ read := func(name string) string {
+ b, err := unitsFS.ReadFile("units/" + name)
+ if err != nil {
+ t.Fatal(err)
+ }
+ return string(b)
+ }
+ p := read("carlos-system-update.path")
+ for _, want := range []string{"PathChanged=/data/system-update-requested", "Unit=carlos-system-update.service", "WantedBy=multi-user.target"} {
+ if !strings.Contains(p, want) {
+ t.Errorf("path unit lacks %q", want)
+ }
+ }
+ u := read("carlos-system-update.service")
+ for _, want := range []string{"Type=oneshot", "StartLimitIntervalSec=0", "TimeoutStartSec=15min", "OnFailure=carlos-system-recover.service",
+ "EnvironmentFile=-/etc/carlos/host.env", "system apply", "carlos.prev", "MemoryMax=256M"} {
+ if !strings.Contains(u, want) {
+ t.Errorf("update unit lacks %q", want)
+ }
+ }
+ for _, never := range []string{"ProtectSystem", "User="} {
+ if strings.Contains(u, never) {
+ t.Errorf("update unit must run as root with /opt writable; found %q", never)
+ }
+ }
+ r := read("carlos-system-recover.service")
+ for _, want := range []string{"Type=oneshot", "Before=carlos-edge.service", "WantedBy=multi-user.target", "TimeoutStartSec=10min", "system apply --recover", "carlos.prev"} {
+ if !strings.Contains(r, want) {
+ t.Errorf("recover unit lacks %q", want)
+ }
+ }
+}
+```
+Update `TestProbedUnitsAreTheOnesBootstrapArms` so its probed set includes `systemUpdatePathUnit` (read the test; it compares probe constants against `enabledUnits`).
+
+- [ ] **Step 2: Run to verify failure**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/host/ -run 'TestBootstrap|TestSystemUnits|TestProbedUnits' -count=1 2>&1); rc=$?; echo "$out" | tail -30; echo rc=$rc`
+Expected: FAIL (missing units, undefined `currentBootstrapPlan`).
+
+- [ ] **Step 3: Write the units**
+
+`internal/host/units/carlos-system-update.path`:
+```ini
+# The relay from the sandboxed agent to a root apply of a new platform
+# binary (spec §4). The agent (User=carlos, ProtectSystem=strict) can fetch
+# and verify the artifact but cannot write /opt or /etc; it touches this
+# marker and root does the rest. Same shape as carlos-stage.path.
+[Unit]
+Description=CARLOS platform self-update on agent request
+
+[Path]
+PathChanged=/data/system-update-requested
+Unit=carlos-system-update.service
+
+[Install]
+WantedBy=multi-user.target
+```
+
+`internal/host/units/carlos-system-update.service`:
+```ini
+# Root apply of a platform self-update (spec §6): re-verify the pointer,
+# copy the candidate into a root-owned dir, journal, swap the binary and
+# units, restart the edge, gate it, roll back to carlos.prev on failure.
+#
+# ExecStart goes through one line of shell so that when a journal is open
+# (a previous apply was cut off) and carlos.prev exists, the RECOVERY runs
+# from carlos.prev rather than from the candidate that did not work.
+#
+# No ProtectSystem, no User=: it must write /opt/carlos and
+# /etc/systemd/system. StartLimitIntervalSec=0 because the path unit can
+# fire in bursts. TimeoutStartSec bounds a hung apply; OnFailure= makes a
+# killed or timed-out apply recover without waiting for a touch.
+[Unit]
+Description=carlos platform self-update (root apply)
+After=network-online.target
+Wants=network-online.target
+StartLimitIntervalSec=0
+OnFailure=carlos-system-recover.service
+
+[Service]
+Type=oneshot
+TimeoutStartSec=15min
+EnvironmentFile=-/etc/carlos/host.env
+ExecStart=/bin/sh -c 'if [ -f /opt/carlos/system/router/journal.json ] && [ -x /opt/carlos/carlos.prev ]; then exec /opt/carlos/carlos.prev system apply; fi; exec /opt/carlos/carlos system apply'
+
+# --- hardening (the carlos-stage.service set) ---
+ProtectHome=yes
+PrivateTmp=yes
+PrivateDevices=yes
+ProtectClock=yes
+ProtectHostname=yes
+ProtectKernelTunables=yes
+ProtectKernelModules=yes
+ProtectKernelLogs=yes
+RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
+RestrictNamespaces=yes
+RestrictRealtime=yes
+LockPersonality=yes
+MemoryDenyWriteExecute=yes
+MemoryMax=256M
+TasksMax=64
+```
+
+`internal/host/units/carlos-system-recover.service`:
+```ini
+# Recovery of an interrupted platform self-update (spec §6, "every run
+# begins with recovery"). Runs once per boot, BEFORE the edge, and on the
+# update oneshot's failure. It restores files only — carlos.prev over
+# carlos, the previous unit set — and never restarts or gates the edge
+# from inside a unit the edge is ordered after: at boot the edge's own
+# start IS the restart. At runtime (edge already started this boot) it
+# issues a non-blocking restart and exits.
+[Unit]
+Description=carlos platform self-update recovery
+Before=carlos-edge.service
+DefaultDependencies=yes
+
+[Service]
+Type=oneshot
+TimeoutStartSec=10min
+EnvironmentFile=-/etc/carlos/host.env
+ExecStart=/bin/sh -c 'if [ -f /opt/carlos/system/router/journal.json ] && [ -x /opt/carlos/carlos.prev ]; then exec /opt/carlos/carlos.prev system apply --recover; fi; exec /opt/carlos/carlos system apply --recover'
+
+ProtectHome=yes
+PrivateTmp=yes
+PrivateDevices=yes
+ProtectClock=yes
+ProtectHostname=yes
+ProtectKernelTunables=yes
+ProtectKernelModules=yes
+ProtectKernelLogs=yes
+RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
+RestrictNamespaces=yes
+RestrictRealtime=yes
+LockPersonality=yes
+MemoryDenyWriteExecute=yes
+MemoryMax=256M
+TasksMax=64
+
+[Install]
+WantedBy=multi-user.target
+```
+
+- [ ] **Step 4: `bootstrap.go` changes**
+
+Replace `enabledUnits` (`:72`):
+```go
+var enabledUnits = []string{"carlos-edge.service", "carlos-adopt.timer", "carlos-adopt.path", "carlos-stage.path", "carlos-restart.path",
+ "carlos-system-update.path", "carlos-system-recover.service"}
+
+// inertUnits are installed everywhere and deliberately never armed by
+// bootstrap or by a self-update apply: restore-verify until an operator
+// stages litestream, hibernate-verify by long-standing choice.
+var inertUnits = []string{"carlos-restore-verify.timer", "carlos-hibernate-verify.timer"}
+
+// bootstrapPlan is what `bootstrap -root <dir> -plan-json` prints: the
+// arming list of THIS binary, so a self-update apply arms what the new
+// binary's bootstrap would arm and never a list an older binary remembers.
+type bootstrapPlan struct {
+ Enabled []string `json:"enabled"`
+ Inert []string `json:"inert"`
+}
+
+func currentBootstrapPlan() bootstrapPlan {
+ return bootstrapPlan{Enabled: append([]string(nil), enabledUnits...), Inert: append([]string(nil), inertUnits...)}
+}
+```
+In `RunBootstrap`, add the flag after `offcloud`:
+```go
+ planJSON := fs.Bool("plan-json", false, "with -root: print the arming plan as JSON on stdout instead of the human summary")
+```
+and pass it through: change `bootstrap(*root, consumers)` to `bootstrapWith(*root, consumers, *planJSON)`, where `bootstrap(root, consumers)` becomes `func bootstrap(root string, consumers []string) error { return bootstrapWith(root, consumers, false) }` and `bootstrapWith` is the old body with the staging print block (`:327-341`) wrapped:
+```go
+ if !live {
+ if planJSON {
+ return json.NewEncoder(os.Stdout).Encode(currentBootstrapPlan())
+ }
+ fmt.Printf("bootstrap: staged under %s — on a real box (--root /) it would also:\n", root)
+ …existing prints…
+ return nil
+ }
+```
+All the `fmt.Printf(" wrote %s\n", path)` / `" created %s\n"` lines in the staging path must go to **stderr** when `planJSON` is set, or the JSON is not alone on stdout: introduce `out := io.Writer(os.Stdout); if planJSON { out = os.Stderr }` at the top of `bootstrapWith` and change those `fmt.Printf` to `fmt.Fprintf(out, …)`. (`-plan-json` with `-root /` is refused: `log.Fatal("bootstrap: -plan-json is for a staging root")`.)
+
+`internal/host/probes.go`, alongside `restartPathActive`:
+```go
+const systemUpdatePathUnit = "carlos-system-update.path"
+
+// systemUpdatePathActive: is the self-update relay armed? nil = unknown.
+func systemUpdatePathActive() *bool {
+ out, _ := exec.Command("systemctl", "is-active", systemUpdatePathUnit).CombinedOutput()
+ return parseIsActive(string(out))
+}
+```
+(copy the exact shape of `restartPathActive` at `probes.go:82`, including how it runs `systemctl`).
+
+- [ ] **Step 5: Run the tests**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/host/ -count=1 2>&1); rc=$?; echo "$out" | tail -30; echo rc=$rc`
+Expected: `ok`. Then, as a real check of the flag: `go run . bootstrap -root /tmp/c/stage -plan-json 2>/dev/null | head -3` prints one JSON object with `enabled` and `inert`.
+
+- [ ] **Step 6: Commit**
+
+```bash
+gofmt -l internal/host/ && git add internal/host/units/carlos-system-update.path internal/host/units/carlos-system-update.service internal/host/units/carlos-system-recover.service internal/host/bootstrap.go internal/host/probes.go internal/host/host_test.go && git commit -q -F - <<'EOF'
+host: the self-update relay, apply and recover units, and a JSON arming plan
+
+bootstrap -root <dir> -plan-json prints this binary's enabledUnits and
+inert list, so an apply arms what the NEW binary's bootstrap would arm.
+Both oneshots exec carlos.prev when a journal is open.
+
+Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
+Claude-Session: https://claude.ai/code/session_015QCUCpfxbCyEAVQ2m51dH5
+EOF
+```
+
+---
+### Task 7: The agent leg (`internal/host/systemleg.go`)
+
+**Files:**
+- Create: `internal/host/systemleg.go`
+- Modify: `internal/host/agent.go` — `fastDeps` (`:1077`), `fastTick` (`:1330`; the serial-move branch `:1352-1367` and the unconditional part before `fastMembers` `:1370`), `fastConvergeLoop` (`:1434-1459`)
+- Test: `internal/host/systemleg_test.go`, `internal/host/agent_test.go` (`fastFixture.deps` at `:1098` may leave the new field nil)
+
+**Interfaces:**
+- Consumes: `release.GetSystemPointer`, `release.FetchToFile`, `release.SystemComponentRouter`, `release.OptDir`, `systemChannel`, `systemRequest`, `readSystemResult`, `readSystemSeen`, `systemCachePath`, `systemRequestPath`, `systemRequestMarker`, `systemRequestEvery`, `systemArtifactLimit`, `validSystemVersion`, `version.Build`.
+- Produces:
+ ```go
+ type systemLeg struct { … }
+ func newSystemLeg(store release.Store, pub ed25519.PublicKey, getenv func(string) string, dataDir, optDir string) *systemLeg
+ func (l *systemLeg) observe(ctx context.Context) // serial move / hourly: read pointer, decide, set pending
+ func (l *systemLeg) tick(ctx context.Context, now time.Time) // every 2 s: fire pending, re-touch, hourly re-observe
+ // fastDeps gains: systemLeg *systemLeg
+ ```
+ Test seams on the struct: `now func() time.Time`, `jitter func(time.Duration) time.Duration`, `build string`, `platform string`.
+
+- [ ] **Step 1: Write the failing tests**
+
+```go
+package host
+
+import (
+ "context"
+ "crypto/sha256"
+ "encoding/hex"
+ "os"
+ "path/filepath"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/carlosframework/platform/internal/release"
+)
+
+type legFixture struct {
+ t *testing.T
+ store *release.DirStore
+ priv ed25519.PrivateKey
+ leg *systemLeg
+ dd string
+ opt string
+ now time.Time
+}
+
+func newLegFixture(t *testing.T, channel, build string) *legFixture {
+ t.Helper()
+ priv, pub, _ := ed25519.GenerateKey(nil)
+ f := &legFixture{t: t, store: release.NewDirStore(t.TempDir()), priv: priv, dd: t.TempDir(), opt: t.TempDir(),
+ now: time.Date(2026, 9, 6, 10, 0, 0, 0, time.UTC)}
+ getenv := func(k string) string {
+ if k == systemChannelEnv {
+ return channel
+ }
+ return ""
+ }
+ f.leg = newSystemLeg(f.store, pub, getenv, f.dd, f.opt)
+ f.leg.build = build
+ f.leg.platform = "linux-arm64"
+ f.leg.now = func() time.Time { return f.now }
+ f.leg.jitter = func(d time.Duration) time.Duration { return d / 2 } // deterministic: half the spread
+ return f
+}
+
+func (f *legFixture) promote(version string, body []byte, spread time.Duration) {
+ f.t.Helper()
+ sum := sha256.Sum256(body)
+ rel := release.SystemRelease{Version: version, Artifacts: map[string]release.SystemArtifact{
+ "linux-arm64": {Key: release.SystemArtifactKey(version, "linux-arm64"), SHA256: hex.EncodeToString(sum[:])}}}
+ if _, err := f.store.Put(context.Background(), rel.Artifacts["linux-arm64"].Key, body, ""); err != nil {
+ f.t.Fatal(err)
+ }
+ if _, err := release.PromoteSystem(context.Background(), f.store, release.RootSigner(f.priv), "router", f.leg.channel, rel,
+ release.SystemPromoteOptions{Spread: spread}); err != nil {
+ f.t.Fatal(err)
+ }
+}
+
+func (f *legFixture) marker() (os.FileInfo, error) { return os.Stat(filepath.Join(f.dd, systemRequestMarker)) }
+
+func (f *legFixture) request() *systemRequest {
+ var r systemRequest
+ if err := readJSONFile(systemRequestPath(f.dd), &r); err != nil {
+ return nil
+ }
+ return &r
+}
+
+func TestSystemLegDoesNothingWithoutAPointer(t *testing.T) {
+ f := newLegFixture(t, "stable", "v1")
+ f.leg.observe(context.Background())
+ f.leg.tick(context.Background(), f.now.Add(time.Hour))
+ if _, err := f.marker(); err == nil {
+ t.Fatal("marker touched with no pointer")
+ }
+}
+
+func TestSystemLegFetchesVerifiesAndRequestsAfterTheSpread(t *testing.T) {
+ f := newLegFixture(t, "stable", "v1")
+ body := []byte("ELF v2")
+ f.promote("v2", body, 10*time.Minute)
+ f.leg.observe(context.Background())
+ if _, err := f.marker(); err == nil {
+ t.Fatal("requested before the spread elapsed")
+ }
+ f.leg.tick(context.Background(), f.now.Add(4*time.Minute)) // jitter = 5m
+ if _, err := f.marker(); err == nil {
+ t.Fatal("requested before the deadline")
+ }
+ f.leg.tick(context.Background(), f.now.Add(5*time.Minute+time.Second))
+ if _, err := f.marker(); err != nil {
+ t.Fatal("marker not touched after the deadline")
+ }
+ got, _ := os.ReadFile(systemCachePath(f.dd, "v2"))
+ if string(got) != "ELF v2" {
+ t.Fatalf("cache = %q", got)
+ }
+ r := f.request()
+ if r == nil || r.Version != "v2" || r.NoOp || r.PromotedAt.IsZero() {
+ t.Fatalf("request = %+v", r)
+ }
+ raw, _ := os.ReadFile(systemRequestPath(f.dd))
+ for _, never := range []string{"channel", "key", "sha", "path"} {
+ if strings.Contains(string(raw), `"`+never+`"`) {
+ t.Errorf("request carries %q — root must not take that from the sandbox", never)
+ }
+ }
+}
+
+func TestSystemLegRetouchesUntilRootAnswers(t *testing.T) {
+ f := newLegFixture(t, "stable", "v1")
+ f.promote("v2", []byte("x"), 0)
+ f.leg.observe(context.Background())
+ f.leg.tick(context.Background(), f.now)
+ first, _ := f.marker()
+ f.leg.tick(context.Background(), f.now.Add(30*time.Second))
+ same, _ := f.marker()
+ if !same.ModTime().Equal(first.ModTime()) {
+ t.Fatal("re-touched inside the minute throttle")
+ }
+ // The touch is a rewrite (PathChanged needs a change), so mtime moves.
+ time.Sleep(20 * time.Millisecond)
+ f.leg.tick(context.Background(), f.now.Add(61*time.Second))
+ later, _ := f.marker()
+ if !later.ModTime().After(first.ModTime()) {
+ t.Fatal("not re-touched after a minute with no result")
+ }
+ // Root answers: a terminal result for this tuple stops the re-touch and
+ // the leg deletes its own cache.
+ r := f.request()
+ os.MkdirAll(systemRootDir(f.opt), 0o755)
+ writeJSONDurable(filepath.Join(systemRootDir(f.opt), "result.json"),
+ systemResult{Channel: "stable", Version: "v2", PromotedAt: r.PromotedAt, Outcome: systemOutcomeRolledBack, Reason: "gate", At: f.now}, 0o644)
+ time.Sleep(20 * time.Millisecond)
+ f.leg.tick(context.Background(), f.now.Add(3*time.Minute))
+ final, _ := f.marker()
+ if !final.ModTime().Equal(later.ModTime()) {
+ t.Fatal("re-touched after a terminal result")
+ }
+ if _, err := os.Stat(systemCachePath(f.dd, "v2")); err == nil {
+ t.Fatal("cache kept after a terminal result")
+ }
+ // And a later observe of the SAME pointer does nothing: refuse to repeat a failure.
+ f.leg.observe(context.Background())
+ f.leg.tick(context.Background(), f.now.Add(10*time.Minute))
+ if f.leg.pending != nil {
+ t.Fatal("re-requested a pointer root already rolled back")
+ }
+}
+
+func TestSystemLegNoOpRequestFetchesNothing(t *testing.T) {
+ f := newLegFixture(t, "stable", "v2")
+ f.promote("v2", []byte("same"), 0)
+ f.leg.observe(context.Background())
+ f.leg.tick(context.Background(), f.now)
+ r := f.request()
+ if r == nil || !r.NoOp {
+ t.Fatalf("request = %+v, want a no-op", r)
+ }
+ if _, err := os.Stat(systemCachePath(f.dd, "v2")); err == nil {
+ t.Fatal("a no-op must not download the artifact")
+ }
+ // Once root has seen it (seen.json advanced to this PromotedAt), nothing more.
+ os.MkdirAll(systemRootDir(f.opt), 0o755)
+ writeJSONDurable(filepath.Join(systemRootDir(f.opt), "seen.json"), systemSeen{"stable": r.PromotedAt}, 0o644)
+ f.leg.pending = nil
+ os.Remove(filepath.Join(f.dd, systemRequestMarker))
+ f.leg.observe(context.Background())
+ f.leg.tick(context.Background(), f.now.Add(2*time.Minute))
+ if _, err := f.marker(); err == nil {
+ t.Fatal("requested again after root recorded the watermark")
+ }
+}
+
+func TestSystemLegIgnoresDevBuildEqualityAndOtherPlatforms(t *testing.T) {
+ f := newLegFixture(t, "stable", "dev")
+ f.promote("v2", []byte("x"), 0)
+ f.leg.platform = "linux-amd64" // no artifact for me
+ f.leg.observe(context.Background())
+ f.leg.tick(context.Background(), f.now)
+ if _, err := f.marker(); err == nil {
+ t.Fatal("requested with no artifact for this platform")
+ }
+}
+
+func TestSystemLegRefusesShaMismatchAndOversize(t *testing.T) {
+ f := newLegFixture(t, "stable", "v1")
+ f.promote("v2", []byte("good"), 0)
+ // Swap the object under the signed sha.
+ f.store.Put(context.Background(), release.SystemArtifactKey("v2", "linux-arm64"), []byte("evil"), "")
+ f.leg.observe(context.Background())
+ f.leg.tick(context.Background(), f.now)
+ if _, err := f.marker(); err == nil {
+ t.Fatal("requested an artifact whose sha does not match")
+ }
+ if _, err := os.Stat(systemCachePath(f.dd, "v2")); err == nil {
+ t.Fatal("mismatched artifact left in the cache")
+ }
+}
+
+func TestSystemLegIgnoresAPointerOlderThanTheWatermark(t *testing.T) {
+ f := newLegFixture(t, "stable", "v1")
+ f.promote("v2", []byte("x"), 0)
+ os.MkdirAll(systemRootDir(f.opt), 0o755)
+ writeJSONDurable(filepath.Join(systemRootDir(f.opt), "seen.json"), systemSeen{"stable": time.Now().Add(time.Hour)}, 0o644)
+ f.leg.observe(context.Background())
+ f.leg.tick(context.Background(), f.now)
+ if _, err := f.marker(); err == nil {
+ t.Fatal("requested a pointer root would refuse as a replay")
+ }
+}
+
+func TestFastTickRunsTheSystemLeg(t *testing.T) {
+ fx := newFastFixture(t)
+ d := fx.deps(nil)
+ observes := 0
+ d.systemLeg = &systemLeg{observeHook: func() { observes++ }}
+ ctx := context.Background()
+ fastTick(ctx, d, time.Now()) // boot seed
+ if observes != 0 {
+ t.Fatalf("boot seed observed %d times", observes)
+ }
+ release.BumpSerial(ctx, fx.store, time.Now())
+ fastTick(ctx, d, time.Now().Add(10*time.Second))
+ if observes != 1 {
+ t.Fatalf("one serial move → %d observes", observes)
+ }
+}
+```
+(`observeHook func()` is a test seam on `systemLeg`: when non-nil, `observe` calls it and returns. Add `"crypto/ed25519"` to the imports.)
+
+- [ ] **Step 2: Run to verify failure**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/host/ -run 'TestSystemLeg|TestFastTickRunsTheSystemLeg' -count=1 2>&1); rc=$?; echo "$out" | tail -20; echo rc=$rc`
+Expected: undefined `systemLeg`, `newSystemLeg`.
+
+- [ ] **Step 3: Implement `internal/host/systemleg.go`**
+
+```go
+package host
+
+import (
+ "context"
+ "crypto/ed25519"
+ "errors"
+ "log"
+ "math/rand/v2"
+ "os"
+ "path/filepath"
+ "runtime"
+ "time"
+
+ "github.com/carlosframework/platform/internal/release"
+ "github.com/carlosframework/platform/internal/version"
+)
+
+// systemLeg is the sandboxed half of the platform self-update (spec §4):
+// notice the channel pointer, fetch and verify the artifact into /data, ask
+// root. Two entry points so the spread has something to run it: observe on
+// a serial move (and hourly), tick every 2 s with no store read.
+type systemLeg struct {
+ store release.Store
+ pub ed25519.PublicKey
+ channel string
+ platform string
+ build string
+ dataDir string
+ optDir string
+
+ now func() time.Time
+ jitter func(time.Duration) time.Duration
+
+ pending *systemPending
+ lastTouch time.Time
+ lastObserve time.Time
+ backoffUntil time.Time
+ backoff time.Duration
+
+ observeHook func() // test seam
+}
+
+type systemPending struct {
+ version string
+ promotedAt time.Time
+ noop bool
+ artifact release.SystemArtifact
+ deadline time.Time
+ requested bool
+}
+
+const systemObserveEvery = time.Hour
+
+func newSystemLeg(store release.Store, pub ed25519.PublicKey, getenv func(string) string, dataDir, optDir string) *systemLeg {
+ return &systemLeg{
+ store: store, pub: pub, channel: systemChannel(getenv),
+ platform: runtime.GOOS + "-" + runtime.GOARCH, build: version.Build,
+ dataDir: dataDir, optDir: optDir,
+ now: time.Now,
+ jitter: func(spread time.Duration) time.Duration { return rand.N(spread + 1) },
+ }
+}
+
+// observe reads the pointer and decides. It never blocks on a download:
+// that happens in tick, after the spread.
+func (l *systemLeg) observe(ctx context.Context) {
+ if l.observeHook != nil {
+ l.observeHook()
+ return
+ }
+ now := l.now()
+ l.lastObserve = now
+ p, _, err := release.GetSystemPointer(ctx, l.store, l.pub, release.SystemComponentRouter, l.channel)
+ if err != nil {
+ if !errors.Is(err, os.ErrNotExist) {
+ log.Printf("agent: system pointer %s: %v", l.channel, err)
+ }
+ return
+ }
+ seen, _ := readSystemSeen(l.optDir)
+ if wm, ok := seen[l.channel]; ok && p.PromotedAt.Before(wm) {
+ return // root would refuse it as a replay
+ }
+ if res, _ := readSystemResult(l.optDir); res.matches(l.channel, p.Current.Version, p.PromotedAt) && res.terminal() {
+ l.forget(p.Current.Version)
+ return // refuse to repeat a failure
+ }
+ noop := p.Current.Version == l.build
+ if noop {
+ if wm, ok := seen[l.channel]; ok && !p.PromotedAt.After(wm) {
+ return // nothing to do, and root has seen this promote
+ }
+ }
+ art, ok := p.Current.Artifacts[l.platform]
+ if !ok && !noop {
+ if l.pending == nil || l.pending.version != p.Current.Version {
+ log.Printf("agent: system pointer %s names %s with no artifact for %s — not for this box", l.channel, p.Current.Version, l.platform)
+ }
+ return
+ }
+ if l.pending != nil && l.pending.version == p.Current.Version && l.pending.promotedAt.Equal(p.PromotedAt) {
+ return // already pending; keep its deadline
+ }
+ l.pending = &systemPending{version: p.Current.Version, promotedAt: p.PromotedAt, noop: noop, artifact: art,
+ deadline: now.Add(l.jitter(p.SpreadDuration()))}
+ if noop {
+ l.pending.deadline = now
+ }
+}
+
+// tick fires a pending request whose deadline has passed and re-touches an
+// unanswered one every minute. It reads no store except for the one fetch.
+func (l *systemLeg) tick(ctx context.Context, now time.Time) {
+ if now.Sub(l.lastObserve) >= systemObserveEvery {
+ l.observe(ctx)
+ }
+ p := l.pending
+ if p == nil {
+ return
+ }
+ if res, _ := readSystemResult(l.optDir); res.matches(l.channel, p.version, p.promotedAt) {
+ switch {
+ case res.terminal():
+ l.forget(p.version)
+ return
+ case res.Outcome == systemOutcomeNoted:
+ l.pending = nil
+ return
+ case res.Outcome == systemOutcomeDeferred && res.At.After(p.deadline):
+ // root could not reach the store; try again with backoff
+ if l.backoff == 0 {
+ l.backoff = fastBackoffBase
+ } else if l.backoff < fastBackoffCap {
+ l.backoff *= 2
+ }
+ p.deadline = now.Add(l.backoff)
+ p.requested = false
+ }
+ }
+ if now.Before(p.deadline) || now.Before(l.backoffUntil) {
+ return
+ }
+ if !p.requested {
+ if !p.noop {
+ if err := l.fetch(ctx, p); err != nil {
+ log.Printf("agent: system update %s: %v", p.version, err)
+ os.Remove(systemCachePath(l.dataDir, p.version))
+ l.backoffUntil = now.Add(fastBackoffBase)
+ return
+ }
+ }
+ if err := l.writeRequest(p, now); err != nil {
+ log.Printf("agent: system update %s: request: %v", p.version, err)
+ return
+ }
+ p.requested = true
+ }
+ if now.Sub(l.lastTouch) >= systemRequestEvery {
+ if err := os.WriteFile(filepath.Join(l.dataDir, systemRequestMarker), []byte(now.UTC().Format(time.RFC3339Nano)+"\n"), 0o644); err != nil {
+ log.Printf("agent: system update %s: cannot ask for a root apply: %v", p.version, err)
+ return
+ }
+ l.lastTouch = now
+ }
+}
+
+func (l *systemLeg) fetch(ctx context.Context, p *systemPending) error {
+ dst := systemCachePath(l.dataDir, p.version)
+ if err := os.MkdirAll(filepath.Dir(dst), 0o755); err != nil {
+ return err
+ }
+ return release.FetchToFile(ctx, l.store, p.artifact.Key, dst, systemArtifactLimit, p.artifact.SHA256)
+}
+
+func (l *systemLeg) writeRequest(p *systemPending, now time.Time) error {
+ if err := os.MkdirAll(systemDataDir(l.dataDir), 0o755); err != nil {
+ return err
+ }
+ return writeJSONDurable(systemRequestPath(l.dataDir), systemRequest{Version: p.version, PromotedAt: p.promotedAt, NoOp: p.noop, SeenAt: now.UTC()}, 0o644)
+}
+
+// forget: root has answered with a terminal outcome. Drop the pending item
+// and the cache — root never deletes under /data, the agent does.
+func (l *systemLeg) forget(version string) {
+ l.pending = nil
+ l.backoff = 0
+ if validSystemVersion(version) {
+ os.RemoveAll(filepath.Dir(systemCachePath(l.dataDir, version)))
+ }
+}
+```
+
+`agent.go` wiring:
+- `fastDeps` gains `systemLeg *systemLeg // the platform self-update leg (spec §4); nil = not armed`.
+- In `fastTick`, inside the serial-move branch after the `scheduleLeg` call (`:1365`): `if d.systemLeg != nil { d.systemLeg.observe(ctx) }`.
+- Before `members := d.fastMembers(now)` (`:1370`), outside the switch: `if d.systemLeg != nil { d.systemLeg.tick(ctx, now) }`.
+- In `fastConvergeLoop`, after `d.scheduleLeg = scheduleLegFor(sd)` (`:1506`): `d.systemLeg = newSystemLeg(store, pub, os.Getenv, dd, optDir)` and extend the "armed" log line with `+ system pointer`.
+- The `adoptLoop` doc comment's DO-NOT list (`:566-600`) gets one sentence: the system leg lives in the fast loop and re-observes hourly on its own; nothing is added here.
+
+- [ ] **Step 4: Run the tests**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/host/ -count=1 2>&1); rc=$?; echo "$out" | tail -20; echo rc=$rc`
+Expected: `ok`.
+
+- [ ] **Step 5: Commit**
+
+```bash
+gofmt -l internal/host/ && git add internal/host/systemleg.go internal/host/systemleg_test.go internal/host/agent.go internal/host/agent_test.go && git commit -q -F - <<'EOF'
+host: the agent notices a platform promote, fetches, verifies, and asks root
+
+observe on a serial move (and hourly), tick every 2 s: spread jitter,
+streamed fetch with the signed sha, a request carrying only version
+and promoted_at, and a marker re-touched every minute until root
+answers. A pointer root already rolled back is never re-requested.
+
+Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
+Claude-Session: https://claude.ai/code/session_015QCUCpfxbCyEAVQ2m51dH5
+EOF
+```
+
+---
+### Task 8a: `carlos system apply` — the refuse path (steps 1–5, no-op, lock)
+
+**Files:**
+- Create: `internal/host/systemapply.go`
+- Test: `internal/host/systemapply_test.go`
+
+**Interfaces:**
+- Consumes: Task 4 state helpers; `release.GetSystemPointer`; `release.SystemComponentRouter`; `systemChannel`.
+- Produces (package `host`):
+ ```go
+ type sysOps struct {
+ systemctl func(args ...string) (string, error)
+ run func(ctx context.Context, timeout time.Duration, name string, args ...string) (string, error)
+ now func() time.Time
+ sleep func(time.Duration)
+ probeEdge func() error
+ edgeStartedThisBoot func() bool
+ logf func(format string, args ...any)
+ }
+ type systemApplier struct {
+ ops sysOps
+ store release.Store
+ pub ed25519.PublicKey
+ channel string
+ optDir, dataDir string
+ etcDir string // "/etc" in production; a temp dir in tests
+ build string // version.Build of THIS process
+ gateWait, soak time.Duration // 20s, 15s in production
+ }
+ func (a *systemApplier) Run(ctx context.Context, recover bool) error
+ const (
+ systemQuiesceUnits = "carlos-adopt.timer carlos-adopt.path carlos-stage.path carlos-restart.path carlos-adopt.service carlos-config-sync.service carlos-stage.service carlos-restart.service"
+ systemEdgeUnit = "carlos-edge"
+ )
+ func (a *systemApplier) rootDir() string; stagingDir() string; lockPath() string
+ func (a *systemApplier) writeResult(req systemRequest, outcome, reason string) error
+ func (a *systemApplier) readRequest() (systemRequest, error) // O_NOFOLLOW walk, 4 KiB cap, validSystemVersion
+ func (a *systemApplier) verifyPointer(ctx) (*release.SystemPointer, error)
+ func (a *systemApplier) stageCandidate(p *release.SystemPointer, req systemRequest) (string, error) // path of staging/carlos
+ func openBeneath(root string, parts ...string) (*os.File, error) // per-component O_DIRECTORY|O_NOFOLLOW, then O_NOFOLLOW
+ var errSystemRefused = errors.New("refused"); var errSystemDeferred = errors.New("deferred")
+ ```
+ In this task `Run` handles: lock → (recovery: Task 8c, stub as "journal present → return error") → read request → verify pointer + replay guard → advance seen → no-op → stage candidate → **stop with `errNotYetImplemented`** (Task 8b continues). `Run` returns nil on `noted`/`refused`/`deferred` after writing `result.json`; exit codes are `RunSystemApply`'s business (8c).
+
+- [ ] **Step 1: Write the failing tests**
+
+```go
+package host
+
+import (
+ "context"
+ "crypto/ed25519"
+ "crypto/sha256"
+ "encoding/hex"
+ "errors"
+ "os"
+ "path/filepath"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/carlosframework/platform/internal/release"
+)
+
+// applyFixture is a fake box: temp opt/data/etc roots, a DirStore with a
+// signed pointer, and a sysOps that records every call and answers like a
+// healthy systemd. The candidate "binary" is answered by the fake run.
+type applyFixture struct {
+ t *testing.T
+ a *systemApplier
+ store *release.DirStore
+ priv ed25519.PrivateKey
+ calls []string // "systemctl is-active carlos-edge", "run <base> version", …
+ answer map[string]string // "version:<path>" → stdout; "plan:<path>" → JSON plan
+ now time.Time
+ edgeUp bool
+ nrestarts string
+ activeEnter string
+ probeErr error
+ edgeStartedThisBoot bool
+}
+
+func newApplyFixture(t *testing.T, channel, build string) *applyFixture {
+ t.Helper()
+ priv, pub, _ := ed25519.GenerateKey(nil)
+ f := &applyFixture{t: t, store: release.NewDirStore(t.TempDir()), priv: priv, answer: map[string]string{},
+ now: time.Date(2026, 9, 6, 11, 0, 0, 0, time.UTC), edgeUp: true, nrestarts: "0", activeEnter: "1000", edgeStartedThisBoot: true}
+ f.a = &systemApplier{
+ store: f.store, pub: pub, channel: channel,
+ optDir: t.TempDir(), dataDir: t.TempDir(), etcDir: t.TempDir(), build: build,
+ }
+ f.a.ops = sysOps{
+ systemctl: func(args ...string) (string, error) {
+ f.calls = append(f.calls, "systemctl "+strings.Join(args, " "))
+ switch {
+ case args[0] == "is-active" && args[1] == systemEdgeUnit:
+ if f.edgeUp {
+ return "active\n", nil
+ }
+ return "failed\n", errors.New("inactive")
+ case args[0] == "show":
+ return f.nrestarts + "\n" + f.activeEnter + "\n", nil
+ }
+ return "", nil
+ },
+ run: func(ctx context.Context, timeout time.Duration, name string, args ...string) (string, error) {
+ f.calls = append(f.calls, "run "+filepath.Base(name)+" "+strings.Join(args, " "))
+ if len(args) == 1 && args[0] == "version" {
+ if v, ok := f.answer["version:"+name]; ok {
+ return v, nil
+ }
+ return "", errors.New("no such binary")
+ }
+ if len(args) >= 3 && args[0] == "bootstrap" && args[1] == "-root" {
+ plan, ok := f.answer["plan:"+name]
+ if !ok {
+ return "", errors.New("bootstrap failed")
+ }
+ units := strings.Split(f.answer["units:"+name], " ")
+ dir := filepath.Join(args[2], "etc", "systemd", "system")
+ os.MkdirAll(dir, 0o755)
+ for _, u := range units {
+ if u != "" {
+ os.WriteFile(filepath.Join(dir, u), []byte("[Unit]\n# "+filepath.Base(name)+"\n"), 0o644)
+ }
+ }
+ os.MkdirAll(filepath.Join(args[2], "etc", "sysusers.d"), 0o755)
+ os.WriteFile(filepath.Join(args[2], "etc", "sysusers.d", "carlos.conf"), []byte("g carlos-sockets -\n"), 0o644)
+ os.MkdirAll(filepath.Join(args[2], "etc", "tmpfiles.d"), 0o755)
+ os.WriteFile(filepath.Join(args[2], "etc", "tmpfiles.d", "carlos.conf"), []byte("d /run/carlos\n"), 0o644)
+ os.MkdirAll(filepath.Join(args[2], "data"), 0o755)
+ os.WriteFile(filepath.Join(args[2], "data", "registry.db"), nil, 0o640)
+ return plan, nil
+ }
+ return "", errors.New("unexpected command")
+ },
+ now: func() time.Time { return f.now },
+ sleep: func(d time.Duration) { f.now = f.now.Add(d) },
+ probeEdge: func() error { return f.probeErr },
+ edgeStartedThisBoot: func() bool { return f.edgeStartedThisBoot },
+ logf: t.Logf,
+ }
+ os.MkdirAll(systemRootDir(f.a.optDir), 0o755)
+ os.MkdirAll(filepath.Join(f.a.etcDir, "systemd", "system"), 0o755)
+ // The on-disk binary answers `version` with build.
+ os.WriteFile(systemBinaryPath(f.a.optDir), []byte("#!/bin/sh\necho carlos "+build+"\n"), 0o755)
+ f.answer["version:"+systemBinaryPath(f.a.optDir)] = "carlos " + build + "\n"
+ f.answer["plan:"+systemBinaryPath(f.a.optDir)] = `{"enabled":["carlos-edge.service","carlos-adopt.timer","carlos-adopt.path","carlos-stage.path","carlos-restart.path","carlos-system-update.path","carlos-system-recover.service"],"inert":["carlos-restore-verify.timer"]}`
+ f.answer["units:"+systemBinaryPath(f.a.optDir)] = "carlos-edge.service carlos-adopt.timer carlos-adopt.path carlos-stage.path carlos-restart.path carlos-system-update.path carlos-system-update.service carlos-system-recover.service carlos-old-only.timer"
+ return f
+}
+
+// promote signs a pointer for f.a.channel naming version with body as the
+// linux-arm64 artifact, stages body into the agent's cache, and writes the
+// request the agent would have written. Returns the request.
+func (f *applyFixture) promote(version string, body []byte, noop bool) systemRequest {
+ f.t.Helper()
+ sum := sha256.Sum256(body)
+ rel := release.SystemRelease{Version: version, Artifacts: map[string]release.SystemArtifact{
+ "linux-arm64": {Key: release.SystemArtifactKey(version, "linux-arm64"), SHA256: hex.EncodeToString(sum[:])}}}
+ f.store.Put(context.Background(), rel.Artifacts["linux-arm64"].Key, body, "")
+ p, err := release.PromoteSystem(context.Background(), f.store, release.RootSigner(f.priv), "router", f.a.channel, rel, release.SystemPromoteOptions{})
+ if err != nil {
+ f.t.Fatal(err)
+ }
+ req := systemRequest{Version: version, PromotedAt: p.PromotedAt, NoOp: noop, SeenAt: f.now}
+ os.MkdirAll(systemDataDir(f.a.dataDir), 0o755)
+ writeJSONDurable(systemRequestPath(f.a.dataDir), req, 0o644)
+ if !noop {
+ os.MkdirAll(filepath.Dir(systemCachePath(f.a.dataDir, version)), 0o755)
+ os.WriteFile(systemCachePath(f.a.dataDir, version), body, 0o644)
+ }
+ return req
+}
+
+func (f *applyFixture) result() *systemResult { r, _ := readSystemResult(f.a.optDir); return r }
+
+func (f *applyFixture) called(prefix string) bool {
+ for _, c := range f.calls {
+ if strings.HasPrefix(c, prefix) {
+ return true
+ }
+ }
+ return false
+}
+
+func TestApplyNoRequestIsNothingToDo(t *testing.T) {
+ f := newApplyFixture(t, "stable", "v1")
+ if err := f.a.Run(context.Background(), false); err != nil {
+ t.Fatal(err)
+ }
+ if f.result() != nil || len(f.calls) != 0 {
+ t.Fatalf("result=%+v calls=%v", f.result(), f.calls)
+ }
+}
+
+func TestApplyLockKeepsTwoRunsApart(t *testing.T) {
+ f := newApplyFixture(t, "stable", "v1")
+ f.promote("v2", []byte("x"), false)
+ // Hold the lock as another process would.
+ held, err := os.OpenFile(f.a.lockPath(), os.O_CREATE|os.O_RDWR, 0o600)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer held.Close()
+ if err := flockExclusiveNonblock(held); err != nil {
+ t.Fatal(err)
+ }
+ if err := f.a.Run(context.Background(), false); err != nil {
+ t.Fatalf("a locked-out path run must exit 0 quietly: %v", err)
+ }
+ if f.result() != nil {
+ t.Fatal("a locked-out run wrote a result")
+ }
+}
+
+func TestApplyRequestIsAHintNotAnInstruction(t *testing.T) {
+ f := newApplyFixture(t, "stable", "v1")
+ // A symlinked parent under /data.
+ os.MkdirAll(filepath.Join(f.a.dataDir, "elsewhere"), 0o755)
+ os.Symlink(filepath.Join(f.a.dataDir, "elsewhere"), filepath.Join(f.a.dataDir, "system"))
+ os.MkdirAll(filepath.Join(f.a.dataDir, "elsewhere", "router"), 0o755)
+ os.WriteFile(filepath.Join(f.a.dataDir, "elsewhere", "router", "requested.json"), []byte(`{"version":"v2"}`), 0o644)
+ if _, err := f.a.readRequest(); err == nil {
+ t.Fatal("followed a symlinked parent under /data")
+ }
+ os.Remove(filepath.Join(f.a.dataDir, "system"))
+ // A symlinked request file.
+ os.MkdirAll(systemDataDir(f.a.dataDir), 0o755)
+ os.WriteFile(filepath.Join(f.a.dataDir, "real.json"), []byte(`{"version":"v2"}`), 0o644)
+ os.Symlink(filepath.Join(f.a.dataDir, "real.json"), systemRequestPath(f.a.dataDir))
+ if _, err := f.a.readRequest(); err == nil {
+ t.Fatal("followed a symlinked request")
+ }
+ os.Remove(systemRequestPath(f.a.dataDir))
+ // A traversal version.
+ writeJSONDurable(systemRequestPath(f.a.dataDir), systemRequest{Version: "../../etc/passwd", PromotedAt: f.now}, 0o644)
+ if _, err := f.a.readRequest(); err == nil {
+ t.Fatal("accepted a traversal version")
+ }
+ // Oversized.
+ os.WriteFile(systemRequestPath(f.a.dataDir), []byte(`{"version":"`+strings.Repeat("a", 5000)+`"}`), 0o644)
+ if _, err := f.a.readRequest(); err == nil {
+ t.Fatal("accepted a 5 KiB request")
+ }
+}
+
+func TestApplyChannelComesFromRootNotTheRequest(t *testing.T) {
+ // The box is on stable; a valid CANARY pointer exists and the agent
+ // requests its version. Root reads control/system/router/stable.json
+ // (missing) and refuses — a signed canary promote does not authorise a
+ // stable box.
+ f := newApplyFixture(t, "stable", "v1")
+ f.a.channel = "canary"
+ req := f.promote("v2", []byte("x"), false) // signs the CANARY pointer
+ f.a.channel = "stable"
+ if err := f.a.Run(context.Background(), false); err != nil {
+ t.Fatal(err)
+ }
+ r := f.result()
+ if r == nil || r.Outcome != systemOutcomeRefused || !r.matches("stable", "v2", req.PromotedAt) {
+ t.Fatalf("result = %+v", r)
+ }
+ if _, err := os.Stat(systemBinaryPath(f.a.optDir) + ".prev"); err == nil {
+ t.Fatal("touched the box")
+ }
+}
+
+func TestApplyStaleTupleIsRefusedAgainstTheRequestsTuple(t *testing.T) {
+ f := newApplyFixture(t, "stable", "v1")
+ old := f.promote("v2", []byte("x"), false)
+ time.Sleep(1100 * time.Millisecond)
+ f.promote("v3", []byte("y"), false)
+ // Put the OLD request back: the agent's spread started under v2.
+ writeJSONDurable(systemRequestPath(f.a.dataDir), old, 0o644)
+ if err := f.a.Run(context.Background(), false); err != nil {
+ t.Fatal(err)
+ }
+ r := f.result()
+ if r == nil || r.Outcome != systemOutcomeRefused || r.Version != "v2" || !r.PromotedAt.Equal(old.PromotedAt) {
+ t.Fatalf("result = %+v (must name the REQUEST's tuple so v3 is not suppressed)", r)
+ }
+}
+
+func TestApplyReplayGuardIsPerChannelAndEqualIsAllowed(t *testing.T) {
+ f := newApplyFixture(t, "stable", "v1")
+ req := f.promote("v2", []byte("x"), true)
+ // Older than the watermark → refused, seen unchanged.
+ writeJSONDurable(filepath.Join(systemRootDir(f.a.optDir), "seen.json"), systemSeen{"stable": req.PromotedAt.Add(time.Hour), "canary": time.Time{}}, 0o644)
+ if err := f.a.Run(context.Background(), false); err != nil {
+ t.Fatal(err)
+ }
+ if r := f.result(); r == nil || r.Outcome != systemOutcomeRefused || !strings.Contains(r.Reason, "replay") {
+ t.Fatalf("result = %+v", r)
+ }
+ // Equal → allowed (a retry of the same promote).
+ writeJSONDurable(filepath.Join(systemRootDir(f.a.optDir), "seen.json"), systemSeen{"stable": req.PromotedAt}, 0o644)
+ if err := f.a.Run(context.Background(), false); err != nil {
+ t.Fatal(err)
+ }
+ if r := f.result(); r == nil || r.Outcome != systemOutcomeNoted {
+ t.Fatalf("result = %+v", r)
+ }
+ // Another channel's newer watermark does not block this channel.
+ writeJSONDurable(filepath.Join(systemRootDir(f.a.optDir), "seen.json"), systemSeen{"canary": req.PromotedAt.Add(time.Hour)}, 0o644)
+ if err := f.a.Run(context.Background(), false); err != nil {
+ t.Fatal(err)
+ }
+ if r := f.result(); r == nil || r.Outcome != systemOutcomeNoted {
+ t.Fatalf("result = %+v", r)
+ }
+ seen, _ := readSystemSeen(f.a.optDir)
+ if !seen["stable"].Equal(req.PromotedAt) {
+ t.Fatalf("seen not advanced: %v", seen)
+ }
+}
+
+func TestApplyNoOpNotesAndTouchesNothing(t *testing.T) {
+ f := newApplyFixture(t, "stable", "v2")
+ f.promote("v2", []byte("x"), true)
+ if err := f.a.Run(context.Background(), false); err != nil {
+ t.Fatal(err)
+ }
+ if r := f.result(); r == nil || r.Outcome != systemOutcomeNoted {
+ t.Fatalf("result = %+v", r)
+ }
+ if _, err := os.Stat(f.a.stagingDir()); err == nil {
+ t.Fatal("a no-op created staging/")
+ }
+ if len(f.calls) != 0 {
+ t.Fatalf("a no-op ran commands: %v", f.calls)
+ }
+}
+
+func TestApplyStoreErrorIsDeferredNotRefused(t *testing.T) {
+ f := newApplyFixture(t, "stable", "v1")
+ req := f.promote("v2", []byte("x"), false)
+ f.a.store = failingStore{f.store}
+ if err := f.a.Run(context.Background(), false); err != nil {
+ t.Fatal(err)
+ }
+ if r := f.result(); r == nil || r.Outcome != systemOutcomeDeferred || !r.matches("stable", "v2", req.PromotedAt) {
+ t.Fatalf("result = %+v", r)
+ }
+}
+
+type failingStore struct{ release.Store }
+
+func (failingStore) Get(context.Context, string) ([]byte, string, error) {
+ return nil, "", errors.New("door restarting")
+}
+
+func TestApplyStagesTheCandidateFromARootOwnedCopy(t *testing.T) {
+ f := newApplyFixture(t, "stable", "v1")
+ body := []byte("#!/bin/sh\necho carlos v2\n")
+ f.promote("v2", body, false)
+ f.answer["version:"+filepath.Join(f.a.stagingDir(), "carlos")] = "carlos v2\n"
+ p, err := f.a.verifyPointer(context.Background())
+ if err != nil {
+ t.Fatal(err)
+ }
+ req, _ := f.a.readRequest()
+ staged, err := f.a.stageCandidate(p, req)
+ if err != nil {
+ t.Fatal(err)
+ }
+ st, _ := os.Stat(staged)
+ if st.Mode().Perm() != 0o755 {
+ t.Fatalf("staged mode = %v after verification", st.Mode().Perm())
+ }
+ if dst, _ := os.Stat(f.a.stagingDir()); dst.Mode().Perm() != 0o700 {
+ t.Fatalf("staging dir mode = %v", dst.Mode().Perm())
+ }
+ // Swapping the cache after staging changes nothing about the copy.
+ os.WriteFile(systemCachePath(f.a.dataDir, "v2"), []byte("evil"), 0o644)
+ got, _ := os.ReadFile(staged)
+ if string(got) != string(body) {
+ t.Fatal("staged copy follows the cache")
+ }
+ // A sha mismatch is refused BEFORE the copy is made executable and
+ // staging/ is cleared.
+ f2 := newApplyFixture(t, "stable", "v1")
+ f2.promote("v2", body, false)
+ os.WriteFile(systemCachePath(f2.a.dataDir, "v2"), []byte("evil"), 0o644)
+ p2, _ := f2.a.verifyPointer(context.Background())
+ req2, _ := f2.a.readRequest()
+ if _, err := f2.a.stageCandidate(p2, req2); err == nil || !strings.Contains(err.Error(), "sha256") {
+ t.Fatalf("err = %v", err)
+ }
+ // A candidate that lies about its version is refused.
+ f3 := newApplyFixture(t, "stable", "v1")
+ f3.promote("v2", body, false)
+ f3.answer["version:"+filepath.Join(f3.a.stagingDir(), "carlos")] = "carlos v9\n"
+ p3, _ := f3.a.verifyPointer(context.Background())
+ req3, _ := f3.a.readRequest()
+ if _, err := f3.a.stageCandidate(p3, req3); err == nil || !strings.Contains(err.Error(), "reports") {
+ t.Fatalf("err = %v", err)
+ }
+ // A symlinked cache file is refused.
+ f4 := newApplyFixture(t, "stable", "v1")
+ f4.promote("v2", body, false)
+ os.Remove(systemCachePath(f4.a.dataDir, "v2"))
+ os.WriteFile(filepath.Join(f4.a.dataDir, "real"), body, 0o644)
+ os.Symlink(filepath.Join(f4.a.dataDir, "real"), systemCachePath(f4.a.dataDir, "v2"))
+ p4, _ := f4.a.verifyPointer(context.Background())
+ req4, _ := f4.a.readRequest()
+ if _, err := f4.a.stageCandidate(p4, req4); err == nil {
+ t.Fatal("followed a symlinked cache file")
+ }
+}
+```
+
+- [ ] **Step 2: Run to verify failure**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/host/ -run 'TestApply' -count=1 2>&1); rc=$?; echo "$out" | tail -20; echo rc=$rc`
+Expected: undefined `systemApplier`, `sysOps`, `flockExclusiveNonblock`.
+
+- [ ] **Step 3: Implement the refuse path in `internal/host/systemapply.go`**
+
+```go
+package host
+
+import (
+ "context"
+ "crypto/ed25519"
+ "crypto/sha256"
+ "encoding/hex"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io"
+ "os"
+ "path/filepath"
+ "strings"
+ "syscall"
+ "time"
+
+ "github.com/carlosframework/platform/internal/release"
+)
+
+// The root half of the platform self-update (spec §6). Runs as root from
+// carlos-system-update.service (or --recover from carlos-system-recover).
+// Every box-changing step is journaled first; every run begins with
+// recovery; root reads ONE file under /data and treats it as a hint.
+
+const (
+ systemEdgeUnit = "carlos-edge"
+ systemLockFile = "lock"
+ systemMaxRequest = 4 << 10
+ systemGateWait = 20 * time.Second
+ systemGateSoak = 15 * time.Second
+ systemVersionWait = 30 * time.Second
+ systemBootstrapWait = 120 * time.Second
+)
+
+// systemQuiesceUnits are the root units that exec /opt/carlos/carlos
+// against the running edge. Stopped (services too — stopping a timer does
+// not stop the pass it already started) before the rename, re-armed after
+// the gate.
+var systemQuiesceUnits = []string{
+ "carlos-adopt.timer", "carlos-adopt.path", "carlos-stage.path", "carlos-restart.path",
+ "carlos-adopt.service", "carlos-config-sync.service", "carlos-stage.service", "carlos-restart.service",
+}
+
+var (
+ errSystemRefused = errors.New("refused")
+ errSystemDeferred = errors.New("deferred")
+ errNotYetImplemented = errors.New("system apply: install path not implemented")
+)
+
+// sysOps is the seam between the apply and the box: systemctl, exec with a
+// deadline, the clock, sleep, the edge probe. Tests record and answer.
+type sysOps struct {
+ systemctl func(args ...string) (string, error)
+ run func(ctx context.Context, timeout time.Duration, name string, args ...string) (string, error)
+ now func() time.Time
+ sleep func(time.Duration)
+ probeEdge func() error
+ edgeStartedThisBoot func() bool
+ logf func(format string, args ...any)
+}
+
+type systemApplier struct {
+ ops sysOps
+ store release.Store
+ pub ed25519.PublicKey
+ channel string
+ optDir string
+ dataDir string
+ etcDir string
+ build string
+ gateWait, soak time.Duration
+}
+
+func (a *systemApplier) rootDir() string { return systemRootDir(a.optDir) }
+func (a *systemApplier) stagingDir() string { return filepath.Join(a.rootDir(), "staging") }
+func (a *systemApplier) lockPath() string { return filepath.Join(a.rootDir(), systemLockFile) }
+func (a *systemApplier) file(name string) string { return filepath.Join(a.rootDir(), name) }
+
+// Run is the whole oneshot. It returns an error only for a failure the
+// caller should exit non-zero on (an apply that rolled back, a rollback
+// that failed, a broken box); refusals and no-ops are recorded in
+// result.json and return nil.
+func (a *systemApplier) Run(ctx context.Context, recover bool) error {
+ if err := os.MkdirAll(a.rootDir(), 0o755); err != nil {
+ return err
+ }
+ lock, err := os.OpenFile(a.lockPath(), os.O_CREATE|os.O_RDWR, 0o600)
+ if err != nil {
+ return err
+ }
+ defer lock.Close()
+ if recover {
+ if err := flockExclusive(lock); err != nil {
+ return err
+ }
+ } else if err := flockExclusiveNonblock(lock); err != nil {
+ a.ops.logf("system apply: another apply is running; leaving it to finish")
+ return nil
+ }
+ defer os.RemoveAll(a.stagingDir())
+
+ if err := a.recoverIfNeeded(ctx); err != nil {
+ return err
+ }
+ if recover {
+ return nil
+ }
+ req, err := a.readRequest()
+ if errors.Is(err, os.ErrNotExist) {
+ return nil
+ }
+ if err != nil {
+ a.ops.logf("system apply: request: %v", err)
+ return nil // nothing to record against: the request itself is unreadable
+ }
+ p, err := a.verifyPointer(ctx)
+ if err != nil {
+ return a.record(req, err)
+ }
+ if req.Version != p.Current.Version || !req.PromotedAt.Equal(p.PromotedAt) {
+ return a.record(req, fmt.Errorf("%w: request names %s@%s, pointer is %s@%s", errSystemRefused,
+ req.Version, req.PromotedAt.Format(time.RFC3339), p.Current.Version, p.PromotedAt.Format(time.RFC3339)))
+ }
+ seen, err := readSystemSeen(a.optDir)
+ if err != nil {
+ return a.record(req, fmt.Errorf("%w: seen.json unreadable: %v", errSystemRefused, err))
+ }
+ if wm, ok := seen[a.channel]; ok && p.PromotedAt.Before(wm) {
+ return a.record(req, fmt.Errorf("%w: replay — pointer promoted %s is older than the %s watermark %s", errSystemRefused,
+ p.PromotedAt.Format(time.RFC3339), a.channel, wm.Format(time.RFC3339)))
+ }
+ seen[a.channel] = p.PromotedAt
+ if err := writeJSONDurable(a.file("seen.json"), seen, 0o644); err != nil {
+ return err
+ }
+ if req.NoOp || p.Current.Version == a.onDiskBuild(ctx) {
+ return a.writeResult(req, systemOutcomeNoted, "")
+ }
+ staged, err := a.stageCandidate(p, req)
+ if err != nil {
+ return a.record(req, fmt.Errorf("%w: %v", errSystemRefused, err))
+ }
+ return a.install(ctx, req, p, staged) // Task 8b
+}
+
+func (a *systemApplier) install(ctx context.Context, req systemRequest, p *release.SystemPointer, staged string) error {
+ return errNotYetImplemented
+}
+
+func (a *systemApplier) recoverIfNeeded(ctx context.Context) error {
+ j, err := readSystemJournal(a.optDir)
+ if err != nil || j == nil {
+ return err
+ }
+ return fmt.Errorf("system apply: journal open at %s — recovery not implemented", j.Phase) // Task 8c
+}
+
+// record turns a refusal or deferral into result.json and a nil return.
+func (a *systemApplier) record(req systemRequest, err error) error {
+ outcome := systemOutcomeRefused
+ if errors.Is(err, errSystemDeferred) {
+ outcome = systemOutcomeDeferred
+ }
+ a.ops.logf("system apply: %s %s: %s: %v", a.channel, req.Version, outcome, err)
+ return a.writeResult(req, outcome, err.Error())
+}
+
+func (a *systemApplier) writeResult(req systemRequest, outcome, reason string) error {
+ return writeJSONDurable(a.file("result.json"), systemResult{
+ Channel: a.channel, Version: req.Version, PromotedAt: req.PromotedAt,
+ Outcome: outcome, Reason: reason, At: a.ops.now().UTC(),
+ }, 0o644)
+}
+
+// readRequest is the one read under /data: a fixed path reached without
+// following a single symlink, a regular file under 4 KiB, a version that
+// passes validSystemVersion. Anything else is an error, and root acts on
+// nothing.
+func (a *systemApplier) readRequest() (systemRequest, error) {
+ f, err := openBeneath(a.dataDir, "system", systemComponent, "requested.json")
+ if err != nil {
+ return systemRequest{}, err
+ }
+ defer f.Close()
+ st, err := f.Stat()
+ if err != nil {
+ return systemRequest{}, err
+ }
+ if !st.Mode().IsRegular() || st.Size() > systemMaxRequest {
+ return systemRequest{}, fmt.Errorf("request: not a regular file under %d bytes", systemMaxRequest)
+ }
+ var req systemRequest
+ if err := json.NewDecoder(io.LimitReader(f, systemMaxRequest)).Decode(&req); err != nil {
+ return systemRequest{}, fmt.Errorf("request: %w", err)
+ }
+ if !validSystemVersion(req.Version) {
+ return systemRequest{}, fmt.Errorf("request: version %q is not a version", req.Version)
+ }
+ if req.PromotedAt.IsZero() {
+ return systemRequest{}, errors.New("request: promoted_at unset")
+ }
+ return req, nil
+}
+
+// openBeneath opens root/parts... one component at a time with
+// O_DIRECTORY|O_NOFOLLOW for every directory and O_NOFOLLOW for the leaf,
+// so no component under a user-owned tree can be a symlink.
+func openBeneath(root string, parts ...string) (*os.File, error) {
+ fd, err := syscall.Open(root, syscall.O_RDONLY|syscall.O_DIRECTORY|syscall.O_CLOEXEC, 0)
+ if err != nil {
+ return nil, fmt.Errorf("open %s: %w", root, err)
+ }
+ for i, part := range parts {
+ flags := syscall.O_RDONLY | syscall.O_NOFOLLOW | syscall.O_CLOEXEC
+ if i < len(parts)-1 {
+ flags |= syscall.O_DIRECTORY
+ }
+ next, err := syscall.Openat(fd, part, flags, 0)
+ syscall.Close(fd)
+ if err != nil {
+ return nil, fmt.Errorf("open %s: %w", filepath.Join(append([]string{root}, parts[:i+1]...)...), err)
+ }
+ fd = next
+ }
+ return os.NewFile(uintptr(fd), filepath.Join(append([]string{root}, parts...)...)), nil
+}
+
+// verifyPointer fetches THIS box's channel pointer and verifies it with
+// the release public key. The channel is root's own (host.env), never the
+// request's. A store error is deferred, a bad pointer refused.
+func (a *systemApplier) verifyPointer(ctx context.Context) (*release.SystemPointer, error) {
+ p, _, err := release.GetSystemPointer(ctx, a.store, a.pub, release.SystemComponentRouter, a.channel)
+ switch {
+ case err == nil:
+ return p, nil
+ case errors.Is(err, os.ErrNotExist):
+ return nil, fmt.Errorf("%w: no %s pointer for this box's channel", errSystemRefused, a.channel)
+ case strings.Contains(err.Error(), "system pointer"):
+ return nil, fmt.Errorf("%w: %v", errSystemRefused, err)
+ default:
+ return nil, fmt.Errorf("%w: %v", errSystemDeferred, err)
+ }
+}
+
+// onDiskBuild asks the installed binary what it is, with a deadline. Not
+// version.Build: after an interrupted apply the two can differ.
+func (a *systemApplier) onDiskBuild(ctx context.Context) string {
+ out, err := a.ops.run(ctx, systemVersionWait, systemBinaryPath(a.optDir), "version")
+ if err != nil {
+ return ""
+ }
+ return strings.TrimSpace(strings.TrimPrefix(strings.TrimSpace(out), "carlos "))
+}
+
+// stageCandidate copies the agent's cache into staging/ (0700) as 0600,
+// verifies the copy's sha against the pointer, makes it executable, and
+// runs its `version`. Nothing under /data is read after the copy.
+func (a *systemApplier) stageCandidate(p *release.SystemPointer, req systemRequest) (string, error) {
+ art, ok := p.Current.Artifacts[a.platform()]
+ if !ok {
+ return "", fmt.Errorf("pointer names no artifact for %s", a.platform())
+ }
+ os.RemoveAll(a.stagingDir())
+ if err := os.MkdirAll(a.stagingDir(), 0o700); err != nil {
+ return "", err
+ }
+ if err := os.Chmod(a.stagingDir(), 0o700); err != nil {
+ return "", err
+ }
+ src, err := openBeneath(a.dataDir, "system", systemComponent, req.Version, "carlos")
+ if err != nil {
+ return "", fmt.Errorf("candidate: %w", err)
+ }
+ defer src.Close()
+ if st, err := src.Stat(); err != nil || !st.Mode().IsRegular() {
+ return "", errors.New("candidate: not a regular file")
+ }
+ staged := filepath.Join(a.stagingDir(), "carlos")
+ dst, err := os.OpenFile(staged, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o600)
+ if err != nil {
+ return "", err
+ }
+ h := sha256.New()
+ n, err := io.Copy(io.MultiWriter(dst, h), io.LimitReader(src, systemArtifactLimit+1))
+ dst.Close()
+ if err != nil {
+ return "", err
+ }
+ if n > systemArtifactLimit {
+ return "", fmt.Errorf("candidate: larger than %d bytes", systemArtifactLimit)
+ }
+ if got := hex.EncodeToString(h.Sum(nil)); got != art.SHA256 {
+ return "", fmt.Errorf("candidate: sha256 %s does not match the signed %s", got, art.SHA256)
+ }
+ if err := os.Chmod(staged, 0o755); err != nil {
+ return "", err
+ }
+ out, err := a.ops.run(context.Background(), systemVersionWait, staged, "version")
+ if err != nil || !strings.Contains(out, "carlos "+p.Current.Version) {
+ return "", fmt.Errorf("candidate reports %q, not %s", strings.TrimSpace(out), p.Current.Version)
+ }
+ return staged, nil
+}
+
+func (a *systemApplier) platform() string { return runtime.GOOS + "-" + runtime.GOARCH }
+
+func flockExclusive(f *os.File) error { return syscall.Flock(int(f.Fd()), syscall.LOCK_EX) }
+func flockExclusiveNonblock(f *os.File) error { return syscall.Flock(int(f.Fd()), syscall.LOCK_EX|syscall.LOCK_NB) }
+```
+Add `"runtime"` to the imports. Tests that stage a candidate need `platform()` to be `linux-arm64`: make `platform` a field set in the constructor (`platform string`, default `runtime.GOOS+"-"+runtime.GOARCH`) rather than a method, and have the fixture set `f.a.platform = "linux-arm64"`. Do that; update the test fixture accordingly.
+
+- [ ] **Step 4: Run the tests**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/host/ -run 'TestApply' -count=1 2>&1); rc=$?; echo "$out" | tail -30; echo rc=$rc`
+Expected: PASS for every test in this task.
+
+- [ ] **Step 5: Commit**
+
+```bash
+gofmt -l internal/host/ && git add internal/host/systemapply.go internal/host/systemapply_test.go && git commit -q -F - <<'EOF'
+host: carlos system apply — the refuse path
+
+Lock, one O_NOFOLLOW-walked request under /data, the channel from
+root's own env, the pointer re-verified, the per-channel replay
+watermark, the no-op, and the candidate copied into a 0700 staging
+dir and verified there before it is made executable.
+
+Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
+Claude-Session: https://claude.ai/code/session_015QCUCpfxbCyEAVQ2m51dH5
+EOF
+```
+
+---
+
+### Task 8b: `carlos system apply` — install, gate, arm, record (steps 6–12)
+
+**Files:**
+- Modify: `internal/host/systemapply.go` (replace the `install` stub)
+- Test: `internal/host/systemapply_test.go` (add)
+
+**Interfaces:**
+- Consumes: `systemctlPlan`, `bootstrapPlan`, `systemQuiesceUnits`, the fixture from 8a.
+- Produces:
+ ```go
+ func (a *systemApplier) install(ctx, req, p, staged) error
+ func (a *systemApplier) decidePrev(ctx) (prevExists bool, err error) // journal backing-up, copy when authorised
+ func (a *systemApplier) stagePlan(ctx, binary, dir string) (bootstrapPlan, []string, error) // runs bootstrap -root dir -plan-json; returns plan + unit file names
+ func (a *systemApplier) journal(j systemJournal) error
+ func (a *systemApplier) installUnits(stagedRoot string, remove []string) error // remove, copy units, sysusers, tmpfiles, daemon-reload
+ func (a *systemApplier) restartEdge() error // reset-failed then restart
+ func (a *systemApplier) gate(ctx, target string) error
+ func (a *systemApplier) arm(plan bootstrapPlan) error // systemctlPlan verbatim
+ func (a *systemApplier) rollback(ctx, req, reason string, j *systemJournal) error // Task 8c; stub returns the reason as an error
+ ```
+
+- [ ] **Step 1: Write the failing tests**
+
+Add to `systemapply_test.go`:
+```go
+// applyHappy sets up a v1 box with a v2 candidate whose fake bootstrap
+// plan adds carlos-new-only.path and drops carlos-old-only.timer.
+func applyHappy(t *testing.T) (*applyFixture, systemRequest) {
+ f := newApplyFixture(t, "stable", "v1")
+ body := []byte("#!/bin/sh\necho carlos v2\n")
+ req := f.promote("v2", body, false)
+ staged := filepath.Join(f.a.stagingDir(), "carlos")
+ f.answer["version:"+staged] = "carlos v2\n"
+ f.answer["plan:"+staged] = `{"enabled":["carlos-edge.service","carlos-adopt.timer","carlos-adopt.path","carlos-stage.path","carlos-restart.path","carlos-system-update.path","carlos-system-recover.service","carlos-new-only.path"],"inert":["carlos-restore-verify.timer"]}`
+ f.answer["units:"+staged] = "carlos-edge.service carlos-adopt.timer carlos-adopt.path carlos-stage.path carlos-restart.path carlos-system-update.path carlos-system-update.service carlos-system-recover.service carlos-new-only.path"
+ // .prev, once made, answers as v1 and stages v1's units.
+ prev := systemPrevPath(f.a.optDir)
+ f.answer["version:"+prev] = "carlos v1\n"
+ f.answer["plan:"+prev] = f.answer["plan:"+systemBinaryPath(f.a.optDir)]
+ f.answer["units:"+prev] = f.answer["units:"+systemBinaryPath(f.a.optDir)]
+ // After the install the on-disk binary answers v2.
+ f.answer["version:"+systemBinaryPath(f.a.optDir)] = "carlos v2\n"
+ return f, req
+}
+
+func TestApplyInstallsGatesArmsAndRecords(t *testing.T) {
+ f, req := applyHappy(t)
+ if err := f.a.Run(context.Background(), false); err != nil {
+ t.Fatal(err)
+ }
+ r := f.result()
+ if r == nil || r.Outcome != systemOutcomeApplied || !r.matches("stable", "v2", req.PromotedAt) {
+ t.Fatalf("result = %+v", r)
+ }
+ ap, _ := readSystemApplied(f.a.optDir)
+ if ap == nil || ap.Version != "v2" || ap.Channel != "stable" {
+ t.Fatalf("applied = %+v", ap)
+ }
+ if j, _ := readSystemJournal(f.a.optDir); j != nil {
+ t.Fatalf("journal left open: %+v", j)
+ }
+ if _, err := os.Stat(f.a.stagingDir()); err == nil {
+ t.Fatal("staging/ left behind")
+ }
+ bin, _ := os.ReadFile(systemBinaryPath(f.a.optDir))
+ if !strings.Contains(string(bin), "carlos v2") {
+ t.Fatal("binary not swapped")
+ }
+ prev, _ := os.ReadFile(systemPrevPath(f.a.optDir))
+ if !strings.Contains(string(prev), "carlos v1") {
+ t.Fatal(".prev is not the old binary")
+ }
+ units := filepath.Join(f.a.etcDir, "systemd", "system")
+ if _, err := os.Stat(filepath.Join(units, "carlos-new-only.path")); err != nil {
+ t.Error("new unit not installed")
+ }
+ if _, err := os.Stat(filepath.Join(units, "carlos-old-only.timer")); err == nil {
+ t.Error("removed unit still installed")
+ }
+ if _, err := os.Stat(filepath.Join(f.a.etcDir, "sysusers.d", "carlos.conf")); err != nil {
+ t.Error("sysusers not installed")
+ }
+ if _, err := os.Stat(filepath.Join(f.a.etcDir, "tmpfiles.d", "carlos.conf")); err != nil {
+ t.Error("tmpfiles not installed")
+ }
+ if _, err := os.Stat(filepath.Join(f.a.etcDir, "registry.db")); err == nil {
+ t.Error("staging data/ copied")
+ }
+ // Order of the systemctl calls is the spec.
+ joined := strings.Join(f.calls, "\n")
+ order := []string{
+ "systemctl stop carlos-adopt.timer carlos-adopt.path carlos-stage.path carlos-restart.path carlos-adopt.service carlos-config-sync.service carlos-stage.service carlos-restart.service",
+ "systemctl disable --now carlos-old-only.timer",
+ "run systemd-sysusers",
+ "run systemd-tmpfiles --create",
+ "systemctl daemon-reload",
+ "systemctl reset-failed carlos-edge",
+ "systemctl restart carlos-edge",
+ "systemctl is-active carlos-edge",
+ "systemctl daemon-reload",
+ "systemctl enable carlos-edge.service carlos-adopt.timer carlos-adopt.path carlos-stage.path carlos-restart.path carlos-system-update.path carlos-system-recover.service carlos-new-only.path",
+ "systemctl start carlos-adopt.timer carlos-adopt.path carlos-stage.path carlos-restart.path carlos-system-update.path carlos-new-only.path",
+ }
+ at := 0
+ for _, want := range order {
+ i := strings.Index(joined[at:], want)
+ if i < 0 {
+ t.Fatalf("call order broken: %q not found after position %d in:\n%s", want, at, joined)
+ }
+ at += i + len(want)
+ }
+ for _, c := range f.calls {
+ if strings.HasPrefix(c, "systemctl start ") && strings.Contains(c, ".service") {
+ t.Fatalf("apply started a service: %s", c)
+ }
+ }
+ if strings.Contains(joined, "systemctl enable --now") {
+ t.Fatal("enable --now is never used")
+ }
+}
+
+func TestApplyGateSoakCatchesAFlap(t *testing.T) {
+ f, req := applyHappy(t)
+ // Edge comes up, then restarts once during the soak.
+ restarts := 0
+ orig := f.a.ops.systemctl
+ f.a.ops.systemctl = func(args ...string) (string, error) {
+ if args[0] == "show" {
+ restarts++
+ if restarts > 1 {
+ return "1\n2000\n", nil
+ }
+ }
+ return orig(args...)
+ }
+ err := f.a.Run(context.Background(), false)
+ if err == nil {
+ t.Fatal("a flapping edge passed the gate")
+ }
+ r := f.result()
+ if r == nil || !r.matches("stable", "v2", req.PromotedAt) || r.Outcome == systemOutcomeApplied {
+ t.Fatalf("result = %+v", r)
+ }
+}
+
+func TestApplyPrevIsRefreshedOnlyWhenTheDiskIsKnownGood(t *testing.T) {
+ // applied.json names v0 but the disk says v1 (a rollback-failed left a
+ // bad v1 in place, .prev is the good v0): .prev must be left alone.
+ f, _ := applyHappy(t)
+ os.WriteFile(systemPrevPath(f.a.optDir), []byte("#!/bin/sh\necho carlos v0\n"), 0o755)
+ f.answer["version:"+systemPrevPath(f.a.optDir)] = "carlos v0\n"
+ writeJSONDurable(filepath.Join(systemRootDir(f.a.optDir), "applied.json"), systemApplied{Version: "v0", Channel: "stable", At: f.now}, 0o644)
+ _ = f.a.Run(context.Background(), false)
+ prev, _ := os.ReadFile(systemPrevPath(f.a.optDir))
+ if !strings.Contains(string(prev), "carlos v0") {
+ t.Fatal(".prev was overwritten by a binary that never passed a gate")
+ }
+}
+
+func TestApplyPrevUnitsComeFromPrevNotFromDisk(t *testing.T) {
+ f, _ := applyHappy(t)
+ os.WriteFile(systemPrevPath(f.a.optDir), []byte("#!/bin/sh\necho carlos v0\n"), 0o755)
+ f.answer["version:"+systemPrevPath(f.a.optDir)] = "carlos v0\n"
+ f.answer["plan:"+systemPrevPath(f.a.optDir)] = `{"enabled":["carlos-edge.service"],"inert":[]}`
+ f.answer["units:"+systemPrevPath(f.a.optDir)] = "carlos-edge.service carlos-v0-only.timer"
+ writeJSONDurable(filepath.Join(systemRootDir(f.a.optDir), "applied.json"), systemApplied{Version: "v0", Channel: "stable", At: f.now}, 0o644)
+ // Inspect the journal mid-flight by failing the gate.
+ f.edgeUp = false
+ _ = f.a.Run(context.Background(), false)
+ // units_removed must be computed against .prev's set (v0-only.timer),
+ // not the on-disk v1's (old-only.timer).
+ found := false
+ for _, c := range f.calls {
+ if c == "run carlos.prev bootstrap -root "+filepath.Join(f.a.stagingDir(), "prev")+" -plan-json" {
+ found = true
+ }
+ }
+ if !found {
+ t.Fatalf("prev units not staged from carlos.prev:\n%s", strings.Join(f.calls, "\n"))
+ }
+}
+```
+
+- [ ] **Step 2: Run to verify failure**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/host/ -run 'TestApplyInstalls|TestApplyGate|TestApplyPrev' -count=1 2>&1); rc=$?; echo "$out" | tail -20; echo rc=$rc`
+Expected: FAIL with `errNotYetImplemented`.
+
+- [ ] **Step 3: Implement the install path**
+
+Replace the `install` stub and add the helpers:
+```go
+func (a *systemApplier) install(ctx context.Context, req systemRequest, p *release.SystemPointer, staged string) error {
+ target := p.Current.Version
+ prevOK, err := a.decidePrev(ctx)
+ if err != nil {
+ return a.record(req, fmt.Errorf("%w: %v", errSystemRefused, err))
+ }
+ if !prevOK {
+ return a.record(req, fmt.Errorf("%w: no known-good carlos.prev to fall back to", errSystemRefused))
+ }
+ newDir, prevDir := filepath.Join(a.stagingDir(), "new"), filepath.Join(a.stagingDir(), "prev")
+ newPlan, newUnits, err := a.stagePlan(ctx, staged, newDir)
+ if err != nil {
+ return a.record(req, fmt.Errorf("%w: candidate bootstrap: %v", errSystemRefused, err))
+ }
+ prevPlan, prevUnits, err := a.stagePlan(ctx, systemPrevPath(a.optDir), prevDir)
+ if err != nil {
+ return a.record(req, fmt.Errorf("%w: carlos.prev bootstrap: %v", errSystemRefused, err))
+ }
+ j := systemJournal{Phase: systemPhaseInstalling, Target: target, PrevBuild: a.prevBuild(ctx),
+ UnitsAdded: diff(newUnits, prevUnits), UnitsRemoved: diff(prevUnits, newUnits), StartedAt: a.ops.now().UTC()}
+ if err := a.journal(j); err != nil {
+ return err
+ }
+ fail := func(step string, err error) error {
+ return a.rollback(ctx, req, fmt.Sprintf("%s: %v", step, err), &j, prevPlan)
+ }
+ if _, err := a.ops.systemctl(append([]string{"stop"}, systemQuiesceUnits...)...); err != nil {
+ a.ops.logf("system apply: quiesce: %v (continuing)", err)
+ }
+ if err := os.Rename(staged, systemBinaryPath(a.optDir)); err != nil {
+ return fail("install", err)
+ }
+ if err := os.Chown(systemBinaryPath(a.optDir), 0, 0); err != nil && !errors.Is(err, os.ErrPermission) {
+ a.ops.logf("system apply: chown root: %v", err)
+ }
+ j.Phase = systemPhaseInstalled
+ if err := a.journal(j); err != nil {
+ return fail("journal", err)
+ }
+ if err := a.installUnits(newDir, j.UnitsRemoved); err != nil {
+ return fail("units", err)
+ }
+ j.Phase = systemPhaseUnits
+ if err := a.journal(j); err != nil {
+ return fail("journal", err)
+ }
+ if err := a.restartEdge(); err != nil {
+ return fail("restart", err)
+ }
+ j.Phase = systemPhaseRestarted
+ if err := a.journal(j); err != nil {
+ return fail("journal", err)
+ }
+ if err := a.gate(ctx, target); err != nil {
+ return fail("gate", err)
+ }
+ j.Phase = systemPhaseGated
+ if err := a.journal(j); err != nil {
+ return fail("journal", err)
+ }
+ return a.finish(req, target, newPlan)
+}
+
+// finish is steps 11–12, and what recovery runs forward from phase gated.
+func (a *systemApplier) finish(req systemRequest, target string, plan bootstrapPlan) error {
+ if err := a.arm(plan); err != nil {
+ a.ops.logf("system apply: arm: %v", err)
+ }
+ if err := writeJSONDurable(a.file("applied.json"), systemApplied{Version: target, Channel: a.channel, At: a.ops.now().UTC()}, 0o644); err != nil {
+ return err
+ }
+ if err := a.writeResult(req, systemOutcomeApplied, ""); err != nil {
+ return err
+ }
+ os.Remove(a.file("journal.json"))
+ return syncDirBestEffort(a.rootDir())
+}
+
+// decidePrev: refresh carlos.prev from the on-disk binary only when that
+// binary is known good — applied.json names it, or nothing has ever been
+// applied and there is no .prev (a hand-rolled box whose running binary
+// passed the script's own post-condition). Otherwise leave .prev alone.
+func (a *systemApplier) decidePrev(ctx context.Context) (bool, error) {
+ prev := systemPrevPath(a.optDir)
+ _, prevErr := os.Stat(prev)
+ applied, err := readSystemApplied(a.optDir)
+ if err != nil {
+ return false, err
+ }
+ disk := a.onDiskBuild(ctx)
+ known := (applied != nil && applied.Version == disk && disk != "") || (applied == nil && prevErr != nil)
+ if !known {
+ return prevErr == nil, nil
+ }
+ if err := a.journal(systemJournal{Phase: systemPhaseBackingUp, Target: "", PrevBuild: disk, StartedAt: a.ops.now().UTC()}); err != nil {
+ return false, err
+ }
+ if err := copyFileDurable(systemBinaryPath(a.optDir), prev, 0o755); err != nil {
+ return false, err
+ }
+ os.Remove(a.file("journal.json"))
+ return true, nil
+}
+
+func (a *systemApplier) prevBuild(ctx context.Context) string {
+ out, err := a.ops.run(ctx, systemVersionWait, systemPrevPath(a.optDir), "version")
+ if err != nil {
+ return ""
+ }
+ return strings.TrimSpace(strings.TrimPrefix(strings.TrimSpace(out), "carlos "))
+}
+
+// stagePlan runs `<binary> bootstrap -root <dir> -plan-json` and returns
+// that binary's arming plan and the unit files it staged.
+func (a *systemApplier) stagePlan(ctx context.Context, binary, dir string) (bootstrapPlan, []string, error) {
+ os.RemoveAll(dir)
+ if err := os.MkdirAll(dir, 0o700); err != nil {
+ return bootstrapPlan{}, nil, err
+ }
+ out, err := a.ops.run(ctx, systemBootstrapWait, binary, "bootstrap", "-root", dir, "-plan-json")
+ if err != nil {
+ return bootstrapPlan{}, nil, err
+ }
+ var plan bootstrapPlan
+ if err := json.Unmarshal([]byte(out), &plan); err != nil {
+ return bootstrapPlan{}, nil, fmt.Errorf("plan: %w", err)
+ }
+ entries, err := os.ReadDir(filepath.Join(dir, "etc", "systemd", "system"))
+ if err != nil {
+ return bootstrapPlan{}, nil, err
+ }
+ var units []string
+ for _, e := range entries {
+ if !e.IsDir() {
+ units = append(units, e.Name())
+ }
+ }
+ return plan, units, nil
+}
+
+func (a *systemApplier) journal(j systemJournal) error {
+ return writeJSONDurable(a.file("journal.json"), j, 0o600)
+}
+
+// installUnits: remove what the target build dropped, copy its units in,
+// then sysusers BEFORE tmpfiles, then daemon-reload. Never the staging
+// root's data/.
+func (a *systemApplier) installUnits(stagedRoot string, remove []string) error {
+ unitDir := filepath.Join(a.etcDir, "systemd", "system")
+ for _, u := range remove {
+ if _, err := a.ops.systemctl("disable", "--now", u); err != nil {
+ a.ops.logf("system apply: disable %s: %v", u, err)
+ }
+ os.Remove(filepath.Join(unitDir, u))
+ }
+ entries, err := os.ReadDir(filepath.Join(stagedRoot, "etc", "systemd", "system"))
+ if err != nil {
+ return err
+ }
+ for _, e := range entries {
+ if e.IsDir() {
+ continue
+ }
+ if err := copyFileDurable(filepath.Join(stagedRoot, "etc", "systemd", "system", e.Name()), filepath.Join(unitDir, e.Name()), 0o644); err != nil {
+ return err
+ }
+ }
+ sys := filepath.Join(stagedRoot, "etc", "sysusers.d", "carlos.conf")
+ if _, err := os.Stat(sys); err == nil {
+ if err := os.MkdirAll(filepath.Join(a.etcDir, "sysusers.d"), 0o755); err != nil {
+ return err
+ }
+ dst := filepath.Join(a.etcDir, "sysusers.d", "carlos.conf")
+ if err := copyFileDurable(sys, dst, 0o644); err != nil {
+ return err
+ }
+ if _, err := a.ops.run(context.Background(), systemVersionWait, "systemd-sysusers", dst); err != nil {
+ return fmt.Errorf("systemd-sysusers: %w", err)
+ }
+ }
+ tmp := filepath.Join(stagedRoot, "etc", "tmpfiles.d", "carlos.conf")
+ if err := os.MkdirAll(filepath.Join(a.etcDir, "tmpfiles.d"), 0o755); err != nil {
+ return err
+ }
+ dst := filepath.Join(a.etcDir, "tmpfiles.d", "carlos.conf")
+ if err := copyFileDurable(tmp, dst, 0o644); err != nil {
+ return err
+ }
+ if _, err := a.ops.run(context.Background(), systemVersionWait, "systemd-tmpfiles", "--create", dst); err != nil {
+ a.ops.logf("system apply: systemd-tmpfiles: %v (continuing, as the script does)", err)
+ }
+ _, err = a.ops.systemctl("daemon-reload")
+ return err
+}
+
+// restartEdge: reset-failed FIRST — a start-limited unit refuses restart —
+// then restart.
+func (a *systemApplier) restartEdge() error {
+ a.ops.systemctl("reset-failed", systemEdgeUnit)
+ _, err := a.ops.systemctl("restart", systemEdgeUnit)
+ return err
+}
+
+// gate: within gateWait the edge is active, answers on loopback, and the
+// installed binary reports the target; then a soak during which neither
+// NRestarts nor ActiveEnterTimestampMonotonic moves.
+func (a *systemApplier) gate(ctx context.Context, target string) error {
+ wait, soak := a.gateWait, a.soak
+ deadline := a.ops.now().Add(wait)
+ var lastErr error
+ for {
+ lastErr = a.gateOnce(ctx, target)
+ if lastErr == nil {
+ break
+ }
+ if !a.ops.now().Before(deadline) {
+ return lastErr
+ }
+ a.ops.sleep(time.Second)
+ }
+ before, err := a.ops.systemctl("show", "-p", "NRestarts", "-p", "ActiveEnterTimestampMonotonic", "--value", systemEdgeUnit)
+ if err != nil {
+ return err
+ }
+ a.ops.sleep(soak)
+ after, err := a.ops.systemctl("show", "-p", "NRestarts", "-p", "ActiveEnterTimestampMonotonic", "--value", systemEdgeUnit)
+ if err != nil {
+ return err
+ }
+ if strings.TrimSpace(before) != strings.TrimSpace(after) {
+ return fmt.Errorf("edge restarted during the soak (%s → %s)", strings.ReplaceAll(strings.TrimSpace(before), "\n", "/"), strings.ReplaceAll(strings.TrimSpace(after), "\n", "/"))
+ }
+ if err := a.gateOnce(ctx, target); err != nil {
+ return err
+ }
+ return nil
+}
+
+func (a *systemApplier) gateOnce(ctx context.Context, target string) error {
+ if out, err := a.ops.systemctl("is-active", systemEdgeUnit); err != nil || strings.TrimSpace(out) != "active" {
+ return fmt.Errorf("%s is %s", systemEdgeUnit, strings.TrimSpace(out))
+ }
+ if err := a.ops.probeEdge(); err != nil {
+ return fmt.Errorf("edge probe: %w", err)
+ }
+ if got := a.onDiskBuild(ctx); got != target {
+ return fmt.Errorf("installed binary reports %q, not %s", got, target)
+ }
+ return nil
+}
+
+// arm is systemctlPlan verbatim: enable all, start timers and paths, never
+// a service — which is why apply can call it from inside itself.
+func (a *systemApplier) arm(plan bootstrapPlan) error {
+ for _, args := range systemctlPlan(plan.Enabled) {
+ if _, err := a.ops.systemctl(args...); err != nil {
+ return fmt.Errorf("systemctl %s: %w", strings.Join(args, " "), err)
+ }
+ }
+ return nil
+}
+
+func (a *systemApplier) rollback(ctx context.Context, req systemRequest, reason string, j *systemJournal, prevPlan bootstrapPlan) error {
+ return errors.New("rollback not implemented: " + reason) // Task 8c
+}
+
+func diff(have, drop []string) []string {
+ skip := map[string]bool{}
+ for _, d := range drop {
+ skip[d] = true
+ }
+ var out []string
+ for _, h := range have {
+ if !skip[h] {
+ out = append(out, h)
+ }
+ }
+ return out
+}
+
+// copyFileDurable is writeFileDurable for a file too large to hold: temp,
+// fsync, rename, directory fsync.
+func copyFileDurable(src, dst string, mode os.FileMode) (err error) {
+ in, err := os.Open(src)
+ if err != nil {
+ return err
+ }
+ defer in.Close()
+ tmp := dst + ".tmp"
+ out, err := os.OpenFile(tmp, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, mode)
+ if err != nil {
+ return err
+ }
+ defer func() {
+ if err != nil {
+ out.Close()
+ os.Remove(tmp)
+ }
+ }()
+ if _, err = io.Copy(out, in); err != nil {
+ return err
+ }
+ if err = out.Chmod(mode); err != nil {
+ return err
+ }
+ if err = out.Sync(); err != nil {
+ return err
+ }
+ if err = out.Close(); err != nil {
+ return err
+ }
+ if err = os.Rename(tmp, dst); err != nil {
+ return err
+ }
+ return syncDirBestEffort(filepath.Dir(dst))
+}
+
+func syncDirBestEffort(dir string) error {
+ d, err := os.Open(dir)
+ if err != nil {
+ return err
+ }
+ defer d.Close()
+ return d.Sync()
+}
+```
+The test for the flap expects `TestApplyGateSoakCatchesAFlap` to end in a rollback; with the 8c stub it ends in an error from `rollback`, and `result.json` is not yet written by the stub. Make the stub write `result.json` with `outcome: rolled-back` and reason, then return the error — that is the 8c contract in miniature and keeps this task's tests honest:
+```go
+func (a *systemApplier) rollback(ctx context.Context, req systemRequest, reason string, j *systemJournal, prevPlan bootstrapPlan) error {
+ _ = a.writeResult(req, systemOutcomeRolledBack, reason)
+ return errors.New("rollback not implemented: " + reason)
+}
+```
+
+- [ ] **Step 4: Run the tests**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/host/ -run 'TestApply' -count=1 2>&1); rc=$?; echo "$out" | tail -30; echo rc=$rc`
+Expected: PASS. `TestApplyInstallsGatesArmsAndRecords` checks the exact systemctl order; if a call is missing, fix the code, not the expected list, unless the spec says otherwise.
+
+- [ ] **Step 5: Commit**
+
+```bash
+gofmt -l internal/host/ && git add internal/host/systemapply.go internal/host/systemapply_test.go && git commit -q -F - <<'EOF'
+host: carlos system apply — install, gate, arm, record
+
+.prev refreshed only when the disk is known good; both unit sets
+staged from the binaries that own them; quiesce before the rename;
+sysusers before tmpfiles; reset-failed before restart; a 15 s soak on
+NRestarts; arming after the gate via systemctlPlan.
+
+Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
+Claude-Session: https://claude.ai/code/session_015QCUCpfxbCyEAVQ2m51dH5
+EOF
+```
+
+---
+
+### Task 8c: `carlos system apply` — rollback, recovery, `--recover`, the real `sysOps`
+
+**Files:**
+- Modify: `internal/host/systemapply.go` (replace `rollback` and `recoverIfNeeded`; add `RunSystemApply` and `productionSysOps`)
+- Test: `internal/host/systemapply_test.go` (add)
+
+**Interfaces:**
+- Produces:
+ ```go
+ func RunSystemApply(args []string) // flags: -recover; exits 1 on a non-nil Run error
+ func productionSysOps() sysOps
+ func (a *systemApplier) rollback(ctx, req, reason, j *systemJournal, prevPlan bootstrapPlan) error
+ func (a *systemApplier) recoverIfNeeded(ctx) error
+ func (a *systemApplier) restoreFiles(ctx, j *systemJournal) (bootstrapPlan, error) // files only: binary, units, sysusers, tmpfiles, daemon-reload, enable
+ ```
+
+- [ ] **Step 1: Write the failing tests**
+
+```go
+func TestApplyRollsBackWhenTheGateFails(t *testing.T) {
+ f, req := applyHappy(t)
+ f.edgeUp = false
+ err := f.a.Run(context.Background(), false)
+ if err == nil {
+ t.Fatal("a failed gate must exit non-zero")
+ }
+ r := f.result()
+ if r == nil || r.Outcome != systemOutcomeRolledBack || !r.matches("stable", "v2", req.PromotedAt) || !strings.Contains(r.Reason, "gate") {
+ t.Fatalf("result = %+v", r)
+ }
+ bin, _ := os.ReadFile(systemBinaryPath(f.a.optDir))
+ if !strings.Contains(string(bin), "carlos v1") {
+ t.Fatal("binary not restored")
+ }
+ if _, err := os.Stat(systemPrevPath(f.a.optDir)); err != nil {
+ t.Fatal(".prev consumed by the rollback")
+ }
+ units := filepath.Join(f.a.etcDir, "systemd", "system")
+ if _, err := os.Stat(filepath.Join(units, "carlos-new-only.path")); err == nil {
+ t.Error("added unit survived the rollback")
+ }
+ if _, err := os.Stat(filepath.Join(units, "carlos-old-only.timer")); err != nil {
+ t.Error("removed unit not restored")
+ }
+ if ap, _ := readSystemApplied(f.a.optDir); ap != nil {
+ t.Fatalf("applied.json written by a rollback: %+v", ap)
+ }
+ if j, _ := readSystemJournal(f.a.optDir); j != nil {
+ t.Fatal("journal left open after rollback")
+ }
+ joined := strings.Join(f.calls, "\n")
+ if !strings.Contains(joined, "systemctl disable --now carlos-new-only.path") {
+ t.Error("added unit not disabled")
+ }
+ // reset-failed precedes the rollback's restart too, and the previous
+ // plan is armed afterwards.
+ i := strings.LastIndex(joined, "systemctl reset-failed carlos-edge")
+ k := strings.LastIndex(joined, "systemctl restart carlos-edge")
+ m := strings.LastIndex(joined, "systemctl start carlos-adopt.timer")
+ if !(i < k && k < m) {
+ t.Fatalf("rollback order wrong:\n%s", joined)
+ }
+}
+
+func TestApplyRollbackFailedLeavesPrevAndAppliedAlone(t *testing.T) {
+ f, req := applyHappy(t)
+ writeJSONDurable(filepath.Join(systemRootDir(f.a.optDir), "applied.json"), systemApplied{Version: "v1", Channel: "stable", At: f.now}, 0o644)
+ f.edgeUp = false // never comes up, on either binary
+ if err := f.a.Run(context.Background(), false); err == nil {
+ t.Fatal("must exit non-zero")
+ }
+ r := f.result()
+ if r == nil || r.Outcome != systemOutcomeRollbackFailed || !r.matches("stable", "v2", req.PromotedAt) {
+ t.Fatalf("result = %+v", r)
+ }
+ ap, _ := readSystemApplied(f.a.optDir)
+ if ap == nil || ap.Version != "v1" {
+ t.Fatalf("applied = %+v, must still name v1", ap)
+ }
+ prev, _ := os.ReadFile(systemPrevPath(f.a.optDir))
+ if !strings.Contains(string(prev), "carlos v1") {
+ t.Fatal(".prev damaged")
+ }
+ // A second apply of a NEW version must not refresh .prev from the
+ // unproven disk binary.
+ f.answer["version:"+systemBinaryPath(f.a.optDir)] = "carlos v1\n" // rollback restored v1 on disk
+ os.Remove(systemRequestPath(f.a.dataDir))
+ f.promote("v3", []byte("#!/bin/sh\necho carlos v3\n"), false)
+ f.answer["version:"+filepath.Join(f.a.stagingDir(), "carlos")] = "carlos v3\n"
+ f.answer["plan:"+filepath.Join(f.a.stagingDir(), "carlos")] = f.answer["plan:"+systemPrevPath(f.a.optDir)]
+ f.answer["units:"+filepath.Join(f.a.stagingDir(), "carlos")] = f.answer["units:"+systemPrevPath(f.a.optDir)]
+ _ = f.a.Run(context.Background(), false)
+ prev2, _ := os.ReadFile(systemPrevPath(f.a.optDir))
+ if string(prev2) != string(prev) {
+ t.Fatal(".prev refreshed after rollback-failed")
+ }
+}
+
+func TestRecoveryRollsBackFilesOnlyAtBootAndRestartsAtRuntime(t *testing.T) {
+ for _, phase := range []string{systemPhaseInstalling, systemPhaseInstalled, systemPhaseUnits, systemPhaseRestarted} {
+ t.Run(phase, func(t *testing.T) {
+ f, _ := applyHappy(t)
+ // Simulate: v2 installed, journal open at phase, box rebooted.
+ os.WriteFile(systemPrevPath(f.a.optDir), []byte("#!/bin/sh\necho carlos v1\n"), 0o755)
+ os.WriteFile(systemBinaryPath(f.a.optDir), []byte("#!/bin/sh\necho carlos v2\n"), 0o755)
+ os.WriteFile(filepath.Join(f.a.etcDir, "systemd", "system", "carlos-new-only.path"), []byte("x"), 0o644)
+ writeJSONDurable(filepath.Join(systemRootDir(f.a.optDir), "journal.json"),
+ systemJournal{Phase: phase, Target: "v2", PrevBuild: "v1", UnitsAdded: []string{"carlos-new-only.path"}, UnitsRemoved: []string{"carlos-old-only.timer"}, StartedAt: f.now}, 0o600)
+ f.edgeStartedThisBoot = false
+ if err := f.a.Run(context.Background(), true); err != nil {
+ t.Fatal(err)
+ }
+ bin, _ := os.ReadFile(systemBinaryPath(f.a.optDir))
+ if !strings.Contains(string(bin), "carlos v1") {
+ t.Fatal("binary not restored")
+ }
+ if _, err := os.Stat(filepath.Join(f.a.etcDir, "systemd", "system", "carlos-new-only.path")); err == nil {
+ t.Error("added unit survived")
+ }
+ if _, err := os.Stat(filepath.Join(f.a.etcDir, "systemd", "system", "carlos-old-only.timer")); err != nil {
+ t.Error("removed unit not restored")
+ }
+ joined := strings.Join(f.calls, "\n")
+ if strings.Contains(joined, "systemctl restart carlos-edge") || strings.Contains(joined, "systemctl start ") {
+ t.Fatalf("boot recovery restarted or started something:\n%s", joined)
+ }
+ if !strings.Contains(joined, "systemctl enable carlos-edge.service") {
+ t.Fatal("previous plan not enabled for the boot")
+ }
+ if r := f.result(); r == nil || r.Outcome != systemOutcomeRolledBack || !strings.Contains(r.Reason, phase) {
+ t.Fatalf("result = %+v", r)
+ }
+ if j, _ := readSystemJournal(f.a.optDir); j != nil {
+ t.Fatal("journal left open")
+ }
+ })
+ }
+ // Runtime (OnFailure=): same, plus a non-blocking restart and the
+ // previous plan's timers/paths started.
+ f, _ := applyHappy(t)
+ os.WriteFile(systemPrevPath(f.a.optDir), []byte("#!/bin/sh\necho carlos v1\n"), 0o755)
+ writeJSONDurable(filepath.Join(systemRootDir(f.a.optDir), "journal.json"),
+ systemJournal{Phase: systemPhaseUnits, Target: "v2", PrevBuild: "v1", StartedAt: f.now}, 0o600)
+ f.edgeStartedThisBoot = true
+ if err := f.a.Run(context.Background(), true); err != nil {
+ t.Fatal(err)
+ }
+ joined := strings.Join(f.calls, "\n")
+ for _, want := range []string{"systemctl stop carlos-adopt.timer", "systemctl reset-failed carlos-edge", "systemctl restart --no-block carlos-edge", "systemctl start carlos-adopt.timer"} {
+ if !strings.Contains(joined, want) {
+ t.Errorf("runtime recovery lacks %q:\n%s", want, joined)
+ }
+ }
+}
+
+func TestRecoveryFinishesForwardFromGated(t *testing.T) {
+ f, req := applyHappy(t)
+ os.WriteFile(systemBinaryPath(f.a.optDir), []byte("#!/bin/sh\necho carlos v2\n"), 0o755)
+ writeJSONDurable(systemRequestPath(f.a.dataDir), req, 0o644)
+ writeJSONDurable(filepath.Join(systemRootDir(f.a.optDir), "journal.json"),
+ systemJournal{Phase: systemPhaseGated, Target: "v2", PrevBuild: "v1", StartedAt: f.now}, 0o600)
+ if err := f.a.Run(context.Background(), true); err != nil {
+ t.Fatal(err)
+ }
+ bin, _ := os.ReadFile(systemBinaryPath(f.a.optDir))
+ if !strings.Contains(string(bin), "carlos v2") {
+ t.Fatal("a gated apply was undone")
+ }
+ ap, _ := readSystemApplied(f.a.optDir)
+ if ap == nil || ap.Version != "v2" {
+ t.Fatalf("applied = %+v", ap)
+ }
+ if r := f.result(); r == nil || r.Outcome != systemOutcomeApplied {
+ t.Fatalf("result = %+v", r)
+ }
+}
+
+func TestRecoveryRemovesTheJournalLast(t *testing.T) {
+ // A restart that fails mid-recovery leaves the journal for the next run.
+ f, _ := applyHappy(t)
+ os.WriteFile(systemPrevPath(f.a.optDir), []byte("#!/bin/sh\necho carlos v1\n"), 0o755)
+ writeJSONDurable(filepath.Join(systemRootDir(f.a.optDir), "journal.json"),
+ systemJournal{Phase: systemPhaseInstalled, Target: "v2", PrevBuild: "v1", StartedAt: f.now}, 0o600)
+ orig := f.a.ops.systemctl
+ f.a.ops.systemctl = func(args ...string) (string, error) {
+ if args[0] == "daemon-reload" {
+ return "", errors.New("systemd busy")
+ }
+ return orig(args...)
+ }
+ if err := f.a.Run(context.Background(), true); err == nil {
+ t.Fatal("a failed recovery must exit non-zero")
+ }
+ if j, _ := readSystemJournal(f.a.optDir); j == nil {
+ t.Fatal("journal removed although recovery did not complete")
+ }
+}
+```
+
+- [ ] **Step 2: Run to verify failure**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/host/ -run 'TestApplyRolls|TestApplyRollback|TestRecovery' -count=1 2>&1); rc=$?; echo "$out" | tail -20; echo rc=$rc`
+Expected: FAIL ("rollback not implemented", "recovery not implemented").
+
+- [ ] **Step 3: Implement**
+
+Replace `rollback` and `recoverIfNeeded`; add `restoreFiles`, `RunSystemApply`, `productionSysOps`:
+```go
+// rollback is step 13: back to carlos.prev and its units, restart, gate,
+// arm the previous plan. applied.json is untouched — the .prev build is
+// what passed a gate, and it is what applied.json already names.
+func (a *systemApplier) rollback(ctx context.Context, req systemRequest, reason string, j *systemJournal, prevPlan bootstrapPlan) error {
+ a.ops.logf("system apply: %s %s: rolling back: %s", a.channel, j.Target, reason)
+ plan, err := a.restoreFiles(ctx, j)
+ if err != nil {
+ _ = a.writeResult(req, systemOutcomeRollbackFailed, reason+"; restore: "+err.Error())
+ os.Remove(a.file("journal.json"))
+ return fmt.Errorf("system apply: rollback of %s failed: %v", j.Target, err)
+ }
+ if err := a.restartEdge(); err != nil {
+ _ = a.writeResult(req, systemOutcomeRollbackFailed, reason+"; restart: "+err.Error())
+ os.Remove(a.file("journal.json"))
+ return fmt.Errorf("system apply: rollback restart failed: %v", err)
+ }
+ if err := a.gate(ctx, j.PrevBuild); err != nil {
+ _ = a.writeResult(req, systemOutcomeRollbackFailed, reason+"; previous build failed its gate too: "+err.Error())
+ os.Remove(a.file("journal.json"))
+ return fmt.Errorf("system apply: rollback gate failed: %v", err)
+ }
+ if err := a.arm(plan); err != nil {
+ a.ops.logf("system apply: arm previous plan: %v", err)
+ }
+ if err := a.writeResult(req, systemOutcomeRolledBack, reason); err != nil {
+ return err
+ }
+ os.Remove(a.file("journal.json"))
+ return fmt.Errorf("system apply: %s rolled back: %s", j.Target, reason)
+}
+
+// restoreFiles is the files-only half shared by rollback and recovery:
+// carlos.prev over carlos (keeping .prev), units_added disabled and
+// removed, the previous unit set copied back (bringing units_removed
+// back), sysusers, tmpfiles, daemon-reload, and `enable` of the previous
+// plan so it comes up at boot. No restart, no start.
+func (a *systemApplier) restoreFiles(ctx context.Context, j *systemJournal) (bootstrapPlan, error) {
+ prev := systemPrevPath(a.optDir)
+ if _, err := os.Stat(prev); err != nil {
+ return bootstrapPlan{}, fmt.Errorf("no carlos.prev: %w", err)
+ }
+ a.ops.systemctl(append([]string{"stop"}, systemQuiesceUnits...)...)
+ if err := copyFileDurable(prev, systemBinaryPath(a.optDir), 0o755); err != nil {
+ return bootstrapPlan{}, err
+ }
+ prevDir := filepath.Join(a.stagingDir(), "prev")
+ plan, _, err := a.stagePlan(ctx, prev, prevDir)
+ if err != nil {
+ return bootstrapPlan{}, fmt.Errorf("carlos.prev bootstrap: %w", err)
+ }
+ if err := a.installUnits(prevDir, j.UnitsAdded); err != nil {
+ return bootstrapPlan{}, err
+ }
+ if _, err := a.ops.systemctl(append([]string{"enable"}, plan.Enabled...)...); err != nil {
+ return bootstrapPlan{}, err
+ }
+ return plan, nil
+}
+
+// recoverIfNeeded: an open journal means a previous run was cut off.
+// Before gated → back, files only, restart only if the edge has started
+// this boot; gated → forward. The journal is removed LAST.
+func (a *systemApplier) recoverIfNeeded(ctx context.Context) error {
+ j, err := readSystemJournal(a.optDir)
+ if err != nil || j == nil {
+ return err
+ }
+ req, _ := a.readRequest()
+ if req.Version == "" {
+ req = systemRequest{Version: j.Target, PromotedAt: j.StartedAt}
+ }
+ a.ops.logf("system apply: recovering an apply of %s interrupted at %s", j.Target, j.Phase)
+ switch j.Phase {
+ case systemPhaseGated:
+ plan, _, err := a.stagePlan(ctx, systemBinaryPath(a.optDir), filepath.Join(a.stagingDir(), "new"))
+ if err != nil {
+ return fmt.Errorf("system apply: recovery: %w", err)
+ }
+ return a.finish(req, j.Target, plan)
+ case systemPhaseBackingUp:
+ os.Remove(systemPrevPath(a.optDir) + ".tmp")
+ os.Remove(a.file("journal.json"))
+ return nil
+ }
+ plan, err := a.restoreFiles(ctx, j)
+ if err != nil {
+ return fmt.Errorf("system apply: recovery: %w", err) // journal kept: the next run recovers again
+ }
+ if a.ops.edgeStartedThisBoot() {
+ a.ops.systemctl("reset-failed", systemEdgeUnit)
+ if _, err := a.ops.systemctl("restart", "--no-block", systemEdgeUnit); err != nil {
+ return fmt.Errorf("system apply: recovery: restart: %w", err)
+ }
+ var armed []string
+ for _, u := range plan.Enabled {
+ if strings.HasSuffix(u, ".timer") || strings.HasSuffix(u, ".path") {
+ armed = append(armed, u)
+ }
+ }
+ if len(armed) > 0 {
+ a.ops.systemctl(append([]string{"start"}, armed...)...)
+ }
+ }
+ if err := a.writeResult(req, systemOutcomeRolledBack, "interrupted at "+j.Phase); err != nil {
+ return err
+ }
+ os.Remove(a.file("journal.json"))
+ return syncDirBestEffort(a.rootDir())
+}
+
+// RunSystemApply is `carlos system apply [--recover]`: the root oneshot.
+func RunSystemApply(args []string) {
+ fs := flag.NewFlagSet("system apply", flag.ExitOnError)
+ recover := fs.Bool("recover", false, "recover an interrupted apply (files only) and exit: the boot and OnFailure unit")
+ fs.Usage = cliflag.Usage(fs, "")
+ _ = fs.Parse(args)
+ if os.Geteuid() != 0 {
+ log.Fatal("carlos system apply is the root oneshot behind carlos-system-update.service; it is not for operators")
+ }
+ ctx := context.Background()
+ store, err := openDeployStore(ctx)
+ if err != nil {
+ log.Fatalf("system apply: %v", err)
+ }
+ pub, err := release.ParsePublicKey(os.Getenv("CARLOS_RELEASE_PUBKEY"), "CARLOS_RELEASE_PUBKEY")
+ if err != nil {
+ log.Fatalf("system apply: %v", err)
+ }
+ a := &systemApplier{
+ ops: productionSysOps(), store: store, pub: pub, channel: systemChannel(os.Getenv),
+ optDir: release.OptDir(), dataDir: dataDir(), etcDir: "/etc", build: version.Build,
+ platform: runtime.GOOS + "-" + runtime.GOARCH, gateWait: systemGateWait, soak: systemGateSoak,
+ }
+ if err := a.Run(ctx, *recover); err != nil {
+ log.Fatalf("%v", err)
+ }
+}
+
+func productionSysOps() sysOps {
+ return sysOps{
+ systemctl: func(args ...string) (string, error) {
+ out, err := exec.Command("systemctl", args...).CombinedOutput()
+ if err != nil {
+ return string(out), fmt.Errorf("systemctl %s: %w: %s", strings.Join(args, " "), err, strings.TrimSpace(string(out)))
+ }
+ return string(out), nil
+ },
+ run: func(ctx context.Context, timeout time.Duration, name string, args ...string) (string, error) {
+ ctx, cancel := context.WithTimeout(ctx, timeout)
+ defer cancel()
+ out, err := exec.CommandContext(ctx, name, args...).Output()
+ if err != nil {
+ return string(out), fmt.Errorf("%s %s: %w", name, strings.Join(args, " "), err)
+ }
+ return string(out), nil
+ },
+ now: time.Now,
+ sleep: time.Sleep,
+ probeEdge: func() error {
+ // Any HTTP answer on loopback proves the process serves; a box
+ // with no route yet must still pass (spec §6 step 10).
+ c := &http.Client{Timeout: 3 * time.Second, CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }}
+ resp, err := c.Get("http://127.0.0.1/")
+ if err != nil {
+ return err
+ }
+ resp.Body.Close()
+ return nil
+ },
+ edgeStartedThisBoot: func() bool {
+ out, err := exec.Command("systemctl", "show", "-p", "ActiveEnterTimestampMonotonic", "--value", systemEdgeUnit).Output()
+ return err == nil && strings.TrimSpace(string(out)) != "" && strings.TrimSpace(string(out)) != "0"
+ },
+ logf: log.Printf,
+ }
+}
+```
+Imports to add: `"flag"`, `"log"`, `"net/http"`, `"os/exec"`, `"github.com/carlosframework/platform/internal/cliflag"`, `"github.com/carlosframework/platform/internal/version"`. In the fixture, set `f.a.platform = "linux-arm64"` (from 8a). The edge's plain-HTTP listener: confirm the port with `grep -n 'ListenAndServe\|":80"\|Listen(' internal/edge/edge.go | head`; if the edge only listens on 443, probe `https://127.0.0.1/` with `InsecureSkipVerify` and a `ServerName` of `localhost` instead, and say which in the commit.
+
+- [ ] **Step 4: Run the whole host package**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/host/ -count=1 2>&1); rc=$?; echo "$out" | tail -30; echo rc=$rc`
+Expected: `ok`.
+
+- [ ] **Step 5: Commit**
+
+```bash
+gofmt -l internal/host/ && git add internal/host/systemapply.go internal/host/systemapply_test.go && git commit -q -F - <<'EOF'
+host: carlos system apply — rollback, recovery, and the real box
+
+Rollback restores carlos.prev and its unit set, gates it, arms the
+previous plan, and leaves applied.json alone. Recovery is files-only
+and goes forward from phase gated; it restarts the edge only when the
+edge has started this boot, and removes the journal last.
+
+Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
+Claude-Session: https://claude.ai/code/session_015QCUCpfxbCyEAVQ2m51dH5
+EOF
+```
+
+---
+### Task 9: `carlos system` dispatch and the `apply` verb
+
+**Files:**
+- Create: `system.go` (package main), `system_test.go`
+- Modify: `main.go` (the `switch cmd` at `:30-134`, `usage()` at `:139-233`), `update.go` (`noHintVerbs` at `:76-85`)
+
+**Interfaces:**
+- Consumes: `host.RunSystemApply` (Task 8c), `main.go` dispatch pattern (`case "ops": ops.Run(args)`), `usage()` section "On the host".
+- Produces:
+ ```go
+ func runSystem(args []string) // carlos system <verb> | carlos system router <verb>
+ func systemVerb(args []string) (verb string, rest []string, component string) // strips an optional leading "router"
+ ```
+ Verbs in this task: `apply` → `host.RunSystemApply(rest)`; `update`/`status`/`rollback` → `log.Fatal("carlos system " + verb + " arrives in a later task")` placeholders that Task 12 replaces; `help`/none → usage, exit 2.
+
+- [ ] **Step 1: Write the failing test**
+
+```go
+package main
+
+import "testing"
+
+func TestSystemVerbAcceptsTheComponentForm(t *testing.T) {
+ cases := []struct {
+ in []string
+ verb, comp string
+ rest int
+ }{
+ {[]string{"update", "--channel", "canary"}, "update", "router", 2},
+ {[]string{"router", "update"}, "update", "router", 0},
+ {[]string{"router", "status"}, "status", "router", 0},
+ {[]string{"apply", "--recover"}, "apply", "router", 1},
+ {[]string{}, "", "router", 0},
+ {[]string{"router"}, "", "router", 0},
+ }
+ for _, c := range cases {
+ verb, rest, comp := systemVerb(c.in)
+ if verb != c.verb || comp != c.comp || len(rest) != c.rest {
+ t.Errorf("systemVerb(%v) = %q %v %q", c.in, verb, rest, comp)
+ }
+ }
+ if verb, _, _ := systemVerb([]string{"console", "update"}); verb != "" {
+ t.Errorf("an unknown component must not be a verb: got %q", verb)
+ }
+}
+```
+
+- [ ] **Step 2: Run to verify failure**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test . -run TestSystemVerb -count=1 2>&1); rc=$?; echo "$out" | tail -5; echo rc=$rc`
+Expected: undefined `systemVerb`.
+
+- [ ] **Step 3: Implement `system.go`**
+
+```go
+package main
+
+import (
+ "fmt"
+ "log"
+ "os"
+
+ "github.com/carlosframework/platform/internal/host"
+)
+
+// carlos system: what is not an app (spec §0, the console-as-an-app
+// dividing line). One component today, the router — the substrate binary.
+// `carlos system router update` is the same verb scoped to it, so the
+// command Paul named works and a future split needs no new verb.
+
+const systemComponentRouter = "router"
+
+func systemVerb(args []string) (verb string, rest []string, component string) {
+ component = systemComponentRouter
+ if len(args) > 0 && args[0] == systemComponentRouter {
+ args = args[1:]
+ }
+ if len(args) == 0 {
+ return "", nil, component
+ }
+ switch args[0] {
+ case "update", "status", "rollback", "apply", "help", "-h", "--help":
+ return args[0], args[1:], component
+ }
+ return "", args, component
+}
+
+func runSystem(args []string) {
+ verb, rest, _ := systemVerb(args)
+ switch verb {
+ case "apply":
+ host.RunSystemApply(rest)
+ case "update":
+ runSystemUpdate(rest)
+ case "status":
+ runSystemStatus(rest)
+ case "rollback":
+ runSystemRollback(rest)
+ case "help", "-h", "--help":
+ systemUsage()
+ default:
+ if len(args) > 0 && verb == "" {
+ fmt.Fprintf(os.Stderr, "system: unknown verb %q\n\n", args[0])
+ }
+ systemUsage()
+ os.Exit(2)
+ }
+}
+
+func systemUsage() {
+ fmt.Fprint(os.Stderr, `usage: carlos system <verb> [flags]
+ carlos system router <verb> [flags]
+
+ update promote a platform release to a channel; every box on it rolls itself
+ status every box: platform, channel, running build, channel target, last apply outcome
+ rollback point a channel back at the release before this one
+ apply (root oneshot behind carlos-system-update.service; not for operators)
+
+The platform is not an app: the console is updated with carlos deploy --app console.
+`)
+}
+
+// Placeholders until Task 12.
+func runSystemUpdate(args []string) { log.Fatal("carlos system update: not yet wired to the console") }
+func runSystemStatus(args []string) { log.Fatal("carlos system status: not yet wired to the console") }
+func runSystemRollback(args []string) { log.Fatal("carlos system rollback: not yet wired to the console") }
+```
+`main.go`: add `case "system": runSystem(args)` after the `ops` case; in `usage()` under "On the host" add ` system platform self-update: update | status | rollback (apply is the root oneshot)`; in `update.go` `noHintVerbs` add `"system": true`.
+
+- [ ] **Step 4: Run and check the binary**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test . -run 'TestSystemVerb' -count=1 2>&1); rc=$?; echo "$out" | tail -5; echo rc=$rc; go build -o /tmp/c/carlos . && /tmp/c/carlos system help 2>&1 | head -3; /tmp/c/carlos system apply 2>&1 | head -1`
+Expected: PASS; usage printed; `apply` as non-root prints "not for operators".
+
+- [ ] **Step 5: Commit**
+
+```bash
+gofmt -l . && git add system.go system_test.go main.go update.go && git commit -q -F - <<'EOF'
+cli: carlos system, with apply wired to the root oneshot
+
+Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
+Claude-Session: https://claude.ai/code/session_015QCUCpfxbCyEAVQ2m51dH5
+EOF
+```
+
+---
+
+### Task 10: The public release feed (`internal/publicrelease`)
+
+**Files:**
+- Create: `internal/publicrelease/feed.go`, `internal/publicrelease/feed_test.go`
+- Modify: `update.go` (`fetchLatest` `:100-119`, `selfReplace` `:295-327`, `fetchUpdateFile` `:358-369`)
+
+**Interfaces:**
+- Consumes: the package vars `updateLatestURL`, `updateDownloadBase`, `cliUpdatePubKey` in `update.go` (kept, since `update_test.go` repoints them).
+- Produces:
+ ```go
+ package publicrelease
+ type Feed struct {
+ LatestURL string // https://apt.carlosframework.com/latest.txt
+ DownloadBase string // https://github.com/carlosframework/releases/releases/download (tag-pinned; never latest/download)
+ PubKey string // base64 ed25519 public key that signs SHA256SUMS.txt
+ Client *http.Client // nil = a 5-minute client
+ }
+ func Default() Feed
+ var TagRe = regexp.MustCompile(`^v[0-9]+\.[0-9]+\.[0-9]+$`)
+ func (f Feed) Latest(ctx context.Context) (string, error)
+ func (f Feed) Sums(ctx context.Context, tag string) (map[string]string, error) // asset → sha, signature verified
+ func (f Feed) Asset(ctx context.Context, tag, asset string, limit int64) (data []byte, sha256hex string, err error) // verified against Sums
+ func AssetName(platform string) string // "carlos-" + platform
+ ```
+
+- [ ] **Step 1: Write the failing tests**
+
+```go
+package publicrelease
+
+import (
+ "context"
+ "crypto/ed25519"
+ "crypto/sha256"
+ "encoding/base64"
+ "encoding/hex"
+ "fmt"
+ "net/http"
+ "net/http/httptest"
+ "strings"
+ "testing"
+)
+
+func testFeed(t *testing.T, sign bool, assets map[string][]byte) (Feed, string) {
+ t.Helper()
+ pub, priv, _ := ed25519.GenerateKey(nil)
+ var sums strings.Builder
+ for name, body := range assets {
+ s := sha256.Sum256(body)
+ fmt.Fprintf(&sums, "%s %s\n", hex.EncodeToString(s[:]), name)
+ }
+ sig := ed25519.Sign(priv, []byte(sums.String()))
+ mux := http.NewServeMux()
+ mux.HandleFunc("/latest.txt", func(w http.ResponseWriter, r *http.Request) { fmt.Fprintln(w, "v0.18.0") })
+ mux.HandleFunc("/dl/v0.18.0/SHA256SUMS.txt", func(w http.ResponseWriter, r *http.Request) { fmt.Fprint(w, sums.String()) })
+ mux.HandleFunc("/dl/v0.18.0/SHA256SUMS.txt.sig", func(w http.ResponseWriter, r *http.Request) {
+ if sign {
+ fmt.Fprintln(w, base64.StdEncoding.EncodeToString(sig))
+ } else {
+ fmt.Fprintln(w, base64.StdEncoding.EncodeToString(make([]byte, ed25519.SignatureSize)))
+ }
+ })
+ for name, body := range assets {
+ body := body
+ mux.HandleFunc("/dl/v0.18.0/"+name, func(w http.ResponseWriter, r *http.Request) { w.Write(body) })
+ }
+ ts := httptest.NewServer(mux)
+ t.Cleanup(ts.Close)
+ return Feed{LatestURL: ts.URL + "/latest.txt", DownloadBase: ts.URL + "/dl", PubKey: base64.StdEncoding.EncodeToString(pub)}, ts.URL
+}
+
+func TestFeedLatestAndVerifiedAsset(t *testing.T) {
+ body := []byte("ELF arm64")
+ f, _ := testFeed(t, true, map[string][]byte{"carlos-linux-arm64": body, "carlos-linux-amd64": []byte("ELF amd64")})
+ ctx := context.Background()
+ tag, err := f.Latest(ctx)
+ if err != nil || tag != "v0.18.0" {
+ t.Fatalf("latest = %q %v", tag, err)
+ }
+ data, sha, err := f.Asset(ctx, tag, AssetName("linux-arm64"), 1<<20)
+ if err != nil || string(data) != string(body) {
+ t.Fatalf("asset: %v", err)
+ }
+ s := sha256.Sum256(body)
+ if sha != hex.EncodeToString(s[:]) {
+ t.Fatalf("sha = %s", sha)
+ }
+ if _, _, err := f.Asset(ctx, tag, "carlos-windows-amd64.zip", 1<<20); err == nil {
+ t.Fatal("an asset absent from the sums was accepted")
+ }
+}
+
+func TestFeedRefusesABadSignatureAndATamperedAsset(t *testing.T) {
+ body := []byte("ELF")
+ f, _ := testFeed(t, false, map[string][]byte{"carlos-linux-arm64": body})
+ if _, err := f.Sums(context.Background(), "v0.18.0"); err == nil || !strings.Contains(err.Error(), "signature") {
+ t.Fatalf("bad signature: %v", err)
+ }
+ // Signed sums, but the served asset differs from the sum.
+ f2, _ := testFeed(t, true, map[string][]byte{"carlos-linux-arm64": body})
+ f2.DownloadBase = f2.DownloadBase // same server
+ pub, _ := base64.StdEncoding.DecodeString(f2.PubKey)
+ _ = pub
+ // Simulate tampering by asking for a limit smaller than the asset: the
+ // truncated download cannot match its sum.
+ if _, _, err := f2.Asset(context.Background(), "v0.18.0", "carlos-linux-arm64", 1); err == nil {
+ t.Fatal("a truncated asset matched its sum")
+ }
+}
+
+func TestFeedLatestRequiresATag(t *testing.T) {
+ ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { fmt.Fprintln(w, "main") }))
+ defer ts.Close()
+ f := Feed{LatestURL: ts.URL, DownloadBase: ts.URL, PubKey: ""}
+ if _, err := f.Latest(context.Background()); err == nil {
+ t.Fatal("accepted a non-tag latest")
+ }
+}
+```
+
+- [ ] **Step 2: Run to verify failure**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/publicrelease/ -count=1 2>&1); rc=$?; echo "$out" | tail -5; echo rc=$rc`
+Expected: package does not exist.
+
+- [ ] **Step 3: Implement `feed.go`**
+
+```go
+// Package publicrelease reads the vendor's public signed release feed: the
+// latest tag, the signed checksum list, and a verified asset. Two callers
+// share it — `carlos update` on a workstation and the console's `carlos
+// system update` promote — so the verification chain is written once.
+package publicrelease
+
+import (
+ "context"
+ "crypto/ed25519"
+ "crypto/sha256"
+ "encoding/base64"
+ "encoding/hex"
+ "errors"
+ "fmt"
+ "io"
+ "net/http"
+ "regexp"
+ "strings"
+ "time"
+)
+
+type Feed struct {
+ LatestURL string
+ DownloadBase string // tag-pinned: <DownloadBase>/<tag>/<file>; never latest/download
+ PubKey string // base64 ed25519 public key over the raw SHA256SUMS.txt bytes
+ Client *http.Client
+}
+
+// Default is the vendor feed. The strings are duplicated in update.go's
+// package vars so update_test.go can repoint them; keep the two in step.
+func Default() Feed {
+ return Feed{
+ LatestURL: "https://apt.carlosframework.com/latest.txt",
+ DownloadBase: "https://github.com/carlosframework/releases/releases/download",
+ PubKey: "R+EO3qQ47VsuKl9Qq/pkDXBucPYvzp5lU2ut4zpnJPc=",
+ }
+}
+
+var TagRe = regexp.MustCompile(`^v[0-9]+\.[0-9]+\.[0-9]+$`)
+
+func AssetName(platform string) string { return "carlos-" + platform }
+
+func (f Feed) client() *http.Client {
+ if f.Client != nil {
+ return f.Client
+ }
+ return &http.Client{Timeout: 5 * time.Minute}
+}
+
+func (f Feed) get(ctx context.Context, url string, limit int64) ([]byte, error) {
+ req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
+ if err != nil {
+ return nil, err
+ }
+ res, err := f.client().Do(req)
+ if err != nil {
+ return nil, err
+ }
+ defer res.Body.Close()
+ if res.StatusCode != http.StatusOK {
+ return nil, fmt.Errorf("%s: status %d", url, res.StatusCode)
+ }
+ return io.ReadAll(io.LimitReader(res.Body, limit))
+}
+
+// Latest is the newest release tag, and only a tag.
+func (f Feed) Latest(ctx context.Context) (string, error) {
+ b, err := f.get(ctx, f.LatestURL, 128)
+ if err != nil {
+ return "", err
+ }
+ tag := strings.TrimSpace(string(b))
+ if !TagRe.MatchString(tag) {
+ return "", fmt.Errorf("%s: %q is not a release tag", f.LatestURL, tag)
+ }
+ return tag, nil
+}
+
+// Sums fetches SHA256SUMS.txt and its .sig for tag, verifies the signature
+// with PubKey, and returns asset → sha256 hex.
+func (f Feed) Sums(ctx context.Context, tag string) (map[string]string, error) {
+ base := f.DownloadBase + "/" + tag + "/"
+ sums, err := f.get(ctx, base+"SHA256SUMS.txt", 1<<20)
+ if err != nil {
+ return nil, err
+ }
+ sig, err := f.get(ctx, base+"SHA256SUMS.txt.sig", 4096)
+ if err != nil {
+ return nil, fmt.Errorf("%v (a release without a signed checksum list predates self-update)", err)
+ }
+ pub, err := base64.StdEncoding.DecodeString(f.PubKey)
+ if err != nil || len(pub) != ed25519.PublicKeySize {
+ return nil, errors.New("release feed public key is invalid")
+ }
+ rawSig, err := base64.StdEncoding.DecodeString(strings.TrimSpace(string(sig)))
+ if err != nil || !ed25519.Verify(ed25519.PublicKey(pub), sums, rawSig) {
+ return nil, errors.New("SHA256SUMS.txt signature does not verify — refusing the release")
+ }
+ out := map[string]string{}
+ for _, line := range strings.Split(string(sums), "\n") {
+ fields := strings.Fields(line)
+ if len(fields) == 2 {
+ out[fields[1]] = fields[0]
+ }
+ }
+ return out, nil
+}
+
+// Asset fetches one named asset for tag and refuses it unless its sha256
+// matches the signed checksum list.
+func (f Feed) Asset(ctx context.Context, tag, asset string, limit int64) ([]byte, string, error) {
+ sums, err := f.Sums(ctx, tag)
+ if err != nil {
+ return nil, "", err
+ }
+ want, ok := sums[asset]
+ if !ok {
+ return nil, "", fmt.Errorf("no %s in the release's checksum list", asset)
+ }
+ data, err := f.get(ctx, f.DownloadBase+"/"+tag+"/"+asset, limit)
+ if err != nil {
+ return nil, "", err
+ }
+ s := sha256.Sum256(data)
+ got := hex.EncodeToString(s[:])
+ if got != want {
+ return nil, "", fmt.Errorf("%s does not match its signed checksum — refusing it", asset)
+ }
+ return data, got, nil
+}
+```
+`update.go`: replace the bodies of `fetchLatest` and the fetch-and-verify half of `selfReplace` (`:296-327`) with calls through a feed built from the package vars, so tests that repoint the vars still work:
+```go
+func updateFeed() publicrelease.Feed {
+ return publicrelease.Feed{LatestURL: updateLatestURL, DownloadBase: updateDownloadBase, PubKey: cliUpdatePubKey}
+}
+```
+`fetchLatest(timeout)` → `updateFeed()` with `Client: &http.Client{Timeout: timeout}` and `.Latest(ctx)`; in `selfReplace`, `binData, _, err := updateFeed().Asset(ctx, latest, asset, 256<<20)`, keeping the workstation-only half (temp file, lie detector, rename) as is. Delete `fetchUpdateFile` if nothing else calls it. Run `go test . -run 'Update' -count=1` to prove `update_test.go` still passes.
+
+- [ ] **Step 4: Run the tests**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/publicrelease/ . -run 'Feed|Update' -count=1 2>&1); rc=$?; echo "$out" | tail -10; echo rc=$rc`
+Expected: `ok` for both packages.
+
+- [ ] **Step 5: Commit**
+
+```bash
+gofmt -l . internal/publicrelease && git add internal/publicrelease update.go && git commit -q -F - <<'EOF'
+publicrelease: the vendor feed's verification chain, written once
+
+carlos update and the console's carlos system update share it: latest
+tag, signed SHA256SUMS, per-asset sha.
+
+Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
+Claude-Session: https://claude.ai/code/session_015QCUCpfxbCyEAVQ2m51dH5
+EOF
+```
+
+---
+
+### Task 11: The console doors (`internal/console/apisystem.go`)
+
+**Files:**
+- Create: `internal/console/apisystem.go`, `internal/console/apisystem_test.go`
+- Modify: `internal/console/authcli.go` `registerCLIRoutes()` (add five `cliRoute` lines), `internal/console/server.go` `Config` (add `SystemFeed publicrelease.Feed`)
+
+**Interfaces:**
+- Consumes: `s.cliRoute(pattern, svcrole.Set{}, handler)`; `principalFrom(ctx)`, `PrincipalUser`; `s.store.Role(OpsAccountID, addr)`, `RoleOwner`; `s.cfg.Releases`, `s.cfg.Key`, `s.cfg.ReleasePub`; `s.bumpSerial(ctx)`; `writeJSON`, `jsonError`, `readJSONStrictLimit`; `status.LoadBoxes`, `release.NewPrefixStore`, `s.openPassRegistry().ListFleets()` (the `fleetMeters` loop, `economicspass.go:648-670`); `release.PromoteSystem`, `release.RollbackSystem`, `release.GetSystemPointer`, `release.RootSigner`; `publicrelease.Feed`; `apiShipMaxBytes`-style limits.
+- Produces (routes, all person-only, ops-owner-only, bare 404 on refusal like `IsOperator` doors):
+ ```
+ GET /api/cli/system/status → {"pointers": {"canary": {...}|null, "stable": ...}, "boxes": [{label, platform, channel, release, target, system:{...}|null, stale}]}
+ POST /api/cli/system/update → {"job": "<id>"}; body {"channel","version"?,"upload"?,"spread"?,"component"?}
+ GET /api/cli/system/jobs/{id} → {"steps": ["resolved v0.18.0", ...], "done": bool, "error": ""}
+ PUT /api/cli/system/upload → {"upload": "<handle>", "size": n}; multipart: meta {"version","platform"}, file
+ POST /api/cli/system/rollback → {"channel","version"}; body {"channel","component"?}
+ ```
+ ```go
+ func (s *Server) apiSystemOwner(w, r) (addr string, ok bool)
+ type systemJob struct{ mu sync.Mutex; Steps []string; Done bool; Err string }
+ func (s *Server) runSystemUpdate(ctx, job *systemJob, actor string, in systemUpdateRequest)
+ ```
+ Job and upload registries are fields on `Server`: `systemJobs map[string]*systemJob`, `systemUploads map[string]systemUpload`, guarded by `systemMu sync.Mutex`; one upload slot (128 MiB cap) held in memory, consumed by the job that names it.
+
+- [ ] **Step 1: Write the failing tests**
+
+```go
+package console
+
+import (
+ "bytes"
+ "context"
+ "crypto/ed25519"
+ "crypto/sha256"
+ "encoding/base64"
+ "encoding/hex"
+ "fmt"
+ "mime/multipart"
+ "net/http"
+ "net/http/httptest"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/carlosframework/platform/internal/publicrelease"
+ "github.com/carlosframework/platform/internal/release"
+ "github.com/carlosframework/platform/internal/status"
+)
+
+// systemFeed serves a signed fake release from an httptest server and
+// points the fixture's console at it.
+func systemFeed(t *testing.T, fx *consoleFixture, assets map[string][]byte) {
+ t.Helper()
+ pub, priv, _ := ed25519.GenerateKey(nil)
+ var sums strings.Builder
+ for name, body := range assets {
+ s := sha256.Sum256(body)
+ fmt.Fprintf(&sums, "%s %s\n", hex.EncodeToString(s[:]), name)
+ }
+ sig := base64.StdEncoding.EncodeToString(ed25519.Sign(priv, []byte(sums.String())))
+ mux := http.NewServeMux()
+ mux.HandleFunc("/latest.txt", func(w http.ResponseWriter, r *http.Request) { fmt.Fprintln(w, "v0.18.0") })
+ mux.HandleFunc("/dl/v0.18.0/SHA256SUMS.txt", func(w http.ResponseWriter, r *http.Request) { fmt.Fprint(w, sums.String()) })
+ mux.HandleFunc("/dl/v0.18.0/SHA256SUMS.txt.sig", func(w http.ResponseWriter, r *http.Request) { fmt.Fprintln(w, sig) })
+ for name, body := range assets {
+ body := body
+ mux.HandleFunc("/dl/v0.18.0/"+name, func(w http.ResponseWriter, r *http.Request) { w.Write(body) })
+ }
+ ts := httptest.NewServer(mux)
+ t.Cleanup(ts.Close)
+ fx.srv.cfg.SystemFeed = publicrelease.Feed{LatestURL: ts.URL + "/latest.txt", DownloadBase: ts.URL + "/dl", PubKey: base64.StdEncoding.EncodeToString(pub)}
+}
+
+func waitJob(t *testing.T, fx *consoleFixture, token, id string) map[string]any {
+ t.Helper()
+ for i := 0; i < 100; i++ {
+ st, out := getJSON(t, fx.ts.URL+"/api/cli/system/jobs/"+id, token)
+ if st != http.StatusOK {
+ t.Fatalf("job = %d %v", st, out)
+ }
+ if out["done"] == true {
+ return out
+ }
+ time.Sleep(20 * time.Millisecond)
+ }
+ t.Fatal("job never finished")
+ return nil
+}
+
+func seedBox(t *testing.T, fx *consoleFixture, label, platform, channel, rel string) {
+ t.Helper()
+ b := status.CollectBox(nil, status.Probes{Release: rel, Platform: platform, SystemChannel: channel}, time.Now())
+ b.Label = label
+ if err := status.WriteBox(context.Background(), fx.rel, b); err != nil {
+ t.Fatal(err)
+ }
+}
+
+func TestSystemDoorsAreForOpsOwnersOnly(t *testing.T) {
+ fx := newConsoleFixture(t)
+ if err := fx.st.AddMember(OpsAccountID, "member@keymail.dev", RoleMember, time.Now()); err != nil {
+ t.Fatal(err)
+ }
+ member := cliToken(t, fx, "member@keymail.dev", "code-sys-member")
+ for _, u := range []string{"/api/cli/system/status"} {
+ if st, _ := getJSON(t, fx.ts.URL+u, member); st != http.StatusNotFound {
+ t.Errorf("GET %s as a member = %d, want 404", u, st)
+ }
+ }
+ if st, _ := postJSON(t, fx.ts.URL+"/api/cli/system/update", map[string]string{"channel": "canary"}, member); st != http.StatusNotFound {
+ t.Errorf("update as a member = %d, want 404", st)
+ }
+ if st, _ := postJSON(t, fx.ts.URL+"/api/cli/system/rollback", map[string]string{"channel": "canary"}, member); st != http.StatusNotFound {
+ t.Errorf("rollback as a member = %d, want 404", st)
+ }
+ if st, _ := getJSON(t, fx.ts.URL+"/api/cli/system/status", "not-a-token"); st == http.StatusOK {
+ t.Error("unauthenticated status answered 200")
+ }
+}
+
+func TestSystemUpdateMirrorsSignsAndBumps(t *testing.T) {
+ fx := newConsoleFixture(t)
+ arm := []byte("ELF arm64")
+ systemFeed(t, fx, map[string][]byte{"carlos-linux-arm64": arm, "carlos-linux-amd64": []byte("ELF amd64")})
+ seedBox(t, fx, "ie", "linux-arm64", "canary", "v0.17.0")
+ owner := cliToken(t, fx, "paul@keymail.dev", "code-sys-owner")
+ before, _, _ := release.ReadSerial(context.Background(), fx.rel)
+ st, out := postJSON(t, fx.ts.URL+"/api/cli/system/update", map[string]any{"channel": "canary", "spread": "2m"}, owner)
+ if st != http.StatusOK {
+ t.Fatalf("update = %d %v", st, out)
+ }
+ job := waitJob(t, fx, owner, str(t, out, "job"))
+ if job["error"] != "" {
+ t.Fatalf("job failed: %v", job)
+ }
+ p, _, err := release.GetSystemPointer(context.Background(), fx.rel, fx.pub, "router", "canary")
+ if err != nil {
+ t.Fatal(err)
+ }
+ if p.Current.Version != "v0.18.0" || p.Spread != "2m0s" || p.Current.Source != "release:v0.18.0" {
+ t.Fatalf("pointer = %+v", p)
+ }
+ a, ok := p.Current.Artifacts["linux-arm64"]
+ if !ok {
+ t.Fatalf("no arm64 artifact: %+v", p.Current.Artifacts)
+ }
+ if _, ok := p.Current.Artifacts["linux-amd64"]; ok {
+ t.Fatal("mirrored a platform no box reports")
+ }
+ body, _, err := fx.rel.Get(context.Background(), a.Key)
+ if err != nil || !bytes.Equal(body, arm) {
+ t.Fatalf("artifact not mirrored: %v", err)
+ }
+ s := sha256.Sum256(arm)
+ if a.SHA256 != hex.EncodeToString(s[:]) {
+ t.Fatalf("sha = %s", a.SHA256)
+ }
+ after, _, _ := release.ReadSerial(context.Background(), fx.rel)
+ if after.Serial == before.Serial {
+ t.Fatal("serial not bumped")
+ }
+ // A second promote of the same version to stable copies nothing and
+ // refuses different bytes under the same version.
+ st, out = postJSON(t, fx.ts.URL+"/api/cli/system/update", map[string]any{"channel": "stable", "version": "v0.18.0"}, owner)
+ if st != http.StatusOK {
+ t.Fatalf("second update = %d %v", st, out)
+ }
+ if job := waitJob(t, fx, owner, str(t, out, "job")); job["error"] != "" {
+ t.Fatalf("second job: %v", job)
+ }
+ if _, err := fx.rel.Put(context.Background(), a.Key, []byte("different"), ""); err == nil {
+ // Overwrote the object out from under the pointer; a re-promote must refuse.
+ st, out = postJSON(t, fx.ts.URL+"/api/cli/system/update", map[string]any{"channel": "canary", "version": "v0.18.0"}, owner)
+ job := waitJob(t, fx, owner, str(t, out, "job"))
+ if job["error"] == "" || !strings.Contains(job["error"].(string), "immutable") {
+ t.Fatalf("re-promote over different bytes: %v", job)
+ }
+ }
+}
+
+func TestSystemUpdateFromAnUpload(t *testing.T) {
+ fx := newConsoleFixture(t)
+ owner := cliToken(t, fx, "paul@keymail.dev", "code-sys-upload")
+ var buf bytes.Buffer
+ mw := multipart.NewWriter(&buf)
+ meta, _ := mw.CreateFormField("meta")
+ meta.Write([]byte(`{"version":"66587fa","platform":"linux-arm64"}`))
+ file, _ := mw.CreateFormFile("file", "carlos-linux-arm64")
+ file.Write([]byte("ELF local"))
+ mw.Close()
+ req, _ := http.NewRequest(http.MethodPut, fx.ts.URL+"/api/cli/system/upload", &buf)
+ req.Header.Set("Content-Type", mw.FormDataContentType())
+ req.Header.Set("Authorization", "Bearer "+owner)
+ res, err := http.DefaultClient.Do(req)
+ if err != nil {
+ t.Fatal(err)
+ }
+ var up map[string]any
+ if res.StatusCode != http.StatusOK {
+ t.Fatalf("upload = %d", res.StatusCode)
+ }
+ if err := json.NewDecoder(res.Body).Decode(&up); err != nil {
+ t.Fatal(err)
+ }
+ st, out := postJSON(t, fx.ts.URL+"/api/cli/system/update", map[string]any{"channel": "canary", "upload": up["upload"]}, owner)
+ if st != http.StatusOK {
+ t.Fatalf("update = %d %v", st, out)
+ }
+ if job := waitJob(t, fx, owner, str(t, out, "job")); job["error"] != "" {
+ t.Fatalf("job: %v", job)
+ }
+ p, _, err := release.GetSystemPointer(context.Background(), fx.rel, fx.pub, "router", "canary")
+ if err != nil || p.Current.Version != "66587fa" || p.Current.Source != "file" {
+ t.Fatalf("pointer = %+v %v", p, err)
+ }
+ // Upload without a platform is refused at the door.
+ var buf2 bytes.Buffer
+ mw2 := multipart.NewWriter(&buf2)
+ m2, _ := mw2.CreateFormField("meta")
+ m2.Write([]byte(`{"version":"abc"}`))
+ f2, _ := mw2.CreateFormFile("file", "x")
+ f2.Write([]byte("x"))
+ mw2.Close()
+ req2, _ := http.NewRequest(http.MethodPut, fx.ts.URL+"/api/cli/system/upload", &buf2)
+ req2.Header.Set("Content-Type", mw2.FormDataContentType())
+ req2.Header.Set("Authorization", "Bearer "+owner)
+ res2, _ := http.DefaultClient.Do(req2)
+ if res2.StatusCode != http.StatusBadRequest {
+ t.Fatalf("upload without platform = %d", res2.StatusCode)
+ }
+}
+
+func TestSystemStatusAndRollback(t *testing.T) {
+ fx := newConsoleFixture(t)
+ owner := cliToken(t, fx, "paul@keymail.dev", "code-sys-status")
+ seedBox(t, fx, "ie", "linux-arm64", "canary", "v1")
+ sg := release.RootSigner(fx.priv)
+ rel := func(v string) release.SystemRelease {
+ return release.SystemRelease{Version: v, Artifacts: map[string]release.SystemArtifact{"linux-arm64": {Key: release.SystemArtifactKey(v, "linux-arm64"), SHA256: strings.Repeat("a", 64)}}}
+ }
+ release.PromoteSystem(context.Background(), fx.rel, sg, "router", "canary", rel("v1"), release.SystemPromoteOptions{})
+ time.Sleep(1100 * time.Millisecond)
+ release.PromoteSystem(context.Background(), fx.rel, sg, "router", "canary", rel("v2"), release.SystemPromoteOptions{})
+ st, out := getJSON(t, fx.ts.URL+"/api/cli/system/status", owner)
+ if st != http.StatusOK {
+ t.Fatalf("status = %d %v", st, out)
+ }
+ boxes := out["boxes"].([]any)
+ if len(boxes) != 1 {
+ t.Fatalf("boxes = %v", boxes)
+ }
+ b := boxes[0].(map[string]any)
+ if b["label"] != "ie" || b["release"] != "v1" || b["target"] != "v2" || b["channel"] != "canary" {
+ t.Fatalf("box row = %v", b)
+ }
+ time.Sleep(1100 * time.Millisecond)
+ st, out = postJSON(t, fx.ts.URL+"/api/cli/system/rollback", map[string]string{"channel": "canary"}, owner)
+ if st != http.StatusOK || str(t, out, "version") != "v1" {
+ t.Fatalf("rollback = %d %v", st, out)
+ }
+ if st, out = postJSON(t, fx.ts.URL+"/api/cli/system/rollback", map[string]string{"channel": "canary"}, owner); st != http.StatusConflict {
+ t.Fatalf("second rollback = %d %v (nothing left in history)", st, out)
+ }
+}
+```
+(add `"encoding/json"` to the imports.)
+
+- [ ] **Step 2: Run to verify failure**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/console/ -run 'TestSystem' -count=1 2>&1); rc=$?; echo "$out" | tail -10; echo rc=$rc`
+Expected: build failure (`SystemFeed` undefined) or 404s.
+
+- [ ] **Step 3: Implement**
+
+`server.go` `Config` gains, next to `Releases`:
+```go
+ // SystemFeed is where `carlos system update` fetches a public release
+ // from. Zero value = the vendor feed; tests point it at a stub.
+ SystemFeed publicrelease.Feed
+```
+and `Server` gains `systemMu sync.Mutex; systemJobs map[string]*systemJob; systemUploads map[string]systemUpload`.
+
+`authcli.go` `registerCLIRoutes()`:
+```go
+ // carlos system (spec §5): ops-account owners, person sessions only.
+ s.cliRoute("GET /api/cli/system/status", svcrole.Set{}, s.handleAPISystemStatus)
+ s.cliRoute("POST /api/cli/system/update", svcrole.Set{}, s.handleAPISystemUpdate)
+ s.cliRoute("GET /api/cli/system/jobs/{id}", svcrole.Set{}, s.handleAPISystemJob)
+ s.cliRoute("PUT /api/cli/system/upload", svcrole.Set{}, s.handleAPISystemUpload)
+ s.cliRoute("POST /api/cli/system/rollback", svcrole.Set{}, s.handleAPISystemRollback)
+```
+
+`apisystem.go`:
+```go
+package console
+
+import (
+ "context"
+ "crypto/rand"
+ "crypto/sha256"
+ "encoding/hex"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io"
+ "log"
+ "net/http"
+ "sort"
+ "strings"
+ "sync"
+ "time"
+
+ "github.com/carlosframework/platform/internal/publicrelease"
+ "github.com/carlosframework/platform/internal/release"
+ "github.com/carlosframework/platform/internal/status"
+)
+
+// The console half of `carlos system` (spec §5). Every door here is for an
+// OWNER of the ops account on a PERSON session, checked on every call;
+// refusal is a bare 404 like the operator doors, so a non-owner learns
+// nothing. The console holds the bucket, the signing key and the egress;
+// the operator's laptop holds none of them.
+
+const (
+ systemUploadMaxBytes = int64(128 << 20)
+ systemAssetMaxBytes = int64(256 << 20)
+ systemJobTTL = time.Hour
+)
+
+var systemPlatformRe = targetRe // apicli.go: ^[a-z0-9]+-[a-z0-9]+$
+
+type systemJob struct {
+ mu sync.Mutex
+ Steps []string
+ Done bool
+ Err string
+ at time.Time
+}
+
+func (j *systemJob) step(format string, args ...any) {
+ j.mu.Lock()
+ j.Steps = append(j.Steps, fmt.Sprintf(format, args...))
+ j.mu.Unlock()
+}
+
+func (j *systemJob) finish(err error) {
+ j.mu.Lock()
+ j.Done = true
+ if err != nil {
+ j.Err = err.Error()
+ }
+ j.mu.Unlock()
+}
+
+type systemUpload struct {
+ version, platform string
+ data []byte
+ at time.Time
+}
+
+func (s *Server) apiSystemOwner(w http.ResponseWriter, r *http.Request) (string, bool) {
+ p, ok := principalFrom(r.Context())
+ if !ok || p.Kind != PrincipalUser {
+ http.NotFound(w, r)
+ return "", false
+ }
+ role, err := s.store.Role(OpsAccountID, p.Address)
+ if err != nil {
+ log.Printf("console: api system: role: %v", err)
+ http.NotFound(w, r)
+ return "", false
+ }
+ if role != RoleOwner {
+ http.NotFound(w, r)
+ return "", false
+ }
+ return p.Address, true
+}
+
+func (s *Server) systemFeed() publicrelease.Feed {
+ if s.cfg.SystemFeed.DownloadBase == "" {
+ return publicrelease.Default()
+ }
+ return s.cfg.SystemFeed
+}
+
+// systemBoxes reads every heartbeat: the platform prefix and each fleet's.
+func (s *Server) systemBoxes(ctx context.Context) ([]status.Box, error) {
+ boxes, err := status.LoadBoxes(ctx, s.cfg.Releases)
+ if err != nil {
+ return nil, err
+ }
+ reg, err := s.openPassRegistry()
+ if err != nil {
+ return boxes, nil // no registry: platform boxes only
+ }
+ fleets, err := reg.ListFleets()
+ if err != nil {
+ return boxes, nil
+ }
+ for _, f := range fleets {
+ fb, err := status.LoadBoxes(ctx, release.NewPrefixStore(s.cfg.Releases, "fleets/"+f.Name+"/"))
+ if err != nil {
+ log.Printf("console: api system: fleet %s boxes: %v", f.Name, err)
+ continue
+ }
+ boxes = append(boxes, fb...)
+ }
+ return boxes, nil
+}
+
+type systemUpdateRequest struct {
+ Component string `json:"component"`
+ Channel string `json:"channel"`
+ Version string `json:"version"`
+ Upload string `json:"upload"`
+ Spread string `json:"spread"`
+}
+
+func (s *Server) handleAPISystemUpdate(w http.ResponseWriter, r *http.Request) {
+ addr, ok := s.apiSystemOwner(w, r)
+ if !ok {
+ return
+ }
+ if s.cfg.Key == nil {
+ jsonError(w, http.StatusServiceUnavailable, "no signing key on this host — the console is running read-only")
+ return
+ }
+ var in systemUpdateRequest
+ if !readJSONStrictLimit(w, r, &in, 4<<10) {
+ return
+ }
+ if in.Component == "" {
+ in.Component = release.SystemComponentRouter
+ }
+ if in.Component != release.SystemComponentRouter {
+ jsonError(w, http.StatusBadRequest, "unknown component "+in.Component)
+ return
+ }
+ if in.Channel != release.RungCanary && in.Channel != release.RungStable {
+ jsonError(w, http.StatusBadRequest, "channel must be canary or stable")
+ return
+ }
+ spread := 5 * time.Minute
+ if in.Spread != "" {
+ d, err := time.ParseDuration(in.Spread)
+ if err != nil || d < 0 || d > 24*time.Hour {
+ jsonError(w, http.StatusBadRequest, "spread must be a duration between 0 and 24h")
+ return
+ }
+ spread = d
+ }
+ id := s.newSystemJob()
+ job := s.systemJobs[id]
+ go s.runSystemUpdate(context.WithoutCancel(r.Context()), job, addr, in, spread)
+ writeJSON(w, http.StatusOK, map[string]string{"job": id})
+}
+
+func (s *Server) newSystemJob() string {
+ b := make([]byte, 8)
+ rand.Read(b)
+ id := hex.EncodeToString(b)
+ s.systemMu.Lock()
+ defer s.systemMu.Unlock()
+ if s.systemJobs == nil {
+ s.systemJobs = map[string]*systemJob{}
+ }
+ for k, j := range s.systemJobs {
+ if time.Since(j.at) > systemJobTTL {
+ delete(s.systemJobs, k)
+ }
+ }
+ s.systemJobs[id] = &systemJob{at: time.Now()}
+ return id
+}
+
+// runSystemUpdate is the promote: resolve, ingest (or take the upload),
+// mirror, sign, bump. Every step is idempotent, so a lost job is re-run.
+func (s *Server) runSystemUpdate(ctx context.Context, job *systemJob, actor string, in systemUpdateRequest, spread time.Duration) {
+ err := func() error {
+ var rel release.SystemRelease
+ if in.Upload != "" {
+ s.systemMu.Lock()
+ up, ok := s.systemUploads[in.Upload]
+ delete(s.systemUploads, in.Upload)
+ s.systemMu.Unlock()
+ if !ok {
+ return errors.New("upload handle unknown or already consumed")
+ }
+ if in.Version != "" && in.Version != up.version {
+ return fmt.Errorf("upload is %s, request says %s", up.version, in.Version)
+ }
+ job.step("using uploaded %s for %s", up.version, up.platform)
+ sum := sha256.Sum256(up.data)
+ rel = release.SystemRelease{Version: up.version, Source: "file", Artifacts: map[string]release.SystemArtifact{}}
+ if err := s.mirrorSystemArtifact(ctx, job, up.version, up.platform, up.data, hex.EncodeToString(sum[:])); err != nil {
+ return err
+ }
+ rel.Artifacts[up.platform] = release.SystemArtifact{Key: release.SystemArtifactKey(up.version, up.platform), SHA256: hex.EncodeToString(sum[:])}
+ } else {
+ feed := s.systemFeed()
+ tag := in.Version
+ if tag == "" {
+ var err error
+ if tag, err = feed.Latest(ctx); err != nil {
+ return err
+ }
+ }
+ if !publicrelease.TagRe.MatchString(tag) {
+ return fmt.Errorf("%q is not a release tag; use --file for a local build", tag)
+ }
+ job.step("resolved %s", tag)
+ platforms, err := s.systemPlatforms(ctx)
+ if err != nil {
+ return err
+ }
+ rel = release.SystemRelease{Version: tag, Source: "release:" + tag, Artifacts: map[string]release.SystemArtifact{}}
+ for _, plat := range platforms {
+ data, sha, err := feed.Asset(ctx, tag, publicrelease.AssetName(plat), systemAssetMaxBytes)
+ if err != nil {
+ return fmt.Errorf("%s: %w", plat, err)
+ }
+ job.step("verified %s (%d bytes)", publicrelease.AssetName(plat), len(data))
+ if err := s.mirrorSystemArtifact(ctx, job, tag, plat, data, sha); err != nil {
+ return err
+ }
+ rel.Artifacts[plat] = release.SystemArtifact{Key: release.SystemArtifactKey(tag, plat), SHA256: sha}
+ }
+ }
+ p, err := release.PromoteSystem(ctx, s.cfg.Releases, release.RootSigner(s.cfg.Key), in.Component, in.Channel, rel,
+ release.SystemPromoteOptions{Actor: actor, Spread: spread})
+ if err != nil {
+ return err
+ }
+ job.step("pointed %s/%s at %s (spread %s)", in.Component, in.Channel, p.Current.Version, p.Spread)
+ s.audit(nil, "platform", actor, "system-update", "pointed "+in.Component+"/"+in.Channel+" at "+p.Current.Version)
+ s.bumpSerial(ctx)
+ job.step("serial bumped: boxes on %s start their spread", in.Channel)
+ return nil
+ }()
+ job.finish(err)
+}
+
+// systemPlatforms: the distinct platforms the fleet reports; linux-arm64
+// when nothing has reported the field yet.
+func (s *Server) systemPlatforms(ctx context.Context) ([]string, error) {
+ boxes, err := s.systemBoxes(ctx)
+ if err != nil {
+ return nil, err
+ }
+ set := map[string]bool{}
+ for _, b := range boxes {
+ if b.Metrics != nil && b.Metrics.Platform != "" {
+ set[b.Metrics.Platform] = true
+ }
+ }
+ if len(set) == 0 {
+ set["linux-arm64"] = true
+ }
+ var out []string
+ for p := range set {
+ out = append(out, p)
+ }
+ sort.Strings(out)
+ return out, nil
+}
+
+// mirrorSystemArtifact: present with the same sha → skip; present with a
+// different sha → refuse (versions are immutable, spec §3).
+func (s *Server) mirrorSystemArtifact(ctx context.Context, job *systemJob, version, platform string, data []byte, sha string) error {
+ key := release.SystemArtifactKey(version, platform)
+ existing, etag, err := s.cfg.Releases.Get(ctx, key)
+ switch {
+ case err == nil:
+ sum := sha256.Sum256(existing)
+ if hex.EncodeToString(sum[:]) == sha {
+ job.step("%s already mirrored", key)
+ return nil
+ }
+ return fmt.Errorf("%s exists with different bytes — versions are immutable; pick a new version", key)
+ case errors.Is(err, os.ErrNotExist):
+ etag = ""
+ default:
+ return err
+ }
+ if _, err := s.cfg.Releases.Put(ctx, key, data, etag); err != nil {
+ return err
+ }
+ job.step("mirrored %s", key)
+ return nil
+}
+
+func (s *Server) handleAPISystemJob(w http.ResponseWriter, r *http.Request) {
+ if _, ok := s.apiSystemOwner(w, r); !ok {
+ return
+ }
+ s.systemMu.Lock()
+ job, ok := s.systemJobs[r.PathValue("id")]
+ s.systemMu.Unlock()
+ if !ok {
+ jsonError(w, http.StatusNotFound, "no such job — the console may have restarted; re-run, every step is idempotent")
+ return
+ }
+ job.mu.Lock()
+ defer job.mu.Unlock()
+ writeJSON(w, http.StatusOK, map[string]any{"steps": append([]string(nil), job.Steps...), "done": job.Done, "error": job.Err})
+}
+
+func (s *Server) handleAPISystemUpload(w http.ResponseWriter, r *http.Request) {
+ if _, ok := s.apiSystemOwner(w, r); !ok {
+ return
+ }
+ r.Body = http.MaxBytesReader(w, r.Body, systemUploadMaxBytes+64<<10)
+ mr, err := r.MultipartReader()
+ if err != nil {
+ jsonError(w, http.StatusBadRequest, "multipart body required")
+ return
+ }
+ var meta struct {
+ Version string `json:"version"`
+ Platform string `json:"platform"`
+ }
+ first, err := mr.NextPart()
+ if err != nil || first.FormName() != "meta" {
+ jsonError(w, http.StatusBadRequest, "first part must be meta")
+ return
+ }
+ if err := json.NewDecoder(io.LimitReader(first, 4<<10)).Decode(&meta); err != nil {
+ jsonError(w, http.StatusBadRequest, "meta: "+err.Error())
+ return
+ }
+ if !release.ValidVersion(meta.Version) || meta.Version == "dev" {
+ jsonError(w, http.StatusBadRequest, "meta.version is required and must be a version")
+ return
+ }
+ if !systemPlatformRe.MatchString(meta.Platform) {
+ jsonError(w, http.StatusBadRequest, "meta.platform is required (e.g. linux-arm64): the console cannot run a foreign binary to ask it")
+ return
+ }
+ part, err := mr.NextPart()
+ if err != nil || part.FormName() != "file" {
+ jsonError(w, http.StatusBadRequest, "second part must be file")
+ return
+ }
+ data, err := io.ReadAll(io.LimitReader(part, systemUploadMaxBytes+1))
+ if err != nil {
+ jsonError(w, http.StatusBadRequest, "file: "+err.Error())
+ return
+ }
+ if int64(len(data)) > systemUploadMaxBytes {
+ jsonError(w, http.StatusRequestEntityTooLarge, "file exceeds 128 MiB")
+ return
+ }
+ b := make([]byte, 8)
+ rand.Read(b)
+ handle := hex.EncodeToString(b)
+ s.systemMu.Lock()
+ if s.systemUploads == nil {
+ s.systemUploads = map[string]systemUpload{}
+ }
+ for k := range s.systemUploads { // one slot: a forgotten upload is not a memory leak
+ delete(s.systemUploads, k)
+ }
+ s.systemUploads[handle] = systemUpload{version: meta.Version, platform: meta.Platform, data: data, at: time.Now()}
+ s.systemMu.Unlock()
+ writeJSON(w, http.StatusOK, map[string]any{"upload": handle, "size": len(data)})
+}
+
+func (s *Server) handleAPISystemRollback(w http.ResponseWriter, r *http.Request) {
+ addr, ok := s.apiSystemOwner(w, r)
+ if !ok {
+ return
+ }
+ if s.cfg.Key == nil {
+ jsonError(w, http.StatusServiceUnavailable, "no signing key on this host — the console is running read-only")
+ return
+ }
+ var in struct {
+ Component string `json:"component"`
+ Channel string `json:"channel"`
+ }
+ if !readJSONStrictLimit(w, r, &in, 1<<10) {
+ return
+ }
+ if in.Component == "" {
+ in.Component = release.SystemComponentRouter
+ }
+ p, err := release.RollbackSystem(r.Context(), s.cfg.Releases, release.RootSigner(s.cfg.Key), in.Component, in.Channel, addr)
+ switch {
+ case errors.Is(err, release.ErrNoSystemHistory):
+ jsonError(w, http.StatusConflict, err.Error())
+ return
+ case errors.Is(err, os.ErrNotExist):
+ jsonError(w, http.StatusNotFound, "no "+in.Channel+" pointer to roll back")
+ return
+ case err != nil:
+ jsonError(w, http.StatusInternalServerError, err.Error())
+ return
+ }
+ s.audit(nil, "platform", addr, "system-rollback", "pointed "+in.Component+"/"+in.Channel+" back at "+p.Current.Version)
+ s.bumpSerial(r.Context())
+ writeJSON(w, http.StatusOK, map[string]string{"channel": in.Channel, "version": p.Current.Version})
+}
+
+func (s *Server) handleAPISystemStatus(w http.ResponseWriter, r *http.Request) {
+ if _, ok := s.apiSystemOwner(w, r); !ok {
+ return
+ }
+ ctx := r.Context()
+ pointers := map[string]any{}
+ targets := map[string]string{}
+ for _, ch := range []string{release.RungCanary, release.RungStable} {
+ p, _, err := release.GetSystemPointer(ctx, s.cfg.Releases, s.cfg.ReleasePub, release.SystemComponentRouter, ch)
+ if err != nil {
+ pointers[ch] = nil
+ continue
+ }
+ pointers[ch] = map[string]any{"version": p.Current.Version, "promoted_at": p.PromotedAt, "spread": p.Spread, "source": p.Current.Source,
+ "history": len(p.History)}
+ targets[ch] = p.Current.Version
+ }
+ boxes, err := s.systemBoxes(ctx)
+ if err != nil {
+ jsonError(w, http.StatusBadGateway, "box heartbeats unreadable: "+err.Error())
+ return
+ }
+ now := time.Now()
+ rows := make([]map[string]any, 0, len(boxes))
+ for _, b := range boxes {
+ row := map[string]any{"label": b.Label, "region": b.Region, "stale": status.BoxStale(b, now, status.SnapshotPeriod),
+ "platform": "", "channel": "", "release": "", "target": "", "system": nil}
+ if b.Metrics != nil {
+ ch := b.Metrics.SystemChannel
+ if ch == "" {
+ ch = "stable"
+ }
+ row["platform"], row["channel"], row["release"] = b.Metrics.Platform, ch, b.Metrics.Release
+ row["target"] = targets[ch]
+ if b.Metrics.System != nil {
+ row["system"] = b.Metrics.System
+ }
+ }
+ rows = append(rows, row)
+ }
+ sort.Slice(rows, func(i, j int) bool { return rows[i]["label"].(string) < rows[j]["label"].(string) })
+ writeJSON(w, http.StatusOK, map[string]any{"pointers": pointers, "boxes": rows})
+}
+```
+Check three things against the code while writing: `s.audit`'s signature (`apicli.go:855` passes `(acct, app, addr, action, detail)`; if `acct` cannot be nil, use the ops account fetched by id, or add a `s.auditSystem(addr, action, detail)` writing an event row against `OpsAccountID`); `status.BoxStale`'s signature (`fleet.go:137`); `os` import. `readJSONStrictLimit` is at `authcli.go:849`.
+
+- [ ] **Step 4: Run the console tests**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/console/ -count=1 2>&1); rc=$?; echo "$out" | tail -20; echo rc=$rc`
+Expected: `ok`. `TestEveryCLIDoorNamesItsRoles` and `TestEveryClosedDoorRefusesAServiceCredential` cover the five new doors automatically; if either fails, the route line is wrong, not the test.
+
+- [ ] **Step 5: Commit**
+
+```bash
+gofmt -l internal/console/ && git add internal/console/apisystem.go internal/console/apisystem_test.go internal/console/authcli.go internal/console/server.go && git commit -q -F - <<'EOF'
+console: the carlos system doors — status, update job, upload, rollback
+
+Ops-account owners on a person session, bare 404 otherwise. The
+update is a job: resolve, ingest from the signed public feed (or an
+upload), mirror per reported platform, sign the pointer, bump the
+serial. Versions are immutable in the bucket.
+
+Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
+Claude-Session: https://claude.ai/code/session_015QCUCpfxbCyEAVQ2m51dH5
+EOF
+```
+
+---
+### Task 12: `carlos system update | status | rollback` (API mode)
+
+**Files:**
+- Modify: `system.go` (replace the three placeholders), `system_test.go`
+
+**Interfaces:**
+- Consumes: `requireDeployAPISession(consoleFlag) *apiSession` (`deploy.go:1151`); `(*apiSession).getJSON/postJSON` (`apimode.go:737,799`); `(*consoleClient).doMultipart` (`auth.go:485`) via `sess.c`; `writeFilePart(mw, name, path)` (`apimode.go:3009`); `interactiveTerminal`, `promptYes` (`account_choose.go`); `stdout`, `colorFor`, `mutedf` (`color.go`); `flagUsage`.
+- Produces:
+ ```go
+ func runSystemUpdate(args []string) // --console --channel --version --file --platform --spread --yes
+ func runSystemStatus(args []string) // --console --channel
+ func runSystemRollback(args []string) // --console --channel --yes
+ func printSystemStatus(w io.Writer, resp systemStatusResponse, channel string)
+ type systemStatusResponse struct{ Pointers map[string]*systemPointerRow; Boxes []systemBoxRow }
+ ```
+
+- [ ] **Step 1: Write the failing test** (the pure renderer; the doors are covered in Task 11)
+
+```go
+func TestPrintSystemStatusSaysWhatEachBoxIsDoing(t *testing.T) {
+ resp := systemStatusResponse{
+ Pointers: map[string]*systemPointerRow{"canary": {Version: "v2", Spread: "5m0s"}, "stable": nil},
+ Boxes: []systemBoxRow{
+ {Label: "ie", Platform: "linux-arm64", Channel: "canary", Release: "v2", Target: "v2"},
+ {Label: "us", Platform: "linux-arm64", Channel: "canary", Release: "v1", Target: "v2",
+ System: &systemStateRow{Version: "v2", Outcome: "rolled-back", Reason: "gate: carlos-edge is failed"}},
+ {Label: "au", Platform: "", Channel: "stable", Release: "v1", Target: "", Stale: true},
+ },
+ }
+ var b strings.Builder
+ printSystemStatus(&b, resp, "")
+ out := b.String()
+ for _, want := range []string{"canary v2", "stable (no pointer)", "ie ", "current", "us ", "rolled-back", "gate: carlos-edge is failed", "au ", "UNREACHABLE", "pre-upgrade"} {
+ if !strings.Contains(out, want) {
+ t.Errorf("status output lacks %q:\n%s", want, out)
+ }
+ }
+ var only strings.Builder
+ printSystemStatus(&only, resp, "stable")
+ if strings.Contains(only.String(), "ie ") {
+ t.Error("--channel stable still listed a canary box")
+ }
+}
+```
+
+- [ ] **Step 2: Run to verify failure**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test . -run TestPrintSystemStatus -count=1 2>&1); rc=$?; echo "$out" | tail -5; echo rc=$rc`
+Expected: undefined types.
+
+- [ ] **Step 3: Implement**
+
+Replace the placeholders in `system.go`:
+```go
+type systemPointerRow struct {
+ Version string `json:"version"`
+ PromotedAt time.Time `json:"promoted_at"`
+ Spread string `json:"spread"`
+ Source string `json:"source"`
+ History int `json:"history"`
+}
+
+type systemStateRow struct {
+ Channel string `json:"channel"`
+ Version string `json:"version"`
+ PromotedAt time.Time `json:"promoted_at"`
+ Outcome string `json:"outcome"`
+ Reason string `json:"reason"`
+ At time.Time `json:"at"`
+}
+
+type systemBoxRow struct {
+ Label string `json:"label"`
+ Region string `json:"region"`
+ Platform string `json:"platform"`
+ Channel string `json:"channel"`
+ Release string `json:"release"`
+ Target string `json:"target"`
+ System *systemStateRow `json:"system"`
+ Stale bool `json:"stale"`
+}
+
+type systemStatusResponse struct {
+ Pointers map[string]*systemPointerRow `json:"pointers"`
+ Boxes []systemBoxRow `json:"boxes"`
+}
+
+func runSystemUpdate(args []string) {
+ fs := flag.NewFlagSet("system update", flag.ExitOnError)
+ consoleFlag := fs.String("console", "", "console origin (default: the only logged-in one)")
+ channel := fs.String("channel", "stable", "canary or stable")
+ version := fs.String("version", "", "release tag to promote (default: the latest public release); with --file, the version the binary reports")
+ file := fs.String("file", "", "a locally built binary instead of a public release (dev and pre-release)")
+ platform := fs.String("platform", "linux-arm64", "with --file: the binary's <goos>-<goarch>")
+ spread := fs.String("spread", "5m", "boxes jitter their restarts over this; 0 for a hotfix")
+ yes := fs.Bool("yes", false, "do not ask")
+ fs.Usage = flagUsage(fs, "")
+ fs.Parse(args)
+ if *file != "" && *version == "" {
+ log.Fatal("--file needs --version: the console cannot run a foreign-arch binary to ask it")
+ }
+ sess := requireDeployAPISession(*consoleFlag)
+ ctx := context.Background()
+
+ // What this reaches, before it is done.
+ st, body, err := sess.getJSON(ctx, "/api/cli/system/status")
+ if err != nil || st != http.StatusOK {
+ log.Fatalf("carlos system status: %s", apiErrorFromBody(st, body, err))
+ }
+ var cur systemStatusResponse
+ _ = json.Unmarshal(body, &cur)
+ var reach []string
+ for _, b := range cur.Boxes {
+ if b.Channel == *channel {
+ reach = append(reach, fmt.Sprintf("%s (%s, running %s)", b.Label, b.Platform, orDash(b.Release)))
+ }
+ }
+ what := "the latest public release"
+ if *version != "" {
+ what = *version
+ }
+ fmt.Fprintf(stdout, "promote %s to %s/%s — %d box(es) will roll over %s:\n", what, systemComponentRouter, *channel, len(reach), *spread)
+ for _, r := range reach {
+ fmt.Fprintf(stdout, " %s\n", r)
+ }
+ if len(reach) == 0 {
+ mutedf(stdout, " (no box reports channel %s yet; boxes on an older build count as stable)\n", *channel)
+ }
+ if !*yes {
+ if !interactiveTerminal() {
+ log.Fatal("no terminal to confirm on; pass --yes")
+ }
+ if !promptYes(stdout, "go ahead? [y/N] ") {
+ return
+ }
+ }
+
+ in := map[string]any{"channel": *channel, "spread": *spread}
+ if *version != "" {
+ in["version"] = *version
+ }
+ if *file != "" {
+ handle, err := sess.uploadSystemBinary(ctx, *file, *version, *platform)
+ if err != nil {
+ log.Fatalf("upload: %v", err)
+ }
+ in["upload"] = handle
+ }
+ st, body, err = sess.postJSON(ctx, "/api/cli/system/update", in)
+ if err != nil || st != http.StatusOK {
+ log.Fatalf("carlos system update: %s", apiErrorFromBody(st, body, err))
+ }
+ var started struct{ Job string `json:"job"` }
+ _ = json.Unmarshal(body, &started)
+ if err := sess.watchSystemJob(ctx, started.Job, stdout); err != nil {
+ log.Fatal(err)
+ }
+ fmt.Fprintf(stdout, "%s boxes on %s roll themselves; watch with: carlos system status --channel %s\n", colorFor(stdout).ok("promoted"), *channel, *channel)
+}
+
+// watchSystemJob polls the job and prints each step once, as deploy's
+// watchVersion does for a route.
+func (s *apiSession) watchSystemJob(ctx context.Context, id string, w io.Writer) error {
+ printed := 0
+ deadline := time.Now().Add(15 * time.Minute)
+ for {
+ st, body, err := s.getJSON(ctx, "/api/cli/system/jobs/"+id)
+ if err != nil || st != http.StatusOK {
+ return fmt.Errorf("job %s: %s", id, apiErrorFromBody(st, body, err))
+ }
+ var j struct {
+ Steps []string `json:"steps"`
+ Done bool `json:"done"`
+ Error string `json:"error"`
+ }
+ _ = json.Unmarshal(body, &j)
+ for ; printed < len(j.Steps); printed++ {
+ fmt.Fprintf(w, " %s\n", j.Steps[printed])
+ }
+ if j.Done {
+ if j.Error != "" {
+ return fmt.Errorf("carlos system update: %s", j.Error)
+ }
+ return nil
+ }
+ if time.Now().After(deadline) {
+ return errors.New("carlos system update: the console's job did not finish in 15 minutes")
+ }
+ time.Sleep(time.Second)
+ }
+}
+
+func (s *apiSession) uploadSystemBinary(ctx context.Context, path, version, platform string) (string, error) {
+ st, body, err := s.c.doMultipart(ctx, "/api/cli/system/upload", func(mw *multipart.Writer) error {
+ meta, err := mw.CreateFormField("meta")
+ if err != nil {
+ return err
+ }
+ if err := json.NewEncoder(meta).Encode(map[string]string{"version": version, "platform": platform}); err != nil {
+ return err
+ }
+ return writeFilePart(mw, "file", path)
+ })
+ if err != nil || st != http.StatusOK {
+ return "", fmt.Errorf("%s", apiErrorFromBody(st, body, err))
+ }
+ var out struct{ Upload string `json:"upload"` }
+ _ = json.Unmarshal(body, &out)
+ return out.Upload, nil
+}
+
+func runSystemStatus(args []string) {
+ fs := flag.NewFlagSet("system status", flag.ExitOnError)
+ consoleFlag := fs.String("console", "", "console origin (default: the only logged-in one)")
+ channel := fs.String("channel", "", "only boxes on this channel")
+ fs.Usage = flagUsage(fs, "")
+ fs.Parse(args)
+ sess := requireDeployAPISession(*consoleFlag)
+ st, body, err := sess.getJSON(context.Background(), "/api/cli/system/status")
+ if err != nil || st != http.StatusOK {
+ log.Fatalf("carlos system status: %s", apiErrorFromBody(st, body, err))
+ }
+ var resp systemStatusResponse
+ if err := json.Unmarshal(body, &resp); err != nil {
+ log.Fatalf("carlos system status: %v", err)
+ }
+ printSystemStatus(stdout, resp, *channel)
+}
+
+func printSystemStatus(w io.Writer, resp systemStatusResponse, channel string) {
+ for _, ch := range []string{"canary", "stable"} {
+ if channel != "" && ch != channel {
+ continue
+ }
+ p := resp.Pointers[ch]
+ if p == nil {
+ fmt.Fprintf(w, "%-7s (no pointer)\n", ch)
+ continue
+ }
+ fmt.Fprintf(w, "%-7s %s promoted %s spread %s %s\n", ch, p.Version, p.PromotedAt.Format(time.RFC3339), p.Spread, p.Source)
+ }
+ fmt.Fprintln(w)
+ c := colorFor(w)
+ for _, b := range resp.Boxes {
+ if channel != "" && b.Channel != channel {
+ continue
+ }
+ state := ""
+ switch {
+ case b.Stale:
+ state = c.bad("UNREACHABLE")
+ case b.Platform == "":
+ state = c.warn("pre-upgrade (no platform reported; counts as stable)")
+ case b.Target == "":
+ state = "no target"
+ case b.Release == b.Target:
+ state = c.ok("current")
+ default:
+ state = c.warn("behind → " + b.Target)
+ }
+ fmt.Fprintf(w, "%-20s %-12s %-7s %-10s %s", b.Label, orDash(b.Platform), b.Channel, orDash(b.Release), state)
+ if b.System != nil && b.System.Outcome != "applied" && b.System.Outcome != "noted" {
+ fmt.Fprintf(w, " %s %s: %s", c.bad(b.System.Outcome), b.System.Version, b.System.Reason)
+ }
+ fmt.Fprintln(w)
+ }
+}
+
+func runSystemRollback(args []string) {
+ fs := flag.NewFlagSet("system rollback", flag.ExitOnError)
+ consoleFlag := fs.String("console", "", "console origin (default: the only logged-in one)")
+ channel := fs.String("channel", "", "canary or stable (required)")
+ yes := fs.Bool("yes", false, "do not ask")
+ fs.Usage = flagUsage(fs, "")
+ fs.Parse(args)
+ if *channel == "" {
+ log.Fatal("--channel is required")
+ }
+ sess := requireDeployAPISession(*consoleFlag)
+ if !*yes {
+ if !interactiveTerminal() {
+ log.Fatal("no terminal to confirm on; pass --yes")
+ }
+ if !promptYes(stdout, fmt.Sprintf("point %s back at the release before this one? every box on it rolls back [y/N] ", *channel)) {
+ return
+ }
+ }
+ st, body, err := sess.postJSON(context.Background(), "/api/cli/system/rollback", map[string]string{"channel": *channel})
+ if err != nil || st != http.StatusOK {
+ log.Fatalf("carlos system rollback: %s", apiErrorFromBody(st, body, err))
+ }
+ var out struct{ Version string `json:"version"` }
+ _ = json.Unmarshal(body, &out)
+ fmt.Fprintf(stdout, "%s %s → %s\n", colorFor(stdout).ok("rolled back"), *channel, out.Version)
+}
+```
+Imports: `context`, `encoding/json`, `errors`, `flag`, `io`, `mime/multipart`, `net/http`, `time`. `orDash` exists in `cli.go:3327+`; `apiErrorFromBody` exists in `apimode.go` (confirm its signature with `grep -n 'func apiErrorFromBody' *.go`; adapt the three-argument call if it takes fewer). `sess.c` is the `*consoleClient` field on `apiSession` (confirm with `grep -n 'type apiSession struct' -A 6 apimode.go`).
+
+- [ ] **Step 4: Run the tests and try the binary against a fixture console**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test . -run 'TestPrintSystemStatus|TestSystemVerb' -count=1 2>&1); rc=$?; echo "$out" | tail -5; echo rc=$rc; go vet . `
+Expected: PASS, vet clean.
+
+- [ ] **Step 5: Commit**
+
+```bash
+gofmt -l . && git add system.go system_test.go && git commit -q -F - <<'EOF'
+cli: carlos system update, status, rollback through the console
+
+Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
+Claude-Session: https://claude.ai/code/session_015QCUCpfxbCyEAVQ2m51dH5
+EOF
+```
+
+---
+
+### Task 13: The hourly belt — `carlos adopt` arms the self-update units
+
+**Files:**
+- Modify: `internal/host/agent.go` `RunAdopt` (`:2811`; insert before `openServiceRegistry` at `:2825`)
+- Create: `internal/host/systemunits.go`, `internal/host/systemunits_test.go`
+
+**Interfaces:**
+- Consumes: `unitsFS`, `systemctlPlan`, `copyFileDurable`, a `sysOps`-shaped `systemctl` func.
+- Produces:
+ ```go
+ // ensureSystemUnits installs and arms carlos-system-update.path/.service
+ // and carlos-system-recover.service when any is missing from etcDir or
+ // the path unit is inactive / the recover unit not enabled. Non-fatal.
+ func ensureSystemUnits(etcDir string, systemctl func(args ...string) (string, error), logf func(string, ...any))
+ ```
+
+- [ ] **Step 1: Write the failing test**
+
+```go
+package host
+
+import (
+ "os"
+ "path/filepath"
+ "strings"
+ "testing"
+)
+
+func TestEnsureSystemUnitsInstallsAndArmsWhatIsMissing(t *testing.T) {
+ etc := t.TempDir()
+ os.MkdirAll(filepath.Join(etc, "systemd", "system"), 0o755)
+ var calls []string
+ systemctl := func(args ...string) (string, error) {
+ calls = append(calls, strings.Join(args, " "))
+ switch args[0] {
+ case "is-active":
+ return "inactive\n", nil
+ case "is-enabled":
+ return "disabled\n", nil
+ }
+ return "", nil
+ }
+ ensureSystemUnits(etc, systemctl, t.Logf)
+ for _, u := range []string{"carlos-system-update.path", "carlos-system-update.service", "carlos-system-recover.service"} {
+ if _, err := os.Stat(filepath.Join(etc, "systemd", "system", u)); err != nil {
+ t.Errorf("%s not installed", u)
+ }
+ }
+ joined := strings.Join(calls, "\n")
+ for _, want := range []string{"daemon-reload", "enable carlos-system-update.path carlos-system-recover.service", "start carlos-system-update.path"} {
+ if !strings.Contains(joined, want) {
+ t.Errorf("calls lack %q:\n%s", want, joined)
+ }
+ }
+ if strings.Contains(joined, "start carlos-system-recover.service") || strings.Contains(joined, "start carlos-system-update.service") {
+ t.Fatal("a service was started")
+ }
+ // All present and armed: no writes, no systemctl beyond the two checks.
+ calls = nil
+ systemctl2 := func(args ...string) (string, error) {
+ calls = append(calls, strings.Join(args, " "))
+ switch args[0] {
+ case "is-active":
+ return "active\n", nil
+ case "is-enabled":
+ return "enabled\n", nil
+ }
+ return "", nil
+ }
+ ensureSystemUnits(etc, systemctl2, t.Logf)
+ if len(calls) != 2 {
+ t.Fatalf("an armed box got %v", calls)
+ }
+}
+```
+
+- [ ] **Step 2: Run to verify failure**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/host/ -run TestEnsureSystemUnits -count=1 2>&1); rc=$?; echo "$out" | tail -5; echo rc=$rc`
+
+- [ ] **Step 3: Implement**
+
+```go
+package host
+
+import (
+ "os"
+ "path/filepath"
+ "strings"
+)
+
+var systemUnitFiles = []string{"carlos-system-update.path", "carlos-system-update.service", "carlos-system-recover.service"}
+
+// ensureSystemUnits is the second belt of spec §7: a box the roll script
+// forgot to arm is armed within the hour by the root adopt pass. It
+// installs the embedded units when missing and arms them through
+// systemctlPlan (never starting a service). Non-fatal: it runs before
+// RunAdopt's registry and pubkey gates, and logs instead of exiting.
+func ensureSystemUnits(etcDir string, systemctl func(args ...string) (string, error), logf func(string, ...any)) {
+ unitDir := filepath.Join(etcDir, "systemd", "system")
+ missing := false
+ for _, u := range systemUnitFiles {
+ if _, err := os.Stat(filepath.Join(unitDir, u)); err != nil {
+ missing = true
+ }
+ }
+ active, _ := systemctl("is-active", systemUpdatePathUnit)
+ enabled, _ := systemctl("is-enabled", "carlos-system-recover.service")
+ if !missing && strings.TrimSpace(active) == "active" && strings.TrimSpace(enabled) == "enabled" {
+ return
+ }
+ for _, u := range systemUnitFiles {
+ b, err := unitsFS.ReadFile("units/" + u)
+ if err != nil {
+ logf("adopt: system units: %v", err)
+ return
+ }
+ if err := writeFileDurable(filepath.Join(unitDir, u), b, 0o644); err != nil {
+ logf("adopt: system units: %v", err)
+ return
+ }
+ }
+ for _, args := range systemctlPlan([]string{systemUpdatePathUnit, "carlos-system-recover.service"}) {
+ if _, err := systemctl(args...); err != nil {
+ logf("adopt: system units: systemctl %s: %v", strings.Join(args, " "), err)
+ return
+ }
+ }
+ logf("adopt: self-update units installed and armed")
+}
+```
+In `RunAdopt`, immediately after `fs.Parse(args)` and before `openServiceRegistry`:
+```go
+ // Spec §7 belt 2: arm the self-update relay if the last hand roll did
+ // not. Before the fatal gates below, and never fatal itself.
+ if !*restartOnly {
+ ensureSystemUnits("/etc", productionSysOps().systemctl, log.Printf)
+ }
+```
+
+- [ ] **Step 4: Run**
+
+Run: `export TMPDIR=/tmp/c; out=$(go test ./internal/host/ -count=1 2>&1); rc=$?; echo "$out" | tail -10; echo rc=$rc`
+
+- [ ] **Step 5: Commit**
+
+```bash
+gofmt -l internal/host/ && git add internal/host/systemunits.go internal/host/systemunits_test.go internal/host/agent.go && git commit -q -F - <<'EOF'
+host: the hourly adopt pass arms the self-update units if the roll script did not
+
+Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
+Claude-Session: https://claude.ai/code/session_015QCUCpfxbCyEAVQ2m51dH5
+EOF
+```
+
+---
+
+### Task 14: Docs, the env contract, and the last two script lines
+
+**Files:**
+- Modify: `docs/site/cli.md` (after `### carlos status`, ~line 930), `AGENTS.md` (the section that names `carlos-stage.path` / `update-platform-binary.sh`, ~lines 246-277)
+- Out of repo, hand off (do not edit from this checkout): `platform-infrastructure/scripts/update-platform-binary.sh` and the Tito copy in `amadan.net/tito/carlos-infrastructure/scripts/update-platform-binary.sh`
+
+- [ ] **Step 1: `docs/site/cli.md`** — add under `## Your deployment`, after `### carlos status`:
+
+````markdown
+### carlos system
+
+The platform's own release channel. `update` promotes a public release (or,
+with `--file`, a binary you built) to `canary` or `stable`; every box on that
+channel fetches it through its normal store, verifies it against the signed
+pointer, installs it, restarts its edge, and rolls back to the previous
+binary if the new one does not come up and stay up. `status` shows each box's
+platform, channel, running build, the channel's target, and the last apply
+outcome with its reason. `rollback` points a channel back at the release
+before this one, and every box on it rolls back the same way it rolled
+forward. All three talk to your console; you need to be an owner of the ops
+account.
+
+```sh
+carlos system update --channel canary
+carlos system status --channel canary
+carlos system update --channel stable --spread 10m
+carlos system rollback --channel stable
+```
+
+A box is put on canary with `CARLOS_SYSTEM_CHANNEL=canary` in its host.env.
+Boxes that have not reported a platform yet count as stable. `carlos system
+apply` is the root oneshot behind `carlos-system-update.service`; it is not
+for operators. The console is an app — update it with `carlos deploy --app
+console`.
+````
+
+- [ ] **Step 2: `AGENTS.md`** — in the section describing the root relays, add after the `carlos-restart.path` paragraph:
+
+```markdown
+- `carlos-system-update.path` relays the agent's platform self-update request
+ to `carlos system apply` (root; spec 2026-09-06-carlos-system-update-design.md
+ §6). `carlos-system-recover.service` runs once per boot, before the edge,
+ and on the apply's failure. Both must be armed; the last run of
+ `update-platform-binary.sh` does it, and the hourly `carlos adopt` pass
+ installs and arms them if it did not. A box follows `CARLOS_SYSTEM_CHANNEL`
+ (canary|stable, default stable) in host.env.
+```
+
+- [ ] **Step 3: The pinfra hand-off** — write `docs/superpowers/plans/2026-09-06-carlos-system-update-pinfra.md` with exactly this content, for whoever edits the two scripts (they are in other repos; this checkout does not touch them):
+
+```markdown
+# update-platform-binary.sh: the last two lines it needs
+
+After the `carlos-restart.path` block, guarded the same way:
+
+ if [ -f /etc/systemd/system/carlos-system-update.path ]; then
+ systemctl enable --now carlos-system-update.path
+ systemctl enable carlos-system-recover.service
+ fi
+
+And to the post-conditions at the end:
+
+ systemctl is-active carlos-system-update.path
+ systemctl is-enabled carlos-system-recover.service
+
+Same change in amadan.net/tito/carlos-infrastructure/scripts/update-platform-binary.sh.
+After every deployment has been hand-rolled once onto a binary carrying the
+system-update spec, the script is retired (spec §10).
+```
+
+- [ ] **Step 4: Full gate**
+
+Run: `export TMPDIR=/tmp/c; out=$(make test 2>&1); rc=$?; echo "$out" | tail -30; echo rc=$rc`
+Expected: rc=0 (vet, gofmt, every package).
+
+- [ ] **Step 5: Commit, push, describe, land**
+
+```bash
+git add docs/site/cli.md AGENTS.md docs/superpowers/plans/2026-09-06-carlos-system-update-pinfra.md && git commit -q -F - <<'EOF'
+docs: carlos system, the env contract, and the roll script's last two lines
+
+Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
+Claude-Session: https://claude.ai/code/session_015QCUCpfxbCyEAVQ2m51dH5
+EOF
+git push -u origin carlos-system-update
+amadan branch describe carlos/platform carlos-system-update -body - <<'EOF'
+carlos system update: the platform rolls itself. Spec: docs/superpowers/specs/2026-09-06-carlos-system-update-design.md. Plan: docs/superpowers/plans/2026-09-06-carlos-system-update.md.
+EOF
+# then: amadan ci status carlos/platform -branch carlos-system-update (until 0)
+# then: amadan branch merge carlos/platform carlos-system-update -expect <full sha>
+```
+
+---
+
+## Self-review
+
+**Spec coverage.** §3 pointer, keys, immutability, spread, history-of-releases, rollback-is-new-promote, door grants, streaming read → Tasks 1, 2, 3. §4 leg (observe/tick, re-touch, watermark check, suppression, jitter, streamed fetch, request shape, deferred retry), units, exec shim, `enabledUnits` → Tasks 6, 7. §5 authorization, job, upload, update/status/rollback, `router` alias, no console verb → Tasks 9, 11, 12. §6 root state files, durability, lock, recovery-first (forward from gated), request walk, channel from env, replay guard per channel, no-op, staging copy 0600→0755, `.prev` decision, both unit sets from their binaries, quiesce incl. services, removed units, sysusers→tmpfiles→daemon-reload, reset-failed, gate+soak, arm after gate via `systemctlPlan`, record, rollback, rollback-failed → Tasks 4, 8a–8c. §7 first-roll arming (script + hourly belt), env contract → Tasks 13, 14. §9/§10 nothing to build. §11 unit-level proof: every bullet maps to a test in Tasks 1–13; the live checks are the operator's after landing.
+
+**Gaps found and fixed in review:** the spec's `release.Artifact` name collides with an existing type — Task 1 uses `SystemArtifact`; the spec's "interface gains `Open`" would break 48 fakes — Task 2 uses an `Opener` side-interface; the spec's 403 on the system doors is a bare 404 here, matching the console's operator doors; the spec's "ops-account owner" is `Role(OpsAccountID) == RoleOwner`, not `IsOperator` — Task 11 says which.
+
+**Placeholder scan.** Tasks 9 and 8a/8b carry explicit stubs that the next task replaces (`errNotYetImplemented`, the placeholder verbs, the stub `rollback`); each is named and replaced in a listed task. No "TBD", no "similar to".
+
+**Type consistency.** `systemRequest{Version, PromotedAt, NoOp, SeenAt}` is written by Task 7 and read by 8a; `systemResult.matches/terminal` are used identically in 7 and 8a; `bootstrapPlan{Enabled, Inert}` is produced by Task 6 and consumed by 8b/8c/13; `sysOps` fields are the same in 8a's fixture and 8c's `productionSysOps`; `release.SystemPromoteOptions{Actor, Spread}` in 1, 7's fixture and 11; `systemStatusResponse` in 12 matches the JSON Task 11 writes (`pointers`, `boxes` with `label, region, platform, channel, release, target, system, stale`).