carlos / platform Public

Clone
git clone https://amadan.net/carlos/platform

Plain git — no account needed to clone.

Download

Download this file

Console release defences, Plan 1 of 4: D1 (roll back a named unit that will not come up)

> For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: A box that adopts a named-unit build (today: the console) checks that the new process actually comes up. If it doesn't, the box rolls back, refuses that promotion until carlos promote --retry, and alerts.

Architecture: adoptHost (internal/release/adopt.go) already rolls back when its restart callback errors. This plan makes the named-unit restart return an honest error by settling the unit after try-restart, through a small unitSystem seam over systemctl. It adds a durable refusal record in the bake directory, scoped to the routes that share one process. It also adds a --retry promote, and an alert plus a status --live line fed from the refusal records.

Tech Stack: Go (stdlib, os/exec systemctl, net/http over TCP or Unix sockets), the existing release/registry/host/status packages.

Spec: docs/superpowers/specs/2026-10-06-console-release-defences-design.md (rev 5): §2 (D1) and §7's D1 and retry tests. Plans 2 (D2), 3 (D3) and 4 (D4, process and scripts) follow; this plan stands alone.

Global Constraints

  • Settle window CARLOS_UNIT_SETTLE: default 30 s, floor 5 s, poll every 500 ms (spec §2.3).
  • Readiness follows the effective upstream: Socket if set, else Addr (spec §2.4).
  • No-probe readiness: the unit stays active within one invocation for 5 s (spec §2.4).
  • A unit not running before the restart is never checked or started (spec §2.3).
  • A binding mismatch (ExecStart not live/<host>) skips the settle check only; the try-restart error still propagates (spec §2.2, codex 12).
  • D1 rollback runs reset-failed then restart; every other makeStartRoute caller keeps start (spec §2.3, codex 1).
  • A rollback group is the bound route plus unstamped same-app, same-account routes with the same non-empty upstream; two empty upstreams never match (spec §2.3).
  • A refusal blocks while the pointer's Version and PromotedAt equal the record's (spec §2.5). See Task 1's note on keying.
  • If systemctl show fails (for example, the sandboxed agent may not query the unit), behave exactly as today: plain try-restart, return its error.
  • Gate locally with GOFLAGS=-buildvcs=false; in this sandbox set GOCACHE=$TMPDIR/gocache per command (AGENTS.md path is read-only here).
  • Commits go on branch design-console-release-defences's implementation branch (named at execution), as the agent identity, ending with the Co-Authored-By line.

Review Focus

  1. A console that takes longer than the window to start (cold registry, slow disk). Expect: it is rolled back only after the full window, the window is configurable, and the log says "not ready after 30s", not "crashed". Test in Task 4 (TestSettleDeadlineSaysNotReady).
  2. The sandboxed agent cannot run systemctl show. Expect: today's behaviour exactly, with no false rollback. Test in Task 6 (TestNamedRestartShowFailsFallsBack).
  3. An operator stopped the console on purpose during an adoption. Expect: no check, no start, and it stays stopped. Test in Task 4 (TestSettleSkipsAUnitThatWasNotRunning).
  4. A corrupt or half-written refusal file. Expect: treated as absent and logged, never a permanent block or a panic. Test in Task 1 (TestReadRefusalCorruptIsAbsent).
  5. Two instances of one app on different upstreams. Expect: a rollback of one never touches the other. Test in Task 3 (TestRollbackGroupLeavesOtherUpstreamAlone).

---

Task 1: Refusal records

Files:

  • Create: internal/release/refusal.go
  • Test: internal/release/refusal_test.go

Interfaces:

  • Produces: - type Refusal struct { Schema int; Host, App, Version, SHA256, Platform string; PointerPromotedAt, RefusedAt time.Time; Reason string } (JSON keys below) - func RefusalDir(bakeDir string) string - func WriteRefusal(bakeDir string, r Refusal) error - func ReadRefusal(bakeDir, host string) (*Refusal, error) — (nil, nil) when absent or unparseable - func ListRefusals(bakeDir string) ([]Refusal, error) - func ClearRefusal(bakeDir, host string) error — nil when absent - func (r *Refusal) Blocks(p *ChannelPointer) bool

Keying note (for the reviewer): spec §2.5 keys on the artifact sha256. This plan blocks on (Version, PointerPromotedAt) and records SHA256 for display. That is equivalent: a version's manifest is immutable while a pointer names it, and reusing a name for new bytes needs a new promotion, which changes PromotedAt and clears the block. It also avoids a manifest GET on every fast-loop tick for a refused route.

  • [ ] Step 1: Write the failing tests
package release

import (
	"os"
	"path/filepath"
	"testing"
	"time"
)

func TestRefusalRoundTripAndBlocks(t *testing.T) {
	dir := t.TempDir()
	at := time.Date(2026, 10, 6, 9, 30, 30, 0, time.UTC)
	r := Refusal{Host: "console.example.com", App: "console", Version: "v2",
		SHA256: "abc", Platform: "linux-arm64", PointerPromotedAt: at,
		RefusedAt: at.Add(10 * time.Second), Reason: "exited: status=2"}
	if err := WriteRefusal(dir, r); err != nil {
		t.Fatal(err)
	}
	got, err := ReadRefusal(dir, "console.example.com")
	if err != nil || got == nil {
		t.Fatalf("ReadRefusal = %+v, %v", got, err)
	}
	if got.Schema != 1 || got.Version != "v2" || !got.PointerPromotedAt.Equal(at) {
		t.Fatalf("round trip = %+v", got)
	}
	if !got.Blocks(&ChannelPointer{Version: "v2", PromotedAt: at}) {
		t.Fatal("the same promotion is not blocked")
	}
	if got.Blocks(&ChannelPointer{Version: "v2", PromotedAt: at.Add(time.Second)}) {
		t.Fatal("a re-signed pointer (retry) is still blocked")
	}
	if got.Blocks(&ChannelPointer{Version: "v3", PromotedAt: at}) {
		t.Fatal("a different version is blocked")
	}
	var nilR *Refusal
	if nilR.Blocks(&ChannelPointer{Version: "v2", PromotedAt: at}) {
		t.Fatal("a nil refusal blocks")
	}
}

func TestReadRefusalAbsentIsNil(t *testing.T) {
	got, err := ReadRefusal(t.TempDir(), "nobody.example.com")
	if got != nil || err != nil {
		t.Fatalf("absent = %+v, %v; want nil, nil", got, err)
	}
}

func TestReadRefusalCorruptIsAbsent(t *testing.T) {
	dir := t.TempDir()
	p := filepath.Join(RefusalDir(dir), "console.example.com.json")
	if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
		t.Fatal(err)
	}
	if err := os.WriteFile(p, []byte(`{"schema":1,"host":`), 0o644); err != nil {
		t.Fatal(err)
	}
	got, err := ReadRefusal(dir, "console.example.com")
	if got != nil || err != nil {
		t.Fatalf("corrupt = %+v, %v; want nil, nil (a torn file must never block)", got, err)
	}
}

func TestRefusalRejectsAPathInTheHost(t *testing.T) {
	if err := WriteRefusal(t.TempDir(), Refusal{Host: "../etc/x"}); err == nil {
		t.Fatal("a host with a path separator was written")
	}
}

func TestListAndClearRefusals(t *testing.T) {
	dir := t.TempDir()
	for _, h := range []string{"a.example.com", "b.example.com"} {
		if err := WriteRefusal(dir, Refusal{Host: h, Version: "v2"}); err != nil {
			t.Fatal(err)
		}
	}
	if err := ClearRefusal(dir, "a.example.com"); err != nil {
		t.Fatal(err)
	}
	if err := ClearRefusal(dir, "never.example.com"); err != nil {
		t.Fatalf("clearing an absent record: %v", err)
	}
	all, err := ListRefusals(dir)
	if err != nil || len(all) != 1 || all[0].Host != "b.example.com" {
		t.Fatalf("ListRefusals = %+v, %v", all, err)
	}
}
  • [ ] Step 2: Run them to verify they fail

Run: GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/release -run 'Refusal' -count=1 Expected: FAIL, undefined: Refusal.

  • [ ] Step 3: Implement
// Refusal records — D1 of the console release defences (spec
// 2026-10-06-console-release-defences-design.md §2.5).
//
// When a named unit does not come up on a new build, the root pass rolls it
// back and writes one of these per host in the rolled-back process group.
// Every later pass, the agent's fast loop and root alike, reads it before
// staging and skips the promotion it names. Without it the very next pass
// would adopt the same build and bounce the service again.
//
// The record blocks a PROMOTION, not a version name: it matches while the
// channel pointer still has the version and PromotedAt it had when the build
// was refused. `carlos promote --retry` re-signs the pointer with a fresh
// PromotedAt, and so does any move to another version; either clears it.
//
// Root writes these under the bake directory, which the sandboxed agent can
// read. A file that does not parse reads as absent: a torn write must never
// become a permanent block.
package release

import (
	"encoding/json"
	"errors"
	"fmt"
	"log"
	"os"
	"path/filepath"
	"sort"
	"strings"
	"time"
)

const refusalSchema = 1

type Refusal struct {
	Schema            int       `json:"schema"`
	Host              string    `json:"host"`
	App               string    `json:"app"`
	Version           string    `json:"version"`
	SHA256            string    `json:"sha256,omitempty"`
	Platform          string    `json:"platform,omitempty"`
	PointerPromotedAt time.Time `json:"pointer_promoted_at"`
	RefusedAt         time.Time `json:"refused_at"`
	Reason            string    `json:"reason"`
}

func RefusalDir(bakeDir string) string { return filepath.Join(bakeDir, "refusals") }

func refusalPath(bakeDir, host string) (string, error) {
	if host == "" || strings.ContainsAny(host, `/\`) || host == "." || host == ".." {
		return "", fmt.Errorf("refusal: invalid host %q", host)
	}
	return filepath.Join(RefusalDir(bakeDir), host+".json"), nil
}

// WriteRefusal writes atomically (temp file + rename) and world-readable,
// because the reader is the sandboxed agent.
func WriteRefusal(bakeDir string, r Refusal) error {
	p, err := refusalPath(bakeDir, r.Host)
	if err != nil {
		return err
	}
	r.Schema = refusalSchema
	b, err := json.MarshalIndent(r, "", "  ")
	if err != nil {
		return err
	}
	if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
		return err
	}
	tmp, err := os.CreateTemp(filepath.Dir(p), ".refusal-*")
	if err != nil {
		return err
	}
	defer os.Remove(tmp.Name())
	if _, err := tmp.Write(append(b, '\n')); err != nil {
		tmp.Close()
		return err
	}
	if err := tmp.Chmod(0o644); err != nil {
		tmp.Close()
		return err
	}
	if err := tmp.Close(); err != nil {
		return err
	}
	return os.Rename(tmp.Name(), p)
}

func ReadRefusal(bakeDir, host string) (*Refusal, error) {
	p, err := refusalPath(bakeDir, host)
	if err != nil {
		return nil, err
	}
	b, err := os.ReadFile(p)
	if errors.Is(err, os.ErrNotExist) {
		return nil, nil
	}
	if err != nil {
		return nil, err
	}
	var r Refusal
	if err := json.Unmarshal(b, &r); err != nil || r.Schema != refusalSchema {
		log.Printf("refusal: %s does not parse (%v), treating it as absent", p, err)
		return nil, nil
	}
	return &r, nil
}

func ListRefusals(bakeDir string) ([]Refusal, error) {
	ents, err := os.ReadDir(RefusalDir(bakeDir))
	if errors.Is(err, os.ErrNotExist) {
		return nil, nil
	}
	if err != nil {
		return nil, err
	}
	var out []Refusal
	for _, e := range ents {
		name := e.Name()
		if e.IsDir() || !strings.HasSuffix(name, ".json") || strings.HasPrefix(name, ".") {
			continue
		}
		r, err := ReadRefusal(bakeDir, strings.TrimSuffix(name, ".json"))
		if err != nil || r == nil {
			continue
		}
		out = append(out, *r)
	}
	sort.Slice(out, func(i, j int) bool { return out[i].Host < out[j].Host })
	return out, nil
}

func ClearRefusal(bakeDir, host string) error {
	p, err := refusalPath(bakeDir, host)
	if err != nil {
		return err
	}
	if err := os.Remove(p); err != nil && !errors.Is(err, os.ErrNotExist) {
		return err
	}
	return nil
}

// Blocks reports whether this record still refuses the promotion p names.
func (r *Refusal) Blocks(p *ChannelPointer) bool {
	return r != nil && p != nil && r.Version == p.Version && r.PointerPromotedAt.Equal(p.PromotedAt)
}
  • [ ] Step 4: Run the tests to verify they pass

Run: GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/release -run 'Refusal' -count=1 Expected: PASS.

  • [ ] Step 5: Commit
git add internal/release/refusal.go internal/release/refusal_test.go
git commit -m "release: refusal records for builds a box rolled back (console defences D1)"

---

Task 2: adoptHost honours a refusal before it stages or relays

Files:

  • Modify: internal/release/adopt.go (in adoptHost, after the poisoned[ver] check and before ensureStaged; and after the final SetVersions success)
  • Test: internal/release/adopt_test.go (append)

Interfaces:

  • Consumes: Task 1's ReadRefusal, ClearRefusal, (*Refusal).Blocks; passConfig.bakeDir (set by BakeDir(dir)).
  • Produces: nothing new; the printed line refused: <ver> did not come up on this box at <time> (<reason>); retry with: carlos promote --retry <ver> <channel> is what Task 8 and operators grep.
  • [ ] Step 1: Write the failing tests
// TestRefusedPromotionIsNeitherStagedNorRelayed: a refusal naming the current
// pointer stops adoptHost before ensureStaged, so the agent never relays a
// root pass for it and nothing restarts.
func TestRefusedPromotionIsNeitherStagedNorRelayed(t *testing.T) {
	host := "console.example.com"
	ctx, reg, s, pub, opt := startUnitFixture(t, host)
	bake := t.TempDir()
	ptr, err := Channel(ctx, s, pub, "bab", "console", "canary/x")
	if err != nil {
		t.Fatal(err)
	}
	if err := WriteRefusal(bake, Refusal{Host: host, App: "console", Version: ptr.Version,
		PointerPromotedAt: ptr.PromotedAt, Reason: "exited: status=2"}); err != nil {
		t.Fatal(err)
	}
	restart := func(h string) error { t.Fatalf("restart consulted for refused %s", h); return nil }
	relayed := false
	var out bytes.Buffer
	withCapturedStdout(t, &out, func() {
		if err := Pass(ctx, reg, s, pub, opt, restart, BakeDir(bake),
			OnStageBlocked(func(string, string, error) { relayed = true })); err != nil {
			t.Fatalf("Pass: %v", err)
		}
	})
	if v, _ := LiveVersion(opt, host); v != "old" {
		t.Fatalf("live = %q, want old", v)
	}
	if relayed {
		t.Fatal("a refused promotion was relayed to root")
	}
	if !strings.Contains(out.String(), "promote --retry") {
		t.Fatalf("printed %q, want the retry hint", out.String())
	}
}

// TestRePromotionClearsTheBlock: a pointer re-signed for the same version
// (what promote --retry does) is adopted, and the stale record is removed.
func TestRePromotionClearsTheBlock(t *testing.T) {
	host := "console.example.com"
	ctx, reg, s, pub, opt := startUnitFixture(t, host)
	bake := t.TempDir()
	if err := WriteRefusal(bake, Refusal{Host: host, App: "console", Version: "new",
		PointerPromotedAt: time.Unix(1, 0).UTC(), Reason: "older promotion"}); err != nil {
		t.Fatal(err)
	}
	if err := Pass(ctx, reg, s, pub, opt, func(string) error { return nil }, BakeDir(bake)); err != nil {
		t.Fatalf("Pass: %v", err)
	}
	if v, _ := LiveVersion(opt, host); v != "new" {
		t.Fatalf("live = %q, want new (the record names an older promotion)", v)
	}
	if r, _ := ReadRefusal(bake, host); r != nil {
		t.Fatalf("stale refusal survived a successful adoption: %+v", r)
	}
}

Channel (internal/release/release.go:1047) is the verified pointer read.

  • [ ] Step 2: Run them to verify they fail

Run: GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/release -run 'TestRefusedPromotion|TestRePromotionClears' -count=1 Expected: FAIL. The first test's restart fatals, and the second still finds the record.

  • [ ] Step 3: Implement the gate

In adoptHost, immediately after:

	if poisoned[ver] {
		fmt.Printf("  %s: skipping %s (failed on another host this pass)\n", rt.Host, ver)
		return
	}

insert:

	// D1's refusal (refusal.go): this box rolled this exact promotion back
	// because the unit did not come up. Checked before ensureStaged so a
	// refused build is neither downloaded nor relayed to a root pass.
	if cfg.bakeDir != "" {
		if r, err := ReadRefusal(cfg.bakeDir, rt.Host); err != nil {
			fmt.Printf("  %s: reading refusal record: %v\n", rt.Host, err)
		} else if r.Blocks(ptr) {
			fmt.Printf("  %s: refused: %s did not come up on this box at %s (%s); retry with: carlos promote --retry %s %s\n",
				rt.Host, ver, r.RefusedAt.Format(time.RFC3339), r.Reason, ver, channel)
			return
		}
	}

and immediately after the final successful reg.SetVersions(rt.Host, ver, cur) (the one followed by the updated %s -> %s print), insert:

	// A record left from an older promotion is stale once a build is live.
	// Best-effort: the agent cannot remove root's file, the root pass can.
	if cfg.bakeDir != "" {
		_ = ClearRefusal(cfg.bakeDir, rt.Host)
	}
  • [ ] Step 4: Run the release package tests

Run: GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/release -count=1 Expected: PASS (new tests and every existing adopt test).

  • [ ] Step 5: Commit
git add internal/release/adopt.go internal/release/adopt_test.go
git commit -m "release: adoptHost skips a refused promotion before staging or relaying"

---

Task 3: The process-group rollback when a unit does not come up

Files:

  • Create: internal/release/rollback.go
  • Modify: internal/release/adopt.go (passConfig, new WithRollbackRestart, the rerr != nil branch of adoptHost)
  • Test: internal/release/rollback_test.go

Interfaces:

  • Produces: - type UnitNotUpError struct { Unit, Reason, Journal string } with Error() string. The host package's restart returns this (Task 6) and adoptHost tests it with errors.As. - func upstream(rt registry.Route) string: rt.Socket if non-empty, else rt.Addr. - func processGroup(reg *registry.Registry, bound registry.Route) []registry.Route: excludes bound. - func WithRollbackRestart(fn func(host string) error) PassOption: replaces startUnit on the D1 path only.
  • Consumes: Task 1's WriteRefusal; LiveVersion, PointLive, RouteAccount, reg.List, reg.SetVersions.
  • [ ] Step 1: Write the failing tests
package release

import (
	"bytes"
	"testing"

	"github.com/carlosframework/platform/internal/registry"
)

func groupFixtureRoute(t *testing.T, reg *registry.Registry, host, socket, addr, unit string) {
	t.Helper()
	if err := reg.Upsert(registry.Route{Host: host, Kind: "instance", App: "console",
		Channel: "canary/x", Socket: socket, Addr: addr, Unit: unit}); err != nil {
		t.Fatalf("upsert %s: %v", host, err)
	}
}

func TestProcessGroupKeysOnTheEffectiveUpstream(t *testing.T) {
	reg := testRegistry(t)
	groupFixtureRoute(t, reg, "platform.example.com", "", "127.0.0.1:8090", "console.service")
	groupFixtureRoute(t, reg, "console.example.com", "", "127.0.0.1:8090", "")
	groupFixtureRoute(t, reg, "other.example.com", "", "127.0.0.1:9000", "")
	groupFixtureRoute(t, reg, "sock.example.com", "/run/x.sock", "127.0.0.1:8090", "")
	bound, _ := reg.Get("platform.example.com")
	got := processGroup(reg, *bound)
	if len(got) != 1 || got[0].Host != "console.example.com" {
		t.Fatalf("group = %+v, want only console.example.com (socket wins over addr; 9000 is another process)", got)
	}
}

func TestProcessGroupNeverMatchesEmptyUpstreams(t *testing.T) {
	reg := testRegistry(t)
	groupFixtureRoute(t, reg, "a.example.com", "", "", "a.service")
	groupFixtureRoute(t, reg, "b.example.com", "", "", "")
	bound, _ := reg.Get("a.example.com")
	if got := processGroup(reg, *bound); len(got) != 0 {
		t.Fatalf("group = %+v, want none: two empty upstreams are not one process", got)
	}
}

// The flagship shape: three hosts, one process, the unit runs the LAST
// host's link. Hosts before it were repointed and stamped; the bound host's
// restart reports the unit did not come up.
func TestNotUpRollsTheWholeGroupBackAndRefusesEveryHost(t *testing.T) {
	ctx, reg, s, pub, opt := startUnitFixture(t, "console.example.com")
	if err := reg.SetUnit("console.example.com", ""); err != nil {
		t.Fatal(err)
	}
	groupFixtureRoute(t, reg, "platform.example.com", "", "127.0.0.1:8090", "console.service")
	if err := reg.SetUpstreams("console.example.com", "127.0.0.1:8090", "", ""); err != nil {
		t.Fatal(err)
	}
	if err := PointLive(opt, "platform.example.com", "bab", "old", "console"); err != nil {
		t.Fatal(err)
	}
	bake := t.TempDir()
	restart := func(h string) error {
		if h == "platform.example.com" {
			return &UnitNotUpError{Unit: "console.service", Reason: "exited: status=2"}
		}
		return nil
	}
	var rolledBackHost string
	rb := func(h string) error { rolledBackHost = h; return nil }
	var out bytes.Buffer
	withCapturedStdout(t, &out, func() {
		if err := Pass(ctx, reg, s, pub, opt, restart, BakeDir(bake), WithRollbackRestart(rb)); err != nil {
			t.Fatalf("Pass: %v", err)
		}
	})
	for _, h := range []string{"console.example.com", "platform.example.com"} {
		if v, _ := LiveVersion(opt, h); v != "old" {
			t.Fatalf("%s live = %q, want old", h, v)
		}
		rt, _ := reg.Get(h)
		if rt.Version == "new" {
			t.Fatalf("%s still stamped new: %+v", h, rt)
		}
		if r, _ := ReadRefusal(bake, h); r == nil || r.Version != "new" {
			t.Fatalf("%s refusal = %+v, want one naming new", h, r)
		}
	}
	if rolledBackHost != "platform.example.com" {
		t.Fatalf("rollback restart ran for %q, want the bound host", rolledBackHost)
	}
}

func TestRollbackGroupLeavesOtherUpstreamAlone(t *testing.T) {
	ctx, reg, s, pub, opt := startUnitFixture(t, "console.example.com")
	if err := reg.SetUnit("console.example.com", "console.service"); err != nil {
		t.Fatal(err)
	}
	if err := reg.SetUpstreams("console.example.com", "127.0.0.1:8090", "", ""); err != nil {
		t.Fatal(err)
	}
	groupFixtureRoute(t, reg, "zz-other.example.com", "", "127.0.0.1:9000", "")
	if err := StageArtifact(opt, "bab", "new", "console", []byte("bad build")); err != nil {
		t.Fatal(err)
	}
	if err := PointLive(opt, "zz-other.example.com", "bab", "new", "console"); err != nil {
		t.Fatal(err)
	}
	restart := func(h string) error {
		if h == "console.example.com" {
			return &UnitNotUpError{Unit: "console.service", Reason: "not ready after 30s"}
		}
		return nil
	}
	bake := t.TempDir()
	if err := Pass(ctx, reg, s, pub, opt, restart, BakeDir(bake), WithRollbackRestart(func(string) error { return nil })); err != nil {
		t.Fatalf("Pass: %v", err)
	}
	if v, _ := LiveVersion(opt, "zz-other.example.com"); v != "new" {
		t.Fatalf("another process's route was rolled back to %q", v)
	}
	if r, _ := ReadRefusal(bake, "zz-other.example.com"); r != nil {
		t.Fatalf("another process's route was refused: %+v", r)
	}
}

Check the fixture's assumptions before running: reg.SetUnit and reg.SetUpstreams(host, addr, addrs, health) exist at internal/registry/registry.go:1231 and :1315, and startUnitFixture's route starts with no Unit.

  • [ ] Step 2: Run them to verify they fail

Run: GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/release -run 'ProcessGroup|NotUpRolls|RollbackGroup' -count=1 Expected: FAIL, undefined: UnitNotUpError.

  • [ ] Step 3: Implement rollback.go
// The D1 rollback (spec 2026-10-06-console-release-defences-design.md §2.3):
// what adoptHost does when a named unit's restart reports that the new build
// did not come up, as opposed to systemctl itself failing.
package release

import (
	"fmt"
	"strings"

	"github.com/carlosframework/platform/internal/registry"
)

// UnitNotUpError is a named unit that was restarted onto a new build and did
// not settle: it failed, restarted again, or was not ready by the deadline.
// The host package's settle check returns it (internal/host/unitsettle.go).
type UnitNotUpError struct {
	Unit    string
	Reason  string
	Journal string // the unit's last journal lines, for the rollback line and the alert
}

func (e *UnitNotUpError) Error() string {
	return fmt.Sprintf("%s did not come up: %s", e.Unit, e.Reason)
}

// upstream is a route's effective upstream: Socket wins over Addr, as at the
// edge (registry.Route.Socket's doc).
func upstream(rt registry.Route) string {
	if rt.Socket != "" {
		return rt.Socket
	}
	return rt.Addr
}

// processGroup is every OTHER route served by bound's process: unstamped,
// same account and app, same non-empty upstream. A bound route with no
// upstream is a group of one.
func processGroup(reg *registry.Registry, bound registry.Route) []registry.Route {
	up := upstream(bound)
	if up == "" {
		return nil
	}
	all, err := reg.List()
	if err != nil {
		fmt.Printf("  %s: listing routes for the rollback group: %v (rolling back this host only)\n", bound.Host, err)
		return nil
	}
	var out []registry.Route
	for _, r := range all {
		if r.Host == bound.Host || r.Unit != "" || r.App != bound.App ||
			RouteAccount(r) != RouteAccount(bound) || upstream(r) != up {
			continue
		}
		out = append(out, r)
	}
	return out
}

// rollbackGroup puts every group route whose link names the rejected version
// onto restored (what bound now runs), stamps it to match, and writes a
// refusal for bound and every group host.
func rollbackGroup(reg *registry.Registry, optDir string, cfg passConfig, bound registry.Route,
	ptr *ChannelPointer, rejected, restored, sha, platform string, nu *UnitNotUpError, now time.Time) {
	hosts := []string{bound.Host}
	for _, g := range processGroup(reg, bound) {
		hosts = append(hosts, g.Host)
		v, err := LiveVersion(optDir, g.Host)
		if err != nil || v != rejected || restored == "" {
			continue
		}
		if err := PointLive(optDir, g.Host, RouteAccount(g), restored, g.App); err != nil {
			fmt.Printf("  %s: could not roll the symlink back to %s with its process: %v\n", g.Host, restored, err)
			continue
		}
		if err := reg.SetVersions(g.Host, restored, g.PrevVersion); err != nil {
			fmt.Printf("  %s: rolled back to %s but recording the version failed: %v\n", g.Host, restored, err)
		}
	}
	if cfg.bakeDir == "" {
		return
	}
	reason := nu.Reason
	if j := strings.TrimSpace(nu.Journal); j != "" {
		lines := strings.Split(j, "\n")
		reason += " — " + lines[len(lines)-1]
	}
	for _, h := range hosts {
		r := Refusal{Host: h, App: bound.App, Version: rejected, SHA256: sha, Platform: platform,
			PointerPromotedAt: ptr.PromotedAt, RefusedAt: now.UTC(), Reason: reason}
		if err := WriteRefusal(cfg.bakeDir, r); err != nil {
			fmt.Printf("  %s: writing refusal record: %v\n", h, err)
		}
	}
}

now is the now time.Time that adoptHost already receives. Add "time" to rollback.go's imports.

  • [ ] Step 4: Wire it into adoptHost and passConfig

In passConfig (adopt.go:34) add:

	// rollbackRestart replaces startUnit on D1's path only — a named unit
	// whose restart returned *UnitNotUpError. It must REPLACE whatever is
	// running (reset-failed + restart), because a deadline failure can leave
	// the rejected build running unready and `start` is a no-op on it.
	rollbackRestart func(host string) error

Below WithStartUnit, add:

// WithRollbackRestart sets D1's rollback restart (see passConfig).
func WithRollbackRestart(fn func(host string) error) PassOption {
	return func(c *passConfig) { c.rollbackRestart = fn }
}

In the rerr != nil branch of adoptHost, right after poisoned[ver] = true, record whether this is D1's case:

			var notUp *UnitNotUpError
			isNotUp := errors.As(rerr, &notUp)

Replace the final start block

			if cfg.startUnit != nil && rolledBack && !daemonRestart {
				if serr := cfg.startUnit(rt.Host); serr != nil {

with:

			start := cfg.startUnit
			if isNotUp && cfg.rollbackRestart != nil {
				start = cfg.rollbackRestart
			}
			if isNotUp {
				sha := ""
				if m != nil {
					sha = m.Artifacts[adoptPlatform].SHA256
				}
				rollbackGroup(reg, optDir, cfg, rt, ptr, ver, cur, sha, adoptPlatform, notUp, now)
				if notUp.Journal != "" {
					fmt.Printf("  %s: %s's last lines:\n%s\n", rt.Host, notUp.Unit, indent(notUp.Journal, "      "))
				}
			}
			if start != nil && rolledBack && !daemonRestart {
				if serr := start(rt.Host); serr != nil {

Keep the rest of that block (the "could not start it" print) unchanged. Add indent to rollback.go if the package has no such helper:

func indent(s, pre string) string {
	return pre + strings.ReplaceAll(strings.TrimRight(s, "\n"), "\n", "\n"+pre)
}

adoptPlatform (adopt.go:26, runtime.GOOS + "-" + runtime.GOARCH) is the box's own artifact key.

  • [ ] Step 5: Run the release package tests

Run: GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/release -count=1 Expected: PASS, including every pre-existing restart-failure test. Those return plain errors, so isNotUp is false and their behaviour is unchanged.

  • [ ] Step 6: Commit
git add internal/release/rollback.go internal/release/rollback_test.go internal/release/adopt.go
git commit -m "release: roll back a unit's whole process group and refuse the build when it does not come up"

---

Task 4: The settle check

Files:

  • Create: internal/host/unitsettle.go
  • Test: internal/host/unitsettle_test.go

Interfaces:

  • Produces: - type unitProps struct { ActiveState, InvocationID string; MainPID, NRestarts int; ExecPath string } - type unitSystem interface { Show(unit string) (unitProps, error); TryRestart(unit string) error; Restart(unit string) error; ResetFailed(unit string) error; JournalTail(unit string, n int) string; Exe(pid int) (string, error) } - type systemdUnits struct{} implementing it with systemctl/journalctl/os.Readlink("/proc/<pid>/exe") - func parseShow(out string) unitProps - type settleOpts struct { Timeout, Poll, Hold time.Duration; WantDir string; Ready func() error; Now func() time.Time; Sleep func(time.Duration) } - func settleUnit(sys unitSystem, unit, beforeInvocation string, o settleOpts) error: nil or *release.UnitNotUpError - func unitSettleTimeout(getenv func(string) string) time.Duration
  • [ ] Step 1: Write the failing tests
package host

import (
	"errors"
	"strings"
	"testing"
	"time"

	"github.com/carlosframework/platform/internal/release"
)

type fakeUnits struct {
	shows    []unitProps // returned in order; the last repeats
	n        int
	exe      map[int]string
	journal  string
	restarts []string
}

func (f *fakeUnits) Show(string) (unitProps, error) {
	p := f.shows[min(f.n, len(f.shows)-1)]
	f.n++
	return p, nil
}
func (f *fakeUnits) TryRestart(u string) error  { f.restarts = append(f.restarts, "try-restart "+u); return nil }
func (f *fakeUnits) Restart(u string) error     { f.restarts = append(f.restarts, "restart "+u); return nil }
func (f *fakeUnits) ResetFailed(u string) error { f.restarts = append(f.restarts, "reset-failed "+u); return nil }
func (f *fakeUnits) JournalTail(string, int) string { return f.journal }
func (f *fakeUnits) Exe(pid int) (string, error) {
	if e, ok := f.exe[pid]; ok {
		return e, nil
	}
	return "", errors.New("no such pid")
}

// settleClock, not fakeClock: stage_relay_test.go already declares a
// fakeClock type in this package.
func settleClock() (func() time.Time, func(time.Duration)) {
	t := time.Unix(0, 0)
	return func() time.Time { return t }, func(d time.Duration) { t = t.Add(d) }
}

func opts(ready func() error) settleOpts {
	now, sleep := settleClock()
	return settleOpts{Timeout: 30 * time.Second, Poll: 500 * time.Millisecond, Hold: 5 * time.Second,
		WantDir: "/opt/carlos/versions/new", Ready: ready, Now: now, Sleep: sleep}
}

func TestParseShow(t *testing.T) {
	p := parseShow("ActiveState=active\nInvocationID=abc\nMainPID=42\nNRestarts=0\n" +
		"ExecStart={ path=/opt/carlos/live/platform.example.com ; argv[]=/opt/carlos/live/platform.example.com serve ; ignore_errors=no }\n")
	if p.ActiveState != "active" || p.InvocationID != "abc" || p.MainPID != 42 ||
		p.ExecPath != "/opt/carlos/live/platform.example.com" {
		t.Fatalf("parseShow = %+v", p)
	}
}

func TestSettleHealthyNewInvocation(t *testing.T) {
	f := &fakeUnits{shows: []unitProps{
		{ActiveState: "activating", InvocationID: "new"},
		{ActiveState: "active", InvocationID: "new", MainPID: 7},
	}, exe: map[int]string{7: "/opt/carlos/versions/new/console"}}
	if err := settleUnit(f, "console.service", "old", opts(func() error { return nil })); err != nil {
		t.Fatalf("settle = %v, want healthy", err)
	}
}

// systemd resets NRestarts on a manual start: a healthy unit that had
// auto-restarted before must still pass (spec §2.3, codex 4).
func TestSettleIgnoresThePreviousInvocationsCounter(t *testing.T) {
	f := &fakeUnits{shows: []unitProps{{ActiveState: "active", InvocationID: "new", MainPID: 7, NRestarts: 0}},
		exe: map[int]string{7: "/opt/carlos/versions/new/console"}}
	if err := settleUnit(f, "console.service", "old", opts(func() error { return nil })); err != nil {
		t.Fatalf("settle = %v", err)
	}
}

// Today's outage: the process exits in 55 ms and systemd cycles it.
func TestSettleCatchesARestartLoop(t *testing.T) {
	f := &fakeUnits{shows: []unitProps{
		{ActiveState: "active", InvocationID: "new1", MainPID: 7},
		{ActiveState: "activating", InvocationID: "new2", NRestarts: 1},
	}, exe: map[int]string{7: "/opt/carlos/versions/new/console"}, journal: "usage: carlos <command> [flags]"}
	err := settleUnit(f, "console.service", "old", opts(func() error { return errors.New("connection refused") }))
	var nu *release.UnitNotUpError
	if !errors.As(err, &nu) || !strings.Contains(nu.Journal, "usage: carlos") {
		t.Fatalf("settle = %v, want UnitNotUpError carrying the journal", err)
	}
}

func TestSettleCatchesFailed(t *testing.T) {
	f := &fakeUnits{shows: []unitProps{{ActiveState: "failed", InvocationID: "new"}}}
	var nu *release.UnitNotUpError
	if err := settleUnit(f, "console.service", "old", opts(nil)); !errors.As(err, &nu) {
		t.Fatalf("settle = %v, want UnitNotUpError", err)
	}
}

func TestSettleDeadlineSaysNotReady(t *testing.T) {
	f := &fakeUnits{shows: []unitProps{{ActiveState: "active", InvocationID: "new", MainPID: 7}},
		exe: map[int]string{7: "/opt/carlos/versions/new/console"}}
	err := settleUnit(f, "console.service", "old", opts(func() error { return errors.New("dial: refused") }))
	var nu *release.UnitNotUpError
	if !errors.As(err, &nu) || !strings.Contains(nu.Reason, "not ready after 30s") {
		t.Fatalf("settle = %v, want a not-ready deadline", err)
	}
}

func TestSettleRejectsTheOldBinary(t *testing.T) {
	f := &fakeUnits{shows: []unitProps{{ActiveState: "active", InvocationID: "new", MainPID: 7}},
		exe: map[int]string{7: "/opt/carlos/versions/old/console"}}
	var nu *release.UnitNotUpError
	if err := settleUnit(f, "console.service", "old", opts(func() error { return nil })); !errors.As(err, &nu) {
		t.Fatalf("settle = %v, want not-up: the process runs the old build", err)
	}
}

func TestSettleWithoutReadyHoldsFiveSeconds(t *testing.T) {
	f := &fakeUnits{shows: []unitProps{{ActiveState: "active", InvocationID: "new", MainPID: 7}},
		exe: map[int]string{7: "/opt/carlos/versions/new/console"}}
	o := opts(nil)
	start := o.Now()
	if err := settleUnit(f, "console.service", "old", o); err != nil {
		t.Fatalf("settle = %v", err)
	}
	if got := o.Now().Sub(start); got < 5*time.Second {
		t.Fatalf("settled after %v, want at least the 5s hold", got)
	}
}

func TestSettleSkipsAUnitThatWasNotRunning(t *testing.T) {
	// beforeInvocation "" means the caller saw the unit inactive; settleUnit
	// is not called at all in that case (Task 6). This pins the caller's
	// contract from this side: an inactive unit that stays inactive after
	// try-restart is "nothing started", not a failure.
	f := &fakeUnits{shows: []unitProps{{ActiveState: "inactive"}}}
	if err := settleUnit(f, "console.service", "", opts(nil)); err != nil {
		t.Fatalf("settle on a stopped unit = %v, want nil", err)
	}
}

func TestUnitSettleTimeoutFloor(t *testing.T) {
	env := func(v string) func(string) string { return func(string) string { return v } }
	for in, want := range map[string]time.Duration{"": 30 * time.Second, "1s": 5 * time.Second,
		"45s": 45 * time.Second, "junk": 30 * time.Second} {
		if got := unitSettleTimeout(env(in)); got != want {
			t.Fatalf("CARLOS_UNIT_SETTLE=%q -> %v, want %v", in, got, want)
		}
	}
}
  • [ ] Step 2: Run them to verify they fail

Run: GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/host -run 'Settle|ParseShow' -count=1 Expected: FAIL, undefined: unitProps.

  • [ ] Step 3: Implement unitsettle.go
// The settle check — D1 of the console release defences (spec
// 2026-10-06-console-release-defences-design.md §2.2-§2.4).
//
// `systemctl try-restart` of a Type=simple unit returns 0 as soon as the start
// job is queued, so it says nothing about whether the new build runs. On
// 2026-10-06 the console's new process printed the CLI's usage and exited 2
// after 55 ms while try-restart reported success, and adoptHost's rollback
// never fired. settleUnit watches the unit after the restart and returns
// *release.UnitNotUpError when the new build does not come up, which drives
// that rollback.
package host

import (
	"fmt"
	"os"
	"os/exec"
	"regexp"
	"strconv"
	"strings"
	"time"

	"github.com/carlosframework/platform/internal/release"
)

type unitProps struct {
	ActiveState  string
	InvocationID string
	MainPID      int
	NRestarts    int
	ExecPath     string // ExecStart's executable path
}

type unitSystem interface {
	Show(unit string) (unitProps, error)
	TryRestart(unit string) error
	Restart(unit string) error
	ResetFailed(unit string) error
	JournalTail(unit string, n int) string
	Exe(pid int) (string, error)
}

type systemdUnits struct{}

func (systemdUnits) Show(unit string) (unitProps, error) {
	out, err := exec.Command("systemctl", "show", unit,
		"-p", "ActiveState", "-p", "InvocationID", "-p", "MainPID", "-p", "NRestarts", "-p", "ExecStart").Output()
	if err != nil {
		return unitProps{}, fmt.Errorf("systemctl show %s: %w", unit, err)
	}
	return parseShow(string(out)), nil
}

func (systemdUnits) TryRestart(unit string) error { return systemctlTryRestartUnit(unit) }
func (systemdUnits) Restart(unit string) error    { return systemctlVerb("restart", unit) }
func (systemdUnits) ResetFailed(unit string) error { return systemctlVerb("reset-failed", unit) }

func (systemdUnits) JournalTail(unit string, n int) string {
	out, _ := exec.Command("journalctl", "-u", unit, "-o", "cat", "-n", strconv.Itoa(n), "--no-pager").Output()
	return strings.TrimSpace(string(out))
}

func (systemdUnits) Exe(pid int) (string, error) { return os.Readlink(fmt.Sprintf("/proc/%d/exe", pid)) }

func systemctlVerb(verb, unit string) error {
	out, err := exec.Command("systemctl", verb, unit).CombinedOutput()
	if err != nil {
		return fmt.Errorf("systemctl %s %s: %w: %s", verb, unit, err, strings.TrimSpace(string(out)))
	}
	return nil
}

var execPathRE = regexp.MustCompile(`path=([^ ;]+)`)

func parseShow(out string) unitProps {
	var p unitProps
	for _, line := range strings.Split(out, "\n") {
		k, v, ok := strings.Cut(strings.TrimSpace(line), "=")
		if !ok {
			continue
		}
		switch k {
		case "ActiveState":
			p.ActiveState = v
		case "InvocationID":
			p.InvocationID = v
		case "MainPID":
			p.MainPID, _ = strconv.Atoi(v)
		case "NRestarts":
			p.NRestarts, _ = strconv.Atoi(v)
		case "ExecStart":
			if m := execPathRE.FindStringSubmatch(v); m != nil {
				p.ExecPath = m[1]
			}
		}
	}
	return p
}

type settleOpts struct {
	Timeout time.Duration // the whole window (CARLOS_UNIT_SETTLE)
	Poll    time.Duration // 500ms
	Hold    time.Duration // no-probe readiness: active this long in one invocation
	WantDir string        // the new version's directory; MainPID's exe must be under it
	Ready   func() error  // nil = no upstream to probe; Hold applies
	Now     func() time.Time
	Sleep   func(time.Duration)
}

// settleUnit watches unit after a try-restart. beforeInvocation is the
// InvocationID before the restart; "" means the unit was not running, and an
// inactive unit is then "nothing started", not a failure.
func settleUnit(sys unitSystem, unit, beforeInvocation string, o settleOpts) error {
	deadline := o.Now().Add(o.Timeout)
	var inv string        // the new invocation, once seen active
	var activeSince time.Time
	var lastReady error
	notUp := func(reason string) error {
		return &release.UnitNotUpError{Unit: unit, Reason: reason, Journal: sys.JournalTail(unit, 20)}
	}
	for {
		p, err := sys.Show(unit)
		if err != nil {
			return notUp(err.Error())
		}
		if beforeInvocation == "" && p.ActiveState == "inactive" {
			return nil
		}
		switch {
		case p.ActiveState == "failed":
			return notUp("unit entered failed state")
		case inv != "" && p.InvocationID != inv:
			return notUp("systemd restarted it (new invocation)")
		case inv != "" && p.NRestarts > 0:
			return notUp(fmt.Sprintf("systemd restarted it %d time(s)", p.NRestarts))
		case inv == "" && p.ActiveState == "active" && p.InvocationID != beforeInvocation:
			inv, activeSince = p.InvocationID, o.Now()
		}
		if inv != "" && p.ActiveState == "active" {
			exe, err := sys.Exe(p.MainPID)
			if err == nil && o.WantDir != "" && !strings.HasPrefix(exe, strings.TrimSuffix(o.WantDir, "/")+"/") {
				return notUp(fmt.Sprintf("running %s, not the new build in %s", exe, o.WantDir))
			}
			if o.Ready != nil {
				if lastReady = o.Ready(); lastReady == nil {
					return nil
				}
			} else if o.Now().Sub(activeSince) >= o.Hold {
				return nil
			}
		}
		if !o.Now().Before(deadline) {
			if lastReady != nil {
				return notUp(fmt.Sprintf("not ready after %s: %v", o.Timeout, lastReady))
			}
			return notUp(fmt.Sprintf("not ready after %s", o.Timeout))
		}
		o.Sleep(o.Poll)
	}
}

func unitSettleTimeout(getenv func(string) string) time.Duration {
	d, err := time.ParseDuration(strings.TrimSpace(getenv("CARLOS_UNIT_SETTLE")))
	switch {
	case err != nil || d <= 0:
		return 30 * time.Second
	case d < 5*time.Second:
		return 5 * time.Second
	}
	return d
}

systemctlTryRestartUnit already exists in agent.go and is reused, not duplicated.

  • [ ] Step 4: Run the tests to verify they pass

Run: GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/host -run 'Settle|ParseShow' -count=1 Expected: PASS.

  • [ ] Step 5: Commit
git add internal/host/unitsettle.go internal/host/unitsettle_test.go
git commit -m "host: settle check for a named unit after try-restart"

---

Task 5: Readiness on the effective upstream

Files:

  • Modify: internal/host/unitsettle.go (append)
  • Test: internal/host/unitsettle_test.go (append)

Interfaces:

  • Produces: func routeReady(rt registry.Route, timeout time.Duration) func() error, which returns nil when the route has no upstream.
  • Consumes: provision.Health and .WithDefaults() (internal/provision/provision.go:189).
  • [ ] Step 1: Write the failing tests
func TestRouteReadyProbesTheUnixSocket(t *testing.T) {
	sock := filepath.Join(t.TempDir(), "s.sock")
	ln, err := net.Listen("unix", sock)
	if err != nil {
		t.Fatal(err)
	}
	srv := &http.Server{Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		if r.URL.Path == "/healthz" && r.Host == "console.example.com" {
			w.WriteHeader(200)
			return
		}
		w.WriteHeader(500)
	})}
	go srv.Serve(ln)
	t.Cleanup(func() { srv.Close() })
	rt := registry.Route{Host: "console.example.com", Socket: sock, Addr: "127.0.0.1:1",
		Health: `{"path":"/healthz","status":200}`}
	if err := routeReady(rt, time.Second)(); err != nil {
		t.Fatalf("ready over the socket = %v (Addr must be ignored when Socket is set)", err)
	}
}

func TestRouteReadyWrongStatus(t *testing.T) {
	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(503) }))
	t.Cleanup(srv.Close)
	rt := registry.Route{Host: "x.example.com", Addr: strings.TrimPrefix(srv.URL, "http://"),
		Health: `{"path":"/healthz","status":200}`}
	if err := routeReady(rt, time.Second)(); err == nil || !strings.Contains(err.Error(), "503") {
		t.Fatalf("ready = %v, want a 503 failure", err)
	}
}

func TestRouteReadyNoProbeIsAConnect(t *testing.T) {
	ln, err := net.Listen("tcp", "127.0.0.1:0")
	if err != nil {
		t.Fatal(err)
	}
	t.Cleanup(func() { ln.Close() })
	if err := routeReady(registry.Route{Host: "x", Addr: ln.Addr().String()}, time.Second)(); err != nil {
		t.Fatalf("connect readiness = %v", err)
	}
}

func TestRouteReadyNoUpstreamIsNil(t *testing.T) {
	if routeReady(registry.Route{Host: "x"}, time.Second) != nil {
		t.Fatal("a route with no upstream got a ready check; settle must use the hold instead")
	}
}

Add net, net/http, net/http/httptest, path/filepath and github.com/carlosframework/platform/internal/registry to the test imports.

  • [ ] Step 2: Run them to verify they fail

Run: GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/host -run 'RouteReady' -count=1 Expected: FAIL, undefined: routeReady.

  • [ ] Step 3: Implement

Append to unitsettle.go (add context, encoding/json, io, net, net/http, registry and provision imports):

// routeReady is the readiness check for rt's effective upstream (spec §2.4):
// Socket wins over Addr. With a configured probe it is an HTTP GET; without
// one, a connect. nil when the route has no upstream: settle then requires
// the unit to stay active for its Hold.
func routeReady(rt registry.Route, timeout time.Duration) func() error {
	network, address := "unix", rt.Socket
	if address == "" {
		network, address = "tcp", rt.Addr
	}
	if address == "" {
		return nil
	}
	if rt.Health == "" {
		return func() error {
			c, err := net.DialTimeout(network, address, timeout)
			if err != nil {
				return fmt.Errorf("connect %s: %w", address, err)
			}
			return c.Close()
		}
	}
	var h provision.Health
	if err := json.Unmarshal([]byte(rt.Health), &h); err != nil {
		return func() error { return fmt.Errorf("route health probe does not parse: %w", err) }
	}
	h = h.WithDefaults()
	hostHdr := h.HostHeader
	if hostHdr == "" {
		hostHdr = rt.Host
	}
	client := &http.Client{Timeout: timeout, Transport: &http.Transport{
		DialContext: func(ctx context.Context, _, _ string) (net.Conn, error) {
			var d net.Dialer
			return d.DialContext(ctx, network, address)
		},
	}}
	return func() error {
		req, err := http.NewRequest(http.MethodGet, "http://upstream"+h.Path, nil)
		if err != nil {
			return err
		}
		req.Host = hostHdr
		resp, err := client.Do(req)
		if err != nil {
			return fmt.Errorf("dial: %v", err)
		}
		io.Copy(io.Discard, io.LimitReader(resp.Body, 4<<10))
		resp.Body.Close()
		if resp.StatusCode != h.Status {
			return fmt.Errorf("status %d (want %d)", resp.StatusCode, h.Status)
		}
		return nil
	}
}
  • [ ] Step 4: Run the tests to verify they pass

Run: GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/host -run 'RouteReady|Settle' -count=1 Expected: PASS.

  • [ ] Step 5: Commit
git add internal/host/unitsettle.go internal/host/unitsettle_test.go
git commit -m "host: readiness on a route's effective upstream, socket before addr"

---

Task 6: Wire the checked restart and the rollback restart

Files:

  • Modify: internal/host/agent.go: makeRestartRoute (3101) gets tryRestartUnit func(rt registry.Route) error; new checkedUnitRestart and rollbackUnitRestart; callers at 820, 1931, 3243; RunAdopt pass options at 3660; the agent loop options at 871 and 1500.
  • Modify: internal/host/agent_test.go: the seven makeRestartRoute( call sites wrap their unit fake with unitOnly(...).
  • Test: internal/host/unitsettle_test.go (append)

Interfaces:

  • Consumes: Tasks 4 and 5 (unitSystem, settleUnit, settleOpts, routeReady, unitSettleTimeout); Task 3's release.WithRollbackRestart.
  • Produces: - func unitOnly(f func(unit string) error) func(rt registry.Route) error - func checkedUnitRestart(sys unitSystem, optDir string, getenv func(string) string) func(rt registry.Route) error - func rollbackUnitRestart(reg *registry.Registry, sys unitSystem, optDir string, getenv func(string) string) func(host string) error
  • [ ] Step 1: Write the failing tests
func boundRoute() registry.Route {
	return registry.Route{Host: "platform.example.com", App: "console", Unit: "console.service", Addr: "127.0.0.1:1"}
}

func liveTo(t *testing.T, opt, host, ver string) {
	t.Helper()
	dir := filepath.Join(opt, "versions", ver)
	if err := os.MkdirAll(dir, 0o755); err != nil {
		t.Fatal(err)
	}
	if err := os.WriteFile(filepath.Join(dir, "console"), []byte("x"), 0o755); err != nil {
		t.Fatal(err)
	}
	if err := os.MkdirAll(filepath.Join(opt, "live"), 0o755); err != nil {
		t.Fatal(err)
	}
	os.Remove(filepath.Join(opt, "live", host))
	if err := os.Symlink(filepath.Join(dir, "console"), filepath.Join(opt, "live", host)); err != nil {
		t.Fatal(err)
	}
}

func TestNamedRestartBindingMismatchKeepsTheError(t *testing.T) {
	opt := t.TempDir()
	liveTo(t, opt, "platform.example.com", "new")
	f := &fakeUnits{shows: []unitProps{{ActiveState: "active", InvocationID: "old",
		ExecPath: filepath.Join(opt, "live", "console.example.com")}}}
	failing := &failingTry{fakeUnits: f}
	err := checkedUnitRestart(failing, opt, func(string) string { return "" })(boundRoute())
	if err == nil || !strings.Contains(err.Error(), "try-restart failed") {
		t.Fatalf("err = %v, want the try-restart error propagated (codex 12)", err)
	}
}

type failingTry struct{ *fakeUnits }

func (failingTry) TryRestart(string) error { return errors.New("try-restart failed") }

func TestNamedRestartShowFailsFallsBack(t *testing.T) {
	opt := t.TempDir()
	sys := &showFails{fakeUnits: &fakeUnits{}}
	if err := checkedUnitRestart(sys, opt, func(string) string { return "" })(boundRoute()); err != nil {
		t.Fatalf("err = %v, want today's behaviour (plain try-restart, its nil)", err)
	}
	if len(sys.restarts) != 1 || sys.restarts[0] != "try-restart console.service" {
		t.Fatalf("restarts = %v", sys.restarts)
	}
}

type showFails struct{ *fakeUnits }

func (showFails) Show(string) (unitProps, error) { return unitProps{}, errors.New("Access denied") }

func TestNamedRestartStoppedUnitIsNotChecked(t *testing.T) {
	opt := t.TempDir()
	liveTo(t, opt, "platform.example.com", "new")
	f := &fakeUnits{shows: []unitProps{{ActiveState: "inactive", ExecPath: filepath.Join(opt, "live", "platform.example.com")}}}
	if err := checkedUnitRestart(f, opt, func(string) string { return "" })(boundRoute()); err != nil {
		t.Fatalf("err = %v, want nil for an operator-stopped unit", err)
	}
}

func TestRollbackRestartResetsThenRestarts(t *testing.T) {
	opt := t.TempDir()
	liveTo(t, opt, "platform.example.com", "old")
	reg := restartTestRegistry(t)
	if err := reg.Upsert(boundRoute()); err != nil {
		t.Fatal(err)
	}
	f := &fakeUnits{shows: []unitProps{
		{ActiveState: "active", InvocationID: "bad"},
		{ActiveState: "active", InvocationID: "restored", MainPID: 9},
	}, exe: map[int]string{9: filepath.Join(opt, "versions", "old", "console")}}
	rb := rollbackUnitRestart(reg, f, opt, func(string) string { return "" })
	if err := rb("platform.example.com"); err != nil {
		t.Fatalf("rollback = %v", err)
	}
	if strings.Join(f.restarts, ",") != "reset-failed console.service,restart console.service" {
		t.Fatalf("restarts = %v, want reset-failed then restart (never start)", f.restarts)
	}
}

restartTestRegistry is agent_test.go's existing constructor (line 539). In TestRollbackRestartResetsThenRestarts, upsert boundRoute() with Addr cleared (r := boundRoute(); r.Addr = ""). With no upstream, routeReady returns nil and settle uses the hold instead of dialling. rollbackUnitRestart uses the real clock, so this test takes about 5 s, which is acceptable for one test. The binding test keeps Addr because it returns before readiness.

  • [ ] Step 2: Run them to verify they fail

Run: GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/host -run 'NamedRestart|RollbackRestart' -count=1 Expected: FAIL, undefined: checkedUnitRestart.

  • [ ] Step 3: Implement

In agent.go, change makeRestartRoute's signature and its first arm:

func makeRestartRoute(reg *registry.Registry, tryRestart func(host string), tryRestartUnit func(rt registry.Route) error, bounce func(host string) bool) func(host string) error {
	return func(host string) error {
		rt, err := reg.Get(host)
		if err != nil {
			log.Printf("adopt: registry lookup %s: %v (restarting as a unit)", host, err)
		}
		if err == nil && rt != nil && rt.Unit != "" {
			return tryRestartUnit(*rt)
		}

(the rest unchanged). Add, next to systemctlTryRestartUnit:

// unitOnly adapts a unit-name restart to makeRestartRoute's route-aware arm.
func unitOnly(f func(unit string) error) func(rt registry.Route) error {
	return func(rt registry.Route) error { return f(rt.Unit) }
}

// checkedUnitRestart is the named-unit arm with D1's settle check (spec
// §2.2-§2.3). The binding is verified first: the check judges a restart only
// when the unit's ExecStart runs THIS route's live link. Anything it cannot
// establish (show denied, wrong binding, unit stopped) falls back to today's
// plain try-restart and returns that command's own error.
func checkedUnitRestart(sys unitSystem, optDir string, getenv func(string) string) func(rt registry.Route) error {
	return func(rt registry.Route) error {
		before, err := sys.Show(rt.Unit)
		if err != nil {
			log.Printf("adopt: %s: cannot read %s (%v); restart not checked", rt.Host, rt.Unit, err)
			return sys.TryRestart(rt.Unit)
		}
		link := filepath.Join(optDir, "live", rt.Host)
		if before.ExecPath != link {
			log.Printf("adopt: binding: %s runs %s, but the unit stamp is on %s; restart not checked", rt.Unit, before.ExecPath, rt.Host)
			return sys.TryRestart(rt.Unit)
		}
		if before.ActiveState != "active" {
			return sys.TryRestart(rt.Unit) // stopped by an operator: stays a no-op
		}
		want, err := filepath.EvalSymlinks(link)
		if err != nil {
			return sys.TryRestart(rt.Unit)
		}
		if err := sys.TryRestart(rt.Unit); err != nil {
			return err
		}
		timeout := unitSettleTimeout(getenv)
		err = settleUnit(sys, rt.Unit, before.InvocationID, settleOpts{
			Timeout: timeout, Poll: 500 * time.Millisecond, Hold: 5 * time.Second,
			WantDir: filepath.Dir(want), Ready: routeReady(rt, 2*time.Second),
			Now: time.Now, Sleep: time.Sleep})
		if err == nil {
			log.Printf("adopt: %s: %s settled on %s", rt.Host, rt.Unit, filepath.Dir(want))
		}
		return err
	}
}

// rollbackUnitRestart is D1's rollback restart (release.WithRollbackRestart):
// the live link already names the restored build; replace whatever is
// running (a deadline failure can leave the rejected build up) and check
// the restored build comes up too.
func rollbackUnitRestart(reg *registry.Registry, sys unitSystem, optDir string, getenv func(string) string) func(host string) error {
	return func(host string) error {
		rt, err := reg.Get(host)
		if err != nil || rt == nil || rt.Unit == "" {
			return nil
		}
		before, _ := sys.Show(rt.Unit)
		if err := sys.ResetFailed(rt.Unit); err != nil {
			log.Printf("adopt: %s: reset-failed %s: %v (continuing)", host, rt.Unit, err)
		}
		if err := sys.Restart(rt.Unit); err != nil {
			return err
		}
		want, err := filepath.EvalSymlinks(filepath.Join(optDir, "live", host))
		if err != nil {
			return err
		}
		if err := settleUnit(sys, rt.Unit, before.InvocationID, settleOpts{
			Timeout: unitSettleTimeout(getenv), Poll: 500 * time.Millisecond, Hold: 5 * time.Second,
			WantDir: filepath.Dir(want), Ready: routeReady(*rt, 2*time.Second),
			Now: time.Now, Sleep: time.Sleep}); err != nil {
			return fmt.Errorf("rollback did not come up either: %w", err)
		}
		return nil
	}
}

settleUnit treats beforeInvocation == "" with an inactive unit as nothing started. In rollbackUnitRestart the unit has just been restarted, so before.InvocationID is whatever ran before. A failed unit keeps its last InvocationID, and a new one appears on restart, which is what settle keys on.

Callers:

  • line 820 and line 1931: replace systemctlTryRestartUnit with checkedUnitRestart(systemdUnits{}, optDir, os.Getenv), using whatever opt-dir variable is in scope there (d.optDir on the fast path).
  • line 3243 rootRestartRoute: the same, with the opt dir its caller uses (RunAdopt's). Add an optDir string parameter to rootRestartRoute and pass it from its one production caller. Update its three test callers too: internal/host/agent_test.go:718, :836 and :963, each passing that test's own opt dir (or t.TempDir() where the test has none).
  • RunAdopt's passOpts (3660): append release.WithRollbackRestart(rollbackUnitRestart(reg, systemdUnits{}, optDir, os.Getenv)).
  • the agent loop options at 871 and 1500: append the same option. The sandboxed agent's reset-failed/restart may be denied; that is logged and the rollback line says "could not start it", the same as today's start path there.

In agent_test.go, change each makeRestartRoute(reg, X, Y, Z) whose third argument is a func(string) error to makeRestartRoute(reg, X, unitOnly(Y), Z).

  • [ ] Step 4: Run the host and release suites

Run: GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/host ./internal/release -count=1 Expected: PASS. Per [[cpu-budget-docs-t1]], run them niced and one at a time if the box is busy: nice -n 10.

  • [ ] Step 5: Commit
git add internal/host/agent.go internal/host/agent_test.go internal/host/unitsettle_test.go
git commit -m "host: named-unit restarts settle and roll back by restart (console defences D1)"

---

Task 7: carlos promote --retry

Files:

  • Modify: internal/release/release.go: PromoteOptions (786) adds Retry bool; Promotion (144) adds Retry bool \json:"retry,omitempty"\`; PromoteOnLadder`'s repeat branch.
  • Modify: cli.go (2356: flag; 2397 and 2417: pass it)
  • Modify: apimode.go:947 (promote payload)
  • Modify: internal/console/apicli.go (743: request field; 899: option)
  • Test: internal/release/release_test.go (append), internal/console/apicli_test.go (append)

Interfaces:

  • Produces: PromoteOptions.Retry. With Retry, a repeat is re-signed with a fresh PromotedAt. Without a repeat it is an error: promote --retry renews a promotion: <channel> names <cur>, not <version>.
  • [ ] Step 1: Write the failing tests
func TestPromoteRetryRenewsPromotedAt(t *testing.T) {
	s := NewDirStore(t.TempDir())
	ctx := context.Background()
	priv, pub := testKeys(t)
	shipRelease(t, s, "bab", "console", "v1", adoptPlatform, []byte("b"))
	if _, err := PromoteOnLadder(ctx, s, priv, "bab", "console", "canary/x", "v1", PromoteOptions{}); err != nil {
		t.Fatal(err)
	}
	before, _ := Channel(ctx, s, pub, "bab", "console", "canary/x")
	time.Sleep(1100 * time.Millisecond) // PromotedAt is truncated to the second
	res, err := PromoteOnLadder(ctx, s, priv, "bab", "console", "canary/x", "v1", PromoteOptions{})
	if err != nil || !res.NoOp {
		t.Fatalf("plain repeat = %+v, %v; want the existing no-op", res, err)
	}
	same, _ := Channel(ctx, s, pub, "bab", "console", "canary/x")
	if !same.PromotedAt.Equal(before.PromotedAt) {
		t.Fatal("a plain repeat rewrote the pointer")
	}
	if _, err := PromoteOnLadder(ctx, s, priv, "bab", "console", "canary/x", "v1", PromoteOptions{Retry: true}); err != nil {
		t.Fatalf("retry: %v", err)
	}
	after, _ := Channel(ctx, s, pub, "bab", "console", "canary/x")
	if after.Version != "v1" || !after.PromotedAt.After(before.PromotedAt) {
		t.Fatalf("after retry = %+v, want v1 with a later PromotedAt", after)
	}
}

func TestPromoteRetryNeverChangesTheVersion(t *testing.T) {
	s := NewDirStore(t.TempDir())
	ctx := context.Background()
	priv, _ := testKeys(t)
	shipRelease(t, s, "bab", "console", "v1", adoptPlatform, []byte("a"))
	shipRelease(t, s, "bab", "console", "v2", adoptPlatform, []byte("b"))
	if _, err := PromoteOnLadder(ctx, s, priv, "bab", "console", "canary/x", "v1", PromoteOptions{}); err != nil {
		t.Fatal(err)
	}
	_, err := PromoteOnLadder(ctx, s, priv, "bab", "console", "canary/x", "v2", PromoteOptions{Retry: true})
	if err == nil || !strings.Contains(err.Error(), "renews a promotion") {
		t.Fatalf("retry onto a different version = %v, want a refusal", err)
	}
}

For the API door, add one test beside the existing handleAPIPromote tests in internal/console/apicli_test.go. Copy the nearest existing promote test's setup verbatim, send {"version":"v1","channel":"canary/x","retry":true} after a first promote, and assert 200 and that the stored pointer's PromotedAt moved.

  • [ ] Step 2: Run them to verify they fail

Run: GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/release -run 'PromoteRetry' -count=1 Expected: FAIL, unknown field Retry.

  • [ ] Step 3: Implement

PromoteOptions: add

	// Retry re-signs a pointer that already names this version, with a
	// fresh PromotedAt: the way past a box's D1 refusal (refusal.go), which
	// blocks one promotion, not a version. Never changes the version.
	Retry bool

Promotion: add Retry bool \json:"retry,omitempty"\``.

In PromoteOnLadder's repeat branch, before res.NoOp = true, insert:

			if opt.Retry {
				res.From = res.To
				rec := Promotion{From: res.To, To: res.To, Actor: opt.Actor, Retry: true}
				if !time.Now().UTC().Truncate(time.Second).After(p.PromotedAt) {
					time.Sleep(time.Until(p.PromotedAt.Add(time.Second)))
				}
				if err := sg.promote(ctx, s, acct, app, channel, version, opt.Authorize, rec); err != nil {
					return res, err
				}
				return res, nil
			}

and after the repeat branch's closing brace (so it is reached only when the pointer is absent or names another version), insert:

	if opt.Retry {
		cur := "nothing"
		if c, _, err := s.Get(ctx, ChannelKey(acct, app, channel)); err == nil {
			if p, verr := verifyPointer(pub, c, acct, app, channel); verr == nil {
				cur = p.Version
			}
		}
		return res, fmt.Errorf("promote --retry renews a promotion: %s names %s, not %s", channel, cur, version)
	}

cli.go: next to hotfix := fs.Bool("hotfix", ...) (2356) add

	retry := fs.Bool("retry", false, "re-sign the channel's current promotion of this version with a fresh time, so a box that refused it tries again")

and pass it: release.PromoteOptions{Hotfix: *hotfix, Retry: *retry, Actor: promoteActor()} (2417), and in API mode sess.promote(..., *hotfix, *retry, message).

apimode.go:947: add a retry bool parameter after hotfix, and a Retry bool \json:"retry,omitempty"\` field in the payload. Update its one other caller if any (grep sess.promote( and .promote(ctx`).

internal/console/apicli.go: add Retry bool \json:"retry"\` to the request struct (743), and Retry: req.Retry to the PromoteOptions at 899. The retry goes through the same Authorize` and scope checks as any promote.

  • [ ] Step 4: Run the tests

Run: GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/release ./internal/console -run 'Promote' -count=1 && GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go build ./... Expected: PASS, and the build succeeds.

  • [ ] Step 5: Commit
git add internal/release/release.go internal/release/release_test.go cli.go apimode.go internal/console/apicli.go internal/console/apicli_test.go
git commit -m "promote --retry: re-sign a promotion so a box retries a refused build"

---

Task 8: The alert and the status --live line

Files:

  • Create: internal/host/refusalwatch.go
  • Modify: internal/host/alertsink.go (add deliverRelease)
  • Modify: internal/status/snapshot.go (ReleaseRefusal type; Probes.ReleaseRefusals; BoxMetrics.ReleaseRefusals; the copy at 719 and the emptiness check at 700)
  • Modify: internal/host/agent.go:2939 (feed the probe), and the metrics loop (internal/host/metrics_loop.go:511 neighbourhood) to call the watcher each tick
  • Modify: cli.go:3418 printLiveBoxes
  • Test: internal/host/refusalwatch_test.go, statuslive_test.go (the existing printLiveBoxes tests, extended)

Interfaces:

  • Consumes: Task 1's release.ListRefusals.
  • Produces: - type ReleaseRefusal struct { Host, Version, Reason string; RefusedAt time.Time } (status package, JSON host, version, reason, refused_at) - type refusalWatch struct { seen map[string]time.Time } with func (w *refusalWatch) tick(ctx context.Context, bakeDir string, sinks *alertSinks, now time.Time) []status.ReleaseRefusal - func (a *alertSinks) deliverRelease(ctx context.Context, r release.Refusal)
  • [ ] Step 1: Write the failing tests
package host

import (
	"context"
	"strings"
	"testing"
	"time"

	"github.com/carlosframework/platform/internal/release"
)

func TestRefusalWatchAlertsOncePerRecord(t *testing.T) {
	bake := t.TempDir()
	now := time.Date(2026, 10, 6, 9, 31, 0, 0, time.UTC)
	if err := release.WriteRefusal(bake, release.Refusal{Host: "console.example.com", App: "console",
		Version: "v2", RefusedAt: now.Add(-time.Minute), Reason: "exited: status=2"}); err != nil {
		t.Fatal(err)
	}
	var posts []string
	sinks := newAlertSinks("https://hook.example", func(_ string, b []byte) error { posts = append(posts, string(b)); return nil }, "", nil, &alertCounters{})
	w := &refusalWatch{}
	got := w.tick(context.Background(), bake, sinks, now)
	w.tick(context.Background(), bake, sinks, now.Add(time.Minute))
	if len(posts) != 1 || !strings.Contains(posts[0], `"release_refused"`) || !strings.Contains(posts[0], "v2") {
		t.Fatalf("posts = %v, want exactly one release_refused alert", posts)
	}
	if len(got) != 1 || got[0].Host != "console.example.com" {
		t.Fatalf("status rows = %+v", got)
	}
}

func TestRefusalWatchSkipsOldRecordsAtStartup(t *testing.T) {
	bake := t.TempDir()
	now := time.Date(2026, 10, 6, 12, 0, 0, 0, time.UTC)
	if err := release.WriteRefusal(bake, release.Refusal{Host: "a.example.com", Version: "v2",
		RefusedAt: now.Add(-2 * time.Hour)}); err != nil {
		t.Fatal(err)
	}
	var posts int
	sinks := newAlertSinks("https://hook.example", func(string, []byte) error { posts++; return nil }, "", nil, &alertCounters{})
	(&refusalWatch{}).tick(context.Background(), bake, sinks, now)
	if posts != 0 {
		t.Fatalf("an agent restart re-alerted a refusal from two hours ago")
	}
}

In statuslive_test.go, extend the existing printLiveBoxes test with a box whose Metrics.ReleaseRefusals has one row. Assert that the output contains refused console.example.com v2 since 2026-10-06T09:30:40Z: exited: status=2.

  • [ ] Step 2: Run them to verify they fail

Run: GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/host -run 'RefusalWatch' -count=1 Expected: FAIL, undefined: refusalWatch.

  • [ ] Step 3: Implement

internal/status/snapshot.go, beside ConfigRefusal:

// ReleaseRefusal is a build this box rolled back because its unit did not
// come up (D1, release.Refusal). PRIVATE like the rest of Box.Metrics: Host
// is a real route name.
type ReleaseRefusal struct {
	Host      string    `json:"host"`
	Version   string    `json:"version"`
	Reason    string    `json:"reason"`
	RefusedAt time.Time `json:"refused_at"`
}

Add ReleaseRefusals []ReleaseRefusal to Probes and to the metrics struct that ConfigRefusals is copied into at line 719, with JSON key release_refusals,omitempty. Copy it there the same way, and add len(p.ReleaseRefusals) > 0 || to the emptiness check at 700.

internal/host/alertsink.go, after deliver:

// deliverRelease sends a D1 refusal to every configured sink. The webhook
// body is the record with kind "release_refused"; SNS gets a sentence.
func (a *alertSinks) deliverRelease(ctx context.Context, r release.Refusal) {
	if a == nil {
		return
	}
	if a.webhookURL != "" && a.post != nil {
		body, err := json.Marshal(struct {
			Kind string `json:"kind"`
			release.Refusal
		}{"release_refused", r})
		if err == nil {
			if err := a.post(a.webhookURL, body); err != nil {
				a.counts.webhook.Add(1)
				log.Printf("host: alert webhook post (release refusal %s): %v", r.Host, err)
			}
		}
	}
	if a.topicARN != "" && a.pub != nil {
		subject := fmt.Sprintf("%s: %s did not start, rolled back", r.Host, r.Version)
		if len(subject) > 100 { // SNS subject limit
			subject = subject[:100]
		}
		msg := fmt.Sprintf("%s refused %s at %s: %s\n\nRetry with: carlos promote --retry %s <channel>",
			r.Host, r.Version, r.RefusedAt.Format(time.RFC3339), r.Reason, r.Version)
		pctx, cancel := context.WithTimeout(ctx, alertPublishTimeout)
		defer cancel()
		if _, err := a.pub.Publish(pctx, &sns.PublishInput{TopicArn: &a.topicARN, Subject: &subject, Message: &msg}); err != nil {
			a.counts.sns.Add(1)
			log.Printf("host: alert sns publish (release refusal %s): %v", r.Host, err)
		}
	}
}

internal/host/refusalwatch.go:

// refusalWatch turns D1's refusal records (release.Refusal, written by the
// root pass) into one alert each and a status row each. It runs on the
// agent's metrics tick. A record older than refusalAlertWindow when first
// seen is not alerted, so an agent restart does not re-send old news.
package host

import (
	"context"
	"log"
	"time"

	"github.com/carlosframework/platform/internal/release"
	"github.com/carlosframework/platform/internal/status"
)

const refusalAlertWindow = 30 * time.Minute

type refusalWatch struct{ seen map[string]time.Time }

func (w *refusalWatch) tick(ctx context.Context, bakeDir string, sinks *alertSinks, now time.Time) []status.ReleaseRefusal {
	if w.seen == nil {
		w.seen = map[string]time.Time{}
	}
	recs, err := release.ListRefusals(bakeDir)
	if err != nil {
		log.Printf("agent: refusals: %v", err)
		return nil
	}
	var rows []status.ReleaseRefusal
	for _, r := range recs {
		rows = append(rows, status.ReleaseRefusal{Host: r.Host, Version: r.Version, Reason: r.Reason, RefusedAt: r.RefusedAt})
		if last, ok := w.seen[r.Host]; ok && !r.RefusedAt.After(last) {
			continue
		}
		w.seen[r.Host] = r.RefusedAt
		if now.Sub(r.RefusedAt) <= refusalAlertWindow {
			sinks.deliverRelease(ctx, r)
		}
	}
	return rows
}

Wire it. In the metrics loop, keep one refusalWatch for the life of the loop. Each tick, call rows := watch.tick(ctx, dataDir(), sinks, now) and store the rows where the probe at agent.go:2939 can read them. Follow the configRefusalProbe pattern: a package-level atomic.Pointer[[]status.ReleaseRefusal] written by the tick and read by a releaseRefusalProbe() that fills p.ReleaseRefusals.

cli.go printLiveBoxes: after the box's main Fprintf, add

		for _, r := range m.ReleaseRefusals {
			fmt.Fprintf(w, "  refused %s %s since %s: %s\n", r.Host, r.Version, r.RefusedAt.UTC().Format(time.RFC3339), r.Reason)
		}
  • [ ] Step 4: Run the tests

Run: GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/host ./internal/status . -run 'RefusalWatch|LiveBoxes|Snapshot' -count=1 Expected: PASS.

  • [ ] Step 5: Commit
git add internal/host/refusalwatch.go internal/host/refusalwatch_test.go internal/host/alertsink.go internal/host/agent.go internal/host/metrics_loop.go internal/status/snapshot.go cli.go statuslive_test.go
git commit -m "host: alert on a refused build and show it in status --live"

---

Task 9: Full gate and the README operator note

Files:

  • Modify: README.md: under "Releasing the platform", in the release-notes area that lists operator checks, add the note below.
  • [ ] Step 1: Add the note
**Named units now settle (console release defences, D1).** After adopting a
build for a route with a named unit (the console), the box waits up to 30 s
(`CARLOS_UNIT_SETTLE`) for the unit to stay up on the new build and pass its
health probe. If it does not, the box rolls back, refuses that promotion and
alerts. Retry with `carlos promote --retry <version> <channel>`. The check
only runs when the unit's `ExecStart` names the stamped route's own live
link; `adopt` logs `binding: … restart not checked` where it does not, and
that box needs its stamp moved (2026-08-23 console design §4c).

Keep its sentences plain and avoid em dashes in new prose ([[copy-voice-no-em-dashes]]). It is README prose, so invoke the copy-review skill on it before committing and apply Paul's edits verbatim.

  • [ ] Step 2: Run the full gate

Run, niced and serial ([[cpu-budget-docs-t1]]):

GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false nice -n 10 go vet ./... && \
GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false nice -n 10 go test ./... -count=1 && \
GOOS=darwin GOARCH=arm64 GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go build ./... && \
GOOS=windows GOARCH=amd64 GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go build ./...

Expected: all pass. unitsettle.go uses no linux-only syscalls (exec and /proc reads only), so it needs no build tag ([[cross-compile-gate]]). If the cross build fails on it anyway, split the systemdUnits methods into unitsettle_linux.go with a !linux stub returning an error from Show.

  • [ ] Step 3: Commit
git add README.md
git commit -m "README: operator note for the named-unit settle check"