carlos / platform Public
Console as an App — Implementation Plan
> For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.
Goal: Make the CARLOS console an ordinary app deployed by carlos deploy, retiring roll-console.sh to break-glass.
Architecture: The console already ships as a release, rides a channel and is adopted through a route — what it lacks is an app row (blocked by #79), an honest way to verify which build is running (the edge's X-Carlos-Version is stamped from the route row and lies), and a rollback that leaves the box running. Tasks 1–4 give the app a row and a control-plane flag that gates destruction; tasks 5–6 replace header-watching with a probe the console process answers itself; tasks 7–10 close three failure modes the pinfra script has been papering over. Then a runbook rolls it out, tito first.
Tech Stack: Go 1.24, SQLite (modernc driver), systemd, AWS SSM (runbook only, never from carlos).
Spec: docs/superpowers/specs/2026-08-23-console-as-an-app-design.md — read it first; this plan argues from it.
Global Constraints
go build ./...,go vet ./...,go test ./... -count=1clean, every package, before any commit lands.gofmt -lclean for files you touch. Three pre-existing unformatted files elsewhere are left alone.GOOS=darwin go build ./...must pass —internal/hostlinks into the CLI, which people run on Macs.- Sandbox:
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off. - Wire-shape changes are additive. Anything written before your change must still parse.
- Console gate order never moves: session → CSRF → owner (byte-identical 404) →
requireFreshAuthLAST. - A test that shells out to the
carlosbinary must build its env throughcarlosTestEnvand setcmd.Dirto a scratch dir. - Every change lands through a PR. Nothing merges to
mainlocally. - Branch:
console-as-an-app(already exists, carries the spec).
---
Task 1: Two columns on apps, and the migration trap
Files:
- Modify:
internal/console/store.go:189-231(the additive ALTER block),internal/console/store.go:266-288(migrateAppsKey'sapps_new) - Modify:
internal/console/store.go(new accessors nearSetAppPlacement, ~line 1170) - Test:
internal/console/migrate_test.go
Interfaces:
- Consumes: nothing.
- Produces: -
func (s *Store) AppFlags(accountID int64, app string) (controlPlane bool, buildProbe string, err error)-func (s *Store) SetAppControlPlane(accountID int64, app string, on bool) error-func (s *Store) SetAppBuildProbe(accountID int64, app, path string) error
Why both columns land together: they are one migration, and splitting them means touching migrateAppsKey's copy list twice.
The trap (spec §8): the ALTER block at store.go:189 runs before migrateAppsKey() at line 231, and that rebuild copies an enumerated column list into apps_new. A column added only to the ALTER block is silently dropped for any database that has not yet been rebuilt — a self-hoster jumping versions. deleted_at and placement are already in both places for exactly this reason; the comment at store.go:280 says so. Follow that pattern: add to both.
- [ ] Step 1: Write the failing test
Add to internal/console/migrate_test.go:
// TestAppFlagsSurviveTheAppsKeyRebuild: control_plane and build_probe are
// added by the ALTER block, which runs BEFORE migrateAppsKey's table rebuild
// (store.go:189 vs :231). The rebuild copies an ENUMERATED column list, so a
// column missing from apps_new is silently dropped on any database that has
// not been rebuilt yet — a self-hoster jumping versions. deleted_at and
// placement are in both places for this reason (store.go:280); these must be
// too, and this test is what proves it rather than a comment.
func TestAppFlagsSurviveTheAppsKeyRebuild(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "old.db")
// A pre-rebuild database: PRIMARY KEY(app), the shape migrateAppsKey exists
// to replace. Built by hand because no released binary produces it any more.
raw, err := sql.Open("sqlite", path)
if err != nil {
t.Fatal(err)
}
if _, err := raw.Exec(`CREATE TABLE apps (
app TEXT PRIMARY KEY,
account_id INTEGER NOT NULL DEFAULT 1,
created_by TEXT NOT NULL,
created_at INTEGER NOT NULL,
deleted_at INTEGER NOT NULL DEFAULT 0,
placement TEXT NOT NULL DEFAULT ''
);`); err != nil {
t.Fatal(err)
}
if _, err := raw.Exec(`INSERT INTO apps (app, account_id, created_by, created_at)
VALUES ('console', 1, 'paul@keymail.dev', 1)`); err != nil {
t.Fatal(err)
}
if err := raw.Close(); err != nil {
t.Fatal(err)
}
st, err := Open(path)
if err != nil {
t.Fatalf("Open (which runs migrate) on a pre-rebuild database: %v", err)
}
defer st.Close()
if err := st.SetAppControlPlane(OpsAccountID, "console", true); err != nil {
t.Fatalf("SetAppControlPlane: %v — the column did not survive the rebuild", err)
}
if err := st.SetAppBuildProbe(OpsAccountID, "console", "/healthz"); err != nil {
t.Fatalf("SetAppBuildProbe: %v", err)
}
cp, probe, err := st.AppFlags(OpsAccountID, "console")
if err != nil {
t.Fatalf("AppFlags: %v", err)
}
if !cp || probe != "/healthz" {
t.Fatalf("AppFlags = %v %q, want true \"/healthz\"", cp, probe)
}
}
// TestAppFlagsDefaultOff: every row that existed before this change means
// "not the control plane, no probe" — the additive-default idiom the kind and
// service-principal columns used.
func TestAppFlagsDefaultOff(t *testing.T) {
fx := newConsoleFixture(t)
if _, err := fx.st.CreateAppIn(OpsAccountID, "titogo", "paul@keymail.dev", time.Now()); err != nil {
t.Fatal(err)
}
cp, probe, err := fx.st.AppFlags(OpsAccountID, "titogo")
if err != nil {
t.Fatal(err)
}
if cp || probe != "" {
t.Fatalf("a freshly claimed app = %v %q, want false \"\"", cp, probe)
}
}Check migrate_test.go's existing imports; add database/sql, path/filepath, time if absent, and confirm the sqlite driver import matches what store.go uses (_ "modernc.org/sqlite" or the package's existing alias).
- [ ] Step 2: Run the test to verify it fails
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
go test ./internal/console/ -run 'TestAppFlags' -count=1Expected: FAIL — st.SetAppControlPlane undefined.
- [ ] Step 3: Add the columns to the ALTER block
In internal/console/store.go, append to the additive slice that ends with the accounts … tier statement (~line 226):
// Control-plane apps (console-as-an-app spec §6): the flag REFUSES
// destruction — trash, purge, rename, transfer — and refuses a
// service principal's promote. It deliberately does NOT gate a
// person's promote: `stable`'s fresh-auth 403 has no API-side
// step-up and `carlos deploy` log.Fatalfs on a failed promote AFTER
// the ship has landed (deploy.go:206-208), so gating promotion would
// break `carlos deploy -app console` — the command the flag exists to
// make safe. Deploying the console should be easy; destroying it
// should be hard.
`ALTER TABLE apps ADD COLUMN control_plane INTEGER NOT NULL DEFAULT 0;`,
// The path `carlos deploy` polls for the build the app's own PROCESS
// reports, instead of X-Carlos-Version — which the edge stamps from
// the route row (edge.go:1336) and which therefore lies from the
// moment adoption swaps the symlink until something restarts the
// process. "" keeps today's header behaviour byte for byte.
`ALTER TABLE apps ADD COLUMN build_probe TEXT NOT NULL DEFAULT '';`,- [ ] Step 4: Add the same columns to
migrateAppsKey's rebuild
In internal/console/store.go, migrateAppsKey, extend apps_new and the copy — both, or the rebuild drops them:
`CREATE TABLE apps_new (
app TEXT NOT NULL,
account_id INTEGER NOT NULL DEFAULT 1,
created_by TEXT NOT NULL,
created_at INTEGER NOT NULL,
deleted_at INTEGER NOT NULL DEFAULT 0,
placement TEXT NOT NULL DEFAULT '',
control_plane INTEGER NOT NULL DEFAULT 0,
build_probe TEXT NOT NULL DEFAULT '',
PRIMARY KEY (account_id, app)
);`,
`INSERT INTO apps_new (app, account_id, created_by, created_at, deleted_at, placement, control_plane, build_probe)
SELECT app, account_id, created_by, created_at, deleted_at, placement, control_plane, build_probe FROM apps;`,Extend the existing comment above the INSERT to name the two new columns alongside deleted_at and placement.
- [ ] Step 5: Add the accessors
In internal/console/store.go, after SetAppPlacement:
// AppFlags reads the two per-app deployment facts the control-plane work
// added: whether this app IS the control plane, and the path `carlos deploy`
// should poll for a build the app's own process reports.
//
// One query for both because both doors that read them read them together
// (the ship gate and the deploy watch), and a second round trip to the same
// row buys nothing.
func (s *Store) AppFlags(accountID int64, app string) (controlPlane bool, buildProbe string, err error) {
err = s.db.QueryRow(`SELECT control_plane, build_probe FROM apps WHERE account_id = ? AND app = ?`,
accountID, app).Scan(&controlPlane, &buildProbe)
if errors.Is(err, sql.ErrNoRows) {
// No row is not an error here: every caller has already been through
// apiAccountApp's byte-identical 404, so reaching this with no row
// means a race with a purge, and "not the control plane, no probe" is
// the right reading of an app that no longer exists.
return false, "", nil
}
return controlPlane, buildProbe, err
}
// SetAppControlPlane marks (or unmarks) an app as the control plane. The row
// is guaranteed to exist: both callers reach this straight after a successful
// CreateAppIn or behind apiAccountApp's gate.
func (s *Store) SetAppControlPlane(accountID int64, app string, on bool) error {
_, err := s.db.Exec(`UPDATE apps SET control_plane = ? WHERE account_id = ? AND app = ?`,
on, accountID, app)
return err
}
// SetAppBuildProbe stamps the deploy watch's probe path. "" clears it back to
// header-watching.
func (s *Store) SetAppBuildProbe(accountID int64, app, path string) error {
_, err := s.db.Exec(`UPDATE apps SET build_probe = ? WHERE account_id = ? AND app = ?`,
path, accountID, app)
return err
}Confirm errors and database/sql are already imported in store.go; add if not.
- [ ] Step 6: Run the tests to verify they pass
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
go test ./internal/console/ -run 'TestAppFlags' -count=1 -vExpected: both PASS.
- [ ] Step 7: Full gate
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off sh -c '
go build ./... && go vet ./... && GOOS=darwin go build ./... &&
go test ./... -count=1'
gofmt -l internal/console/store.go internal/console/migrate_test.goExpected: no failures, no gofmt output.
- [ ] Step 8: Commit
git add internal/console/store.go internal/console/migrate_test.go
git commit -m "console: control_plane and build_probe on the app row
Both in the ALTER block AND in migrateAppsKey's apps_new copy list. The
rebuild copies an enumerated column list, so a column in only the first
place is silently dropped on a database that has not been rebuilt yet —
which is a self-hoster jumping versions, not a hypothetical. deleted_at
and placement are already in both for this reason (store.go:280); the new
test builds a pre-rebuild database by hand and proves it."---
Task 2: The #79 owner exemption
Files:
- Modify:
internal/console/server.go:2426(claimApp),internal/console/server.go:2518-2545(handleNewApp),internal/console/apicli.go:394(handleAPICreateApp) - Test:
internal/console/apps_claim_test.go(create)
Interfaces:
- Consumes: nothing from Task 1.
- Produces:
claimAppgains a final parameter —func (s *Server) claimApp(ctx context.Context, acct *Account, name, addr, placement string, ownerClaim bool) (outcome claimOutcome, placed string).
The reasoning (spec §2): apps/<sqid>/<app>/ is namespaced by account, so objects under apps/bab/ can only be ops's. An owner of bab claiming a name under bab's own prefix is adopting their own objects, not inheriting a stranger's. The guard is right for a stranger and wrong for the owner.
- [ ] Step 1: Write the failing tests
Create internal/console/apps_claim_test.go:
package console
import (
"context"
"net/http"
"testing"
"time"
"github.com/carlosframework/platform/internal/svcrole"
)
// seedBucketApp puts an object under an account's own app prefix WITHOUT a
// row, which is exactly what a direct-bucket ship does (direct mode skips the
// claim) and exactly the state that makes #79 unrecoverable: the objects
// block the claim, and no pass can discover a row that does not exist.
func seedBucketApp(t *testing.T, fx *consoleFixture, sqid, app string) {
t.Helper()
key := "apps/" + sqid + "/" + app + "/channels/canary-console.json"
if err := fx.rel.Put(context.Background(), key, []byte(`{}`)); err != nil {
t.Fatalf("seed %s: %v", key, err)
}
}
// TestOwnerClaimsAppWhoseObjectsExist is #79's fix. Before it, an app first
// shipped direct-to-bucket could never be claimed and so was permanently
// invisible to the console, with no recovery but deleting its release history.
func TestOwnerClaimsAppWhoseObjectsExist(t *testing.T) {
fx := newConsoleFixture(t)
seedBucketApp(t, fx, "bab", "console")
acct, err := fx.st.Account(OpsAccountID)
if err != nil {
t.Fatal(err)
}
outcome, _ := fx.srv.claimApp(context.Background(), acct, "console", "paul@keymail.dev", "", true)
if outcome != claimOK {
t.Fatalf("owner claim over existing objects = %v, want claimOK", outcome)
}
has, err := fx.st.HasAppIn(OpsAccountID, "console")
if err != nil || !has {
t.Fatalf("HasAppIn after claim = %v %v, want true", has, err)
}
}
// TestNonOwnerStillRefusedOverExistingObjects: the exemption is OWNERSHIP, not
// membership. Adopting existing release history is an ownership act.
func TestNonOwnerStillRefusedOverExistingObjects(t *testing.T) {
fx := newConsoleFixture(t)
seedBucketApp(t, fx, "bab", "console")
acct, err := fx.st.Account(OpsAccountID)
if err != nil {
t.Fatal(err)
}
outcome, _ := fx.srv.claimApp(context.Background(), acct, "console", "member@keymail.dev", "", false)
if outcome != claimTaken {
t.Fatalf("non-owner claim over existing objects = %v, want claimTaken", outcome)
}
}
// TestLegacyPrefixStillBlocksAnOwner: the exemption covers the ACCOUNT-
// QUALIFIED prefix only. Legacy objects at apps/<app>/ predate account scoping
// and are owed an `accounts migrate -from legacy`; handing the name out would
// put an app in the way of that migration.
func TestLegacyPrefixStillBlocksAnOwner(t *testing.T) {
fx := newConsoleFixture(t)
if err := fx.rel.Put(context.Background(), "apps/oldsite/channels/stable.json", []byte(`{}`)); err != nil {
t.Fatal(err)
}
acct, err := fx.st.Account(OpsAccountID)
if err != nil {
t.Fatal(err)
}
outcome, _ := fx.srv.claimApp(context.Background(), acct, "oldsite", "paul@keymail.dev", "", true)
if outcome != claimTaken {
t.Fatalf("owner claim over a LEGACY prefix = %v, want claimTaken — those objects are owed a migrate", outcome)
}
}
// TestGrantStillBlocksAnOwner: a CLI-granted app in ops stays claim-proof.
// Verified live on flagship 2026-08-23: derp, meep and platform-site hold
// grants; console does not, which is why the exemption is sufficient for it.
func TestGrantStillBlocksAnOwner(t *testing.T) {
fx := newConsoleFixture(t)
if err := fx.st.Grant("someone@keymail.dev", "derp"); err != nil {
t.Fatal(err)
}
acct, err := fx.st.Account(OpsAccountID)
if err != nil {
t.Fatal(err)
}
outcome, _ := fx.srv.claimApp(context.Background(), acct, "derp", "paul@keymail.dev", "", true)
if outcome != claimTaken {
t.Fatalf("owner claim over a GRANTED name = %v, want claimTaken", outcome)
}
}
// TestServicePrincipalNeverClaimsOverObjects: a service has no role at all, so
// ownerClaim must be pinned FALSE for it. An account-scoped PrincipalService
// legitimately reaches claimApp (apicli.go blocks only app-scoped ones), so
// without the pin the exemption would silently hand every CI credential the
// power to adopt release history.
func TestServicePrincipalNeverClaimsOverObjects(t *testing.T) {
fx := newConsoleFixture(t)
seedBucketApp(t, fx, "bab", "console")
svc := serviceToken(t, fx.st, OpsAccountID, "hub", "", svcrole.Publish)
status, out := postJSON(t, fx.ts.URL+"/api/cli/apps/bab", map[string]string{"name": "console"}, svc)
if status != http.StatusConflict {
t.Fatalf("service claim over existing objects = %d %v, want 409 (taken)", status, out)
}
has, err := fx.st.HasAppIn(OpsAccountID, "console")
if err != nil || has {
t.Fatalf("HasAppIn after a service claim = %v %v, want false — a service must never adopt release history", has, err)
}
}Two things to check against the codebase before running, and fix in the test if they differ: the release-store Put helper's name on fx.rel (grep fx.rel. in existing tests), and handleAPICreateApp's success/failure status codes and request path (read apicli.go around line 394 and the route registration in authcli.go). Use whatever the existing create tests use.
- [ ] Step 2: Run to verify they fail
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
go test ./internal/console/ -run 'Claim|ClaimsApp|Refused|Blocks|ServicePrincipalNever' -count=1Expected: FAIL — too many arguments in call to fx.srv.claimApp.
- [ ] Step 3: Add the parameter and the exemption
In internal/console/server.go, change the signature and the bucket guard. Extend claimApp's doc comment with the new paragraph:
// ownerClaim exempts the ACCOUNT-QUALIFIED prefix check below, and nothing
// else. The guard exists so a stranger cannot inherit objects somebody else
// shipped — but apps/<sqid>/<app>/ is namespaced BY ACCOUNT, so objects under
// a given sqid can only be that account's. An OWNER of that account claiming
// a name under its own prefix is adopting their own release history, which is
// the whole of issue #79: an app first shipped direct-to-bucket (direct mode
// skips the claim) could otherwise never be given a row, and so stayed
// invisible to the console forever with no recovery but deleting its history.
//
// It is OWNER, not member: adopting existing release history is an ownership
// act. It is FALSE for every service principal — a service has no role at
// all, and an account-scoped one legitimately reaches this function, so
// without an explicit pin the exemption would hand every CI credential the
// same power.
func (s *Server) claimApp(ctx context.Context, acct *Account, name, addr, placement string, ownerClaim bool) (outcome claimOutcome, placed string) {Then the prefix loop. The legacy prefix must keep refusing, so the exemption applies per-prefix, not to the loop:
for _, prefix := range prefixes {
keys, err := s.cfg.Releases.List(ctx, prefix)
if err != nil {
return claimStoreFailed, ""
}
if len(keys) == 0 {
continue
}
// The owner exemption covers the account's OWN prefix only. The
// legacy unqualified one (ops only) still refuses whoever is asking:
// those objects are owed an `accounts migrate -from legacy`, and a
// name handed out over them puts an app in that migration's way.
if ownerClaim && prefix != release.LegacyAppPrefix(name) {
continue
}
return claimTaken, ""
}- [ ] Step 4: Update both call sites
internal/console/apicli.go:394 — a service principal is pinned false:
// ownerClaim: a PERSON who owns this account may adopt objects already
// under its own prefix (#79). A service principal never may — it holds no
// account role, and handing a CI credential the power to adopt release
// history is exactly the widening svcrole exists to prevent.
ownerClaim := p.Kind == PrincipalUser && acct.Role == RoleOwner
outcome, placed := s.claimApp(r.Context(), acct, name, addr, place, ownerClaim)internal/console/server.go:2542 — the browser door already has the role and was discarding it:
addr, raw, acct, role, ok := s.account(w, r) outcome, _ := s.claimApp(r.Context(), acct, name, addr, "", role == RoleOwner)And correct handleNewApp's doc comment, whose stated policy this changes:
// handleNewApp is the browser door onto claimApp: the account page's New app
// form. Any member may claim a FREE name — creation grants nothing promotion
// doesn't already gate — and unlike the CLI door it carries no rate limit: a
// session behind CSRF is not the stolen-credential shape the API budget
// exists for, and this form is the escape hatch when that budget is honestly
// spent.
//
// An OWNER may additionally claim a name whose objects already exist under
// this account's own prefix (#79, claimApp's ownerClaim). A plain member
// cannot: adopting existing release history is an ownership act.- [ ] Step 5: Run the tests to verify they pass
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
go test ./internal/console/ -run 'Claim|ClaimsApp|Refused|Blocks|ServicePrincipalNever' -count=1 -vExpected: all five PASS.
- [ ] Step 6: Full gate, then commit
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off sh -c '
go build ./... && go vet ./... && GOOS=darwin go build ./... && go test ./... -count=1'
gofmt -l internal/console/server.go internal/console/apicli.go internal/console/apps_claim_test.go
git add internal/console/server.go internal/console/apicli.go internal/console/apps_claim_test.go
git commit -m "console: an owner may claim a name whose own-account objects exist (#79)
apps/<sqid>/<app>/ is namespaced BY ACCOUNT, so objects under a sqid can
only be that account's — an owner claiming there is adopting their own
release history, not inheriting a stranger's. That is the whole of #79: an
app first shipped direct-to-bucket was otherwise unclaimable forever, with
no recovery but deleting its history.
Owner, not member. Pinned FALSE for every service principal: an
account-scoped one reaches claimApp and holds no account role, so without
the pin every CI credential could adopt release history. The legacy
unqualified prefix still refuses everyone — those objects are owed a
migrate and the name must stay out of its way."---
Task 3: Set the flags at claim time
Files:
- Modify:
internal/console/apicli.go(handleAPICreateApprequest struct and post-claim stamping) - Modify:
apps.go:379(runAppsCreate) - Test:
internal/console/apps_claim_test.go,apps_test.go
Interfaces:
- Consumes:
Store.SetAppControlPlane,Store.SetAppBuildProbe(Task 1);claimApp(..., ownerClaim)(Task 2). - Produces:
carlos apps createaccepts-control-planeand-build-probe <path>; the API accepts"control_plane": booland"build_probe": string.
Why at claim time, not a settings page: the runbook claims ops/console with the flag already set (spec §10 step 10). A row that exists without the flag is a window in which any ops member or Publish service principal can roll the production console. One step, no window. A settings-page toggle is deferred.
- [ ] Step 1: Write the failing tests
Add to internal/console/apps_claim_test.go:
// TestCreateAppStampsControlPlaneAndProbe: the flags are set in the SAME call
// that claims the name. Claiming first and flagging second would leave a
// window in which the production console is an ordinary app any ops member
// could roll.
func TestCreateAppStampsControlPlaneAndProbe(t *testing.T) {
fx := newConsoleFixture(t)
seedBucketApp(t, fx, "bab", "console")
token := cliToken(t, fx, "paul@keymail.dev", "code-create-cp")
status, out := postJSON(t, fx.ts.URL+"/api/cli/apps/bab", map[string]any{
"name": "console", "control_plane": true, "build_probe": "/healthz",
}, token)
if status != http.StatusCreated {
t.Fatalf("create = %d %v, want 201", status, out)
}
cp, probe, err := fx.st.AppFlags(OpsAccountID, "console")
if err != nil {
t.Fatal(err)
}
if !cp || probe != "/healthz" {
t.Fatalf("flags after create = %v %q, want true \"/healthz\"", cp, probe)
}
}
// TestOnlyAnOwnerSetsControlPlane: the flag refuses destruction, so setting it
// is an ownership act — a member who could set it could also make an app
// undeletable by its owner.
func TestOnlyAnOwnerSetsControlPlane(t *testing.T) {
fx := newConsoleFixture(t)
if err := fx.st.AddMember(OpsAccountID, "member@keymail.dev", RoleMember, time.Now()); err != nil {
t.Fatal(err)
}
token := cliToken(t, fx, "member@keymail.dev", "code-create-cp-member")
status, out := postJSON(t, fx.ts.URL+"/api/cli/apps/bab", map[string]any{
"name": "notyours", "control_plane": true,
}, token)
if status != http.StatusForbidden {
t.Fatalf("member setting control_plane = %d %v, want 403", status, out)
}
has, err := fx.st.HasAppIn(OpsAccountID, "notyours")
if err != nil || has {
t.Fatalf("the app was claimed anyway (%v) — a refused flag must refuse the WHOLE create, like -place does", has)
}
}
// TestBuildProbeMustBeAnAbsolutePath: the probe is a URL path the deploy watch
// dials. Anything else is a typo that would otherwise surface as a deploy that
// times out for reasons nobody can see.
func TestBuildProbeMustBeAnAbsolutePath(t *testing.T) {
fx := newConsoleFixture(t)
token := cliToken(t, fx, "paul@keymail.dev", "code-create-probe-bad")
for _, bad := range []string{"healthz", "http://x/healthz", "/healthz?x=1", "/health z"} {
status, out := postJSON(t, fx.ts.URL+"/api/cli/apps/bab", map[string]any{
"name": "probe" + bad[:1], "build_probe": bad,
}, token)
if status != http.StatusBadRequest {
t.Fatalf("build_probe %q = %d %v, want 400", bad, status, out)
}
}
}Confirm RoleMember's actual identifier by grepping RoleOwner in internal/console; use whatever the member constant is really called.
- [ ] Step 2: Run to verify they fail
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
go test ./internal/console/ -run 'CreateAppStamps|OnlyAnOwnerSets|BuildProbeMust' -count=1Expected: FAIL — flags ignored, 201 where 403/400 expected.
- [ ] Step 3: Extend the API door
In internal/console/apicli.go, handleAPICreateApp, extend the request struct and validate before claimApp — the create limiter charges attempts, so a shape the server is certain to refuse must not cost a claim:
var req struct {
Name string `json:"name"`
Place string `json:"place"`
ControlPlane bool `json:"control_plane"`
BuildProbe string `json:"build_probe"`
} // Validated BEFORE claimApp, like place: a refused flag refuses the WHOLE
// create rather than leaving a claimed name with the wrong shape, and a
// shape the server is certain to refuse costs no claim attempt.
if req.ControlPlane && acct.Role != RoleOwner {
jsonError(w, http.StatusForbidden,
"only an owner can mark an app as the control plane")
return
}
probe := strings.TrimSpace(req.BuildProbe)
if probe != "" && !validBuildProbe(probe) {
jsonError(w, http.StatusBadRequest,
`build_probe must be an absolute URL path with no query, e.g. "/healthz"`)
return
}Add the validator near normalizeAppName in server.go:
// validBuildProbe bounds the deploy watch's probe path to what it actually
// dials: an absolute path, no query, no spaces. The watch appends its own
// cache-busting query parameter (deploy.go), so a probe carrying one would
// produce a URL with two — and a probe that is a full URL would silently
// point the watch at another host entirely, which is the one mistake here
// that could report a DIFFERENT deployment's build as this one's.
func validBuildProbe(p string) bool {
if !strings.HasPrefix(p, "/") || strings.ContainsAny(p, "?# \t") {
return false
}
return len(p) <= 256
}Then stamp after a successful claim, beside the existing placement stamp:
if req.ControlPlane {
if err := s.store.SetAppControlPlane(acct.ID, name, true); err != nil {
log.Printf("console: claim %s/%s: set control_plane: %v", acct.Sqid, name, err)
}
}
if probe != "" {
if err := s.store.SetAppBuildProbe(acct.ID, name, probe); err != nil {
log.Printf("console: claim %s/%s: set build_probe: %v", acct.Sqid, name, err)
}
}Echo both in the response body alongside placement, for the same reason placed is echoed: a caller that cannot see what was actually stored cannot tell "claimed and flagged" from "claimed, flag silently didn't".
- [ ] Step 4: Extend the CLI
In apps.go, runAppsCreate:
controlPlane := fs.Bool("control-plane", false,
"mark the app as the control plane: it refuses trash, purge, rename and transfer, "+
"and refuses a service credential's promote (owner only)")
buildProbe := fs.String("build-probe", "",
`path whose response body carries the running build, e.g. "/healthz" — `+
"`carlos deploy` polls it instead of the X-Carlos-Version header, which the edge "+
"stamps from the route row and which therefore lies until the process restarts")Extend the usage string with [-control-plane] [-build-probe /path], send both fields in the request body, and print what came back in the echo so the operator sees the stored truth rather than what they asked for.
- [ ] Step 5: Run the tests to verify they pass
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
go test ./internal/console/ ./ -run 'CreateAppStamps|OnlyAnOwnerSets|BuildProbeMust|AppsCreate' -count=1 -vExpected: PASS.
- [ ] Step 6: Full gate, then commit
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off sh -c '
go build ./... && go vet ./... && GOOS=darwin go build ./... && go test ./... -count=1'
gofmt -l internal/console/apicli.go internal/console/server.go apps.go internal/console/apps_claim_test.go
git add -A
git commit -m "apps create: -control-plane and -build-probe, stamped with the claim
Set in the SAME call that claims the name. Claiming first and flagging
second leaves a window in which the production console is an ordinary app
any ops member — or any Publish service credential — could roll.
Both validated before claimApp, like -place: a refused flag refuses the
whole create rather than leaving a claimed name with the wrong shape, and
costs no attempt against the per-day create budget. Both echoed in the
response, so a caller can tell 'claimed and flagged' from 'claimed, flag
silently didn't'."---
Task 4: The flag refuses destruction, and refuses a service's promote
Files:
- Modify:
internal/console/ownership.go:293(handlePurgeApp),internal/console/appsettings.go(handleAppRename,handleAppDelete,handleAppMoveAccount) - Modify:
internal/console/apicli.go(handleAPITrashApp,handleAPIPromote,handleAPIRollback) - Test:
internal/console/controlplane_test.go(create)
Interfaces:
- Consumes:
Store.AppFlags(Task 1). - Produces:
func (s *Server) refuseControlPlane(w http.ResponseWriter, accountID int64, app, verb string) bool— returns true when it wrote a refusal and the caller must stop.
The inversion (spec §6): an earlier draft gated promotion behind fresh auth. That would have made carlos deploy -app console ship and then die every time, because stable's 403 has no API-side step-up and deploy.go:206-208 log.Fatalfs after the ship lands. Deploying the console should be easy; destroying it should be hard — and handlePurgeApp can currently delete apps/bab/console/**, the release history the fleet is running, behind an ordinary confirmation.
- [ ] Step 1: Write the failing tests
Create internal/console/controlplane_test.go:
package console
import (
"net/http"
"testing"
"time"
"github.com/carlosframework/platform/internal/svcrole"
)
func controlPlaneFixture(t *testing.T, label string) (*consoleFixture, string) {
t.Helper()
fx := newConsoleFixture(t)
if _, err := fx.st.CreateAppIn(OpsAccountID, "console", "paul@keymail.dev", time.Now()); err != nil {
t.Fatal(err)
}
if err := fx.st.SetAppControlPlane(OpsAccountID, "console", true); err != nil {
t.Fatal(err)
}
return fx, cliToken(t, fx, "paul@keymail.dev", label)
}
// TestControlPlaneAppRefusesTrash: the console's release history is not
// something to delete behind a confirmation dialog. Refused outright, not
// "with fresh auth" — a gesture is not the guard here, the flag is.
func TestControlPlaneAppRefusesTrash(t *testing.T) {
fx, token := controlPlaneFixture(t, "code-cp-trash")
status, out := postJSON(t, fx.ts.URL+"/api/cli/apps/bab/console/delete", map[string]string{}, token)
if status != http.StatusForbidden {
t.Fatalf("trash a control-plane app = %d %v, want 403", status, out)
}
if msg := str(t, out, "error"); !strings.Contains(msg, "control plane") {
t.Fatalf("refusal %q does not name the flag the operator has to clear", msg)
}
trashed, err := fx.st.HasAppIn(OpsAccountID, "console")
if err != nil || !trashed {
t.Fatalf("app row after a refused trash: HasAppIn = %v %v, want true (untouched)", trashed, err)
}
}
// TestControlPlaneAppAllowsAPersonToPromote is the other half, and the one
// that matters most: gating promotion is what an earlier draft got wrong.
// `stable`'s fresh-auth 403 has no API-side step-up and carlos deploy
// log.Fatalfs on a failed promote AFTER the ship has landed, so a promote gate
// would break the very command this work exists to create.
func TestControlPlaneAppAllowsAPersonToPromote(t *testing.T) {
fx, token := controlPlaneFixture(t, "code-cp-promote")
status, out := postShip(t, fx.ts.URL+"/api/cli/apps/bab/console/releases", token,
[]shipPart{metaPart(t, "cp00001", "binary"), {name: "linux-arm64", body: []byte("x")}})
if status != http.StatusCreated {
t.Fatalf("ship = %d %v", status, out)
}
status, out = postJSON(t, fx.ts.URL+"/api/cli/apps/bab/console/promote",
map[string]string{"version": "cp00001", "channel": "canary/console"}, token)
if status != http.StatusOK {
t.Fatalf("a PERSON promoting a control-plane app = %d %v, want 200 — gating this breaks carlos deploy", status, out)
}
}
// TestControlPlaneAppRefusesAServicePromote: a person with ops membership may
// roll the console; a CI credential may not. This is what the flag buys on the
// promote path instead of a fresh-auth gate.
func TestControlPlaneAppRefusesAServicePromote(t *testing.T) {
fx, token := controlPlaneFixture(t, "code-cp-svc")
status, out := postShip(t, fx.ts.URL+"/api/cli/apps/bab/console/releases", token,
[]shipPart{metaPart(t, "cp00002", "binary"), {name: "linux-arm64", body: []byte("x")}})
if status != http.StatusCreated {
t.Fatalf("ship = %d %v", status, out)
}
svc := serviceToken(t, fx.st, OpsAccountID, "ci", "console", svcrole.Publish)
status, out = postJSON(t, fx.ts.URL+"/api/cli/apps/bab/console/promote",
map[string]string{"version": "cp00002", "channel": "canary/console"}, svc)
if status != http.StatusForbidden {
t.Fatalf("a SERVICE promoting a control-plane app = %d %v, want 403", status, out)
}
}
// TestOrdinaryAppUnaffected: every refusal above is scoped to the flag.
func TestOrdinaryAppUnaffected(t *testing.T) {
fx := newConsoleFixture(t)
if _, err := fx.st.CreateAppIn(OpsAccountID, "titogo", "paul@keymail.dev", time.Now()); err != nil {
t.Fatal(err)
}
token := cliToken(t, fx, "paul@keymail.dev", "code-cp-ordinary")
status, out := postJSON(t, fx.ts.URL+"/api/cli/apps/bab/titogo/delete", map[string]string{}, token)
if status == http.StatusForbidden {
t.Fatalf("trash an ORDINARY app = %d %v, want it to work as before", status, out)
}
}
// TestControlPlaneRefusalIsNotAMembershipOracle: a non-member must meet the
// byte-identical 404 in apiAccountApp before the flag is ever consulted, or
// the refusal becomes a way to learn which accounts hold a control-plane app.
func TestControlPlaneRefusalIsNotAMembershipOracle(t *testing.T) {
fx, _ := controlPlaneFixture(t, "code-cp-oracle")
stranger := cliToken(t, fx, "eve@keymail.dev", "code-cp-stranger")
status, out := postJSON(t, fx.ts.URL+"/api/cli/apps/bab/console/delete", map[string]string{}, stranger)
if status != http.StatusNotFound {
t.Fatalf("a non-member = %d %v, want the byte-identical 404", status, out)
}
}Add strings to the imports. Check handleAPIPromote's real request field names (read apicli.go around the promote handler) and use them; the ones above are a guess and must be corrected to match.
- [ ] Step 2: Run to verify they fail
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
go test ./internal/console/ -run 'ControlPlane|OrdinaryAppUnaffected' -count=1Expected: FAIL — trash succeeds, service promote succeeds.
- [ ] Step 3: Add the shared refusal
In internal/console/controlplane.go (create):
// Control-plane apps: the flag that says "this app IS the platform".
//
// It gates DESTRUCTION, not promotion, and the asymmetry is the whole design.
// Deploying the console should be as ordinary as deploying any app — that is
// what this slice exists to make true — so a promote gate would defeat it:
// `stable`'s fresh-auth 403 has no API-side step-up, and `carlos deploy`
// log.Fatalfs on a failed promote AFTER the ship has already landed
// (deploy.go:206-208), so any gate here would leave every console deploy
// half-done. Destroying the console is the act that deserves a wall, and it
// had none: handlePurgeApp will delete apps/<sqid>/console/** — the release
// history the fleet is currently running — behind an owner check, a typed
// name and a passkey, none of which knows what this app is.
//
// Refused OUTRIGHT rather than behind fresh auth. A gesture proves intent,
// and intent is not the thing in doubt: an operator who means to delete the
// console can clear the flag first, which is a separate, deliberate act with
// its own record.
package console
import (
"log"
"net/http"
)
// refuseControlPlane writes a refusal and returns true when app is the control
// plane. Callers MUST be past their own membership gate first — the
// byte-identical 404 protects account membership, and consulting the flag
// before it would turn this refusal into an oracle for which accounts hold a
// control-plane app.
//
// A lookup failure refuses too. The alternative — treating an unreadable row
// as "not the control plane" — makes a database blip the way past the guard.
func (s *Server) refuseControlPlane(w http.ResponseWriter, accountID int64, app, verb string) bool {
cp, _, err := s.store.AppFlags(accountID, app)
if err != nil {
log.Printf("console: control-plane check %s/%d: %v", app, accountID, err)
jsonError(w, http.StatusServiceUnavailable, "could not check this app's control-plane status — nothing was changed")
return true
}
if !cp {
return false
}
jsonError(w, http.StatusForbidden, app+" is marked as the control plane, so it cannot be "+verb+
" — clear the control-plane flag first if you really mean to")
return true
}For the browser doors, which render pages rather than JSON, add a sibling that sets a flash and redirects, following whatever pattern handleAppRename already uses for a refusal.
- [ ] Step 4: Wire the four destructive doors and the promote pair
Each call goes after the existing membership/owner gate and before any write or any fresh-auth prompt — refusing before a passkey round trip is both faster and clearer than asking for a gesture that cannot help:
internal/console/apicli.go,handleAPITrashApp:if s.refuseControlPlane(w, acct.ID, app, "trashed") { return }internal/console/appsettings.go,handleAppDelete: refuse with"trashed"internal/console/appsettings.go,handleAppRename: refuse with"renamed"internal/console/appsettings.go,handleAppMoveAccount: refuse with"moved to another account"internal/console/ownership.go,handlePurgeApp: refuse with"deleted"— this door takes the app from the trash form, so read the name it resolved
For the promote pair, in handleAPIPromote and handleAPIRollback, after the existing gates and before PromoteOnLadder:
// A person with ops membership may roll the control plane; a service
// credential may not. This is the flag's promote-side job — the whole of
// it. Gating a PERSON here would break `carlos deploy -app console`, the
// command this exists to make ordinary.
if p.Kind != PrincipalUser {
cp, _, err := s.store.AppFlags(acct.ID, app)
if err != nil {
jsonError(w, http.StatusServiceUnavailable, "could not check this app's control-plane status — nothing was changed")
return
}
if cp {
jsonError(w, http.StatusForbidden,
app+" is the control plane — a service credential cannot promote it; a person must")
return
}
}- [ ] Step 5: Run the tests to verify they pass
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
go test ./internal/console/ -run 'ControlPlane|OrdinaryAppUnaffected' -count=1 -vExpected: all five PASS.
- [ ] Step 6: Full gate, then commit
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off sh -c '
go build ./... && go vet ./... && GOOS=darwin go build ./... && go test ./... -count=1'
gofmt -l internal/console/controlplane.go internal/console/apicli.go internal/console/appsettings.go internal/console/ownership.go internal/console/controlplane_test.go
git add -A
git commit -m "console: the control-plane flag refuses destruction, not promotion
Trash, purge, rename and transfer are refused outright for a flagged app;
a service credential cannot promote one; a PERSON promotes it with
ordinary ops membership.
The asymmetry is the design. Gating promotion would have broken
carlos deploy -app console — stable's fresh-auth 403 has no API-side
step-up and deploy log.Fatalfs on a failed promote after the ship has
landed. Meanwhile purge could delete the release history the fleet is
running, behind a confirmation that had no idea what the app was."---
Task 5: /healthz reports the build
Files:
- Modify:
internal/console/server.go:794-796 - Test:
internal/console/server_test.go:212(extend the existing assertion)
Interfaces:
- Consumes: nothing.
- Produces:
GET /healthz→ok <build>\n, from the console process.
Existing consumers checked (CLAUDE.md's parse-the-shape rule): internal/vet and the edge health probe assert status only, so appending is additive-safe. Re-verify with grep -rn '/healthz' --include=*.go . before changing it, and if any consumer compares the body exactly, fix that consumer in this task.
- [ ] Step 1: Write the failing test
Add to internal/console/server_test.go:
// TestHealthzCarriesTheBuild: the console's own build, from the PROCESS.
//
// This is the deploy watch's exit criterion, and it exists because both cheap
// answers lie. X-Carlos-Version is stamped by the EDGE from the route row
// (edge.go:1336), so adoption sets it while the old process keeps serving;
// `systemctl is-active` says "active" for a process started from the previous
// version just as happily (tito, 2026-08-09, seconds after adopt and restart
// both reported success). Only the running process can answer this.
func TestHealthzCarriesTheBuild(t *testing.T) {
fx := newConsoleFixture(t)
status, _, body := get(t, nil, fx.ts.URL+"/healthz")
if status != 200 {
t.Fatalf("/healthz = %d, want 200", status)
}
if !strings.Contains(body, "ok") {
t.Fatalf("/healthz body = %q, want it to still start with ok (vet and the edge probe read this)", body)
}
fields := strings.Fields(body)
if len(fields) != 2 || fields[1] != version.Build {
t.Fatalf("/healthz body = %q, want exactly \"ok %s\" — the watch matches a whitespace-delimited token", body, version.Build)
}
}Match get's real signature from the neighbouring test at server_test.go:212, and import internal/version and strings.
- [ ] Step 2: Run to verify it fails
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
go test ./internal/console/ -run TestHealthzCarriesTheBuild -count=1Expected: FAIL — body is "ok\n", two fields wanted.
- [ ] Step 3: Implement
internal/console/server.go:
// "ok <build>" — the build comes from THIS PROCESS (ldflags, at link
// time), which is the whole point: X-Carlos-Version is stamped by the
// edge from the route row, so it reports the version adoption INTENDED
// from the moment the symlink moves, whether or not anything restarted.
// `carlos deploy` polls this instead for any app declaring a build probe.
//
// "ok" stays the first token: internal/vet and the edge health probe read
// this endpoint and check status, not body, but keeping the prefix means
// a human or a script eyeballing it sees what it always saw.
s.mux.HandleFunc("GET /healthz", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "ok", version.Build)
})Import github.com/carlosframework/platform/internal/version in server.go if not present.
- [ ] Step 4: Run the tests to verify they pass
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
go test ./internal/console/ -run 'Healthz|TestServer' -count=1Expected: PASS, including the pre-existing /healthz assertion at line 212.
- [ ] Step 5: Full gate, then commit
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off sh -c '
go build ./... && go vet ./... && GOOS=darwin go build ./... && go test ./... -count=1'
gofmt -l internal/console/server.go internal/console/server_test.go
git add internal/console/server.go internal/console/server_test.go
git commit -m "console: /healthz reports the build the PROCESS is running
X-Carlos-Version is stamped by the edge from the route row, so it reports
what adoption intended from the moment the symlink moves — whether or not
anything restarted. systemctl is-active is no better. Only the process can
answer this honestly, and now it does.
'ok' stays the first token: vet and the edge probe read status not body,
but a human eyeballing it should see what they always saw."---
Task 6: carlos deploy polls the probe instead of the header
Files:
- Modify:
deploy.go(watchTarget,watchVersion, and the call site that builds targets) - Modify:
internal/console/apicli.go(servebuild_probeon an existing per-app GET the deploy path already calls) - Test:
deploy_test.go
Interfaces:
- Consumes:
Store.AppFlags(Task 1);/healthzshape (Task 5). - Produces:
watchTargetgainsProbe string; when non-empty the watch pollsOrigin + Probeand matches the body, otherwise it polls/and matchesX-Carlos-Versionexactly as today.
Two lies this must not reintroduce (spec §3b): the ldflags trap (a binary built without -X …version.Build reports dev forever, so a healthy deploy reports failure) — closed by naming the body in the timeout, and at the door in Task 8; and the edge cache — watchVersion already busts every poll because the edge caches a proxied 2xx for up to five minutes, and the probe leg must inherit that.
- [ ] Step 1: Write the failing test
Add to deploy_test.go:
// TestWatchProbeMatchesAWholeToken: the probe body must carry the build as a
// whitespace-delimited token. Substring matching would let abc123 satisfy a
// watch for abc1234 — and short git shas make that a real collision, not a
// theoretical one.
func TestWatchProbeMatchesAWholeToken(t *testing.T) {
var body atomic.Value
body.Store("ok abc123")
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/healthz" {
t.Errorf("watch dialled %q, want /healthz", r.URL.Path)
}
if r.URL.Query().Get("carlos-deploy-probe") == "" {
t.Error("probe poll carried no cache-buster — the edge caches a proxied 2xx for up to 5 minutes")
}
fmt.Fprintln(w, body.Load().(string))
}))
defer srv.Close()
old := watchOrigin
watchOrigin = func(host string) string { return srv.URL }
defer func() { watchOrigin = old }()
targets := []watchTarget{{Host: "console.example", Origin: srv.URL, Probe: "/healthz"}}
err := watchVersion(io.Discard, targets, "abc1234", 500*time.Millisecond, 50*time.Millisecond, nil)
if err == nil {
t.Fatal("watch for abc1234 succeeded against a body of \"ok abc123\" — substring match, not token match")
}
body.Store("ok abc1234")
if err := watchVersion(io.Discard, targets, "abc1234", 2*time.Second, 50*time.Millisecond, nil); err != nil {
t.Fatalf("watch against the matching build = %v, want success", err)
}
}
// TestWatchProbeTimeoutNamesTheBody: a binary built without the version
// ldflag reports "dev" forever, so the deploy fails while the roll was
// perfectly healthy. The report has to make that self-diagnosing rather than
// saying "timed out".
func TestWatchProbeTimeoutNamesTheBody(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "ok dev")
}))
defer srv.Close()
var buf bytes.Buffer
targets := []watchTarget{{Host: "console.example", Origin: srv.URL, Probe: "/healthz"}}
err := watchVersion(&buf, targets, "abc1234", 300*time.Millisecond, 50*time.Millisecond, nil)
if err == nil {
t.Fatal("want a timeout")
}
out := buf.String() + err.Error()
if !strings.Contains(out, "dev") {
t.Fatalf("timeout report %q does not name what the probe actually said — the ldflags trap stays invisible", out)
}
}
// TestWatchWithoutAProbeStillReadsTheHeader: an app that declares no probe
// keeps today's behaviour byte for byte.
func TestWatchWithoutAProbeStillReadsTheHeader(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("X-Carlos-Version", "abc1234")
w.WriteHeader(200)
}))
defer srv.Close()
targets := []watchTarget{{Host: "app.example", Origin: srv.URL}}
if err := watchVersion(io.Discard, targets, "abc1234", 2*time.Second, 50*time.Millisecond, nil); err != nil {
t.Fatalf("header watch = %v, want success", err)
}
}Read watchTarget's real fields and watchVersion's signature in deploy.go first and adjust; add sync/atomic, bytes, strings to imports as needed.
- [ ] Step 2: Run to verify they fail
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
go test . -run 'TestWatch(Probe|Without)' -count=1Expected: FAIL — unknown field Probe.
- [ ] Step 3: Implement the probe leg
Add Probe string to watchTarget with a comment naming why it exists. In watchVersion's per-target goroutine, branch on it, keeping the cache-buster on both paths:
n := attempt.Add(1)
path := "/"
if t.Probe != "" {
path = t.Probe
}
url := fmt.Sprintf("%s%s?carlos-deploy-probe=%d", t.Origin, path, n)
resp, err := client.Get(url)
if err != nil {
results[i] = result{diag: "unreachable: " + err.Error()}
return
}
if t.Probe != "" {
// The app's own process answers here, so the body IS the
// truth — no status/header disagreement to reconcile the
// way the header path has to. Read a bounded amount: this
// is a health endpoint, and a target that answers with a
// stream is misconfigured, not something to buffer.
b, rerr := io.ReadAll(io.LimitReader(resp.Body, 4096))
resp.Body.Close()
if rerr != nil {
results[i] = result{diag: "probe unreadable: " + rerr.Error()}
return
}
got := strings.TrimSpace(string(b))
if resp.StatusCode/100 != 2 {
results[i] = result{diag: fmt.Sprintf("%s answered HTTP %d", path, resp.StatusCode)}
return
}
// Whole-token, never substring: abc123 must not satisfy a
// watch for abc1234, and short git shas collide like that
// in practice.
for _, f := range strings.Fields(got) {
if f == ver {
results[i] = result{live: true}
return
}
}
// Naming the body is what makes the ldflags trap
// self-diagnosing: a binary built without
// -X …version.Build reports "dev" forever, and "timed
// out" would send the operator hunting the wrong thing.
results[i] = result{diag: fmt.Sprintf("%s reports %q, waiting for %s", path, got, ver)}
return
}
io.CopyN(io.Discard, resp.Body, 4096)
resp.Body.Close()
got := resp.Header.Get("X-Carlos-Version")
// ... existing header switch unchanged ...- [ ] Step 4: Serve and consume the probe
Server side: include "build_probe" in the JSON of the per-app GET the deploy path already calls. Read deploy.go to find which call that is (the instance list, or channels) and extend that handler and its client struct rather than adding a door. State in a comment that the probe is app-level and read from the app ROW — not from provision.Record, whose handleAPIInstanceUpdate is a read-modify-write that would silently drop an unknown field.
Client side: set Probe on every watchTarget the deploy builds from that response.
- [ ] Step 5: Run the tests
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
go test . ./internal/console/ -run 'Watch|Deploy' -count=1 -vExpected: PASS, existing deploy tests included.
- [ ] Step 6: Full gate, then commit
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off sh -c '
go build ./... && go vet ./... && GOOS=darwin go build ./... && go test ./... -count=1'
gofmt -l deploy.go deploy_test.go internal/console/apicli.go
git add -A
git commit -m "carlos deploy: watch the app's own build report, not the edge's header
An app declaring a build probe is watched by polling that path and
matching the build as a WHOLE TOKEN in the body — substring matching would
let abc123 satisfy a watch for abc1234, which short git shas make a real
collision. No probe declared keeps the header path byte for byte.
The probe leg inherits the cache-buster: the edge caches a proxied 2xx for
up to five minutes, longer than the watch's own timeout. And the timeout
NAMES what the probe said, so a binary built without -X version.Build
diagnoses itself instead of looking like a failed deploy."---
Task 7: Adoption starts the unit again after a rolled-back restart
Files:
- Modify:
internal/release/adopt.go:781-815(the restart-failure block) - Modify:
internal/host/agent.go(makeRestartRoute, to expose a start) - Test:
internal/release/adopt_test.go
Interfaces:
- Consumes: nothing.
- Produces: the restart callback type gains a start capability. Prefer the smallest change that keeps
Pass's signature stable — e.g. aPassOptioncarryingstartUnit func(host string) error, so callers that pass no start keep today's behaviour.
This is a tenant fix and the most important defect in the slice (spec §4b). Verified:
pass 1: PointLive(new) → try-restart FAILS → poison, PointLive(old), no stamp
unit: failed. console: DOWN. header: honest (old).
pass 2: PointLive(new) → try-restart on a NOT-RUNNING unit → NO-OP, exit 0
→ SetVersions(new)
unit: still failed. console: STILL DOWN. header: claims the new build.try-restart being a no-op on a stopped unit is deliberate — it is what lets a deliberately-stopped console stay stopped (makeRestartRoute's comment). The bug is that the rollback leaves the unit stopped in the first place.
- [ ] Step 1: Write the failing test
Add to internal/release/adopt_test.go, following the file's existing fixture style for a registry + store + route:
// TestFailedRestartLeavesTheUnitRUNNINGOnTheOldVersion.
//
// The rollback repointed the symlink and stopped there, which left the box
// with a correct symlink and no process. Worse, try-restart is a NO-OP on a
// unit that is not running — so the NEXT pass exited 0, stamped SetVersions,
// and produced exactly the lying header the whole mechanism exists to
// prevent, over a service that was not running at all.
func TestFailedRestartLeavesTheUnitRUNNINGOnTheOldVersion(t *testing.T) {
// Build the fixture the way the neighbouring adopt tests do: a registry
// with one route on a channel, a store holding two versions, "old"
// already live and recorded.
fx := newAdoptFixture(t)
fx.setLive("console.example", "old")
fx.promote("console.example", "new")
var started []string
restart := func(host string) error { return errors.New("Job for console.service failed") }
start := func(host string) error { started = append(started, host); return nil }
if err := Pass(context.Background(), fx.reg, fx.store, fx.pub, fx.optDir, restart,
WithStartUnit(start)); err != nil {
t.Fatalf("pass: %v", err)
}
if got := fx.liveVersion("console.example"); got != "old" {
t.Fatalf("symlink = %q, want it rolled back to old", got)
}
if len(started) != 1 || started[0] != "console.example" {
t.Fatalf("started = %v, want the unit started once on the restored version — a rollback that leaves the service down is not a rollback", started)
}
if got := fx.recordedVersion("console.example"); got == "new" {
t.Fatal("SetVersions stamped the failed build")
}
}
// TestSecondPassOverAStillBrokenBuildDoesNotStamp: the second-order failure.
// With the unit running again after pass 1, pass 2's try-restart is a real
// restart that fails again — so it rolls back again and still does not stamp.
func TestSecondPassOverAStillBrokenBuildDoesNotStamp(t *testing.T) {
fx := newAdoptFixture(t)
fx.setLive("console.example", "old")
fx.promote("console.example", "new")
restart := func(host string) error { return errors.New("still broken") }
start := func(host string) error { return nil }
for i := 0; i < 2; i++ {
if err := Pass(context.Background(), fx.reg, fx.store, fx.pub, fx.optDir, restart,
WithStartUnit(start)); err != nil {
t.Fatalf("pass %d: %v", i, err)
}
}
if got := fx.recordedVersion("console.example"); got == "new" {
t.Fatal("the second pass stamped a build that has never run")
}
if got := fx.liveVersion("console.example"); got != "old" {
t.Fatalf("symlink after two passes = %q, want old", got)
}
}newAdoptFixture and its helpers are illustrative — read internal/release/adopt_test.go and use the fixture that file already has, or add these helpers to it if none fits.
- [ ] Step 2: Run to verify they fail
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
go test ./internal/release/ -run 'FailedRestartLeaves|SecondPassOver' -count=1Expected: FAIL — undefined: WithStartUnit.
- [ ] Step 3: Add the option and use it in the rollback
Add WithStartUnit alongside the existing PassOptions, then in adoptHost's restart-failure block, after the symlink has gone back and any capability restore has run:
// Start the service again on the version the symlink NOW names.
// The rollback above put the bytes back; without this the box is
// left with a correct symlink and no process — and because
// try-restart is a no-op on a unit that is not running (which is
// what lets a deliberately-stopped service stay stopped), the
// NEXT pass's restart would exit 0, SetVersions would stamp the
// build that has never run, and the header would claim it over a
// service that is down. Start, not try-restart: the whole premise
// here is that the unit is not running.
//
// A start that also fails changes nothing about the refusal — the
// version stays poisoned and unstamped — but it is printed,
// because at that point the box is genuinely broken and the
// operator needs to know that rather than reading a rollback
// message as "recovered".
if cfg.startUnit != nil {
if serr := cfg.startUnit(rt.Host); serr != nil {
fmt.Printf(" %s: rolled back to %s but could not start it: %v — this host is serving nothing\n",
rt.Host, cur, serr)
}
}Then wire it at the agent: makeRestartRoute's neighbours already have tryRestartUnit; add a startUnit sibling that resolves the route's Unit and runs systemctl start, and pass WithStartUnit where the agent builds its pass options. A route with no Unit has nothing to start — return nil, matching arm 1's "this arm takes no part in the fallthrough" rule.
- [ ] Step 4: Run the tests
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
go test ./internal/release/ ./internal/host/ -count=1Expected: PASS.
- [ ] Step 5: Full gate, then commit
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off sh -c '
go build ./... && go vet ./... && GOOS=darwin go build ./... && go test ./... -count=1'
gofmt -l internal/release/adopt.go internal/host/agent.go internal/release/adopt_test.go
git add -A
git commit -m "adopt: a rolled-back restart starts the service again
The rollback repointed the symlink and stopped, leaving a correct symlink
and no process. And because try-restart is a no-op on a unit that is not
running — the property that lets a deliberately-stopped service stay
stopped — the NEXT pass exited 0 and stamped SetVersions over a service
that was not running at all: the exact lying header the mechanism exists
to prevent, now with nothing behind it.
Every unit-backed route has this. It stayed hidden because the pinfra
roll script's explicit post-pass restart papers over it on the only
unit-stamped route in the fleet."---
Task 8: Ship-time refusals for a control-plane app
Files:
- Modify:
internal/console/apicli.go(handleAPIShip, after the artifact loop) - Test:
internal/console/apicli_test.go
Interfaces:
- Consumes:
Store.AppFlags(Task 1). - Produces:
func checkControlPlaneArtifacts(files map[string][]byte, version string) errorininternal/console.
Both checks verified against a real arm64 console build — stdlib only, and -s -w does not strip either:
debug/buildinfo → Settings["-ldflags"] = "-s -w -X …/internal/version.Build=deccd44"
debug/elf → Machine = EM_AARCH64Scoped to control-plane apps deliberately: a tenant may legitimately ship an artifact we would guess wrong about, and refusing their ship on our inference is not our call.
- [ ] Step 1: Write the failing test
Add to internal/console/apicli_test.go:
// TestControlPlaneShipRefusesTheWrongArchitecture: failure mode #1, an amd64
// binary shipped to Graviton boxes. -target linux-arm64 is carlos deploy's
// default, which protects the careful caller; this closes the path a hand-run
// ship, a CI job or a future script takes into the same channel.
func TestControlPlaneShipRefusesTheWrongArchitecture(t *testing.T) {
fx, token := controlPlaneFixture(t, "code-cp-arch")
liftShipBudget(fx)
amd64ELF := buildTestELF(t, "linux", "amd64", "cp00003")
status, out := postShip(t, fx.ts.URL+"/api/cli/apps/bab/console/releases", token,
[]shipPart{metaPart(t, "cp00003", "binary"), {name: "linux-arm64", body: amd64ELF}})
if status != http.StatusBadRequest {
t.Fatalf("amd64 ELF as linux-arm64 = %d %v, want 400", status, out)
}
msg := str(t, out, "error")
if !strings.Contains(msg, "linux-arm64") || !strings.Contains(msg, "amd64") {
t.Fatalf("refusal %q must name both the declared target and what was found", msg)
}
}
// TestControlPlaneShipRefusesAMismatchedEmbeddedBuild: closes the ldflags trap
// at the door. Build without -X …version.Build and the roll succeeds while the
// probe reports "dev" forever — deploy reports failure on a healthy deploy,
// the exact inverse of the lie the probe exists to kill.
func TestControlPlaneShipRefusesAMismatchedEmbeddedBuild(t *testing.T) {
fx, token := controlPlaneFixture(t, "code-cp-embed")
liftShipBudget(fx)
wrong := buildTestELF(t, "linux", "arm64", "notthesha")
status, out := postShip(t, fx.ts.URL+"/api/cli/apps/bab/console/releases", token,
[]shipPart{metaPart(t, "cp00004", "binary"), {name: "linux-arm64", body: wrong}})
if status != http.StatusBadRequest {
t.Fatalf("embedded build != -version = %d %v, want 400", status, out)
}
if msg := str(t, out, "error"); !strings.Contains(msg, "notthesha") || !strings.Contains(msg, "cp00004") {
t.Fatalf("refusal %q must name both builds", msg)
}
}
// TestControlPlaneShipAcceptsAMatchingArtifact.
func TestControlPlaneShipAcceptsAMatchingArtifact(t *testing.T) {
fx, token := controlPlaneFixture(t, "code-cp-ok")
liftShipBudget(fx)
good := buildTestELF(t, "linux", "arm64", "cp00005")
status, out := postShip(t, fx.ts.URL+"/api/cli/apps/bab/console/releases", token,
[]shipPart{metaPart(t, "cp00005", "binary"), {name: "linux-arm64", body: good}})
if status != http.StatusCreated {
t.Fatalf("matching artifact = %d %v, want 201", status, out)
}
}
// TestOrdinaryAppShipIsNotArchChecked: a tenant may legitimately ship an
// artifact we would guess wrong about; refusing on our inference is not our
// call. Only the app that IS the control plane, on boxes we own and know are
// arm64, gets the check.
func TestOrdinaryAppShipIsNotArchChecked(t *testing.T) {
fx := newConsoleFixture(t)
if _, err := fx.st.CreateAppIn(OpsAccountID, "titogo", "paul@keymail.dev", time.Now()); err != nil {
t.Fatal(err)
}
token := cliToken(t, fx, "paul@keymail.dev", "code-cp-tenant")
liftShipBudget(fx)
status, out := postShip(t, fx.ts.URL+"/api/cli/apps/bab/titogo/releases", token,
[]shipPart{metaPart(t, "t0000001", "binary"), {name: "linux-arm64", body: []byte("not an ELF at all")}})
if status != http.StatusCreated {
t.Fatalf("ordinary app shipping a non-ELF = %d %v, want 201 (unaffected)", status, out)
}
}buildTestELF compiles a tiny program with go build at the requested GOOS/GOARCH and ldflags, caching per (goos, goarch, build) in a package-level sync.Map so the suite compiles each combination once. Write it in this file, t.Skip if the toolchain is unavailable, and give it a comment saying why a real binary is used rather than a handcrafted ELF header (the point is to prove debug/buildinfo reads what the real build recipe writes).
- [ ] Step 2: Run to verify they fail
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
go test ./internal/console/ -run 'ControlPlaneShip|OrdinaryAppShip' -count=1Expected: FAIL — all ships accepted.
- [ ] Step 3: Implement the checks
Add to internal/console/controlplane.go:
// targetMachine maps an artifact part name — which IS the declared target,
// "linux-arm64" and friends — to the ELF machine a binary for it must carry.
// Unknown targets are not refused: this guard exists to catch a wrong build
// of a target we know, not to bound the target vocabulary, which targetRe
// already does.
var targetMachine = map[string]elf.Machine{
"linux-arm64": elf.EM_AARCH64,
"linux-amd64": elf.EM_X86_64,
}
// checkControlPlaneArtifacts refuses two ways a control-plane ship can be
// wrong in a manner nothing downstream would catch.
//
// ARCHITECTURE. An amd64 binary on a Graviton box has happened and briefly
// broke one. `carlos deploy` defaults -target to linux-arm64, which protects
// the careful caller and nobody else; this closes the same door against a
// hand-run ship, a CI job, or a script written next year.
//
// EMBEDDED BUILD. version.Build is set by ldflags the operator has to
// remember. Forget it and the artifact says "dev" forever: the roll SUCCEEDS,
// the deploy watch's probe never matches, and a perfectly healthy deploy
// reports failure — the exact inverse of the lie the probe exists to kill.
// Caught here it is one sentence at the door instead of a timeout nobody can
// explain.
//
// Both read the bytes that actually arrived, and both are stdlib. Verified
// against a real arm64 console build that -s -w strips neither the ELF header
// nor the build info.
func checkControlPlaneArtifacts(files map[string][]byte, version string) error {
for name, b := range files {
want, known := targetMachine[name]
if !known {
continue
}
f, err := elf.NewFile(bytes.NewReader(b))
if err != nil {
return fmt.Errorf("artifact %s is not a readable ELF binary: %w", name, err)
}
if f.Machine != want {
return fmt.Errorf("artifact %s declares target %s but is a %s binary",
name, name, machineName(f.Machine))
}
bi, err := buildinfo.Read(bytes.NewReader(b))
if err != nil {
return fmt.Errorf("artifact %s carries no Go build info: %w", name, err)
}
got := embeddedBuild(bi)
if got != version {
return fmt.Errorf("artifact %s was built with version.Build=%q but is being shipped as %q — "+
`rebuild with -ldflags "-X github.com/carlosframework/platform/internal/version.Build=%s"`,
name, got, version, version)
}
}
return nil
}Write embeddedBuild(bi *debug.BuildInfo) string to find the -ldflags setting and extract the value after internal/version.Build=, returning "" when absent — and refuse "" with a message naming the missing ldflag specifically, because that is the common case and deserves its own sentence. Write machineName(elf.Machine) string mapping the two known machines to amd64/arm64 and falling back to the elf.Machine string.
Call it in handleAPIShip after the artifact loop and before release.Ship:
// After the parts are read (the bytes have to exist to be inspected) and
// before anything is written. A control-plane app only.
if cp, _, err := s.store.AppFlags(acct.ID, app); err != nil {
jsonError(w, http.StatusServiceUnavailable, "could not check this app's control-plane status — nothing was shipped")
return
} else if cp && meta.Kind != "static" {
if err := checkControlPlaneArtifacts(files, version); err != nil {
jsonError(w, http.StatusBadRequest, err.Error())
return
}
}version here must be the resolved version (after minting), not meta.Version — read the surrounding code and use whatever holds the final value.
- [ ] Step 4: Run the tests, then the full gate, then commit
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
go test ./internal/console/ -run 'ControlPlaneShip|OrdinaryAppShip' -count=1 -v
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off sh -c '
go build ./... && go vet ./... && GOOS=darwin go build ./... && go test ./... -count=1'
gofmt -l internal/console/controlplane.go internal/console/apicli.go internal/console/apicli_test.go
git add -A
git commit -m "ship: a control-plane artifact must match its target and its version
Two refusals read from the bytes that arrived, stdlib only. debug/elf
catches an amd64 binary declared linux-arm64 — which has happened and
briefly broke a box, and which -target only protects the careful caller
from. debug/buildinfo catches a binary whose embedded version.Build is not
the version being shipped, which is the ldflags trap: forget the flag and
the roll succeeds while the probe reports 'dev' forever, so a healthy
deploy reports failure.
Control-plane apps only. A tenant may legitimately ship an artifact we
would guess wrong about, and refusing their ship on our inference is not
our call."---
Task 9: carlos deploy refuses a derived version from a dirty tree
Files:
- Modify:
deploy.go:52-54 - Test:
deploy_test.go
Interfaces:
- Consumes: nothing.
- Produces: nothing new;
runDeploygains a refusal.
roll-console.sh refuses a dirty tree because the stamp is the git short sha, and a stamp naming a commit the binary does not match is how a false "verified" happens. An explicit -version is the operator saying they mean it and is not second-guessed. This is a general tenant fix.
- [ ] Step 1: Write the failing test
Add to deploy_test.go, following the file's existing pattern for driving runDeploy against a fake console mux, and using a t.TempDir() git repo so the working tree state is controlled:
// TestDeployRefusesADerivedVersionFromADirtyTree: the default -version is the
// git short sha, so a dirty tree ships a stamp naming a commit the binary was
// not built from — which is precisely how a false "verified" happens, and why
// roll-console.sh refused it. An EXPLICIT -version is the operator saying they
// mean it and is not second-guessed.
func TestDeployRefusesADerivedVersionFromADirtyTree(t *testing.T) {
dir := initDirtyGitRepo(t)
err := deployPreflight(dir, "" /* no explicit -version */)
if err == nil {
t.Fatal("a dirty tree with a derived version was accepted")
}
if !strings.Contains(err.Error(), "-version") {
t.Fatalf("refusal %q should name the flag that overrides it", err)
}
if err := deployPreflight(dir, "abc1234"); err != nil {
t.Fatalf("an explicit -version from a dirty tree = %v, want accepted", err)
}
}Factor the check into a small testable deployPreflight(dir, explicitVersion string) error rather than testing through runDeploy, which log.Fatals. initDirtyGitRepo runs git init, commits a file, then modifies it; skip the test if git is unavailable.
- [ ] Step 2: Run to verify it fails
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
go test . -run TestDeployRefusesADerivedVersion -count=1Expected: FAIL — undefined: deployPreflight.
- [ ] Step 3: Implement
// deployPreflight refuses the one combination that produces a release whose
// name is a lie: a version DERIVED from git (the -version default) taken from
// a tree with uncommitted changes. The sha then names a commit the artifact
// was not built from, and every later "which build is this" answer —
// including the honest ones — inherits the lie.
//
// An explicit -version is untouched: that is the operator naming the release
// themselves, and second-guessing it would break every workflow that versions
// by something other than a sha.
func deployPreflight(dir, explicitVersion string) error {
if explicitVersion != "" {
return nil
}
out, err := exec.Command("git", "-C", dir, "status", "--porcelain").Output()
if err != nil {
return nil // not a git checkout, or no git: nothing to be dirty
}
if len(bytes.TrimSpace(out)) == 0 {
return nil
}
return errors.New("this tree has uncommitted changes and -version defaults to the git short sha, " +
"which would name a commit this build does not match — commit, or pass -version explicitly")
}Call it in runDeploy immediately after -version resolution, before anything ships. Track whether -version was explicitly set by testing the flag's value before the gitShortHead() default is applied.
- [ ] Step 4: Run, gate, commit
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off go test . -run Deploy -count=1
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off sh -c '
go build ./... && go vet ./... && GOOS=darwin go build ./... && go test ./... -count=1'
gofmt -l deploy.go deploy_test.go
git add deploy.go deploy_test.go
git commit -m "carlos deploy: refuse a git-derived version from a dirty tree
-version defaults to the git short sha, so a dirty tree ships a stamp
naming a commit the artifact was not built from — the false-verified shape
roll-console.sh has always refused. An explicit -version is the operator
naming the release themselves and is left alone."---
Task 10: The console signals readiness
Files:
- Modify:
cmd/console/main.go:274-276 - Create:
internal/host/sdnotify.go(orcmd/console/sdnotify.goif nothing else will use it) - Test:
internal/host/sdnotify_test.go
Interfaces:
- Consumes: nothing.
- Produces:
func NotifyReady() error— writesREADY=1to$NOTIFY_SOCKET, a no-op returning nil when the variable is unset.
Why (spec §4a): systemctl try-restart on a Type=simple unit succeeds the moment the process is exec'd, so a console that starts and then panics on its first request exits 0 — SetVersions stamps the new build and nothing rolls back. Readiness signalled after the listener is open converts that into a failed start, which Task 7's machinery already handles. The unit-file half (Type=notify) is a pinfra change in the runbook; this task is the code half and is inert until then.
- [ ] Step 1: Write the failing test
// TestNotifyReadyWritesToTheSocket: the whole contract. systemd hands the
// process a unixgram socket path in NOTIFY_SOCKET and waits for READY=1
// before calling the unit started — which is what turns "exec'd" into
// "actually serving" for the adopt pass's restart check.
func TestNotifyReadyWritesToTheSocket(t *testing.T) {
dir := t.TempDir()
sock := filepath.Join(dir, "notify.sock")
conn, err := net.ListenPacket("unixgram", sock)
if err != nil {
t.Skipf("unixgram unavailable here: %v", err)
}
defer conn.Close()
t.Setenv("NOTIFY_SOCKET", sock)
if err := NotifyReady(); err != nil {
t.Fatalf("NotifyReady: %v", err)
}
buf := make([]byte, 64)
conn.SetReadDeadline(time.Now().Add(2 * time.Second))
n, _, err := conn.ReadFrom(buf)
if err != nil {
t.Fatalf("nothing arrived on the notify socket: %v", err)
}
if got := string(buf[:n]); got != "READY=1" {
t.Fatalf("sent %q, want READY=1", got)
}
}
// TestNotifyReadyIsANoOpOutsideSystemd: the console runs under `go run`, in
// tests and on a laptop, and none of those set NOTIFY_SOCKET. Absent is
// normal, not an error.
func TestNotifyReadyIsANoOpOutsideSystemd(t *testing.T) {
t.Setenv("NOTIFY_SOCKET", "")
if err := NotifyReady(); err != nil {
t.Fatalf("NotifyReady with no socket = %v, want nil", err)
}
}- [ ] Step 2: Run to verify it fails
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
go test ./internal/host/ -run TestNotifyReady -count=1Expected: FAIL — undefined: NotifyReady.
- [ ] Step 3: Implement
// NotifyReady tells systemd this process is ready to serve, for a unit
// declared Type=notify.
//
// It exists so that "the process was exec'd" and "the process is serving"
// stop being the same fact to systemd. With Type=simple, `systemctl restart`
// succeeds the instant the binary is exec'd — so a console that starts and
// then dies on its first request, or half-applies a migration, restarts
// SUCCESSFULLY as far as the adopt pass can tell, and the pass then stamps a
// version nothing is really running. Signalled after the listener is open,
// a broken build fails its restart instead, and adoption's existing rollback
// takes over.
//
// No dependency: the protocol is one datagram on the socket systemd names in
// NOTIFY_SOCKET. An unset variable means "not under systemd" — a laptop, a
// test, `go run` — and is not an error.
//
// An "@" prefix denotes a Linux abstract socket, which net.Dial expects as a
// leading NUL.
func NotifyReady() error {
sock := os.Getenv("NOTIFY_SOCKET")
if sock == "" {
return nil
}
if strings.HasPrefix(sock, "@") {
sock = "\x00" + sock[1:]
}
conn, err := net.Dial("unixgram", sock)
if err != nil {
return err
}
defer conn.Close()
_, err = conn.Write([]byte("READY=1"))
return err
}- [ ] Step 4: Call it after the listener is open
cmd/console/main.go — the current last line is log.Fatal(http.ListenAndServe(addr, srv)), which cannot signal readiness because it never returns. Split it:
addr := envOr("CONSOLE_ADDR", "127.0.0.1:8090")
ln, err := net.Listen("tcp", addr)
if err != nil {
log.Fatal(err)
}
log.Printf("console: %s listening on %s", origin, addr)
// AFTER the listener is open and the store has migrated (openConsoleDB
// and NewServer both ran above), so readiness means what it says. With
// console.service declared Type=notify, a build that dies before this
// line fails its restart — and the adopt pass rolls the symlink back and
// starts the previous version, instead of stamping a build that never
// served (internal/release/adopt.go).
if err := host.NotifyReady(); err != nil {
log.Printf("console: notifying systemd of readiness: %v", err)
}
log.Fatal(http.Serve(ln, srv))A failed notify is logged, not fatal: under Type=notify systemd will time the start out anyway, and under any other Type= the console must keep serving.
- [ ] Step 5: Run, gate, commit
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off \
go test ./internal/host/ -run TestNotifyReady -count=1 -v
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off sh -c '
go build ./... && go vet ./... && GOOS=darwin go build ./... && go test ./... -count=1'
gofmt -l cmd/console/main.go internal/host/sdnotify.go internal/host/sdnotify_test.go
git add -A
git commit -m "console: signal readiness to systemd after the listener is open
With Type=simple, systemctl restart succeeds the instant the binary is
exec'd — so a console that dies on its first request restarts
'successfully' as far as the adopt pass can tell, and the pass stamps a
version nothing is running. Readiness signalled after migrate + listen
makes a broken build fail its restart, which adoption's rollback already
handles.
Inert until console.service is declared Type=notify, which is the pinfra
half of this change."---
Rollout runbook
Not TDD tasks — an ordered operational sequence. Do not start it until tasks 1–10 are merged. Each step states how to know it worked. Tito first throughout: smaller blast radius, and its console going down does not take carloku with it.
R0 — Survey both boxes (read-only)
Flagship's readings are in the spec's appendix. Tito's are not, and it is deployed first. For each box collect: systemctl show -p ExecStart -p Type --value console; every console route with its channel and unit (sqlite3 /data/registry.db); sqlite3 /data/console/console.db "select count(*) from owners where app='console'"; whether apps/console/ (legacy, unqualified) exists in the bucket; the bucket production flag on console.
Stop and re-plan if: owners has a console row (the grant refuses the claim before the owner exemption is reached), or a legacy apps/console/ prefix exists.
R1 — Deliver the code by the legacy path, one last time
THREE deliveries, not one. Corrected 2026-08-28, after the branch merged and setting up the roll exposed the gap: roll-console.sh builds ./cmd/console for the box and a native carlos used only to drive ship/promote locally (its lines 52–53). It never touches the box's root /opt/carlos/carlos binary.
| Delivery | Carries | How | |---|---|---| | console | the control-plane flag and its refusal doors, #79, the claim flags, /healthz, the ship-door checks, the probe echo | roll-console.sh <clean-checkout> <box> | | root binary | §4b's restart recovery and its flap bound — they live in internal/release/adopt.go and internal/host/agent.go | build linux/arm64 with the version ldflag, upload to s3://<deployment-bucket>/bootstrap/carlos, then update-platform-binary.sh <profile> <instance> <bucket> <sha256> | | CLI | the build-probe watch, the dirty-tree guard | a release, or a locally built carlos for R5–R6 |
Skipping the root-binary delivery is not cosmetic. R5's first carlos deploy -app console would run against the OLD adoption pass — so if that deploy failed, the recovery path would be the exact defect §4b exists to fix: the rollback leaving the console down and the next pass stamping over it. Deliver all three before R5.
update-platform-binary.sh restarts carlos-edge, the front door for every tenant on the box — the one tenant-visible step in this runbook. Do one box at a time and verify the edge is serving a real tenant route before moving on.
Verify: /proc/<pid>/exe names the sha for the console; carlos version on the box names it for the root binary; systemctl is-active carlos-edge carlos-adopt.timer; a real tenant hostname still answers 200; and curl -s https://<origin>/healthz returns ok <sha> — the first proof Task 5 is live.
R2 — pinfra: Type=notify on console.service
DO NOT flip this at the first opportunity. Found in the Task 9/10 review, and it inverts the naive ordering:
Once the unit is Type=notify, a start only succeeds if the started binary sends READY=1. Adoption's restart-failure path (§4b, WithStartUnit) rolls the symlink back and starts the previous version — and if that previous version predates the readiness commit, it never signals, so the rollback's start TIMES OUT and fails. The box is then left with no console AND a poisoned new version: the §4b hole again, but worse, because the recovery path is now the thing that fails.
So the precondition is two consecutive readiness-capable console versions on disk under /opt/carlos/versions/ — i.e. R2 happens only after a SECOND post-R1 roll, so that whatever the rollback lands on can also signal. Until then console.service stays Type=simple and §4a's residual (a build that starts and then misbehaves) stays open, which is the state we are in today anyway.
Then: add Type=notify and a TimeoutStartSec comfortably exceeding console startup to console.service in the pinfra repo; apply to both boxes; systemctl daemon-reload and restart once.
Verify: systemctl show -p Type --value console says notify; the console is active and serving; and ls /opt/carlos/versions/ shows at least two readiness-capable builds.
R3 — Stamp the right route
From R0: the Unit stamp must be on the host named in ExecStart, and on exactly one route. Flagship today has these disagreeing — ExecStart names platform.carlosframework.com, the stamp is on console.carloku.com.
carlos route -registry /data/registry.db -host <execstart-host> -unit console.service
carlos route -registry /data/registry.db -host <previously-stamped-host> -unit ""Run as carlos, not root. Verify by reading the routes back, then run one carlos adopt and confirm it is a no-op.
R4 — Claim ops/console with the flag, in one step
carlos apps create -app console -account ops -control-plane -build-probe /healthz \
-console https://carlos.tito.iothen the same against https://platform.carlosframework.com. One step, not two: a row without the flag is a window in which any ops member or Publish service credential can roll the production console.
Verify: carlos channels -app console -account ops -console <origin> lists the existing history — proof the claim adopted the objects rather than creating an empty app.
Expect a pipeline object to appear. Since #254, every fresh claim stamps a default one-channel edge pipeline, so this claim writes apps/bab/console/… pipeline state the earlier steps never mention. That is the normal shape of a claim now, not a sign the adoption went wrong — do not investigate it mid-runbook.
It does not gate anything this runbook does: canary/* sits outside every pipeline by design, so canary/console promotes and the adoptions in R5–R7 are ungated by it, and stable keeps its passkey 403 with or without the pipeline.
R5 — First carlos deploy -app console, on tito
Build and deploy in the recipe the spec's failure-mode table demands — cmd/console, not the root binary (the 8-minute outage of 2026-08-08):
GOOS=linux GOARCH=arm64 CGO_ENABLED=0 go build \
-ldflags "-s -w -X github.com/carlosframework/platform/internal/version.Build=$(git rev-parse --short HEAD)" \
-o /tmp/console ./cmd/console
carlos deploy -app console -account ops -console https://carlos.tito.io \
-channel canary/console -host <tito-execstart-host> -target linux-arm64 /tmp/consoleVerify: the command reports live off the probe, and independently verify-console.sh tito <sha> agrees via /proc. Both, this once — that agreement is what retires the script.
If it fails: roll-console.sh <checkout> tito is the break-glass path.
R6 — Flagship
The same, against https://platform.carlosframework.com. Verify identically.
R7 — Prove the ordinary apps
For kass, keymail, platform-site: apply R3's precondition first (their Unit stamps are unsurveyed, and kass/keymail are addr-backed bespoke services). An unstamped keymail "proof" run is the #110 shape — symlink swapped, nothing restarted, header lies — and would certify a deploy that did not happen. Then carlos deploy -app <name> -account <acct> for each.
R8 — Documentation
Update CLAUDE.md's "Deploy reality", which currently sends the reader to the pinfra scripts for all four apps. Reclassify roll-console.sh and verify-console.sh as break-glass in their header comments. File the two follow-ups: adoption deduping named-unit restarts to the end of a pass (spec §4c), and how a fresh deployment gets its ops/console row (spec §11).
---
Self-review
Spec coverage. §2 → Task 2. §3/§3a/§3b → Tasks 5, 6, and 8 (the ldflags trap is closed at the door as well as in the timeout). §4a → Task 10 + R2. §4b → Task 7. §4c → R0/R3. §5 → Task 8. §6 → Tasks 1, 3, 4. §7 → Task 9. §8 → Task 1. §9 → R8. §10 → the runbook, same order. §11's three open questions are carried into R0 (stop conditions) and R8 (filed follow-ups).
Placeholders. None. Four places name a verification the implementer must do against the codebase rather than a value I could read: handleAPIPromote's request field names (Task 4 step 1), the release-store Put helper and create-door status codes (Task 2 step 1), the member-role constant (Task 3 step 1), and internal/release/adopt_test.go's existing fixture (Task 7 step 1). Each says what to check and what to do with the answer — these are "match the surrounding code", not "decide the design".
Type consistency. AppFlags(accountID int64, app string) (bool, string, error) is used identically in Tasks 3, 4, 6 and 8. refuseControlPlane(w, accountID, app, verb) bool is defined in Task 4 and used only there. watchTarget.Probe is added in Task 6 and read only there. NotifyReady() error is defined in Task 10 and called once. claimApp's new trailing ownerClaim bool is added in Task 2 and both call sites are updated in the same task.
One deliberate deviation from the spec, flagged for the reviewer: §3a argues the build probe onto "the app row" without naming a column; this plan adds build_probe TEXT in the same migration as control_plane rather than in a separate task, because splitting them means touching migrateAppsKey's copy list twice and that list is exactly where the spec's §8 trap lives.