amadan

carlos / platform Public

plan: dashboard truth + scale implementation

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Paul Campbell f5cf49c0dba54f1ee0d77ce6aaefd13cd9add0fb parent 0c8f3e7
1 file changed, +1327 −0
  • docs/superpowers/plans/2026-08-11-dashboard-truth-and-scale.md +1327 −0
diff --git a/docs/superpowers/plans/2026-08-11-dashboard-truth-and-scale.md b/docs/superpowers/plans/2026-08-11-dashboard-truth-and-scale.md
new file mode 100644
index 000000000..027aeaf4c
--- /dev/null
+++ b/docs/superpowers/plans/2026-08-11-dashboard-truth-and-scale.md
@@ -0,0 +1,1327 @@
+# Dashboard Truth + Scale Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Make the app dashboard's health reporting truthful (readings for bespoke-unit routes, an honest "No readings" state, a route-granular account-mismatch banner) and scale the page to 100s of instances (flow-diagram rollup, Running-tab search/filter/pagination, API + CLI parity).
+
+**Architecture:** Box side, the 60s metrics sweep additionally samples the cgroup of any registry route with a stamped `Unit` (the #110 column). Console side, `OwnerAppMetrics` learns that third match key, `healthWord` gains an `expectReadings` distinction, the mismatch banner is computed per-route, and a shared list-query helper (q / health / opaque cursor) drives the Running tab, the instances API, and the CLI.
+
+**Tech Stack:** Go, html/template server-rendered console, SQLite registry, bucket release store. No JS added.
+
+**Spec:** `docs/superpowers/specs/2026-08-10-dashboard-truth-and-scale-design.md` — read it first.
+
+## Global Constraints
+
+- Branch: `dashboard-truth-and-scale`, based on origin/main `db76025`. Rebase onto origin/main before opening the PR (a parallel session was rolling PR #140 onto main).
+- Gate before EVERY commit: `go build ./... && go vet ./... && go test ./... -count=1` all clean, plus `gofmt -l` clean on touched files. Sandbox: `GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off`; go commands may need the sandbox disabled ("read-only file system" is the tell).
+- CI also cross-compiles `GOOS=darwin go build ./...` — no Linux-only type assumptions in anything linked into the CLI (`internal/host` is).
+- Wire shapes are additive only: readings chunks (`metrics/boxes/<box>/raw/*.json`), the instances API response, and the registry schema all have live producers/consumers. New JSON fields only; never rename, move, or re-type an existing field. When a storage shape changes, the consumer list is everything that PARSES the shape.
+- Production channel names are pinned in `internal/release/laddercheck_test.go` — do not touch channel semantics.
+- Any test that shells out to the carlos binary builds env through `carlosTestEnv` AND sets `cmd.Dir` to a scratch dir.
+- Health-word vocabulary (exact strings, used in templates, filters, and URLs): `Running`, `Serving`, `Setting up`, `Asleep`, `No readings`, `Not responding`, `Failed`.
+- Comment style: comments state constraints the code can't show; match the codebase's density (it is high and deliberate).
+- Commit messages end with `Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>`.
+- Delegation (Paul's rule): suggested executor model is noted per task; name it explicitly when dispatching. Opus reviews Tasks 4, 5, and 9 (authority/production-touching).
+
+---
+
+### Task 1: RouteView carries Unit, Backing, Provisioned
+
+**Model:** sonnet
+
+**Files:**
+- Modify: `internal/console/view.go` (RouteView struct ~line 112-133; Assemble's RouteView literal ~line 377-383)
+- Test: `internal/console/view_test.go`
+
+**Interfaces:**
+- Consumes: `registry.Route.Unit`, `.Backing`, `.Provisioned` (already scanned by `registry.List()`).
+- Produces: `RouteView.Unit string`, `RouteView.Backing string`, `RouteView.Provisioned bool` — Tasks 2, 3, 6, 9 read these.
+
+- [ ] **Step 1: Write the failing test**
+
+In `internal/console/view_test.go`, add (follow `buildFixtures`' existing registry-seeding pattern in this file — it builds a real SQLite registry; use `registry.Open` + the same route-insert helper it uses, adding a route with `Unit`, `Backing`, and `Provisioned` set — check the exact insert call `buildFixtures` makes and mirror it):
+
+```go
+func TestAssembleCarriesProcessOwnership(t *testing.T) {
+ // Seed one route with Unit "titogo@demo.service", Backing
+ // registry.BackingUnit, Provisioned true (via buildFixtures' insert
+ // path), then Assemble and find its RouteView.
+ views := assembleFixtureViews(t) // or inline Assemble(...) as sibling tests do
+ rt := findRoute(t, views, "unit-stamped.example.test")
+ if rt.Unit != "titogo@demo.service" || rt.Backing != registry.BackingUnit || !rt.Provisioned {
+ t.Fatalf("RouteView lost process ownership: %+v", rt)
+ }
+}
+```
+
+(If `view_test.go` has no `findRoute` helper, write the loop inline — sibling tests show the pattern.)
+
+- [ ] **Step 2: Run it, verify it fails** — `go test ./internal/console/ -run TestAssembleCarriesProcessOwnership -count=1` → FAIL (fields don't exist / are zero).
+
+- [ ] **Step 3: Implement** — add to `RouteView` (after `Caps`):
+
+```go
+ // Unit is the bespoke systemd unit bound to this route
+ // (registry.Route.Unit, #110): set when an operator-run service owns the
+ // route's process. The sweep samples stamped units (Task 5), so a stamped
+ // route's silence is meaningful — see expectsReadings.
+ Unit string
+ // Backing mirrors registry.Route.Backing: "" = an exec child the
+ // activator spawns, registry.BackingUnit = a carlos-app@<host> unit.
+ Backing string
+ // Provisioned mirrors registry.Route.Provisioned: minted from a bucket
+ // instance record, so the activator runs its process by construction.
+ Provisioned bool
+```
+
+and to Assemble's `RouteView{...}` literal: `Unit: rt.Unit, Backing: rt.Backing, Provisioned: rt.Provisioned,`.
+
+- [ ] **Step 4: Run the test, verify PASS**, then the full package: `go test ./internal/console/ -count=1`.
+
+- [ ] **Step 5: Commit** — `git add internal/console/view.go internal/console/view_test.go && git commit` — `console: RouteView carries Unit/Backing/Provisioned`.
+
+---
+
+### Task 2: healthWord tells "Not responding" from "No readings"
+
+**Model:** sonnet
+
+**Files:**
+- Modify: `internal/console/overview.go` (`healthWord` ~line 258, `assembleInstanceRows` ~line 246, new `expectsReadings`)
+- Modify: `internal/console/flow.go` (`addressTone` ~line 100)
+- Modify: `internal/console/templates/app_instances.html` (Health cell, line 14)
+- Test: `internal/console/overview_test.go`
+
+**Interfaces:**
+- Consumes: Task 1's `RouteView.Unit/Backing/Provisioned`.
+- Produces: `func expectsReadings(provisioned bool, unit, backing string) bool` and `func healthWord(kind, status string, hibernate, reporting, expectReadings bool) string` — Tasks 3, 8, 9 call these. New health word `"No readings"`.
+
+- [ ] **Step 1: Write the failing tests** — extend the existing healthWord table test in `overview_test.go` (it exists; the a332e7f commit row-scoped it). Add cases:
+
+```go
+// silent + expected to report stays the red word
+{kind: "instance", status: "ready", hibernate: false, reporting: false, expect: true, want: "Not responding"},
+// silent + nobody could sample it = honest ignorance, not an outage
+{kind: "instance", status: "ready", hibernate: false, reporting: false, expect: false, want: "No readings"},
+// hibernation still wins over both
+{kind: "instance", status: "ready", hibernate: true, reporting: false, expect: false, want: "Asleep"},
+```
+
+And a direct test of the rule:
+
+```go
+func TestExpectsReadings(t *testing.T) {
+ cases := []struct {
+ provisioned bool
+ unit, backing string
+ want bool
+ }{
+ {true, "", "", true}, // instances-API-minted: activator-run
+ {false, "titogo@x.service", "", true}, // stamped: the sweep samples it
+ {false, "", registry.BackingUnit, true}, // carlos-app@ tenant
+ {false, "", "", false}, // manual exec-shaped row: can't tell
+ }
+ for _, c := range cases {
+ if got := expectsReadings(c.provisioned, c.unit, c.backing); got != c.want {
+ t.Fatalf("expectsReadings(%v,%q,%q) = %v", c.provisioned, c.unit, c.backing, got)
+ }
+ }
+}
+```
+
+- [ ] **Step 2: Run, verify FAIL** (new parameter doesn't exist).
+
+- [ ] **Step 3: Implement** in `overview.go`:
+
+```go
+// expectsReadings is the one rule for whether silence from a route's process
+// is meaningful: the platform runs the process (a provisioned instance or a
+// carlos-app@ tenant) or a stamped unit tells the sweep where to look.
+// Everything else — a manually-added route whose process some bespoke,
+// unstamped unit owns — is honest ignorance, not an outage, and must not
+// render red (the 2026-08-10 tito false alarm: 18 serving routes flagged
+// "Not responding" because their titogo@* units matched no sweep key).
+func expectsReadings(provisioned bool, unit, backing string) bool {
+ return provisioned || unit != "" || backing == registry.BackingUnit
+}
+```
+
+(add `"github.com/carlosframework/platform/internal/registry"` to overview.go's imports). Change `healthWord`:
+
+```go
+func healthWord(kind, status string, hibernate, reporting, expectReadings bool) string {
+ switch status {
+ case "pending", "creating", "starting", "issuing":
+ return "Setting up"
+ case "failed":
+ return "Failed"
+ }
+ if kind != "instance" {
+ return "Serving"
+ }
+ switch {
+ case reporting:
+ return "Running"
+ case hibernate:
+ return "Asleep"
+ case expectReadings:
+ return "Not responding"
+ default:
+ return "No readings"
+ }
+}
+```
+
+In `assembleInstanceRows`, the call becomes:
+
+```go
+HealthWord: healthWord(rt.Kind, status, rt.Hibernate, byHost[rt.Host].Reporting(),
+ expectsReadings(rt.Provisioned, rt.Unit, rt.Backing)),
+```
+
+In `flow.go` `addressTone`, add `"No readings"` explicitly to the idle arm (it already falls there via default; naming it documents that it is deliberate):
+
+```go
+ case "Asleep", "No readings":
+ return "idle"
+```
+
+(Adjust: today `Asleep` is in the default; keep whatever structure is there, just ensure both words yield `"idle"` and the comment covers "No readings".) In `app_instances.html`'s Health cell add after the Asleep hint:
+
+```html
+{{if eq .HealthWord "No readings"}}<span class="small"> · stamp its unit (carlos route -unit) to see health here</span>{{end}}
+```
+
+- [ ] **Step 4: Run** `go test ./internal/console/ -count=1` → PASS (fix any other healthWord call sites the compiler finds — `grep -rn "healthWord(" internal/console/`).
+
+- [ ] **Step 5: Commit** — `console: healthWord distinguishes Not responding from No readings`.
+
+---
+
+### Task 3: Overview tiles stop counting the unknowable
+
+**Model:** sonnet
+
+**Files:**
+- Modify: `internal/console/overview.go` (`OverviewStats`, `assembleOverview`, `healthVerdict`)
+- Modify: `internal/console/templates/app_overview.html` (Running tile lines 37-41, Web addresses tile line 50)
+- Test: `internal/console/overview_test.go`
+
+**Interfaces:**
+- Consumes: Task 2's `expectsReadings`.
+- Produces: `OverviewStats.InstanceNoReadings int`, `OverviewStats.InstanceCounted int` (= Expected − NoReadings). Templates and Task 8's header read these.
+
+- [ ] **Step 1: Failing test** — in `overview_test.go`, next to existing `assembleOverview` tests (mirror their AppView/metrics construction):
+
+```go
+func TestOverviewSeparatesNoReadings(t *testing.T) {
+ view := &AppView{Routes: []RouteView{
+ {Host: "a.x", Kind: "instance", Account: "acct1", Provisioned: true}, // reporting
+ {Host: "b.x", Kind: "instance", Account: "acct1", Provisioned: true}, // silent, expected
+ {Host: "c.x", Kind: "instance", Account: "acct1"}, // silent, bespoke-unstamped
+ {Host: "d.x", Kind: "instance", Account: "acct1", Hibernate: true}, // asleep
+ }, Domains: []DomainView{{}, {}, {}, {}}}
+ metrics := []InstanceMetrics{{Host: "a.x", Box: "box1"}}
+ st := assembleOverview(view, "acct1", metrics, nil)
+ if st.InstanceExpected != 4 || st.InstanceReporting != 1 || st.InstanceAsleep != 1 ||
+ st.InstanceNoReadings != 1 || st.InstanceCounted != 3 {
+ t.Fatalf("counts: %+v", st)
+ }
+}
+
+func TestVerdictIgnoresNoReadings(t *testing.T) {
+ // 1 reporting + 1 asleep + 1 no-readings out of 3 counted... construct an
+ // OverviewStats where Reporting+Asleep == InstanceCounted and assert the
+ // verdict is NOT "attention" on account of the no-readings row.
+ st := OverviewStats{HasStable: true, InstanceExpected: 3, InstanceCounted: 2,
+ InstanceReporting: 1, InstanceAsleep: 1, InstanceNoReadings: 1}
+ if v, _ := healthVerdict(st); v != "ok" {
+ t.Fatalf("no-readings rows must not trip the verdict: %s", v)
+ }
+}
+```
+
+- [ ] **Step 2: Run, verify FAIL.**
+
+- [ ] **Step 3: Implement** — `OverviewStats` gains (after `InstanceAsleep`):
+
+```go
+ // InstanceNoReadings counts instance-kind routes nobody could sample: not
+ // provisioned, no carlos-app@ backing, no stamped unit. They are honest
+ // unknowns — the tile names them separately and the verdict must not
+ // treat them as down (the 2026-08-10 tito false alarm).
+ InstanceNoReadings int
+ // InstanceCounted is the Running tile's denominator: InstanceExpected
+ // minus the unknowable. Templates render facts, never compute them.
+ InstanceCounted int
+```
+
+`assembleOverview`'s route loop:
+
+```go
+ if rt.Kind == "instance" {
+ st.InstanceExpected++
+ if reporting[rt.Host] {
+ st.InstanceReporting++
+ } else if rt.Hibernate {
+ st.InstanceAsleep++
+ } else if !expectsReadings(rt.Provisioned, rt.Unit, rt.Backing) {
+ st.InstanceNoReadings++
+ }
+ }
+```
+
+then before the `healthVerdict` call: `st.InstanceCounted = st.InstanceExpected - st.InstanceNoReadings`. In `healthVerdict`, the shortfall case becomes:
+
+```go
+ case st.InstanceReporting+st.InstanceAsleep < st.InstanceCounted:
+ return "attention", "Needs attention"
+```
+
+Template — Running tile value/detail lines become:
+
+```html
+ <p class="tile-value">{{if eq .InstanceTotal 0}}—{{else if eq .InstanceExpected 0}}Served{{else if .MetricsUnavailable}}?{{else if eq .InstanceCounted 0}}?{{else}}{{.InstanceReporting}} of {{.InstanceCounted}}{{end}}</p>
+ <p class="tile-detail">{{if eq .InstanceTotal 0}}nothing running yet{{else if eq .InstanceExpected 0}}static — the platform serves it, nothing to run{{else if .MetricsUnavailable}}no data right now{{else if eq .InstanceCounted 0}}no readings for {{.InstanceNoReadings}} copies — stamp their units to see health{{else}}copies reporting{{if .InstanceAsleep}} · {{.InstanceAsleep}} asleep{{end}}{{if .InstanceNoReadings}} · {{.InstanceNoReadings}} no readings{{end}}{{end}}</p>
+```
+
+Web addresses tile: replace `all working` with `DNS &amp; certs OK`.
+
+- [ ] **Step 4: Run** `go test ./internal/console/ -count=1` → PASS. Fix any existing test asserting the old copy (`grep -rn "all working\|copies responding" internal/console/`).
+
+- [ ] **Step 5: Commit** — `console: Running tile separates not-tracked copies; Web addresses says what it measures`.
+
+---
+
+### Task 4: Account-mismatch banner at route granularity
+
+**Model:** sonnet — **Opus reviews** (this banner gates deploy-confidence messaging).
+
+**Files:**
+- Modify: `internal/console/view.go` (`AppView`, `AccountMismatch`, `Assemble` ~lines 345-425)
+- Modify: `internal/console/templates/app_overview.html` (lines 5-15) and `internal/console/templates/app_releases.html` (its twin banner, lines 5-12)
+- Test: `internal/console/view_test.go` (assembly), `internal/console/overview_test.go` (rendering)
+
+**Interfaces:**
+- Produces: `type MismatchedRoute struct{ Host, Account string }`, `AppView.Mismatched []MismatchedRoute`, `AppView.AllMismatched bool`. `AppView.RouteAccount` keeps its meaning (first mismatched account, `""` when none) so `AccountMismatch()` semantics survive for any other caller.
+
+- [ ] **Step 1: Failing tests** — in `view_test.go`, three cases via the fixture registry (seed routes stamped to another account; mirror how buildFixtures stamps accounts — `release.RouteAccount` reads the row's sqid):
+
+```go
+func TestMismatchListsRoutes(t *testing.T) {
+ // App with 3 non-platform routes, 1 stamped to "otheracct":
+ // Mismatched == [{host1, otheracct}], AllMismatched == false.
+}
+func TestMismatchAllRoutes(t *testing.T) {
+ // All 3 stamped elsewhere: len(Mismatched) == 3, AllMismatched == true.
+}
+func TestMismatchIgnoresPlatformRoutes(t *testing.T) {
+ // A platform-minted route stamped elsewhere contributes nothing.
+}
+```
+
+Write real assertions (the sketches above name the expected values); construct via the same fixture path Task 1 used.
+
+- [ ] **Step 2: Run, verify FAIL.**
+
+- [ ] **Step 3: Implement** — in `view.go`:
+
+```go
+// MismatchedRoute is one route whose stamped account disagrees with the
+// account whose page is being viewed: host plus where it actually reads
+// releases from. The banner names these — a warning that says WHICH routes
+// is a diagnostic; one that says "this app" is an alarm, and on tito
+// (2026-08-10) it was a false one.
+type MismatchedRoute struct{ Host, Account string }
+```
+
+`AppView` gains:
+
+```go
+ // Mismatched is every non-platform route stamped to another account;
+ // AllMismatched is whether that is ALL of them — only then is "this app
+ // is not deploying" a claim the page can back.
+ Mismatched []MismatchedRoute
+ AllMismatched bool
+```
+
+`AccountMismatch()` becomes `return len(v.Mismatched) > 0`. In `Assemble`, replace the `routeAccount` map with:
+
+```go
+ mismatchedByApp := map[string][]MismatchedRoute{}
+ routableByApp := map[string]int{}
+```
+
+and in the route loop (replacing the `routeAccount[rt.App]` block):
+
+```go
+ if rt.App != "" && !rt.Platform {
+ routableByApp[rt.App]++
+ if a := release.RouteAccount(rt); a != acct {
+ mismatchedByApp[rt.App] = append(mismatchedByApp[rt.App], MismatchedRoute{Host: rt.Host, Account: a})
+ }
+ }
+```
+
+In the per-app assembly (replacing the `mismatch :=` block):
+
+```go
+ mm := mismatchedByApp[app]
+ mismatch := ""
+ if len(mm) > 0 {
+ mismatch = mm[0].Account
+ }
+```
+
+and the `AppView{...}` literal gains `Mismatched: mm, AllMismatched: len(mm) > 0 && len(mm) == routableByApp[app],` (keeping `RouteAccount: mismatch`). Templates — in BOTH `app_overview.html` and `app_releases.html`, replace the banner block with:
+
+```html
+{{if .Detail.AccountMismatch}}
+{{if .Detail.AllMismatched}}
+<p class="banner error">
+ <strong>This app is not deploying.</strong> Every one of its addresses reads
+ releases from account <code>{{.Detail.RouteAccount}}</code>, but this page
+ writes them to <code>{{.Account.Sqid}}</code> — so a promote here succeeds
+ and reaches nothing. Affected: {{range .Detail.Mismatched}}<code>{{.Host}}</code> {{end}}
+ Finish the move on this app's
+ <a href="{{.Nav}}/apps/{{.AppName}}/settings">Settings</a> page.
+ <span class="small">If it is the routes that are in the wrong account, move
+ those back instead — whichever of the two is right, they have to agree.</span>
+</p>
+{{else}}
+<p class="banner">
+ {{len .Detail.Mismatched}} of this app's addresses read releases from another
+ account and won't follow promotes made here:
+ {{range .Detail.Mismatched}}<code>{{.Host}}</code> (reads <code>{{.Account}}</code>) {{end}}
+ — the rest deploy normally. Fix on
+ <a href="{{.Nav}}/apps/{{.AppName}}/settings">Settings</a>.
+</p>
+{{end}}
+{{end}}
+```
+
+- [ ] **Step 4: Run** `go test ./internal/console/ -count=1` → PASS (existing banner tests will need the new copy — update assertions to match the new strings, not the other way around).
+
+- [ ] **Step 5: Commit** — `console: account-mismatch banner names routes, claims "not deploying" only when true`.
+
+---
+
+### Task 5: Box sweep samples stamped units
+
+**Model:** opus (root-side, fleet-wide blast radius) — **Opus reviews.**
+
+**Files:**
+- Modify: `internal/host/metrics_probes.go` (new `listStampedUnits`)
+- Modify: `internal/host/metrics_loop.go` (`realUnitReadings` ~line 639, `realSampleProbes` ~line 621, `metricsLoop` ~line 705 — thread a `stampedUnits func() []string` through)
+- Modify: `internal/host/agent.go` (~line 194, the `go metricsLoop(cgm.Root())` call; new `stampedUnitNames`)
+- Test: `internal/host/metrics_probes_test.go` (or wherever `listCarlosUnits`' tests live — find with `grep -rn "listCarlosUnits" internal/host/*_test.go`)
+
+**Interfaces:**
+- Consumes: `registry.Route.Unit` via `reg.List()`; `unitCgroup{name, dir}`; `readUnitRaw`.
+- Produces: readings for stamped units appear in `metrics.Sample.Units` keyed by the REAL unit name (e.g. `titogo@vicky.service`) — the same additive wire shape; Task 6 matches on it.
+
+- [ ] **Step 1: Failing test** — temp-dir fixture, same style as the existing `listCarlosUnits` test:
+
+```go
+func TestListStampedUnits(t *testing.T) {
+ root := t.TempDir()
+ // flat bespoke unit
+ mustMkdir(t, filepath.Join(root, "console.service"))
+ // template instance inside its implicit per-template slice
+ mustMkdir(t, filepath.Join(root, "system-titogo\\x2dcanary.slice", "titogo-canary@demo.service"))
+ // present on disk but NOT stamped — must not be returned
+ mustMkdir(t, filepath.Join(root, "postgres.service"))
+
+ got := listStampedUnits(root, []string{"console.service", "titogo-canary@demo.service", "absent@x.service"})
+ want := map[string]bool{"console.service": true, "titogo-canary@demo.service": true}
+ if len(got) != 2 {
+ t.Fatalf("got %d units: %+v", len(got), got)
+ }
+ for _, u := range got {
+ if !want[u.name] {
+ t.Fatalf("unexpected unit %q", u.name)
+ }
+ if _, err := os.Stat(u.dir); err != nil {
+ t.Fatalf("dir %q not the real cgroup dir: %v", u.dir, err)
+ }
+ }
+}
+```
+
+(`mustMkdir` = `os.MkdirAll` + `t.Fatal` on error; add if absent. Note the literal directory name contains `\x2d` — systemd's escaping — written in Go as `"system-titogo\\x2dcanary.slice"`.)
+
+- [ ] **Step 2: Run, verify FAIL** (`listStampedUnits` undefined).
+
+- [ ] **Step 3: Implement** in `metrics_probes.go`, after `listCarlosUnits`:
+
+```go
+// listStampedUnits finds the cgroup directories of operator-stamped bespoke
+// units — registry Route.Unit (#110), the routes the carlos- naming
+// convention can't see (tito's titogo@*, an operator's console.service).
+// Same two-level walk as listCarlosUnits, for the same reason: flat units
+// sit under root, template instances inside their implicit system-*.slice.
+// A stamped unit with no cgroup dir simply isn't returned — no reading is
+// the honest report for a unit that isn't running, and the console renders
+// its absence, never a guessed zero.
+func listStampedUnits(root string, stamped []string) []unitCgroup {
+ if len(stamped) == 0 {
+ return nil
+ }
+ want := make(map[string]bool, len(stamped))
+ for _, u := range stamped {
+ want[u] = true
+ }
+ entries, err := os.ReadDir(root)
+ if err != nil {
+ return nil
+ }
+ var units []unitCgroup
+ for _, entry := range entries {
+ if !entry.IsDir() {
+ continue
+ }
+ name := entry.Name()
+ if want[name] {
+ units = append(units, unitCgroup{name: name, dir: filepath.Join(root, name)})
+ continue
+ }
+ if !strings.HasPrefix(name, "system-") || !strings.HasSuffix(name, ".slice") {
+ continue
+ }
+ sliceDir := filepath.Join(root, name)
+ nested, err := os.ReadDir(sliceDir)
+ if err != nil {
+ continue // an unreadable slice loses its units, never the whole scan
+ }
+ for _, child := range nested {
+ if child.IsDir() && want[child.Name()] {
+ units = append(units, unitCgroup{name: child.Name(), dir: filepath.Join(sliceDir, child.Name())})
+ }
+ }
+ }
+ return units
+}
+```
+
+- [ ] **Step 4: Run the new test → PASS.**
+
+- [ ] **Step 5: Thread it through the loop.** Read `metrics_loop.go`'s `realUnitReadings` (line ~639), `realSampleProbes` (~621), `metricsLoop` (~705) and the `sampler`/`sampleProbes` types they build, then:
+ - `realUnitReadings(execRoot string)` → `realUnitReadings(execRoot string, stamped []string)`; its unit list becomes `listCarlosUnits(cgroupSliceDir)`, then `append(units, listStampedUnits(cgroupSliceDir, stamped)...)`, then `append(units, listExecCgroups(execRoot)...)` (the `out[u.name] =` map write already dedupes a stamped unit that happens to match the carlos convention).
+ - `realSampleProbes` and `metricsLoop` gain the parameter: `metricsLoop(execRoot string, stampedUnits func() []string)`, calling `stampedUnits()` once per tick and passing the slice down. A nil func is treated as none (guard: `var stamped []string; if stampedUnits != nil { stamped = stampedUnits() }`).
+ - Update every test double / call site the compiler flags (`go build ./internal/host/`).
+
+- [ ] **Step 6: Registry read + agent wiring.** In `agent.go`, next to the other loops:
+
+```go
+// stampedUnitNames is the sweep's one registry read: every distinct bespoke
+// unit stamped on a route (Route.Unit). Called once per 60s tick; a read
+// failure costs this tick's stamped units, never the sweep or the agent.
+func stampedUnitNames(reg *registry.Registry) []string {
+ routes, err := reg.List()
+ if err != nil {
+ return nil
+ }
+ seen := map[string]bool{}
+ var out []string
+ for _, rt := range routes {
+ if rt.Unit == "" || seen[rt.Unit] {
+ continue
+ }
+ seen[rt.Unit] = true
+ out = append(out, rt.Unit)
+ }
+ sort.Strings(out)
+ return out
+}
+```
+
+and change the launch (line ~194): `go metricsLoop(cgm.Root(), func() []string { return stampedUnitNames(reg) })`. Add a small test for `stampedUnitNames` against a fixture registry (dedupe + empty-unit skip) if `internal/host` has registry-fixture tests; if none exist, the `listStampedUnits` + loop-plumbing coverage suffices — note it in the commit message.
+
+- [ ] **Step 7: Full gate** — `go build ./... && go vet ./... && go test ./... -count=1` and `GOOS=darwin go build ./...` (host package links into the CLI). → all PASS.
+
+- [ ] **Step 8: Commit** — `host: metrics sweep samples registry-stamped bespoke units`.
+
+---
+
+### Task 6: Console matches stamped units
+
+**Model:** sonnet
+
+**Files:**
+- Modify: `internal/console/metricsview.go` (`OwnerAppMetrics` ~line 709)
+- Modify: `internal/console/server.go` (`appResourceMetrics` line 1475)
+- Test: `internal/console/metricsview_test.go` (find `OwnerAppMetrics`' existing tests: `grep -rn "OwnerAppMetrics" internal/console/*_test.go`)
+
+**Interfaces:**
+- Consumes: Task 1's `RouteView.Unit`; sweep entries keyed by real unit names (Task 5).
+- Produces: `OwnerAppMetrics(ctx context.Context, hosts []string, unitByHost map[string]string) ([]InstanceMetrics, error)` — Task 9 also calls this.
+
+- [ ] **Step 1: Failing test** — mirror the existing OwnerAppMetrics test's fixture (it fabricates `BoxMetricsView`/sweep data); add:
+
+```go
+func TestOwnerAppMetricsMatchesStampedUnit(t *testing.T) {
+ // Sweep contains a unit row named "titogo@vicky.service" on box "b1".
+ // OwnerAppMetrics(ctx, []string{"vicky.go.tito.io"},
+ // map[string]string{"vicky.go.tito.io": "titogo@vicky.service"})
+ // must return that host with Box "b1" (Reporting true).
+ // And WITHOUT the unitByHost entry, the same host must come back
+ // Reporting false — the third key is opt-in per host, not a wildcard.
+}
+```
+
+Write the real body against the fixture helpers the sibling test uses.
+
+- [ ] **Step 2: Run, verify FAIL** (signature).
+
+- [ ] **Step 3: Implement** — `OwnerAppMetrics` gains the param and one block in the allowlist loop:
+
+```go
+ want[activator.UnitName(h)] = h
+ want[activator.ExecName(h)] = h
+ // The third legal key: an operator-stamped bespoke unit
+ // (registry Route.Unit, #110). Opt-in per host — the caller maps
+ // host → stamped unit from the route rows it already holds; an
+ // empty map costs nothing and matches nothing new.
+ if u := unitByHost[h]; u != "" {
+ want[u] = h
+ }
+```
+
+(A nil map indexes fine in Go — no guard needed.) `appResourceMetrics` builds it from the fields Task 1 added:
+
+```go
+ unitByHost := make(map[string]string, len(view.Routes))
+ for _, rt := range view.Routes {
+ if rt.Account != acct.Sqid || rt.Unit == "" {
+ continue
+ }
+ unitByHost[rt.Host] = rt.Unit
+ }
+ am, err := s.OwnerAppMetrics(ctx, hosts, unitByHost)
+```
+
+Fix every other `OwnerAppMetrics` call site the compiler flags (pass `nil` where no route rows are in hand).
+
+- [ ] **Step 4: Run** `go test ./internal/console/ -count=1` → PASS.
+
+- [ ] **Step 5: Commit** — `console: OwnerAppMetrics matches operator-stamped units`.
+
+---
+
+### Task 7: Flow diagram rolls up above 8 addresses
+
+**Model:** sonnet
+
+**Files:**
+- Modify: `internal/console/flow.go` (`FlowStage`, `assembleFlow` ~line 126)
+- Modify: `internal/console/server.go` (`assembleFlow` call in `handleApp`, ~line 1517 — `grep -n "assembleFlow" internal/console/server.go`)
+- Modify: `internal/console/templates/flow.html` (`flow_stage` template)
+- Test: `internal/console/flow_test.go`
+
+**Interfaces:**
+- Consumes: `InstanceRow.HealthWord` vocabulary (Task 2).
+- Produces: `FlowStage.RolledUp bool, Total, Attention, Hidden int, MoreURL string`; `assembleFlow(view *AppView, rows []InstanceRow, latest, latestAt, instancesPath string) FlowView`.
+
+- [ ] **Step 1: Failing tests** (pure function — exact code, adapt only the AppView/rows construction to sibling tests' style):
+
+```go
+func TestFlowRollsUpBigStages(t *testing.T) {
+ rows := make([]InstanceRow, 0, 12)
+ for i := 0; i < 12; i++ {
+ h := fmt.Sprintf("i%02d.example.test", i)
+ w := "Running"
+ if i == 3 || i == 7 {
+ w = "Not responding"
+ }
+ rows = append(rows, InstanceRow{Host: h, Channel: "edge", HealthWord: w})
+ }
+ view := &AppView{Channels: []ChannelView{{Name: "edge", Version: "v1"}}}
+ f := assembleFlow(view, rows, "v1", "", "/a/x/apps/app/instances")
+ st := f.Line[0] // edge
+ if !st.RolledUp || st.Total != 12 || st.Attention != 2 || st.Hidden != 0 {
+ t.Fatalf("rollup: %+v", st)
+ }
+ if len(st.Addresses) != 2 || st.Addresses[0].Health != "Not responding" {
+ t.Fatalf("exceptions: %+v", st.Addresses)
+ }
+ if st.MoreURL != "/a/x/apps/app/instances" {
+ t.Fatalf("MoreURL: %q", st.MoreURL)
+ }
+}
+
+func TestFlowSmallStagesUnchanged(t *testing.T) {
+ // 8 addresses exactly: RolledUp false, all 8 listed.
+}
+
+func TestFlowRollupCapsExceptions(t *testing.T) {
+ // 20 addresses, 9 "Not responding": Attention 9, len(Addresses) 5, Hidden 4.
+}
+```
+
+- [ ] **Step 2: Run, verify FAIL.**
+
+- [ ] **Step 3: Implement** — constants + filter in `flow.go`:
+
+```go
+// flowAddrMax is the most addresses a stage lists in full. Below it the
+// diagram looks exactly as it always has; above it, a page anticipating
+// hundreds of per-customer instances (2026-08-10 design) shows counts and
+// problems, never an unbounded list.
+const flowAddrMax = 8
+
+// flowExceptionMax caps the needs-attention list inside a rolled-up stage;
+// the Running tab is where the full story lives.
+const flowExceptionMax = 5
+
+// needsAttention is the rollup's filter: the words worth naming when there
+// are too many addresses to name them all. Asleep and No readings are
+// deliberately not here — counted, not listed (hibernation is the system
+// working; no-readings is ignorance, not alarm).
+func needsAttention(health string) bool {
+ switch health {
+ case "Not responding", "Failed", "Setting up":
+ return true
+ }
+ return false
+}
+```
+
+`FlowStage` gains the fields (comment block from Interfaces above). `assembleFlow` gains the `instancesPath string` param; inside `stage()`, after `Addresses: byChannel[channel]`:
+
+```go
+ if addrs := s.Addresses; len(addrs) > flowAddrMax {
+ s.RolledUp = true
+ s.Total = len(addrs)
+ s.MoreURL = instancesPath
+ var exceptions []FlowAddress
+ for _, a := range addrs {
+ if !needsAttention(a.Health) {
+ continue
+ }
+ s.Attention++
+ if len(exceptions) < flowExceptionMax {
+ exceptions = append(exceptions, a)
+ }
+ }
+ s.Hidden = s.Attention - len(exceptions)
+ s.Addresses = exceptions
+ }
+```
+
+`handleApp`'s call site passes `p.Nav+"/apps/"+p.AppName+"/instances"` (check the actual field expressions used two lines up in the same handler for Nav/AppName spelling). `flow_stage` template — wrap the addresses block:
+
+```html
+ {{if .RolledUp}}
+ <p class="flow-note">{{.Total}} addresses{{if .Attention}} · <span class="error">{{.Attention}} need{{if eq .Attention 1}}s{{end}} attention</span>{{end}}</p>
+ {{if .Addresses}}
+ <ul class="flow-addrs">
+ {{range .Addresses}}
+ <li class="flow-addr flow-tone-{{.Tone}}">
+ <a href="https://{{.Host}}">{{.Host}}</a>
+ <span class="flow-health">{{if eq .Tone "bad"}}{{icon "triangle-alert"}}{{end}}{{.Health}}</span>
+ </li>
+ {{end}}
+ </ul>
+ {{end}}
+ <p class="flow-note"><a href="{{.MoreURL}}">{{if .Hidden}}+{{.Hidden}} more · {{end}}all addresses</a></p>
+ {{else}}{{if .Addresses}}
+ <ul class="flow-addrs">
+ ... (the existing block, unchanged)
+ </ul>
+ {{end}}{{end}}
+```
+
+- [ ] **Step 4: Run** `go test ./internal/console/ -count=1` → PASS (other `assembleFlow` call sites/tests get the new arg — compiler finds them).
+
+- [ ] **Step 5: Commit** — `console: flow diagram rolls up stages above 8 addresses`.
+
+---
+
+### Task 8: Running tab — search, health filter, cursor pagination
+
+**Model:** sonnet
+
+**Files:**
+- Create: `internal/console/listquery.go`
+- Modify: `internal/console/server.go` (`handleAppInstances` ~line 1599; `page` struct ~line 526)
+- Modify: `internal/console/templates/app_instances.html`
+- Test: Create `internal/console/listquery_test.go`; extend `internal/console/overview_test.go` (page rendering)
+
+**Interfaces:**
+- Consumes: `InstanceRow` (host-ordered), health vocabulary.
+- Produces: `parseInstanceQuery(url.Values) instanceQuery`, `pageInstanceRows(rows []InstanceRow, q instanceQuery, limit int) instancePage`, `encodeInstanceCursor/decodeInstanceCursor`, `healthSlug/healthFromSlug`, `healthWords` — Task 9 reuses the query/cursor/slug pieces. `const instancePageSize = 50`.
+
+- [ ] **Step 1: Failing tests** in `listquery_test.go` (pure — exact code):
+
+```go
+package console
+
+import (
+ "net/url"
+ "testing"
+)
+
+func rowsFor(hosts ...string) []InstanceRow {
+ out := make([]InstanceRow, len(hosts))
+ for i, h := range hosts {
+ out[i] = InstanceRow{Host: h, HealthWord: "Running"}
+ }
+ return out
+}
+
+func TestHealthSlugRoundTrips(t *testing.T) {
+ for _, w := range healthWords {
+ if got := healthFromSlug(healthSlug(w)); got != w {
+ t.Fatalf("%q -> %q -> %q", w, healthSlug(w), got)
+ }
+ }
+ if healthFromSlug("nonsense") != "" {
+ t.Fatal("unknown slug must mean no filter, not an error")
+ }
+}
+
+func TestPageFiltersAndCounts(t *testing.T) {
+ rows := rowsFor("a.x", "ab.x", "b.x")
+ rows[2].HealthWord = "Asleep"
+ p := pageInstanceRows(rows, instanceQuery{Q: "a"}, 50)
+ if p.Total != 2 || len(p.Rows) != 2 || p.NextCursor != "" {
+ t.Fatalf("%+v", p)
+ }
+ if p.Counts["Running"] != 2 || p.Counts["Asleep"] != 1 {
+ t.Fatalf("counts are over ALL rows, filtered or not: %+v", p.Counts)
+ }
+ p = pageInstanceRows(rows, instanceQuery{Health: "Asleep"}, 50)
+ if p.Total != 1 || p.Rows[0].Host != "b.x" {
+ t.Fatalf("%+v", p)
+ }
+}
+
+func TestPageCursors(t *testing.T) {
+ rows := rowsFor("a.x", "b.x", "c.x", "d.x", "e.x")
+ p1 := pageInstanceRows(rows, instanceQuery{}, 2)
+ if len(p1.Rows) != 2 || p1.NextCursor == "" || p1.Rows[1].Host != "b.x" {
+ t.Fatalf("page1: %+v", p1)
+ }
+ p2 := pageInstanceRows(rows, instanceQuery{Cursor: p1.NextCursor}, 2)
+ if len(p2.Rows) != 2 || p2.Rows[0].Host != "c.x" {
+ t.Fatalf("page2: %+v", p2)
+ }
+ p3 := pageInstanceRows(rows, instanceQuery{Cursor: p2.NextCursor}, 2)
+ if len(p3.Rows) != 1 || p3.Rows[0].Host != "e.x" || p3.NextCursor != "" {
+ t.Fatalf("page3: %+v", p3)
+ }
+}
+
+func TestPageGarbageCursorStartsOver(t *testing.T) {
+ rows := rowsFor("a.x", "b.x")
+ p := pageInstanceRows(rows, instanceQuery{Cursor: "!!not-base64!!"}, 50)
+ if len(p.Rows) != 2 {
+ t.Fatalf("bad cursor must degrade to page one: %+v", p)
+ }
+}
+
+func TestParseInstanceQuery(t *testing.T) {
+ q := parseInstanceQuery(url.Values{"q": {" api "}, "health": {"not-responding"}, "cursor": {"abc"}})
+ if q.Q != "api" || q.Health != "Not responding" || q.Cursor != "abc" {
+ t.Fatalf("%+v", q)
+ }
+}
+```
+
+- [ ] **Step 2: Run, verify FAIL.**
+
+- [ ] **Step 3: Implement `listquery.go`** (exact):
+
+```go
+// The Running tab's and the instances API's shared list controls: a
+// substring match, a health-word filter, and an opaque after-this-host
+// cursor. GET-param driven and server-computed on purpose — a filter the
+// server applies is one the API and CLI can apply identically (every carlos
+// command works for a user with no box access), and a bookmarked filtered
+// URL keeps meaning the same thing.
+package console
+
+import (
+ "encoding/base64"
+ "encoding/json"
+ "net/url"
+ "strings"
+)
+
+// instancePageSize is the Running tab's page length.
+const instancePageSize = 50
+
+// healthWords is every word healthWord can emit, in the order the header
+// lists them. A new word joins healthWord, this list, and nothing else —
+// slugs, filters and links all derive from here.
+var healthWords = []string{"Running", "Serving", "Setting up", "Asleep", "No readings", "Not responding", "Failed"}
+
+// instanceQuery is one request's list controls. Zero value = everything,
+// first page.
+type instanceQuery struct {
+ Q string // substring match on host, trimmed
+ Health string // canonical health word ("" = all)
+ Cursor string // opaque; list resumes after the host inside it
+}
+
+func parseInstanceQuery(v url.Values) instanceQuery {
+ return instanceQuery{
+ Q: strings.TrimSpace(v.Get("q")),
+ Health: healthFromSlug(v.Get("health")),
+ Cursor: v.Get("cursor"),
+ }
+}
+
+// healthSlug is a health word's URL form: "Not responding" ↔
+// "not-responding".
+func healthSlug(word string) string {
+ return strings.ReplaceAll(strings.ToLower(word), " ", "-")
+}
+
+// healthFromSlug inverts healthSlug. An unknown slug returns "" — no filter:
+// a mistyped URL shows everything rather than a mysteriously empty list.
+func healthFromSlug(slug string) string {
+ for _, w := range healthWords {
+ if healthSlug(w) == slug {
+ return w
+ }
+ }
+ return ""
+}
+
+type instanceCursor struct {
+ Host string `json:"host"`
+}
+
+func encodeInstanceCursor(host string) string {
+ b, _ := json.Marshal(instanceCursor{Host: host})
+ return base64.RawURLEncoding.EncodeToString(b)
+}
+
+// decodeInstanceCursor degrades on any malformed input to "no cursor" —
+// page one, never an error page (applog.DecodeCursor's rule).
+func decodeInstanceCursor(s string) (string, bool) {
+ b, err := base64.RawURLEncoding.DecodeString(s)
+ if err != nil {
+ return "", false
+ }
+ var c instanceCursor
+ if json.Unmarshal(b, &c) != nil || c.Host == "" {
+ return "", false
+ }
+ return c.Host, true
+}
+
+// healthLink is one header chip: a word, its count over ALL the app's rows,
+// and the link that filters to it. Precomputed — templates render facts.
+type healthLink struct {
+ Word, Slug string
+ Count int
+ Active bool
+}
+
+// instancePage is one screen of the list: the rows to render plus the
+// numbers and links the header states.
+type instancePage struct {
+ Rows []InstanceRow
+ Counts map[string]int // health word -> count over ALL rows
+ Total int // rows matching Q+Health, across all pages
+ Links []healthLink // non-zero words, healthWords order
+ NextCursor string // "" on the last page
+ NextURL string // the whole next-page href; "" on the last page
+}
+
+// pageInstanceRows applies q to rows (which arrive in host order and stay in
+// it) and cuts one page. Counts cover every row regardless of filter — the
+// header's chips are the app's whole health story, and double as filters.
+func pageInstanceRows(rows []InstanceRow, q instanceQuery, limit int) instancePage {
+ p := instancePage{Counts: map[string]int{}}
+ var matched []InstanceRow
+ for _, r := range rows {
+ p.Counts[r.HealthWord]++
+ if q.Q != "" && !strings.Contains(r.Host, q.Q) {
+ continue
+ }
+ if q.Health != "" && r.HealthWord != q.Health {
+ continue
+ }
+ matched = append(matched, r)
+ }
+ p.Total = len(matched)
+ start := 0
+ if after, ok := decodeInstanceCursor(q.Cursor); ok {
+ for i, r := range matched {
+ if r.Host > after {
+ start = i
+ break
+ }
+ start = i + 1
+ }
+ }
+ end := start + limit
+ if end < len(matched) {
+ p.NextCursor = encodeInstanceCursor(matched[end-1].Host)
+ } else {
+ end = len(matched)
+ }
+ p.Rows = matched[start:end]
+ for _, w := range healthWords {
+ if p.Counts[w] == 0 {
+ continue
+ }
+ p.Links = append(p.Links, healthLink{Word: w, Slug: healthSlug(w), Count: p.Counts[w], Active: q.Health == w})
+ }
+ if p.NextCursor != "" {
+ v := url.Values{}
+ if q.Q != "" {
+ v.Set("q", q.Q)
+ }
+ if q.Health != "" {
+ v.Set("health", healthSlug(q.Health))
+ }
+ v.Set("cursor", p.NextCursor)
+ p.NextURL = "?" + v.Encode()
+ }
+ return p
+}
+```
+
+- [ ] **Step 4: Run listquery tests → PASS.**
+
+- [ ] **Step 5: Wire the handler and template.** `page` struct gains `InstancePage instancePage` and `InstanceQuery instanceQuery`. `handleAppInstances` becomes:
+
+```go
+ q := parseInstanceQuery(r.URL.Query())
+ rows := assembleInstanceRows(p.Detail, acct, p.AppMetrics)
+ pg := pageInstanceRows(rows, q, instancePageSize)
+ p.InstanceRows = pg.Rows
+ p.InstancePage = pg
+ p.InstanceQuery = q
+ if len(rows) == 0 {
+ p.AttachCmd, p.AttachEnable = s.attachHint(r.Context(), acct.Sqid, p.AppName, "stable")
+ }
+```
+
+(NOTE: the empty-state check moves to `len(rows)` — a filter that matches nothing must show "no matches", not the attach onboarding.) Template, between `<h2>Running</h2>` and the table (and only when the app has rows at all):
+
+```html
+{{if or .InstanceRows .InstanceQuery.Q .InstanceQuery.Health}}
+<form method="get" class="listctl">
+ <input type="search" name="q" value="{{.InstanceQuery.Q}}" placeholder="Filter addresses" aria-label="Filter addresses">
+ {{if .InstanceQuery.Health}}<input type="hidden" name="health" value="{{healthSlug .InstanceQuery.Health}}">{{end}}
+ <button>Filter</button>
+</form>
+<p class="small">
+ <a href="?{{if .InstanceQuery.Q}}q={{.InstanceQuery.Q}}{{end}}"{{if not .InstanceQuery.Health}} aria-current="true"{{end}}>All</a>
+ {{range .InstancePage.Links}}
+ · <a href="?health={{.Slug}}{{if $.InstanceQuery.Q}}&amp;q={{$.InstanceQuery.Q}}{{end}}"{{if .Active}} aria-current="true"{{end}}>{{.Word}} {{.Count}}</a>
+ {{end}}
+</p>
+{{end}}
+```
+
+Register `healthSlug` in the template FuncMap (find it: `grep -n "FuncMap" internal/console/*.go`; add `"healthSlug": healthSlug`). After the table, the pager and the filtered-empty state:
+
+```html
+{{if .InstancePage.NextURL}}<p class="small"><a href="{{.InstancePage.NextURL}}">Next {{len .InstanceRows}} shown of {{.InstancePage.Total}} →</a></p>{{end}}
+```
+
+and change the `{{if .InstanceRows}}` empty branch: when `.InstanceQuery.Q` or `.InstanceQuery.Health` is set render `<p>No addresses match.</p>` instead of the attach hint.
+
+- [ ] **Step 6: Handler test** — extend `overview_test.go` next to `TestOverviewInstancesPageRendersTable` (same fixture helpers `newConsoleFixture`/`signInPaul`/`get`/`fx.u`):
+
+```go
+func TestInstancesPageFilters(t *testing.T) {
+ fx := newConsoleFixture(t)
+ c := signInPaul(t, fx)
+ status, _, body := get(t, c, fx.u("/apps/kass/instances?q=zzz-no-such-host"))
+ if status != http.StatusOK || !strings.Contains(body, "No addresses match.") {
+ t.Fatalf("filtered-empty state missing (%d):\n%s", status, body)
+ }
+ status, _, body = get(t, c, fx.u("/apps/kass/instances?health=running"))
+ if status != http.StatusOK {
+ t.Fatalf("health filter = %d", status)
+ }
+}
+```
+
+- [ ] **Step 7: Run** `go test ./internal/console/ -count=1` → PASS.
+
+- [ ] **Step 8: Commit** — `console: Running tab gains search, health filter and cursor pagination`.
+
+---
+
+### Task 9: Instances API — q/health/limit/cursor + health field
+
+**Model:** opus (auth-gated API surface) — **Opus reviews.**
+
+**Files:**
+- Modify: `internal/console/apiinstances.go` (`handleAPIInstanceList` line ~227, `instanceListRecord` line ~156, new `instanceHealth`)
+- Test: `internal/console/apiinstances_test.go`
+
+**Interfaces:**
+- Consumes: `parseInstanceQuery`, cursor + slug helpers (Task 8); `healthWord`/`expectsReadings` (Task 2); `OwnerAppMetrics` with `unitByHost` (Task 6); `openRegistry(s.cfg.Registry)`, `registry.Route`.
+- Produces (wire, ADDITIVE): each record gains `"health"`; the response object gains `"total"`, `"counts"`, `"next_cursor"`. Existing fields untouched; a request with no new params returns every record it did before (default limit stays `instanceListMax`).
+
+- [ ] **Step 1: Failing tests** in `apiinstances_test.go` (reuse its `instanceFixture` + `rawReq` bearer-request helper; create a few records through the same seeding path existing list tests use):
+
+```go
+func TestAPIInstanceListHealthAndFilters(t *testing.T) {
+ // Seed 3 records: hosts "a.app.x", "ab.app.x", "b.app.x".
+ // 1) GET /instances → 3 records, each with a "health"
+ // string field; body contains "total":3 and a "counts" object.
+ // 2) GET /instances?q=a → total 2, hosts a.app.x, ab.app.x.
+ // 3) GET /instances?limit=1 → 1 record + non-empty next_cursor;
+ // following the cursor returns the next host, no overlap.
+ // 4) GET /instances?health=setting-up → only records whose health is
+ // "Setting up" (a record with no registry route yet reads as that).
+ // 5) No params → response still parses into the OLD client shape
+ // (instances + truncated) — additive check.
+}
+```
+
+Write the real body: unmarshal into `map[string]any` / small local structs, assert exactly.
+
+- [ ] **Step 2: Run, verify FAIL.**
+
+- [ ] **Step 3: Implement.** `instanceListRecord` gains `Health string \`json:"health"\``. New helper in `apiinstances.go`:
+
+```go
+// instanceHealth computes each record host's health word the same way the
+// Running tab does — registry row joined with the metrics sweep — so the
+// API, the CLI and the page can never disagree about an instance's state.
+// A record with no registry row yet is still provisioning: "Setting up".
+// Failures degrade to that same word rather than erroring the list: health
+// here is an annotation on the record, not the record.
+func (s *Server) instanceHealth(ctx context.Context, recs []provision.Record) map[string]string {
+ out := make(map[string]string, len(recs))
+ for _, rec := range recs {
+ out[rec.Host] = "Setting up"
+ }
+ reg, err := openRegistry(s.cfg.Registry)
+ if err != nil {
+ return out
+ }
+ routes, err := reg.List()
+ if err != nil {
+ return out
+ }
+ byHost := make(map[string]registry.Route, len(routes))
+ for _, rt := range routes {
+ byHost[rt.Host] = rt
+ }
+ var hosts []string
+ unitByHost := map[string]string{}
+ for _, rec := range recs {
+ rt, ok := byHost[rec.Host]
+ if !ok {
+ continue
+ }
+ hosts = append(hosts, rec.Host)
+ if rt.Unit != "" {
+ unitByHost[rec.Host] = rt.Unit
+ }
+ }
+ am, err := s.OwnerAppMetrics(ctx, hosts, unitByHost)
+ if err != nil {
+ am = nil
+ }
+ rep := make(map[string]bool, len(am))
+ for _, m := range am {
+ if m.Reporting() {
+ rep[m.Host] = true
+ }
+ }
+ for _, rec := range recs {
+ rt, ok := byHost[rec.Host]
+ if !ok {
+ continue
+ }
+ status := rt.Status
+ if status == "" {
+ status = "ready" // registry contract: empty status means ready
+ }
+ out[rec.Host] = healthWord(rt.Kind, status, rt.Hibernate, rep[rec.Host],
+ expectsReadings(rt.Provisioned, rt.Unit, rt.Backing))
+ }
+ return out
+}
+```
+
+`handleAPIInstanceList`, after the existing `recs, truncated` + `versions` reads (keep gate, timeout, error paths byte-identical):
+
+```go
+ q := parseInstanceQuery(r.URL.Query())
+ limit := instanceListMax
+ if raw := r.URL.Query().Get("limit"); raw != "" {
+ if n, err := strconv.Atoi(raw); err == nil && n > 0 && n < instanceListMax {
+ limit = n
+ }
+ }
+ health := s.instanceHealth(ctx, recs)
+ counts := map[string]int{}
+ matched := make([]instanceListRecord, 0, len(recs))
+ for _, rec := range recs {
+ h := health[rec.Host]
+ counts[h]++
+ if q.Q != "" && !strings.Contains(rec.Host, q.Q) {
+ continue
+ }
+ if q.Health != "" && h != q.Health {
+ continue
+ }
+ matched = append(matched, instanceListRecord{Record: rec, StagedVersion: versions[rec.Host], Health: h})
+ }
+ total := len(matched)
+ start := 0
+ if after, ok := decodeInstanceCursor(q.Cursor); ok {
+ for i, rec := range matched {
+ if rec.Host > after {
+ start = i
+ break
+ }
+ start = i + 1
+ }
+ }
+ next := ""
+ end := start + limit
+ if end < total {
+ next = encodeInstanceCursor(matched[end-1].Host)
+ } else {
+ end = total
+ }
+ writeJSON(w, http.StatusOK, map[string]any{
+ "instances": matched[start:end], "truncated": truncated,
+ "total": total, "counts": counts, "next_cursor": next,
+ })
+```
+
+(`provision.ListRecords` already returns records host-sorted — the cursor leans on that; do not re-sort.)
+
+- [ ] **Step 4: Run** `go test ./internal/console/ -count=1` → PASS.
+
+- [ ] **Step 5: Commit** — `console: instances API reports health, filters and paginates (additive)`.
+
+---
+
+### Task 10: CLI — `carlos instances list -match -health`
+
+**Model:** sonnet
+
+**Files:**
+- Modify: `instances_cli.go` (`runInstancesList` ~line 255)
+- Modify: `apimode.go` (`instanceList` ~line 1066, `apiInstanceRecord` ~line 1019, `apiPath` ~line 217)
+- Test: extend the CLI's existing tests near `instances` (find: `grep -rn "instanceList\|instances list" *_test.go`); use `carlosTestEnv` + scratch `cmd.Dir` for any subprocess test.
+
+**Interfaces:**
+- Consumes: API params + fields from Task 9.
+- Produces: `-match <substr>`, `-health <slug>` flags; a HEALTH column; `instanceList(ctx, a, app, match, health string) (recs []apiInstanceRecord, truncated bool, total int, err error)`.
+
+- [ ] **Step 1: Failing test** — unit-test the request construction (an `httptest.Server` capturing the query; construct the `apiSession` the way the nearest apimode test does — check `apimode_test.go` / `auth_e2e_test.go` for the pattern):
+
+```go
+func TestInstanceListSendsFilters(t *testing.T) {
+ // Fake console returns {"instances":[],"truncated":false,"total":0}.
+ // Call instanceList with match "api", health "not-responding" and assert
+ // the request URL had q=api and health=not-responding, and that an
+ // -account-id session (a.Exact) still produces a well-formed URL with
+ // BOTH exact=sqid and the filters (the "?"+"&" join).
+}
+```
+
+- [ ] **Step 2: Run, verify FAIL.**
+
+- [ ] **Step 3: Implement.** `apiPath` learns to join (this is the bug the test above pins):
+
+```go
+func apiPath(a apiAcct, app, tail string) string {
+ p := "/api/cli/apps/" + url.PathEscape(a.Seg) + "/" + url.PathEscape(app) + tail
+ if a.Exact {
+ sep := "?"
+ if strings.Contains(p, "?") {
+ sep = "&"
+ }
+ p += sep + "exact=sqid"
+ }
+ return p
+}
+```
+
+`apiInstanceRecord` gains `Health string \`json:"health"\``. `instanceList` gains params and returns total:
+
+```go
+func (s *apiSession) instanceList(ctx context.Context, a apiAcct, app, match, health string) ([]apiInstanceRecord, bool, int, error) {
+ tail := "/instances"
+ v := url.Values{}
+ if match != "" {
+ v.Set("q", match)
+ }
+ if health != "" {
+ v.Set("health", health)
+ }
+ if enc := v.Encode(); enc != "" {
+ tail += "?" + enc
+ }
+ status, body, err := s.getJSON(ctx, apiPath(a, app, tail))
+ ...
+ var res struct {
+ Instances []apiInstanceRecord `json:"instances"`
+ Truncated bool `json:"truncated"`
+ Total int `json:"total"`
+ }
+ ...
+ return res.Instances, res.Truncated, res.Total, nil
+}
+```
+
+(keep the existing error paths verbatim; a pre-Task-9 console returns no `total` → 0, which the printer treats as "unknown" and omits). `runInstancesList` gains:
+
+```go
+ match := fs.String("match", "", "only hosts containing this substring")
+ health := fs.String("health", "", "only this state: running, serving, setting-up, asleep, no-readings, not-responding, failed")
+```
+
+table header becomes `HOST\tREGION\tCHANNEL\tPOOL\tVERSION\tHEALTH\tCREATED\tBY`, the row printf gains `r.Health` in that position (em-dash `—` when empty — an old console), and after the flush:
+
+```go
+ if (*match != "" || *health != "") && total > len(recs) {
+ mutedf(stdout, "(showing %d of %d matches)", len(recs), total)
+ }
+```
+
+Update `instancesRecordsUsage` to document both flags.
+
+- [ ] **Step 4: Run** `go test ./... -count=1` (CLI tests live at the repo root package) → PASS.
+
+- [ ] **Step 5: Commit** — `cli: instances list -match/-health, server-side, with HEALTH column`.
+
+---
+
+### Task 11: Whole-branch gate, rebase, PR
+
+**Model:** run in the main session (Fable) — whole-branch final review per the delegation policy.
+
+- [ ] **Step 1:** `git fetch origin && git rebase origin/main` (a parallel session merged PR #140 onto main mid-branch — resolve conflicts toward main's shapes).
+- [ ] **Step 2:** Full gate: `go build ./... && go vet ./... && go test ./... -count=1`, `GOOS=darwin go build ./...`, `gofmt -l` on every touched file.
+- [ ] **Step 3:** Re-read the spec end to end against the diff — every section must map to landed code (the self-review checklist at the bottom of this plan).
+- [ ] **Step 4:** Push and open the PR: title `Dashboard truth + scale: honest health, route-granular mismatch banner, instance list controls`, body summarizing the spec's Diagnosis + Design sections, `Closes` nothing automatically (the tito stamping and 404 investigation are operational follow-ups), ending with the standard generation footer.
+
+---
+
+## Rollout runbook (operator / main session — NOT a subagent task)
+
+Order matters; each step verifies against the thing it changed with the binary it built.
+
+1. Merge the PR. Deploy **boxes first** (`update-platform-binary.sh` on flagship + `carlos-tito-ie`) — sampling stamped units is harmless while nothing is stamped. Then both consoles (`roll-console.sh`; console = the `cmd/console` build, NOT the root binary — both boxes are arm64). Coordinate on the current deploy lock discipline (another session was rolling PR #140; check open PRs and parallel-session deploys BEFORE deploying).
+2. Stamp tito's bespoke-unit routes, as user carlos on `carlos-tito-ie`, one per route (registry `-registry /data/titogo/registry.db` — CHECK which registry the tito console reads; the box has two: `/data/registry.db` platform, `/data/titogo/registry.db` titogo):
+ `carlos route -registry <path> -host vicky.go.tito.io -unit titogo@vicky.service` (canaries use `titogo-canary@<name>.service` — confirm each unit's real name with `systemctl list-units 'titogo*'` first; a wrong stamp yields no reading, not an error).
+3. Verify: the titogo Overview on the tito console shows the stamped routes **Running with real CPU/memory** within ~2 sweep ticks; the Running tile reads `N of N reporting`; `curl -sI --resolve` untouched — this change is console-content only.
+4. Confirm the banner: after the roll it must name the actual `bab`-stamped rows (if any remain) instead of the app-wide alarm. Fix those rows (re-stamp/migrate) as a separate operational step.
+5. Investigate `titogo.bae.platform.tito.io` (404, no version header) — genuinely broken, pre-existing, not addressed by this branch.
+6. Website/tap batch per the standing rule if any member-facing docs mention the instances list.
+
+## Self-Review (run after writing all code)
+
+1. **Spec coverage:** §1 readings+vocabulary → Tasks 1,2,5,6; §1 tiles → Task 3; §2 banner → Task 4; §3 rollup → Task 7; §3 Running tab → Task 8; §3 API → Task 9; §3 CLI → Task 10; §4 rollout → runbook; §5 tests → inside each task.
+2. **Placeholder scan:** none of "TBD/TODO/similar to Task N"; the only "check the actual X" notes point at named files for signatures this plan could not verify (fixture seeding helpers, apiSession construction) — resolve them by reading the named file, not by inventing.
+3. **Type consistency:** `expectsReadings(provisioned bool, unit, backing string)`; `healthWord(kind, status string, hibernate, reporting, expectReadings bool)`; `OwnerAppMetrics(ctx, hosts []string, unitByHost map[string]string)`; `assembleFlow(view, rows, latest, latestAt, instancesPath string)`; `pageInstanceRows(rows []InstanceRow, q instanceQuery, limit int) instancePage`; `instanceList(ctx, a, app, match, health string) ([]apiInstanceRecord, bool, int, error)` — used identically everywhere above.