| 1 | --- |
| 2 | target: CARLOS console (internal/console/templates) |
| 3 | total_score: 19 |
| 4 | max_score: 40 |
| 5 | na_heuristics: |
| 6 | p0_count: 2 |
| 7 | p1_count: 3 |
| 8 | timestamp: 2026-08-09T15-36-14Z |
| 9 | slug: internal-console-templates |
| 10 | --- |
| 11 | # CARLOS console — ease-of-understanding critique |
| 12 | |
| 13 | Method: dual-agent (A: design review · B: deterministic detector) |
| 14 | |
| 15 | ## Design Health Score |
| 16 | |
| 17 | | # | Heuristic | Score | Key Issue | |
| 18 | |---|-----------|-------|-----------| |
| 19 | | 1 | Visibility of System Status | 2 | State everywhere, verdict nowhere; `Status` reports cert provisioning, not app health | |
| 20 | | 2 | Match System / Real World | 1 | Nine internal terms in one 24-word sentence (ladder.html:3-4); pointer/manifest/bake/adopt/sqid rendered verbatim | |
| 21 | | 3 | User Control and Freedom | 3 | Confirm cards, rollback, 30-day trash all strong; restore silently doesn't restore serving | |
| 22 | | 4 | Consistency and Standards | 2 | "Instances" = three different numbers on three screens; bake has two names; delete/retire/destroy all used | |
| 23 | | 5 | Error Prevention | 3 | Best-in-class confirm/step-up/type-the-name; fast-track styled like a routine button | |
| 24 | | 6 | Recognition Rather Than Recall | 1 | CLI commands to transcribe; exec/unit/static undefined; canary defined only in a vanishing empty state | |
| 25 | | 7 | Flexibility and Efficiency | 2 | Deep links good; no search/filter/restart; consequences hidden in title= tooltips | |
| 26 | | 8 | Aesthetic and Minimalist Design | 1 | Releases page ≈85 controls; Overview 6 tiles/3 destinations; 60-75-word always-on paragraphs | |
| 27 | | 9 | Error Recovery | 2 | Two excellent diagnostics beside raw `err.Error()` passthrough | |
| 28 | | 10 | Help and Documentation | 2 | Huge volume of inline help, zero structure; no glossary; footer is a disclaimer, not a docs link | |
| 29 | | **Total** | | **19/40** | **Poor — major clarity overhaul needed (craft is high; language misaimed)** | |
| 30 | |
| 31 | ## Design Specificity Verdict |
| 32 | |
| 33 | **LLM assessment:** Authored, emphatically — but for the wrong reader. The em-dash discipline enforced in the funcmap, the documented chip grammar (bordered = fact, filled = outage, exactly one filled variant), the JS-off account switcher: a real designer's fingerprint. But every good decision serves epistemic honesty about *platform internals* — the operator's value, not the app owner's. The console reads as a beautifully documented internal tool. |
| 34 | |
| 35 | **Deterministic scan:** 0 findings across all 23 templates (exit 0, clean). The detector confirms markup hygiene is not the problem; nothing it can measure is wrong. The problem lives entirely in vocabulary and information architecture. |
| 36 | |
| 37 | **Visual overlays:** skipped — no browser automation in this session. |
| 38 | |
| 39 | ## Priority Issues |
| 40 | |
| 41 | **[P0] No health verdict anywhere, and the column that looks like one is lying.** `Status` on apps.html:9 / app_instances.html:10 renders the provisioning enum (`pending|creating|issuing|ready|failed`); `ready` means the cert issued, not the app is up. `Route.LastError` never surfaces. "healthy" appears only on the unlinked operator fleet page. Fix: computed verdict chip (`All good` / `Needs attention` / `Not deployed yet` / `Can't tell right now`) on apps list + Overview from data already in overview.go; rename Status→Health with values Running/Asleep/Starting up/Not responding/Setting up/Failed; move provisioning state to Settings›Domains. |
| 42 | |
| 43 | **[P0] "Supply-chain bake" is the highest-cost term in the product.** Appears on 6 templates + confirm.go, explained differently each time, plus a second alias ("flagged production"). Fix: rename globally to **safety delay**, one canonical line: "Safety delay: on — new versions wait 24h on beta and 72h on live before machines pick them up." Fast-track → "Skip the safety delay", styled as danger. |
| 44 | |
| 45 | **[P1] "Instances" means three different things.** Apps-list column counts kind=="instance" routes; Overview tile and Instances tab count all routes; "Inside the app" is a fourth unrelated concept. Fix: tab → **Running**; tile → **Machines** ("3 of 4 responding · 1 asleep"); drop the list column for a "Running at" URL; app-reported table → "Reported by the app itself" behind a details. |
| 46 | |
| 47 | **[P1] Releases page has no progressive reveal; the metaphor is taught before the answer.** Ladder + 4 tables + 2 shell blocks + ~85 controls. Fix: first glance = "Live: d388035 · 2 days" / "Next: e91ac2 on beta — ready in 4h" + one button; All versions (20) behind details with one Promote… button per row (channel chosen in the confirm card); full ladder + channels + canaries + tags behind a "Release pipeline" disclosure. |
| 48 | |
| 49 | **[P1] Load-bearing content in title= tooltips and muted .small text.** The consequence of disabling the safety delay exists only as a tooltip (ladder.html:26); bake state, "Promotions disabled", and the column legend are muted-small. Fix: promote consequences into visible text or the confirm cards; reserve .small for provenance. |
| 50 | |
| 51 | **[P2] The console tells users to run box-only commands, violating the product's own no-infra rule.** app_settings.html:50-57 (operator carlos add + "on the platform's roadmap"), app_instances.html:32 (carlos add) vs app_releases.html:89-92 (carlos instances create) — conflicting instructions for the same job. Fix: one documented path (carlos instances create); never show a command the reader cannot run. |
| 52 | |
| 53 | **[P2] Raw internal errors rendered into prose.** "Promote refused: "+err.Error(), "flag unreadable ({{.Err}})", "following a pointer that does not exist". Fix: plain first sentence + technical detail behind details. |
| 54 | |
| 55 | **[P3] `summarize` doesn't normalize empty status** → "2 , 1 ready" on the primary screen (server.go:2413-2423 vs overview.go:178-180). |
| 56 | |
| 57 | **[P3] Footer LLM disclaimer is the last thing read on every destructive-confirm screen.** Move to /about. |
| 58 | |
| 59 | ## Persona Red Flags |
| 60 | |
| 61 | **Jordan (first-timer, the owner's worry):** h1 says "Promote it" before Jordan knows what CARLOS does; apps table = 4 jargon columns of 6; empty Overview = six tiles of dashes with the actual next step on another tab; ladder lede = nine terms in one sentence; reads `Status: ready` as "app is fine" (it means cert issued); "restoring with no routes" undecipherable; the one rescuing sentence ("Stable is what every user of X gets") appears only after committing to a promote. |
| 62 | |
| 63 | **Alex (power user):** no app search/filter; no restart control anywhere in the console though carlos restart shipped (#113); promote = 3 clicks + re-deciding a made decision; absolute timestamps title=-only; no copy button on pre blocks containing shell substitutions; two different "Settings" in the header; fleet pages unlinked from any nav. |
| 64 | |
| 65 | **Sam (screen reader/keyboard):** no thead/scope on ~14 tables (the 11-column fleet table is unnavigable); unlabeled inputs on canary form, 20× per-row promote selects, account delete confirm, team invite, cli_approve code; N× bare "Revoke"/"Remove"/"Restore"/"Roll back" buttons with no row subject; title=-only consequences inaccessible; sparkline SVGs untitled and not aria-hidden; no skip link, no :focus-visible anywhere; metrics/fleet render with no nav at all. |
| 66 | |
| 67 | ## Minor Observations |
| 68 | |
| 69 | app_new pattern allows uppercase while help text says lowercase; "canary" defined only in an empty state that vanishes on first use; literal `default` env chip on every unbound route; Activity promises completeness but excludes CLI actions; `From` column renders bare IP; N stacked delete forms in trash; two duration formats (~24h vs 3d4h); `R`/`W` unexpanded in metrics; pagehead one-sentence pattern applied to only half the screens — applying it uniformly is near-free orientation. |
| 70 | |
| 71 | ## Questions to Consider |
| 72 | |
| 73 | 1. If the ladder diagram were deleted from the console (kept in CLI + specs), what would break? Gates are enforced server-side regardless; a "Live / Next up / one button" strip answers the same question in two lines. |
| 74 | 2. Is this one console or two? App pages speak operator language while the operator page is the only one saying "healthy" — the vocabularies are exactly backwards. |
| 75 | 3. The console explains its own uncertainty better than its own state ("authoritative clock is each box's own first-seen record"). Is the honest move to explain the approximation — or to stop showing a number it cannot know and say "Goes live in about 4 hours"? |
| 76 | |
amadan