amadan
spec: record Paul's four rulings on errors and analytics
Analytics off by default, settable per account with per-app override, asked about in the getting-started skill; the reserved prefix is /__carlos/ (static, not per-app random: the path is public by construction and the Origin check, caps and door switch are what bound abuse); warn level is a setting and level is filterable everywhere; the daily salt lives 0600 beside the registry DB, Fable's recommendation. The two per-app switches become one telemetry.json shape at account and app level, resolved app -> account -> default.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PvHDGSqVe2EbPJgqYMHDa
1 file changed,
+129
−65
docs/superpowers/specs/2026-09-07-errors-and-analytics-design.md+129 −65
CI passed — run details
diff --git a/docs/superpowers/specs/2026-09-07-errors-and-analytics-design.md b/docs/superpowers/specs/2026-09-07-errors-and-analytics-design.md| index a6594b56c..6cbdff7ce 100644 |
| --- a/docs/superpowers/specs/2026-09-07-errors-and-analytics-design.md |
| +++ b/docs/superpowers/specs/2026-09-07-errors-and-analytics-design.md |
| @@ -3,9 +3,10 @@ |
| **Date:** 2026-09-07 |
| **Status:** DRAFT for Paul's review. Nothing here has been approved. Every |
| choice is marked **(decision)** and can be overturned cheaply; none is a |
| -ruling. Drafted by Claude Fable, then reviewed by a second Fable context |
| -and by Codex; both returned needs-rework on the first draft and §9 records |
| -what they changed. Line numbers cite `origin/main` at `09a22d6`. |
| +ruling except the four in §8, which Paul answered on 2026-09-07 and |
| +which are marked **(ruling)** where they land. Drafted by Claude Fable, |
| +then reviewed by a second Fable context and by Codex; both returned |
| +needs-rework on the first draft and §9 records what they changed. Line numbers cite `origin/main` at `09a22d6`. |
| ## 1. The idea, and why it fits this platform |
| @@ -118,13 +119,14 @@ Sources and where they enter: |
| | source | errors | analytics | |
| |---------|----------------------------------------------------------|---------------------------------------------| |
| -| log | applog record with level `error` or `fatal`, read-derived | none | |
| +| log | applog record at or above `errors.log_level`, read-derived | none | |
| | edge | upstream 5xx, gateway 502/504, wake failure | every HTML 200 GET becomes a pageview | |
| | server | `POST /errors` on the control socket | `POST /events` on the control socket | |
| -| browser | `POST /_carlos/e` on the app's own host | `POST /_carlos/a` on the app's own host | |
| +| browser | `POST /__carlos/e` on the app's own host | `POST /__carlos/a` on the app's own host | |
| -`log` and `edge` need no integration at all. They are why the tabs are not |
| -empty on day one for every app in the fleet. |
| +`log` and `edge` need no integration at all. They are why the Errors tab |
| +is not empty on day one for any app in the fleet, and why Analytics |
| +starts counting the moment an account turns it on (§3.3). |
| **Sanitisation before buffering (decision, Codex).** Errors carry |
| messages, stacks, paths and caller fields, and those carry secrets in |
| @@ -144,9 +146,14 @@ review 7).** The first draft mirrored `level=error` lines into a second |
| family. `ReadWindow` with `Filter.Level="error"` (`filter.go:22`) over |
| `logs/` yields the same rows with the same retention, and a mirror would |
| have had to choose a side of the token budget at `sink.go:95`. So the |
| -Errors tab reads two families and merges: `logs/` filtered to |
| -`error`/`fatal`, and `errors/` for the other three sources. A JSON line |
| -whose `level` is unrecognised is not an error; `ParseAppLine` never |
| +Errors tab reads two families and merges: `logs/` filtered to the |
| +levels the `errors.log_level` setting names (§3.3; `error` means |
| +`error`+`fatal`, `warn` adds `warn`; **a setting, per Paul's ruling of |
| +2026-09-07**), and `errors/` for the other three sources. Whatever the |
| +setting, every row carries its level and the tab, API and CLI filter on |
| +it, so a member on `warn` can look at errors only and a member on |
| +`error` can widen one read to `warn` without changing the setting. A |
| +JSON line whose `level` is unrecognised is not an error; `ParseAppLine` never |
| guesses, and neither does this. A plain-text `panic:` on stderr carries no |
| level and is not a `log` error either: the crash reaches the tab through |
| the `edge` source when the next request meets a dead instance, which is |
| @@ -251,7 +258,7 @@ requests with `Sec-Purpose: prefetch` or `Purpose: prefetch` are not; |
| `fetch()`, which is how htmx and SPAs load HTML fragments, and a fragment |
| is not a page; an absent header is an old UA or a bot, and the bot list |
| handles the latter). Requests under |
| -`/_carlos/` are never counted. A request whose user agent matches the |
| +`/__carlos/` are never counted. A request whose user agent matches the |
| vendored bot list is counted into `bots` and nowhere else; the list is |
| incomplete by nature and the tab shows `bots` so the member can see what |
| was excluded. |
| @@ -265,9 +272,15 @@ host of `RemoteAddr`. A header the client can set never reaches it. |
| **Visitor (decision, amended per Codex 1 and Fable 10).** A visitor is |
| `sha256(day-salt ‖ route host ‖ clientIP ‖ user agent)[:8]`. The salt is |
| -per box per UTC day, generated at first use, written 0600 to the box's state |
| -directory beside the registry (the agent already owns it) so a restart |
| -mid-day does not re-salt and double-count, and deleted on rotation; it is never written to the bucket. |
| +per box per UTC day, generated at first use, written 0600 to the box's |
| +state directory beside the registry DB (`/data/`, which the agent |
| +already owns) so a restart mid-day does not re-salt and double-count, |
| +and deleted on rotation; it is never written to the bucket. That |
| +location is Fable's recommendation, confirmed against who owns the |
| +directory; Codex's ask was for a restart-stable, app-scoped salt shared |
| +by every box serving the app, which the persisted file gives for the |
| +first half and which the home-box rule makes unnecessary for the |
| +second. |
| No cookie is set, no IP is stored, and the platform holds nothing that |
| joins a hash across days. Because a route is served by its home box (a |
| box without the row backhauls, `edge.go:1216`), a visitor is normally |
| @@ -343,19 +356,43 @@ five-minute `usageLoop` cadence flushes them in the same tick as usage, |
| as a second CAS write (two objects, two writes; the first draft's "one |
| round-trip" was wrong). |
| -**The switch (decision, Fable 9).** `features.json` is console-only and |
| -no box reads it. The switches are one bucket object |
| -`apps/<sqid>/<app>/analytics.json` |
| -`{"count": "edge" | "off", "door": "on" | "off"}`, written by the console |
| -with a serial bump and converged by boxes on the next pass like |
| -`instances/*.json`. Absent means `edge` and `on`. `count: off` stops |
| +**The settings object (decision, Fable 9; default is a ruling).** |
| +`features.json` is console-only and no box reads it. The switches are one |
| +bucket object at two levels with the same shape: |
| + |
| +``` |
| +accounts/<sqid>/telemetry.json account default |
| +apps/<sqid>/<app>/telemetry.json per-app override |
| +{"analytics": {"count": "edge" | "off", "door": "on" | "off"}, |
| + "errors": {"log_level": "error" | "warn"}} |
| +``` |
| + |
| +Each key resolves app → account → built-in default, so an account can |
| +turn analytics on once for every app it has and a single app can still |
| +differ. The console writes either object with a serial bump and boxes |
| +converge on the next pass like `instances/*.json`; the account object |
| +lives beside `account.json`, which stays identity-only. Built-in |
| +defaults: **`count: off` (ruling, Paul, 2026-09-07)**, `door: on`, |
| +`log_level: error`. Analytics is therefore opt-in: the visitor design is |
| +built to be defensible without consent, but whether it is consent-free |
| +is the deployment's assessment and the member's, not the platform's, so |
| +the platform does not switch it on for anyone. `count: off` stops |
| counting at the observing writer and makes both doors drop analytics |
| -events (`/_carlos/a` and the socket's `/events`) with `202`; existing day |
| -files stay until retention. `door: off` closes the browser door |
| +events (`/__carlos/a` and the socket's `/events`) with `202`; existing |
| +day files stay until retention. `door: off` closes the browser door |
| entirely, errors included, for an owner who wants no public ingest |
| surface at all. `"count": "browser"` is reserved for a later mode (§5) |
| so the shape is additive. |
| +Because the default is off, the tab is not empty when it is off: it |
| +shows the switch, what turning it on records, and the one-line privacy |
| +reading, so a member decides from the tab rather than finding a setting |
| +by accident. The `getting-started` skill that `carlos skills` fetches |
| +from carlosframework.com (published from the website repo, not here) |
| +gains one question in the same step: whether to turn analytics on for |
| +the account, with the same one-line reading. That is where an agent |
| +setting up a member's first app will ask it. |
| + |
| ### 3.4 The browser door |
| **Where in the edge (decision, Fable 1, Codex 6).** The first draft said |
| @@ -379,16 +416,16 @@ nothing. Because |
| `Origin` must equal that hostname, the sender is already a page served |
| from it: same origin, same trust boundary the browser already grants |
| those siblings. An unknown hostname takes the existing `!ok` branch and |
| -is never answered by this code. The lookup must never accept `_carlos` as |
| +is never answered by this code. The lookup must never accept `__carlos` as |
| a prefix key; a test pins that. |
| **Endpoint (decision).** |
| ``` |
| -POST /_carlos/e one error {kind, name, stack, path, fields, fp} |
| -POST /_carlos/a one event {name, fields} | {"name":"pageview","path":…} |
| -GET /_carlos/carlos.js loader, Cache-Control: max-age=300, must-revalidate |
| -GET /_carlos/carlos.<hash>.js the snippet, immutable |
| +POST /__carlos/e one error {kind, name, stack, path, fields, fp} |
| +POST /__carlos/a one event {name, fields} | {"name":"pageview","path":…} |
| +GET /__carlos/carlos.js loader, Cache-Control: max-age=300, must-revalidate |
| +GET /__carlos/carlos.<hash>.js the snippet, immutable |
| ``` |
| Both POSTs answer `202` with an empty body in every case: accepted, |
| @@ -396,7 +433,7 @@ rate-limited, rejected, malformed. The endpoint reveals nothing, and the |
| browser never sees a failure worth retrying. Every drop increments the |
| app's `dropped.rate` (or `dropped.origin`, `dropped.size`), which the |
| tab shows, so silence outward is not silence to the owner (Codex 11). |
| -`/_carlos/` is reserved on all routes and never proxied upstream from |
| +`/__carlos/` is reserved on all routes and never proxied upstream from |
| this change on. No app in this repo serves it; §8 asks about the |
| deployment roots. |
| @@ -426,8 +463,8 @@ with a large count and, if it exceeds the bucket, an honest `dropped`. |
| **The snippet (decision).** `carlos.js` is vendored into the edge binary |
| the way Lucide icons are vendored into the console: in-code, never a |
| CDN. It installs `window.onerror` and `unhandledrejection` handlers that |
| -post to `/_carlos/e` after §3.1 sanitisation, and exposes |
| -`carlos.track(name, fields)` which posts to `/_carlos/a` via |
| +post to `/__carlos/e` after §3.1 sanitisation, and exposes |
| +`carlos.track(name, fields)` which posts to `/__carlos/a` via |
| `navigator.sendBeacon`. **It does not send pageviews in edge mode.** The |
| edge already counted the page when it served the HTML; a JS pageview |
| would double every number. What edge mode misses, stated on the tab: SPA |
| @@ -438,7 +475,7 @@ SPA may call `carlos.track("route", {path})` and see it as a named |
| event. Members include the snippet with one line: |
| ```html |
| -<script src="/_carlos/carlos.js" defer></script> |
| +<script src="/__carlos/carlos.js" defer></script> |
| ``` |
| ### 3.5 The server door |
| @@ -479,7 +516,8 @@ through `appPage(w, r, "errors")`, template `app_errors.html`, |
| One row per fingerprint: source badge, kind, name, count in window, first |
| seen, last seen, hosts affected, releases seen. Rows sort by last seen. A |
| row expands in place to the stored sample: stack, path, status, fields, |
| -and a link to Logs at that timestamp. Filters: source, host, release. |
| +and a link to Logs at that timestamp. Filters: source, level, host, |
| +release. |
| Grouping happens on read over the window. `ReadWindow` fetches the whole |
| window and discards beyond `logsCollectCap` (50,000), which bounds |
| memory, not bucket reads and not totals (Codex 14): when the cap is hit |
| @@ -508,16 +546,18 @@ which go through `appPage`; the API paths have no session and no CSRF by |
| construction. The tabs render from the same functions the API answers |
| with. |
| -**CLI.** `carlos errors -app X [-since 24h] [-source browser] [-host h] |
| -[-release v] [-fp <id>]` lists groups, `-fp` expands one; |
| +**CLI.** `carlos errors -app X [-since 24h] [-source browser] [-level warn] |
| +[-host h] [-release v] [-fp <id>]` lists groups, `-fp` expands one; |
| `carlos analytics -app X [-range 7d]` prints the headline, chart as a |
| text sparkline, and the three tables. Both get headings in |
| `docs/site/cli.md` in the same branch, or `internal/docsite` fails the |
| gate. |
| -**Settings.** `analytics: on|off` on the Settings tab and via |
| -`carlos apps`, writing the object in §3.3. Errors have no switch: an app |
| -that is failing is an operational fact the owner needs. A member may |
| +**Settings.** The three keys of §3.3 on the app Settings tab and on the |
| +account settings page, and via `carlos apps` and `carlos accounts`, |
| +writing the two objects. Error *capture* has no switch: an app that is |
| +failing is an operational fact the owner needs; `log_level` only widens |
| +what the `log` source counts. A member may |
| delete an error group from the tab or the API (`DELETE |
| …/errors/{fp}`, owner on the page, `Operate` token on the API). This |
| writes a tombstone `apps/<sqid>/<app>/errors/tombstones/<fp>.json` |
| @@ -541,7 +581,7 @@ should never wonder whether silence means no errors or no wiring. |
| Every path in §3 is checked against the scheduled-ticks rule: |
| -- `/_carlos/*` is answered between route lookup and the meter wrap, before |
| +- `/__carlos/*` is answered between route lookup and the meter wrap, before |
| the cache and the activator. A beacon to a hibernating app returns `202` |
| from the edge; `beginRequest` is never called and the idle clock is not |
| stamped. The test reads `a.entries[host].lastActive` directly, as |
| @@ -611,7 +651,7 @@ capped at 1,000 fingerprints. |
| byte counts unchanged for every existing usage test. |
| - **Pageview definition:** `httptest` through `Edge.ServeHTTP` with a fake |
| upstream: HTML 200 GET counts; JSON 200, HTML 404, HTML 302, 304, HEAD, |
| - `Sec-Purpose: prefetch`, `/_carlos/carlos.js` do not; a cache hit |
| + `Sec-Purpose: prefetch`, `/__carlos/carlos.js` do not; a cache hit |
| counts; `Sec-Fetch-Dest: empty` does not; a known bot UA counts only |
| into `bots`; a `status` route counts nothing; `count: off` counts |
| nothing. |
| @@ -638,9 +678,9 @@ capped at 1,000 fingerprints. |
| post in a minute does not; each drop increments the right `dropped` |
| key; every case answers `202`. A hibernating route receives a beacon |
| and `lastActive` is unchanged. On a path-routed hostname the beacon |
| - lands on the app owning its `path` prefix. `_carlos` is refused as a |
| + lands on the app owning its `path` prefix. `__carlos` is refused as a |
| prefix-map key. |
| -- **`/_carlos/` reservation:** a route whose upstream serves `/_carlos/x` |
| +- **`/__carlos/` reservation:** a route whose upstream serves `/__carlos/x` |
| never sees the request; the loader revalidates and the hashed asset is |
| immutable. |
| - **Server door:** a bad bearer is refused; a good one lands a record; |
| @@ -656,10 +696,15 @@ capped at 1,000 fingerprints. |
| - **Console gates:** both pages give a non-member the byte-identical 404 |
| the Logs tab gives; both API paths refuse a token without `Instance` or |
| `Operate`. |
| -- **Switch convergence:** writing `analytics.json` with `count: off` and |
| - bumping the serial stops counting on the box within the next pass; the |
| - browser door drops `/_carlos/a` and still accepts `/_carlos/e`; |
| - `door: off` drops both. |
| +- **Settings resolution:** with no object at either level nothing is |
| + counted and `log_level` is `error`; an account object with |
| + `count: edge` turns counting on for every app; an app object with |
| + `count: off` under that account turns it off for that app alone; |
| + writing either and bumping the serial converges on the box within the |
| + next pass; the browser door drops `/__carlos/a` and still accepts |
| + `/__carlos/e` when off; `door: off` drops both; `log_level: warn` |
| + brings warn lines into the Errors read and `-level error` filters |
| + them back out. |
| - **Docsite:** the gate passes with both new CLI headings present and |
| fails with either removed. |
| @@ -674,16 +719,19 @@ before promotion, in the ladder the fleet already runs. |
| the fleet gets a populated Errors tab. Requires the chunk-family |
| refactor, the store-door allow-list, the observing writer (for edge |
| errors), and the coalescing sink. |
| -2. **Analytics counting at the edge, tab, API, CLI, the switch.** Every |
| - app gets pageviews and visitors with no JavaScript. Shares the |
| - observing writer with step 1, so it lands second. |
| +2. **Analytics counting at the edge, tab, API, CLI, the settings |
| + objects at both levels, and the skill question.** An account that |
| + turns it on gets pageviews and visitors for every app with no |
| + JavaScript. Shares the observing writer with step 1, so it lands |
| + second. The `getting-started` skill change is a companion branch in |
| + the website repo, landed the same day. |
| 3. **Browser door and `carlos.js`.** Members opt in with one script tag; |
| JS errors and named events appear. |
| 4. **Server door on the control socket.** Exec-backed apps post |
| structured errors and events. |
| Wire shapes are additive at every step: `logs/` is untouched, `errors/` |
| -and `analytics/` are new prefixes, `analytics.json` absent means on, and |
| +and `analytics/` are new prefixes, `telemetry.json` absent means off, and |
| a box on an older build writes none of them. A mixed-version fleet shows |
| the coverage line per host so a missing source is visible rather than |
| puzzling. |
| @@ -691,22 +739,38 @@ puzzling. |
| Binaries roll as usual: promote → adopt → restart for the console; edge |
| and agent pick up on adoption. |
| -## 8. Open questions for Paul |
| - |
| -1. **Default for analytics: on or off?** On, above, because the feature is |
| - invisible if nobody turns it on. The visitor design is built to be |
| - consent-free under the reading Plausible defends (no cookie, no stored |
| - identifier, no cross-day join), but that is a reading, not a ruling, |
| - and Codex is right that it is the deployment's assessment to make, not |
| - the spec's. If the deployment serves EU members who would rather opt |
| - in, off-by-default costs one setting. |
| -2. **Does `/_carlos/` collide with anything in `platform-infrastructure`?** |
| - Nothing in this repo serves it; the deployment roots are not here. |
| -3. **Should the `log` source include `warn`?** No, above; a warn-level |
| - Errors tab is noise on day one. |
| -4. **Salt persistence location.** Beside the registry DB is the |
| - suggestion; if the box state directory has a better home for a |
| - day-scoped secret, use it. |
| +## 8. Rulings (Paul, 2026-09-07) |
| + |
| +Four questions were open in the draft; Paul answered all four. |
| + |
| +1. **Analytics is off by default (ruling).** Settable at the account |
| + level so one switch covers every app, overridable per app, and asked |
| + about in the `getting-started` skill. §3.3 carries the objects and |
| + the resolution order; §7 step 2 carries the skill change. |
| +2. **The reserved prefix is `/__carlos/` (ruling).** Paul asked whether a |
| + random per-app identifier would be better for unpredictability. The |
| + recommendation is no, and the spec stays static: the path is not a |
| + secret and cannot be one, because the snippet URL is in every page's |
| + source and the beacon endpoint is public by construction (§3.4). |
| + What bounds abuse is the Origin check, the caps, `dropped`, and the |
| + `door` switch, none of which a random path would strengthen. A |
| + random path would also cost the one-line snippet its one line and |
| + would make the reservation rule ("this prefix is never proxied |
| + upstream") per-app state instead of a constant. The double underscore |
| + buys the collision safety Paul wanted: no app in this repo serves it, |
| + and nothing conventional does. |
| +3. **Warn level is a setting, and level is always filterable (ruling).** |
| + `errors.log_level` in §3.3 at either level, `error` by default; the |
| + tab, API and CLI filter on level regardless (§3.2, §3.6). |
| +4. **The salt lives beside the registry DB, 0600 (Fable's |
| + recommendation, taken).** §3.3 records both reviewers' positions: |
| + Fable named the location and confirmed the agent owns it; Codex asked |
| + for restart-stability and cross-box sharing, and the persisted file |
| + gives the first while the home-box rule makes the second unnecessary. |
| + |
| +The `platform-infrastructure` collision check for `/__carlos/` is still |
| +worth one grep before step 3 ships; the double underscore makes a hit |
| +unlikely enough that it is a checklist line, not an open question. |
| ## 9. Review round 1 |