amadan
plan: analytics step 3, the browser door and carlos.js, task by task
Ten tasks from the switch shape to the docs: both switches in one map, the /__carlos/ reservation answered before the observer, the door's token buckets, the vendored snippet behind a revalidating loader and an immutable hashed asset, beacon shapes and the Origin rule, drop counts on the meter, the handler, the no-wake test, and the tabs, CLI and docs. Seven planning rulings are listed for Paul to overturn.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
1 file changed,
+1484
−0
docs/superpowers/plans/2026-09-07-analytics-step3-browser-door.md+1484 −0
CI passed — run details
diff --git a/docs/superpowers/plans/2026-09-07-analytics-step3-browser-door.md b/docs/superpowers/plans/2026-09-07-analytics-step3-browser-door.md| new file mode 100644 |
| index 000000000..5568115e2 |
| --- /dev/null |
| +++ b/docs/superpowers/plans/2026-09-07-analytics-step3-browser-door.md |
| @@ -0,0 +1,1484 @@ |
| +# Errors & Analytics — Step 3: the browser door and `carlos.js` — Implementation Plan |
| + |
| +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. |
| + |
| +**Goal:** A member adds one script tag and their app's JavaScript errors appear on the Errors tab and their named events on the Analytics tab, with no wake, no cookie, silent `202`s outward and honest `dropped` counts inward. |
| + |
| +**Architecture:** The edge reserves `/__carlos/` on every route between the route lookup and the observing writer: the loader and the hashed snippet are served from the binary, and `POST /__carlos/e` and `/__carlos/a` are validated (Origin, size, shape, rate), attributed to the app owning the beacon's `path` on that hostname, and handed to the step 1 error sink and the step 2 analytics meter. The host's telemetry pass now pushes both switches (`count`, `door`) per app; the console renders the `door` switch it already accepts; the tabs and the CLI show the new drop reasons and the one-line snippet instruction. |
| + |
| +**Tech Stack:** Go stdlib (net/http, crypto/sha256, hash/fnv, container/list), html/template, one vendored JavaScript string. No new dependencies. |
| + |
| +**Spec:** `docs/superpowers/specs/2026-09-07-errors-and-analytics-design.md` — read §3.1, §3.4, §3.6 ("Coverage line"), §4, §6 ("Browser door", "`/__carlos/` reservation", "Settings resolution"), §8 ruling 2 before the task you are on. The spec wins where this plan and it disagree; say so in the commit if you find such a place. Steps 1 and 2 are on main (`internal/apperr`, `internal/analytics`, `internal/telemetry`, `internal/edge/analytics.go`, `internal/host/telemetry_pass.go`); read them before touching them. |
| + |
| +## Global Constraints |
| + |
| +- Reserved prefix is the static `/__carlos/` (§8 ruling 2). It is answered by the edge on every route kind and never proxied upstream from this branch on. |
| +- The intercept sits AFTER `lookupUpstreamPath` and BEFORE the `countingWriter` wrap in `Edge.ServeHTTP` (§3.4): beacons are unbilled, unmetered, uncompressed, uncached and never reach the activator (§4). `beginRequest` is never called; `lastActive` never moves. |
| +- Both POSTs answer `202` with an empty body in every case: accepted, dropped, rejected, malformed, switched off (§3.4). |
| +- `Origin` must be `https` and resolve through `lookupIn(e.current(), originHost)` to an entry whose `route.Host` equals the request's `up.route.Host`; `Referer` is consulted only when `Origin` is absent; `Sec-Fetch-Site: cross-site` is rejected outright (§3.4). |
| +- Caps: body 16 KB; `stack` 8 KB (`apperr.MaxStack`); `fields` 32 keys × 256 bytes; `kind` 64 bytes; `name` 1024 bytes; `fp` 64 bytes (the sink caps it). Rate: per app 600/min for `/e` and 600/min for `/a` separately; per visitor 60/min; visitor state ≤ 10,000 entries per app, LRU (§3.4). |
| +- Every drop increments the app's `dropped.e|a.{rate,size,origin,malformed}` (§3.4, `analytics.DropCounts`), through the meter, when the app's `count` switch is on. Drops on a count-off app are not recorded anywhere (ruling below). |
| +- The snippet sends NO pageviews in edge mode (§3.4). A `pageview` arriving at `/a` is answered `202` and not counted. |
| +- `door: off` drops both POSTs; `count: off` drops `/a` and still accepts `/e` (§6 "Settings resolution"). Neither is a recorded drop: the owner chose it. |
| +- `carlos.js` is vendored in-code (a Go string), never a CDN; the loader is `Cache-Control: max-age=300, must-revalidate`, the hashed asset `public, max-age=31536000, immutable` (§3.4). |
| +- Sanitisation floor (§3.1) applies in the snippet before sending AND in the sink on arrival; the sink's `Add` already sanitises name, path, stack and fields. |
| +- Additive wire shapes: `apperr.Record` and `analytics.Day` do not change. `analytics.DropCounts` keys already exist. |
| +- Every commit: `gofmt -l` clean, `go vet ./...`, `go test ./internal/edge/ ./internal/analytics/ ./internal/host/ ./internal/console/ ./internal/domains/ .`; before push `GOOS=darwin GOARCH=arm64 go build ./...` and `GOOS=windows GOARCH=amd64 go build ./...`. Env: `GOFLAGS=-mod=mod GOPROXY=off TMPDIR=/tmp/g` (mkdir it). |
| +- Never `git stash`. Commit with `git commit -F <file>` and the `Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>` trailer. |
| + |
| +**Rulings taken while planning (Paul to overturn if wrong):** |
| +1. Switches travel as one map of `telemetry.Switches{Count, Door bool}` replacing step 2's `map[string]bool`. Cost if wrong: one type rename. |
| +2. Drops are recorded only when the attributed app's `count` is on: with counting off there is no Analytics tab to show them and nothing is written under `analytics/`. Cost if wrong: one `if`. |
| +3. Drops that happen before attribution (origin, size) are charged to the hostname's root app (`up`), because the body has not been read. Cost if wrong: a miscount on path-routed hostnames only. |
| +4. The per-visitor rate key is `fnv64(clientIP ‖ UA)` in edge memory, not the salted visitor hash: it is never stored and needs no salt. Cost if wrong: none visible. |
| +5. The loader is a two-line script that appends the hashed asset, so a page that cached the loader for five minutes still picks up a new snippet within five minutes of a roll. Cost if wrong: one redirect instead. |
| +6. `validPathPrefix` (domains) refuses `__carlos`, AND `lookupUpstreamPath` ignores a `__carlos` key defensively. Cost if wrong: one redundant check. |
| +7. Analytics tab coverage line: "Sources: edge" plus ", browser" once the range has a named event or a browser drop. Errors tab keeps its record-derived Sources line. |
| + |
| +--- |
| + |
| +## File structure |
| + |
| +| File | Responsibility | |
| +|---|---| |
| +| `internal/telemetry/telemetry.go` | `Switches` struct and `Resolved.Switches()` | |
| +| `internal/edge/analytics.go` | `SetSwitches`, `switchesFor(rt)`; the `AnalyticsMeter` interface grows `Track` and `Drop` | |
| +| `internal/host/telemetry_pass.go` | resolves both switches per app | |
| +| `internal/host/agent.go:456` | `setAnalyticsSink(e.SetSwitches)` | |
| +| `internal/edge/reserved.go` | `ReservedPrefix`, the intercept `serveReserved`, the 404 default | |
| +| `internal/edge/edge.go` | the one intercept line in `ServeHTTP`; the `__carlos` guard in `lookupUpstreamPath` | |
| +| `internal/domains/claims.go` | `validPathPrefix` refuses `__carlos` | |
| +| `internal/edge/doorlimit.go` | token buckets per app/door and per visitor, LRU-bounded | |
| +| `internal/edge/carlosjs.go` | the vendored snippet, its hash, the loader and asset handlers | |
| +| `internal/edge/browserdoor.go` | beacon parsing, the Origin rule, the POST handler | |
| +| `internal/analytics/meter.go` | `Drop` | |
| +| `internal/console/templates/{app_settings,account_settings,app_analytics,app_errors}.html`, `analyticsview.go` | door switch rendered; coverage line; drop reasons; snippet instruction | |
| +| `analytics_cmd.go`, `docs/site/cli.md` | drop breakdown; snippet paragraph | |
| + |
| +--- |
| + |
| +### Task 1: Both switches travel together |
| + |
| +**Files:** |
| +- Modify: `internal/telemetry/telemetry.go` (after `Resolved`) |
| +- Modify: `internal/edge/analytics.go:34-60` |
| +- Modify: `internal/host/telemetry_pass.go` |
| +- Modify: `internal/host/agent.go:456` |
| +- Test: `internal/telemetry/telemetry_test.go`, `internal/edge/analytics_test.go` (every `SetAnalyticsOn` caller), `internal/host/telemetry_pass_test.go` |
| + |
| +**Interfaces:** |
| +- Consumes: `telemetry.Resolved` (step 2), `Edge.analyticsOn atomic.Pointer[map[string]bool]` (step 2, replaced). |
| +- Produces: |
| + ```go |
| + // package telemetry |
| + type Switches struct{ Count, Door bool } |
| + func (r Resolved) Switches() Switches // Count: r.Count == CountEdge; Door: r.Door == DoorOn |
| + // package edge |
| + func (e *Edge) SetSwitches(m map[string]telemetry.Switches) // copies; nil clears |
| + func (e *Edge) switchesFor(rt registry.Route) telemetry.Switches // zero value when unset |
| + // package host |
| + func telemetrySwitches(ctx, s, reg) (map[string]telemetry.Switches, error) |
| + func setAnalyticsSink(fn func(map[string]telemetry.Switches)) |
| + ``` |
| + `SetAnalyticsOn` is REMOVED; `analyticsEnabled(rt)` becomes `e.switchesFor(rt).Count`. |
| + |
| +- [ ] **Step 1: Failing tests** |
| + |
| +`internal/telemetry/telemetry_test.go` — add: |
| +```go |
| +func TestResolvedSwitches(t *testing.T) { |
| + r := Resolved{Count: CountEdge, Door: DoorOff} |
| + if sw := r.Switches(); !sw.Count || sw.Door { |
| + t.Fatalf("got %+v", sw) |
| + } |
| + r = Resolved{Count: CountOff, Door: DoorOn} |
| + if sw := r.Switches(); sw.Count || !sw.Door { |
| + t.Fatalf("got %+v", sw) |
| + } |
| +} |
| +``` |
| +`internal/edge/analytics_test.go` — replace every `e.SetAnalyticsOn(map[string]bool{k: true})` with `e.SetSwitches(map[string]telemetry.Switches{k: {Count: true, Door: true}})` and add: |
| +```go |
| +func TestSetSwitchesCopiesAndClears(t *testing.T) { |
| + e := &Edge{} |
| + rt := registry.Route{App: "web"} |
| + m := map[string]telemetry.Switches{release.OpsAccount + "/web": {Count: true}} |
| + e.SetSwitches(m) |
| + delete(m, release.OpsAccount+"/web") |
| + if !e.switchesFor(rt).Count { |
| + t.Fatal("stored map must be a copy") |
| + } |
| + e.SetSwitches(nil) |
| + if sw := e.switchesFor(rt); sw.Count || sw.Door { |
| + t.Fatal("nil clears to the zero switches") |
| + } |
| +} |
| +``` |
| +`internal/host/telemetry_pass_test.go` — the existing test that asserts `on[key] == true` becomes `on[key] == telemetry.Switches{Count: true, Door: true}`; add a case with an app object `{"analytics":{"door":"off"}}` expecting `{Count: true, Door: false}` under an account object `{"analytics":{"count":"edge"}}`. |
| + |
| +- [ ] **Step 2: Run** `go test ./internal/telemetry/ ./internal/edge/ -run 'Switches' ./internal/host/ -run Telemetry` — FAIL: undefined `Switches`, `SetSwitches`. |
| + |
| +- [ ] **Step 3: Implement** |
| + |
| +`telemetry.go`: |
| +```go |
| +// Switches is what a box acts on for one app: the two booleans the edge |
| +// consults per request. Resolved carries provenance for the page; this |
| +// carries none, so it is what crosses into the edge. |
| +type Switches struct{ Count, Door bool } |
| + |
| +func (r Resolved) Switches() Switches { return Switches{Count: r.Count == CountEdge, Door: r.Door == DoorOn} } |
| +``` |
| +`edge/analytics.go`: field `switches atomic.Pointer[map[string]telemetry.Switches]` (rename `analyticsOn` in the Edge struct, `edge.go`), `SetSwitches` with the same copy semantics and doc as `SetAnalyticsOn` had, `switchesFor` returning `(*m)[release.RouteAccount(rt)+"/"+rt.App]`, and `analyticsEnabled` returning `e.switchesFor(rt).Count`. Delete `SetAnalyticsOn`. |
| +`host/telemetry_pass.go`: `analyticsSink atomic.Pointer[func(map[string]telemetry.Switches)]`; `telemetrySwitches` builds `map[string]telemetry.Switches` with `on[key] = res.Switches()` (a pair that fails to resolve is left ABSENT, which is the zero value: off and door closed — keep the log line). `agent.go:456`: `setAnalyticsSink(e.SetSwitches)`. |
| + |
| +- [ ] **Step 4: Run** the same tests plus `go build ./... && go vet ./internal/edge/ ./internal/host/` — PASS. |
| +- [ ] **Step 5: Commit** `analytics: the switch map carries count and door together`. |
| + |
| +--- |
| + |
| +### Task 2: `/__carlos/` is reserved on every route |
| + |
| +**Files:** |
| +- Create: `internal/edge/reserved.go` |
| +- Modify: `internal/edge/edge.go` (`ServeHTTP` after line 1271, `lookupUpstreamPath` at 1075-1090) |
| +- Modify: `internal/domains/claims.go:365` (`validPathPrefix`) |
| +- Test: `internal/edge/reserved_test.go`, `internal/domains/claims_test.go` |
| + |
| +**Interfaces:** |
| +- Produces: |
| + ```go |
| + // package edge |
| + const ReservedPrefix = "/__carlos/" |
| + const reservedSegment = "__carlos" |
| + // serveReserved answers everything under ReservedPrefix. This task: 404 for all. |
| + // Tasks 4 and 7 add the snippet and the beacon dispatch inside it. |
| + func (e *Edge) serveReserved(w http.ResponseWriter, r *http.Request, host string, up upstream) |
| + ``` |
| + |
| +- [ ] **Step 1: Failing tests** |
| + |
| +`internal/edge/reserved_test.go`: |
| +```go |
| +package edge |
| + |
| +import ( |
| + "net/http" |
| + "net/http/httptest" |
| + "sync/atomic" |
| + "testing" |
| + |
| + "github.com/carlosframework/platform/internal/registry" |
| +) |
| + |
| +// A route's upstream never sees /__carlos/*, whatever it would have served. |
| +func TestReservedPrefixNeverProxied(t *testing.T) { |
| + var hits atomic.Int32 |
| + backend := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { |
| + hits.Add(1) |
| + w.Write([]byte("upstream")) |
| + })) |
| + t.Cleanup(backend.Close) |
| + reg := openReg(t) |
| + if err := reg.Upsert(registry.Route{Host: "app.example.com", Kind: "service", Addr: backend.Listener.Addr().String(), Status: "ready", App: "web"}); err != nil { |
| + t.Fatal(err) |
| + } |
| + e, err := New(reg, nil) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + srv := httptest.NewServer(e) |
| + t.Cleanup(srv.Close) |
| + for _, p := range []string{"/__carlos/x", "/__carlos/", "/__carlos/e"} { |
| + req, _ := http.NewRequest("GET", srv.URL+p, nil) |
| + req.Host = "app.example.com" |
| + resp, err := http.DefaultClient.Do(req) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + resp.Body.Close() |
| + if resp.StatusCode != http.StatusNotFound { |
| + t.Fatalf("%s: status %d, want 404", p, resp.StatusCode) |
| + } |
| + } |
| + if hits.Load() != 0 { |
| + t.Fatalf("upstream saw %d reserved requests", hits.Load()) |
| + } |
| + // The prefix is exact: /__carlosx and /__carlos (no slash) are the app's. |
| + req, _ := http.NewRequest("GET", srv.URL+"/__carlosx", nil) |
| + req.Host = "app.example.com" |
| + resp, _ := http.DefaultClient.Do(req) |
| + resp.Body.Close() |
| + if hits.Load() != 1 { |
| + t.Fatal("/__carlosx must reach the upstream") |
| + } |
| +} |
| + |
| +// A hibernating route's activator is never invoked for a reserved path. |
| +func TestReservedPrefixNeverWakes(t *testing.T) { |
| + reg := openReg(t) |
| + if err := reg.Upsert(registry.Route{Host: "sleepy.example.com", Kind: "instance", Addr: "127.0.0.1:1", Status: "ready", App: "web", Hibernate: true}); err != nil { |
| + t.Fatal(err) |
| + } |
| + act := &fakeActivator{} |
| + e, err := New(reg, act) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + rec := httptest.NewRecorder() |
| + req := httptest.NewRequest("POST", "http://sleepy.example.com/__carlos/a", nil) |
| + e.ServeHTTP(rec, req) |
| + if act.calls() != 0 { |
| + t.Fatal("the activator must not see a reserved path") |
| + } |
| +} |
| + |
| +// A prefix map cannot steer the reserved segment: a key "__carlos" is ignored. |
| +func TestLookupIgnoresReservedPrefixKey(t *testing.T) { |
| + root := upstream{route: registry.Route{Host: "shared.example.com", App: "root"}, paths: map[string]string{"__carlos": "other.example.com", "team": "other.example.com"}} |
| + other := upstream{route: registry.Route{Host: "other.example.com", App: "other"}} |
| + e := &Edge{} |
| + e.upstreams.Store(&map[string]upstream{"shared.example.com": root, "other.example.com": other}) |
| + if up, _ := e.lookupUpstreamPath("shared.example.com", "/__carlos/e"); up.route.App != "root" { |
| + t.Fatalf("__carlos steered to %q", up.route.App) |
| + } |
| + if up, _ := e.lookupUpstreamPath("shared.example.com", "/team/x"); up.route.App != "other" { |
| + t.Fatalf("ordinary prefix broke: %q", up.route.App) |
| + } |
| +} |
| +``` |
| +Read `fakeActivator` in `edge_test.go:251` for the call counter it exposes (add a `calls()` accessor if it lacks one) and how the live map is stored (`e.current()` — find the atomic field it loads and use it in the third test; if the map is only reachable through `Reload`, build the two routes via the registry with a claim path map instead and say so in the commit). |
| + |
| +`internal/domains/claims_test.go` — add to the existing `validPathPrefix` table (find `TestValidPathPrefix` or the nearest): `{"__carlos", false}`, `{"__carlosx", true}`. |
| + |
| +- [ ] **Step 2: Run** `go test ./internal/edge/ -run Reserved ./internal/domains/ -run PathPrefix` — FAIL. |
| + |
| +- [ ] **Step 3: Implement** |
| + |
| +`reserved.go`: |
| +```go |
| +package edge |
| + |
| +import "net/http" |
| + |
| +// ReservedPrefix is the platform's own path space on every route (spec |
| +// 2026-09-07 §3.4, §8 ruling 2): the browser door and its snippet. It is |
| +// answered here, between the route lookup and the observing writer, so |
| +// nothing under it is billed, compressed, cached, proxied or able to wake |
| +// an instance (§4). |
| +const ReservedPrefix = "/__carlos/" |
| + |
| +const reservedSegment = "__carlos" |
| + |
| +// serveReserved answers a request under ReservedPrefix for the route the |
| +// lookup resolved. Anything not explicitly served is a plain 404 — never |
| +// the upstream's answer. |
| +func (e *Edge) serveReserved(w http.ResponseWriter, r *http.Request, host string, up upstream) { |
| + http.NotFound(w, r) |
| +} |
| +``` |
| +`edge.go` `ServeHTTP`, immediately after the `if !ok { … }` block that ends at line 1271 and before the `var bytes int64` observer wrap: |
| +```go |
| + // The platform's own path space (reserved.go). Placed AFTER the lookup |
| + // (attribution needs up.route, and an unknown host must still backhaul |
| + // or 404 exactly as before) and BEFORE the observing writer, so a beacon |
| + // is neither metered nor compressed nor cached and never reaches the |
| + // activator branches below (spec §3.4, §4). |
| + if strings.HasPrefix(r.URL.Path, ReservedPrefix) { |
| + e.serveReserved(w, r, host, up) |
| + return |
| + } |
| +``` |
| +`lookupUpstreamPath`: after computing `seg` and `raw`, before consulting `up.paths`: |
| +```go |
| + if seg == reservedSegment || raw == reservedSegment { |
| + return up, true // the reserved segment belongs to the hostname's root app, never a prefix target |
| + } |
| +``` |
| +`domains/claims.go` `validPathPrefix`: `return p != "." && p != ".." && p != "__carlos"` with a comment naming the edge's reservation. |
| + |
| +- [ ] **Step 4: Run** `go test ./internal/edge/ ./internal/domains/` — PASS (every existing edge test still passes; none serves `/__carlos/`). |
| +- [ ] **Step 5: Commit** `edge: /__carlos/ is reserved on every route, answered before the observer`. |
| + |
| +--- |
| + |
| +### Task 3: The door's rate limiter |
| + |
| +**Files:** |
| +- Create: `internal/edge/doorlimit.go` |
| +- Test: `internal/edge/doorlimit_test.go` |
| + |
| +**Interfaces:** |
| +- Produces: |
| + ```go |
| + const ( |
| + doorAppPerMinute = 600 |
| + doorVisitorPerMinute = 60 |
| + doorVisitorEntries = 10000 |
| + ) |
| + type doorLimiter struct { mu sync.Mutex; apps map[string]*doorAppState } |
| + func newDoorLimiter() *doorLimiter |
| + // allow spends one token from the app's bucket for door ('e' or 'a') AND |
| + // one from the visitor's bucket; false when either is empty (neither is |
| + // spent then). now is injected for tests. |
| + func (l *doorLimiter) allow(app string, door byte, visitor uint64, now time.Time) bool |
| + ``` |
| + Token bucket: capacity = per-minute figure, refill = capacity/60 per second, continuous (`tokens = min(cap, tokens + elapsed.Seconds()*cap/60)`). Visitor buckets per app live in a `container/list` LRU capped at `doorVisitorEntries`; the oldest is evicted on insert. |
| + |
| +- [ ] **Step 1: Failing test** `doorlimit_test.go`: |
| +```go |
| +package edge |
| + |
| +import ( |
| + "testing" |
| + "time" |
| +) |
| + |
| +func TestDoorLimiterAppBucket(t *testing.T) { |
| + l := newDoorLimiter() |
| + now := time.Unix(1_700_000_000, 0) |
| + for i := 0; i < doorAppPerMinute; i++ { |
| + if !l.allow("a/web", 'e', uint64(i), now) { // a fresh visitor each time: only the app bucket can refuse |
| + t.Fatalf("post %d refused", i+1) |
| + } |
| + } |
| + if l.allow("a/web", 'e', 99999, now) { |
| + t.Fatal("the 601st error in a minute must be refused") |
| + } |
| + if !l.allow("a/web", 'a', 99999, now) { |
| + t.Fatal("the events bucket is separate") |
| + } |
| + if !l.allow("a/other", 'e', 99999, now) { |
| + t.Fatal("another app is unaffected") |
| + } |
| + if !l.allow("a/web", 'e', 99999, now.Add(6*time.Second)) { |
| + t.Fatal("six seconds refill 60 tokens") |
| + } |
| +} |
| + |
| +func TestDoorLimiterVisitorBucketAndLRU(t *testing.T) { |
| + l := newDoorLimiter() |
| + now := time.Unix(1_700_000_000, 0) |
| + for i := 0; i < doorVisitorPerMinute; i++ { |
| + if !l.allow("a/web", 'a', 7, now) { |
| + t.Fatalf("visitor post %d refused", i+1) |
| + } |
| + } |
| + if l.allow("a/web", 'a', 7, now) { |
| + t.Fatal("the 61st post from one visitor must be refused") |
| + } |
| + if !l.allow("a/web", 'a', 8, now) { |
| + t.Fatal("another visitor is unaffected") |
| + } |
| + // Refusal spends nothing: the app bucket still has its tokens. |
| + if got := l.apps["a/web"].bucket['a'].tokens; got < float64(doorAppPerMinute-61) { |
| + t.Fatalf("app tokens %v, refused visitor posts must not spend them", got) |
| + } |
| + for i := uint64(100); i < 100+doorVisitorEntries; i++ { |
| + l.allow("a/web", 'a', i, now) |
| + } |
| + if n := l.apps["a/web"].visitors.Len(); n != doorVisitorEntries { |
| + t.Fatalf("visitor table %d entries, want the cap", n) |
| + } |
| + if _, ok := l.apps["a/web"].visitorIdx[7]; ok { |
| + t.Fatal("the oldest visitor must have been evicted") |
| + } |
| +} |
| +``` |
| + |
| +- [ ] **Step 2: Run** `go test ./internal/edge/ -run DoorLimiter` — FAIL. |
| + |
| +- [ ] **Step 3: Implement** `doorlimit.go`: |
| +```go |
| +package edge |
| + |
| +import ( |
| + "container/list" |
| + "sync" |
| + "time" |
| +) |
| + |
| +// The browser door's rate (spec §3.4): token buckets per app for each |
| +// door and per visitor, all in edge memory, all dropping silently and |
| +// counting dropped.rate. Visitor state is bounded per app with LRU |
| +// eviction so a scan of a million addresses costs a fixed table. |
| +const ( |
| + doorAppPerMinute = 600 |
| + doorVisitorPerMinute = 60 |
| + doorVisitorEntries = 10000 |
| +) |
| + |
| +type tokenBucket struct { |
| + tokens float64 |
| + cap float64 |
| + last time.Time |
| +} |
| + |
| +func (b *tokenBucket) take(now time.Time) bool { |
| + if b.last.IsZero() { |
| + b.tokens, b.last = b.cap, now |
| + } |
| + if el := now.Sub(b.last); el > 0 { |
| + b.tokens += el.Seconds() * b.cap / 60 |
| + if b.tokens > b.cap { |
| + b.tokens = b.cap |
| + } |
| + b.last = now |
| + } |
| + if b.tokens < 1 { |
| + return false |
| + } |
| + b.tokens-- |
| + return true |
| +} |
| + |
| +func (b *tokenBucket) has(now time.Time) bool { // peek: refill, but spend nothing |
| + if b.last.IsZero() { |
| + b.tokens, b.last = b.cap, now |
| + } |
| + if el := now.Sub(b.last); el > 0 { |
| + b.tokens += el.Seconds() * b.cap / 60 |
| + if b.tokens > b.cap { |
| + b.tokens = b.cap |
| + } |
| + b.last = now |
| + } |
| + return b.tokens >= 1 |
| +} |
| + |
| +type visitorEntry struct { |
| + key uint64 |
| + b tokenBucket |
| +} |
| + |
| +type doorAppState struct { |
| + bucket map[byte]*tokenBucket // 'e', 'a' |
| + visitors *list.List // front = most recent; *visitorEntry values |
| + visitorIdx map[uint64]*list.Element |
| +} |
| + |
| +type doorLimiter struct { |
| + mu sync.Mutex |
| + apps map[string]*doorAppState |
| +} |
| + |
| +func newDoorLimiter() *doorLimiter { return &doorLimiter{apps: map[string]*doorAppState{}} } |
| + |
| +// allow spends one token from the app's door bucket and one from the |
| +// visitor's bucket; when either is empty nothing is spent and the post is |
| +// refused. Both are checked before either is charged. |
| +func (l *doorLimiter) allow(app string, door byte, visitor uint64, now time.Time) bool { |
| + l.mu.Lock() |
| + defer l.mu.Unlock() |
| + st := l.apps[app] |
| + if st == nil { |
| + st = &doorAppState{ |
| + bucket: map[byte]*tokenBucket{'e': {cap: doorAppPerMinute}, 'a': {cap: doorAppPerMinute}}, |
| + visitors: list.New(), |
| + visitorIdx: map[uint64]*list.Element{}, |
| + } |
| + l.apps[app] = st |
| + } |
| + ab := st.bucket[door] |
| + if ab == nil { |
| + return false |
| + } |
| + el := st.visitorIdx[visitor] |
| + if el == nil { |
| + if st.visitors.Len() >= doorVisitorEntries { |
| + old := st.visitors.Back() |
| + st.visitors.Remove(old) |
| + delete(st.visitorIdx, old.Value.(*visitorEntry).key) |
| + } |
| + el = st.visitors.PushFront(&visitorEntry{key: visitor, b: tokenBucket{cap: doorVisitorPerMinute}}) |
| + st.visitorIdx[visitor] = el |
| + } else { |
| + st.visitors.MoveToFront(el) |
| + } |
| + vb := &el.Value.(*visitorEntry).b |
| + if !ab.has(now) || !vb.has(now) { |
| + return false |
| + } |
| + return ab.take(now) && vb.take(now) |
| +} |
| +``` |
| + |
| +- [ ] **Step 4: Run** `go test ./internal/edge/ -run DoorLimiter -race` — PASS. |
| +- [ ] **Step 5: Commit** `edge: the browser door's token buckets, per app, per door, per visitor`. |
| + |
| +--- |
| + |
| +### Task 4: `carlos.js`, vendored, hashed, served |
| + |
| +**Files:** |
| +- Create: `internal/edge/carlosjs.go` |
| +- Modify: `internal/edge/reserved.go` (dispatch the two GETs) |
| +- Test: `internal/edge/carlosjs_test.go` |
| + |
| +**Interfaces:** |
| +- Produces: |
| + ```go |
| + const carlosJS = `…` // the snippet source (below) |
| + var carlosJSHash string // 12 hex chars of sha256(carlosJS), computed in init |
| + func carlosJSAssetPath() string // "/__carlos/carlos." + carlosJSHash + ".js" |
| + func serveCarlosJS(w http.ResponseWriter, r *http.Request) bool // true when it answered (loader, asset, or 404 for a stale hash) |
| + ``` |
| + `serveReserved` becomes: `if r.Method == GET|HEAD && strings.HasPrefix(path, "/__carlos/carlos.") → serveCarlosJS`, else 404 (Task 7 adds the POSTs). |
| + |
| +- [ ] **Step 1: Failing test** `carlosjs_test.go`: |
| +```go |
| +package edge |
| + |
| +import ( |
| + "net/http" |
| + "net/http/httptest" |
| + "strings" |
| + "testing" |
| +) |
| + |
| +func TestCarlosJSLoaderAndAsset(t *testing.T) { |
| + e := &Edge{} |
| + get := func(p string, hdr map[string]string) *httptest.ResponseRecorder { |
| + rec := httptest.NewRecorder() |
| + req := httptest.NewRequest("GET", "http://app.example.com"+p, nil) |
| + for k, v := range hdr { |
| + req.Header.Set(k, v) |
| + } |
| + e.serveReserved(rec, req, "app.example.com", upstream{}) |
| + return rec |
| + } |
| + rec := get("/__carlos/carlos.js", nil) |
| + if rec.Code != 200 || rec.Header().Get("Cache-Control") != "max-age=300, must-revalidate" { |
| + t.Fatalf("loader: %d %q", rec.Code, rec.Header().Get("Cache-Control")) |
| + } |
| + if !strings.Contains(rec.Body.String(), carlosJSAssetPath()) { |
| + t.Fatal("loader must reference the hashed asset") |
| + } |
| + if ct := rec.Header().Get("Content-Type"); !strings.HasPrefix(ct, "text/javascript") { |
| + t.Fatalf("content-type %q", ct) |
| + } |
| + etag := rec.Header().Get("ETag") |
| + if etag == "" { |
| + t.Fatal("loader carries an ETag so revalidation is a 304") |
| + } |
| + if rec := get("/__carlos/carlos.js", map[string]string{"If-None-Match": etag}); rec.Code != 304 { |
| + t.Fatalf("revalidate: %d", rec.Code) |
| + } |
| + rec = get(carlosJSAssetPath(), nil) |
| + if rec.Code != 200 || rec.Header().Get("Cache-Control") != "public, max-age=31536000, immutable" { |
| + t.Fatalf("asset: %d %q", rec.Code, rec.Header().Get("Cache-Control")) |
| + } |
| + if rec.Body.String() != carlosJS { |
| + t.Fatal("asset body is the snippet verbatim") |
| + } |
| + if rec := get("/__carlos/carlos.deadbeef0000.js", nil); rec.Code != 404 { |
| + t.Fatalf("stale hash: %d", rec.Code) |
| + } |
| + if len(carlosJSHash) != 12 { |
| + t.Fatalf("hash %q", carlosJSHash) |
| + } |
| +} |
| + |
| +// The snippet's contract (spec §3.4), pinned as strings: handlers for both |
| +// error events, sendBeacon for events, no pageview send in edge mode. |
| +func TestCarlosJSContract(t *testing.T) { |
| + for _, want := range []string{"'error'", "'unhandledrejection'", "navigator.sendBeacon", "/__carlos/e", "/__carlos/a", "window.carlos", "track:", "<email>", "<redacted>", "<hex>", "location.pathname"} { |
| + if !strings.Contains(carlosJS, want) { |
| + t.Errorf("snippet lacks %s", want) |
| + } |
| + } |
| + if strings.Contains(carlosJS, `"pageview"`) || strings.Contains(carlosJS, `'pageview'`) { |
| + t.Error("the snippet must not send pageviews in edge mode") |
| + } |
| +} |
| +``` |
| + |
| +- [ ] **Step 2: Run** `go test ./internal/edge/ -run CarlosJS` — FAIL. |
| + |
| +- [ ] **Step 3: Implement** `carlosjs.go`: |
| +```go |
| +package edge |
| + |
| +import ( |
| + "crypto/sha256" |
| + "encoding/hex" |
| + "net/http" |
| + "strings" |
| +) |
| + |
| +// carlos.js is vendored here the way the console vendors its icons: in |
| +// code, never a CDN (spec §3.4). The page includes ONE line — |
| +// |
| +// <script src="/__carlos/carlos.js" defer></script> |
| +// |
| +// — the loader, revalidated every five minutes, which appends the hashed |
| +// asset below; the asset is immutable for a year because its URL changes |
| +// with its content. Edge mode: window.onerror and unhandledrejection post |
| +// to /__carlos/e after the §3.1 sanitisation floor; carlos.track(name, |
| +// fields) posts to /__carlos/a via sendBeacon. No pageview is ever sent — |
| +// the edge counted the page when it served the HTML. |
| +const carlosJS = `(function(){ |
| +if(window.carlos&&window.carlos.__v)return; |
| +var E='/__carlos/e',A='/__carlos/a'; |
| +function san(s){s=String(s==null?'':s);if(s.length>8192)s=s.slice(0,8192); |
| +s=s.replace(/[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}/g,'<email>'); |
| +s=s.replace(/\b(bearer|basic)\s+[A-Za-z0-9._~+\/=-]+/gi,'$1 <redacted>'); |
| +s=s.replace(/\b(token|key|secret|password|passwd|pwd|api_key|apikey|access_token|auth)("?\s*[=:]\s*"?)[^\s&"',;]+/gi,'$1$2<redacted>'); |
| +s=s.replace(/\b[A-Fa-f0-9]{32,}\b/g,'<hex>'); |
| +s=s.replace(/\b(?=[A-Za-z0-9+\/]*\d)(?=[A-Za-z0-9+\/]*[A-Z])(?=[A-Za-z0-9+\/]*[a-z])[A-Za-z0-9+\/]{32,}={0,2}/g,'<hex>'); |
| +return s;} |
| +function send(u,b){try{var s=JSON.stringify(b); |
| +if(navigator.sendBeacon){navigator.sendBeacon(u,new Blob([s],{type:'application/json'}));} |
| +else{fetch(u,{method:'POST',body:s,headers:{'Content-Type':'application/json'},keepalive:true,credentials:'omit'});}}catch(e){}} |
| +function fields(f){var o={},n=0;if(f&&typeof f==='object'){for(var k in f){if(!Object.prototype.hasOwnProperty.call(f,k))continue;if(n++>=32)break;var v=f[k];if(v==null)continue; |
| +o[String(k).slice(0,64)]=san(typeof v==='object'?JSON.stringify(v):v).slice(0,256);}}return o;} |
| +function err(kind,name,stack,f){send(E,{kind:String(kind||'Error').slice(0,64),name:san(name).slice(0,1024),stack:san(stack||'').slice(0,8192),path:location.pathname,fields:fields(f)});} |
| +window.addEventListener('error',function(ev){var e=ev.error;err(e&&e.name||'Error',ev.message||(e&&e.message)||'error',e&&e.stack||'',{file:ev.filename||'',line:String(ev.lineno||'')});}); |
| +window.addEventListener('unhandledrejection',function(ev){var r=ev.reason;var m=r&&r.message||(typeof r==='string'?r:'unhandled rejection');err(r&&r.name||'UnhandledRejection',m,r&&r.stack||'',{});}); |
| +window.carlos={__v:1, |
| +track:function(name,f){send(A,{name:String(name||'').slice(0,64),path:location.pathname,fields:fields(f)});}, |
| +error:function(e,f){err(e&&e.name||'Error',e&&e.message||String(e),e&&e.stack||'',f);}}; |
| +})(); |
| +` |
| + |
| +var carlosJSHash string |
| + |
| +func init() { |
| + sum := sha256.Sum256([]byte(carlosJS)) |
| + carlosJSHash = hex.EncodeToString(sum[:])[:12] |
| +} |
| + |
| +func carlosJSAssetPath() string { return ReservedPrefix + "carlos." + carlosJSHash + ".js" } |
| + |
| +// serveCarlosJS answers the loader and the hashed asset. It reports false |
| +// for any other path so serveReserved can 404 it. HEAD is served like GET |
| +// (the stdlib drops the body). |
| +func serveCarlosJS(w http.ResponseWriter, r *http.Request) bool { |
| + if r.Method != http.MethodGet && r.Method != http.MethodHead { |
| + return false |
| + } |
| + switch r.URL.Path { |
| + case ReservedPrefix + "carlos.js": |
| + etag := `"` + carlosJSHash + `"` |
| + w.Header().Set("Content-Type", "text/javascript; charset=utf-8") |
| + w.Header().Set("Cache-Control", "max-age=300, must-revalidate") |
| + w.Header().Set("ETag", etag) |
| + w.Header().Set("X-Content-Type-Options", "nosniff") |
| + if strings.Contains(r.Header.Get("If-None-Match"), carlosJSHash) { |
| + w.WriteHeader(http.StatusNotModified) |
| + return true |
| + } |
| + w.Write([]byte("(function(){var s=document.createElement('script');s.src='" + carlosJSAssetPath() + "';s.async=true;document.head.appendChild(s);})();\n")) |
| + return true |
| + case carlosJSAssetPath(): |
| + w.Header().Set("Content-Type", "text/javascript; charset=utf-8") |
| + w.Header().Set("Cache-Control", "public, max-age=31536000, immutable") |
| + w.Header().Set("X-Content-Type-Options", "nosniff") |
| + w.Write([]byte(carlosJS)) |
| + return true |
| + } |
| + if strings.HasPrefix(r.URL.Path, ReservedPrefix+"carlos.") { |
| + http.NotFound(w, r) // a stale hash from an old loader |
| + return true |
| + } |
| + return false |
| +} |
| +``` |
| +`reserved.go` `serveReserved`: `if serveCarlosJS(w, r) { return }` before the 404. |
| + |
| +- [ ] **Step 4: Run** `go test ./internal/edge/ -run 'CarlosJS|Reserved'` — PASS. Also paste the snippet into `node -e` if node is installed (`node -e "$(sed -n '/^const carlosJS = \`/,/^\`$/p' internal/edge/carlosjs.go | sed '1s/.*`//;$d')" 2>&1 | head -3` — a ReferenceError for `window` is fine, a SyntaxError is not); if node is absent, say so in the commit. |
| +- [ ] **Step 5: Commit** `edge: carlos.js vendored, a revalidating loader and an immutable hashed asset`. |
| + |
| +--- |
| + |
| +### Task 5: Beacon shapes and the Origin rule |
| + |
| +**Files:** |
| +- Create: `internal/edge/browserdoor.go` (parsing and origin only; the handler is Task 7) |
| +- Test: `internal/edge/browserdoor_test.go` |
| + |
| +**Interfaces:** |
| +- Consumes: `lookupIn`, `upstream`, `apperr.Record`, `apperr.MaxStack`. |
| +- Produces: |
| + ```go |
| + const ( |
| + maxBeaconBody = 16 << 10 |
| + maxBeaconKind = 64 |
| + maxBeaconName = 1024 |
| + maxBeaconFields = 32 |
| + maxBeaconField = 256 |
| + ) |
| + type errorBeacon struct { Kind, Name, Stack, Path, FP string; Fields map[string]string } |
| + type eventBeacon struct { Name, Path string } |
| + // parseErrorBeacon decodes and caps; false when not a JSON object or name empty. |
| + func parseErrorBeacon(b []byte) (errorBeacon, bool) |
| + // parseEventBeacon decodes; false when not a JSON object or name empty. fields are accepted and dropped (§5). |
| + func parseEventBeacon(b []byte) (eventBeacon, bool) |
| + // beaconOriginOK applies §3.4: Sec-Fetch-Site cross-site → false; Origin present → must be https and resolve |
| + // via lookupIn(m, host) to route.Host == want; Origin absent → Referer's host through the same rule; neither → false. |
| + func beaconOriginOK(m map[string]upstream, r *http.Request, want string) bool |
| + ``` |
| + |
| +- [ ] **Step 1: Failing tests** `browserdoor_test.go`: |
| +```go |
| +package edge |
| + |
| +import ( |
| + "net/http" |
| + "net/http/httptest" |
| + "strings" |
| + "testing" |
| + |
| + "github.com/carlosframework/platform/internal/registry" |
| +) |
| + |
| +func TestParseErrorBeaconCaps(t *testing.T) { |
| + long := strings.Repeat("x", 9000) |
| + b, ok := parseErrorBeacon([]byte(`{"kind":"` + long + `","name":"boom","stack":"` + long + `","path":"/p?q=1","fp":"abc","fields":{"a":"` + long + `"}}`)) |
| + if !ok || len(b.Kind) != maxBeaconKind || len(b.Stack) != 8<<10 || len(b.Fields["a"]) != maxBeaconField || b.Path != "/p?q=1" { |
| + t.Fatalf("caps: %+v ok=%v", b, ok) |
| + } |
| + if _, ok := parseErrorBeacon([]byte(`{"kind":"E"}`)); ok { |
| + t.Fatal("a beacon without a name is malformed") |
| + } |
| + if _, ok := parseErrorBeacon([]byte(`[1,2]`)); ok { |
| + t.Fatal("not an object") |
| + } |
| + many := "{" |
| + for i := 0; i < 40; i++ { |
| + many += `"k` + string(rune('a'+i%26)) + string(rune('0'+i/26)) + `":"v",` |
| + } |
| + b, _ = parseErrorBeacon([]byte(`{"name":"n","fields":` + many + `"z":"v"}}`)) |
| + if len(b.Fields) != maxBeaconFields { |
| + t.Fatalf("fields capped to %d, got %d", maxBeaconFields, len(b.Fields)) |
| + } |
| +} |
| + |
| +func TestParseEventBeacon(t *testing.T) { |
| + b, ok := parseEventBeacon([]byte(`{"name":"Signup","path":"/x","fields":{"plan":"pro"}}`)) |
| + if !ok || b.Name != "Signup" || b.Path != "/x" { |
| + t.Fatalf("%+v %v", b, ok) |
| + } |
| + if _, ok := parseEventBeacon([]byte(`{"path":"/x"}`)); ok { |
| + t.Fatal("no name") |
| + } |
| +} |
| + |
| +func TestBeaconOriginRule(t *testing.T) { |
| + m := map[string]upstream{ |
| + "app.example.com": {route: registry.Route{Host: "app.example.com"}}, |
| + "www.example.com": {route: registry.Route{Host: "app.example.com"}, alias: true}, |
| + "other.example.com": {route: registry.Route{Host: "other.example.com"}}, |
| + } |
| + req := func(hdr map[string]string) *http.Request { |
| + r := httptest.NewRequest("POST", "http://app.example.com/__carlos/e", nil) |
| + for k, v := range hdr { |
| + r.Header.Set(k, v) |
| + } |
| + return r |
| + } |
| + cases := []struct { |
| + name string |
| + hdr map[string]string |
| + ok bool |
| + }{ |
| + {"origin matches", map[string]string{"Origin": "https://app.example.com"}, true}, |
| + {"origin with port", map[string]string{"Origin": "https://app.example.com:443"}, true}, |
| + {"alias origin shares the route", map[string]string{"Origin": "https://www.example.com"}, true}, |
| + {"http origin", map[string]string{"Origin": "http://app.example.com"}, false}, |
| + {"foreign origin", map[string]string{"Origin": "https://other.example.com"}, false}, |
| + {"unknown origin", map[string]string{"Origin": "https://evil.example.net"}, false}, |
| + {"null origin", map[string]string{"Origin": "null"}, false}, |
| + {"foreign origin, matching referer", map[string]string{"Origin": "https://other.example.com", "Referer": "https://app.example.com/page"}, false}, |
| + {"no origin, matching referer", map[string]string{"Referer": "https://app.example.com/page"}, true}, |
| + {"no origin, foreign referer", map[string]string{"Referer": "https://other.example.com/"}, false}, |
| + {"neither", map[string]string{}, false}, |
| + {"cross-site", map[string]string{"Origin": "https://app.example.com", "Sec-Fetch-Site": "cross-site"}, false}, |
| + {"same-origin fetch site", map[string]string{"Origin": "https://app.example.com", "Sec-Fetch-Site": "same-origin"}, true}, |
| + } |
| + for _, c := range cases { |
| + if got := beaconOriginOK(m, req(c.hdr), "app.example.com"); got != c.ok { |
| + t.Errorf("%s: got %v want %v", c.name, got, c.ok) |
| + } |
| + } |
| +} |
| +``` |
| + |
| +- [ ] **Step 2: Run** `go test ./internal/edge/ -run 'Beacon'` — FAIL. |
| + |
| +- [ ] **Step 3: Implement** `browserdoor.go`: |
| +```go |
| +package edge |
| + |
| +import ( |
| + "encoding/json" |
| + "net/http" |
| + "net/url" |
| + "sort" |
| + "strings" |
| + |
| + "github.com/carlosframework/platform/internal/apperr" |
| +) |
| + |
| +// The browser door (spec 2026-09-07 §3.4): what a page may post, how much, |
| +// and who may post it. Parsing caps everything before the sink sees it; |
| +// the sink sanitises on arrival regardless of what the snippet did. |
| +const ( |
| + maxBeaconBody = 16 << 10 |
| + maxBeaconKind = 64 |
| + maxBeaconName = 1024 |
| + maxBeaconFields = 32 |
| + maxBeaconField = 256 |
| +) |
| + |
| +type errorBeacon struct { |
| + Kind string `json:"kind"` |
| + Name string `json:"name"` |
| + Stack string `json:"stack"` |
| + Path string `json:"path"` |
| + FP string `json:"fp"` |
| + Fields map[string]string `json:"fields"` |
| +} |
| + |
| +type eventBeacon struct { |
| + Name string `json:"name"` |
| + Path string `json:"path"` |
| +} |
| + |
| +func capStr(s string, n int) string { |
| + if len(s) > n { |
| + return s[:n] |
| + } |
| + return s |
| +} |
| + |
| +func parseErrorBeacon(b []byte) (errorBeacon, bool) { |
| + var e errorBeacon |
| + if err := json.Unmarshal(b, &e); err != nil || e.Name == "" { |
| + return errorBeacon{}, false |
| + } |
| + e.Kind = capStr(e.Kind, maxBeaconKind) |
| + e.Name = capStr(e.Name, maxBeaconName) |
| + e.Stack = capStr(e.Stack, apperr.MaxStack) |
| + e.FP = capStr(e.FP, 64) |
| + if len(e.Fields) > 0 { |
| + keys := make([]string, 0, len(e.Fields)) |
| + for k := range e.Fields { |
| + keys = append(keys, k) |
| + } |
| + sort.Strings(keys) // deterministic under the cap |
| + f := make(map[string]string, maxBeaconFields) |
| + for i, k := range keys { |
| + if i >= maxBeaconFields { |
| + break |
| + } |
| + f[capStr(k, 64)] = capStr(e.Fields[k], maxBeaconField) |
| + } |
| + e.Fields = f |
| + } |
| + return e, true |
| +} |
| + |
| +func parseEventBeacon(b []byte) (eventBeacon, bool) { |
| + var e eventBeacon |
| + if err := json.Unmarshal(b, &e); err != nil || e.Name == "" { |
| + return eventBeacon{}, false |
| + } |
| + return e, true |
| +} |
| + |
| +// beaconOriginOK is identification, not authentication (§3.4): the page |
| +// that posts must be one this hostname's route serves. Origin wins when |
| +// present — a bad Origin is refused even if Referer would pass — and |
| +// Referer is consulted only when Origin is absent. Both must be https and |
| +// must resolve through the SAME map lookup the request took to an entry |
| +// whose route.Host equals want; aliases share the primary's route verbatim |
| +// and so pass with no registry call. |
| +func beaconOriginOK(m map[string]upstream, r *http.Request, want string) bool { |
| + if strings.EqualFold(r.Header.Get("Sec-Fetch-Site"), "cross-site") { |
| + return false |
| + } |
| + src := r.Header.Get("Origin") |
| + if src == "" { |
| + src = r.Header.Get("Referer") |
| + } |
| + if src == "" { |
| + return false |
| + } |
| + u, err := url.Parse(src) |
| + if err != nil || u.Scheme != "https" || u.Hostname() == "" { |
| + return false |
| + } |
| + up, ok := lookupIn(m, strings.ToLower(u.Hostname())) |
| + return ok && up.route.Host == want |
| +} |
| +``` |
| + |
| +- [ ] **Step 4: Run** `go test ./internal/edge/ -run Beacon` — PASS. |
| +- [ ] **Step 5: Commit** `edge: beacon shapes with their caps, and the Origin rule`. |
| + |
| +--- |
| + |
| +### Task 6: The meter records drops; the edge's meter contract grows |
| + |
| +**Files:** |
| +- Modify: `internal/analytics/meter.go` (after `Track`) |
| +- Modify: `internal/edge/analytics.go:19-21` (`AnalyticsMeter`) |
| +- Test: `internal/analytics/meter_test.go`, `internal/edge/analytics_test.go` (`memAnalytics`) |
| + |
| +**Interfaces:** |
| +- Produces: |
| + ```go |
| + // package analytics |
| + const ( |
| + DoorErrors = 'e'; DoorEvents = 'a' |
| + DropRate = "rate"; DropSize = "size"; DropOrigin = "origin"; DropMalformed = "malformed" |
| + ) |
| + // Drop counts one refused beacon in the app's current-day Dropped block. |
| + func (m *Meter) Drop(acct, app string, door byte, reason string, when time.Time) |
| + // package edge |
| + type AnalyticsMeter interface { |
| + Observe(analytics.Obs) |
| + Track(acct, app, name string, when time.Time) |
| + Drop(acct, app string, door byte, reason string, when time.Time) |
| + } |
| + ``` |
| + |
| +- [ ] **Step 1: Failing tests** |
| + |
| +`meter_test.go` — add: |
| +```go |
| +func TestMeterDropCountsByDoorAndReason(t *testing.T) { |
| + now := time.Date(2026, 9, 7, 10, 0, 0, 0, time.UTC) |
| + m := NewMeter(t.TempDir(), "box1", func() time.Time { return now }) |
| + m.Drop("acct", "web", DoorErrors, DropRate, now) |
| + m.Drop("acct", "web", DoorErrors, DropSize, now) |
| + m.Drop("acct", "web", DoorEvents, DropOrigin, now) |
| + m.Drop("acct", "web", DoorEvents, DropMalformed, now) |
| + m.Drop("acct", "web", DoorEvents, DropMalformed, now) |
| + m.Drop("acct", "web", 'x', DropRate, now) // unknown door: ignored, never a panic |
| + ds := m.Drain()[AppKey{Acct: "acct", App: "web"}] |
| + if len(ds) != 1 { |
| + t.Fatalf("deltas %d", len(ds)) |
| + } |
| + d := ds[0].Dropped |
| + if d.E.Rate != 1 || d.E.Size != 1 || d.A.Origin != 1 || d.A.Malformed != 2 || d.E.Origin != 0 { |
| + t.Fatalf("dropped %+v", d) |
| + } |
| +} |
| +``` |
| +`edge/analytics_test.go` — `memAnalytics` gains: |
| +```go |
| +type trackCall struct{ Acct, App, Name string } |
| +type dropCall struct{ Acct, App string; Door byte; Reason string } |
| +// fields: tracks []trackCall; drops []dropCall (under mu) |
| +func (m *memAnalytics) Track(acct, app, name string, _ time.Time) { m.mu.Lock(); m.tracks = append(m.tracks, trackCall{acct, app, name}); m.mu.Unlock() } |
| +func (m *memAnalytics) Drop(acct, app string, door byte, reason string, _ time.Time) { m.mu.Lock(); m.drops = append(m.drops, dropCall{acct, app, door, reason}); m.mu.Unlock() } |
| +``` |
| +(with `snapTracks()`/`snapDrops()` accessors mirroring `snap()`). |
| + |
| +- [ ] **Step 2: Run** `go test ./internal/analytics/ -run Drop ./internal/edge/` — FAIL (interface unsatisfied / undefined). |
| + |
| +- [ ] **Step 3: Implement** in `meter.go`: |
| +```go |
| +// Doors and drop reasons (spec §3.4): the browser door names them when it |
| +// refuses a beacon; the day file's Dropped block carries the counts. |
| +const ( |
| + DoorErrors = 'e' |
| + DoorEvents = 'a' |
| + |
| + DropRate = "rate" |
| + DropSize = "size" |
| + DropOrigin = "origin" |
| + DropMalformed = "malformed" |
| +) |
| + |
| +// Drop counts one refused beacon for the app's current day. An unknown |
| +// door or reason counts nothing: the caller is this package's own door |
| +// constants, and a typo must not panic the serving path. |
| +func (m *Meter) Drop(acct, app string, door byte, reason string, when time.Time) { |
| + m.mu.Lock() |
| + defer m.mu.Unlock() |
| + d := m.delta(AppKey{Acct: acct, App: app}, usage.DayOf(when)) |
| + var c *DropCounts |
| + switch door { |
| + case DoorErrors: |
| + c = &d.Dropped.E |
| + case DoorEvents: |
| + c = &d.Dropped.A |
| + default: |
| + return |
| + } |
| + switch reason { |
| + case DropRate: |
| + c.Rate++ |
| + case DropSize: |
| + c.Size++ |
| + case DropOrigin: |
| + c.Origin++ |
| + case DropMalformed: |
| + c.Malformed++ |
| + } |
| +} |
| +``` |
| +`edge/analytics.go`: widen the interface as above (doc: "analytics.Meter satisfies it; the door calls Track and Drop, the observer calls Observe"). |
| + |
| +- [ ] **Step 4: Run** `go test ./internal/analytics/ ./internal/edge/ ./internal/host/` — PASS. |
| +- [ ] **Step 5: Commit** `analytics: the meter records door drops by door and reason`. |
| + |
| +--- |
| + |
| +### Task 7: The door handler |
| + |
| +**Files:** |
| +- Modify: `internal/edge/browserdoor.go` (add the handler) |
| +- Modify: `internal/edge/reserved.go` (dispatch the POSTs) |
| +- Modify: `internal/edge/edge.go` (Edge struct: `doorLimit *doorLimiter`; `New` sets `doorLimit: newDoorLimiter()`) |
| +- Test: `internal/edge/browserdoor_test.go` (append) |
| + |
| +**Interfaces:** |
| +- Consumes: Tasks 1–6. |
| +- Produces: `func (e *Edge) serveBeacon(w http.ResponseWriter, r *http.Request, host string, up upstream, door byte)`. |
| + |
| +The flow, in this order, every branch ending in `202` with an empty body: |
| +1. `up.route.Kind == "status" || up.route.App == ""` → 202, nothing (no app to attribute). |
| +2. `!beaconOriginOK(e.current(), r, up.route.Host)` → drop `origin` against `up`'s app → 202. |
| +3. Body: `io.ReadAll(io.LimitReader(r.Body, maxBeaconBody+1))`; `len > maxBeaconBody` → drop `size` against `up`'s app. |
| +4. Parse (`parseErrorBeacon` / `parseEventBeacon`) → failure is drop `malformed` against `up`'s app. |
| +5. Attribute: `tu := up`; if the beacon's `Path` is non-empty, `tu, _ = e.lookupUpstreamPath(host, beacon.Path)` (the raw path, §3.4). `acct, app := release.RouteAccount(tu.route), tu.route.App`. |
| +6. Switches `sw := e.switchesFor(tu.route)`: door `e` requires `sw.Door`; door `a` requires `sw.Door && sw.Count`; otherwise 202, nothing recorded. |
| +7. Rate: `visitor := fnv64(clientIP(r) + "\x00" + r.Header.Get("User-Agent"))` (hash/fnv 64a); `!e.doorLimit.allow(acct+"/"+app, door, visitor, now)` → drop `rate`. |
| +8. `e`: `errSink.Add(acct, app, apperr.Record{Host: tu.route.Host, Source: apperr.SourceBrowser, Kind: b.Kind, Name: b.Name, Stack: b.Stack, Path: b.Path, Fields: b.Fields, FP: b.FP, Release: tu.route.Version})` (FPV: set to `apperr.FPVersion` only when `b.FP != ""`; the sink stamps it otherwise). `a`: `name == "pageview"` → 202, nothing (edge mode); else `meter.Track(acct, app, b.Name, now)`. |
| + A drop is recorded via `meter.Drop` ONLY when `e.switchesFor(<the charged route>).Count` is true (ruling 2); a nil meter or nil sink is a no-op. |
| +9. `w.WriteHeader(http.StatusAccepted)`. |
| + |
| +- [ ] **Step 1: Failing tests** — append to `browserdoor_test.go`: |
| +```go |
| +// doorFixture: one counted, door-open app on app.example.com with a fake |
| +// upstream that must never be reached, a memory sink and a memory meter. |
| +type memSink struct { |
| + mu sync.Mutex |
| + recs []struct{ Acct, App string; R apperr.Record } |
| +} |
| + |
| +func (s *memSink) Add(acct, app string, r apperr.Record) { |
| + s.mu.Lock() |
| + s.recs = append(s.recs, struct{ Acct, App string; R apperr.Record }{acct, app, r}) |
| + s.mu.Unlock() |
| +} |
| + |
| +func doorFixture(t *testing.T) (*Edge, *httptest.Server, *memSink, *memAnalytics) { |
| + t.Helper() |
| + backend := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { t.Errorf("upstream reached: %s", r.URL.Path) })) |
| + t.Cleanup(backend.Close) |
| + reg := openReg(t) |
| + if err := reg.Upsert(registry.Route{Host: "app.example.com", Kind: "service", Addr: backend.Listener.Addr().String(), Status: "ready", App: "web", Version: "v7"}); err != nil { |
| + t.Fatal(err) |
| + } |
| + e, err := New(reg, nil) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + sink, meter := &memSink{}, &memAnalytics{} |
| + e.EnableErrors(sink) |
| + e.EnableAnalytics(meter) |
| + e.SetSwitches(map[string]telemetry.Switches{release.OpsAccount + "/web": {Count: true, Door: true}}) |
| + srv := httptest.NewServer(e) |
| + t.Cleanup(srv.Close) |
| + return e, srv, sink, meter |
| +} |
| + |
| +func postBeacon(t *testing.T, srv *httptest.Server, path, body string, hdr map[string]string) int { |
| + t.Helper() |
| + req, _ := http.NewRequest("POST", srv.URL+path, strings.NewReader(body)) |
| + req.Host = "app.example.com" |
| + req.Header.Set("Origin", "https://app.example.com") |
| + for k, v := range hdr { |
| + if v == "" { |
| + req.Header.Del(k) |
| + } else { |
| + req.Header.Set(k, v) |
| + } |
| + } |
| + resp, err := http.DefaultClient.Do(req) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + b, _ := io.ReadAll(resp.Body) |
| + resp.Body.Close() |
| + if len(b) != 0 { |
| + t.Fatalf("beacon answered with a body: %q", b) |
| + } |
| + return resp.StatusCode |
| +} |
| + |
| +func TestDoorAcceptsAndAttributes(t *testing.T) { |
| + _, srv, sink, meter := doorFixture(t) |
| + if st := postBeacon(t, srv, "/__carlos/e", `{"kind":"TypeError","name":"x is not a function","stack":"at a (app.js:1:2)","path":"/checkout?step=2","fields":{"user":"a@b.co"}}`, nil); st != 202 { |
| + t.Fatalf("status %d", st) |
| + } |
| + if len(sink.recs) != 1 { |
| + t.Fatalf("records %d", len(sink.recs)) |
| + } |
| + r := sink.recs[0] |
| + if r.Acct != release.OpsAccount || r.App != "web" || r.R.Source != apperr.SourceBrowser || r.R.Kind != "TypeError" || r.R.Host != "app.example.com" || r.R.Release != "v7" || r.R.Path != "/checkout?step=2" { |
| + t.Fatalf("record %+v", r) |
| + } |
| + if st := postBeacon(t, srv, "/__carlos/a", `{"name":"signup","path":"/"}`, nil); st != 202 { |
| + t.Fatalf("status %d", st) |
| + } |
| + if tr := meter.snapTracks(); len(tr) != 1 || tr[0].Name != "signup" || tr[0].App != "web" { |
| + t.Fatalf("tracks %+v", tr) |
| + } |
| + postBeacon(t, srv, "/__carlos/a", `{"name":"pageview","path":"/"}`, nil) |
| + if len(meter.snapTracks()) != 1 { |
| + t.Fatal("a browser pageview is not counted in edge mode") |
| + } |
| + if len(meter.snapDrops()) != 0 { |
| + t.Fatalf("nothing should be dropped: %+v", meter.snapDrops()) |
| + } |
| +} |
| + |
| +func TestDoorDropsAndAlways202(t *testing.T) { |
| + _, srv, sink, meter := doorFixture(t) |
| + cases := []struct { |
| + name, path, body string |
| + hdr map[string]string |
| + door byte |
| + reason string |
| + }{ |
| + {"foreign origin", "/__carlos/e", `{"name":"n"}`, map[string]string{"Origin": "https://evil.example.net"}, 'e', "origin"}, |
| + {"foreign origin, matching referer", "/__carlos/e", `{"name":"n"}`, map[string]string{"Origin": "https://evil.example.net", "Referer": "https://app.example.com/"}, 'e', "origin"}, |
| + {"cross-site", "/__carlos/a", `{"name":"n"}`, map[string]string{"Sec-Fetch-Site": "cross-site"}, 'a', "origin"}, |
| + {"no origin no referer", "/__carlos/a", `{"name":"n"}`, map[string]string{"Origin": ""}, 'a', "origin"}, |
| + {"17 KB", "/__carlos/e", `{"name":"` + strings.Repeat("x", 17<<10) + `"}`, nil, 'e', "size"}, |
| + {"not json", "/__carlos/e", `nope`, nil, 'e', "malformed"}, |
| + {"no name", "/__carlos/a", `{"path":"/"}`, nil, 'a', "malformed"}, |
| + } |
| + for _, c := range cases { |
| + before := len(meter.snapDrops()) |
| + if st := postBeacon(t, srv, c.path, c.body, c.hdr); st != 202 { |
| + t.Errorf("%s: status %d", c.name, st) |
| + } |
| + drops := meter.snapDrops() |
| + if len(drops) != before+1 || drops[before].Door != c.door || drops[before].Reason != c.reason { |
| + t.Errorf("%s: drops %+v", c.name, drops[before:]) |
| + } |
| + } |
| + if len(sink.recs) != 0 || len(meter.snapTracks()) != 0 { |
| + t.Fatal("dropped beacons must not land") |
| + } |
| + // Missing Origin with a matching Referer counts. |
| + if st := postBeacon(t, srv, "/__carlos/a", `{"name":"ok"}`, map[string]string{"Origin": "", "Referer": "https://app.example.com/p"}); st != 202 || len(meter.snapTracks()) != 1 { |
| + t.Fatal("referer fallback") |
| + } |
| +} |
| + |
| +func TestDoorRateLimit(t *testing.T) { |
| + _, srv, sink, meter := doorFixture(t) |
| + for i := 0; i < doorVisitorPerMinute; i++ { |
| + postBeacon(t, srv, "/__carlos/e", `{"name":"n"}`, nil) |
| + } |
| + postBeacon(t, srv, "/__carlos/e", `{"name":"n"}`, nil) |
| + if len(sink.recs) != doorVisitorPerMinute { |
| + t.Fatalf("landed %d", len(sink.recs)) |
| + } |
| + if d := meter.snapDrops(); len(d) != 1 || d[0].Reason != "rate" || d[0].Door != 'e' { |
| + t.Fatalf("drops %+v", d) |
| + } |
| + // The app bucket: 600 across distinct visitors, the 601st dropped. |
| + e, srv2, sink2, meter2 := doorFixture(t) |
| + _ = e |
| + for i := 0; i <= doorAppPerMinute; i++ { |
| + postBeacon(t, srv2, "/__carlos/a", `{"name":"n"}`, map[string]string{"User-Agent": "ua-" + strconv.Itoa(i)}) |
| + } |
| + if len(meter2.snapTracks()) != doorAppPerMinute || len(meter2.snapDrops()) != 1 { |
| + t.Fatalf("tracked %d dropped %d", len(meter2.snapTracks()), len(meter2.snapDrops())) |
| + } |
| + _ = sink2 |
| +} |
| + |
| +func TestDoorSwitches(t *testing.T) { |
| + e, srv, sink, meter := doorFixture(t) |
| + key := release.OpsAccount + "/web" |
| + e.SetSwitches(map[string]telemetry.Switches{key: {Count: false, Door: true}}) |
| + postBeacon(t, srv, "/__carlos/a", `{"name":"n"}`, nil) |
| + postBeacon(t, srv, "/__carlos/e", `{"name":"n"}`, nil) |
| + if len(meter.snapTracks()) != 0 || len(sink.recs) != 1 || len(meter.snapDrops()) != 0 { |
| + t.Fatal("count off: /a dropped silently, /e accepted, nothing recorded as a drop") |
| + } |
| + e.SetSwitches(map[string]telemetry.Switches{key: {Count: true, Door: false}}) |
| + postBeacon(t, srv, "/__carlos/a", `{"name":"n"}`, nil) |
| + postBeacon(t, srv, "/__carlos/e", `{"name":"n"}`, nil) |
| + if len(meter.snapTracks()) != 0 || len(sink.recs) != 1 || len(meter.snapDrops()) != 0 { |
| + t.Fatal("door off: both dropped silently") |
| + } |
| + // Count off: an origin drop is not recorded either (ruling 2). |
| + e.SetSwitches(map[string]telemetry.Switches{key: {Count: false, Door: true}}) |
| + postBeacon(t, srv, "/__carlos/e", `{"name":"n"}`, map[string]string{"Origin": "https://evil.example.net"}) |
| + if len(meter.snapDrops()) != 0 { |
| + t.Fatal("drops are recorded only for a counted app") |
| + } |
| +} |
| + |
| +// On a path-routed hostname the beacon lands on the app owning its path. |
| +func TestDoorAttributesByPathPrefix(t *testing.T) { |
| + e, srv, sink, _ := doorFixture(t) |
| + reg := e.reg |
| + if err := reg.Upsert(registry.Route{Host: "team.internal", Kind: "service", Addr: "127.0.0.1:1", Status: "ready", App: "team"}); err != nil { |
| + t.Fatal(err) |
| + } |
| + // Give app.example.com a prefix map: /team/* → team.internal. Use the |
| + // same seam the prefix-routing tests use (grep "paths:" in edge_test.go |
| + // / claims tests for how a map reaches the live upstream; if only a |
| + // Reload from claims does it, set the claim and Reload). |
| + setPathMap(t, e, "app.example.com", map[string]string{"team": "team.internal"}) |
| + e.SetSwitches(map[string]telemetry.Switches{release.OpsAccount + "/web": {Count: true, Door: true}, release.OpsAccount + "/team": {Count: true, Door: true}}) |
| + postBeacon(t, srv, "/__carlos/e", `{"name":"n","path":"/team/page"}`, nil) |
| + postBeacon(t, srv, "/__carlos/e", `{"name":"n","path":"/other"}`, nil) |
| + postBeacon(t, srv, "/__carlos/e", `{"name":"n"}`, nil) |
| + if len(sink.recs) != 3 || sink.recs[0].App != "team" || sink.recs[1].App != "web" || sink.recs[2].App != "web" { |
| + t.Fatalf("attribution %+v", sink.recs) |
| + } |
| +} |
| +``` |
| +`setPathMap` is a test helper this task writes: find how `upstream.paths` is populated (grep `paths:` and `paths =` in `internal/edge/*.go`) and either drive it through the real path (a claim with `Paths`, then `Reload`) or, if the map build takes a source the tests can substitute, use that. Document the choice in the commit. |
| + |
| +- [ ] **Step 2: Run** `go test ./internal/edge/ -run Door` — FAIL. |
| + |
| +- [ ] **Step 3: Implement** — append to `browserdoor.go`: |
| +```go |
| +// serveBeacon is POST /__carlos/e (door 'e') and /__carlos/a (door 'a'). |
| +// Every branch answers 202 with no body (§3.4): the endpoint reveals |
| +// nothing, and the owner's dropped counters are where refusals show. |
| +func (e *Edge) serveBeacon(w http.ResponseWriter, r *http.Request, host string, up upstream, door byte) { |
| + defer w.WriteHeader(http.StatusAccepted) |
| + if up.route.Kind == "status" || up.route.App == "" { |
| + return |
| + } |
| + now := time.Now() |
| + drop := func(rt registry.Route, reason string) { |
| + if ref := e.analyticsMeter.Load(); ref != nil && e.switchesFor(rt).Count { |
| + ref.Drop(release.RouteAccount(rt), rt.App, door, reason, now) |
| + } |
| + } |
| + if !beaconOriginOK(e.current(), r, up.route.Host) { |
| + drop(up.route, analytics.DropOrigin) |
| + return |
| + } |
| + body, err := io.ReadAll(io.LimitReader(r.Body, maxBeaconBody+1)) |
| + if err != nil || len(body) > maxBeaconBody { |
| + drop(up.route, analytics.DropSize) |
| + return |
| + } |
| + var path string |
| + var eb errorBeacon |
| + var ab eventBeacon |
| + var ok bool |
| + if door == analytics.DoorErrors { |
| + eb, ok = parseErrorBeacon(body) |
| + path = eb.Path |
| + } else { |
| + ab, ok = parseEventBeacon(body) |
| + path = ab.Path |
| + } |
| + if !ok { |
| + drop(up.route, analytics.DropMalformed) |
| + return |
| + } |
| + tu := up |
| + if path != "" { |
| + if t, found := e.lookupUpstreamPath(host, path); found { |
| + tu = t |
| + } |
| + } |
| + sw := e.switchesFor(tu.route) |
| + if !sw.Door || (door == analytics.DoorEvents && !sw.Count) { |
| + return |
| + } |
| + acct, app := release.RouteAccount(tu.route), tu.route.App |
| + h := fnv.New64a() |
| + h.Write([]byte(clientIP(r))) |
| + h.Write([]byte{0}) |
| + h.Write([]byte(r.Header.Get("User-Agent"))) |
| + if !e.doorLimit.allow(acct+"/"+app, door, h.Sum64(), now) { |
| + drop(tu.route, analytics.DropRate) |
| + return |
| + } |
| + if door == analytics.DoorErrors { |
| + ref := e.errSink.Load() |
| + if ref == nil { |
| + return |
| + } |
| + rec := apperr.Record{Host: tu.route.Host, Source: apperr.SourceBrowser, Kind: eb.Kind, Name: eb.Name, Stack: eb.Stack, Path: eb.Path, Fields: eb.Fields, FP: eb.FP, Release: tu.route.Version} |
| + if rec.FP != "" { |
| + rec.FPV = apperr.FPVersion |
| + } |
| + ref.Add(acct, app, rec) |
| + return |
| + } |
| + if ab.Name == "pageview" { |
| + return // edge mode: the HTML response was the pageview (§3.4) |
| + } |
| + if ref := e.analyticsMeter.Load(); ref != nil { |
| + ref.Track(acct, app, ab.Name, now) |
| + } |
| +} |
| +``` |
| +`reserved.go` `serveReserved`: |
| +```go |
| + if r.Method == http.MethodPost { |
| + switch r.URL.Path { |
| + case ReservedPrefix + "e": |
| + e.serveBeacon(w, r, host, up, analytics.DoorErrors) |
| + return |
| + case ReservedPrefix + "a": |
| + e.serveBeacon(w, r, host, up, analytics.DoorEvents) |
| + return |
| + } |
| + } |
| + if serveCarlosJS(w, r) { |
| + return |
| + } |
| + http.NotFound(w, r) |
| +``` |
| +`edge.go`: `doorLimit *doorLimiter` on `Edge`; `New` sets it; a hand-built `&Edge{}` in tests gets it lazily — make `serveBeacon` tolerate nil by calling `e.doorLimiter()` which lazily stores one under `sync.Once` (field `doorOnce sync.Once`). |
| + |
| +- [ ] **Step 4: Run** `go test ./internal/edge/ -race` — PASS, every existing test included. |
| +- [ ] **Step 5: Commit** `edge: the browser door — origin, size, shape, rate, attribution, then the sink or the meter`. |
| + |
| +--- |
| + |
| +### Task 8: A beacon never wakes or keeps awake |
| + |
| +**Files:** |
| +- Test: `internal/activator/activator_test.go` (append) |
| + |
| +**Interfaces:** consumes `edge.New(reg, a)` with the real `*Service` as the `edge.Activator`, `testActivator` (`activator_test.go:350`), `arrangeAwake`, `a.entries[host].lastActive` (the seam `TestServeScheduledDoesNotStampActivity` at `:1370` uses). |
| + |
| +- [ ] **Step 1: Write the test** |
| +```go |
| +// A beacon to a hibernating route is answered by the edge before the |
| +// activator branch: lastActive is untouched and nothing is started |
| +// (spec §4: nothing here wakes an instance). |
| +func TestBrowserBeaconDoesNotStampActivity(t *testing.T) { |
| + const host = "beacon.bab.oncarlos.com" |
| + a, b, clk, reg := testActivator(t, time.Hour) |
| + rt := testRoute(host) |
| + arrangeAwake(t, a, rt) |
| + a.mu.Lock() |
| + before := a.entries[rt.Host].lastActive |
| + a.mu.Unlock() |
| + clk.advance(time.Minute) |
| + _, _, starts0 := b.counts() |
| + |
| + e, err := edge.New(reg, a) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + rec := httptest.NewRecorder() |
| + req := httptest.NewRequest("POST", "https://"+host+"/__carlos/a", strings.NewReader(`{"name":"n"}`)) |
| + req.Host = host |
| + req.Header.Set("Origin", "https://"+host) |
| + e.ServeHTTP(rec, req) |
| + if rec.Code != http.StatusAccepted { |
| + t.Fatalf("status %d", rec.Code) |
| + } |
| + a.mu.Lock() |
| + after := a.entries[rt.Host].lastActive |
| + a.mu.Unlock() |
| + if !after.Equal(before) { |
| + t.Errorf("lastActive moved %v -> %v", before, after) |
| + } |
| + if _, _, starts := b.counts(); starts != starts0 { |
| + t.Errorf("a beacon started an instance") |
| + } |
| +} |
| +``` |
| +If `testActivator`'s registry does not hold `rt` (read `arrangeAwake` and `testRoute`), `reg.Upsert(rt)` before `edge.New`. If the route the fixture makes is not `Hibernate: true` in the registry, set it so the edge takes the hibernating branch for ordinary requests — the test's point is that the reserved path never gets there. |
| + |
| +- [ ] **Step 2: Run** `go test ./internal/activator/ -run BrowserBeacon` — PASS on first run is expected (Task 2 placed the intercept before the activator); if it FAILS, the intercept is in the wrong place — fix `edge.go`, not the test. |
| +- [ ] **Step 3: Commit** `activator: a beacon to a hibernating route leaves lastActive alone`. |
| + |
| +--- |
| + |
| +### Task 9: The switch, the tabs, the CLI and the docs say what is now true |
| + |
| +**Files:** |
| +- Modify: `internal/console/templates/app_settings.html:197-207`, `account_settings.html:161-172` (door select) |
| +- Modify: `internal/console/templates/app_analytics.html` (drop reasons, coverage line, snippet line), `app_errors.html:77` (snippet line replaces "coming in a later step") |
| +- Modify: `internal/console/analyticsview.go` (`analyticsWindow.Sources []string`, filled in `readAnalytics`) |
| +- Modify: `internal/console/apianalytics.go` (`sources` in the response), `apimode.go:2905` (mirror), `analytics_cmd.go:102` (breakdown line + sources) |
| +- Modify: `docs/site/cli.md` (`### carlos analytics`, `### carlos errors`) |
| +- Test: `internal/console/analyticsview_test.go` (or the nearest existing analytics console test), `analytics_cmd_test.go` |
| + |
| +**Interfaces:** |
| +- Produces: `analyticsWindow.Sources []string` — `["edge"]`, plus `"browser"` when `Sum.Events` is non-empty or any `Sum.Dropped.E`/`.A` counter is non-zero (ruling 7); JSON `"sources":["edge","browser"]` on the API and CLI responses. |
| + |
| +- [ ] **Step 1: Failing tests** |
| + |
| +Console: find the test that renders the analytics window (grep `readAnalytics\|analyticsWindow` in `internal/console/*_test.go`) and add: a summary with one named event yields `Sources == []string{"edge","browser"}`; an empty summary yields `[]string{"edge"}`. Template tests: the app and account settings pages contain `name="door"`; the analytics page for a window with `Dropped.E.Origin == 3` contains `origin` and `3` in the dropped table; the errors page contains `/__carlos/carlos.js`. |
| + |
| +CLI (`analytics_cmd_test.go`): a response with `Dropped.A.Rate = 2` prints a line containing `events: rate 2`; with all zero, no `dropped by reason` line. |
| + |
| +- [ ] **Step 2: Run** the console and root package tests named — FAIL. |
| + |
| +- [ ] **Step 3: Implement** |
| + |
| +Settings forms — add after the count `<select>` in both templates: |
| +```html |
| + <label>Browser door (<code>/__carlos/</code> beacons): |
| + <select name="door"> |
| + <option value="on"{{if eq .AnalyticsSetting.Door "on"}} selected{{end}}>on</option> |
| + <option value="off"{{if eq .AnalyticsSetting.Door "off"}} selected{{end}}>off</option> |
| + </select> |
| + </label> |
| +``` |
| +and extend the "currently …" sentence with `; door {{.AnalyticsSetting.Door}}, from the {{.AnalyticsSetting.DoorFrom}}`. `analyticsFormValues` already accepts both. |
| + |
| +Analytics tab — replace the headline `Dropped` cell's value with the total `{{template "dropTotal" .Sum.Dropped}}` (a small define summing the nine counters is impossible in a template: compute `DroppedTotal int64` on `analyticsWindow` in Go instead and render that) and add under the headline table: |
| +```html |
| +{{if .DroppedTotal}}<p class="small">Dropped by reason — errors: rate {{.Sum.Dropped.E.Rate}}, size {{.Sum.Dropped.E.Size}}, origin {{.Sum.Dropped.E.Origin}}, malformed {{.Sum.Dropped.E.Malformed}}; events: rate {{.Sum.Dropped.A.Rate}}, size {{.Sum.Dropped.A.Size}}, origin {{.Sum.Dropped.A.Origin}}, malformed {{.Sum.Dropped.A.Malformed}}; flush {{.Sum.Dropped.Flush}}.</p>{{end}} |
| +<p class="small">Sources live: {{range $i, $s := .Sources}}{{if $i}}, {{end}}{{$s}}{{end}}.</p> |
| +``` |
| +and, after the named-events table: |
| +```html |
| +<p class="small">Named events and JavaScript errors need one line in the page: <code><script src="/__carlos/carlos.js" defer></script></code>. Then <code>carlos.track("signup", {plan: "pro"})</code> counts an event here (fields are accepted and not yet shown). Page views are counted at the edge already; the snippet never sends them. What edge counting misses: single-page-app route changes, pages served by a CDN in front of CARLOS, and back/forward-cache restores.</p> |
| +``` |
| +Errors tab line 77 becomes: `Log lines at <code>{{.ErrWin.LogLevel}}</code> and above and responses of 500 or higher appear here on their own. A JavaScript error needs one line in the page: <code><script src="/__carlos/carlos.js" defer></script></code> — messages, stacks and fields are redacted for addresses, credentials and long tokens before they leave the page and again on arrival (a floor, not a guarantee). A server-side error needs the hook (next step). The level is set on the Settings tab. …` |
| + |
| +`analyticsview.go`: `Sources []string` and `DroppedTotal int64` on `analyticsWindow`; fill in `readAnalytics` after the summary is built: |
| +```go |
| + win.Sources = []string{"edge"} |
| + d := win.Sum.Dropped |
| + win.DroppedTotal = d.E.Rate + d.E.Size + d.E.Origin + d.E.Malformed + d.A.Rate + d.A.Size + d.A.Origin + d.A.Malformed + d.Flush |
| + if len(win.Sum.Events) > 0 || win.DroppedTotal-d.Flush > 0 { |
| + win.Sources = append(win.Sources, "browser") |
| + } |
| +``` |
| +API: `Sources []string \`json:"sources"\`` on `analyticsResponse` and its `apimode.go` mirror. CLI: after the headline, when the breakdown is non-zero, print `dropped by reason — errors: rate N, size N, origin N, malformed N; events: …; flush N` muted, and `sources: edge, browser`. |
| + |
| +`docs/site/cli.md` — under `### carlos analytics`, after the existing flags, add a paragraph: "Named events come from the page: include `<script src="/__carlos/carlos.js" defer></script>` and call `carlos.track("name", {field: "value"})`. The same snippet reports JavaScript errors to the Errors tab. `--set-door off` closes both beacons for the app; page views are counted at the edge regardless." Under `### carlos errors`, add one sentence naming the `browser` source and the same script tag. |
| + |
| +- [ ] **Step 4: Run** `go test ./internal/console/ ./internal/docsite/ . ` — PASS. |
| +- [ ] **Step 5: Commit** `console, cli, docs: the door switch, drop reasons, the coverage line and the one-line snippet`. |
| + |
| +--- |
| + |
| +### Task 10: Gate, spec status, branch description (controller) |
| + |
| +- [ ] `gofmt -l .` clean (excluding `.claude/`); `go vet ./...`; `go test ./...` with `TMPDIR=/tmp/g`; `GOOS=darwin GOARCH=arm64 go build ./...`; `GOOS=windows GOARCH=amd64 go build ./...`. |
| +- [ ] `grep -rn '__carlos' /home/paulca/github.com/carlosframework/platform-infrastructure` is empty (§8's checklist line; it was on 2026-09-07). |
| +- [ ] Spec status line: step 3 built on branch `analytics-step3`; note rulings 1–7 above in §8 as "planning rulings, step 3". |
| +- [ ] Whole-branch review (Opus) against §3.4, §4, §6; fix; re-review. |
| +- [ ] Push, `amadan ci status`, `amadan branch describe carlos/platform analytics-step3 -body -` with: what lands, the rulings, the review record, deferred minors, and the follow-up for step 4. |
| + |
| +--- |
| + |
| +## Self-review |
| + |
| +**Spec coverage.** §3.4 placement (T2), attribution on shared hostname (T7, T2's lookup guard), endpoint shapes and 202 (T7), Origin/Referer/Sec-Fetch-Site (T5), caps (T5, T7), rate and LRU (T3, T7), snippet contract and vendoring, loader/asset caching (T4), `dropped.*` (T6, T7, T9), `door` and `count` switches (T1, T7), §4 no-wake (T2, T8), §3.6 coverage line and settings rendering (T9), §6 "Browser door" list: correct Origin counts (T7), missing Origin + Referer counts (T7), foreign Origin + Referer does not (T7), cross-site does not (T7), 17 KB does not (T7), 601st does not (T7 app bucket), each drop increments the right key (T6, T7), every case 202 (T7), hibernating `lastActive` (T8), path-routed attribution (T7), `__carlos` prefix key refused (T2); "`/__carlos/` reservation": never proxied, loader revalidates, asset immutable (T2, T4); "Settings resolution": `/a` dropped and `/e` accepted when off, `door: off` drops both (T7). Not in this step: server door (step 4), tombstones, `Group.Level` badge, browser-counted mode (§5). |
| + |
| +**Placeholders.** None: every step carries its code or the exact grep that finds the seam (T7's `setPathMap`, T8's fixture) with the fallback stated. |
| + |
| +**Type consistency.** `telemetry.Switches{Count, Door}` (T1) is what `SetSwitches`, `switchesFor` (T1, T7) and the host pass (T1) use. `AnalyticsMeter` = `Observe/Track/Drop` (T6) matches `memAnalytics` (T6) and `serveBeacon` (T7). `analytics.DoorErrors/DoorEvents/Drop*` (T6) are the names T7 and T9 use. `doorLimiter.allow(app, door, visitor, now)` (T3) is called with `acct+"/"+app` (T7). `serveReserved(w, r, host, up)` (T2) is what T4 and T7 extend; `serveCarlosJS(w, r) bool` (T4); `parseErrorBeacon`/`parseEventBeacon`/`beaconOriginOK` (T5) as used in T7. |