Analytics, step 2: counting at the edge — Implementation Plan

> For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: An account that turns analytics on gets pageviews, daily visitors, bytes, top pages, top referers and named events for every app, counted at the edge with no JavaScript and no cookie, on an Analytics tab, an API and carlos analytics.

Architecture: A new leaf package internal/analytics owns the sketches (HyperLogLog for visitors, Misra–Gries for top keys), the in-memory per-app meter the edge feeds, the day-file wire shape and its CAS merge with per-box seq idempotency, the daily salt file, the bot list, and the cross-box range read. The edge's existing observer defer builds one observation per response and hands it to the meter when the route's app is switched on; the switch map is pushed into the edge by a host pass that resolves telemetry.json for the routes the box serves. The agent flushes the meter on the usage loop's five-minute tick as a second CAS write. The console gains the tab, GET …/analytics, the switches at both levels (and PUT …/telemetry learns the analytics keys, with a serial bump), and the CLI gains carlos analytics.

Tech Stack: Go 1.22+, html/template, inline SVG computed in Go, release.Store bucket, sqlite registry; no new module dependencies.

Spec: docs/superpowers/specs/2026-09-07-errors-and-analytics-design.md — §3.3 in full (where the edge sees it; what is counted; client IP; visitor; cardinality; page paths; storage; named events; accumulation; the settings object), §3.6 (Analytics tab, API, CLI, Settings), §4, §6, §7 step 2, §8 rulings 1 and 4.

Global Constraints

• Defaults (ruling): analytics.count = off, analytics.door = on. Nothing is counted for an app until an account or app object says count: edge.
• Pageview = method GET, final status 200, media type text/html; not a redirect, 304, 204/205, HEAD; not Sec-Purpose: prefetch / Purpose: prefetch; Sec-Fetch-Dest when present must be document; never under /__carlos/; a bot UA counts into bots only; status routes never counted.
• Visitor = sha256(day-salt ‖ route host ‖ clientIP ‖ user agent)[:8]; salt per box per UTC day, 0600 in the box data dir (dataDir(), default /data), generated at first use, deleted on rotation, never in the bucket. No cookie; no IP stored.
• clientIP(r) = last element of the joined X-Forwarded-For when hopVerified(ctx), else host of RemoteAddr.
• Cardinality: HLL with 2¹¹ registers, encoded base64 with a one-byte version prefix (1), merged by register max. Pages/referers/events are Misra–Gries with 500 counters, keys capped at 256 bytes; the reader re-folds to 500 + _other; above 500 distinct keys the table is labelled approximate.
• Page path keys are apperr.SanitizePath(path). Referer keys are host-only; empty → direct.
• Storage: apps/<sqid>/<app>/analytics/<box>/days/<date>.json; buckets keyed by event time; drain returns deltas; a failed flush re-queues once (second consecutive failure drops the older window and increments dropped.flush); every delta batch carries a box-monotonic seq, the day file records seq per box, a merge with seq ≤ recorded is a no-op.
• Day file shape (v 1): {"v","date","hours":{"HH":{"pv","bots","bytes"}},"visitors","pages","referers","events","seq":{box:n},"dropped":{"e":{…},"a":{…},"flush"},"countries":{}} exactly as spec §3.3; dropped.e/dropped.a are written as zeros in this step (the doors arrive in steps 3–4).
• Switch convergence: the console writes telemetry.json and bumps the serial; boxes resolve count app → account → default on the serial-triggered full pass and on the five-minute instances loop; count: off stops counting at the observer.
• Accumulation: flushes ride the existing usageLoop tick (five minutes) as a second CAS write.
• API: GET /api/cli/apps/{account}/{app}/analytics with svcrole.Allow(svcrole.Instance, svcrole.Operate) through instanceGate; PUT …/telemetry gains analytics (Operate service; owner person), bumps the serial when analytics changed.
• Tab: range today / 7d / 30d; headline pageviews, visitors ("daily visitors, summed" for ranges), bytes, bots, dropped; one inline SVG chart computed in Go (per hour today, per day otherwise), no library; three tables; a "counts since the last flush may be missing" note only when a restart happened that day; approximate label above 500 keys. When count is off the tab shows the switch and the one-line privacy reading instead of numbers.
• CLI: carlos analytics --app X [--range today|7d|30d] [--set-count edge|off] [--set-door on|off]; heading ### carlos analytics in docs/site/cli.md or the docsite gate fails.
• Wire shapes additive; nothing wakes an instance; the edge wrapper order is unchanged (the tap is a read of ob.status/ob.ctype/bytes in the existing defer).
• Go: GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off TMPDIR=/tmp/g (mkdir /tmp/g); retry once with the sandbox disabled on "read-only file system". Before push: GOOS=darwin GOARCH=arm64 go build ./... and GOOS=windows GOARCH=amd64 go build ./... (the salt file code must build on all three).
• Commit trailer: Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>. Branch analytics-step2, cut from main at 1a99060. Push to origin (the forge). Never gh.
• The getting-started skill question (spec §7 step 2) lives in the carlosframework/website repo and is NOT a task here; Task 10 files it as a follow-up in the branch description.

File structure

| File | Responsibility | |---|---| | internal/analytics/hll.go | HLL sketch: add, merge, count, encode/decode | | internal/analytics/topk.go | TopK Misra–Gries summary: add, merge, fold | | internal/analytics/bots.go | IsBot(ua) vendored substring list | | internal/analytics/salt.go | DaySalt(dir, date): 0600 file, rotation | | internal/analytics/day.go | Day wire shape, keys, Delta, Merge, UpsertDay | | internal/analytics/meter.go | Meter: Obs, Observe, Track, Drain, Requeue | | internal/analytics/read.go | ReadRange across boxes, Summary | | internal/edge/analytics.go | AnalyticsMeter iface, EnableAnalytics, SetAnalyticsOn, clientIP, observeAnalytics | | internal/edge/edge.go | one call in the observer defer | | internal/host/telemetry_pass.go | resolve switches for the box's routes, push to the edge | | internal/host/usage_loop.go, agent.go | meter construction, flush on the usage tick, pass wiring | | internal/console/analyticsview.go | readAnalytics, chart points, page + settings handlers | | internal/console/apianalytics.go | GET …/analytics; apierrors.go PUT accepts analytics | | internal/console/templates/app_analytics.html, appnav.html, app_settings.html, account_settings.html | tab, nav, switches | | analytics_cmd.go, apimode.go, main.go, docs/site/cli.md | CLI |

---

Task 1: HyperLogLog and Misra–Gries sketches

Files:

• Create: internal/analytics/hll.go, internal/analytics/topk.go
• Test: internal/analytics/hll_test.go, internal/analytics/topk_test.go

Interfaces:

• Produces: ``go const HLLPrecision = 11; const hllRegisters = 1 << HLLPrecision type HLL struct { reg [hllRegisters]uint8 } func (h *HLL) Add(hash uint64) func (h *HLL) Merge(o *HLL) func (h *HLL) Count() float64 func (h *HLL) Empty() bool func (h *HLL) Encode() string // base64(std) of [1, reg...] func DecodeHLL(s string) (*HLL, error) // "" → empty sketch const TopKSize = 500; const OtherKey = "_other"; const MaxKeyLen = 256 type TopK struct { counts map[string]int64; other int64 } func NewTopK() *TopK func (t *TopK) Add(key string, n int64) func (t *TopK) Merge(o *TopK) func (t *TopK) Fold() map[string]int64 // ≤500 keys plus OtherKey when other>0 func TopKFrom(m map[string]int64) *TopK // inverse of Fold (OtherKey → other) func (t *TopK) Approximate() bool // other > 0 ``

• [ ] Step 1: Failing tests

hll_test.go:

package analytics

import (
	"hash/fnv"
	"math"
	"testing"
)

func h64(s string) uint64 { f := fnv.New64a(); f.Write([]byte(s)); return f.Sum64() }

func TestHLLEstimatesWithinThreeSigma(t *testing.T) {
	for _, n := range []int{1000, 10000, 100000} {
		var h HLL
		for i := 0; i < n; i++ {
			h.Add(h64("v" + itoa(i)))
		}
		got := h.Count()
		sigma := 1.04 / math.Sqrt(hllRegisters) * float64(n)
		if math.Abs(got-float64(n)) > 3*sigma {
			t.Errorf("n=%d: estimate %.0f off by more than 3σ (%.0f)", n, got, 3*sigma)
		}
	}
}

func TestHLLMergeIsUnionAndIdempotent(t *testing.T) {
	var a, b HLL
	for i := 0; i < 5000; i++ {
		a.Add(h64("a" + itoa(i)))
		b.Add(h64("b" + itoa(i)))
	}
	var ab HLL
	ab.Merge(&a)
	ab.Merge(&b)
	ab.Merge(&b) // idempotent
	if got := ab.Count(); math.Abs(got-10000) > 3*(1.04/math.Sqrt(hllRegisters))*10000 {
		t.Fatalf("union estimate %.0f", got)
	}
	var ba HLL
	ba.Merge(&b)
	ba.Merge(&a)
	if ab.reg != ba.reg {
		t.Fatal("merge must be commutative")
	}
}

func TestHLLEncodeRoundTripAndVersion(t *testing.T) {
	var h HLL
	h.Add(h64("x"))
	enc := h.Encode()
	if enc[:2] != "AQ" { // base64 of a leading 0x01 byte
		t.Fatalf("version prefix missing: %q", enc[:4])
	}
	d, err := DecodeHLL(enc)
	if err != nil || d.reg != h.reg {
		t.Fatalf("round trip: %v", err)
	}
	if e, err := DecodeHLL(""); err != nil || !e.Empty() {
		t.Fatal("empty string must decode to an empty sketch")
	}
	if _, err := DecodeHLL("AgAA"); err == nil {
		t.Fatal("unknown version must error")
	}
}

func itoa(i int) string { return string(rune('0'+i%10)) + string(rune('0'+(i/10)%10)) + string(rune('0'+(i/100)%10)) + string(rune('0'+(i/1000)%10)) + string(rune('0'+(i/10000)%10)) + string(rune('0'+(i/100000)%10)) }

topk_test.go:

package analytics

import "testing"

func TestTopKExactBelowCapAndBoundedAbove(t *testing.T) {
	k := NewTopK()
	for i := 0; i < 100; i++ {
		k.Add("/p"+itoa(i), int64(i+1))
	}
	f := k.Fold()
	if len(f) != 100 || f["/p99"] != 100 || k.Approximate() {
		t.Fatalf("below cap must be exact: len=%d p99=%d approx=%v", len(f), f["/p99"], k.Approximate())
	}
	for i := 0; i < 2000; i++ {
		k.Add("/q"+itoa(i), 1)
	}
	f = k.Fold()
	if len(f) > TopKSize+1 || !k.Approximate() || f[OtherKey] == 0 {
		t.Fatalf("above cap: len=%d approx=%v other=%d", len(f), k.Approximate(), f[OtherKey])
	}
	// Total mass is conserved between counters and _other.
	var total int64
	for _, v := range f {
		total += v
	}
	if total != 100*101/2+2000 {
		t.Fatalf("mass %d", total)
	}
	// A heavy hitter survives.
	if f["/p99"] < 100-2000/TopKSize {
		t.Fatalf("heavy hitter undercounted: %d", f["/p99"])
	}
}

func TestTopKMergeRefolds(t *testing.T) {
	a, b := NewTopK(), NewTopK()
	for i := 0; i < 400; i++ {
		a.Add("/a"+itoa(i), 2)
		b.Add("/b"+itoa(i), 3)
	}
	a.Merge(b)
	f := a.Fold()
	if len(f) > TopKSize+1 || f[OtherKey] == 0 {
		t.Fatalf("merged 800 keys must fold to ≤501: %d", len(f))
	}
	r := TopKFrom(f)
	if r.other != f[OtherKey] || len(r.counts) != len(f)-1 {
		t.Fatal("TopKFrom must invert Fold")
	}
	long := NewTopK()
	long.Add(string(make([]byte, 300)), 1)
	for key := range long.Fold() {
		if len(key) > MaxKeyLen {
			t.Fatal("keys must be capped at 256 bytes")
		}
	}
}

• [ ] Step 2: Run, expect compile failure

• [ ] Step 3: Implement hll.go

// Package analytics is the pure half of the Analytics tab (spec
// 2026-09-07-errors-and-analytics-design.md §3.3): the visitor and top-key
// sketches, the in-memory meter the edge feeds, the day-file wire shape
// and its idempotent CAS merge, the daily salt, the bot list and the
// cross-box range read. internal/edge observes, internal/host flushes,
// internal/console reads — the applog/apperr split.
package analytics

import (
	"encoding/base64"
	"errors"
	"math"
	"math/bits"
)

// HLLPrecision is the register-index width: 2^11 registers, 2 KB, ~2.3%
// standard error (spec §3.3). One sketch per app per day per box.
const HLLPrecision = 11
const hllRegisters = 1 << HLLPrecision

// HLL is a HyperLogLog cardinality sketch over 64-bit hashes.
type HLL struct{ reg [hllRegisters]uint8 }

// Add records one hashed element.
func (h *HLL) Add(hash uint64) {
	idx := hash >> (64 - HLLPrecision)
	rest := hash<<HLLPrecision | 1<<(HLLPrecision-1) // guard bit bounds the run
	rank := uint8(bits.LeadingZeros64(rest)) + 1
	if rank > h.reg[idx] {
		h.reg[idx] = rank
	}
}

// Merge folds o into h by register max — the union of the two sets.
func (h *HLL) Merge(o *HLL) {
	for i := range h.reg {
		if o.reg[i] > h.reg[i] {
			h.reg[i] = o.reg[i]
		}
	}
}

// Empty reports whether nothing was added.
func (h *HLL) Empty() bool {
	for _, r := range h.reg {
		if r != 0 {
			return false
		}
	}
	return true
}

// Count estimates the cardinality (Flajolet et al. 2007 with the small-
// range linear-counting correction).
func (h *HLL) Count() float64 {
	m := float64(hllRegisters)
	alpha := 0.7213 / (1 + 1.079/m)
	var sum float64
	zeros := 0
	for _, r := range h.reg {
		sum += math.Pow(2, -float64(r))
		if r == 0 {
			zeros++
		}
	}
	est := alpha * m * m / sum
	if est <= 2.5*m && zeros > 0 {
		est = m * math.Log(m/float64(zeros))
	}
	return est
}

const hllVersion = 1

// Encode renders the sketch as base64 of a version byte plus the registers.
func (h *HLL) Encode() string {
	b := make([]byte, 1+hllRegisters)
	b[0] = hllVersion
	copy(b[1:], h.reg[:])
	return base64.StdEncoding.EncodeToString(b)
}

// DecodeHLL parses Encode's output; "" is an empty sketch, an unknown
// version is an error (a future shape this build cannot read).
func DecodeHLL(s string) (*HLL, error) {
	h := &HLL{}
	if s == "" {
		return h, nil
	}
	b, err := base64.StdEncoding.DecodeString(s)
	if err != nil {
		return nil, err
	}
	if len(b) != 1+hllRegisters || b[0] != hllVersion {
		return nil, errors.New("analytics: unknown visitors sketch version or size")
	}
	copy(h.reg[:], b[1:])
	return h, nil
}

• [ ] Step 4: Implement topk.go

package analytics

// TopKSize bounds a top-keys summary (pages, referers, events) per app
// per day per box (spec §3.3). Misra–Gries: exact below the cap, a lower
// bound within the summary's error above it, mergeable, with the evicted
// mass kept in OtherKey so totals are conserved.
const TopKSize = 500

// OtherKey is the fold-over row for keys beyond the cap.
const OtherKey = "_other"

// MaxKeyLen caps a key's bytes.
const MaxKeyLen = 256

// TopK is a Misra–Gries summary with TopKSize counters.
type TopK struct {
	counts map[string]int64
	other  int64
}

// NewTopK returns an empty summary.
func NewTopK() *TopK { return &TopK{counts: make(map[string]int64)} }

// Add counts n occurrences of key. When the summary is full and key is
// new, every counter is decremented by the smallest amount that frees a
// slot, and the decremented mass moves to other.
func (t *TopK) Add(key string, n int64) {
	if len(key) > MaxKeyLen {
		key = key[:MaxKeyLen]
	}
	if n <= 0 {
		return
	}
	if _, ok := t.counts[key]; ok || len(t.counts) < TopKSize {
		t.counts[key] += n
		return
	}
	// Full and new: take the smallest counter as the decrement.
	min := int64(-1)
	for _, v := range t.counts {
		if min < 0 || v < min {
			min = v
		}
	}
	if n <= min {
		t.other += n
		return
	}
	for k, v := range t.counts {
		if v <= min {
			delete(t.counts, k)
			t.other += v
		} else {
			t.counts[k] = v - min
			t.other += min
		}
	}
	t.counts[key] = n - min
	t.other += min
}

// Merge folds o into t: counters add, other adds, then t re-folds to the
// cap so a merge of two full summaries stays bounded.
func (t *TopK) Merge(o *TopK) {
	merged := NewTopK()
	merged.other = t.other + o.other
	sum := make(map[string]int64, len(t.counts)+len(o.counts))
	for k, v := range t.counts {
		sum[k] += v
	}
	for k, v := range o.counts {
		sum[k] += v
	}
	for k, v := range sum { // Add re-applies the cap
		merged.Add(k, v)
	}
	t.counts, t.other = merged.counts, merged.other
}

// Fold returns the counters plus OtherKey when any mass fell over.
func (t *TopK) Fold() map[string]int64 {
	out := make(map[string]int64, len(t.counts)+1)
	for k, v := range t.counts {
		out[k] = v
	}
	if t.other > 0 {
		out[OtherKey] = t.other
	}
	return out
}

// TopKFrom rebuilds a summary from a folded map (the day file's shape).
func TopKFrom(m map[string]int64) *TopK {
	t := NewTopK()
	for k, v := range m {
		if k == OtherKey {
			t.other += v
			continue
		}
		t.Add(k, v)
	}
	return t
}

// Approximate reports whether counters are lower bounds (mass fell over).
func (t *TopK) Approximate() bool { return t.other > 0 }

• [ ] Step 5: Run go test ./internal/analytics/ -count=1 -v, PASS; gofmt; vet. Commit "analytics: HyperLogLog and Misra–Gries sketches (spec §3.3)".

---

Task 2: Bot list, day salt, day-file shape and idempotent CAS merge

Files:

• Create: internal/analytics/bots.go, internal/analytics/salt.go, internal/analytics/day.go
• Test: internal/analytics/bots_test.go, internal/analytics/salt_test.go, internal/analytics/day_test.go

Interfaces:

• Produces: ``go func IsBot(ua string) bool func DaySalt(dir, date string) ([]byte, error) // 32 bytes; creates 0600 <dir>/analytics-salt-<date>; removes other analytics-salt-* files type HourRow struct { PV int64 json:"pv"; Bots int64 json:"bots"; Bytes int64 json:"bytes" } type Dropped struct { E DropCounts json:"e"; A DropCounts json:"a"; Flush int64 json:"flush" } type DropCounts struct { Rate, Size, Origin, Malformed int64 } // json tags rate,size,origin,malformed type Day struct { V int; Date string; Hours map[string]HourRow; Visitors string; Pages, Referers, Events map[string]int64; Seq map[string]int64; Dropped Dropped; Countries map[string]int64; Restarted bool json:"restarted,omitempty" } type Delta struct { Date string; Box string; Seq int64; Hours map[string]HourRow; Visitors *HLL; Pages, Referers, Events *TopK; Dropped Dropped; Restarted bool } func Prefix(sqid, app string) string // applog.FamilyPrefix(FamilyAnalytics, …) func BoxPrefix(sqid, app, box string) string // Prefix + box + "/days/" func DayKey(sqid, app, box, date string) string func (d *Day) Merge(x Delta) (applied bool, err error) // seq ≤ recorded → false,nil func UpsertDay(ctx, s release.Store, sqid, app string, x Delta) error // CAS ×3 ``

• [ ] Step 1: Failing tests

bots_test.go:

package analytics

import "testing"

func TestIsBot(t *testing.T) {
	yes := []string{
		"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)",
		"Mozilla/5.0 (compatible; bingbot/2.0)", "curl/8.4.0", "Wget/1.21",
		"python-requests/2.31", "Go-http-client/1.1", "Mozilla/5.0 HeadlessChrome/120",
		"facebookexternalhit/1.1", "Slackbot-LinkExpanding", "AhrefsBot/7.0", "",
	}
	no := []string{
		"Mozilla/5.0 (Macintosh; Intel Mac OS X 14_0) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.0 Safari/605.1.15",
		"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0 Safari/537.36",
		"Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) AppleWebKit/605.1.15 Mobile/15E148",
	}
	for _, ua := range yes {
		if !IsBot(ua) {
			t.Errorf("IsBot(%q) = false", ua)
		}
	}
	for _, ua := range no {
		if IsBot(ua) {
			t.Errorf("IsBot(%q) = true", ua)
		}
	}
}

salt_test.go:

package analytics

import (
	"os"
	"path/filepath"
	"runtime"
	"testing"
)

func TestDaySaltIsStableAndRotates(t *testing.T) {
	dir := t.TempDir()
	a, err := DaySalt(dir, "2026-09-07")
	if err != nil || len(a) != 32 {
		t.Fatalf("salt: %v len=%d", err, len(a))
	}
	b, _ := DaySalt(dir, "2026-09-07")
	if string(a) != string(b) {
		t.Fatal("same day must return the same salt (restart-stable)")
	}
	if runtime.GOOS != "windows" {
		st, _ := os.Stat(filepath.Join(dir, "analytics-salt-2026-09-07"))
		if st.Mode().Perm() != 0o600 {
			t.Fatalf("mode %v", st.Mode().Perm())
		}
	}
	c, _ := DaySalt(dir, "2026-09-08")
	if string(c) == string(a) {
		t.Fatal("a new day must salt afresh")
	}
	if _, err := os.Stat(filepath.Join(dir, "analytics-salt-2026-09-07")); !os.IsNotExist(err) {
		t.Fatal("rotation must delete the previous day's salt")
	}
}

day_test.go:

package analytics

import (
	"context"
	"encoding/json"
	"testing"

	"github.com/carlosframework/platform/internal/release"
)

func delta(box string, seq int64, pv int64) Delta {
	h := &HLL{}
	h.Add(h64(box + itoa(int(seq))))
	p := NewTopK()
	p.Add("/", pv)
	return Delta{Date: "2026-09-07", Box: box, Seq: seq, Hours: map[string]HourRow{"09": {PV: pv, Bytes: pv * 100}},
		Visitors: h, Pages: p, Referers: NewTopK(), Events: NewTopK()}
}

func TestDayMergeIsSeqIdempotent(t *testing.T) {
	var d Day
	if ok, err := d.Merge(delta("box-a", 5, 10)); !ok || err != nil {
		t.Fatalf("first merge: %v %v", ok, err)
	}
	if ok, _ := d.Merge(delta("box-a", 5, 10)); ok {
		t.Fatal("replaying the same seq must be a no-op")
	}
	if ok, _ := d.Merge(delta("box-a", 4, 10)); ok {
		t.Fatal("an older seq must be a no-op")
	}
	if ok, _ := d.Merge(delta("box-b", 1, 7)); !ok {
		t.Fatal("another box's seq is independent")
	}
	if d.Hours["09"].PV != 17 || d.Pages["/"] != 17 || d.Seq["box-a"] != 5 || d.Seq["box-b"] != 1 || d.V != 1 || d.Date != "2026-09-07" {
		t.Fatalf("merged = %+v", d)
	}
	vis, _ := DecodeHLL(d.Visitors)
	if c := vis.Count(); c < 1.5 || c > 2.5 {
		t.Fatalf("visitors ≈ 2, got %.2f", c)
	}
	b, _ := json.Marshal(d)
	for _, want := range []string{`"v":1`, `"hours"`, `"visitors"`, `"pages"`, `"referers"`, `"events"`, `"seq"`, `"dropped"`, `"countries"`} {
		if !contains(string(b), want) {
			t.Errorf("wire missing %s in %s", want, b)
		}
	}
}

func TestUpsertDayCASAcrossBoxes(t *testing.T) {
	ctx := context.Background()
	s := release.NewDirStore(t.TempDir())
	if err := UpsertDay(ctx, s, "bab", "web", delta("box-a", 1, 3)); err != nil {
		t.Fatal(err)
	}
	if err := UpsertDay(ctx, s, "bab", "web", delta("box-a", 2, 4)); err != nil {
		t.Fatal(err)
	}
	if err := UpsertDay(ctx, s, "bab", "web", delta("box-a", 2, 4)); err != nil { // retry of an ambiguous put
		t.Fatal(err)
	}
	raw, _, err := s.Get(ctx, DayKey("bab", "web", "box-a", "2026-09-07"))
	if err != nil {
		t.Fatal(err)
	}
	var d Day
	json.Unmarshal(raw, &d)
	if d.Hours["09"].PV != 7 || d.Seq["box-a"] != 2 {
		t.Fatalf("day = %+v", d)
	}
	if DayKey("bab", "web", "box-a", "2026-09-07") != "apps/bab/web/analytics/box-a/days/2026-09-07.json" {
		t.Fatal(DayKey("bab", "web", "box-a", "2026-09-07"))
	}
}

func contains(s, sub string) bool { return len(s) >= len(sub) && (s == sub || len(s) > 0 && indexOf(s, sub) >= 0) }
func indexOf(s, sub string) int {
	for i := 0; i+len(sub) <= len(s); i++ {
		if s[i:i+len(sub)] == sub {
			return i
		}
	}
	return -1
}

• [ ] Step 2: Run, expect compile failure

• [ ] Step 3: Implement bots.go

package analytics

import "strings"

// botMarkers is the vendored user-agent list (spec §3.3): incomplete by
// nature, so the tab shows the bots count rather than pretending. Matched
// case-insensitively as substrings. An empty UA is a bot: no browser sends
// none.
var botMarkers = []string{
	"bot", "crawler", "spider", "slurp", "curl/", "wget/", "python-requests",
	"go-http-client", "headlesschrome", "phantomjs", "facebookexternalhit",
	"ahrefs", "semrush", "mj12", "dotbot", "petalbot", "bytespider", "gptbot",
	"claudebot", "ccbot", "applebot", "yandex", "baiduspider", "duckduckbot",
	"linkedinbot", "twitterbot", "discordbot", "telegrambot", "whatsapp",
	"pingdom", "uptimerobot", "statuscake", "site24x7", "newrelicpinger",
	"lighthouse", "pagespeed", "gtmetrix", "httpclient", "okhttp", "java/",
	"libwww", "scrapy", "node-fetch", "axios/", "postmanruntime", "insomnia",
}

// IsBot reports whether ua looks like an automated client.
func IsBot(ua string) bool {
	if ua == "" {
		return true
	}
	l := strings.ToLower(ua)
	for _, m := range botMarkers {
		if strings.Contains(l, m) {
			return true
		}
	}
	return false
}

• [ ] Step 4: Implement salt.go

package analytics

import (
	"crypto/rand"
	"errors"
	"os"
	"path/filepath"
	"strings"
)

const saltPrefix = "analytics-salt-"

// DaySalt returns the box's visitor salt for date (YYYY-MM-DD), creating
// <dir>/analytics-salt-<date> with 32 random bytes at 0600 when absent and
// deleting any other day's file (spec §3.3: restart-stable within the day,
// gone on rotation, never in the bucket). dir is the box data directory.
func DaySalt(dir, date string) ([]byte, error) {
	path := filepath.Join(dir, saltPrefix+date)
	if b, err := os.ReadFile(path); err == nil && len(b) == 32 {
		return b, nil
	}
	b := make([]byte, 32)
	if _, err := rand.Read(b); err != nil {
		return nil, err
	}
	f, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o600)
	if err != nil {
		if errors.Is(err, os.ErrExist) { // a sibling process won the race
			return os.ReadFile(path)
		}
		return nil, err
	}
	_, werr := f.Write(b)
	cerr := f.Close()
	if werr != nil || cerr != nil {
		os.Remove(path)
		return nil, errors.Join(werr, cerr)
	}
	// Rotation: every other day's salt goes, so nothing on disk can join a
	// hash across days.
	entries, _ := os.ReadDir(dir)
	for _, e := range entries {
		if n := e.Name(); strings.HasPrefix(n, saltPrefix) && n != saltPrefix+date {
			os.Remove(filepath.Join(dir, n))
		}
	}
	return b, nil
}

• [ ] Step 5: Implement day.go

package analytics

import (
	"context"
	"encoding/json"
	"errors"
	"fmt"
	"os"

	"github.com/carlosframework/platform/internal/applog"
	"github.com/carlosframework/platform/internal/release"
)

// HourRow is one UTC hour's counters.
type HourRow struct {
	PV    int64 `json:"pv"`
	Bots  int64 `json:"bots"`
	Bytes int64 `json:"bytes"`
}

// DropCounts is why events were dropped at a door (steps 3–4 fill them).
type DropCounts struct {
	Rate      int64 `json:"rate"`
	Size      int64 `json:"size"`
	Origin    int64 `json:"origin"`
	Malformed int64 `json:"malformed"`
}

// Dropped is the honesty block (spec §3.3).
type Dropped struct {
	E     DropCounts `json:"e"`
	A     DropCounts `json:"a"`
	Flush int64      `json:"flush"`
}

// Day is apps/<sqid>/<app>/analytics/<box>/days/<date>.json (spec §3.3):
// one box's counting for one app for one UTC day. Additive shape.
type Day struct {
	V         int                `json:"v"`
	Date      string             `json:"date"`
	Hours     map[string]HourRow `json:"hours"`
	Visitors  string             `json:"visitors"`
	Pages     map[string]int64   `json:"pages"`
	Referers  map[string]int64   `json:"referers"`
	Events    map[string]int64   `json:"events"`
	Seq       map[string]int64   `json:"seq"`
	Dropped   Dropped            `json:"dropped"`
	Countries map[string]int64   `json:"countries"`
	Restarted bool               `json:"restarted,omitempty"`
}

// Delta is one flush batch from one box for one app-day.
type Delta struct {
	Date      string
	Box       string
	Seq       int64
	Hours     map[string]HourRow
	Visitors  *HLL
	Pages     *TopK
	Referers  *TopK
	Events    *TopK
	Dropped   Dropped
	Restarted bool
}

// Prefix is the app's analytics/ subtree.
func Prefix(sqid, app string) string { return applog.FamilyPrefix(applog.FamilyAnalytics, sqid, app) }

// BoxPrefix is one box's day files.
func BoxPrefix(sqid, app, box string) string { return Prefix(sqid, app) + box + "/days/" }

// DayKey is one day file.
func DayKey(sqid, app, box, date string) string { return BoxPrefix(sqid, app, box) + date + ".json" }

// Merge folds x into d unless x.Seq is not above the recorded seq for
// x.Box — the idempotency rule that makes a retried conditional put safe.
func (d *Day) Merge(x Delta) (bool, error) {
	if d.Seq == nil {
		d.Seq = map[string]int64{}
	}
	if x.Seq <= d.Seq[x.Box] {
		return false, nil
	}
	d.V, d.Date = 1, x.Date
	if d.Hours == nil {
		d.Hours = map[string]HourRow{}
	}
	for h, row := range x.Hours {
		cur := d.Hours[h]
		cur.PV += row.PV
		cur.Bots += row.Bots
		cur.Bytes += row.Bytes
		d.Hours[h] = cur
	}
	vis, err := DecodeHLL(d.Visitors)
	if err != nil {
		return false, err
	}
	if x.Visitors != nil {
		vis.Merge(x.Visitors)
	}
	d.Visitors = vis.Encode()
	d.Pages = mergeTop(d.Pages, x.Pages)
	d.Referers = mergeTop(d.Referers, x.Referers)
	d.Events = mergeTop(d.Events, x.Events)
	d.Dropped.E = addDrops(d.Dropped.E, x.Dropped.E)
	d.Dropped.A = addDrops(d.Dropped.A, x.Dropped.A)
	d.Dropped.Flush += x.Dropped.Flush
	d.Restarted = d.Restarted || x.Restarted
	if d.Countries == nil {
		d.Countries = map[string]int64{}
	}
	d.Seq[x.Box] = x.Seq
	return true, nil
}

func mergeTop(m map[string]int64, t *TopK) map[string]int64 {
	k := TopKFrom(m)
	if t != nil {
		k.Merge(t)
	}
	return k.Fold()
}

func addDrops(a, b DropCounts) DropCounts {
	return DropCounts{a.Rate + b.Rate, a.Size + b.Size, a.Origin + b.Origin, a.Malformed + b.Malformed}
}

// UpsertDay CAS-merges x into the app-day object for x.Box: read (etag),
// Merge, conditional put; on ErrCASConflict re-read and retry, three
// attempts. A no-op merge (seq already recorded) writes nothing.
func UpsertDay(ctx context.Context, s release.Store, sqid, app string, x Delta) error {
	key := DayKey(sqid, app, x.Box, x.Date)
	for attempt := 0; attempt < 3; attempt++ {
		var day Day
		raw, etag, err := s.Get(ctx, key)
		switch {
		case err == nil:
			if uerr := json.Unmarshal(raw, &day); uerr != nil {
				return fmt.Errorf("analytics: parse %s: %w", key, uerr)
			}
		case errors.Is(err, os.ErrNotExist):
			etag = ""
		default:
			return err
		}
		applied, err := day.Merge(x)
		if err != nil {
			return err
		}
		if !applied {
			return nil
		}
		out, err := json.Marshal(day)
		if err != nil {
			return err
		}
		if _, err := s.Put(ctx, key, out, etag); err == nil {
			return nil
		} else if !errors.Is(err, release.ErrCASConflict) {
			return err
		}
	}
	return fmt.Errorf("analytics: %s: lost CAS race three times", key)
}

Ensure Day zero-value fields marshal as {} not null for hours, pages, referers, events, seq, countries (initialise nil maps in Merge; mergeTop returns a non-nil map).

• [ ] Step 6: Run, PASS (also GOOS=windows go build ./internal/analytics/ to prove the salt file compiles everywhere). Commit "analytics: bot list, day salt, day file with seq-idempotent CAS merge (spec §3.3)".

---

Task 3: The meter the edge feeds

Files:

• Create: internal/analytics/meter.go
• Test: internal/analytics/meter_test.go

Interfaces:

• Produces: ``go type Obs struct { Acct, App, Host, Method string; Status int; ContentType, Path, RefererHost, UA, ClientIP string; Bytes int64; Prefetch bool; FetchDest string; When time.Time } type AppKey = applog.AppKey func NewMeter(saltDir, box string, now func() time.Time) *Meter func (m *Meter) Observe(o Obs) // decides pageview/bot; counts bytes for every response func (m *Meter) Track(acct, app, name string, when time.Time) // named events (steps 3–4 call it) func (m *Meter) Drain() map[AppKey][]Delta // one Delta per app-day touched, seq stamped func (m *Meter) Requeue(pending map[AppKey][]Delta) // failed flush: kept once, then dropped with Dropped.Flush++ func IsPageview(o Obs) (pageview bool, bot bool) // pure rule from §3.3 func VisitorHash(salt []byte, host, ip, ua string) uint64 func EventName(s string) (string, bool) // lowercased, ≤64 bytes of [a-z0-9_.-] ``

• [ ] Step 1: Failing tests

package analytics

import (
	"testing"
	"time"
)

func obs(method string, status int, ctype string) Obs {
	return Obs{Acct: "bab", App: "web", Host: "h1", Method: method, Status: status, ContentType: ctype,
		Path: "/pricing?x=1", RefererHost: "news.ycombinator.com", UA: "Mozilla/5.0 Safari", ClientIP: "203.0.113.5",
		Bytes: 1200, When: time.Date(2026, 9, 7, 9, 30, 0, 0, time.UTC)}
}

func TestIsPageviewRule(t *testing.T) {
	cases := []struct {
		name string
		o    Obs
		pv   bool
		bot  bool
	}{
		{"html 200", obs("GET", 200, "text/html; charset=utf-8"), true, false},
		{"json", obs("GET", 200, "application/json"), false, false},
		{"404", obs("GET", 404, "text/html"), false, false},
		{"redirect", obs("GET", 302, "text/html"), false, false},
		{"304", obs("GET", 304, ""), false, false},
		{"head", obs("HEAD", 200, "text/html"), false, false},
		{"post", obs("POST", 200, "text/html"), false, false},
		{"prefetch", func() Obs { o := obs("GET", 200, "text/html"); o.Prefetch = true; return o }(), false, false},
		{"fetch dest empty", func() Obs { o := obs("GET", 200, "text/html"); o.FetchDest = "empty"; return o }(), false, false},
		{"fetch dest document", func() Obs { o := obs("GET", 200, "text/html"); o.FetchDest = "document"; return o }(), true, false},
		{"carlos prefix", func() Obs { o := obs("GET", 200, "text/html"); o.Path = "/__carlos/a"; return o }(), false, false},
		{"bot", func() Obs { o := obs("GET", 200, "text/html"); o.UA = "Googlebot/2.1"; return o }(), false, true},
	}
	for _, c := range cases {
		pv, bot := IsPageview(c.o)
		if pv != c.pv || bot != c.bot {
			t.Errorf("%s: pv=%v bot=%v, want %v %v", c.name, pv, bot, c.pv, c.bot)
		}
	}
}

func TestMeterCountsAndDrains(t *testing.T) {
	now := time.Date(2026, 9, 7, 9, 30, 0, 0, time.UTC)
	m := NewMeter(t.TempDir(), "box1", func() time.Time { return now })
	m.Observe(obs("GET", 200, "text/html"))
	m.Observe(obs("GET", 200, "text/html"))
	o := obs("GET", 200, "text/html")
	o.ClientIP = "198.51.100.9"
	m.Observe(o)
	m.Observe(obs("GET", 200, "application/json")) // bytes only
	b := obs("GET", 200, "text/html")
	b.UA = "curl/8"
	m.Observe(b)
	m.Track("bab", "web", "Signup", now)
	m.Track("bab", "web", "bad name!", now) // rejected
	out := m.Drain()
	ds := out[AppKey{Acct: "bab", App: "web"}]
	if len(ds) != 1 {
		t.Fatalf("deltas = %d", len(ds))
	}
	d := ds[0]
	if d.Date != "2026-09-07" || d.Box != "box1" || d.Seq == 0 {
		t.Fatalf("delta = %+v", d)
	}
	if h := d.Hours["09"]; h.PV != 3 || h.Bots != 1 || h.Bytes != 6000 {
		t.Fatalf("hour = %+v", h)
	}
	if c := d.Visitors.Count(); c < 1.5 || c > 2.5 {
		t.Fatalf("visitors ≈ 2, got %.2f", c)
	}
	if p := d.Pages.Fold(); p["/pricing"] != 3 {
		t.Fatalf("pages = %v", p)
	}
	if r := d.Referers.Fold(); r["news.ycombinator.com"] != 3 {
		t.Fatalf("referers = %v", r)
	}
	if e := d.Events.Fold(); e["signup"] != 1 || len(e) != 1 {
		t.Fatalf("events = %v", e)
	}
	if len(m.Drain()) != 0 {
		t.Fatal("second drain must be empty")
	}
	// Event-time bucketing: an observation at 09:59 flushed at 10:00 lands in 09.
	m.Observe(func() Obs { o := obs("GET", 200, "text/html"); o.When = now.Add(29 * time.Minute); return o }())
	now = now.Add(31 * time.Minute)
	if d := m.Drain()[AppKey{Acct: "bab", App: "web"}][0]; d.Hours["09"].PV != 1 || d.Hours["10"].PV != 0 {
		t.Fatalf("event-time bucket: %+v", d.Hours)
	}
}

func TestMeterRequeueSurvivesOnce(t *testing.T) {
	now := time.Date(2026, 9, 7, 9, 30, 0, 0, time.UTC)
	m := NewMeter(t.TempDir(), "box1", func() time.Time { return now })
	m.Observe(obs("GET", 200, "text/html"))
	first := m.Drain()
	m.Requeue(first) // flush failed
	m.Observe(obs("GET", 200, "text/html"))
	second := m.Drain()
	ds := second[AppKey{Acct: "bab", App: "web"}]
	if len(ds) != 2 || ds[0].Seq >= ds[1].Seq {
		t.Fatalf("requeued delta must come back first with its own seq: %+v", ds)
	}
	m.Requeue(second) // failed again: the older window drops
	m.Observe(obs("GET", 200, "text/html"))
	third := m.Drain()[AppKey{Acct: "bab", App: "web"}]
	if len(third) != 2 || third[1].Dropped.Flush != 1 {
		t.Fatalf("second failure must drop the oldest and count it: %+v", third)
	}
}

func TestVisitorHashAndEventName(t *testing.T) {
	s1, s2 := []byte("salt-a"), []byte("salt-b")
	if VisitorHash(s1, "h", "ip", "ua") != VisitorHash(s1, "h", "ip", "ua") {
		t.Fatal("deterministic")
	}
	if VisitorHash(s1, "h", "ip", "ua") == VisitorHash(s2, "h", "ip", "ua") {
		t.Fatal("salt must change the hash")
	}
	for in, want := range map[string]string{"Signup": "signup", "checkout.done": "checkout.done", "a b": "", "": "", "x/y": ""} {
		got, ok := EventName(in)
		if (want == "") == ok || got != want {
			t.Errorf("EventName(%q) = %q,%v", in, got, ok)
		}
	}
}

• [ ] Step 2: Run, expect compile failure

• [ ] Step 3: Implement meter.go

package analytics

import (
	"crypto/sha256"
	"encoding/binary"
	"log"
	"regexp"
	"strings"
	"sync"
	"time"

	"github.com/carlosframework/platform/internal/apperr"
	"github.com/carlosframework/platform/internal/applog"
	"github.com/carlosframework/platform/internal/usage"
)

// Obs is one served response as the edge saw it (spec §3.3). The edge
// fills it from the observer defer; nothing here touches the request.
type Obs struct {
	Acct, App, Host string
	Method          string
	Status          int
	ContentType     string
	Path            string // raw; sanitised here
	RefererHost     string // host only, "" = direct
	UA              string
	ClientIP        string
	Bytes           int64
	Prefetch        bool   // Sec-Purpose/Purpose: prefetch
	FetchDest       string // Sec-Fetch-Dest, "" when absent
	When            time.Time
}

// AppKey is the (account, app) the meter buckets by.
type AppKey = applog.AppKey

// IsPageview applies §3.3's rule: GET, 200, text/html, not prefetch, dest
// document or absent, not under /__carlos/. A bot UA is reported
// separately and never a pageview.
func IsPageview(o Obs) (pageview, bot bool) {
	if o.Method != "GET" || o.Status != 200 {
		return false, false
	}
	mt := strings.ToLower(strings.TrimSpace(strings.SplitN(o.ContentType, ";", 2)[0]))
	if mt != "text/html" {
		return false, false
	}
	if o.Prefetch || (o.FetchDest != "" && o.FetchDest != "document") {
		return false, false
	}
	if strings.HasPrefix(o.Path, "/__carlos/") {
		return false, false
	}
	if IsBot(o.UA) {
		return false, true
	}
	return true, false
}

// VisitorHash is sha256(salt ‖ host ‖ ip ‖ ua) folded to 64 bits for the
// sketch; the platform never stores the inputs.
func VisitorHash(salt []byte, host, ip, ua string) uint64 {
	h := sha256.New()
	h.Write(salt)
	h.Write([]byte{0})
	h.Write([]byte(host))
	h.Write([]byte{0})
	h.Write([]byte(ip))
	h.Write([]byte{0})
	h.Write([]byte(ua))
	return binary.BigEndian.Uint64(h.Sum(nil)[:8])
}

var eventName = regexp.MustCompile(`^[a-z0-9_.-]{1,64}$`)

// EventName normalises a named event: lowercased, 64 bytes of [a-z0-9_.-].
func EventName(s string) (string, bool) {
	s = strings.ToLower(strings.TrimSpace(s))
	if !eventName.MatchString(s) {
		return "", false
	}
	return s, true
}

// Meter accumulates per-app deltas between flushes. One per edge process;
// Observe sits on the serving path (one mutex, no I/O except the first
// salt read of a day).
type Meter struct {
	box     string
	saltDir string
	now     func() time.Time

	mu      sync.Mutex
	cur     map[AppKey]map[string]*Delta // app → date → delta being built
	pending map[AppKey][]Delta           // re-queued after a failed flush (one window)
	seq     int64
	saltDay string
	salt    []byte
	started string // date the process started: Restarted is set on that day's first delta
}

// NewMeter returns a meter whose salts live in saltDir (the box data dir).
func NewMeter(saltDir, box string, now func() time.Time) *Meter {
	return &Meter{box: box, saltDir: saltDir, now: now, cur: map[AppKey]map[string]*Delta{},
		pending: map[AppKey][]Delta{}, seq: now().UnixMilli(), started: usage.DayOf(now())}
}

func (m *Meter) saltFor(date string) []byte {
	if m.saltDay == date && m.salt != nil {
		return m.salt
	}
	s, err := DaySalt(m.saltDir, date)
	if err != nil {
		log.Printf("analytics: day salt: %v (visitors not counted until it works)", err)
		return nil
	}
	m.saltDay, m.salt = date, s
	return s
}

func (m *Meter) delta(k AppKey, date string) *Delta {
	days := m.cur[k]
	if days == nil {
		days = map[string]*Delta{}
		m.cur[k] = days
	}
	d := days[date]
	if d == nil {
		d = &Delta{Date: date, Box: m.box, Hours: map[string]HourRow{}, Visitors: &HLL{},
			Pages: NewTopK(), Referers: NewTopK(), Events: NewTopK(), Restarted: date == m.started}
		days[date] = d
	}
	return d
}

// Observe counts one response. Bytes count for every response; pageview,
// bot, visitor, page and referer only when the rule says so.
func (m *Meter) Observe(o Obs) {
	pv, bot := IsPageview(o)
	date, hour := usage.DayOf(o.When), usage.HourLabel(o.When)
	path := apperr.SanitizePath(o.Path)
	m.mu.Lock()
	defer m.mu.Unlock()
	d := m.delta(AppKey{Acct: o.Acct, App: o.App}, date)
	row := d.Hours[hour]
	row.Bytes += o.Bytes
	if bot {
		row.Bots++
	}
	if pv {
		row.PV++
		if salt := m.saltFor(date); salt != nil {
			d.Visitors.Add(VisitorHash(salt, o.Host, o.ClientIP, o.UA))
		}
		d.Pages.Add(path, 1)
		ref := o.RefererHost
		if ref == "" {
			ref = "direct"
		}
		d.Referers.Add(strings.ToLower(ref), 1)
	}
	d.Hours[hour] = row
}

// Track counts one named event (the doors call it; nothing does in step 2).
func (m *Meter) Track(acct, app, name string, when time.Time) {
	n, ok := EventName(name)
	if !ok {
		return
	}
	m.mu.Lock()
	defer m.mu.Unlock()
	m.delta(AppKey{Acct: acct, App: app}, usage.DayOf(when)).Events.Add(n, 1)
}

// Drain returns everything buffered — re-queued deltas first, then the
// current ones, each stamped with a fresh box-monotonic seq — and resets.
func (m *Meter) Drain() map[AppKey][]Delta {
	m.mu.Lock()
	defer m.mu.Unlock()
	out := map[AppKey][]Delta{}
	for k, ds := range m.pending {
		out[k] = append(out[k], ds...)
	}
	m.pending = map[AppKey][]Delta{}
	for k, days := range m.cur {
		for _, d := range days {
			m.seq++
			d.Seq = m.seq
			out[k] = append(out[k], *d)
		}
	}
	m.cur = map[AppKey]map[string]*Delta{}
	if m.started != "" && usage.DayOf(m.now()) != m.started {
		m.started = "" // the restart note is only for the day it happened
	}
	return out
}

// Requeue holds a failed flush for the next drain. One window survives:
// a delta already re-queued once is dropped and counted in Dropped.Flush
// on the newest delta for that app.
func (m *Meter) Requeue(failed map[AppKey][]Delta) {
	m.mu.Lock()
	defer m.mu.Unlock()
	for k, ds := range failed {
		if n := len(m.pending[k]); n > 0 {
			dropped := int64(n)
			m.pending[k] = nil
			if len(ds) > 0 {
				ds[len(ds)-1].Dropped.Flush += dropped
			}
		}
		m.pending[k] = append(m.pending[k], ds...)
	}
}

Note: Requeue must not re-stamp Seq; Drain stamps fresh seqs only on current deltas, and re-queued ones keep the seq they were drained with (they may have landed: the day file's seq rule makes the replay a no-op). Re-check the second test: after the first failed flush, the re-queued delta (seq 1) comes back before the new delta (seq 2); on the second failure both are re-queued, pending[k] had 0 entries at that moment (Drain cleared it), so nothing is dropped yet — adjust the test's third expectation: the drop happens when a delta is re-queued while pending still holds an earlier window, i.e. Requeue(first), then Requeue(second) without a Drain between. Rewrite TestMeterRequeueSurvivesOnce's tail to: m.Requeue(first); m.Requeue(second_without_drain) — concretely: after second := m.Drain(), call m.Requeue(second), then immediately m.Requeue(map[AppKey][]Delta{k: {obsDelta}}) with a hand-built delta, then Drain() and assert the first window is gone and Dropped.Flush == 2 on the last delta. Write whichever exact sequence matches the implementation and state it in the report.

• [ ] Step 4: Run, PASS; gofmt; vet. Commit "analytics: the meter the edge feeds — pageview rule, visitors, top keys, requeue (spec §3.3)".

---

Task 4: Cross-box range read

Files:

• Create: internal/analytics/read.go
• Test: internal/analytics/read_test.go

Interfaces:

• Produces: ``go type DaySummary struct { Date string; PV, Bots, Bytes int64; Visitors int64; Hours [24]int64; Restarted bool } type Summary struct { From, To string; Days []DaySummary; PV, Bots, Bytes, Visitors int64; Pages, Referers, Events map[string]int64; PagesApprox, ReferersApprox, EventsApprox bool; Dropped Dropped; Boxes []string } func ReadRange(ctx, s release.Store, sqid, app string, from, to time.Time) (Summary, error) // inclusive UTC dates func Dates(from, to time.Time) []string ``

• [ ] Step 1: Failing test

package analytics

import (
	"context"
	"testing"
	"time"

	"github.com/carlosframework/platform/internal/release"
)

func TestReadRangeMergesBoxesAndDays(t *testing.T) {
	ctx := context.Background()
	s := release.NewDirStore(t.TempDir())
	d1 := delta("box-a", 1, 10)
	d2 := delta("box-b", 1, 5)
	d3 := delta("box-a", 2, 7)
	d3.Date = "2026-09-08"
	d3.Restarted = true
	for _, d := range []Delta{d1, d2, d3} {
		if err := UpsertDay(ctx, s, "bab", "web", d); err != nil {
			t.Fatal(err)
		}
	}
	from := time.Date(2026, 9, 7, 0, 0, 0, 0, time.UTC)
	sum, err := ReadRange(ctx, s, "bab", "web", from, from.Add(24*time.Hour))
	if err != nil {
		t.Fatal(err)
	}
	if len(sum.Days) != 2 || sum.Days[0].Date != "2026-09-07" || sum.Days[0].PV != 15 || sum.Days[1].PV != 7 {
		t.Fatalf("days = %+v", sum.Days)
	}
	if sum.PV != 22 || sum.Pages["/"] != 22 || sum.Days[0].Hours[9] != 15 || !sum.Days[1].Restarted || sum.Days[0].Restarted {
		t.Fatalf("summary = %+v", sum)
	}
	if sum.Visitors != sum.Days[0].Visitors+sum.Days[1].Visitors || sum.Days[0].Visitors < 1 {
		t.Fatalf("visitors must be daily sums: %+v", sum)
	}
	if len(sum.Boxes) != 2 {
		t.Fatalf("boxes = %v", sum.Boxes)
	}
	// An app with no day files reads as an empty summary, not an error.
	empty, err := ReadRange(ctx, s, "bab", "nothing", from, from)
	if err != nil || empty.PV != 0 || len(empty.Days) != 1 {
		t.Fatalf("empty = %+v %v", empty, err)
	}
}

• [ ] Step 2: Run, expect compile failure

• [ ] Step 3: Implement read.go

package analytics

import (
	"context"
	"encoding/json"
	"errors"
	"math"
	"os"
	"sort"
	"strconv"
	"strings"
	"time"

	"github.com/carlosframework/platform/internal/release"
	"github.com/carlosframework/platform/internal/usage"
)

// DaySummary is one UTC day across every box.
type DaySummary struct {
	Date      string
	PV        int64
	Bots      int64
	Bytes     int64
	Visitors  int64 // HLL estimate, rounded
	Hours     [24]int64
	Restarted bool
}

// Summary is what the tab and the API render: per-day rows plus totals.
// Visitors is the SUM of daily estimates (spec §3.3: "daily visitors,
// summed"), never a cross-day union.
type Summary struct {
	From, To       string
	Days           []DaySummary
	PV, Bots       int64
	Bytes          int64
	Visitors       int64
	Pages          map[string]int64
	Referers       map[string]int64
	Events         map[string]int64
	PagesApprox    bool
	ReferersApprox bool
	EventsApprox   bool
	Dropped        Dropped
	Boxes          []string
}

// Dates lists the UTC dates from from to to inclusive.
func Dates(from, to time.Time) []string {
	var out []string
	for d := from.UTC().Truncate(24 * time.Hour); !d.After(to.UTC()); d = d.Add(24 * time.Hour) {
		out = append(out, usage.DayOf(d))
	}
	return out
}

// ReadRange lists the app's boxes and reads each day file in the range,
// merging across boxes: counters sum, visitor sketches union per day,
// top-key summaries merge and re-fold. A missing day file is an empty
// day; a store error fails the read.
func ReadRange(ctx context.Context, s release.Store, sqid, app string, from, to time.Time) (Summary, error) {
	sum := Summary{From: usage.DayOf(from), To: usage.DayOf(to)}
	boxDirs, err := s.ListDirs(ctx, Prefix(sqid, app))
	if err != nil {
		return sum, err
	}
	for _, dir := range boxDirs {
		sum.Boxes = append(sum.Boxes, boxName(dir))
	}
	sort.Strings(sum.Boxes)
	pages, refs, events := NewTopK(), NewTopK(), NewTopK()
	for _, date := range Dates(from, to) {
		ds := DaySummary{Date: date}
		vis := &HLL{}
		for _, box := range sum.Boxes {
			raw, _, err := s.Get(ctx, DayKey(sqid, app, box, date))
			if errors.Is(err, os.ErrNotExist) {
				continue
			}
			if err != nil {
				return sum, err
			}
			var day Day
			if err := json.Unmarshal(raw, &day); err != nil {
				return sum, err
			}
			for h, row := range day.Hours {
				if i, err := strconv.Atoi(h); err == nil && i >= 0 && i < 24 {
					ds.Hours[i] += row.PV
				}
				ds.PV += row.PV
				ds.Bots += row.Bots
				ds.Bytes += row.Bytes
			}
			if v, err := DecodeHLL(day.Visitors); err == nil {
				vis.Merge(v)
			}
			pages.Merge(TopKFrom(day.Pages))
			refs.Merge(TopKFrom(day.Referers))
			events.Merge(TopKFrom(day.Events))
			sum.Dropped.E = addDrops(sum.Dropped.E, day.Dropped.E)
			sum.Dropped.A = addDrops(sum.Dropped.A, day.Dropped.A)
			sum.Dropped.Flush += day.Dropped.Flush
			ds.Restarted = ds.Restarted || day.Restarted
		}
		if !vis.Empty() {
			ds.Visitors = int64(math.Round(vis.Count()))
		}
		sum.Days = append(sum.Days, ds)
		sum.PV += ds.PV
		sum.Bots += ds.Bots
		sum.Bytes += ds.Bytes
		sum.Visitors += ds.Visitors
	}
	sum.Pages, sum.PagesApprox = pages.Fold(), pages.Approximate()
	sum.Referers, sum.ReferersApprox = refs.Fold(), refs.Approximate()
	sum.Events, sum.EventsApprox = events.Fold(), events.Approximate()
	return sum, nil
}

func boxName(dir string) string {
	t := strings.TrimSuffix(dir, "/")
	if i := strings.LastIndexByte(t, '/'); i >= 0 {
		return t[i+1:]
	}
	return t
}

• [ ] Step 4: Run, PASS. Commit "analytics: range read across boxes and days".

---

Task 5: The edge tap and the switch map

Files:

• Create: internal/edge/analytics.go
• Modify: internal/edge/edge.go (fields; one call in the observer defer)
• Test: internal/edge/analytics_test.go

Interfaces:

• Produces: ``go type AnalyticsMeter interface { Observe(analytics.Obs) } func (e *Edge) EnableAnalytics(m AnalyticsMeter) func (e *Edge) SetAnalyticsOn(on map[string]bool) // key sqid+"/"+app; nil = nothing on func clientIP(r *http.Request) string func (e *Edge) observeAnalytics(rt registry.Route, r *http.Request, ob *countingWriter, bytes int64) ``

• [ ] Step 1: Failing tests

package edge

import (
	"net/http"
	"net/http/httptest"
	"sync"
	"testing"

	"github.com/carlosframework/platform/internal/analytics"
	"github.com/carlosframework/platform/internal/registry"
	"github.com/carlosframework/platform/internal/release"
)

type memAnalytics struct {
	mu  sync.Mutex
	obs []analytics.Obs
}

func (m *memAnalytics) Observe(o analytics.Obs) { m.mu.Lock(); m.obs = append(m.obs, o); m.mu.Unlock() }
func (m *memAnalytics) snap() []analytics.Obs {
	m.mu.Lock()
	defer m.mu.Unlock()
	return append([]analytics.Obs(nil), m.obs...)
}

func TestClientIP(t *testing.T) {
	r := httptest.NewRequest("GET", "http://h/", nil)
	r.RemoteAddr = "203.0.113.5:4444"
	r.Header.Set("X-Forwarded-For", "198.51.100.1")
	if ip := clientIP(r); ip != "203.0.113.5" {
		t.Fatalf("unverified: %q", ip)
	}
	r2 := r.WithContext(withHopVerified(r.Context())) // helper: mark ctx as a verified hop (use hopVerifiedKey)
	r2.Header.Set("X-Forwarded-For", "10.0.0.1, 198.51.100.7")
	if ip := clientIP(r2); ip != "198.51.100.7" {
		t.Fatalf("verified hop: %q", ip)
	}
}

func TestEdgeObservesOnlyWhenSwitchedOn(t *testing.T) {
	backend := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		w.Header().Set("Content-Type", "text/html; charset=utf-8")
		w.Write([]byte("<h1>hi</h1>"))
	}))
	t.Cleanup(backend.Close)
	reg := openReg(t)
	rt := registry.Route{Host: "app.example.com", Kind: "service", Addr: backend.Listener.Addr().String(), Status: "ready", App: "web"}
	if err := reg.Upsert(rt); err != nil {
		t.Fatal(err)
	}
	e, err := New(reg, nil)
	if err != nil {
		t.Fatal(err)
	}
	m := &memAnalytics{}
	e.EnableAnalytics(m)
	srv := httptest.NewServer(e)
	t.Cleanup(srv.Close)
	get := func(path string, hdr map[string]string) {
		req, _ := http.NewRequest("GET", srv.URL+path, nil)
		req.Host = "app.example.com"
		for k, v := range hdr {
			req.Header.Set(k, v)
		}
		resp, err := http.DefaultClient.Do(req)
		if err != nil {
			t.Fatal(err)
		}
		resp.Body.Close()
	}
	get("/", nil)
	if len(m.snap()) != 0 {
		t.Fatal("nothing is observed while the app is off")
	}
	e.SetAnalyticsOn(map[string]bool{release.OpsAccount + "/web": true})
	get("/pricing?x=1", map[string]string{"Referer": "https://news.ycombinator.com/item?id=1", "User-Agent": "Mozilla/5.0 Safari", "Sec-Fetch-Dest": "document"})
	obs := m.snap()
	if len(obs) != 1 {
		t.Fatalf("observed %d", len(obs))
	}
	o := obs[0]
	if o.Acct != release.OpsAccount || o.App != "web" || o.Host != "app.example.com" || o.Method != "GET" || o.Status != 200 ||
		o.ContentType != "text/html; charset=utf-8" || o.Path != "/pricing" || o.RefererHost != "news.ycombinator.com" ||
		o.FetchDest != "document" || o.Bytes == 0 || o.ClientIP == "" || o.When.IsZero() {
		t.Fatalf("obs = %+v", o)
	}
	get("/x", map[string]string{"Purpose": "prefetch"})
	if !m.snap()[1].Prefetch {
		t.Fatal("prefetch header must be carried")
	}
	e.SetAnalyticsOn(nil)
	get("/", nil)
	if len(m.snap()) != 2 {
		t.Fatal("switching off stops observation")
	}
}

If there is no withHopVerified helper, add a tiny unexported one in analytics.go (func withHopVerified(ctx context.Context) context.Context { return context.WithValue(ctx, hopVerifiedKey{}, true) }) matching how acceptHop sets the marker (read homemap.go:767-798 for the exact key and value).

• [ ] Step 2: Run, expect compile failure

• [ ] Step 3: Implement internal/edge/analytics.go

package edge

import (
	"net"
	"net/http"
	"net/url"
	"strings"
	"sync/atomic"
	"time"

	"github.com/carlosframework/platform/internal/analytics"
	"github.com/carlosframework/platform/internal/registry"
	"github.com/carlosframework/platform/internal/release"
)

// AnalyticsMeter is where the edge files one observation per response
// for an app whose analytics are switched on (spec §3.3). analytics.Meter
// satisfies it; the interface keeps the dependency on the contract.
type AnalyticsMeter interface {
	Observe(analytics.Obs)
}

type analyticsMeterRef struct{ AnalyticsMeter }

// EnableAnalytics wires the meter in; nil is a valid no-op.
func (e *Edge) EnableAnalytics(m AnalyticsMeter) {
	if m == nil {
		e.analyticsMeter.Store(nil)
		return
	}
	e.analyticsMeter.Store(&analyticsMeterRef{m})
}

// SetAnalyticsOn replaces the switch map: sqid+"/"+app → counted. The
// host's telemetry pass pushes it on every serial-triggered full pass and
// every five minutes (spec §3.3, "converged by boxes on the next pass").
func (e *Edge) SetAnalyticsOn(on map[string]bool) {
	if on == nil {
		e.analyticsOn.Store(nil)
		return
	}
	e.analyticsOn.Store(&on)
}

func (e *Edge) analyticsEnabled(rt registry.Route) bool {
	m := e.analyticsOn.Load()
	if m == nil {
		return false
	}
	return (*m)[release.RouteAccount(rt)+"/"+rt.App]
}

// clientIP is the address the visitor hash uses (spec §3.3): the last
// element of the verified inbound X-Forwarded-For chain on a proven hop,
// otherwise RemoteAddr's host. A header the client set never reaches it.
func clientIP(r *http.Request) string {
	if hopVerified(r.Context()) {
		chain := strings.Join(r.Header.Values("X-Forwarded-For"), ", ")
		if i := strings.LastIndexByte(chain, ','); i >= 0 {
			chain = chain[i+1:]
		}
		if ip := strings.TrimSpace(chain); ip != "" {
			return ip
		}
	}
	host, _, err := net.SplitHostPort(r.RemoteAddr)
	if err != nil {
		return r.RemoteAddr
	}
	return host
}

// observeAnalytics files one Obs when the route's app is switched on.
// Runs in the observer defer after the response completed; reads only.
func (e *Edge) observeAnalytics(rt registry.Route, r *http.Request, ob *countingWriter, bytes int64) {
	ref := e.analyticsMeter.Load()
	if ref == nil || !e.analyticsEnabled(rt) {
		return
	}
	refHost := ""
	if u, err := url.Parse(r.Header.Get("Referer")); err == nil {
		refHost = u.Hostname()
	}
	prefetch := strings.EqualFold(r.Header.Get("Sec-Purpose"), "prefetch") || strings.EqualFold(r.Header.Get("Purpose"), "prefetch")
	ref.Observe(analytics.Obs{
		Acct: release.RouteAccount(rt), App: rt.App, Host: rt.Host,
		Method: r.Method, Status: ob.status, ContentType: ob.ctype,
		Path: r.URL.Path, RefererHost: refHost, UA: r.Header.Get("User-Agent"),
		ClientIP: clientIP(r), Bytes: bytes,
		Prefetch: prefetch, FetchDest: strings.ToLower(r.Header.Get("Sec-Fetch-Dest")),
		When: time.Now(),
	})
}

Add to Edge:

	analyticsMeter atomic.Pointer[analyticsMeterRef]
	analyticsOn    atomic.Pointer[map[string]bool]

and in the observer defer (edge.go ~1300), after the edge-error line:

			e.observeAnalytics(up.route, r, ob, bytes)

ob.status == 0 (nothing written) is never a pageview by IsPageview (status must be 200), and bytes are 0 — the meter still records an app-day with zero counts; acceptable.

• [ ] Step 4: Run go test ./internal/edge/ -count=1 (TMPDIR=/tmp/g), PASS including every pre-existing test. Commit "edge: observe responses for switched-on apps (spec §3.3)".

---

Task 6: Host — switch pass, meter, flush on the usage tick

Files:

• Create: internal/host/telemetry_pass.go
• Modify: internal/host/usage_loop.go, internal/host/agent.go
• Test: internal/host/telemetry_pass_test.go, internal/host/usage_loop_test.go (append)

Interfaces:

• Produces: ``go func telemetrySwitches(ctx, s release.Store, reg *registry.Registry) (map[string]bool, error) // sqid/app → count==edge for the routes this box serves func setAnalyticsSink(fn func(map[string]bool)) // agent.go callback registers e.SetAnalyticsOn func telemetryPass(ctx, s, reg) // resolves and pushes; logs errors func analyticsFlush(ctx, s release.Store, sqid... ) — folded into usageFlush: usageFlush(ctx, s, box, drain, am *analytics.Meter, now) error ``

• [ ] Step 1: Failing tests

telemetry_pass_test.go:

package host

import (
	"context"
	"testing"

	"github.com/carlosframework/platform/internal/registry"
	"github.com/carlosframework/platform/internal/release"
	"github.com/carlosframework/platform/internal/telemetry"
)

func TestTelemetrySwitchesResolvePerRoute(t *testing.T) {
	ctx := context.Background()
	s := release.NewDirStore(t.TempDir())
	reg := openTestRegistry(t) // use whatever helper host tests use to open a temp registry
	for _, rt := range []registry.Route{
		{Host: "a.example", Kind: "instance", Account: "bab", App: "web", Status: "ready"},
		{Host: "b.example", Kind: "service", Account: "bab", App: "api", Status: "ready"},
		{Host: "c.example", Kind: "static", Account: "zzz", App: "site", Status: "ready"},
		{Host: "status.example", Kind: "status", Status: "ready"},
	} {
		if err := reg.Upsert(rt); err != nil {
			t.Fatal(err)
		}
	}
	telemetry.Update(ctx, s, telemetry.AccountKey("bab"), func(st *telemetry.Settings) { st.Analytics = &telemetry.Analytics{Count: telemetry.CountEdge} })
	telemetry.Update(ctx, s, telemetry.AppKey("bab", "api"), func(st *telemetry.Settings) { st.Analytics = &telemetry.Analytics{Count: telemetry.CountOff} })
	on, err := telemetrySwitches(ctx, s, reg)
	if err != nil {
		t.Fatal(err)
	}
	if !on["bab/web"] || on["bab/api"] || on["zzz/site"] {
		t.Fatalf("switches = %v", on)
	}
	if _, ok := on["/"]; ok {
		t.Fatal("status routes have no app and must not appear")
	}
}

Find the helper host tests use to open a temporary registry (grep registry.Open(filepath.Join(t.TempDir() in internal/host/*_test.go); use it or open one inline. The route's account column name: check registry.Route for the field release.RouteAccount reads (Account), and set it so RouteAccount returns the sqid.

usage_loop_test.go append:

func TestUsageFlushAlsoFlushesAnalytics(t *testing.T) {
	ctx := context.Background()
	s := release.NewDirStore(t.TempDir())
	now := time.Date(2026, 9, 7, 9, 30, 0, 0, time.UTC)
	am := analytics.NewMeter(t.TempDir(), "box1", func() time.Time { return now })
	am.Observe(analytics.Obs{Acct: "bab", App: "web", Host: "h", Method: "GET", Status: 200, ContentType: "text/html", Path: "/", UA: "Mozilla/5.0 Safari", ClientIP: "1.2.3.4", Bytes: 10, When: now})
	if err := usageFlush(ctx, s, "box1", func() map[string]usage.EdgeRow { return nil }, am, now); err != nil {
		t.Fatal(err)
	}
	sum, err := analytics.ReadRange(ctx, s, "bab", "web", now, now)
	if err != nil || sum.PV != 1 {
		t.Fatalf("flushed day: %+v %v", sum, err)
	}
	// A failed flush re-queues; the next flush lands it.
	am.Observe(analytics.Obs{Acct: "bab", App: "web", Host: "h", Method: "GET", Status: 200, ContentType: "text/html", Path: "/", UA: "Mozilla/5.0 Safari", ClientIP: "1.2.3.4", Bytes: 10, When: now})
	_ = usageFlush(ctx, failingStore{}, "box1", func() map[string]usage.EdgeRow { return nil }, am, now)
	if err := usageFlush(ctx, s, "box1", func() map[string]usage.EdgeRow { return nil }, am, now); err != nil {
		t.Fatal(err)
	}
	if sum, _ := analytics.ReadRange(ctx, s, "bab", "web", now, now); sum.PV != 2 {
		t.Fatalf("after requeue: %d", sum.PV)
	}
}

(failingStore exists in logship_test.go; if its Get returns an error other than not-found, UpsertDay returns that error, which is what re-queues. Check failingStore's shape and adapt.)

• [ ] Step 2: Run, expect compile failure

• [ ] Step 3: Implement telemetry_pass.go

// The analytics switch pass (spec 2026-09-07 §3.3): resolve
// telemetry.json app → account → default for every (account, app) this
// box serves and hand the edge a fresh switch map. Rides the serial-
// triggered full pass and the five-minute instances loop, so a console
// write lands within seconds and the hourly floor stands underneath.
package host

import (
	"context"
	"log"
	"sync/atomic"

	"github.com/carlosframework/platform/internal/registry"
	"github.com/carlosframework/platform/internal/release"
	"github.com/carlosframework/platform/internal/telemetry"
)

var analyticsSink atomic.Pointer[func(map[string]bool)]

// setAnalyticsSink registers where a resolved switch map goes (the edge's
// SetAnalyticsOn), the way setUpstreamSource publishes the prober table.
func setAnalyticsSink(fn func(map[string]bool)) { analyticsSink.Store(&fn) }

// telemetrySwitches resolves the count switch for each distinct
// (account, app) among the box's routes. Status routes carry no app and
// are skipped. One Resolve (two Gets) per pair; a pair that fails to
// resolve is left OFF and logged — off is the safe side of this switch.
func telemetrySwitches(ctx context.Context, s release.Store, reg *registry.Registry) (map[string]bool, error) {
	routes, err := reg.List()
	if err != nil {
		return nil, err
	}
	on := map[string]bool{}
	seen := map[string]bool{}
	for _, rt := range routes {
		if rt.Kind == "status" || rt.App == "" {
			continue
		}
		sqid := release.RouteAccount(rt)
		key := sqid + "/" + rt.App
		if seen[key] {
			continue
		}
		seen[key] = true
		res, err := telemetry.Resolve(ctx, s, sqid, rt.App)
		if err != nil {
			log.Printf("agent: telemetry resolve %s: %v (analytics stays off)", key, err)
			continue
		}
		if res.Count == telemetry.CountEdge {
			on[key] = true
		}
	}
	return on, nil
}

// telemetryPass resolves and pushes; errors are one log line each.
func telemetryPass(ctx context.Context, s release.Store, reg *registry.Registry) {
	sink := analyticsSink.Load()
	if sink == nil {
		return
	}
	on, err := telemetrySwitches(ctx, s, reg)
	if err != nil {
		log.Printf("agent: telemetry pass: %v", err)
		return
	}
	(*sink)(on)
}

Confirm reg.List()'s signature (edge.go:435 calls it) and the Route field the account lives in.

• [ ] Step 4: Wire the pass and the flush in agent.go and usage_loop.go

usage_loop.go:

// usageFlush drains the edge meter and CAS-merges the deltas into the
// box's usage/edge day object, then flushes the analytics meter's deltas
// into their app-day objects (spec §3.3: the same tick, a second CAS
// write). A failed analytics batch is re-queued for the next tick.
func usageFlush(ctx context.Context, s release.Store, box string, drain func() map[string]usage.EdgeRow, am *analytics.Meter, now time.Time) error {
	err := usage.UpsertEdgeDay(ctx, s, box, now, drain())
	if am == nil {
		return err
	}
	failed := map[analytics.AppKey][]analytics.Delta{}
	for k, ds := range am.Drain() {
		for _, d := range ds {
			if uerr := analytics.UpsertDay(ctx, s, k.Acct, k.App, d); uerr != nil {
				log.Printf("agent: analytics flush %s/%s %s: %v (re-queued)", k.Acct, k.App, d.Date, uerr)
				failed[k] = append(failed[k], d)
			}
		}
	}
	if len(failed) > 0 {
		am.Requeue(failed)
	}
	return err
}

usageLoop(drain, am) passes am through. In agent.go: construct am := analytics.NewMeter(dataDir(), status.Label(readMachineID()), time.Now) beside the usage meter inside the RunWithActivatorReady callback; e.EnableAnalytics(am); setAnalyticsSink(e.SetAnalyticsOn); go usageLoop(m.Drain, am). Note: usageLoop opens its own store; the analytics flush uses the same s and the keys are NOT rebased (they are the app's own data under apps/…, like logs — read logship.go:54-61's reasoning; on a fleet box the door admits analytics/ since step 1's ACL). If usageLoop wraps its store with NewPrefixStore(raw, resolveStatusPrefix(...)), the analytics writes must go to the UNPREFIXED raw store — pass both, or open the raw store for analytics the way logshipLoop does. Do that: usageLoop keeps s (prefixed) for usage and raw for analytics.

Pass wiring: in d.fullPass (agent.go ~1505) add telemetryPass(ctx, store, reg) after reconcileSerialized; in instancesLoop's tick add the same call after its reconcile. Both have store and reg in scope (verify; instancesLoop(reg) may open its own store — read its body and use that variable).

• [ ] Step 5: go build ./... && go test ./internal/host/ -count=1 PASS. Commit "host: resolve analytics switches per pass; flush the analytics meter on the usage tick".

---

Task 7: Console — settings switches, PUT accepts analytics, serial bump

Files:

• Modify: internal/console/apierrors.go (handleAPITelemetryPut), internal/console/errorsview.go or new internal/console/analyticsview.go (two settings handlers), internal/console/server.go (routes, page fields AnalyticsSetting telemetry.Resolved, AnalyticsUnread bool, populate in both settings handlers beside ErrLevelSetting), templates/app_settings.html, templates/account_settings.html
• Test: internal/console/apianalytics_test.go (settings part)

Interfaces:

• Produces: POST /a/{sqid}/apps/{app}/analytics/setting and POST /a/{sqid}/analytics/setting with form fields count (edge|off) and door (on|off), owner + CSRF, audited, serial bump; PUT …/telemetry accepts {"analytics":{"count":…,"door":…}} (validated: count ∈ {edge,off}, door ∈ {on,off}; either key optional) alongside errors, bumps the serial when analytics was present.

• [ ] Step 1: Failing test

func TestAPITelemetryPutSetsAnalytics(t *testing.T) {
	fx, _ := instanceFixture(t, "code-analytics-set")
	op := serviceToken(t, fx.st, OpsAccountID, "ops", "titogo", svcrole.Operate)
	put := func(body string) (int, string) {
		st, _, b := rawReq(t, http.MethodPut, fx.ts.URL+"/api/cli/apps/"+fx.sqid+"/titogo/telemetry", op, body)
		return st, b
	}
	if st, b := put(`{"analytics":{"count":"edge"}}`); st != 200 {
		t.Fatalf("count=edge: %d %s", st, b)
	}
	r, _ := telemetry.Resolve(context.Background(), fx.rel, fx.sqid, "titogo")
	if r.Count != "edge" || r.CountFrom != "app" || r.Door != "on" {
		t.Fatalf("resolved = %+v", r)
	}
	if st, _ := put(`{"analytics":{"count":"browser"}}`); st != 400 {
		t.Fatal("reserved mode must be refused for now")
	}
	if st, _ := put(`{"analytics":{"door":"sideways"}}`); st != 400 {
		t.Fatal("bad door")
	}
	if st, _ := put(`{"analytics":{"door":"off"},"errors":{"log_level":"warn"}}`); st != 200 {
		t.Fatal("both halves in one PUT")
	}
	r, _ = telemetry.Resolve(context.Background(), fx.rel, fx.sqid, "titogo")
	if r.Door != "off" || r.LogLevel != "warn" || r.Count != "edge" {
		t.Fatalf("after both: %+v", r)
	}
	// The serial moved: boxes converge on it.
	if doc, _, err := release.ReadSerial(context.Background(), fx.rel); err != nil || doc.Serial == 0 {
		t.Fatalf("serial not bumped: %+v %v", doc, err)
	}
}

func TestSettingsPagesShowAnalyticsSwitch(t *testing.T) {
	fx, _ := instanceFixture(t, "code-analytics-page")
	fx.setUser("paul@keymail.dev")
	c := newBrowser(t)
	signUp(t, fx, c, "code-analytics-signin")
	status, _, body := get(t, c, fx.u("/apps/titogo/settings"))
	if status != 200 || !strings.Contains(body, "analytics/setting") || !strings.Contains(body, "currently off, from the default") {
		t.Fatalf("app settings: %d", status)
	}
	// Post the account-level switch on; the app page shows it inherited.
	status, _, _ = postForm(t, c, fx.u("/analytics/setting"), url.Values{"csrf": {csrfFor(t, c, fx)}, "count": {"edge"}, "door": {"on"}})
	if status != 303 {
		t.Fatalf("account switch: %d", status)
	}
	_, _, body = get(t, c, fx.u("/apps/titogo/settings"))
	if !strings.Contains(body, "currently edge, from the account") {
		t.Fatal("inheritance not shown")
	}
}

Use the form-post and CSRF helpers the existing settings tests use (grep csrf in appsettings_test.go for how a test obtains the token and posts a form; adapt the two helper names). Check release.ReadSerial's return shape (serial.go:32).

• [ ] Step 2: Run, expect failure

• [ ] Step 3: Implement

In handleAPITelemetryPut: replace the body.Analytics != nil → 400 block with validation:

	if a := body.Analytics; a != nil {
		if a.Count != "" && a.Count != telemetry.CountEdge && a.Count != telemetry.CountOff {
			jsonError(w, http.StatusBadRequest, "analytics.count: want edge or off")
			return
		}
		if a.Door != "" && a.Door != telemetry.DoorOn && a.Door != telemetry.DoorOff {
			jsonError(w, http.StatusBadRequest, "analytics.door: want on or off")
			return
		}
	}
	if body.Errors == nil && body.Analytics == nil { 400 "nothing to set" }
	if body.Errors != nil && !telemetry.ValidLogLevel(...) { 400 as today }

and in the Update closure set st.Errors when given and merge st.Analytics field-by-field (only the keys present, keeping the other). After a successful write with body.Analytics != nil, call s.bumpSerial(ctx) (box-consumed; the errors level is console-read and needs none). Audit text names what changed.

New handlers (in analyticsview.go, mirroring handleAppErrorLevel/handleAccountErrorLevel exactly, including the nil-Releases guard): read count and door form values, validate as above (empty = leave unchanged), telemetry.Update on AppKey/AccountKey merging into st.Analytics, s.audit(...), s.bumpSerial(r.Context()), flash "Analytics counting is now edge for this app" / "…off…", redirect back to the settings page. Routes beside the errors ones.

Page fields and population: beside ErrLevelSetting/ErrLevelUnread add AnalyticsSetting telemetry.Resolved and AnalyticsUnread bool; since Resolve already returns both halves, reuse the one call: set p.AnalyticsSetting = p.ErrLevelSetting and p.AnalyticsUnread = p.ErrLevelUnread in both settings handlers (comment that they are one object).

Templates: after the Errors section in both files:

<h2>Analytics</h2>
{{if .AnalyticsUnread}}
<p class="small">This app&rsquo;s analytics setting could not be read just now. Reload before concluding anything.</p>
{{else if .IsOwner}}
<form method="post" action="{{.Base}}/apps/{{.Detail.App}}/analytics/setting" class="settings-row">
  <input type="hidden" name="csrf" value="{{.CSRF}}">
  <label>Count page views at the edge:
    <select name="count">
      <option value="off"{{if eq .AnalyticsSetting.Count "off"}} selected{{end}}>off</option>
      <option value="edge"{{if eq .AnalyticsSetting.Count "edge"}} selected{{end}}>on (edge)</option>
    </select>
  </label>
  <button type="submit" class="btn-sm">Save</button>
  <span class="small">currently {{.AnalyticsSetting.Count}}, from the {{.AnalyticsSetting.CountFrom}}</span>
</form>
{{else}}
<p class="small">Analytics counting is <code>{{.AnalyticsSetting.Count}}</code> for this app, from the {{.AnalyticsSetting.CountFrom}}. Only the account owner can change it.</p>
{{end}}
<p class="small">When on, the edge counts HTML page views and daily visitors as it serves them: no JavaScript, no cookie, no stored IP. A visitor is a salted hash that changes every day and cannot be joined across days. Whether that needs consent where your visitors are is your call, which is why it is off until you turn it on.</p>

The account form: same fields, action {{.Base}}/analytics/setting, copy "for every app without its own setting". The door select is NOT rendered in step 2 (no door exists yet); the handlers accept it for step 3.

• [ ] Step 4: go test ./internal/console/ -count=1 PASS. Commit "console: the analytics switch at both levels; telemetry PUT accepts analytics and bumps the serial".

---

Task 8: Console — Analytics tab and API

Files:

• Create: internal/console/analyticsview.go (add to the file from Task 7), internal/console/apianalytics.go, templates/app_analytics.html
• Modify: server.go (route GET /a/{sqid}/apps/{app}/analytics, page fields), authcli.go (cliRoute), appnav.html
• Test: internal/console/apianalytics_test.go (tab + API parts)

Interfaces:

• Produces: ``go type analyticsWindow struct { Sum analytics.Summary; Range string; On bool; Setting telemetry.Resolved; ChartBars []chartBar; ChartMax int64 } type chartBar struct { Label string; Value int64; X, Y, W, H float64 } func (s *Server) readAnalytics(ctx, sqid, app, rng string, now time.Time) (analyticsWindow, error) // rng: today|7d|30d func chartBars(vals []int64, labels []string, w, h float64) ([]chartBar, int64) type analyticsResponse struct { Range string; From, To string; On bool; Count, CountFrom string; PV, Visitors, Bytes, Bots int64; Days []analytics.DaySummary; Pages, Referers, Events map[string]int64; PagesApprox, ReferersApprox, EventsApprox bool; Dropped analytics.Dropped } // json tags snake_case ` Routes: GET /api/cli/apps/{account}/{app}/analytics?range=today|7d|30d (Instance, Operate); page GET /a/{sqid}/apps/{app}/analytics?range=…`.

• [ ] Step 1: Failing tests

func seedAnalytics(t *testing.T, fx *consoleFixture, now time.Time) {
	t.Helper()
	m := analytics.NewMeter(t.TempDir(), "box1", func() time.Time { return now })
	for i := 0; i < 5; i++ {
		m.Observe(analytics.Obs{Acct: release.OpsAccount, App: "titogo", Host: "titogo.example.com", Method: "GET", Status: 200,
			ContentType: "text/html", Path: "/pricing", RefererHost: "news.ycombinator.com", UA: "Mozilla/5.0 Safari",
			ClientIP: "203.0.113." + strconv.Itoa(i), Bytes: 1000, When: now.Add(-time.Hour)})
	}
	m.Observe(analytics.Obs{Acct: release.OpsAccount, App: "titogo", Host: "titogo.example.com", Method: "GET", Status: 200,
		ContentType: "text/html", Path: "/", UA: "Googlebot/2.1", ClientIP: "1.1.1.1", Bytes: 500, When: now.Add(-time.Hour)})
	m.Track(release.OpsAccount, "titogo", "signup", now.Add(-time.Hour))
	for k, ds := range m.Drain() {
		for _, d := range ds {
			if err := analytics.UpsertDay(context.Background(), fx.rel, k.Acct, k.App, d); err != nil {
				t.Fatal(err)
			}
		}
	}
}

func TestAPIAnalyticsSummary(t *testing.T) {
	fx, token := instanceFixture(t, "code-analytics-api")
	now := time.Now().UTC()
	seedAnalytics(t, fx, now)
	status, _, body := rawReq(t, http.MethodGet, fx.ts.URL+"/api/cli/apps/"+fx.sqid+"/titogo/analytics?range=today", token, "")
	if status != 200 {
		t.Fatalf("%d %s", status, body)
	}
	var resp analyticsResponse
	json.Unmarshal([]byte(body), &resp)
	if resp.PV != 5 || resp.Bots != 1 || resp.Bytes != 5500 || resp.Visitors < 4 || resp.Visitors > 6 ||
		resp.Pages["/pricing"] != 5 || resp.Referers["news.ycombinator.com"] != 5 || resp.Events["signup"] != 1 || resp.On {
		t.Fatalf("resp = %+v", resp)
	}
	if st, _, _ := rawReq(t, http.MethodGet, fx.ts.URL+"/api/cli/apps/"+fx.sqid+"/titogo/analytics?range=fortnight", token, ""); st != 400 {
		t.Fatal("bad range")
	}
	if st, _, b := rawReq(t, http.MethodGet, fx.ts.URL+"/api/cli/apps/"+fx.sqid+"/titogo/analytics?range=7d", token, ""); st != 200 {
		t.Fatalf("7d: %d %s", st, b)
	}
}

func TestAppAnalyticsPage(t *testing.T) {
	fx, _ := instanceFixture(t, "code-analytics-tab")
	now := time.Now().UTC()
	seedAnalytics(t, fx, now)
	fx.setUser("paul@keymail.dev")
	c := newBrowser(t)
	signUp(t, fx, c, "code-analytics-tab-signin")
	status, _, body := get(t, c, fx.u("/apps/titogo/analytics"))
	if status != 200 {
		t.Fatalf("%d %s", status, body)
	}
	// Off: the switch and the reading, and the numbers that exist already.
	for _, want := range []string{"Analytics is off for this app", "no JavaScript, no cookie", "/pricing", "news.ycombinator.com", "signup", "<svg"} {
		if !strings.Contains(body, want) {
			t.Errorf("page missing %q", want)
		}
	}
	telemetry.Update(context.Background(), fx.rel, telemetry.AppKey(fx.sqid, "titogo"), func(st *telemetry.Settings) {
		st.Analytics = &telemetry.Analytics{Count: telemetry.CountEdge}
	})
	_, _, body = get(t, c, fx.u("/apps/titogo/analytics?range=7d"))
	if strings.Contains(body, "Analytics is off") || !strings.Contains(body, "daily visitors, summed") || !strings.Contains(body, "carlos analytics --app titogo") {
		t.Fatal("on + 7d page")
	}
}

• [ ] Step 2: Run, expect failure

• [ ] Step 3: Implement

readAnalytics: parse rng (today → from = today 00:00 UTC, to = now; 7d → 6 days ago 00:00 to now; 30d → 29 days ago), telemetry.Resolve for On/Setting, analytics.ReadRange, then chart: for today the 24 hourly PVs of Days[0] with labels "00"…"23"; otherwise one bar per day labelled MM-DD. chartBars(vals, labels, 480, 120) computes rect geometry: W = 480/len - 2, H = v/max*120, Y = 120-H, X = i*480/len; ChartMax for the axis note.

handleAppAnalytics via appPage(w, r, "analytics"), range default today, invalid → today; AnalyticsWin analyticsWindow, AnalyticsRange, AnalyticsErr page fields. handleAPIAnalytics via instanceGate, range param validated (400 range: want today, 7d or 30d), readAnalytics, then analyticsResponse (nil maps → {}), writeJSON.

Template app_analytics.html:

{{define "app_analytics"}}{{template "top" .}}
{{if .Flash}}<p class="banner">{{.Flash}}</p>{{end}}
{{if .FlashErr}}<p class="banner error">{{.FlashErr}}</p>{{end}}
{{template "appnav" .}}
<h2>Analytics</h2>
{{if .AnalyticsErr}}<p class="banner error">{{.AnalyticsErr}}</p>{{end}}
{{if not .AnalyticsWin.On}}
<p><strong>Analytics is off for this app</strong> (from the {{.AnalyticsWin.Setting.CountFrom}}).
When on, the edge counts HTML page views and daily visitors as it serves them: no JavaScript, no cookie,
no stored IP. A visitor is a salted hash that changes every day. Turn it on for this app or for the whole
account on the <a href="{{.Base}}/apps/{{.AppName}}/settings">Settings</a> tab.</p>
{{end}}
<form method="get" class="small">
  <label>range
    <select name="range">
      <option value="today"{{if eq .AnalyticsRange "today"}} selected{{end}}>today</option>
      <option value="7d"{{if eq .AnalyticsRange "7d"}} selected{{end}}>7 days</option>
      <option value="30d"{{if eq .AnalyticsRange "30d"}} selected{{end}}>30 days</option>
    </select>
  </label>
  <button type="submit">Update</button>
</form>
{{with .AnalyticsWin}}
<div class="tablewrap"><table>
  <tr><th>Page views</th><th>Visitors{{if ne $.AnalyticsRange "today"}} (daily visitors, summed){{end}}</th><th>Bytes served</th><th>Bots excluded</th><th>Dropped</th></tr>
  <tr><td>{{.Sum.PV}}</td><td>{{.Sum.Visitors}}</td><td>{{.Sum.Bytes}}</td><td>{{.Sum.Bots}}</td><td>{{.Sum.Dropped.Flush}}</td></tr>
</table></div>
{{if .ChartBars}}
<svg viewBox="0 0 480 140" width="100%" role="img" aria-label="Page views">
  {{range .ChartBars}}<rect x="{{.X}}" y="{{.Y}}" width="{{.W}}" height="{{.H}}" fill="currentColor"><title>{{.Label}}: {{.Value}}</title></rect>{{end}}
  <text x="0" y="136" class="small">{{if eq $.AnalyticsRange "today"}}per hour, UTC{{else}}per day{{end}}; peak {{.ChartMax}}</text>
</svg>
{{end}}
{{range .Sum.Days}}{{if .Restarted}}<p class="small">{{.Date}}: counts since the last flush may be missing (the edge restarted that day).</p>{{end}}{{end}}
<h3>Top pages{{if .Sum.PagesApprox}} <span class="small">approximate: more than 500 distinct keys</span>{{end}}</h3>
{{template "kvtable" .Sum.Pages}}
<h3>Top referers{{if .Sum.ReferersApprox}} <span class="small">approximate</span>{{end}}</h3>
{{template "kvtable" .Sum.Referers}}
<h3>Named events{{if .Sum.EventsApprox}} <span class="small">approximate</span>{{end}}</h3>
{{template "kvtable" .Sum.Events}}
{{end}}
<p class="small">Times are UTC. The same view from a terminal: <code>carlos analytics --app {{.AppName}} --range {{.AnalyticsRange}}</code>.</p>
{{template "bottom" .}}{{end}}
{{define "kvtable"}}{{if .}}<div class="tablewrap"><table>{{range $k, $v := sortedKV .}}<tr><td>{{$k.Key}}</td><td>{{$k.Value}}</td></tr>{{end}}</table></div>{{else}}<p class="small">Nothing yet.</p>{{end}}{{end}}

Add a template func sortedKV (map → slice of {Key string; Value int64} sorted by value desc, _other last) in server.go's funcMap beside logLines; fix the range variable usage accordingly ({{range sortedKV .}}<tr><td>{{.Key}}</td><td>{{.Value}}</td></tr>{{end}}).

appnav.html: add Analytics after Errors. authcli.go: the cliRoute beside /errors. svcroutes_test.go: add the analytics door to exemptFromLegacyPin with the same reasoning as errors.

• [ ] Step 4: go test ./internal/console/ -count=1 PASS. Commit "console: Analytics tab and API (spec §3.6)".

---

Task 9: carlos analytics and the docs heading

Files:

• Create: analytics_cmd.go, analytics_cmd_test.go
• Modify: apimode.go (apiAnalyticsResponse, analyticsGet), main.go (dispatch + help line), docs/site/cli.md (### carlos analytics after ### carlos errors)

• [ ] Step 1: Failing test TestPrintAnalytics — feed an apiAnalyticsResponse with PV 5, Visitors 4, two pages, one referer, one event, Range: "7d", and assert the output contains "5", "4", "daily visitors, summed", "/pricing", "news.ycombinator.com", "signup".

• [ ] Step 2: Implement runAnalytics(args): flags --app, --range (default today), --set-count edge|off, --set-door on|off, plus console/account flags as errors_cmd.go. --set-* short-circuit into sess.telemetryPut(ctx, acct, app2, map[string]any{"analytics": map[string]string{...}}) and print "Analytics counting for X is now edge/off." Otherwise sess.analyticsGet(ctx, acct, app2, url.Values{"range": …}) and printAnalytics(stdout, res): a headline line (page views N · visitors N (daily visitors, summed) · bytes N · bots N · dropped N), a text sparkline of Days (or hours for today) using the block characters ▁▂▃▄▅▆▇█, then the three tables sorted by value desc with _other last, and, when !res.On, a muted line "analytics is off for this app — turn it on with carlos analytics --app X --set-count edge". apiAnalyticsResponse mirrors analyticsResponse's JSON tags.

docs/site/cli.md:

### carlos analytics

Page views, daily visitors, bytes served, top pages, top referers and named
events, counted at the edge as it serves your app: no JavaScript, no cookie,
no stored IP. Off until you turn it on, for one app or for the whole account.

```sh
carlos analytics --app hello --set-count edge
carlos analytics --app hello --range 7d

--range is today, 7d or 30d; multi-day visitor totals are daily visitors summed. The same view is the Analytics tab on the app's dashboard, and the switch is on the Settings tab too.

- [ ] **Step 3: `go build ./... && go test . -run TestPrintAnalytics -count=1 && go test ./internal/docsite/ -count=1`; prove the docsite gate fails with the heading removed, restore. Commit** "carlos analytics: the app's page views and visitors from a terminal, and the switch".

---

### Task 10: Whole-tree gate, push, branch description

- [ ] `go build ./... && go vet ./... && gofmt -l . && go test ./... -count=1` (TMPDIR=/tmp/g), then `GOOS=darwin GOARCH=arm64 go build ./...` and `GOOS=windows GOARCH=amd64 go build ./...`.
- [ ] Update the spec's Status line: "steps 1 and 2 implemented (step 2 on branch analytics-step2)". Commit.
- [ ] `git push origin analytics-step2`; `amadan branch describe carlos/platform analytics-step2 -body -` with a description that names the follow-up in `carlosframework/website` (the `getting-started` skill question: "Do you want analytics on for this account? It counts HTML page views and daily visitors at the edge, no JavaScript, no cookie, no stored IP; off by default"); `amadan ci status carlos/platform -branch analytics-step2` until passed.
- [ ] Do not merge.

## Self-review against the spec

- §3.3 observing writer: already in place (step 1); the tap reads `ob.status`/`ob.ctype` — Task 5. Pageview rule, bots, `/__carlos/` exclusion — Task 3 `IsPageview`. Client IP — Task 5. Visitor hash + salt — Tasks 2–3. HLL 2¹¹ with version byte, Misra–Gries 500, 256-byte keys, re-fold and approximate label — Tasks 1, 4, 8. Page paths sanitised — Task 3. Storage key, event-time buckets, requeue-once with `dropped.flush`, per-box `seq` idempotency, `restarted` note, shape with `countries` and `dropped` — Tasks 2–3. Named events via `Track` (no caller yet) — Task 3. Accumulation on the usage tick — Task 6. Settings object, resolution, off by default, serial bump, box convergence on the pass — Tasks 6–7.
- §3.6 tab (ranges, headline, chart in Go, three tables, off-state copy, restart note), API path and roles, CLI, settings at both levels — Tasks 7–9.
- §4: no new activator call; Task 5's tap is in the existing defer.
- Type consistency: `analytics.Obs` fields used in Tasks 3, 5, 6, 8 match; `Delta`/`Day` in 2, 3, 4, 6; `Summary` in 4, 8, 9; `telemetry.Resolved.Count/CountFrom/Door` in 6–9.
- Deferred: `door` UI (step 3), `Track` callers (steps 3–4), `dropped.e/a` producers (steps 3–4), GeoIP (`countries` stays empty), the website-repo skill question.