carlos / platform Public

Clone
git clone https://amadan.net/carlos/platform

Plain git — no account needed to clone.

Download

Download this file

Errors tab, step 1: log and edge sources — Implementation Plan

> For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Every app in the fleet gets a populated Errors tab, API and CLI with zero integration: error-level log lines and edge-observed 5xx responses, grouped by fingerprint and coalesced on the box.

Architecture: A new leaf package internal/apperr owns the error record, sanitisation, fingerprinting, the coalescing box sink, grouping, and the bucket read. It stores chunks in a sibling family of logs/ (apps/<sqid>/<app>/errors/<box>/<hour>/…jsonl.gz) through family-parameterised helpers added to internal/applog. The edge's countingWriter learns to observe status and emits an edge error for upstream 5xx; the agent ships the sink like it ships logs. The console reads two families (errors/ plus logs/ filtered to a level from a new internal/telemetry settings object) and serves a tab, a cliRoute API and carlos errors.

Tech Stack: Go 1.22+ (net/http patterns, generics), html/template, sqlite registry, release.Store bucket, no JS framework.

Spec: docs/superpowers/specs/2026-09-07-errors-and-analytics-design.md — §3.1 (shape, sanitisation), §3.2 (log source read-derived, edge errors, fingerprint, coalescing, storage, store door), §3.3 (observing writer, settings object), §3.6 (tab, API, CLI, settings, retention), §4, §6, §7 step 1, §8 rulings.

Global Constraints

  • Wire shapes are additive: logs/ chunks must be byte-identical after the refactor; existing internal/applog tests pass unchanged.
  • Edge wrapper order is load-bearing: recorder above compressor, counter below; nothing moves, one wrapper learns to look (internal/edge/edge.go:1241).
  • Nothing here may enter activator.Service.ServeHTTP or stamp lastActive.
  • Store door write grant becomes a family allow-list: logs, errors, analytics (spec §3.2), each segment-exact.
  • Coalescing cap: 1,000 records per app per UTC hour counting (fp, host, release); overflow folds into one _overflow row.
  • Fingerprint: sha256(len-delimited(source, kind, norm(name), norm(top frame)))[:16] hex, fpv: 1; 1–3 digit tokens survive normalisation, 4+ digits, hex 8+, UUIDs and quoted strings become placeholders.
  • Sanitisation before buffering on every source: query and fragment off paths; emails → <email>; Bearer/Basic credentials and token=/key=/secret=/password= values → <redacted>; hex or base64 runs of 32+ → <hex>.
  • Defaults (ruling): errors.log_level = error. Settings object: accounts/<sqid>/telemetry.json and apps/<sqid>/<app>/telemetry.json, resolved app → account → default.
  • API: GET /api/cli/apps/{account}/{app}/errors with svcrole.Allow(svcrole.Instance, svcrole.Operate); pages go through appPage (session → CSRF → owner → fresh auth).
  • New carlos errors needs a ### carlos errors heading in docs/site/cli.md or internal/docsite fails the gate.
  • Go commands: GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off; sandbox may need disabling on "read-only file system". Before push: GOOS=darwin GOARCH=arm64 go build ./... and GOOS=windows GOARCH=amd64 go build ./....
  • Every commit: go build ./... && go vet ./... && gofmt -l . clean for the packages touched; full go test ./... -count=1 before the final push.
  • Commit trailer on every commit: Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>.
  • Branch: spec-errors-analytics (already exists, carries the spec). Push to origin (the amadan forge). Never gh.

File structure

| File | Responsibility | |---|---| | internal/applog/chunk.go | + Family, FamilyPrefix/BoxPrefix/HourPrefix/ChunkKey, generic EncodeJSONL/DecodeJSONL; existing funcs become thin wrappers | | internal/applog/read.go | + generic ReadChunks[T]; ReadWindow re-based on it | | internal/edge/storedoor.go | aclWriteAppLogs → aclWriteAppFamily allow-list | | internal/apperr/record.go | Record, source/kind constants, FromLog | | internal/apperr/sanitize.go | Sanitize, SanitizePath | | internal/apperr/fingerprint.go | Normalize, TopFrame, Fingerprint, FPVersion | | internal/apperr/sink.go | coalescing Sink | | internal/apperr/read.go | Filter, ReadWindow, chunk key helpers | | internal/apperr/group.go | Group, GroupRecords | | internal/activator/answered.go | WithAnswerFlag, MarkAnswered | | internal/activator/activator.go | mark the three wake-path 503s | | internal/edge/usagemeter.go | countingWriter observes status and content type | | internal/edge/edge.go | ErrorSink, EnableErrors, unconditional writer, edge error emission | | internal/host/logship.go | logshipPrune → chunkPrune(fam, …) | | internal/host/errship.go | error sink ship loop | | internal/host/agent.go | construct apperr.Sink, wire edge + loop | | internal/telemetry/telemetry.go | settings object, Resolve, Update | | internal/console/errorsview.go | window read merging both families; page fold | | internal/console/apierrors.go | GET …/errors, PUT …/telemetry | | internal/console/server.go | routes, page fields, handleAppErrors, settings handlers | | internal/console/templates/app_errors.html, appnav.html, app_settings.html, account_settings.html | tab, nav link, settings forms | | errors_cmd.go, apimode.go, main.go | carlos errors | | docs/site/cli.md | ### carlos errors |

---

Task 1: applog chunk families and a generic codec

Files:

  • Modify: internal/applog/chunk.go
  • Modify: internal/applog/read.go
  • Test: internal/applog/chunk_test.go (append), internal/applog/read_test.go (append)

Interfaces:

  • Produces: type Family string; FamilyLogs, FamilyErrors, FamilyAnalytics; FamilyPrefix(fam Family, acct, app string) string; FamilyBoxPrefix(fam, acct, app, box); FamilyHourPrefix(fam, acct, app, box, hour); FamilyChunkKey(fam, acct, app, box, hour string, seq uint64, size int) string; EncodeJSONL[T any](recs []T) ([]byte, error); DecodeJSONL[T any](b []byte) ([]T, error); ReadChunks[T any](ctx, s release.Store, prefix string, since, until time.Time, keep func(T) bool, collectCap int) (perBox [][]T, trimmed bool, err error); HourNameOfDir(dir string) string (exported rename of hourNameOfDir).
  • Existing Prefix, BoxPrefix, HourPrefix, ChunkKey, EncodeChunk, DecodeChunk, ReadWindow keep their signatures and behaviour.
  • [ ] Step 1: Write the failing tests

Append to internal/applog/chunk_test.go:

func TestFamilyKeysMirrorLogs(t *testing.T) {
	if got, want := FamilyPrefix(FamilyLogs, "bab", "web"), Prefix("bab", "web"); got != want {
		t.Fatalf("FamilyPrefix(logs) = %q, Prefix = %q", got, want)
	}
	if got, want := FamilyChunkKey(FamilyLogs, "bab", "web", "box1", "2026-09-07-10", 42, 7), ChunkKey("bab", "web", "box1", "2026-09-07-10", 42, 7); got != want {
		t.Fatalf("FamilyChunkKey(logs) = %q, ChunkKey = %q", got, want)
	}
	got := FamilyChunkKey(FamilyErrors, "bab", "web", "box1", "2026-09-07-10", 42, 7)
	want := "apps/bab/web/errors/box1/2026-09-07-10/0000000000000042-7.jsonl.gz"
	if got != want {
		t.Fatalf("errors key = %q, want %q", got, want)
	}
	if n, ok := ChunkKeySize(got); !ok || n != 7 {
		t.Fatalf("ChunkKeySize on an errors key = %d,%v", n, ok)
	}
}

func TestEncodeJSONLRoundTripsAnyShape(t *testing.T) {
	type rec struct {
		A string `json:"a"`
		N int    `json:"n,omitempty"`
	}
	b, err := EncodeJSONL([]rec{{A: "x", N: 1}, {A: "y"}})
	if err != nil {
		t.Fatal(err)
	}
	out, err := DecodeJSONL[rec](b)
	if err != nil {
		t.Fatal(err)
	}
	if len(out) != 2 || out[0].A != "x" || out[0].N != 1 || out[1].A != "y" || out[1].N != 0 {
		t.Fatalf("round trip = %+v", out)
	}
	// EncodeChunk must be byte-identical to EncodeJSONL over the same
	// records: the logs family's wire shape does not move.
	recs := []Record{{TS: time.Date(2026, 9, 7, 10, 0, 0, 0, time.UTC), Msg: "hi", Seq: 1}}
	a, _ := EncodeChunk(recs)
	c, _ := EncodeJSONL(recs)
	da, _ := DecodeChunk(a)
	dc, _ := DecodeJSONL[Record](c)
	if len(da) != 1 || len(dc) != 1 || da[0].Msg != dc[0].Msg || !da[0].TS.Equal(dc[0].TS) {
		t.Fatalf("EncodeChunk and EncodeJSONL disagree: %+v vs %+v", da, dc)
	}
}

Append to internal/applog/read_test.go (use the same store construction the file already uses — release.NewDirStore(t.TempDir()) — and its existing helper for writing a chunk if one exists; otherwise s.Put(ctx, ChunkKey(...), body, "")):

func TestReadChunksWalksAnyFamily(t *testing.T) {
	ctx := context.Background()
	s := release.NewDirStore(t.TempDir())
	type ev struct {
		TS   time.Time `json:"ts"`
		Name string    `json:"name"`
	}
	now := time.Date(2026, 9, 7, 10, 30, 0, 0, time.UTC)
	body, _ := EncodeJSONL([]ev{{TS: now, Name: "boom"}, {TS: now.Add(time.Minute), Name: "skip"}})
	key := FamilyChunkKey(FamilyErrors, "bab", "web", "box1", HourOf(now), 1, len(body))
	if _, err := s.Put(ctx, key, body, ""); err != nil {
		t.Fatal(err)
	}
	perBox, trimmed, err := ReadChunks[ev](ctx, s, FamilyPrefix(FamilyErrors, "bab", "web"),
		now.Add(-time.Hour), now.Add(time.Hour), func(e ev) bool { return e.Name != "skip" }, 100)
	if err != nil || trimmed {
		t.Fatalf("ReadChunks: %v trimmed=%v", err, trimmed)
	}
	if len(perBox) != 1 || len(perBox[0]) != 1 || perBox[0][0].Name != "boom" {
		t.Fatalf("perBox = %+v", perBox)
	}
	// An hour outside the window is never listed.
	perBox, _, err = ReadChunks[ev](ctx, s, FamilyPrefix(FamilyErrors, "bab", "web"),
		now.Add(2*time.Hour), time.Time{}, func(ev) bool { return true }, 100)
	if err != nil || len(perBox) != 1 || len(perBox[0]) != 0 {
		t.Fatalf("out-of-window read = %+v, %v", perBox, err)
	}
}
  • [ ] Step 2: Run tests to verify they fail

Run: go test ./internal/applog/ -run 'TestFamilyKeys|TestEncodeJSONL|TestReadChunks' -count=1 Expected: FAIL to compile (undefined: FamilyPrefix etc.).

  • [ ] Step 3: Implement families and the generic codec in chunk.go

Replace the Prefix/BoxPrefix/HourPrefix/ChunkKey block and the codec with:

// Family names one chunk family under an app — the path segment after
// apps/<sqid>/<app>/. logs/ is the original; errors/ and analytics/ are
// the siblings the errors-and-analytics spec adds (§3.2). The key shape
// below the family segment is identical for every family, which is what
// lets one prune loop, one store-door grant and one reader serve all three.
type Family string

const (
	FamilyLogs      Family = "logs"
	FamilyErrors    Family = "errors"
	FamilyAnalytics Family = "analytics"
)

// FamilyPrefix is the app's whole subtree for one family — under
// release.AppPrefix on purpose (spec §3): it is the app's data, so the
// owner gate maps onto the prefix and app trash/purge sweeps it for free.
func FamilyPrefix(fam Family, acct, app string) string {
	return release.AppPrefix(acct, app) + string(fam) + "/"
}

// FamilyBoxPrefix is one box's subtree within a family — the unit of
// write-side ownership: each box prunes only what it wrote.
func FamilyBoxPrefix(fam Family, acct, app, box string) string {
	return FamilyPrefix(fam, acct, app) + box + "/"
}

// FamilyHourPrefix is one UTC hour within a box's subtree.
func FamilyHourPrefix(fam Family, acct, app, box, hour string) string {
	return FamilyBoxPrefix(fam, acct, app, box) + hour + "/"
}

// FamilyChunkKey names one flushed chunk in any family. See ChunkKey for
// why seq and size live in the key.
func FamilyChunkKey(fam Family, acct, app, box, hour string, seq uint64, size int) string {
	return fmt.Sprintf("%s%016d-%d.jsonl.gz", FamilyHourPrefix(fam, acct, app, box, hour), seq, size)
}

// Prefix is the app's whole log subtree: FamilyPrefix for logs.
func Prefix(acct, app string) string { return FamilyPrefix(FamilyLogs, acct, app) }

// BoxPrefix is one box's subtree within an app's logs.
func BoxPrefix(acct, app, box string) string { return FamilyBoxPrefix(FamilyLogs, acct, app, box) }

// HourPrefix is one UTC hour within a box's log subtree.
func HourPrefix(acct, app, box, hour string) string {
	return FamilyHourPrefix(FamilyLogs, acct, app, box, hour)
}

// ChunkKey names one flushed log chunk: seq is the first record's per-box
// sequence number, zero-padded (16 digits — seq is milliseconds-based, see
// NewSink) so lexical listing order is numeric order, and size is the
// object's own byte length. Size lives IN the key on purpose:
// release.Store's List returns keys only, and retention's byte cap needs
// sizes — embedding them costs a suffix, extending the Store interface
// costs every implementation. Chunks are written once and never appended
// or rewritten (spec §3).
func ChunkKey(acct, app, box, hour string, seq uint64, size int) string {
	return FamilyChunkKey(FamilyLogs, acct, app, box, hour, seq, size)
}

Keep ChunkKeySize as is. Replace EncodeChunk/DecodeChunk with:

// EncodeJSONL renders any record shape as gzipped JSONL, one per line —
// the chunk codec every family shares.
func EncodeJSONL[T any](recs []T) ([]byte, error) {
	var buf bytes.Buffer
	zw := gzip.NewWriter(&buf)
	enc := json.NewEncoder(zw)
	for _, r := range recs {
		if err := enc.Encode(r); err != nil {
			return nil, err
		}
	}
	if err := zw.Close(); err != nil {
		return nil, err
	}
	return buf.Bytes(), nil
}

// DecodeJSONL parses a chunk, skipping malformed lines rather than failing
// the whole object — one corrupt line must not cost the member the rest of
// the hour. Corrupt gzip is an error: that is a broken object, not a line.
func DecodeJSONL[T any](b []byte) ([]T, error) {
	zr, err := gzip.NewReader(bytes.NewReader(b))
	if err != nil {
		return nil, err
	}
	defer zr.Close()
	var out []T
	sc := bufio.NewScanner(zr)
	sc.Buffer(nil, 1<<20) // lines are capped well below this at the tee
	for sc.Scan() {
		var r T
		if json.Unmarshal(sc.Bytes(), &r) != nil {
			continue
		}
		out = append(out, r)
	}
	if err := sc.Err(); err != nil && err != io.EOF {
		return out, err
	}
	return out, nil
}

// EncodeChunk renders log records: EncodeJSONL over Record.
func EncodeChunk(recs []Record) ([]byte, error) { return EncodeJSONL(recs) }

// DecodeChunk parses a log chunk: DecodeJSONL over Record.
func DecodeChunk(b []byte) ([]Record, error) { return DecodeJSONL[Record](b) }
  • [ ] Step 4: Implement ReadChunks in read.go and re-base ReadWindow on it
// ReadChunks walks every box's hour directories under prefix (a
// FamilyPrefix) whose hours intersect [since, until], decodes each chunk as
// T, keeps what passes keep, and returns one slice per box in listing
// order. Hour pruning happens at the PREFIX level — an hour outside the
// window is never listed, which is what makes a 1h read over 7d of history
// cheap (spec §3). collectCap bounds memory per box: past it the OLDEST
// records go and trimmed says so. Store errors fail the read — a partial
// answer that looks whole is the one thing this must never produce.
func ReadChunks[T any](ctx context.Context, s release.Store, prefix string, since, until time.Time, keep func(T) bool, collectCap int) ([][]T, bool, error) {
	boxDirs, err := s.ListDirs(ctx, prefix)
	if err != nil {
		return nil, false, err
	}
	trimmed := false
	perBox := make([][]T, 0, len(boxDirs))
	for _, boxDir := range boxDirs {
		hourDirs, err := s.ListDirs(ctx, boxDir)
		if err != nil {
			return nil, false, err
		}
		var recs []T
		for _, hourDir := range hourDirs { // lexically sorted = chronological
			if !hourIntersects(HourNameOfDir(hourDir), since, until) {
				continue
			}
			keys, err := s.List(ctx, hourDir)
			if err != nil {
				return nil, false, err
			}
			for _, key := range keys { // lexically sorted = seq order
				body, _, err := s.Get(ctx, key)
				if err != nil {
					return nil, false, err
				}
				chunk, err := DecodeJSONL[T](body)
				if err != nil {
					return nil, false, err
				}
				for _, r := range chunk {
					if keep(r) {
						recs = append(recs, r)
					}
				}
				if over := len(recs) - collectCap; over > 0 {
					recs = recs[over:]
					trimmed = true
				}
			}
		}
		perBox = append(perBox, recs)
	}
	return perBox, trimmed, nil
}

Then ReadWindow becomes:

func ReadWindow(ctx context.Context, s release.Store, acct, app string, f Filter, collectCap int) ([]Record, bool, error) {
	perBox, trimmed, err := ReadChunks[Record](ctx, s, Prefix(acct, app), f.Since, f.Until, f.Match, collectCap)
	if err != nil {
		return nil, false, err
	}
	merged := MergeRecords(perBox...)
	if over := len(merged) - collectCap; over > 0 {
		merged = merged[over:]
		trimmed = true
	}
	return merged, trimmed, nil
}

Rename hourNameOfDir to HourNameOfDir (exported; the host package's hourNameOf duplicate goes away in Task 6) and update its one caller.

  • [ ] Step 5: Run the whole package

Run: go test ./internal/applog/ -count=1 Expected: PASS, every pre-existing test included.

  • [ ] Step 6: Commit
git add internal/applog
git commit -m "applog: chunk families and a generic JSONL codec, logs/ unchanged

Prefix/ChunkKey/EncodeChunk/ReadWindow keep their signatures and bytes;
FamilyPrefix, FamilyChunkKey, EncodeJSONL/DecodeJSONL and ReadChunks are
the seams the errors/ family (spec 2026-09-07 §3.2) builds on.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>"

---

Task 2: Store door family allow-list

Files:

  • Modify: internal/edge/storedoor.go:503-518 (aclWriteAppLogs)
  • Test: internal/edge/storedoor_test.go (rows in TestStoreDoorACLMatrix)

Interfaces:

  • Produces: aclWriteAppFamily(sqid, key string) bool replacing aclWriteAppLogs; aclWrite calls it.
  • [ ] Step 1: Add matrix rows

In TestStoreDoorACLMatrix, after the existing logs rows (around line 138), add:

		// --- sibling chunk families (errors-and-analytics spec §3.2) ---
		{"apps/ab12/web/errors/box-a/2026090710/0000000000000001-42.jsonl.gz", opWrite, true, "own app's error chunk"},
		{"apps/ab12/web/errors/box-a/2026090710/0000000000000001-42.jsonl.gz", opDelete, true, "errship prunes its own subtree hourly"},
		{"apps/ab12/web/analytics/box-a/days/2026-09-07.json", opWrite, true, "own app's analytics day file"},
		{"apps/zz99/web/errors/box-a/x.jsonl.gz", opWrite, false, "another account's error chunks"},
		{"apps/ab12/errors/box-a/x.jsonl.gz", opWrite, false, "errors must sit under an app segment"},
		{"apps/ab12/web/errors", opWrite, false, "the bare family segment is not a chunk"},
		{"apps/ab12/web/errors/", opWrite, false, "nor with a slash"},
		{"apps/ab12/web/other/box-a/x.jsonl.gz", opWrite, false, "the family list is exact: logs, errors, analytics"},
		{"apps/ab12/web/channels/errors/stable.json", opWrite, false, "\"errors\" deeper in the key grants nothing"},
		{"apps/ab12/web/errorsx/box-a/x.jsonl.gz", opWrite, false, "the segment is errors/, not a prefix of a longer name"},
  • [ ] Step 2: Run to verify the new rows fail

Run: go test ./internal/edge/ -run TestStoreDoorACLMatrix -count=1 Expected: FAIL on the errors and analytics write rows (refused today).

  • [ ] Step 3: Implement

Replace aclWriteAppLogs with:

// chunkFamilies is the allow-list of app subtrees a box may write: the
// chunk and day-file families it produces about its own tenant's apps
// (applog.Family). Exact segment names — "logs" is not "logsx".
var chunkFamilies = map[string]bool{"logs": true, "errors": true, "analytics": true}

// aclWriteAppFamily implements aclWrite's second scope: exactly
// apps/<sqid>/<app>/<family>/<something>, with <app> a single non-empty
// segment, <family> one of chunkFamilies, and at least one key component
// below the family. Anything else — a shorter key, a second path shape,
// a family name as the app segment — is refused, so the grant can never be
// widened by a key that merely contains the word.
func aclWriteAppFamily(sqid, key string) bool {
	if sqid == "" {
		return false
	}
	rest, ok := strings.CutPrefix(key, "apps/"+sqid+"/")
	if !ok {
		return false
	}
	app, tail, ok := strings.Cut(rest, "/")
	if !ok || app == "" {
		return false
	}
	fam, below, ok := strings.Cut(tail, "/")
	return ok && chunkFamilies[fam] && below != ""
}

Update aclWrite to call aclWriteAppFamily, and update the doc comment above aclWrite (the "logs scope is SEGMENT-EXACT" paragraph) to say "family scope" and name the three families.

  • [ ] Step 4: Run the matrix and the package

Run: go test ./internal/edge/ -run TestStoreDoor -count=1 Expected: PASS.

  • [ ] Step 5: Commit
git add internal/edge/storedoor.go internal/edge/storedoor_test.go
git commit -m "store door: the app write grant is a family allow-list (logs, errors, analytics)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>"

---

Task 3: internal/apperr record, sanitisation, fingerprint

Files:

  • Create: internal/apperr/record.go, internal/apperr/sanitize.go, internal/apperr/fingerprint.go
  • Test: internal/apperr/sanitize_test.go, internal/apperr/fingerprint_test.go

Interfaces:

  • Produces: ``go const (SourceLog = "log"; SourceEdge = "edge"; SourceServer = "server"; SourceBrowser = "browser") const (KindHTTP = "http"; KindLog = "log") const FPVersion = 1 type Record struct { TS time.Time; Box, Host, Source, Kind, Name, Level, Path string; Status int; Release, Stack string; Fields map[string]string; FP string; FPV int; Count int; First, Last time.Time; Seq uint64 } func FromLog(r applog.Record) Record func Sanitize(s string) string func SanitizePath(p string) string func Normalize(s string) string func TopFrame(stack string) string func Fingerprint(source, kind, name, stack string) string ``
  • [ ] Step 1: Write failing tests

internal/apperr/sanitize_test.go:

package apperr

import "testing"

func TestSanitize(t *testing.T) {
	cases := map[string]string{
		"user paul@tito.io not found":                        "user <email> not found",
		"Authorization: Bearer abc.def-ghi":                  "Authorization: Bearer <redacted>",
		"basic dXNlcjpwYXNz failed":                          "basic <redacted> failed",
		"GET /x?token=abc123&y=1":                            "GET /x?token=<redacted>&y=1",
		`{"password": "hunter2", "n": 3}`:                    `{"password": "<redacted>", "n": 3}`,
		"api_key=sk-live-1234 ok":                            "api_key=<redacted> ok",
		"digest 0123456789abcdef0123456789abcdef done":       "digest <hex> done",
		"b64 QUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVo0MTIzNA== ok": "b64 <hex> ok",
		"short 4123 stays":                                   "short 4123 stays",
		"nothing here":                                       "nothing here",
	}
	for in, want := range cases {
		if got := Sanitize(in); got != want {
			t.Errorf("Sanitize(%q) = %q, want %q", in, got, want)
		}
	}
}

func TestSanitizePath(t *testing.T) {
	cases := map[string]string{
		"/reset/8f3a9c1e2b4d6f708a9b0c1d2e3f4a5b?x=1#frag":           "/reset/:token",
		"/users/paul@tito.io/edit":                                    "/users/:token/edit",
		"/items/550e8400-e29b-41d4-a716-446655440000":                 "/items/:token",
		"/s/QUJDREVGR0hJSktMTU5PUFFSU1RV":                             "/s/:token",
		"/blog/2024-review":                                           "/blog/2024-review",
		"/api/v2/orders/42":                                           "/api/v2/orders/42",
		"/":                                                           "/",
		"":                                                            "/",
	}
	for in, want := range cases {
		if got := SanitizePath(in); got != want {
			t.Errorf("SanitizePath(%q) = %q, want %q", in, got, want)
		}
	}
}

internal/apperr/fingerprint_test.go:

package apperr

import "testing"

func TestNormalize(t *testing.T) {
	cases := map[string]string{
		"user 4123 not found":                    "user <n> not found",
		"user 9911 not found":                    "user <n> not found",
		"HTTP 503 from upstream":                 "HTTP 503 from upstream",
		"retry 3 failed":                         "retry 3 failed",
		"v2 handler panicked":                    "v2 handler panicked",
		"E404 for id 550e8400-e29b-41d4-a716-446655440000": "E404 for id <uuid>",
		`missing key "stripe_secret"`:            `missing key "?"`,
		"trace 0123456789abcdef bad":             "trace <hex> bad",
		"deadbeef cafe":                          "<hex> cafe",
	}
	for in, want := range cases {
		if got := Normalize(in); got != want {
			t.Errorf("Normalize(%q) = %q, want %q", in, got, want)
		}
	}
}

func TestTopFrame(t *testing.T) {
	cases := map[string]string{
		"TypeError: x\n    at render (https://app.example/assets/main.3f2a9c1b.js:12:34)\n    at y": "at render (https://app.example/assets/main.js)",
		"\n  app/models/user.rb:42:in `find'\n  app/x.rb:1":                                    "app/models/user.rb:in `find'",
		"":                                                                                      "",
		"single line only":                                                                      "single line only",
	}
	for in, want := range cases {
		if got := TopFrame(in); got != want {
			t.Errorf("TopFrame(%q) = %q, want %q", in, got, want)
		}
	}
}

func TestFingerprint(t *testing.T) {
	a := Fingerprint(SourceLog, KindLog, "user 4123 not found", "")
	b := Fingerprint(SourceLog, KindLog, "user 9911 not found", "")
	c := Fingerprint(SourceLog, KindLog, "user gone", "")
	d := Fingerprint(SourceEdge, KindLog, "user 4123 not found", "")
	if a != b {
		t.Fatalf("digit runs must group: %s != %s", a, b)
	}
	if a == c || a == d {
		t.Fatalf("different names or sources must not group: %s %s %s", a, c, d)
	}
	if len(a) != 16 {
		t.Fatalf("fingerprint length %d, want 16 hex", len(a))
	}
	// Length-delimited: ("ab","c") and ("a","bc") differ.
	if Fingerprint("ab", "c", "n", "") == Fingerprint("a", "bc", "n", "") {
		t.Fatal("components must be length-delimited")
	}
	// A redeploy's bundle hash and line numbers do not split a browser group.
	s1 := "TypeError: x\n    at render (https://a/main.3f2a9c1b.js:12:34)"
	s2 := "TypeError: x\n    at render (https://a/main.9b8c7d6e.js:99:1)"
	if Fingerprint(SourceBrowser, "TypeError", "x", s1) != Fingerprint(SourceBrowser, "TypeError", "x", s2) {
		t.Fatal("frame normalisation must drop bundle hash and line:col")
	}
}
  • [ ] Step 2: Run to verify failure

Run: go test ./internal/apperr/ -count=1 Expected: FAIL to compile (package does not exist).

  • [ ] Step 3: Write record.go
// Package apperr is the pure half of the Errors tab (spec
// 2026-09-07-errors-and-analytics-design.md §3.1–3.2): the wire record,
// sanitisation, fingerprinting, the box-side coalescing sink, the chunk
// read and the grouping fold. No I/O and no clock of its own — the same
// split internal/applog keeps: internal/host ships, internal/edge and the
// activator produce, internal/console reads.
package apperr

import (
	"time"

	"github.com/carlosframework/platform/internal/applog"
)

// Sources an error can come from (spec §3.1). log is never stored in the
// errors/ family — it is derived on read from logs/ (§3.2).
const (
	SourceLog     = "log"
	SourceEdge    = "edge"
	SourceServer  = "server"
	SourceBrowser = "browser"
)

// Kinds the platform itself produces. A server or browser caller sets its
// own kind (an exception class, typically).
const (
	KindHTTP = "http" // an edge-observed 5xx
	KindLog  = "log"  // a log line at or above the configured level
)

// FPVersion stamps the fingerprint algorithm so it can change without
// merging old groups into new ones.
const FPVersion = 1

// Record is one coalesced error on the wire — one JSONL line inside an
// errors/ chunk. Additive shape (AGENTS.md rule): every field but TS,
// Source, Name and FP is omitempty, unknown keys are ignored on read.
// Count, First and Last are the coalescing state: one record stands for
// Count occurrences of FP on Host under Release between First and Last;
// the other fields are the first occurrence's sample.
type Record struct {
	TS      time.Time         `json:"ts"`
	Box     string            `json:"box,omitempty"`
	Host    string            `json:"host,omitempty"`
	Source  string            `json:"source"`
	Kind    string            `json:"kind,omitempty"`
	Name    string            `json:"name"`
	Level   string            `json:"level,omitempty"` // log source: warn | error
	Path    string            `json:"path,omitempty"`
	Status  int               `json:"status,omitempty"`
	Release string            `json:"release,omitempty"`
	Stack   string            `json:"stack,omitempty"`
	Fields  map[string]string `json:"fields,omitempty"`
	FP      string            `json:"fp"`
	FPV     int               `json:"fpv,omitempty"`
	Count   int               `json:"count,omitempty"`
	First   time.Time         `json:"first,omitempty"`
	Last    time.Time         `json:"last,omitempty"`
	Seq     uint64            `json:"seq,omitempty"`
}

// MaxStack bounds a stored stack (spec §3.1).
const MaxStack = 8 << 10

// FromLog derives the log-source record for one applog line (spec §3.2:
// read-derived, never stored twice). Fingerprinted on the normalised
// message alone — a log line has no frame.
func FromLog(r applog.Record) Record {
	name := Sanitize(r.Msg)
	return Record{
		TS: r.TS, Box: r.Box, Host: r.Host,
		Source: SourceLog, Kind: KindLog, Name: name, Level: r.Level,
		Fields: r.Fields,
		FP:     Fingerprint(SourceLog, KindLog, name, ""), FPV: FPVersion,
		Count:  1, First: r.TS, Last: r.TS, Seq: r.Seq,
	}
}
  • [ ] Step 4: Write sanitize.go
package apperr

import (
	"regexp"
	"strings"
)

// Sanitisation is a floor, not a guarantee (spec §3.1): the patterns below
// catch what secrets look like in practice — addresses, bearer tokens,
// key=value credentials, long opaque runs — before anything is buffered.
var (
	reEmail = regexp.MustCompile(`[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}`)
	reAuth  = regexp.MustCompile(`(?i)\b(bearer|basic)\s+[A-Za-z0-9._~+/=-]+`)
	// key=value and "key": "value" shapes, value up to the next delimiter.
	reKV = regexp.MustCompile(`(?i)\b(token|key|secret|password|passwd|pwd|api_key|apikey|access_token|auth)("?\s*[=:]\s*"?)[^\s&"',;]+`)
	// A hex or base64 run of 32+ characters: a digest, a session id, a key.
	reLong = regexp.MustCompile(`\b[A-Za-z0-9+/_-]{32,}={0,2}`)
)

// Sanitize redacts secrets in free text: messages, stacks, field values.
func Sanitize(s string) string {
	s = reEmail.ReplaceAllString(s, "<email>")
	s = reAuth.ReplaceAllStringFunc(s, func(m string) string {
		word := m[:strings.IndexAny(m, " \t")]
		return word + " <redacted>"
	})
	s = reKV.ReplaceAllString(s, "${1}${2}<redacted>")
	s = reLong.ReplaceAllString(s, "<hex>")
	return s
}

var (
	reUUID    = regexp.MustCompile(`^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$`)
	reHex16   = regexp.MustCompile(`^[0-9a-fA-F]{16,}$`)
	reB64ish  = regexp.MustCompile(`^[A-Za-z0-9+/_-]{20,}={0,2}$`)
	reEmailNW = regexp.MustCompile(`^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}$`)
)

// SanitizePath strips query and fragment and collapses token-shaped
// segments to ":token" (spec §3.3): a UUID, a hex run of 16+, a base64-
// looking run of 20+, or an email. "/blog/2024-review" and "/orders/42"
// survive; "/reset/8f3a…" does not. Empty is "/".
func SanitizePath(p string) string {
	if i := strings.IndexAny(p, "?#"); i >= 0 {
		p = p[:i]
	}
	if p == "" {
		return "/"
	}
	segs := strings.Split(p, "/")
	for i, s := range segs {
		if s == "" {
			continue
		}
		if reUUID.MatchString(s) || reHex16.MatchString(s) || reEmailNW.MatchString(s) ||
			(reB64ish.MatchString(s) && strings.ContainsAny(s, "0123456789") && strings.ContainsAny(s, "ABCDEFGHIJKLMNOPQRSTUVWXYZ")) {
			segs[i] = ":token"
		}
	}
	return strings.Join(segs, "/")
}

The base64 segment rule requires both a digit and an upper-case letter so that ordinary long slugs (some-long-page-name-here) survive; the tests pin /s/QUJD… as a token and /blog/2024-review as not.

  • [ ] Step 5: Write fingerprint.go
package apperr

import (
	"crypto/sha256"
	"encoding/hex"
	"regexp"
	"strconv"
	"strings"
)

var (
	reQuoted   = regexp.MustCompile(`"[^"]*"|'[^']*'`)
	reUUIDAny  = regexp.MustCompile(`\b[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}\b`)
	reHexRun   = regexp.MustCompile(`\b[0-9a-fA-F]{8,}\b`)
	reDigits4  = regexp.MustCompile(`\b\d{4,}\b`)
	reLineCol  = regexp.MustCompile(`:\d+(:\d+)?`)
	reBundleJS = regexp.MustCompile(`\.[0-9a-fA-F]{8,}\.js`)
)

// Normalize folds the parts of a message that vary per occurrence into
// placeholders on token boundaries (spec §3.2): quoted strings, UUIDs, hex
// runs of 8+, digit runs of 4+. Tokens of 1–3 digits and mixed tokens
// (v2, E404) survive, so "HTTP 503" and "retry 3" still separate groups
// while "user 4123" and "user 9911" do not. Small ids under 1,000 will
// split groups; that is the stated trade-off, and a caller-supplied FP
// overrides it.
func Normalize(s string) string {
	s = reQuoted.ReplaceAllString(s, `"?"`)
	s = reUUIDAny.ReplaceAllString(s, "<uuid>")
	s = reHexRun.ReplaceAllStringFunc(s, func(m string) string {
		if _, err := strconv.ParseUint(m, 10, 64); err == nil {
			return "<n>" // all digits: a digit run, not a hex run
		}
		return "<hex>"
	})
	s = reDigits4.ReplaceAllString(s, "<n>")
	return s
}

// TopFrame is the first non-empty line of a stack with its line, column
// and bundle hash removed, so a redeploy does not split every group.
func TopFrame(stack string) string {
	for _, line := range strings.Split(stack, "\n") {
		line = strings.TrimSpace(line)
		if line == "" {
			continue
		}
		line = reBundleJS.ReplaceAllString(line, ".js")
		line = reLineCol.ReplaceAllString(line, "")
		return line
	}
	return ""
}

// Fingerprint groups occurrences: sha256 over length-delimited (source,
// kind, Normalize(name), Normalize(TopFrame(stack)))[:16] as hex. For a
// stack with a message line first (JS "TypeError: x\n at …"), the top
// frame is the second line; callers pass the message as name and the
// whole stack here.
func Fingerprint(source, kind, name, stack string) string {
	h := sha256.New()
	frame := TopFrame(stack)
	if strings.HasPrefix(frame, name) || strings.Contains(frame, ": "+name) {
		// The stack's first line restates the message; take the next.
		rest := strings.TrimSpace(stack)
		if i := strings.IndexByte(rest, '\n'); i >= 0 {
			frame = TopFrame(rest[i+1:])
		}
	}
	for _, part := range []string{source, kind, Normalize(name), Normalize(frame)} {
		h.Write([]byte(strconv.Itoa(len(part))))
		h.Write([]byte{':'})
		h.Write([]byte(part))
	}
	return hex.EncodeToString(h.Sum(nil))[:16]
}

Note for TestTopFrame's first case: the expected value is the message line (TypeError: x) is NOT the frame — adjust the test's first case to expect TypeError: x from TopFrame (it is the first non-empty line), and rely on TestFingerprint's last assertion for the message-skipping behaviour. Concretely, change that case's expected string to "TypeError: x".

  • [ ] Step 6: Run tests, iterate until green

Run: go test ./internal/apperr/ -count=1 -v Expected: PASS. If a regex case disagrees with a test, fix the regex, not the test, unless the test contradicts the spec's stated rule.

  • [ ] Step 7: Commit
git add internal/apperr
git commit -m "apperr: error record, sanitisation and fingerprinting (spec §3.1–3.2)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>"

---

Task 4: internal/apperr coalescing sink and bucket read

Files:

  • Create: internal/apperr/sink.go, internal/apperr/read.go
  • Test: internal/apperr/sink_test.go, internal/apperr/read_test.go

Interfaces:

  • Produces: ``go const MaxGroupsPerAppHour = 1000 const OverflowFP = "_overflow" func NewSink(box string, now func() time.Time) *Sink func (s *Sink) Add(acct, app string, r Record) // sanitises, fingerprints if FP == "", coalesces func (s *Sink) Drain() map[applog.AppKey][]Record func (s *Sink) Known() []applog.AppKey func Prefix(acct, app string) string // applog.FamilyPrefix(FamilyErrors, …) func ChunkKey(acct, app, box, hour string, seq uint64, size int) string func EncodeChunk(recs []Record) ([]byte, error) func DecodeChunk(b []byte) ([]Record, error) type Filter struct { Since, Until time.Time; Host, Source, Release, FP string } func (f Filter) Match(r Record) bool // [First,Last] overlaps [Since,Until] func ReadWindow(ctx, s release.Store, acct, app string, f Filter, collectCap int) ([]Record, bool, error) func MergeRecords(slices ...[]Record) []Record // by (Last, Box, Seq) ``
  • [ ] Step 1: Write failing tests

internal/apperr/sink_test.go:

package apperr

import (
	"fmt"
	"testing"
	"time"

	"github.com/carlosframework/platform/internal/applog"
)

func TestSinkCoalescesByFingerprintHostRelease(t *testing.T) {
	now := time.Date(2026, 9, 7, 10, 0, 0, 0, time.UTC)
	clock := now
	s := NewSink("box1", func() time.Time { return clock })
	for i := 0; i < 10000; i++ {
		clock = now.Add(time.Duration(i) * time.Millisecond)
		s.Add("bab", "web", Record{Host: "h1", Source: SourceEdge, Kind: KindHTTP, Name: "500 GET /x", Status: 500, Release: "v1"})
	}
	s.Add("bab", "web", Record{Host: "h2", Source: SourceEdge, Kind: KindHTTP, Name: "500 GET /x", Status: 500, Release: "v1"})
	out := s.Drain()
	recs := out[applog.AppKey{Acct: "bab", App: "web"}]
	if len(recs) != 2 {
		t.Fatalf("records = %d, want 2 (one per host)", len(recs))
	}
	var h1 Record
	for _, r := range recs {
		if r.Host == "h1" {
			h1 = r
		}
	}
	if h1.Count != 10000 || !h1.First.Equal(now) || !h1.Last.Equal(now.Add(9999*time.Millisecond)) {
		t.Fatalf("h1 = count %d first %v last %v", h1.Count, h1.First, h1.Last)
	}
	if h1.FP == "" || h1.FPV != FPVersion || h1.Box != "box1" || h1.Seq == 0 {
		t.Fatalf("stamping: %+v", h1)
	}
	if len(s.Drain()) != 0 {
		t.Fatal("second drain must be empty")
	}
}

func TestSinkCapsGroupsPerAppPerHour(t *testing.T) {
	now := time.Date(2026, 9, 7, 10, 0, 0, 0, time.UTC)
	s := NewSink("box1", func() time.Time { return now })
	for i := 0; i < MaxGroupsPerAppHour+5; i++ {
		s.Add("bab", "web", Record{Source: SourceEdge, Kind: KindHTTP, Name: fmt.Sprintf("500 GET /p%d-x", i), FP: fmt.Sprintf("fp%d", i)})
	}
	s.Add("bab", "other", Record{Source: SourceEdge, Kind: KindHTTP, Name: "unaffected", FP: "z"})
	out := s.Drain()
	web := out[applog.AppKey{Acct: "bab", App: "web"}]
	if len(web) != MaxGroupsPerAppHour+1 {
		t.Fatalf("web records = %d, want cap+1 (the overflow row)", len(web))
	}
	var over *Record
	for i := range web {
		if web[i].FP == OverflowFP {
			over = &web[i]
		}
	}
	if over == nil || over.Count != 5 || over.Name != "500 GET /p1000-x" {
		t.Fatalf("overflow = %+v", over)
	}
	if len(out[applog.AppKey{Acct: "bab", App: "other"}]) != 1 {
		t.Fatal("a neighbour app must not be suppressed")
	}
	// The cap counts per hour: the next hour admits new groups again.
	now = now.Add(time.Hour)
	s.Add("bab", "web", Record{Source: SourceEdge, Kind: KindHTTP, Name: "fresh", FP: "fresh"})
	if recs := s.Drain()[applog.AppKey{Acct: "bab", App: "web"}]; len(recs) != 1 || recs[0].FP != "fresh" {
		t.Fatalf("next hour = %+v", recs)
	}
}

func TestSinkSanitisesAndFingerprintsOnAdd(t *testing.T) {
	now := time.Date(2026, 9, 7, 10, 0, 0, 0, time.UTC)
	s := NewSink("box1", func() time.Time { return now })
	long := make([]byte, MaxStack+100)
	for i := range long {
		long[i] = 'x'
	}
	s.Add("bab", "web", Record{Source: SourceServer, Kind: "RuntimeError", Name: "mail to paul@tito.io bounced",
		Path: "/reset/8f3a9c1e2b4d6f708a9b0c1d2e3f4a5b?t=1", Stack: string(long),
		Fields: map[string]string{"user": "paul@tito.io", "k": "v"}})
	r := s.Drain()[applog.AppKey{Acct: "bab", App: "web"}][0]
	if r.Name != "mail to <email> bounced" || r.Path != "/reset/:token" || r.Fields["user"] != "<email>" || r.Fields["k"] != "v" {
		t.Fatalf("sanitised = %+v", r)
	}
	if len(r.Stack) != MaxStack {
		t.Fatalf("stack len %d, want %d", len(r.Stack), MaxStack)
	}
	if r.FP != Fingerprint(SourceServer, "RuntimeError", "mail to <email> bounced", r.Stack) || r.FPV != FPVersion {
		t.Fatalf("fp = %q", r.FP)
	}
	if r.Count != 1 || !r.First.Equal(now) || !r.Last.Equal(now) || !r.TS.Equal(now) {
		t.Fatalf("stamps = %+v", r)
	}
}

internal/apperr/read_test.go:

package apperr

import (
	"context"
	"testing"
	"time"

	"github.com/carlosframework/platform/internal/release"
)

func TestReadWindowMergesAndFilters(t *testing.T) {
	ctx := context.Background()
	s := release.NewDirStore(t.TempDir())
	now := time.Date(2026, 9, 7, 10, 30, 0, 0, time.UTC)
	put := func(box string, recs []Record) {
		body, err := EncodeChunk(recs)
		if err != nil {
			t.Fatal(err)
		}
		if _, err := s.Put(ctx, ChunkKey("bab", "web", box, HourOf(recs[0].TS), recs[0].Seq, len(body)), body, ""); err != nil {
			t.Fatal(err)
		}
	}
	put("box1", []Record{
		{TS: now, Box: "box1", Host: "h1", Source: SourceEdge, Name: "a", FP: "fa", Count: 3, First: now.Add(-4 * time.Minute), Last: now, Seq: 1},
		{TS: now, Box: "box1", Host: "h1", Source: SourceServer, Name: "b", FP: "fb", Count: 1, First: now, Last: now, Seq: 2},
	})
	put("box2", []Record{
		{TS: now.Add(-3 * time.Hour), Box: "box2", Host: "h2", Source: SourceEdge, Name: "old", FP: "fo", Count: 1, First: now.Add(-3 * time.Hour), Last: now.Add(-3 * time.Hour), Seq: 1},
	})
	recs, trimmed, err := ReadWindow(ctx, s, "bab", "web", Filter{Since: now.Add(-time.Hour)}, 100)
	if err != nil || trimmed {
		t.Fatalf("read: %v %v", err, trimmed)
	}
	if len(recs) != 2 || recs[0].FP != "fa" || recs[1].FP != "fb" {
		t.Fatalf("window = %+v", recs)
	}
	// A record whose [First, Last] overlaps the window is in, whole.
	recs, _, _ = ReadWindow(ctx, s, "bab", "web", Filter{Since: now.Add(-2 * time.Minute), Until: now.Add(-time.Minute)}, 100)
	if len(recs) != 1 || recs[0].FP != "fa" || recs[0].Count != 3 {
		t.Fatalf("overlap = %+v", recs)
	}
	recs, _, _ = ReadWindow(ctx, s, "bab", "web", Filter{Since: now.Add(-time.Hour), Source: SourceServer}, 100)
	if len(recs) != 1 || recs[0].FP != "fb" {
		t.Fatalf("source filter = %+v", recs)
	}
	recs, _, _ = ReadWindow(ctx, s, "bab", "web", Filter{Since: now.Add(-4 * time.Hour), Host: "h2"}, 100)
	if len(recs) != 1 || recs[0].FP != "fo" {
		t.Fatalf("host filter = %+v", recs)
	}
}
  • [ ] Step 2: Run to verify failure

Run: go test ./internal/apperr/ -count=1 Expected: FAIL to compile.

  • [ ] Step 3: Write sink.go
package apperr

import (
	"sync"
	"time"

	"github.com/carlosframework/platform/internal/applog"
)

// MaxGroupsPerAppHour caps coalesced records per app per UTC hour, counting
// (fp, host, release) combinations (spec §3.2). Per app, not per box: one
// tenant's storm must not suppress a neighbour's.
const MaxGroupsPerAppHour = 1000

// OverflowFP names the one row the excess beyond the cap folds into. Its
// Count is occurrences, and the tab labels it as such.
const OverflowFP = "_overflow"

// Sink coalesces errors between ship-loop flushes: per app, one record per
// (fp, host, release) carrying count/first/last and the first sample. An
// error storm is the moment this must work — ten thousand identical panics
// in a minute become one record. Safe for concurrent use; the edge, the
// activator and the ship loop share one Sink per agent process.
type Sink struct {
	box string
	now func() time.Time

	mu   sync.Mutex
	apps map[applog.AppKey]*appBuf
	seq  uint64
}

type groupKey struct{ fp, host, release string }

type appBuf struct {
	groups   map[groupKey]*Record
	order    []groupKey // insertion order, for a deterministic drain
	overflow *Record
	hour     string // the UTC hour `admitted` counts for
	admitted int    // distinct groups admitted this hour, cap counter
}

// NewSink returns a Sink stamping records with box and timestamps from now.
// seq starts at UnixMilli for the reason applog.NewSink gives: chunk keys
// embed the first record's seq, and a restart must sort after last boot.
func NewSink(box string, now func() time.Time) *Sink {
	return &Sink{box: box, now: now, apps: make(map[applog.AppKey]*appBuf), seq: uint64(now().UnixMilli())}
}

// Add buffers one occurrence for (acct, app). r is sanitised here (§3.1),
// fingerprinted when it carries no FP, and folded into its group.
func (s *Sink) Add(acct, app string, r Record) {
	now := s.now().UTC()
	r.Name = Sanitize(r.Name)
	r.Path = SanitizePath(r.Path)
	if r.Path == "/" && r.Status == 0 && r.Source != SourceEdge {
		r.Path = ""
	}
	if len(r.Stack) > MaxStack {
		r.Stack = r.Stack[:MaxStack]
	}
	r.Stack = Sanitize(r.Stack)
	if len(r.Fields) > 0 {
		f := make(map[string]string, len(r.Fields))
		for k, v := range r.Fields {
			f[k] = Sanitize(v)
		}
		r.Fields = f
	}
	if r.FP == "" {
		r.FP = Fingerprint(r.Source, r.Kind, r.Name, r.Stack)
		r.FPV = FPVersion
	}
	if len(r.FP) > 64 {
		r.FP = r.FP[:64]
	}

	s.mu.Lock()
	defer s.mu.Unlock()
	b := s.apps[applog.AppKey{Acct: acct, App: app}]
	if b == nil {
		b = &appBuf{groups: make(map[groupKey]*Record)}
		s.apps[applog.AppKey{Acct: acct, App: app}] = b
	}
	if h := applog.HourOf(now); h != b.hour {
		b.hour, b.admitted = h, 0
	}
	k := groupKey{r.FP, r.Host, r.Release}
	if g := b.groups[k]; g != nil {
		g.Count++
		g.Last = now
		return
	}
	if b.admitted >= MaxGroupsPerAppHour {
		if b.overflow == nil {
			o := r
			o.FP = OverflowFP
			o.TS, o.First, o.Last, o.Count = now, now, now, 0
			o.Box = s.box
			s.seq++
			o.Seq = s.seq
			b.overflow = &o
		}
		b.overflow.Count++
		b.overflow.Last = now
		return
	}
	b.admitted++
	s.seq++
	r.TS, r.First, r.Last, r.Count = now, now, now, 1
	r.Box = s.box
	r.Seq = s.seq
	b.groups[k] = &r
	b.order = append(b.order, k)
}

// Drain returns everything buffered, keyed by app, and resets the groups
// (the per-hour cap counter and seq persist — they are the hour's and the
// process's, not the batch's). Empty apps are omitted.
func (s *Sink) Drain() map[applog.AppKey][]Record {
	s.mu.Lock()
	defer s.mu.Unlock()
	out := make(map[applog.AppKey][]Record)
	for k, b := range s.apps {
		if len(b.order) == 0 && b.overflow == nil {
			continue
		}
		recs := make([]Record, 0, len(b.order)+1)
		for _, gk := range b.order {
			recs = append(recs, *b.groups[gk])
		}
		if b.overflow != nil {
			recs = append(recs, *b.overflow)
		}
		out[k] = recs
		b.groups = make(map[groupKey]*Record)
		b.order = nil
		b.overflow = nil
	}
	return out
}

// Known lists every app this sink has ever buffered for — the prune
// worklist, as applog.Sink.Known is for logs.
func (s *Sink) Known() []applog.AppKey {
	s.mu.Lock()
	defer s.mu.Unlock()
	out := make([]applog.AppKey, 0, len(s.apps))
	for k := range s.apps {
		out = append(out, k)
	}
	return out
}
  • [ ] Step 4: Write read.go
package apperr

import (
	"context"
	"sort"
	"time"

	"github.com/carlosframework/platform/internal/applog"
	"github.com/carlosframework/platform/internal/release"
)

// Prefix is the app's errors/ subtree (spec §3.2): a sibling family of
// logs/ with the same key shape below it.
func Prefix(acct, app string) string { return applog.FamilyPrefix(applog.FamilyErrors, acct, app) }

// BoxPrefix is one box's subtree within an app's errors.
func BoxPrefix(acct, app, box string) string {
	return applog.FamilyBoxPrefix(applog.FamilyErrors, acct, app, box)
}

// ChunkKey names one flushed errors chunk; see applog.ChunkKey.
func ChunkKey(acct, app, box, hour string, seq uint64, size int) string {
	return applog.FamilyChunkKey(applog.FamilyErrors, acct, app, box, hour, seq, size)
}

// HourOf is applog.HourOf, re-exported so shippers need one import.
func HourOf(t time.Time) string { return applog.HourOf(t) }

// EncodeChunk renders records as gzipped JSONL.
func EncodeChunk(recs []Record) ([]byte, error) { return applog.EncodeJSONL(recs) }

// DecodeChunk parses a chunk, skipping malformed lines.
func DecodeChunk(b []byte) ([]Record, error) { return applog.DecodeJSONL[Record](b) }

// Filter is the read-side match. A coalesced record belongs to a window
// when [First, Last] overlaps [Since, Until] and is counted whole (spec
// §3.2: a window edge is wrong by at most one flush interval).
type Filter struct {
	Since, Until time.Time
	Host         string
	Source       string
	Release      string
	FP           string
}

// Match reports whether r survives f.
func (f Filter) Match(r Record) bool {
	last, first := r.Last, r.First
	if last.IsZero() {
		last = r.TS
	}
	if first.IsZero() {
		first = r.TS
	}
	if !f.Since.IsZero() && last.Before(f.Since) {
		return false
	}
	if !f.Until.IsZero() && first.After(f.Until) {
		return false
	}
	if f.Host != "" && r.Host != f.Host {
		return false
	}
	if f.Source != "" && r.Source != f.Source {
		return false
	}
	if f.Release != "" && r.Release != f.Release {
		return false
	}
	if f.FP != "" && r.FP != f.FP {
		return false
	}
	return true
}

// ReadWindow assembles one app's errors/ records across boxes: every hour
// intersecting the window, filtered through f, merged by (Last, Box, Seq).
// collectCap bounds memory; past it the oldest go and the bool says so.
func ReadWindow(ctx context.Context, s release.Store, acct, app string, f Filter, collectCap int) ([]Record, bool, error) {
	perBox, trimmed, err := applog.ReadChunks[Record](ctx, s, Prefix(acct, app), f.Since, f.Until, f.Match, collectCap)
	if err != nil {
		return nil, false, err
	}
	merged := MergeRecords(perBox...)
	if over := len(merged) - collectCap; over > 0 {
		merged = merged[over:]
		trimmed = true
	}
	return merged, trimmed, nil
}

// MergeRecords folds per-source slices into one timeline ordered by
// (Last, Box, Seq), oldest first.
func MergeRecords(slices ...[]Record) []Record {
	var out []Record
	for _, s := range slices {
		out = append(out, s...)
	}
	sort.SliceStable(out, func(i, j int) bool {
		a, b := out[i], out[j]
		if !a.Last.Equal(b.Last) {
			return a.Last.Before(b.Last)
		}
		if a.Box != b.Box {
			return a.Box < b.Box
		}
		return a.Seq < b.Seq
	})
	return out
}

Note on ReadChunks and the hour window: a record's chunk lives under the hour of its TS (the first occurrence); a window that starts after that hour but before Last will not list the hour. Since the flush interval is at most ten seconds plus the hour boundary, the reader widens Since by one hour when listing: in ReadWindow, pass f.Since.Add(-time.Hour) as the listing since (when non-zero) and keep f.Match as the exact test. Add that line and a comment saying why.

  • [ ] Step 5: Run tests

Run: go test ./internal/apperr/ -count=1 -v Expected: PASS.

  • [ ] Step 6: Commit
git add internal/apperr
git commit -m "apperr: coalescing sink and errors/ chunk read (spec §3.2)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>"

---

Task 5: internal/apperr grouping for the tab

Files:

  • Create: internal/apperr/group.go
  • Test: internal/apperr/group_test.go

Interfaces:

  • Produces: ``go type Group struct { FP, Source, Kind, Name, Level string; Count int; First, Last time.Time; Hosts, Releases []string; Sample Record } func GroupRecords(recs []Record) []Group // newest Last first ``
  • [ ] Step 1: Failing test
package apperr

import (
	"testing"
	"time"
)

func TestGroupRecords(t *testing.T) {
	now := time.Date(2026, 9, 7, 10, 0, 0, 0, time.UTC)
	recs := []Record{
		{FP: "a", Source: SourceEdge, Kind: KindHTTP, Name: "500 GET /x", Host: "h1", Release: "v1", Count: 3, First: now.Add(-10 * time.Minute), Last: now.Add(-5 * time.Minute), Stack: "s1"},
		{FP: "a", Source: SourceEdge, Kind: KindHTTP, Name: "500 GET /x", Host: "h2", Release: "v2", Count: 2, First: now.Add(-20 * time.Minute), Last: now.Add(-1 * time.Minute)},
		{FP: "b", Source: SourceLog, Kind: KindLog, Name: "boom", Level: "error", Host: "h1", Count: 1, First: now.Add(-30 * time.Minute), Last: now.Add(-30 * time.Minute)},
		{FP: OverflowFP, Source: SourceEdge, Name: "500 GET /z", Host: "h1", Count: 40, First: now, Last: now},
	}
	gs := GroupRecords(recs)
	if len(gs) != 3 {
		t.Fatalf("groups = %d", len(gs))
	}
	if gs[0].FP != OverflowFP || gs[1].FP != "a" || gs[2].FP != "b" {
		t.Fatalf("order by Last desc: %s %s %s", gs[0].FP, gs[1].FP, gs[2].FP)
	}
	a := gs[1]
	if a.Count != 5 || !a.First.Equal(now.Add(-20*time.Minute)) || !a.Last.Equal(now.Add(-1*time.Minute)) {
		t.Fatalf("a = %+v", a)
	}
	if len(a.Hosts) != 2 || a.Hosts[0] != "h1" || a.Hosts[1] != "h2" || len(a.Releases) != 2 {
		t.Fatalf("a hosts/releases = %v %v", a.Hosts, a.Releases)
	}
	if a.Sample.Stack != "s1" {
		t.Fatal("sample must be the earliest record")
	}
}
  • [ ] Step 2: Run, expect compile failure
  • [ ] Step 3: Implement
package apperr

import (
	"sort"
	"time"
)

// Group is one row of the Errors tab: every record sharing a fingerprint
// in the window, summed (spec §3.6).
type Group struct {
	FP       string    `json:"fp"`
	Source   string    `json:"source"`
	Kind     string    `json:"kind,omitempty"`
	Name     string    `json:"name"`
	Level    string    `json:"level,omitempty"`
	Count    int       `json:"count"`
	First    time.Time `json:"first"`
	Last     time.Time `json:"last"`
	Hosts    []string  `json:"hosts"`
	Releases []string  `json:"releases,omitempty"`
	Sample   Record    `json:"sample"`
}

// GroupRecords folds records into groups by FP, newest Last first. The
// sample is the earliest record of the group (its stack and fields are
// the first occurrence's).
func GroupRecords(recs []Record) []Group {
	byFP := make(map[string]*Group)
	var order []string
	for _, r := range recs {
		first, last := r.First, r.Last
		if first.IsZero() {
			first = r.TS
		}
		if last.IsZero() {
			last = r.TS
		}
		count := r.Count
		if count == 0 {
			count = 1
		}
		g := byFP[r.FP]
		if g == nil {
			g = &Group{FP: r.FP, Source: r.Source, Kind: r.Kind, Name: r.Name, Level: r.Level, First: first, Last: last, Sample: r}
			byFP[r.FP] = g
			order = append(order, r.FP)
		}
		g.Count += count
		if first.Before(g.First) {
			g.First = first
			g.Sample = r
		}
		if last.After(g.Last) {
			g.Last = last
		}
		g.Hosts = addUnique(g.Hosts, r.Host)
		g.Releases = addUnique(g.Releases, r.Release)
	}
	out := make([]Group, 0, len(order))
	for _, fp := range order {
		g := byFP[fp]
		sort.Strings(g.Hosts)
		sort.Strings(g.Releases)
		out = append(out, *g)
	}
	sort.SliceStable(out, func(i, j int) bool { return out[i].Last.After(out[j].Last) })
	return out
}

func addUnique(list []string, v string) []string {
	if v == "" {
		return list
	}
	for _, x := range list {
		if x == v {
			return list
		}
	}
	return append(list, v)
}
  • [ ] Step 4: Run, expect PASS. Commit
git add internal/apperr/group.go internal/apperr/group_test.go
git commit -m "apperr: group records by fingerprint for the Errors tab

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>"

---

Task 6: Ship the error sink from the agent; prune by family

Files:

  • Modify: internal/host/logship.go (logshipPrune → chunkPrune; remove hourNameOf)
  • Create: internal/host/errship.go
  • Modify: internal/host/agent.go:234-237, 384
  • Test: internal/host/logship_test.go (update two callers), internal/host/errship_test.go

Interfaces:

  • Consumes: apperr.Sink, apperr.EncodeChunk, apperr.ChunkKey, apperr.BoxPrefix, applog.FamilyBoxPrefix, applog.HourNameOfDir.
  • Produces: chunkPrune(ctx, s release.Store, fam applog.Family, box string, k applog.AppKey, now time.Time); errshipLoop(sink *apperr.Sink, box string); errshipFlush(ctx, s, pending map[applog.AppKey][]apperr.Record); package-level errSink *apperr.Sink is NOT used — the sink is constructed in runAgent beside logSink and passed to the edge in the RunWithActivatorReady callback (Task 7 adds EnableErrors).
  • [ ] Step 1: Refactor prune to take a family

In logship.go, rename logshipPrune to chunkPrune with signature (ctx context.Context, s release.Store, fam applog.Family, box string, k applog.AppKey, now time.Time); inside, prefix := applog.FamilyBoxPrefix(fam, k.Acct, k.App, box) and hourName := applog.HourNameOfDir(h.dir); delete hourNameOf. Update the call in logshipLoop to chunkPrune(ctx, s, applog.FamilyLogs, box, k, time.Now()) and the two test callers in logship_test.go likewise. Update the doc comment: "for ONE app's ONE family on THIS box's subtree".

Run: go test ./internal/host/ -run Logship -count=1 — PASS.

  • [ ] Step 2: Write the failing errship test

internal/host/errship_test.go:

package host

import (
	"context"
	"testing"
	"time"

	"github.com/carlosframework/platform/internal/apperr"
	"github.com/carlosframework/platform/internal/applog"
	"github.com/carlosframework/platform/internal/release"
)

func TestErrshipFlushWritesOneChunkPerApp(t *testing.T) {
	s := release.NewDirStore(t.TempDir())
	now := time.Date(2026, 9, 7, 10, 0, 0, 0, time.UTC)
	sink := apperr.NewSink("box1", func() time.Time { return now })
	sink.Add("bab", "web", apperr.Record{Source: apperr.SourceEdge, Kind: apperr.KindHTTP, Name: "500 GET /x", Host: "h1"})
	sink.Add("bab", "web", apperr.Record{Source: apperr.SourceEdge, Kind: apperr.KindHTTP, Name: "500 GET /x", Host: "h1"})
	pending := map[applog.AppKey][]apperr.Record{}
	errshipGather(sink, pending)
	errshipFlush(context.Background(), s, pending)
	if len(pending) != 0 {
		t.Fatalf("pending after flush = %d", len(pending))
	}
	recs, _, err := apperr.ReadWindow(context.Background(), s, "bab", "web", apperr.Filter{Since: now.Add(-time.Minute)}, 100)
	if err != nil || len(recs) != 1 || recs[0].Count != 2 {
		t.Fatalf("read back = %+v, %v", recs, err)
	}
}

func TestErrshipFlushHoldsOnFailure(t *testing.T) {
	pending := map[applog.AppKey][]apperr.Record{{Acct: "bab", App: "web"}: {{TS: time.Now(), Source: apperr.SourceEdge, Name: "x", FP: "f", Box: "box1", Seq: 1}}}
	errshipFlush(context.Background(), failingStore{}, pending)
	if len(pending) != 1 {
		t.Fatal("a failed flush must keep its records for the next round")
	}
}

(failingStore already exists in logship_test.go.)

  • [ ] Step 3: Write errship.go
// The error shipping loop — the box-side pump of the Errors tab (spec
// 2026-09-07-errors-and-analytics-design.md §3.2), shaped exactly like
// logshipLoop: the edge and the activator coalesce into one apperr.Sink;
// every tick this drains it and writes one immutable chunk per app under
// apps/<sqid>/<app>/errors/<box>/<hour>/; on each hour boundary it prunes
// its own subtree with the log retention (§3.6: errors follow log
// retention, age and byte cap).
package host

import (
	"context"
	"errors"
	"log"
	"os"
	"time"

	"github.com/carlosframework/platform/internal/apperr"
	"github.com/carlosframework/platform/internal/applog"
	"github.com/carlosframework/platform/internal/release"
)

const errshipTick = 10 * time.Second

func errshipLoop(sink *apperr.Sink, box string) {
	defer func() {
		if r := recover(); r != nil {
			log.Printf("agent: errship loop panic, error shipping stopped for this process: %v", r)
		}
	}()
	ctx := context.Background()
	s, target, err := openStatusStore(ctx, os.Getenv)
	if err != nil {
		if errors.Is(err, errNoStatusStore) {
			log.Printf("agent: errship loop disabled: neither CARLOS_STATUS_BUCKET nor CARLOS_DEPLOYMENT_BUCKET is set")
		} else {
			log.Printf("agent: errship loop disabled: %v", err)
		}
		return
	}
	log.Printf("agent: errship loop armed (10s ticks), publishing to %s", target)

	pending := make(map[applog.AppKey][]apperr.Record)
	pruned := ""
	ticker := time.NewTicker(errshipTick)
	defer ticker.Stop()
	for range ticker.C {
		errshipGather(sink, pending)
		errshipFlush(ctx, s, pending)
		if hour := applog.HourOf(time.Now()); hour != pruned {
			for _, k := range errshipKnownApps(pending, sink) {
				chunkPrune(ctx, s, applog.FamilyErrors, box, k, time.Now())
			}
			pruned = hour
		}
	}
}

// errshipGather drains the sink into pending under the same per-app cap
// logship uses — oldest first.
func errshipGather(sink *apperr.Sink, pending map[applog.AppKey][]apperr.Record) {
	for k, recs := range sink.Drain() {
		merged := append(pending[k], recs...)
		if over := len(merged) - logshipMaxPending; over > 0 {
			merged = merged[over:]
		}
		pending[k] = merged
	}
}

// errshipFlush writes one create-only chunk per pending app and forgets
// what landed; a failed app keeps its records for the next round.
func errshipFlush(ctx context.Context, s release.Store, pending map[applog.AppKey][]apperr.Record) {
	for k, recs := range pending {
		if len(recs) == 0 {
			delete(pending, k)
			continue
		}
		body, err := apperr.EncodeChunk(recs)
		if err != nil {
			log.Printf("agent: errship encode %s/%s: %v (dropping %d records)", k.Acct, k.App, err, len(recs))
			delete(pending, k)
			continue
		}
		key := apperr.ChunkKey(k.Acct, k.App, recs[0].Box, apperr.HourOf(recs[0].TS), recs[0].Seq, len(body))
		if _, err := s.Put(ctx, key, body, ""); err != nil {
			log.Printf("agent: errship flush %s/%s: %v (%d records held)", k.Acct, k.App, err, len(recs))
			continue
		}
		delete(pending, k)
	}
}

func errshipKnownApps(pending map[applog.AppKey][]apperr.Record, sink *apperr.Sink) []applog.AppKey {
	seen := make(map[applog.AppKey]bool)
	var out []applog.AppKey
	for k := range pending {
		if !seen[k] {
			seen[k] = true
			out = append(out, k)
		}
	}
	for _, k := range sink.Known() {
		if !seen[k] {
			seen[k] = true
			out = append(out, k)
		}
	}
	return out
}
  • [ ] Step 4: Wire in agent.go

Beside logSink (line 234-237):

	var logSink *applog.Sink
	var errSink *apperr.Sink
	if _, _, err := openStatusStore(context.Background(), os.Getenv); err == nil {
		logSink = applog.NewSink(status.Label(readMachineID()), time.Now)
		errSink = apperr.NewSink(status.Label(readMachineID()), time.Now)
	}

Beside go logshipLoop(...) (line 384): if errSink != nil { go errshipLoop(errSink, status.Label(readMachineID())) } — match however logshipLoop is gated there (read the surrounding lines and mirror the condition exactly).

In the RunWithActivatorReady callback after e.EnableUsage(m): if errSink != nil { e.EnableErrors(errSink) } — EnableErrors arrives in Task 7; add this line in Task 7 instead if the build order matters. Order the tasks so Task 7 lands before this file compiles: do Task 7's edge change first if convenient, or add the EnableErrors call in Task 7. Decision: add the agent.go callback line in Task 7, and only the sink construction and loop here.

  • [ ] Step 5: Test and commit

Run: go build ./... && go test ./internal/host/ -count=1 Expected: PASS.

git add internal/host
git commit -m "host: ship coalesced errors like logs; prune by chunk family

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>"

---

Task 7: The edge observes status and emits edge errors

Files:

  • Create: internal/activator/answered.go
  • Modify: internal/activator/activator.go:1298-1330 (three http.Error sites in serve)
  • Modify: internal/edge/usagemeter.go (countingWriter)
  • Modify: internal/edge/edge.go (Edge fields, EnableErrors, ServeHTTP at 1241-1250 and the hibernating branch)
  • Modify: internal/host/agent.go (callback line)
  • Test: internal/activator/answered_test.go, internal/edge/usagemeter_test.go, internal/edge/edgeerrors_test.go

Interfaces:

  • Produces: activator.WithAnswerFlag(r *http.Request) (*http.Request, *bool); activator.MarkAnswered(r *http.Request); edge.ErrorSink interface{ Add(acct, app string, r apperr.Record) }; (*Edge).EnableErrors(ErrorSink); countingWriter gains status int, ctype string, wroteHdr bool and a WriteHeader.
  • [ ] Step 1: Failing tests

internal/activator/answered_test.go:

package activator

import (
	"net/http/httptest"
	"testing"
)

func TestAnswerFlagRoundTrip(t *testing.T) {
	r := httptest.NewRequest("GET", "http://h/", nil)
	MarkAnswered(r) // no flag: a no-op, never a panic
	r, flag := WithAnswerFlag(r)
	if *flag {
		t.Fatal("flag must start false")
	}
	MarkAnswered(r)
	if !*flag {
		t.Fatal("MarkAnswered must set the flag the edge holds")
	}
}

Append to internal/edge/usagemeter_test.go:

func TestCountingWriterObservesStatusAndType(t *testing.T) {
	var n int64
	rec := httptest.NewRecorder()
	w := &countingWriter{ResponseWriter: rec, n: &n}
	w.Header().Set("Content-Type", "text/html; charset=utf-8")
	w.WriteHeader(103) // informational: forwarded, not latched
	w.WriteHeader(http.StatusBadGateway)
	w.WriteHeader(http.StatusOK) // superfluous second final status: ignored
	if w.status != http.StatusBadGateway || w.ctype != "text/html; charset=utf-8" {
		t.Fatalf("latched %d %q", w.status, w.ctype)
	}
	// Implicit 200 at first Write.
	rec2 := httptest.NewRecorder()
	w2 := &countingWriter{ResponseWriter: rec2, n: &n}
	w2.Write([]byte("x"))
	if w2.status != http.StatusOK {
		t.Fatalf("implicit status = %d", w2.status)
	}
	// Nothing written at all: status stays 0 (the caller treats it as a
	// bodiless 200 that is never a pageview and never an error).
	w3 := &countingWriter{ResponseWriter: httptest.NewRecorder(), n: &n}
	if w3.status != 0 {
		t.Fatal("untouched writer must report 0")
	}
}

internal/edge/edgeerrors_test.go:

package edge

import (
	"net/http"
	"net/http/httptest"
	"strings"
	"sync"
	"testing"

	"github.com/carlosframework/platform/internal/activator"
	"github.com/carlosframework/platform/internal/apperr"
	"github.com/carlosframework/platform/internal/registry"
)

type memErrSink struct {
	mu   sync.Mutex
	recs []apperr.Record
	apps []string
}

func (m *memErrSink) Add(acct, app string, r apperr.Record) {
	m.mu.Lock()
	defer m.mu.Unlock()
	m.recs = append(m.recs, r)
	m.apps = append(m.apps, acct+"/"+app)
}

func TestEdgeEmitsErrorOnUpstream5xx(t *testing.T) {
	backend := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		switch r.URL.Path {
		case "/boom":
			http.Error(w, "kaboom", http.StatusInternalServerError)
		case "/missing":
			http.NotFound(w, r)
		default:
			w.Header().Set("Content-Type", "text/html")
			w.Write([]byte("<p>ok</p>"))
		}
	}))
	t.Cleanup(backend.Close)
	reg := openReg(t)
	rt := registry.Route{Host: "app.example.com", Kind: "service", Addr: backend.Listener.Addr().String(), Status: "ready", App: "web", Version: "v7"}
	if err := reg.Upsert(rt); err != nil {
		t.Fatal(err)
	}
	e, err := New(reg, nil)
	if err != nil {
		t.Fatal(err)
	}
	sink := &memErrSink{}
	e.EnableErrors(sink)
	srv := httptest.NewServer(e)
	t.Cleanup(srv.Close)

	do := func(method, path string) int {
		req, _ := http.NewRequest(method, srv.URL+path, nil)
		req.Host = "app.example.com"
		resp, err := http.DefaultClient.Do(req)
		if err != nil {
			t.Fatal(err)
		}
		resp.Body.Close()
		return resp.StatusCode
	}
	if do("POST", "/boom?token=abc") != 500 || do("GET", "/missing") != 404 || do("GET", "/") != 200 {
		t.Fatal("upstream statuses must pass through unchanged")
	}
	if len(sink.recs) != 1 {
		t.Fatalf("errors emitted = %d, want 1 (the 500 only)", len(sink.recs))
	}
	r := sink.recs[0]
	if r.Source != apperr.SourceEdge || r.Kind != apperr.KindHTTP || r.Status != 500 || r.Host != "app.example.com" || r.Release != "v7" {
		t.Fatalf("record = %+v", r)
	}
	if r.Name != "500 POST /boom" || r.Path != "/boom" || strings.Contains(r.Name, "token") {
		t.Fatalf("name/path = %q %q", r.Name, r.Path)
	}
}

// answering503 is an Activator that answers every request itself with the
// wake-path 503 the real one writes, and marks it as the activator's own.
type answering503 struct{}

func (answering503) ServeHTTP(w http.ResponseWriter, r *http.Request, rt registry.Route, proxy http.Handler) bool {
	activator.MarkAnswered(r)
	http.Error(w, "instance failed to wake", http.StatusServiceUnavailable)
	return true
}
func (answering503) RetireExcept(map[string]bool) {}

func TestEdgeSkipsActivatorAnswered503(t *testing.T) {
	reg := openReg(t)
	rt := registry.Route{Host: "sleepy.example.com", Kind: "instance", Socket: t.TempDir() + "/x.sock", Status: "ready", Hibernate: true, App: "web"}
	if err := reg.Upsert(rt); err != nil {
		t.Fatal(err)
	}
	e, err := New(reg, answering503{})
	if err != nil {
		t.Fatal(err)
	}
	sink := &memErrSink{}
	e.EnableErrors(sink)
	rec := httptest.NewRecorder()
	req := httptest.NewRequest("GET", "http://sleepy.example.com/", nil)
	e.ServeHTTP(rec, req)
	if rec.Code != 503 {
		t.Fatalf("status = %d", rec.Code)
	}
	if len(sink.recs) != 0 {
		t.Fatalf("an activator-answered 503 is already a platform log error; got %d edge errors", len(sink.recs))
	}
}

If registry.Route needs different fields for a hibernating instance route to load into the edge's map (check TestHibernat* tests in edge_test.go or activator_e2e_test.go for the minimal upsert they use), copy that shape.

  • [ ] Step 2: Run to confirm failure

Run: go test ./internal/activator/ -run TestAnswerFlag -count=1; go test ./internal/edge/ -run 'TestCountingWriterObserves|TestEdgeEmits|TestEdgeSkips' -count=1 Expected: compile failures.

  • [ ] Step 3: internal/activator/answered.go
package activator

import (
	"context"
	"net/http"
)

// answeredKey carries the flag the edge attaches so it can tell a 503 the
// activator wrote on the wake path from one the upstream returned. The
// wake failure already reaches the member as a platform log error
// (emitEvent, logsink.go); the edge must not also file it as an edge
// error (spec 2026-09-07 §3.2, "exactly once").
type answeredKey struct{}

// WithAnswerFlag returns r carrying a fresh flag and the flag itself.
func WithAnswerFlag(r *http.Request) (*http.Request, *bool) {
	f := new(bool)
	return r.WithContext(context.WithValue(r.Context(), answeredKey{}, f)), f
}

// MarkAnswered sets the flag when r carries one; a no-op otherwise.
func MarkAnswered(r *http.Request) {
	if f, ok := r.Context().Value(answeredKey{}).(*bool); ok {
		*f = true
	}
}

In serve (activator.go), call MarkAnswered(r) immediately before each of the four http.Error(w, …, http.StatusServiceUnavailable) calls (sidecar-asleep, waking elsewhere, waking, failed to wake).

  • [ ] Step 4: countingWriter observes

In usagemeter.go:

// countingWriter counts response body bytes and OBSERVES the final status
// and Content-Type (spec 2026-09-07 §3.3): it is the one wrapper installed
// at a single site for every response shape — cache hit, plain proxy,
// hibernating, upgrade — so it is where the edge learns that an upstream
// answered 5xx (an edge error) or served HTML (a pageview, step 2). It
// sits below the compressor, so Content-Type is the synced value
// gzipWriter.syncHeader copied down before calling this WriteHeader.
// Deliberately does NOT implement io.ReaderFrom: io.Copy falls back to
// Write, which counts. Hijack/Flush forward via the interfaces the wrapped
// writer provides, mirroring recordingResponseWriter (edge.go).
type countingWriter struct {
	http.ResponseWriter
	n        *int64
	status   int    // final status (>= 200); 0 until one is written
	ctype    string // Content-Type at the final WriteHeader
	wroteHdr bool
}

// WriteHeader latches the first final status. 1xx are forwarded and never
// latched (recordingResponseWriter.WriteHeader's reasoning).
func (c *countingWriter) WriteHeader(status int) {
	if status < 200 {
		c.ResponseWriter.WriteHeader(status)
		return
	}
	if !c.wroteHdr {
		c.wroteHdr = true
		c.status = status
		c.ctype = c.ResponseWriter.Header().Get("Content-Type")
	}
	c.ResponseWriter.WriteHeader(status)
}

func (c *countingWriter) Write(p []byte) (int, error) {
	if !c.wroteHdr {
		c.wroteHdr = true
		c.status = http.StatusOK
		c.ctype = c.ResponseWriter.Header().Get("Content-Type")
	}
	n, err := c.ResponseWriter.Write(p)
	*c.n += int64(n)
	return n, err
}

Keep Hijack and Flush as they are.

  • [ ] Step 5: Edge wiring

In edge.go:

Add near the usageMeter field:

	// errSink, when set, receives edge-observed errors (an upstream 5xx)
	// — spec 2026-09-07 §3.2. Nil disables emission; nothing else changes.
	errSink atomic.Pointer[errorSinkRef]

and:

// ErrorSink is where the edge files an error it observed on the serving
// path. apperr.Sink satisfies it; the interface lives here so this package
// depends on the contract, not the buffer.
type ErrorSink interface {
	Add(acct, app string, r apperr.Record)
}

type errorSinkRef struct{ ErrorSink }

// EnableErrors wires the error sink in. Called once at startup by the
// agent; nil is a valid no-op.
func (e *Edge) EnableErrors(s ErrorSink) {
	if s == nil {
		e.errSink.Store(nil)
		return
	}
	e.errSink.Store(&errorSinkRef{s})
}

// emitEdgeError files one edge-observed 5xx for rt. The path is sanitised
// (query and token segments off) before it becomes the record's name.
func (e *Edge) emitEdgeError(rt registry.Route, r *http.Request, status int) {
	ref := e.errSink.Load()
	if ref == nil {
		return
	}
	path := apperr.SanitizePath(r.URL.Path)
	ref.Add(release.RouteAccount(rt), rt.App, apperr.Record{
		Host:    rt.Host,
		Source:  apperr.SourceEdge,
		Kind:    apperr.KindHTTP,
		Name:    fmt.Sprintf("%d %s %s", status, r.Method, path),
		Path:    path,
		Status:  status,
		Release: rt.Version,
	})
}

Replace the meter block at edge.go:1241-1245 with:

	// The observing writer is installed UNCONDITIONALLY now — the usage
	// meter is one of its two consumers, the errors sink the other — at
	// the same single site, below everything that follows, so all three
	// dispatch shapes plus the upgrade path pass through it. Meter and
	// sink are each nil-safe. Status routes are neither billed nor observed.
	//
	// (hopped attribution paragraph: keep the existing comment text here.)
	var bytes int64
	ob := &countingWriter{ResponseWriter: w, n: &bytes}
	w = ob
	var answered *bool
	if up.hibernating {
		// The activator marks the wake-path 503s it writes itself; those
		// are already the member's platform log error and are not filed
		// again as edge errors (spec §3.2).
		r, answered = activator.WithAnswerFlag(r)
	}
	if up.route.Kind != "status" {
		defer func() {
			if m := e.usageMeter.Load(); m != nil {
				m.Record(up.route, 1, bytes, hopped)
			}
			if ob.status >= 500 && (answered == nil || !*answered) {
				e.emitEdgeError(up.route, r, ob.status)
			}
		}()
	}

Add "github.com/carlosframework/platform/internal/activator" and "github.com/carlosframework/platform/internal/apperr" to the imports. Confirm no cycle: go build ./... (activator must not import edge; it does not today).

Existing usage tests that construct the edge without a meter must still pass: Record is skipped when the meter is nil.

  • [ ] Step 6: agent.go callback

After e.EnableUsage(m):

		if errSink != nil {
			e.EnableErrors(errSink)
		}
  • [ ] Step 7: Run edge, activator, host packages

Run: go build ./... && go vet ./internal/edge/ ./internal/activator/ ./internal/host/ && go test ./internal/edge/ ./internal/activator/ ./internal/host/ -count=1 Expected: PASS, including every pre-existing usage, compress and cache test.

  • [ ] Step 8: Commit
git add internal/activator/answered.go internal/activator/activator.go internal/activator/answered_test.go internal/edge internal/host/agent.go
git commit -m "edge: the counting writer observes status; upstream 5xx become edge errors

Installed unconditionally at the one site every response shape passes;
the activator marks its own wake-path 503s so they are filed once, as the
platform log error they already are (spec 2026-09-07 §3.2–3.3).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>"

---

Task 8: internal/telemetry settings object

Files:

  • Create: internal/telemetry/telemetry.go
  • Test: internal/telemetry/telemetry_test.go

Interfaces:

  • Produces: ``go const (LogLevelError = "error"; LogLevelWarn = "warn"; CountEdge = "edge"; CountOff = "off"; DoorOn = "on"; DoorOff = "off") type Analytics struct { Count string json:"count,omitempty"; Door string json:"door,omitempty" } type Errors struct { LogLevel string json:"log_level,omitempty" } type Settings struct { Analytics *Analytics json:"analytics,omitempty"; Errors *Errors json:"errors,omitempty" } func AccountKey(sqid string) string func AppKey(sqid, app string) string func Read(ctx, s release.Store, key string) (Settings, string, error) // etag; os.ErrNotExist → zero Settings, "", nil func Update(ctx, s release.Store, key string, fn func(*Settings)) error // read-modify-CAS-write, 3 attempts type Resolved struct { LogLevel, LogLevelFrom, Count, CountFrom, Door, DoorFrom string } func Resolve(ctx, s release.Store, sqid, app string) (Resolved, error) func ValidLogLevel(s string) bool ``
  • [ ] Step 1: Failing test
package telemetry

import (
	"context"
	"testing"

	"github.com/carlosframework/platform/internal/release"
)

func TestResolveAppThenAccountThenDefault(t *testing.T) {
	ctx := context.Background()
	s := release.NewDirStore(t.TempDir())
	r, err := Resolve(ctx, s, "bab", "web")
	if err != nil {
		t.Fatal(err)
	}
	if r.LogLevel != LogLevelError || r.LogLevelFrom != "default" || r.Count != CountOff || r.CountFrom != "default" || r.Door != DoorOn {
		t.Fatalf("defaults = %+v", r)
	}
	if err := Update(ctx, s, AccountKey("bab"), func(st *Settings) {
		st.Errors = &Errors{LogLevel: LogLevelWarn}
		st.Analytics = &Analytics{Count: CountEdge}
	}); err != nil {
		t.Fatal(err)
	}
	r, _ = Resolve(ctx, s, "bab", "web")
	if r.LogLevel != LogLevelWarn || r.LogLevelFrom != "account" || r.Count != CountEdge || r.CountFrom != "account" || r.Door != DoorOn || r.DoorFrom != "default" {
		t.Fatalf("account level = %+v", r)
	}
	if err := Update(ctx, s, AppKey("bab", "web"), func(st *Settings) {
		st.Analytics = &Analytics{Count: CountOff}
	}); err != nil {
		t.Fatal(err)
	}
	r, _ = Resolve(ctx, s, "bab", "web")
	if r.LogLevel != LogLevelWarn || r.Count != CountOff || r.CountFrom != "app" {
		t.Fatalf("app override = %+v", r)
	}
	// Update preserves the other key.
	if err := Update(ctx, s, AppKey("bab", "web"), func(st *Settings) { st.Errors = &Errors{LogLevel: LogLevelError} }); err != nil {
		t.Fatal(err)
	}
	got, _, _ := Read(ctx, s, AppKey("bab", "web"))
	if got.Analytics == nil || got.Analytics.Count != CountOff || got.Errors == nil || got.Errors.LogLevel != LogLevelError {
		t.Fatalf("read back = %+v", got)
	}
}
  • [ ] Step 2: Run, expect compile failure
  • [ ] Step 3: Implement
// Package telemetry is the settings object behind the Errors and Analytics
// tabs (spec 2026-09-07-errors-and-analytics-design.md §3.3): one JSON
// shape at two levels — accounts/<sqid>/telemetry.json and
// apps/<sqid>/<app>/telemetry.json — resolved app → account → built-in
// default. Defaults are rulings (Paul, 2026-09-07): analytics off, the
// browser door on, the log level error.
package telemetry

import (
	"context"
	"encoding/json"
	"errors"
	"fmt"
	"os"

	"github.com/carlosframework/platform/internal/release"
)

const (
	LogLevelError = "error"
	LogLevelWarn  = "warn"
	CountEdge     = "edge"
	CountOff      = "off"
	DoorOn        = "on"
	DoorOff       = "off"
)

// Analytics is the analytics half. Empty strings mean "not set here".
type Analytics struct {
	Count string `json:"count,omitempty"`
	Door  string `json:"door,omitempty"`
}

// Errors is the errors half.
type Errors struct {
	LogLevel string `json:"log_level,omitempty"`
}

// Settings is the wire shape at either level. Additive: unknown keys are
// ignored, absent halves inherit.
type Settings struct {
	Analytics *Analytics `json:"analytics,omitempty"`
	Errors    *Errors    `json:"errors,omitempty"`
}

// AccountKey is the account-level object, beside account.json (which stays
// identity-only).
func AccountKey(sqid string) string { return "accounts/" + sqid + "/telemetry.json" }

// AppKey is the per-app override.
func AppKey(sqid, app string) string { return release.AppPrefix(sqid, app) + "telemetry.json" }

// ValidLogLevel reports whether s is a level the setting accepts.
func ValidLogLevel(s string) bool { return s == LogLevelError || s == LogLevelWarn }

// Read returns the object at key and its etag; an absent object is the
// zero Settings with an empty etag and no error.
func Read(ctx context.Context, s release.Store, key string) (Settings, string, error) {
	body, etag, err := s.Get(ctx, key)
	if err != nil {
		if errors.Is(err, os.ErrNotExist) {
			return Settings{}, "", nil
		}
		return Settings{}, "", err
	}
	var st Settings
	if err := json.Unmarshal(body, &st); err != nil {
		return Settings{}, "", fmt.Errorf("telemetry: %s: %w", key, err)
	}
	return st, etag, nil
}

// Update applies fn to the object at key under compare-and-swap, three
// attempts, so two owners editing different halves at once both land.
func Update(ctx context.Context, s release.Store, key string, fn func(*Settings)) error {
	var last error
	for attempt := 0; attempt < 3; attempt++ {
		st, etag, err := Read(ctx, s, key)
		if err != nil {
			return err
		}
		fn(&st)
		body, err := json.Marshal(st)
		if err != nil {
			return err
		}
		if _, err := s.Put(ctx, key, body, etag); err != nil {
			if errors.Is(err, release.ErrCASConflict) {
				last = err
				continue
			}
			return err
		}
		return nil
	}
	return last
}

// Resolved is what a reader acts on, with each value's provenance so a
// settings page can say "inherited from the account".
type Resolved struct {
	LogLevel, LogLevelFrom string
	Count, CountFrom       string
	Door, DoorFrom         string
}

// Resolve reads both levels and folds them over the defaults. A store
// error fails the resolve; an absent object at either level is fine.
func Resolve(ctx context.Context, s release.Store, sqid, app string) (Resolved, error) {
	r := Resolved{LogLevel: LogLevelError, LogLevelFrom: "default", Count: CountOff, CountFrom: "default", Door: DoorOn, DoorFrom: "default"}
	if s == nil {
		return r, nil
	}
	acct, _, err := Read(ctx, s, AccountKey(sqid))
	if err != nil {
		return r, err
	}
	appSt, _, err := Read(ctx, s, AppKey(sqid, app))
	if err != nil {
		return r, err
	}
	for _, lvl := range []struct {
		from string
		st   Settings
	}{{"account", acct}, {"app", appSt}} {
		if lvl.st.Errors != nil && ValidLogLevel(lvl.st.Errors.LogLevel) {
			r.LogLevel, r.LogLevelFrom = lvl.st.Errors.LogLevel, lvl.from
		}
		if lvl.st.Analytics != nil {
			if c := lvl.st.Analytics.Count; c == CountEdge || c == CountOff {
				r.Count, r.CountFrom = c, lvl.from
			}
			if d := lvl.st.Analytics.Door; d == DoorOn || d == DoorOff {
				r.Door, r.DoorFrom = d, lvl.from
			}
		}
	}
	return r, nil
}

Check how DirStore.Put treats the ifMatch argument ("" = unconditional? or create-only?). Read internal/release/store.go around func (d *DirStore) Put and confirm that Put(ctx, key, body, "") on an existing key succeeds or is create-only; if create-only, Update must pass the etag read back (which it does) and for a first write the empty etag must mean create — verify with the test.

  • [ ] Step 4: Run, PASS, commit
git add internal/telemetry
git commit -m "telemetry: the two-level settings object behind Errors and Analytics (spec §3.3)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>"

---

Task 9: Console — Errors API, tab, and the log-level setting

Files:

  • Create: internal/console/errorsview.go, internal/console/apierrors.go, internal/console/templates/app_errors.html
  • Modify: internal/console/server.go (routes near 1050 and 1071; page struct near 1370; handleAppErrors), internal/console/authcli.go:595 (two cliRoutes), internal/console/templates/appnav.html, internal/console/templates/app_settings.html, internal/console/templates/account_settings.html
  • Test: internal/console/apierrors_test.go

Interfaces:

  • Consumes: apperr.*, applog.ReadWindow, telemetry.Resolve/Update, s.instanceGate, s.appPage, s.accountApp, s.account, s.audit, s.setFlash, logsCollectCap, instanceStoreTimeout.
  • Produces: ``go type errorsWindow struct { Groups []apperr.Group; Records []apperr.Record; Truncated bool; LogLevel, LogLevelFrom string; Sources []string } func (s *Server) readErrors(ctx, sqid, app string, f apperr.Filter, level string) (errorsWindow, error) type errorsResponse struct { Groups []apperr.Group json:"groups"; Records []apperr.Record json:"records,omitempty"; Truncated bool json:"truncated"; LogLevel string json:"loglevel"; Sources []string json:"sources" } ` Routes: GET /api/cli/apps/{account}/{app}/errors (Instance, Operate); PUT /api/cli/apps/{account}/{app}/telemetry (Operate) body {"errors":{"loglevel":"warn"}}; GET /a/{sqid}/apps/{app}/errors; POST /a/{sqid}/apps/{app}/errors/level; POST /a/{sqid}/errors/level`.
  • [ ] Step 1: Failing tests — internal/console/apierrors_test.go
package console

import (
	"context"
	"encoding/json"
	"net/http"
	"strings"
	"testing"
	"time"

	"github.com/carlosframework/platform/internal/apperr"
	"github.com/carlosframework/platform/internal/applog"
	"github.com/carlosframework/platform/internal/release"
	"github.com/carlosframework/platform/internal/telemetry"
)

func errorsURL(fx *consoleFixture, acct, app, query string) string {
	u := fx.ts.URL + "/api/cli/apps/" + acct + "/" + app + "/errors"
	if query != "" {
		u += "?" + query
	}
	return u
}

// seedErrors ships one coalesced edge error and, via seedLogChunks, one
// error-level platform line and one warn-level app line.
func seedErrors(t *testing.T, fx *consoleFixture, now time.Time) {
	t.Helper()
	seedLogChunks(t, fx, now)
	warn := []applog.Record{{TS: now.Add(-7 * time.Minute), Box: "box1", Host: "titogo.example.com",
		Stream: applog.StreamApp, Level: applog.LevelWarn, Msg: "slow query 4123ms", Seq: 150}}
	body, _ := applog.EncodeChunk(warn)
	if _, err := fx.rel.Put(context.Background(), applog.ChunkKey(release.OpsAccount, "titogo", "box1", applog.HourOf(warn[0].TS), 150, len(body)), body, ""); err != nil {
		t.Fatal(err)
	}
	e := []apperr.Record{{TS: now.Add(-3 * time.Minute), Box: "box1", Host: "titogo.example.com",
		Source: apperr.SourceEdge, Kind: apperr.KindHTTP, Name: "500 POST /checkout", Path: "/checkout", Status: 500,
		Release: "v3", FP: "edge500", FPV: 1, Count: 12, First: now.Add(-9 * time.Minute), Last: now.Add(-3 * time.Minute), Seq: 200}}
	eb, _ := apperr.EncodeChunk(e)
	if _, err := fx.rel.Put(context.Background(), apperr.ChunkKey(release.OpsAccount, "titogo", "box1", apperr.HourOf(e[0].TS), 200, len(eb)), eb, ""); err != nil {
		t.Fatal(err)
	}
}

func TestAPIErrorsGroupsBothFamilies(t *testing.T) {
	fx, token := instanceFixture(t, "code-errors-1")
	now := time.Now().UTC()
	seedErrors(t, fx, now)
	get := func(query string) errorsResponse {
		t.Helper()
		status, _, body := rawReq(t, http.MethodGet, errorsURL(fx, fx.sqid, "titogo", query), token, "")
		if status != http.StatusOK {
			t.Fatalf("GET ?%s = %d: %s", query, status, body)
		}
		var resp errorsResponse
		if err := json.Unmarshal([]byte(body), &resp); err != nil {
			t.Fatal(err)
		}
		return resp
	}
	resp := get("")
	if len(resp.Groups) != 2 {
		t.Fatalf("groups = %+v", resp.Groups)
	}
	if resp.Groups[0].FP != "edge500" || resp.Groups[0].Count != 12 || resp.Groups[0].Releases[0] != "v3" {
		t.Fatalf("newest first, coalesced count: %+v", resp.Groups[0])
	}
	if resp.Groups[1].Source != apperr.SourceLog || !strings.Contains(resp.Groups[1].Name, "wake failed") {
		t.Fatalf("log-derived group: %+v", resp.Groups[1])
	}
	if resp.LogLevel != "error" || len(resp.Sources) != 2 {
		t.Fatalf("meta = %q %v", resp.LogLevel, resp.Sources)
	}
	// A warn read widens without changing the setting.
	if resp = get("level=warn"); len(resp.Groups) != 3 {
		t.Fatalf("level=warn groups = %d", len(resp.Groups))
	}
	// Source and fp filters.
	if resp = get("source=edge"); len(resp.Groups) != 1 || resp.Groups[0].FP != "edge500" {
		t.Fatalf("source filter = %+v", resp.Groups)
	}
	if resp = get("fp=edge500"); len(resp.Records) != 1 || resp.Records[0].Status != 500 {
		t.Fatalf("fp expand = %+v", resp.Records)
	}
}

func TestAPITelemetryPutSetsLogLevel(t *testing.T) {
	fx, token := instanceFixture(t, "code-errors-2")
	now := time.Now().UTC()
	seedErrors(t, fx, now)
	// The instance token may read but not write settings: Operate only.
	status, _, body := rawReq(t, http.MethodPut, fx.ts.URL+"/api/cli/apps/"+fx.sqid+"/titogo/telemetry", token, `{"errors":{"log_level":"warn"}}`)
	if status != http.StatusForbidden && status != http.StatusNotFound {
		t.Fatalf("instance token PUT = %d %s", status, body)
	}
	op := serviceToken(t, fx.st, OpsAccountID, "ops", "titogo", svcrole.Operate)
	status, _, body = rawReq(t, http.MethodPut, fx.ts.URL+"/api/cli/apps/"+fx.sqid+"/titogo/telemetry", op, `{"errors":{"log_level":"warn"}}`)
	if status != http.StatusOK {
		t.Fatalf("PUT = %d %s", status, body)
	}
	r, _ := telemetry.Resolve(context.Background(), fx.rel, fx.sqid, "titogo")
	if r.LogLevel != "warn" || r.LogLevelFrom != "app" {
		t.Fatalf("resolved = %+v", r)
	}
	status, _, body = rawReq(t, http.MethodGet, errorsURL(fx, fx.sqid, "titogo", ""), token, "")
	var resp errorsResponse
	json.Unmarshal([]byte(body), &resp)
	if status != 200 || resp.LogLevel != "warn" || len(resp.Groups) != 3 {
		t.Fatalf("after set: %d level=%q groups=%d", status, resp.LogLevel, len(resp.Groups))
	}
	status, _, body = rawReq(t, http.MethodPut, fx.ts.URL+"/api/cli/apps/"+fx.sqid+"/titogo/telemetry", op, `{"errors":{"log_level":"loud"}}`)
	if status != http.StatusBadRequest {
		t.Fatalf("bad level = %d %s", status, body)
	}
}

func TestAppErrorsPage(t *testing.T) {
	fx, _ := instanceFixture(t, "code-errors-page")
	now := time.Now().UTC()
	seedErrors(t, fx, now)
	fx.setUser("paul@keymail.dev")
	c := newBrowser(t)
	signUp(t, fx, c, "code-errors-signin")
	status, _, body := get(t, c, fx.u("/apps/titogo/errors"))
	if status != http.StatusOK {
		t.Fatalf("errors page: %d %s", status, body)
	}
	for _, want := range []string{"500 POST /checkout", "wake failed", "carlos errors --app titogo", "Sources live: log, edge", "to the nearest flush"} {
		if !strings.Contains(body, want) {
			t.Errorf("page missing %q", want)
		}
	}
	if strings.Contains(body, "slow query") {
		t.Error("warn lines must not show at the default level")
	}
	status, _, body = get(t, c, fx.u("/apps/titogo/errors?level=warn"))
	if status != 200 || !strings.Contains(body, "slow query") {
		t.Fatalf("level=warn page: %d", status)
	}
	status, _, body = get(t, c, fx.u("/apps/titogo/errors?fp=edge500"))
	if status != 200 || !strings.Contains(body, "/checkout") || !strings.Contains(body, "v3") {
		t.Fatalf("expanded group: %d", status)
	}
	status, _, body = get(t, c, fx.u("/apps/titogo/errors?since=1ns"))
	if status != 200 || !strings.Contains(body, "No errors in this window") {
		t.Fatalf("empty window: %d", status)
	}
}

operateToken(t, fx): find how existing tests mint a service token with the Operate role (grep svcrole.Operate in internal/console/*_test.go; there is a helper minting tokens with a role list for the apicli tests). Use that helper; if its name differs, use it directly in place of operateToken. If no such helper exists, mint through the same store call instanceFixture uses with svcrole.Operate substituted.

  • [ ] Step 2: Run, expect compile failure
  • [ ] Step 3: errorsview.go
package console

// The Errors tab's read (spec 2026-09-07 §3.2, §3.6): errors/ records
// plus log lines at or above the resolved level, derived on read and
// never stored twice, grouped by fingerprint. The tab and the CLI's API
// share this function — the API is not an afterthought.

import (
	"context"
	"regexp"
	"sort"
	"time"

	"github.com/carlosframework/platform/internal/apperr"
	"github.com/carlosframework/platform/internal/applog"
	"github.com/carlosframework/platform/internal/telemetry"
)

// errorsWindow is one assembled read.
type errorsWindow struct {
	Groups       []apperr.Group
	Records      []apperr.Record // when f.FP is set: that group's records, oldest first
	Truncated    bool
	LogLevel     string // resolved setting
	LogLevelFrom string
	Sources      []string // sources with at least one record in the window
}

// readErrors assembles the window. level overrides the resolved setting
// for THIS read only ("" = use the setting); it is what lets a member on
// error widen one look to warn without changing anything (§8 ruling 3).
func (s *Server) readErrors(ctx context.Context, sqid, app string, f apperr.Filter, level string) (errorsWindow, error) {
	var out errorsWindow
	res, err := telemetry.Resolve(ctx, s.cfg.Releases, sqid, app)
	if err != nil {
		return out, err
	}
	out.LogLevel, out.LogLevelFrom = res.LogLevel, res.LogLevelFrom
	if level == "" {
		level = res.LogLevel
	}
	stored, trimmed, err := apperr.ReadWindow(ctx, s.cfg.Releases, sqid, app, f, logsCollectCap)
	if err != nil {
		return out, err
	}
	out.Truncated = trimmed
	var derived []apperr.Record
	if f.Source == "" || f.Source == apperr.SourceLog {
		lf := applog.Filter{Since: f.Since, Until: f.Until, Host: f.Host, Level: level}
		lines, ltrim, err := applog.ReadWindow(ctx, s.cfg.Releases, sqid, app, lf, logsCollectCap)
		if err != nil {
			return out, err
		}
		out.Truncated = out.Truncated || ltrim
		for _, l := range lines {
			r := apperr.FromLog(l)
			if (f.FP == "" || r.FP == f.FP) && (f.Release == "" || r.Release == f.Release) {
				derived = append(derived, r)
			}
		}
	}
	all := apperr.MergeRecords(stored, derived)
	if f.FP != "" {
		out.Records = all
	}
	out.Groups = apperr.GroupRecords(all)
	seen := map[string]bool{}
	for _, r := range all {
		if !seen[r.Source] {
			seen[r.Source] = true
			out.Sources = append(out.Sources, r.Source)
		}
	}
	sort.Strings(out.Sources)
	return out, nil
}

// errorsFilterFromQuery reads the tab's and the API's shared params:
// since/until/source/host/release/fp/level. The empty badParam means they
// parsed.
func errorsFilterFromQuery(q map[string][]string, now time.Time) (f apperr.Filter, level string, badParam string) {
	get := func(k string) string {
		if v := q[k]; len(v) > 0 {
			return v[0]
		}
		return ""
	}
	var err error
	if f.Since, err = applog.ParseTimeArg(get("since"), now); err != nil {
		return f, "", "since: " + err.Error()
	}
	if f.Until, err = applog.ParseTimeArg(get("until"), now); err != nil {
		return f, "", "until: " + err.Error()
	}
	if f.Since.IsZero() {
		f.Since = now.Add(-24 * time.Hour)
	}
	switch v := get("source"); v {
	case "", apperr.SourceLog, apperr.SourceEdge, apperr.SourceServer, apperr.SourceBrowser:
		f.Source = v
	default:
		return f, "", "source: want log, edge, server or browser"
	}
	f.Host, f.Release, f.FP = get("host"), get("release"), get("fp")
	if !validFP.MatchString(f.FP) {
		return f, "", "fp: want a fingerprint id"
	}
	if level = get("level"); level != "" && !telemetry.ValidLogLevel(level) {
		return f, "", "level: want error or warn"
	}
	return f, level, ""
}

var validFP = regexp.MustCompile(`^[A-Za-z0-9_.-]{0,64}$`)
  • [ ] Step 4: apierrors.go
package console

import (
	"context"
	"encoding/json"
	"log"
	"net/http"
	"time"

	"github.com/carlosframework/platform/internal/apperr"
	"github.com/carlosframework/platform/internal/telemetry"
)

// errorsResponse is the wire shape `carlos errors` and the tab read.
type errorsResponse struct {
	Groups    []apperr.Group  `json:"groups"`
	Records   []apperr.Record `json:"records,omitempty"`
	Truncated bool            `json:"truncated"`
	LogLevel  string          `json:"log_level"`
	Sources   []string        `json:"sources"`
}

// handleAPIErrors answers `carlos errors`: the instance family's gate
// (an app token may read its own errors), the shared filter, one read.
func (s *Server) handleAPIErrors(w http.ResponseWriter, r *http.Request) {
	_, acct, app, ok := s.instanceGate(w, r)
	if !ok {
		return
	}
	f, level, bad := errorsFilterFromQuery(r.URL.Query(), time.Now())
	if bad != "" {
		jsonError(w, http.StatusBadRequest, bad)
		return
	}
	ctx, cancel := context.WithTimeout(r.Context(), instanceStoreTimeout)
	defer cancel()
	win, err := s.readErrors(ctx, acct.Sqid, app, f, level)
	if err != nil {
		log.Printf("console: api errors %s/%s: %v", acct.Sqid, app, err)
		jsonError(w, http.StatusBadGateway, "could not read the errors")
		return
	}
	resp := errorsResponse{Groups: win.Groups, Records: win.Records, Truncated: win.Truncated, LogLevel: win.LogLevel, Sources: win.Sources}
	if resp.Groups == nil {
		resp.Groups = []apperr.Group{}
	}
	if resp.Sources == nil {
		resp.Sources = []string{}
	}
	writeJSON(w, http.StatusOK, resp)
}

// handleAPITelemetryPut sets the app-level telemetry object: today the
// errors log level; the analytics keys arrive with step 2. Operate only —
// an app's own token must not widen what its owner sees.
func (s *Server) handleAPITelemetryPut(w http.ResponseWriter, r *http.Request) {
	principal, acct, app, ok := s.instanceGate(w, r)
	if !ok {
		return
	}
	var body telemetry.Settings
	if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096)).Decode(&body); err != nil {
		jsonError(w, http.StatusBadRequest, "body: want {\"errors\":{\"log_level\":\"error|warn\"}}")
		return
	}
	if body.Errors == nil || !telemetry.ValidLogLevel(body.Errors.LogLevel) {
		jsonError(w, http.StatusBadRequest, "errors.log_level: want error or warn")
		return
	}
	if body.Analytics != nil {
		jsonError(w, http.StatusBadRequest, "analytics: not settable yet")
		return
	}
	ctx, cancel := context.WithTimeout(r.Context(), instanceStoreTimeout)
	defer cancel()
	if err := telemetry.Update(ctx, s.cfg.Releases, telemetry.AppKey(acct.Sqid, app), func(st *telemetry.Settings) {
		st.Errors = &telemetry.Errors{LogLevel: body.Errors.LogLevel}
	}); err != nil {
		log.Printf("console: api telemetry %s/%s: %v", acct.Sqid, app, err)
		jsonError(w, http.StatusBadGateway, "could not write the setting")
		return
	}
	s.audit(acct, app, principal, "errors-level", "set the Errors log level to "+body.Errors.LogLevel)
	writeJSON(w, http.StatusOK, map[string]string{"log_level": body.Errors.LogLevel})
}

principal is the first return of instanceGate (a string naming the token's principal — check the type; if it is not a string suitable for audit's addr, pass what the logs/instances handlers pass).

Register in authcli.go beside the logs route:

	s.cliRoute("GET /api/cli/apps/{account}/{app}/errors", svcrole.Allow(svcrole.Instance, svcrole.Operate), s.handleAPIErrors)
	s.cliRoute("PUT /api/cli/apps/{account}/{app}/telemetry", svcrole.Allow(svcrole.Operate), s.handleAPITelemetryPut)
  • [ ] Step 5: Page handler, fields, routes, templates

Page struct (near line 1370), add:

	// The Errors tab (errorsview.go).
	ErrWin    errorsWindow
	ErrSince  string // "1h" | "24h" | "168h"
	ErrSource string
	ErrLevel  string // "" = the setting
	ErrHost   string
	ErrFP     string // expanded group
	ErrsErr   string

Routes (near 1050 and 1071):

	s.mux.HandleFunc("GET /a/{sqid}/apps/{app}/errors", s.handleAppErrors)
	s.mux.HandleFunc("POST /a/{sqid}/apps/{app}/errors/level", s.handleAppErrorLevel)
	s.mux.HandleFunc("POST /a/{sqid}/errors/level", s.handleAccountErrorLevel)

Handlers (in errorsview.go):

// handleAppErrors renders the Errors tab: the same read the API answers,
// with the filters as GET params so a view is a URL.
func (s *Server) handleAppErrors(w http.ResponseWriter, r *http.Request) {
	p, acct, ok := s.appPage(w, r, "errors")
	if !ok {
		return
	}
	q := r.URL.Query()
	p.ErrSince = q.Get("since")
	if p.ErrSince != "1h" && p.ErrSince != "168h" {
		p.ErrSince = "24h"
	}
	q.Set("since", p.ErrSince)
	f, level, bad := errorsFilterFromQuery(q, time.Now())
	if bad != "" {
		f, level, _ = errorsFilterFromQuery(map[string][]string{"since": {p.ErrSince}}, time.Now())
	}
	p.ErrSource, p.ErrLevel, p.ErrHost, p.ErrFP = f.Source, level, f.Host, f.FP
	if s.cfg.Releases != nil {
		ctx, cancel := context.WithTimeout(r.Context(), instanceStoreTimeout)
		defer cancel()
		win, err := s.readErrors(ctx, acct.Sqid, p.AppName, f, level)
		if err != nil {
			log.Printf("console: app errors %s/%s: %v", acct.Sqid, p.AppName, err)
			p.ErrsErr = "The errors could not be read just now — try again in a moment."
		}
		p.ErrWin = win
	}
	s.render(w, http.StatusOK, "app_errors", p)
}

// handleAppErrorLevel is the Settings tab's form: owner, CSRF, one key.
func (s *Server) handleAppErrorLevel(w http.ResponseWriter, r *http.Request) {
	addr, raw, acct, role, app, ok := s.accountApp(w, r)
	if !ok {
		return
	}
	if subtle.ConstantTimeCompare([]byte(r.FormValue("csrf")), []byte(raw)) != 1 {
		s.renderMessage(w, r, http.StatusForbidden, "Refused", "Cross-site request blocked.")
		return
	}
	if role != RoleOwner {
		http.NotFound(w, r)
		return
	}
	level := r.FormValue("level")
	back := "/a/" + url.PathEscape(acct.Sqid) + "/apps/" + url.PathEscape(app) + "/settings"
	if !telemetry.ValidLogLevel(level) {
		s.setFlash(w, "err", "Choose error or warn.")
		http.Redirect(w, r, back, http.StatusSeeOther)
		return
	}
	if err := telemetry.Update(r.Context(), s.cfg.Releases, telemetry.AppKey(acct.Sqid, app), func(st *telemetry.Settings) {
		st.Errors = &telemetry.Errors{LogLevel: level}
	}); err != nil {
		log.Printf("console: errors level %s/%s: %v", acct.Sqid, app, err)
		s.setFlash(w, "err", "The setting could not be saved — try again in a moment.")
		http.Redirect(w, r, back, http.StatusSeeOther)
		return
	}
	s.audit(acct, app, addr, "errors-level", "set the Errors log level to "+level)
	s.setFlash(w, "ok", "Errors now include "+level+" lines and above for this app.")
	http.Redirect(w, r, back, http.StatusSeeOther)
}

// handleAccountErrorLevel is the account settings form: the default every
// app without its own setting inherits.
func (s *Server) handleAccountErrorLevel(w http.ResponseWriter, r *http.Request) {
	addr, raw, acct, role, ok := s.account(w, r)
	if !ok {
		return
	}
	if subtle.ConstantTimeCompare([]byte(r.FormValue("csrf")), []byte(raw)) != 1 {
		s.renderMessage(w, r, http.StatusForbidden, "Refused", "Cross-site request blocked.")
		return
	}
	if role != RoleOwner {
		http.NotFound(w, r)
		return
	}
	level := r.FormValue("level")
	back := "/a/" + url.PathEscape(acct.Sqid) + "/settings"
	if !telemetry.ValidLogLevel(level) {
		s.setFlash(w, "err", "Choose error or warn.")
		http.Redirect(w, r, back, http.StatusSeeOther)
		return
	}
	if err := telemetry.Update(r.Context(), s.cfg.Releases, telemetry.AccountKey(acct.Sqid), func(st *telemetry.Settings) {
		st.Errors = &telemetry.Errors{LogLevel: level}
	}); err != nil {
		log.Printf("console: account errors level %s: %v", acct.Sqid, err)
		s.setFlash(w, "err", "The setting could not be saved — try again in a moment.")
		http.Redirect(w, r, back, http.StatusSeeOther)
		return
	}
	s.audit(acct, "", addr, "errors-level", "set the account's Errors log level to "+level)
	s.setFlash(w, "ok", "Errors now include "+level+" lines and above for every app without its own setting.")
	http.Redirect(w, r, back, http.StatusSeeOther)
}

Confirm s.account's return arity by reading handleAccountSettings (addr, raw, acct, role, ok). Add p.ErrLevelSetting telemetry.Resolved to the page struct and populate it in handleAppSettings (p.ErrLevelSetting, _ = telemetry.Resolve(ctx, s.cfg.Releases, acct.Sqid, app)) and in handleAccountSettings (Resolve(ctx, s.cfg.Releases, acct.Sqid, "") — with an empty app name AppKey reads a key that does not exist, which returns defaults; that is acceptable for the account page and should be noted in a comment).

appnav.html: insert after the Logs link:

  <a href="{{.Nav}}/apps/{{.AppName}}/errors"{{if eq .ActiveTab "errors"}} aria-current="page"{{end}}>Errors</a>

app_errors.html:

{{define "app_errors"}}{{template "top" .}}
{{if .Flash}}<p class="banner">{{.Flash}}</p>{{end}}
{{if .FlashErr}}<p class="banner error">{{.FlashErr}}</p>{{end}}
{{template "appnav" .}}

<h2>Errors</h2>
{{if .ErrsErr}}<p class="banner error">{{.ErrsErr}}</p>{{end}}
<form method="get" class="small">
  <label>from the last
    <select name="since">
      <option value="1h"{{if eq .ErrSince "1h"}} selected{{end}}>hour</option>
      <option value="24h"{{if eq .ErrSince "24h"}} selected{{end}}>day</option>
      <option value="168h"{{if eq .ErrSince "168h"}} selected{{end}}>week</option>
    </select>
  </label>
  <label>source
    <select name="source">
      <option value=""{{if eq .ErrSource ""}} selected{{end}}>all</option>
      <option value="log"{{if eq .ErrSource "log"}} selected{{end}}>log</option>
      <option value="edge"{{if eq .ErrSource "edge"}} selected{{end}}>edge</option>
      <option value="server"{{if eq .ErrSource "server"}} selected{{end}}>server</option>
      <option value="browser"{{if eq .ErrSource "browser"}} selected{{end}}>browser</option>
    </select>
  </label>
  <label>log level
    <select name="level">
      <option value=""{{if eq .ErrLevel ""}} selected{{end}}>setting ({{.ErrWin.LogLevel}})</option>
      <option value="error"{{if eq .ErrLevel "error"}} selected{{end}}>error</option>
      <option value="warn"{{if eq .ErrLevel "warn"}} selected{{end}}>warn and above</option>
    </select>
  </label>
  <label>host <input name="host" value="{{.ErrHost}}" size="24"></label>
  <button type="submit">Update</button>
</form>
<p class="small">Sources live: {{if .ErrWin.Sources}}{{range $i, $s := .ErrWin.Sources}}{{if $i}}, {{end}}{{$s}}{{end}}{{else}}none yet{{end}}.
Counts are to the nearest flush. Messages are redacted for addresses and credentials before they are stored; treat that as a floor, not a guarantee.</p>

{{if .ErrFP}}
{{with .ErrWin.Records}}
<h3>Occurrences of <code>{{$.ErrFP}}</code></h3>
<p class="small"><a href="?since={{$.ErrSince}}">All groups</a></p>
{{range .}}
<div class="tablewrap"><table>
  <tr><th>Last</th><td>{{.Last.UTC.Format "2006-01-02 15:04:05"}}</td></tr>
  <tr><th>Count</th><td>{{.Count}}</td></tr>
  <tr><th>Source</th><td>{{.Source}}{{if .Kind}} / {{.Kind}}{{end}}</td></tr>
  <tr><th>Host</th><td>{{.Host}}</td></tr>
  {{if .Release}}<tr><th>Release</th><td>{{.Release}}</td></tr>{{end}}
  {{if .Path}}<tr><th>Path</th><td><code>{{.Path}}</code>{{if .Status}} → {{.Status}}{{end}}</td></tr>{{end}}
  <tr><th>Message</th><td>{{.Name}}</td></tr>
  {{if .Stack}}<tr><th>Stack</th><td><pre>{{.Stack}}</pre></td></tr>{{end}}
  {{if .Fields}}<tr><th>Fields</th><td>{{range $k, $v := .Fields}}<code>{{$k}}={{$v}}</code> {{end}}</td></tr>{{end}}
  <tr><th>Logs</th><td><a href="{{$.Nav}}/apps/{{$.AppName}}/logs?since=1h&host={{.Host}}">around this time</a></td></tr>
</table></div>
{{end}}
{{else}}<p>No occurrences of this group in the window.</p>{{end}}
{{else if .ErrWin.Groups}}
{{if .ErrWin.Truncated}}<p class="small">Showing the most recent 50,000 records of a larger window — counts are within what was read.</p>{{end}}
<div class="tablewrap">
<table>
  <tr><th>Last seen</th><th>Count</th><th>Source</th><th>Error</th><th>Hosts</th><th>Releases</th></tr>
  {{range .ErrWin.Groups}}<tr>
    <td>{{.Last.UTC.Format "2006-01-02 15:04"}}</td>
    <td>{{.Count}}{{if eq .FP "_overflow"}} <span class="small" title="more kinds than the hourly cap; occurrences">(overflow)</span>{{end}}</td>
    <td>{{.Source}}{{if .Level}} <span class="small">{{.Level}}</span>{{end}}</td>
    <td><a href="?since={{$.ErrSince}}&fp={{.FP}}{{if $.ErrLevel}}&level={{$.ErrLevel}}{{end}}">{{.Name}}</a></td>
    <td>{{range $i, $h := .Hosts}}{{if $i}}, {{end}}{{$h}}{{end}}</td>
    <td>{{range $i, $r := .Releases}}{{if $i}}, {{end}}<code>{{$r}}</code>{{end}}</td>
  </tr>{{end}}
</table>
</div>
<p class="small">Times are UTC. The same view from a terminal: <code>carlos errors --app {{.AppName}} --since {{.ErrSince}}</code>.</p>
{{else if not .ErrsErr}}
<p>No errors in this window.</p>
<p class="small">Log lines at <code>{{.ErrWin.LogLevel}}</code> and above and responses of 500 or higher appear here on their own; a JavaScript or server-side error needs the hook (coming in a later step). The level is set on the Settings tab. The same view from a terminal: <code>carlos errors --app {{.AppName}}</code>.</p>
{{end}}
{{template "bottom" .}}{{end}}

app_settings.html: before the Feature flags section add:

<h2>Errors</h2>
<form method="post" action="{{.Base}}/apps/{{.Detail.App}}/errors/level" class="settings-row">
  <input type="hidden" name="csrf" value="{{.CSRF}}">
  <label>Log lines counted as errors:
    <select name="level">
      <option value="error"{{if eq .ErrLevelSetting.LogLevel "error"}} selected{{end}}>error and above</option>
      <option value="warn"{{if eq .ErrLevelSetting.LogLevel "warn"}} selected{{end}}>warn and above</option>
    </select>
  </label>
  <button type="submit">Save</button>
  <span class="small">currently {{.ErrLevelSetting.LogLevel}}, from the {{.ErrLevelSetting.LogLevelFrom}}</span>
</form>
<p class="small">The Errors tab always lets you widen one look to warn without changing this.</p>

Check the CSRF field name and value the rename form uses ({{.CSRF}}, {{.Raw}} or similar) and match it exactly.

account_settings.html: before "Delete this account" add the same form posting to {{.Base}}/errors/level, with the copy "for every app without its own setting".

  • [ ] Step 6: Run console tests

Run: go build ./... && go vet ./internal/console/ && go test ./internal/console/ -count=1 Expected: PASS (the whole package; template parse errors surface at server construction in every test).

  • [ ] Step 7: Commit
git add internal/console
git commit -m "console: Errors tab, errors API, and the log-level setting at both levels (spec §3.6)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>"

---

Task 10: carlos errors and the docs heading

Files:

  • Create: errors_cmd.go
  • Modify: apimode.go (beside logsGet), main.go:80 (dispatch) and the help text near line 204
  • Modify: docs/site/cli.md after ### carlos logs
  • Test: errors_cmd_test.go (format only) and the docsite gate

Interfaces:

  • Consumes: requireAPISession, resolvedAccount, resolvedApp, appFlag, flagUsage, apiPath, getJSON, putJSON (check its name; apimode.go has a JSON PUT/POST helper used by features set), colorFor, mutedf, stdout.
  • Produces: runErrors(args []string); (*apiSession).errorsGet(ctx, a apiAcct, app string, q url.Values) (apiErrorsResponse, error); (*apiSession).telemetryPut(ctx, a apiAcct, app string, body any) error; printErrorGroups(w io.Writer, groups []apperr.Group); printErrorRecords(w io.Writer, recs []apperr.Record).
  • [ ] Step 1: Failing format test — errors_cmd_test.go
package main

import (
	"bytes"
	"strings"
	"testing"
	"time"

	"github.com/carlosframework/platform/internal/apperr"
)

func TestPrintErrorGroups(t *testing.T) {
	now := time.Date(2026, 9, 7, 10, 0, 0, 0, time.UTC)
	var buf bytes.Buffer
	printErrorGroups(&buf, []apperr.Group{
		{FP: "abc123", Source: "edge", Name: "500 POST /checkout", Count: 12, Last: now, Hosts: []string{"a.example"}, Releases: []string{"v3"}},
		{FP: "_overflow", Source: "edge", Name: "500 GET /z", Count: 40, Last: now},
	})
	out := buf.String()
	for _, want := range []string{"2026-09-07 10:00", "12", "edge", "500 POST /checkout", "abc123", "a.example", "v3", "overflow"} {
		if !strings.Contains(out, want) {
			t.Errorf("output missing %q:\n%s", want, out)
		}
	}
}
  • [ ] Step 2: Implement errors_cmd.go
// `carlos errors` — an app's error groups with no box access (spec
// 2026-09-07-errors-and-analytics-design.md §3.6): log lines at the
// configured level and edge-observed 5xx, coalesced by fingerprint, read
// through the console API. API mode only, like `carlos logs`.
package main

import (
	"context"
	"flag"
	"fmt"
	"io"
	"log"
	"net/url"
	"sort"
	"strconv"

	"github.com/carlosframework/platform/internal/apperr"
)

const errorsUsage = `usage: carlos errors --app <a> [flags]

Lists error groups, newest first: one row per fingerprint with its count in
the window, the source it came from (log, edge, server, browser), the hosts
and releases it was seen on. --fp <id> expands one group to its occurrences.
--set-level error|warn changes which log lines count for this app.
`

func runErrors(args []string) {
	fs := flag.NewFlagSet("errors", flag.ExitOnError)
	app := appFlag(fs, "app whose errors to read")
	since := fs.String("since", "24h", "window start: duration back from now, or RFC3339")
	until := fs.String("until", "", "window end (default: now)")
	source := fs.String("source", "", "only one source: log | edge | server | browser")
	level := fs.String("level", "", "log lines at this level and above for this read: error | warn (default: the app's setting)")
	host := fs.String("host", "", "only one instance (route host)")
	rel := fs.String("release", "", "only one release")
	fp := fs.String("fp", "", "expand one group: its occurrences, oldest first")
	setLevel := fs.String("set-level", "", "set the app's log level for errors: error | warn")
	consoleFlag := fs.String("console", "", "console to read through (default: the only logged-in one)")
	acctRaw := fs.String("account", "", "account (sqid or name) the app belongs to (default: your only account, or the configured/chosen one)")
	acctIDRaw := fs.String("account-id", "", "account sqid, matched as a sqid ONLY — supersedes --account")
	fs.Usage = flagUsage(fs, errorsUsage)
	fs.Parse(args)

	sess := requireAPISession("errors", *consoleFlag)
	acct := resolvedAccount(sess, acctRaw, acctIDRaw)
	app2 := resolvedApp(sess, acct, *app, errorsUsage)
	ctx := context.Background()

	if *setLevel != "" {
		if *setLevel != "error" && *setLevel != "warn" {
			log.Fatalf("errors: --set-level wants error or warn")
		}
		if err := sess.telemetryPut(ctx, acct, app2, map[string]any{"errors": map[string]string{"log_level": *setLevel}}); err != nil {
			log.Fatalf("errors: %v", err)
		}
		fmt.Fprintf(stdout, "Errors for %s now include %s lines and above.\n", app2, *setLevel)
		return
	}

	q := url.Values{}
	for k, v := range map[string]string{"since": *since, "until": *until, "source": *source, "level": *level, "host": *host, "release": *rel, "fp": *fp} {
		if v != "" {
			q.Set(k, v)
		}
	}
	res, err := sess.errorsGet(ctx, acct, app2, q)
	if err != nil {
		log.Fatalf("errors: %v", err)
	}
	if *fp != "" {
		printErrorRecords(stdout, res.Records)
		if len(res.Records) == 0 {
			mutedf(stdout, "no occurrences of %s in this window", *fp)
		}
		return
	}
	printErrorGroups(stdout, res.Groups)
	if len(res.Groups) == 0 {
		mutedf(stdout, "no errors in this window (log level %s; sources live: %s) — try --since 168h or --level warn", res.LogLevel, joinOr(res.Sources, "none yet"))
	} else {
		mutedf(stdout, "log level %s; sources live: %s; counts are to the nearest flush", res.LogLevel, joinOr(res.Sources, "none"))
	}
	if res.Truncated {
		mutedf(stdout, "(the window held more than could be read; counts are within what was read — narrow it)")
	}
}

func joinOr(xs []string, empty string) string {
	if len(xs) == 0 {
		return empty
	}
	out := xs[0]
	for _, x := range xs[1:] {
		out += ", " + x
	}
	return out
}

// printErrorGroups renders one row per group: last seen, count, source,
// message, then the fingerprint, hosts and releases dimmed.
func printErrorGroups(w io.Writer, groups []apperr.Group) {
	c := colorFor(w)
	for _, g := range groups {
		count := strconv.Itoa(g.Count)
		if g.FP == apperr.OverflowFP {
			count += " (overflow)"
		}
		line := c.muted(g.Last.UTC().Format("2006-01-02 15:04")) + "  " + c.head(fmt.Sprintf("%6s", count)) + "  " +
			c.muted(fmt.Sprintf("%-7s", g.Source)) + "  " + c.bad(g.Name)
		line += "\n" + "                          " + c.muted("fp="+g.FP)
		if len(g.Hosts) > 0 {
			line += c.muted(" hosts=" + joinOr(g.Hosts, ""))
		}
		if len(g.Releases) > 0 {
			line += c.muted(" releases=" + joinOr(g.Releases, ""))
		}
		fmt.Fprintln(w, line)
	}
}

// printErrorRecords renders one group's occurrences, oldest first.
func printErrorRecords(w io.Writer, recs []apperr.Record) {
	c := colorFor(w)
	for _, r := range recs {
		fmt.Fprintf(w, "%s  %s  %s  %s\n", c.muted(r.Last.UTC().Format("2006-01-02 15:04:05")),
			c.head(fmt.Sprintf("x%d", r.Count)), c.muted(r.Host), c.bad(r.Name))
		if r.Path != "" {
			fmt.Fprintf(w, "    %s\n", c.muted(fmt.Sprintf("%s → %d", r.Path, r.Status)))
		}
		if r.Release != "" {
			fmt.Fprintf(w, "    %s\n", c.muted("release "+r.Release))
		}
		if r.Stack != "" {
			fmt.Fprintf(w, "    %s\n", r.Stack)
		}
		keys := make([]string, 0, len(r.Fields))
		for k := range r.Fields {
			keys = append(keys, k)
		}
		sort.Strings(keys)
		for _, k := range keys {
			fmt.Fprintf(w, "    %s\n", c.muted(k+"="+r.Fields[k]))
		}
	}
}

In apimode.go, beside logsGet:

type apiErrorsResponse struct {
	Groups    []apperr.Group  `json:"groups"`
	Records   []apperr.Record `json:"records"`
	Truncated bool            `json:"truncated"`
	LogLevel  string          `json:"log_level"`
	Sources   []string        `json:"sources"`
}

func (s *apiSession) errorsGet(ctx context.Context, a apiAcct, app string, query url.Values) (apiErrorsResponse, error) {
	path := apiPath(a, app, "/errors")
	if enc := query.Encode(); enc != "" {
		if strings.Contains(path, "?") {
			path += "&" + enc
		} else {
			path += "?" + enc
		}
	}
	var res apiErrorsResponse
	status, body, err := s.getJSON(ctx, path)
	if err != nil {
		return res, err
	}
	if status != http.StatusOK {
		return res, apiErrorFromBody(status, body)
	}
	if err := json.Unmarshal(body, &res); err != nil {
		return res, fmt.Errorf("errors: unreadable response: %w", err)
	}
	return res, nil
}

func (s *apiSession) telemetryPut(ctx context.Context, a apiAcct, app string, body any) error {
	status, resp, err := s.putJSON(ctx, apiPath(a, app, "/telemetry"), body)
	if err != nil {
		return err
	}
	if status != http.StatusOK {
		return apiErrorFromBody(status, resp)
	}
	return nil
}

Find the existing JSON-body request helper in apimode.go (grep func (s \*apiSession) .*JSON — postJSON/doJSON/sendJSON) and use its real name and signature.

main.go: add case "errors": runErrors(args) beside logs, and in the help block near line 204 a line: errors error groups for an app: log lines at the set level plus 5xx the edge saw.

docs/site/cli.md after the ### carlos logs section:

### carlos errors

One row per distinct error, newest first, with how many times it happened
in the window and where. Two sources need no wiring at all: log lines the
app tagged `error` (or `warn`, if you set the level) and responses of 500
or above that the edge saw. Server-side and browser hooks come later.

```sh
carlos errors --app hello --since 24h
carlos errors --app hello --fp 3f2a9c1b5e7d0a4c
carlos errors --app hello --set-level warn

--fp expands one group to its occurrences. --level warn widens a single read without changing the setting; --set-level changes it for the app. The same view is the Errors tab on the app's dashboard.


- [ ] **Step 3: Run the CLI test, docsite gate, whole build**

Run: `go build ./... && go test . -run TestPrintErrorGroups -count=1 && go test ./internal/docsite/ -count=1`
Expected: PASS; the docsite gate passes with the new heading (and would fail without it — try removing the heading once to see the failure, then restore it).

- [ ] **Step 4: Commit**

```bash
git add errors_cmd.go errors_cmd_test.go apimode.go main.go docs/site/cli.md
git commit -m "carlos errors: the app's error groups from a terminal, and the log-level setting

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>"

---

Task 11: Whole-tree gate, cross-compile, push

Files: none new.

  • [ ] Step 1: Full gate

Run:

go build ./... && go vet ./... && gofmt -l . && go test ./... -count=1
GOOS=darwin GOARCH=arm64 go build ./... && GOOS=windows GOARCH=amd64 go build ./...

Expected: all clean; gofmt -l . prints nothing.

  • [ ] Step 2: Spec status line

In docs/superpowers/specs/2026-09-07-errors-and-analytics-design.md, change the **Status:** line to approved (Paul, 2026-09-07: "go ahead with that and plan, build review cycle overnight"); step 1 implemented on branch spec-errors-analytics, plan docs/superpowers/plans/2026-09-07-errors-step1-log-and-edge.md. Commit with the plan file:

git add docs/superpowers
git commit -m "spec: approved; step 1 plan

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>"
  • [ ] Step 3: Push and describe
git push origin spec-errors-analytics
amadan branch describe carlos/platform spec-errors-analytics -body - < <description file>
amadan ci status carlos/platform -branch spec-errors-analytics

Do not merge. The branch waits for Paul's review of the diff.

Self-review against the spec

  • §3.1 shape: Task 3 Record carries every field but ua/referer (browser/edge analytics fields; step 2/3). Sanitisation: Task 3/4. ✔
  • §3.2 log source read-derived: Task 9 readErrors. Edge errors with activator suppression: Task 7. Fingerprint rules: Task 3. Coalescing cap per app-hour on (fp, host, release) with _overflow: Task 4. Storage family and consumer list (ChunkKeySize, HourNameOfDir, chunkPrune, aclWriteAppFamily): Tasks 1, 2, 6. ✔
  • §3.3 observing writer, unconditional, status routes excluded, 1xx ignored, implicit 200, Content-Type latched: Task 7. Settings object and resolution: Task 8. ✔
  • §3.6 tab (window 1h/24h/7d, source/level/host filters, expand, coverage line, "nearest flush", collect-cap disclosure), API paths and roles, CLI flags including -level, settings at both levels, retention via chunkPrune: Tasks 6, 9, 10. Tombstone DELETE …/errors/{fp} is deferred to a follow-up branch and is called out in the branch description. -release filter on the CLI: Task 10 (--release). ✔
  • §4 hibernation: nothing here calls the activator; Task 7's flag path is the only touch and it is a context value. ✔
  • §6 tests: sanitisation, fingerprint, coalescing, log-derived rows, edge errors on the plain path, chunk family byte-identity, store door, observing writer, settings resolution, console gates (page 404 for non-member is inherited from appPage; add one get as a non-member in TestAppErrorsPage if the logs page test has an equivalent — mirror it), docsite. ✔
  • Type consistency: apperr.Filter fields (Since, Until, Host, Source, Release, FP) are used identically in Tasks 4 and 9; errorsResponse fields match apiErrorsResponse; chunkPrune(ctx, s, fam, box, k, now) is called with that order in Tasks 6 and its tests. ✔