carlos / platform Public
Errors tab, step 1: log and edge sources — Implementation Plan
> For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.
Goal: Every app in the fleet gets a populated Errors tab, API and CLI with zero integration: error-level log lines and edge-observed 5xx responses, grouped by fingerprint and coalesced on the box.
Architecture: A new leaf package internal/apperr owns the error record, sanitisation, fingerprinting, the coalescing box sink, grouping, and the bucket read. It stores chunks in a sibling family of logs/ (apps/<sqid>/<app>/errors/<box>/<hour>/…jsonl.gz) through family-parameterised helpers added to internal/applog. The edge's countingWriter learns to observe status and emits an edge error for upstream 5xx; the agent ships the sink like it ships logs. The console reads two families (errors/ plus logs/ filtered to a level from a new internal/telemetry settings object) and serves a tab, a cliRoute API and carlos errors.
Tech Stack: Go 1.22+ (net/http patterns, generics), html/template, sqlite registry, release.Store bucket, no JS framework.
Spec: docs/superpowers/specs/2026-09-07-errors-and-analytics-design.md — §3.1 (shape, sanitisation), §3.2 (log source read-derived, edge errors, fingerprint, coalescing, storage, store door), §3.3 (observing writer, settings object), §3.6 (tab, API, CLI, settings, retention), §4, §6, §7 step 1, §8 rulings.
Global Constraints
- Wire shapes are additive:
logs/chunks must be byte-identical after the refactor; existinginternal/applogtests pass unchanged. - Edge wrapper order is load-bearing: recorder above compressor, counter below; nothing moves, one wrapper learns to look (
internal/edge/edge.go:1241). - Nothing here may enter
activator.Service.ServeHTTPor stamplastActive. - Store door write grant becomes a family allow-list:
logs,errors,analytics(spec §3.2), each segment-exact. - Coalescing cap: 1,000 records per app per UTC hour counting
(fp, host, release); overflow folds into one_overflowrow. - Fingerprint:
sha256(len-delimited(source, kind, norm(name), norm(top frame)))[:16]hex,fpv: 1; 1–3 digit tokens survive normalisation, 4+ digits, hex 8+, UUIDs and quoted strings become placeholders. - Sanitisation before buffering on every source: query and fragment off paths; emails →
<email>;Bearer/Basiccredentials andtoken=/key=/secret=/password=values →<redacted>; hex or base64 runs of 32+ →<hex>. - Defaults (ruling):
errors.log_level=error. Settings object:accounts/<sqid>/telemetry.jsonandapps/<sqid>/<app>/telemetry.json, resolved app → account → default. - API:
GET /api/cli/apps/{account}/{app}/errorswithsvcrole.Allow(svcrole.Instance, svcrole.Operate); pages go throughappPage(session → CSRF → owner → fresh auth). - New
carlos errorsneeds a### carlos errorsheading indocs/site/cli.mdorinternal/docsitefails the gate. - Go commands:
GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off; sandbox may need disabling on "read-only file system". Before push:GOOS=darwin GOARCH=arm64 go build ./...andGOOS=windows GOARCH=amd64 go build ./.... - Every commit:
go build ./... && go vet ./... && gofmt -l .clean for the packages touched; fullgo test ./... -count=1before the final push. - Commit trailer on every commit:
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>. - Branch:
spec-errors-analytics(already exists, carries the spec). Push toorigin(the amadan forge). Nevergh.
File structure
| File | Responsibility | |---|---| | internal/applog/chunk.go | + Family, FamilyPrefix/BoxPrefix/HourPrefix/ChunkKey, generic EncodeJSONL/DecodeJSONL; existing funcs become thin wrappers | | internal/applog/read.go | + generic ReadChunks[T]; ReadWindow re-based on it | | internal/edge/storedoor.go | aclWriteAppLogs → aclWriteAppFamily allow-list | | internal/apperr/record.go | Record, source/kind constants, FromLog | | internal/apperr/sanitize.go | Sanitize, SanitizePath | | internal/apperr/fingerprint.go | Normalize, TopFrame, Fingerprint, FPVersion | | internal/apperr/sink.go | coalescing Sink | | internal/apperr/read.go | Filter, ReadWindow, chunk key helpers | | internal/apperr/group.go | Group, GroupRecords | | internal/activator/answered.go | WithAnswerFlag, MarkAnswered | | internal/activator/activator.go | mark the three wake-path 503s | | internal/edge/usagemeter.go | countingWriter observes status and content type | | internal/edge/edge.go | ErrorSink, EnableErrors, unconditional writer, edge error emission | | internal/host/logship.go | logshipPrune → chunkPrune(fam, …) | | internal/host/errship.go | error sink ship loop | | internal/host/agent.go | construct apperr.Sink, wire edge + loop | | internal/telemetry/telemetry.go | settings object, Resolve, Update | | internal/console/errorsview.go | window read merging both families; page fold | | internal/console/apierrors.go | GET …/errors, PUT …/telemetry | | internal/console/server.go | routes, page fields, handleAppErrors, settings handlers | | internal/console/templates/app_errors.html, appnav.html, app_settings.html, account_settings.html | tab, nav link, settings forms | | errors_cmd.go, apimode.go, main.go | carlos errors | | docs/site/cli.md | ### carlos errors |
---
Task 1: applog chunk families and a generic codec
Files:
- Modify:
internal/applog/chunk.go - Modify:
internal/applog/read.go - Test:
internal/applog/chunk_test.go(append),internal/applog/read_test.go(append)
Interfaces:
- Produces:
type Family string;FamilyLogs,FamilyErrors,FamilyAnalytics;FamilyPrefix(fam Family, acct, app string) string;FamilyBoxPrefix(fam, acct, app, box);FamilyHourPrefix(fam, acct, app, box, hour);FamilyChunkKey(fam, acct, app, box, hour string, seq uint64, size int) string;EncodeJSONL[T any](recs []T) ([]byte, error);DecodeJSONL[T any](b []byte) ([]T, error);ReadChunks[T any](ctx, s release.Store, prefix string, since, until time.Time, keep func(T) bool, collectCap int) (perBox [][]T, trimmed bool, err error);HourNameOfDir(dir string) string(exported rename ofhourNameOfDir). - Existing
Prefix,BoxPrefix,HourPrefix,ChunkKey,EncodeChunk,DecodeChunk,ReadWindowkeep their signatures and behaviour.
- [ ] Step 1: Write the failing tests
Append to internal/applog/chunk_test.go:
func TestFamilyKeysMirrorLogs(t *testing.T) {
if got, want := FamilyPrefix(FamilyLogs, "bab", "web"), Prefix("bab", "web"); got != want {
t.Fatalf("FamilyPrefix(logs) = %q, Prefix = %q", got, want)
}
if got, want := FamilyChunkKey(FamilyLogs, "bab", "web", "box1", "2026-09-07-10", 42, 7), ChunkKey("bab", "web", "box1", "2026-09-07-10", 42, 7); got != want {
t.Fatalf("FamilyChunkKey(logs) = %q, ChunkKey = %q", got, want)
}
got := FamilyChunkKey(FamilyErrors, "bab", "web", "box1", "2026-09-07-10", 42, 7)
want := "apps/bab/web/errors/box1/2026-09-07-10/0000000000000042-7.jsonl.gz"
if got != want {
t.Fatalf("errors key = %q, want %q", got, want)
}
if n, ok := ChunkKeySize(got); !ok || n != 7 {
t.Fatalf("ChunkKeySize on an errors key = %d,%v", n, ok)
}
}
func TestEncodeJSONLRoundTripsAnyShape(t *testing.T) {
type rec struct {
A string `json:"a"`
N int `json:"n,omitempty"`
}
b, err := EncodeJSONL([]rec{{A: "x", N: 1}, {A: "y"}})
if err != nil {
t.Fatal(err)
}
out, err := DecodeJSONL[rec](b)
if err != nil {
t.Fatal(err)
}
if len(out) != 2 || out[0].A != "x" || out[0].N != 1 || out[1].A != "y" || out[1].N != 0 {
t.Fatalf("round trip = %+v", out)
}
// EncodeChunk must be byte-identical to EncodeJSONL over the same
// records: the logs family's wire shape does not move.
recs := []Record{{TS: time.Date(2026, 9, 7, 10, 0, 0, 0, time.UTC), Msg: "hi", Seq: 1}}
a, _ := EncodeChunk(recs)
c, _ := EncodeJSONL(recs)
da, _ := DecodeChunk(a)
dc, _ := DecodeJSONL[Record](c)
if len(da) != 1 || len(dc) != 1 || da[0].Msg != dc[0].Msg || !da[0].TS.Equal(dc[0].TS) {
t.Fatalf("EncodeChunk and EncodeJSONL disagree: %+v vs %+v", da, dc)
}
}Append to internal/applog/read_test.go (use the same store construction the file already uses — release.NewDirStore(t.TempDir()) — and its existing helper for writing a chunk if one exists; otherwise s.Put(ctx, ChunkKey(...), body, "")):
func TestReadChunksWalksAnyFamily(t *testing.T) {
ctx := context.Background()
s := release.NewDirStore(t.TempDir())
type ev struct {
TS time.Time `json:"ts"`
Name string `json:"name"`
}
now := time.Date(2026, 9, 7, 10, 30, 0, 0, time.UTC)
body, _ := EncodeJSONL([]ev{{TS: now, Name: "boom"}, {TS: now.Add(time.Minute), Name: "skip"}})
key := FamilyChunkKey(FamilyErrors, "bab", "web", "box1", HourOf(now), 1, len(body))
if _, err := s.Put(ctx, key, body, ""); err != nil {
t.Fatal(err)
}
perBox, trimmed, err := ReadChunks[ev](ctx, s, FamilyPrefix(FamilyErrors, "bab", "web"),
now.Add(-time.Hour), now.Add(time.Hour), func(e ev) bool { return e.Name != "skip" }, 100)
if err != nil || trimmed {
t.Fatalf("ReadChunks: %v trimmed=%v", err, trimmed)
}
if len(perBox) != 1 || len(perBox[0]) != 1 || perBox[0][0].Name != "boom" {
t.Fatalf("perBox = %+v", perBox)
}
// An hour outside the window is never listed.
perBox, _, err = ReadChunks[ev](ctx, s, FamilyPrefix(FamilyErrors, "bab", "web"),
now.Add(2*time.Hour), time.Time{}, func(ev) bool { return true }, 100)
if err != nil || len(perBox) != 1 || len(perBox[0]) != 0 {
t.Fatalf("out-of-window read = %+v, %v", perBox, err)
}
}- [ ] Step 2: Run tests to verify they fail
Run: go test ./internal/applog/ -run 'TestFamilyKeys|TestEncodeJSONL|TestReadChunks' -count=1 Expected: FAIL to compile (undefined: FamilyPrefix etc.).
- [ ] Step 3: Implement families and the generic codec in
chunk.go
Replace the Prefix/BoxPrefix/HourPrefix/ChunkKey block and the codec with:
// Family names one chunk family under an app — the path segment after
// apps/<sqid>/<app>/. logs/ is the original; errors/ and analytics/ are
// the siblings the errors-and-analytics spec adds (§3.2). The key shape
// below the family segment is identical for every family, which is what
// lets one prune loop, one store-door grant and one reader serve all three.
type Family string
const (
FamilyLogs Family = "logs"
FamilyErrors Family = "errors"
FamilyAnalytics Family = "analytics"
)
// FamilyPrefix is the app's whole subtree for one family — under
// release.AppPrefix on purpose (spec §3): it is the app's data, so the
// owner gate maps onto the prefix and app trash/purge sweeps it for free.
func FamilyPrefix(fam Family, acct, app string) string {
return release.AppPrefix(acct, app) + string(fam) + "/"
}
// FamilyBoxPrefix is one box's subtree within a family — the unit of
// write-side ownership: each box prunes only what it wrote.
func FamilyBoxPrefix(fam Family, acct, app, box string) string {
return FamilyPrefix(fam, acct, app) + box + "/"
}
// FamilyHourPrefix is one UTC hour within a box's subtree.
func FamilyHourPrefix(fam Family, acct, app, box, hour string) string {
return FamilyBoxPrefix(fam, acct, app, box) + hour + "/"
}
// FamilyChunkKey names one flushed chunk in any family. See ChunkKey for
// why seq and size live in the key.
func FamilyChunkKey(fam Family, acct, app, box, hour string, seq uint64, size int) string {
return fmt.Sprintf("%s%016d-%d.jsonl.gz", FamilyHourPrefix(fam, acct, app, box, hour), seq, size)
}
// Prefix is the app's whole log subtree: FamilyPrefix for logs.
func Prefix(acct, app string) string { return FamilyPrefix(FamilyLogs, acct, app) }
// BoxPrefix is one box's subtree within an app's logs.
func BoxPrefix(acct, app, box string) string { return FamilyBoxPrefix(FamilyLogs, acct, app, box) }
// HourPrefix is one UTC hour within a box's log subtree.
func HourPrefix(acct, app, box, hour string) string {
return FamilyHourPrefix(FamilyLogs, acct, app, box, hour)
}
// ChunkKey names one flushed log chunk: seq is the first record's per-box
// sequence number, zero-padded (16 digits — seq is milliseconds-based, see
// NewSink) so lexical listing order is numeric order, and size is the
// object's own byte length. Size lives IN the key on purpose:
// release.Store's List returns keys only, and retention's byte cap needs
// sizes — embedding them costs a suffix, extending the Store interface
// costs every implementation. Chunks are written once and never appended
// or rewritten (spec §3).
func ChunkKey(acct, app, box, hour string, seq uint64, size int) string {
return FamilyChunkKey(FamilyLogs, acct, app, box, hour, seq, size)
}Keep ChunkKeySize as is. Replace EncodeChunk/DecodeChunk with:
// EncodeJSONL renders any record shape as gzipped JSONL, one per line —
// the chunk codec every family shares.
func EncodeJSONL[T any](recs []T) ([]byte, error) {
var buf bytes.Buffer
zw := gzip.NewWriter(&buf)
enc := json.NewEncoder(zw)
for _, r := range recs {
if err := enc.Encode(r); err != nil {
return nil, err
}
}
if err := zw.Close(); err != nil {
return nil, err
}
return buf.Bytes(), nil
}
// DecodeJSONL parses a chunk, skipping malformed lines rather than failing
// the whole object — one corrupt line must not cost the member the rest of
// the hour. Corrupt gzip is an error: that is a broken object, not a line.
func DecodeJSONL[T any](b []byte) ([]T, error) {
zr, err := gzip.NewReader(bytes.NewReader(b))
if err != nil {
return nil, err
}
defer zr.Close()
var out []T
sc := bufio.NewScanner(zr)
sc.Buffer(nil, 1<<20) // lines are capped well below this at the tee
for sc.Scan() {
var r T
if json.Unmarshal(sc.Bytes(), &r) != nil {
continue
}
out = append(out, r)
}
if err := sc.Err(); err != nil && err != io.EOF {
return out, err
}
return out, nil
}
// EncodeChunk renders log records: EncodeJSONL over Record.
func EncodeChunk(recs []Record) ([]byte, error) { return EncodeJSONL(recs) }
// DecodeChunk parses a log chunk: DecodeJSONL over Record.
func DecodeChunk(b []byte) ([]Record, error) { return DecodeJSONL[Record](b) }- [ ] Step 4: Implement
ReadChunksinread.goand re-baseReadWindowon it
// ReadChunks walks every box's hour directories under prefix (a
// FamilyPrefix) whose hours intersect [since, until], decodes each chunk as
// T, keeps what passes keep, and returns one slice per box in listing
// order. Hour pruning happens at the PREFIX level — an hour outside the
// window is never listed, which is what makes a 1h read over 7d of history
// cheap (spec §3). collectCap bounds memory per box: past it the OLDEST
// records go and trimmed says so. Store errors fail the read — a partial
// answer that looks whole is the one thing this must never produce.
func ReadChunks[T any](ctx context.Context, s release.Store, prefix string, since, until time.Time, keep func(T) bool, collectCap int) ([][]T, bool, error) {
boxDirs, err := s.ListDirs(ctx, prefix)
if err != nil {
return nil, false, err
}
trimmed := false
perBox := make([][]T, 0, len(boxDirs))
for _, boxDir := range boxDirs {
hourDirs, err := s.ListDirs(ctx, boxDir)
if err != nil {
return nil, false, err
}
var recs []T
for _, hourDir := range hourDirs { // lexically sorted = chronological
if !hourIntersects(HourNameOfDir(hourDir), since, until) {
continue
}
keys, err := s.List(ctx, hourDir)
if err != nil {
return nil, false, err
}
for _, key := range keys { // lexically sorted = seq order
body, _, err := s.Get(ctx, key)
if err != nil {
return nil, false, err
}
chunk, err := DecodeJSONL[T](body)
if err != nil {
return nil, false, err
}
for _, r := range chunk {
if keep(r) {
recs = append(recs, r)
}
}
if over := len(recs) - collectCap; over > 0 {
recs = recs[over:]
trimmed = true
}
}
}
perBox = append(perBox, recs)
}
return perBox, trimmed, nil
}Then ReadWindow becomes:
func ReadWindow(ctx context.Context, s release.Store, acct, app string, f Filter, collectCap int) ([]Record, bool, error) {
perBox, trimmed, err := ReadChunks[Record](ctx, s, Prefix(acct, app), f.Since, f.Until, f.Match, collectCap)
if err != nil {
return nil, false, err
}
merged := MergeRecords(perBox...)
if over := len(merged) - collectCap; over > 0 {
merged = merged[over:]
trimmed = true
}
return merged, trimmed, nil
}Rename hourNameOfDir to HourNameOfDir (exported; the host package's hourNameOf duplicate goes away in Task 6) and update its one caller.
- [ ] Step 5: Run the whole package
Run: go test ./internal/applog/ -count=1 Expected: PASS, every pre-existing test included.
- [ ] Step 6: Commit
git add internal/applog
git commit -m "applog: chunk families and a generic JSONL codec, logs/ unchanged
Prefix/ChunkKey/EncodeChunk/ReadWindow keep their signatures and bytes;
FamilyPrefix, FamilyChunkKey, EncodeJSONL/DecodeJSONL and ReadChunks are
the seams the errors/ family (spec 2026-09-07 §3.2) builds on.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>"---
Task 2: Store door family allow-list
Files:
- Modify:
internal/edge/storedoor.go:503-518(aclWriteAppLogs) - Test:
internal/edge/storedoor_test.go(rows inTestStoreDoorACLMatrix)
Interfaces:
- Produces:
aclWriteAppFamily(sqid, key string) boolreplacingaclWriteAppLogs;aclWritecalls it.
- [ ] Step 1: Add matrix rows
In TestStoreDoorACLMatrix, after the existing logs rows (around line 138), add:
// --- sibling chunk families (errors-and-analytics spec §3.2) ---
{"apps/ab12/web/errors/box-a/2026090710/0000000000000001-42.jsonl.gz", opWrite, true, "own app's error chunk"},
{"apps/ab12/web/errors/box-a/2026090710/0000000000000001-42.jsonl.gz", opDelete, true, "errship prunes its own subtree hourly"},
{"apps/ab12/web/analytics/box-a/days/2026-09-07.json", opWrite, true, "own app's analytics day file"},
{"apps/zz99/web/errors/box-a/x.jsonl.gz", opWrite, false, "another account's error chunks"},
{"apps/ab12/errors/box-a/x.jsonl.gz", opWrite, false, "errors must sit under an app segment"},
{"apps/ab12/web/errors", opWrite, false, "the bare family segment is not a chunk"},
{"apps/ab12/web/errors/", opWrite, false, "nor with a slash"},
{"apps/ab12/web/other/box-a/x.jsonl.gz", opWrite, false, "the family list is exact: logs, errors, analytics"},
{"apps/ab12/web/channels/errors/stable.json", opWrite, false, "\"errors\" deeper in the key grants nothing"},
{"apps/ab12/web/errorsx/box-a/x.jsonl.gz", opWrite, false, "the segment is errors/, not a prefix of a longer name"},- [ ] Step 2: Run to verify the new rows fail
Run: go test ./internal/edge/ -run TestStoreDoorACLMatrix -count=1 Expected: FAIL on the errors and analytics write rows (refused today).
- [ ] Step 3: Implement
Replace aclWriteAppLogs with:
// chunkFamilies is the allow-list of app subtrees a box may write: the
// chunk and day-file families it produces about its own tenant's apps
// (applog.Family). Exact segment names — "logs" is not "logsx".
var chunkFamilies = map[string]bool{"logs": true, "errors": true, "analytics": true}
// aclWriteAppFamily implements aclWrite's second scope: exactly
// apps/<sqid>/<app>/<family>/<something>, with <app> a single non-empty
// segment, <family> one of chunkFamilies, and at least one key component
// below the family. Anything else — a shorter key, a second path shape,
// a family name as the app segment — is refused, so the grant can never be
// widened by a key that merely contains the word.
func aclWriteAppFamily(sqid, key string) bool {
if sqid == "" {
return false
}
rest, ok := strings.CutPrefix(key, "apps/"+sqid+"/")
if !ok {
return false
}
app, tail, ok := strings.Cut(rest, "/")
if !ok || app == "" {
return false
}
fam, below, ok := strings.Cut(tail, "/")
return ok && chunkFamilies[fam] && below != ""
}Update aclWrite to call aclWriteAppFamily, and update the doc comment above aclWrite (the "logs scope is SEGMENT-EXACT" paragraph) to say "family scope" and name the three families.
- [ ] Step 4: Run the matrix and the package
Run: go test ./internal/edge/ -run TestStoreDoor -count=1 Expected: PASS.
- [ ] Step 5: Commit
git add internal/edge/storedoor.go internal/edge/storedoor_test.go
git commit -m "store door: the app write grant is a family allow-list (logs, errors, analytics)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>"---
Task 3: internal/apperr record, sanitisation, fingerprint
Files:
- Create:
internal/apperr/record.go,internal/apperr/sanitize.go,internal/apperr/fingerprint.go - Test:
internal/apperr/sanitize_test.go,internal/apperr/fingerprint_test.go
Interfaces:
- Produces: ``
go const (SourceLog = "log"; SourceEdge = "edge"; SourceServer = "server"; SourceBrowser = "browser") const (KindHTTP = "http"; KindLog = "log") const FPVersion = 1 type Record struct { TS time.Time; Box, Host, Source, Kind, Name, Level, Path string; Status int; Release, Stack string; Fields map[string]string; FP string; FPV int; Count int; First, Last time.Time; Seq uint64 } func FromLog(r applog.Record) Record func Sanitize(s string) string func SanitizePath(p string) string func Normalize(s string) string func TopFrame(stack string) string func Fingerprint(source, kind, name, stack string) string``
- [ ] Step 1: Write failing tests
internal/apperr/sanitize_test.go:
package apperr
import "testing"
func TestSanitize(t *testing.T) {
cases := map[string]string{
"user paul@tito.io not found": "user <email> not found",
"Authorization: Bearer abc.def-ghi": "Authorization: Bearer <redacted>",
"basic dXNlcjpwYXNz failed": "basic <redacted> failed",
"GET /x?token=abc123&y=1": "GET /x?token=<redacted>&y=1",
`{"password": "hunter2", "n": 3}`: `{"password": "<redacted>", "n": 3}`,
"api_key=sk-live-1234 ok": "api_key=<redacted> ok",
"digest 0123456789abcdef0123456789abcdef done": "digest <hex> done",
"b64 QUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVo0MTIzNA== ok": "b64 <hex> ok",
"short 4123 stays": "short 4123 stays",
"nothing here": "nothing here",
}
for in, want := range cases {
if got := Sanitize(in); got != want {
t.Errorf("Sanitize(%q) = %q, want %q", in, got, want)
}
}
}
func TestSanitizePath(t *testing.T) {
cases := map[string]string{
"/reset/8f3a9c1e2b4d6f708a9b0c1d2e3f4a5b?x=1#frag": "/reset/:token",
"/users/paul@tito.io/edit": "/users/:token/edit",
"/items/550e8400-e29b-41d4-a716-446655440000": "/items/:token",
"/s/QUJDREVGR0hJSktMTU5PUFFSU1RV": "/s/:token",
"/blog/2024-review": "/blog/2024-review",
"/api/v2/orders/42": "/api/v2/orders/42",
"/": "/",
"": "/",
}
for in, want := range cases {
if got := SanitizePath(in); got != want {
t.Errorf("SanitizePath(%q) = %q, want %q", in, got, want)
}
}
}internal/apperr/fingerprint_test.go:
package apperr
import "testing"
func TestNormalize(t *testing.T) {
cases := map[string]string{
"user 4123 not found": "user <n> not found",
"user 9911 not found": "user <n> not found",
"HTTP 503 from upstream": "HTTP 503 from upstream",
"retry 3 failed": "retry 3 failed",
"v2 handler panicked": "v2 handler panicked",
"E404 for id 550e8400-e29b-41d4-a716-446655440000": "E404 for id <uuid>",
`missing key "stripe_secret"`: `missing key "?"`,
"trace 0123456789abcdef bad": "trace <hex> bad",
"deadbeef cafe": "<hex> cafe",
}
for in, want := range cases {
if got := Normalize(in); got != want {
t.Errorf("Normalize(%q) = %q, want %q", in, got, want)
}
}
}
func TestTopFrame(t *testing.T) {
cases := map[string]string{
"TypeError: x\n at render (https://app.example/assets/main.3f2a9c1b.js:12:34)\n at y": "at render (https://app.example/assets/main.js)",
"\n app/models/user.rb:42:in `find'\n app/x.rb:1": "app/models/user.rb:in `find'",
"": "",
"single line only": "single line only",
}
for in, want := range cases {
if got := TopFrame(in); got != want {
t.Errorf("TopFrame(%q) = %q, want %q", in, got, want)
}
}
}
func TestFingerprint(t *testing.T) {
a := Fingerprint(SourceLog, KindLog, "user 4123 not found", "")
b := Fingerprint(SourceLog, KindLog, "user 9911 not found", "")
c := Fingerprint(SourceLog, KindLog, "user gone", "")
d := Fingerprint(SourceEdge, KindLog, "user 4123 not found", "")
if a != b {
t.Fatalf("digit runs must group: %s != %s", a, b)
}
if a == c || a == d {
t.Fatalf("different names or sources must not group: %s %s %s", a, c, d)
}
if len(a) != 16 {
t.Fatalf("fingerprint length %d, want 16 hex", len(a))
}
// Length-delimited: ("ab","c") and ("a","bc") differ.
if Fingerprint("ab", "c", "n", "") == Fingerprint("a", "bc", "n", "") {
t.Fatal("components must be length-delimited")
}
// A redeploy's bundle hash and line numbers do not split a browser group.
s1 := "TypeError: x\n at render (https://a/main.3f2a9c1b.js:12:34)"
s2 := "TypeError: x\n at render (https://a/main.9b8c7d6e.js:99:1)"
if Fingerprint(SourceBrowser, "TypeError", "x", s1) != Fingerprint(SourceBrowser, "TypeError", "x", s2) {
t.Fatal("frame normalisation must drop bundle hash and line:col")
}
}- [ ] Step 2: Run to verify failure
Run: go test ./internal/apperr/ -count=1 Expected: FAIL to compile (package does not exist).
- [ ] Step 3: Write
record.go
// Package apperr is the pure half of the Errors tab (spec
// 2026-09-07-errors-and-analytics-design.md §3.1–3.2): the wire record,
// sanitisation, fingerprinting, the box-side coalescing sink, the chunk
// read and the grouping fold. No I/O and no clock of its own — the same
// split internal/applog keeps: internal/host ships, internal/edge and the
// activator produce, internal/console reads.
package apperr
import (
"time"
"github.com/carlosframework/platform/internal/applog"
)
// Sources an error can come from (spec §3.1). log is never stored in the
// errors/ family — it is derived on read from logs/ (§3.2).
const (
SourceLog = "log"
SourceEdge = "edge"
SourceServer = "server"
SourceBrowser = "browser"
)
// Kinds the platform itself produces. A server or browser caller sets its
// own kind (an exception class, typically).
const (
KindHTTP = "http" // an edge-observed 5xx
KindLog = "log" // a log line at or above the configured level
)
// FPVersion stamps the fingerprint algorithm so it can change without
// merging old groups into new ones.
const FPVersion = 1
// Record is one coalesced error on the wire — one JSONL line inside an
// errors/ chunk. Additive shape (AGENTS.md rule): every field but TS,
// Source, Name and FP is omitempty, unknown keys are ignored on read.
// Count, First and Last are the coalescing state: one record stands for
// Count occurrences of FP on Host under Release between First and Last;
// the other fields are the first occurrence's sample.
type Record struct {
TS time.Time `json:"ts"`
Box string `json:"box,omitempty"`
Host string `json:"host,omitempty"`
Source string `json:"source"`
Kind string `json:"kind,omitempty"`
Name string `json:"name"`
Level string `json:"level,omitempty"` // log source: warn | error
Path string `json:"path,omitempty"`
Status int `json:"status,omitempty"`
Release string `json:"release,omitempty"`
Stack string `json:"stack,omitempty"`
Fields map[string]string `json:"fields,omitempty"`
FP string `json:"fp"`
FPV int `json:"fpv,omitempty"`
Count int `json:"count,omitempty"`
First time.Time `json:"first,omitempty"`
Last time.Time `json:"last,omitempty"`
Seq uint64 `json:"seq,omitempty"`
}
// MaxStack bounds a stored stack (spec §3.1).
const MaxStack = 8 << 10
// FromLog derives the log-source record for one applog line (spec §3.2:
// read-derived, never stored twice). Fingerprinted on the normalised
// message alone — a log line has no frame.
func FromLog(r applog.Record) Record {
name := Sanitize(r.Msg)
return Record{
TS: r.TS, Box: r.Box, Host: r.Host,
Source: SourceLog, Kind: KindLog, Name: name, Level: r.Level,
Fields: r.Fields,
FP: Fingerprint(SourceLog, KindLog, name, ""), FPV: FPVersion,
Count: 1, First: r.TS, Last: r.TS, Seq: r.Seq,
}
}- [ ] Step 4: Write
sanitize.go
package apperr
import (
"regexp"
"strings"
)
// Sanitisation is a floor, not a guarantee (spec §3.1): the patterns below
// catch what secrets look like in practice — addresses, bearer tokens,
// key=value credentials, long opaque runs — before anything is buffered.
var (
reEmail = regexp.MustCompile(`[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}`)
reAuth = regexp.MustCompile(`(?i)\b(bearer|basic)\s+[A-Za-z0-9._~+/=-]+`)
// key=value and "key": "value" shapes, value up to the next delimiter.
reKV = regexp.MustCompile(`(?i)\b(token|key|secret|password|passwd|pwd|api_key|apikey|access_token|auth)("?\s*[=:]\s*"?)[^\s&"',;]+`)
// A hex or base64 run of 32+ characters: a digest, a session id, a key.
reLong = regexp.MustCompile(`\b[A-Za-z0-9+/_-]{32,}={0,2}`)
)
// Sanitize redacts secrets in free text: messages, stacks, field values.
func Sanitize(s string) string {
s = reEmail.ReplaceAllString(s, "<email>")
s = reAuth.ReplaceAllStringFunc(s, func(m string) string {
word := m[:strings.IndexAny(m, " \t")]
return word + " <redacted>"
})
s = reKV.ReplaceAllString(s, "${1}${2}<redacted>")
s = reLong.ReplaceAllString(s, "<hex>")
return s
}
var (
reUUID = regexp.MustCompile(`^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$`)
reHex16 = regexp.MustCompile(`^[0-9a-fA-F]{16,}$`)
reB64ish = regexp.MustCompile(`^[A-Za-z0-9+/_-]{20,}={0,2}$`)
reEmailNW = regexp.MustCompile(`^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}$`)
)
// SanitizePath strips query and fragment and collapses token-shaped
// segments to ":token" (spec §3.3): a UUID, a hex run of 16+, a base64-
// looking run of 20+, or an email. "/blog/2024-review" and "/orders/42"
// survive; "/reset/8f3a…" does not. Empty is "/".
func SanitizePath(p string) string {
if i := strings.IndexAny(p, "?#"); i >= 0 {
p = p[:i]
}
if p == "" {
return "/"
}
segs := strings.Split(p, "/")
for i, s := range segs {
if s == "" {
continue
}
if reUUID.MatchString(s) || reHex16.MatchString(s) || reEmailNW.MatchString(s) ||
(reB64ish.MatchString(s) && strings.ContainsAny(s, "0123456789") && strings.ContainsAny(s, "ABCDEFGHIJKLMNOPQRSTUVWXYZ")) {
segs[i] = ":token"
}
}
return strings.Join(segs, "/")
}The base64 segment rule requires both a digit and an upper-case letter so that ordinary long slugs (some-long-page-name-here) survive; the tests pin /s/QUJD… as a token and /blog/2024-review as not.
- [ ] Step 5: Write
fingerprint.go
package apperr
import (
"crypto/sha256"
"encoding/hex"
"regexp"
"strconv"
"strings"
)
var (
reQuoted = regexp.MustCompile(`"[^"]*"|'[^']*'`)
reUUIDAny = regexp.MustCompile(`\b[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}\b`)
reHexRun = regexp.MustCompile(`\b[0-9a-fA-F]{8,}\b`)
reDigits4 = regexp.MustCompile(`\b\d{4,}\b`)
reLineCol = regexp.MustCompile(`:\d+(:\d+)?`)
reBundleJS = regexp.MustCompile(`\.[0-9a-fA-F]{8,}\.js`)
)
// Normalize folds the parts of a message that vary per occurrence into
// placeholders on token boundaries (spec §3.2): quoted strings, UUIDs, hex
// runs of 8+, digit runs of 4+. Tokens of 1–3 digits and mixed tokens
// (v2, E404) survive, so "HTTP 503" and "retry 3" still separate groups
// while "user 4123" and "user 9911" do not. Small ids under 1,000 will
// split groups; that is the stated trade-off, and a caller-supplied FP
// overrides it.
func Normalize(s string) string {
s = reQuoted.ReplaceAllString(s, `"?"`)
s = reUUIDAny.ReplaceAllString(s, "<uuid>")
s = reHexRun.ReplaceAllStringFunc(s, func(m string) string {
if _, err := strconv.ParseUint(m, 10, 64); err == nil {
return "<n>" // all digits: a digit run, not a hex run
}
return "<hex>"
})
s = reDigits4.ReplaceAllString(s, "<n>")
return s
}
// TopFrame is the first non-empty line of a stack with its line, column
// and bundle hash removed, so a redeploy does not split every group.
func TopFrame(stack string) string {
for _, line := range strings.Split(stack, "\n") {
line = strings.TrimSpace(line)
if line == "" {
continue
}
line = reBundleJS.ReplaceAllString(line, ".js")
line = reLineCol.ReplaceAllString(line, "")
return line
}
return ""
}
// Fingerprint groups occurrences: sha256 over length-delimited (source,
// kind, Normalize(name), Normalize(TopFrame(stack)))[:16] as hex. For a
// stack with a message line first (JS "TypeError: x\n at …"), the top
// frame is the second line; callers pass the message as name and the
// whole stack here.
func Fingerprint(source, kind, name, stack string) string {
h := sha256.New()
frame := TopFrame(stack)
if strings.HasPrefix(frame, name) || strings.Contains(frame, ": "+name) {
// The stack's first line restates the message; take the next.
rest := strings.TrimSpace(stack)
if i := strings.IndexByte(rest, '\n'); i >= 0 {
frame = TopFrame(rest[i+1:])
}
}
for _, part := range []string{source, kind, Normalize(name), Normalize(frame)} {
h.Write([]byte(strconv.Itoa(len(part))))
h.Write([]byte{':'})
h.Write([]byte(part))
}
return hex.EncodeToString(h.Sum(nil))[:16]
}Note for TestTopFrame's first case: the expected value is the message line (TypeError: x) is NOT the frame — adjust the test's first case to expect TypeError: x from TopFrame (it is the first non-empty line), and rely on TestFingerprint's last assertion for the message-skipping behaviour. Concretely, change that case's expected string to "TypeError: x".
- [ ] Step 6: Run tests, iterate until green
Run: go test ./internal/apperr/ -count=1 -v Expected: PASS. If a regex case disagrees with a test, fix the regex, not the test, unless the test contradicts the spec's stated rule.
- [ ] Step 7: Commit
git add internal/apperr
git commit -m "apperr: error record, sanitisation and fingerprinting (spec §3.1–3.2)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>"---
Task 4: internal/apperr coalescing sink and bucket read
Files:
- Create:
internal/apperr/sink.go,internal/apperr/read.go - Test:
internal/apperr/sink_test.go,internal/apperr/read_test.go
Interfaces:
- Produces: ``
go const MaxGroupsPerAppHour = 1000 const OverflowFP = "_overflow" func NewSink(box string, now func() time.Time) *Sink func (s *Sink) Add(acct, app string, r Record) // sanitises, fingerprints if FP == "", coalesces func (s *Sink) Drain() map[applog.AppKey][]Record func (s *Sink) Known() []applog.AppKey func Prefix(acct, app string) string // applog.FamilyPrefix(FamilyErrors, …) func ChunkKey(acct, app, box, hour string, seq uint64, size int) string func EncodeChunk(recs []Record) ([]byte, error) func DecodeChunk(b []byte) ([]Record, error) type Filter struct { Since, Until time.Time; Host, Source, Release, FP string } func (f Filter) Match(r Record) bool // [First,Last] overlaps [Since,Until] func ReadWindow(ctx, s release.Store, acct, app string, f Filter, collectCap int) ([]Record, bool, error) func MergeRecords(slices ...[]Record) []Record // by (Last, Box, Seq)``
- [ ] Step 1: Write failing tests
internal/apperr/sink_test.go:
package apperr
import (
"fmt"
"testing"
"time"
"github.com/carlosframework/platform/internal/applog"
)
func TestSinkCoalescesByFingerprintHostRelease(t *testing.T) {
now := time.Date(2026, 9, 7, 10, 0, 0, 0, time.UTC)
clock := now
s := NewSink("box1", func() time.Time { return clock })
for i := 0; i < 10000; i++ {
clock = now.Add(time.Duration(i) * time.Millisecond)
s.Add("bab", "web", Record{Host: "h1", Source: SourceEdge, Kind: KindHTTP, Name: "500 GET /x", Status: 500, Release: "v1"})
}
s.Add("bab", "web", Record{Host: "h2", Source: SourceEdge, Kind: KindHTTP, Name: "500 GET /x", Status: 500, Release: "v1"})
out := s.Drain()
recs := out[applog.AppKey{Acct: "bab", App: "web"}]
if len(recs) != 2 {
t.Fatalf("records = %d, want 2 (one per host)", len(recs))
}
var h1 Record
for _, r := range recs {
if r.Host == "h1" {
h1 = r
}
}
if h1.Count != 10000 || !h1.First.Equal(now) || !h1.Last.Equal(now.Add(9999*time.Millisecond)) {
t.Fatalf("h1 = count %d first %v last %v", h1.Count, h1.First, h1.Last)
}
if h1.FP == "" || h1.FPV != FPVersion || h1.Box != "box1" || h1.Seq == 0 {
t.Fatalf("stamping: %+v", h1)
}
if len(s.Drain()) != 0 {
t.Fatal("second drain must be empty")
}
}
func TestSinkCapsGroupsPerAppPerHour(t *testing.T) {
now := time.Date(2026, 9, 7, 10, 0, 0, 0, time.UTC)
s := NewSink("box1", func() time.Time { return now })
for i := 0; i < MaxGroupsPerAppHour+5; i++ {
s.Add("bab", "web", Record{Source: SourceEdge, Kind: KindHTTP, Name: fmt.Sprintf("500 GET /p%d-x", i), FP: fmt.Sprintf("fp%d", i)})
}
s.Add("bab", "other", Record{Source: SourceEdge, Kind: KindHTTP, Name: "unaffected", FP: "z"})
out := s.Drain()
web := out[applog.AppKey{Acct: "bab", App: "web"}]
if len(web) != MaxGroupsPerAppHour+1 {
t.Fatalf("web records = %d, want cap+1 (the overflow row)", len(web))
}
var over *Record
for i := range web {
if web[i].FP == OverflowFP {
over = &web[i]
}
}
if over == nil || over.Count != 5 || over.Name != "500 GET /p1000-x" {
t.Fatalf("overflow = %+v", over)
}
if len(out[applog.AppKey{Acct: "bab", App: "other"}]) != 1 {
t.Fatal("a neighbour app must not be suppressed")
}
// The cap counts per hour: the next hour admits new groups again.
now = now.Add(time.Hour)
s.Add("bab", "web", Record{Source: SourceEdge, Kind: KindHTTP, Name: "fresh", FP: "fresh"})
if recs := s.Drain()[applog.AppKey{Acct: "bab", App: "web"}]; len(recs) != 1 || recs[0].FP != "fresh" {
t.Fatalf("next hour = %+v", recs)
}
}
func TestSinkSanitisesAndFingerprintsOnAdd(t *testing.T) {
now := time.Date(2026, 9, 7, 10, 0, 0, 0, time.UTC)
s := NewSink("box1", func() time.Time { return now })
long := make([]byte, MaxStack+100)
for i := range long {
long[i] = 'x'
}
s.Add("bab", "web", Record{Source: SourceServer, Kind: "RuntimeError", Name: "mail to paul@tito.io bounced",
Path: "/reset/8f3a9c1e2b4d6f708a9b0c1d2e3f4a5b?t=1", Stack: string(long),
Fields: map[string]string{"user": "paul@tito.io", "k": "v"}})
r := s.Drain()[applog.AppKey{Acct: "bab", App: "web"}][0]
if r.Name != "mail to <email> bounced" || r.Path != "/reset/:token" || r.Fields["user"] != "<email>" || r.Fields["k"] != "v" {
t.Fatalf("sanitised = %+v", r)
}
if len(r.Stack) != MaxStack {
t.Fatalf("stack len %d, want %d", len(r.Stack), MaxStack)
}
if r.FP != Fingerprint(SourceServer, "RuntimeError", "mail to <email> bounced", r.Stack) || r.FPV != FPVersion {
t.Fatalf("fp = %q", r.FP)
}
if r.Count != 1 || !r.First.Equal(now) || !r.Last.Equal(now) || !r.TS.Equal(now) {
t.Fatalf("stamps = %+v", r)
}
}internal/apperr/read_test.go:
package apperr
import (
"context"
"testing"
"time"
"github.com/carlosframework/platform/internal/release"
)
func TestReadWindowMergesAndFilters(t *testing.T) {
ctx := context.Background()
s := release.NewDirStore(t.TempDir())
now := time.Date(2026, 9, 7, 10, 30, 0, 0, time.UTC)
put := func(box string, recs []Record) {
body, err := EncodeChunk(recs)
if err != nil {
t.Fatal(err)
}
if _, err := s.Put(ctx, ChunkKey("bab", "web", box, HourOf(recs[0].TS), recs[0].Seq, len(body)), body, ""); err != nil {
t.Fatal(err)
}
}
put("box1", []Record{
{TS: now, Box: "box1", Host: "h1", Source: SourceEdge, Name: "a", FP: "fa", Count: 3, First: now.Add(-4 * time.Minute), Last: now, Seq: 1},
{TS: now, Box: "box1", Host: "h1", Source: SourceServer, Name: "b", FP: "fb", Count: 1, First: now, Last: now, Seq: 2},
})
put("box2", []Record{
{TS: now.Add(-3 * time.Hour), Box: "box2", Host: "h2", Source: SourceEdge, Name: "old", FP: "fo", Count: 1, First: now.Add(-3 * time.Hour), Last: now.Add(-3 * time.Hour), Seq: 1},
})
recs, trimmed, err := ReadWindow(ctx, s, "bab", "web", Filter{Since: now.Add(-time.Hour)}, 100)
if err != nil || trimmed {
t.Fatalf("read: %v %v", err, trimmed)
}
if len(recs) != 2 || recs[0].FP != "fa" || recs[1].FP != "fb" {
t.Fatalf("window = %+v", recs)
}
// A record whose [First, Last] overlaps the window is in, whole.
recs, _, _ = ReadWindow(ctx, s, "bab", "web", Filter{Since: now.Add(-2 * time.Minute), Until: now.Add(-time.Minute)}, 100)
if len(recs) != 1 || recs[0].FP != "fa" || recs[0].Count != 3 {
t.Fatalf("overlap = %+v", recs)
}
recs, _, _ = ReadWindow(ctx, s, "bab", "web", Filter{Since: now.Add(-time.Hour), Source: SourceServer}, 100)
if len(recs) != 1 || recs[0].FP != "fb" {
t.Fatalf("source filter = %+v", recs)
}
recs, _, _ = ReadWindow(ctx, s, "bab", "web", Filter{Since: now.Add(-4 * time.Hour), Host: "h2"}, 100)
if len(recs) != 1 || recs[0].FP != "fo" {
t.Fatalf("host filter = %+v", recs)
}
}- [ ] Step 2: Run to verify failure
Run: go test ./internal/apperr/ -count=1 Expected: FAIL to compile.
- [ ] Step 3: Write
sink.go
package apperr
import (
"sync"
"time"
"github.com/carlosframework/platform/internal/applog"
)
// MaxGroupsPerAppHour caps coalesced records per app per UTC hour, counting
// (fp, host, release) combinations (spec §3.2). Per app, not per box: one
// tenant's storm must not suppress a neighbour's.
const MaxGroupsPerAppHour = 1000
// OverflowFP names the one row the excess beyond the cap folds into. Its
// Count is occurrences, and the tab labels it as such.
const OverflowFP = "_overflow"
// Sink coalesces errors between ship-loop flushes: per app, one record per
// (fp, host, release) carrying count/first/last and the first sample. An
// error storm is the moment this must work — ten thousand identical panics
// in a minute become one record. Safe for concurrent use; the edge, the
// activator and the ship loop share one Sink per agent process.
type Sink struct {
box string
now func() time.Time
mu sync.Mutex
apps map[applog.AppKey]*appBuf
seq uint64
}
type groupKey struct{ fp, host, release string }
type appBuf struct {
groups map[groupKey]*Record
order []groupKey // insertion order, for a deterministic drain
overflow *Record
hour string // the UTC hour `admitted` counts for
admitted int // distinct groups admitted this hour, cap counter
}
// NewSink returns a Sink stamping records with box and timestamps from now.
// seq starts at UnixMilli for the reason applog.NewSink gives: chunk keys
// embed the first record's seq, and a restart must sort after last boot.
func NewSink(box string, now func() time.Time) *Sink {
return &Sink{box: box, now: now, apps: make(map[applog.AppKey]*appBuf), seq: uint64(now().UnixMilli())}
}
// Add buffers one occurrence for (acct, app). r is sanitised here (§3.1),
// fingerprinted when it carries no FP, and folded into its group.
func (s *Sink) Add(acct, app string, r Record) {
now := s.now().UTC()
r.Name = Sanitize(r.Name)
r.Path = SanitizePath(r.Path)
if r.Path == "/" && r.Status == 0 && r.Source != SourceEdge {
r.Path = ""
}
if len(r.Stack) > MaxStack {
r.Stack = r.Stack[:MaxStack]
}
r.Stack = Sanitize(r.Stack)
if len(r.Fields) > 0 {
f := make(map[string]string, len(r.Fields))
for k, v := range r.Fields {
f[k] = Sanitize(v)
}
r.Fields = f
}
if r.FP == "" {
r.FP = Fingerprint(r.Source, r.Kind, r.Name, r.Stack)
r.FPV = FPVersion
}
if len(r.FP) > 64 {
r.FP = r.FP[:64]
}
s.mu.Lock()
defer s.mu.Unlock()
b := s.apps[applog.AppKey{Acct: acct, App: app}]
if b == nil {
b = &appBuf{groups: make(map[groupKey]*Record)}
s.apps[applog.AppKey{Acct: acct, App: app}] = b
}
if h := applog.HourOf(now); h != b.hour {
b.hour, b.admitted = h, 0
}
k := groupKey{r.FP, r.Host, r.Release}
if g := b.groups[k]; g != nil {
g.Count++
g.Last = now
return
}
if b.admitted >= MaxGroupsPerAppHour {
if b.overflow == nil {
o := r
o.FP = OverflowFP
o.TS, o.First, o.Last, o.Count = now, now, now, 0
o.Box = s.box
s.seq++
o.Seq = s.seq
b.overflow = &o
}
b.overflow.Count++
b.overflow.Last = now
return
}
b.admitted++
s.seq++
r.TS, r.First, r.Last, r.Count = now, now, now, 1
r.Box = s.box
r.Seq = s.seq
b.groups[k] = &r
b.order = append(b.order, k)
}
// Drain returns everything buffered, keyed by app, and resets the groups
// (the per-hour cap counter and seq persist — they are the hour's and the
// process's, not the batch's). Empty apps are omitted.
func (s *Sink) Drain() map[applog.AppKey][]Record {
s.mu.Lock()
defer s.mu.Unlock()
out := make(map[applog.AppKey][]Record)
for k, b := range s.apps {
if len(b.order) == 0 && b.overflow == nil {
continue
}
recs := make([]Record, 0, len(b.order)+1)
for _, gk := range b.order {
recs = append(recs, *b.groups[gk])
}
if b.overflow != nil {
recs = append(recs, *b.overflow)
}
out[k] = recs
b.groups = make(map[groupKey]*Record)
b.order = nil
b.overflow = nil
}
return out
}
// Known lists every app this sink has ever buffered for — the prune
// worklist, as applog.Sink.Known is for logs.
func (s *Sink) Known() []applog.AppKey {
s.mu.Lock()
defer s.mu.Unlock()
out := make([]applog.AppKey, 0, len(s.apps))
for k := range s.apps {
out = append(out, k)
}
return out
}- [ ] Step 4: Write
read.go
package apperr
import (
"context"
"sort"
"time"
"github.com/carlosframework/platform/internal/applog"
"github.com/carlosframework/platform/internal/release"
)
// Prefix is the app's errors/ subtree (spec §3.2): a sibling family of
// logs/ with the same key shape below it.
func Prefix(acct, app string) string { return applog.FamilyPrefix(applog.FamilyErrors, acct, app) }
// BoxPrefix is one box's subtree within an app's errors.
func BoxPrefix(acct, app, box string) string {
return applog.FamilyBoxPrefix(applog.FamilyErrors, acct, app, box)
}
// ChunkKey names one flushed errors chunk; see applog.ChunkKey.
func ChunkKey(acct, app, box, hour string, seq uint64, size int) string {
return applog.FamilyChunkKey(applog.FamilyErrors, acct, app, box, hour, seq, size)
}
// HourOf is applog.HourOf, re-exported so shippers need one import.
func HourOf(t time.Time) string { return applog.HourOf(t) }
// EncodeChunk renders records as gzipped JSONL.
func EncodeChunk(recs []Record) ([]byte, error) { return applog.EncodeJSONL(recs) }
// DecodeChunk parses a chunk, skipping malformed lines.
func DecodeChunk(b []byte) ([]Record, error) { return applog.DecodeJSONL[Record](b) }
// Filter is the read-side match. A coalesced record belongs to a window
// when [First, Last] overlaps [Since, Until] and is counted whole (spec
// §3.2: a window edge is wrong by at most one flush interval).
type Filter struct {
Since, Until time.Time
Host string
Source string
Release string
FP string
}
// Match reports whether r survives f.
func (f Filter) Match(r Record) bool {
last, first := r.Last, r.First
if last.IsZero() {
last = r.TS
}
if first.IsZero() {
first = r.TS
}
if !f.Since.IsZero() && last.Before(f.Since) {
return false
}
if !f.Until.IsZero() && first.After(f.Until) {
return false
}
if f.Host != "" && r.Host != f.Host {
return false
}
if f.Source != "" && r.Source != f.Source {
return false
}
if f.Release != "" && r.Release != f.Release {
return false
}
if f.FP != "" && r.FP != f.FP {
return false
}
return true
}
// ReadWindow assembles one app's errors/ records across boxes: every hour
// intersecting the window, filtered through f, merged by (Last, Box, Seq).
// collectCap bounds memory; past it the oldest go and the bool says so.
func ReadWindow(ctx context.Context, s release.Store, acct, app string, f Filter, collectCap int) ([]Record, bool, error) {
perBox, trimmed, err := applog.ReadChunks[Record](ctx, s, Prefix(acct, app), f.Since, f.Until, f.Match, collectCap)
if err != nil {
return nil, false, err
}
merged := MergeRecords(perBox...)
if over := len(merged) - collectCap; over > 0 {
merged = merged[over:]
trimmed = true
}
return merged, trimmed, nil
}
// MergeRecords folds per-source slices into one timeline ordered by
// (Last, Box, Seq), oldest first.
func MergeRecords(slices ...[]Record) []Record {
var out []Record
for _, s := range slices {
out = append(out, s...)
}
sort.SliceStable(out, func(i, j int) bool {
a, b := out[i], out[j]
if !a.Last.Equal(b.Last) {
return a.Last.Before(b.Last)
}
if a.Box != b.Box {
return a.Box < b.Box
}
return a.Seq < b.Seq
})
return out
}Note on ReadChunks and the hour window: a record's chunk lives under the hour of its TS (the first occurrence); a window that starts after that hour but before Last will not list the hour. Since the flush interval is at most ten seconds plus the hour boundary, the reader widens Since by one hour when listing: in ReadWindow, pass f.Since.Add(-time.Hour) as the listing since (when non-zero) and keep f.Match as the exact test. Add that line and a comment saying why.
- [ ] Step 5: Run tests
Run: go test ./internal/apperr/ -count=1 -v Expected: PASS.
- [ ] Step 6: Commit
git add internal/apperr
git commit -m "apperr: coalescing sink and errors/ chunk read (spec §3.2)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>"---
Task 5: internal/apperr grouping for the tab
Files:
- Create:
internal/apperr/group.go - Test:
internal/apperr/group_test.go
Interfaces:
- Produces: ``
go type Group struct { FP, Source, Kind, Name, Level string; Count int; First, Last time.Time; Hosts, Releases []string; Sample Record } func GroupRecords(recs []Record) []Group // newest Last first``
- [ ] Step 1: Failing test
package apperr
import (
"testing"
"time"
)
func TestGroupRecords(t *testing.T) {
now := time.Date(2026, 9, 7, 10, 0, 0, 0, time.UTC)
recs := []Record{
{FP: "a", Source: SourceEdge, Kind: KindHTTP, Name: "500 GET /x", Host: "h1", Release: "v1", Count: 3, First: now.Add(-10 * time.Minute), Last: now.Add(-5 * time.Minute), Stack: "s1"},
{FP: "a", Source: SourceEdge, Kind: KindHTTP, Name: "500 GET /x", Host: "h2", Release: "v2", Count: 2, First: now.Add(-20 * time.Minute), Last: now.Add(-1 * time.Minute)},
{FP: "b", Source: SourceLog, Kind: KindLog, Name: "boom", Level: "error", Host: "h1", Count: 1, First: now.Add(-30 * time.Minute), Last: now.Add(-30 * time.Minute)},
{FP: OverflowFP, Source: SourceEdge, Name: "500 GET /z", Host: "h1", Count: 40, First: now, Last: now},
}
gs := GroupRecords(recs)
if len(gs) != 3 {
t.Fatalf("groups = %d", len(gs))
}
if gs[0].FP != OverflowFP || gs[1].FP != "a" || gs[2].FP != "b" {
t.Fatalf("order by Last desc: %s %s %s", gs[0].FP, gs[1].FP, gs[2].FP)
}
a := gs[1]
if a.Count != 5 || !a.First.Equal(now.Add(-20*time.Minute)) || !a.Last.Equal(now.Add(-1*time.Minute)) {
t.Fatalf("a = %+v", a)
}
if len(a.Hosts) != 2 || a.Hosts[0] != "h1" || a.Hosts[1] != "h2" || len(a.Releases) != 2 {
t.Fatalf("a hosts/releases = %v %v", a.Hosts, a.Releases)
}
if a.Sample.Stack != "s1" {
t.Fatal("sample must be the earliest record")
}
}- [ ] Step 2: Run, expect compile failure
- [ ] Step 3: Implement
package apperr
import (
"sort"
"time"
)
// Group is one row of the Errors tab: every record sharing a fingerprint
// in the window, summed (spec §3.6).
type Group struct {
FP string `json:"fp"`
Source string `json:"source"`
Kind string `json:"kind,omitempty"`
Name string `json:"name"`
Level string `json:"level,omitempty"`
Count int `json:"count"`
First time.Time `json:"first"`
Last time.Time `json:"last"`
Hosts []string `json:"hosts"`
Releases []string `json:"releases,omitempty"`
Sample Record `json:"sample"`
}
// GroupRecords folds records into groups by FP, newest Last first. The
// sample is the earliest record of the group (its stack and fields are
// the first occurrence's).
func GroupRecords(recs []Record) []Group {
byFP := make(map[string]*Group)
var order []string
for _, r := range recs {
first, last := r.First, r.Last
if first.IsZero() {
first = r.TS
}
if last.IsZero() {
last = r.TS
}
count := r.Count
if count == 0 {
count = 1
}
g := byFP[r.FP]
if g == nil {
g = &Group{FP: r.FP, Source: r.Source, Kind: r.Kind, Name: r.Name, Level: r.Level, First: first, Last: last, Sample: r}
byFP[r.FP] = g
order = append(order, r.FP)
}
g.Count += count
if first.Before(g.First) {
g.First = first
g.Sample = r
}
if last.After(g.Last) {
g.Last = last
}
g.Hosts = addUnique(g.Hosts, r.Host)
g.Releases = addUnique(g.Releases, r.Release)
}
out := make([]Group, 0, len(order))
for _, fp := range order {
g := byFP[fp]
sort.Strings(g.Hosts)
sort.Strings(g.Releases)
out = append(out, *g)
}
sort.SliceStable(out, func(i, j int) bool { return out[i].Last.After(out[j].Last) })
return out
}
func addUnique(list []string, v string) []string {
if v == "" {
return list
}
for _, x := range list {
if x == v {
return list
}
}
return append(list, v)
}- [ ] Step 4: Run, expect PASS. Commit
git add internal/apperr/group.go internal/apperr/group_test.go
git commit -m "apperr: group records by fingerprint for the Errors tab
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>"---
Task 6: Ship the error sink from the agent; prune by family
Files:
- Modify:
internal/host/logship.go(logshipPrune→chunkPrune; removehourNameOf) - Create:
internal/host/errship.go - Modify:
internal/host/agent.go:234-237, 384 - Test:
internal/host/logship_test.go(update two callers),internal/host/errship_test.go
Interfaces:
- Consumes:
apperr.Sink,apperr.EncodeChunk,apperr.ChunkKey,apperr.BoxPrefix,applog.FamilyBoxPrefix,applog.HourNameOfDir. - Produces:
chunkPrune(ctx, s release.Store, fam applog.Family, box string, k applog.AppKey, now time.Time);errshipLoop(sink *apperr.Sink, box string);errshipFlush(ctx, s, pending map[applog.AppKey][]apperr.Record); package-levelerrSink *apperr.Sinkis NOT used — the sink is constructed inrunAgentbesidelogSinkand passed to the edge in theRunWithActivatorReadycallback (Task 7 addsEnableErrors).
- [ ] Step 1: Refactor prune to take a family
In logship.go, rename logshipPrune to chunkPrune with signature (ctx context.Context, s release.Store, fam applog.Family, box string, k applog.AppKey, now time.Time); inside, prefix := applog.FamilyBoxPrefix(fam, k.Acct, k.App, box) and hourName := applog.HourNameOfDir(h.dir); delete hourNameOf. Update the call in logshipLoop to chunkPrune(ctx, s, applog.FamilyLogs, box, k, time.Now()) and the two test callers in logship_test.go likewise. Update the doc comment: "for ONE app's ONE family on THIS box's subtree".
Run: go test ./internal/host/ -run Logship -count=1 — PASS.
- [ ] Step 2: Write the failing errship test
internal/host/errship_test.go:
package host
import (
"context"
"testing"
"time"
"github.com/carlosframework/platform/internal/apperr"
"github.com/carlosframework/platform/internal/applog"
"github.com/carlosframework/platform/internal/release"
)
func TestErrshipFlushWritesOneChunkPerApp(t *testing.T) {
s := release.NewDirStore(t.TempDir())
now := time.Date(2026, 9, 7, 10, 0, 0, 0, time.UTC)
sink := apperr.NewSink("box1", func() time.Time { return now })
sink.Add("bab", "web", apperr.Record{Source: apperr.SourceEdge, Kind: apperr.KindHTTP, Name: "500 GET /x", Host: "h1"})
sink.Add("bab", "web", apperr.Record{Source: apperr.SourceEdge, Kind: apperr.KindHTTP, Name: "500 GET /x", Host: "h1"})
pending := map[applog.AppKey][]apperr.Record{}
errshipGather(sink, pending)
errshipFlush(context.Background(), s, pending)
if len(pending) != 0 {
t.Fatalf("pending after flush = %d", len(pending))
}
recs, _, err := apperr.ReadWindow(context.Background(), s, "bab", "web", apperr.Filter{Since: now.Add(-time.Minute)}, 100)
if err != nil || len(recs) != 1 || recs[0].Count != 2 {
t.Fatalf("read back = %+v, %v", recs, err)
}
}
func TestErrshipFlushHoldsOnFailure(t *testing.T) {
pending := map[applog.AppKey][]apperr.Record{{Acct: "bab", App: "web"}: {{TS: time.Now(), Source: apperr.SourceEdge, Name: "x", FP: "f", Box: "box1", Seq: 1}}}
errshipFlush(context.Background(), failingStore{}, pending)
if len(pending) != 1 {
t.Fatal("a failed flush must keep its records for the next round")
}
}(failingStore already exists in logship_test.go.)
- [ ] Step 3: Write
errship.go
// The error shipping loop — the box-side pump of the Errors tab (spec
// 2026-09-07-errors-and-analytics-design.md §3.2), shaped exactly like
// logshipLoop: the edge and the activator coalesce into one apperr.Sink;
// every tick this drains it and writes one immutable chunk per app under
// apps/<sqid>/<app>/errors/<box>/<hour>/; on each hour boundary it prunes
// its own subtree with the log retention (§3.6: errors follow log
// retention, age and byte cap).
package host
import (
"context"
"errors"
"log"
"os"
"time"
"github.com/carlosframework/platform/internal/apperr"
"github.com/carlosframework/platform/internal/applog"
"github.com/carlosframework/platform/internal/release"
)
const errshipTick = 10 * time.Second
func errshipLoop(sink *apperr.Sink, box string) {
defer func() {
if r := recover(); r != nil {
log.Printf("agent: errship loop panic, error shipping stopped for this process: %v", r)
}
}()
ctx := context.Background()
s, target, err := openStatusStore(ctx, os.Getenv)
if err != nil {
if errors.Is(err, errNoStatusStore) {
log.Printf("agent: errship loop disabled: neither CARLOS_STATUS_BUCKET nor CARLOS_DEPLOYMENT_BUCKET is set")
} else {
log.Printf("agent: errship loop disabled: %v", err)
}
return
}
log.Printf("agent: errship loop armed (10s ticks), publishing to %s", target)
pending := make(map[applog.AppKey][]apperr.Record)
pruned := ""
ticker := time.NewTicker(errshipTick)
defer ticker.Stop()
for range ticker.C {
errshipGather(sink, pending)
errshipFlush(ctx, s, pending)
if hour := applog.HourOf(time.Now()); hour != pruned {
for _, k := range errshipKnownApps(pending, sink) {
chunkPrune(ctx, s, applog.FamilyErrors, box, k, time.Now())
}
pruned = hour
}
}
}
// errshipGather drains the sink into pending under the same per-app cap
// logship uses — oldest first.
func errshipGather(sink *apperr.Sink, pending map[applog.AppKey][]apperr.Record) {
for k, recs := range sink.Drain() {
merged := append(pending[k], recs...)
if over := len(merged) - logshipMaxPending; over > 0 {
merged = merged[over:]
}
pending[k] = merged
}
}
// errshipFlush writes one create-only chunk per pending app and forgets
// what landed; a failed app keeps its records for the next round.
func errshipFlush(ctx context.Context, s release.Store, pending map[applog.AppKey][]apperr.Record) {
for k, recs := range pending {
if len(recs) == 0 {
delete(pending, k)
continue
}
body, err := apperr.EncodeChunk(recs)
if err != nil {
log.Printf("agent: errship encode %s/%s: %v (dropping %d records)", k.Acct, k.App, err, len(recs))
delete(pending, k)
continue
}
key := apperr.ChunkKey(k.Acct, k.App, recs[0].Box, apperr.HourOf(recs[0].TS), recs[0].Seq, len(body))
if _, err := s.Put(ctx, key, body, ""); err != nil {
log.Printf("agent: errship flush %s/%s: %v (%d records held)", k.Acct, k.App, err, len(recs))
continue
}
delete(pending, k)
}
}
func errshipKnownApps(pending map[applog.AppKey][]apperr.Record, sink *apperr.Sink) []applog.AppKey {
seen := make(map[applog.AppKey]bool)
var out []applog.AppKey
for k := range pending {
if !seen[k] {
seen[k] = true
out = append(out, k)
}
}
for _, k := range sink.Known() {
if !seen[k] {
seen[k] = true
out = append(out, k)
}
}
return out
}- [ ] Step 4: Wire in
agent.go
Beside logSink (line 234-237):
var logSink *applog.Sink
var errSink *apperr.Sink
if _, _, err := openStatusStore(context.Background(), os.Getenv); err == nil {
logSink = applog.NewSink(status.Label(readMachineID()), time.Now)
errSink = apperr.NewSink(status.Label(readMachineID()), time.Now)
}Beside go logshipLoop(...) (line 384): if errSink != nil { go errshipLoop(errSink, status.Label(readMachineID())) } — match however logshipLoop is gated there (read the surrounding lines and mirror the condition exactly).
In the RunWithActivatorReady callback after e.EnableUsage(m): if errSink != nil { e.EnableErrors(errSink) } — EnableErrors arrives in Task 7; add this line in Task 7 instead if the build order matters. Order the tasks so Task 7 lands before this file compiles: do Task 7's edge change first if convenient, or add the EnableErrors call in Task 7. Decision: add the agent.go callback line in Task 7, and only the sink construction and loop here.
- [ ] Step 5: Test and commit
Run: go build ./... && go test ./internal/host/ -count=1 Expected: PASS.
git add internal/host
git commit -m "host: ship coalesced errors like logs; prune by chunk family
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>"---
Task 7: The edge observes status and emits edge errors
Files:
- Create:
internal/activator/answered.go - Modify:
internal/activator/activator.go:1298-1330(threehttp.Errorsites inserve) - Modify:
internal/edge/usagemeter.go(countingWriter) - Modify:
internal/edge/edge.go(Edgefields,EnableErrors,ServeHTTPat 1241-1250 and the hibernating branch) - Modify:
internal/host/agent.go(callback line) - Test:
internal/activator/answered_test.go,internal/edge/usagemeter_test.go,internal/edge/edgeerrors_test.go
Interfaces:
- Produces:
activator.WithAnswerFlag(r *http.Request) (*http.Request, *bool);activator.MarkAnswered(r *http.Request);edge.ErrorSink interface{ Add(acct, app string, r apperr.Record) };(*Edge).EnableErrors(ErrorSink);countingWritergainsstatus int,ctype string,wroteHdr booland aWriteHeader.
- [ ] Step 1: Failing tests
internal/activator/answered_test.go:
package activator
import (
"net/http/httptest"
"testing"
)
func TestAnswerFlagRoundTrip(t *testing.T) {
r := httptest.NewRequest("GET", "http://h/", nil)
MarkAnswered(r) // no flag: a no-op, never a panic
r, flag := WithAnswerFlag(r)
if *flag {
t.Fatal("flag must start false")
}
MarkAnswered(r)
if !*flag {
t.Fatal("MarkAnswered must set the flag the edge holds")
}
}Append to internal/edge/usagemeter_test.go:
func TestCountingWriterObservesStatusAndType(t *testing.T) {
var n int64
rec := httptest.NewRecorder()
w := &countingWriter{ResponseWriter: rec, n: &n}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
w.WriteHeader(103) // informational: forwarded, not latched
w.WriteHeader(http.StatusBadGateway)
w.WriteHeader(http.StatusOK) // superfluous second final status: ignored
if w.status != http.StatusBadGateway || w.ctype != "text/html; charset=utf-8" {
t.Fatalf("latched %d %q", w.status, w.ctype)
}
// Implicit 200 at first Write.
rec2 := httptest.NewRecorder()
w2 := &countingWriter{ResponseWriter: rec2, n: &n}
w2.Write([]byte("x"))
if w2.status != http.StatusOK {
t.Fatalf("implicit status = %d", w2.status)
}
// Nothing written at all: status stays 0 (the caller treats it as a
// bodiless 200 that is never a pageview and never an error).
w3 := &countingWriter{ResponseWriter: httptest.NewRecorder(), n: &n}
if w3.status != 0 {
t.Fatal("untouched writer must report 0")
}
}internal/edge/edgeerrors_test.go:
package edge
import (
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
"github.com/carlosframework/platform/internal/activator"
"github.com/carlosframework/platform/internal/apperr"
"github.com/carlosframework/platform/internal/registry"
)
type memErrSink struct {
mu sync.Mutex
recs []apperr.Record
apps []string
}
func (m *memErrSink) Add(acct, app string, r apperr.Record) {
m.mu.Lock()
defer m.mu.Unlock()
m.recs = append(m.recs, r)
m.apps = append(m.apps, acct+"/"+app)
}
func TestEdgeEmitsErrorOnUpstream5xx(t *testing.T) {
backend := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/boom":
http.Error(w, "kaboom", http.StatusInternalServerError)
case "/missing":
http.NotFound(w, r)
default:
w.Header().Set("Content-Type", "text/html")
w.Write([]byte("<p>ok</p>"))
}
}))
t.Cleanup(backend.Close)
reg := openReg(t)
rt := registry.Route{Host: "app.example.com", Kind: "service", Addr: backend.Listener.Addr().String(), Status: "ready", App: "web", Version: "v7"}
if err := reg.Upsert(rt); err != nil {
t.Fatal(err)
}
e, err := New(reg, nil)
if err != nil {
t.Fatal(err)
}
sink := &memErrSink{}
e.EnableErrors(sink)
srv := httptest.NewServer(e)
t.Cleanup(srv.Close)
do := func(method, path string) int {
req, _ := http.NewRequest(method, srv.URL+path, nil)
req.Host = "app.example.com"
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatal(err)
}
resp.Body.Close()
return resp.StatusCode
}
if do("POST", "/boom?token=abc") != 500 || do("GET", "/missing") != 404 || do("GET", "/") != 200 {
t.Fatal("upstream statuses must pass through unchanged")
}
if len(sink.recs) != 1 {
t.Fatalf("errors emitted = %d, want 1 (the 500 only)", len(sink.recs))
}
r := sink.recs[0]
if r.Source != apperr.SourceEdge || r.Kind != apperr.KindHTTP || r.Status != 500 || r.Host != "app.example.com" || r.Release != "v7" {
t.Fatalf("record = %+v", r)
}
if r.Name != "500 POST /boom" || r.Path != "/boom" || strings.Contains(r.Name, "token") {
t.Fatalf("name/path = %q %q", r.Name, r.Path)
}
}
// answering503 is an Activator that answers every request itself with the
// wake-path 503 the real one writes, and marks it as the activator's own.
type answering503 struct{}
func (answering503) ServeHTTP(w http.ResponseWriter, r *http.Request, rt registry.Route, proxy http.Handler) bool {
activator.MarkAnswered(r)
http.Error(w, "instance failed to wake", http.StatusServiceUnavailable)
return true
}
func (answering503) RetireExcept(map[string]bool) {}
func TestEdgeSkipsActivatorAnswered503(t *testing.T) {
reg := openReg(t)
rt := registry.Route{Host: "sleepy.example.com", Kind: "instance", Socket: t.TempDir() + "/x.sock", Status: "ready", Hibernate: true, App: "web"}
if err := reg.Upsert(rt); err != nil {
t.Fatal(err)
}
e, err := New(reg, answering503{})
if err != nil {
t.Fatal(err)
}
sink := &memErrSink{}
e.EnableErrors(sink)
rec := httptest.NewRecorder()
req := httptest.NewRequest("GET", "http://sleepy.example.com/", nil)
e.ServeHTTP(rec, req)
if rec.Code != 503 {
t.Fatalf("status = %d", rec.Code)
}
if len(sink.recs) != 0 {
t.Fatalf("an activator-answered 503 is already a platform log error; got %d edge errors", len(sink.recs))
}
}If registry.Route needs different fields for a hibernating instance route to load into the edge's map (check TestHibernat* tests in edge_test.go or activator_e2e_test.go for the minimal upsert they use), copy that shape.
- [ ] Step 2: Run to confirm failure
Run: go test ./internal/activator/ -run TestAnswerFlag -count=1; go test ./internal/edge/ -run 'TestCountingWriterObserves|TestEdgeEmits|TestEdgeSkips' -count=1 Expected: compile failures.
- [ ] Step 3:
internal/activator/answered.go
package activator
import (
"context"
"net/http"
)
// answeredKey carries the flag the edge attaches so it can tell a 503 the
// activator wrote on the wake path from one the upstream returned. The
// wake failure already reaches the member as a platform log error
// (emitEvent, logsink.go); the edge must not also file it as an edge
// error (spec 2026-09-07 §3.2, "exactly once").
type answeredKey struct{}
// WithAnswerFlag returns r carrying a fresh flag and the flag itself.
func WithAnswerFlag(r *http.Request) (*http.Request, *bool) {
f := new(bool)
return r.WithContext(context.WithValue(r.Context(), answeredKey{}, f)), f
}
// MarkAnswered sets the flag when r carries one; a no-op otherwise.
func MarkAnswered(r *http.Request) {
if f, ok := r.Context().Value(answeredKey{}).(*bool); ok {
*f = true
}
}In serve (activator.go), call MarkAnswered(r) immediately before each of the four http.Error(w, …, http.StatusServiceUnavailable) calls (sidecar-asleep, waking elsewhere, waking, failed to wake).
- [ ] Step 4:
countingWriterobserves
In usagemeter.go:
// countingWriter counts response body bytes and OBSERVES the final status
// and Content-Type (spec 2026-09-07 §3.3): it is the one wrapper installed
// at a single site for every response shape — cache hit, plain proxy,
// hibernating, upgrade — so it is where the edge learns that an upstream
// answered 5xx (an edge error) or served HTML (a pageview, step 2). It
// sits below the compressor, so Content-Type is the synced value
// gzipWriter.syncHeader copied down before calling this WriteHeader.
// Deliberately does NOT implement io.ReaderFrom: io.Copy falls back to
// Write, which counts. Hijack/Flush forward via the interfaces the wrapped
// writer provides, mirroring recordingResponseWriter (edge.go).
type countingWriter struct {
http.ResponseWriter
n *int64
status int // final status (>= 200); 0 until one is written
ctype string // Content-Type at the final WriteHeader
wroteHdr bool
}
// WriteHeader latches the first final status. 1xx are forwarded and never
// latched (recordingResponseWriter.WriteHeader's reasoning).
func (c *countingWriter) WriteHeader(status int) {
if status < 200 {
c.ResponseWriter.WriteHeader(status)
return
}
if !c.wroteHdr {
c.wroteHdr = true
c.status = status
c.ctype = c.ResponseWriter.Header().Get("Content-Type")
}
c.ResponseWriter.WriteHeader(status)
}
func (c *countingWriter) Write(p []byte) (int, error) {
if !c.wroteHdr {
c.wroteHdr = true
c.status = http.StatusOK
c.ctype = c.ResponseWriter.Header().Get("Content-Type")
}
n, err := c.ResponseWriter.Write(p)
*c.n += int64(n)
return n, err
}Keep Hijack and Flush as they are.
- [ ] Step 5: Edge wiring
In edge.go:
Add near the usageMeter field:
// errSink, when set, receives edge-observed errors (an upstream 5xx)
// — spec 2026-09-07 §3.2. Nil disables emission; nothing else changes.
errSink atomic.Pointer[errorSinkRef]and:
// ErrorSink is where the edge files an error it observed on the serving
// path. apperr.Sink satisfies it; the interface lives here so this package
// depends on the contract, not the buffer.
type ErrorSink interface {
Add(acct, app string, r apperr.Record)
}
type errorSinkRef struct{ ErrorSink }
// EnableErrors wires the error sink in. Called once at startup by the
// agent; nil is a valid no-op.
func (e *Edge) EnableErrors(s ErrorSink) {
if s == nil {
e.errSink.Store(nil)
return
}
e.errSink.Store(&errorSinkRef{s})
}
// emitEdgeError files one edge-observed 5xx for rt. The path is sanitised
// (query and token segments off) before it becomes the record's name.
func (e *Edge) emitEdgeError(rt registry.Route, r *http.Request, status int) {
ref := e.errSink.Load()
if ref == nil {
return
}
path := apperr.SanitizePath(r.URL.Path)
ref.Add(release.RouteAccount(rt), rt.App, apperr.Record{
Host: rt.Host,
Source: apperr.SourceEdge,
Kind: apperr.KindHTTP,
Name: fmt.Sprintf("%d %s %s", status, r.Method, path),
Path: path,
Status: status,
Release: rt.Version,
})
}Replace the meter block at edge.go:1241-1245 with:
// The observing writer is installed UNCONDITIONALLY now — the usage
// meter is one of its two consumers, the errors sink the other — at
// the same single site, below everything that follows, so all three
// dispatch shapes plus the upgrade path pass through it. Meter and
// sink are each nil-safe. Status routes are neither billed nor observed.
//
// (hopped attribution paragraph: keep the existing comment text here.)
var bytes int64
ob := &countingWriter{ResponseWriter: w, n: &bytes}
w = ob
var answered *bool
if up.hibernating {
// The activator marks the wake-path 503s it writes itself; those
// are already the member's platform log error and are not filed
// again as edge errors (spec §3.2).
r, answered = activator.WithAnswerFlag(r)
}
if up.route.Kind != "status" {
defer func() {
if m := e.usageMeter.Load(); m != nil {
m.Record(up.route, 1, bytes, hopped)
}
if ob.status >= 500 && (answered == nil || !*answered) {
e.emitEdgeError(up.route, r, ob.status)
}
}()
}Add "github.com/carlosframework/platform/internal/activator" and "github.com/carlosframework/platform/internal/apperr" to the imports. Confirm no cycle: go build ./... (activator must not import edge; it does not today).
Existing usage tests that construct the edge without a meter must still pass: Record is skipped when the meter is nil.
- [ ] Step 6:
agent.gocallback
After e.EnableUsage(m):
if errSink != nil {
e.EnableErrors(errSink)
}- [ ] Step 7: Run edge, activator, host packages
Run: go build ./... && go vet ./internal/edge/ ./internal/activator/ ./internal/host/ && go test ./internal/edge/ ./internal/activator/ ./internal/host/ -count=1 Expected: PASS, including every pre-existing usage, compress and cache test.
- [ ] Step 8: Commit
git add internal/activator/answered.go internal/activator/activator.go internal/activator/answered_test.go internal/edge internal/host/agent.go
git commit -m "edge: the counting writer observes status; upstream 5xx become edge errors
Installed unconditionally at the one site every response shape passes;
the activator marks its own wake-path 503s so they are filed once, as the
platform log error they already are (spec 2026-09-07 §3.2–3.3).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>"---
Task 8: internal/telemetry settings object
Files:
- Create:
internal/telemetry/telemetry.go - Test:
internal/telemetry/telemetry_test.go
Interfaces:
- Produces: ``
go const (LogLevelError = "error"; LogLevelWarn = "warn"; CountEdge = "edge"; CountOff = "off"; DoorOn = "on"; DoorOff = "off") type Analytics struct { Count stringjson:"count,omitempty"; Door stringjson:"door,omitempty"} type Errors struct { LogLevel stringjson:"log_level,omitempty"} type Settings struct { Analytics *Analyticsjson:"analytics,omitempty"; Errors *Errorsjson:"errors,omitempty"} func AccountKey(sqid string) string func AppKey(sqid, app string) string func Read(ctx, s release.Store, key string) (Settings, string, error) // etag; os.ErrNotExist → zero Settings, "", nil func Update(ctx, s release.Store, key string, fn func(*Settings)) error // read-modify-CAS-write, 3 attempts type Resolved struct { LogLevel, LogLevelFrom, Count, CountFrom, Door, DoorFrom string } func Resolve(ctx, s release.Store, sqid, app string) (Resolved, error) func ValidLogLevel(s string) bool``
- [ ] Step 1: Failing test
package telemetry
import (
"context"
"testing"
"github.com/carlosframework/platform/internal/release"
)
func TestResolveAppThenAccountThenDefault(t *testing.T) {
ctx := context.Background()
s := release.NewDirStore(t.TempDir())
r, err := Resolve(ctx, s, "bab", "web")
if err != nil {
t.Fatal(err)
}
if r.LogLevel != LogLevelError || r.LogLevelFrom != "default" || r.Count != CountOff || r.CountFrom != "default" || r.Door != DoorOn {
t.Fatalf("defaults = %+v", r)
}
if err := Update(ctx, s, AccountKey("bab"), func(st *Settings) {
st.Errors = &Errors{LogLevel: LogLevelWarn}
st.Analytics = &Analytics{Count: CountEdge}
}); err != nil {
t.Fatal(err)
}
r, _ = Resolve(ctx, s, "bab", "web")
if r.LogLevel != LogLevelWarn || r.LogLevelFrom != "account" || r.Count != CountEdge || r.CountFrom != "account" || r.Door != DoorOn || r.DoorFrom != "default" {
t.Fatalf("account level = %+v", r)
}
if err := Update(ctx, s, AppKey("bab", "web"), func(st *Settings) {
st.Analytics = &Analytics{Count: CountOff}
}); err != nil {
t.Fatal(err)
}
r, _ = Resolve(ctx, s, "bab", "web")
if r.LogLevel != LogLevelWarn || r.Count != CountOff || r.CountFrom != "app" {
t.Fatalf("app override = %+v", r)
}
// Update preserves the other key.
if err := Update(ctx, s, AppKey("bab", "web"), func(st *Settings) { st.Errors = &Errors{LogLevel: LogLevelError} }); err != nil {
t.Fatal(err)
}
got, _, _ := Read(ctx, s, AppKey("bab", "web"))
if got.Analytics == nil || got.Analytics.Count != CountOff || got.Errors == nil || got.Errors.LogLevel != LogLevelError {
t.Fatalf("read back = %+v", got)
}
}- [ ] Step 2: Run, expect compile failure
- [ ] Step 3: Implement
// Package telemetry is the settings object behind the Errors and Analytics
// tabs (spec 2026-09-07-errors-and-analytics-design.md §3.3): one JSON
// shape at two levels — accounts/<sqid>/telemetry.json and
// apps/<sqid>/<app>/telemetry.json — resolved app → account → built-in
// default. Defaults are rulings (Paul, 2026-09-07): analytics off, the
// browser door on, the log level error.
package telemetry
import (
"context"
"encoding/json"
"errors"
"fmt"
"os"
"github.com/carlosframework/platform/internal/release"
)
const (
LogLevelError = "error"
LogLevelWarn = "warn"
CountEdge = "edge"
CountOff = "off"
DoorOn = "on"
DoorOff = "off"
)
// Analytics is the analytics half. Empty strings mean "not set here".
type Analytics struct {
Count string `json:"count,omitempty"`
Door string `json:"door,omitempty"`
}
// Errors is the errors half.
type Errors struct {
LogLevel string `json:"log_level,omitempty"`
}
// Settings is the wire shape at either level. Additive: unknown keys are
// ignored, absent halves inherit.
type Settings struct {
Analytics *Analytics `json:"analytics,omitempty"`
Errors *Errors `json:"errors,omitempty"`
}
// AccountKey is the account-level object, beside account.json (which stays
// identity-only).
func AccountKey(sqid string) string { return "accounts/" + sqid + "/telemetry.json" }
// AppKey is the per-app override.
func AppKey(sqid, app string) string { return release.AppPrefix(sqid, app) + "telemetry.json" }
// ValidLogLevel reports whether s is a level the setting accepts.
func ValidLogLevel(s string) bool { return s == LogLevelError || s == LogLevelWarn }
// Read returns the object at key and its etag; an absent object is the
// zero Settings with an empty etag and no error.
func Read(ctx context.Context, s release.Store, key string) (Settings, string, error) {
body, etag, err := s.Get(ctx, key)
if err != nil {
if errors.Is(err, os.ErrNotExist) {
return Settings{}, "", nil
}
return Settings{}, "", err
}
var st Settings
if err := json.Unmarshal(body, &st); err != nil {
return Settings{}, "", fmt.Errorf("telemetry: %s: %w", key, err)
}
return st, etag, nil
}
// Update applies fn to the object at key under compare-and-swap, three
// attempts, so two owners editing different halves at once both land.
func Update(ctx context.Context, s release.Store, key string, fn func(*Settings)) error {
var last error
for attempt := 0; attempt < 3; attempt++ {
st, etag, err := Read(ctx, s, key)
if err != nil {
return err
}
fn(&st)
body, err := json.Marshal(st)
if err != nil {
return err
}
if _, err := s.Put(ctx, key, body, etag); err != nil {
if errors.Is(err, release.ErrCASConflict) {
last = err
continue
}
return err
}
return nil
}
return last
}
// Resolved is what a reader acts on, with each value's provenance so a
// settings page can say "inherited from the account".
type Resolved struct {
LogLevel, LogLevelFrom string
Count, CountFrom string
Door, DoorFrom string
}
// Resolve reads both levels and folds them over the defaults. A store
// error fails the resolve; an absent object at either level is fine.
func Resolve(ctx context.Context, s release.Store, sqid, app string) (Resolved, error) {
r := Resolved{LogLevel: LogLevelError, LogLevelFrom: "default", Count: CountOff, CountFrom: "default", Door: DoorOn, DoorFrom: "default"}
if s == nil {
return r, nil
}
acct, _, err := Read(ctx, s, AccountKey(sqid))
if err != nil {
return r, err
}
appSt, _, err := Read(ctx, s, AppKey(sqid, app))
if err != nil {
return r, err
}
for _, lvl := range []struct {
from string
st Settings
}{{"account", acct}, {"app", appSt}} {
if lvl.st.Errors != nil && ValidLogLevel(lvl.st.Errors.LogLevel) {
r.LogLevel, r.LogLevelFrom = lvl.st.Errors.LogLevel, lvl.from
}
if lvl.st.Analytics != nil {
if c := lvl.st.Analytics.Count; c == CountEdge || c == CountOff {
r.Count, r.CountFrom = c, lvl.from
}
if d := lvl.st.Analytics.Door; d == DoorOn || d == DoorOff {
r.Door, r.DoorFrom = d, lvl.from
}
}
}
return r, nil
}Check how DirStore.Put treats the ifMatch argument ("" = unconditional? or create-only?). Read internal/release/store.go around func (d *DirStore) Put and confirm that Put(ctx, key, body, "") on an existing key succeeds or is create-only; if create-only, Update must pass the etag read back (which it does) and for a first write the empty etag must mean create — verify with the test.
- [ ] Step 4: Run, PASS, commit
git add internal/telemetry
git commit -m "telemetry: the two-level settings object behind Errors and Analytics (spec §3.3)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>"---
Task 9: Console — Errors API, tab, and the log-level setting
Files:
- Create:
internal/console/errorsview.go,internal/console/apierrors.go,internal/console/templates/app_errors.html - Modify:
internal/console/server.go(routes near 1050 and 1071; page struct near 1370;handleAppErrors),internal/console/authcli.go:595(twocliRoutes),internal/console/templates/appnav.html,internal/console/templates/app_settings.html,internal/console/templates/account_settings.html - Test:
internal/console/apierrors_test.go
Interfaces:
- Consumes:
apperr.*,applog.ReadWindow,telemetry.Resolve/Update,s.instanceGate,s.appPage,s.accountApp,s.account,s.audit,s.setFlash,logsCollectCap,instanceStoreTimeout. - Produces: ``
go type errorsWindow struct { Groups []apperr.Group; Records []apperr.Record; Truncated bool; LogLevel, LogLevelFrom string; Sources []string } func (s *Server) readErrors(ctx, sqid, app string, f apperr.Filter, level string) (errorsWindow, error) type errorsResponse struct { Groups []apperr.Groupjson:"groups"; Records []apperr.Recordjson:"records,omitempty"; Truncated booljson:"truncated"; LogLevel stringjson:"loglevel"; Sources []stringjson:"sources"}`Routes:GET /api/cli/apps/{account}/{app}/errors(Instance, Operate);PUT /api/cli/apps/{account}/{app}/telemetry(Operate) body{"errors":{"loglevel":"warn"}};GET /a/{sqid}/apps/{app}/errors;POST /a/{sqid}/apps/{app}/errors/level;POST /a/{sqid}/errors/level`.
- [ ] Step 1: Failing tests —
internal/console/apierrors_test.go
package console
import (
"context"
"encoding/json"
"net/http"
"strings"
"testing"
"time"
"github.com/carlosframework/platform/internal/apperr"
"github.com/carlosframework/platform/internal/applog"
"github.com/carlosframework/platform/internal/release"
"github.com/carlosframework/platform/internal/telemetry"
)
func errorsURL(fx *consoleFixture, acct, app, query string) string {
u := fx.ts.URL + "/api/cli/apps/" + acct + "/" + app + "/errors"
if query != "" {
u += "?" + query
}
return u
}
// seedErrors ships one coalesced edge error and, via seedLogChunks, one
// error-level platform line and one warn-level app line.
func seedErrors(t *testing.T, fx *consoleFixture, now time.Time) {
t.Helper()
seedLogChunks(t, fx, now)
warn := []applog.Record{{TS: now.Add(-7 * time.Minute), Box: "box1", Host: "titogo.example.com",
Stream: applog.StreamApp, Level: applog.LevelWarn, Msg: "slow query 4123ms", Seq: 150}}
body, _ := applog.EncodeChunk(warn)
if _, err := fx.rel.Put(context.Background(), applog.ChunkKey(release.OpsAccount, "titogo", "box1", applog.HourOf(warn[0].TS), 150, len(body)), body, ""); err != nil {
t.Fatal(err)
}
e := []apperr.Record{{TS: now.Add(-3 * time.Minute), Box: "box1", Host: "titogo.example.com",
Source: apperr.SourceEdge, Kind: apperr.KindHTTP, Name: "500 POST /checkout", Path: "/checkout", Status: 500,
Release: "v3", FP: "edge500", FPV: 1, Count: 12, First: now.Add(-9 * time.Minute), Last: now.Add(-3 * time.Minute), Seq: 200}}
eb, _ := apperr.EncodeChunk(e)
if _, err := fx.rel.Put(context.Background(), apperr.ChunkKey(release.OpsAccount, "titogo", "box1", apperr.HourOf(e[0].TS), 200, len(eb)), eb, ""); err != nil {
t.Fatal(err)
}
}
func TestAPIErrorsGroupsBothFamilies(t *testing.T) {
fx, token := instanceFixture(t, "code-errors-1")
now := time.Now().UTC()
seedErrors(t, fx, now)
get := func(query string) errorsResponse {
t.Helper()
status, _, body := rawReq(t, http.MethodGet, errorsURL(fx, fx.sqid, "titogo", query), token, "")
if status != http.StatusOK {
t.Fatalf("GET ?%s = %d: %s", query, status, body)
}
var resp errorsResponse
if err := json.Unmarshal([]byte(body), &resp); err != nil {
t.Fatal(err)
}
return resp
}
resp := get("")
if len(resp.Groups) != 2 {
t.Fatalf("groups = %+v", resp.Groups)
}
if resp.Groups[0].FP != "edge500" || resp.Groups[0].Count != 12 || resp.Groups[0].Releases[0] != "v3" {
t.Fatalf("newest first, coalesced count: %+v", resp.Groups[0])
}
if resp.Groups[1].Source != apperr.SourceLog || !strings.Contains(resp.Groups[1].Name, "wake failed") {
t.Fatalf("log-derived group: %+v", resp.Groups[1])
}
if resp.LogLevel != "error" || len(resp.Sources) != 2 {
t.Fatalf("meta = %q %v", resp.LogLevel, resp.Sources)
}
// A warn read widens without changing the setting.
if resp = get("level=warn"); len(resp.Groups) != 3 {
t.Fatalf("level=warn groups = %d", len(resp.Groups))
}
// Source and fp filters.
if resp = get("source=edge"); len(resp.Groups) != 1 || resp.Groups[0].FP != "edge500" {
t.Fatalf("source filter = %+v", resp.Groups)
}
if resp = get("fp=edge500"); len(resp.Records) != 1 || resp.Records[0].Status != 500 {
t.Fatalf("fp expand = %+v", resp.Records)
}
}
func TestAPITelemetryPutSetsLogLevel(t *testing.T) {
fx, token := instanceFixture(t, "code-errors-2")
now := time.Now().UTC()
seedErrors(t, fx, now)
// The instance token may read but not write settings: Operate only.
status, _, body := rawReq(t, http.MethodPut, fx.ts.URL+"/api/cli/apps/"+fx.sqid+"/titogo/telemetry", token, `{"errors":{"log_level":"warn"}}`)
if status != http.StatusForbidden && status != http.StatusNotFound {
t.Fatalf("instance token PUT = %d %s", status, body)
}
op := serviceToken(t, fx.st, OpsAccountID, "ops", "titogo", svcrole.Operate)
status, _, body = rawReq(t, http.MethodPut, fx.ts.URL+"/api/cli/apps/"+fx.sqid+"/titogo/telemetry", op, `{"errors":{"log_level":"warn"}}`)
if status != http.StatusOK {
t.Fatalf("PUT = %d %s", status, body)
}
r, _ := telemetry.Resolve(context.Background(), fx.rel, fx.sqid, "titogo")
if r.LogLevel != "warn" || r.LogLevelFrom != "app" {
t.Fatalf("resolved = %+v", r)
}
status, _, body = rawReq(t, http.MethodGet, errorsURL(fx, fx.sqid, "titogo", ""), token, "")
var resp errorsResponse
json.Unmarshal([]byte(body), &resp)
if status != 200 || resp.LogLevel != "warn" || len(resp.Groups) != 3 {
t.Fatalf("after set: %d level=%q groups=%d", status, resp.LogLevel, len(resp.Groups))
}
status, _, body = rawReq(t, http.MethodPut, fx.ts.URL+"/api/cli/apps/"+fx.sqid+"/titogo/telemetry", op, `{"errors":{"log_level":"loud"}}`)
if status != http.StatusBadRequest {
t.Fatalf("bad level = %d %s", status, body)
}
}
func TestAppErrorsPage(t *testing.T) {
fx, _ := instanceFixture(t, "code-errors-page")
now := time.Now().UTC()
seedErrors(t, fx, now)
fx.setUser("paul@keymail.dev")
c := newBrowser(t)
signUp(t, fx, c, "code-errors-signin")
status, _, body := get(t, c, fx.u("/apps/titogo/errors"))
if status != http.StatusOK {
t.Fatalf("errors page: %d %s", status, body)
}
for _, want := range []string{"500 POST /checkout", "wake failed", "carlos errors --app titogo", "Sources live: log, edge", "to the nearest flush"} {
if !strings.Contains(body, want) {
t.Errorf("page missing %q", want)
}
}
if strings.Contains(body, "slow query") {
t.Error("warn lines must not show at the default level")
}
status, _, body = get(t, c, fx.u("/apps/titogo/errors?level=warn"))
if status != 200 || !strings.Contains(body, "slow query") {
t.Fatalf("level=warn page: %d", status)
}
status, _, body = get(t, c, fx.u("/apps/titogo/errors?fp=edge500"))
if status != 200 || !strings.Contains(body, "/checkout") || !strings.Contains(body, "v3") {
t.Fatalf("expanded group: %d", status)
}
status, _, body = get(t, c, fx.u("/apps/titogo/errors?since=1ns"))
if status != 200 || !strings.Contains(body, "No errors in this window") {
t.Fatalf("empty window: %d", status)
}
}operateToken(t, fx): find how existing tests mint a service token with the Operate role (grep svcrole.Operate in internal/console/*_test.go; there is a helper minting tokens with a role list for the apicli tests). Use that helper; if its name differs, use it directly in place of operateToken. If no such helper exists, mint through the same store call instanceFixture uses with svcrole.Operate substituted.
- [ ] Step 2: Run, expect compile failure
- [ ] Step 3:
errorsview.go
package console
// The Errors tab's read (spec 2026-09-07 §3.2, §3.6): errors/ records
// plus log lines at or above the resolved level, derived on read and
// never stored twice, grouped by fingerprint. The tab and the CLI's API
// share this function — the API is not an afterthought.
import (
"context"
"regexp"
"sort"
"time"
"github.com/carlosframework/platform/internal/apperr"
"github.com/carlosframework/platform/internal/applog"
"github.com/carlosframework/platform/internal/telemetry"
)
// errorsWindow is one assembled read.
type errorsWindow struct {
Groups []apperr.Group
Records []apperr.Record // when f.FP is set: that group's records, oldest first
Truncated bool
LogLevel string // resolved setting
LogLevelFrom string
Sources []string // sources with at least one record in the window
}
// readErrors assembles the window. level overrides the resolved setting
// for THIS read only ("" = use the setting); it is what lets a member on
// error widen one look to warn without changing anything (§8 ruling 3).
func (s *Server) readErrors(ctx context.Context, sqid, app string, f apperr.Filter, level string) (errorsWindow, error) {
var out errorsWindow
res, err := telemetry.Resolve(ctx, s.cfg.Releases, sqid, app)
if err != nil {
return out, err
}
out.LogLevel, out.LogLevelFrom = res.LogLevel, res.LogLevelFrom
if level == "" {
level = res.LogLevel
}
stored, trimmed, err := apperr.ReadWindow(ctx, s.cfg.Releases, sqid, app, f, logsCollectCap)
if err != nil {
return out, err
}
out.Truncated = trimmed
var derived []apperr.Record
if f.Source == "" || f.Source == apperr.SourceLog {
lf := applog.Filter{Since: f.Since, Until: f.Until, Host: f.Host, Level: level}
lines, ltrim, err := applog.ReadWindow(ctx, s.cfg.Releases, sqid, app, lf, logsCollectCap)
if err != nil {
return out, err
}
out.Truncated = out.Truncated || ltrim
for _, l := range lines {
r := apperr.FromLog(l)
if (f.FP == "" || r.FP == f.FP) && (f.Release == "" || r.Release == f.Release) {
derived = append(derived, r)
}
}
}
all := apperr.MergeRecords(stored, derived)
if f.FP != "" {
out.Records = all
}
out.Groups = apperr.GroupRecords(all)
seen := map[string]bool{}
for _, r := range all {
if !seen[r.Source] {
seen[r.Source] = true
out.Sources = append(out.Sources, r.Source)
}
}
sort.Strings(out.Sources)
return out, nil
}
// errorsFilterFromQuery reads the tab's and the API's shared params:
// since/until/source/host/release/fp/level. The empty badParam means they
// parsed.
func errorsFilterFromQuery(q map[string][]string, now time.Time) (f apperr.Filter, level string, badParam string) {
get := func(k string) string {
if v := q[k]; len(v) > 0 {
return v[0]
}
return ""
}
var err error
if f.Since, err = applog.ParseTimeArg(get("since"), now); err != nil {
return f, "", "since: " + err.Error()
}
if f.Until, err = applog.ParseTimeArg(get("until"), now); err != nil {
return f, "", "until: " + err.Error()
}
if f.Since.IsZero() {
f.Since = now.Add(-24 * time.Hour)
}
switch v := get("source"); v {
case "", apperr.SourceLog, apperr.SourceEdge, apperr.SourceServer, apperr.SourceBrowser:
f.Source = v
default:
return f, "", "source: want log, edge, server or browser"
}
f.Host, f.Release, f.FP = get("host"), get("release"), get("fp")
if !validFP.MatchString(f.FP) {
return f, "", "fp: want a fingerprint id"
}
if level = get("level"); level != "" && !telemetry.ValidLogLevel(level) {
return f, "", "level: want error or warn"
}
return f, level, ""
}
var validFP = regexp.MustCompile(`^[A-Za-z0-9_.-]{0,64}$`)- [ ] Step 4:
apierrors.go
package console
import (
"context"
"encoding/json"
"log"
"net/http"
"time"
"github.com/carlosframework/platform/internal/apperr"
"github.com/carlosframework/platform/internal/telemetry"
)
// errorsResponse is the wire shape `carlos errors` and the tab read.
type errorsResponse struct {
Groups []apperr.Group `json:"groups"`
Records []apperr.Record `json:"records,omitempty"`
Truncated bool `json:"truncated"`
LogLevel string `json:"log_level"`
Sources []string `json:"sources"`
}
// handleAPIErrors answers `carlos errors`: the instance family's gate
// (an app token may read its own errors), the shared filter, one read.
func (s *Server) handleAPIErrors(w http.ResponseWriter, r *http.Request) {
_, acct, app, ok := s.instanceGate(w, r)
if !ok {
return
}
f, level, bad := errorsFilterFromQuery(r.URL.Query(), time.Now())
if bad != "" {
jsonError(w, http.StatusBadRequest, bad)
return
}
ctx, cancel := context.WithTimeout(r.Context(), instanceStoreTimeout)
defer cancel()
win, err := s.readErrors(ctx, acct.Sqid, app, f, level)
if err != nil {
log.Printf("console: api errors %s/%s: %v", acct.Sqid, app, err)
jsonError(w, http.StatusBadGateway, "could not read the errors")
return
}
resp := errorsResponse{Groups: win.Groups, Records: win.Records, Truncated: win.Truncated, LogLevel: win.LogLevel, Sources: win.Sources}
if resp.Groups == nil {
resp.Groups = []apperr.Group{}
}
if resp.Sources == nil {
resp.Sources = []string{}
}
writeJSON(w, http.StatusOK, resp)
}
// handleAPITelemetryPut sets the app-level telemetry object: today the
// errors log level; the analytics keys arrive with step 2. Operate only —
// an app's own token must not widen what its owner sees.
func (s *Server) handleAPITelemetryPut(w http.ResponseWriter, r *http.Request) {
principal, acct, app, ok := s.instanceGate(w, r)
if !ok {
return
}
var body telemetry.Settings
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096)).Decode(&body); err != nil {
jsonError(w, http.StatusBadRequest, "body: want {\"errors\":{\"log_level\":\"error|warn\"}}")
return
}
if body.Errors == nil || !telemetry.ValidLogLevel(body.Errors.LogLevel) {
jsonError(w, http.StatusBadRequest, "errors.log_level: want error or warn")
return
}
if body.Analytics != nil {
jsonError(w, http.StatusBadRequest, "analytics: not settable yet")
return
}
ctx, cancel := context.WithTimeout(r.Context(), instanceStoreTimeout)
defer cancel()
if err := telemetry.Update(ctx, s.cfg.Releases, telemetry.AppKey(acct.Sqid, app), func(st *telemetry.Settings) {
st.Errors = &telemetry.Errors{LogLevel: body.Errors.LogLevel}
}); err != nil {
log.Printf("console: api telemetry %s/%s: %v", acct.Sqid, app, err)
jsonError(w, http.StatusBadGateway, "could not write the setting")
return
}
s.audit(acct, app, principal, "errors-level", "set the Errors log level to "+body.Errors.LogLevel)
writeJSON(w, http.StatusOK, map[string]string{"log_level": body.Errors.LogLevel})
}principal is the first return of instanceGate (a string naming the token's principal — check the type; if it is not a string suitable for audit's addr, pass what the logs/instances handlers pass).
Register in authcli.go beside the logs route:
s.cliRoute("GET /api/cli/apps/{account}/{app}/errors", svcrole.Allow(svcrole.Instance, svcrole.Operate), s.handleAPIErrors)
s.cliRoute("PUT /api/cli/apps/{account}/{app}/telemetry", svcrole.Allow(svcrole.Operate), s.handleAPITelemetryPut)- [ ] Step 5: Page handler, fields, routes, templates
Page struct (near line 1370), add:
// The Errors tab (errorsview.go).
ErrWin errorsWindow
ErrSince string // "1h" | "24h" | "168h"
ErrSource string
ErrLevel string // "" = the setting
ErrHost string
ErrFP string // expanded group
ErrsErr stringRoutes (near 1050 and 1071):
s.mux.HandleFunc("GET /a/{sqid}/apps/{app}/errors", s.handleAppErrors)
s.mux.HandleFunc("POST /a/{sqid}/apps/{app}/errors/level", s.handleAppErrorLevel)
s.mux.HandleFunc("POST /a/{sqid}/errors/level", s.handleAccountErrorLevel)Handlers (in errorsview.go):
// handleAppErrors renders the Errors tab: the same read the API answers,
// with the filters as GET params so a view is a URL.
func (s *Server) handleAppErrors(w http.ResponseWriter, r *http.Request) {
p, acct, ok := s.appPage(w, r, "errors")
if !ok {
return
}
q := r.URL.Query()
p.ErrSince = q.Get("since")
if p.ErrSince != "1h" && p.ErrSince != "168h" {
p.ErrSince = "24h"
}
q.Set("since", p.ErrSince)
f, level, bad := errorsFilterFromQuery(q, time.Now())
if bad != "" {
f, level, _ = errorsFilterFromQuery(map[string][]string{"since": {p.ErrSince}}, time.Now())
}
p.ErrSource, p.ErrLevel, p.ErrHost, p.ErrFP = f.Source, level, f.Host, f.FP
if s.cfg.Releases != nil {
ctx, cancel := context.WithTimeout(r.Context(), instanceStoreTimeout)
defer cancel()
win, err := s.readErrors(ctx, acct.Sqid, p.AppName, f, level)
if err != nil {
log.Printf("console: app errors %s/%s: %v", acct.Sqid, p.AppName, err)
p.ErrsErr = "The errors could not be read just now — try again in a moment."
}
p.ErrWin = win
}
s.render(w, http.StatusOK, "app_errors", p)
}
// handleAppErrorLevel is the Settings tab's form: owner, CSRF, one key.
func (s *Server) handleAppErrorLevel(w http.ResponseWriter, r *http.Request) {
addr, raw, acct, role, app, ok := s.accountApp(w, r)
if !ok {
return
}
if subtle.ConstantTimeCompare([]byte(r.FormValue("csrf")), []byte(raw)) != 1 {
s.renderMessage(w, r, http.StatusForbidden, "Refused", "Cross-site request blocked.")
return
}
if role != RoleOwner {
http.NotFound(w, r)
return
}
level := r.FormValue("level")
back := "/a/" + url.PathEscape(acct.Sqid) + "/apps/" + url.PathEscape(app) + "/settings"
if !telemetry.ValidLogLevel(level) {
s.setFlash(w, "err", "Choose error or warn.")
http.Redirect(w, r, back, http.StatusSeeOther)
return
}
if err := telemetry.Update(r.Context(), s.cfg.Releases, telemetry.AppKey(acct.Sqid, app), func(st *telemetry.Settings) {
st.Errors = &telemetry.Errors{LogLevel: level}
}); err != nil {
log.Printf("console: errors level %s/%s: %v", acct.Sqid, app, err)
s.setFlash(w, "err", "The setting could not be saved — try again in a moment.")
http.Redirect(w, r, back, http.StatusSeeOther)
return
}
s.audit(acct, app, addr, "errors-level", "set the Errors log level to "+level)
s.setFlash(w, "ok", "Errors now include "+level+" lines and above for this app.")
http.Redirect(w, r, back, http.StatusSeeOther)
}
// handleAccountErrorLevel is the account settings form: the default every
// app without its own setting inherits.
func (s *Server) handleAccountErrorLevel(w http.ResponseWriter, r *http.Request) {
addr, raw, acct, role, ok := s.account(w, r)
if !ok {
return
}
if subtle.ConstantTimeCompare([]byte(r.FormValue("csrf")), []byte(raw)) != 1 {
s.renderMessage(w, r, http.StatusForbidden, "Refused", "Cross-site request blocked.")
return
}
if role != RoleOwner {
http.NotFound(w, r)
return
}
level := r.FormValue("level")
back := "/a/" + url.PathEscape(acct.Sqid) + "/settings"
if !telemetry.ValidLogLevel(level) {
s.setFlash(w, "err", "Choose error or warn.")
http.Redirect(w, r, back, http.StatusSeeOther)
return
}
if err := telemetry.Update(r.Context(), s.cfg.Releases, telemetry.AccountKey(acct.Sqid), func(st *telemetry.Settings) {
st.Errors = &telemetry.Errors{LogLevel: level}
}); err != nil {
log.Printf("console: account errors level %s: %v", acct.Sqid, err)
s.setFlash(w, "err", "The setting could not be saved — try again in a moment.")
http.Redirect(w, r, back, http.StatusSeeOther)
return
}
s.audit(acct, "", addr, "errors-level", "set the account's Errors log level to "+level)
s.setFlash(w, "ok", "Errors now include "+level+" lines and above for every app without its own setting.")
http.Redirect(w, r, back, http.StatusSeeOther)
}Confirm s.account's return arity by reading handleAccountSettings (addr, raw, acct, role, ok). Add p.ErrLevelSetting telemetry.Resolved to the page struct and populate it in handleAppSettings (p.ErrLevelSetting, _ = telemetry.Resolve(ctx, s.cfg.Releases, acct.Sqid, app)) and in handleAccountSettings (Resolve(ctx, s.cfg.Releases, acct.Sqid, "") — with an empty app name AppKey reads a key that does not exist, which returns defaults; that is acceptable for the account page and should be noted in a comment).
appnav.html: insert after the Logs link:
<a href="{{.Nav}}/apps/{{.AppName}}/errors"{{if eq .ActiveTab "errors"}} aria-current="page"{{end}}>Errors</a>app_errors.html:
{{define "app_errors"}}{{template "top" .}}
{{if .Flash}}<p class="banner">{{.Flash}}</p>{{end}}
{{if .FlashErr}}<p class="banner error">{{.FlashErr}}</p>{{end}}
{{template "appnav" .}}
<h2>Errors</h2>
{{if .ErrsErr}}<p class="banner error">{{.ErrsErr}}</p>{{end}}
<form method="get" class="small">
<label>from the last
<select name="since">
<option value="1h"{{if eq .ErrSince "1h"}} selected{{end}}>hour</option>
<option value="24h"{{if eq .ErrSince "24h"}} selected{{end}}>day</option>
<option value="168h"{{if eq .ErrSince "168h"}} selected{{end}}>week</option>
</select>
</label>
<label>source
<select name="source">
<option value=""{{if eq .ErrSource ""}} selected{{end}}>all</option>
<option value="log"{{if eq .ErrSource "log"}} selected{{end}}>log</option>
<option value="edge"{{if eq .ErrSource "edge"}} selected{{end}}>edge</option>
<option value="server"{{if eq .ErrSource "server"}} selected{{end}}>server</option>
<option value="browser"{{if eq .ErrSource "browser"}} selected{{end}}>browser</option>
</select>
</label>
<label>log level
<select name="level">
<option value=""{{if eq .ErrLevel ""}} selected{{end}}>setting ({{.ErrWin.LogLevel}})</option>
<option value="error"{{if eq .ErrLevel "error"}} selected{{end}}>error</option>
<option value="warn"{{if eq .ErrLevel "warn"}} selected{{end}}>warn and above</option>
</select>
</label>
<label>host <input name="host" value="{{.ErrHost}}" size="24"></label>
<button type="submit">Update</button>
</form>
<p class="small">Sources live: {{if .ErrWin.Sources}}{{range $i, $s := .ErrWin.Sources}}{{if $i}}, {{end}}{{$s}}{{end}}{{else}}none yet{{end}}.
Counts are to the nearest flush. Messages are redacted for addresses and credentials before they are stored; treat that as a floor, not a guarantee.</p>
{{if .ErrFP}}
{{with .ErrWin.Records}}
<h3>Occurrences of <code>{{$.ErrFP}}</code></h3>
<p class="small"><a href="?since={{$.ErrSince}}">All groups</a></p>
{{range .}}
<div class="tablewrap"><table>
<tr><th>Last</th><td>{{.Last.UTC.Format "2006-01-02 15:04:05"}}</td></tr>
<tr><th>Count</th><td>{{.Count}}</td></tr>
<tr><th>Source</th><td>{{.Source}}{{if .Kind}} / {{.Kind}}{{end}}</td></tr>
<tr><th>Host</th><td>{{.Host}}</td></tr>
{{if .Release}}<tr><th>Release</th><td>{{.Release}}</td></tr>{{end}}
{{if .Path}}<tr><th>Path</th><td><code>{{.Path}}</code>{{if .Status}} → {{.Status}}{{end}}</td></tr>{{end}}
<tr><th>Message</th><td>{{.Name}}</td></tr>
{{if .Stack}}<tr><th>Stack</th><td><pre>{{.Stack}}</pre></td></tr>{{end}}
{{if .Fields}}<tr><th>Fields</th><td>{{range $k, $v := .Fields}}<code>{{$k}}={{$v}}</code> {{end}}</td></tr>{{end}}
<tr><th>Logs</th><td><a href="{{$.Nav}}/apps/{{$.AppName}}/logs?since=1h&host={{.Host}}">around this time</a></td></tr>
</table></div>
{{end}}
{{else}}<p>No occurrences of this group in the window.</p>{{end}}
{{else if .ErrWin.Groups}}
{{if .ErrWin.Truncated}}<p class="small">Showing the most recent 50,000 records of a larger window — counts are within what was read.</p>{{end}}
<div class="tablewrap">
<table>
<tr><th>Last seen</th><th>Count</th><th>Source</th><th>Error</th><th>Hosts</th><th>Releases</th></tr>
{{range .ErrWin.Groups}}<tr>
<td>{{.Last.UTC.Format "2006-01-02 15:04"}}</td>
<td>{{.Count}}{{if eq .FP "_overflow"}} <span class="small" title="more kinds than the hourly cap; occurrences">(overflow)</span>{{end}}</td>
<td>{{.Source}}{{if .Level}} <span class="small">{{.Level}}</span>{{end}}</td>
<td><a href="?since={{$.ErrSince}}&fp={{.FP}}{{if $.ErrLevel}}&level={{$.ErrLevel}}{{end}}">{{.Name}}</a></td>
<td>{{range $i, $h := .Hosts}}{{if $i}}, {{end}}{{$h}}{{end}}</td>
<td>{{range $i, $r := .Releases}}{{if $i}}, {{end}}<code>{{$r}}</code>{{end}}</td>
</tr>{{end}}
</table>
</div>
<p class="small">Times are UTC. The same view from a terminal: <code>carlos errors --app {{.AppName}} --since {{.ErrSince}}</code>.</p>
{{else if not .ErrsErr}}
<p>No errors in this window.</p>
<p class="small">Log lines at <code>{{.ErrWin.LogLevel}}</code> and above and responses of 500 or higher appear here on their own; a JavaScript or server-side error needs the hook (coming in a later step). The level is set on the Settings tab. The same view from a terminal: <code>carlos errors --app {{.AppName}}</code>.</p>
{{end}}
{{template "bottom" .}}{{end}}app_settings.html: before the Feature flags section add:
<h2>Errors</h2>
<form method="post" action="{{.Base}}/apps/{{.Detail.App}}/errors/level" class="settings-row">
<input type="hidden" name="csrf" value="{{.CSRF}}">
<label>Log lines counted as errors:
<select name="level">
<option value="error"{{if eq .ErrLevelSetting.LogLevel "error"}} selected{{end}}>error and above</option>
<option value="warn"{{if eq .ErrLevelSetting.LogLevel "warn"}} selected{{end}}>warn and above</option>
</select>
</label>
<button type="submit">Save</button>
<span class="small">currently {{.ErrLevelSetting.LogLevel}}, from the {{.ErrLevelSetting.LogLevelFrom}}</span>
</form>
<p class="small">The Errors tab always lets you widen one look to warn without changing this.</p>Check the CSRF field name and value the rename form uses ({{.CSRF}}, {{.Raw}} or similar) and match it exactly.
account_settings.html: before "Delete this account" add the same form posting to {{.Base}}/errors/level, with the copy "for every app without its own setting".
- [ ] Step 6: Run console tests
Run: go build ./... && go vet ./internal/console/ && go test ./internal/console/ -count=1 Expected: PASS (the whole package; template parse errors surface at server construction in every test).
- [ ] Step 7: Commit
git add internal/console
git commit -m "console: Errors tab, errors API, and the log-level setting at both levels (spec §3.6)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>"---
Task 10: carlos errors and the docs heading
Files:
- Create:
errors_cmd.go - Modify:
apimode.go(besidelogsGet),main.go:80(dispatch) and the help text near line 204 - Modify:
docs/site/cli.mdafter### carlos logs - Test:
errors_cmd_test.go(format only) and the docsite gate
Interfaces:
- Consumes:
requireAPISession,resolvedAccount,resolvedApp,appFlag,flagUsage,apiPath,getJSON,putJSON(check its name;apimode.gohas a JSON PUT/POST helper used byfeatures set),colorFor,mutedf,stdout. - Produces:
runErrors(args []string);(*apiSession).errorsGet(ctx, a apiAcct, app string, q url.Values) (apiErrorsResponse, error);(*apiSession).telemetryPut(ctx, a apiAcct, app string, body any) error;printErrorGroups(w io.Writer, groups []apperr.Group);printErrorRecords(w io.Writer, recs []apperr.Record).
- [ ] Step 1: Failing format test —
errors_cmd_test.go
package main
import (
"bytes"
"strings"
"testing"
"time"
"github.com/carlosframework/platform/internal/apperr"
)
func TestPrintErrorGroups(t *testing.T) {
now := time.Date(2026, 9, 7, 10, 0, 0, 0, time.UTC)
var buf bytes.Buffer
printErrorGroups(&buf, []apperr.Group{
{FP: "abc123", Source: "edge", Name: "500 POST /checkout", Count: 12, Last: now, Hosts: []string{"a.example"}, Releases: []string{"v3"}},
{FP: "_overflow", Source: "edge", Name: "500 GET /z", Count: 40, Last: now},
})
out := buf.String()
for _, want := range []string{"2026-09-07 10:00", "12", "edge", "500 POST /checkout", "abc123", "a.example", "v3", "overflow"} {
if !strings.Contains(out, want) {
t.Errorf("output missing %q:\n%s", want, out)
}
}
}- [ ] Step 2: Implement
errors_cmd.go
// `carlos errors` — an app's error groups with no box access (spec
// 2026-09-07-errors-and-analytics-design.md §3.6): log lines at the
// configured level and edge-observed 5xx, coalesced by fingerprint, read
// through the console API. API mode only, like `carlos logs`.
package main
import (
"context"
"flag"
"fmt"
"io"
"log"
"net/url"
"sort"
"strconv"
"github.com/carlosframework/platform/internal/apperr"
)
const errorsUsage = `usage: carlos errors --app <a> [flags]
Lists error groups, newest first: one row per fingerprint with its count in
the window, the source it came from (log, edge, server, browser), the hosts
and releases it was seen on. --fp <id> expands one group to its occurrences.
--set-level error|warn changes which log lines count for this app.
`
func runErrors(args []string) {
fs := flag.NewFlagSet("errors", flag.ExitOnError)
app := appFlag(fs, "app whose errors to read")
since := fs.String("since", "24h", "window start: duration back from now, or RFC3339")
until := fs.String("until", "", "window end (default: now)")
source := fs.String("source", "", "only one source: log | edge | server | browser")
level := fs.String("level", "", "log lines at this level and above for this read: error | warn (default: the app's setting)")
host := fs.String("host", "", "only one instance (route host)")
rel := fs.String("release", "", "only one release")
fp := fs.String("fp", "", "expand one group: its occurrences, oldest first")
setLevel := fs.String("set-level", "", "set the app's log level for errors: error | warn")
consoleFlag := fs.String("console", "", "console to read through (default: the only logged-in one)")
acctRaw := fs.String("account", "", "account (sqid or name) the app belongs to (default: your only account, or the configured/chosen one)")
acctIDRaw := fs.String("account-id", "", "account sqid, matched as a sqid ONLY — supersedes --account")
fs.Usage = flagUsage(fs, errorsUsage)
fs.Parse(args)
sess := requireAPISession("errors", *consoleFlag)
acct := resolvedAccount(sess, acctRaw, acctIDRaw)
app2 := resolvedApp(sess, acct, *app, errorsUsage)
ctx := context.Background()
if *setLevel != "" {
if *setLevel != "error" && *setLevel != "warn" {
log.Fatalf("errors: --set-level wants error or warn")
}
if err := sess.telemetryPut(ctx, acct, app2, map[string]any{"errors": map[string]string{"log_level": *setLevel}}); err != nil {
log.Fatalf("errors: %v", err)
}
fmt.Fprintf(stdout, "Errors for %s now include %s lines and above.\n", app2, *setLevel)
return
}
q := url.Values{}
for k, v := range map[string]string{"since": *since, "until": *until, "source": *source, "level": *level, "host": *host, "release": *rel, "fp": *fp} {
if v != "" {
q.Set(k, v)
}
}
res, err := sess.errorsGet(ctx, acct, app2, q)
if err != nil {
log.Fatalf("errors: %v", err)
}
if *fp != "" {
printErrorRecords(stdout, res.Records)
if len(res.Records) == 0 {
mutedf(stdout, "no occurrences of %s in this window", *fp)
}
return
}
printErrorGroups(stdout, res.Groups)
if len(res.Groups) == 0 {
mutedf(stdout, "no errors in this window (log level %s; sources live: %s) — try --since 168h or --level warn", res.LogLevel, joinOr(res.Sources, "none yet"))
} else {
mutedf(stdout, "log level %s; sources live: %s; counts are to the nearest flush", res.LogLevel, joinOr(res.Sources, "none"))
}
if res.Truncated {
mutedf(stdout, "(the window held more than could be read; counts are within what was read — narrow it)")
}
}
func joinOr(xs []string, empty string) string {
if len(xs) == 0 {
return empty
}
out := xs[0]
for _, x := range xs[1:] {
out += ", " + x
}
return out
}
// printErrorGroups renders one row per group: last seen, count, source,
// message, then the fingerprint, hosts and releases dimmed.
func printErrorGroups(w io.Writer, groups []apperr.Group) {
c := colorFor(w)
for _, g := range groups {
count := strconv.Itoa(g.Count)
if g.FP == apperr.OverflowFP {
count += " (overflow)"
}
line := c.muted(g.Last.UTC().Format("2006-01-02 15:04")) + " " + c.head(fmt.Sprintf("%6s", count)) + " " +
c.muted(fmt.Sprintf("%-7s", g.Source)) + " " + c.bad(g.Name)
line += "\n" + " " + c.muted("fp="+g.FP)
if len(g.Hosts) > 0 {
line += c.muted(" hosts=" + joinOr(g.Hosts, ""))
}
if len(g.Releases) > 0 {
line += c.muted(" releases=" + joinOr(g.Releases, ""))
}
fmt.Fprintln(w, line)
}
}
// printErrorRecords renders one group's occurrences, oldest first.
func printErrorRecords(w io.Writer, recs []apperr.Record) {
c := colorFor(w)
for _, r := range recs {
fmt.Fprintf(w, "%s %s %s %s\n", c.muted(r.Last.UTC().Format("2006-01-02 15:04:05")),
c.head(fmt.Sprintf("x%d", r.Count)), c.muted(r.Host), c.bad(r.Name))
if r.Path != "" {
fmt.Fprintf(w, " %s\n", c.muted(fmt.Sprintf("%s → %d", r.Path, r.Status)))
}
if r.Release != "" {
fmt.Fprintf(w, " %s\n", c.muted("release "+r.Release))
}
if r.Stack != "" {
fmt.Fprintf(w, " %s\n", r.Stack)
}
keys := make([]string, 0, len(r.Fields))
for k := range r.Fields {
keys = append(keys, k)
}
sort.Strings(keys)
for _, k := range keys {
fmt.Fprintf(w, " %s\n", c.muted(k+"="+r.Fields[k]))
}
}
}In apimode.go, beside logsGet:
type apiErrorsResponse struct {
Groups []apperr.Group `json:"groups"`
Records []apperr.Record `json:"records"`
Truncated bool `json:"truncated"`
LogLevel string `json:"log_level"`
Sources []string `json:"sources"`
}
func (s *apiSession) errorsGet(ctx context.Context, a apiAcct, app string, query url.Values) (apiErrorsResponse, error) {
path := apiPath(a, app, "/errors")
if enc := query.Encode(); enc != "" {
if strings.Contains(path, "?") {
path += "&" + enc
} else {
path += "?" + enc
}
}
var res apiErrorsResponse
status, body, err := s.getJSON(ctx, path)
if err != nil {
return res, err
}
if status != http.StatusOK {
return res, apiErrorFromBody(status, body)
}
if err := json.Unmarshal(body, &res); err != nil {
return res, fmt.Errorf("errors: unreadable response: %w", err)
}
return res, nil
}
func (s *apiSession) telemetryPut(ctx context.Context, a apiAcct, app string, body any) error {
status, resp, err := s.putJSON(ctx, apiPath(a, app, "/telemetry"), body)
if err != nil {
return err
}
if status != http.StatusOK {
return apiErrorFromBody(status, resp)
}
return nil
}Find the existing JSON-body request helper in apimode.go (grep func (s \*apiSession) .*JSON — postJSON/doJSON/sendJSON) and use its real name and signature.
main.go: add case "errors": runErrors(args) beside logs, and in the help block near line 204 a line: errors error groups for an app: log lines at the set level plus 5xx the edge saw.
docs/site/cli.md after the ### carlos logs section:
### carlos errors
One row per distinct error, newest first, with how many times it happened
in the window and where. Two sources need no wiring at all: log lines the
app tagged `error` (or `warn`, if you set the level) and responses of 500
or above that the edge saw. Server-side and browser hooks come later.
```sh
carlos errors --app hello --since 24h
carlos errors --app hello --fp 3f2a9c1b5e7d0a4c
carlos errors --app hello --set-level warn--fp expands one group to its occurrences. --level warn widens a single read without changing the setting; --set-level changes it for the app. The same view is the Errors tab on the app's dashboard.
- [ ] **Step 3: Run the CLI test, docsite gate, whole build**
Run: `go build ./... && go test . -run TestPrintErrorGroups -count=1 && go test ./internal/docsite/ -count=1`
Expected: PASS; the docsite gate passes with the new heading (and would fail without it — try removing the heading once to see the failure, then restore it).
- [ ] **Step 4: Commit**
```bash
git add errors_cmd.go errors_cmd_test.go apimode.go main.go docs/site/cli.md
git commit -m "carlos errors: the app's error groups from a terminal, and the log-level setting
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>"---
Task 11: Whole-tree gate, cross-compile, push
Files: none new.
- [ ] Step 1: Full gate
Run:
go build ./... && go vet ./... && gofmt -l . && go test ./... -count=1
GOOS=darwin GOARCH=arm64 go build ./... && GOOS=windows GOARCH=amd64 go build ./...Expected: all clean; gofmt -l . prints nothing.
- [ ] Step 2: Spec status line
In docs/superpowers/specs/2026-09-07-errors-and-analytics-design.md, change the **Status:** line to approved (Paul, 2026-09-07: "go ahead with that and plan, build review cycle overnight"); step 1 implemented on branch spec-errors-analytics, plan docs/superpowers/plans/2026-09-07-errors-step1-log-and-edge.md. Commit with the plan file:
git add docs/superpowers
git commit -m "spec: approved; step 1 plan
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>"- [ ] Step 3: Push and describe
git push origin spec-errors-analytics
amadan branch describe carlos/platform spec-errors-analytics -body - < <description file>
amadan ci status carlos/platform -branch spec-errors-analyticsDo not merge. The branch waits for Paul's review of the diff.
Self-review against the spec
- §3.1 shape: Task 3
Recordcarries every field butua/referer(browser/edge analytics fields; step 2/3). Sanitisation: Task 3/4. ✔ - §3.2 log source read-derived: Task 9
readErrors. Edge errors with activator suppression: Task 7. Fingerprint rules: Task 3. Coalescing cap per app-hour on(fp, host, release)with_overflow: Task 4. Storage family and consumer list (ChunkKeySize,HourNameOfDir,chunkPrune,aclWriteAppFamily): Tasks 1, 2, 6. ✔ - §3.3 observing writer, unconditional, status routes excluded, 1xx ignored, implicit 200, Content-Type latched: Task 7. Settings object and resolution: Task 8. ✔
- §3.6 tab (window 1h/24h/7d, source/level/host filters, expand, coverage line, "nearest flush", collect-cap disclosure), API paths and roles, CLI flags including
-level, settings at both levels, retention viachunkPrune: Tasks 6, 9, 10. TombstoneDELETE …/errors/{fp}is deferred to a follow-up branch and is called out in the branch description.-releasefilter on the CLI: Task 10 (--release). ✔ - §4 hibernation: nothing here calls the activator; Task 7's flag path is the only touch and it is a context value. ✔
- §6 tests: sanitisation, fingerprint, coalescing, log-derived rows, edge errors on the plain path, chunk family byte-identity, store door, observing writer, settings resolution, console gates (page 404 for non-member is inherited from
appPage; add onegetas a non-member inTestAppErrorsPageif the logs page test has an equivalent — mirror it), docsite. ✔ - Type consistency:
apperr.Filterfields (Since, Until, Host, Source, Release, FP) are used identically in Tasks 4 and 9;errorsResponsefields matchapiErrorsResponse;chunkPrune(ctx, s, fam, box, k, now)is called with that order in Tasks 6 and its tests. ✔