amadan
carlos / todomvc-bakeoff Public
fix: sum turns and time across result events; security-finding extraction; tokens --since
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
20 files changed,
+588
−6
builders/post-build.sh+2 −1builders/reviews/go-rastrillo.1.md+7 −0builders/reviews/go-rastrillo.2.md+1 −0builders/reviews/rust.1.md+3 −0builders/reviews/rust.2.md+3 −0builders/reviews/swift.1.md+3 −0builders/reviews/swift.2.md+1 −0builders/reviews/zig.1.md+3 −0builders/reviews/zig.2.md+1 −0builders/reviews/zig.3.md+1 −0builders/security-findings.py+10 −0harness/cmd/bakeoff/main.go+9 −2harness/harness_test.go+12 −0harness/testdata/claude-multi.jsonl+3 −0harness/tokens.go+10 −3results/go-rastrillo.json+104 −0results/ocaml.json+102 −0results/rust.json+105 −0results/swift.json+104 −0results/zig.json+104 −0
diff --git a/builders/post-build.sh b/builders/post-build.sh| index bc072dd..83b987d 100755 |
| --- a/builders/post-build.sh |
| +++ b/builders/post-build.sh |
| @@ -28,7 +28,8 @@ fi |
| builders/run-review.sh "$slug" 2 |
| last=2 |
| if ! bakeoff findings --slug "$slug" --file "builders/reviews/$slug.2.md"; then |
| - grep -E '^\s*[0-9]+\.\s+\[security\]' -A20 "builders/reviews/$slug.2.md" | awk '/^\s*[0-9]+\.\s+\[/{keep=/\[security\]/} keep' > "$TMPDIR/$slug.security.md" |
| + builders/security-findings.py "builders/reviews/$slug.2.md" > "$TMPDIR/$slug.security.md" |
| + [ -s "$TMPDIR/$slug.security.md" ] || { echo "$slug: could not extract security findings" >&2; exit 1; } |
| builders/run-fix.sh "$slug" security "$TMPDIR/$slug.security.md" |
| rm -f "$TMPDIR/$slug.security.md" |
| bakeoff tokens --slug "$slug" --phase security |
diff --git a/builders/reviews/go-rastrillo.1.md b/builders/reviews/go-rastrillo.1.md| new file mode 100644 |
| index 0000000..566c37d |
| --- /dev/null |
| +++ b/builders/reviews/go-rastrillo.1.md |
| @@ -0,0 +1,7 @@ |
| +1. [spec] `apps/go-rastrillo/internal/todo/origin.go:28`: Delegating to `csrf.Protect` lets `Sec-Fetch-Site: same-origin` or `none` override a foreign `Origin`. Such POSTs execute instead of returning the required 403. Compare the supplied Origin directly with the configured origin. |
| + |
| +2. [correctness] `apps/go-rastrillo/internal/todo/handlers.go:79`: Form parsing errors are ignored. An edit body exceeding the 1 MiB limit yields an empty title, so the existing todo is deleted and the response reports success. Reject parsing errors before changing data. |
| + |
| +3. [correctness] `apps/go-rastrillo/internal/todo/static/app.js:36`: An add response discards the current input unconditionally. Submit a todo, then type a replacement title while the request is pending: the response erases that new text. Clear the submitted value when submitting and preserve subsequent typing. |
| + |
| +4. [spec] `apps/go-rastrillo/cmd/todo/main.go:42`: Explicit listen addresses are never restricted to loopback. `--listen 0.0.0.0:8080` passes through to Rastrillo’s TCP listener and exposes the app on every interface, contrary to the brief. Reject addresses whose host is not `127.0.0.1`. |
| \ No newline at end of file |
diff --git a/builders/reviews/go-rastrillo.2.md b/builders/reviews/go-rastrillo.2.md| new file mode 100644 |
| index 0000000..c54809c |
| --- /dev/null |
| +++ b/builders/reviews/go-rastrillo.2.md |
| @@ -0,0 +1 @@ |
| +1. [correctness] `apps/go-rastrillo/internal/todo/static/app.js:42` — Every action response replaces the entire app, preserving only the new-todo input. With a request pending, double-click another todo and start editing: when the earlier response arrives, the editor disappears and its unsaved text is lost. Preserve the active editor’s value and focus across replacement, or defer replacement until editing finishes. |
| \ No newline at end of file |
diff --git a/builders/reviews/rust.1.md b/builders/reviews/rust.1.md| new file mode 100644 |
| index 0000000..a310c2a |
| --- /dev/null |
| +++ b/builders/reviews/rust.1.md |
| @@ -0,0 +1,3 @@ |
| +1. [correctness] [apps/rust/static/app.js:32](/home/paulca/amadan.net/carlos/todomvc-bakeoff/.worktrees/rust/apps/rust/static/app.js:32): Every action response replaces the entire list, discarding edits started while the request was pending. With a delayed response, toggle one todo, then edit another: when the toggle response arrives, the edit disappears without saving. Preserve the active edit and its draft across updates. |
| + |
| +2. [spec] [apps/rust/Makefile:33](/home/paulca/amadan.net/carlos/todomvc-bakeoff/.worktrees/rust/apps/rust/Makefile:33): `clean-deps` removes Cargo’s target directory but leaves Zig’s compiled libraries in its shared cache. `build-arm64` uses `cargo zigbuild` without configuring an app-local Zig cache, so subsequent cold builds can reuse compiled dependencies. Put Zig’s compiled cache under this app’s `.cache/` and remove it in `clean-deps`. |
| \ No newline at end of file |
diff --git a/builders/reviews/rust.2.md b/builders/reviews/rust.2.md| new file mode 100644 |
| index 0000000..661e316 |
| --- /dev/null |
| +++ b/builders/reviews/rust.2.md |
| @@ -0,0 +1,3 @@ |
| +1. [correctness] [apps/rust/static/app.js:10](/home/paulca/amadan.net/carlos/todomvc-bakeoff/.worktrees/rust/apps/rust/static/app.js:10): Filter navigation can discard pending changes. Requests wait in a promise queue, but filter links navigate immediately. With a toggle request pending, edit another todo and click a filter: blur queues the save, then navigation destroys the queue before that save reaches the server. Wait for pending actions before navigating. |
| + |
| +2. [spec] [apps/rust/src/main.rs:45](/home/paulca/amadan.net/carlos/todomvc-bakeoff/.worktrees/rust/apps/rust/src/main.rs:45): `--listen` accepts every loopback address, including `::1` and `127.0.0.2`. The brief explicitly restricts binding to `127.0.0.1`. Validate that exact address. |
| \ No newline at end of file |
diff --git a/builders/reviews/swift.1.md b/builders/reviews/swift.1.md| new file mode 100644 |
| index 0000000..07893fb |
| --- /dev/null |
| +++ b/builders/reviews/swift.1.md |
| @@ -0,0 +1,3 @@ |
| +1. [correctness] `apps/swift/static/app.js:10` and `:20` — Adding a todo can erase the next title. Submit “first”, then type “second” before the response arrives: the first response resets the input, deleting “second”. Pressing Enter meanwhile also silently discards that submission because `dataset.sending` is set. |
| + |
| +2. [correctness] `apps/swift/static/app.js:19` — Concurrent actions replace the list in response-arrival order, allowing an older snapshot to overwrite newer state. Toggle two different items quickly: if the earlier response arrives last, an item can appear unchecked and the counter and toggle-all state become stale despite both changes being saved. Serialize actions or prevent stale responses from replacing newer state. |
| \ No newline at end of file |
diff --git a/builders/reviews/swift.2.md b/builders/reviews/swift.2.md| new file mode 100644 |
| index 0000000..e4dc4d9 |
| --- /dev/null |
| +++ b/builders/reviews/swift.2.md |
| @@ -0,0 +1 @@ |
| +1. [correctness] [apps/swift/static/app.js:32](/home/paulca/amadan.net/carlos/todomvc-bakeoff/.worktrees/swift/apps/swift/static/app.js:32): Every action response removes the entire list, including any active editor. Toggle one item, then start editing another before the response arrives: the replacement discards the draft and focus without saving. Serializing requests does not protect unsent edits. Preserve active editor contents and focus across updates, or defer replacement until editing finishes. |
| \ No newline at end of file |
diff --git a/builders/reviews/zig.1.md b/builders/reviews/zig.1.md| new file mode 100644 |
| index 0000000..e70a7bc |
| --- /dev/null |
| +++ b/builders/reviews/zig.1.md |
| @@ -0,0 +1,3 @@ |
| +1. [correctness] `apps/zig/static/app.js:24–25`: Every response replaces the entire list, including any open editor. With a request pending, double-click another todo and start editing; when the earlier response arrives, the editor disappears and its unsaved text is lost. Preserve pending edits when applying responses. |
| + |
| +2. [spec] `apps/zig/src/app.zig:172` and `:185`: Adding and editing trim only ASCII whitespace. Unicode whitespace such as U+00A0 or U+2003 remains, so whitespace-only titles create items, and editing an item to whitespace does not delete it. Both handlers need Unicode whitespace trimming. |
| \ No newline at end of file |
diff --git a/builders/reviews/zig.2.md b/builders/reviews/zig.2.md| new file mode 100644 |
| index 0000000..d72fc1b |
| --- /dev/null |
| +++ b/builders/reviews/zig.2.md |
| @@ -0,0 +1 @@ |
| +1. [security] `apps/zig/src/app.zig:218`: A POST to `/todos` with `title=a%80%80%80%80` makes the backward trimming scan pass five bytes to `std.unicode.utf8Decode`, which reaches `unreachable` and aborts the ReleaseSafe server. One unauthenticated request can therefore terminate the shared instance. Validate UTF-8 before trimming, or check the sequence length and leading byte before decoding. |
| \ No newline at end of file |
diff --git a/builders/reviews/zig.3.md b/builders/reviews/zig.3.md| new file mode 100644 |
| index 0000000..ea30bf8 |
| --- /dev/null |
| +++ b/builders/reviews/zig.3.md |
| @@ -0,0 +1 @@ |
| +1. [security] [apps/zig/src/app.zig:218](apps/zig/src/app.zig#L218): A POST to `/todos` with body `title=a%80%80%80%80` passes a five-byte slice to `std.unicode.utf8Decode`, whose unsupported-length branch is `unreachable`. In the configured ReleaseSafe build, this aborts the entire server, denying service to every visitor. Validate UTF-8 before trimming and reject malformed titles without panicking. |
| \ No newline at end of file |
diff --git a/builders/security-findings.py b/builders/security-findings.py| new file mode 100755 |
| index 0000000..9312db9 |
| --- /dev/null |
| +++ b/builders/security-findings.py |
| @@ -0,0 +1,10 @@ |
| +#!/usr/bin/env python3 |
| +"""Prints only the [security] items of a numbered review list.""" |
| +import re, sys |
| +items, cur = [], None |
| +for line in open(sys.argv[1]).read().splitlines(): |
| + if re.match(r"\s*\d+\.\s+", line): |
| + cur = [line]; items.append(cur) |
| + elif cur is not None: |
| + cur.append(line) |
| +print("\n".join("\n".join(i) for i in items if re.match(r"\s*\d+\.\s+\[security\]", i[0]))) |
diff --git a/harness/cmd/bakeoff/main.go b/harness/cmd/bakeoff/main.go| index 398c420..0b69614 100644 |
| --- a/harness/cmd/bakeoff/main.go |
| +++ b/harness/cmd/bakeoff/main.go |
| @@ -35,6 +35,7 @@ func main() { |
| levels := fs.String("levels", "1,16,64", "load: concurrency levels") |
| phase := fs.String("phase", "", "tokens: build|fix|security|review") |
| logs := fs.String("logs", "builders/logs", "tokens: log directory") |
| + since := fs.String("since", "", "tokens: log whose cumulative totals this resumed phase starts from (default: the previous phase's)") |
| file := fs.String("file", "", "behaviour: Playwright JSON report") |
| var rec b.Record |
| fs.StringVar(&rec.Language, "language", "", "record: display name") |
| @@ -70,7 +71,7 @@ func main() { |
| case "build": |
| err = build(*slug, *dir, *runs) |
| case "tokens": |
| - err = tokens(*slug, *logs, *phase) |
| + err = tokens(*slug, *logs, *phase, *since) |
| case "behaviour": |
| err = behaviour(*slug, *file) |
| case "report": |
| @@ -215,7 +216,7 @@ func build(slug, dir string, runs int) error { |
| return b.Update(slug, func(r *b.AppResult) { r.Build = st }) |
| } |
| -func tokens(slug, logs, phase string) error { |
| +func tokens(slug, logs, phase, since string) error { |
| parse := b.ParseClaudeLog |
| if phase == "review" { |
| parse = b.ParseCodexLog |
| @@ -246,8 +247,14 @@ func tokens(slug, logs, phase string) error { |
| if phase == "security" { |
| prev = []string{"fix", "build"} |
| } |
| + if since != "" { |
| + prev = []string{since} |
| + } |
| for _, p := range prev { |
| f := filepath.Join(logs, slug+"."+p+".jsonl") |
| + if since != "" { |
| + f = since |
| + } |
| fh, err := os.Open(f) |
| if err != nil { |
| continue |
diff --git a/harness/harness_test.go b/harness/harness_test.go| index c5a5953..9be9ca3 100644 |
| --- a/harness/harness_test.go |
| +++ b/harness/harness_test.go |
| @@ -262,3 +262,15 @@ func TestApplyRecordKeepsUnsetFields(t *testing.T) { |
| t.Fatalf("got %+v", r) |
| } |
| } |
| + |
| +func TestParseClaudeLogSumsTurnsAcrossResults(t *testing.T) { |
| + f, _ := os.Open("testdata/claude-multi.jsonl") |
| + defer f.Close() |
| + tk, err := ParseClaudeLog(f) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if tk.Turns != 5 || tk.Seconds != 1.5 || tk.Output != 140 || tk.CacheRead != 30 || tk.CostUSD != 1.5 { |
| + t.Fatalf("got %+v (usage and cost are cumulative: take the last; turns and time are per result: sum)", tk) |
| + } |
| +} |
diff --git a/harness/testdata/claude-multi.jsonl b/harness/testdata/claude-multi.jsonl| new file mode 100644 |
| index 0000000..20ae861 |
| --- /dev/null |
| +++ b/harness/testdata/claude-multi.jsonl |
| @@ -0,0 +1,3 @@ |
| +{"type":"system","subtype":"init","session_id":"s"} |
| +{"type":"result","num_turns":3,"duration_ms":1000,"total_cost_usd":1.0,"usage":{"output_tokens":100},"modelUsage":{"claude-opus-5-5":{"inputTokens":1,"outputTokens":100,"cacheReadInputTokens":10,"cacheCreationInputTokens":5}}} |
| +{"type":"result","num_turns":2,"duration_ms":500,"total_cost_usd":1.5,"usage":{"output_tokens":40},"modelUsage":{"claude-opus-5-5":{"inputTokens":2,"outputTokens":140,"cacheReadInputTokens":30,"cacheCreationInputTokens":6}}} |
diff --git a/harness/tokens.go b/harness/tokens.go| index ce45e5f..7d8fdea 100644 |
| --- a/harness/tokens.go |
| +++ b/harness/tokens.go |
| @@ -30,9 +30,12 @@ func scanLines(r io.Reader, f func([]byte)) error { |
| return sc.Err() |
| } |
| -// ParseClaudeLog reads `claude -p --output-format stream-json` output and |
| -// returns the final result event's totals. Per-model usage (which includes |
| -// any subagents) is summed when present. |
| +// ParseClaudeLog reads `claude -p --output-format stream-json` output. A |
| +// session can emit several result events (it resumes itself when |
| +// background work finishes): their per-model usage and cost are |
| +// cumulative for the whole session, so the last one counts, while turns |
| +// and duration are per result, so they are summed. Per-model usage, which |
| +// includes any subagents, is preferred to the top-level usage. |
| func ParseClaudeLog(r io.Reader) (Tokens, error) { |
| type usage struct { |
| Input int64 `json:"input_tokens"` |
| @@ -47,6 +50,7 @@ func ParseClaudeLog(r io.Reader) (Tokens, error) { |
| CacheWrite int64 `json:"cacheCreationInputTokens"` |
| } |
| var last *Tokens |
| + turns, seconds := 0, 0.0 |
| err := scanLines(r, func(b []byte) { |
| var e struct { |
| Type string `json:"type"` |
| @@ -70,6 +74,8 @@ func ParseClaudeLog(r io.Reader) (Tokens, error) { |
| t.CacheWrite += m.CacheWrite |
| } |
| } |
| + turns += e.Turns |
| + seconds += e.DurationMs / 1000 |
| last = &t |
| }) |
| if err != nil { |
| @@ -78,6 +84,7 @@ func ParseClaudeLog(r io.Reader) (Tokens, error) { |
| if last == nil { |
| return Tokens{}, errors.New("no result event in claude log") |
| } |
| + last.Turns, last.Seconds = turns, seconds |
| return *last, nil |
| } |
diff --git a/results/go-rastrillo.json b/results/go-rastrillo.json| new file mode 100644 |
| index 0000000..1b3e9b7 |
| --- /dev/null |
| +++ b/results/go-rastrillo.json |
| @@ -0,0 +1,104 @@ |
| +{ |
| + "Slug": "go-rastrillo", |
| + "Language": "Go + Rastrillo", |
| + "Status": "LIVE", |
| + "URL": "https://todo-go-rastrillo.bfj.oncarlos.com", |
| + "Version": "1364616", |
| + "Stack": "", |
| + "Source": "", |
| + "BehaviourJSOn": { |
| + "Passed": 35, |
| + "Failed": 0, |
| + "Skipped": 0 |
| + }, |
| + "BehaviourJSOff": { |
| + "Passed": 32, |
| + "Failed": 0, |
| + "Skipped": 3 |
| + }, |
| + "TTFBEmpty": { |
| + "N": 0, |
| + "P50": 0, |
| + "P95": 0, |
| + "P99": 0, |
| + "Min": 0, |
| + "Max": 0 |
| + }, |
| + "TTFB50": { |
| + "N": 0, |
| + "P50": 0, |
| + "P95": 0, |
| + "P99": 0, |
| + "Min": 0, |
| + "Max": 0 |
| + }, |
| + "Wake": null, |
| + "Load": null, |
| + "Weight": { |
| + "HTMLBytes": 0, |
| + "JSBytes": 0 |
| + }, |
| + "Code": { |
| + "Server": 0, |
| + "Templates": 0, |
| + "JS": 0, |
| + "Tests": 0, |
| + "Deps": 0 |
| + }, |
| + "Build": { |
| + "ColdSeconds": 0, |
| + "WarmSeconds": 0, |
| + "EditSeconds": 0, |
| + "Bytes": 0, |
| + "Host": "" |
| + }, |
| + "TokensBuild": { |
| + "Input": 124, |
| + "Output": 53908, |
| + "CacheRead": 7871388, |
| + "CacheWrite": 166924, |
| + "Turns": 65, |
| + "Seconds": 756.874, |
| + "CostUSD": 3.9883256 |
| + }, |
| + "TokensFix": { |
| + "Input": 22, |
| + "Output": 8277, |
| + "CacheRead": 2045481, |
| + "CacheWrite": 13112, |
| + "Turns": 12, |
| + "Seconds": 209.902, |
| + "CostUSD": 0.6796201999999991 |
| + }, |
| + "TokensSecurity": { |
| + "Input": 0, |
| + "Output": 0, |
| + "CacheRead": 0, |
| + "CacheWrite": 0, |
| + "Turns": 0, |
| + "Seconds": 0, |
| + "CostUSD": 0 |
| + }, |
| + "TokensReview": { |
| + "Input": 684516, |
| + "Output": 3432, |
| + "CacheRead": 582400, |
| + "CacheWrite": 0, |
| + "Turns": 2, |
| + "Seconds": 0, |
| + "CostUSD": 0 |
| + }, |
| + "Readability": { |
| + "Naming": 4, |
| + "Structure": 4, |
| + "Traceability": 4, |
| + "Errors": 4, |
| + "Comments": 5, |
| + "Overall": 8, |
| + "Notes": "Strongest: comments consistently explain why (Resolve vs Run and DBPath reset, the 1 MiB form cap preventing an over-long edit reading as delete, originCheck vs csrf.Protect, toggleAll as one statement). The owned()/afterItemWrite/done seam makes a toggle easy to follow from route to SQL to fragment or 303. Weakest: renderStatus swallows template errors without logging while fail() logs with a ref, so one 500 path is silent; several discarded second returns (sessions.UserID, ParseInt) rely on reader trust in middleware; App/Router/Mux is a thin three-layer wrapper and ErrorPage/errorPage differ only by case; the \"make bump edits this line\" aside is noise." |
| + }, |
| + "ReviewFindings": [ |
| + "[correctness] `apps/go-rastrillo/internal/todo/static/app.js:42` — Every action response replaces the entire app, preserving only the new-todo input. With a request pending, double-click another todo and start editing: when the earlier response arrives, the editor disappears and its unsaved text is lost. Preserve the active editor’s value and focus across replacement, or defer replacement until editing finishes." |
| + ], |
| + "Notes": null |
| +} |
diff --git a/results/ocaml.json b/results/ocaml.json| new file mode 100644 |
| index 0000000..67c94a8 |
| --- /dev/null |
| +++ b/results/ocaml.json |
| @@ -0,0 +1,102 @@ |
| +{ |
| + "Slug": "ocaml", |
| + "Language": "", |
| + "Status": "", |
| + "URL": "", |
| + "Version": "", |
| + "Stack": "", |
| + "Source": "", |
| + "BehaviourJSOn": { |
| + "Passed": 0, |
| + "Failed": 0, |
| + "Skipped": 0 |
| + }, |
| + "BehaviourJSOff": { |
| + "Passed": 0, |
| + "Failed": 0, |
| + "Skipped": 0 |
| + }, |
| + "TTFBEmpty": { |
| + "N": 0, |
| + "P50": 0, |
| + "P95": 0, |
| + "P99": 0, |
| + "Min": 0, |
| + "Max": 0 |
| + }, |
| + "TTFB50": { |
| + "N": 0, |
| + "P50": 0, |
| + "P95": 0, |
| + "P99": 0, |
| + "Min": 0, |
| + "Max": 0 |
| + }, |
| + "Wake": null, |
| + "Load": null, |
| + "Weight": { |
| + "HTMLBytes": 0, |
| + "JSBytes": 0 |
| + }, |
| + "Code": { |
| + "Server": 0, |
| + "Templates": 0, |
| + "JS": 0, |
| + "Tests": 0, |
| + "Deps": 0 |
| + }, |
| + "Build": { |
| + "ColdSeconds": 0, |
| + "WarmSeconds": 0, |
| + "EditSeconds": 0, |
| + "Bytes": 0, |
| + "Host": "" |
| + }, |
| + "TokensBuild": { |
| + "Input": 202, |
| + "Output": 83667, |
| + "CacheRead": 12568981, |
| + "CacheWrite": 175533, |
| + "Turns": 103, |
| + "Seconds": 1110.2859999999998, |
| + "CostUSD": 5.5922082 |
| + }, |
| + "TokensFix": { |
| + "Input": 0, |
| + "Output": 0, |
| + "CacheRead": 0, |
| + "CacheWrite": 0, |
| + "Turns": 0, |
| + "Seconds": 0, |
| + "CostUSD": 0 |
| + }, |
| + "TokensSecurity": { |
| + "Input": 0, |
| + "Output": 0, |
| + "CacheRead": 0, |
| + "CacheWrite": 0, |
| + "Turns": 0, |
| + "Seconds": 0, |
| + "CostUSD": 0 |
| + }, |
| + "TokensReview": { |
| + "Input": 0, |
| + "Output": 0, |
| + "CacheRead": 0, |
| + "CacheWrite": 0, |
| + "Turns": 0, |
| + "Seconds": 0, |
| + "CostUSD": 0 |
| + }, |
| + "Readability": { |
| + "Naming": 0, |
| + "Structure": 0, |
| + "Traceability": 0, |
| + "Errors": 0, |
| + "Comments": 0, |
| + "Overall": 0, |
| + "Notes": "" |
| + }, |
| + "ReviewFindings": null, |
| + "Notes": null |
| +} |
diff --git a/results/rust.json b/results/rust.json| new file mode 100644 |
| index 0000000..96dc9ea |
| --- /dev/null |
| +++ b/results/rust.json |
| @@ -0,0 +1,105 @@ |
| +{ |
| + "Slug": "rust", |
| + "Language": "Rust", |
| + "Status": "LIVE", |
| + "URL": "https://todo-rust.bfj.oncarlos.com", |
| + "Version": "f78a916", |
| + "Stack": "", |
| + "Source": "", |
| + "BehaviourJSOn": { |
| + "Passed": 35, |
| + "Failed": 0, |
| + "Skipped": 0 |
| + }, |
| + "BehaviourJSOff": { |
| + "Passed": 32, |
| + "Failed": 0, |
| + "Skipped": 3 |
| + }, |
| + "TTFBEmpty": { |
| + "N": 0, |
| + "P50": 0, |
| + "P95": 0, |
| + "P99": 0, |
| + "Min": 0, |
| + "Max": 0 |
| + }, |
| + "TTFB50": { |
| + "N": 0, |
| + "P50": 0, |
| + "P95": 0, |
| + "P99": 0, |
| + "Min": 0, |
| + "Max": 0 |
| + }, |
| + "Wake": null, |
| + "Load": null, |
| + "Weight": { |
| + "HTMLBytes": 0, |
| + "JSBytes": 0 |
| + }, |
| + "Code": { |
| + "Server": 0, |
| + "Templates": 0, |
| + "JS": 0, |
| + "Tests": 0, |
| + "Deps": 0 |
| + }, |
| + "Build": { |
| + "ColdSeconds": 0, |
| + "WarmSeconds": 0, |
| + "EditSeconds": 0, |
| + "Bytes": 0, |
| + "Host": "" |
| + }, |
| + "TokensBuild": { |
| + "Input": 94, |
| + "Output": 46373, |
| + "CacheRead": 3981174, |
| + "CacheWrite": 102899, |
| + "Turns": 48, |
| + "Seconds": 1019.941, |
| + "CostUSD": 2.5472628 |
| + }, |
| + "TokensFix": { |
| + "Input": 14, |
| + "Output": 6770, |
| + "CacheRead": 839821, |
| + "CacheWrite": 8710, |
| + "Turns": 7, |
| + "Seconds": 397.575, |
| + "CostUSD": 0.3731002000000001 |
| + }, |
| + "TokensSecurity": { |
| + "Input": 0, |
| + "Output": 0, |
| + "CacheRead": 0, |
| + "CacheWrite": 0, |
| + "Turns": 0, |
| + "Seconds": 0, |
| + "CostUSD": 0 |
| + }, |
| + "TokensReview": { |
| + "Input": 619228, |
| + "Output": 2898, |
| + "CacheRead": 529920, |
| + "CacheWrite": 0, |
| + "Turns": 2, |
| + "Seconds": 0, |
| + "CostUSD": 0 |
| + }, |
| + "Readability": { |
| + "Naming": 4, |
| + "Structure": 4, |
| + "Traceability": 4, |
| + "Errors": 4, |
| + "Comments": 5, |
| + "Overall": 8, |
| + "Notes": "Strongest: the `act` helper gives every action one pipeline (resolve visitor, transaction, 404/303/fragment), so a request is easy to follow from route to db to markup; comments explain real why (body-read before 403 behind the edge, mutex over a pool, append-only migrations, editing li preservation). db.rs is one query per function with scoped list ids. Weakest: handler params are single letters (s, h, q, f) and the `found`/`Outcome` indirection adds a hop; `swap` in app.js is dense DOM surgery that needs careful reading; errors are only rusqlite wrapped in AppError and logged with eprintln, with a few `expect`s standing in for proper failure paths." |
| + }, |
| + "ReviewFindings": [ |
| + "[correctness] [apps/rust/static/app.js:10](/home/paulca/amadan.net/carlos/todomvc-bakeoff/.worktrees/rust/apps/rust/static/app.js:10): Filter navigation can discard pending changes. Requests wait in a promise queue, but filter links navigate immediately. With a toggle request pending, edit another todo and click a filter: blur queues the save, then navigation destroys the queue before that save reaches the server. Wait for pending actions before navigating.", |
| + "[spec] [apps/rust/src/main.rs:45](/home/paulca/amadan.net/carlos/todomvc-bakeoff/.worktrees/rust/apps/rust/src/main.rs:45): `--listen` accepts every loopback address, including `::1` and `127.0.0.2`. The brief explicitly restricts binding to `127.0.0.1`. Validate that exact address." |
| + ], |
| + "Notes": null |
| +} |
diff --git a/results/swift.json b/results/swift.json| new file mode 100644 |
| index 0000000..28dc2c2 |
| --- /dev/null |
| +++ b/results/swift.json |
| @@ -0,0 +1,104 @@ |
| +{ |
| + "Slug": "swift", |
| + "Language": "Swift", |
| + "Status": "LIVE", |
| + "URL": "https://todo-swift.bfj.oncarlos.com", |
| + "Version": "354c88a", |
| + "Stack": "", |
| + "Source": "", |
| + "BehaviourJSOn": { |
| + "Passed": 35, |
| + "Failed": 0, |
| + "Skipped": 0 |
| + }, |
| + "BehaviourJSOff": { |
| + "Passed": 32, |
| + "Failed": 0, |
| + "Skipped": 3 |
| + }, |
| + "TTFBEmpty": { |
| + "N": 0, |
| + "P50": 0, |
| + "P95": 0, |
| + "P99": 0, |
| + "Min": 0, |
| + "Max": 0 |
| + }, |
| + "TTFB50": { |
| + "N": 0, |
| + "P50": 0, |
| + "P95": 0, |
| + "P99": 0, |
| + "Min": 0, |
| + "Max": 0 |
| + }, |
| + "Wake": null, |
| + "Load": null, |
| + "Weight": { |
| + "HTMLBytes": 0, |
| + "JSBytes": 0 |
| + }, |
| + "Code": { |
| + "Server": 0, |
| + "Templates": 0, |
| + "JS": 0, |
| + "Tests": 0, |
| + "Deps": 0 |
| + }, |
| + "Build": { |
| + "ColdSeconds": 0, |
| + "WarmSeconds": 0, |
| + "EditSeconds": 0, |
| + "Bytes": 0, |
| + "Host": "" |
| + }, |
| + "TokensBuild": { |
| + "Input": 120, |
| + "Output": 68280, |
| + "CacheRead": 6750077, |
| + "CacheWrite": 154900, |
| + "Turns": 67, |
| + "Seconds": 1813.84, |
| + "CostUSD": 3.9552954 |
| + }, |
| + "TokensFix": { |
| + "Input": 24, |
| + "Output": 6564, |
| + "CacheRead": 2080923, |
| + "CacheWrite": 11946, |
| + "Turns": 12, |
| + "Seconds": 434.384, |
| + "CostUSD": 0.6431286000000007 |
| + }, |
| + "TokensSecurity": { |
| + "Input": 0, |
| + "Output": 0, |
| + "CacheRead": 0, |
| + "CacheWrite": 0, |
| + "Turns": 0, |
| + "Seconds": 0, |
| + "CostUSD": 0 |
| + }, |
| + "TokensReview": { |
| + "Input": 688335, |
| + "Output": 3734, |
| + "CacheRead": 591488, |
| + "CacheWrite": 0, |
| + "Turns": 2, |
| + "Seconds": 0, |
| + "CostUSD": 0 |
| + }, |
| + "Readability": { |
| + "Naming": 4, |
| + "Structure": 4, |
| + "Traceability": 5, |
| + "Errors": 4, |
| + "Comments": 5, |
| + "Overall": 8, |
| + "Notes": "Strongest: a toggle is trivially traceable (route closure → `action` → `Store.toggle` → `changed` → `Page.body`), each file has one clear job, and comments explain decisions (single SQLite connection, append-only migrations, 400-day cookie, the partial header) without restating code. Weakest: `OriginCheck` lives in Visitors.swift and `trimmingWhitespace` is split across Routes.swift and Store.swift; `action` is a generic name for the central request helper; `Store.todos` silently defaults malformed rows (`?? 0`, `?? \"\"`) rather than failing; a few `try?` cleanups are justified but unlogged, and the logger is otherwise unused. `bumpNote` is build tooling leaking into app code, though documented." |
| + }, |
| + "ReviewFindings": [ |
| + "[correctness] [apps/swift/static/app.js:32](/home/paulca/amadan.net/carlos/todomvc-bakeoff/.worktrees/swift/apps/swift/static/app.js:32): Every action response removes the entire list, including any active editor. Toggle one item, then start editing another before the response arrives: the replacement discards the draft and focus without saving. Serializing requests does not protect unsent edits. Preserve active editor contents and focus across updates, or defer replacement until editing finishes." |
| + ], |
| + "Notes": null |
| +} |
diff --git a/results/zig.json b/results/zig.json| new file mode 100644 |
| index 0000000..89cd706 |
| --- /dev/null |
| +++ b/results/zig.json |
| @@ -0,0 +1,104 @@ |
| +{ |
| + "Slug": "zig", |
| + "Language": "Zig", |
| + "Status": "LIVE", |
| + "URL": "https://todo-zig.bfj.oncarlos.com", |
| + "Version": "576c3b8", |
| + "Stack": "", |
| + "Source": "", |
| + "BehaviourJSOn": { |
| + "Passed": 35, |
| + "Failed": 0, |
| + "Skipped": 0 |
| + }, |
| + "BehaviourJSOff": { |
| + "Passed": 32, |
| + "Failed": 0, |
| + "Skipped": 3 |
| + }, |
| + "TTFBEmpty": { |
| + "N": 0, |
| + "P50": 0, |
| + "P95": 0, |
| + "P99": 0, |
| + "Min": 0, |
| + "Max": 0 |
| + }, |
| + "TTFB50": { |
| + "N": 0, |
| + "P50": 0, |
| + "P95": 0, |
| + "P99": 0, |
| + "Min": 0, |
| + "Max": 0 |
| + }, |
| + "Wake": null, |
| + "Load": null, |
| + "Weight": { |
| + "HTMLBytes": 0, |
| + "JSBytes": 0 |
| + }, |
| + "Code": { |
| + "Server": 0, |
| + "Templates": 0, |
| + "JS": 0, |
| + "Tests": 0, |
| + "Deps": 0 |
| + }, |
| + "Build": { |
| + "ColdSeconds": 0, |
| + "WarmSeconds": 0, |
| + "EditSeconds": 0, |
| + "Bytes": 0, |
| + "Host": "" |
| + }, |
| + "TokensBuild": { |
| + "Input": 172, |
| + "Output": 62991, |
| + "CacheRead": 9913615, |
| + "CacheWrite": 158246, |
| + "Turns": 92, |
| + "Seconds": 1791.585, |
| + "CostUSD": 4.509198999999999 |
| + }, |
| + "TokensFix": { |
| + "Input": 26, |
| + "Output": 8226, |
| + "CacheRead": 2312162, |
| + "CacheWrite": 15063, |
| + "Turns": 13, |
| + "Seconds": 285.943, |
| + "CostUSD": 0.7475604000000002 |
| + }, |
| + "TokensSecurity": { |
| + "Input": 8, |
| + "Output": 1030, |
| + "CacheRead": 749296, |
| + "CacheWrite": 3731, |
| + "Turns": 4, |
| + "Seconds": 55.56, |
| + "CostUSD": 0.20033920000000016 |
| + }, |
| + "TokensReview": { |
| + "Input": 899608, |
| + "Output": 3958, |
| + "CacheRead": 760064, |
| + "CacheWrite": 0, |
| + "Turns": 3, |
| + "Seconds": 0, |
| + "CostUSD": 0 |
| + }, |
| + "Readability": { |
| + "Naming": 4, |
| + "Structure": 4, |
| + "Traceability": 5, |
| + "Errors": 4, |
| + "Comments": 5, |
| + "Overall": 8, |
| + "Notes": "Strongest: traceability. A toggle is easy to follow from app.js's form submit through handle/post, itemAction, Store.toggle's single UPDATE, and back via view.render, with clear module boundaries (main/app/db/view) and good why-comments on header lifetime, keep-alive, token hashing, and signal-safe shutdown. Weakest: a few small opacities. post() is one long if-chain mixing routing and logic; readerExpectContinue errors are remapped to WriteFailed with no explanation; receiveHead failures and the ROLLBACK/deleteFile errors are silently dropped; sqlite errors collapse to a single error.Sqlite with logging only at some layers; the health_body comment references an unexplained 'make bump'; and the 'Each of these answers false' doc sits on toggle alone but means three functions." |
| + }, |
| + "ReviewFindings": [ |
| + "[security] [apps/zig/src/app.zig:218](apps/zig/src/app.zig#L218): A POST to `/todos` with body `title=a%80%80%80%80` passes a five-byte slice to `std.unicode.utf8Decode`, whose unsupported-length branch is `unreachable`. In the configured ReleaseSafe build, this aborts the entire server, denying service to every visitor. Validate UTF-8 before trimming and reject malformed titles without panicking." |
| + ], |
| + "Notes": null |
| +} |