🤖 TodoMVC bakeoff on Carloku — design

Status: draft for Paul's approval, 2026-10-23. Reviewed by Astra (builder check).

Why

CARLOS apps don't have to be written in Go, and the CARLOS Platform runs any binary that keeps its process contract. This bakeoff shows that with working apps. It's driven by curiosity and acts as a showcase. It isn't meant to pick a winner or feed a decision.

We build the same app, TodoMVC, in nine language setups. Each one renders HTML on the server and layers JavaScript on top, and all of them are deployed to the Carloku bakeoff account (bfj). A report page on the same account compares them.

Goals

• Nine working TodoMVC apps, live on Carloku, one per setup: 1. Go, without Rastrillo 2. Go, with Rastrillo 3. Rust 4. Zig 5. OCaml 6. Haskell 7. Swift 8. Nim (replaced Mojo, Paul's call 2026-10-24: Mojo can't make a single-file binary; see docs/toolchains.md) 9. Spinel (a Ruby compiler that produces native binaries)
• Every app passes the same behaviour tests with JavaScript on and with it off.
• A report page that puts honest numbers side by side: what each build cost in tokens, how readable the code is, how fast it responds, how it holds up under load, and the other measures listed below.

Non-goals

• Choosing a "best" language. The report presents numbers and doesn't crown anyone.
• Tuning one app beyond the others after the build. Every app gets the same build process and the same review-and-fix round, and nothing more.
• Rich features beyond TodoMVC: no sign-in, no sharing, no sync between devices.

Anti-goals (failure even if everything technically works)

• A report that looks rigged. One app gets a different UI, a different database, a bigger instance, more tuning, extra review rounds, or quietly drops part of the spec.
• An app missing from the report without comment. If a setup can't be built or can't run on Carloku, the report says so plainly and says why.
• Working around Carloku. If the platform lacks something an app needs (for example, a binary that needs files beside it), we file a request against the platform and report the app as blocked on it. We don't build workarounds. (Paul's standing rule.)

Decided with Paul (2026-10-23)

• One shared UI. Every app uses TodoMVC's standard markup and CSS (tastejs/todomvc-app-template, todomvc-app-css). For the bakeoff this overrides the CARLOS skill's default that Rastrillo apps use Rastrillo's own UI components. The Go + Rastrillo app uses the TodoMVC CSS too.
• Server-rendered with progressive enhancement. No client-side app framework. The page is HTML from the server and still works with JavaScript turned off.
• Each agent chooses its own stack. Web server, router, templating, SQLite binding and so on are each agent's own best recommendation for its language.
• The builder is Opus 5.5 for every app. Astra reviews every app through the codex CLI, and the same Opus agent fixes what the review raises.
• Spinel is already supported on Carloku. The Spinel builder starts from lpay's working setup (paulca/lpay, directory spinel/): tep (a Sinatra-style layer that drives Spinel), the tep-carlos.patch for the socket contract, and the zig-based cross-build to a static aarch64-musl binary in BUILD-arm64.md. Mojo is expected to work if it compiles to a single binary.
• Arm64 builds: try docs-t1 first (cross-compiling or an emulated arm64 container), and fall back to tito-studio over Tailscale.
• After the one fix round: remaining correctness gaps ship as they are, and the report shows them. Security findings get a second fix round, counted separately, because the apps are public.
• Downloads approved: GHC (via ghcup) and Mojo (via pixi) may be installed under ~ on docs-t1.
• Everything lives on the bakeoff account (bfj): nine apps and the report page.

What every app must do (the shared brief)

Each builder gets the same written brief. It contains:

Behaviour

• The TodoMVC app spec (tastejs/todomvc/app-spec.md), with these changes because the server holds the data: - Persistence: in SQLite on the server, not in localStorage. - Routes: real paths /, /active and /completed, not #/ fragments. The selected filter survives a reload because it's in the URL. - Editing state isn't persisted (same as the spec).
• With JavaScript off, everything works through plain forms and links: add, toggle, toggle all, edit, delete, clear completed, and the three filters. Each action is a POST that redirects back to the current filter (post/redirect/get). Editing without JS uses a link to an edit view of that item, such as /?edit=<id>, rendered with the edit field open.
• With JavaScript on, the page behaves like the reference TodoMVC: double-click to edit, Enter saves, Escape cancels, blur saves, an empty title deletes, toggles happen without a full page reload, and the counter and the "Clear completed" button update in place. The JS sends the same POSTs and swaps in server-rendered HTML or applies a small JSON reply (the builder chooses). There's no build step and no framework. Plain ES modules are allowed, and so is vendored VanJS (the one reactive library CARLOS sanctions).
• Who owns a list: the whole app is one instance. Each visitor gets their own list through an anonymous, random, HttpOnly, SameSite=Lax cookie set on the first visit. There's no sign-in. Forms are protected against cross-site posts (by SameSite plus an Origin check, or a CSRF token; the builder chooses).

Platform contract

• One binary for linux-arm64 (the platform default; carlos deploy --target), started as <bin> --socket <path> --db <path>. It listens on the unix socket, has no $PORT, and serves GET /healthz and GET /api/version.
• Linked statically, or with only libc. Instances run sandboxed and see only their own build, so the builder must not assume any shared library exists on the box. carlos vet --boot (a local check that the binary starts and answers /healthz) must pass before the first deploy.
• Reads CARLOS_HOST when it needs its own hostname. Never parses the socket path.
• Uses SQLite in WAL mode. Migrations only ever add.
• The app starts quickly after hibernation (sleeping when idle) and shuts down cleanly on SIGTERM.
• Configures the build probe (carlos apps create --build-probe /api/version) so carlos deploy confirms that the new build is what's actually serving.

Repo shape for each app

• apps/<slug>/ in this bakeoff repo. The slugs are go, go-rastrillo, rust, zig, ocaml, haskell, swift, mojo and spinel.
• AGENTS.md first. It holds the one rule and the tenancy declaration: one instance, anonymous lists by cookie, trust class C (plaintext todo text, chosen and written down).
• A README.md with how to build, run locally, test and deploy. One command for each (a Makefile or a script).
• The app's own tests, whatever is normal for its language.
• The Carloku app name is todo-<slug>, e.g. todo-rust.

What the builder may not do

• Change the markup or the CSS beyond adding hooks the JS needs, such as data- attributes or wrapping each toggle in a <form>. The rendered page must look identical to the reference.
• Add caching layers, a CDN or edge tricks that the other apps don't have. Each app may set ordinary HTTP cache headers on its static assets.
• Spend effort on performance tuning beyond what an experienced developer would write the first time. "Write it the way you'd normally write it well" is the instruction.

How the apps get built

1. Shared tooling first, built by the orchestrator (this session) before any builder starts, so every app is judged by the same tools: - The brief, as one file every builder reads. - A behaviour test suite adapted from TodoMVC's Cypress tests (tastejs/todomvc/cypress). It runs against any URL, once with JS on and once with JS off. The JS-off run uses forms only and skips the parts that are JS-only, such as double-click editing (replaced by the edit link). Builders run it locally against their own app. - A load test script and a metrics collector (see "Metrics").
2. Toolchains installed per user, under ~ (rustup, Zig tarball, opam, ghcup, Swift's static Linux SDK, Mojo through pixi, Spinel built from source). GHC and Mojo each take several GB, so we ask Paul before downloading them (machine rule).
3. Nine Opus 5.5 builders run in parallel, each in its own headless claude session with JSON output so token use is recorded. Each starts from a fresh context with the same brief and the CARLOS skills (carlosframework/skills), plus Rastrillo's own SKILL.md for Go + Rastrillo. Each builds, tests locally, then deploys to bfj with carlos deploy and checks it on the live URL.
4. Astra reviews each app through codex exec against the brief, the spec and the code. The review is limited to correctness, spec compliance and security. It doesn't ask for performance tuning.
5. One fix round. The same Opus builder session fixes what Astra raised, then redeploys. Only one round, for every app. The one exception is security: if Astra still finds a security problem after that round, there's a second round for security fixes only, with its tokens reported separately.
6. Blind readability scoring by Fable. Each app's code is shown with the language named but nothing else identifying, and scored against a fixed rubric. Astra doesn't score readability because it already reviewed the code.
7. Measurement and report (below).

All local servers bind to 127.0.0.1. Scratch space goes in $TMPDIR and is removed at the end.

Metrics

All of them are collected by scripts in this repo and can be re-run.

| Measure | How | |---|---| | Tokens to build | From each builder's session log: input, output, cache read and cache write, plus turns and wall-clock time. The first build and the fix round are reported separately, and so is the Astra review. | | Correctness | Behaviour tests passed, JS on and JS off, against the live URL. Spec gaps found by review. | | Readability | Fable's rubric score (naming, structure, how easy it is to follow one request end to end, error handling, comments), with a short written justification. | | Size of the code | Lines of code split three ways: server code, templates, and JS. Also the number of third-party dependencies. | | Build | Three build times for the arm64 binary that ships: cold (the app and every third-party library compiled from already-downloaded sources; compiler and standard library as shipped), warm (libraries already compiled, app rebuilt from clean), and one-line change (rebuild after a trivial edit in the request-handling code). Plus binary size. Paul asked for cold and warm, 2026-10-23. Measured on docs-t1 where the app cross-compiles there; otherwise on the machine that built it, with the machine named beside the number and kept out of the ranking. | | Response time | Time to the first byte for GET / with an empty list and with 50 items, from docs-t1, median of repeated runs. | | Wake time | Time to the first byte on the first request after the instance has gone to sleep. | | Under load | A ramp for reads (GET / on a 50-item list) and for writes (toggling an item): 1, 2, 4 … 1,024 concurrent connections, 10 s per step. Reports peak requests per second, latency at every step, and where it slows down: the most connections at which doubling the load still raised throughput by 10% or more. A ramp stops early after two flat doublings or more than 1% errors. Paul asked for this, 2026-10-23, replacing fixed loads at 1, 16 and 64. | | Memory | Peak and idle memory from the platform, if it exposes them. If it doesn't, we file a request against the platform rather than measure inside the app. | | Page weight | Bytes of HTML and JS sent for GET /. |

Load tests run from docs-t1 over the public internet, so the numbers include network time. That's the same for every app, and the report says so. Every app runs on the same box class and instance size, and we run the apps one after another, never at the same time.

The report page

• A static site on bfj (carlos deploy --kind static), named bakeoff.
• One page with a summary table of every measure for every app, and a chart per performance measure. Each app gets a short section covering its stack, notable choices, Astra's main findings, Fable's readability notes, a link to the live app and a link to its source.
• A "How we measured" section and a "What this does and doesn't show" section, including the network caveat, Spinel's single-threaded server and any app that's blocked.
• Built from the metrics files by a script, so a re-run regenerates it.
• Report copy goes through the copy-review gate before it ships.

Known risks

• Everything targets linux-arm64, and docs-t1 is x86-64. Rust, Zig, Go and Swift (static Linux SDK) cross-compile well, and so does Spinel (lpay's zig recipe). OCaml, Haskell and Mojo are harder. Each tries docs-t1 first, by cross-compiling or in an emulated arm64 container, and falls back to tito-studio. If neither works within the effort limit, that app is reported as blocked.
• Mojo may need runtime files beside the binary. The binary kind takes a single file. If Mojo can't produce one, we file a platform request and report Mojo as blocked.
• Spinel has no threads. lpay's notes flag its aarch64 fiber context switch as untested at runtime. Whatever concurrency model the Spinel server ends up with, the report states it next to the load numbers.
• Load testing shared boxes can affect other people's apps on the same box. We keep load runs short, one app at a time.

Effort limit

Every builder gets the same limit. If an app still doesn't have a local build passing the behaviour tests after 4 hours of wall-clock time, the builder stops and writes down where it got stuck, and the app is reported as blocked. All tokens a builder spends are reported, including for blocked apps and time spent getting toolchains working.

Left to the builders

• Each app's libraries, framework, server, templating and SQLite binding.
• How the JS talks to the server (HTML fragments or small JSON).
• Internal code layout, within the repo shape above.

Calls made by the orchestrator, not by Paul

• Real paths instead of #/ routes, and per-visitor lists by anonymous cookie.
• Fable for blind readability scoring, separate from Astra's review.
• One review-and-fix round per app.
• All apps in one repo under apps/<slug>/.
• App names todo-<slug>, and the report app named bakeoff.
• The shared test suite and load test are written once by the orchestrator, not by each builder.
• The effort limit: 4 hours of wall-clock time per builder.
• Build time for apps that can't cross-compile on docs-t1 is measured where they're built, with the machine named, and isn't ranked against the others.