carlos / todomvc-bakeoff Public

fix: say whether Carloku turned requests away (503) or the app failed (502)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Paul Campbell pushed by claude 19 hours ago e364812ad7258bf8f47845cec1a1a5f1f5a35e73 parent b24ef50

6 files changed, +66 −7

copy-review/strings.json +25 −1
harness/harness_test.go +16 −0
harness/report/report.go +4 −1
harness/report/report_test.go +3 −2
harness/sweep.go +9 −0
report/site/index.html +9 −3

diff --git a/copy-review/strings.json b/copy-review/strings.json

index dc909a2..bafee0c 100644

--- a/copy-review/strings.json

+++ b/copy-review/strings.json

@@ -285,6 +285,22 @@

"context": "The line under that chart saying exactly what was measured.",

"notes": "One sentence or two."

+ {

+ "id": "chart.reads_after_knee.title",

+ "section": "Charts",

+ "label": "Page loads after writes: slow-down title",

+ "text": "Page loads after the writes: where it slows down",

+ "context": "Title of the slow-down chart for the second read ramp.",

+ "notes": "Short."

+ },

+ {

+ "id": "chart.reads_after_knee.note",

+ "section": "Charts",

+ "label": "Page loads after writes: slow-down note",

+ "text": "The same rule, for the read ramp run straight after the writes.",

+ "context": "Note under that chart.",

+ "notes": ""

+ },

{

"id": "chart.binary.title",

"section": "Charts",

@@ -461,6 +477,14 @@

"context": "Appended after a slow-down value when, at the next step, Carloku's edge started answering some requests with 503 (its limit of 64 requests in flight per home box). E.g. \"64 connections · Carloku turned some requests away from 128\".",

"notes": "Starts with a space and a dot. Keep {n}."

+ {

+ "id": "label.failed_from",

+ "section": "Labels",

+ "label": "Failures marker",

+ "text": " · requests started failing from {n}",

+ "context": "Appended after a slow-down value when, at the next step, the app itself stopped answering some requests (a 502 from Carloku's edge), as opposed to the edge turning them away.",

+ "notes": "Starts with a space and a dot. Keep {n}."

+ },

{

"id": "label.still_scaling",

"section": "Labels",

@@ -644,7 +668,7 @@

"id": "caveats.body",

"section": "Caveats",

"label": "Body",

- "text": "- **The network is in every speed number.** Requests travelled from our test machine to Carloku over the internet, and the base delay swung between about 17 and 37 milliseconds during the runs. Small differences in first-byte times mean little.\n- **Carloku caps traffic at the edge.** When a request reaches an edge server that isn't the one running the app, Carloku lets at most 64 such requests be in flight at once and turns the rest away with a \"try again shortly\" response. Above 64 connections, the load test partly measures that limit, and whether it applied depended on how our traffic was routed at the time.\n- **Every app ran on Carloku's default size.** Throughput limits reflect that size, not what each language could do on a bigger server.\n- **One build per language.** Another session with the same model could produce a different app, faster or slower.\n- **Readability scores cluster.** Most apps scored 8 out of 10. Treat the score as a rough signal.\n- **Three apps were built under emulation.** OCaml, Haskell and Spinel had no straightforward way to compile for ARM from our x86 machine, so they were built inside an emulated ARM system. Their build times are much longer for that reason, and don't compare with the others.\n- **Spinel restarted under heavy load.** Its server process exited during the load test and Carloku started it again each time.\n- **Mojo was dropped.** Mojo can't produce a single self-contained program file, which is what Carloku runs, so we replaced it with Nim. The platform question is filed as [an idea](https://amadan.net/carlos/platform/-/discussion/360).\n- **Fresh copies for measuring.** To give every app an empty database, each was measured on a new copy of itself, which is the one the live links point to.",

+ "text": "- **The network is in every speed number.** Requests travelled from our test machine to Carloku over the internet, and the base delay swung between about 17 and 37 milliseconds during the runs. Small differences in first-byte times mean little.\n- **Carloku caps traffic at the edge.** When a request reaches an edge server that isn't the one running the app, Carloku lets at most 64 such requests be in flight at once and turns the rest away with a \"try again shortly\" response. Above 64 connections, the load test partly measures that limit, and whether it applied depended on how our traffic was routed at the time.\n- **Some apps failed outright at very high load.** At 256 or 512 connections, Swift, Nim and Spinel stopped answering some requests, which Carloku's edge reported as errors (502) rather than turning them away. The charts mark where that started.\n- **Every app ran on Carloku's default size.** Throughput limits reflect that size, not what each language could do on a bigger server.\n- **One build per language.** Another session with the same model could produce a different app, faster or slower.\n- **Readability scores cluster.** Most apps scored 8 out of 10. Treat the score as a rough signal.\n- **Three apps were built under emulation.** OCaml, Haskell and Spinel had no straightforward way to compile for ARM from our x86 machine, so they were built inside an emulated ARM system. Their build times are much longer for that reason, and don't compare with the others.\n- **Spinel restarted under heavy load.** Its server process exited during the load test and Carloku started it again each time.\n- **Mojo was dropped.** Mojo can't produce a single self-contained program file, which is what Carloku runs, so we replaced it with Nim. The platform question is filed as [an idea](https://amadan.net/carlos/platform/-/discussion/360).\n- **Fresh copies for measuring.** To give every app an empty database, each was measured on a new copy of itself, which is the one the live links point to.",

"context": "Plain list of what to be careful about when reading the numbers.",

"notes": "Markdown list."

}

diff --git a/harness/harness_test.go b/harness/harness_test.go

index 09a6288..986169a 100644

--- a/harness/harness_test.go

+++ b/harness/harness_test.go

@@ -331,6 +331,9 @@ func TestKneeIgnoresAOneStepBlip(t *testing.T) {

func TestKneeStopsAtErrors(t *testing.T) {

sweep := []LoadResult{lr(16, 400, 60, 0), lr(32, 800, 70, 0), lr(64, 1300, 90, 0), lr(128, 1360, 370, 0.40)}

k := Knee(sweep)

+ if k.ErrorsStatus != "" {

+ t.Fatalf("no breakdown recorded: status should be empty, got %q", k.ErrorsStatus)

+ }

if !k.Found || k.Concurrency != 64 || k.ErrorsAt != 128 || k.PeakRPS != 1300 {

t.Fatalf("got %+v: want slow-down 64 (last clean step), errors from 128, peak from clean steps only", k)

}

@@ -389,3 +392,16 @@ func TestMeasureMemoryOfRefapp(t *testing.T) {

t.Fatalf("implausible %+v", m)

}

+func TestKneeNamesTheFailingStatus(t *testing.T) {

+ bad := lr(128, 1360, 370, 0.40)

+ bad.Status = map[string]int{"200": 600, "503": 400}

+ k := Knee([]LoadResult{lr(32, 800, 70, 0), lr(64, 1300, 90, 0), bad})

+ if k.ErrorsStatus != "503" {

+ t.Fatalf("want 503, got %q", k.ErrorsStatus)

+ }

+ bad.Status = map[string]int{"200": 600, "502": 300, "503": 100}

+ if k := Knee([]LoadResult{lr(64, 1300, 90, 0), bad}); k.ErrorsStatus != "502" {

+ t.Fatalf("want the most common failing status 502, got %q", k.ErrorsStatus)

+ }

diff --git a/harness/report/report.go b/harness/report/report.go

index 2ccf988..0ae0fac 100644

--- a/harness/report/report.go

+++ b/harness/report/report.go

@@ -99,8 +99,10 @@ func kneeChart(rs []b.AppResult, c Copy, scenario, id string) Chart {

k := knee(r, scenario)

switch {

case !live(r):

- case k.ErrorsAt > 0:

+ case k.ErrorsAt > 0 && k.ErrorsStatus == "503":

ch.Rows[i].Suffix = c.T("label.errors_from", "n", num(float64(k.ErrorsAt)))

+ case k.ErrorsAt > 0:

+ ch.Rows[i].Suffix = c.T("label.failed_from", "n", num(float64(k.ErrorsAt)))

case !k.Found && k.MaxTested > 0:

ch.Rows[i].Missing = c.T("label.still_scaling", "n", num(float64(k.MaxTested)))

}

@@ -155,6 +157,7 @@ func charts(rs []b.AppResult, c Copy) []Chart {

chart(rs, c, "toggles_peak", num, func(r b.AppResult) (float64, bool) { return pos(knee(r, "toggle").PeakRPS) }),

kneeChart(rs, c, "toggle", "toggles_knee"),

chart(rs, c, "reads_after", num, func(r b.AppResult) (float64, bool) { return pos(knee(r, "read-after").PeakRPS) }),

+ kneeChart(rs, c, "read-after", "reads_after_knee"),

chart(rs, c, "memory_idle", kb, func(r b.AppResult) (float64, bool) { return pos(float64(r.Memory.IdleKB)) }),

chart(rs, c, "memory_peak", kb, func(r b.AppResult) (float64, bool) { return pos(float64(r.Memory.PeakKB)) }),

chart(rs, c, "binary", mb, func(r b.AppResult) (float64, bool) { return pos(float64(r.Build.Bytes)) }),

diff --git a/harness/report/report_test.go b/harness/report/report_test.go

index 050c5c8..0377f80 100644

--- a/harness/report/report_test.go

+++ b/harness/report/report_test.go

@@ -22,7 +22,8 @@ func fixture() []b.AppResult {

Wake: []float64{120, 140, 130},

Load: []b.LoadResult{{Scenario: "read", Concurrency: 1, RPS: 20}, {Scenario: "read", Concurrency: 2, RPS: 40},

{Scenario: "read", Concurrency: 4, RPS: 41}, {Scenario: "toggle", Concurrency: 1, RPS: 15},

- {Scenario: "toggle", Concurrency: 2, RPS: 30}, {Scenario: "toggle", Concurrency: 4, RPS: 31, ErrorRate: 0.4}},

+ {Scenario: "toggle", Concurrency: 2, RPS: 30}, {Scenario: "toggle", Concurrency: 4, RPS: 31, ErrorRate: 0.4, Status: map[string]int{"303": 6, "503": 4}},

+ {Scenario: "read-after", Concurrency: 1, RPS: 20}, {Scenario: "read-after", Concurrency: 2, RPS: 9, ErrorRate: 0.5, Status: map[string]int{"200": 5, "502": 5}}},

Weight: b.Weight{HTMLBytes: 9000, JSBytes: 2400},

Code: b.CodeStats{Server: 300, Templates: 80, JS: 120, Tests: 200, Deps: 1},

Build: b.Build{ColdSeconds: 1097.4, WarmSeconds: 2, EditSeconds: 1.1, Bytes: 9_400_000, Host: "docs-t1"},

@@ -51,7 +52,7 @@ func TestRenderAllApps(t *testing.T) {

out := buf.String()

for _, want := range []string{

"Go", "Mojo", "single-file binaries only", "https://todo-go.bfj.oncarlos.com", "<svg",

- "<p>Nine apps.</p>", `<a href="https://example.com">link</a>`, "Clear & tidy <ok>", "9.4 MB", "2 connections", "2 connections · Carloku turned some requests away from 4", "35/35 · 33/33", "18.3 min",

+ "<p>Nine apps.</p>", `<a href="https://example.com">link</a>`, "Clear & tidy <ok>", "9.4 MB", "2 connections", "2 connections · Carloku turned some requests away from 4", "35/35 · 33/33", "18.3 min", "1 connections · requests started failing from 2",

} {

if !strings.Contains(out, want) {

t.Errorf("output lacks %q", want)

diff --git a/harness/sweep.go b/harness/sweep.go

index 42bf3fe..e54fb09 100644

--- a/harness/sweep.go

+++ b/harness/sweep.go

@@ -18,6 +18,9 @@ type KneeResult struct {

P50Ms float64 // median latency at Concurrency

MaxTested int

ErrorsAt int // the first level with more than 1% errors, if any

+ // ErrorsStatus is the commonest failing status at ErrorsAt: "503" when

+ // Carloku's edge turned requests away, "502" when the app didn't answer.

+ ErrorsStatus string

}

// Knee finds where the app slows down: the fewest connections from which

@@ -31,6 +34,12 @@ func Knee(sweep []LoadResult) KneeResult {

for i, r := range sweep {

if r.ErrorRate > maxErrorRate {

k.ErrorsAt = r.Concurrency

+ n := 0

+ for code, c := range r.Status {

+ if code != "200" && code != "303" && c > n {

+ k.ErrorsStatus, n = code, c

+ }

sweep = sweep[:i]

break

}

diff --git a/report/site/index.html b/report/site/index.html

index 57a8c00..60fd557 100644

--- a/report/site/index.html

+++ b/report/site/index.html

@@ -142,7 +142,7 @@ dt { color: var(--muted); } dd { margin: 0; color: var(--ink-2); }

<h3>Page loads: where it slows down</h3>

<p class="note">The fewest simultaneous connections from which adding more never raised throughput by 10% or more.</p>

-<svg class="chart" viewBox="0 0 680 286" role="img" aria-label="Page loads: where it slows down"><line class="axis" x1="150" x2="150" y1="8" y2="278"/><text class="label" x="140" y="28">Go</text><path class="s1" d="M150.0 15.0H154.7Q158.7 15.0 158.7 19.0V27.0Q158.7 31.0 154.7 31.0H150.0Z"><title>Go: 8 connections</title></path><text class="value" x="164.7" y="28">8 connections</text><text class="label" x="140" y="58">Go + Rastrillo</text><path class="s1" d="M150.0 45.0H163.4Q167.4 45.0 167.4 49.0V57.0Q167.4 61.0 163.4 61.0H150.0Z"><title>Go + Rastrillo: 16 connections</title></path><text class="value" x="173.4" y="58">16 connections</text><text class="label" x="140" y="88">Rust</text><path class="s1" d="M150.0 75.0H215.6Q219.6 75.0 219.6 79.0V87.0Q219.6 91.0 215.6 91.0H150.0Z"><title>Rust: 64 connections</title></path><text class="value" x="225.6" y="88">64 connections · Carloku turned some requests away from 128</text><text class="label" x="140" y="118">Zig</text><path class="s1" d="M150.0 105.0H180.8Q184.8 105.0 184.8 109.0V117.0Q184.8 121.0 180.8 121.0H150.0Z"><title>Zig: 32 connections</title></path><text class="value" x="190.8" y="118">32 connections</text><text class="label" x="140" y="148">OCaml</text><path class="s1" d="M150.0 135.0H215.6Q219.6 135.0 219.6 139.0V147.0Q219.6 151.0 215.6 151.0H150.0Z"><title>OCaml: 64 connections</title></path><text class="value" x="225.6" y="148">64 connections · Carloku turned some requests away from 256</text><text class="label" x="140" y="178">Haskell</text><path class="s1" d="M150.0 165.0H180.8Q184.8 165.0 184.8 169.0V177.0Q184.8 181.0 180.8 181.0H150.0Z"><title>Haskell: 32 connections</title></path><text class="value" x="190.8" y="178">32 connections</text><text class="label" x="140" y="208">Swift</text><path class="s1" d="M150.0 195.0H215.6Q219.6 195.0 219.6 199.0V207.0Q219.6 211.0 215.6 211.0H150.0Z"><title>Swift: 64 connections</title></path><text class="value" x="225.6" y="208">64 connections · Carloku turned some requests away from 512</text><text class="label" x="140" y="238">Nim</text><path class="s1" d="M150.0 225.0H180.8Q184.8 225.0 184.8 229.0V237.0Q184.8 241.0 180.8 241.0H150.0Z"><title>Nim: 32 connections</title></path><text class="value" x="190.8" y="238">32 connections</text><text class="label" x="140" y="268">Spinel</text><path class="s1" d="M150.0 255.0H215.6Q219.6 255.0 219.6 259.0V267.0Q219.6 271.0 215.6 271.0H150.0Z"><title>Spinel: 64 connections</title></path><text class="value" x="225.6" y="268">64 connections · Carloku turned some requests away from 128</text></svg>

+<svg class="chart" viewBox="0 0 680 286" role="img" aria-label="Page loads: where it slows down"><line class="axis" x1="150" x2="150" y1="8" y2="278"/><text class="label" x="140" y="28">Go</text><path class="s1" d="M150.0 15.0H154.7Q158.7 15.0 158.7 19.0V27.0Q158.7 31.0 154.7 31.0H150.0Z"><title>Go: 8 connections</title></path><text class="value" x="164.7" y="28">8 connections</text><text class="label" x="140" y="58">Go + Rastrillo</text><path class="s1" d="M150.0 45.0H163.4Q167.4 45.0 167.4 49.0V57.0Q167.4 61.0 163.4 61.0H150.0Z"><title>Go + Rastrillo: 16 connections</title></path><text class="value" x="173.4" y="58">16 connections</text><text class="label" x="140" y="88">Rust</text><path class="s1" d="M150.0 75.0H215.6Q219.6 75.0 219.6 79.0V87.0Q219.6 91.0 215.6 91.0H150.0Z"><title>Rust: 64 connections</title></path><text class="value" x="225.6" y="88">64 connections · Carloku turned some requests away from 128</text><text class="label" x="140" y="118">Zig</text><path class="s1" d="M150.0 105.0H180.8Q184.8 105.0 184.8 109.0V117.0Q184.8 121.0 180.8 121.0H150.0Z"><title>Zig: 32 connections</title></path><text class="value" x="190.8" y="118">32 connections</text><text class="label" x="140" y="148">OCaml</text><path class="s1" d="M150.0 135.0H215.6Q219.6 135.0 219.6 139.0V147.0Q219.6 151.0 215.6 151.0H150.0Z"><title>OCaml: 64 connections</title></path><text class="value" x="225.6" y="148">64 connections · Carloku turned some requests away from 256</text><text class="label" x="140" y="178">Haskell</text><path class="s1" d="M150.0 165.0H180.8Q184.8 165.0 184.8 169.0V177.0Q184.8 181.0 180.8 181.0H150.0Z"><title>Haskell: 32 connections</title></path><text class="value" x="190.8" y="178">32 connections</text><text class="label" x="140" y="208">Swift</text><path class="s1" d="M150.0 195.0H215.6Q219.6 195.0 219.6 199.0V207.0Q219.6 211.0 215.6 211.0H150.0Z"><title>Swift: 64 connections</title></path><text class="value" x="225.6" y="208">64 connections · requests started failing from 512</text><text class="label" x="140" y="238">Nim</text><path class="s1" d="M150.0 225.0H180.8Q184.8 225.0 184.8 229.0V237.0Q184.8 241.0 180.8 241.0H150.0Z"><title>Nim: 32 connections</title></path><text class="value" x="190.8" y="238">32 connections</text><text class="label" x="140" y="268">Spinel</text><path class="s1" d="M150.0 255.0H215.6Q219.6 255.0 219.6 259.0V267.0Q219.6 271.0 215.6 271.0H150.0Z"><title>Spinel: 64 connections</title></path><text class="value" x="225.6" y="268">64 connections · Carloku turned some requests away from 128</text></svg>

</div>

@@ -154,7 +154,7 @@ dt { color: var(--muted); } dd { margin: 0; color: var(--ink-2); }

<h3>Toggles: where it slows down</h3>

<p class="note">The same rule as for page loads.</p>

-<svg class="chart" viewBox="0 0 680 286" role="img" aria-label="Toggles: where it slows down"><line class="axis" x1="150" x2="150" y1="8" y2="278"/><text class="label" x="140" y="28">Go</text><path class="s1" d="M150.0 15.0H161.5Q165.5 15.0 165.5 19.0V27.0Q165.5 31.0 161.5 31.0H150.0Z"><title>Go: 32 connections</title></path><text class="value" x="171.5" y="28">32 connections · Carloku turned some requests away from 128</text><text class="label" x="140" y="58">Go + Rastrillo</text><path class="s1" d="M150.0 45.0H153.9Q157.8 45.0 157.8 48.9V57.1Q157.8 61.0 153.9 61.0H150.0Z"><title>Go + Rastrillo: 16 connections</title></path><text class="value" x="163.8" y="58">16 connections</text><text class="label" x="140" y="88">Rust</text><path class="s1" d="M150.0 75.0H177.0Q181.0 75.0 181.0 79.0V87.0Q181.0 91.0 177.0 91.0H150.0Z"><title>Rust: 64 connections</title></path><text class="value" x="187.0" y="88">64 connections · Carloku turned some requests away from 128</text><text class="label" x="140" y="118">Zig</text><path class="s1" d="M150.0 105.0H177.0Q181.0 105.0 181.0 109.0V117.0Q181.0 121.0 177.0 121.0H150.0Z"><title>Zig: 64 connections</title></path><text class="value" x="187.0" y="118">64 connections</text><text class="label" x="140" y="148">OCaml</text><path class="s1" d="M150.0 135.0H177.0Q181.0 135.0 181.0 139.0V147.0Q181.0 151.0 177.0 151.0H150.0Z"><title>OCaml: 64 connections</title></path><text class="value" x="187.0" y="148">64 connections · Carloku turned some requests away from 128</text><text class="label" x="140" y="178">Haskell</text><path class="s1" d="M150.0 165.0H161.5Q165.5 165.0 165.5 169.0V177.0Q165.5 181.0 161.5 181.0H150.0Z"><title>Haskell: 32 connections</title></path><text class="value" x="171.5" y="178">32 connections · Carloku turned some requests away from 128</text><text class="label" x="140" y="208">Swift</text><path class="s1" d="M150.0 195.0H208.0Q212.0 195.0 212.0 199.0V207.0Q212.0 211.0 208.0 211.0H150.0Z"><title>Swift: 128 connections</title></path><text class="value" x="218.0" y="208">128 connections</text><text class="label" x="140" y="238">Nim</text><path class="s1" d="M150.0 225.0H208.0Q212.0 225.0 212.0 229.0V237.0Q212.0 241.0 208.0 241.0H150.0Z"><title>Nim: 128 connections</title></path><text class="value" x="218.0" y="238">128 connections · Carloku turned some requests away from 256</text><text class="label" x="140" y="268">Spinel</text><path class="s1" d="M150.0 255.0H177.0Q181.0 255.0 181.0 259.0V267.0Q181.0 271.0 177.0 271.0H150.0Z"><title>Spinel: 64 connections</title></path><text class="value" x="187.0" y="268">64 connections · Carloku turned some requests away from 256</text></svg>

+<svg class="chart" viewBox="0 0 680 286" role="img" aria-label="Toggles: where it slows down"><line class="axis" x1="150" x2="150" y1="8" y2="278"/><text class="label" x="140" y="28">Go</text><path class="s1" d="M150.0 15.0H163.4Q167.4 15.0 167.4 19.0V27.0Q167.4 31.0 163.4 31.0H150.0Z"><title>Go: 32 connections</title></path><text class="value" x="173.4" y="28">32 connections · Carloku turned some requests away from 128</text><text class="label" x="140" y="58">Go + Rastrillo</text><path class="s1" d="M150.0 45.0H154.7Q158.7 45.0 158.7 49.0V57.0Q158.7 61.0 154.7 61.0H150.0Z"><title>Go + Rastrillo: 16 connections</title></path><text class="value" x="164.7" y="58">16 connections</text><text class="label" x="140" y="88">Rust</text><path class="s1" d="M150.0 75.0H180.8Q184.8 75.0 184.8 79.0V87.0Q184.8 91.0 180.8 91.0H150.0Z"><title>Rust: 64 connections</title></path><text class="value" x="190.8" y="88">64 connections · Carloku turned some requests away from 128</text><text class="label" x="140" y="118">Zig</text><path class="s1" d="M150.0 105.0H180.8Q184.8 105.0 184.8 109.0V117.0Q184.8 121.0 180.8 121.0H150.0Z"><title>Zig: 64 connections</title></path><text class="value" x="190.8" y="118">64 connections</text><text class="label" x="140" y="148">OCaml</text><path class="s1" d="M150.0 135.0H180.8Q184.8 135.0 184.8 139.0V147.0Q184.8 151.0 180.8 151.0H150.0Z"><title>OCaml: 64 connections</title></path><text class="value" x="190.8" y="148">64 connections · Carloku turned some requests away from 128</text><text class="label" x="140" y="178">Haskell</text><path class="s1" d="M150.0 165.0H163.4Q167.4 165.0 167.4 169.0V177.0Q167.4 181.0 163.4 181.0H150.0Z"><title>Haskell: 32 connections</title></path><text class="value" x="173.4" y="178">32 connections · Carloku turned some requests away from 128</text><text class="label" x="140" y="208">Swift</text><path class="s1" d="M150.0 195.0H215.6Q219.6 195.0 219.6 199.0V207.0Q219.6 211.0 215.6 211.0H150.0Z"><title>Swift: 128 connections</title></path><text class="value" x="225.6" y="208">128 connections</text><text class="label" x="140" y="238">Nim</text><path class="s1" d="M150.0 225.0H215.6Q219.6 225.0 219.6 229.0V237.0Q219.6 241.0 215.6 241.0H150.0Z"><title>Nim: 128 connections</title></path><text class="value" x="225.6" y="238">128 connections · requests started failing from 256</text><text class="label" x="140" y="268">Spinel</text><path class="s1" d="M150.0 255.0H180.8Q184.8 255.0 184.8 259.0V267.0Q184.8 271.0 180.8 271.0H150.0Z"><title>Spinel: 64 connections</title></path><text class="value" x="190.8" y="268">64 connections · requests started failing from 256</text></svg>

</div>

@@ -163,6 +163,12 @@ dt { color: var(--muted); } dd { margin: 0; color: var(--ink-2); }

</div>

+<div class="card">

+<h3>Page loads after the writes: where it slows down</h3>

+<p class="note">The same rule, for the read ramp run straight after the writes.</p>

+<svg class="chart" viewBox="0 0 680 286" role="img" aria-label="Page loads after the writes: where it slows down"><line class="axis" x1="150" x2="150" y1="8" y2="278"/><text class="label" x="140" y="28">Go</text><path class="s1" d="M150.0 15.0H152.2Q154.3 15.0 154.3 17.2V28.8Q154.3 31.0 152.2 31.0H150.0Z"><title>Go: 8 connections</title></path><text class="value" x="160.3" y="28">8 connections</text><text class="label" x="140" y="58">Go + Rastrillo</text><path class="s1" d="M150.0 45.0H154.7Q158.7 45.0 158.7 49.0V57.0Q158.7 61.0 154.7 61.0H150.0Z"><title>Go + Rastrillo: 16 connections</title></path><text class="value" x="164.7" y="58">16 connections</text><text class="label" x="140" y="88">Rust</text><path class="s1" d="M150.0 75.0H215.6Q219.6 75.0 219.6 79.0V87.0Q219.6 91.0 215.6 91.0H150.0Z"><title>Rust: 128 connections</title></path><text class="value" x="225.6" y="88">128 connections</text><text class="label" x="140" y="118">Zig</text><path class="s1" d="M150.0 105.0H163.4Q167.4 105.0 167.4 109.0V117.0Q167.4 121.0 163.4 121.0H150.0Z"><title>Zig: 32 connections</title></path><text class="value" x="173.4" y="118">32 connections</text><text class="label" x="140" y="148">OCaml</text><path class="s1" d="M150.0 135.0H163.4Q167.4 135.0 167.4 139.0V147.0Q167.4 151.0 163.4 151.0H150.0Z"><title>OCaml: 32 connections</title></path><text class="value" x="173.4" y="148">32 connections · Carloku turned some requests away from 128</text><text class="label" x="140" y="178">Haskell</text><path class="s1" d="M150.0 165.0H152.2Q154.3 165.0 154.3 167.2V178.8Q154.3 181.0 152.2 181.0H150.0Z"><title>Haskell: 8 connections</title></path><text class="value" x="160.3" y="178">8 connections</text><text class="label" x="140" y="208">Swift</text><path class="s1" d="M150.0 195.0H180.8Q184.8 195.0 184.8 199.0V207.0Q184.8 211.0 180.8 211.0H150.0Z"><title>Swift: 64 connections</title></path><text class="value" x="190.8" y="208">64 connections</text><text class="label" x="140" y="238">Nim</text><path class="s1" d="M150.0 225.0H163.4Q167.4 225.0 167.4 229.0V237.0Q167.4 241.0 163.4 241.0H150.0Z"><title>Nim: 32 connections</title></path><text class="value" x="173.4" y="238">32 connections</text><text class="label" x="140" y="268">Spinel</text><path class="s1" d="M150.0 255.0H180.8Q184.8 255.0 184.8 259.0V267.0Q184.8 271.0 180.8 271.0H150.0Z"><title>Spinel: 64 connections</title></path><text class="value" x="190.8" y="268">64 connections · requests started failing from 256</text></svg>

+</div>

<h3>Memory at rest</h3>

<p class="note">Resident memory just after start, measured on our own machine because Carloku doesn't show members memory figures.</p>

@@ -288,7 +294,7 @@ dt { color: var(--muted); } dd { margin: 0; color: var(--ink-2); }

<ul><li><strong>The same starting point.</strong> Every builder got the same brief, the same rules for routes and forms, TodoMVC's own page and styles, and the same tests. It chose its own libraries.<li><strong>The same tests.</strong> TodoMVC's 29 standard tests, run with JavaScript on and again with it off, plus checks for unsafe input, foreign characters, very long titles, one visitor reaching another's list, and a screenshot comparison with TodoMVC's own page.<li><strong>One build each.</strong> Opus 5.5 built each app in a single unattended session, with a four-hour limit.<li><strong>One round of review.</strong> Astra reviewed each app for bugs, missed requirements and security problems, and the same builder fixed what it found. A second round was allowed only for security problems; Zig and Nim needed one.<li><strong>Readability.</strong> Fable scored each app's code out of 10 three times, without knowing which app it was. We show the median.<li><strong>Speed.</strong> Measured from one test machine over the internet, one app at a time, each on a fresh copy of the app with an empty database. Time to first byte is the median of 30 page loads. The load test doubles the number of simultaneous connections from 1 upwards, 10 seconds per step, first loading a 50-item list, then ticking a to-do on and off, then loading the list again.<li><strong>Waking up.</strong> Carloku puts an app to sleep after 15 quiet minutes. We timed the first request after each app fell asleep, three times.<li><strong>Builds.</strong> Every app is built for the 64-bit ARM servers Carloku runs on. A cold build compiles every library from scratch, once per app. A warm build and a one-line change are each the median of three.<li><strong>Cost.</strong> What each build and its fixes cost at list price, from the builders' own session records.</ul>

<h2>What this does and doesn't show</h2>

-<ul><li><strong>The network is in every speed number.</strong> Requests travelled from our test machine to Carloku over the internet, and the base delay swung between about 17 and 37 milliseconds during the runs. Small differences in first-byte times mean little.<li><strong>Carloku caps traffic at the edge.</strong> When a request reaches an edge server that isn't the one running the app, Carloku lets at most 64 such requests be in flight at once and turns the rest away with a "try again shortly" response. Above 64 connections, the load test partly measures that limit, and whether it applied depended on how our traffic was routed at the time.<li><strong>Every app ran on Carloku's default size.</strong> Throughput limits reflect that size, not what each language could do on a bigger server.<li><strong>One build per language.</strong> Another session with the same model could produce a different app, faster or slower.<li><strong>Readability scores cluster.</strong> Most apps scored 8 out of 10. Treat the score as a rough signal.<li><strong>Three apps were built under emulation.</strong> OCaml, Haskell and Spinel had no straightforward way to compile for ARM from our x86 machine, so they were built inside an emulated ARM system. Their build times are much longer for that reason, and don't compare with the others.<li><strong>Spinel restarted under heavy load.</strong> Its server process exited during the load test and Carloku started it again each time.<li><strong>Mojo was dropped.</strong> Mojo can't produce a single self-contained program file, which is what Carloku runs, so we replaced it with Nim. The platform question is filed as <a href="https://amadan.net/carlos/platform/-/discussion/360">an idea</a>.<li><strong>Fresh copies for measuring.</strong> To give every app an empty database, each was measured on a new copy of itself, which is the one the live links point to.</ul>

+<ul><li><strong>The network is in every speed number.</strong> Requests travelled from our test machine to Carloku over the internet, and the base delay swung between about 17 and 37 milliseconds during the runs. Small differences in first-byte times mean little.<li><strong>Carloku caps traffic at the edge.</strong> When a request reaches an edge server that isn't the one running the app, Carloku lets at most 64 such requests be in flight at once and turns the rest away with a "try again shortly" response. Above 64 connections, the load test partly measures that limit, and whether it applied depended on how our traffic was routed at the time.<li><strong>Some apps failed outright at very high load.</strong> At 256 or 512 connections, Swift, Nim and Spinel stopped answering some requests, which Carloku's edge reported as errors (502) rather than turning them away. The charts mark where that started.<li><strong>Every app ran on Carloku's default size.</strong> Throughput limits reflect that size, not what each language could do on a bigger server.<li><strong>One build per language.</strong> Another session with the same model could produce a different app, faster or slower.<li><strong>Readability scores cluster.</strong> Most apps scored 8 out of 10. Treat the score as a rough signal.<li><strong>Three apps were built under emulation.</strong> OCaml, Haskell and Spinel had no straightforward way to compile for ARM from our x86 machine, so they were built inside an emulated ARM system. Their build times are much longer for that reason, and don't compare with the others.<li><strong>Spinel restarted under heavy load.</strong> Its server process exited during the load test and Carloku started it again each time.<li><strong>Mojo was dropped.</strong> Mojo can't produce a single self-contained program file, which is what Carloku runs, so we replaced it with Nim. The platform question is filed as <a href="https://amadan.net/carlos/platform/-/discussion/360">an idea</a>.<li><strong>Fresh copies for measuring.</strong> To give every app an empty database, each was measured on a new copy of itself, which is the one the live links point to.</ul>

</main>

</body>