5566d4f · carlos/todomvc-bakeoff

diff --git a/docs/toolchains.md b/docs/toolchains.md

new file mode 100644

index 0000000..0ad9860

--- /dev/null

+++ b/docs/toolchains.md

@@ -0,0 +1,53 @@

+# 🤖 Toolchains

+

+Installed on docs-t1 (x86-64, Debian 13) under `$HOME`, with no `sudo`,

+by `scripts/toolchains.sh`. Put them on `PATH` with

+`source scripts/env.sh`. Carloku runs **linux-arm64**, so each language

+also needs a way to produce a static (or libc-only) arm64 binary.

+`scripts/smoke.sh <lang>` proves that route with a hello-world: it builds

+the binary, checks it's aarch64 and static, and runs it under qemu.

+

+Checked 2026-10-23.

+

+|---|---|---|---|

+| Zig | Zig 0.17.0 | `-target aarch64-linux-musl` | ok, static |

+| Mojo | Mojo 1.2.0 (pixi) | Build **inside the arm64 root** with `/root/.pixi/bin/mojo build`, which takes about 16 s for hello-world | **builds and runs, but isn't one file**: every Mojo binary needs `libKGENCompilerRTShared.so` beside it, and Mojo ships no static version. Cross-linking from x86 fails |

+| Spinel | spinel `abf54280` (tep's pin), tep, spinelkit, Ruby 4.0.9 + prism (pixi) | Either build Spinel **inside the arm64 root** and compile with `--cc="cc -static"`, or follow lpay's zig cross-build (`paulca/lpay` `spinel/BUILD-arm64.md`) | ok, static (arm64 root route) |

+

+## The arm64 root

+

+`scripts/arm64.sh <command>` runs a command in a Debian 13 arm64 root

+filesystem (`~/.local/share/bakeoff-toolchains/arm64root`) through proot

+and qemu-user. It needs no root access and no binfmt. The repo, `$TMPDIR`

+and the toolchains directory are visible inside at the same paths.

+Everything in it runs emulated, so builds are several times slower than

+native. Install extra Debian packages with

+`scripts/arm64.sh apt-get install -y <pkg>`.

+

+To run any arm64 binary on docs-t1:

+`qemu-aarch64-static -L ~/.local/share/bakeoff-toolchains/arm64root <binary>`.

+

+## Notes for builders

+

+- **OCaml and Haskell:** the x86 compiler versions differ from the arm64

+ root's (OCaml 5.3.0 in both; GHC 9.10.3 on x86 but 9.6.6 in the root).

+ Choose library versions that build with the arm64 root's compiler, or

+ install a matching one inside the root (ghcup and opam both work

+ there, emulated).

+- **Static glibc** (OCaml, Haskell, Spinel via the arm64 root) warns about

+ `getaddrinfo` and `dlopen`. That's fine for an app that doesn't resolve

+ hostnames itself.

+- **Mojo:** the platform's binary kind takes a single file

+ (`carlos deploy --target`, "binary kind, single file"). If the Mojo

+ builder can't find a supported way to make a self-contained binary, the

+ app is `BLOCKED platform: single-file binaries only (Mojo needs

+ libKGENCompilerRTShared.so)`. Don't work around it.

+- **Fallback:** if a route above fails for a real app, the builder writes

+ `BLOCKED needs-arm64-host`, and the orchestrator arranges a native

+ arm64 machine.

diff --git a/scripts/arm64.sh b/scripts/arm64.sh

new file mode 100755

index 0000000..e097d9f

--- /dev/null

+++ b/scripts/arm64.sh

@@ -0,0 +1,11 @@

+#!/usr/bin/env bash

+# Runs a command inside the emulated Debian arm64 root (proot + qemu-user),

+# with the bakeoff repo mounted at the same path and $HOME/.cache bound in.

+# scripts/arm64.sh <command> [args...]

+set -euo pipefail

+TC=${BAKEOFF_TOOLCHAINS:-$HOME/.local/share/bakeoff-toolchains}

+repo=$(cd "$(dirname "$0")/.." && pwd)

+exec "$TC/proot/proot" -0 -q "$TC/qemu/qemu-aarch64-static" -r "$TC/arm64root" \

+ -b /dev -b /proc -b /sys -b /etc/resolv.conf -b "$repo" -b "${TMPDIR:-/tmp}" -b "$TC" \

+ -w "$PWD" /usr/bin/env -i HOME=/root PATH=/root/.local/bin:/root/.ghcup/bin:/root/.cargo/bin:/usr/local/bin:/usr/bin:/bin \

+ TERM=dumb LANG=C.UTF-8 "$@"

diff --git a/scripts/env.sh b/scripts/env.sh

new file mode 100755

index 0000000..48be8db

--- /dev/null

+++ b/scripts/env.sh

@@ -0,0 +1,5 @@

+# Source this to put the bakeoff toolchains on PATH: source scripts/env.sh

+export PATH="$HOME/.local/bin:$HOME/.pixi/bin:$HOME/.cargo/bin:$HOME/.ghcup/bin:$PATH"

+if command -v opam >/dev/null 2>&1 && [ -d "$HOME/.opam/bakeoff" ]; then

+ eval "$(opam env --switch bakeoff --set-switch 2>/dev/null)"

+fi

diff --git a/scripts/smoke-spinel.sh b/scripts/smoke-spinel.sh

new file mode 100755

index 0000000..cd95849

--- /dev/null

+++ b/scripts/smoke-spinel.sh

@@ -0,0 +1,19 @@

+#!/usr/bin/env bash

+# Builds a Ruby file with Spinel into a static linux-arm64 binary, using a

+# Spinel compiler built natively inside the emulated arm64 root.

+# scripts/smoke-spinel.sh <main.rb> <output>

+set -euo pipefail

+here=$(cd "$(dirname "$0")" && pwd)

+TC=${BAKEOFF_TOOLCHAINS:-$HOME/.local/share/bakeoff-toolchains}

+sp="$TC/spinel-arm64"

+if ! [ -x "$sp/bin/spinel" ]; then

+ rm -rf "$sp" && cp -r "$TC/spinel-src/spinel" "$sp"

+ (cd "$sp" && git clean -fdxq -e vendor/ && "$here/arm64.sh" make -j8 >/dev/null)

+fi

+src=$(cd "$(dirname "$1")" && pwd)/$(basename "$1")

+out=$(cd "$(dirname "$2")" && pwd)/$(basename "$2")

+work=$(mktemp -d -p "${TMPDIR:-$HOME/.cache}")

+trap 'rm -rf "$work"' EXIT

+cp "$src" "$work/main.rb"

+(cd "$work" && "$here/arm64.sh" "$sp/bin/spinel" main.rb -o app --cc="cc -static")

+cp "$work/app" "$out"

diff --git a/scripts/smoke.sh b/scripts/smoke.sh

new file mode 100755

index 0000000..adebbcc

--- /dev/null

+++ b/scripts/smoke.sh

@@ -0,0 +1,36 @@

+#!/usr/bin/env bash

+# Proves a toolchain can make a runnable linux-arm64 binary: builds the

+# hello-world in scripts/smoke/<lang>, checks it is aarch64 and static (or

+# libc-only), and runs it under qemu. Prints "<lang>: ok" on success.

+set -euo pipefail

+here=$(cd "$(dirname "$0")" && pwd)

+source "$here/env.sh"

+lang=$1

+src="$here/smoke/$lang"

+out=$(mktemp -d -p "${TMPDIR:-$HOME/.cache}")

+trap 'rm -rf "$out"' EXIT

+bin="$out/smoke"

+case $lang in

+ go) (cd "$src" && CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -o "$bin" .) ;;

+ rust) (cd "$src" && CARGO_TARGET_DIR="$out/target" cargo zigbuild -q --release --target aarch64-unknown-linux-musl &&

+ cp "$out/target/aarch64-unknown-linux-musl/release/smoke" "$bin") ;;

+ zig) (cd "$out" && zig build-exe -target aarch64-linux-musl -O ReleaseSmall -femit-bin="$bin" "$src/main.zig") ;;

+ swift) (cd "$src" && swift build -q -c release --swift-sdk aarch64-swift-linux-musl --scratch-path "$out/build" &&

+ cp "$out/build/aarch64-swift-linux-musl/release/smoke" "$bin") ;;

+ ocaml) cp "$src/main.ml" "$out/" && (cd "$out" && "$here/arm64.sh" ocamlfind ocamlopt -package unix -linkpkg -ccopt -static main.ml -o smoke) ;;

+ haskell) cp "$src/Main.hs" "$out/" && (cd "$out" && "$here/arm64.sh" ghc -O -static -optl-static -optl-pthread Main.hs -o smoke >/dev/null) ;;

+ mojo) cp "$src/main.mojo" "$out/" && (cd "$out" && "$here/arm64.sh" /root/.pixi/bin/mojo build main.mojo -o smoke 2>/dev/null) ;;

+ spinel) "$here/smoke-spinel.sh" "$src/main.rb" "$bin" ;;

+ *) echo "unknown language $lang" >&2; exit 2 ;;

+esac

+desc=$(file -b "$bin")

+echo "$lang: $desc" | cut -c1-160

+case $desc in *aarch64*) ;; *) echo "$lang: not an aarch64 binary" >&2; exit 1 ;; esac

+case $desc in *"statically linked"*|*"static-pie"*) ;; *)

+ extra=$(readelf -d "$bin" | sed -n 's/.*(NEEDED).*\[$.*$\]/\1/p' | grep -Ev '^(libc|libm|libpthread|libdl|librt|ld-linux-aarch64)\.so' || true)

+ [ -z "$extra" ] || { echo "$lang: needs shared libraries beyond libc: $extra" >&2; exit 1; } ;;

+esac

+got=$(qemu-aarch64-static -L "${BAKEOFF_TOOLCHAINS:-$HOME/.local/share/bakeoff-toolchains}/arm64root" "$bin")

+[ "$got" = ok ] || { echo "$lang: printed '$got', want ok" >&2; exit 1; }

+echo "$lang: ok"

diff --git a/scripts/smoke/go/go.mod b/scripts/smoke/go/go.mod

new file mode 100644

index 0000000..3bcd72f

--- /dev/null

+++ b/scripts/smoke/go/go.mod

@@ -0,0 +1,3 @@

+module smoke

+

+go 1.26

diff --git a/scripts/smoke/go/main.go b/scripts/smoke/go/main.go

new file mode 100644

index 0000000..becbb8d

--- /dev/null

+++ b/scripts/smoke/go/main.go

@@ -0,0 +1,5 @@

+package main

+

+import "fmt"

+

+func main() { fmt.Println("ok") }

diff --git a/scripts/smoke/haskell/Main.hs b/scripts/smoke/haskell/Main.hs

new file mode 100644

index 0000000..34671a3

--- /dev/null

+++ b/scripts/smoke/haskell/Main.hs

@@ -0,0 +1,2 @@

+main :: IO ()

+main = putStrLn "ok"

diff --git a/scripts/smoke/mojo/main.mojo b/scripts/smoke/mojo/main.mojo

new file mode 100644

index 0000000..3a7d734

--- /dev/null

+++ b/scripts/smoke/mojo/main.mojo

@@ -0,0 +1,2 @@

+def main():

+ print("ok")

diff --git a/scripts/smoke/ocaml/main.ml b/scripts/smoke/ocaml/main.ml

new file mode 100644

index 0000000..75b7fcf

--- /dev/null

+++ b/scripts/smoke/ocaml/main.ml

@@ -0,0 +1 @@

+let () = print_endline "ok"

diff --git a/scripts/smoke/rust/Cargo.lock b/scripts/smoke/rust/Cargo.lock

new file mode 100644

index 0000000..cda7d8d

--- /dev/null

+++ b/scripts/smoke/rust/Cargo.lock

@@ -0,0 +1,7 @@

+# This file is automatically @generated by Cargo.

+# It is not intended for manual editing.

+version = 4

+

+[[package]]

+name = "smoke"

+version = "0.1.0"

diff --git a/scripts/smoke/rust/Cargo.toml b/scripts/smoke/rust/Cargo.toml

new file mode 100644

index 0000000..4a8d6b4

--- /dev/null

+++ b/scripts/smoke/rust/Cargo.toml

@@ -0,0 +1,4 @@

+[package]

+name = "smoke"

+version = "0.1.0"

+edition = "2021"

diff --git a/scripts/smoke/rust/src/main.rs b/scripts/smoke/rust/src/main.rs

new file mode 100644

index 0000000..87daa7a

--- /dev/null

+++ b/scripts/smoke/rust/src/main.rs

@@ -0,0 +1,3 @@

+fn main() {

+ println!("ok");

+}

diff --git a/scripts/smoke/spinel/main.rb b/scripts/smoke/spinel/main.rb

new file mode 100644

index 0000000..ed1c3a2

--- /dev/null

+++ b/scripts/smoke/spinel/main.rb

@@ -0,0 +1 @@

+puts "ok"

diff --git a/scripts/smoke/swift/Package.swift b/scripts/smoke/swift/Package.swift

new file mode 100644

index 0000000..3d0f7d1

--- /dev/null

+++ b/scripts/smoke/swift/Package.swift

@@ -0,0 +1,4 @@

+// swift-tools-version:6.0

+import PackageDescription

+

+let package = Package(name: "smoke", targets: [.executableTarget(name: "smoke")])

diff --git a/scripts/smoke/swift/Sources/smoke/main.swift b/scripts/smoke/swift/Sources/smoke/main.swift

new file mode 100644

index 0000000..c9f9b6d

--- /dev/null

+++ b/scripts/smoke/swift/Sources/smoke/main.swift

@@ -0,0 +1 @@

+print("ok")

diff --git a/scripts/smoke/zig/main.zig b/scripts/smoke/zig/main.zig

new file mode 100644

index 0000000..aa5d619

--- /dev/null

+++ b/scripts/smoke/zig/main.zig

@@ -0,0 +1,8 @@

+const std = @import("std");

+

+pub fn main(init: std.process.Init) !void {

+ var buf: [64]u8 = undefined;

+ var out: std.Io.File.Writer = .init(.stdout(), init.io, &buf);

+ try out.interface.writeAll("ok\n");

+ try out.interface.flush();

+}

diff --git a/scripts/toolchains.sh b/scripts/toolchains.sh

new file mode 100755

index 0000000..a44249f

--- /dev/null

+++ b/scripts/toolchains.sh

@@ -0,0 +1,121 @@

+#!/usr/bin/env bash

+# Installs every bakeoff toolchain under $HOME (no sudo), idempotently.

+# scripts/toolchains.sh [section...] sections: base qemu zig rust swift ocaml haskell mojo spinel arm64root

+# Then `source scripts/env.sh` puts them on PATH. Versions land in docs/toolchains.md.

+set -euo pipefail

+export PIXI_NO_TELEMETRY=1

+here=$(cd "$(dirname "$0")" && pwd)

+TC=${BAKEOFF_TOOLCHAINS:-$HOME/.local/share/bakeoff-toolchains}

+mkdir -p "$TC" "$HOME/.local/bin"

+dl=$(mktemp -d -p "${TMPDIR:-$HOME/.cache}")

+trap 'rm -rf "$dl"' EXIT

+source "$here/env.sh"

+

+have() { command -v "$1" >/dev/null 2>&1; }

+

+sec_base() { # pixi, plus conda-forge packages for headers and tools Debian lacks here

+ have pixi || curl -fsSL https://pixi.sh/install.sh | PIXI_NO_PATH_UPDATE=1 bash

+ pixi global install --environment bakeoff-base sqlite unzip pkg-config ruby gmp libffi yaml

+}

+

+sec_qemu() { # user-mode arm64 emulator and proot, run explicitly (no binfmt, no root)

+ if ! [ -x "$TC/qemu/qemu-aarch64-static" ]; then

+ # Debian 13's qemu-user ships statically linked binaries (qemu-user-static is transitional)

+ (cd "$dl" && apt-get download qemu-user >/dev/null)

+ dpkg-deb -x "$dl"/qemu-user_*.deb "$dl/q"

+ mkdir -p "$TC/qemu"

+ cp "$dl/q/usr/bin/qemu-aarch64" "$TC/qemu/qemu-aarch64-static"

+ fi

+ ln -sf "$TC/qemu/qemu-aarch64-static" "$HOME/.local/bin/qemu-aarch64-static"

+ if ! [ -x "$TC/proot/proot" ]; then

+ mkdir -p "$TC/proot"

+ curl -fsSL -o "$TC/proot/proot" https://proot.gitlab.io/proot/bin/proot

+ chmod +x "$TC/proot/proot"

+ fi

+ ln -sf "$TC/proot/proot" "$HOME/.local/bin/proot"

+}

+

+sec_zig() {

+ if ! [ -x "$TC/zig/zig" ]; then

+ url=$(curl -fsSL https://ziglang.org/download/index.json |

+ python3 -c 'import json,sys; d=json.load(sys.stdin); v=[k for k in d if k!="master"]; v.sort(key=lambda s:[int(x) for x in s.split(".")]); print(d[v[-1]]["x86_64-linux"]["tarball"])')

+ curl -fsSL "$url" | tar -xJ -C "$dl"

+ rm -rf "$TC/zig" && mv "$dl"/zig-* "$TC/zig"

+ fi

+ ln -sf "$TC/zig/zig" "$HOME/.local/bin/zig"

+}

+

+sec_rust() {

+ have rustup || curl -fsSL https://sh.rustup.rs | sh -s -- -y --no-modify-path --profile minimal

+ rustup toolchain install stable --profile minimal

+ rustup target add aarch64-unknown-linux-musl x86_64-unknown-linux-musl

+ have cargo-zigbuild || cargo install --locked cargo-zigbuild

+}

+

+sec_swift() { # Swift 6.2 is installed; add the Static Linux SDK (musl, x86_64 + aarch64)

+ swift sdk list 2>/dev/null | grep -q static-linux && return

+ v=$(swift --version 2>&1 | sed -n 's/.*Swift version $[0-9.]*$.*/\1/p')

+ idx=$(curl -fsSL "https://www.swift.org/api/v1/install/releases.json")

+ read -r url sum < <(python3 -c '

+import json,sys; v=sys.argv[1]; d=json.loads(sys.argv[2])

+r=[x for x in d if x["name"]==v][0]

+s=[p for p in r["platforms"] if p.get("platform")=="static-sdk" or p.get("name","").lower().startswith("static")][0]

+print("https://download.swift.org/swift-%s-release/static-sdk/swift-%s-RELEASE/swift-%s-RELEASE_static-linux-0.0.1.artifactbundle.tar.gz"%(v,v,v), s["checksum"])' "$v" "$idx")

+ swift sdk install "$url" --checksum "$sum"

+}

+

+sec_ocaml() { # opam + OCaml 5 for x86-64; arm64 builds run in the arm64 root (see arm64root)

+ if ! have opam; then

+ curl -fsSL -o "$HOME/.local/bin/opam" "$(curl -fsSL https://api.github.com/repos/ocaml/opam/releases/latest |

+ python3 -c 'import json,sys; print([a["browser_download_url"] for a in json.load(sys.stdin)["assets"] if a["name"].endswith("x86_64-linux")][0])')"

+ chmod +x "$HOME/.local/bin/opam"

+ fi

+ [ -d "$HOME/.opam" ] || opam init --bare -n --disable-sandboxing

+ opam switch list -s | grep -qx bakeoff || opam switch create bakeoff ocaml-base-compiler.5.3.0 -y

+ opam install --switch bakeoff -y dune

+}

+

+sec_haskell() { # ghcup + recommended GHC and cabal for x86-64

+ if ! [ -x "$HOME/.ghcup/bin/ghcup" ]; then

+ curl -fsSL https://get-ghcup.haskell.org | BOOTSTRAP_HASKELL_NONINTERACTIVE=1 BOOTSTRAP_HASKELL_MINIMAL=1 sh

+ fi

+ ghcup install ghc recommended --set

+ ghcup install cabal recommended --set

+}

+

+sec_mojo() { # Mojo through pixi, as a global environment

+ pixi global install --environment mojo -c https://conda.modular.com/max -c conda-forge mojo

+}

+

+sec_spinel() { # spinel + tep + spinelkit at the pins lpay uses (paulca/lpay spinel/)

+ src="$TC/spinel-src"; mkdir -p "$src"

+ [ -d "$src/lpay" ] || gh repo clone paulca/lpay "$src/lpay" -- --depth 1

+ [ -d "$src/tep" ] || git clone https://github.com/OriPekelman/tep "$src/tep"

+ [ -d "$src/spinelkit" ] || git clone https://github.com/OriPekelman/spinelkit "$src/spinelkit"

+ [ -d "$src/spinel" ] || git clone https://github.com/matz/spinel "$src/spinel"

+ pin=$(head -1 "$src/tep/SPINEL_PIN")

+ (cd "$src/spinel" && git fetch -q origin && git checkout -q "$pin" && make deps && make)

+ ln -sf "$src/spinel/bin/spinel" "$HOME/.local/bin/spinel"

+ ln -sf "$src/tep/bin/tep" "$HOME/.local/bin/tep"

+}

+

+sec_arm64root() { # a Debian arm64 root filesystem, run through proot + qemu (no root needed)

+ root="$TC/arm64root"

+ if ! [ -x "$root/bin/sh" ]; then

+ url=$(curl -fsSL https://images.linuxcontainers.org/meta/1.0/index-system |

+ awk -F';' '$1=="debian" && $2=="trixie" && $3=="arm64" && $4=="default" {print "https://images.linuxcontainers.org"$6"rootfs.tar.xz"}' | tail -1)

+ mkdir -p "$root"

+ curl -fsSL "$url" | tar -xJ -C "$root" --exclude='dev/*'

+ fi

+ rm -f "$root/etc/resolv.conf"; cp /etc/resolv.conf "$root/etc/resolv.conf"

+ "$here/arm64.sh" sh -c 'apt-get update -qq && apt-get install -y -qq build-essential curl git ca-certificates libgmp-dev libsqlite3-dev zlib1g-dev libffi-dev pkg-config xz-utils file ocaml-nox ocaml-findlib opam ghc cabal-install libghc-network-dev >/dev/null'

+ # Mojo for arm64, natively inside the root (pixi's linux-aarch64 build)

+ "$here/arm64.sh" sh -c '[ -x /root/.pixi/bin/mojo ] || { export PIXI_NO_TELEMETRY=1; curl -fsSL https://pixi.sh/install.sh | PIXI_NO_PATH_UPDATE=1 bash >/dev/null; /root/.pixi/bin/pixi global install --environment mojo -c https://conda.modular.com/max -c conda-forge mojo; }'

+}

+

+sections=("$@")

+[ ${#sections[@]} -gt 0 ] || sections=(base qemu zig rust swift ocaml haskell mojo spinel arm64root)

+for s in "${sections[@]}"; do

+ echo "== $s"

+ "sec_$s"

+done

carlos / todomvc-bakeoff Public

chore: toolchains and arm64 smoke builds