amadan
Merge remote-tracking branch 'origin/main' into egress-and-minting-rulings
10 files changed,
+498
−262
.amadan/README.md+73 −36.amadan/ci.d/10-gate+43 −12.amadan/ci.d/{20-cross@3 => 20-cross@5}+15 −6.amadan/ci.d/30-minio-contract+323 −0.github/workflows/ci.yml+0 −186.gitignore+5 −0AGENTS.md+24 −14Makefile+8 −2internal/host/statfs_unix.go+1 −1internal/objstore/contract_test.go+6 −5
diff --git a/.amadan/README.md b/.amadan/README.md| index e35f10661..8d81510a1 100644 |
| --- a/.amadan/README.md |
| +++ b/.amadan/README.md |
| @@ -1,41 +1,78 @@ |
| # platform's CI on amadan |
| -`ci.d/` is a second home for this repo's gate, running on a self-hosted |
| -amadan runner. **`.github/workflows/ci.yml` remains the required check |
| -and remains the definition of green.** Where the two disagree, the |
| -workflow is the one that gates a merge, and the difference is a bug |
| -here. |
| +`ci.d/` is this repo's gate, running on a self-hosted amadan runner. |
| +**It is the only one.** `.github/workflows/ci.yml` was deleted on |
| +2026-08-24; nothing stands behind these three steps. A green run here |
| +is the whole of what "green" means for this repo, and a red one is not |
| +a discrepancy to reconcile against a more official run somewhere else. |
| -| workflow job | here | | |
| +The workflow went for two reasons that arrived together. The steps here |
| +had caught up with it and then gone past it — the last gap, `gofmt`, |
| +closed by *widening*, so this gate is now strictly the tighter of the |
| +two (see below). And GitHub Actions on the `carlosframework` account |
| +had stopped starting jobs at all over billing, so the "required check" |
| +had been gating nothing for some time before anyone removed it. The |
| +evidence for the first half is the run on `cd47f38`, green on the real |
| +arm64 worker with all four sections in the log: `=== gate ===` |
| +including `gofmt -l .: clean`, `=== cross ===` with all five targets, |
| +and `=== minio-contract ===` with `--- PASS: TestDeliveredContractMinio` |
| +and its five subtests. |
| + |
| +## Where each step came from |
| + |
| +This table is provenance, not a checklist against a live file. Every |
| +step here is shaped the way it is because of the job it descends from, |
| +and dropping that history makes the scripts read as arbitrary choices |
| +rather than as ported ones. The left column names a job in a file that |
| +no longer exists. |
| + |
| +| descended from | here | | |
| |---|---|---| |
| | `gate` | `10-gate` | build · vet · `go test ./... -count=1` | |
| -| `cross` | `20-cross@3` | one target per shard | |
| -| `minio-contract` | **not ported** | see below | |
| -| `gofmt` | **not ported** | see below | |
| - |
| -## What is deliberately missing, and why |
| - |
| -Two jobs are absent. Writing that down is the point of this file: a |
| -`ci.d/` directory that quietly covers less than the workflow next to it |
| -is worth less than no `ci.d/` at all, because it invites people to read |
| -a green run as meaning more than it does. |
| - |
| -**`minio-contract`** starts a MinIO server in Docker and runs |
| -`TestDeliveredContractMinio` against it. amadan CI jobs run under |
| -bubblewrap with the host filesystem read-only and no container runtime; |
| -`-isolate oci` is designed but unbuilt, precisely because "what |
| -container access does a job get, and does that leave any isolation at |
| -all" is unresolved. So this job stays on GitHub, where a service |
| -container is free. It is the only place that S3 contract runs live. |
| - |
| -**`gofmt`** checks only the files a PR changed, deliberately — three |
| -pre-existing unformatted files elsewhere are left alone, and a |
| -repo-wide check would fail on arrival and teach everyone to ignore the |
| -job. That needs the PR's **base SHA**, which is a GitHub concept. amadan |
| -CI hands a job its commit and its branch and has no notion of what it |
| -is a pull request against, so the job cannot be reproduced faithfully |
| -here. Reproducing it *unfaithfully* — repo-wide, or diffed against a |
| -guessed base — is worse than not running it. |
| - |
| -Both are gaps in this runner, not in the gate. The gate is still whole |
| -on GitHub. |
| +| `cross` | `20-cross@5` | one target per shard, all five | |
| +| `minio-contract` | `30-minio-contract` | MinIO on loopback, no container | |
| +| `gofmt` | `10-gate` | repo-wide, via `make test` — see below | |
| + |
| +## `gofmt`: ported by widening it, not by reproducing it |
| + |
| +This section is the reason it was safe to delete the workflow, so it is |
| +worth keeping even now that the comparison is with a dead file. The |
| +whole risk in a `ci.d/` that covers *less* than what it replaces is |
| +that it invites people to read a green run as meaning more than it |
| +does — and that risk did not end when the other gate did. It moved: |
| +there is nothing left to compare against, so a gap here is now a gap |
| +full stop. This file is the check on that, and it has to stay true. |
| + |
| +**`gofmt`** was the last gap, and it closed by widening rather than by |
| +imitation. The workflow checked only the files a PR changed, because it |
| +needed a **base SHA** — a GitHub concept. amadan CI hands a job its |
| +commit and its branch and has no notion of what it is a pull request |
| +against, so that job could not be reproduced faithfully here. |
| + |
| +It did not need to be. The scoping existed to protect three |
| +pre-existing unformatted files, and those are gone: `gofmt -l .` over |
| +the whole tree is clean today. So `10-gate` runs `make test`, whose |
| +recipe already contains a **repo-wide** gofmt check — strictly stronger |
| +than the job it replaced, and one definition shared with what a |
| +developer runs locally rather than a second list that can drift. |
| + |
| +The rule that survives from the old scoping is the one worth keeping: |
| +if a repo-wide check ever starts failing on arrival, fix the tree |
| +rather than narrowing the check, because a gate people learn to ignore |
| +is not a gate. |
| + |
| +## `minio-contract`, and why it turned out to be portable |
| + |
| +It was listed here as impossible: it started MinIO in Docker, and amadan |
| +jobs get no container runtime. That conflated the job with its |
| +packaging. MinIO and `mc` are single static Go binaries, and |
| +`TestDeliveredContractMinio` only wants an endpoint and credentials over |
| +HTTP — it has no opinion about what is serving. So `30-minio-contract` |
| +downloads both binaries at pinned versions, verifies them against pinned |
| +sha256 digests, and runs the server on loopback inside the job's |
| +existing bubblewrap sandbox. Nothing persists on the box. |
| + |
| +The test **skips itself** unless `CARLOS_MINIO_TEST=1`, and a skipped |
| +test exits 0. The step therefore requires the parent's `--- PASS:` line |
| +to appear in the output and fails without it, so a dropped env var |
| +cannot turn this into a green step that exercised nothing. |
diff --git a/.amadan/ci.d/10-gate b/.amadan/ci.d/10-gate| index 9276c1635..434170661 100755 |
| --- a/.amadan/ci.d/10-gate |
| +++ b/.amadan/ci.d/10-gate |
| @@ -3,18 +3,49 @@ |
| # ./... -count=1 — all clean, every package, before any commit lands" — |
| # as amadan CI runs it. |
| # |
| -# A SECOND home for that gate, not a replacement. |
| -# .github/workflows/ci.yml stays the required check and stays the |
| -# definition of green; where the two disagree, the workflow is the one |
| -# that gates a merge and the difference is a bug in this file. |
| -# |
| -# -count=1 is the same rule for the same reason it is there: never |
| -# satisfy the gate from a cached pass. The 30m timeout is above Go's |
| -# 10m default because the root package's e2e tests compile the carlos |
| -# binary many times over — copied from the workflow rather than |
| -# re-derived, so a change there is a change in one place. |
| +# THIS IS THE GATE. Not a second home for one, not a copy kept beside a |
| +# more authoritative run somewhere else: the definition of green for this |
| +# repo is what these three steps say it is. |
| +# |
| +# It did not start that way, and the change is recent enough to be worth |
| +# writing down. Until 2026-08-24 the authority was |
| +# .github/workflows/ci.yml, and this file said so in as many words — |
| +# where the two disagreed, the workflow won and the difference was a bug |
| +# here. That workflow has been deleted. Two things made it safe to |
| +# delete, and they arrived together: |
| +# |
| +# 1. This directory caught up, and then went past it. `10-gate`, |
| +# `20-cross@5` and `30-minio-contract` cover every job the workflow |
| +# ran — build/vet/test, all five cross-compile targets, the live |
| +# MinIO contract — and the gofmt check here is STRICTER: `make |
| +# test` runs `gofmt -l .` over the whole tree, where the workflow |
| +# only ever checked the files a PR happened to touch. There is no |
| +# direction in which the deleted file was the tighter gate. |
| +# |
| +# 2. It had stopped running anyway. GitHub Actions on the |
| +# carlosframework account has been refusing to start jobs over |
| +# billing, so the "required check" was gating nothing well before |
| +# anyone removed it — which is the worse of the two failure modes, |
| +# because a gate that is merely absent is at least honest about it. |
| +# |
| +# So: a red step here is not a discrepancy to reconcile against a more |
| +# official run. It is the answer, and nothing behind it is going to catch |
| +# what it misses. Keep it true. |
| +# |
| +# One definition, not two. Everything past the build is `make test` — the |
| +# same target a maintainer runs on a laptop — so CI and the local gate |
| +# cannot drift apart, and a check added to one is a check added to both. |
| +# That is how the gofmt check gets here at all: `make test` runs vet, a |
| +# repo-wide `gofmt -l .`, then the tests, and this file no longer restates |
| +# any of it. |
| +# |
| +# `go build ./...` stays spelled out because `make test` does not do it — |
| +# the Makefile's test target starts at vet. |
| +# |
| +# -count=1 and -timeout 30m moved to the Makefile's go test line along with |
| +# the reasoning for them; they are still in force here, just no longer |
| +# written down twice. |
| set -eu |
| go build ./... |
| -go vet ./... |
| -go test ./... -count=1 -timeout 30m |
| +exec make test |
diff --git a/.amadan/ci.d/20-cross@3 b/.amadan/ci.d/20-cross@5| similarity index 66% |
| rename from .amadan/ci.d/20-cross@3 |
| rename to .amadan/ci.d/20-cross@5 |
| index 6f79790c5..9dec4b177 100755 |
| --- a/.amadan/ci.d/20-cross@3 |
| +++ b/.amadan/ci.d/20-cross@5 |
| @@ -9,9 +9,16 @@ |
| # built it was on Linux. Cross-compiling catches it in seconds and needs |
| # no Mac. |
| # |
| -# The @3 is amadan's whole sharding interface: it runs this script three |
| +# Windows is a CLIENT target: `carlos deploy` from a Windows workstation |
| +# against a Linux fleet. The box-side packages still have to COMPILE |
| +# there, because the CLI is the same binary as the agent, so unix-only |
| +# syscalls live behind per-OS files (internal/edge/purge_listen_*.go, |
| +# internal/host/{statfs,flock}_*.go). These two shards are what stops |
| +# the next syscall.Foo from silently un-shipping the Windows build. |
| +# |
| +# The @5 is amadan's whole sharding interface: it runs this script five |
| # times concurrently and tells each copy which one it is. One target per |
| -# shard rather than three builds in a row, because they are independent |
| +# shard rather than five builds in a row, because they are independent |
| # and this box has the cores — and because a failing target then names |
| # itself in the shard that failed, instead of being the first line of a |
| # combined log. |
| @@ -22,12 +29,12 @@ shards=${AMADAN_SHARD_COUNT:?this step must be run by amadan CI, which sets AMAD |
| # The fence. The targets below are a fixed list, and the shard count in |
| # this file's name is what deals them out — so the two MUST agree. Rename |
| -# this to @2 and, without this check, one platform silently stops being |
| +# this to @4 and, without this check, one platform silently stops being |
| # compiled while CI stays green. That is the exact shape of the bug the |
| # job exists to catch, arriving through the job that catches it. |
| -if [ "$shards" -ne 3 ]; then |
| - echo "this step deals 3 fixed targets but was run with $shards shards —" |
| - echo "rename the file back to 20-cross@3, or add a target below" |
| +if [ "$shards" -ne 5 ]; then |
| + echo "this step deals 5 fixed targets but was run with $shards shards —" |
| + echo "rename the file back to 20-cross@5, or add a target below" |
| exit 1 |
| fi |
| @@ -35,6 +42,8 @@ case "$shard" in |
| 0) GOOS=darwin GOARCH=arm64 ;; |
| 1) GOOS=darwin GOARCH=amd64 ;; |
| 2) GOOS=linux GOARCH=arm64 ;; |
| +3) GOOS=windows GOARCH=amd64 ;; |
| +4) GOOS=windows GOARCH=arm64 ;; |
| *) echo "no target for shard $shard"; exit 1 ;; |
| esac |
diff --git a/.amadan/ci.d/30-minio-contract b/.amadan/ci.d/30-minio-contract| new file mode 100755 |
| index 000000000..37cac5ee0 |
| --- /dev/null |
| +++ b/.amadan/ci.d/30-minio-contract |
| @@ -0,0 +1,323 @@ |
| +#!/bin/sh |
| +# The S3 delivered-contract gate: internal/objstore's |
| +# TestDeliveredContractMinio, run against a real MinIO, with no container |
| +# runtime anywhere in sight. |
| +# |
| +# THIS IS THE ONLY PLACE THIS TEST RUNS LIVE. It began as a port of the |
| +# `minio-contract` job in .github/workflows/ci.yml, with that workflow |
| +# holding the final say; the workflow has since been deleted (10-gate's |
| +# header records why), so there is no longer a second run of this |
| +# anywhere. If this step does not exercise the contract against a real |
| +# server, nothing does — `TestDeliveredContractMinio` skips itself |
| +# without CARLOS_MINIO_TEST, and a skipped test is a green test. |
| +# |
| +# References to "the workflow" below are provenance: they say where a |
| +# choice came from and where this deliberately diverged from it. They |
| +# describe a file that no longer exists. |
| +# |
| +# WHY THIS CAN EXIST HERE AT ALL |
| +# |
| +# .amadan/README.md used to list this job as "not ported", on the grounds |
| +# that it starts MinIO in Docker and amadan jobs get no container |
| +# runtime. That reasoning conflated the job with its packaging. MinIO and |
| +# mc are single static Go binaries, published per-arch at pinned archive |
| +# URLs on dl.min.io, and contract_test.go only ever wanted MINIO_ENDPOINT |
| +# plus credentials over HTTP — it has no opinion about what is serving. |
| +# So the container was never load-bearing; it was just how GitHub happens |
| +# to hand you a MinIO. |
| +# |
| +# Running the server as a plain process inside the job's existing |
| +# bubblewrap sandbox was, if anything, tighter than the container the |
| +# workflow used, and still is the tighter of the two shapes: it binds |
| +# loopback only, its data directory is the job's private /tmp tmpfs, and |
| +# it dies with the job's PID namespace. Nothing persists on the box. |
| +set -eu |
| + |
| +# Fail on a missing tool by name rather than by a confusing error forty |
| +# lines down. Both are present on the runner; this is here so that a |
| +# stripped-down box says which one it is missing. |
| +for tool in curl sha256sum; do |
| + command -v "$tool" >/dev/null 2>&1 || { |
| + echo "$tool is required by this step but was not found on PATH" >&2 |
| + exit 1 |
| + } |
| +done |
| + |
| +# --------------------------------------------------------------------- |
| +# The pins. |
| +# |
| +# The workflow pinned its MinIO image and its mc download, and said why: |
| +# a floating minio tag has already broken this exact job class once, when |
| +# mc's admin-policy syntax churned across versions. This started as a |
| +# port of that pin, not an upgrade — RELEASE.2025-09-07T16-13-09Z is the |
| +# same server release that image digest resolved to, and the mc release |
| +# is the same one it downloaded. |
| +# |
| +# But a version string is a weaker pin than a container digest, which the |
| +# workflow got for free and a plain URL does not: |
| +# dl.min.io could serve different bytes at the same archive path and |
| +# nothing would notice. So each binary carries its sha256, verified after |
| +# download and again on every cache hit, and a mismatch refuses to run |
| +# rather than testing whatever arrived. The four digests below were taken |
| +# from upstream's own published .sha256sum files alongside each archive |
| +# (and independently confirmed against the downloaded bytes) on |
| +# 2026-08-23. |
| +# |
| +# Bump the two releases together, and re-derive all four digests when you |
| +# do. |
| +MINIO_RELEASE=RELEASE.2025-09-07T16-13-09Z |
| +MC_RELEASE=RELEASE.2025-08-13T08-35-41Z |
| + |
| +# Arch-aware on purpose. The amadan runner is arm64; the workflow's |
| +# ubuntu-latest was amd64, and it hardwired linux-amd64 for its mc |
| +# download — harmless there, wrong here, and a bug not to carry across. |
| +# The runner fleet is also the kind of thing that changes under you, so |
| +# this reads uname rather than trusting a note about what the box is. |
| +case "$(uname -m)" in |
| +x86_64 | amd64) |
| + arch=amd64 |
| + minio_sha=7c5bd8512c6e966455b1d198209358b2d191c77a83ab377c4073281065fb855f |
| + mc_sha=01f866e9c5f9b87c2b09116fa5d7c06695b106242d829a8bb32990c00312e891 |
| + ;; |
| +aarch64 | arm64) |
| + arch=arm64 |
| + minio_sha=5c83cd2cf151717ba0243f73e1c7802ff36e272b67144bdd7f1f7d684fd6f03d |
| + mc_sha=14c8c9616cfce4636add161304353244e8de383b2e2752c0e9dad01d4c27c12c |
| + ;; |
| +*) |
| + echo "no pinned MinIO build for $(uname -m) — add one above, do not guess" >&2 |
| + exit 1 |
| + ;; |
| +esac |
| + |
| +echo "minio contract: $arch, minio $MINIO_RELEASE, mc $MC_RELEASE" |
| + |
| +# --------------------------------------------------------------------- |
| +# The cache. |
| +# |
| +# amadan reuses one workspace per repo across jobs and does not clean |
| +# untracked files between them by default, so an untracked directory in |
| +# the checkout IS the cache — no separate cache action, no key, nothing |
| +# to expire. ~140MB of binaries then downloads once per worker rather |
| +# than once per run. |
| +# |
| +# The dot prefix is deliberate: the Go toolchain skips dot-directories |
| +# when walking ./..., so this cannot leak into `go build ./...` or |
| +# `go vet ./...` in 10-gate no matter what ends up inside it. The digest |
| +# check below is what makes reusing the cache safe — a truncated or |
| +# half-written binary from an interrupted job fails verification and is |
| +# re-fetched, so a cache hit is never taken on trust. |
| +cache="$PWD/.amadan-cache/minio-$arch" |
| +mkdir -p "$cache" |
| + |
| +fetch_pinned() { |
| + fp_name=$1 |
| + fp_url=$2 |
| + fp_want=$3 |
| + fp_dest=$4 |
| + |
| + if [ -f "$fp_dest" ] && [ "$(sha256sum <"$fp_dest" | cut -d' ' -f1)" = "$fp_want" ]; then |
| + echo " $fp_name: cached, digest verified" |
| + return 0 |
| + fi |
| + |
| + echo " $fp_name: downloading $fp_url" |
| + curl -sfL --retry 3 -o "$fp_dest.part" "$fp_url" |
| + |
| + fp_got=$(sha256sum <"$fp_dest.part" | cut -d' ' -f1) |
| + if [ "$fp_got" != "$fp_want" ]; then |
| + rm -f "$fp_dest.part" |
| + echo "$fp_name digest mismatch — refusing to run" >&2 |
| + echo " want $fp_want" >&2 |
| + echo " got $fp_got" >&2 |
| + echo " from $fp_url" >&2 |
| + exit 1 |
| + fi |
| + |
| + chmod +x "$fp_dest.part" |
| + mv "$fp_dest.part" "$fp_dest" |
| + echo " $fp_name: downloaded, digest verified" |
| +} |
| + |
| +fetch_pinned minio \ |
| + "https://dl.min.io/server/minio/release/linux-$arch/archive/minio.$MINIO_RELEASE" \ |
| + "$minio_sha" "$cache/minio" |
| +fetch_pinned mc \ |
| + "https://dl.min.io/client/mc/release/linux-$arch/archive/mc.$MC_RELEASE" \ |
| + "$mc_sha" "$cache/mc" |
| + |
| +minio=$cache/minio |
| +mc=$cache/mc |
| + |
| +# --------------------------------------------------------------------- |
| +# Scratch state, and getting rid of it. |
| +# |
| +# Everything mutable this step creates lives under one directory: MinIO's |
| +# data, mc's config (MC_CONFIG_DIR, so mc never touches $HOME), the |
| +# policy document, and the server log. /tmp inside an amadan job is a |
| +# private tmpfs that the sandbox discards when the job ends, so this is |
| +# scratch in the strongest sense available. |
| +rundir=$(mktemp -d "${TMPDIR:-/tmp}/carlos-minio-contract-XXXXXX") |
| +MC_CONFIG_DIR=$rundir/mc |
| +export MC_CONFIG_DIR |
| + |
| +minio_pid= |
| + |
| +# The trap runs on the success path and on every failure path, because |
| +# `set -eu` exits through it — which is the point. A step that leaves a |
| +# MinIO listening on a shared box after it fails would poison every later |
| +# run on that worker, and the failure that caused it is exactly when |
| +# nobody is watching. |
| +cleanup() { |
| + if [ -n "$minio_pid" ]; then |
| + kill "$minio_pid" 2>/dev/null || true |
| + wait "$minio_pid" 2>/dev/null || true |
| + fi |
| + rm -rf "$rundir" |
| +} |
| +trap cleanup EXIT INT TERM |
| + |
| +# Root credentials are minted per run rather than hardcoded. They are not |
| +# a secret — the server is loopback-only and lives for seconds — but |
| +# random ones make the identity check below meaningful: creds that only |
| +# this process knows cannot authenticate against somebody else's MinIO. |
| +root_user=root$$ |
| +root_pass=$(head -c 18 /dev/urandom | od -An -tx1 | tr -d ' \n') |
| + |
| +# --------------------------------------------------------------------- |
| +# Start the server. |
| +# |
| +# The workflow used a fixed port 9000 because its MinIO was in a |
| +# container with its own network. amadan jobs deliberately do NOT get |
| +# --unshare-net (they need the Go module proxy), so a job shares the |
| +# host's network |
| +# namespace with the worker and with anything else on it. 9000 is a |
| +# popular port; assuming it is free is assuming something this step does |
| +# not know. |
| +# |
| +# So: pick a high port, and if MinIO cannot take it, pick another. The |
| +# liveness probe alone would not be enough to tell the two cases apart — |
| +# a foreign MinIO already on the port answers /health/live perfectly |
| +# happily, and we would then test somebody else's server. Requiring OUR |
| +# process to still be alive is what distinguishes "it started" from "the |
| +# port was taken"; `mc alias set` with the random root credentials above |
| +# is the second, independent check that the thing answering is ours. |
| +port= |
| +attempt=1 |
| +while [ "$attempt" -le 5 ]; do |
| + candidate=$((20000 + ($$ + attempt * 997) % 20000)) |
| + rm -rf "$rundir/data" |
| + mkdir -p "$rundir/data" |
| + |
| + MINIO_ROOT_USER=$root_user MINIO_ROOT_PASSWORD=$root_pass \ |
| + "$minio" server "$rundir/data" \ |
| + --address "127.0.0.1:$candidate" \ |
| + --console-address "127.0.0.1:0" \ |
| + >"$rundir/minio.log" 2>&1 & |
| + minio_pid=$! |
| + |
| + waited=0 |
| + while [ "$waited" -lt 30 ]; do |
| + if ! kill -0 "$minio_pid" 2>/dev/null; then |
| + break # died — almost certainly could not bind |
| + fi |
| + if curl -sf "http://127.0.0.1:$candidate/minio/health/live" >/dev/null; then |
| + port=$candidate |
| + break |
| + fi |
| + sleep 1 |
| + waited=$((waited + 1)) |
| + done |
| + |
| + [ -n "$port" ] && break |
| + |
| + kill "$minio_pid" 2>/dev/null || true |
| + wait "$minio_pid" 2>/dev/null || true |
| + minio_pid= |
| + echo " minio did not come up on 127.0.0.1:$candidate, trying another port" |
| + attempt=$((attempt + 1)) |
| +done |
| + |
| +if [ -z "$port" ]; then |
| + echo "minio never became healthy after 5 attempts; last log:" >&2 |
| + cat "$rundir/minio.log" >&2 |
| + exit 1 |
| +fi |
| + |
| +echo " minio healthy on 127.0.0.1:$port" |
| + |
| +# --------------------------------------------------------------------- |
| +# The fixture: two buckets and a scoped user granted exactly one of them. |
| +# |
| +# Carried over verbatim from the workflow's `mint scoped user` step — |
| +# same bucket names, same policy, same user — because the whole value of |
| +# the test is that cross-bucket denial is enforced by real IAM rather |
| +# than by the client politely not asking. Nothing checks this policy |
| +# against a second copy any more, so loosening it here is invisible: |
| +# grant `denied-bucket` by accident and the test still passes while |
| +# proving nothing. Change it only on purpose. |
| +"$mc" --quiet alias set local "http://127.0.0.1:$port" "$root_user" "$root_pass" |
| +"$mc" --quiet mb local/contract-bucket local/denied-bucket |
| + |
| +cat >"$rundir/policy.json" <<'EOF' |
| +{"Version": "2012-10-17", "Statement": [ |
| + {"Effect": "Allow", "Action": ["s3:ListBucket"], "Resource": ["arn:aws:s3:::contract-bucket"]}, |
| + {"Effect": "Allow", "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"], "Resource": ["arn:aws:s3:::contract-bucket/*"]} |
| +]} |
| +EOF |
| + |
| +"$mc" --quiet admin policy create local contract-only "$rundir/policy.json" |
| +"$mc" --quiet admin user add local scopeduser scopedsecret1 |
| +"$mc" --quiet admin policy attach local contract-only --user scopeduser |
| + |
| +echo " buckets and scoped user ready" |
| + |
| +# --------------------------------------------------------------------- |
| +# Run it. |
| +# |
| +# Same env var names the test reads (contract_test.go's requireEnv calls), |
| +# and -count=1 for the same reason 10-gate uses it: never satisfy a gate |
| +# from a cached pass. Output goes to a file rather than through a pipe |
| +# because POSIX sh has no pipefail, and this step needs the test's own |
| +# exit status, not tee's. |
| +out=$rundir/test.log |
| +set +e |
| +CARLOS_MINIO_TEST=1 \ |
| + MINIO_ENDPOINT="http://127.0.0.1:$port" \ |
| + MINIO_SCOPED_KEY=scopeduser \ |
| + MINIO_SCOPED_SECRET=scopedsecret1 \ |
| + MINIO_BUCKET=contract-bucket \ |
| + MINIO_DENIED_BUCKET=denied-bucket \ |
| + go test ./internal/objstore/ -run TestDeliveredContractMinio -count=1 -v >"$out" 2>&1 |
| +test_status=$? |
| +set -e |
| + |
| +cat "$out" |
| + |
| +if [ "$test_status" -ne 0 ]; then |
| + exit "$test_status" |
| +fi |
| + |
| +# THE ASSERTION. Do not delete this because the test above already |
| +# "passed". |
| +# |
| +# TestDeliveredContractMinio skips itself unless CARLOS_MINIO_TEST=1, and |
| +# a skipped Go test exits 0. So every interesting way this step can break |
| +# — a typo'd env var, a MinIO that started but was never reachable, a |
| +# rename of the test, a future edit that drops the env block — produces a |
| +# green step that tested nothing at all. That is strictly worse than no |
| +# step, because it reports coverage this repo does not have. |
| +# |
| +# Requiring the parent test's PASS line to actually appear closes it. The |
| +# trailing " (" anchors on the parent rather than its subtests, whose |
| +# PASS lines are indented anyway. |
| +if ! grep -q '^--- PASS: TestDeliveredContractMinio (' "$out"; then |
| + echo >&2 |
| + echo "the contract test did not PASS — it was skipped, filtered out, or renamed." >&2 |
| + echo "go test exited 0, which for this test means nothing was exercised: it" >&2 |
| + echo "skips unless CARLOS_MINIO_TEST=1. Check the env block above and that" >&2 |
| + echo "TestDeliveredContractMinio still exists in internal/objstore." >&2 |
| + exit 1 |
| +fi |
| + |
| +echo "minio contract: PASS" |
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml| deleted file mode 100644 |
| index b2d1ff026..000000000 |
| --- a/.github/workflows/ci.yml |
| +++ /dev/null |
| @@ -1,186 +0,0 @@ |
| -# The build gate from CLAUDE.md, enforced by a machine rather than by |
| -# whoever happens to be at a keyboard. The gate itself is not new; what is |
| -# new is that it now runs somewhere other than a maintainer's Linux laptop. |
| -# |
| -# The cross-compile job exists because of a specific bug: readDiskFreeBytes |
| -# in internal/host/probes.go multiplied an int64 by syscall.Statfs_t.Bsize, |
| -# which is int64 on Linux and uint32 on darwin. That is a compile error on |
| -# macOS for every GOARCH, and internal/host is imported by the carlos CLI — |
| -# the laptop tool the README's developer story is built around. It sat in |
| -# main undetected because every person who built it was on Linux. A |
| -# `GOOS=darwin go build ./...` costs seconds on a Linux runner and would |
| -# have caught it at the commit that introduced it. |
| -name: CI |
| - |
| -on: |
| - push: |
| - branches: [main] |
| - pull_request: |
| - |
| -permissions: |
| - contents: read |
| - |
| -# A second push to a PR makes the first run's answer irrelevant. |
| -concurrency: |
| - group: ci-${{ github.workflow }}-${{ github.ref }} |
| - cancel-in-progress: true |
| - |
| -jobs: |
| - gate: |
| - name: build · vet · test (linux) |
| - runs-on: ubuntu-latest |
| - steps: |
| - - uses: actions/checkout@v4 |
| - - uses: actions/setup-go@v5 |
| - with: |
| - go-version-file: go.mod |
| - cache: true |
| - - run: go build ./... |
| - - run: go vet ./... |
| - # -count=1 to match CLAUDE.md: never satisfy the gate from a cached |
| - # pass. The explicit timeout is above the 10m default because the |
| - # root package's e2e tests compile the carlos binary many times over. |
| - - run: go test ./... -count=1 -timeout 30m |
| - |
| - cross: |
| - name: compile ${{ matrix.goos }}/${{ matrix.goarch }} |
| - runs-on: ubuntu-latest |
| - strategy: |
| - # One platform breaking must not hide the state of the others. |
| - fail-fast: false |
| - matrix: |
| - include: |
| - - {goos: darwin, goarch: arm64} |
| - - {goos: darwin, goarch: amd64} |
| - - {goos: linux, goarch: arm64} |
| - # Windows is a CLIENT target: `carlos deploy` from a Windows |
| - # workstation against a Linux fleet. The box-side packages still |
| - # have to COMPILE there because the CLI is the same binary as the |
| - # agent, so unix-only syscalls live behind per-OS files |
| - # (internal/edge/purge_listen_*.go, internal/host/{statfs,flock}_*.go). |
| - # This job is what stops the next syscall.Foo from silently |
| - # un-shipping the Windows build. |
| - - {goos: windows, goarch: amd64} |
| - - {goos: windows, goarch: arm64} |
| - steps: |
| - - uses: actions/checkout@v4 |
| - - uses: actions/setup-go@v5 |
| - with: |
| - go-version-file: go.mod |
| - cache: true |
| - # Compile only — these binaries are never run, so no runner for the |
| - # target OS is needed. CGO_ENABLED=0 is already the default when |
| - # cross-compiling; it is explicit here so that adding a cgo dependency |
| - # fails loudly in this job rather than silently changing what is |
| - # covered. The tree is pure Go today (modernc.org/sqlite, no import "C"). |
| - - run: go build ./... |
| - env: |
| - GOOS: ${{ matrix.goos }} |
| - GOARCH: ${{ matrix.goarch }} |
| - CGO_ENABLED: "0" |
| - |
| - minio-contract: |
| - # Exercises internal/objstore's TestDeliveredContractMinio: the S3 |
| - # contract a member app actually gets from delivered grant env (static |
| - # creds, path-style, us-east-1, checksum WhenRequired), against a real |
| - # S3-compatible server plus a scoped user IAM never granted the second |
| - # bucket. This is the only place in the gate that runs that test live — |
| - # see internal/objstore/contract_test.go's doc comment. |
| - name: minio contract |
| - runs-on: ubuntu-latest |
| - steps: |
| - - uses: actions/checkout@v4 |
| - - uses: actions/setup-go@v5 |
| - with: |
| - go-version-file: go.mod |
| - cache: true |
| - - name: start minio |
| - # Pinned rather than :latest — everything else in this file pins |
| - # (go-version-file, action majors), and a floating minio tag has |
| - # already broken this exact job class once (mc's own admin-policy |
| - # syntax churned across versions). Verified live against the quay.io |
| - # registry API on 2026-08-10: RELEASE.2025-09-07T16-13-09Z is the |
| - # image quay.io/minio/minio:latest currently resolves to (identical |
| - # manifest digest sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e). |
| - # Bump deliberately, not silently — re-verify against |
| - # https://quay.io/api/v1/repository/minio/minio/tag/ before moving it. |
| - run: | |
| - docker run -d --name minio -p 9000:9000 \ |
| - -e MINIO_ROOT_USER=root -e MINIO_ROOT_PASSWORD=rootsecret \ |
| - quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z server /data |
| - for i in $(seq 1 30); do |
| - if curl -sf http://127.0.0.1:9000/minio/health/live; then |
| - break |
| - fi |
| - if [ "$i" -eq 30 ]; then |
| - echo "minio never became healthy after 30s" >&2 |
| - docker logs minio || true |
| - exit 1 |
| - fi |
| - sleep 1 |
| - done |
| - - name: mint scoped user |
| - run: | |
| - # Pinned to the mc release published alongside the minio server |
| - # image above (verified live against the minio/mc GitHub releases |
| - # API and dl.min.io's archive endpoint on 2026-08-10) rather than |
| - # the floating linux-amd64/mc path, for the same reproducibility |
| - # reason. Bump together with the minio image pin above. |
| - curl -sfL https://dl.min.io/client/mc/release/linux-amd64/archive/mc.RELEASE.2025-08-13T08-35-41Z -o mc && chmod +x mc |
| - ./mc --version |
| - ./mc alias set local http://127.0.0.1:9000 root rootsecret |
| - ./mc mb local/contract-bucket local/denied-bucket |
| - cat > /tmp/policy.json <<'EOF' |
| - {"Version": "2012-10-17", "Statement": [ |
| - {"Effect": "Allow", "Action": ["s3:ListBucket"], "Resource": ["arn:aws:s3:::contract-bucket"]}, |
| - {"Effect": "Allow", "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"], "Resource": ["arn:aws:s3:::contract-bucket/*"]} |
| - ]} |
| - EOF |
| - ./mc admin policy create local contract-only /tmp/policy.json |
| - ./mc admin user add local scopeduser scopedsecret1 |
| - ./mc admin policy attach local contract-only --user scopeduser |
| - - name: contract test |
| - env: |
| - CARLOS_MINIO_TEST: "1" |
| - MINIO_ENDPOINT: http://127.0.0.1:9000 |
| - MINIO_SCOPED_KEY: scopeduser |
| - MINIO_SCOPED_SECRET: scopedsecret1 |
| - MINIO_BUCKET: contract-bucket |
| - MINIO_DENIED_BUCKET: denied-bucket |
| - run: go test ./internal/objstore/ -run TestDeliveredContractMinio -count=1 -v |
| - |
| - gofmt: |
| - name: gofmt (changed files) |
| - runs-on: ubuntu-latest |
| - # Only meaningful against a base to diff from. |
| - if: github.event_name == 'pull_request' |
| - steps: |
| - - uses: actions/checkout@v4 |
| - with: |
| - fetch-depth: 0 |
| - - uses: actions/setup-go@v5 |
| - with: |
| - go-version-file: go.mod |
| - # CLAUDE.md's rule is "gofmt -l clean for files you touch", not for the |
| - # tree: three pre-existing unformatted files are deliberately left |
| - # alone, so a repo-wide check would fail on arrival and teach everyone |
| - # to ignore this job. Scoped to the files the PR actually changes. |
| - - name: gofmt changed files |
| - env: |
| - BASE_SHA: ${{ github.event.pull_request.base.sha }} |
| - run: | |
| - set -euo pipefail |
| - files=$(git diff --name-only --diff-filter=ACMR "$BASE_SHA"...HEAD -- '*.go') |
| - if [ -z "$files" ]; then |
| - echo "no Go files changed" |
| - exit 0 |
| - fi |
| - echo "checking:" |
| - echo "$files" | sed 's/^/ /' |
| - unformatted=$(gofmt -l $files) |
| - if [ -n "$unformatted" ]; then |
| - echo |
| - echo "needs gofmt -w:" |
| - echo "$unformatted" | sed 's/^/ /' |
| - exit 1 |
| - fi |
diff --git a/.gitignore b/.gitignore| index 0df49b9a3..6b209b585 100644 |
| --- a/.gitignore |
| +++ b/.gitignore |
| @@ -3,3 +3,8 @@ platform |
| *.tfstate* |
| .terraform/ |
| .claude/worktrees/ |
| + |
| +# Pinned MinIO/mc binaries cached by .amadan/ci.d/30-minio-contract. |
| +# amadan reuses one workspace per repo across jobs, so an untracked |
| +# directory in the checkout is that step's cache — see the step for why. |
| +.amadan-cache/ |
diff --git a/AGENTS.md b/AGENTS.md| index efe0aa0ee..78b2f8c32 100644 |
| --- a/AGENTS.md |
| +++ b/AGENTS.md |
| @@ -74,26 +74,36 @@ Use the judgement the rest of this file asks for: a PR that is still |
| collecting review, that is gated on something (`#62`), or whose diff you |
| would want a second pair of eyes on, waits for a person. |
| -The reason is timing, not ceremony. `.github/workflows/ci.yml` triggers on |
| -`pull_request` *and* on `push` to `main`, so a locally merged change does |
| -still meet the gate — but only **after** it is on `main`, where a red run |
| -is an incident to clean up rather than a merge that simply doesn't happen. |
| -The PR is the one place the gate runs against the merge result while the |
| -change can still be turned away, and the one place the branch's diff is |
| -read as a single piece rather than commit by commit. |
| +The reason is timing, not ceremony. A locally merged change does still |
| +meet the gate — but only **after** it is on `main`, where a red run is an |
| +incident to clean up rather than a merge that simply doesn't happen. The |
| +PR is the one place the branch's diff is read as a single piece rather |
| +than commit by commit, while the change can still be turned away. |
| ## Build & test |
| - `go build ./...`, `go vet ./...`, `go test ./... -count=1` — all clean, |
| every package (21 as of 2026-08-08; the count grows), before any commit |
| lands. |
| -- `gofmt -l` clean for files you touch (three pre-existing unformatted files |
| - elsewhere are left alone). |
| -- `.github/workflows/ci.yml` runs that same gate on every PR, plus a |
| - `GOOS=darwin` compile. **The gate is not Linux-only**: `internal/host` is |
| - linked into the CLI, which people run on Macs, so a Linux-only type |
| - assumption breaks `go build .` for them. Cross-compiling is enough to catch |
| - it — no Mac required. |
| +- `gofmt -l .` clean over the whole tree — `make test` checks it, and the |
| + three pre-existing unformatted files that used to be exempted are gone. |
| +- **`.amadan/ci.d/` is the only CI this repo has.** The GitHub workflow |
| + that used to run this gate was deleted on 2026-08-24, once the amadan |
| + steps covered everything it did and more; `.amadan/README.md` records |
| + what each step descends from, and `.amadan/ci.d/10-gate`'s header |
| + records why the switch was safe. `10-gate` is `go build ./...` then |
| + `make test`, so CI and what you run locally are one definition rather |
| + than two that can drift. |
| +- **CI is triggered by a push to the `amadan` remote, not by GitHub.** |
| + `origin` is still where PRs live and where `main` is source of truth, |
| + but nothing on GitHub runs a build any more. A branch pushed only to |
| + `origin` has been gated by exactly nothing — push it to `amadan` too, |
| + and read the run, before calling it green. |
| +- `20-cross@5` adds `GOOS=darwin`/`windows`/`linux-arm64` compiles, one |
| + per shard. **The gate is not Linux-only**: `internal/host` is linked |
| + into the CLI, which people run on Macs, so a Linux-only type assumption |
| + breaks `go build .` for them. Cross-compiling is enough to catch it — |
| + no Mac required. |
| - Sandbox note: use `GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod |
| GOPROXY=off`; go commands may need the sandbox disabled ("read-only file |
| system" from the toolchain is the tell). |
diff --git a/Makefile b/Makefile| index 7e858f35a..5a963c1e4 100644 |
| --- a/Makefile |
| +++ b/Makefile |
| @@ -5,10 +5,16 @@ GOARCH ?= |
| .PHONY: test build fmt |
| +# The build gate: vet, format, tests. `.amadan/ci.d/10-gate` runs this same |
| +# target, so what CI checks and what you check are one definition. |
| +# |
| +# -count=1 is CLAUDE.md's rule: never satisfy the gate from a cached pass. |
| +# The 30m timeout is above Go's 10m default because the root package's e2e |
| +# tests compile the carlos binary many times over. |
| test: |
| go vet ./... |
| - @fmt_out=$$(gofmt -l .); if [ -n "$$fmt_out" ]; then echo "gofmt needed:"; echo "$$fmt_out"; exit 1; fi |
| - go test ./... |
| + @fmt_out=$$(gofmt -l .); if [ -n "$$fmt_out" ]; then echo "gofmt needed:"; echo "$$fmt_out"; exit 1; fi; echo "gofmt -l .: clean" |
| + go test ./... -count=1 -timeout 30m |
| build: |
| CGO_ENABLED=0 GOOS=$(GOOS) GOARCH=$(GOARCH) go build -ldflags "$(LDFLAGS)" -o carlos . |
diff --git a/internal/host/statfs_unix.go b/internal/host/statfs_unix.go| index 4c9916994..a57f9ceba 100644 |
| --- a/internal/host/statfs_unix.go |
| +++ b/internal/host/statfs_unix.go |
| @@ -9,7 +9,7 @@ import "syscall" |
| // |
| // It returns BYTES, not the raw statfs fields, and that is the point: Bsize |
| // is int64 on Linux and uint32 on darwin, a difference that already broke |
| -// the CI build once (see .github/workflows/ci.yml's header) because the |
| +// the CI build once (see .amadan/ci.d/20-cross@5's header) because the |
| // multiplication happened at every call site instead of once here. Callers |
| // now get two plain uint64s and no per-OS arithmetic. |
| func diskBytes(dir string) (total, avail uint64, err error) { |
diff --git a/internal/objstore/contract_test.go b/internal/objstore/contract_test.go| index 6a9489a57..ae328d283 100644 |
| --- a/internal/objstore/contract_test.go |
| +++ b/internal/objstore/contract_test.go |
| @@ -28,13 +28,14 @@ import ( |
| // been exercised anywhere in this tree is the actual wire contract a member |
| // app's client.Options() produces from delivered env, against a real |
| // S3-compatible implementation. Skipped by default because it needs a |
| -// running Minio; CI's minio-contract job (.github/workflows/ci.yml) sets |
| -// CARLOS_MINIO_TEST=1 and stands one up. There is no docker in this |
| -// sandbox, so the live path here has only ever been verified in CI — see |
| -// the commit body. |
| +// running Minio; CI's 30-minio-contract step (.amadan/ci.d/30-minio-contract) |
| +// sets CARLOS_MINIO_TEST=1 and stands one up, as a plain process on |
| +// loopback rather than a container. That step is now the only place this |
| +// test runs live anywhere, so treat a change to it as a change to what |
| +// this test is worth. |
| func TestDeliveredContractMinio(t *testing.T) { |
| if os.Getenv("CARLOS_MINIO_TEST") != "1" { |
| - t.Skip("set CARLOS_MINIO_TEST=1 (with a running Minio and the MINIO_* env vars) to run the delivered-contract test; see .github/workflows/ci.yml's minio-contract job") |
| + t.Skip("set CARLOS_MINIO_TEST=1 (with a running Minio and the MINIO_* env vars) to run the delivered-contract test; see .amadan/ci.d/30-minio-contract, which stands one up") |
| } |
| endpoint := requireEnv(t, "MINIO_ENDPOINT") |