CI failed 6m 31s
- Branch
region-ha-1b-next-build- Commit
03212a8bPaul Campbell
- Runner
ci-aws~paul@keymail.dev/ip-172-31-30-149- Reported by
ci-aws~paul@keymail.dev/ip-172-31-30-149- Queued
- Started
- Waited for a runner
- 3m 5s
- Last change
- Attempt
- 1 of 2
Steps run in order, one after another.
-
gate failed 6m 30s
cd launchctx && go vet ./... && go test ./... -count=1 ok amadan.net/carlos/platform.git/launchctx 0.011s ? amadan.net/carlos/platform.git/launchctx/controlname [no test files] ? amadan.net/carlos/platform.git/launchctx/wireid [no test files] go vet ./... gofmt -l .: clean go test ./... -count=1 -timeout 30m ok github.com/carlosframework/platform 339.457s ? github.com/carlosframework/platform/cmd/changelog [no test files] ok github.com/carlosframework/platform/cmd/console 3.175s ok github.com/carlosframework/platform/cmd/moveprobe 93.227s ok github.com/carlosframework/platform/internal/activator 234.538s ok github.com/carlosframework/platform/internal/agentsock 0.014s ok github.com/carlosframework/platform/internal/alerting 0.005s ok github.com/carlosframework/platform/internal/analytics 0.020s ok github.com/carlosframework/platform/internal/appconfig 2.328s ok github.com/carlosframework/platform/internal/apperr 0.078s ok github.com/carlosframework/platform/internal/applog 0.160s ok github.com/carlosframework/platform/internal/awsinv 0.005s ok github.com/carlosframework/platform/internal/boxchoice 0.007s ok github.com/carlosframework/platform/internal/boxid 0.671s ? github.com/carlosframework/platform/internal/boxid/boxidtest [no test files] ? github.com/carlosframework/platform/internal/boxid/internal/signtest [no test files] ok github.com/carlosframework/platform/internal/boxvolumes 0.010s ok github.com/carlosframework/platform/internal/caps 0.012s ok github.com/carlosframework/platform/internal/certwarm 0.002s ok github.com/carlosframework/platform/internal/cgroup 0.191s ok github.com/carlosframework/platform/internal/changelog 0.246s ok github.com/carlosframework/platform/internal/clamd 0.055s ? github.com/carlosframework/platform/internal/cliflag [no test files] ok github.com/carlosframework/platform/internal/console 364.342s ? github.com/carlosframework/platform/internal/controlname [no test files] ? github.com/carlosframework/platform/internal/controlsock [no test files] ok github.com/carlosframework/platform/internal/dbdoor 5.794s ok github.com/carlosframework/platform/internal/dbdoor/dbdoortest 0.009s ok github.com/carlosframework/platform/internal/dbverify 0.004s ok github.com/carlosframework/platform/internal/dnsrec 0.005s ok github.com/carlosframework/platform/internal/docsite 0.004s ok github.com/carlosframework/platform/internal/domains 2.115s ok github.com/carlosframework/platform/internal/doorkey 0.006s ok github.com/carlosframework/platform/internal/economics 0.004s ok github.com/carlosframework/platform/internal/edge 26.053s ? github.com/carlosframework/platform/internal/edgeproof [no test files] ok github.com/carlosframework/platform/internal/edgesfile 0.007s ok github.com/carlosframework/platform/internal/fleetapi 0.123s ok github.com/carlosframework/platform/internal/gate 1.654s ok github.com/carlosframework/platform/internal/genselect 0.015s ok github.com/carlosframework/platform/internal/geo 0.015s ok github.com/carlosframework/platform/internal/geoip 0.029s ok github.com/carlosframework/platform/internal/host 74.035s ok github.com/carlosframework/platform/internal/hostactors 0.366s ok github.com/carlosframework/platform/internal/hostbox 1.857s ok github.com/carlosframework/platform/internal/hostclaim 0.510s ok github.com/carlosframework/platform/internal/hostevidence 0.016s ok github.com/carlosframework/platform/internal/hostexec 0.305s ok github.com/carlosframework/platform/internal/hostfence 0.038s ok github.com/carlosframework/platform/internal/hostfiles 10.423s ok github.com/carlosframework/platform/internal/hostfloor 0.015s ok github.com/carlosframework/platform/internal/hostguard 0.144s ok github.com/carlosframework/platform/internal/hostops 1.408s ? github.com/carlosframework/platform/internal/hostops/fencerelease [no test files] ok github.com/carlosframework/platform/internal/hostops/internal/fencerelease 0.015s ok github.com/carlosframework/platform/internal/hostquarantine 0.015s ok github.com/carlosframework/platform/internal/hostrelocate 0.447s ok github.com/carlosframework/platform/internal/hostresolve 0.149s ok github.com/carlosframework/platform/internal/hostwrite 0.015s ok github.com/carlosframework/platform/internal/infraview 0.016s ok github.com/carlosframework/platform/internal/instancectl 1.706s ok github.com/carlosframework/platform/internal/instanceexec 2.697s ok github.com/carlosframework/platform/internal/instancefs 0.015s ok github.com/carlosframework/platform/internal/instancekey 0.015s ok github.com/carlosframework/platform/internal/instanceupdates 0.025s ok github.com/carlosframework/platform/internal/layout 1.421s ok github.com/carlosframework/platform/internal/lease 9.442s ok github.com/carlosframework/platform/internal/lease/leasetest 0.005s ok github.com/carlosframework/platform/internal/ledger 0.011s ok github.com/carlosframework/platform/internal/mail 0.170s ok github.com/carlosframework/platform/internal/maintenance 0.077s ok github.com/carlosframework/platform/internal/maintenanceexec 5.504s ok github.com/carlosframework/platform/internal/maintenanceguard 0.068s ok github.com/carlosframework/platform/internal/metrics 0.005s ok github.com/carlosframework/platform/internal/moves 20.969s ok github.com/carlosframework/platform/internal/moves/archive 0.761s ok github.com/carlosframework/platform/internal/objstore 0.174s ok github.com/carlosframework/platform/internal/ops 0.167s ok github.com/carlosframework/platform/internal/park 0.201s ok github.com/carlosframework/platform/internal/privatefile 0.003s ok github.com/carlosframework/platform/internal/provision 3.317s ok github.com/carlosframework/platform/internal/publicrelease 0.006s ? github.com/carlosframework/platform/internal/regionname [no test files] ok github.com/carlosframework/platform/internal/registry 4.010s ok github.com/carlosframework/platform/internal/relay 7.229s awake.example.com: updated -> v1 (canary/x) sleepy.example.com: repointed -> v1 (canary/x; hibernating — runs it on next wake) awake.example.com: updated -> v1 (canary/x) sleepy.example.com: repointed -> v1 (canary/x; hibernating — runs it on next wake) awake.example.com: updated -> v1 (canary/x) sleepy.example.com: repointed -> v1 (canary/x; hibernating — bounced if awake, else runs it on next wake) zzz-awake.example.com: updated -> v1 (canary/x) aaa-sleepy.example.com: repointed -> v1 (canary/x; hibernating — bounced if awake, else runs it on next wake) sleepy.example.com: repointed -> v1 (canary/x; hibernating — bounced if awake, else runs it on next wake) sleepy.example.com: update to v2 failed: bounce failed — rolled the symlink back awake.example.com: updated -> v1 (canary/x) awake.example.com: updated -> v1 (canary/x) a.example.com: update to v2 failed: unit failed — rolled the symlink back b.example.com: skipping v2 (failed on another host this pass) h.example.com: v1 (stable) is in its supply-chain bake window until 2026-10-16 10:54 UTC myapp.example.com: updated -> v1 (canary/x) kass.example.com: updated -> v2 (canary/x) ops.example.com: updated -> v1 (canary/x) kass.example.com: updated -> v1 (canary/x) ops.example.com: updated -> v1 (canary/x) h.example.com: release v1 has no linux-arm64 build — skipping canary.example.com: updated -> v1 (canary/x) old-canary.example.com: updated -> v1 (canary/x) awake-a.example.com: updated -> v1 (canary/x) awake-b.example.com: updated -> v1 (canary/x) sleepy.example.com: repointed -> v1 (canary/x; hibernating — runs it on next wake) old-canary.example.com: updated -> v1 (canary/x) fresh-canary.example.com: updated -> v1 (canary/x) h.example.com: stage manifest sibling for v1abc: is a directory h.example.com: back-fill manifest for v1abc: dirstore: apps/bab/keymail/releases/v1abc/manifest.json: file does not exist — if this deployment predates account-scoped bucket keys its objects are still at apps/keymail/, which the runtime no longer reads; move them once with: carlos accounts migrate --app keymail --from legacy --to bab keymail.dev: carlos-app@keymail.dev.socket was not enabled — enabled it, or its ListenStream binds nothing after a reboot keymail.dev: carlos-app@keymail.dev.service was not enabled — enabled it, so it comes back after a reboot keymail.dev: capability drop-ins converged and the unit restarted keymail.dev: updated v1 -> v2 (canary/x) a.example.com: carlos-app@a.example.com.socket was not enabled — enabled it, or its ListenStream binds nothing after a reboot a.example.com: carlos-app@a.example.com.service was not enabled — enabled it, so it comes back after a reboot a.example.com: capability drop-ins converged and the unit restarted a.example.com: updated -> v1 (canary/x) b.example.com: updated -> v1 (canary/x) exec.example.com: updated -> v1 (canary/x) unit.example.com: v1 declares capabilities and this pass does not write unit configuration — it runs against the drop-ins the previous version left until the root adopt pass (carlos-adopt.service, hourly) converges them unit.example.com: updated -> v1 (canary/x) sleepy.example.com: port:25 is granted on a hibernating route — a connection to that port starts the unit directly, bypassing the activator's health-gated wake; the edge's clean 503 on a broken boot does not cover this path sleepy.example.com: carlos-app@sleepy.example.com.socket was not enabled — enabled it, or its ListenStream binds nothing after a reboot sleepy.example.com: capability drop-ins converged and the unit restarted sleepy.example.com: repointed -> v1 (canary/x; hibernating — runs it on next wake) keymail.dev: back-fill manifest for v2: dirstore: apps/bab/keymail/releases/v2/manifest.json: file does not exist — if this deployment predates account-scoped bucket keys its objects are still at apps/keymail/, which the runtime no longer reads; move them once with: carlos accounts migrate --app keymail --from legacy --to bab keymail.dev: updated v1 -> v2 (canary/x) keymail.dev: carlos-app@keymail.dev.socket was not enabled — enabled it, or its ListenStream binds nothing after a reboot keymail.dev: carlos-app@keymail.dev.service was not enabled — enabled it, so it comes back after a reboot keymail.dev: capability drop-ins converged and the unit restarted keymail.dev: updated -> v1 (canary/x) keymail.dev: no capability materializes here any more — removed /tmp/TestAdoptRemovesDropInsWhenTheNewVersionDeclaresNothing2766118413/004/carlos-app@keymail.dev.service.d/10-carlos-capabilities.conf keymail.dev: no capability materializes here any more — removed /tmp/TestAdoptRemovesDropInsWhenTheNewVersionDeclaresNothing2766118413/004/carlos-app@keymail.dev.socket.d/10-carlos-capabilities.conf keymail.dev: capability drop-ins converged and the unit restarted keymail.dev: updated v1 -> v2 (canary/x) keymail.dev: carlos-app@keymail.dev.socket was not enabled — enabled it, or its ListenStream binds nothing after a reboot keymail.dev: carlos-app@keymail.dev.service was not enabled — enabled it, so it comes back after a reboot keymail.dev: capability drop-ins converged and the unit restarted keymail.dev: updated -> v1 (canary/x) keymail.dev: capability drop-ins converged keymail.dev: updated v1 -> v2 (canary/x) keymail.dev: carlos-app@keymail.dev.service was not enabled — enabled it, so it comes back after a reboot keymail.dev: unit state: unit masked keymail.dev: no capability materializes here any more — removed /tmp/TestConvergeFailureAfterItsReloadRestoresTheDropIns1559584099/004/carlos-app@keymail.dev.service.d/10-carlos-capabilities.conf keymail.dev: no capability materializes here any more — removed /tmp/TestConvergeFailureAfterItsReloadRestoresTheDropIns1559584099/004/carlos-app@keymail.dev.socket.d/10-carlos-capabilities.conf keymail.dev: capability drop-ins rolled back to the previous version's keymail.dev: capabilities for v2 were refused: unit masked — not pointed live sleepy.example.com: port:25 is granted on a hibernating route — a connection to that port starts the unit directly, bypassing the activator's health-gated wake; the edge's clean 503 on a broken boot does not cover this path sleepy.example.com: carlos-app@sleepy.example.com.socket was not enabled — enabled it, or its ListenStream binds nothing after a reboot sleepy.example.com: capability drop-ins converged and the unit restarted sleepy.example.com: repointed -> v1 (canary/x; hibernating — runs it on next wake) sleepy.example.com: capability daemon and hibernation contradict — nothing written and no unit state changed; fix it with: carlos route --host sleepy.example.com --hibernate=false sleepy.example.com: capabilities for v2 were refused: capability daemon and hibernation contradict — not pointed live keymail.dev: carlos-app@keymail.dev.socket was not enabled — enabled it, or its ListenStream binds nothing after a reboot keymail.dev: carlos-app@keymail.dev.service was not enabled — enabled it, so it comes back after a reboot keymail.dev: capability drop-ins converged and the unit restarted keymail.dev: updated -> v1 (canary/x) keymail.dev: carlos-app@keymail.dev.socket was not enabled — enabled it, or its ListenStream binds nothing after a reboot keymail.dev: carlos-app@keymail.dev.service was not enabled — enabled it, so it comes back after a reboot keymail.dev: capability drop-ins converged and the unit restarted keymail.dev: updated v1 -> v2 (canary/x) keymail.dev: carlos-app@keymail.dev.socket was not enabled — enabled it, or its ListenStream binds nothing after a reboot keymail.dev: carlos-app@keymail.dev.service was not enabled — enabled it, so it comes back after a reboot keymail.dev: capability drop-ins converged and the unit restarted keymail.dev: updated -> v1 (canary/x) h.example.com: updated -> v1 (stable) h.example.com: v1 (stable) is in its supply-chain bake window until 2026-10-16 17:24 UTC h.example.com: updated -> v1 (stable) bab/myapp: production flag unreadable (production flag wrapper: invalid character 'o' in literal null (expecting 'u')) — treating as production, holds apply h.example.com: v1 (stable) is in its supply-chain bake window until 2026-10-16 13:00 UTC h.example.com: v1 (stable) is in its supply-chain bake window until 2026-10-16 03:25 UTC h.example.com: v1 (stable) is in its supply-chain bake window until 2026-10-16 03:25 UTC cx.example.com: updated -> v1 (canary/x) cy.example.com: updated -> v1 (canary/y) s.example.com: v1 (stable) is in its supply-chain bake window until 2026-10-16 08:49 UTC cx.example.com: updated -> v1 (canary/x) s.example.com: v1 (stable) is in its supply-chain bake window until 2026-10-16 02:40 UTC h.example.com: v1 (stable) is in its supply-chain bake window until 2026-10-16 14:25 UTC h.example.com: v1 (stable) is in its supply-chain bake window until 2026-10-16 14:25 UTC e.example.com: updated -> v1 (edge) p.example.com: v1 (production) is in its supply-chain bake window until 2026-10-16 11:56 UTC p.example.com: updated -> v1 (production) p.example.com: v2 (production) is in its supply-chain bake window until 2026-10-14 04:30 UTC p.example.com: updated v1 -> v2 (production) c.example.com: updated -> v1 (canary/x) p.example.com: v1 (production) is in its supply-chain bake window until 2026-10-16 18:43 UTC bab/myapp: pipeline unreadable (pipeline wrapper: invalid character 'o' in literal null (expecting 'u')) — remembered windows still apply; anything unremembered falls to legacy holds p.example.com: v1 (production) is in its supply-chain bake window until 2026-10-14 17:47 UTC e.example.com: v1 (edge) is in its supply-chain bake window until 2026-10-14 03:51 UTC bab/myapp: pipeline unreadable (pipeline wrapper: invalid character 'o' in literal null (expecting 'u')) — remembered windows still apply; anything unremembered falls to legacy holds e.example.com: v1 (edge) is in its supply-chain bake window until 2026-10-14 03:51 UTC e.example.com: v1 (edge) is in its supply-chain bake window until 2026-10-14 06:08 UTC e.example.com: v1 (edge) is in its supply-chain bake window until 2026-10-14 06:08 UTC e.example.com: v1 (edge) is in its supply-chain bake window until 2026-10-14 06:08 UTC e.example.com: updated -> v1 (edge) p.example.com: v1 (production) is in its supply-chain bake window until 2026-10-16 01:25 UTC p.example.com: updated -> v1 (production) p.example.com: v2 (production) is in its supply-chain bake window until 2026-10-14 00:59 UTC bab/myapp: pipeline unreadable (pipeline wrapper: invalid character 'o' in literal null (expecting 'u')) — remembered windows still apply; anything unremembered falls to legacy holds p.example.com: v2 (production) is in its supply-chain bake window until 2026-10-16 01:25 UTC p.example.com: v1 (production) is in its supply-chain bake window until 2026-10-16 09:22 UTC p.example.com: updated -> v1 (production) p.example.com: v2 (production) is in its supply-chain bake window until 2026-10-14 03:39 UTC p.example.com: v2 (production) is in its supply-chain bake window until 2026-10-16 09:23 UTC p.example.com: updated v1 -> v2 (production) p.example.com: v1 (production) is in its supply-chain bake window until 2026-10-16 17:07 UTC p.example.com: updated -> v1 (production) p.example.com: v2 (production) is in its supply-chain bake window until 2026-10-14 06:13 UTC p.example.com: v2 (production) is in its supply-chain bake window until 2026-10-16 17:07 UTC p.example.com: v2 (production) is in its supply-chain bake window until 2026-10-16 17:07 UTC p.example.com: v2 (production) is in its supply-chain bake window until 2026-10-16 17:07 UTC e.example.com: v1 (edge) is in its supply-chain bake window until 2026-10-14 03:10 UTC bab/myapp: pipeline unreadable (pipeline wrapper: invalid character 'o' in literal null (expecting 'u')) — remembered windows still apply; anything unremembered falls to legacy holds e.example.com: v1 (edge) is in its supply-chain bake window until 2026-10-14 03:10 UTC e.example.com: v1 (edge) is in its supply-chain bake window until 2026-10-14 05:19 UTC e.example.com: v1 (edge) is in its supply-chain bake window until 2026-10-14 05:19 UTC e.example.com: updated -> v1 (edge) e.example.com: updated -> v1 (edge) bab/myapp: pipeline unreadable (pipeline wrapper: invalid character 'o' in literal null (expecting 'u')) — remembered windows still apply; anything unremembered falls to legacy holds e.example.com: updated -> v1 (edge) cx.example.com: updated -> v1 (canary/x) cy.example.com: updated -> v1 (canary/y) e.example.com: updated -> v1 (edge) s.example.com: updated -> v1 (stable) console.example.com: update to new failed: Job for console.service failed — rolled the symlink back console.example.com: update to new failed: still broken — rolled the symlink back console.example.com: update to new failed: still broken — rolled the symlink back console.example.com: updated old -> new (canary/x) console.example.com: update to new failed: Job for console.service failed — rolled the symlink back console.example.com: updated old -> new (canary/x) h1.example.com: updated -> v1 (canary/x) h1.example.com: updated -> v1 (canary/x) h1.example.com: updated -> v1 (canary/x) a.example.com: updated -> v1 (canary/x) a.example.com: updated -> v2 (canary/x) a.example.com: updated -> v1 (canary/x) a.example.com: updated -> v1 (canary/x) a.example.com: updated -> v1 (canary/x) p.example.com: v1 (production) is in its supply-chain bake window until 2026-10-16 02:51 UTC p.example.com: updated -> v1 (production) p.example.com: v2 (production) is in its supply-chain bake window until 2026-10-14 01:28 UTC a.example.com: v1 recovered from this box's verified staged copy (v1: promotion record: s3: 503 slow down) a.example.com: v1 recovered from this box's verified staged copy (manifest for km/v1: s3: 503 slow down) app.example: repoint failed: release selection changed app.example: could not roll the symlink back to v1: release selection changed app.example: update to v2 failed: restart failed — could not roll the symlink back app.example: update to v1 failed: hostwrite: lifecycle provider unavailable — rolled the symlink back app.example: updated -> v1 (canary/x) app.example: update to v1 failed: release selection changed — rolled the symlink back app.example: update to v2 failed: restart failed — rolled the symlink back app.example: rolled back to v1 but could not start it: release selection changed — this host may be serving nothing h1.example.com: updated -> v1 (canary/x) h1.example.com: updated -> v1 (canary/x) 2026/10/13 00:47:07 healthy history: /tmp/TestHealthyCorruptReadsEmpty2213477926/001/healthy/bab/console.json does not parse (unexpected end of JSON input), treating it as empty held stage request "held.example.com.json": refused: release: the held release is not authorized: no promotion record and no signed pointer names it: bab/console at v3; removed held stage request "held.example.com.json": refused: manifest for console/v1: release: manifest does not hash to the digest its signer recorded: console/v1 hashes to 9c2d216b7a41, the pointer signed 05cd819d301c; removed held.example.com: held stage of v1 for mv_abcdefghijklmnopqrstuvwxyz: release: held stage of bab/console at v1: reading channel canary/x: transient read failure; kept for the next pass 2026/10/13 00:47:07 release: pointer digest ratchet: open directory /tmp/TestHeldStageRetriesARatchetThatCannotTellthe_mark_could_not_be3246880805/002/ro/bab/km: permission denied 2026/10/13 00:47:07 release: pointer digest ratchet: open directory /tmp/TestHeldAuthorizedAdmitsOnlyNamingPointers1686759976/003/ro/bab/km: permission denied held.example.com: v1: the staged bytes do not match the proof (staged sha256 22e7ffce966280e2f2c9765e5ac4feaa19c2883fc9cddf0a97610e92cda7f6e5, dev proved 683a664a34bd9c537ce6ebbcd5878689db9147dd15d292152be4af4a5a0ab168); not pointed live held.example.com: v1: the staged bytes do not match the proof (staged sha256 22e7ffce966280e2f2c9765e5ac4feaa19c2883fc9cddf0a97610e92cda7f6e5, dev proved 683a664a34bd9c537ce6ebbcd5878689db9147dd15d292152be4af4a5a0ab168); not pointed live held.example.com: v1: the staged bytes do not match the proof (staged sha256 22e7ffce966280e2f2c9765e5ac4feaa19c2883fc9cddf0a97610e92cda7f6e5, dev proved 683a664a34bd9c537ce6ebbcd5878689db9147dd15d292152be4af4a5a0ab168); not pointed live held.example.com: v1: the staged bytes do not match the proof (staged sha256 487b54327a2fde78118d00e9d25a33dec1689d85ab81c72e671b69cd3fd8b5d2, dev proved 683a664a34bd9c537ce6ebbcd5878689db9147dd15d292152be4af4a5a0ab168); not pointed live held.example.com: v1 (sha256 683a664a34bd9c537ce6ebbcd5878689db9147dd15d292152be4af4a5a0ab168) is not proven on dev; staying on v1 (proof from dev: proof signature does not verify against the configured key) proof policy /tmp/TestVerifyHeldLiveRechecksAnExistingLink3741064768/004/proof-policy.json: does not parse: invalid character 'n' looking for beginning of object key string; every named-unit route waits until it is fixed held.example.com: held stage of v1: proof policy /tmp/TestVerifyHeldLiveRechecksAnExistingLink3741064768/004/proof-policy.json: does not parse: invalid character 'n' looking for beginning of object key string; not staged --- FAIL: TestVerifyHeldLiveUnderReadsUnderTheClaim (0.00s) heldverify_test.go:221: VerifyHeldLive under the claim: <nil>, want ErrHeldClaimWired mat.test: updated -> abc1234 (canary/dev) mat.test: abc1234 (beta) is in its supply-chain bake window until 2026-10-14 03:24 UTC bab/matapp/canary/dev: channel has no release yet mat.test: release 0.1.0 has no linux-arm64 build — skipping bab/matapp/canary/dev: channel has no release yet held.example.com: v1 (sha256 683a664a34bd9c537ce6ebbcd5878689db9147dd15d292152be4af4a5a0ab168) is not proven on dev; staying on v0 (no proof from dev) held.example.com: v1: the staged bytes do not match the proof (staged sha256 3a2d06131202d057a22794871fefac411198fd6d3ff8515a59b60d0a27493699, dev proved 683a664a34bd9c537ce6ebbcd5878689db9147dd15d292152be4af4a5a0ab168); not pointed live held stage request "held.example.com.json": refused: release: held stage request no longer matches its move: the claim names mv_abcdefghijklmnopqrstuvwxyz; removed held stage request "nul.json": refused: its host is not its file name; removed held stage request "held.example.com.json": refused: malformed version; removed held stage request "held.example.com.json": refused: malformed account; removed held stage request "held.example.com.json": refused: malformed app; removed held stage request "held.example.com.json": refused: no move op; removed held stage request "held.example.com.json": refused: no box; removed held stage request "evil.json": refused: its host is not its file name; removed held stage request "held.example.com.json": refused: its host is not its file name; removed held stage request "held.example.com.json": not a request; removed held stage request "held.example.com.json": unreadable: held.example.com.json: not a regular file with one link; removed held.example.com: held stage of v9 for mv_abcdefghijklmnopqrstuvwxyz: manifest for console/v9: dirstore: apps/bab/console/releases/v9/manifest.json: file does not exist — if this deployment predates account-scoped bucket keys its objects are still at apps/console/, which the runtime no longer reads; move them once with: carlos accounts migrate --app console --from legacy --to bab; kept for the next pass held.example.com: held stage of v1 for mv_abcdefghijklmnopqrstuvwxyz: live held stage request "held.example.com.json": refused at publication: release: held stage request no longer matches its move: the claim names another op; removed held stage request "held.example.com.json": unreadable: held.example.com.json: not a regular file with one link; removed 2026/10/13 00:47:08 program check: manifest console/v4: declares wrong app/version 2026/10/13 00:47:08 program check: manifest console/v3: declares wrong app/version 2026/10/13 00:47:08 program check: manifest console/v2: validation error: unsupported manifest schema 2 2026/10/13 00:47:08 program check: manifest console/v2: decode error: invalid character 'o' in literal null (expecting 'u') console.example.com: updated vb1 -> vb2 (canary/x) console.example.com: updated -> va (canary/x) console.example.com: updated va -> vb2 (canary/x) console.example.com: updated -> va (canary/x) console.example.com: updated va -> vb (canary/x) console.example.com: updated -> old (canary/x) console.example.com: new is example.com/carlos, but this route has only run example.com/console — not pointed live (ship a new version with carlos ship --program-change and promote it, if the change is intended) console.example.com: updated -> va (canary/x) console.example.com: updated va -> vb (canary/x) console.example.com: updated vb -> va (canary/x) console.example.com: updated -> va (canary/x) console.example.com: updated va -> vb (canary/x) console.example.com: updated vb -> va (canary/x) console.example.com: updated -> va (canary/x) console.example.com: updated va -> vb (canary/x) console.example.com: updated vb -> va (canary/x) console.example.com: updated vb -> va (canary/x) x.example.com: updated old -> new (canary/x) sleeper.example.com: new (sha256 c0e6c71ad4b1f424bc3bbf19f5e5ca5e0d8b1b6a8df6e59ff7049b4f2cd61426) is not proven on dev; staying on old (no proof from dev) proof policy /tmp/TestProofGateRequires499295981/005/proof-policy.json: does not parse: unexpected EOF; every named-unit route waits until it is fixed half.example.com: download v1: dirstore: apps/bab/half/releases/v1/linux-arm64: file does not exist — if this deployment predates account-scoped bucket keys its objects are still at apps/half/, which the runtime no longer reads; move them once with: carlos accounts migrate --app half --from legacy --to bab 2026/10/13 00:47:13 release: pointer digest ratchet: open directory /tmp/TestRatchetFailsClosedWhenTheMarkCannotBePersisted2390364720/001/ro/bab/km: permission denied 2026/10/13 00:47:13 release: pointer digest ratchet: is a directory 2026/10/13 00:47:13 refusal: /tmp/TestReadRefusalCorruptIsAbsent3139189418/001/refusals/console.example.com.json does not parse (unexpected end of JSON input), treating it as absent console.example.com: update to new failed: console.service did not come up: not ready after 30s — rolled the symlink back awake.example.com: updated -> v1 (canary/x) web.example.com: updated -> v1 (canary/x) bot.example.com: repointed -> v1 (canary/x; sidecar, its worker moves onto it on the agent's drift check) nap.example.com: repointed -> v1 (canary/x; sidecar, its worker moves onto it on the agent's drift check) web.example.com: updated -> v1 (canary/x) bot.example.com: repointed -> v1 (canary/x; sidecar, its worker moves onto it on the agent's drift check) web.example.com: update to v2 failed: did not come up — rolled the symlink back bot.example.com: skipping v2 (failed on another host this pass) bot.example.com: v1 (stable) is in its supply-chain bake window until 2026-10-16 04:22 UTC bot.example.com: repointed -> v1 (canary/x; sidecar, its worker moves onto it on the agent's drift check) web.example.com: updated -> v1 (canary/x) nap.example.com: repointed -> v1 (canary/x; hibernating — bounced if awake, else runs it on next wake) web.example.com: updated -> v1 (canary/x) nap.example.com: repointed -> v1 (canary/x; hibernating — bounced if awake, else runs it on next wake) bot.example.com: repointed -> v1 (canary/x; sidecar, its worker moves onto it on the agent's drift check) blocked.example.com: updated -> v1 (canary/x) blocked.example.com: repoint failed: symlink /tmp/TestPassRelaysBlockedLiveSwapForStagedRelease4280306519/002/versions/v2/myapp /tmp/TestPassRelaysBlockedLiveSwapForStagedRelease4280306519/002/live/blocked.example.com.tmp: permission denied blocked.example.com: updated v1 -> v2 (canary/x) h1.example.com: updated -> v1 (canary/x) h1.example.com: updated -> v1 (canary/x) h1.example.com: updated -> v1 (canary/x) h1.example.com: updated -> v1 (canary/x) h1.example.com: updated -> v1 (canary/x) h1.example.com: updated -> v1 (canary/x) h1.example.com: updated -> v1 (canary/x) h1.example.com: updated -> v1 (canary/x) h1.example.com: updated -> v1 (canary/x) h1.example.com: updated -> v1 (canary/x) h1.example.com: updated -> v2 (canary/x) h1.example.com: updated -> v1 (canary/x) h1.example.com: updated -> v1 (canary/x) h1.example.com: updated -> v1 (canary/x) h1.example.com: updated -> v1 (canary/x) h1.example.com: updated -> v1 (canary/x) h1.example.com: updated -> v1 (canary/x) h1.example.com: updated -> v1 (canary/x) h1.example.com: updated -> v1 (canary/x) FAIL FAIL github.com/carlosframework/platform/internal/release 19.656s ok github.com/carlosframework/platform/internal/rollout 3.240s ok github.com/carlosframework/platform/internal/rolloutwords 0.003s ok github.com/carlosframework/platform/internal/rootaws 0.007s ok github.com/carlosframework/platform/internal/s3compat 0.005s ok github.com/carlosframework/platform/internal/safefile 0.004s ok github.com/carlosframework/platform/internal/schedule 2.266s ok github.com/carlosframework/platform/internal/sim/hostlife 8.452s ok github.com/carlosframework/platform/internal/sim/litestreamfacts 3.965s ok github.com/carlosframework/platform/internal/status 0.038s ok github.com/carlosframework/platform/internal/steering 0.006s ok github.com/carlosframework/platform/internal/stopevidence 0.216s ok github.com/carlosframework/platform/internal/svcrole 0.002s ok github.com/carlosframework/platform/internal/telemetry 0.004s ok github.com/carlosframework/platform/internal/termfence 0.004s ok github.com/carlosframework/platform/internal/tunnelclient 0.004s ok github.com/carlosframework/platform/internal/unixdial 0.002s ok github.com/carlosframework/platform/internal/updatewords 0.004s ok github.com/carlosframework/platform/internal/usage 0.004s ok github.com/carlosframework/platform/internal/version 0.002s ok github.com/carlosframework/platform/internal/vet 1.546s FAIL make: *** [Makefile:18: test] Error 1 amadan: step gate failed: exit status 2 -
cross not run
No output.
-
minio-contract not run
No output.
Whole run log
=== gate ===
cd launchctx && go vet ./... && go test ./... -count=1
ok amadan.net/carlos/platform.git/launchctx 0.011s
? amadan.net/carlos/platform.git/launchctx/controlname [no test files]
? amadan.net/carlos/platform.git/launchctx/wireid [no test files]
go vet ./...
gofmt -l .: clean
go test ./... -count=1 -timeout 30m
ok github.com/carlosframework/platform 339.457s
? github.com/carlosframework/platform/cmd/changelog [no test files]
ok github.com/carlosframework/platform/cmd/console 3.175s
ok github.com/carlosframework/platform/cmd/moveprobe 93.227s
ok github.com/carlosframework/platform/internal/activator 234.538s
ok github.com/carlosframework/platform/internal/agentsock 0.014s
ok github.com/carlosframework/platform/internal/alerting 0.005s
ok github.com/carlosframework/platform/internal/analytics 0.020s
ok github.com/carlosframework/platform/internal/appconfig 2.328s
ok github.com/carlosframework/platform/internal/apperr 0.078s
ok github.com/carlosframework/platform/internal/applog 0.160s
ok github.com/carlosframework/platform/internal/awsinv 0.005s
ok github.com/carlosframework/platform/internal/boxchoice 0.007s
ok github.com/carlosframework/platform/internal/boxid 0.671s
? github.com/carlosframework/platform/internal/boxid/boxidtest [no test files]
? github.com/carlosframework/platform/internal/boxid/internal/signtest [no test files]
ok github.com/carlosframework/platform/internal/boxvolumes 0.010s
ok github.com/carlosframework/platform/internal/caps 0.012s
ok github.com/carlosframework/platform/internal/certwarm 0.002s
ok github.com/carlosframework/platform/internal/cgroup 0.191s
ok github.com/carlosframework/platform/internal/changelog 0.246s
ok github.com/carlosframework/platform/internal/clamd 0.055s
? github.com/carlosframework/platform/internal/cliflag [no test files]
ok github.com/carlosframework/platform/internal/console 364.342s
? github.com/carlosframework/platform/internal/controlname [no test files]
? github.com/carlosframework/platform/internal/controlsock [no test files]
ok github.com/carlosframework/platform/internal/dbdoor 5.794s
ok github.com/carlosframework/platform/internal/dbdoor/dbdoortest 0.009s
ok github.com/carlosframework/platform/internal/dbverify 0.004s
ok github.com/carlosframework/platform/internal/dnsrec 0.005s
ok github.com/carlosframework/platform/internal/docsite 0.004s
ok github.com/carlosframework/platform/internal/domains 2.115s
ok github.com/carlosframework/platform/internal/doorkey 0.006s
ok github.com/carlosframework/platform/internal/economics 0.004s
ok github.com/carlosframework/platform/internal/edge 26.053s
? github.com/carlosframework/platform/internal/edgeproof [no test files]
ok github.com/carlosframework/platform/internal/edgesfile 0.007s
ok github.com/carlosframework/platform/internal/fleetapi 0.123s
ok github.com/carlosframework/platform/internal/gate 1.654s
ok github.com/carlosframework/platform/internal/genselect 0.015s
ok github.com/carlosframework/platform/internal/geo 0.015s
ok github.com/carlosframework/platform/internal/geoip 0.029s
ok github.com/carlosframework/platform/internal/host 74.035s
ok github.com/carlosframework/platform/internal/hostactors 0.366s
ok github.com/carlosframework/platform/internal/hostbox 1.857s
ok github.com/carlosframework/platform/internal/hostclaim 0.510s
ok github.com/carlosframework/platform/internal/hostevidence 0.016s
ok github.com/carlosframework/platform/internal/hostexec 0.305s
ok github.com/carlosframework/platform/internal/hostfence 0.038s
ok github.com/carlosframework/platform/internal/hostfiles 10.423s
ok github.com/carlosframework/platform/internal/hostfloor 0.015s
ok github.com/carlosframework/platform/internal/hostguard 0.144s
ok github.com/carlosframework/platform/internal/hostops 1.408s
? github.com/carlosframework/platform/internal/hostops/fencerelease [no test files]
ok github.com/carlosframework/platform/internal/hostops/internal/fencerelease 0.015s
ok github.com/carlosframework/platform/internal/hostquarantine 0.015s
ok github.com/carlosframework/platform/internal/hostrelocate 0.447s
ok github.com/carlosframework/platform/internal/hostresolve 0.149s
ok github.com/carlosframework/platform/internal/hostwrite 0.015s
ok github.com/carlosframework/platform/internal/infraview 0.016s
ok github.com/carlosframework/platform/internal/instancectl 1.706s
ok github.com/carlosframework/platform/internal/instanceexec 2.697s
ok github.com/carlosframework/platform/internal/instancefs 0.015s
ok github.com/carlosframework/platform/internal/instancekey 0.015s
ok github.com/carlosframework/platform/internal/instanceupdates 0.025s
ok github.com/carlosframework/platform/internal/layout 1.421s
ok github.com/carlosframework/platform/internal/lease 9.442s
ok github.com/carlosframework/platform/internal/lease/leasetest 0.005s
ok github.com/carlosframework/platform/internal/ledger 0.011s
ok github.com/carlosframework/platform/internal/mail 0.170s
ok github.com/carlosframework/platform/internal/maintenance 0.077s
ok github.com/carlosframework/platform/internal/maintenanceexec 5.504s
ok github.com/carlosframework/platform/internal/maintenanceguard 0.068s
ok github.com/carlosframework/platform/internal/metrics 0.005s
ok github.com/carlosframework/platform/internal/moves 20.969s
ok github.com/carlosframework/platform/internal/moves/archive 0.761s
ok github.com/carlosframework/platform/internal/objstore 0.174s
ok github.com/carlosframework/platform/internal/ops 0.167s
ok github.com/carlosframework/platform/internal/park 0.201s
ok github.com/carlosframework/platform/internal/privatefile 0.003s
ok github.com/carlosframework/platform/internal/provision 3.317s
ok github.com/carlosframework/platform/internal/publicrelease 0.006s
? github.com/carlosframework/platform/internal/regionname [no test files]
ok github.com/carlosframework/platform/internal/registry 4.010s
ok github.com/carlosframework/platform/internal/relay 7.229s
awake.example.com: updated -> v1 (canary/x)
sleepy.example.com: repointed -> v1 (canary/x; hibernating — runs it on next wake)
awake.example.com: updated -> v1 (canary/x)
sleepy.example.com: repointed -> v1 (canary/x; hibernating — runs it on next wake)
awake.example.com: updated -> v1 (canary/x)
sleepy.example.com: repointed -> v1 (canary/x; hibernating — bounced if awake, else runs it on next wake)
zzz-awake.example.com: updated -> v1 (canary/x)
aaa-sleepy.example.com: repointed -> v1 (canary/x; hibernating — bounced if awake, else runs it on next wake)
sleepy.example.com: repointed -> v1 (canary/x; hibernating — bounced if awake, else runs it on next wake)
sleepy.example.com: update to v2 failed: bounce failed — rolled the symlink back
awake.example.com: updated -> v1 (canary/x)
awake.example.com: updated -> v1 (canary/x)
a.example.com: update to v2 failed: unit failed — rolled the symlink back
b.example.com: skipping v2 (failed on another host this pass)
h.example.com: v1 (stable) is in its supply-chain bake window until 2026-10-16 10:54 UTC
myapp.example.com: updated -> v1 (canary/x)
kass.example.com: updated -> v2 (canary/x)
ops.example.com: updated -> v1 (canary/x)
kass.example.com: updated -> v1 (canary/x)
ops.example.com: updated -> v1 (canary/x)
h.example.com: release v1 has no linux-arm64 build — skipping
canary.example.com: updated -> v1 (canary/x)
old-canary.example.com: updated -> v1 (canary/x)
awake-a.example.com: updated -> v1 (canary/x)
awake-b.example.com: updated -> v1 (canary/x)
sleepy.example.com: repointed -> v1 (canary/x; hibernating — runs it on next wake)
old-canary.example.com: updated -> v1 (canary/x)
fresh-canary.example.com: updated -> v1 (canary/x)
h.example.com: stage manifest sibling for v1abc: is a directory
h.example.com: back-fill manifest for v1abc: dirstore: apps/bab/keymail/releases/v1abc/manifest.json: file does not exist — if this deployment predates account-scoped bucket keys its objects are still at apps/keymail/, which the runtime no longer reads; move them once with: carlos accounts migrate --app keymail --from legacy --to bab
keymail.dev: carlos-app@keymail.dev.socket was not enabled — enabled it, or its ListenStream binds nothing after a reboot
keymail.dev: carlos-app@keymail.dev.service was not enabled — enabled it, so it comes back after a reboot
keymail.dev: capability drop-ins converged and the unit restarted
keymail.dev: updated v1 -> v2 (canary/x)
a.example.com: carlos-app@a.example.com.socket was not enabled — enabled it, or its ListenStream binds nothing after a reboot
a.example.com: carlos-app@a.example.com.service was not enabled — enabled it, so it comes back after a reboot
a.example.com: capability drop-ins converged and the unit restarted
a.example.com: updated -> v1 (canary/x)
b.example.com: updated -> v1 (canary/x)
exec.example.com: updated -> v1 (canary/x)
unit.example.com: v1 declares capabilities and this pass does not write unit configuration — it runs against the drop-ins the previous version left until the root adopt pass (carlos-adopt.service, hourly) converges them
unit.example.com: updated -> v1 (canary/x)
sleepy.example.com: port:25 is granted on a hibernating route — a connection to that port starts the unit directly, bypassing the activator's health-gated wake; the edge's clean 503 on a broken boot does not cover this path
sleepy.example.com: carlos-app@sleepy.example.com.socket was not enabled — enabled it, or its ListenStream binds nothing after a reboot
sleepy.example.com: capability drop-ins converged and the unit restarted
sleepy.example.com: repointed -> v1 (canary/x; hibernating — runs it on next wake)
keymail.dev: back-fill manifest for v2: dirstore: apps/bab/keymail/releases/v2/manifest.json: file does not exist — if this deployment predates account-scoped bucket keys its objects are still at apps/keymail/, which the runtime no longer reads; move them once with: carlos accounts migrate --app keymail --from legacy --to bab
keymail.dev: updated v1 -> v2 (canary/x)
keymail.dev: carlos-app@keymail.dev.socket was not enabled — enabled it, or its ListenStream binds nothing after a reboot
keymail.dev: carlos-app@keymail.dev.service was not enabled — enabled it, so it comes back after a reboot
keymail.dev: capability drop-ins converged and the unit restarted
keymail.dev: updated -> v1 (canary/x)
keymail.dev: no capability materializes here any more — removed /tmp/TestAdoptRemovesDropInsWhenTheNewVersionDeclaresNothing2766118413/004/carlos-app@keymail.dev.service.d/10-carlos-capabilities.conf
keymail.dev: no capability materializes here any more — removed /tmp/TestAdoptRemovesDropInsWhenTheNewVersionDeclaresNothing2766118413/004/carlos-app@keymail.dev.socket.d/10-carlos-capabilities.conf
keymail.dev: capability drop-ins converged and the unit restarted
keymail.dev: updated v1 -> v2 (canary/x)
keymail.dev: carlos-app@keymail.dev.socket was not enabled — enabled it, or its ListenStream binds nothing after a reboot
keymail.dev: carlos-app@keymail.dev.service was not enabled — enabled it, so it comes back after a reboot
keymail.dev: capability drop-ins converged and the unit restarted
keymail.dev: updated -> v1 (canary/x)
keymail.dev: capability drop-ins converged
keymail.dev: updated v1 -> v2 (canary/x)
keymail.dev: carlos-app@keymail.dev.service was not enabled — enabled it, so it comes back after a reboot
keymail.dev: unit state: unit masked
keymail.dev: no capability materializes here any more — removed /tmp/TestConvergeFailureAfterItsReloadRestoresTheDropIns1559584099/004/carlos-app@keymail.dev.service.d/10-carlos-capabilities.conf
keymail.dev: no capability materializes here any more — removed /tmp/TestConvergeFailureAfterItsReloadRestoresTheDropIns1559584099/004/carlos-app@keymail.dev.socket.d/10-carlos-capabilities.conf
keymail.dev: capability drop-ins rolled back to the previous version's
keymail.dev: capabilities for v2 were refused: unit masked — not pointed live
sleepy.example.com: port:25 is granted on a hibernating route — a connection to that port starts the unit directly, bypassing the activator's health-gated wake; the edge's clean 503 on a broken boot does not cover this path
sleepy.example.com: carlos-app@sleepy.example.com.socket was not enabled — enabled it, or its ListenStream binds nothing after a reboot
sleepy.example.com: capability drop-ins converged and the unit restarted
sleepy.example.com: repointed -> v1 (canary/x; hibernating — runs it on next wake)
sleepy.example.com: capability daemon and hibernation contradict — nothing written and no unit state changed; fix it with: carlos route --host sleepy.example.com --hibernate=false
sleepy.example.com: capabilities for v2 were refused: capability daemon and hibernation contradict — not pointed live
keymail.dev: carlos-app@keymail.dev.socket was not enabled — enabled it, or its ListenStream binds nothing after a reboot
keymail.dev: carlos-app@keymail.dev.service was not enabled — enabled it, so it comes back after a reboot
keymail.dev: capability drop-ins converged and the unit restarted
keymail.dev: updated -> v1 (canary/x)
keymail.dev: carlos-app@keymail.dev.socket was not enabled — enabled it, or its ListenStream binds nothing after a reboot
keymail.dev: carlos-app@keymail.dev.service was not enabled — enabled it, so it comes back after a reboot
keymail.dev: capability drop-ins converged and the unit restarted
keymail.dev: updated v1 -> v2 (canary/x)
keymail.dev: carlos-app@keymail.dev.socket was not enabled — enabled it, or its ListenStream binds nothing after a reboot
keymail.dev: carlos-app@keymail.dev.service was not enabled — enabled it, so it comes back after a reboot
keymail.dev: capability drop-ins converged and the unit restarted
keymail.dev: updated -> v1 (canary/x)
h.example.com: updated -> v1 (stable)
h.example.com: v1 (stable) is in its supply-chain bake window until 2026-10-16 17:24 UTC
h.example.com: updated -> v1 (stable)
bab/myapp: production flag unreadable (production flag wrapper: invalid character 'o' in literal null (expecting 'u')) — treating as production, holds apply
h.example.com: v1 (stable) is in its supply-chain bake window until 2026-10-16 13:00 UTC
h.example.com: v1 (stable) is in its supply-chain bake window until 2026-10-16 03:25 UTC
h.example.com: v1 (stable) is in its supply-chain bake window until 2026-10-16 03:25 UTC
cx.example.com: updated -> v1 (canary/x)
cy.example.com: updated -> v1 (canary/y)
s.example.com: v1 (stable) is in its supply-chain bake window until 2026-10-16 08:49 UTC
cx.example.com: updated -> v1 (canary/x)
s.example.com: v1 (stable) is in its supply-chain bake window until 2026-10-16 02:40 UTC
h.example.com: v1 (stable) is in its supply-chain bake window until 2026-10-16 14:25 UTC
h.example.com: v1 (stable) is in its supply-chain bake window until 2026-10-16 14:25 UTC
e.example.com: updated -> v1 (edge)
p.example.com: v1 (production) is in its supply-chain bake window until 2026-10-16 11:56 UTC
p.example.com: updated -> v1 (production)
p.example.com: v2 (production) is in its supply-chain bake window until 2026-10-14 04:30 UTC
p.example.com: updated v1 -> v2 (production)
c.example.com: updated -> v1 (canary/x)
p.example.com: v1 (production) is in its supply-chain bake window until 2026-10-16 18:43 UTC
bab/myapp: pipeline unreadable (pipeline wrapper: invalid character 'o' in literal null (expecting 'u')) — remembered windows still apply; anything unremembered falls to legacy holds
p.example.com: v1 (production) is in its supply-chain bake window until 2026-10-14 17:47 UTC
e.example.com: v1 (edge) is in its supply-chain bake window until 2026-10-14 03:51 UTC
bab/myapp: pipeline unreadable (pipeline wrapper: invalid character 'o' in literal null (expecting 'u')) — remembered windows still apply; anything unremembered falls to legacy holds
e.example.com: v1 (edge) is in its supply-chain bake window until 2026-10-14 03:51 UTC
e.example.com: v1 (edge) is in its supply-chain bake window until 2026-10-14 06:08 UTC
e.example.com: v1 (edge) is in its supply-chain bake window until 2026-10-14 06:08 UTC
e.example.com: v1 (edge) is in its supply-chain bake window until 2026-10-14 06:08 UTC
e.example.com: updated -> v1 (edge)
p.example.com: v1 (production) is in its supply-chain bake window until 2026-10-16 01:25 UTC
p.example.com: updated -> v1 (production)
p.example.com: v2 (production) is in its supply-chain bake window until 2026-10-14 00:59 UTC
bab/myapp: pipeline unreadable (pipeline wrapper: invalid character 'o' in literal null (expecting 'u')) — remembered windows still apply; anything unremembered falls to legacy holds
p.example.com: v2 (production) is in its supply-chain bake window until 2026-10-16 01:25 UTC
p.example.com: v1 (production) is in its supply-chain bake window until 2026-10-16 09:22 UTC
p.example.com: updated -> v1 (production)
p.example.com: v2 (production) is in its supply-chain bake window until 2026-10-14 03:39 UTC
p.example.com: v2 (production) is in its supply-chain bake window until 2026-10-16 09:23 UTC
p.example.com: updated v1 -> v2 (production)
p.example.com: v1 (production) is in its supply-chain bake window until 2026-10-16 17:07 UTC
p.example.com: updated -> v1 (production)
p.example.com: v2 (production) is in its supply-chain bake window until 2026-10-14 06:13 UTC
p.example.com: v2 (production) is in its supply-chain bake window until 2026-10-16 17:07 UTC
p.example.com: v2 (production) is in its supply-chain bake window until 2026-10-16 17:07 UTC
p.example.com: v2 (production) is in its supply-chain bake window until 2026-10-16 17:07 UTC
e.example.com: v1 (edge) is in its supply-chain bake window until 2026-10-14 03:10 UTC
bab/myapp: pipeline unreadable (pipeline wrapper: invalid character 'o' in literal null (expecting 'u')) — remembered windows still apply; anything unremembered falls to legacy holds
e.example.com: v1 (edge) is in its supply-chain bake window until 2026-10-14 03:10 UTC
e.example.com: v1 (edge) is in its supply-chain bake window until 2026-10-14 05:19 UTC
e.example.com: v1 (edge) is in its supply-chain bake window until 2026-10-14 05:19 UTC
e.example.com: updated -> v1 (edge)
e.example.com: updated -> v1 (edge)
bab/myapp: pipeline unreadable (pipeline wrapper: invalid character 'o' in literal null (expecting 'u')) — remembered windows still apply; anything unremembered falls to legacy holds
e.example.com: updated -> v1 (edge)
cx.example.com: updated -> v1 (canary/x)
cy.example.com: updated -> v1 (canary/y)
e.example.com: updated -> v1 (edge)
s.example.com: updated -> v1 (stable)
console.example.com: update to new failed: Job for console.service failed — rolled the symlink back
console.example.com: update to new failed: still broken — rolled the symlink back
console.example.com: update to new failed: still broken — rolled the symlink back
console.example.com: updated old -> new (canary/x)
console.example.com: update to new failed: Job for console.service failed — rolled the symlink back
console.example.com: updated old -> new (canary/x)
h1.example.com: updated -> v1 (canary/x)
h1.example.com: updated -> v1 (canary/x)
h1.example.com: updated -> v1 (canary/x)
a.example.com: updated -> v1 (canary/x)
a.example.com: updated -> v2 (canary/x)
a.example.com: updated -> v1 (canary/x)
a.example.com: updated -> v1 (canary/x)
a.example.com: updated -> v1 (canary/x)
p.example.com: v1 (production) is in its supply-chain bake window until 2026-10-16 02:51 UTC
p.example.com: updated -> v1 (production)
p.example.com: v2 (production) is in its supply-chain bake window until 2026-10-14 01:28 UTC
a.example.com: v1 recovered from this box's verified staged copy (v1: promotion record: s3: 503 slow down)
a.example.com: v1 recovered from this box's verified staged copy (manifest for km/v1: s3: 503 slow down)
app.example: repoint failed: release selection changed
app.example: could not roll the symlink back to v1: release selection changed
app.example: update to v2 failed: restart failed — could not roll the symlink back
app.example: update to v1 failed: hostwrite: lifecycle provider unavailable — rolled the symlink back
app.example: updated -> v1 (canary/x)
app.example: update to v1 failed: release selection changed — rolled the symlink back
app.example: update to v2 failed: restart failed — rolled the symlink back
app.example: rolled back to v1 but could not start it: release selection changed — this host may be serving nothing
h1.example.com: updated -> v1 (canary/x)
h1.example.com: updated -> v1 (canary/x)
2026/10/13 00:47:07 healthy history: /tmp/TestHealthyCorruptReadsEmpty2213477926/001/healthy/bab/console.json does not parse (unexpected end of JSON input), treating it as empty
held stage request "held.example.com.json": refused: release: the held release is not authorized: no promotion record and no signed pointer names it: bab/console at v3; removed
held stage request "held.example.com.json": refused: manifest for console/v1: release: manifest does not hash to the digest its signer recorded: console/v1 hashes to 9c2d216b7a41, the pointer signed 05cd819d301c; removed
held.example.com: held stage of v1 for mv_abcdefghijklmnopqrstuvwxyz: release: held stage of bab/console at v1: reading channel canary/x: transient read failure; kept for the next pass
2026/10/13 00:47:07 release: pointer digest ratchet: open directory /tmp/TestHeldStageRetriesARatchetThatCannotTellthe_mark_could_not_be3246880805/002/ro/bab/km: permission denied
2026/10/13 00:47:07 release: pointer digest ratchet: open directory /tmp/TestHeldAuthorizedAdmitsOnlyNamingPointers1686759976/003/ro/bab/km: permission denied
held.example.com: v1: the staged bytes do not match the proof (staged sha256 22e7ffce966280e2f2c9765e5ac4feaa19c2883fc9cddf0a97610e92cda7f6e5, dev proved 683a664a34bd9c537ce6ebbcd5878689db9147dd15d292152be4af4a5a0ab168); not pointed live
held.example.com: v1: the staged bytes do not match the proof (staged sha256 22e7ffce966280e2f2c9765e5ac4feaa19c2883fc9cddf0a97610e92cda7f6e5, dev proved 683a664a34bd9c537ce6ebbcd5878689db9147dd15d292152be4af4a5a0ab168); not pointed live
held.example.com: v1: the staged bytes do not match the proof (staged sha256 22e7ffce966280e2f2c9765e5ac4feaa19c2883fc9cddf0a97610e92cda7f6e5, dev proved 683a664a34bd9c537ce6ebbcd5878689db9147dd15d292152be4af4a5a0ab168); not pointed live
held.example.com: v1: the staged bytes do not match the proof (staged sha256 487b54327a2fde78118d00e9d25a33dec1689d85ab81c72e671b69cd3fd8b5d2, dev proved 683a664a34bd9c537ce6ebbcd5878689db9147dd15d292152be4af4a5a0ab168); not pointed live
held.example.com: v1 (sha256 683a664a34bd9c537ce6ebbcd5878689db9147dd15d292152be4af4a5a0ab168) is not proven on dev; staying on v1 (proof from dev: proof signature does not verify against the configured key)
proof policy /tmp/TestVerifyHeldLiveRechecksAnExistingLink3741064768/004/proof-policy.json: does not parse: invalid character 'n' looking for beginning of object key string; every named-unit route waits until it is fixed
held.example.com: held stage of v1: proof policy /tmp/TestVerifyHeldLiveRechecksAnExistingLink3741064768/004/proof-policy.json: does not parse: invalid character 'n' looking for beginning of object key string; not staged
--- FAIL: TestVerifyHeldLiveUnderReadsUnderTheClaim (0.00s)
heldverify_test.go:221: VerifyHeldLive under the claim: <nil>, want ErrHeldClaimWired
mat.test: updated -> abc1234 (canary/dev)
mat.test: abc1234 (beta) is in its supply-chain bake window until 2026-10-14 03:24 UTC
bab/matapp/canary/dev: channel has no release yet
mat.test: release 0.1.0 has no linux-arm64 build — skipping
bab/matapp/canary/dev: channel has no release yet
held.example.com: v1 (sha256 683a664a34bd9c537ce6ebbcd5878689db9147dd15d292152be4af4a5a0ab168) is not proven on dev; staying on v0 (no proof from dev)
held.example.com: v1: the staged bytes do not match the proof (staged sha256 3a2d06131202d057a22794871fefac411198fd6d3ff8515a59b60d0a27493699, dev proved 683a664a34bd9c537ce6ebbcd5878689db9147dd15d292152be4af4a5a0ab168); not pointed live
held stage request "held.example.com.json": refused: release: held stage request no longer matches its move: the claim names mv_abcdefghijklmnopqrstuvwxyz; removed
held stage request "nul.json": refused: its host is not its file name; removed
held stage request "held.example.com.json": refused: malformed version; removed
held stage request "held.example.com.json": refused: malformed account; removed
held stage request "held.example.com.json": refused: malformed app; removed
held stage request "held.example.com.json": refused: no move op; removed
held stage request "held.example.com.json": refused: no box; removed
held stage request "evil.json": refused: its host is not its file name; removed
held stage request "held.example.com.json": refused: its host is not its file name; removed
held stage request "held.example.com.json": not a request; removed
held stage request "held.example.com.json": unreadable: held.example.com.json: not a regular file with one link; removed
held.example.com: held stage of v9 for mv_abcdefghijklmnopqrstuvwxyz: manifest for console/v9: dirstore: apps/bab/console/releases/v9/manifest.json: file does not exist — if this deployment predates account-scoped bucket keys its objects are still at apps/console/, which the runtime no longer reads; move them once with: carlos accounts migrate --app console --from legacy --to bab; kept for the next pass
held.example.com: held stage of v1 for mv_abcdefghijklmnopqrstuvwxyz: live
held stage request "held.example.com.json": refused at publication: release: held stage request no longer matches its move: the claim names another op; removed
held stage request "held.example.com.json": unreadable: held.example.com.json: not a regular file with one link; removed
2026/10/13 00:47:08 program check: manifest console/v4: declares wrong app/version
2026/10/13 00:47:08 program check: manifest console/v3: declares wrong app/version
2026/10/13 00:47:08 program check: manifest console/v2: validation error: unsupported manifest schema 2
2026/10/13 00:47:08 program check: manifest console/v2: decode error: invalid character 'o' in literal null (expecting 'u')
console.example.com: updated vb1 -> vb2 (canary/x)
console.example.com: updated -> va (canary/x)
console.example.com: updated va -> vb2 (canary/x)
console.example.com: updated -> va (canary/x)
console.example.com: updated va -> vb (canary/x)
console.example.com: updated -> old (canary/x)
console.example.com: new is example.com/carlos, but this route has only run example.com/console — not pointed live (ship a new version with carlos ship --program-change and promote it, if the change is intended)
console.example.com: updated -> va (canary/x)
console.example.com: updated va -> vb (canary/x)
console.example.com: updated vb -> va (canary/x)
console.example.com: updated -> va (canary/x)
console.example.com: updated va -> vb (canary/x)
console.example.com: updated vb -> va (canary/x)
console.example.com: updated -> va (canary/x)
console.example.com: updated va -> vb (canary/x)
console.example.com: updated vb -> va (canary/x)
console.example.com: updated vb -> va (canary/x)
x.example.com: updated old -> new (canary/x)
sleeper.example.com: new (sha256 c0e6c71ad4b1f424bc3bbf19f5e5ca5e0d8b1b6a8df6e59ff7049b4f2cd61426) is not proven on dev; staying on old (no proof from dev)
proof policy /tmp/TestProofGateRequires499295981/005/proof-policy.json: does not parse: unexpected EOF; every named-unit route waits until it is fixed
half.example.com: download v1: dirstore: apps/bab/half/releases/v1/linux-arm64: file does not exist — if this deployment predates account-scoped bucket keys its objects are still at apps/half/, which the runtime no longer reads; move them once with: carlos accounts migrate --app half --from legacy --to bab
2026/10/13 00:47:13 release: pointer digest ratchet: open directory /tmp/TestRatchetFailsClosedWhenTheMarkCannotBePersisted2390364720/001/ro/bab/km: permission denied
2026/10/13 00:47:13 release: pointer digest ratchet: is a directory
2026/10/13 00:47:13 refusal: /tmp/TestReadRefusalCorruptIsAbsent3139189418/001/refusals/console.example.com.json does not parse (unexpected end of JSON input), treating it as absent
console.example.com: update to new failed: console.service did not come up: not ready after 30s — rolled the symlink back
awake.example.com: updated -> v1 (canary/x)
web.example.com: updated -> v1 (canary/x)
bot.example.com: repointed -> v1 (canary/x; sidecar, its worker moves onto it on the agent's drift check)
nap.example.com: repointed -> v1 (canary/x; sidecar, its worker moves onto it on the agent's drift check)
web.example.com: updated -> v1 (canary/x)
bot.example.com: repointed -> v1 (canary/x; sidecar, its worker moves onto it on the agent's drift check)
web.example.com: update to v2 failed: did not come up — rolled the symlink back
bot.example.com: skipping v2 (failed on another host this pass)
bot.example.com: v1 (stable) is in its supply-chain bake window until 2026-10-16 04:22 UTC
bot.example.com: repointed -> v1 (canary/x; sidecar, its worker moves onto it on the agent's drift check)
web.example.com: updated -> v1 (canary/x)
nap.example.com: repointed -> v1 (canary/x; hibernating — bounced if awake, else runs it on next wake)
web.example.com: updated -> v1 (canary/x)
nap.example.com: repointed -> v1 (canary/x; hibernating — bounced if awake, else runs it on next wake)
bot.example.com: repointed -> v1 (canary/x; sidecar, its worker moves onto it on the agent's drift check)
blocked.example.com: updated -> v1 (canary/x)
blocked.example.com: repoint failed: symlink /tmp/TestPassRelaysBlockedLiveSwapForStagedRelease4280306519/002/versions/v2/myapp /tmp/TestPassRelaysBlockedLiveSwapForStagedRelease4280306519/002/live/blocked.example.com.tmp: permission denied
blocked.example.com: updated v1 -> v2 (canary/x)
h1.example.com: updated -> v1 (canary/x)
h1.example.com: updated -> v1 (canary/x)
h1.example.com: updated -> v1 (canary/x)
h1.example.com: updated -> v1 (canary/x)
h1.example.com: updated -> v1 (canary/x)
h1.example.com: updated -> v1 (canary/x)
h1.example.com: updated -> v1 (canary/x)
h1.example.com: updated -> v1 (canary/x)
h1.example.com: updated -> v1 (canary/x)
h1.example.com: updated -> v1 (canary/x)
h1.example.com: updated -> v2 (canary/x)
h1.example.com: updated -> v1 (canary/x)
h1.example.com: updated -> v1 (canary/x)
h1.example.com: updated -> v1 (canary/x)
h1.example.com: updated -> v1 (canary/x)
h1.example.com: updated -> v1 (canary/x)
h1.example.com: updated -> v1 (canary/x)
h1.example.com: updated -> v1 (canary/x)
h1.example.com: updated -> v1 (canary/x)
FAIL
FAIL github.com/carlosframework/platform/internal/release 19.656s
ok github.com/carlosframework/platform/internal/rollout 3.240s
ok github.com/carlosframework/platform/internal/rolloutwords 0.003s
ok github.com/carlosframework/platform/internal/rootaws 0.007s
ok github.com/carlosframework/platform/internal/s3compat 0.005s
ok github.com/carlosframework/platform/internal/safefile 0.004s
ok github.com/carlosframework/platform/internal/schedule 2.266s
ok github.com/carlosframework/platform/internal/sim/hostlife 8.452s
ok github.com/carlosframework/platform/internal/sim/litestreamfacts 3.965s
ok github.com/carlosframework/platform/internal/status 0.038s
ok github.com/carlosframework/platform/internal/steering 0.006s
ok github.com/carlosframework/platform/internal/stopevidence 0.216s
ok github.com/carlosframework/platform/internal/svcrole 0.002s
ok github.com/carlosframework/platform/internal/telemetry 0.004s
ok github.com/carlosframework/platform/internal/termfence 0.004s
ok github.com/carlosframework/platform/internal/tunnelclient 0.004s
ok github.com/carlosframework/platform/internal/unixdial 0.002s
ok github.com/carlosframework/platform/internal/updatewords 0.004s
ok github.com/carlosframework/platform/internal/usage 0.004s
ok github.com/carlosframework/platform/internal/version 0.002s
ok github.com/carlosframework/platform/internal/vet 1.546s
FAIL
make: *** [Makefile:18: test] Error 1
amadan: step gate failed: exit status 2
amadan: job failed at step gate — its output is above, not at the tail of this log
Watching for changes…