amadan

carlos / platform Public

console: claim loss-record paging (pending loss and possibly lost timers), hourly and durably rate-limited, inert unless CARLOS_HOST_CLAIM

Copy HC5-N4..N7 pending Paul's review. The loss-record model has no
acknowledgement transition, so pending records keep paging hourly.
Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
codex pushed by claude d918c3630d11437d63f27bb3cb7b83d9e0252281 parent 2ad5412
11 files changed, +429 −8
  • approved_moves_copy_test.go +3 −0
  • cmd/console/main.go +3 −4
  • cmd/console/movesswitch_test.go +3 −3
  • internal/console/claim_authority_census_test.go +5 −1
  • internal/console/claim_loss_pages.go +96 −0
  • internal/console/claim_loss_pages_test.go +290 −0
  • internal/console/claim_runtime.go +1 −0
  • internal/console/store_gate.go +7 −0
  • internal/console/testdata/claim-authority-sites.txt +5 −0
  • internal/hostops/recovery.go +7 −0
  • testdata/claim-moves-copy.md +9 −0

CI failed — run details

CI log 
=== gate ===
cd launchctx && go vet ./... && go test ./... -count=1
ok  	amadan.net/carlos/platform.git/launchctx	0.011s
?   	amadan.net/carlos/platform.git/launchctx/controlname	[no test files]
?   	amadan.net/carlos/platform.git/launchctx/wireid	[no test files]
go vet ./...
gofmt -l .: clean
go test ./... -count=1 -timeout 30m
ok  	github.com/carlosframework/platform	311.290s
?   	github.com/carlosframework/platform/cmd/changelog	[no test files]
ok  	github.com/carlosframework/platform/cmd/console	3.013s
ok  	github.com/carlosframework/platform/cmd/moveprobe	93.285s
ok  	github.com/carlosframework/platform/internal/activator	208.365s
ok  	github.com/carlosframework/platform/internal/agentsock	0.021s
ok  	github.com/carlosframework/platform/internal/alerting	0.004s
ok  	github.com/carlosframework/platform/internal/analytics	0.020s
ok  	github.com/carlosframework/platform/internal/appconfig	2.336s
ok  	github.com/carlosframework/platform/internal/apperr	0.099s
ok  	github.com/carlosframework/platform/internal/applog	0.229s
ok  	github.com/carlosframework/platform/internal/awsinv	0.006s
ok  	github.com/carlosframework/platform/internal/boxchoice	0.004s
ok  	github.com/carlosframework/platform/internal/boxid	0.865s
?   	github.com/carlosframework/platform/internal/boxid/boxidtest	[no test files]
?   	github.com/carlosframework/platform/internal/boxid/internal/signtest	[no test files]
ok  	github.com/carlosframework/platform/internal/boxvolumes	0.016s
ok  	github.com/carlosframework/platform/internal/caps	0.012s
ok  	github.com/carlosframework/platform/internal/certwarm	0.003s
ok  	github.com/carlosframework/platform/internal/cgroup	0.206s
ok  	github.com/carlosframework/platform/internal/changelog	0.281s
ok  	github.com/carlosframework/platform/internal/clamd	0.060s
?   	github.com/carlosframework/platform/internal/cliflag	[no test files]
ok  	github.com/carlosframework/platform/internal/console	352.302s
?   	github.com/carlosframework/platform/internal/controlname	[no test files]
?   	github.com/carlosframework/platform/internal/controlsock	[no test files]
ok  	github.com/carlosframework/platform/internal/dbdoor	5.676s
ok  	github.com/carlosframework/platform/internal/dbdoor/dbdoortest	0.009s
ok  	github.com/carlosframework/platform/internal/dbverify	0.004s
ok  	github.com/carlosframework/platform/internal/dnsrec	0.003s
ok  	github.com/carlosframework/platform/internal/docsite	0.006s
ok  	github.com/carlosframework/platform/internal/domains	1.970s
ok  	github.com/carlosframework/platform/internal/doorkey	0.006s
ok  	github.com/carlosframework/platform/internal/economics	0.005s
ok  	github.com/carlosframework/platform/internal/edge	25.744s
?   	github.com/carlosframework/platform/internal/edgeproof	[no test files]
ok  	github.com/carlosframework/platform/internal/edgesfile	0.009s
ok  	github.com/carlosframework/platform/internal/fleetapi	0.127s
ok  	github.com/carlosframework/platform/internal/gate	1.672s
ok  	github.com/carlosframework/platform/internal/genselect	0.017s
ok  	github.com/carlosframework/platform/internal/geo	0.013s
ok  	github.com/carlosframework/platform/internal/geoip	0.030s
ok  	github.com/carlosframework/platform/internal/host	53.367s
ok  	github.com/carlosframework/platform/internal/hostactors	0.530s
ok  	github.com/carlosframework/platform/internal/hostbox	1.284s
ok  	github.com/carlosframework/platform/internal/hostclaim	0.243s
ok  	github.com/carlosframework/platform/internal/hostevidence	0.006s
ok  	github.com/carlosframework/platform/internal/hostexec	0.306s
ok  	github.com/carlosframework/platform/internal/hostfence	0.046s
ok  	github.com/carlosframework/platform/internal/hostfiles	4.176s
ok  	github.com/carlosframework/platform/internal/hostfloor	0.009s
ok  	github.com/carlosframework/platform/internal/hostguard	0.135s
ok  	github.com/carlosframework/platform/internal/hostops	1.328s
?   	github.com/carlosframework/platform/internal/hostops/fencerelease	[no test files]
ok  	github.com/carlosframework/platform/internal/hostops/internal/fencerelease	0.003s
ok  	github.com/carlosframework/platform/internal/hostquarantine	0.003s
ok  	github.com/carlosframework/platform/internal/hostresolve	0.135s
ok  	github.com/carlosframework/platform/internal/hostwrite	0.005s
ok  	github.com/carlosframework/platform/internal/infraview	0.002s
ok  	github.com/carlosframework/platform/internal/instancectl	1.703s
ok  	github.com/carlosframework/platform/internal/instanceexec	2.692s
ok  	github.com/carlosframework/platform/internal/instancefs	0.008s
ok  	github.com/carlosframework/platform/internal/instancekey	0.003s
ok  	github.com/carlosframework/platform/internal/instanceupdates	0.006s
ok  	github.com/carlosframework/platform/internal/layout	1.301s
ok  	github.com/carlosframework/platform/internal/lease	9.422s
ok  	github.com/carlosframework/platform/internal/lease/leasetest	0.009s
ok  	github.com/carlosframework/platform/internal/ledger	0.013s
ok  	github.com/carlosframework/platform/internal/mail	0.150s
ok  	github.com/carlosframework/platform/internal/maintenance	0.095s
ok  	github.com/carlosframework/platform/internal/maintenanceexec	5.484s
ok  	github.com/carlosframework/platform/internal/maintenanceguard	0.069s
ok  	github.com/carlosframework/platform/internal/metrics	0.007s
ok  	github.com/carlosframework/platform/internal/moves	20.878s
ok  	github.com/carlosframework/platform/internal/moves/archive	0.735s
ok  	github.com/carlosframework/platform/internal/objstore	0.165s
ok  	github.com/carlosframework/platform/internal/ops	0.156s
ok  	github.com/carlosframework/platform/internal/park	0.194s
ok  	github.com/carlosframework/platform/internal/privatefile	0.004s
2026/10/12 13:06:30 provision: listing apps/bab/blog/instances/: list refused — its records are unseen this pass
2026/10/12 13:06:30 provision: listing apps/bab/: listdirs refused — its records are unseen this pass
2026/10/12 13:06:30 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:30 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:30 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:30 provision: listing apps/bac/tito/instances/: list refused — its records are unseen this pass
2026/10/12 13:06:30 provision: listing apps/bac/tito/instances/: list refused — its records are unseen this pass
2026/10/12 13:06:30 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/12 13:06:30 provision: remove retired instance route shop.example: provision: claim admission unavailable
2026/10/12 13:06:30 provision: remove retired instance route shop.example: provision: claim admission unavailable
2026/10/12 13:06:30 provision: remove retired sidecar route worker.sidecar.internal: provision: claim admission unavailable
2026/10/12 13:06:30 provision: remove retired sidecar route worker.sidecar.internal: provision: claim admission unavailable
2026/10/12 13:06:30 provision: claim install acme.go.tito.io: provision: claim admission unavailable
2026/10/12 13:06:31 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: claim install acme.go.tito.io: registry: claim row ownership changed or unknown
2026/10/12 13:06:31 provision: claim install bot.eleven.bac.sidecar.internal: provision: claim admission unavailable
2026/10/12 13:06:31 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["run"])
2026/10/12 13:06:31 provision: claim install bot.eleven.bac.sidecar.internal: registry: claim row ownership changed or unknown
2026/10/12 13:06:31 provision: refusing to mint route acme.go.tito.io (app "tito" acct "bac"): this box declares CARLOS_INBOUND=none (no-inbound) and cannot serve routed instance traffic
2026/10/12 13:06:31 provision: acme.go.tito.io: not minting this pass: the box's layout intent could not be read: no sealed config
2026/10/12 13:06:31 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: REFUSING to mint sidecar bot.eleven.bac.sidecar.internal: bot.eleven.bac.sidecar.internal-sidecar.env is staged on this box — the legacy file co-spawn and a native sidecar row are two mechanisms for one host; remove the file to migrate this host onto sidecar records
2026/10/12 13:06:31 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["run"])
2026/10/12 13:06:31 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["run"])
2026/10/12 13:06:31 provision: refusing to mint route acme.go.tito.io (app "tito" acct "bac"): this box declares CARLOS_INBOUND=none (no-inbound) and cannot serve routed instance traffic
2026/10/12 13:06:31 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: REFUSING to mint sidecar bot.eleven.bac.sidecar.internal: bot.eleven.bac.sidecar.internal-sidecar.env is staged on this box — the legacy file co-spawn and a native sidecar row are two mechanisms for one host; remove the file to migrate this host onto sidecar records
2026/10/12 13:06:31 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["run"])
2026/10/12 13:06:31 provision: acme.go.tito.io exists but is not the route this record describes (app "other"/"tito") — leaving it alone
2026/10/12 13:06:31 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: claim install acme.go.tito.io: registry: claim row ownership changed or unknown
2026/10/12 13:06:31 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: claim install acme.go.tito.io: registry: claim row ownership changed or unknown
2026/10/12 13:06:31 provision: bot.eleven.bac.sidecar.internal exists but is not a sidecar row this pass owns (kind "sidecar" provisioned false app "eleven" acct "bac") — leaving it alone
2026/10/12 13:06:31 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["run"])
2026/10/12 13:06:31 provision: claim install bot.eleven.bac.sidecar.internal: registry: claim row ownership changed or unknown
2026/10/12 13:06:31 provision: acme.go.tito.io exists but is not the route this record describes (app "other"/"tito") — leaving it alone
2026/10/12 13:06:31 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: adopted existing route acme.go.tito.io (app "tito" acct "bac") — this record now owns its channel and environment
2026/10/12 13:06:31 provision: bot.eleven.bac.sidecar.internal exists but is not a sidecar row this pass owns (kind "sidecar" provisioned false app "eleven" acct "bac") — leaving it alone
2026/10/12 13:06:31 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/12 13:06:31 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: skipping record apps/bac/tito/instances/acme.go.tito.io.json: malformed: provision: invalid maintenance projection or protected fields
2026/10/12 13:06:31 provision: skipping record apps/bac/tito/instances/new.go.tito.io.json: malformed: provision: invalid maintenance projection or protected fields
2026/10/12 13:06:31 provision: not removing acme.go.tito.io: its record exists but could not be read cleanly this pass
2026/10/12 13:06:31 provision: skipping record apps/bac/tito/instances/acme.go.tito.io.json: malformed: provision: invalid maintenance projection or protected fields
2026/10/12 13:06:31 provision: skipping record apps/bac/tito/instances/new.go.tito.io.json: malformed: provision: invalid maintenance projection or protected fields
2026/10/12 13:06:31 provision: minted instance route rc.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route rd.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route re.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route rf.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route rg.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route ri.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route rk.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route rl.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route ra.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route rb.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route rh.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route rj.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route ra.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route rb.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route rc.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route rd.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: skipping record apps/bac/tito/instances/rb.go.tito.io.json: read failed: read refused
2026/10/12 13:06:31 provision: skipping record apps/bac/tito/instances/rb.go.tito.io.json: read failed: read refused
2026/10/12 13:06:31 provision: minted instance route re.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route x.appa.example (app "appa" acct "bac")
2026/10/12 13:06:31 provision: minted instance route x.appb.example (app "appb" acct "bac")
2026/10/12 13:06:31 provision: minted instance route x.appc.example (app "appc" acct "bac")
2026/10/12 13:06:31 provision: minted instance route x.appe.example (app "appe" acct "bac")
2026/10/12 13:06:31 provision: minted instance route x.appf.example (app "appf" acct "bac")
2026/10/12 13:06:31 provision: minted instance route x.appg.example (app "appg" acct "bac")
2026/10/12 13:06:31 provision: minted instance route x.apph.example (app "apph" acct "bac")
2026/10/12 13:06:31 provision: minted instance route x.appj.example (app "appj" acct "bac")
2026/10/12 13:06:31 provision: minted instance route x.appd.example (app "appd" acct "bac")
2026/10/12 13:06:31 provision: minted instance route x.appi.example (app "appi" acct "bac")
2026/10/12 13:06:31 provision: bac/tito is in the trash — not minting instance routes for it
2026/10/12 13:06:31 provision: minted instance route r0019.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0041.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0047.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0095.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0100.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0056.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0106.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0021.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0029.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0073.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0076.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0086.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0089.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0120.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0096.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0108.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0053.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0003.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0008.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0014.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0068.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0070.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0072.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0080.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0017.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0032.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0078.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0091.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0122.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0026.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0059.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0114.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0081.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0113.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0117.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0057.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0097.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0121.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0007.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0024.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0052.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0067.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0071.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0090.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0101.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0105.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0013.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0015.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0043.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0062.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0054.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0036.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0038.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0088.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0025.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0085.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0102.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0001.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0022.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0044.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0066.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0079.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0127.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0107.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0039.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0045.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0049.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0069.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0126.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0018.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0063.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0109.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0034.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0092.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0111.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0128.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0016.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0037.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0051.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0087.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0005.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0084.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0112.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0006.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0048.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0083.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0094.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0099.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0125.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0004.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0020.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0023.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0035.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0058.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0064.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0075.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0082.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0118.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0115.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0119.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0028.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0040.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0055.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0065.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0123.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0104.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0002.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0098.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0103.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0129.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0000.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0009.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0010.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0012.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0030.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0031.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0042.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0060.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0027.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0050.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0074.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0116.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0046.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0061.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0077.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0110.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0011.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0033.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0093.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:31 provision: minted instance route r0124.go.tito.io (app "tito" acct "bac")
--- FAIL: TestPlan9B8RecordWriteCensus (0.42s)
    plan9_b8_census_test.go:371: record-write census rejected 1 site(s); update the explicit inventory only after reviewing the source:
          unlisted site internal/console/claim_loss_pages.go:Server.pageClaimLossesAt [key:LossRecordKey]: review it, classify it, and update its explicit inventory entry
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: refusing to mint route acme.go.tito.io (app "tito" acct "bac"): this box declares CARLOS_INBOUND=none (no-inbound) and cannot serve routed instance traffic
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: minted static route site.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: minted service route svc.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: minted instance route whole.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: minted instance route console.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: minted service route ha.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: minted instance route shared.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: converged acme.go.tito.io channel stable -> edge
2026/10/12 13:06:32 provision: converged acme.go.tito.io channel edge -> canary/rehearsal
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: skipping record apps/bac/tito/instances/acme.go.tito.io.json: empty channel
2026/10/12 13:06:32 provision: not removing acme.go.tito.io: its record exists but could not be read cleanly this pass
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: converged acme.go.tito.io environment "" -> "staging"
2026/10/12 13:06:32 provision: converged acme.go.tito.io environment "staging" -> ""
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: converged acme.go.tito.io steering "" -> "latency"
2026/10/12 13:06:32 provision: converged acme.go.tito.io steering "latency" -> ""
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: converged acme.go.tito.io hibernate false -> true
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: converged acme.go.tito.io hibernate true -> false (replication "daily" cleared: only a hibernating route's rung is read)
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: converged acme.go.tito.io status-named false -> true
2026/10/12 13:06:32 provision: converged acme.go.tito.io status-named true -> false
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: minted service route ha.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: converged ha.go.tito.io upstreams addr "" -> "", addrs "127.0.0.1:7001,127.0.0.1:7002" -> "127.0.0.1:7001,127.0.0.1:7003" (probe "{\"path\":\"/\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}" -> "{\"path\":\"/\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}")
2026/10/12 13:06:32 provision: converged ha.go.tito.io upstreams addr "" -> "", addrs "127.0.0.1:7001,127.0.0.1:7003" -> "127.0.0.1:7003,127.0.0.1:7001" (probe "{\"path\":\"/\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}" -> "{\"path\":\"/\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}")
2026/10/12 13:06:32 provision: converged ha.go.tito.io upstreams addr "" -> "", addrs "127.0.0.1:7003,127.0.0.1:7001" -> "127.0.0.1:7003,127.0.0.1:7001" (probe "{\"path\":\"/\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}" -> "{\"path\":\"/healthz\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}")
2026/10/12 13:06:32 provision: minted service route whole.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: converged whole.go.tito.io upstreams addr "127.0.0.1:9443" -> "", addrs "" -> "127.0.0.1:9443,127.0.0.1:9444" (probe "" -> "{\"path\":\"/\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}")
2026/10/12 13:06:32 provision: converged whole.go.tito.io upstreams addr "" -> "", addrs "127.0.0.1:9443,127.0.0.1:9444" -> "" (probe "{\"path\":\"/\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}" -> "")
2026/10/12 13:06:32 provision: whole.go.tito.io: record describes a different route SHAPE than the row (kind "instance"/"service" unit ""/"" socket "/run/carlos/whole.go.tito.io.sock"/"" db "/data/whole.go.tito.io.db"/"") — not converged in place; `carlos remove whole.go.tito.io` purges this route's capability grants (registry.Delete's contract, not a bug) before the next pass re-mints it, so treat that as a last resort, not a routine fix
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: minted service route whole.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: acme.go.tito.io exists but is not the route this record describes (app "other"/"tito") — leaving it alone
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: removed retired instance route acme.go.tito.io (record gone)
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: skipping accounts/bac/account.json: malformed account manifest: invalid character 'o' in literal null (expecting 'u')
2026/10/12 13:06:32 provision: removed retired instance route acme.go.tito.io (record gone)
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: manifest bac/tito: read failed: provision: manifest bac/tito: invalid character 'o' in literal null (expecting 'u')
2026/10/12 13:06:32 provision: skipping record apps/bac/tito/instances/acme.go.tito.io.json: disagrees with its own path (host="wrong-host.go.tito.io" account="bac" app="tito")
2026/10/12 13:06:32 provision: skipping record apps/bac/tito/instances/acme$evil.go.tito.io.json: host "acme$evil.go.tito.io" is not a hostname
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: skipping record apps/bac/tito/instances/acme$evil.go.tito.io.json: host "acme$evil.go.tito.io" is not a hostname
2026/10/12 13:06:32 provision: not removing acme$evil.go.tito.io: its record exists but could not be read cleanly this pass
2026/10/12 13:06:32 provision: skipping record apps/bac/tito/instances/bad-service.go.tito.io.json: invalid shape: kind service with no addr: the platform only proxies to a service, so the record must name the upstream it proxies to
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: skipping record apps/bac/tito/instances/bad-service.go.tito.io.json: invalid shape: kind service with no addr: the platform only proxies to a service, so the record must name the upstream it proxies to
2026/10/12 13:06:32 provision: not removing bad-service.go.tito.io: its record exists but could not be read cleanly this pass
2026/10/12 13:06:32 provision: skipping record apps/bac/tito/instances/no-channel.go.tito.io.json: empty channel
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: skipping record apps/bac/tito/instances/no-channel.go.tito.io.json: empty channel
2026/10/12 13:06:32 provision: not removing no-channel.go.tito.io: its record exists but could not be read cleanly this pass
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: skipping record apps/bac/tito/instances/acme.go.tito.io.json: malformed: provision: invalid maintenance projection or protected fields
2026/10/12 13:06:32 provision: not removing acme.go.tito.io: its record exists but could not be read cleanly this pass
2026/10/12 13:06:32 provision: removed retired instance route ghost.go.tito.io (record gone)
2026/10/12 13:06:32 provision: duplicate host dup.go.tito.io: keeping apps/bac/tito/instances/dup.go.tito.io.json, skipping apps/bad/tito/instances/dup.go.tito.io.json
2026/10/12 13:06:32 provision: minted instance route dup.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: bac/tito is in the trash — not minting instance routes for it
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: bac/tito is in the trash — not minting instance routes for it
2026/10/12 13:06:32 provision: removed retired instance route acme.go.tito.io (app in the trash)
2026/10/12 13:06:32 provision: minted static route site.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: bac/tito is in the trash — not minting instance routes for it
2026/10/12 13:06:32 provision: removed retired static route site.go.tito.io (app in the trash)
2026/10/12 13:06:32 provision: trash marker for bac/tito unreadable: release: malformed trash marker for bac/tito: invalid character 'n' looking for beginning of object key string — not minting for this app
2026/10/12 13:06:32 provision: bac/tito is in the trash — not minting instance routes for it
2026/10/12 13:06:32 provision: removed retired instance route acme.go.tito.io (app in the trash)
2026/10/12 13:06:32 provision: bac/tito is in the trash — not minting instance routes for it
2026/10/12 13:06:32 provision: trash marker for bac/tito unreadable: release: malformed trash marker for bac/tito: invalid character 'n' looking for beginning of object key string — not minting for this app
2026/10/12 13:06:32 provision: adopted existing route acme.go.tito.io (app "tito" acct "bac") — this record now owns its channel and environment
2026/10/12 13:06:32 provision: converged acme.go.tito.io channel stable -> edge
2026/10/12 13:06:32 provision: adopted existing route acme.go.tito.io (app "tito" acct "bac") — this record now owns its channel and environment
2026/10/12 13:06:32 provision: converged acme.go.tito.io channel canary/rehearsal -> edge
2026/10/12 13:06:32 provision: converged acme.go.tito.io environment "staging" -> ""
2026/10/12 13:06:32 provision: acme.go.tito.io exists but is not the route this record describes (app "other"/"tito") — leaving it alone
2026/10/12 13:06:32 provision: acme.go.tito.io exists but is not the route this record describes (backing "unit" (box-scoped, not expressible in a record)) — leaving it alone
2026/10/12 13:06:32 provision: acme.go.tito.io exists but is not the route this record describes (tunnel "pi-fleet" (box-scoped, not expressible in a record)) — leaving it alone
2026/10/12 13:06:32 provision: acme.go.tito.io exists but is not the route this record describes (caps "port:25" (box-scoped, not expressible in a record)) — leaving it alone
2026/10/12 13:06:32 provision: acme.go.tito.io exists but is not the route this record describes (platform (owned by the domains reconcile pass)) — leaving it alone
2026/10/12 13:06:32 provision: acme.go.tito.io exists but is not the route this record describes (canary (owned by the domains reconcile pass)) — leaving it alone
2026/10/12 13:06:32 provision: acme.go.tito.io exists but is not the route this record describes (db "/data/other.db"/"/data/acme.go.tito.io.db") — leaving it alone
2026/10/12 13:06:32 provision: adopted existing route acme.go.tito.io (app "tito" acct "bac") — this record now owns its channel and environment
2026/10/12 13:06:32 provision: removed retired instance route acme.go.tito.io (record gone)
2026/10/12 13:06:32 provision: dry-run: would adopt existing route adopt.go.tito.io (app "tito" acct "bac") — this record would own its channel and environment
2026/10/12 13:06:32 provision: dry-run: would converge adopt.go.tito.io channel stable -> edge
2026/10/12 13:06:32 provision: dry-run: would converge adopt.go.tito.io environment "" -> "staging"
2026/10/12 13:06:32 provision: dry-run: would converge owned.go.tito.io channel stable -> edge
2026/10/12 13:06:32 provision: dry-run: would converge owned.go.tito.io environment "" -> "staging"
2026/10/12 13:06:32 provision: dry-run: would converge owned.go.tito.io hibernate false -> true
2026/10/12 13:06:32 provision: dry-run: would converge owned.go.tito.io seed
2026/10/12 13:06:32 provision: dry-run: would mint instance route acme.go.tito.io (app "tito" acct "bac" channel "stable")
2026/10/12 13:06:32 provision: dry-run: would remove retired instance route gone.go.tito.io (record gone)
2026/10/12 13:06:32 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:32 provision: adopted existing route adopt.go.tito.io (app "tito" acct "bac") — this record now owns its channel and environment
2026/10/12 13:06:32 provision: converged adopt.go.tito.io channel stable -> edge
2026/10/12 13:06:32 provision: converged adopt.go.tito.io environment "" -> "staging"
2026/10/12 13:06:32 provision: converged owned.go.tito.io channel stable -> edge
2026/10/12 13:06:32 provision: converged owned.go.tito.io environment "" -> "staging"
2026/10/12 13:06:32 provision: converged owned.go.tito.io hibernate false -> true
2026/10/12 13:06:32 provision: removed retired instance route gone.go.tito.io (record gone)
2026/10/12 13:06:33 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: adopted existing route story.carlosframework.com (app "story" acct "bad") — this record now owns its channel and environment
2026/10/12 13:06:33 provision: story.carlosframework.com exists but is not the route this record describes (kind "instance"/"static") — leaving it alone
2026/10/12 13:06:33 provision: minted static route site.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted service route pubs.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted service route pub.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted instance route unit.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted instance route exec.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted service route loop.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted service route named.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted service route priv.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted static route site.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted static route pisite.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted static route site.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: removed retired static route site.go.tito.io (record gone)
2026/10/12 13:06:33 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted instance route acme-uk.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: converged acme.go.tito.io replicas "ie,uk" -> "uk"
2026/10/12 13:06:33 provision: converged acme.go.tito.io replicas "uk" -> ""
2026/10/12 13:06:33 provision: minted instance route acme-uk.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: converged acme-uk.go.tito.io replica mark true -> false
2026/10/12 13:06:33 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted instance route router.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: acme.go.tito.io: row is on the nested layout (db "/data/instances/acme.go.tito.io/acme.go.tito.io.db" socket "/run/carlos/instances/acme.go.tito.io/app.sock"); the record's shared paths are not drift
2026/10/12 13:06:33 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted instance route router.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted instance route router.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted instance route router.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: acme.go.tito.io: not minting this pass: the box's layout intent could not be read: store down
2026/10/12 13:06:33 provision: router.go.tito.io: not minting this pass: the box's layout intent could not be read: store down
2026/10/12 13:06:33 provision: adopted existing route acme.go.tito.io (app "tito" acct "bac") — this record now owns its channel and environment
2026/10/12 13:06:33 provision: converged acme.go.tito.io channel stable -> edge
2026/10/12 13:06:33 provision: minted instance route acme.go.tito.io held landing by mv_acme (app "tito" acct "bac")
2026/10/12 13:06:33 provision: not removing acme.go.tito.io (record gone): participant of a running operation
2026/10/12 13:06:33 provision: removed retired instance route acme.go.tito.io (record gone)
2026/10/12 13:06:33 provision: minted instance route acme.go.tito.io held landing by mv_acme (app "tito" acct "bac")
2026/10/12 13:06:33 provision: refusing to mint held route acme.go.tito.io (app "tito" acct "bac"): this box declares CARLOS_INBOUND=none (no-inbound) and cannot serve routed instance traffic
2026/10/12 13:06:33 provision: held acme.go.tito.io landing by mv_acme (was ready)
2026/10/12 13:06:33 provision: acme.go.tito.io: held restoring by rb_other, not by mv_acme; leaving it
2026/10/12 13:06:33 provision: acme.go.tito.io: the row is not this pass's (app "other" acct ""); not holding it for mv_acme
2026/10/12 13:06:33 provision: acme.go.tito.io exists but is not the route this record describes (app "other"/"tito") — leaving it alone
2026/10/12 13:06:33 provision: dry-run: would mint instance route acme.go.tito.io held landing by mv_acme (app "tito" acct "bac" channel "stable")
2026/10/12 13:06:33 provision: acme.go.tito.io: not held landing by mv_acme: the row's hold changed under this pass; the next pass judges it afresh
2026/10/12 13:06:33 provision: held acme.go.tito.io landing by mv_acme (was ready)
2026/10/12 13:06:33 provision: removed retired sidecar route gone.eleven.bac.sidecar.internal (record gone)
2026/10/12 13:06:33 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/12 13:06:33 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/12 13:06:33 provision: converged sidecar bot.eleven.bac.sidecar.internal args ["sidecar","run"] -> ["worker","run"]
2026/10/12 13:06:33 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/12 13:06:33 provision: converged sidecar bot.eleven.bac.sidecar.internal ceilings 300/3600 -> 60/120
2026/10/12 13:06:33 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/12 13:06:33 provision: converged sidecar bot.eleven.bac.sidecar.internal channel stable -> canary/eleven
2026/10/12 13:06:33 provision: converged sidecar bot.eleven.bac.sidecar.internal environment "other" -> "relay"
2026/10/12 13:06:33 provision: converged sidecar bot.eleven.bac.sidecar.internal hibernate true -> false
2026/10/12 13:06:33 provision: skipping sidecar record apps/bac/eleven/sidecars/bot.json: read failed: failStore: reads disabled under apps/bac/eleven/sidecars/
2026/10/12 13:06:33 provision: skipping sidecar record apps/bac/eleven/sidecars/bot.json: malformed: invalid character 'n' looking for beginning of object key string
2026/10/12 13:06:33 provision: not removing sidecar bot.eleven.bac.sidecar.internal: its record exists but could not be read cleanly this pass
2026/10/12 13:06:33 provision: skipping sidecar record apps/bac/eleven/sidecars/bot.json: empty channel
2026/10/12 13:06:33 provision: skipping sidecar record apps/bac/eleven/sidecars/held.json: empty channel
2026/10/12 13:06:33 provision: not removing sidecar held.eleven.bac.sidecar.internal: its record exists but could not be read cleanly this pass
2026/10/12 13:06:33 provision: removed retired sidecar route old.eleven.bac.sidecar.internal (record gone)
2026/10/12 13:06:33 provision: bac/eleven has sidecar records but no sidecars manifest — not minting
2026/10/12 13:06:33 provision: bac/eleven is in the trash — not minting sidecar routes for it
2026/10/12 13:06:33 provision: sidecar bot.eleven.bac.sidecar.internal is not placeable here: it asks for "no-inbound" placement and this box accepts inbound work (CARLOS_INBOUND="")
2026/10/12 13:06:33 provision: sidecar hub.relay.bac.sidecar.internal is not placeable here: it asks for "no-inbound" placement and this box accepts inbound work (CARLOS_INBOUND="")
2026/10/12 13:06:33 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/12 13:06:33 provision: minted sidecar route hub.relay.bac.sidecar.internal (app "relay" acct "bac" args ["relay"])
2026/10/12 13:06:33 provision: sidecar bot.eleven.bac.sidecar.internal is not placeable here: it asks for "no-inbound" placement and this box accepts inbound work (CARLOS_INBOUND="")
2026/10/12 13:06:33 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/12 13:06:33 provision: sidecar bot.eleven.bac.sidecar.internal is not placeable here: it asks for "no-inbound" placement and this box accepts inbound work (CARLOS_INBOUND="")
2026/10/12 13:06:33 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/12 13:06:33 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/12 13:06:33 provision: sidecar bot.eleven.bac.sidecar.internal is not placeable here: it asks for "no-inbound" placement and this box accepts inbound work (CARLOS_INBOUND="")
2026/10/12 13:06:33 provision: removed retired sidecar route bot.eleven.bac.sidecar.internal (record gone)
2026/10/12 13:06:33 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/12 13:06:33 provision: sidecar bot.eleven.bac.sidecar.internal is not placeable here: it asks for "no-inbound" placement and this box accepts inbound work (CARLOS_INBOUND="")
2026/10/12 13:06:33 provision: skipping sidecar record apps/bac/relay/sidecars/hub.json: read failed: failStore: reads disabled under apps/bac/relay/sidecars/
2026/10/12 13:06:33 provision: REFUSING to mint sidecar bot.eleven.bac.sidecar.internal: bot.eleven.bac.sidecar.internal-sidecar.env is staged on this box — the legacy file co-spawn and a native sidecar row are two mechanisms for one host; remove the file to migrate this host onto sidecar records
2026/10/12 13:06:33 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/12 13:06:33 provision: dry-run: would mint sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" channel "stable" args ["sidecar","run"])
2026/10/12 13:06:33 provision: dry-run: would remove retired sidecar route gone.eleven.bac.sidecar.internal (record gone)
2026/10/12 13:06:33 provision: bot.eleven.bac.sidecar.internal exists but is not a sidecar row this pass owns (kind "sidecar" provisioned false app "eleven" acct "bac") — leaving it alone
2026/10/12 13:06:33 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: converged acme.go.tito.io update settings "{\"updates\":\"manual\",\"channel\":\"stable\"}" -> ""
2026/10/12 13:06:33 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: manifest bac/tito: read failed: provision: manifest bac/tito: unexpected end of JSON input
2026/10/12 13:06:33 provision: manifest bac/tito: read failed: provision: manifest bac/tito: unexpected end of JSON input
2026/10/12 13:06:33 provision: acme.go.tito.io: not minted this pass: the app's update policy could not be read
2026/10/12 13:06:33 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/12 13:06:33 provision: manifest bac/tito: read failed: provision: manifest bac/tito: unexpected end of JSON input
2026/10/12 13:06:33 provision: acme.go.tito.io: not minted this pass: the app's update policy could not be read
2026/10/12 13:06:33 provision: minted instance route acme.go.tito.io held landing by mv_acme (app "tito" acct "bac")
FAIL
FAIL	github.com/carlosframework/platform/internal/provision	3.084s
ok  	github.com/carlosframework/platform/internal/publicrelease	0.006s
?   	github.com/carlosframework/platform/internal/regionname	[no test files]
ok  	github.com/carlosframework/platform/internal/registry	3.675s
ok  	github.com/carlosframework/platform/internal/relay	7.213s
ok  	github.com/carlosframework/platform/internal/release	17.600s
ok  	github.com/carlosframework/platform/internal/rollout	1.468s
ok  	github.com/carlosframework/platform/internal/rolloutwords	0.003s
ok  	github.com/carlosframework/platform/internal/rootaws	0.009s
ok  	github.com/carlosframework/platform/internal/s3compat	0.005s
ok  	github.com/carlosframework/platform/internal/safefile	0.004s
ok  	github.com/carlosframework/platform/internal/schedule	2.195s
ok  	github.com/carlosframework/platform/internal/sim/hostlife	2.269s
ok  	github.com/carlosframework/platform/internal/sim/litestreamfacts	4.388s
ok  	github.com/carlosframework/platform/internal/status	0.039s
ok  	github.com/carlosframework/platform/internal/steering	0.005s
ok  	github.com/carlosframework/platform/internal/stopevidence	0.127s
ok  	github.com/carlosframework/platform/internal/svcrole	0.002s
ok  	github.com/carlosframework/platform/internal/telemetry	0.003s
ok  	github.com/carlosframework/platform/internal/termfence	0.004s
ok  	github.com/carlosframework/platform/internal/tunnelclient	0.003s
ok  	github.com/carlosframework/platform/internal/unixdial	0.002s
ok  	github.com/carlosframework/platform/internal/updatewords	0.002s
ok  	github.com/carlosframework/platform/internal/usage	0.004s
ok  	github.com/carlosframework/platform/internal/version	0.001s
ok  	github.com/carlosframework/platform/internal/vet	1.546s
FAIL
make: *** [Makefile:18: test] Error 1

amadan: step gate failed: exit status 2

amadan: job failed at step gate — its output is above, not at the tail of this log
diff --git a/approved_moves_copy_test.go b/approved_moves_copy_test.go
index eef047380..e15cbdf49 100644
--- a/approved_moves_copy_test.go
+++ b/approved_moves_copy_test.go
@@ -41,6 +41,7 @@ const movesCopyRecordPath = "docs/superpowers/plans/2026-10-06-moves-phase1-copy
var movesCopyIDs = []string{
"MV-N391",
"HC5-N1", "HC5-N2", "HC5-N3",
+ "HC5-N4", "HC5-N5", "HC5-N6", "HC5-N7",
"M-C1", "M-C2", "M-C3", "M-C4", "M-C5", "M-C6", "M-C7", "M-C8", "M-C9",
"M-C10", "M-C11", "M-C12", "M-C13", "M-C14", "M-C15", "M-C16", "M-C17",
"M-C18", "M-C19", "M-C20", "M-C21", "M-C22", "MV-N1", "MV-N2", "MV-N3",
@@ -1152,6 +1153,8 @@ func TestMovesTextAppliedPlaceholders(t *testing.T) {
// copy" task appends the files it touched; TestMovesScanFilesCoverTheDiff
// fails when the list falls behind.
var movesScanFiles = []string{
+ // Claim-owned pages share the alert transport; keep their drafts in the copy census.
+ "internal/console/claim_loss_pages.go",
"internal/host/claimhandover_crossbox.go",
"internal/host/claimhandover_recovery.go",
"internal/host/claimretire_local.go",
diff --git a/cmd/console/main.go b/cmd/console/main.go
index f4b243819..07d52df85 100644
--- a/cmd/console/main.go
+++ b/cmd/console/main.go
@@ -1350,11 +1350,10 @@ func applyMovesSwitch(cfg *console.Config, getenv func(string) string) {
}
// armMovesPaging arms alerting's sinks from CARLOS_ALERT_WEBHOOK and
-// CARLOS_ALERT_SNS_TOPIC_ARN, the agent's variables, for the console's move
-// pass (moves Plan 5 Task 7, delta M14). Only a host-claim console with the
-// moves switch on runs that pass, so only it arms; it reports whether it did.
+// CARLOS_ALERT_SNS_TOPIC_ARN for claim recovery and its optional move pass.
+// Claim loss pages run even when moves are disabled.
func armMovesPaging(ctx context.Context, cfg console.Config, getenv func(string) string) bool {
- if !cfg.HostClaim || !cfg.MovesEnabled {
+ if !cfg.HostClaim {
return false
}
alerting.Arm(alerting.FromEnv(ctx, getenv, nil))
diff --git a/cmd/console/movesswitch_test.go b/cmd/console/movesswitch_test.go
index 6173a520a..48730c28d 100644
--- a/cmd/console/movesswitch_test.go
+++ b/cmd/console/movesswitch_test.go
@@ -103,17 +103,17 @@ func isIdent(e ast.Expr, name string) bool {
// The console pages from its move pass (moves Plan 5 Task 7, delta M14):
// armMovesPaging arms alerting's sinks from the agent's variables on a
-// host-claim console with the moves switch on, and nowhere else; serve()
+// host-claim console, with or without moves; serve()
// calls it with os.Getenv before ConfigureClaimRuntime, so the recovery
// pass never runs unarmed.
-func TestArmMovesPagingOnlyWithBothSwitches(t *testing.T) {
+func TestArmMovesPagingWithClaims(t *testing.T) {
env := func(k string) string {
if k == "CARLOS_ALERT_WEBHOOK" {
return "http://example.test/hook"
}
return ""
}
- for _, tc := range []struct{ claim, moves, want bool }{{false, false, false}, {true, false, false}, {false, true, false}, {true, true, true}} {
+ for _, tc := range []struct{ claim, moves, want bool }{{false, false, false}, {true, false, true}, {false, true, false}, {true, true, true}} {
cfg := console.Config{HostClaim: tc.claim, MovesEnabled: tc.moves}
if got := armMovesPaging(context.Background(), cfg, env); got != tc.want {
t.Errorf("claim %v moves %v: armed %v, want %v", tc.claim, tc.moves, got, tc.want)
diff --git a/internal/console/claim_authority_census_test.go b/internal/console/claim_authority_census_test.go
index ee731d684..18161f0ab 100644
--- a/internal/console/claim_authority_census_test.go
+++ b/internal/console/claim_authority_census_test.go
@@ -73,7 +73,11 @@ func claimAuthoritySites(t *testing.T) []string {
case *ast.Ident:
method = c.Name
}
- if !selected[method] {
+ // Loss paging reads signed control evidence and writes only its
+ // NonApp delivery journal. Inventory those sites explicitly.
+ lossPageSite := e.Name() == "claim_loss_pages.go" && (method == "List" || method == "ReadLossRecord" || method == "Exec" || method == "ClaimLossPageAttempt")
+ lossPageSchema := e.Name() == "store_gate.go" && fn.Name.Name == "OpenGated" && method == "Exec"
+ if !selected[method] && !lossPageSite && !lossPageSchema {
return true
}
var b bytes.Buffer
diff --git a/internal/console/claim_loss_pages.go b/internal/console/claim_loss_pages.go
new file mode 100644
index 000000000..270600621
--- /dev/null
+++ b/internal/console/claim_loss_pages.go
@@ -0,0 +1,96 @@
+package console
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "strings"
+ "time"
+
+ "github.com/carlosframework/platform/internal/alerting"
+ "github.com/carlosframework/platform/internal/hostclaim"
+ "github.com/carlosframework/platform/internal/hostops"
+)
+
+const claimLossPageInterval = time.Hour
+
+func (s *Server) pageClaimLosses(ctx context.Context) error {
+ return s.pageClaimLossesAt(ctx, time.Now())
+}
+
+// Scan records independently of claims: loss evidence outlives a holder or
+// operation, and may still need attention after its claim has gone.
+func (s *Server) pageClaimLossesAt(ctx context.Context, now time.Time) error {
+ if !s.cfg.HostClaim {
+ return nil
+ }
+ if s.cfg.Releases == nil || s.store == nil || len(s.cfg.ReleasePub) == 0 {
+ return hostops.ErrContract
+ }
+ keys, err := s.cfg.Releases.List(ctx, "control/host-loss/")
+ if err != nil {
+ return err
+ }
+ var errs []error
+ for _, key := range keys {
+ if err := ctx.Err(); err != nil {
+ return errors.Join(append(errs, err)...)
+ }
+ id := strings.TrimSuffix(strings.TrimPrefix(key, "control/host-loss/"), ".json")
+ exact, err := hostclaim.LossRecordKey(id)
+ if err != nil || exact != key {
+ errs = append(errs, hostclaim.ErrLossRecord)
+ continue
+ }
+ observed, err := hostclaim.ReadLossRecord(ctx, s.cfg.Releases, s.cfg.ReleasePub, id)
+ if errors.Is(err, hostclaim.ErrLossRecordAbsent) {
+ continue
+ }
+ if err != nil {
+ errs = append(errs, err)
+ continue
+ }
+ r := observed.Record
+ if r.AlertsPending {
+ subject, body := claimLossPendingPage(r.Host, r.ID)
+ errs = append(errs, s.pageClaimLoss(ctx, "pending", r, now, subject, body))
+ }
+ if r.Timers.Knowledge == hostclaim.TimersPartial || r.Timers.Knowledge == hostclaim.TimersUnknown {
+ subject, body := claimTimerLossPage(r.Host, r.ID)
+ errs = append(errs, s.pageClaimLoss(ctx, "timers", r, now, subject, body))
+ }
+ }
+ return errors.Join(errs...)
+}
+
+func (s *Server) pageClaimLoss(ctx context.Context, condition string, r hostclaim.LossRecord, now time.Time, subject, body string) error {
+ attempted, err := s.store.ClaimLossPageAttempt(condition+" "+r.ID, now)
+ if err != nil || !attempted {
+ return err
+ }
+ return alerting.Page(ctx, alerting.LevelWarning, subject, body)
+}
+
+// ClaimLossPageAttempt reserves the attempt before delivery, including failed
+// sends. The conditional write also serializes overlapping recovery passes.
+func (s *Store) ClaimLossPageAttempt(rule string, now time.Time) (bool, error) {
+ result, err := s.db.Exec(`INSERT INTO claim_loss_pages (rule, attempted_at) VALUES (?, ?)
+ON CONFLICT(rule) DO UPDATE SET attempted_at = excluded.attempted_at
+WHERE claim_loss_pages.attempted_at <= ?`, rule, now.UnixNano(), now.Add(-claimLossPageInterval).UnixNano())
+ if err != nil {
+ return false, err
+ }
+ n, err := result.RowsAffected()
+ return n == 1, err
+}
+
+// Draft operator copy. The controller collects these strings for approval.
+func claimLossPendingPage(host, id string) (subject, body string) {
+ return fmt.Sprintf("Loss recorded for %s", host),
+ fmt.Sprintf("A loss was recorded for host %s. Review loss record %s and check what needs recovery.", host, id)
+}
+
+func claimTimerLossPage(host, id string) (subject, body string) {
+ return fmt.Sprintf("Timers may be lost for %s", host),
+ fmt.Sprintf("Timers may have been lost with the box serving host %s. Check loss record %s. Check which timers are missing and re-arm them.", host, id)
+}
diff --git a/internal/console/claim_loss_pages_test.go b/internal/console/claim_loss_pages_test.go
new file mode 100644
index 000000000..bb6d42c38
--- /dev/null
+++ b/internal/console/claim_loss_pages_test.go
@@ -0,0 +1,290 @@
+package console
+
+import (
+ "crypto/ed25519"
+ "crypto/sha256"
+ "encoding/hex"
+ "encoding/json"
+ "errors"
+ "path/filepath"
+ "strings"
+ "sync"
+ "sync/atomic"
+ "testing"
+ "time"
+
+ "github.com/carlosframework/platform/internal/alerting"
+ "github.com/carlosframework/platform/internal/hostclaim"
+ "github.com/carlosframework/platform/internal/release"
+)
+
+func lossPageRecord(id string, pending bool, knowledge hostclaim.TimerKnowledge) hostclaim.LossRecord {
+ at := time.Date(2026, 10, 12, 12, 0, 0, 0, time.UTC)
+ r := hostclaim.LossRecord{V: 1, ID: id, State: "prepared", Actor: "operator", AcceptedAt: at,
+ Host: "host.example", HolderIncarnation: strings.Repeat("a", 64), HolderStamp: 1,
+ ExpectedClaimGen: 1, ExpectedClaimETag: "claim", OpID: "operation", OpKind: hostclaim.KindMoving,
+ OpPhase: "precommit", Box: "box-a", Enrollment: "enrollment", QuarantineRevision: 1, FencedAt: at,
+ CloudObservation: hostclaim.LossCloudObservation{ObservedAt: at, Account: "123", Region: "eu-west-1", InstanceID: "i-1", State: "stopped", RequestID: "request"},
+ OperatorAttestation: hostclaim.LossOperatorAttestation{MaintainComputeFence: true, ColdQuarantineStartOnly: true},
+ ConfirmedConsequences: hostclaim.LossConsequences{RecoveryIntent: json.RawMessage(`{"kind":"lost-target"}`)},
+ AlertsPending: pending, Timers: hostclaim.TimerLoss{Knowledge: knowledge}}
+ if knowledge != hostclaim.TimersUnknown {
+ r.Timers.Source = &hostclaim.TimerSource{Bucket: "bucket", Key: "timers", VersionID: "v1", SHA256: strings.Repeat("b", 64), CapturedAt: at}
+ }
+ return r
+}
+
+func lossPageWorld(t *testing.T) (*Server, *release.DirStore, release.Signer, *[]map[string]string) {
+ t.Helper()
+ raw := release.NewDirStore(t.TempDir())
+ _, key, err := ed25519.GenerateKey(nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ publishRestoreRoster(t, raw, key)
+ s := newEnabledRestoreConsole(t, raw, key, crConsole)
+ var pages []map[string]string
+ alerting.Arm(&alerting.Sinks{WebhookURL: "http://unused", Post: func(_ string, body []byte) error {
+ var page map[string]string
+ if err := json.Unmarshal(body, &page); err != nil {
+ return err
+ }
+ pages = append(pages, page)
+ return nil
+ }})
+ t.Cleanup(func() { alerting.Arm(nil) })
+ return s, raw, release.RootSigner(key), &pages
+}
+
+func putLossPageRecord(t *testing.T, raw release.Store, signer release.Signer, r hostclaim.LossRecord) {
+ t.Helper()
+ if _, err := hostclaim.CreateLossRecord(t.Context(), raw, signer, r); err != nil {
+ t.Fatal(err)
+ }
+}
+
+func TestClaimLossPagesIntervalAndRestart(t *testing.T) {
+ s, raw, signer, pages := lossPageWorld(t)
+ for _, id := range []string{"loss-one", "loss-two"} {
+ putLossPageRecord(t, raw, signer, lossPageRecord(id, true, hostclaim.TimersComplete))
+ }
+ now := time.Date(2026, 10, 12, 13, 0, 0, 123, time.UTC)
+ pass := func() {
+ t.Helper()
+ if err := s.pageClaimLossesAt(t.Context(), now); err != nil {
+ t.Fatal(err)
+ }
+ }
+ pass()
+ pass()
+ if len(*pages) != 2 {
+ t.Fatalf("first passes: %v", *pages)
+ }
+ // Reopen the actual SQLite file; no in-memory limiter survives this.
+ path := s.store.gate.path
+ if err := s.store.Close(); err != nil {
+ t.Fatal(err)
+ }
+ st, err := OpenGated(path)
+ if err != nil {
+ t.Fatal(err)
+ }
+ t.Cleanup(func() { st.Close() })
+ s = &Server{cfg: s.cfg, store: st}
+ now = now.Add(time.Hour - time.Nanosecond)
+ pass()
+ if len(*pages) != 2 {
+ t.Fatalf("restart lost rate limit: %v", *pages)
+ }
+ now = now.Add(time.Nanosecond)
+ pass()
+ if len(*pages) != 4 {
+ t.Fatalf("interval did not repeat each record: %v", *pages)
+ }
+ key, _ := hostclaim.LossRecordKey("loss-one")
+ if err := raw.Delete(t.Context(), key); err != nil {
+ t.Fatal(err)
+ }
+ now = now.Add(time.Hour)
+ pass()
+ if len(*pages) != 5 || !strings.Contains((*pages)[4]["body"], "loss-two") {
+ t.Fatalf("absent pending record paged: %v", *pages)
+ }
+}
+
+func TestClaimLossPagesClearedAndTimers(t *testing.T) {
+ for _, knowledge := range []hostclaim.TimerKnowledge{hostclaim.TimersComplete, hostclaim.TimersPartial, hostclaim.TimersUnknown} {
+ t.Run(string(knowledge), func(t *testing.T) {
+ s, raw, signer, pages := lossPageWorld(t)
+ r := lossPageRecord("loss-timer", false, knowledge)
+ putLossPageRecord(t, raw, signer, r)
+ now := r.AcceptedAt
+ if err := s.pageClaimLossesAt(t.Context(), now); err != nil {
+ t.Fatal(err)
+ }
+ want := 1
+ if knowledge == hostclaim.TimersComplete {
+ want = 0
+ }
+ if len(*pages) != want {
+ t.Fatalf("pages: %v", *pages)
+ }
+ if want != 0 && (!strings.Contains((*pages)[0]["body"], r.Host) || !strings.Contains((*pages)[0]["body"], "re-arm")) {
+ t.Fatalf("timer instructions: %v", *pages)
+ }
+ key, _ := hostclaim.LossRecordKey(r.ID)
+ if err := raw.Delete(t.Context(), key); err != nil {
+ t.Fatal(err)
+ }
+ if err := s.pageClaimLossesAt(t.Context(), now.Add(2*time.Hour)); err != nil {
+ t.Fatal(err)
+ }
+ if len(*pages) != want {
+ t.Fatal("cleared record paged")
+ }
+ })
+ }
+}
+
+func TestClaimLossPagesRecoveryWiringWithoutMovesOrClaims(t *testing.T) {
+ s, raw, signer, pages := lossPageWorld(t)
+ putLossPageRecord(t, raw, signer, lossPageRecord("orphan-loss", true, hostclaim.TimersComplete))
+ if s.cfg.MovesEnabled {
+ t.Fatal("fixture enabled moves")
+ }
+ if err := s.RunClaimRecoveryPass(t.Context()); err != nil {
+ t.Fatal(err)
+ }
+ if len(*pages) != 1 {
+ t.Fatalf("record without a claim was not paged: %v", *pages)
+ }
+}
+
+func TestClaimLossPagesFlagOff(t *testing.T) {
+ calls := 0
+ alerting.Arm(&alerting.Sinks{WebhookURL: "http://unused", Post: func(string, []byte) error { calls++; return nil }})
+ t.Cleanup(func() { alerting.Arm(nil) })
+ s := &Server{}
+ if err := s.pageClaimLossesAt(t.Context(), time.Now()); err != nil {
+ t.Fatal(err)
+ }
+ if err := s.RunClaimRecoveryPass(t.Context()); err != nil {
+ t.Fatal(err)
+ }
+ if calls != 0 {
+ t.Fatal("disabled runtime paged")
+ }
+ st, err := Open(filepath.Join(t.TempDir(), "off.db"))
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer st.Close()
+ var count int
+ if err := st.db.QueryRow(`SELECT count(*) FROM sqlite_master WHERE name = 'claim_loss_pages'`).Scan(&count); err != nil || count != 0 {
+ t.Fatalf("flag-off schema changed: %d %v", count, err)
+ }
+}
+
+func TestClaimLossPageFailedDeliveryIsRateLimited(t *testing.T) {
+ s, raw, signer, _ := lossPageWorld(t)
+ putLossPageRecord(t, raw, signer, lossPageRecord("failed-page", true, hostclaim.TimersComplete))
+ calls := 0
+ failure := errors.New("delivery failed")
+ alerting.Arm(&alerting.Sinks{WebhookURL: "http://unused", Post: func(string, []byte) error { calls++; return failure }})
+ now := time.Now()
+ if err := s.pageClaimLossesAt(t.Context(), now); !errors.Is(err, failure) {
+ t.Fatalf("delivery: %v", err)
+ }
+ if err := s.pageClaimLossesAt(t.Context(), now.Add(time.Minute)); err != nil {
+ t.Fatal(err)
+ }
+ if calls != 1 {
+ t.Fatalf("attempts: %d", calls)
+ }
+}
+
+func TestClaimLossPagesAutomaticReportingDoesNotAcknowledge(t *testing.T) {
+ s, raw, _, pages := lossPageWorld(t)
+ ref := hostclaim.FenceRef{Enrollment: strings.Repeat("a", 32), SHA256: strings.Repeat("b", 64), Key: "control/box-fences/" + strings.Repeat("a", 32) + ".json"}
+ sum := sha256.Sum256([]byte("operation\x00box-a\x00" + ref.Enrollment + "\x00" + ref.SHA256))
+ id := "auto-" + hex.EncodeToString(sum[:16])
+ loss := hostclaim.AutomaticLoss{ID: id, Box: "box-a", Enrollment: ref.Enrollment, Fence: ref,
+ ObservedAt: time.Now().UTC().Format(time.RFC3339Nano), Op: "operation", Kind: hostclaim.KindLanding,
+ Phase: hostclaim.PhaseProbing, Role: "target", Incarnation: strings.Repeat("c", 64), Stamp: 1, Timers: hostclaim.TimersUnknown}
+ _, err := hostclaim.Update(t.Context(), raw, hostclaim.Observed{Host: "host.example", Absent: true}, func(c *hostclaim.Claim) error {
+ c.LossRefs = []string{id}
+ c.EvidenceRefs = []hostclaim.AutomaticLoss{loss}
+ return nil
+ })
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := s.RunClaimRecoveryPass(t.Context()); err != nil {
+ t.Fatal(err)
+ }
+ if len(*pages) != 2 {
+ t.Fatalf("materialized record must page both conditions in the same pass: %v", *pages)
+ }
+ c, err := hostclaim.Read(t.Context(), raw, "host.example")
+ if err != nil || !c.Claim.EvidenceRefs[0].Reported {
+ t.Fatalf("reporting did not finish: %+v %v", c, err)
+ }
+ if err := s.pageClaimLossesAt(t.Context(), time.Now().Add(2*time.Hour)); err != nil {
+ t.Fatal(err)
+ }
+ if len(*pages) != 4 {
+ t.Fatal("claim reporting silenced pending operator alerts")
+ }
+}
+
+func TestClaimLossPageReservationSerializesPasses(t *testing.T) {
+ st := openGatedStore(t)
+ now := time.Now()
+ var wg sync.WaitGroup
+ var attempts atomic.Int64
+ for i := 0; i < 12; i++ {
+ wg.Add(1)
+ go func() {
+ defer wg.Done()
+ ok, err := st.ClaimLossPageAttempt("pending loss", now)
+ if err != nil {
+ t.Error(err)
+ }
+ if ok {
+ attempts.Add(1)
+ }
+ }()
+ }
+ wg.Wait()
+ if attempts.Load() != 1 {
+ t.Fatalf("reserved %d attempts", attempts.Load())
+ }
+}
+
+func TestClaimLossPagesRejectBadEvidenceAndContinue(t *testing.T) {
+ s, raw, signer, pages := lossPageWorld(t)
+ if _, err := raw.Put(t.Context(), "control/host-loss/bad.json", []byte(`{}`), ""); err != nil {
+ t.Fatal(err)
+ }
+ putLossPageRecord(t, raw, signer, lossPageRecord("valid", true, hostclaim.TimersComplete))
+ if err := s.pageClaimLossesAt(t.Context(), time.Now()); !errors.Is(err, hostclaim.ErrLossRecord) {
+ t.Fatalf("invalid signature accepted: %v", err)
+ }
+ if len(*pages) != 1 {
+ t.Fatalf("bad record blocked good record: %v", *pages)
+ }
+}
+
+func TestClaimLossPagesJournalFailureDoesNotSend(t *testing.T) {
+ s, raw, signer, pages := lossPageWorld(t)
+ putLossPageRecord(t, raw, signer, lossPageRecord("loss", true, hostclaim.TimersComplete))
+ if err := s.store.Close(); err != nil {
+ t.Fatal(err)
+ }
+ if err := s.pageClaimLossesAt(t.Context(), time.Now()); err == nil {
+ t.Fatal("journal error hidden")
+ }
+ if len(*pages) != 0 {
+ t.Fatal("sent without durable reservation")
+ }
+}
diff --git a/internal/console/claim_runtime.go b/internal/console/claim_runtime.go
index 186476dfd..b606fa928 100644
--- a/internal/console/claim_runtime.go
+++ b/internal/console/claim_runtime.go
@@ -127,6 +127,7 @@ func (s *Server) ConfigureClaimRuntime(ctx context.Context) error {
moveLogs = s.drainMoveLog
}
s.cfg.HostClaimRecovery = &hostops.Recovery{
+ LossPages: s.pageClaimLosses,
ReportAutomaticLosses: func(ctx context.Context, host string) error {
return a.Control.ReportAutomaticLosses(ctx, a.Signer, host)
},
diff --git a/internal/console/store_gate.go b/internal/console/store_gate.go
index 6a90ecb46..134c9b725 100644
--- a/internal/console/store_gate.go
+++ b/internal/console/store_gate.go
@@ -209,6 +209,7 @@ var claimSQLTables = map[string]claimSQLTable{
"memberships": {NonApp: "account membership, keyed by account and address; names no app"},
"invites": {NonApp: "account invites, keyed by account and address; names no app"},
"alert_seen": {NonApp: "operator alert memory, keyed by box, rule and address"},
+ "claim_loss_pages": {NonApp: "claim recovery page attempts per loss record and condition; no app state or lifecycle authority"},
"move_pages": {NonApp: "the move pass's last page attempt per finding (rule names the host and the move); no app column"},
"move_log_outbox": {NonApp: "a moves audit outbox that stores no app name: rows are keyed by host, move and entry and bound to an instance incarnation, verified against the host's claim and record at every write, written only by the moves recovery and log passes, and purged with the incarnation"},
"edge_proof_seen": {NonApp: "single-row edge proof marker"},
@@ -1789,6 +1790,12 @@ func OpenGated(path string) (*Store, error) {
if err := s.migrate(); err != nil {
return err
}
+ if _, err := s.db.Exec(`CREATE TABLE IF NOT EXISTS claim_loss_pages (
+ rule TEXT PRIMARY KEY,
+ attempted_at INTEGER NOT NULL
+);`); err != nil {
+ return err
+ }
return s.migrateClaimAppIdentity()
})
}
diff --git a/internal/console/testdata/claim-authority-sites.txt b/internal/console/testdata/claim-authority-sites.txt
index 9e1fbf855..d4c863cb4 100644
--- a/internal/console/testdata/claim-authority-sites.txt
+++ b/internal/console/testdata/claim-authority-sites.txt
@@ -125,3 +125,8 @@ store_gate.go|openGated|sql.OpenDB(claimConnector{g: g, trusted: true})
store_gate.go|openGated|sql.OpenDB(claimConnector{g: g})
store_gate.go|withClaimSQL|s.withPreparedClaimSQL(ctx, scopes, check, func(ctx context.Context, _ claimSQLScope) error { return ctx.Err() }, nil, false, fn)
view.go|openRegistry|registry.Open(path)
+claim_loss_pages.go|ClaimLossPageAttempt|s.db.Exec(`INSERT INTO claim_loss_pages (rule, attempted_at) VALUES (?, ?) ON CONFLICT(rule) DO UPDATE SET attempted_at = excluded.attempted_at WHERE claim_loss_pages.attempted_at <= ?`, rule, now.UnixNano(), now.Add(-claimLossPageInterval).UnixNano())
+claim_loss_pages.go|pageClaimLossesAt|hostclaim.ReadLossRecord(ctx, s.cfg.Releases, s.cfg.ReleasePub, id)
+claim_loss_pages.go|pageClaimLossesAt|s.cfg.Releases.List(ctx, "control/host-loss/")
+claim_loss_pages.go|pageClaimLoss|s.store.ClaimLossPageAttempt(condition+" "+r.ID, now)
+store_gate.go|OpenGated|s.db.Exec(`CREATE TABLE IF NOT EXISTS claim_loss_pages ( rule TEXT PRIMARY KEY, attempted_at INTEGER NOT NULL );`)
diff --git a/internal/hostops/recovery.go b/internal/hostops/recovery.go
index 32b6735ac..a92885a9b 100644
--- a/internal/hostops/recovery.go
+++ b/internal/hostops/recovery.go
@@ -38,6 +38,8 @@ func ListClaims(ctx context.Context, s release.Store) ([]string, error) {
}
type Recovery struct {
+ // LossPages scans retained loss records once per pass, after materialization.
+ LossPages func(context.Context) error
ReportAutomaticLosses func(context.Context, string) error
MoveFenceAuthority hostclaim.FenceAuthority
Moves func(context.Context, *MoveRecoveryGrant) error
@@ -121,6 +123,11 @@ func (r Recovery) Pass(ctx context.Context) error {
all = append(all, err)
}
}
+ if r.LossPages != nil {
+ if err := r.LossPages(ctx); err != nil {
+ all = append(all, err)
+ }
+ }
if r.MoveLogs != nil {
if err := runMoveLogDrain(ctx, r.MoveLogs); err != nil && !errors.Is(err, ErrPending) {
all = append(all, err)
diff --git a/testdata/claim-moves-copy.md b/testdata/claim-moves-copy.md
index 6038aa4cb..532c335e0 100644
--- a/testdata/claim-moves-copy.md
+++ b/testdata/claim-moves-copy.md
@@ -5,3 +5,12 @@
| HC5-N1 | internal/console/claim_retire_helpers.go | instance moves are not available on this console yet: they need the host-claim Moves protocol | approved |
| HC5-N2 | internal/console/apiinstancemove.go | invalid resolution request | approved |
| HC5-N3 | internal/console/apiinstancemove.go | could not read the instance | approved |
+
+## Claim loss pages (pending Paul's review)
+
+| id | file | text | status |
+|---|---|---|---|
+| HC5-N4 | internal/console/claim_loss_pages.go | Loss recorded for <host> | pending |
+| HC5-N5 | internal/console/claim_loss_pages.go | A loss was recorded for host <host>. Review loss record <id> and check what needs recovery. | pending |
+| HC5-N6 | internal/console/claim_loss_pages.go | Timers may be lost for <host> | pending |
+| HC5-N7 | internal/console/claim_loss_pages.go | Timers may have been lost with the box serving host <host>. Check loss record <id>. Check which timers are missing and re-arm them. | pending |