harness: Run 10 B2 large instance PASS: 296 s offline (seal 69 s, flip 85 s, first wake 133 s, all scale with replica history); legacy dir parked with the source; R10-1, R10-2
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
4 files changed,
+509
−6
harness/RUNLOG.md+77 −6harness/evidence/run10/b2-dennis-wake.log+34 −0harness/evidence/run10/b2-grow.log+40 −0harness/evidence/run10/b2-move.log+358 −0
CI failed — run details
CI log
=== gate ===
cd launchctx && go vet ./... && go test ./... -count=1
ok amadan.net/carlos/platform.git/launchctx 0.011s
? amadan.net/carlos/platform.git/launchctx/controlname [no test files]
? amadan.net/carlos/platform.git/launchctx/wireid [no test files]
go vet ./...
gofmt -l .: clean
go test ./... -count=1 -timeout 30m
ok github.com/carlosframework/platform 309.732s
? github.com/carlosframework/platform/cmd/changelog [no test files]
ok github.com/carlosframework/platform/cmd/console 2.740s
ok github.com/carlosframework/platform/cmd/moveprobe 92.233s
ok github.com/carlosframework/platform/cmd/zddprobe 0.708s
? github.com/carlosframework/platform/harness/apps/wakecheck [no test files]
ok github.com/carlosframework/platform/internal/activator 250.731s
ok github.com/carlosframework/platform/internal/agentsock 0.015s
ok github.com/carlosframework/platform/internal/alerting 0.003s
ok github.com/carlosframework/platform/internal/analytics 0.021s
ok github.com/carlosframework/platform/internal/appconfig 2.451s
ok github.com/carlosframework/platform/internal/apperr 0.075s
ok github.com/carlosframework/platform/internal/applog 0.165s
ok github.com/carlosframework/platform/internal/awsinv 0.003s
ok github.com/carlosframework/platform/internal/boxchoice 0.006s
ok github.com/carlosframework/platform/internal/boxid 0.828s
? github.com/carlosframework/platform/internal/boxid/boxidtest [no test files]
? github.com/carlosframework/platform/internal/boxid/internal/signtest [no test files]
ok github.com/carlosframework/platform/internal/boxvolumes 0.008s
ok github.com/carlosframework/platform/internal/caps 0.012s
ok github.com/carlosframework/platform/internal/certwarm 0.003s
ok github.com/carlosframework/platform/internal/cgroup 0.717s
ok github.com/carlosframework/platform/internal/changelog 0.317s
ok github.com/carlosframework/platform/internal/claimactivation 0.797s
ok github.com/carlosframework/platform/internal/clamd 0.056s
? github.com/carlosframework/platform/internal/cliflag [no test files]
ok github.com/carlosframework/platform/internal/console 408.803s
? github.com/carlosframework/platform/internal/controlname [no test files]
? github.com/carlosframework/platform/internal/controlsock [no test files]
ok github.com/carlosframework/platform/internal/dbdoor 5.783s
ok github.com/carlosframework/platform/internal/dbdoor/dbdoortest 0.009s
ok github.com/carlosframework/platform/internal/dbverify 0.006s
ok github.com/carlosframework/platform/internal/dnsrec 0.006s
ok github.com/carlosframework/platform/internal/docsite 0.006s
ok github.com/carlosframework/platform/internal/domains 2.372s
ok github.com/carlosframework/platform/internal/doorkey 0.008s
ok github.com/carlosframework/platform/internal/economics 0.005s
ok github.com/carlosframework/platform/internal/edge 44.776s
? github.com/carlosframework/platform/internal/edgeproof [no test files]
ok github.com/carlosframework/platform/internal/edgesfile 0.009s
ok github.com/carlosframework/platform/internal/fleetapi 0.147s
ok github.com/carlosframework/platform/internal/gate 1.656s
ok github.com/carlosframework/platform/internal/genselect 0.017s
ok github.com/carlosframework/platform/internal/geo 0.016s
ok github.com/carlosframework/platform/internal/geoip 0.038s
ok github.com/carlosframework/platform/internal/historyreadback 0.159s
ok github.com/carlosframework/platform/internal/host 180.242s
ok github.com/carlosframework/platform/internal/hostactors 6.303s
ok github.com/carlosframework/platform/internal/hostactors/awsmembership 0.005s
ok github.com/carlosframework/platform/internal/hostbox 2.678s
ok github.com/carlosframework/platform/internal/hostclaim 0.542s
ok github.com/carlosframework/platform/internal/hostenv 0.004s
ok github.com/carlosframework/platform/internal/hostevidence 0.005s
ok github.com/carlosframework/platform/internal/hostexec 4.929s
ok github.com/carlosframework/platform/internal/hostfence 0.031s
ok github.com/carlosframework/platform/internal/hostfiles 10.533s
ok github.com/carlosframework/platform/internal/hostfloor 0.015s
ok github.com/carlosframework/platform/internal/hostguard 1.029s
ok github.com/carlosframework/platform/internal/hostops 1.565s
? github.com/carlosframework/platform/internal/hostops/fencerelease [no test files]
ok github.com/carlosframework/platform/internal/hostops/internal/fencerelease 0.005s
ok github.com/carlosframework/platform/internal/hostquarantine 0.002s
ok github.com/carlosframework/platform/internal/hostrelocate 0.451s
ok github.com/carlosframework/platform/internal/hostresolve 0.152s
ok github.com/carlosframework/platform/internal/hostwrite 0.016s
ok github.com/carlosframework/platform/internal/infraview 0.002s
ok github.com/carlosframework/platform/internal/instancectl 5.248s
ok github.com/carlosframework/platform/internal/instanceexec 2.730s
ok github.com/carlosframework/platform/internal/instancefs 0.018s
ok github.com/carlosframework/platform/internal/instancekey 0.018s
ok github.com/carlosframework/platform/internal/instanceupdates 0.025s
ok github.com/carlosframework/platform/internal/layout 1.540s
ok github.com/carlosframework/platform/internal/lbhealth 0.002s
ok github.com/carlosframework/platform/internal/lease 9.836s
ok github.com/carlosframework/platform/internal/lease/leasetest 0.007s
ok github.com/carlosframework/platform/internal/ledger 0.012s
ok github.com/carlosframework/platform/internal/mail 0.165s
ok github.com/carlosframework/platform/internal/maintenance 0.075s
ok github.com/carlosframework/platform/internal/maintenanceexec 5.513s
ok github.com/carlosframework/platform/internal/maintenanceguard 0.067s
ok github.com/carlosframework/platform/internal/metrics 0.005s
ok github.com/carlosframework/platform/internal/moves 21.049s
ok github.com/carlosframework/platform/internal/moves/archive 0.777s
ok github.com/carlosframework/platform/internal/objstore 0.169s
ok github.com/carlosframework/platform/internal/ops 0.168s
ok github.com/carlosframework/platform/internal/park 0.198s
ok github.com/carlosframework/platform/internal/privatefile 0.003s
2026/10/20 22:13:50 provision: listing apps/bab/blog/instances/: list refused — its records are unseen this pass
2026/10/20 22:13:50 provision: listing apps/bab/: listdirs refused — its records are unseen this pass
2026/10/20 22:13:50 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:50 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:50 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:50 provision: listing apps/bac/tito/instances/: list refused — its records are unseen this pass
2026/10/20 22:13:50 provision: listing apps/bac/tito/instances/: list refused — its records are unseen this pass
2026/10/20 22:13:50 provision: skipping record apps/bac/tito/instances/one.example.com.json: unreadable: read refused
2026/10/20 22:13:50 provision: listing apps/bac/tito/instances/: list refused — its records are unseen this pass
2026/10/20 22:13:50 provision: an app's records could not be listed: they are skipped
2026/10/20 22:13:50 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/20 22:13:50 provision: remove retired instance route shop.example: provision: claim admission unavailable
2026/10/20 22:13:50 provision: remove retired instance route shop.example: provision: claim admission unavailable
2026/10/20 22:13:50 provision: remove retired sidecar route worker.sidecar.internal: provision: claim admission unavailable
2026/10/20 22:13:50 provision: remove retired sidecar route worker.sidecar.internal: provision: claim admission unavailable
2026/10/20 22:13:50 provision: claim install acme.go.tito.io: provision: claim admission unavailable
2026/10/20 22:13:50 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:50 provision: claim install acme.go.tito.io: registry: claim row ownership changed or unknown
2026/10/20 22:13:50 provision: claim install bot.eleven.bac.sidecar.internal: provision: claim admission unavailable
2026/10/20 22:13:50 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["run"])
2026/10/20 22:13:50 provision: claim install bot.eleven.bac.sidecar.internal: registry: claim row ownership changed or unknown
2026/10/20 22:13:50 provision: refusing to mint route acme.go.tito.io (app "tito" acct "bac"): this box declares CARLOS_INBOUND=none (no-inbound) and cannot serve routed instance traffic
2026/10/20 22:13:50 provision: acme.go.tito.io: not minting this pass: the box's layout intent could not be read: no sealed config
2026/10/20 22:13:50 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:50 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:50 provision: REFUSING to mint sidecar bot.eleven.bac.sidecar.internal: bot.eleven.bac.sidecar.internal-sidecar.env is staged on this box — the legacy file co-spawn and a native sidecar row are two mechanisms for one host; remove the file to migrate this host onto sidecar records
2026/10/20 22:13:50 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["run"])
2026/10/20 22:13:50 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["run"])
2026/10/20 22:13:50 provision: refusing to mint route acme.go.tito.io (app "tito" acct "bac"): this box declares CARLOS_INBOUND=none (no-inbound) and cannot serve routed instance traffic
2026/10/20 22:13:50 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:50 provision: REFUSING to mint sidecar bot.eleven.bac.sidecar.internal: bot.eleven.bac.sidecar.internal-sidecar.env is staged on this box — the legacy file co-spawn and a native sidecar row are two mechanisms for one host; remove the file to migrate this host onto sidecar records
2026/10/20 22:13:50 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["run"])
2026/10/20 22:13:50 provision: acme.go.tito.io exists but is not the route this record describes (app "other"/"tito") — leaving it alone
2026/10/20 22:13:50 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:50 provision: claim install acme.go.tito.io: registry: claim row ownership changed or unknown
2026/10/20 22:13:51 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: claim install acme.go.tito.io: registry: claim row ownership changed or unknown
2026/10/20 22:13:51 provision: bot.eleven.bac.sidecar.internal exists but is not a sidecar row this pass owns (kind "sidecar" provisioned false app "eleven" acct "bac") — leaving it alone
2026/10/20 22:13:51 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["run"])
2026/10/20 22:13:51 provision: claim install bot.eleven.bac.sidecar.internal: registry: claim row ownership changed or unknown
2026/10/20 22:13:51 provision: acme.go.tito.io exists but is not the route this record describes (app "other"/"tito") — leaving it alone
2026/10/20 22:13:51 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: adopted existing route acme.go.tito.io (app "tito" acct "bac") — this record now owns its channel and environment
2026/10/20 22:13:51 provision: bot.eleven.bac.sidecar.internal exists but is not a sidecar row this pass owns (kind "sidecar" provisioned false app "eleven" acct "bac") — leaving it alone
2026/10/20 22:13:51 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/20 22:13:51 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: skipping record apps/bac/tito/instances/acme.go.tito.io.json: malformed: provision: invalid maintenance projection or protected fields
2026/10/20 22:13:51 provision: skipping record apps/bac/tito/instances/new.go.tito.io.json: malformed: provision: invalid maintenance projection or protected fields
2026/10/20 22:13:51 provision: not removing acme.go.tito.io: its record exists but could not be read cleanly this pass
2026/10/20 22:13:51 provision: skipping record apps/bac/tito/instances/acme.go.tito.io.json: malformed: provision: invalid maintenance projection or protected fields
2026/10/20 22:13:51 provision: skipping record apps/bac/tito/instances/new.go.tito.io.json: malformed: provision: invalid maintenance projection or protected fields
2026/10/20 22:13:51 provision: minted instance route rj.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route rk.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route rc.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route re.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route rf.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route rh.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route ri.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route rl.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route ra.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route rb.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route rd.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route rg.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route ra.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route rb.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route rc.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route rd.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: skipping record apps/bac/tito/instances/rb.go.tito.io.json: read failed: read refused
2026/10/20 22:13:51 provision: skipping record apps/bac/tito/instances/rb.go.tito.io.json: read failed: read refused
2026/10/20 22:13:51 provision: minted instance route re.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route x.appd.example (app "appd" acct "bac")
2026/10/20 22:13:51 provision: minted instance route x.appe.example (app "appe" acct "bac")
2026/10/20 22:13:51 provision: minted instance route x.appj.example (app "appj" acct "bac")
2026/10/20 22:13:51 provision: minted instance route x.appa.example (app "appa" acct "bac")
2026/10/20 22:13:51 provision: minted instance route x.appf.example (app "appf" acct "bac")
2026/10/20 22:13:51 provision: minted instance route x.appg.example (app "appg" acct "bac")
2026/10/20 22:13:51 provision: minted instance route x.apph.example (app "apph" acct "bac")
2026/10/20 22:13:51 provision: minted instance route x.appi.example (app "appi" acct "bac")
2026/10/20 22:13:51 provision: minted instance route x.appb.example (app "appb" acct "bac")
2026/10/20 22:13:51 provision: minted instance route x.appc.example (app "appc" acct "bac")
2026/10/20 22:13:51 provision: bac/tito is in the trash — not minting instance routes for it
2026/10/20 22:13:51 provision: minted instance route r0035.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0082.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0092.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0104.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0108.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0031.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0075.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0078.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0101.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0106.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0114.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0009.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0019.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0071.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0074.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0087.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0010.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0024.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0053.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0083.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0100.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0109.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0111.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0041.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0085.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0033.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0036.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0039.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0062.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0045.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0057.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0072.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0073.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0079.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0091.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0004.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0028.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0043.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0098.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0006.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0016.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0067.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0070.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0096.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0129.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0040.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0064.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0077.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0044.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0060.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0094.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0112.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0021.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0061.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0093.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0097.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0115.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0118.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0128.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0005.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0014.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0088.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0120.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0022.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0029.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0034.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0046.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0002.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0027.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0080.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0003.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0050.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0055.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0026.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0001.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0017.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0020.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0047.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0056.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0068.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0084.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0011.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0018.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0023.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0113.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0126.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0127.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0000.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0025.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0102.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0107.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0012.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0013.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0032.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0081.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0058.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0090.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0116.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0122.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0063.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0099.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0042.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0103.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0015.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0121.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0123.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0059.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0066.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0038.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0051.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0052.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0086.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0089.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0048.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0065.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0125.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0037.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0049.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0095.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0105.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0117.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0124.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0007.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0008.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0069.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0076.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0110.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0119.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0030.go.tito.io (app "tito" acct "bac")
2026/10/20 22:13:51 provision: minted instance route r0054.go.tito.io (app "tito" acct "bac")
--- FAIL: TestPlan14RootReadCensus (4.57s)
plan14_root_reads_test.go:940: root read census: 142 discrepancies
changed root read internal/hostfence/writer.go:OpenRootFenceStore; review class, task, sites and digest
changed root source envelope harness
changed root source envelope harness/apps/wakecheck
changed root source envelope harness/boxes
changed root source envelope harness/boxes/shims
changed root source envelope harness/checks
changed root source envelope harness/console
changed root source envelope harness/infra
changed root source envelope harness/infra-copies/747f699/dev
changed root source envelope harness/infra-copies/a0b3f5a/dev
changed root source envelope harness/infra-copies/a0b3f5a/scripts
changed root source envelope harness/infra-copies/pr47
changed root source envelope harness/operator
changed root source envelope harness/release
changed root source envelope harness/scenarios
changed root source envelope harness/scenarios/cohort1
changed root source envelope harness/scenarios/cohort1/blocks
changed root source envelope harness/stand-ins
changed root source envelope internal/host
changed root source envelope internal/hostfence
changed root source envelope internal/publicrelease
changed root source envelope internal/s3compat
unclassified root read harness/apps/wakecheck/main.go:main file:Exec@333,file:Exec@804,file:Open@277,file:QueryRow@870
unclassified root read harness/boxes/boxes-up.sh:external file:external@0
unclassified root read harness/boxes/dmi-shim.sh:external file:external@0
unclassified root read harness/boxes/layout-probe.sh:external file:external@0
unclassified root read harness/boxes/leo-ext4-from.sh:external file:external@0
unclassified root read harness/boxes/leo-ext4.sh:external file:external@0
unclassified root read harness/boxes/leo-swap.sh:external file:external@0
unclassified root read harness/boxes/shims/systemd-zram-setup@.service:external file:external@0
unclassified root read harness/boxes/vendor-shims.sh:external file:external@0
unclassified root read harness/checks/check2.sh:external file:external@0
unclassified root read harness/checks/check5.sh:external file:external@0
unclassified root read harness/checks/check6.sh:external file:external@0
unclassified root read harness/checks/check6b.sh:external file:external@0
unclassified root read harness/checks/check7.sh:external file:external@0
unclassified root read harness/checks/check8.sh:external file:external@0
unclassified root read harness/checks/fix5.sh:external file:external@0
unclassified root read harness/checks/run6b.sh:external file:external@0
unclassified root read harness/checks/snap-base.sh:external file:external@0
unclassified root read harness/checks/snap-boot.sh:external file:external@0
unclassified root read harness/checks/ssmsmoke.sh:external file:external@0
unclassified root read harness/console/console-ext4.sh:external file:external@0
unclassified root read harness/console/console-setup.sh:external file:external@0
unclassified root read harness/console/signin.sh:external file:external@0
unclassified root read harness/console/system-update.sh:external file:external@0
unclassified root read harness/donor-user-data.sh:external file:external@0
unclassified root read harness/evidence.sh:external file:external@0
unclassified root read harness/host-setup.sh:external file:external@0
unclassified root read harness/host-user-data.sh:external file:external@0
unclassified root read harness/hrun.sh:external file:external@0
unclassified root read harness/infra-copies/747f699/dev/host-init.sh:external file:external@0
unclassified root read harness/infra-copies/a0b3f5a/dev/host-init.sh:external file:external@0
unclassified root read harness/infra-copies/a0b3f5a/scripts/bootstrap-box.sh:external file:external@0
unclassified root read harness/infra-copies/a0b3f5a/scripts/ship-app.sh:external file:external@0
unclassified root read harness/infra-copies/a0b3f5a/scripts/update-platform-binary.sh:external file:external@0
unclassified root read harness/infra-copies/pr47/probe.sh:external file:external@0
unclassified root read harness/infra-copies/pr47/remediation.sh:external file:external@0
unclassified root read harness/infra/infra-setup.sh:external file:external@0
unclassified root read harness/operator/operator-setup.sh:external file:external@0
unclassified root read harness/release/release-setup.sh:external file:external@0
unclassified root read harness/release/release-window.sh:external file:external@0
unclassified root read harness/release/step7-operator.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/50-publish-record.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/51-open-signer.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/51-roster-resume.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/51-roster.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/61-sign.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/61a-install-channel.box.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/62-guard-arm.box.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/63-run.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/63-sophie.box.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/7-console-report.box.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/7-evidence-key.box.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/7-run.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/7-runtime.box.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/74-run.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/7a-leo.box.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/7a-run.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/acceptance.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/b1.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/b2-c8.leo.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/b2-list.leo.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/b2.box.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/b4-after.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/b4-pair.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/b4-resume.sophie.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/b4.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/b5.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/g4-roll.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/g5.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/g9-check.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/p9.box.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/place-parent.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/pw-seed.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/r10-a1-normal.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/r10-console-upgrade.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/r10-grow.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/r10-move-precheck.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/r10-move-setup.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/r10-move.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/r10-prep.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/r10-report.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/r9-console-e1.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/r9-console.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/r9-devcheck.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/r9-move-precheck.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/r9-move-setup.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/r9-move.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/r9-normal.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/r9-rearm.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/r9-release.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/r9-setup.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/rearm-roll.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/rearm-strict.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/rearm-strict.leo.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/rearm-strict2.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/rearm.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/rearm2.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/rearm3.host.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/reports.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/run-leo.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/w-env.box.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/w-identity.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/w-open-signer.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/w-sums.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/w81.box.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/w81b.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/w82-f4.box.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/w82-f5.box.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/w82.box.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/w82b.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/w84.box.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/blocks/w9.box.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/box-helpers.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/signer-helpers.sh:external file:external@0
unclassified root read harness/scenarios/cohort1/ssm.sh:external file:external@0
unclassified root read harness/scenarios/sync.sh:external file:external@0
unclassified root read harness/snapshot.sh:external file:external@0
unclassified root read harness/stand-ins/certs.sh:external file:external@0
unclassified root read internal/s3compat/harness.go:readHarnessFile file:ReadFile@189,file:Stat@55
changed reclassified declaration internal/hostfence/writer.go:OpenRootFenceStore
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: refusing to mint route acme.go.tito.io (app "tito" acct "bac"): this box declares CARLOS_INBOUND=none (no-inbound) and cannot serve routed instance traffic
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: minted instance route console.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: minted service route ha.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: minted static route site.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: minted service route svc.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: minted instance route whole.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: minted instance route shared.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: converged acme.go.tito.io channel stable -> edge
2026/10/20 22:14:22 provision: converged acme.go.tito.io channel edge -> canary/rehearsal
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: skipping record apps/bac/tito/instances/acme.go.tito.io.json: empty channel
2026/10/20 22:14:22 provision: not removing acme.go.tito.io: its record exists but could not be read cleanly this pass
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: converged acme.go.tito.io environment "" -> "staging"
2026/10/20 22:14:22 provision: converged acme.go.tito.io environment "staging" -> ""
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: converged acme.go.tito.io steering "" -> "latency"
2026/10/20 22:14:22 provision: converged acme.go.tito.io steering "latency" -> ""
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: converged acme.go.tito.io hibernate false -> true
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: converged acme.go.tito.io hibernate true -> false (replication "daily" cleared: only a hibernating route's rung is read)
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: converged acme.go.tito.io status-named false -> true
2026/10/20 22:14:22 provision: converged acme.go.tito.io status-named true -> false
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: minted service route ha.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: converged ha.go.tito.io upstreams addr "" -> "", addrs "127.0.0.1:7001,127.0.0.1:7002" -> "127.0.0.1:7001,127.0.0.1:7003" (probe "{\"path\":\"/\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}" -> "{\"path\":\"/\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}")
2026/10/20 22:14:22 provision: converged ha.go.tito.io upstreams addr "" -> "", addrs "127.0.0.1:7001,127.0.0.1:7003" -> "127.0.0.1:7003,127.0.0.1:7001" (probe "{\"path\":\"/\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}" -> "{\"path\":\"/\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}")
2026/10/20 22:14:22 provision: converged ha.go.tito.io upstreams addr "" -> "", addrs "127.0.0.1:7003,127.0.0.1:7001" -> "127.0.0.1:7003,127.0.0.1:7001" (probe "{\"path\":\"/\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}" -> "{\"path\":\"/healthz\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}")
2026/10/20 22:14:22 provision: minted service route whole.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: converged whole.go.tito.io upstreams addr "127.0.0.1:9443" -> "", addrs "" -> "127.0.0.1:9443,127.0.0.1:9444" (probe "" -> "{\"path\":\"/\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}")
2026/10/20 22:14:22 provision: converged whole.go.tito.io upstreams addr "" -> "", addrs "127.0.0.1:9443,127.0.0.1:9444" -> "" (probe "{\"path\":\"/\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}" -> "")
2026/10/20 22:14:22 provision: whole.go.tito.io: record describes a different route SHAPE than the row (kind "instance"/"service" unit ""/"" socket "/run/carlos/whole.go.tito.io.sock"/"" db "/data/whole.go.tito.io.db"/"") — not converged in place; `carlos remove whole.go.tito.io` purges this route's capability grants (registry.Delete's contract, not a bug) before the next pass re-mints it, so treat that as a last resort, not a routine fix
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: minted service route whole.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: acme.go.tito.io exists but is not the route this record describes (app "other"/"tito") — leaving it alone
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: removed retired instance route acme.go.tito.io (record gone)
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: skipping accounts/bac/account.json: malformed account manifest: invalid character 'o' in literal null (expecting 'u')
2026/10/20 22:14:22 provision: removed retired instance route acme.go.tito.io (record gone)
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: manifest bac/tito: read failed: provision: manifest bac/tito: invalid character 'o' in literal null (expecting 'u')
2026/10/20 22:14:22 provision: skipping record apps/bac/tito/instances/acme.go.tito.io.json: disagrees with its own path (host="wrong-host.go.tito.io" account="bac" app="tito")
2026/10/20 22:14:22 provision: skipping record apps/bac/tito/instances/acme$evil.go.tito.io.json: host "acme$evil.go.tito.io" is not a hostname
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: skipping record apps/bac/tito/instances/acme$evil.go.tito.io.json: host "acme$evil.go.tito.io" is not a hostname
2026/10/20 22:14:22 provision: not removing acme$evil.go.tito.io: its record exists but could not be read cleanly this pass
2026/10/20 22:14:22 provision: skipping record apps/bac/tito/instances/bad-service.go.tito.io.json: invalid shape: kind service with no addr: the platform only proxies to a service, so the record must name the upstream it proxies to
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: skipping record apps/bac/tito/instances/bad-service.go.tito.io.json: invalid shape: kind service with no addr: the platform only proxies to a service, so the record must name the upstream it proxies to
2026/10/20 22:14:22 provision: not removing bad-service.go.tito.io: its record exists but could not be read cleanly this pass
2026/10/20 22:14:22 provision: skipping record apps/bac/tito/instances/no-channel.go.tito.io.json: empty channel
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: skipping record apps/bac/tito/instances/no-channel.go.tito.io.json: empty channel
2026/10/20 22:14:22 provision: not removing no-channel.go.tito.io: its record exists but could not be read cleanly this pass
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: skipping record apps/bac/tito/instances/acme.go.tito.io.json: malformed: provision: invalid maintenance projection or protected fields
2026/10/20 22:14:22 provision: not removing acme.go.tito.io: its record exists but could not be read cleanly this pass
2026/10/20 22:14:22 provision: removed retired instance route ghost.go.tito.io (record gone)
2026/10/20 22:14:22 provision: duplicate host dup.go.tito.io: keeping apps/bac/tito/instances/dup.go.tito.io.json, skipping apps/bad/tito/instances/dup.go.tito.io.json
2026/10/20 22:14:22 provision: minted instance route dup.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: bac/tito is in the trash — not minting instance routes for it
2026/10/20 22:14:22 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: bac/tito is in the trash — not minting instance routes for it
2026/10/20 22:14:22 provision: removed retired instance route acme.go.tito.io (app in the trash)
2026/10/20 22:14:22 provision: minted static route site.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:22 provision: bac/tito is in the trash — not minting instance routes for it
2026/10/20 22:14:22 provision: removed retired static route site.go.tito.io (app in the trash)
2026/10/20 22:14:22 provision: trash marker for bac/tito unreadable: release: malformed trash marker for bac/tito: invalid character 'n' looking for beginning of object key string — not minting for this app
2026/10/20 22:14:22 provision: bac/tito is in the trash — not minting instance routes for it
2026/10/20 22:14:22 provision: removed retired instance route acme.go.tito.io (app in the trash)
2026/10/20 22:14:22 provision: bac/tito is in the trash — not minting instance routes for it
2026/10/20 22:14:22 provision: trash marker for bac/tito unreadable: release: malformed trash marker for bac/tito: invalid character 'n' looking for beginning of object key string — not minting for this app
2026/10/20 22:14:22 provision: adopted existing route acme.go.tito.io (app "tito" acct "bac") — this record now owns its channel and environment
2026/10/20 22:14:22 provision: converged acme.go.tito.io channel stable -> edge
2026/10/20 22:14:22 provision: adopted existing route acme.go.tito.io (app "tito" acct "bac") — this record now owns its channel and environment
2026/10/20 22:14:22 provision: converged acme.go.tito.io channel canary/rehearsal -> edge
2026/10/20 22:14:22 provision: converged acme.go.tito.io environment "staging" -> ""
2026/10/20 22:14:22 provision: acme.go.tito.io exists but is not the route this record describes (app "other"/"tito") — leaving it alone
2026/10/20 22:14:22 provision: acme.go.tito.io exists but is not the route this record describes (backing "unit" (box-scoped, not expressible in a record)) — leaving it alone
2026/10/20 22:14:22 provision: acme.go.tito.io exists but is not the route this record describes (tunnel "pi-fleet" (box-scoped, not expressible in a record)) — leaving it alone
2026/10/20 22:14:22 provision: acme.go.tito.io exists but is not the route this record describes (caps "port:25" (box-scoped, not expressible in a record)) — leaving it alone
2026/10/20 22:14:22 provision: acme.go.tito.io exists but is not the route this record describes (platform (owned by the domains reconcile pass)) — leaving it alone
2026/10/20 22:14:23 provision: acme.go.tito.io exists but is not the route this record describes (canary (owned by the domains reconcile pass)) — leaving it alone
2026/10/20 22:14:23 provision: acme.go.tito.io exists but is not the route this record describes (db "/data/other.db"/"/data/acme.go.tito.io.db") — leaving it alone
2026/10/20 22:14:23 provision: adopted existing route acme.go.tito.io (app "tito" acct "bac") — this record now owns its channel and environment
2026/10/20 22:14:23 provision: removed retired instance route acme.go.tito.io (record gone)
2026/10/20 22:14:23 provision: dry-run: would mint instance route acme.go.tito.io (app "tito" acct "bac" channel "stable")
2026/10/20 22:14:23 provision: dry-run: would adopt existing route adopt.go.tito.io (app "tito" acct "bac") — this record would own its channel and environment
2026/10/20 22:14:23 provision: dry-run: would converge adopt.go.tito.io channel stable -> edge
2026/10/20 22:14:23 provision: dry-run: would converge adopt.go.tito.io environment "" -> "staging"
2026/10/20 22:14:23 provision: dry-run: would converge owned.go.tito.io channel stable -> edge
2026/10/20 22:14:23 provision: dry-run: would converge owned.go.tito.io environment "" -> "staging"
2026/10/20 22:14:23 provision: dry-run: would converge owned.go.tito.io hibernate false -> true
2026/10/20 22:14:23 provision: dry-run: would converge owned.go.tito.io seed
2026/10/20 22:14:23 provision: dry-run: would remove retired instance route gone.go.tito.io (record gone)
2026/10/20 22:14:23 provision: converged owned.go.tito.io channel stable -> edge
2026/10/20 22:14:23 provision: converged owned.go.tito.io environment "" -> "staging"
2026/10/20 22:14:23 provision: converged owned.go.tito.io hibernate false -> true
2026/10/20 22:14:23 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: adopted existing route adopt.go.tito.io (app "tito" acct "bac") — this record now owns its channel and environment
2026/10/20 22:14:23 provision: converged adopt.go.tito.io channel stable -> edge
2026/10/20 22:14:23 provision: converged adopt.go.tito.io environment "" -> "staging"
2026/10/20 22:14:23 provision: removed retired instance route gone.go.tito.io (record gone)
2026/10/20 22:14:23 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: adopted existing route story.carlosframework.com (app "story" acct "bad") — this record now owns its channel and environment
2026/10/20 22:14:23 provision: story.carlosframework.com exists but is not the route this record describes (kind "instance"/"static") — leaving it alone
2026/10/20 22:14:23 provision: minted static route site.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: minted service route pub.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: minted service route pubs.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: minted instance route exec.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: minted service route loop.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: minted service route named.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: minted service route priv.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: minted instance route unit.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: minted static route site.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: minted static route pisite.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: minted static route site.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: removed retired static route site.go.tito.io (record gone)
2026/10/20 22:14:23 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: minted instance route acme-uk.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: converged acme.go.tito.io replicas "ie,uk" -> "uk"
2026/10/20 22:14:23 provision: converged acme.go.tito.io replicas "uk" -> ""
2026/10/20 22:14:23 provision: minted instance route acme-uk.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: converged acme-uk.go.tito.io replica mark true -> false
2026/10/20 22:14:23 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: minted instance route router.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: acme.go.tito.io: row is on the nested layout (db "/data/instances/acme.go.tito.io/acme.go.tito.io.db" socket "/run/carlos/instances/acme.go.tito.io/app.sock"); the record's shared paths are not drift
2026/10/20 22:14:23 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: minted instance route router.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: minted instance route router.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: minted instance route router.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:23 provision: acme.go.tito.io: not minting this pass: the box's layout intent could not be read: store down
2026/10/20 22:14:23 provision: router.go.tito.io: not minting this pass: the box's layout intent could not be read: store down
2026/10/20 22:14:23 provision: adopted existing route acme.go.tito.io (app "tito" acct "bac") — this record now owns its channel and environment
2026/10/20 22:14:23 provision: converged acme.go.tito.io channel stable -> edge
2026/10/20 22:14:23 provision: minted instance route acme.go.tito.io held landing by mv_acme (app "tito" acct "bac")
2026/10/20 22:14:23 provision: not removing acme.go.tito.io (record gone): participant of a running operation
2026/10/20 22:14:23 provision: removed retired instance route acme.go.tito.io (record gone)
2026/10/20 22:14:23 provision: minted instance route acme.go.tito.io held landing by mv_acme (app "tito" acct "bac")
2026/10/20 22:14:23 provision: refusing to mint held route acme.go.tito.io (app "tito" acct "bac"): this box declares CARLOS_INBOUND=none (no-inbound) and cannot serve routed instance traffic
2026/10/20 22:14:23 provision: held acme.go.tito.io landing by mv_acme (was ready)
2026/10/20 22:14:23 provision: acme.go.tito.io: held restoring by rb_other, not by mv_acme; leaving it
2026/10/20 22:14:23 provision: acme.go.tito.io: the row is not this pass's (app "other" acct ""); not holding it for mv_acme
2026/10/20 22:14:23 provision: acme.go.tito.io exists but is not the route this record describes (app "other"/"tito") — leaving it alone
2026/10/20 22:14:23 provision: dry-run: would mint instance route acme.go.tito.io held landing by mv_acme (app "tito" acct "bac" channel "stable")
2026/10/20 22:14:23 provision: acme.go.tito.io: not held landing by mv_acme: the row's hold changed under this pass; the next pass judges it afresh
2026/10/20 22:14:23 provision: held acme.go.tito.io landing by mv_acme (was ready)
2026/10/20 22:14:23 provision: minted instance route acme.example.com (app "shop" acct "bac")
2026/10/20 22:14:23 provision: minted instance route plain.example.com (app "shop" acct "bac")
2026/10/20 22:14:23 provision: skipping record apps/bac/shop/instances/garbled.shop.example.com.json: malformed: provision: invalid maintenance projection or protected fields
2026/10/20 22:14:23 provision: skipping record apps/bac/shop/instances/liar.shop.example.com.json: disagrees with its own path (host="other.shop.example.com" account="bac" app="shop")
2026/10/20 22:14:23 provision: skipping record apps/bac/shop/instances/garbled.shop.example.com.json: malformed: provision: invalid maintenance projection or protected fields
2026/10/20 22:14:23 provision: skipping record apps/bac/shop/instances/liar.shop.example.com.json: disagrees with its own path (host="other.shop.example.com" account="bac" app="shop")
2026/10/20 22:14:23 provision: skipping record apps/bac/shop/instances/garbled.shop.example.com.json: malformed: provision: invalid maintenance projection or protected fields
2026/10/20 22:14:23 provision: skipping record apps/bac/shop/instances/liar.shop.example.com.json: disagrees with its own path (host="other.shop.example.com" account="bac" app="shop")
2026/10/20 22:14:23 provision: skipping record apps/bac/shop/instances/garbled.shop.example.com.json: malformed: provision: invalid maintenance projection or protected fields
2026/10/20 22:14:23 provision: listing apps/bac/shop/instances/: list refused — its records are unseen this pass
2026/10/20 22:14:24 provision: a.shop.example.com: swapped for import sd_64pas4h5iquycsshflnk5oj26i (the original route is journaled)
2026/10/20 22:14:24 provision: a.shop.example.com: import sd_64pas4h5iquycsshflnk5oj26i committed: its gate is cleared and its journal row dropped
2026/10/20 22:14:24 provision: a.shop.example.com: row is on the nested layout (db "/data/instances/a.shop.example.com/a.shop.example.com.db" socket "/run/carlos/instances/a.shop.example.com/app.sock"); the record's shared paths are not drift
2026/10/20 22:14:24 provision: a.shop.example.com: import sd_vt3lagcwellzwapbd4pdlvutn4 rolled back to its original route
2026/10/20 22:14:24 provision: a.shop.example.com: import sd_vt3lagcwellzwapbd4pdlvutn4: the host's replica is left alone: the record apps/bac/shop/instances/a.shop.example.com.json holds the host
2026/10/20 22:14:24 provision: a.shop.example.com: import sd_vt3lagcwellzwapbd4pdlvutn4 is cleaned up and its journal row retired
2026/10/20 22:14:24 provision: a.shop.example.com: import sd_l5flxj44c3b5lpaujzay7rgcda rolled back to its original route
2026/10/20 22:14:24 provision: a.shop.example.com: import sd_l5flxj44c3b5lpaujzay7rgcda: the host's replica is left alone: the record apps/bac/shop/instances/a.shop.example.com.json holds the host
2026/10/20 22:14:24 provision: a.shop.example.com: import sd_l5flxj44c3b5lpaujzay7rgcda is cleaned up and its journal row retired
2026/10/20 22:14:24 provision: dry-run: would redo a.shop.example.com's swap record CAS for import sd_e3vurn7k3ixukysqq6lv3t3bim
2026/10/20 22:14:24 provision: dry-run: would clear a.shop.example.com's import gate
2026/10/20 22:14:24 provision: dry-run: would converge a.shop.example.com seed source
2026/10/20 22:14:24 provision: a.shop.example.com: row is on the nested layout (db "/data/instances/a.shop.example.com/a.shop.example.com.db" socket "/run/carlos/instances/a.shop.example.com/app.sock"); the record's shared paths are not drift
2026/10/20 22:14:24 provision: a.shop.example.com: the import's record apps/bac/shop/instances/a.shop.example.com.json is not one this pass can use (connection reset)
2026/10/20 22:14:24 provision: a.shop.example.com: the import's record apps/bac/shop/instances/a.shop.example.com.json is not one this pass can use (provision: invalid maintenance projection or protected fields)
2026/10/20 22:14:24 provision: a.shop.example.com: the import's record apps/bac/shop/instances/a.shop.example.com.json is not one this pass can use (the record under apps/bac/shop/instances/a.shop.example.com.json is apps/bac/shop/instances/b.shop.example.com.json's)
2026/10/20 22:14:24 provision: a.shop.example.com: import sd_xkf4pcmzghhtnp7hrshfchh6vm cannot roll back on this box: the activator is not wired; it stays journaled and gated
2026/10/20 22:14:24 provision: a.shop.example.com: import sd_6pycbbshfp6hsetlkurhczw4rm cannot roll back on this box: the activator is not wired; it stays journaled and gated
2026/10/20 22:14:24 provision: a.shop.example.com: import sd_kcwybvij4qqpt7dkisydnadhmq is journaled, but this box cannot act on imports; its row is left alone
2026/10/20 22:14:24 provision: a.shop.example.com: import sd_w2xccv5etc5efrycuue5ptgqtq is journaled, but this box cannot act on imports; its row is left alone
2026/10/20 22:14:24 provision: a.shop.example.com: import sd_crjyv2vhfbrlwwqzkd2b44v7ci is journaled, but this box cannot act on imports; its row is left alone
2026/10/20 22:14:24 provision: the import journal could not be read: registry: import journal a.shop.example.com: original row: unexpected end of JSON input (no import is acted on and no route is removed this pass)
2026/10/20 22:14:24 provision: b.shop.example.com: import sd_3ocslrtxs7dvsdaefh2h3d4ehe left alone this pass: the import journal could not be read
2026/10/20 22:14:24 provision: c.shop.example.com: import sd_kd56wwr7gkeyiqe67ittmp543q left alone this pass: the import journal could not be read
2026/10/20 22:14:24 provision: a.shop.example.com: import sd_36765f3emmoou6gt3oc2fz67j4 committed: its gate is cleared and its journal row dropped
2026/10/20 22:14:24 provision: a.shop.example.com: not converging import sd_vvqkkbstjatekgoqsttjfhejwe: it is swapped but not committed, and the row on this box has no import gate
2026/10/20 22:14:24 provision: a.shop.example.com: not converging import sd_w4oxqygdtvz5d5ulo44chq7dbq: it is swapped but not committed, and the row on this box has no import gate
2026/10/20 22:14:24 provision: a.shop.example.com: not converging import sd_egg4con2sltvv7qeenordnvhbm: it is placed but not committed, and the row on this box has no import gate
2026/10/20 22:14:24 provision: a.shop.example.com: not converging import sd_fchvyij4bf3552hp4e6kmptkze: it is placed but not committed, and the row on this box has no import gate
2026/10/20 22:14:24 provision: a.shop.example.com: row is on the nested layout (db "/data/instances/a.shop.example.com/a.shop.example.com.db" socket "/run/carlos/instances/a.shop.example.com/app.sock"); the record's shared paths are not drift
2026/10/20 22:14:24 provision: converge seed source on a.shop.example.com: constraint failed: seed_source is read-only here (1811)
2026/10/20 22:14:24 provision: a.shop.example.com: row is on the nested layout (db "/data/instances/a.shop.example.com/a.shop.example.com.db" socket "/run/carlos/instances/a.shop.example.com/app.sock"); the record's shared paths are not drift
2026/10/20 22:14:24 provision: a.shop.example.com: import sd_4yukcat57o4rxpl7wlz6pwb2sm committed: its gate is cleared and its journal row dropped
2026/10/20 22:14:24 provision: a.shop.example.com: row is on the nested layout (db "/data/instances/a.shop.example.com/a.shop.example.com.db" socket "/run/carlos/instances/a.shop.example.com/app.sock"); the record's shared paths are not drift
2026/10/20 22:14:24 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:24 provision: acme.go.tito.io: row is on the nested layout (db "/data/instances/acme.go.tito.io/acme.go.tito.io.db" socket "/run/carlos/instances/acme.go.tito.io/app.sock"); the record's shared paths are not drift
2026/10/20 22:14:24 provision: acme.go.tito.io: not minting a seeded instance: this box does not mint nested (<nil>)
2026/10/20 22:14:24 provision: acme.go.tito.io: not minting a seeded instance: claim control is on for this box
2026/10/20 22:14:24 provision: acme.go.tito.io: aborted seed sd_p3hz5g56do75hqu77vjrqqpemu: this box cannot read seed markers, and places no seed; nothing local is removed
2026/10/20 22:14:24 provision: acme.go.tito.io: aborted seed sd_p3hz5g56do75hqu77vjrqqpemu rolled back (no box had swapped it)
2026/10/20 22:14:24 provision: acme.go.tito.io: not minting a seeded instance: its record does not describe a hibernating instance
2026/10/20 22:14:24 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:24 provision: removed instance route acme.go.tito.io (its seed sd_feyist2fohjd5g4x6g7h55jj5q was aborted); the next pass rolls it back
2026/10/20 22:14:24 provision: acme.go.tito.io: aborted seed sd_feyist2fohjd5g4x6g7h55jj5q: this box cannot read seed markers, and places no seed; nothing local is removed
2026/10/20 22:14:24 provision: acme.go.tito.io: aborted seed sd_feyist2fohjd5g4x6g7h55jj5q rolled back (no box had swapped it)
2026/10/20 22:14:24 provision: acme.go.tito.io: aborted seed sd_7y2pfns3mo5v4sdvlkkjbbxjjm has a row this pass may not remove; leaving the seed aborted
2026/10/20 22:14:24 provision: acme.go.tito.io: aborted seed sd_kvch4ubhcnrgiwtzbo5fowshnq rolled back (no box had swapped it)
2026/10/20 22:14:25 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:25 provision: removed instance route acme.go.tito.io (its seed sd_xh5uv3shyogop2xebazwlxcuxm was aborted); the next pass rolls it back
2026/10/20 22:14:25 provision: acme.go.tito.io: aborted seed sd_xh5uv3shyogop2xebazwlxcuxm has no route on this box: its local data is removed
2026/10/20 22:14:25 provision: acme.go.tito.io: aborted seed sd_xh5uv3shyogop2xebazwlxcuxm rolled back (no box had swapped it)
2026/10/20 22:14:25 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_diqflvagyls2fwkvfyqargnp34 (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_diqflvagyls2fwkvfyqargnp34 rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_diqflvagyls2fwkvfyqargnp34 is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_snter3bw4b7ud335h3cgalxsai (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_snter3bw4b7ud335h3cgalxsai rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_snter3bw4b7ud335h3cgalxsai is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_shcsempewbykgkf623v5zhtsfq (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_shcsempewbykgkf623v5zhtsfq rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_shcsempewbykgkf623v5zhtsfq: the host's replica is left alone: its record no longer names this seed
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_shcsempewbykgkf623v5zhtsfq is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_xf3rmqfmal4guy2pl7butzlpcm (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_xf3rmqfmal4guy2pl7butzlpcm rollback deferred: the host is not fenced and stopped yet (<nil>)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_xf3rmqfmal4guy2pl7butzlpcm rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_xf3rmqfmal4guy2pl7butzlpcm is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_ljfmeic33zp53pdxcmucxkgtvi (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_ljfmeic33zp53pdxcmucxkgtvi rollback deferred: the host is not fenced and stopped yet (stop unproven)
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_psumqvzy4o66e44f5zt5blvbxm (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_psumqvzy4o66e44f5zt5blvbxm is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_g3emxyl6zxmohmvlc5k26humyy (the original route is journaled)
2026/10/20 22:14:25 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: malformed: provision: invalid maintenance projection or protected fields
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_g3emxyl6zxmohmvlc5k26humyy left alone this pass: its record could not be read
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_va5o5cptebqkpzsase4f3isgt4 (the original route is journaled)
2026/10/20 22:14:25 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: read failed: connection reset
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_va5o5cptebqkpzsase4f3isgt4 left alone this pass: its record could not be read
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_4vje6y7kcrufv2nbs54ryl3zn4 (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_4vje6y7kcrufv2nbs54ryl3zn4: close the import gate: constraint failed: injected (1811)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_4vje6y7kcrufv2nbs54ryl3zn4 rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_4vje6y7kcrufv2nbs54ryl3zn4 is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_z6h5lyllm7wzpeg5kweg42blmu (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_z6h5lyllm7wzpeg5kweg42blmu: close the import gate: constraint failed: injected (1811)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_z6h5lyllm7wzpeg5kweg42blmu rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_z6h5lyllm7wzpeg5kweg42blmu is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_uzvwszb3wv7cyhy5rw6ctom6vm (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_uzvwszb3wv7cyhy5rw6ctom6vm rollback deferred: the host is not fenced and stopped yet (injected failure at fence)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_uzvwszb3wv7cyhy5rw6ctom6vm rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_uzvwszb3wv7cyhy5rw6ctom6vm is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_7ki4hpfq3bvhadlfebulgs232a (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_7ki4hpfq3bvhadlfebulgs232a rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_7ki4hpfq3bvhadlfebulgs232a is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_7swleorqnjzyuytnyyo2x33f7e (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_7swleorqnjzyuytnyyo2x33f7e: retire its instance key: injected failure at retire-key
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_7swleorqnjzyuytnyyo2x33f7e rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_7swleorqnjzyuytnyyo2x33f7e is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_fugama6mwb6rbyusnysu5p5m6y (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_fugama6mwb6rbyusnysu5p5m6y rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_fugama6mwb6rbyusnysu5p5m6y is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_krmg23tv6waycjhcdsvvwosc5e (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_krmg23tv6waycjhcdsvvwosc5e: remove its local data: injected failure at remove-local
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_krmg23tv6waycjhcdsvvwosc5e rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_krmg23tv6waycjhcdsvvwosc5e is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_o2ojipcuf46nobfixfjckgy5ye (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_o2ojipcuf46nobfixfjckgy5ye rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_o2ojipcuf46nobfixfjckgy5ye is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_stxls6g7xv4qagv5y6ykyfs5pa (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_stxls6g7xv4qagv5y6ykyfs5pa: remove its local data: injected failure at remove-local-done
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_stxls6g7xv4qagv5y6ykyfs5pa rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_stxls6g7xv4qagv5y6ykyfs5pa is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_yjyqrj44i32zq7hapeutyy5lya (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_yjyqrj44i32zq7hapeutyy5lya rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_yjyqrj44i32zq7hapeutyy5lya is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_7otrft4fnt3ekruimknohwbavm (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_7otrft4fnt3ekruimknohwbavm: restore the original route: constraint failed: injected (1811)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_7otrft4fnt3ekruimknohwbavm rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_7otrft4fnt3ekruimknohwbavm is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_5u5v3yyqaixw7p37o3bqtbmv7u (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_5u5v3yyqaixw7p37o3bqtbmv7u: restore the original route: constraint failed: injected (1811)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_5u5v3yyqaixw7p37o3bqtbmv7u rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_5u5v3yyqaixw7p37o3bqtbmv7u is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_l5meg4iuedoi6nb7vmfw3c7zue (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_l5meg4iuedoi6nb7vmfw3c7zue is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_27dx7a524mc57fbsbwx5sypyey (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_27dx7a524mc57fbsbwx5sypyey rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_27dx7a524mc57fbsbwx5sypyey: delete its snapshot: injected failure at delete-snapshot
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_27dx7a524mc57fbsbwx5sypyey is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_r2bdtryksbf5ojqj45lvbw32ge (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_r2bdtryksbf5ojqj45lvbw32ge rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_r2bdtryksbf5ojqj45lvbw32ge is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_lrg62euoll3wu5n7y42hhvfgwe (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_lrg62euoll3wu5n7y42hhvfgwe rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_lrg62euoll3wu5n7y42hhvfgwe: delete the host's replica: injected failure at delete-replica
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_lrg62euoll3wu5n7y42hhvfgwe is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_6uil6vxa65mvcez2fxymq32hri (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_6uil6vxa65mvcez2fxymq32hri rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_6uil6vxa65mvcez2fxymq32hri is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_mjzhhgwb4jp35i4qe2qvjhld3m (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_mjzhhgwb4jp35i4qe2qvjhld3m rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_mjzhhgwb4jp35i4qe2qvjhld3m: retire its journal row: constraint failed: injected (1811)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_mjzhhgwb4jp35i4qe2qvjhld3m is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_jua7mnoxclvmovkxchsbbzwv7y (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_jua7mnoxclvmovkxchsbbzwv7y rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_jua7mnoxclvmovkxchsbbzwv7y: retire its journal row: constraint failed: injected (1811)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_jua7mnoxclvmovkxchsbbzwv7y is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_gzvx7speaezlephcp4ja7idmqe (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_gzvx7speaezlephcp4ja7idmqe rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_gzvx7speaezlephcp4ja7idmqe is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: mark import sd_gzvx7speaezlephcp4ja7idmqe rolled back: injected failure at cas
2026/10/20 22:14:25 provision: a.shop.example.com: aborted seed sd_gzvx7speaezlephcp4ja7idmqe rolled back (no box had swapped it)
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_uktmsuhsus7g6vt5pi4rjhdz7y (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_uktmsuhsus7g6vt5pi4rjhdz7y rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_uktmsuhsus7g6vt5pi4rjhdz7y is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: aborted seed sd_uktmsuhsus7g6vt5pi4rjhdz7y rolled back (no box had swapped it)
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_sq6y2kmcyyz766eclxeceqx6ue (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_sq6y2kmcyyz766eclxeceqx6ue rollback deferred: the host is not fenced and stopped yet (<nil>)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_sq6y2kmcyyz766eclxeceqx6ue rollback deferred: the host is not fenced and stopped yet (<nil>)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_sq6y2kmcyyz766eclxeceqx6ue rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_sq6y2kmcyyz766eclxeceqx6ue is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_hpvyzdkg6an6h4gd2j74iu3jty (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_hpvyzdkg6an6h4gd2j74iu3jty rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_hpvyzdkg6an6h4gd2j74iu3jty is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_tkip55ina5shcg4sbxbeyjeuky (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_tkip55ina5shcg4sbxbeyjeuky rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_tkip55ina5shcg4sbxbeyjeuky is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: minted instance route a.shop.example.com (app "shop" acct "bac")
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_xi3cvq3gyq54uingt3vfwqclya (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_xi3cvq3gyq54uingt3vfwqclya rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_xi3cvq3gyq54uingt3vfwqclya is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: minted instance route a.shop.example.com (app "shop" acct "bac")
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_2ed6ffo7dsl3legdpwefu5wyne (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_2ed6ffo7dsl3legdpwefu5wyne rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_2ed6ffo7dsl3legdpwefu5wyne is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: minted instance route a.shop.example.com (app "shop" acct "bac")
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_jzfkmlog7kptgrmfftww2vbuq4 (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_jzfkmlog7kptgrmfftww2vbuq4 rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_jzfkmlog7kptgrmfftww2vbuq4 is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: minted instance route a.shop.example.com (app "other" acct "bac")
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_tchaavvrmhqijbzgurfhit4hza (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_tchaavvrmhqijbzgurfhit4hza rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_tchaavvrmhqijbzgurfhit4hza is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: minted instance route a.shop.example.com (app "other" acct "bac")
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_lpclunaiptwusgqgbvovl4ibwm (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_lpclunaiptwusgqgbvovl4ibwm rolled back to its original route
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_lpclunaiptwusgqgbvovl4ibwm is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: minted instance route a.shop.example.com (app "other" acct "bac")
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_ccosmtq4zzsyzd52epasctd22a (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_ccosmtq4zzsyzd52epasctd22a has no route on this box: its local data is removed and no route is put back
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_ccosmtq4zzsyzd52epasctd22a is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_audex4neuszobuhlr3km45ot2m (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_ne7ypkmbuvqptm7n74c24tumvq (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_dzj4brnecmcyuepyc4zgml3fae (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_dzj4brnecmcyuepyc4zgml3fae cannot roll back on this box: the activator is not wired; it stays journaled and gated
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_kzoogy2kzri5bw5gnjrh5swqcm (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: import sd_kzoogy2kzri5bw5gnjrh5swqcm is not rolled back: the route changed under the fence, so nothing there is touched
2026/10/20 22:14:25 provision: a.shop.example.com: import "../not-a-seed" names no snapshot and no replica; only its journal row is cleaned up
2026/10/20 22:14:25 provision: a.shop.example.com: import ../not-a-seed is cleaned up and its journal row retired
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_q6svkzne2gpmpqenx77yzuyrui (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_l4c3r6tixebl5j7imc3avhijb4 (the original route is journaled)
2026/10/20 22:14:25 provision: a.shop.example.com: swapped for import sd_5ebt7vkhfnrzm4jiwihtm2ubnq (the original route is journaled)
2026/10/20 22:14:26 provision: a.shop.example.com: swapped for import sd_i4ew64bq5c272varkais552bty (the original route is journaled)
2026/10/20 22:14:26 provision: a.shop.example.com: swapped for import sd_agy7ghdlh5h635ghjkcu4mjtde (the original route is journaled)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_agy7ghdlh5h635ghjkcu4mjtde rollback deferred: the host is not fenced and stopped yet (stop evidence unreadable)
2026/10/20 22:14:26 provision: a.shop.example.com: swapped for import sd_t26tiiupbu4jffsrnvcbkmvs5e (the original route is journaled)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_t26tiiupbu4jffsrnvcbkmvs5e has no route on this box: its local data is removed and no route is put back
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_t26tiiupbu4jffsrnvcbkmvs5e is cleaned up and its journal row retired
2026/10/20 22:14:26 provision: minted instance route a.shop.example.com (app "shop" acct "bac")
2026/10/20 22:14:26 provision: a.shop.example.com: swapped for import sd_wgidlb3dmvnkchkrtsah6saaka (the original route is journaled)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_wgidlb3dmvnkchkrtsah6saaka: remove its local data: injected failure at remove-local
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_wgidlb3dmvnkchkrtsah6saaka is not rolled back: the route at the host is not the import's, so nothing there is touched
2026/10/20 22:14:26 provision: a.shop.example.com: swapped for import sd_pquznuim4sawq35om2bqgp6hyq (the original route is journaled)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_pquznuim4sawq35om2bqgp6hyq: a route appeared at the host, so its local database is left and the rollback waits
2026/10/20 22:14:26 provision: a.shop.example.com: swapped for import sd_ryq7imwjfuyauxmdbdin243aai (the original route is journaled)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_ryq7imwjfuyauxmdbdin243aai rollback deferred: no fence generation: registry busy
2026/10/20 22:14:26 provision: a.shop.example.com: swapped for import sd_4tk6slueqf6su5c5r6xj7ukcku (the original route is journaled)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_4tk6slueqf6su5c5r6xj7ukcku rollback deferred: no fence generation: the generation source answered 0
2026/10/20 22:14:26 provision: a.shop.example.com: swapped for import sd_jpm2baphspjsvaqvvyzkhjroam (the original route is journaled)
2026/10/20 22:14:26 provision: a.shop.example.com: swapped for import sd_g2zca65lrff724f2bngoaavyni (the original route is journaled)
2026/10/20 22:14:26 provision: a.shop.example.com: swapped for import sd_hayj4uisc757s3crbr5e6v5aia (the original route is journaled)
2026/10/20 22:14:26 provision: a.shop.example.com: import deferred: the instance's env file could not be read: a symlink
2026/10/20 22:14:26 provision: a.shop.example.com: import deferred: this box cannot read the instance's env file
2026/10/20 22:14:26 provision: a.shop.example.com: swapped for import sd_7hgsbxyiz4me3mnn5m743azkfm (the original route is journaled)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_7hgsbxyiz4me3mnn5m743azkfm rolled back to its original route
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_7hgsbxyiz4me3mnn5m743azkfm is cleaned up and its journal row retired
2026/10/20 22:14:26 provision: a.shop.example.com: swapped for import sd_uln77w2nbxenkag6ajgeqyioii (the original route is journaled)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_uln77w2nbxenkag6ajgeqyioii rolled back to its original route
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_uln77w2nbxenkag6ajgeqyioii is cleaned up and its journal row retired
2026/10/20 22:14:26 provision: a.shop.example.com: swapped for import sd_ocwukisqgkrwbpxcrvycslob3y (the original route is journaled)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_ocwukisqgkrwbpxcrvycslob3y rolled back to its original route
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_ocwukisqgkrwbpxcrvycslob3y is cleaned up and its journal row retired
2026/10/20 22:14:26 provision: a.shop.example.com: swapped for import sd_dzgxrml65s54cpggpmpzseizqq (the original route is journaled)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_dzgxrml65s54cpggpmpzseizqq: retire its instance key: the env file is a symlink
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_dzgxrml65s54cpggpmpzseizqq rolled back to its original route
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_dzgxrml65s54cpggpmpzseizqq is cleaned up and its journal row retired
2026/10/20 22:14:26 provision: a.shop.example.com: swapped for import sd_a4pc3z6kevugzgjw7xxz7a7f3e (the original route is journaled)
2026/10/20 22:14:26 provision: a.shop.example.com: row is on the nested layout (db "/data/instances/a.shop.example.com/a.shop.example.com.db" socket "/run/carlos/instances/a.shop.example.com/app.sock"); the record's shared paths are not drift
2026/10/20 22:14:26 provision: a.shop.example.com: swapped for import sd_l6uqcg37udcq2gj2oa6uy54ha4 (the original route is journaled)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_auozlrfewufqbzloxokbqzzhda refused: there is no route at a.shop.example.com to replace
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_otu5agyckqfg6n2gluac2lbysa refused: the route's address is 127.0.0.1:9, not 127.0.0.1:8401
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_k75jh36quakkr7k44sphrdthna refused: the route's address is 127.0.0.1:8401,127.0.0.1:8402, not 127.0.0.1:8401
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_oat5jrnzztay2kjgmp2oa5shui refused: the route's address is 127.0.0.1:9, not 127.0.0.1:8401
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_vzn2cohn44aswmimyknlsxbix4 refused: the route belongs to app other
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_jvwte4iibts52662abqccwmavi refused: the route is filed under another account
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_mvpvxt4t4rkrbgeh2n4ed3melu refused: the route is filed under the operator account, and no assignment maps shop to this app
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_uacjc5mrvc56spyvgzsoaf54gu refused: the route is filed under the operator account, and no assignment maps shop to this app
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_pj3osbzcxktyvy2u4ekttglzwy refused: the route is filed under the operator account, and no assignment maps shop to this app
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_kqnasf3k5sawl56in4lq3cx25m refused: the route is filed under the operator account, and no assignment maps shop to this app
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_7j5x3g5ltowckng3evatj55ufm refused: the route is not an address route this import can take over (socket)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_x3q2q23eez5k6u52yxspen3usa refused: the route is not an address route this import can take over (no address)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_loiybxgla2zncwpdhvg2cwytj4 refused: the route is not an address route this import can take over (provisioned)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_mvn4nsnxceeb5a35ntwe7z6mrm refused: the route is not an address route this import can take over (kind static)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_fbwretcdjmved4ttvcm6l3ry3y refused: the route is not an address route this import can take over (backing)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_z5alwqbsddok2ap3cxqrfn6lfm refused: the route is not an address route this import can take over (tunnel)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_nmp2h643aq73hvpag7ivaxmz3y refused: the route is not an address route this import can take over (caps)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_ofs5p6sivibaogpa7eomn35ctq refused: the route is not an address route this import can take over (platform)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_vqd6bglieqproklul2ewk46we4 refused: the route is not an address route this import can take over (canary)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_g442wtdhqtvdvyqmrzkcxpd7ua refused: the route is not an address route this import can take over (kind sidecar)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_ugx6jcvosewslnmi763kpc6p6e refused: the route is not an address route this import can take over (region)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_bc5x6on4twpskixzasxupj3nzy refused: the route is not an address route this import can take over (fleet_box)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_27xwfnslv7zqs3hqzwt5wuxdia refused: the route is not an address route this import can take over (hibernate)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_h7ad6m7hwx3de3u5ggov4owtti refused: the route is not an address route this import can take over (claim_owner)
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_bciprlyyo2vxbot5rpymnhqasa refused: this box does not run instances nested
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_xikqcehjpn4qcobn5emjksqjmm refused: this box does not run instances nested
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_6glo5vj244oanrjccjjc4n4ewm refused: claim control is on for this box
2026/10/20 22:14:26 provision: a.shop.example.com: swapped for import sd_kq46yvcblcsfn4qasrtchoji5y (the original route is journaled)
2026/10/20 22:14:26 provision: a.shop.example.com: swapped for import sd_5vfqomch35omktl22hbhqm7pzy (the original route is journaled)
2026/10/20 22:14:26 provision: a.shop.example.com: import deferred: this box cannot read the legacy route assignments
2026/10/20 22:14:26 provision: a.shop.example.com: import deferred: the legacy route assignments could not be read: store down
2026/10/20 22:14:26 provision: a.shop.example.com: not swapping for import sd_eg2slhemp6ersv742r2rywahr4: its verify hash is not a sha256
2026/10/20 22:14:26 provision: a.shop.example.com: not swapping for import sd_negkgfhzkz2oqwtb6uqin7vyyq: its verify hash is not a sha256
2026/10/20 22:14:26 provision: a.shop.example.com: not swapping for import sd_pgizuij3d6y2oxnff7lliijxai: its verify hash is not a sha256
2026/10/20 22:14:26 provision: a.shop.example.com: not swapping for import sd_../../x: its host or seed id is malformed (seeds: not a seed id)
2026/10/20 22:14:26 provision: A.shop.example.com/..: not swapping for import sd_23n2qqznknwan4e7gpzs3tzygi: its host or seed id is malformed (seeds: not a host)
2026/10/20 22:14:26 provision: a.shop.example.com: not swapping for import sd_sekihm3idybqbyrwzl3vlbvgsm: its record does not describe a nested, hibernating instance
2026/10/20 22:14:26 provision: a.shop.example.com: not swapping for import sd_75xunyykfwb5wu2pey7grvk66m: its record does not describe a nested, hibernating instance
2026/10/20 22:14:26 provision: a.shop.example.com: not swapping for import sd_vj3w6renblxbxhcyenbu7zch2y: its record does not describe a nested, hibernating instance
2026/10/20 22:14:26 provision: a.shop.example.com: not swapping for import sd_yowa7veyblbb5dov7pz52rcqhm: its record does not describe a nested, hibernating instance
2026/10/20 22:14:26 provision: a.shop.example.com: import deferred: the box's layout intent could not be read: store down
2026/10/20 22:14:26 provision: a.shop.example.com: swapped for import sd_oioeu7gbivzcjouabv6gqh24xy (the original route is journaled)
2026/10/20 22:14:26 provision: a.shop.example.com: import deferred: the box's nested probe did not answer: ready probe timed out
2026/10/20 22:14:26 provision: a.shop.example.com: swapped for import sd_7khg7kwkavy754vc4t62zxpyy4 (the original route is journaled)
2026/10/20 22:14:26 provision: a.shop.example.com: swapped for import sd_qjct5jv5lrbgqnvmkbb6kitm24 (the original route is journaled)
2026/10/20 22:14:26 provision: a.shop.example.com: import deferred: the route is held by operation op_1
2026/10/20 22:14:26 provision: dry-run: would consider swapping a.shop.example.com for import sd_ynrstvqadwou6kxaxndb2kqsle
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_hawxs6ahyw7dadocpvtqkk2hgi rolled back to its original route
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_hawxs6ahyw7dadocpvtqkk2hgi: the host's replica is left alone: the record apps/bac/shop/instances/a.shop.example.com.json holds the host
2026/10/20 22:14:26 provision: a.shop.example.com: import sd_hawxs6ahyw7dadocpvtqkk2hgi is cleaned up and its journal row retired
2026/10/20 22:14:27 provision: a.shop.example.com: import sd_p2iym4rvnnwr5naluebukj4thi left alone: the import journal holds seed sd_reyn2xtcm3ljzkk56dswcmnvee at swapped
2026/10/20 22:14:27 provision: a.shop.example.com: not minting import sd_va343ko4dzckrdhmehse7mitf4: it is swapped but not committed, and only its swap makes its row
2026/10/20 22:14:27 provision: a.shop.example.com: not minting import sd_w2t62pedg2sqvlvwl57sckfzte: it is placed but not committed, and only its swap makes its row
2026/10/20 22:14:27 provision: a.shop.example.com: swapped for import sd_wvn7xsooiw4ak6tsr5uouyft2q (the original route is journaled)
2026/10/20 22:14:27 provision: a.shop.example.com: not minting import sd_wvn7xsooiw4ak6tsr5uouyft2q: it is swapped but not committed, and only its swap makes its row
2026/10/20 22:14:27 provision: a.shop.example.com: not minting import sd_wvn7xsooiw4ak6tsr5uouyft2q: it is placed but not committed, and only its swap makes its row
2026/10/20 22:14:27 provision: minted instance route a.shop.example.com (app "shop" acct "bac")
2026/10/20 22:14:27 provision: removed retired sidecar route gone.eleven.bac.sidecar.internal (record gone)
2026/10/20 22:14:27 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/20 22:14:27 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/20 22:14:27 provision: converged sidecar bot.eleven.bac.sidecar.internal args ["sidecar","run"] -> ["worker","run"]
2026/10/20 22:14:27 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/20 22:14:27 provision: converged sidecar bot.eleven.bac.sidecar.internal ceilings 300/3600 -> 60/120
2026/10/20 22:14:27 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/20 22:14:27 provision: converged sidecar bot.eleven.bac.sidecar.internal channel stable -> canary/eleven
2026/10/20 22:14:27 provision: converged sidecar bot.eleven.bac.sidecar.internal environment "other" -> "relay"
2026/10/20 22:14:27 provision: converged sidecar bot.eleven.bac.sidecar.internal hibernate true -> false
2026/10/20 22:14:27 provision: skipping sidecar record apps/bac/eleven/sidecars/bot.json: read failed: failStore: reads disabled under apps/bac/eleven/sidecars/
2026/10/20 22:14:27 provision: skipping sidecar record apps/bac/eleven/sidecars/bot.json: malformed: invalid character 'n' looking for beginning of object key string
2026/10/20 22:14:27 provision: not removing sidecar bot.eleven.bac.sidecar.internal: its record exists but could not be read cleanly this pass
2026/10/20 22:14:27 provision: skipping sidecar record apps/bac/eleven/sidecars/bot.json: empty channel
2026/10/20 22:14:27 provision: skipping sidecar record apps/bac/eleven/sidecars/held.json: empty channel
2026/10/20 22:14:27 provision: not removing sidecar held.eleven.bac.sidecar.internal: its record exists but could not be read cleanly this pass
2026/10/20 22:14:27 provision: removed retired sidecar route old.eleven.bac.sidecar.internal (record gone)
2026/10/20 22:14:27 provision: bac/eleven has sidecar records but no sidecars manifest — not minting
2026/10/20 22:14:27 provision: bac/eleven is in the trash — not minting sidecar routes for it
2026/10/20 22:14:27 provision: sidecar bot.eleven.bac.sidecar.internal is not placeable here: it asks for "no-inbound" placement and this box accepts inbound work (CARLOS_INBOUND="")
2026/10/20 22:14:27 provision: sidecar hub.relay.bac.sidecar.internal is not placeable here: it asks for "no-inbound" placement and this box accepts inbound work (CARLOS_INBOUND="")
2026/10/20 22:14:27 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/20 22:14:27 provision: minted sidecar route hub.relay.bac.sidecar.internal (app "relay" acct "bac" args ["relay"])
2026/10/20 22:14:27 provision: sidecar bot.eleven.bac.sidecar.internal is not placeable here: it asks for "no-inbound" placement and this box accepts inbound work (CARLOS_INBOUND="")
2026/10/20 22:14:27 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/20 22:14:27 provision: sidecar bot.eleven.bac.sidecar.internal is not placeable here: it asks for "no-inbound" placement and this box accepts inbound work (CARLOS_INBOUND="")
2026/10/20 22:14:27 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/20 22:14:27 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/20 22:14:27 provision: sidecar bot.eleven.bac.sidecar.internal is not placeable here: it asks for "no-inbound" placement and this box accepts inbound work (CARLOS_INBOUND="")
2026/10/20 22:14:27 provision: removed retired sidecar route bot.eleven.bac.sidecar.internal (record gone)
2026/10/20 22:14:27 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/20 22:14:27 provision: sidecar bot.eleven.bac.sidecar.internal is not placeable here: it asks for "no-inbound" placement and this box accepts inbound work (CARLOS_INBOUND="")
2026/10/20 22:14:27 provision: skipping sidecar record apps/bac/relay/sidecars/hub.json: read failed: failStore: reads disabled under apps/bac/relay/sidecars/
2026/10/20 22:14:27 provision: REFUSING to mint sidecar bot.eleven.bac.sidecar.internal: bot.eleven.bac.sidecar.internal-sidecar.env is staged on this box — the legacy file co-spawn and a native sidecar row are two mechanisms for one host; remove the file to migrate this host onto sidecar records
2026/10/20 22:14:27 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"])
2026/10/20 22:14:27 provision: dry-run: would mint sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" channel "stable" args ["sidecar","run"])
2026/10/20 22:14:27 provision: dry-run: would remove retired sidecar route gone.eleven.bac.sidecar.internal (record gone)
2026/10/20 22:14:27 provision: bot.eleven.bac.sidecar.internal exists but is not a sidecar row this pass owns (kind "sidecar" provisioned false app "eleven" acct "bac") — leaving it alone
2026/10/20 22:14:27 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:27 provision: converged acme.go.tito.io update settings "{\"updates\":\"manual\",\"channel\":\"stable\"}" -> ""
2026/10/20 22:14:27 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:27 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:27 provision: manifest bac/tito: read failed: provision: manifest bac/tito: unexpected end of JSON input
2026/10/20 22:14:27 provision: manifest bac/tito: read failed: provision: manifest bac/tito: unexpected end of JSON input
2026/10/20 22:14:27 provision: acme.go.tito.io: not minted this pass: the app's update policy could not be read
2026/10/20 22:14:27 provision: minted instance route acme.go.tito.io (app "tito" acct "bac")
2026/10/20 22:14:27 provision: manifest bac/tito: read failed: provision: manifest bac/tito: unexpected end of JSON input
2026/10/20 22:14:27 provision: acme.go.tito.io: not minted this pass: the app's update policy could not be read
2026/10/20 22:14:27 provision: minted instance route acme.go.tito.io held landing by mv_acme (app "tito" acct "bac")
FAIL
FAIL github.com/carlosframework/platform/internal/provision 36.863s
ok github.com/carlosframework/platform/internal/publicrelease 0.008s
? github.com/carlosframework/platform/internal/regionname [no test files]
ok github.com/carlosframework/platform/internal/registry 5.511s
ok github.com/carlosframework/platform/internal/relay 7.248s
ok github.com/carlosframework/platform/internal/release 24.362s
ok github.com/carlosframework/platform/internal/rollout 3.816s
ok github.com/carlosframework/platform/internal/rolloutwords 0.003s
ok github.com/carlosframework/platform/internal/rootaws 0.006s
ok github.com/carlosframework/platform/internal/rootparticipation 0.060s
ok github.com/carlosframework/platform/internal/rootstate 0.026s
ok github.com/carlosframework/platform/internal/runs 0.576s
ok github.com/carlosframework/platform/internal/s3compat 0.004s
ok github.com/carlosframework/platform/internal/safefile 0.004s
ok github.com/carlosframework/platform/internal/schedule 2.513s
ok github.com/carlosframework/platform/internal/seeds 0.011s
ok github.com/carlosframework/platform/internal/sim/hostlife 9.764s
ok github.com/carlosframework/platform/internal/sim/litestreamfacts 3.860s
ok github.com/carlosframework/platform/internal/status 0.059s
ok github.com/carlosframework/platform/internal/steering 0.005s
ok github.com/carlosframework/platform/internal/stopevidence 0.233s
ok github.com/carlosframework/platform/internal/svcrole 0.002s
ok github.com/carlosframework/platform/internal/telemetry 0.004s
ok github.com/carlosframework/platform/internal/termfence 0.004s
ok github.com/carlosframework/platform/internal/tunnelclient 0.004s
ok github.com/carlosframework/platform/internal/unixdial 0.002s
ok github.com/carlosframework/platform/internal/updatewords 0.004s
ok github.com/carlosframework/platform/internal/usage 0.004s
ok github.com/carlosframework/platform/internal/version 0.001s
ok github.com/carlosframework/platform/internal/vet 1.556s
FAIL
make: *** [Makefile:18: test] Error 1
amadan: step gate failed: exit status 2
amadan: job failed at step gate — its output is above, not at the tail of this log
diff --git a/harness/RUNLOG.md b/harness/RUNLOG.md| index e88f217ab..b24fe894f 100644 |
| --- a/harness/RUNLOG.md |
| +++ b/harness/RUNLOG.md |
| @@ -2183,9 +2183,80 @@ contiguous; keys and files unchanged; config.env on dennis at 21:47:04.640Z (fli |
| ``` |
| Never moved by the 15-s refresh; only an ordinary pass re-asserts it (once a pass, to now+7 days). |
| -**Finding R10-1 (behaviour, not a correctness defect): after open, the source box's edge answers 404 for the moved |
| -host until finish's cleanup is under way.** B4: through sophie's edge, 200 only 17.9 s after the first 200 on dennis |
| -(17 tries at 1 s); through leo's edge, 200 at once by then. B1: 404 through both at 7 s after open. Sophie keeps the |
| -row `moving` until finish removes it, and its edge serves 404 for a held row rather than backhauling it. On dev and |
| -flagship the record's DNS points where? If any client still reaches an IE edge for the host after open, it gets 404 |
| -for up to the time finish takes (4-5 min). Measured with answer codes in B2. |
| +**Finding R10-1 (behaviour, not a correctness defect): requests that reach an IE edge right after open get 404 for |
| +up to ~20 s.** The edge drops every non-ready row from its route map (`internal/edge/edge.go:654`, `if t.Status != |
| +"ready" { continue }`), so after open sophie's `moving` row is skipped and a request for the host falls to backhaul, |
| +which needs dennis's published host list (`control/edges/noor-marquetry.json` `hosts`) to have reached that edge. |
| +Measured: B1 404 through sophie's and leo's edges 7 s after open; B4 200 through sophie's edge only 17.9 s after |
| +the first 200 on dennis (leo's at once by then); B2 200 through both at once (the first 200 on dennis came 2 min after |
| +open). Runbook note (M11 below): check the host through the target box first; expect the IE edges to follow within |
| +about 20 s. |
| + |
| +### B2. The LARGE instance, shop-large (same app as B1), sophie -> dennis: PASS, offline 296 s (about 5 min) |
| + |
| +Made to match flagship's large instance as far as the harness allows (`blocks/r10-grow.host.sh shop-large.dev.harness.test |
| +535 1536 100`, 21:53:36-21:57:01Z, on the live instance as its owner uid 989, through SQLite): |
| +- **db 636 MiB** (target "about 535"; the insert loop's 4 MiB batches overshot): 487,424 real rows in table `orders` |
| + (created_at, customer, status, ~1 KiB random-hex body), inserted in 29 s, then **1536 MiB of churn** (UPDATE of |
| + `body` in 4 MiB transactions) in 175 s; |
| +- **replica history 3254 MiB in 177 objects** in `harness-data-ie instances/<host>/` at 21:57:01Z (3503 MiB, 187 |
| + objects by the move; the real IE replica is 3.9 GiB), and 3.5 GiB of local LTX under sophie's |
| + `.activator-<host>.db-litestream/ltx`; |
| +- **files tree 101 MiB** (`media/upload-*.bin`, 100 x 1 MiB, plus `instance.key` and a new `report.token`; 102 files, |
| + tree digest `a7d709cc...`); |
| +- **the stale flat legacy dir** `/data/shop-large.dev.harness.test/` on sophie (carlos 0700) holding `instance.key` and |
| + `report.token`, byte-identical to the nested ones (`cmp` both); |
| +- the 1-minute timer (`every-minute`, never hibernates) and **a one-shot timer due during the move** (`once-1` armed |
| + at 21:57:42Z, due 21:58:27Z: inside the seal). |
| +The app counted `worker_writes_failed 6` during the bulk write (lock contention from my 4 MiB transactions; harness |
| +load, carried in its database). |
| + |
| +`r10-move.host.sh shop shop-large move 45`: |
| + |
| +| step | at | took | tries | result | |
| +|---|---|---|---|---| |
| +| start | 21:57:42.231Z | 4.3 s | 1 | exit 0; `activator: ... hibernated` 21:57:44Z | |
| +| **seal** | 21:57:46.491Z | **69.1 s** | 1 | checkpointed (position 184, replica 184) 21:57:53Z; **`sealed at TXID 184 (186 staged objects ...)` 21:58:53Z**: the whole replica history copied to uk staging in ~60 s | |
| +| **flip** | 21:58:55.637Z | **85.0 s** | 1 | dennis: `landing, not yet OK: db, lease, files, ...` 21:58:55Z; `extracted the files and env`, `landed; every item is OK` 22:00:20Z (`fast converge: ... moves 1m19s`) | |
| +| open | 22:00:20.659Z | 4.1 s | 1 | `opened; the row is ready and adoption stages its build` 22:00:23Z | |
| +| **first request on dennis** | 22:00:24.889Z | **133.1 s** | | 404 (22:00:24.99Z), then four 503s at the 30 s wake bound, **200 at 22:02:38.017Z** (7.2 s request). Dennis: `promoting 186 staged object(s) from moves/<host>/ (residence move landing)` 22:00:26Z, `restored ... from s3://harness-data-uk/instances/<host>` 22:02:36Z | |
| +| finish | 22:02:39.200Z | 55.3 s | 1 | `Moved. ... It was offline for 158 seconds.` | |
| + |
| +**Downtime: 295.8 s from start to the first 200 on dennis**: start 4 s + seal 69 s + flip 85 s + open 4 s + first |
| +wake 133 s. **The console's figure (`offline_s 158.4`) leaves out the first wake** (it stops at open), so for a |
| +large instance it understates the outage by the wake (here 2 min 13 s). All three long steps scale with the replica |
| +HISTORY (186 objects, 3.5 GiB), not the 636 MiB database: seal copies it, landing checks it, and the target's first |
| +wake promotes every staged object and restores from it. B1's 33-object history moved in ~20 s. |
| +**Budget for flagship's large instances (300-540 MB, history up to 3.9 GiB): about 5 to 6 minutes offline each**, |
| +at harness speeds (MinIO on one EBS volume; AWS S3 server-side copies will differ, likely faster per object), plus |
| +the operator's gaps. A compacted/trimmed history before the move would shorten seal, flip and the first wake; nothing |
| +in the doors does that today. |
| + |
| +Checks: **PASS on every item.** |
| +- **Rows:** dennis `bg` home 1-2886 then uk 2887-3577, `worker` home 1-1459 then uk 1460-1801, contiguous; the |
| + 487,424 `orders` rows are in the restored 637 MiB database. |
| +- **Keys and files:** `<host>.env` `2b08fa3b...`, instance key sha256 `eac6dfc1...`, `file_key_sha256 b8fc80ec...` |
| + unchanged; **files tree digest `a7d709cc...` on dennis = sophie's** (102 files, 101M), `instance.key` `c2b23e64...`. |
| +- **config.env before the first wake:** `/data/<host>.config.env` at 22:00:20.343Z (landing), first request 22:00:24.9Z. |
| +- **The one-shot timer due during the move:** not fired in ie (fenced); carried; **fired on dennis at the first wake** |
| + (`once/uk n=1`, `once-1 due 1792533507 count 1`, timer `last_status 200` at 22:02:38Z): 4 min 11 s late, once. |
| +- **The legacy flat dir: neither the seal's local-state check nor the source retire tripped on it.** The seal and |
| + every door passed first time; at open sophie's retire **parked it with the instance**: |
| + `/data/.offline-moves/<host>/mv_nxl7zcylejgjyaqh22jyoz5xey/source/shop-large.dev.harness.test/{instance.key,report.token}` |
| + beside `source/instance/` (the nested directory) and `source/<host>.json`; `/data/shop-large.dev.harness.test` is |
| + gone from sophie. On dennis the files land once, in the flat files dir `/data/<host>/` (`instance.key media |
| + report.token`, from the nested tree); the legacy copy did not travel. |
| +- **Defect 8: PASS** (sophie's `.instances/<host>/` empty from open on). **Tombstone:** written at the seal |
| + (21:57:53Z), re-asserted by the passes at 21:58:53Z, 21:58:56Z, 22:00:23Z and 22:02:40Z, unchanged across flip's |
| + 85 s; lifted at finish 22:03:31Z. Data-uk lease `dennis:1359`; record `region uk`, `last_move done`. |
| +- `status` after open says `Its report is not current` for every box: expected (from open on no door judges a |
| + report by age and the 15-s refresh stops), but it reads like a fault (copy note, M12). |
| + |
| +**Disk (harness).** After B2 the host's root was at 98% (1.1 GB free): the move left the full history in BOTH regions |
| +(data-ie 3503 MiB kept at finish, data-uk 4478 MiB = the carried history + dennis's new L1 511 MiB) plus sophie's |
| +parked 3.5 GiB local LTX. To keep the harness alive I removed, by exact path, only this run's own test data: the |
| +parked `.../source/instance/.activator-shop-large.dev.harness.test.db-litestream/ltx` on sophie (3.5 GiB) and the 186 |
| +replica objects under `harness-data-ie/instances/shop-large.dev.harness.test/` (3503 MiB), keeping its `lock.json` |
| +(the lifted tombstone). Root back to 4.5 GB free. **Finding R10-2 (for the plan, not a defect): a finished move |
| +leaves the source region's whole replica history and the parked local LTX in place**: for a 3.9 GiB history that is |
| +~8 GiB per moved instance in IE until the Cleanup recipe runs. |
diff --git a/harness/evidence/run10/b2-dennis-wake.log b/harness/evidence/run10/b2-dennis-wake.log| new file mode 100644 |
| index 000000000..84a926272 |
| --- /dev/null |
| +++ b/harness/evidence/run10/b2-dennis-wake.log |
| @@ -0,0 +1,34 @@ |
| +[hssm] i-06a1ec144aa5226fb 4b96e2dc-edb2-415d-9adc-07df128c45ae Success exit=0 |
| +2026-10-20T21:58:55+00:00 dennis carlos[1359]: 2026/10/20 21:58:55 moves: shop-large.dev.harness.test: offline move mv_nxl7zcylejgjyaqh22jyoz5xey: landing, not yet OK: db, lease, files, env, key-adoption, config, env-pin, release, timers, next-due, replication, row, warm-cache |
| +2026-10-20T21:58:55+00:00 dennis carlos[1359]: 2026/10/20 21:58:55 layout: deferred until asleep -> shop-small.dev.harness.test (awake (stopped for the move in 38m45s unless it sleeps first)), staleapp.dev.harness.test (awake (stopped for the move in 48m36s unless it sleeps first)) |
| +2026-10-20T21:59:01+00:00 dennis carlos[1359]: 2026/10/20 21:59:01 provision: minted instance route shop-large.dev.harness.test held landing by mv_nxl7zcylejgjyaqh22jyoz5xey (app "shop" acct "bab") |
| +2026-10-20T21:59:01+00:00 dennis carlos[1359]: 2026/10/20 21:59:01 agent: shop-large.dev.harness.test: held for a move; asked root to re-judge what it staged |
| +2026-10-20T21:59:01+00:00 dennis systemd[1]: Starting carlos-stage.service - carlos release staging (on agent request, no canary nag)... |
| +2026-10-20T21:59:01+00:00 dennis carlos[1932]: 2026/10/20 21:59:01 adopt: legacy restart (no root census) |
| +2026-10-20T21:59:01+00:00 dennis systemd[1]: carlos-stage.service: Deactivated successfully. |
| +2026-10-20T21:59:01+00:00 dennis systemd[1]: Finished carlos-stage.service - carlos release staging (on agent request, no canary nag). |
| +2026-10-20T22:00:20+00:00 dennis carlos[1359]: 2026/10/20 22:00:20 moves: shop-large.dev.harness.test: offline move mv_nxl7zcylejgjyaqh22jyoz5xey: extracted the files and env |
| +2026-10-20T22:00:20+00:00 dennis carlos[1359]: shop-large.dev.harness.test: config updated |
| +2026-10-20T22:00:20+00:00 dennis carlos[1359]: 2026/10/20 22:00:20 moves: shop-large.dev.harness.test: offline move mv_nxl7zcylejgjyaqh22jyoz5xey: landed; every item is OK |
| +2026-10-20T22:00:20+00:00 dennis carlos[1359]: 2026/10/20 22:00:20 agent: fast converge: serial pass stages took 1m19.1s (reconcile 0s, build states 0s, telemetry 0s, moves 1m19s, runs 0s, aliases 0s, adopt 0s, config 0s) |
| +2026-10-20T22:00:20+00:00 dennis carlos[1359]: 2026/10/20 22:00:20 layout: shop-large.dev.harness.test: deferred -> status landing |
| +2026-10-20T22:00:20+00:00 dennis carlos[1359]: 2026/10/20 22:00:20 layout: deferred until asleep -> shop-small.dev.harness.test (awake (stopped for the move in 37m20s unless it sleeps first)), staleapp.dev.harness.test (awake (stopped for the move in 47m12s unless it sleeps first)) |
| +2026-10-20T22:00:23+00:00 dennis carlos[1359]: 2026/10/20 22:00:23 agent: shop-large.dev.harness.test: held for a move; asked root to re-judge what it staged |
| +2026-10-20T22:00:23+00:00 dennis carlos[1359]: 2026/10/20 22:00:23 moves: shop-large.dev.harness.test: offline move mv_nxl7zcylejgjyaqh22jyoz5xey: opened; the row is ready and adoption stages its build |
| +2026-10-20T22:00:23+00:00 dennis systemd[1]: Starting carlos-stage.service - carlos release staging (on agent request, no canary nag)... |
| +2026-10-20T22:00:23+00:00 dennis carlos[1978]: 2026/10/20 22:00:23 adopt: legacy restart (no root census) |
| +2026-10-20T22:00:24+00:00 dennis carlos[1359]: 2026/10/20 22:00:24 adopt: try-restart carlos-app@shop-large.dev.harness.test: exit status 1: Failed to try-restart carlos-app@shop-large.dev.harness.test.service: Interactive authentication required. |
| +2026-10-20T22:00:24+00:00 dennis carlos[1359]: See system logs and 'systemctl status carlos-app@shop-large.dev.harness.test.service' for details. (continuing) |
| +2026-10-20T22:00:24+00:00 dennis carlos[1359]: shop-large.dev.harness.test: repointed -> mp1 (edge; hibernating — bounced if awake, else runs it on next wake) |
| +2026-10-20T22:00:24+00:00 dennis carlos[1359]: 2026/10/20 22:00:24 layout: shop-large.dev.harness.test: refused -> db is missing at /data/shop-large.dev.harness.test.db |
| +2026-10-20T22:00:24+00:00 dennis carlos[1359]: 2026/10/20 22:00:24 layout: deferred until asleep -> shop-small.dev.harness.test (awake (stopped for the move in 37m17s unless it sleeps first)), staleapp.dev.harness.test (awake (stopped for the move in 47m8s unless it sleeps first)) |
| +2026-10-20T22:00:24+00:00 dennis carlos[1978]: shop-large.dev.harness.test: exec-backed — config delivered by the agent's in-process loop to /data/<host>.config.env, not this pass |
| +2026-10-20T22:00:24+00:00 dennis systemd[1]: carlos-stage.service: Deactivated successfully. |
| +2026-10-20T22:00:24+00:00 dennis systemd[1]: Finished carlos-stage.service - carlos release staging (on agent request, no canary nag). |
| +2026-10-20T22:00:26+00:00 dennis carlos[1359]: 2026/10/20 22:00:26 activator: shop-large.dev.harness.test: promoting 186 staged object(s) from moves/shop-large.dev.harness.test/ (residence move landing) |
| +2026-10-20T22:02:36+00:00 dennis carlos[1359]: 2026/10/20 22:02:36 activator: restored shop-large.dev.harness.test from s3://harness-data-uk/instances/shop-large.dev.harness.test |
| +2026-10-20T22:02:37+00:00 dennis carlos[1359]: 2026/10/20 22:02:37 cgroup: shop-large.dev.harness.test: memory.max=512M memory.high=460M |
| +2026-10-20T22:02:42+00:00 dennis carlos[1359]: 2026/10/20 22:02:42 layout: deferred until asleep -> shop-large.dev.harness.test (awake (stopped for the move in 1h0m0s unless it sleeps first)), shop-small.dev.harness.test (awake (stopped for the move in 34m58s unless it sleeps first)), staleapp.dev.h |
| +2026-10-20T22:02:48+00:00 dennis carlos[1359]: 2026/10/20 22:02:48 INFO detected database behind replica system=store db=shop-large.dev.harness.test.db db_txid=0000000000000000 replica_txid=00000000000000b8 |
| +2026-10-20T22:02:48+00:00 dennis carlos[1359]: 2026/10/20 22:02:48 INFO fetched latest L0 file from replica system=store db=shop-large.dev.harness.test.db min_txid=00000000000000b8 max_txid=00000000000000b8 |
| +shop-large via dennis: region uk boot d6e42da16af84f8b | bg/home n=2886 1-2886 bg/uk n=691 2887-3577 once/uk n=1 0-0 tick/home n=24 0-0 tick/uk n=7 0-0 worker/home n=1459 1-1459 worker/uk n=342 1460-1801 | ticks 31 once [('once-1', 1792533507, 1)] | pk eac6dfc1cd29bf79 fk b8fc80ec801b0fdd | object probe/d6e42da16af84f8b.txt | worker_failed 6 |
diff --git a/harness/evidence/run10/b2-grow.log b/harness/evidence/run10/b2-grow.log| index e69de29bb..8e8eda4a1 100644 |
| --- a/harness/evidence/run10/b2-grow.log |
| +++ b/harness/evidence/run10/b2-grow.log |
| @@ -0,0 +1,40 @@ |
| +[hssm] i-06a1ec144aa5226fb f68eee39-973f-4024-8766-d945b077a0e3 Success exit=0 |
| +== rows and churn (21:53:36) |
| +rows 65536 db 0 MiB wal 87 MiB at 3s |
| +rows 131072 db 0 MiB wal 173 MiB at 7s |
| +rows 196608 db 241 MiB wal 260 MiB at 10s |
| +rows 262144 db 241 MiB wal 346 MiB at 15s |
| +rows 327680 db 241 MiB wal 433 MiB at 18s |
| +rows 393216 db 465 MiB wal 519 MiB at 24s |
| +rows 458752 db 465 MiB wal 606 MiB at 27s |
| +insert done: rows 487424 db 599 MiB in 29s |
| +churn 128 MiB db 636 MiB wal 16 MiB at 50s |
| +churn 256 MiB db 636 MiB wal 189 MiB at 71s |
| +churn 384 MiB db 636 MiB wal 361 MiB at 78s |
| +churn 512 MiB db 636 MiB wal 533 MiB at 93s |
| +churn 640 MiB db 636 MiB wal 705 MiB at 100s |
| +churn 768 MiB db 636 MiB wal 877 MiB at 109s |
| +churn 896 MiB db 636 MiB wal 172 MiB at 138s |
| +churn 1024 MiB db 636 MiB wal 344 MiB at 145s |
| +churn 1152 MiB db 636 MiB wal 516 MiB at 152s |
| +churn 1280 MiB db 636 MiB wal 119 MiB at 186s |
| +churn 1408 MiB db 636 MiB wal 291 MiB at 193s |
| +churn 1536 MiB db 636 MiB wal 463 MiB at 201s |
| +churn done: 1536 MiB, rows 487424, db 636 MiB, 204s |
| +== files tree and the legacy dir (21:57:00) |
| +legacy /data/shop-large.dev.harness.test/: instance.key and report.token = the nested copies |
| +total 12 |
| +drwx------ 1 989 989 48 Oct 20 21:57 . |
| +drwxr-x--- 1 989 989 632 Oct 20 21:57 .. |
| +-rw------- 1 989 989 65 Oct 20 21:57 instance.key |
| +-rw------- 1 989 989 64 Oct 20 21:57 report.token |
| +101M /data/instances/shop-large.dev.harness.test/shop-large.dev.harness.test |
| +637M /data/instances/shop-large.dev.harness.test/shop-large.dev.harness.test.db |
| +463M /data/instances/shop-large.dev.harness.test/shop-large.dev.harness.test.db-wal |
| +3.5G /data/instances/shop-large.dev.harness.test/.activator-shop-large.dev.harness.test.db-litestream |
| +files tree digest: a7d709cc6c115538 |
| +files: 102 |
| +== the replica in harness-data-ie (21:57:01) |
| +objects 177, 3254 MiB |
| +/dev/nvme0n1p1 59G 46G 11G 82% / |
| +/dev/loop0 45G 19G 26G 42% / |
diff --git a/harness/evidence/run10/b2-move.log b/harness/evidence/run10/b2-move.log| new file mode 100644 |
| index 000000000..d75277a8e |
| --- /dev/null |
| +++ b/harness/evidence/run10/b2-move.log |
| @@ -0,0 +1,358 @@ |
| +== shop/shop-large.dev.harness.test move (45) log dir /root/harness/r10-shop-large-move-215739 |
| +== before (21:57:39.749Z) |
| +{ |
| + "host": "shop-large.dev.harness.test", |
| + "region": "home", |
| + "region_from": "unset, treated as home", |
| + "mode": "normal", |
| + "boot_id": "785fbd0f9fe7ecb6", |
| + "rows": [ |
| + { |
| + "kind": "bg", |
| + "region": "home", |
| + "count": 2878, |
| + "min_seq": 1, |
| + "max_seq": 2878 |
| + }, |
| + { |
| + "kind": "tick", |
| + "region": "home", |
| + "count": 24, |
| + "min_seq": 0, |
| + "max_seq": 0 |
| + }, |
| + { |
| + "kind": "worker", |
| + "region": "home", |
| + "count": 1455, |
| + "min_seq": 1, |
| + "max_seq": 1455 |
| + } |
| + ], |
| + "fired": [ |
| + { |
| + "kind": "tick", |
| + "name": "every-minute", |
| + "due_at": 1792531860, |
| + "count": 1 |
| + }, |
| + { |
| + "kind": "tick", |
| + "name": "every-minute", |
| + "due_at": 1792532100, |
| + "count": 1 |
| + }, |
| + { |
| + "kind": "tick", |
| + "name": "every-minute", |
| + "due_at": 1792532160, |
| + "count": 1 |
| + }, |
| + { |
| + "kind": "tick", |
| + "name": "every-minute", |
| + "due_at": 1792532220, |
| + "count": 1 |
| + }, |
| + { |
| + "kind": "tick", |
| + "name": "every-minute", |
| + "due_at": 1792532280, |
| + "count": 1 |
| + }, |
| + { |
| + "kind": "tick", |
| + "name": "every-minute", |
| + "due_at": 1792532340, |
| + "count": |
| +-- sophie |
| + env file: /data/instances/shop-large.dev.harness.test/shop-large.dev.harness.test.env 2b08fa3b2da90a42 key-sha eac6dfc1cd29bf79 |
| + config.env /data/instances/shop-large.dev.harness.test/shop-large.dev.harness.test.config.env (21:32:48.586): HARNESS_NOTE MOVEPROBE_ACCESS_KEY MOVEPROBE_BUCKET MOVEPROBE_ENDPOINT MOVEPROBE_REGION MOVEPROBE_SECRET MOVEPROBE_SECRET_KEY |
| + files: /data/instances/shop-large.dev.harness.test/shop-large.dev.harness.test/instance.key sha c2b23e645e52fdf0 |
| + files: /data/shop-large.dev.harness.test/instance.key sha c2b23e645e52fdf0 |
| + files tree /data/instances/shop-large.dev.harness.test/shop-large.dev.harness.test: a7d709cc6c115538 (102 files, 101M) |
| + db /data/instances/shop-large.dev.harness.test/shop-large.dev.harness.test.db 637M |
| + instance dir: .activator-shop-large.dev.harness.test.db-litestream launch.run.json shop-large.dev.harness.test shop-large.dev.harness.test.config.env shop-large.dev.harness.test.db shop-large.dev.harness.test.db-shm shop-large.dev.harness.test.db-wal shop-large.dev.harness.test.env |
| + .instances dir: .activator-shop-large.dev.harness.test.db-litestream launch.run.json shop-large.dev.harness.test shop-large.dev.harness.test.config.env shop-large.dev.harness.test.db shop-large.dev.harness.test.db-shm shop-large.dev.harness.test.db-wal shop-large.dev.harness.test.env |
| + legacy /data/shop-large.dev.harness.test: instance.key report.token |
| + row: [('ready', '', '', '/data/instances/shop-large.dev.harness.test/shop-large.dev.harness.test.db', '')] |
| + timers: [('shop-large.dev.harness.test', 'every-minute', 'schedule', 'every 1m', '/tick', 1792533480, 1792533421, '200', 29, 0, 1792531809, 1792533480, '647b32f515af863e7f7bf9986010d473', '', 0, '', '')] |
| +-- dennis |
| + env file: absent key-sha |
| + instance dir: |
| + .instances dir: ls: cannot access '/data/.instances/shop-large.dev.harness.test': No such file or directory |
| + legacy /data/shop-large.dev.harness.test: ls: cannot access '/data/shop-large.dev.harness.test': No such file or directory |
| + row: [] |
| + timers: [] |
| + ie lease {"owner":"sophie:1389","generation":1,"expires_at":"2026-10-20T22:08:22.248008893Z"}; uk lease absent |
| +== arm a once-timer in 45 s (21:57:42.083Z) |
| +{"at":"2026-10-20T21:58:27Z","due_at":1792533507,"name":"once-1"} |
| + |
| +== start (21:57:42.231Z) |
| + try 1 21:57:46.483Z exit=0: Moving shop-large.dev.harness.test from ie to uk. shop-large.dev.harness.test is offline from now until the move opens or is aborted. Waiting for ada-letterpress in ie to confirm it has stopped shop-large.dev.harness.test. shop-large.dev.harness.test is offline. Every box in ie has stopped shop-larg |
| +start exit=0 at 21:57:46.487Z took 4.3s |
| +== seal (21:57:46.491Z) |
| + try 1 21:58:55.626Z exit=0: Copying shop-large.dev.harness.test to uk. Copied and checked. Run carlos move offline flip --host shop-large.dev.harness.test to start it in uk. |
| +seal exit=0 at 21:58:55.631Z took 69.1s |
| +== flip (21:58:55.637Z) |
| + try 1 22:00:20.651Z exit=0: Stopped in ie. Starting in uk. uk holds everything shop-large.dev.harness.test needs. Nothing is served yet. Run carlos move offline open --host shop-large.dev.harness.test to serve it from uk. |
| +flip exit=0 at 22:00:20.655Z took 85.0s |
| +== open (22:00:20.659Z) |
| + try 1 22:00:24.795Z exit=0: this move is offline-open. run carlos move offline status --host shop-large.dev.harness.test to see what it waits for Waiting for ada-letterpress to clean up shop-large.dev.harness.test. shop-large.dev.harness.test is served from uk. Run carlos move offline finish --host shop-large.dev.harness.test |
| +open exit=0 at 22:00:24.798Z took 4.1s |
| +== dennis before the first request (22:00:24.802Z): config.env must be there (defect 6) |
| +-rw------- 1 carlos carlos 289 22:00:20.343469089 /data/shop-large.dev.harness.test.config.env |
| +== first request on dennis (22:00:24.889Z) |
| + dennis /ready [404 0.005787s] 22:00:24.991Z |
| + dennis /ready [503 30.006557s] 22:00:56.109Z |
| + dennis /ready [503 30.007195s] 22:01:27.250Z |
| + dennis /ready [503 30.008343s] 22:01:58.585Z |
| + dennis /ready [503 30.006905s] 22:02:29.702Z |
| +dennis /ready [200 7.202746s] at 22:02:38.017Z: OFFLINE 295.8s since start |
| +{ |
| + "host": "shop-large.dev.harness.test", |
| + "region": "uk", |
| + "region_from": "CARLOS_REGION", |
| + "mode": "normal", |
| + "boot_id": "d6e42da16af84f8b", |
| + "rows": [ |
| + { |
| + "kind": "bg", |
| + "region": "home", |
| + "count": 2886, |
| + "min_seq": 1, |
| + "max_seq": 2886 |
| + }, |
| + { |
| + "kind": "bg", |
| + "region": "uk", |
| + "count": 1, |
| + "min_seq": 2887, |
| + "max_seq": 2887 |
| + }, |
| + { |
| + "kind": "once", |
| + "region": "uk", |
| + "count": 1, |
| + "min_seq": 0, |
| + "max_seq": 0 |
| + }, |
| + { |
| + "kind": "tick", |
| + "region": "home", |
| + "count": 24, |
| + "min_seq": 0, |
| + "max_seq": 0 |
| + }, |
| + { |
| + "kind": "tick", |
| + "region": "uk", |
| + "count": 1, |
| + "min_seq": 0, |
| + "max_seq": 0 |
| + }, |
| + { |
| + "kind": "worker", |
| + "region": "home", |
| + "count": 1459, |
| + "min_seq": 1, |
| + "max_seq": 1459 |
| + }, |
| + { |
| + "kind": "worker", |
| + "region": "uk", |
| + "count": 1, |
| + "min_seq": 1460, |
| + "max_seq": 1460 |
| + } |
| + ], |
| + "fired": [ |
| + { |
| + "kind": "once", |
| + "name": "once-1", |
| + "due_at": 1792533507, |
| + "count": 1 |
| + }, |
| + { |
| + "kind": "tick", |
| + "name": "every-minute", |
| + "due_at": 1792531860, |
| + "count": 1 |
| + }, |
| + { |
| + "kind": "tick", |
| + "name": "every-minute", |
| + "due_at": 1792532100, |
| + "count": 1 |
| + }, |
| + { |
| + "kind": "tick", |
| + "name": "every-minute", |
| + "due_at": 1792532160, |
| + "count": 1 |
| + }, |
| + { |
| + "kind": "tick", |
| + "name": "every-minute", |
| + |
| +-- backhaul through sophie's edge: [200 0.010390s] at 22:02:38.268Z, 0.1s after the first 200 on dennis (1 tries; none before) |
| +-- backhaul through leo's edge: [200 0.011327s] at 22:02:38.401Z, 0.1s after the first 200 on dennis (1 tries; none before) |
| +-- sophie after open |
| + env file: absent key-sha |
| + instance dir: |
| + .instances dir: |
| + legacy /data/shop-large.dev.harness.test: ls: cannot access '/data/shop-large.dev.harness.test': No such file or directory |
| + row: [('moving', '', '', '/data/instances/shop-large.dev.harness.test/shop-large.dev.harness.test.db', '')] |
| + timers: [('shop-large.dev.harness.test', 'every-minute', 'schedule', 'every 1m', '/tick', 1792533480, 1792533421, '200', 29, 0, 1792531809, 1792533480, '647b32f515af863e7f7bf9986010d473', '', 0, '', ''), ('shop-large.dev.harness.test', 'once-1', 'once', 'at 1792533507', '/once', 1792533507, 0, '', 0, 0, 1792533462, 1792533507, '9879c7b78ccd6eb33cc3eed61cf442c6', '', 0, '', '')] |
| + parked: /data/.offline-moves/shop-large.dev.harness.test/mv_nxl7zcylejgjyaqh22jyoz5xey/sealer |
| + parked: /data/.offline-moves/shop-large.dev.harness.test/mv_nxl7zcylejgjyaqh22jyoz5xey/source |
| +-- dennis after open |
| + env file: /data/shop-large.dev.harness.test.env 2b08fa3b2da90a42 key-sha eac6dfc1cd29bf79 |
| + config.env /data/shop-large.dev.harness.test.config.env (22:00:20.343): HARNESS_NOTE MOVEPROBE_ACCESS_KEY MOVEPROBE_BUCKET MOVEPROBE_ENDPOINT MOVEPROBE_REGION MOVEPROBE_SECRET MOVEPROBE_SECRET_KEY |
| + files: /data/shop-large.dev.harness.test/instance.key sha c2b23e645e52fdf0 |
| + files tree /data/shop-large.dev.harness.test: a7d709cc6c115538 (102 files, 101M) |
| + db /data/shop-large.dev.harness.test.db 637M |
| + instance dir: |
| + .instances dir: ls: cannot access '/data/.instances/shop-large.dev.harness.test': No such file or directory |
| + legacy /data/shop-large.dev.harness.test: instance.key media report.token |
| + row: [('ready', '', 'uk', '/data/shop-large.dev.harness.test.db', '')] |
| + timers: [('shop-large.dev.harness.test', 'every-minute', 'schedule', 'every 1m', '/tick', 1792533780, 1792533751, '200', 6752, 0, 1792531809, 1792533780, '647b32f515af863e7f7bf9986010d473', '', 0, '', ''), ('shop-large.dev.harness.test', 'once-1', 'once', 'at 1792533507', '/once', 0, 1792533758, '200', 11, 0, 1792533462, 0, '9879c7b78ccd6eb33cc3eed61cf442c6', '', 0, '', '')] |
| +Moving shop-large.dev.harness.test from ie to uk. |
| +shop-large.dev.harness.test is served from uk. Run carlos move offline finish --host shop-large.dev.harness.test once ie has cleaned up. |
| +Item db is ok. |
| +Item db-tail is ok: every source box is at or below TXID 184. |
| +Item build-state is ok: the request record is in the build state the replica prefix carries. |
| +Item lease is ok. |
| +Item files is ok. |
| +Item env is ok. |
| +Item key-adoption is ok. |
| +Item config is ok. |
| +Item env-pin is ok. |
| +Item release is ok. |
| +Item store is unchecked. |
| +Item timers is ok. |
| +Item next-due is ok. |
| +Item replication is ok. |
| +Item row is ok. |
| +Item routing is unchecked. |
| +Item source-retire is unchecked. |
| +Item sidecar is ok: no source box keeps a sidecar file. |
| +Item restore-verify is unchecked. |
| +Item warm-cache is ok. |
| +Item receipts is unchecked. |
| +Item class is ok. |
| +ada-letterpress is sealed as the sealer. Its report is not current. |
| +ada-letterpress is parked as the source. Its report is not current. |
| +kathleen-juggling is parked as the source. Its report is not current. |
| +noor-marquetry is opened as the target. Its report is not current. |
| +ada-letterpress left /data/instances/shop-large.dev.harness.test in place: the mount view of the instance directory: it stays, empty. |
| +ada-letterpress left /data/.instances/shop-large.dev.harness.test in place: the instance directory stays, empty. |
| +ada-letterpress left /opt/carlos/live/shop-large.dev.harness.test in place: root-owned: listed for the census, not renamed. |
| +== finish (22:02:39.200Z) |
| + try 1 22:03:34.464Z exit=0: Waiting for ada-letterpress to clean up shop-large.dev.harness.test. Moved. shop-large.dev.harness.test is served from uk. It was offline for 158 seconds. |
| +finish exit=0 at 22:03:34.471Z took 55.3s |
| +Move mv_nxl7zcylejgjyaqh22jyoz5xey of shop-large.dev.harness.test is done. |
| +-- sophie after finish |
| + env file: absent key-sha |
| + instance dir: |
| + .instances dir: |
| + legacy /data/shop-large.dev.harness.test: ls: cannot access '/data/shop-large.dev.harness.test': No such file or directory |
| + row: [] |
| + timers: [('shop-large.dev.harness.test', 'every-minute', 'schedule', 'every 1m', '/tick', 1792533480, 1792533421, '200', 29, 0, 1792531809, 1792533480, '647b32f515af863e7f7bf9986010d473', '', 0, '', ''), ('shop-large.dev.harness.test', 'once-1', 'once', 'at 1792533507', '/once', 1792533507, 0, '', 0, 0, 1792533462, 1792533507, '9879c7b78ccd6eb33cc3eed61cf442c6', '', 0, '', '')] |
| + parked: /data/.offline-moves/shop-large.dev.harness.test/mv_nxl7zcylejgjyaqh22jyoz5xey/sealer |
| + parked: /data/.offline-moves/shop-large.dev.harness.test/mv_nxl7zcylejgjyaqh22jyoz5xey/source |
| +-- dennis after finish |
| + env file: /data/shop-large.dev.harness.test.env 2b08fa3b2da90a42 key-sha eac6dfc1cd29bf79 |
| + config.env /data/shop-large.dev.harness.test.config.env (22:00:20.343): HARNESS_NOTE MOVEPROBE_ACCESS_KEY MOVEPROBE_BUCKET MOVEPROBE_ENDPOINT MOVEPROBE_REGION MOVEPROBE_SECRET MOVEPROBE_SECRET_KEY |
| + files: /data/shop-large.dev.harness.test/instance.key sha c2b23e645e52fdf0 |
| + files tree /data/shop-large.dev.harness.test: a7d709cc6c115538 (102 files, 101M) |
| + db /data/shop-large.dev.harness.test.db 637M |
| + instance dir: |
| + .instances dir: ls: cannot access '/data/.instances/shop-large.dev.harness.test': No such file or directory |
| + legacy /data/shop-large.dev.harness.test: instance.key media report.token |
| + row: [('ready', '', 'uk', '/data/shop-large.dev.harness.test.db', '')] |
| + timers: [('shop-large.dev.harness.test', 'every-minute', 'schedule', 'every 1m', '/tick', 1792533840, 1792533781, '200', 2571, 0, 1792531809, 1792533840, '647b32f515af863e7f7bf9986010d473', '', 0, '', ''), ('shop-large.dev.harness.test', 'once-1', 'once', 'at 1792533507', '/once', 0, 1792533758, '200', 11, 0, 1792533462, 0, '9879c7b78ccd6eb33cc3eed61cf442c6', '', 0, '', '')] |
| +-- leo after finish |
| + env file: absent key-sha |
| + instance dir: |
| + .instances dir: ls: cannot access '/data/.instances/shop-large.dev.harness.test': No such file or directory |
| + legacy /data/shop-large.dev.harness.test: ls: cannot access '/data/shop-large.dev.harness.test': No such file or directory |
| + row: [] |
| + timers: [] |
| + ie lease {"owner":"moved:mv_nxl7zcylejgjyaqh22jyoz5xey","generation":1,"expires_at":"1970-01-01T00:00:00Z"}; uk lease {"owner":"dennis:1359","generation":1,"expires_at":"2026-10-20T22:15:26.188005048Z"} |
| +{"region":"uk","move":null,"last_move":{"id":"mv_nxl7zcylejgjyaqh22jyoz5xey","outcome":"done","at":"2026-10-20T22:03:33.921872355Z","offline_s":158.355124646}} |
| +== sampler (changes only) |
| +sampler: 21:57:42.231Z ie {"owner":"sophie:1389","generation":1,"expires_at":"2026-10-20T22:08:22.248008893Z"} uk absent sophie:.instances=[.activator-shop-large.dev.harness.test.db-litestream launch.run.json shop-large.dev.harness.test shop-large.dev.harness.test.config.env shop-large.dev.harness.test.db shop-large.dev.harness.test.db-shm shop-large.dev.harness.test.db-wal shop-large.dev.harness.test.env ] |
| +sampler: 21:57:48.923Z ie absent uk absent sophie:.instances=[.activator-shop-large.dev.harness.test.db-litestream launch.run.json shop-large.dev.harness.test shop-large.dev.harness.test.config.env shop-large.dev.harness.test.db shop-large.dev.harness.test.db-shm shop-large.dev.harness.test.db-wal shop-large.dev.harness.test.env ] |
| +sampler: 21:57:55.621Z ie {"owner":"moved:mv_nxl7zcylejgjyaqh22jyoz5xey","generation":1,"expires_at":"2026-10-27T21:57:53.105121351Z"} uk absent sophie:.instances=[.activator-shop-large.dev.harness.test.db-litestream launch.run.json shop-large.dev.harness.test shop-large.dev.harness.test.config.env shop-large.dev.harness.test.db shop-large.dev.harness.test.db-shm shop-large.dev.harness.test.db-wal shop-large.dev.harness.test.env ] |
| +sampler: 21:58:54.779Z ie {"owner":"moved:mv_nxl7zcylejgjyaqh22jyoz5xey","generation":1,"expires_at":"2026-10-27T21:58:53.739987187Z"} uk absent sophie:.instances=[.activator-shop-large.dev.harness.test.db-litestream launch.run.json shop-large.dev.harness.test shop-large.dev.harness.test.config.env shop-large.dev.harness.test.db shop-large.dev.harness.test.db-shm shop-large.dev.harness.test.db-wal shop-large.dev.harness.test.env ] |
| +sampler: 21:59:01.462Z ie {"owner":"moved:mv_nxl7zcylejgjyaqh22jyoz5xey","generation":1,"expires_at":"2026-10-27T21:58:56.767832111Z"} uk absent sophie:.instances=[.activator-shop-large.dev.harness.test.db-litestream launch.run.json shop-large.dev.harness.test shop-large.dev.harness.test.config.env shop-large.dev.harness.test.db shop-large.dev.harness.test.db-shm shop-large.dev.harness.test.db-wal shop-large.dev.harness.test.env ] |
| +sampler: 22:00:22.814Z ie {"owner":"moved:mv_nxl7zcylejgjyaqh22jyoz5xey","generation":1,"expires_at":"2026-10-27T22:00:23.124017871Z"} uk absent sophie:.instances=[] |
| +sampler: 22:00:29.435Z ie {"owner":"moved:mv_nxl7zcylejgjyaqh22jyoz5xey","generation":1,"expires_at":"2026-10-27T22:00:23.124017871Z"} uk {"owner":"dennis:1359","generation":1,"expires_at":"2026-10-20T22:15:26.188005048Z"} sophie:.instances=[] |
| +sampler: 22:02:44.397Z ie {"owner":"moved:mv_nxl7zcylejgjyaqh22jyoz5xey","generation":1,"expires_at":"2026-10-27T22:02:40.732097039Z"} uk {"owner":"dennis:1359","generation":1,"expires_at":"2026-10-20T22:15:26.188005048Z"} sophie:.instances=[] |
| +sampler: 22:03:32.100Z ie {"owner":"moved:mv_nxl7zcylejgjyaqh22jyoz5xey","generation":1,"expires_at":"1970-01-01T00:00:00Z"} uk {"owner":"dennis:1359","generation":1,"expires_at":"2026-10-20T22:15:26.188005048Z"} sophie:.instances=[] |
| +== sophie journal for shop-large.dev.harness.test since start |
| +2026-10-20T21:57:42+00:00 sophie carlos[1389]: 2026/10/20 21:57:42 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b6 txid.db=00000000000000b7 |
| +2026-10-20T21:57:42+00:00 sophie carlos[1389]: 2026/10/20 21:57:42 INFO ltx file uploaded system=store db=shop-large.dev.harness.test.db replica=s3 level=0 minTXID=00000000000000b7 maxTXID=00000000000000b7 size=4784 |
| +2026-10-20T21:57:43+00:00 sophie carlos[1389]: 2026/10/20 21:57:43 provision: held shop-large.dev.harness.test moving by mv_nxl7zcylejgjyaqh22jyoz5xey (was ready) |
| +2026-10-20T21:57:43+00:00 sophie carlos[1389]: 2026/10/20 21:57:43 agent: shop-large.dev.harness.test: held for a move; asked root to re-judge what it staged |
| +2026-10-20T21:57:43+00:00 sophie carlos[1389]: 2026/10/20 21:57:43 provision: shop-large.dev.harness.test: row is on the nested layout (db "/data/instances/shop-large.dev.harness.test/shop-large.dev.harness.test.db" socket "/run/carlos/instances/shop-large.dev |
| +2026-10-20T21:57:44+00:00 sophie carlos[1389]: 2026/10/20 21:57:44 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b7 txid.db=00000000000000b8 |
| +2026-10-20T21:57:44+00:00 sophie carlos[1389]: 2026/10/20 21:57:44 INFO ltx file uploaded system=store db=shop-large.dev.harness.test.db replica=s3 level=0 minTXID=00000000000000b8 maxTXID=00000000000000b8 size=4455 |
| +2026-10-20T21:57:44+00:00 sophie carlos[1389]: 2026/10/20 21:57:44 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T21:57:44+00:00 sophie carlos[1389]: 2026/10/20 21:57:44 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T21:57:44+00:00 sophie carlos[1389]: 2026/10/20 21:57:44 activator: shop-large.dev.harness.test hibernated |
| +2026-10-20T21:57:45+00:00 sophie carlos[1389]: 2026/10/20 21:57:45 provision: shop-large.dev.harness.test: row is on the nested layout (db "/data/instances/shop-large.dev.harness.test/shop-large.dev.harness.test.db" socket "/run/carlos/instances/shop-large.dev |
| +2026-10-20T21:57:45+00:00 sophie carlos[1389]: 2026/10/20 21:57:45 layout: shop-large.dev.harness.test: deferred -> status moving |
| +2026-10-20T21:57:52+00:00 sophie carlos[1389]: 2026/10/20 21:57:52 provision: shop-large.dev.harness.test: row is on the nested layout (db "/data/instances/shop-large.dev.harness.test/shop-large.dev.harness.test.db" socket "/run/carlos/instances/shop-large.dev |
| +2026-10-20T21:57:52+00:00 sophie carlos[1389]: 2026/10/20 21:57:52 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T21:57:52+00:00 sophie carlos[1389]: 2026/10/20 21:57:52 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T21:57:52+00:00 sophie carlos[1389]: 2026/10/20 21:57:52 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T21:57:52+00:00 sophie carlos[1389]: 2026/10/20 21:57:52 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T21:57:53+00:00 sophie carlos[1389]: 2026/10/20 21:57:53 moves: shop-large.dev.harness.test: offline move mv_nxl7zcylejgjyaqh22jyoz5xey: checkpointed the local database (position 184, replica 184) |
| +2026-10-20T21:58:53+00:00 sophie carlos[1389]: 2026/10/20 21:58:53 moves: shop-large.dev.harness.test: offline move mv_nxl7zcylejgjyaqh22jyoz5xey: sealed at TXID 184 (186 staged objects, files sha256:58ed1d6d6c3904a1cc0defc203273571095c184ef0015a5e20bb722d21e6 |
| +2026-10-20T21:58:53+00:00 sophie carlos[1389]: 2026/10/20 21:58:53 moves: shop-large.dev.harness.test: offline move mv_nxl7zcylejgjyaqh22jyoz5xey: sealed |
| +2026-10-20T21:58:53+00:00 sophie carlos[1389]: 2026/10/20 21:58:53 layout: shop-large.dev.harness.test: deferred -> status moving |
| +2026-10-20T21:58:56+00:00 sophie carlos[1389]: 2026/10/20 21:58:56 provision: not removing shop-large.dev.harness.test (record gone): participant of a running operation |
| +2026-10-20T21:58:56+00:00 sophie carlos[1389]: 2026/10/20 21:58:56 layout: shop-large.dev.harness.test: deferred -> status moving |
| +2026-10-20T22:00:22+00:00 sophie carlos[1389]: 2026/10/20 22:00:22 provision: not removing shop-large.dev.harness.test (record gone): participant of a running operation |
| +2026-10-20T22:00:22+00:00 sophie carlos[1389]: 2026/10/20 22:00:22 moves: shop-large.dev.harness.test: offline move mv_nxl7zcylejgjyaqh22jyoz5xey: wake fence lifted |
| +2026-10-20T22:00:23+00:00 sophie carlos[1389]: 2026/10/20 22:00:23 layout: shop-large.dev.harness.test: deferred -> status moving |
| +2026-10-20T22:02:40+00:00 sophie carlos[1389]: 2026/10/20 22:02:40 provision: not removing shop-large.dev.harness.test (record gone): participant of a running operation |
| +2026-10-20T22:02:40+00:00 sophie carlos[1389]: 2026/10/20 22:02:40 moves: shop-large.dev.harness.test: offline move mv_nxl7zcylejgjyaqh22jyoz5xey: waiting for reconcile to remove the row |
| +2026-10-20T22:02:40+00:00 sophie carlos[1389]: 2026/10/20 22:02:40 layout: shop-large.dev.harness.test: deferred -> status moving |
| +2026-10-20T22:03:29+00:00 sophie carlos[1389]: 2026/10/20 22:03:29 provision: removed retired instance route shop-large.dev.harness.test (record gone) |
| +2026-10-20T22:03:31+00:00 sophie carlos[1389]: 2026/10/20 22:03:31 moves: shop-large.dev.harness.test: offline move mv_nxl7zcylejgjyaqh22jyoz5xey: tombstone lifted |
| +== dennis journal for shop-large.dev.harness.test since start |
| +2026-10-20T22:02:42+00:00 dennis carlos[1359]: 2026/10/20 22:02:42 layout: deferred until asleep -> shop-large.dev.harness.test (awake (stopped for the move in 1h0m0s unless it sleeps first)), shop-small.dev.harness.test (awake (stopped for the move in 34m58s |
| +2026-10-20T22:02:48+00:00 dennis carlos[1359]: 2026/10/20 22:02:48 INFO detected database behind replica system=store db=shop-large.dev.harness.test.db db_txid=0000000000000000 replica_txid=00000000000000b8 |
| +2026-10-20T22:02:48+00:00 dennis carlos[1359]: 2026/10/20 22:02:48 INFO fetched latest L0 file from replica system=store db=shop-large.dev.harness.test.db min_txid=00000000000000b8 max_txid=00000000000000b8 |
| +2026-10-20T22:02:48+00:00 dennis carlos[1359]: 2026/10/20 22:02:48 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:02:50+00:00 dennis carlos[1359]: 2026/10/20 22:02:50 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:02:52+00:00 dennis carlos[1359]: 2026/10/20 22:02:52 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:02:54+00:00 dennis carlos[1359]: 2026/10/20 22:02:54 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:02:56+00:00 dennis carlos[1359]: 2026/10/20 22:02:56 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:02:58+00:00 dennis carlos[1359]: 2026/10/20 22:02:58 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:03:00+00:00 dennis carlos[1359]: 2026/10/20 22:03:00 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:03:02+00:00 dennis carlos[1359]: 2026/10/20 22:03:02 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:03:04+00:00 dennis carlos[1359]: 2026/10/20 22:03:04 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:03:06+00:00 dennis carlos[1359]: 2026/10/20 22:03:06 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:03:08+00:00 dennis carlos[1359]: 2026/10/20 22:03:08 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:03:10+00:00 dennis carlos[1359]: 2026/10/20 22:03:10 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:03:12+00:00 dennis carlos[1359]: 2026/10/20 22:03:12 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:03:14+00:00 dennis carlos[1359]: 2026/10/20 22:03:14 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:03:16+00:00 dennis carlos[1359]: 2026/10/20 22:03:16 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:03:18+00:00 dennis carlos[1359]: 2026/10/20 22:03:18 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:03:20+00:00 dennis carlos[1359]: 2026/10/20 22:03:20 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:03:22+00:00 dennis carlos[1359]: 2026/10/20 22:03:22 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:03:24+00:00 dennis carlos[1359]: 2026/10/20 22:03:24 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:03:26+00:00 dennis carlos[1359]: 2026/10/20 22:03:26 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:03:28+00:00 dennis carlos[1359]: 2026/10/20 22:03:28 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:03:30+00:00 dennis carlos[1359]: 2026/10/20 22:03:30 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:03:32+00:00 dennis carlos[1359]: 2026/10/20 22:03:32 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:03:33+00:00 dennis carlos[1359]: 2026/10/20 22:03:33 INFO compaction complete system=store db=shop-large.dev.harness.test.db level=1 txid.min=0000000000000001 txid.max=00000000000000b8 size=511266909 |
| +2026-10-20T22:03:34+00:00 dennis carlos[1359]: 2026/10/20 22:03:34 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000b8 |
| +2026-10-20T22:03:35+00:00 dennis carlos[1359]: 2026/10/20 22:03:35 INFO replica sync system=store db=shop-large.dev.harness.test.db replica=s3 txid.replica=00000000000000b8 txid.db=00000000000000bb |
| +2026-10-20T22:03:36+00:00 dennis carlos[1359]: 2026/10/20 22:03:36 layout: deferred until asleep -> shop-large.dev.harness.test (awake (stopped for the move in 59m6s unless it sleeps first)), shop-small.dev.harness.test (awake (stopped for the move in 34m5s un |
| +/dev/nvme0n1p1 59G 55G 1.5G 98% / |
| +[hssm] i-06a1ec144aa5226fb c765512b-e20b-4455-b6ad-f022818111ba Success exit=0 |