amadan
CI failed 3m 55s
- Branch
self-serve-moves-2- Commit
d0227a68claude
- Runner
ci-aws~paul@keymail.dev/ip-172-31-26-96- Reported by
ci-aws~paul@keymail.dev/ip-172-31-26-96- Queued
- Started
- Last change
- Attempt
- 1 of 2
Steps run in order, one after another.
-
gate failed 3m 54s
.amadan/test-gate.sh gate: 2026-10-24T00:17:32Z on aarch64, 32 cpus, 62142 MB available, go1.26.8 linux/arm64 gate: +0s start launchctx gate: +0s start vet gate: +0s start gofmt gate: +2s start rest gate: +189s every split-package job finished; e2e stage next gate: +190s start e2e@0 gate: +190s start e2e@1 gate: +190s start e2e@2 gate: +190s start e2e@3 gate: +190s start e2e-vet gate: +234s e2e stage finished === launchctx (5 s) ok amadan.net/carlos/platform.git/launchctx 0.023s ? amadan.net/carlos/platform.git/launchctx/controlname [no test files] ? amadan.net/carlos/platform.git/launchctx/wireid [no test files] === vet (16 s) === gofmt (1 s) gofmt -l .: clean === rest (187 s) ? github.com/carlosframework/platform/cmd/changelog [no test files] ok github.com/carlosframework/platform/cmd/console 3.630s ok github.com/carlosframework/platform/cmd/moveprobe 11.767s ok github.com/carlosframework/platform/cmd/zddprobe 0.710s ok github.com/carlosframework/platform/internal/activator 59.862s ok github.com/carlosframework/platform/internal/agentsock 0.019s ok github.com/carlosframework/platform/internal/alerting 0.004s ok github.com/carlosframework/platform/internal/analytics 0.030s ok github.com/carlosframework/platform/internal/appconfig 2.613s ok github.com/carlosframework/platform/internal/apperr 0.096s ok github.com/carlosframework/platform/internal/applog 0.152s ok github.com/carlosframework/platform/internal/awsinv 0.005s ok github.com/carlosframework/platform/internal/boxchoice 0.007s ok github.com/carlosframework/platform/internal/boxid 0.845s ? github.com/carlosframework/platform/internal/boxid/boxidtest [no test files] ? github.com/carlosframework/platform/internal/boxid/internal/signtest [no test files] ok github.com/carlosframework/platform/internal/boxvolumes 0.010s ok github.com/carlosframework/platform/internal/caps 0.026s ok github.com/carlosframework/platform/internal/certwarm 0.004s ok github.com/carlosframework/platform/internal/cgroup 0.698s ok github.com/carlosframework/platform/internal/changelog 0.385s ok github.com/carlosframework/platform/internal/claimactivation 0.881s ok github.com/carlosframework/platform/internal/clamd 0.056s ? github.com/carlosframework/platform/internal/cliflag [no test files] ? github.com/carlosframework/platform/internal/controlname [no test files] ? github.com/carlosframework/platform/internal/controlsock [no test files] ok github.com/carlosframework/platform/internal/dbdoor 5.903s ok github.com/carlosframework/platform/internal/dbdoor/dbdoortest 0.018s ok github.com/carlosframework/platform/internal/dbverify 0.005s ok github.com/carlosframework/platform/internal/dnsrec 0.003s ok github.com/carlosframework/platform/internal/docsite 0.018s ok github.com/carlosframework/platform/internal/domains 2.711s ok github.com/carlosframework/platform/internal/doorkey 0.007s ok github.com/carlosframework/platform/internal/economics 0.004s ok github.com/carlosframework/platform/internal/edge 45.526s ? github.com/carlosframework/platform/internal/edgeproof [no test files] ok github.com/carlosframework/platform/internal/edgesfile 0.010s ok github.com/carlosframework/platform/internal/fleetapi 0.148s ok github.com/carlosframework/platform/internal/gate 1.665s ok github.com/carlosframework/platform/internal/genselect 0.019s ok github.com/carlosframework/platform/internal/geo 0.036s ok github.com/carlosframework/platform/internal/geoip 0.051s ok github.com/carlosframework/platform/internal/historyreadback 0.184s ok github.com/carlosframework/platform/internal/host 161.467s ok github.com/carlosframework/platform/internal/hostactors 6.121s ok github.com/carlosframework/platform/internal/hostactors/awsmembership 0.005s ok github.com/carlosframework/platform/internal/hostbox 2.702s ok github.com/carlosframework/platform/internal/hostclaim 0.538s ok github.com/carlosframework/platform/internal/hostenv 0.004s ok github.com/carlosframework/platform/internal/hostevidence 0.006s ok github.com/carlosframework/platform/internal/hostexec 4.643s ok github.com/carlosframework/platform/internal/hostfence 0.028s ok github.com/carlosframework/platform/internal/hostfiles 10.449s ok github.com/carlosframework/platform/internal/hostfloor 0.016s ok github.com/carlosframework/platform/internal/hostguard 0.991s ok github.com/carlosframework/platform/internal/hostops 1.661s ? github.com/carlosframework/platform/internal/hostops/fencerelease [no test files] ok github.com/carlosframework/platform/internal/hostops/internal/fencerelease 0.002s ok github.com/carlosframework/platform/internal/hostquarantine 0.004s ok github.com/carlosframework/platform/internal/hostrelocate 0.457s ok github.com/carlosframework/platform/internal/hostresolve 0.126s ok github.com/carlosframework/platform/internal/hostwrite 0.002s ok github.com/carlosframework/platform/internal/infraview 0.002s ok github.com/carlosframework/platform/internal/instancectl 5.213s ok github.com/carlosframework/platform/internal/instanceexec 2.729s ok github.com/carlosframework/platform/internal/instancefs 0.005s ok github.com/carlosframework/platform/internal/instancekey 0.017s ok github.com/carlosframework/platform/internal/instanceupdates 0.024s ok github.com/carlosframework/platform/internal/layout 1.622s ok github.com/carlosframework/platform/internal/lbhealth 0.016s ok github.com/carlosframework/platform/internal/lease 10.085s ok github.com/carlosframework/platform/internal/lease/leasetest 0.391s ok github.com/carlosframework/platform/internal/ledger 0.027s ok github.com/carlosframework/platform/internal/mail 0.170s ok github.com/carlosframework/platform/internal/maintenance 0.079s ok github.com/carlosframework/platform/internal/maintenanceexec 5.517s ok github.com/carlosframework/platform/internal/maintenanceguard 0.070s ok github.com/carlosframework/platform/internal/metrics 0.005s ok github.com/carlosframework/platform/internal/moves 21.131s ok github.com/carlosframework/platform/internal/moves/archive 0.786s ok github.com/carlosframework/platform/internal/objstore 0.203s ok github.com/carlosframework/platform/internal/ops 0.186s ok github.com/carlosframework/platform/internal/park 0.213s ok github.com/carlosframework/platform/internal/plans 0.009s ok github.com/carlosframework/platform/internal/privatefile 0.004s 2026/10/24 00:17:59 provision: listing apps/bab/blog/instances/: list refused — its records are unseen this pass 2026/10/24 00:17:59 provision: listing apps/bab/: listdirs refused — its records are unseen this pass 2026/10/24 00:17:59 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:17:59 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:17:59 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:17:59 provision: listing apps/bac/tito/instances/: list refused — its records are unseen this pass 2026/10/24 00:17:59 provision: listing apps/bac/tito/instances/: list refused — its records are unseen this pass 2026/10/24 00:17:59 provision: skipping record apps/bac/tito/instances/one.example.com.json: unreadable: read refused 2026/10/24 00:17:59 provision: listing apps/bac/tito/instances/: list refused — its records are unseen this pass 2026/10/24 00:17:59 provision: an app's records could not be listed: they are skipped 2026/10/24 00:17:59 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"]) 2026/10/24 00:17:59 provision: remove retired instance route shop.example: provision: claim admission unavailable 2026/10/24 00:17:59 provision: remove retired instance route shop.example: provision: claim admission unavailable 2026/10/24 00:17:59 provision: remove retired sidecar route worker.sidecar.internal: provision: claim admission unavailable 2026/10/24 00:17:59 provision: remove retired sidecar route worker.sidecar.internal: provision: claim admission unavailable 2026/10/24 00:17:59 provision: claim install acme.go.tito.io: provision: claim admission unavailable 2026/10/24 00:17:59 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:17:59 provision: claim install acme.go.tito.io: registry: claim row ownership changed or unknown 2026/10/24 00:17:59 provision: claim install bot.eleven.bac.sidecar.internal: provision: claim admission unavailable 2026/10/24 00:17:59 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["run"]) 2026/10/24 00:17:59 provision: claim install bot.eleven.bac.sidecar.internal: registry: claim row ownership changed or unknown 2026/10/24 00:17:59 provision: refusing to mint route acme.go.tito.io (app "tito" acct "bac"): this box declares CARLOS_INBOUND=none (no-inbound) and cannot serve routed instance traffic 2026/10/24 00:17:59 provision: acme.go.tito.io: not minting this pass: the box's layout intent could not be read: no sealed config 2026/10/24 00:17:59 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:17:59 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:17:59 provision: REFUSING to mint sidecar bot.eleven.bac.sidecar.internal: bot.eleven.bac.sidecar.internal-sidecar.env is staged on this box — the legacy file co-spawn and a native sidecar row are two mechanisms for one host; remove the file to migrate this host onto sidecar records 2026/10/24 00:18:00 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["run"]) 2026/10/24 00:18:00 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["run"]) 2026/10/24 00:18:00 provision: refusing to mint route acme.go.tito.io (app "tito" acct "bac"): this box declares CARLOS_INBOUND=none (no-inbound) and cannot serve routed instance traffic 2026/10/24 00:18:00 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: REFUSING to mint sidecar bot.eleven.bac.sidecar.internal: bot.eleven.bac.sidecar.internal-sidecar.env is staged on this box — the legacy file co-spawn and a native sidecar row are two mechanisms for one host; remove the file to migrate this host onto sidecar records 2026/10/24 00:18:00 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["run"]) 2026/10/24 00:18:00 provision: acme.go.tito.io exists but is not the route this record describes (app "other"/"tito") — leaving it alone 2026/10/24 00:18:00 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: claim install acme.go.tito.io: registry: claim row ownership changed or unknown 2026/10/24 00:18:00 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: claim install acme.go.tito.io: registry: claim row ownership changed or unknown 2026/10/24 00:18:00 provision: bot.eleven.bac.sidecar.internal exists but is not a sidecar row this pass owns (kind "sidecar" provisioned false app "eleven" acct "bac") — leaving it alone 2026/10/24 00:18:00 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["run"]) 2026/10/24 00:18:00 provision: claim install bot.eleven.bac.sidecar.internal: registry: claim row ownership changed or unknown 2026/10/24 00:18:00 provision: acme.go.tito.io exists but is not the route this record describes (app "other"/"tito") — leaving it alone 2026/10/24 00:18:00 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: adopted existing route acme.go.tito.io (app "tito" acct "bac") — this record now owns its channel and environment 2026/10/24 00:18:00 provision: bot.eleven.bac.sidecar.internal exists but is not a sidecar row this pass owns (kind "sidecar" provisioned false app "eleven" acct "bac") — leaving it alone 2026/10/24 00:18:00 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"]) 2026/10/24 00:18:00 provision: skipping record apps/bab/shop/instances/shop.example.com.json: unreadable: dirstore: apps/bab/shop/instances/shop.example.com.json: file does not exist 2026/10/24 00:18:00 provision: skipping record apps/bab/shop/instances/shop.example.com.json: unreadable: dirstore: apps/bab/shop/instances/shop.example.com.json: file does not exist 2026/10/24 00:18:00 provision: skipping record apps/bab/shop/instances/shop.example.com.json: unreadable: file does not exist 2026/10/24 00:18:00 provision: skipping record apps/bab/shop/instances/shop.example.com.json: unreadable: file does not exist 2026/10/24 00:18:00 provision: skipping record apps/bab/shop/instances/broken.example.com.json: unreadable: permission denied 2026/10/24 00:18:00 provision: skipping record apps/bab/shop/instances/broken.example.com.json: malformed: provision: record bab/shop/broken.example.com: provision: record is corrupt: provision: invalid maintenance projection or protected fields 2026/10/24 00:18:00 provision: skipping record apps/bab/shop/instances/broken.example.com.json: malformed: provision: record bab/shop/broken.example.com: provision: record is corrupt: provision: invalid maintenance projection or protected fields 2026/10/24 00:18:00 provision: skipping record apps/bab/shop/instances/shop.example.com.json: malformed: provision: invalid maintenance projection or protected fields 2026/10/24 00:18:00 provision: skipping record apps/bab/shop/instances/shop.example.com.json: malformed: provision: invalid maintenance projection or protected fields 2026/10/24 00:18:00 provision: skipping record apps/bab/shop/instances/shop.example.com.json: malformed: provision: invalid maintenance projection or protected fields 2026/10/24 00:18:00 provision: skipping record apps/bab/shop/instances/shop.example.com.json: malformed: provision: invalid maintenance projection or protected fields 2026/10/24 00:18:00 provision: skipping record apps/bab/shop/instances/shop.example.com.json: disagrees with its own path (host="shop.example.com" account="bab" app="other") 2026/10/24 00:18:00 provision: skipping record apps/bab/shop/instances/shop.example.com.json: disagrees with its own path (host="shop.example.com" account="bab" app="other") 2026/10/24 00:18:00 provision: skipping record apps/bab/shop/instances/shop.example.com.json: disagrees with its own path (host="shop.example.com" account="bab" app="other") 2026/10/24 00:18:00 provision: skipping record apps/bab/shop/instances/shop.example.com.json: disagrees with its own path (host="shop.example.com" account="bab" app="other") 2026/10/24 00:18:00 provision: skipping record apps/bab/shop/instances/shop.example.com.json: unreadable: permission denied 2026/10/24 00:18:00 provision: skipping record apps/bab/shop/instances/shop.example.com.json: unreadable: permission denied 2026/10/24 00:18:00 provision: skipping record apps/bab/shop/instances/shop.example.com.json: malformed: provision: record bab/shop/shop.example.com: provision: record is corrupt: provision: invalid maintenance projection or protected fields 2026/10/24 00:18:00 provision: skipping record apps/bab/shop/instances/shop.example.com.json: malformed: provision: record bab/shop/shop.example.com: provision: record is corrupt: provision: invalid maintenance projection or protected fields 2026/10/24 00:18:00 provision: skipping record apps/bab/shop/instances/shop.example.com.json: unreadable: permission denied 2026/10/24 00:18:00 provision: skipping record apps/bab/shop/instances/shop.example.com.json: unreadable: permission denied 2026/10/24 00:18:00 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: skipping record apps/bac/tito/instances/acme.go.tito.io.json: malformed: provision: record bac/tito/acme.go.tito.io: provision: record is corrupt: provision: invalid maintenance projection or protected fields 2026/10/24 00:18:00 provision: skipping record apps/bac/tito/instances/new.go.tito.io.json: malformed: provision: record bac/tito/new.go.tito.io: provision: record is corrupt: provision: invalid maintenance projection or protected fields 2026/10/24 00:18:00 provision: not removing acme.go.tito.io: its record exists but could not be read cleanly this pass 2026/10/24 00:18:00 provision: skipping record apps/bac/tito/instances/acme.go.tito.io.json: malformed: provision: invalid maintenance projection or protected fields 2026/10/24 00:18:00 provision: skipping record apps/bac/tito/instances/new.go.tito.io.json: malformed: provision: invalid maintenance projection or protected fields 2026/10/24 00:18:00 provision: minted instance route rk.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route rl.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route ra.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route rd.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route rf.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route rh.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route rj.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route rb.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route rc.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route re.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route rg.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route ri.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route rc.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route rd.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route ra.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route rb.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: skipping record apps/bac/tito/instances/rb.go.tito.io.json: read failed: read refused 2026/10/24 00:18:00 provision: skipping record apps/bac/tito/instances/rb.go.tito.io.json: read failed: read refused 2026/10/24 00:18:00 provision: minted instance route re.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route x.appc.example (app "appc" acct "bac") 2026/10/24 00:18:00 provision: minted instance route x.appd.example (app "appd" acct "bac") 2026/10/24 00:18:00 provision: minted instance route x.appe.example (app "appe" acct "bac") 2026/10/24 00:18:00 provision: minted instance route x.appf.example (app "appf" acct "bac") 2026/10/24 00:18:00 provision: minted instance route x.appg.example (app "appg" acct "bac") 2026/10/24 00:18:00 provision: minted instance route x.apph.example (app "apph" acct "bac") 2026/10/24 00:18:00 provision: minted instance route x.appa.example (app "appa" acct "bac") 2026/10/24 00:18:00 provision: minted instance route x.appb.example (app "appb" acct "bac") 2026/10/24 00:18:00 provision: minted instance route x.appi.example (app "appi" acct "bac") 2026/10/24 00:18:00 provision: minted instance route x.appj.example (app "appj" acct "bac") 2026/10/24 00:18:00 provision: bac/tito is in the trash — not minting instance routes for it 2026/10/24 00:18:00 provision: minted instance route r0039.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0065.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0082.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0004.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0010.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0055.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0086.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0105.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0029.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0041.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0099.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0007.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0049.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0097.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0119.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0128.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0005.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0023.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0034.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0053.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0074.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0016.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0033.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0011.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0125.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0001.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0014.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0058.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0062.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0067.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0124.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0000.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0059.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0101.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0112.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0057.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0025.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0030.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0042.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0064.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0103.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0021.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0032.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0047.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0070.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0072.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0078.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0095.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0107.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0013.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0040.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0051.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0054.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0061.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0077.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0110.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0111.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0017.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0069.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0092.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0126.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0127.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0012.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0015.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0063.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0118.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0129.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0048.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0085.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0094.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0036.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0038.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0050.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0093.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0106.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0104.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0108.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0117.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0009.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0018.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0022.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0080.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0122.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0002.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0035.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0066.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0081.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0083.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0121.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0043.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0008.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0031.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0037.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0045.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0089.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0091.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0114.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0052.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0076.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0088.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0100.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0116.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0071.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0098.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0019.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0020.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0046.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0084.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0090.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0123.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0006.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0026.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0003.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0027.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0087.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0096.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0120.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0044.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0068.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0079.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0102.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0109.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0113.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0024.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0028.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0056.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0073.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0115.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0060.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:00 provision: minted instance route r0075.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:01 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:01 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:01 provision: refusing to mint route acme.go.tito.io (app "tito" acct "bac"): this box declares CARLOS_INBOUND=none (no-inbound) and cannot serve routed instance traffic 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its host incarnation slot no longer vouches for it 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com (record gone (its lifetime is retired)) 2026/10/24 00:18:02 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its host incarnation slot no longer vouches for it 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its host incarnation slot no longer vouches for it 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com (record gone (its lifetime is retired)) 2026/10/24 00:18:02 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its host incarnation slot no longer vouches for it 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its host incarnation slot no longer vouches for it 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com (record gone (its lifetime is retired)) 2026/10/24 00:18:02 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its host incarnation slot no longer vouches for it 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: empty channel 2026/10/24 00:18:02 provision: not removing a.shop.example.com: its record exists but could not be read cleanly this pass 2026/10/24 00:18:02 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its host incarnation slot no longer vouches for it 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com (record gone (its lifetime is retired)) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its lifetime could not be judged: slot read failed 2026/10/24 00:18:02 provision: not removing a.shop.example.com: its record's lifetime could not be judged this pass: slot read failed 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its lifetime could not be judged: provision: host incarnation slot is malformed: control/host-incarnations/a.shop.example.com.json: failed validation 2026/10/24 00:18:02 provision: not removing a.shop.example.com: its record's lifetime could not be judged this pass: provision: host incarnation slot is malformed: control/host-incarnations/a.shop.example.com.json: failed validation 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: a.shop.example.com exists but is not the route this record describes (app "shop"/"cart") — leaving it alone 2026/10/24 00:18:02 provision: not removing a.shop.example.com: its record's lifetime could not be judged this pass: slot read failed 2026/10/24 00:18:02 provision: a.shop.example.com exists but is not the route this record describes (app "shop"/"cart") — leaving it alone 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com (record gone) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its host incarnation slot no longer vouches for it 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com (record gone (its lifetime is retired)) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its host incarnation slot no longer vouches for it 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com held by mv_1 (record gone (its lifetime is retired)) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: not removing a.shop.example.com (record gone): participant of a running operation 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: a.shop.example.com exists but is not the route this record describes (app "shop"/"cart") — leaving it alone 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com (record gone) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "cart" acct "bac") 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: a.shop.example.com exists but is not the route this record describes (app "shop"/"cart") — leaving it alone 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com (record gone) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "cart" acct "bac") 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: a.shop.example.com exists but is not the route this record describes (app "shop"/"cart") — leaving it alone 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com held by mv_1 (record gone) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "cart" acct "bac") 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: a.shop.example.com exists but is not the route this record describes (app "shop"/"cart") — leaving it alone 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com held by mv_1 (record gone) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "cart" acct "bac") 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its host incarnation slot no longer vouches for it 2026/10/24 00:18:02 provision: a.shop.example.com exists but is not the route this record describes (app "shop"/"cart") — leaving it alone 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com (record gone (its lifetime is retired)) 2026/10/24 00:18:02 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its host incarnation slot no longer vouches for it 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "cart" acct "bac") 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its host incarnation slot no longer vouches for it 2026/10/24 00:18:02 provision: a.shop.example.com exists but is not the route this record describes (app "shop"/"cart") — leaving it alone 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com (record gone (its lifetime is retired)) 2026/10/24 00:18:02 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its host incarnation slot no longer vouches for it 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "cart" acct "bac") 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its host incarnation slot no longer vouches for it 2026/10/24 00:18:02 provision: a.shop.example.com exists but is not the route this record describes (app "shop"/"cart") — leaving it alone 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com held by mv_1 (record gone (its lifetime is retired)) 2026/10/24 00:18:02 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its host incarnation slot no longer vouches for it 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "cart" acct "bac") 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its host incarnation slot no longer vouches for it 2026/10/24 00:18:02 provision: a.shop.example.com exists but is not the route this record describes (app "shop"/"cart") — leaving it alone 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com held by mv_1 (record gone (its lifetime is retired)) 2026/10/24 00:18:02 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its host incarnation slot no longer vouches for it 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "cart" acct "bac") 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: duplicate host a.shop.example.com: keeping apps/bac/cart/instances/a.shop.example.com.json, skipping apps/bac/shop/instances/a.shop.example.com.json 2026/10/24 00:18:02 provision: a.shop.example.com exists but is not the route this record describes (app "shop"/"cart") — leaving it alone 2026/10/24 00:18:02 provision: duplicate host a.shop.example.com: keeping apps/bac/cart/instances/a.shop.example.com.json, skipping apps/bac/shop/instances/a.shop.example.com.json 2026/10/24 00:18:02 provision: a.shop.example.com exists but is not the route this record describes (app "shop"/"cart") — leaving it alone 2026/10/24 00:18:02 provision: a.shop.example.com: import sd_23x5x3tr622jsmoopyhrr4z7bq is journaled, but this box cannot act on imports; its row is left alone 2026/10/24 00:18:02 provision: a.shop.example.com: import sd_23x5x3tr622jsmoopyhrr4z7bq is journaled, but this box cannot act on imports; its row is left alone 2026/10/24 00:18:02 provision: a.shop.example.com exists but is not the route this record describes (app "shop"/"cart") — leaving it alone 2026/10/24 00:18:02 provision: a.shop.example.com exists but is not the route this record describes (app "shop"/"cart") — leaving it alone 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: duplicate host a.shop.example.com: keeping apps/bac/cart/instances/a.shop.example.com.json, skipping apps/bac/shop/instances/a.shop.example.com.json 2026/10/24 00:18:02 provision: a.shop.example.com exists but is not the route this record describes (app "shop"/"cart") — leaving it alone 2026/10/24 00:18:02 provision: bac/shop is in the trash — not minting instance routes for it 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com (app in the trash) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com (record gone) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com (record gone) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com (record gone) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com (record gone) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: not removing a.shop.example.com (record gone): participant of a running operation 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: not removing a.shop.example.com (record gone): participant of a running operation 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: not removing a.shop.example.com (record gone): participant of a running operation 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: not removing a.shop.example.com (record gone): participant of a running operation 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com (record gone) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com (record gone) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: not removing a.shop.example.com (record gone): participant of a running operation 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: not removing a.shop.example.com (record gone): participant of a running operation 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com (record gone) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com (record gone) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: duplicate host a.shop.example.com: keeping apps/bac/cart/instances/a.shop.example.com.json, skipping apps/bac/shop/instances/a.shop.example.com.json 2026/10/24 00:18:02 provision: a.shop.example.com exists but is not the route this record describes (app "shop"/"cart") — leaving it alone 2026/10/24 00:18:02 provision: not removing a.shop.example.com: its record's lifetime could not be judged this pass: provision: record bac/shop/a.shop.example.com: provision: record is corrupt: provision: invalid maintenance projection or protected fields 2026/10/24 00:18:02 provision: duplicate host a.shop.example.com: keeping apps/bac/cart/instances/a.shop.example.com.json, skipping apps/bac/shop/instances/a.shop.example.com.json 2026/10/24 00:18:02 provision: a.shop.example.com exists but is not the route this record describes (app "shop"/"cart") — leaving it alone 2026/10/24 00:18:02 provision: bac/shop is in the trash — not minting instance routes for it 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com (app in the trash) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its host incarnation slot no longer vouches for it 2026/10/24 00:18:02 provision: not removing a.shop.example.com (record gone (its lifetime is retired)): its hold changed since this pass listed it; the next pass judges it afresh 2026/10/24 00:18:02 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its host incarnation slot no longer vouches for it 2026/10/24 00:18:02 provision: removed retired instance route a.shop.example.com held by mv_2 (record gone (its lifetime is retired)) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com held landing by mv_abcdefghijklmnopqrstuvwxyz (app "shop" acct "bac") 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: installed lifetime (1, 0) on a.shop.example.com (migration, was (0, 0)) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: installed lifetime (1, 4) on a.shop.example.com (migration, was (0, 4)) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: installed lifetime (2, 0) on a.shop.example.com (recreation, was (1, 9)) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: held a.shop.example.com moving by mv_zyxwvutsrqponmlkjihgfedcba (was ready) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: held a.shop.example.com moving by mv_abcdefghijklmnopqrstuvwxyz (was ready) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: held a.shop.example.com moving by mv_abcdefghijklmnopqrstuvwxyz (was ready) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: held a.shop.example.com moving by mv_abcdefghijklmnopqrstuvwxyz (was ready) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: held a.shop.example.com moving by mv_abcdefghijklmnopqrstuvwxyz (was ready) 2026/10/24 00:18:02 provision: removed a.shop.example.com's row of lifetime 1 held by mv_abcdefghijklmnopqrstuvwxyz (lifetime 2 carries no such move) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: held a.shop.example.com moving by mv_abcdefghijklmnopqrstuvwxyz (was ready) 2026/10/24 00:18:02 provision: a.shop.example.com: the row of lifetime 1 is held by mv_abcdefghijklmnopqrstuvwxyz, which lifetime 2 does not carry; nothing here can remove it, leaving it 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: held a.shop.example.com moving by mv_abcdefghijklmnopqrstuvwxyz (was ready) 2026/10/24 00:18:02 provision: a.shop.example.com: not removing the row of lifetime 1: its hold by mv_abcdefghijklmnopqrstuvwxyz changed since this pass listed it; the next pass judges it afresh 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: held a.shop.example.com moving by mv_abcdefghijklmnopqrstuvwxyz (was ready) 2026/10/24 00:18:02 provision: remove a.shop.example.com's row of lifetime 1 held by mv_abcdefghijklmnopqrstuvwxyz: registry busy 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: held a.shop.example.com moving by mv_abcdefghijklmnopqrstuvwxyz (was ready) 2026/10/24 00:18:02 provision: dry-run: would remove a.shop.example.com's row of lifetime 1 held by mv_abcdefghijklmnopqrstuvwxyz, then mint lifetime 2 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:02 provision: held a.shop.example.com moving by mv_abcdefghijklmnopqrstuvwxyz (was ready) 2026/10/24 00:18:02 provision: removed a.shop.example.com's row of lifetime 1 held by mv_abcdefghijklmnopqrstuvwxyz (lifetime 2 carries no such move) 2026/10/24 00:18:02 provision: minted instance route a.shop.example.com held moving by mv_zyxwvutsrqponmlkjihgfedcba (app "shop" acct "bac") 2026/10/24 00:18:03 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:03 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:03 provision: held a.shop.example.com moving by mv_abcdefghijklmnopqrstuvwxyz (was ready) 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted instance route console.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted service route ha.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted static route site.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted service route svc.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted instance route whole.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted instance route shared.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: converged acme.go.tito.io channel stable -> edge 2026/10/24 00:18:03 provision: converged acme.go.tito.io channel edge -> canary/rehearsal 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: skipping record apps/bac/tito/instances/acme.go.tito.io.json: empty channel 2026/10/24 00:18:03 provision: not removing acme.go.tito.io: its record exists but could not be read cleanly this pass 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: converged acme.go.tito.io environment "" -> "staging" 2026/10/24 00:18:03 provision: converged acme.go.tito.io environment "staging" -> "" 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: converged acme.go.tito.io steering "" -> "latency" 2026/10/24 00:18:03 provision: converged acme.go.tito.io steering "latency" -> "" 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: converged acme.go.tito.io hibernate false -> true 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: converged acme.go.tito.io hibernate true -> false (replication "daily" cleared: only a hibernating route's rung is read) 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: converged acme.go.tito.io status-named false -> true 2026/10/24 00:18:03 provision: converged acme.go.tito.io status-named true -> false 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted service route ha.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: converged ha.go.tito.io upstreams addr "" -> "", addrs "127.0.0.1:7001,127.0.0.1:7002" -> "127.0.0.1:7001,127.0.0.1:7003" (probe "{\"path\":\"/\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}" -> "{\"path\":\"/\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}") 2026/10/24 00:18:03 provision: converged ha.go.tito.io upstreams addr "" -> "", addrs "127.0.0.1:7001,127.0.0.1:7003" -> "127.0.0.1:7003,127.0.0.1:7001" (probe "{\"path\":\"/\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}" -> "{\"path\":\"/\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}") 2026/10/24 00:18:03 provision: converged ha.go.tito.io upstreams addr "" -> "", addrs "127.0.0.1:7003,127.0.0.1:7001" -> "127.0.0.1:7003,127.0.0.1:7001" (probe "{\"path\":\"/\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}" -> "{\"path\":\"/healthz\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}") 2026/10/24 00:18:03 provision: minted service route whole.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: converged whole.go.tito.io upstreams addr "127.0.0.1:9443" -> "", addrs "" -> "127.0.0.1:9443,127.0.0.1:9444" (probe "" -> "{\"path\":\"/\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}") 2026/10/24 00:18:03 provision: converged whole.go.tito.io upstreams addr "" -> "", addrs "127.0.0.1:9443,127.0.0.1:9444" -> "" (probe "{\"path\":\"/\",\"status\":200,\"interval_ms\":2000,\"fail_after\":3,\"rise_after\":2}" -> "") 2026/10/24 00:18:03 provision: whole.go.tito.io: record describes a different route SHAPE than the row (kind "instance"/"service" unit ""/"" socket "/run/carlos/whole.go.tito.io.sock"/"" db "/data/whole.go.tito.io.db"/"") — not converged in place; `carlos remove whole.go.tito.io` purges this route's capability grants (registry.Delete's contract, not a bug) before the next pass re-mints it, so treat that as a last resort, not a routine fix 2026/10/24 00:18:03 provision: minted service route whole.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: acme.go.tito.io exists but is not the route this record describes (app "other"/"tito") — leaving it alone 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: removed retired instance route acme.go.tito.io (record gone) 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: skipping accounts/bac/account.json: malformed account manifest: invalid character 'o' in literal null (expecting 'u') 2026/10/24 00:18:03 provision: removed retired instance route acme.go.tito.io (record gone) 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: manifest bac/tito: read failed: provision: manifest bac/tito: invalid character 'o' in literal null (expecting 'u') 2026/10/24 00:18:03 provision: skipping record apps/bac/tito/instances/acme.go.tito.io.json: malformed: provision: record bac/tito/acme.go.tito.io: provision: record is corrupt: record disagrees with its own path 2026/10/24 00:18:03 provision: skipping record apps/bac/tito/instances/acme$evil.go.tito.io.json: host "acme$evil.go.tito.io" is not a hostname 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: skipping record apps/bac/tito/instances/acme$evil.go.tito.io.json: host "acme$evil.go.tito.io" is not a hostname 2026/10/24 00:18:03 provision: not removing acme$evil.go.tito.io: its record exists but could not be read cleanly this pass 2026/10/24 00:18:03 provision: skipping record apps/bac/tito/instances/bad-service.go.tito.io.json: invalid shape: kind service with no addr: the platform only proxies to a service, so the record must name the upstream it proxies to 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: skipping record apps/bac/tito/instances/bad-service.go.tito.io.json: invalid shape: kind service with no addr: the platform only proxies to a service, so the record must name the upstream it proxies to 2026/10/24 00:18:03 provision: not removing bad-service.go.tito.io: its record exists but could not be read cleanly this pass 2026/10/24 00:18:03 provision: skipping record apps/bac/tito/instances/no-channel.go.tito.io.json: empty channel 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: skipping record apps/bac/tito/instances/no-channel.go.tito.io.json: empty channel 2026/10/24 00:18:03 provision: not removing no-channel.go.tito.io: its record exists but could not be read cleanly this pass 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: skipping record apps/bac/tito/instances/acme.go.tito.io.json: malformed: provision: record bac/tito/acme.go.tito.io: provision: record is corrupt: provision: invalid maintenance projection or protected fields 2026/10/24 00:18:03 provision: not removing acme.go.tito.io: its record exists but could not be read cleanly this pass 2026/10/24 00:18:03 provision: removed retired instance route ghost.go.tito.io (record gone) 2026/10/24 00:18:03 provision: duplicate host dup.go.tito.io: keeping apps/bac/tito/instances/dup.go.tito.io.json, skipping apps/bad/tito/instances/dup.go.tito.io.json 2026/10/24 00:18:03 provision: minted instance route dup.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: bac/tito is in the trash — not minting instance routes for it 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: bac/tito is in the trash — not minting instance routes for it 2026/10/24 00:18:03 provision: removed retired instance route acme.go.tito.io (app in the trash) 2026/10/24 00:18:03 provision: minted static route site.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: bac/tito is in the trash — not minting instance routes for it 2026/10/24 00:18:03 provision: removed retired static route site.go.tito.io (app in the trash) 2026/10/24 00:18:03 provision: trash marker for bac/tito unreadable: release: malformed trash marker for bac/tito: invalid character 'n' looking for beginning of object key string — not minting for this app 2026/10/24 00:18:03 provision: bac/tito is in the trash — not minting instance routes for it 2026/10/24 00:18:03 provision: removed retired instance route acme.go.tito.io (app in the trash) 2026/10/24 00:18:03 provision: bac/tito is in the trash — not minting instance routes for it 2026/10/24 00:18:03 provision: trash marker for bac/tito unreadable: release: malformed trash marker for bac/tito: invalid character 'n' looking for beginning of object key string — not minting for this app 2026/10/24 00:18:03 provision: adopted existing route acme.go.tito.io (app "tito" acct "bac") — this record now owns its channel and environment 2026/10/24 00:18:03 provision: converged acme.go.tito.io channel stable -> edge 2026/10/24 00:18:03 provision: adopted existing route acme.go.tito.io (app "tito" acct "bac") — this record now owns its channel and environment 2026/10/24 00:18:03 provision: converged acme.go.tito.io channel canary/rehearsal -> edge 2026/10/24 00:18:03 provision: converged acme.go.tito.io environment "staging" -> "" 2026/10/24 00:18:03 provision: acme.go.tito.io exists but is not the route this record describes (app "other"/"tito") — leaving it alone 2026/10/24 00:18:03 provision: acme.go.tito.io exists but is not the route this record describes (backing "unit" (box-scoped, not expressible in a record)) — leaving it alone 2026/10/24 00:18:03 provision: acme.go.tito.io exists but is not the route this record describes (tunnel "pi-fleet" (box-scoped, not expressible in a record)) — leaving it alone 2026/10/24 00:18:03 provision: acme.go.tito.io exists but is not the route this record describes (caps "port:25" (box-scoped, not expressible in a record)) — leaving it alone 2026/10/24 00:18:03 provision: acme.go.tito.io exists but is not the route this record describes (platform (owned by the domains reconcile pass)) — leaving it alone 2026/10/24 00:18:03 provision: acme.go.tito.io exists but is not the route this record describes (canary (owned by the domains reconcile pass)) — leaving it alone 2026/10/24 00:18:03 provision: acme.go.tito.io exists but is not the route this record describes (db "/data/other.db"/"/data/acme.go.tito.io.db") — leaving it alone 2026/10/24 00:18:03 provision: adopted existing route acme.go.tito.io (app "tito" acct "bac") — this record now owns its channel and environment 2026/10/24 00:18:03 provision: removed retired instance route acme.go.tito.io (record gone) 2026/10/24 00:18:03 provision: dry-run: would mint instance route acme.go.tito.io (app "tito" acct "bac" channel "stable") 2026/10/24 00:18:03 provision: dry-run: would adopt existing route adopt.go.tito.io (app "tito" acct "bac") — this record would own its channel and environment 2026/10/24 00:18:03 provision: dry-run: would converge adopt.go.tito.io channel stable -> edge 2026/10/24 00:18:03 provision: dry-run: would converge adopt.go.tito.io environment "" -> "staging" 2026/10/24 00:18:03 provision: dry-run: would converge owned.go.tito.io channel stable -> edge 2026/10/24 00:18:03 provision: dry-run: would converge owned.go.tito.io environment "" -> "staging" 2026/10/24 00:18:03 provision: dry-run: would converge owned.go.tito.io hibernate false -> true 2026/10/24 00:18:03 provision: dry-run: would converge owned.go.tito.io seed 2026/10/24 00:18:03 provision: dry-run: would remove retired instance route gone.go.tito.io (record gone) 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: adopted existing route adopt.go.tito.io (app "tito" acct "bac") — this record now owns its channel and environment 2026/10/24 00:18:03 provision: converged adopt.go.tito.io channel stable -> edge 2026/10/24 00:18:03 provision: converged adopt.go.tito.io environment "" -> "staging" 2026/10/24 00:18:03 provision: converged owned.go.tito.io channel stable -> edge 2026/10/24 00:18:03 provision: converged owned.go.tito.io environment "" -> "staging" 2026/10/24 00:18:03 provision: converged owned.go.tito.io hibernate false -> true 2026/10/24 00:18:03 provision: removed retired instance route gone.go.tito.io (record gone) 2026/10/24 00:18:03 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: adopted existing route story.carlosframework.com (app "story" acct "bad") — this record now owns its channel and environment 2026/10/24 00:18:03 provision: story.carlosframework.com exists but is not the route this record describes (kind "instance"/"static") — leaving it alone 2026/10/24 00:18:03 provision: minted static route site.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted service route pub.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted service route pubs.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted instance route unit.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted instance route exec.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted service route loop.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted service route named.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted service route priv.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted static route site.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:03 provision: minted static route pisite.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:04 provision: minted static route site.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:04 provision: removed retired static route site.go.tito.io (record gone) 2026/10/24 00:18:04 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:04 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:04 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:04 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:04 provision: minted instance route acme-uk.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:04 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:04 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:04 provision: converged acme.go.tito.io replicas "ie,uk" -> "uk" 2026/10/24 00:18:04 provision: converged acme.go.tito.io replicas "uk" -> "" 2026/10/24 00:18:04 provision: minted instance route acme-uk.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:04 provision: converged acme-uk.go.tito.io replica mark true -> false 2026/10/24 00:18:04 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:04 provision: minted instance route router.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:04 provision: acme.go.tito.io: row is on the nested layout (db "/data/instances/acme.go.tito.io/acme.go.tito.io.db" socket "/run/carlos/instances/acme.go.tito.io/app.sock"); the record's shared paths are not drift 2026/10/24 00:18:04 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:04 provision: minted instance route router.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:04 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:04 provision: minted instance route router.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:04 provision: minted instance route router.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:04 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:04 provision: acme.go.tito.io: not minting this pass: the box's layout intent could not be read: store down 2026/10/24 00:18:04 provision: router.go.tito.io: not minting this pass: the box's layout intent could not be read: store down 2026/10/24 00:18:04 provision: adopted existing route acme.go.tito.io (app "tito" acct "bac") — this record now owns its channel and environment 2026/10/24 00:18:04 provision: converged acme.go.tito.io channel stable -> edge 2026/10/24 00:18:04 provision: minted instance route acme.go.tito.io held landing by mv_acme (app "tito" acct "bac") 2026/10/24 00:18:04 provision: not removing acme.go.tito.io (record gone): participant of a running operation 2026/10/24 00:18:04 provision: removed retired instance route acme.go.tito.io (record gone) 2026/10/24 00:18:04 provision: minted instance route acme.go.tito.io held landing by mv_acme (app "tito" acct "bac") 2026/10/24 00:18:04 provision: refusing to mint held route acme.go.tito.io (app "tito" acct "bac"): this box declares CARLOS_INBOUND=none (no-inbound) and cannot serve routed instance traffic 2026/10/24 00:18:04 provision: held acme.go.tito.io landing by mv_acme (was ready) 2026/10/24 00:18:04 provision: acme.go.tito.io: held restoring by rb_other, not by mv_acme; leaving it 2026/10/24 00:18:04 provision: acme.go.tito.io: the row is not this pass's (app "other" acct ""); not holding it for mv_acme 2026/10/24 00:18:04 provision: acme.go.tito.io exists but is not the route this record describes (app "other"/"tito") — leaving it alone 2026/10/24 00:18:04 provision: dry-run: would mint instance route acme.go.tito.io held landing by mv_acme (app "tito" acct "bac" channel "stable") 2026/10/24 00:18:04 provision: acme.go.tito.io: not held landing by mv_acme: the row's hold changed under this pass, or the row belongs to a later lifetime; the next pass judges it afresh 2026/10/24 00:18:04 provision: held acme.go.tito.io landing by mv_acme (was ready) 2026/10/24 00:18:04 provision: minted instance route plain.example.com (app "shop" acct "bac") 2026/10/24 00:18:04 provision: minted instance route acme.example.com (app "shop" acct "bac") 2026/10/24 00:18:04 provision: skipping record apps/bac/shop/instances/garbled.shop.example.com.json: malformed: provision: invalid maintenance projection or protected fields 2026/10/24 00:18:04 provision: skipping record apps/bac/shop/instances/liar.shop.example.com.json: disagrees with its own path (host="other.shop.example.com" account="bac" app="shop") 2026/10/24 00:18:04 provision: skipping record apps/bac/shop/instances/garbled.shop.example.com.json: malformed: provision: invalid maintenance projection or protected fields 2026/10/24 00:18:04 provision: skipping record apps/bac/shop/instances/liar.shop.example.com.json: disagrees with its own path (host="other.shop.example.com" account="bac" app="shop") 2026/10/24 00:18:04 provision: skipping record apps/bac/shop/instances/garbled.shop.example.com.json: malformed: provision: invalid maintenance projection or protected fields 2026/10/24 00:18:04 provision: skipping record apps/bac/shop/instances/liar.shop.example.com.json: disagrees with its own path (host="other.shop.example.com" account="bac" app="shop") 2026/10/24 00:18:04 provision: skipping record apps/bac/shop/instances/garbled.shop.example.com.json: malformed: provision: invalid maintenance projection or protected fields 2026/10/24 00:18:04 provision: listing apps/bac/shop/instances/: list refused — its records are unseen this pass 2026/10/24 00:18:05 provision: a.shop.example.com: swapped for import sd_ddqfa7ze4h3drbbdewfco7ywxu (the original route is journaled) 2026/10/24 00:18:05 provision: a.shop.example.com: import sd_ddqfa7ze4h3drbbdewfco7ywxu committed: its gate is cleared and its journal row dropped 2026/10/24 00:18:05 provision: a.shop.example.com: row is on the nested layout (db "/data/instances/a.shop.example.com/a.shop.example.com.db" socket "/run/carlos/instances/a.shop.example.com/app.sock"); the record's shared paths are not drift 2026/10/24 00:18:05 provision: a.shop.example.com: import sd_3plalbxfx5ojhhr2cebcz2loiq rolled back to its original route 2026/10/24 00:18:05 provision: a.shop.example.com: import sd_3plalbxfx5ojhhr2cebcz2loiq: the host's replica is left alone: the record apps/bac/shop/instances/a.shop.example.com.json holds the host 2026/10/24 00:18:05 provision: a.shop.example.com: import sd_3plalbxfx5ojhhr2cebcz2loiq is cleaned up and its journal row retired 2026/10/24 00:18:05 provision: a.shop.example.com: import sd_s5ixbuggdxm2vvyhv5pvqrlkay rolled back to its original route 2026/10/24 00:18:05 provision: a.shop.example.com: import sd_s5ixbuggdxm2vvyhv5pvqrlkay: the host's replica is left alone: the record apps/bac/shop/instances/a.shop.example.com.json holds the host 2026/10/24 00:18:05 provision: a.shop.example.com: import sd_s5ixbuggdxm2vvyhv5pvqrlkay is cleaned up and its journal row retired 2026/10/24 00:18:05 provision: dry-run: would redo a.shop.example.com's swap record CAS for import sd_6ayxp7jgkmb4xqjm4rp6npltje 2026/10/24 00:18:05 provision: dry-run: would clear a.shop.example.com's import gate 2026/10/24 00:18:05 provision: dry-run: would converge a.shop.example.com seed source 2026/10/24 00:18:05 provision: a.shop.example.com: row is on the nested layout (db "/data/instances/a.shop.example.com/a.shop.example.com.db" socket "/run/carlos/instances/a.shop.example.com/app.sock"); the record's shared paths are not drift 2026/10/24 00:18:05 provision: a.shop.example.com: the import's record apps/bac/shop/instances/a.shop.example.com.json is not one this pass can use (connection reset) 2026/10/24 00:18:05 provision: a.shop.example.com: the import's record apps/bac/shop/instances/a.shop.example.com.json is not one this pass can use (provision: invalid maintenance projection or protected fields) 2026/10/24 00:18:05 provision: a.shop.example.com: the import's record apps/bac/shop/instances/a.shop.example.com.json is not one this pass can use (the record under apps/bac/shop/instances/a.shop.example.com.json is apps/bac/shop/instances/b.shop.example.com.json's) 2026/10/24 00:18:05 provision: a.shop.example.com: import sd_edtdqivpka762jegbm2yvhyj4m cannot roll back on this box: the activator is not wired; it stays journaled and gated 2026/10/24 00:18:05 provision: a.shop.example.com: import sd_l26oeghgjwqm53ktw3hk4oswcu cannot roll back on this box: the activator is not wired; it stays journaled and gated 2026/10/24 00:18:05 provision: a.shop.example.com: import sd_5nq77aogxqyfab4mya3yqe7nje is journaled, but this box cannot act on imports; its row is left alone 2026/10/24 00:18:05 provision: a.shop.example.com: import sd_xmkeqhokuo5zgw2h566h4277by is journaled, but this box cannot act on imports; its row is left alone 2026/10/24 00:18:05 provision: a.shop.example.com: import sd_fvebovj3nw32nxndmevxv6ibs4 is journaled, but this box cannot act on imports; its row is left alone 2026/10/24 00:18:05 provision: the import journal could not be read: registry: import journal a.shop.example.com: original row: unexpected end of JSON input (no import is acted on and no route is removed this pass) 2026/10/24 00:18:05 provision: b.shop.example.com: import sd_j56wckd4fv5m2vthj5xxzaw2um left alone this pass: the import journal could not be read 2026/10/24 00:18:05 provision: c.shop.example.com: import sd_u7yjvgqzufswtdgam4k5exj4lm left alone this pass: the import journal could not be read 2026/10/24 00:18:05 provision: a.shop.example.com: import sd_6jnz2mupianqaphvvnkxjees3m committed: its gate is cleared and its journal row dropped 2026/10/24 00:18:05 provision: a.shop.example.com: not converging import sd_dmfvxrsqzvjdzgpaynqq3swbmq: it is swapped but not committed, and the row on this box has no import gate 2026/10/24 00:18:05 provision: a.shop.example.com: not converging import sd_rqrwcqzshht3xxga55upqnu32y: it is swapped but not committed, and the row on this box has no import gate 2026/10/24 00:18:05 provision: a.shop.example.com: not converging import sd_wnpydyigepddmsrvvuprzvnvce: it is placed but not committed, and the row on this box has no import gate 2026/10/24 00:18:05 provision: a.shop.example.com: not converging import sd_2l3gwa22ea7merdw3n3r3st2ou: it is placed but not committed, and the row on this box has no import gate 2026/10/24 00:18:05 provision: a.shop.example.com: row is on the nested layout (db "/data/instances/a.shop.example.com/a.shop.example.com.db" socket "/run/carlos/instances/a.shop.example.com/app.sock"); the record's shared paths are not drift 2026/10/24 00:18:05 provision: converge seed source on a.shop.example.com: constraint failed: seed_source is read-only here (1811) 2026/10/24 00:18:05 provision: a.shop.example.com: row is on the nested layout (db "/data/instances/a.shop.example.com/a.shop.example.com.db" socket "/run/carlos/instances/a.shop.example.com/app.sock"); the record's shared paths are not drift 2026/10/24 00:18:05 provision: a.shop.example.com: import sd_vbodmwacz6x3nckmllfw5nrao4 committed: its gate is cleared and its journal row dropped 2026/10/24 00:18:05 provision: a.shop.example.com: row is on the nested layout (db "/data/instances/a.shop.example.com/a.shop.example.com.db" socket "/run/carlos/instances/a.shop.example.com/app.sock"); the record's shared paths are not drift 2026/10/24 00:18:05 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its host incarnation slot no longer vouches for it 2026/10/24 00:18:05 provision: a.shop.example.com: import sd_tgwnkmgxwvsq7nxiqdtmmav5ui rolled back to its original route 2026/10/24 00:18:05 provision: a.shop.example.com: import sd_tgwnkmgxwvsq7nxiqdtmmav5ui is cleaned up and its journal row retired 2026/10/24 00:18:05 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its host incarnation slot no longer vouches for it 2026/10/24 00:18:05 provision: a.shop.example.com: import sd_ltkuuy4kz53cfzt52xiyanub7q rolled back to its original route 2026/10/24 00:18:05 provision: a.shop.example.com: import sd_ltkuuy4kz53cfzt52xiyanub7q is cleaned up and its journal row retired 2026/10/24 00:18:05 provision: a.shop.example.com: import sd_fucg4cnlebsef4m537pkbwmsje committed: its gate is cleared and its journal row dropped 2026/10/24 00:18:05 provision: a.shop.example.com: row is on the nested layout (db "/data/instances/a.shop.example.com/a.shop.example.com.db" socket "/run/carlos/instances/a.shop.example.com/app.sock"); the record's shared paths are not drift 2026/10/24 00:18:05 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its lifetime could not be judged: provision: host incarnation slot is malformed: control/host-incarnations/a.shop.example.com.json: failed validation 2026/10/24 00:18:05 provision: a.shop.example.com: import sd_rl6x4p7icuyi5yjtniqaputhw4 left alone this pass: its record could not be read 2026/10/24 00:18:05 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: its host incarnation slot no longer vouches for it 2026/10/24 00:18:05 provision: a.shop.example.com: import sd_jt2vcult7vr5no3ezwtaztxn3a rolled back to its original route 2026/10/24 00:18:05 provision: a.shop.example.com: import sd_jt2vcult7vr5no3ezwtaztxn3a is cleaned up and its journal row retired 2026/10/24 00:18:05 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:05 provision: acme.go.tito.io: row is on the nested layout (db "/data/instances/acme.go.tito.io/acme.go.tito.io.db" socket "/run/carlos/instances/acme.go.tito.io/app.sock"); the record's shared paths are not drift 2026/10/24 00:18:05 provision: acme.go.tito.io: not minting a seeded instance: this box does not mint nested (<nil>) 2026/10/24 00:18:05 provision: acme.go.tito.io: not minting a seeded instance: claim control is on for this box 2026/10/24 00:18:06 provision: acme.go.tito.io: aborted seed sd_fjj7hle4ly4epwm76c47kose2q: this box cannot read seed markers, and places no seed; nothing local is removed 2026/10/24 00:18:06 provision: acme.go.tito.io: aborted seed sd_fjj7hle4ly4epwm76c47kose2q rolled back (no box had swapped it) 2026/10/24 00:18:06 provision: acme.go.tito.io: not minting a seeded instance: its record does not describe a hibernating instance 2026/10/24 00:18:06 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:06 provision: removed instance route acme.go.tito.io (its seed sd_gcu5elccvgouqff2ejnva3xfqm was aborted); the next pass rolls it back 2026/10/24 00:18:06 provision: acme.go.tito.io: aborted seed sd_gcu5elccvgouqff2ejnva3xfqm: this box cannot read seed markers, and places no seed; nothing local is removed 2026/10/24 00:18:06 provision: acme.go.tito.io: aborted seed sd_gcu5elccvgouqff2ejnva3xfqm rolled back (no box had swapped it) 2026/10/24 00:18:06 provision: acme.go.tito.io: aborted seed sd_goasvstxzmct3pchpqpgdzwaz4 has a row this pass may not remove; leaving the seed aborted 2026/10/24 00:18:06 provision: acme.go.tito.io: aborted seed sd_4hh2a5rxsbfqemh5va354tqgza rolled back (no box had swapped it) 2026/10/24 00:18:06 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:06 provision: removed instance route acme.go.tito.io (its seed sd_dm3vb2ptuvyina64bxeihgugme was aborted); the next pass rolls it back 2026/10/24 00:18:06 provision: acme.go.tito.io: aborted seed sd_dm3vb2ptuvyina64bxeihgugme has no route on this box: its local data is removed 2026/10/24 00:18:06 provision: acme.go.tito.io: aborted seed sd_dm3vb2ptuvyina64bxeihgugme rolled back (no box had swapped it) 2026/10/24 00:18:06 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_vvyd6zphaxrlh7qlfbyo3nfrdm (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_vvyd6zphaxrlh7qlfbyo3nfrdm rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_vvyd6zphaxrlh7qlfbyo3nfrdm is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_hz3yy7us5ljdcsqin2ekv2czbq (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_hz3yy7us5ljdcsqin2ekv2czbq rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_hz3yy7us5ljdcsqin2ekv2czbq is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_lymqvdhxkjakrsrzam5mpjb4fi (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_lymqvdhxkjakrsrzam5mpjb4fi rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_lymqvdhxkjakrsrzam5mpjb4fi: the host's replica is left alone: its record no longer names this seed 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_lymqvdhxkjakrsrzam5mpjb4fi is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_622463tjcn2ezpedu6eaxleoem (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_622463tjcn2ezpedu6eaxleoem rollback deferred: the host is not fenced and stopped yet (<nil>) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_622463tjcn2ezpedu6eaxleoem rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_622463tjcn2ezpedu6eaxleoem is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_h5dx4fkqsixo7vwx2sokdd6g4i (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_h5dx4fkqsixo7vwx2sokdd6g4i rollback deferred: the host is not fenced and stopped yet (stop unproven) 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_gkyvk2zs7z32oo6dboik2tufde (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_gkyvk2zs7z32oo6dboik2tufde is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_hxezrthq3bvbjtig3ixgisqsly (the original route is journaled) 2026/10/24 00:18:06 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: malformed: provision: record bac/shop/a.shop.example.com: provision: record is corrupt: provision: invalid maintenance projection or protected fields 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_hxezrthq3bvbjtig3ixgisqsly left alone this pass: its record could not be read 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_3eglyoo33c7x32xu3lrqdhymeq (the original route is journaled) 2026/10/24 00:18:06 provision: skipping record apps/bac/shop/instances/a.shop.example.com.json: read failed: connection reset 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_3eglyoo33c7x32xu3lrqdhymeq left alone this pass: its record could not be read 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_lc5xpkpfmqzcozmb5hxzgzk3lm (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_lc5xpkpfmqzcozmb5hxzgzk3lm: close the import gate: constraint failed: injected (1811) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_lc5xpkpfmqzcozmb5hxzgzk3lm rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_lc5xpkpfmqzcozmb5hxzgzk3lm is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_xz5cskbyphr6f3ypuy7v6hkjuq (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_xz5cskbyphr6f3ypuy7v6hkjuq: close the import gate: constraint failed: injected (1811) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_xz5cskbyphr6f3ypuy7v6hkjuq rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_xz5cskbyphr6f3ypuy7v6hkjuq is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_tilu2g3gzb7wg7bwsfskdnwlwy (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_tilu2g3gzb7wg7bwsfskdnwlwy rollback deferred: the host is not fenced and stopped yet (injected failure at fence) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_tilu2g3gzb7wg7bwsfskdnwlwy rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_tilu2g3gzb7wg7bwsfskdnwlwy is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_wvhh5rzxfggwnzv27ghk4bgj3q (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_wvhh5rzxfggwnzv27ghk4bgj3q rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_wvhh5rzxfggwnzv27ghk4bgj3q is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_w3ujvn5a2ov2zkjhdfwbwpzq2m (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_w3ujvn5a2ov2zkjhdfwbwpzq2m: retire its instance key: injected failure at retire-key 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_w3ujvn5a2ov2zkjhdfwbwpzq2m rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_w3ujvn5a2ov2zkjhdfwbwpzq2m is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_tivis2ialzu4uplvx5bdfc7j6e (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_tivis2ialzu4uplvx5bdfc7j6e rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_tivis2ialzu4uplvx5bdfc7j6e is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_rqcpebaoekniegzeqonl5rywzq (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_rqcpebaoekniegzeqonl5rywzq: remove its local data: injected failure at remove-local 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_rqcpebaoekniegzeqonl5rywzq rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_rqcpebaoekniegzeqonl5rywzq is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_jtlrnjulxyc2hljlbhmij3idhi (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_jtlrnjulxyc2hljlbhmij3idhi rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_jtlrnjulxyc2hljlbhmij3idhi is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_zx7xonc6cw527p2wec3phcmi5u (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_zx7xonc6cw527p2wec3phcmi5u: remove its local data: injected failure at remove-local-done 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_zx7xonc6cw527p2wec3phcmi5u rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_zx7xonc6cw527p2wec3phcmi5u is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_6lr4u6qq52c53qp6ofjzrokopy (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_6lr4u6qq52c53qp6ofjzrokopy rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_6lr4u6qq52c53qp6ofjzrokopy is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_qutmvuofd63j26btvgp2mpexay (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_qutmvuofd63j26btvgp2mpexay: restore the original route: constraint failed: injected (1811) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_qutmvuofd63j26btvgp2mpexay rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_qutmvuofd63j26btvgp2mpexay is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_ciozrr2ioyj3pxlc3qjm2iajra (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_ciozrr2ioyj3pxlc3qjm2iajra: restore the original route: constraint failed: injected (1811) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_ciozrr2ioyj3pxlc3qjm2iajra rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_ciozrr2ioyj3pxlc3qjm2iajra is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_jusjrlfovuipwpnyr4kav3iclu (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_jusjrlfovuipwpnyr4kav3iclu is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_lc5rttun4c4h6blya2hgqnjw6u (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_lc5rttun4c4h6blya2hgqnjw6u rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_lc5rttun4c4h6blya2hgqnjw6u: delete its snapshot: injected failure at delete-snapshot 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_lc5rttun4c4h6blya2hgqnjw6u is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_65sgfvlnbf6s7hbdwlll52wwse (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_65sgfvlnbf6s7hbdwlll52wwse rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_65sgfvlnbf6s7hbdwlll52wwse is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_yoamlkywmpsbug7zzqralkpnai (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_yoamlkywmpsbug7zzqralkpnai rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_yoamlkywmpsbug7zzqralkpnai: delete the host's replica: injected failure at delete-replica 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_yoamlkywmpsbug7zzqralkpnai is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_wpsorbjfs36ly3kplxa2fuceni (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_wpsorbjfs36ly3kplxa2fuceni rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_wpsorbjfs36ly3kplxa2fuceni is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_kmscdiyv2pdwi6mtmhhuztuf7e (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_kmscdiyv2pdwi6mtmhhuztuf7e rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_kmscdiyv2pdwi6mtmhhuztuf7e: retire its journal row: constraint failed: injected (1811) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_kmscdiyv2pdwi6mtmhhuztuf7e is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_5cunww32mqphmhuihrwxnmswbi (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_5cunww32mqphmhuihrwxnmswbi rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_5cunww32mqphmhuihrwxnmswbi: retire its journal row: constraint failed: injected (1811) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_5cunww32mqphmhuihrwxnmswbi is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_ufvag4vp5cvmaiydqe4nf2w7hm (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_ufvag4vp5cvmaiydqe4nf2w7hm rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_ufvag4vp5cvmaiydqe4nf2w7hm is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: mark import sd_ufvag4vp5cvmaiydqe4nf2w7hm rolled back: injected failure at cas 2026/10/24 00:18:06 provision: a.shop.example.com: aborted seed sd_ufvag4vp5cvmaiydqe4nf2w7hm rolled back (no box had swapped it) 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_fkbio4n254kl4pxeisukwnarou (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_fkbio4n254kl4pxeisukwnarou rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_fkbio4n254kl4pxeisukwnarou is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: aborted seed sd_fkbio4n254kl4pxeisukwnarou rolled back (no box had swapped it) 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_tgte7umv6txfwgjjsj73yq65ay (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_tgte7umv6txfwgjjsj73yq65ay rollback deferred: the host is not fenced and stopped yet (<nil>) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_tgte7umv6txfwgjjsj73yq65ay rollback deferred: the host is not fenced and stopped yet (<nil>) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_tgte7umv6txfwgjjsj73yq65ay rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_tgte7umv6txfwgjjsj73yq65ay is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_zsubh4iijmjkk2wrn5evifntm4 (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_zsubh4iijmjkk2wrn5evifntm4 rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_zsubh4iijmjkk2wrn5evifntm4 is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_yybilxfuoffhhtdzqj7kbld2iy (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_yybilxfuoffhhtdzqj7kbld2iy rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_yybilxfuoffhhtdzqj7kbld2iy is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_bfgoev2u6jacmde4auypbozks4 (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_bfgoev2u6jacmde4auypbozks4 rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_bfgoev2u6jacmde4auypbozks4 is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_fu2ychbin6bczmckdtn24l5o74 (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_fu2ychbin6bczmckdtn24l5o74 rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_fu2ychbin6bczmckdtn24l5o74 is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_tjrzf6op7ebw7bgahanzvgnfgy (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_tjrzf6op7ebw7bgahanzvgnfgy rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_tjrzf6op7ebw7bgahanzvgnfgy is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: minted instance route a.shop.example.com (app "other" acct "bac") 2026/10/24 00:18:06 provision: a.shop.example.com: swapped for import sd_vvgvsp6czh3fhwcowdhhqm7ore (the original route is journaled) 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_vvgvsp6czh3fhwcowdhhqm7ore rolled back to its original route 2026/10/24 00:18:06 provision: a.shop.example.com: import sd_vvgvsp6czh3fhwcowdhhqm7ore is cleaned up and its journal row retired 2026/10/24 00:18:06 provision: minted instance route a.shop.example.com (app "other" acct "bac") 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_k7wtwduqxwzpfmdbdrpl45zooy (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_k7wtwduqxwzpfmdbdrpl45zooy rolled back to its original route 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_k7wtwduqxwzpfmdbdrpl45zooy is cleaned up and its journal row retired 2026/10/24 00:18:07 provision: minted instance route a.shop.example.com (app "other" acct "bac") 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_frqzhgbwt5nkjgmxufqlua7moy (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_frqzhgbwt5nkjgmxufqlua7moy has no route on this box: its local data is removed and no route is put back 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_frqzhgbwt5nkjgmxufqlua7moy is cleaned up and its journal row retired 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_m6npjocuq5pcvwm4brgtbl3w5i (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_smhpewl7anbltpqihwy4vrg5lm (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_olgic34p2ij62qcpkvz7pivhhy (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_olgic34p2ij62qcpkvz7pivhhy cannot roll back on this box: the activator is not wired; it stays journaled and gated 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_z65laye7c6d3e3wqdywkcuicom (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_z65laye7c6d3e3wqdywkcuicom is not rolled back: the route changed under the fence, so nothing there is touched 2026/10/24 00:18:07 provision: a.shop.example.com: import "../not-a-seed" names no snapshot and no replica; only its journal row is cleaned up 2026/10/24 00:18:07 provision: a.shop.example.com: import ../not-a-seed is cleaned up and its journal row retired 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_la4yju35jlqp47ltjpm536jfuq (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_k7r44ynmyohigmxaay6oucobe4 (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_kdqx3o5kfrivqbozpel67o5nya (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_v7kkq26sf3ycr2jmmkvis4zljq (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_qizjbrphj5nneem22wirazfhtq (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_qizjbrphj5nneem22wirazfhtq rollback deferred: the host is not fenced and stopped yet (stop evidence unreadable) 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_ahmjlbxejz3uftfbevbuqz5omq (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_ahmjlbxejz3uftfbevbuqz5omq has no route on this box: its local data is removed and no route is put back 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_ahmjlbxejz3uftfbevbuqz5omq is cleaned up and its journal row retired 2026/10/24 00:18:07 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_icfrqy6rcss37bdmw3ai5zcbge (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_icfrqy6rcss37bdmw3ai5zcbge: remove its local data: injected failure at remove-local 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_icfrqy6rcss37bdmw3ai5zcbge is not rolled back: the route at the host is not the import's, so nothing there is touched 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_6cnajjgujfvj73mle63monm46y (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_6cnajjgujfvj73mle63monm46y: a route appeared at the host, so its local database is left and the rollback waits 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_oc3e74uhwkt5ezrtgovdto5ohu (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_oc3e74uhwkt5ezrtgovdto5ohu rollback deferred: no fence generation: registry busy 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_hw6ugn4dwixfu4pwzs5jrqycf4 (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_hw6ugn4dwixfu4pwzs5jrqycf4 rollback deferred: no fence generation: the generation source answered 0 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_2fpir3xgxzzet2pq464pkncvnu (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_zy2w7klvaqsnwrcqmq4cjgf6ze (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_huvbeont7pnx2ry6pv6gcv4w4e (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: import deferred: the instance's env file could not be read: a symlink 2026/10/24 00:18:07 provision: a.shop.example.com: import deferred: this box cannot read the instance's env file 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_7dyh4auuhenn35iutu24ufwp7i (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_7dyh4auuhenn35iutu24ufwp7i rolled back to its original route 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_7dyh4auuhenn35iutu24ufwp7i is cleaned up and its journal row retired 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_zsonagjhkm7cvxvoutfhw5p2ha (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_zsonagjhkm7cvxvoutfhw5p2ha rolled back to its original route 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_zsonagjhkm7cvxvoutfhw5p2ha is cleaned up and its journal row retired 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_zszlgrp3bj6kvlo3uihzgfc44y (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_zszlgrp3bj6kvlo3uihzgfc44y rolled back to its original route 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_zszlgrp3bj6kvlo3uihzgfc44y is cleaned up and its journal row retired 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_o6os7r3vpxv3thmejbaja4drvy (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_o6os7r3vpxv3thmejbaja4drvy: retire its instance key: the env file is a symlink 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_o6os7r3vpxv3thmejbaja4drvy rolled back to its original route 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_o6os7r3vpxv3thmejbaja4drvy is cleaned up and its journal row retired 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_fvwqgmc7grniblygezpyj3sab4 (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: row is on the nested layout (db "/data/instances/a.shop.example.com/a.shop.example.com.db" socket "/run/carlos/instances/a.shop.example.com/app.sock"); the record's shared paths are not drift 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_3bfavdpsfpc45cgrjbt4gblqum (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_yue7hbkrisl27pkhdtdf6k75fm refused: there is no route at a.shop.example.com to replace 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_oehsh2nlyutf6bvavkfzz22seq refused: the route's address is 127.0.0.1:9, not 127.0.0.1:8401 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_w4yqqpki2kahleufwjon4dvq7m refused: the route's address is 127.0.0.1:8401,127.0.0.1:8402, not 127.0.0.1:8401 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_t6hdqto744m4an76l7egmdctsy refused: the route's address is 127.0.0.1:9, not 127.0.0.1:8401 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_voj3gvvaw42gp2q772qcpcq4qi refused: the route belongs to app other 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_gyxlf5erdqxo6jdjrvsfhy3nn4 refused: the route is filed under another account 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_tsj4twfpt7ocxipsocwd6j2ueu refused: the route is filed under the operator account, and no assignment maps shop to this app 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_bsnebi6ikvqzzpbx27rkz4xe3y refused: the route is filed under the operator account, and no assignment maps shop to this app 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_mc56ifmujilqv5nxhx7kzrfhgm refused: the route is filed under the operator account, and no assignment maps shop to this app 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_ripvzxnltp5chwfyneo7gsctey refused: the route is filed under the operator account, and no assignment maps shop to this app 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_v4xklf356yfpueq4vm5ose5m2e refused: the route is not an address route this import can take over (socket) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_ijsn2x5fprdbkhg6hr22miqbua refused: the route is not an address route this import can take over (no address) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_5jfu4u4clc7vxzqeeuuvn6sju4 refused: the route is not an address route this import can take over (provisioned) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_qkxruqb23icxlqd7t6mqklegja refused: the route is not an address route this import can take over (kind static) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_4ylpdruo3kfr4wrqaezn3pjsii refused: the route is not an address route this import can take over (backing) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_wv4o5ouapndhug4fg33nx3xvuq refused: the route is not an address route this import can take over (tunnel) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_i7wuf2up2wn3zoj3sl2scn5qtq refused: the route is not an address route this import can take over (caps) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_crgyawf6r3qxvioclfr55nrlgy refused: the route is not an address route this import can take over (platform) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_j5cpavtmfk3b4z5p5ce3i6rzym refused: the route is not an address route this import can take over (canary) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_46eitegfzuphqni3dp3ftauw2i refused: the route is not an address route this import can take over (kind sidecar) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_pvvvfoutwyo7inbnnmi5obggby refused: the route is not an address route this import can take over (region) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_mtbp4xxbkzpr44lmlxelhxs6jy refused: the route is not an address route this import can take over (fleet_box) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_tnqhmumdofi4aom6iun2w7kxc4 refused: the route is not an address route this import can take over (hibernate) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_lsvfslpahrjjvqfmvurocaezqa refused: the route is not an address route this import can take over (claim_owner) 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_xthjn4lzwtpocxoa2kdwuu2r2i refused: this box does not run instances nested 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_mzkkbrp4x6wkstlyd2f4ly6lla refused: this box does not run instances nested 2026/10/24 00:18:07 provision: a.shop.example.com: import sd_j3ugixb6g32ogzegxgwmg7berm refused: claim control is on for this box 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_hqpjeauu5utwxmwidtf4b6q35m (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: swapped for import sd_qbyyapyz2wfmgvf3azxkm2m754 (the original route is journaled) 2026/10/24 00:18:07 provision: a.shop.example.com: import deferred: this box cannot read the legacy route assignments 2026/10/24 00:18:07 provision: a.shop.example.com: import deferred: the legacy route assignments could not be read: store down 2026/10/24 00:18:07 provision: a.shop.example.com: not swapping for import sd_ikkbmbmzr4y5zjhfol65jbl334: its verify hash is not a sha256 2026/10/24 00:18:07 provision: a.shop.example.com: not swapping for import sd_o7nxwsbkfqpufnjonzztg3aytu: its verify hash is not a sha256 2026/10/24 00:18:08 provision: a.shop.example.com: not swapping for import sd_ju5gc2kzeefud5kij3fc75mude: its verify hash is not a sha256 2026/10/24 00:18:08 provision: a.shop.example.com: not swapping for import sd_../../x: its host or seed id is malformed (seeds: not a seed id) 2026/10/24 00:18:08 provision: A.shop.example.com/..: not swapping for import sd_msgfk44pfm5wsu4umrjev4rco4: its host or seed id is malformed (seeds: not a host) 2026/10/24 00:18:08 provision: a.shop.example.com: not swapping for import sd_brefergp57n4pmogxvmemgcymm: its record does not describe a nested, hibernating instance 2026/10/24 00:18:08 provision: a.shop.example.com: not swapping for import sd_klqdr7s3cg35gbyfdbnoa7drmm: its record does not describe a nested, hibernating instance 2026/10/24 00:18:08 provision: a.shop.example.com: not swapping for import sd_x3j6us35svd5r63qpd352sxpw4: its record does not describe a nested, hibernating instance 2026/10/24 00:18:08 provision: a.shop.example.com: not swapping for import sd_4km3gofik547czlij2dgwujqru: its record does not describe a nested, hibernating instance 2026/10/24 00:18:08 provision: a.shop.example.com: import deferred: the box's layout intent could not be read: store down 2026/10/24 00:18:08 provision: a.shop.example.com: swapped for import sd_uea7wj6ew5rizj3ddt2n3euen4 (the original route is journaled) 2026/10/24 00:18:08 provision: a.shop.example.com: import deferred: the box's nested probe did not answer: ready probe timed out 2026/10/24 00:18:08 provision: a.shop.example.com: swapped for import sd_zfci7jb6amm36lfha4pcwgdwja (the original route is journaled) 2026/10/24 00:18:08 provision: a.shop.example.com: swapped for import sd_nhaw4rffore4yrmfuhdhcwdfai (the original route is journaled) 2026/10/24 00:18:08 provision: a.shop.example.com: import deferred: the route is held by operation op_1 2026/10/24 00:18:08 provision: dry-run: would consider swapping a.shop.example.com for import sd_z6k56ojfvu55vsgaxdcsaahrj4 2026/10/24 00:18:08 provision: a.shop.example.com: import sd_vatio644ccuip3yg3tngpwcu2y rolled back to its original route 2026/10/24 00:18:08 provision: a.shop.example.com: import sd_vatio644ccuip3yg3tngpwcu2y: the host's replica is left alone: the record apps/bac/shop/instances/a.shop.example.com.json holds the host 2026/10/24 00:18:08 provision: a.shop.example.com: import sd_vatio644ccuip3yg3tngpwcu2y is cleaned up and its journal row retired 2026/10/24 00:18:08 provision: a.shop.example.com: import sd_g4htwa4tzqzzdsskwc5v57tf2u left alone: the import journal holds seed sd_ixkb2o3qmpq6h2nyunvwvkusfq at swapped 2026/10/24 00:18:08 provision: a.shop.example.com: not minting import sd_gd3wcikdz2fwp6xv7hlj6emdxq: it is swapped but not committed, and only its swap makes its row 2026/10/24 00:18:08 provision: a.shop.example.com: not minting import sd_gctowlzjurfvloryxbavke6754: it is placed but not committed, and only its swap makes its row 2026/10/24 00:18:08 provision: a.shop.example.com: swapped for import sd_kzxhcr3mtzcl3tbicp3o7lov7i (the original route is journaled) 2026/10/24 00:18:08 provision: a.shop.example.com: not minting import sd_kzxhcr3mtzcl3tbicp3o7lov7i: it is swapped but not committed, and only its swap makes its row 2026/10/24 00:18:08 provision: a.shop.example.com: not minting import sd_kzxhcr3mtzcl3tbicp3o7lov7i: it is placed but not committed, and only its swap makes its row 2026/10/24 00:18:08 provision: minted instance route a.shop.example.com (app "shop" acct "bac") 2026/10/24 00:18:08 provision: removed retired sidecar route gone.eleven.bac.sidecar.internal (record gone) 2026/10/24 00:18:08 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"]) 2026/10/24 00:18:08 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"]) 2026/10/24 00:18:08 provision: converged sidecar bot.eleven.bac.sidecar.internal args ["sidecar","run"] -> ["worker","run"] 2026/10/24 00:18:08 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"]) 2026/10/24 00:18:08 provision: converged sidecar bot.eleven.bac.sidecar.internal ceilings 300/3600 -> 60/120 2026/10/24 00:18:08 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"]) 2026/10/24 00:18:08 provision: converged sidecar bot.eleven.bac.sidecar.internal channel stable -> canary/eleven 2026/10/24 00:18:08 provision: converged sidecar bot.eleven.bac.sidecar.internal environment "other" -> "relay" 2026/10/24 00:18:08 provision: converged sidecar bot.eleven.bac.sidecar.internal hibernate true -> false 2026/10/24 00:18:08 provision: skipping sidecar record apps/bac/eleven/sidecars/bot.json: read failed: failStore: reads disabled under apps/bac/eleven/sidecars/ 2026/10/24 00:18:08 provision: skipping sidecar record apps/bac/eleven/sidecars/bot.json: malformed: invalid character 'n' looking for beginning of object key string 2026/10/24 00:18:08 provision: not removing sidecar bot.eleven.bac.sidecar.internal: its record exists but could not be read cleanly this pass 2026/10/24 00:18:08 provision: skipping sidecar record apps/bac/eleven/sidecars/bot.json: empty channel 2026/10/24 00:18:08 provision: skipping sidecar record apps/bac/eleven/sidecars/held.json: empty channel 2026/10/24 00:18:08 provision: not removing sidecar held.eleven.bac.sidecar.internal: its record exists but could not be read cleanly this pass 2026/10/24 00:18:08 provision: removed retired sidecar route old.eleven.bac.sidecar.internal (record gone) 2026/10/24 00:18:08 provision: bac/eleven has sidecar records but no sidecars manifest — not minting 2026/10/24 00:18:08 provision: bac/eleven is in the trash — not minting sidecar routes for it 2026/10/24 00:18:08 provision: sidecar bot.eleven.bac.sidecar.internal is not placeable here: it asks for "no-inbound" placement and this box accepts inbound work (CARLOS_INBOUND="") 2026/10/24 00:18:08 provision: sidecar hub.relay.bac.sidecar.internal is not placeable here: it asks for "no-inbound" placement and this box accepts inbound work (CARLOS_INBOUND="") 2026/10/24 00:18:08 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"]) 2026/10/24 00:18:08 provision: minted sidecar route hub.relay.bac.sidecar.internal (app "relay" acct "bac" args ["relay"]) 2026/10/24 00:18:08 provision: sidecar bot.eleven.bac.sidecar.internal is not placeable here: it asks for "no-inbound" placement and this box accepts inbound work (CARLOS_INBOUND="") 2026/10/24 00:18:08 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"]) 2026/10/24 00:18:08 provision: sidecar bot.eleven.bac.sidecar.internal is not placeable here: it asks for "no-inbound" placement and this box accepts inbound work (CARLOS_INBOUND="") 2026/10/24 00:18:08 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"]) 2026/10/24 00:18:08 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"]) 2026/10/24 00:18:08 provision: sidecar bot.eleven.bac.sidecar.internal is not placeable here: it asks for "no-inbound" placement and this box accepts inbound work (CARLOS_INBOUND="") 2026/10/24 00:18:08 provision: removed retired sidecar route bot.eleven.bac.sidecar.internal (record gone) 2026/10/24 00:18:08 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"]) 2026/10/24 00:18:08 provision: sidecar bot.eleven.bac.sidecar.internal is not placeable here: it asks for "no-inbound" placement and this box accepts inbound work (CARLOS_INBOUND="") 2026/10/24 00:18:08 provision: skipping sidecar record apps/bac/relay/sidecars/hub.json: read failed: failStore: reads disabled under apps/bac/relay/sidecars/ 2026/10/24 00:18:08 provision: REFUSING to mint sidecar bot.eleven.bac.sidecar.internal: bot.eleven.bac.sidecar.internal-sidecar.env is staged on this box — the legacy file co-spawn and a native sidecar row are two mechanisms for one host; remove the file to migrate this host onto sidecar records 2026/10/24 00:18:08 provision: minted sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" args ["sidecar","run"]) 2026/10/24 00:18:08 provision: dry-run: would mint sidecar route bot.eleven.bac.sidecar.internal (app "eleven" acct "bac" channel "stable" args ["sidecar","run"]) 2026/10/24 00:18:08 provision: dry-run: would remove retired sidecar route gone.eleven.bac.sidecar.internal (record gone) 2026/10/24 00:18:08 provision: bot.eleven.bac.sidecar.internal exists but is not a sidecar row this pass owns (kind "sidecar" provisioned false app "eleven" acct "bac") — leaving it alone 2026/10/24 00:18:08 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:08 provision: converged acme.go.tito.io update settings "{\"updates\":\"manual\",\"channel\":\"stable\"}" -> "" 2026/10/24 00:18:08 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:08 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:08 provision: manifest bac/tito: read failed: provision: manifest bac/tito: unexpected end of JSON input 2026/10/24 00:18:08 provision: manifest bac/tito: read failed: provision: manifest bac/tito: unexpected end of JSON input 2026/10/24 00:18:08 provision: acme.go.tito.io: not minted this pass: the app's update policy could not be read 2026/10/24 00:18:08 provision: minted instance route acme.go.tito.io (app "tito" acct "bac") 2026/10/24 00:18:08 provision: manifest bac/tito: read failed: provision: manifest bac/tito: unexpected end of JSON input 2026/10/24 00:18:08 provision: acme.go.tito.io: not minted this pass: the app's update policy could not be read 2026/10/24 00:18:08 provision: minted instance route acme.go.tito.io held landing by mv_acme (app "tito" acct "bac") --- FAIL: TestPlan14RootReadCensus (8.74s) plan14_root_reads_test.go:1377: root read census: 1 discrepancies changed root source envelope internal/console FAIL FAIL github.com/carlosframework/platform/internal/provision 17.742s ok github.com/carlosframework/platform/internal/publicrelease 0.014s ? github.com/carlosframework/platform/internal/regionname [no test files] ok github.com/carlosframework/platform/internal/registry 5.613s ok github.com/carlosframework/platform/internal/relay 7.226s ok github.com/carlosframework/platform/internal/release 24.320s ok github.com/carlosframework/platform/internal/rollout 3.623s ok github.com/carlosframework/platform/internal/rolloutwords 0.003s ok github.com/carlosframework/platform/internal/rootaws 0.007s ok github.com/carlosframework/platform/internal/rootparticipation 0.058s ok github.com/carlosframework/platform/internal/rootstate 0.027s ok github.com/carlosframework/platform/internal/runs 0.577s ok github.com/carlosframework/platform/internal/s3compat 0.005s ok github.com/carlosframework/platform/internal/safefile 0.005s ok github.com/carlosframework/platform/internal/schedule 2.610s ok github.com/carlosframework/platform/internal/seeds 0.012s ok github.com/carlosframework/platform/internal/sim/hostlife 8.498s ok github.com/carlosframework/platform/internal/sim/litestreamfacts 4.071s ok github.com/carlosframework/platform/internal/status 0.069s ok github.com/carlosframework/platform/internal/steering 0.006s ok github.com/carlosframework/platform/internal/stopevidence 0.223s ok github.com/carlosframework/platform/internal/svcrole 0.002s ok github.com/carlosframework/platform/internal/telemetry 0.004s ok github.com/carlosframework/platform/internal/termfence 0.004s ok github.com/carlosframework/platform/internal/tunnelclient 0.004s ok github.com/carlosframework/platform/internal/unixdial 0.002s ok github.com/carlosframework/platform/internal/updatewords 0.010s ok github.com/carlosframework/platform/internal/usage 0.005s ok github.com/carlosframework/platform/internal/version 0.003s ok github.com/carlosframework/platform/internal/vet 1.567s FAIL === e2e@0 (44 s) ok github.com/carlosframework/platform/internal/e2e/selfserve 41.772s === e2e@1 (34 s) ok github.com/carlosframework/platform/internal/e2e/selfserve 32.381s === e2e@2 (43 s) 2026/10/24 00:20:44 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:44 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:44 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:44 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:44 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:44 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:44 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:44 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:44 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:44 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:44 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:44 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:44 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:44 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:44 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:44 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:44 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:44 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:44 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:44 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:44 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:44 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument" 2026/10/24 00:20:44 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:44 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaab: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaab: ok true 2026/10/24 00:20:44 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaab: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged 2026/10/24 00:20:44 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaaab (was ready) shop.example.com: config updated 2026/10/24 00:20:44 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:44 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:44 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:44 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:44 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:44 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:20:44 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:20:44 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:20:44 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:20:44 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:20:44 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000003.ltx: read upload data failed: write snapshot ltx: read database page 4: invalid argument" 2026/10/24 00:20:44 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaab: checkpointed the local database (position 2, replica 2) 2026/10/24 00:20:44 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaab: sealed at TXID 3 (3 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:20:44 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaab: sealed 2026/10/24 00:20:44 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:44 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaaab (app "app" acct "bac") 2026/10/24 00:20:44 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaab: extracted the files and env shop.example.com: config updated 2026/10/24 00:20:44 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaab: landed; every item is OK 2026/10/24 00:20:44 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:44 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:44 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaab: wake fence lifted 2026/10/24 00:20:44 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaab: opened; the row is ready and adoption stages its build 2026/10/24 00:20:44 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:44 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:44 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:44 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:44 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaab: waiting for reconcile to remove the row 2026/10/24 00:20:44 provision: removed retired instance route shop.example.com (record gone) 2026/10/24 00:20:44 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaab: tombstone lifted 2026/10/24 00:20:44 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaab (shop.example.com): done 2026/10/24 00:20:44 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:44 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:44 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:44 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:44 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:44 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:44 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:44 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:44 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:44 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:44 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:44 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:44 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:44 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:44 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:44 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:44 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:44 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:44 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:44 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:44 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:44 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:45 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaac: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaac: ok true 2026/10/24 00:20:45 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaac: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:20:45 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaaac (was ready) shop.example.com: config updated 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:45 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:20:45 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:20:45 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaac: checkpointed the local database (position 2, replica 2) 2026/10/24 00:20:45 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaac: sealed at TXID 3 (5 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:20:45 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaac: sealed 2026/10/24 00:20:45 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:45 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaaac (app "app" acct "bac") 2026/10/24 00:20:45 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaac: extracted the files and env shop.example.com: config updated 2026/10/24 00:20:45 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaac: landed; every item is OK 2026/10/24 00:20:45 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaac: open write: context deadline exceeded (confirming from the record) 2026/10/24 00:20:45 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:45 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:45 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaac: wake fence lifted 2026/10/24 00:20:45 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaac: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:20:45 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaac: tombstone lifted on abort 2026/10/24 00:20:45 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:45 moves: parked /tmp/TestOpenUncertainNeverReplaysAnExpiredOpenthe_put_lost3032962841/007/shop.example.com at /tmp/TestOpenUncertainNeverReplaysAnExpiredOpenthe_put_lost3032962841/007/.offline-moves/shop.example.com/mv_mvjobaaaaaaaaaaaaaaaaaaaac/target/shop.example.com 2026/10/24 00:20:45 moves: parked /tmp/TestOpenUncertainNeverReplaysAnExpiredOpenthe_put_lost3032962841/007/shop.example.com.env at /tmp/TestOpenUncertainNeverReplaysAnExpiredOpenthe_put_lost3032962841/007/.offline-moves/shop.example.com/mv_mvjobaaaaaaaaaaaaaaaaaaaac/target/shop.example.com.env 2026/10/24 00:20:45 moves: parked /tmp/TestOpenUncertainNeverReplaysAnExpiredOpenthe_put_lost3032962841/007/shop.example.com.config.env at /tmp/TestOpenUncertainNeverReplaysAnExpiredOpenthe_put_lost3032962841/007/.offline-moves/shop.example.com/mv_mvjobaaaaaaaaaaaaaaaaaaaac/target/shop.example.com.config.env 2026/10/24 00:20:45 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaac: the target's held row is deleted 2026/10/24 00:20:45 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaac (shop.example.com): aborted could not determine the open on uk-box 2026/10/24 00:20:45 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:45 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:45 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:45 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:45 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:45 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000002 size=841 2026/10/24 00:20:45 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:45 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaad: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaad: ok true 2026/10/24 00:20:45 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaad: staged round: 3 object(s) changed, 2479 bytes copied; 3 of 3 objects staged 2026/10/24 00:20:45 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaaad (was ready) shop.example.com: config updated 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:45 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:20:45 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:20:45 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaad: checkpointed the local database (position 2, replica 2) 2026/10/24 00:20:45 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaad: sealed at TXID 3 (5 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:20:45 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaad: sealed 2026/10/24 00:20:45 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:45 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaaad (app "app" acct "bac") 2026/10/24 00:20:45 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaad: extracted the files and env shop.example.com: config updated 2026/10/24 00:20:45 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaad: landed; every item is OK 2026/10/24 00:20:45 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaad: open write: context deadline exceeded (confirming from the record) 2026/10/24 00:20:45 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:45 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaad: wake fence lifted 2026/10/24 00:20:45 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaad: opened; the row is ready and adoption stages its build 2026/10/24 00:20:45 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:45 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:45 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:45 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:45 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaad: waiting for reconcile to remove the row 2026/10/24 00:20:45 provision: removed retired instance route shop.example.com (record gone) 2026/10/24 00:20:45 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaad: tombstone lifted 2026/10/24 00:20:45 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaad (shop.example.com): done 2026/10/24 00:20:45 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:45 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:45 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:45 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:45 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:45 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:45 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:46 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:46 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaae: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaae: ok true 2026/10/24 00:20:46 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaae: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged 2026/10/24 00:20:46 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaaae (was ready) shop.example.com: config updated 2026/10/24 00:20:46 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaae: seal write: faultStore: the reply was lost (confirming from the record) 2026/10/24 00:20:46 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaae: seal write: faultStore: the reply was lost (confirming from the record) 2026/10/24 00:20:46 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaae: seal write: faultStore: the reply was lost (confirming from the record) 2026/10/24 00:20:46 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaae: seal write: faultStore: the reply was lost (confirming from the record) 2026/10/24 00:20:46 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaae (shop.example.com): faultStore: the reply was lost 2026/10/24 00:20:46 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:46 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:46 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:46 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:46 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:46 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:20:46 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:20:46 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:20:46 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:20:46 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000003 size=880 2026/10/24 00:20:46 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:20:46 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000002 size=841 2026/10/24 00:20:46 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaae: checkpointed the local database (position 2, replica 2) 2026/10/24 00:20:46 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaae: sealed at TXID 3 (5 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:20:46 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaae: sealed 2026/10/24 00:20:46 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:46 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaaae (app "app" acct "bac") 2026/10/24 00:20:46 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaae: extracted the files and env shop.example.com: config updated 2026/10/24 00:20:46 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaae: landed; every item is OK 2026/10/24 00:20:46 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:46 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:46 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaae: wake fence lifted 2026/10/24 00:20:46 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaae: opened; the row is ready and adoption stages its build 2026/10/24 00:20:46 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:46 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:46 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:46 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:46 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaae: waiting for reconcile to remove the row 2026/10/24 00:20:46 provision: removed retired instance route shop.example.com (record gone) 2026/10/24 00:20:46 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaae: tombstone lifted 2026/10/24 00:20:46 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaae (shop.example.com): done 2026/10/24 00:20:46 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:46 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:46 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:46 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:46 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:46 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:46 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:46 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:46 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:46 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:46 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:46 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:46 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:46 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:46 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:46 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:46 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:46 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:20:46 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:46 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:46 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:46 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:46 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:46 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaaf: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaaf: ok true 2026/10/24 00:20:46 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaaf: staged round: 3 object(s) changed, 2479 bytes copied; 3 of 3 objects staged 2026/10/24 00:20:46 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaaaf (was ready) shop.example.com: config updated 2026/10/24 00:20:46 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:46 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:46 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:46 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:46 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:46 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:20:46 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:20:46 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:20:46 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:20:46 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:20:46 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaaf: checkpointed the local database (position 2, replica 2) 2026/10/24 00:20:46 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaaf: sealed at TXID 3 (4 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:20:46 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaaf: sealed 2026/10/24 00:20:46 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:46 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaaaf (app "app" acct "bac") 2026/10/24 00:20:46 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaaf: extracted the files and env shop.example.com: config updated 2026/10/24 00:20:46 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaaf: landed; every item is OK 2026/10/24 00:20:46 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaaf: open write: context deadline exceeded (confirming from the record) 2026/10/24 00:20:46 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaaf: open write: context deadline exceeded (confirming from the record) 2026/10/24 00:20:46 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaaf: open write: context deadline exceeded (confirming from the record) 2026/10/24 00:20:46 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaaf: open write: context deadline exceeded (confirming from the record) 2026/10/24 00:20:46 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaaf (shop.example.com): context deadline exceeded 2026/10/24 00:20:46 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:46 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaaf: wake fence lifted 2026/10/24 00:20:46 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaaf: opened; the row is ready and adoption stages its build 2026/10/24 00:20:46 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:46 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:46 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:46 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:20:46 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaaf: waiting for reconcile to remove the row 2026/10/24 00:20:46 provision: removed retired instance route shop.example.com (record gone) 2026/10/24 00:20:46 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaaf: tombstone lifted 2026/10/24 00:20:47 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaaf (shop.example.com): done 2026/10/24 00:20:47 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:47 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:47 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:47 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:47 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:47 INFO compaction complete system=store db=shop.example.com.db level=2 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:47 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000002 txid.max=0000000000000002 size=841 2026/10/24 00:20:47 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:47 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaag: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaag: ok true 2026/10/24 00:20:47 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:47 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:47 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:47 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:47 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:47 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000002 size=841 2026/10/24 00:20:47 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:47 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaah: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaah: ok true 2026/10/24 00:20:47 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:47 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:47 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:47 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:47 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:47 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument" 2026/10/24 00:20:47 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:47 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaai: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaai: ok true 2026/10/24 00:20:47 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:47 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:47 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:47 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:47 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:47 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument" 2026/10/24 00:20:47 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:47 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaaj: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaaj: ok true 2026/10/24 00:20:47 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:47 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:47 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:47 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:47 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:47 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument" 2026/10/24 00:20:47 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:47 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaak: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaak: ok true 2026/10/24 00:20:47 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:47 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:47 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:47 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:47 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:47 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument" 2026/10/24 00:20:47 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:47 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaal: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaal: ok true 2026/10/24 00:20:47 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:47 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:47 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:47 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:48 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:48 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:48 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:48 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument" 2026/10/24 00:20:48 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:48 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaam: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaam: ok true 2026/10/24 00:20:48 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:48 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:48 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:48 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:48 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:48 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:20:48 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:48 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaan: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaan: ok true 2026/10/24 00:20:48 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:48 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:48 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:48 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:48 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:48 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument" 2026/10/24 00:20:48 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000002 size=841 2026/10/24 00:20:48 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:48 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaao: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaao: ok true 2026/10/24 00:20:48 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:48 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:48 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:48 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:48 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:48 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:48 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:48 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaap: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaap: ok true 2026/10/24 00:20:48 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:48 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:48 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:48 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:48 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:48 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:48 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument" 2026/10/24 00:20:48 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:48 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaaq: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaaq: ok true 2026/10/24 00:20:48 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:48 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:48 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:48 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:48 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:48 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:48 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument" 2026/10/24 00:20:48 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:20:48 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:48 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaar: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaar: ok true 2026/10/24 00:20:48 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:48 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:48 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:48 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:48 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:48 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:48 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:48 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument" 2026/10/24 00:20:49 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:49 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaas: queue write: faultStore: the reply was lost (confirming from the record) 2026/10/24 00:20:49 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaas: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaas: ok true 2026/10/24 00:20:49 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:49 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:49 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:49 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:49 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:49 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:49 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:20:49 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:49 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaat: queue write: context deadline exceeded (confirming from the record) 2026/10/24 00:20:49 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaat: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaat: ok true 2026/10/24 00:20:49 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:49 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:49 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:49 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:49 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:49 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument" 2026/10/24 00:20:49 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:20:49 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:49 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaau: queue write: context deadline exceeded (confirming from the record) 2026/10/24 00:20:49 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaau: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaau: ok true 2026/10/24 00:20:49 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:49 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:49 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:49 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:49 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:49 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:49 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaav: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaav: ok true 2026/10/24 00:20:49 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaav: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged 2026/10/24 00:20:49 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:49 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:49 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:49 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:49 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:20:49 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:49 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:20:49 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:49 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaaw: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaaw: ok true 2026/10/24 00:20:49 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaaw: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:20:49 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:49 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:49 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:49 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:49 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:49 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:49 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:49 INFO compaction complete system=store db=shop.example.com.db level=2 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:49 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:49 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:50 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaax: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaax: ok true 2026/10/24 00:20:50 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaax: staged round: 5 object(s) changed, 4029 bytes copied; 5 of 5 objects staged 2026/10/24 00:20:50 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:50 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:50 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:50 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:50 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:50 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument" 2026/10/24 00:20:50 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:50 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaay: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaay: ok true 2026/10/24 00:20:50 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaay: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged 2026/10/24 00:20:50 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:50 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:50 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:50 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:50 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:50 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:50 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:50 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:20:50 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:50 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaaz: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaaz: ok true 2026/10/24 00:20:50 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaaz: staged round: 5 object(s) changed, 4073 bytes copied; 5 of 5 objects staged 2026/10/24 00:20:50 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:50 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:50 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:50 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:50 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:50 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:50 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaba: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaba: ok true 2026/10/24 00:20:50 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaba: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged 2026/10/24 00:20:50 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:50 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:50 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:50 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000001.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument" 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:50 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:50 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:50 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:50 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:20:50 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:50 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:20:51 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:51 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabb: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabb: ok true 2026/10/24 00:20:51 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabb: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:20:51 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:51 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:51 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:51 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000001.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument" 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:51 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:51 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:51 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument" 2026/10/24 00:20:51 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:51 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabc: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabc: ok true 2026/10/24 00:20:51 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabc: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged 2026/10/24 00:20:51 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:51 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:51 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:51 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:51 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:51 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:51 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabd: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabd: ok true 2026/10/24 00:20:51 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabd: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:20:51 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:51 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:51 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:51 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000001.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument" 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:51 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:51 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:51 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:51 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:20:51 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:51 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabe: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabe: ok true 2026/10/24 00:20:51 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabe: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:20:51 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:51 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:51 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:51 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:52 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:52 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:52 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:20:52 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument" 2026/10/24 00:20:52 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:52 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabf: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabf: ok true 2026/10/24 00:20:52 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabf: staged round: 4 object(s) changed, 3232 bytes copied; 4 of 4 objects staged 2026/10/24 00:20:52 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:52 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:52 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:52 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000001.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument" 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:52 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:52 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:52 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:52 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:20:52 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:52 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabg: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabg: ok true 2026/10/24 00:20:52 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabg: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:20:52 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:52 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:52 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:52 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:52 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:52 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:52 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabh: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabh: ok true 2026/10/24 00:20:52 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabh: staged round: 3 object(s) changed, 2479 bytes copied; 3 of 3 objects staged 2026/10/24 00:20:52 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabh: estimate write: faultStore: the reply was lost (confirming from the record) 2026/10/24 00:20:52 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:52 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:52 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:52 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:52 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:52 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:20:52 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:52 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabi: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabi: ok true 2026/10/24 00:20:52 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabi: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:20:52 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabi: estimate write: context deadline exceeded (confirming from the record) 2026/10/24 00:20:52 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:52 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:52 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:52 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:52 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:52 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:52 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:53 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:53 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabj: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabj: ok true 2026/10/24 00:20:53 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabj: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:20:53 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabj: estimate write: context deadline exceeded (confirming from the record) 2026/10/24 00:20:53 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:53 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:53 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:53 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:53 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:53 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:53 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabk: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabk: ok true 2026/10/24 00:20:53 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabk: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:20:53 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabk: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:20:53 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabk (shop.example.com): aborted the move was cancelled 2026/10/24 00:20:53 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:53 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:53 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:53 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:53 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:53 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument" 2026/10/24 00:20:53 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:53 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabl: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabl: ok true 2026/10/24 00:20:53 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabl: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged 2026/10/24 00:20:53 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabl: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:20:53 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabl (shop.example.com): aborted the move was cancelled 2026/10/24 00:20:53 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:53 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:53 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:53 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000001.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument" 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:53 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:53 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:53 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:53 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 4: invalid argument" 2026/10/24 00:20:53 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:20:53 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:53 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabm: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabm: ok true 2026/10/24 00:20:53 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabm: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:20:53 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabm: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:20:53 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabm (shop.example.com): aborted the move was cancelled 2026/10/24 00:20:54 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:54 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:54 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:54 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:54 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:54 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:54 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabn: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabn: ok true 2026/10/24 00:20:54 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabn: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:20:54 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabn: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:20:54 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabn (shop.example.com): aborted the move was cancelled 2026/10/24 00:20:54 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:54 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:54 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:54 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000001.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument" 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:54 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:54 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:54 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument" 2026/10/24 00:20:54 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:54 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabo: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabo: ok true 2026/10/24 00:20:54 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabo: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged 2026/10/24 00:20:54 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabo: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:20:54 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabo (shop.example.com): aborted the move was cancelled 2026/10/24 00:20:54 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:54 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:54 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:54 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:54 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:54 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:54 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:54 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabp: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabp: ok true 2026/10/24 00:20:54 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabp: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:20:54 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabp: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:20:54 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabp (shop.example.com): aborted the move was cancelled 2026/10/24 00:20:54 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:54 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:54 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:54 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:54 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:54 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:54 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000002 size=841 2026/10/24 00:20:54 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:54 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabq: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabq: ok true 2026/10/24 00:20:54 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabq: staged round: 3 object(s) changed, 2479 bytes copied; 3 of 3 objects staged 2026/10/24 00:20:55 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabq: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:20:55 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabq (shop.example.com): aborted the move was cancelled 2026/10/24 00:20:55 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:55 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:55 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:55 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000001.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument" 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:55 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:55 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:55 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 4: invalid argument" 2026/10/24 00:20:55 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:55 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabr: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabr: ok true 2026/10/24 00:20:55 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabr: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged 2026/10/24 00:20:55 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabr: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:20:55 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabr (shop.example.com): aborted the move was cancelled 2026/10/24 00:20:55 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:55 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:55 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:55 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:55 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:55 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:20:55 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:20:55 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:55 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabs: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabs: ok true 2026/10/24 00:20:55 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabs: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:20:55 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabs: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:20:55 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabs (shop.example.com): aborted the move was cancelled 2026/10/24 00:20:55 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:55 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:55 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:55 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:55 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:55 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:55 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument" 2026/10/24 00:20:55 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:55 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabt: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabt: ok true 2026/10/24 00:20:55 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabt: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:20:55 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabt: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:20:55 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabt (shop.example.com): aborted the move was cancelled 2026/10/24 00:20:55 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:55 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:55 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:55 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument" 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:55 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:55 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:55 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:55 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:20:55 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:55 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabu: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabu: ok true 2026/10/24 00:20:56 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabu: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:20:56 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabu: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:20:56 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabu (shop.example.com): aborted the move was cancelled 2026/10/24 00:20:56 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:56 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:56 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:56 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:56 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:56 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:20:56 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:56 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabv: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabv: ok true 2026/10/24 00:20:56 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabv: staged round: 3 object(s) changed, 2479 bytes copied; 3 of 3 objects staged 2026/10/24 00:20:56 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabv: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:20:56 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabv (shop.example.com): aborted the move was cancelled 2026/10/24 00:20:56 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:56 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:56 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:56 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:56 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:56 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:56 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000002 size=841 2026/10/24 00:20:56 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:56 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabw: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabw: ok true 2026/10/24 00:20:56 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabw: staged round: 4 object(s) changed, 3320 bytes copied; 4 of 4 objects staged 2026/10/24 00:20:56 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabw: abort write: faultStore: the reply was lost (confirming from the record) 2026/10/24 00:20:56 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabw: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:20:56 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabw (shop.example.com): aborted the move was cancelled 2026/10/24 00:20:56 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:56 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:56 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:56 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:56 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:56 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument" 2026/10/24 00:20:56 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:56 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabx: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabx: ok true 2026/10/24 00:20:56 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabx: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:20:56 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabx: abort write: context deadline exceeded (confirming from the record) 2026/10/24 00:20:56 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabx: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:20:56 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabx (shop.example.com): aborted the move was cancelled 2026/10/24 00:20:56 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:56 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:56 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:56 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:56 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:56 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:56 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 4: invalid argument" 2026/10/24 00:20:56 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000002 size=841 2026/10/24 00:20:56 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:57 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaby: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaby: ok true 2026/10/24 00:20:57 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaby: staged round: 3 object(s) changed, 2479 bytes copied; 3 of 3 objects staged 2026/10/24 00:20:57 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaby: abort write: context deadline exceeded (confirming from the record) 2026/10/24 00:20:57 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaby: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:20:57 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaby (shop.example.com): aborted the move was cancelled 2026/10/24 00:20:57 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:57 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:57 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:57 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:57 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:57 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:57 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabz: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabz: ok true 2026/10/24 00:20:57 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabz: staged round: 3 object(s) changed, 2479 bytes copied; 3 of 3 objects staged 2026/10/24 00:20:57 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaabz (was ready) shop.example.com: config updated 2026/10/24 00:20:57 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:57 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:57 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:57 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:57 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:57 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:57 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:57 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaca: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaca: ok true 2026/10/24 00:20:57 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaca: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:20:57 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaaca (was ready) shop.example.com: config updated 2026/10/24 00:20:57 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:57 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:57 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:57 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:57 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:57 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:20:57 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:20:57 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:57 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:58 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacb: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacb: ok true 2026/10/24 00:20:58 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacb: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:20:58 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacb (was ready) shop.example.com: config updated 2026/10/24 00:20:58 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:58 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:58 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:58 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:58 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:58 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:58 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:58 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:58 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:58 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:58 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:58 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:58 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:20:58 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:58 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:20:58 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:58 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacc: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacc: ok true 2026/10/24 00:20:58 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacc: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:20:58 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacc (was ready) shop.example.com: config updated 2026/10/24 00:20:58 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:58 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:58 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:58 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:58 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:58 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:58 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:58 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:58 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:58 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:58 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:58 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:58 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:20:58 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:58 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:20:58 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:58 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacd: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacd: ok true 2026/10/24 00:20:58 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacd: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:20:58 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacd (was ready) shop.example.com: config updated 2026/10/24 00:20:58 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:58 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:58 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:58 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:58 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:58 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:58 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:59 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:59 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:59 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:59 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:59 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:59 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:59 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument" 2026/10/24 00:20:59 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:59 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaace: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaace: ok true 2026/10/24 00:20:59 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaace: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:20:59 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaace (was ready) shop.example.com: config updated 2026/10/24 00:20:59 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:59 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:59 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:59 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:59 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:59 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:59 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:59 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:59 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:59 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:59 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:59 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:59 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:59 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument" 2026/10/24 00:20:59 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:59 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacf: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacf: ok true 2026/10/24 00:20:59 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacf: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:20:59 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacf (was ready) shop.example.com: config updated 2026/10/24 00:20:59 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:20:59 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:59 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:59 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:59 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:59 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:20:59 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:20:59 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:20:59 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:20:59 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:20:59 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:20:59 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:20:59 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:20:59 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:20:59 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000002 size=841 2026/10/24 00:20:59 console: materialized account bac (fixture) from the bucket 2026/10/24 00:20:59 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacg: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacg: ok true 2026/10/24 00:20:59 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacg: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:21:00 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacg (was ready) shop.example.com: config updated 2026/10/24 00:21:00 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:00 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:00 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:00 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:00 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:00 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 4: invalid argument" 2026/10/24 00:21:00 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:00 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaach: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaach: ok true 2026/10/24 00:21:00 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaach: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged 2026/10/24 00:21:00 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaach (was ready) shop.example.com: config updated 2026/10/24 00:21:00 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:00 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:00 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:00 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:00 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:00 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:00 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaci: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaci: ok true 2026/10/24 00:21:00 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaci: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged 2026/10/24 00:21:00 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaaci (was ready) shop.example.com: config updated 2026/10/24 00:21:00 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:00 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:00 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:00 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:00 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:00 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:00 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:00 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:01 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacj: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacj: ok true 2026/10/24 00:21:01 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacj: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:21:01 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacj (was ready) shop.example.com: config updated 2026/10/24 00:21:01 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:01 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:01 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:01 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:01 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:01 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:01 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:01 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:01 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:01 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:01 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:01 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:01 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:01 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:01 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:01 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:01 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaack: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaack: ok true 2026/10/24 00:21:01 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaack: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:21:01 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaack (was ready) shop.example.com: config updated 2026/10/24 00:21:01 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:01 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:01 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:01 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:01 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:01 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:01 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:01 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:01 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:01 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:01 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:01 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:01 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:01 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:01 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacl: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacl: ok true 2026/10/24 00:21:01 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacl: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged 2026/10/24 00:21:02 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaacl: start write: faultStore: the reply was lost (confirming from the record) 2026/10/24 00:21:02 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacl (was ready) shop.example.com: config updated 2026/10/24 00:21:02 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:02 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:02 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:02 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:02 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:02 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:02 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:02 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:02 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacm: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacm: ok true 2026/10/24 00:21:02 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacm: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:21:02 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaacm: start write: context deadline exceeded (confirming from the record) 2026/10/24 00:21:02 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacm (was ready) shop.example.com: config updated 2026/10/24 00:21:02 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:02 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:02 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:02 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:02 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:02 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:02 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:02 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacn: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacn: ok true 2026/10/24 00:21:02 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacn: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:21:02 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaacn: start write: context deadline exceeded (confirming from the record) 2026/10/24 00:21:02 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacn (was ready) shop.example.com: config updated 2026/10/24 00:21:02 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:02 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:02 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:02 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:02 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:02 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:02 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:02 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:02 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:03 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaco: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaco: ok true 2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaco: staged round: 5 object(s) changed, 4073 bytes copied; 5 of 5 objects staged 2026/10/24 00:21:03 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaaco (was ready) shop.example.com: config updated 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:03 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:21:03 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaco: checkpointed the local database (position 2, replica 2) 2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaco: sealed at TXID 3 (6 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaco: sealed 2026/10/24 00:21:03 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:03 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaaco (app "app" acct "bac") 2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaco: extracted the files and env shop.example.com: config updated 2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaco: landed; every item is OK 2026/10/24 00:21:03 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:03 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:03 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:03 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:03 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:03 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:03 ERROR retention enforcement failed system=store db=shop.example.com.db error="enforce snapshot retention: remove ltx files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed" 2026/10/24 00:21:03 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:03 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacp: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacp: ok true 2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacp: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:21:03 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacp (was ready) shop.example.com: config updated 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:03 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:21:03 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacp: checkpointed the local database (position 2, replica 2) 2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacp: sealed at TXID 3 (6 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacp: sealed 2026/10/24 00:21:03 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:03 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaacp (app "app" acct "bac") 2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacp: extracted the files and env shop.example.com: config updated 2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacp: landed; every item is OK 2026/10/24 00:21:03 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:03 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:03 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:03 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:03 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:03 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:03 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:04 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacq: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacq: ok true 2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacq: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:21:04 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacq (was ready) shop.example.com: config updated 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:04 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:21:04 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:04 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument" 2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacq: checkpointed the local database (position 2, replica 2) 2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacq: sealed at TXID 3 (4 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacq: sealed 2026/10/24 00:21:04 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:04 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaacq (app "app" acct "bac") 2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacq: extracted the files and env shop.example.com: config updated 2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacq: landed; every item is OK 2026/10/24 00:21:04 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:04 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:04 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:04 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:04 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:04 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:04 ERROR retention enforcement failed system=store db=shop.example.com.db error="enforce snapshot retention: remove ltx files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed" 2026/10/24 00:21:04 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:04 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacr: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacr: ok true 2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacr: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:21:04 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacr (was ready) shop.example.com: config updated 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:04 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:21:04 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacr: checkpointed the local database (position 2, replica 2) 2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacr: sealed at TXID 3 (5 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacr: sealed 2026/10/24 00:21:04 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:04 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaacr (app "app" acct "bac") 2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacr: extracted the files and env shop.example.com: config updated 2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacr: landed; every item is OK 2026/10/24 00:21:04 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:04 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:04 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:04 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:04 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:04 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:04 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:05 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacs: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacs: ok true 2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacs: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:21:05 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacs (was ready) shop.example.com: config updated 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:05 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:21:05 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacs: checkpointed the local database (position 2, replica 2) 2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacs: sealed at TXID 3 (4 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacs: sealed 2026/10/24 00:21:05 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:05 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaacs (app "app" acct "bac") 2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacs: extracted the files and env shop.example.com: config updated 2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacs: landed; every item is OK 2026/10/24 00:21:05 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:05 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:05 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:05 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument" 2026/10/24 00:21:05 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:05 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:05 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:05 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:05 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaact: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaact: ok true 2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaact: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:21:05 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaact (was ready) shop.example.com: config updated 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:05 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:21:05 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaact: checkpointed the local database (position 2, replica 2) 2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaact: sealed at TXID 3 (5 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaact: sealed 2026/10/24 00:21:05 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:05 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaact (app "app" acct "bac") 2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaact: extracted the files and env shop.example.com: config updated 2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaact: landed; every item is OK 2026/10/24 00:21:05 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:05 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:05 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:05 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:05 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:05 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:05 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:05 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:06 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacu: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacu: ok true 2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacu: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:21:06 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacu (was ready) shop.example.com: config updated 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:06 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:21:06 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:06 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000003 size=880 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacu: checkpointed the local database (position 2, replica 2) 2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacu: sealed at TXID 3 (5 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacu: sealed 2026/10/24 00:21:06 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:06 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaacu (app "app" acct "bac") 2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacu: extracted the files and env shop.example.com: config updated 2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacu: landed; every item is OK 2026/10/24 00:21:06 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:06 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:06 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:06 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:06 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:06 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:06 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:06 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacv: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacv: ok true 2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacv: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:21:06 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacv (was ready) shop.example.com: config updated 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:06 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:21:06 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacv: checkpointed the local database (position 2, replica 2) 2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacv: sealed at TXID 3 (5 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacv: sealed 2026/10/24 00:21:06 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:06 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaacv (app "app" acct "bac") 2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacv: extracted the files and env shop.example.com: config updated 2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacv: landed; every item is OK 2026/10/24 00:21:06 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:06 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:06 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:06 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:06 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:06 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:06 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:06 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:06 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:07 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacw: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacw: ok true 2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacw: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:21:07 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacw (was ready) shop.example.com: config updated 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:07 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:21:07 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:07 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000003 size=880 2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacw: checkpointed the local database (position 2, replica 2) 2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacw: sealed at TXID 3 (6 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacw: sealed 2026/10/24 00:21:07 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:07 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaacw (app "app" acct "bac") 2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacw: extracted the files and env shop.example.com: config updated 2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacw: landed; every item is OK 2026/10/24 00:21:07 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:07 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:07 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:07 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:07 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:07 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:07 ERROR retention enforcement failed system=store db=shop.example.com.db error="enforce snapshot retention: remove ltx files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed" 2026/10/24 00:21:07 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:07 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:07 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacx: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacx: ok true 2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacx: staged round: 5 object(s) changed, 4073 bytes copied; 5 of 5 objects staged 2026/10/24 00:21:07 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacx (was ready) shop.example.com: config updated 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:07 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:21:07 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:07 INFO compaction complete system=store db=shop.example.com.db level=2 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacx: checkpointed the local database (position 2, replica 2) 2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacx: sealed at TXID 3 (7 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacx: sealed 2026/10/24 00:21:07 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:07 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaacx (app "app" acct "bac") 2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacx: extracted the files and env shop.example.com: config updated 2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacx: landed; every item is OK 2026/10/24 00:21:07 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:07 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:07 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:07 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:07 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:07 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:07 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:07 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:08 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacy: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacy: ok true 2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacy: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:21:08 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacy (was ready) shop.example.com: config updated 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:08 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:21:08 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:08 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000003.ltx: read upload data failed: write snapshot ltx: read database page 4: invalid argument" 2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacy: checkpointed the local database (position 2, replica 2) 2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacy: sealed at TXID 3 (4 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacy: sealed 2026/10/24 00:21:08 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:08 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaacy (app "app" acct "bac") 2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacy: extracted the files and env shop.example.com: config updated 2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacy: landed; every item is OK 2026/10/24 00:21:08 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:08 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:08 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:08 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:08 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:08 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:08 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:08 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument" 2026/10/24 00:21:08 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacz: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacz: ok true 2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacz: staged round: 4 object(s) changed, 3232 bytes copied; 4 of 4 objects staged 2026/10/24 00:21:08 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacz (was ready) shop.example.com: config updated 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:08 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:21:08 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:08 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument" 2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacz: checkpointed the local database (position 2, replica 2) 2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacz: sealed at TXID 3 (5 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacz: sealed 2026/10/24 00:21:08 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:08 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaacz (app "app" acct "bac") 2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacz: extracted the files and env shop.example.com: config updated 2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacz: landed; every item is OK 2026/10/24 00:21:08 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:08 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:08 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:08 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:08 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:08 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:08 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:08 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:08 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:09 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaada: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaada: ok true 2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaada: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:21:09 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaada (was ready) shop.example.com: config updated 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:09 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:09 ERROR retention enforcement failed system=store db=shop.example.com.db error="enforce snapshot retention: remove ltx files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed" 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:21:09 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaada: checkpointed the local database (position 2, replica 2) 2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaada: sealed at TXID 3 (5 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaada: sealed 2026/10/24 00:21:09 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaada: flip write: faultStore: the reply was lost (confirming from the record) 2026/10/24 00:21:09 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:09 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaada (app "app" acct "bac") 2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaada: extracted the files and env shop.example.com: config updated 2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaada: landed; every item is OK 2026/10/24 00:21:09 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:09 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:09 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:09 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:09 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:09 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:09 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadb: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadb: ok true 2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadb: staged round: 3 object(s) changed, 2479 bytes copied; 3 of 3 objects staged 2026/10/24 00:21:09 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaadb (was ready) shop.example.com: config updated 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:09 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:21:09 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadb: checkpointed the local database (position 2, replica 2) 2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadb: sealed at TXID 3 (4 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadb: sealed 2026/10/24 00:21:09 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadb: flip write: context deadline exceeded (confirming from the record) 2026/10/24 00:21:09 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:09 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaadb (app "app" acct "bac") 2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadb: extracted the files and env shop.example.com: config updated 2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadb: landed; every item is OK 2026/10/24 00:21:09 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:09 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:09 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:09 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:09 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:09 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:09 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:09 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument" 2026/10/24 00:21:09 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:10 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadc: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadc: ok true 2026/10/24 00:21:10 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadc: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:21:10 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaadc (was ready) shop.example.com: config updated 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:10 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:21:10 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:21:10 INFO compaction complete system=store db=shop.example.com.db level=2 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:10 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000003 size=880 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:10 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadc: checkpointed the local database (position 2, replica 2) 2026/10/24 00:21:10 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadc: sealed at TXID 3 (6 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:21:10 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadc: sealed 2026/10/24 00:21:10 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadc: flip write: context deadline exceeded (confirming from the record) 2026/10/24 00:21:10 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:10 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaadc (app "app" acct "bac") 2026/10/24 00:21:10 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadc: extracted the files and env shop.example.com: config updated 2026/10/24 00:21:10 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadc: landed; every item is OK 2026/10/24 00:21:10 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:10 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:10 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:10 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:10 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument" 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:10 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:10 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:10 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:10 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadd: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadd: ok true 2026/10/24 00:21:10 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadd (shop.example.com): aborting: could not determine the replica's size on ie-box 2026/10/24 00:21:10 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadd (shop.example.com): aborted could not determine the replica's size on ie-box 2026/10/24 00:21:10 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:10 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:10 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:10 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:10 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:10 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:10 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:10 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:10 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaade: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaade: ok true 2026/10/24 00:21:10 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaade (shop.example.com): aborting: could not determine the replica's size on ie-box 2026/10/24 00:21:10 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaade (shop.example.com): aborted could not determine the replica's size on ie-box 2026/10/24 00:21:10 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:10 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:10 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:10 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:10 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:10 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:10 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:10 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadf: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadf: ok true 2026/10/24 00:21:10 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadf (shop.example.com): aborting: this account's files are too large to move this way 2026/10/24 00:21:10 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadf (shop.example.com): aborted this account's files are too large to move this way 2026/10/24 00:21:10 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:10 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:10 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:10 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:10 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:10 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:10 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadg: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadg: ok true 2026/10/24 00:21:10 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:10 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:10 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:10 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000001.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument" 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:10 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:10 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:10 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:10 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:10 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:11 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:11 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadh: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadh: ok true 2026/10/24 00:21:11 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadh: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:21:11 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadh: waiting: Waiting for ie-box in ie to confirm it has stopped shop.example.com. shop.example.com is offline. 2026/10/24 00:21:11 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadh: waiting: Waiting for ie-box in ie to confirm it has stopped shop.example.com. shop.example.com is offline. 2026/10/24 00:21:11 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadh: waiting: Waiting for ie-box in ie to confirm it has stopped shop.example.com. shop.example.com is offline. 2026/10/24 00:21:11 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:11 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:11 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:11 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:11 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:11 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument" 2026/10/24 00:21:11 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:11 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadi: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadi: ok true 2026/10/24 00:21:11 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadi: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:21:11 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:11 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:11 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:11 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:11 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:11 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:11 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:11 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:11 ERROR retention enforcement failed system=store db=shop.example.com.db error="enforce snapshot retention: remove ltx files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed" 2026/10/24 00:21:11 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:11 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadj: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadj: ok true 2026/10/24 00:21:11 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadj: staged round: 5 object(s) changed, 4073 bytes copied; 5 of 5 objects staged 2026/10/24 00:21:11 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadj (shop.example.com): sealed, the seal not read: the copy's bytes are unknown (no copy timing) 2026/10/24 00:21:11 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadj: waiting: this move is offline-sealed. run carlos move offline status --host shop.example.com to see what it waits for 2026/10/24 00:21:11 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadj: waiting: this move is offline-sealed. run carlos move offline status --host shop.example.com to see what it waits for 2026/10/24 00:21:11 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:11 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:11 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:11 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:11 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:11 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:11 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:11 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:11 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadk: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadk: ok true 2026/10/24 00:21:11 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadk: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:21:11 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:12 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:12 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:12 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument" 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:12 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:12 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:12 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:12 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:12 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadl: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadl: ok true 2026/10/24 00:21:12 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadl: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:21:12 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadl: waiting: this move is offline-landed. run carlos move offline status --host shop.example.com to see what it waits for 2026/10/24 00:21:12 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadl: waiting: this move is offline-landed. run carlos move offline status --host shop.example.com to see what it waits for 2026/10/24 00:21:12 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadl: waiting: this move is offline-landed. run carlos move offline status --host shop.example.com to see what it waits for 2026/10/24 00:21:12 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:12 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:12 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:12 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:12 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:12 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:12 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument" 2026/10/24 00:21:12 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:12 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadm: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadm: ok true 2026/10/24 00:21:12 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadm: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:21:12 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:12 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:12 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:12 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:12 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:12 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:12 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadn: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadn: ok true 2026/10/24 00:21:12 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadn: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged 2026/10/24 00:21:12 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:12 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:12 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:12 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:12 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:12 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:12 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:12 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:12 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaado: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaado: ok true 2026/10/24 00:21:12 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaado: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:21:13 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:13 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:13 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:13 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:13 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:13 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:13 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:13 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:13 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadp: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadp: ok true 2026/10/24 00:21:13 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadp: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:21:13 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadp (shop.example.com): an operator started the move; the job yields it 2026/10/24 00:21:13 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:13 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:13 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:13 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:13 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:13 ERROR enforce L0 time retention system=store db=shop.example.com.db error="remove expired l0 files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed" 2026/10/24 00:21:13 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000002 size=841 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:13 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:13 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadq: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadq: ok true 2026/10/24 00:21:13 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadq: staged round: 4 object(s) changed, 3320 bytes copied; 4 of 4 objects staged 2026/10/24 00:21:13 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadq (shop.example.com): an operator started the move; the job yields it 2026/10/24 00:21:13 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:13 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:13 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:13 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:13 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:13 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:13 ERROR retention enforcement failed system=store db=shop.example.com.db error="enforce snapshot retention: remove ltx files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed" 2026/10/24 00:21:13 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:13 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:13 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:13 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadr: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadr: ok true 2026/10/24 00:21:13 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadr: staged round: 5 object(s) changed, 4073 bytes copied; 5 of 5 objects staged 2026/10/24 00:21:13 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadr (shop.example.com): an operator started the move; the job yields it 2026/10/24 00:21:13 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:13 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:13 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:13 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:13 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:13 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:13 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:13 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:13 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaads: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaads: ok true 2026/10/24 00:21:13 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaads: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:21:14 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaads (shop.example.com): an operator started the move; the job yields it 2026/10/24 00:21:14 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:14 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:14 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:14 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:14 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:14 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument" 2026/10/24 00:21:14 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:14 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadt: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadt: ok true 2026/10/24 00:21:14 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadt: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged 2026/10/24 00:21:14 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadt (shop.example.com): an operator started the move; the job yields it 2026/10/24 00:21:14 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:14 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:14 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:14 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:14 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:14 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:14 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:14 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:14 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadu: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadu: ok true 2026/10/24 00:21:14 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadu: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:21:14 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadu: waiting: Waiting for ie-box in ie to confirm it has stopped shop.example.com. shop.example.com is offline. 2026/10/24 00:21:14 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadu: waiting: Waiting for ie-box in ie to confirm it has stopped shop.example.com. shop.example.com is offline. 2026/10/24 00:21:14 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadu: waiting: Waiting for ie-box in ie to confirm it has stopped shop.example.com. shop.example.com is offline. 2026/10/24 00:21:14 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:14 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:14 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:14 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:14 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:14 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:14 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:14 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:14 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadv: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadv: ok true 2026/10/24 00:21:14 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadv: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:21:14 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadv: waiting: Waiting for ie-box in ie to confirm it has stopped shop.example.com. shop.example.com is offline. 2026/10/24 00:21:14 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadv: waiting: Waiting for ie-box in ie to confirm it has stopped shop.example.com. shop.example.com is offline. 2026/10/24 00:21:14 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadv: waiting: Waiting for ie-box in ie to confirm it has stopped shop.example.com. shop.example.com is offline. 2026/10/24 00:21:14 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:14 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:14 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:14 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:14 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:14 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:14 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:14 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:14 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:15 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadw: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadw: ok true 2026/10/24 00:21:15 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadw: staged round: 4 object(s) changed, 3232 bytes copied; 4 of 4 objects staged 2026/10/24 00:21:15 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadw: waiting: this move is offline-sealed. run carlos move offline status --host shop.example.com to see what it waits for 2026/10/24 00:21:15 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadw: waiting: this move is offline-sealed. run carlos move offline status --host shop.example.com to see what it waits for 2026/10/24 00:21:15 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadw: waiting: this move is offline-sealed. run carlos move offline status --host shop.example.com to see what it waits for 2026/10/24 00:21:15 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadw: waiting: this move is offline-sealed. run carlos move offline status --host shop.example.com to see what it waits for 2026/10/24 00:21:15 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:15 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:15 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:15 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:15 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:15 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:15 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:15 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:15 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadx: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadx: ok true 2026/10/24 00:21:15 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadx: staged round: 5 object(s) changed, 4073 bytes copied; 5 of 5 objects staged 2026/10/24 00:21:15 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:15 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:15 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:15 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:15 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:15 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:15 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:15 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaady: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaady: ok true 2026/10/24 00:21:15 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaady: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:21:15 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:15 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:15 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:15 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:15 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:15 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument" 2026/10/24 00:21:15 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:15 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadz: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadz: ok true 2026/10/24 00:21:15 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadz: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged 2026/10/24 00:21:15 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:15 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:15 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:15 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:15 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:15 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:15 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:15 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:16 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaea: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaea: ok true 2026/10/24 00:21:16 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaea: staged round: 3 object(s) changed, 2479 bytes copied; 3 of 3 objects staged 2026/10/24 00:21:16 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:16 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:16 ERROR enforce L0 time retention system=store db=shop.example.com.db error="remove expired l0 files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed" 2026/10/24 00:21:16 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:16 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:16 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:16 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:16 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:16 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaec (shop.example.com): aborted the move could not be started 2026/10/24 00:21:16 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:16 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:16 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:16 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:16 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaee: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:21:16 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaed (shop.example.com): aborted another move is in flight 2026/10/24 00:21:16 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:16 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:16 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:16 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaeg: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:21:16 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaef (shop.example.com): aborted another move is in flight 2026/10/24 00:21:16 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:16 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:16 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:16 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaeh: staged round: 4 object(s) changed, 3232 bytes copied; 4 of 4 objects staged 2026/10/24 00:21:16 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:16 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000001.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument" 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:16 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:16 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaei (shop.example.com): the record is offline-queued, behind the row's barrier; the row is left where it is 2026/10/24 00:21:16 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaei (shop.example.com): the record is offline-queued, behind the row's barrier; the row is left where it is 2026/10/24 00:21:16 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaei (shop.example.com): the record is offline-queued, behind the row's barrier; the row is left where it is 2026/10/24 00:21:16 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:16 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:16 INFO compaction complete system=store db=shop.example.com.db level=2 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:16 ERROR enforce L0 time retention system=store db=shop.example.com.db error="remove expired l0 files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed" 2026/10/24 00:21:16 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:16 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:16 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaej: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaej: ok true 2026/10/24 00:21:16 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaej: staged round: 5 object(s) changed, 4073 bytes copied; 5 of 5 objects staged 2026/10/24 00:21:17 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:17 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument" 2026/10/24 00:21:17 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:17 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaek (shop.example.com): aborted the instance was recreated 2026/10/24 00:21:17 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:17 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:17 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:17 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaael (shop.example.com): aborting: a run is open on this instance: run_1. wait for it to finish or cancel it, then move the instance 2026/10/24 00:21:17 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaael (shop.example.com): aborted a run is open on this instance: run_1. wait for it to finish or cancel it, then move the instance 2026/10/24 00:21:17 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:17 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:17 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:17 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaem (shop.example.com): aborting: shop.example.com is steered or runs under a unit. offline moves cannot carry either yet 2026/10/24 00:21:17 provision: skipping record apps/bac/app/instances/shop.example.com.json: invalid shape: steering "canary": want "" or "latency" 2026/10/24 00:21:17 provision: skipping record apps/bac/app/instances/shop.example.com.json: invalid shape: steering "canary": want "" or "latency" 2026/10/24 00:21:17 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaem (shop.example.com): aborted shop.example.com is steered or runs under a unit. offline moves cannot carry either yet 2026/10/24 00:21:17 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:17 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:17 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaen: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaen: ok true 2026/10/24 00:21:17 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaen: stage write: faultStore: the reply was lost (confirming from the record) 2026/10/24 00:21:17 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaen: stage write: faultStore: the reply was lost (confirming from the record) 2026/10/24 00:21:17 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaen: stage write: faultStore: the reply was lost (confirming from the record) 2026/10/24 00:21:17 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaen: stage write: faultStore: the reply was lost (confirming from the record) 2026/10/24 00:21:17 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaen (shop.example.com): faultStore: the reply was lost 2026/10/24 00:21:17 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:17 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:17 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:17 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:17 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaeo: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaeo: ok true 2026/10/24 00:21:17 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaeo: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:21:17 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:17 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:18 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaep: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaep: ok true 2026/10/24 00:21:18 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaep: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged 2026/10/24 00:21:18 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaep: abort write: provision: record apps/bac/app/instances/shop.example.com.json: incarnation 1 record_seq 4, expected 1 and 3: store: compare-and-swap conflict (confirming from the record) 2026/10/24 00:21:18 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaep: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:21:18 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaep (shop.example.com): aborted the move was cancelled 2026/10/24 00:21:18 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:18 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:18 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:18 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:18 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:18 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:18 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:18 ERROR retention enforcement failed system=store db=shop.example.com.db error="enforce snapshot retention: remove ltx files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed" 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:18 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:18 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaeq: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaeq: ok true 2026/10/24 00:21:18 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaeq: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:21:18 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:18 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:18 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:18 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:18 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:18 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:18 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:18 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:18 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaer: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaer: ok true 2026/10/24 00:21:18 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaer: staged round: 5 object(s) changed, 4073 bytes copied; 5 of 5 objects staged 2026/10/24 00:21:18 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:18 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:18 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:18 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:18 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:18 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:18 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument" 2026/10/24 00:21:18 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:18 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaes: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaes: ok true 2026/10/24 00:21:18 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaes: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged 2026/10/24 00:21:18 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaes: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:21:19 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaes (shop.example.com): aborted the move was cancelled 2026/10/24 00:21:19 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:19 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:19 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:19 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:19 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:19 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:19 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument" 2026/10/24 00:21:19 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:19 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaet: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaet: ok true 2026/10/24 00:21:19 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaet: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:21:19 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaet: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:21:19 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaet (shop.example.com): aborted the move was cancelled 2026/10/24 00:21:19 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:19 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:19 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:19 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:19 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:19 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000002 size=841 2026/10/24 00:21:19 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:19 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaeu (shop.example.com): aborted the move was cancelled 2026/10/24 00:21:19 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:19 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:19 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:19 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:19 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:19 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:19 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:19 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaev (shop.example.com): aborted the move was cancelled 2026/10/24 00:21:19 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:19 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:19 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:19 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000001.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument" 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:19 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:19 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:19 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:19 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:19 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaew: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaew: ok true 2026/10/24 00:21:19 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaew (shop.example.com): aborted the move was cancelled 2026/10/24 00:21:19 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:19 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:19 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:19 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:19 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:19 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:19 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:19 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:19 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:19 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:19 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaex: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaex: ok true 2026/10/24 00:21:19 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaex: staged round: 5 object(s) changed, 4073 bytes copied; 5 of 5 objects staged 2026/10/24 00:21:19 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaex: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:21:19 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaex (shop.example.com): aborted the move was cancelled 2026/10/24 00:21:20 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:20 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:20 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:20 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:20 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:20 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:20 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:20 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:20 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:20 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:20 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:20 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:20 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:20 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument" 2026/10/24 00:21:20 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaey: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaey: ok true 2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaey: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged 2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaey: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:21:20 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaey (shop.example.com): aborted the move was cancelled 2026/10/24 00:21:20 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:20 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:20 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:20 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:20 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:20 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:20 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:20 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument" 2026/10/24 00:21:20 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:20 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:20 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:20 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:20 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:20 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:20 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:20 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaez: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaez: ok true 2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaez: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged 2026/10/24 00:21:20 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaaez (was ready) shop.example.com: config updated 2026/10/24 00:21:20 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:20 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:20 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:20 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:20 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:21:20 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:20 INFO compaction complete system=store db=shop.example.com.db level=2 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:20 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000003.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument" 2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaez: checkpointed the local database (position 2, replica 2) 2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaez: sealed at TXID 3 (6 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaez: sealed 2026/10/24 00:21:20 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:20 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaaez (app "app" acct "bac") 2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaez: extracted the files and env shop.example.com: config updated 2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaez: landed; every item is OK 2026/10/24 00:21:20 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:20 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaez: wake fence lifted 2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaez: opened; the row is ready and adoption stages its build 2026/10/24 00:21:20 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:20 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:20 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:20 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaez: waiting for reconcile to remove the row 2026/10/24 00:21:20 provision: removed retired instance route shop.example.com (record gone) 2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaez: tombstone lifted 2026/10/24 00:21:20 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaez (shop.example.com): done 2026/10/24 00:21:20 moves: shop.example.com: reported the leftovers of move mv_mvjobaaaaaaaaaaaaaaaaaaaez for return request 1792801280914498284-console-e2e-job-f6668b83 2026/10/24 00:21:20 moves: shop.example.com: reported the leftovers of move mv_mvjobaaaaaaaaaaaaaaaaaaaez for return request 1792801280936195449-console-e2e-job-a79abc6f 2026/10/24 00:21:20 moves: shop.example.com: reported the leftovers of move mv_mvjobaaaaaaaaaaaaaaaaaaaez for return request 1792801280953331764-console-e2e-job-5c5b0e9c 2026/10/24 00:21:21 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:21 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:21 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:21 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:21 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:21 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:21 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafa: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaafa: ok true 2026/10/24 00:21:21 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafa: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged 2026/10/24 00:21:21 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaafa (was ready) shop.example.com: config updated 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:21 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:21:21 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:21 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000003.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument" 2026/10/24 00:21:21 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafa: checkpointed the local database (position 2, replica 2) 2026/10/24 00:21:21 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafa: sealed at TXID 3 (3 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:21:21 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafa: sealed 2026/10/24 00:21:21 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaafa (shop.example.com): sealed, the seal not read: the copy's bytes are unknown (no copy timing) 2026/10/24 00:21:21 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:21 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaafa (app "app" acct "bac") 2026/10/24 00:21:21 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafa: extracted the files and env shop.example.com: config updated 2026/10/24 00:21:21 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafa: landed; every item is OK 2026/10/24 00:21:21 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:21 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:21 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafa: wake fence lifted 2026/10/24 00:21:21 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafa: opened; the row is ready and adoption stages its build 2026/10/24 00:21:21 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:21 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:21 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:21 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:21 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafa: waiting for reconcile to remove the row 2026/10/24 00:21:21 provision: removed retired instance route shop.example.com (record gone) 2026/10/24 00:21:21 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafa: tombstone lifted 2026/10/24 00:21:21 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaafa (shop.example.com): done 2026/10/24 00:21:21 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:21 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:21 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:21 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:21 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:21 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:21 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:21 ERROR retention enforcement failed system=store db=shop.example.com.db error="enforce snapshot retention: remove ltx files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed" 2026/10/24 00:21:21 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:21 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:21 moves: shop.example.com: offline move mv_p5ayh5372ejinhnvjatvjy5eyu: lane probe of data-uk/moves/_lane/mv_p5ayh5372ejinhnvjatvjy5eyu: ok true 2026/10/24 00:21:21 moves: shop.example.com: offline move mv_p5ayh5372ejinhnvjatvjy5eyu: staged round: 5 object(s) changed, 4073 bytes copied; 5 of 5 objects staged 2026/10/24 00:21:21 moves: shop.example.com: offline move mv_p5ayh5372ejinhnvjatvjy5eyu: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:21:21 console: self-serve move mv_p5ayh5372ejinhnvjatvjy5eyu (shop.example.com): aborted the move was cancelled 2026/10/24 00:21:21 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:21 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:21 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:21 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:21 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:21 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:21 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument" 2026/10/24 00:21:22 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:22 moves: shop.example.com: offline move mv_otlxpjmww64k4wtx67snax3ose: lane probe of data-uk/moves/_lane/mv_otlxpjmww64k4wtx67snax3ose: ok true 2026/10/24 00:21:22 moves: shop.example.com: offline move mv_otlxpjmww64k4wtx67snax3ose: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged 2026/10/24 00:21:22 console: self-serve move mv_otlxpjmww64k4wtx67snax3ose (shop.example.com): aborting: the target box already holds data for shop.example.com: /tmp/TestSelfServeAbortByFailedPrecheck2204483583/007/shop.example.com.env 2026/10/24 00:21:22 moves: shop.example.com: offline move mv_otlxpjmww64k4wtx67snax3ose: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:21:22 moves: parked /tmp/TestSelfServeAbortByFailedPrecheck2204483583/007/shop.example.com.env at /tmp/TestSelfServeAbortByFailedPrecheck2204483583/007/.offline-moves/shop.example.com/mv_otlxpjmww64k4wtx67snax3ose/target/shop.example.com.env 2026/10/24 00:21:22 console: self-serve move mv_otlxpjmww64k4wtx67snax3ose (shop.example.com): aborted the target box already holds data for shop.example.com: /tmp/TestSelfServeAbortByFailedPrecheck2204483583/007/shop.example.com.env 2026/10/24 00:21:22 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:22 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:22 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:22 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:22 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:22 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:22 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:22 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument" 2026/10/24 00:21:22 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:22 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:22 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:22 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:22 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:22 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:22 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:22 moves: shop.example.com: offline move mv_enq4u76p6dfv6ue2yfyfhvx3uy: lane probe of data-uk/moves/_lane/mv_enq4u76p6dfv6ue2yfyfhvx3uy: ok true 2026/10/24 00:21:22 moves: shop.example.com: offline move mv_enq4u76p6dfv6ue2yfyfhvx3uy: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged 2026/10/24 00:21:22 provision: held shop.example.com moving by mv_enq4u76p6dfv6ue2yfyfhvx3uy (was ready) shop.example.com: config updated 2026/10/24 00:21:22 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:22 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:22 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:22 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:22 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:21:22 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000003 size=880 2026/10/24 00:21:22 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:22 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000003 size=880 2026/10/24 00:21:22 moves: shop.example.com: offline move mv_enq4u76p6dfv6ue2yfyfhvx3uy: checkpointed the local database (position 2, replica 2) 2026/10/24 00:21:22 moves: shop.example.com: offline move mv_enq4u76p6dfv6ue2yfyfhvx3uy: sealed at TXID 3 (5 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:21:22 moves: shop.example.com: offline move mv_enq4u76p6dfv6ue2yfyfhvx3uy: sealed 2026/10/24 00:21:22 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:22 provision: minted instance route shop.example.com held landing by mv_enq4u76p6dfv6ue2yfyfhvx3uy (app "app" acct "bac") 2026/10/24 00:21:22 moves: shop.example.com: offline move mv_enq4u76p6dfv6ue2yfyfhvx3uy: extracted the files and env shop.example.com: config updated 2026/10/24 00:21:22 moves: shop.example.com: offline move mv_enq4u76p6dfv6ue2yfyfhvx3uy: landed; every item is OK 2026/10/24 00:21:22 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:22 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:22 moves: shop.example.com: offline move mv_enq4u76p6dfv6ue2yfyfhvx3uy: wake fence lifted 2026/10/24 00:21:22 moves: shop.example.com: offline move mv_enq4u76p6dfv6ue2yfyfhvx3uy: opened; the row is ready and adoption stages its build 2026/10/24 00:21:22 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:22 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:22 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:22 moves: shop.example.com: offline move mv_enq4u76p6dfv6ue2yfyfhvx3uy: waiting for reconcile to remove the row 2026/10/24 00:21:22 provision: removed retired instance route shop.example.com (record gone) 2026/10/24 00:21:22 moves: shop.example.com: offline move mv_enq4u76p6dfv6ue2yfyfhvx3uy: tombstone lifted 2026/10/24 00:21:22 console: self-serve move mv_enq4u76p6dfv6ue2yfyfhvx3uy (shop.example.com): done 2026/10/24 00:21:22 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:22 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:22 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:22 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:22 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:22 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:22 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:22 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:22 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:22 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:22 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:22 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:22 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:22 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:22 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently 2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently --- FAIL: TestClaimsOffInstanceIDIsPublished (0.97s) selfserve_operator_test.go:337: after 60 rounds, still not in staging 2026/10/24 00:21:23 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:23 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:23 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:23 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:23 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:23 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:23 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:23 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:23 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:23 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:23 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:23 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:23 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:23 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:23 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:23 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:23 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:23 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:23 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:23 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:23 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:23 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:23 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:23 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:23 moves: shop.example.com: offline move mv_oukeoqnajqhy2orte7e2hw5p7u: lane probe of data-uk/moves/_lane/mv_oukeoqnajqhy2orte7e2hw5p7u: ok true 2026/10/24 00:21:24 moves: shop.example.com: offline move mv_oukeoqnajqhy2orte7e2hw5p7u: staged round: 5 object(s) changed, 4073 bytes copied; 5 of 5 objects staged 2026/10/24 00:21:24 provision: held shop.example.com moving by mv_oukeoqnajqhy2orte7e2hw5p7u (was ready) shop.example.com: config updated 2026/10/24 00:21:24 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaafc (c.shop.example.com): aborting: box x-box was replaced 2026/10/24 00:21:24 provision: minted instance route b.shop.example.com (app "app" acct "bac") 2026/10/24 00:21:24 provision: minted instance route c.shop.example.com (app "app" acct "bac") 2026/10/24 00:21:24 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:24 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:24 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:24 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:24 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003 2026/10/24 00:21:24 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880 2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003 2026/10/24 00:21:24 moves: shop.example.com: offline move mv_oukeoqnajqhy2orte7e2hw5p7u: checkpointed the local database (position 2, replica 2) 2026/10/24 00:21:24 moves: shop.example.com: offline move mv_oukeoqnajqhy2orte7e2hw5p7u: sealed at TXID 3 (7 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b) 2026/10/24 00:21:24 moves: shop.example.com: offline move mv_oukeoqnajqhy2orte7e2hw5p7u: sealed 2026/10/24 00:21:24 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaafc (c.shop.example.com): aborted box x-box was replaced 2026/10/24 00:21:24 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:24 provision: minted instance route shop.example.com held landing by mv_oukeoqnajqhy2orte7e2hw5p7u (app "app" acct "bac") 2026/10/24 00:21:24 moves: shop.example.com: offline move mv_oukeoqnajqhy2orte7e2hw5p7u: extracted the files and env shop.example.com: config updated 2026/10/24 00:21:24 moves: shop.example.com: offline move mv_oukeoqnajqhy2orte7e2hw5p7u: landed; every item is OK 2026/10/24 00:21:24 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:24 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:24 moves: shop.example.com: offline move mv_oukeoqnajqhy2orte7e2hw5p7u: wake fence lifted 2026/10/24 00:21:24 moves: shop.example.com: offline move mv_oukeoqnajqhy2orte7e2hw5p7u: opened; the row is ready and adoption stages its build 2026/10/24 00:21:24 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:24 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:24 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:24 provision: not removing shop.example.com (record gone): participant of a running operation 2026/10/24 00:21:24 moves: shop.example.com: offline move mv_oukeoqnajqhy2orte7e2hw5p7u: waiting for reconcile to remove the row 2026/10/24 00:21:24 provision: removed retired instance route shop.example.com (record gone) 2026/10/24 00:21:24 moves: shop.example.com: offline move mv_oukeoqnajqhy2orte7e2hw5p7u: tombstone lifted 2026/10/24 00:21:24 console: self-serve move mv_oukeoqnajqhy2orte7e2hw5p7u (shop.example.com): done 2026/10/24 00:21:24 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:24 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:24 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:24 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:24 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:24 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:24 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:24 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797 2026/10/24 00:21:24 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:24 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:24 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:24 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:24 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:24 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841 2026/10/24 00:21:24 ERROR retention enforcement failed system=store db=shop.example.com.db error="enforce snapshot retention: remove ltx files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed" 2026/10/24 00:21:24 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:24 ERROR enforce L0 time retention system=store db=shop.example.com.db error="remove expired l0 files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed" 2026/10/24 00:21:24 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797 2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:24 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:24 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaafd: queue write: provision: record apps/bac/app/instances/shop.example.com.json: incarnation 1 record_seq 1, expected 1 and 0: store: compare-and-swap conflict (confirming from the record) 2026/10/24 00:21:24 provision: converged shop.example.com steering "" -> "latency" 2026/10/24 00:21:24 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaafd (shop.example.com): aborting: shop.example.com is steered or runs under a unit. offline moves cannot carry either yet 2026/10/24 00:21:24 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaafd (shop.example.com): aborted shop.example.com is steered or runs under a unit. offline moves cannot carry either yet 2026/10/24 00:21:24 provision: minted instance route shop.example.com (app "app" acct "bac") 2026/10/24 00:21:24 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:24 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:24 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:24 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:24 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001 2026/10/24 00:21:24 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797 2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001 2026/10/24 00:21:24 INFO starting compaction monitor system=store level=1 interval=1h0m0s 2026/10/24 00:21:24 INFO starting compaction monitor system=store level=3 interval=24h0m0s 2026/10/24 00:21:24 INFO starting compaction monitor system=store level=2 interval=6h0m0s 2026/10/24 00:21:24 INFO starting compaction monitor system=store level=9 interval=24h0m0s 2026/10/24 00:21:24 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s 2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002 2026/10/24 00:21:24 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841 2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002 2026/10/24 00:21:24 console: materialized account bac (fixture) from the bucket 2026/10/24 00:21:24 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafe: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaafe: ok true 2026/10/24 00:21:24 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafe: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged 2026/10/24 00:21:25 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaafe: start write: provision: record apps/bac/app/instances/shop.example.com.json: incarnation 1 record_seq 8, expected 1 and 7: store: compare-and-swap conflict (confirming from the record) 2026/10/24 00:21:25 provision: converged shop.example.com steering "" -> "latency" 2026/10/24 00:21:25 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaafe (shop.example.com): aborting: shop.example.com is steered or runs under a unit. offline moves cannot carry either yet 2026/10/24 00:21:25 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafe: the staged copy in data-uk/moves/shop.example.com/ is deleted 2026/10/24 00:21:25 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaafe (shop.example.com): aborted shop.example.com is steered or runs under a unit. offline moves cannot carry either yet FAIL FAIL github.com/carlosframework/platform/internal/e2e/selfserve 40.806s FAIL === e2e@3 (31 s) ok github.com/carlosframework/platform/internal/e2e/selfserve 29.174s === e2e-vet (2 s) FAILED: rest e2e@2 make: *** [Makefile:19: test] Error 1 amadan: step gate failed: exit status 2 -
cross not run
No output.
-
minio-contract not run
No output.
Whole run log
amadan: log truncated — showing the last 406991 bytes of 641489
2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:51 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:51 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabd: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabd: ok true
2026/10/24 00:20:51 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabd: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:20:51 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:51 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:51 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:51 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:51 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:51 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:51 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:51 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000001.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument"
2026/10/24 00:20:51 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:51 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:51 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:51 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:51 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:51 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:51 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:51 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:20:51 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:51 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabe: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabe: ok true
2026/10/24 00:20:51 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabe: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:20:51 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:51 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:51 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:51 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:51 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:51 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:51 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:51 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:52 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:52 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:52 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:20:52 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument"
2026/10/24 00:20:52 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:52 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabf: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabf: ok true
2026/10/24 00:20:52 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabf: staged round: 4 object(s) changed, 3232 bytes copied; 4 of 4 objects staged
2026/10/24 00:20:52 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:52 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:52 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:52 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000001.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument"
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:52 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:52 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:52 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:52 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:20:52 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:52 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabg: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabg: ok true
2026/10/24 00:20:52 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabg: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:20:52 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:52 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:52 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:52 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:52 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:52 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:52 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabh: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabh: ok true
2026/10/24 00:20:52 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabh: staged round: 3 object(s) changed, 2479 bytes copied; 3 of 3 objects staged
2026/10/24 00:20:52 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabh: estimate write: faultStore: the reply was lost (confirming from the record)
2026/10/24 00:20:52 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:52 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:52 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:52 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:52 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:52 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:20:52 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:52 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabi: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabi: ok true
2026/10/24 00:20:52 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabi: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:20:52 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabi: estimate write: context deadline exceeded (confirming from the record)
2026/10/24 00:20:52 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:52 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:52 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:52 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:52 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:52 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:52 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:52 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:53 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:53 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabj: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabj: ok true
2026/10/24 00:20:53 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabj: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:20:53 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabj: estimate write: context deadline exceeded (confirming from the record)
2026/10/24 00:20:53 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:53 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:53 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:53 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:53 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:53 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:53 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabk: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabk: ok true
2026/10/24 00:20:53 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabk: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:20:53 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabk: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:20:53 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabk (shop.example.com): aborted the move was cancelled
2026/10/24 00:20:53 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:53 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:53 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:53 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:53 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:53 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument"
2026/10/24 00:20:53 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:53 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabl: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabl: ok true
2026/10/24 00:20:53 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabl: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged
2026/10/24 00:20:53 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabl: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:20:53 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabl (shop.example.com): aborted the move was cancelled
2026/10/24 00:20:53 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:53 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:53 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:53 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000001.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument"
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:53 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:53 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:53 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:53 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:53 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 4: invalid argument"
2026/10/24 00:20:53 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:20:53 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:53 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabm: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabm: ok true
2026/10/24 00:20:53 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabm: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:20:53 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabm: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:20:53 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabm (shop.example.com): aborted the move was cancelled
2026/10/24 00:20:54 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:54 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:54 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:54 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:54 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:54 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:54 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabn: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabn: ok true
2026/10/24 00:20:54 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabn: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:20:54 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabn: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:20:54 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabn (shop.example.com): aborted the move was cancelled
2026/10/24 00:20:54 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:54 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:54 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:54 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000001.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument"
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:54 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:54 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:54 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument"
2026/10/24 00:20:54 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:54 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabo: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabo: ok true
2026/10/24 00:20:54 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabo: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged
2026/10/24 00:20:54 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabo: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:20:54 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabo (shop.example.com): aborted the move was cancelled
2026/10/24 00:20:54 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:54 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:54 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:54 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:54 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:54 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:54 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:54 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabp: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabp: ok true
2026/10/24 00:20:54 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabp: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:20:54 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabp: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:20:54 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabp (shop.example.com): aborted the move was cancelled
2026/10/24 00:20:54 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:54 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:54 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:54 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:54 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:54 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:54 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:54 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000002 size=841
2026/10/24 00:20:54 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:54 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabq: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabq: ok true
2026/10/24 00:20:54 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabq: staged round: 3 object(s) changed, 2479 bytes copied; 3 of 3 objects staged
2026/10/24 00:20:55 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabq: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:20:55 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabq (shop.example.com): aborted the move was cancelled
2026/10/24 00:20:55 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:55 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:55 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:55 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000001.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument"
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:55 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:55 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:55 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 4: invalid argument"
2026/10/24 00:20:55 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:55 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabr: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabr: ok true
2026/10/24 00:20:55 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabr: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged
2026/10/24 00:20:55 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabr: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:20:55 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabr (shop.example.com): aborted the move was cancelled
2026/10/24 00:20:55 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:55 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:55 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:55 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:55 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:55 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:20:55 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:20:55 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:55 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabs: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabs: ok true
2026/10/24 00:20:55 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabs: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:20:55 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabs: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:20:55 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabs (shop.example.com): aborted the move was cancelled
2026/10/24 00:20:55 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:55 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:55 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:55 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:55 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:55 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:55 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument"
2026/10/24 00:20:55 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:55 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabt: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabt: ok true
2026/10/24 00:20:55 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabt: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:20:55 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabt: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:20:55 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabt (shop.example.com): aborted the move was cancelled
2026/10/24 00:20:55 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:55 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:55 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:55 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument"
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:55 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:55 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:55 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:55 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:55 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:20:55 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:55 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabu: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabu: ok true
2026/10/24 00:20:56 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabu: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:20:56 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabu: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:20:56 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabu (shop.example.com): aborted the move was cancelled
2026/10/24 00:20:56 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:56 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:56 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:56 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:56 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:56 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:20:56 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:56 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabv: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabv: ok true
2026/10/24 00:20:56 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabv: staged round: 3 object(s) changed, 2479 bytes copied; 3 of 3 objects staged
2026/10/24 00:20:56 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabv: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:20:56 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabv (shop.example.com): aborted the move was cancelled
2026/10/24 00:20:56 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:56 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:56 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:56 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:56 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:56 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:56 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000002 size=841
2026/10/24 00:20:56 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:56 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabw: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabw: ok true
2026/10/24 00:20:56 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabw: staged round: 4 object(s) changed, 3320 bytes copied; 4 of 4 objects staged
2026/10/24 00:20:56 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabw: abort write: faultStore: the reply was lost (confirming from the record)
2026/10/24 00:20:56 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabw: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:20:56 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabw (shop.example.com): aborted the move was cancelled
2026/10/24 00:20:56 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:56 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:56 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:56 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:56 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:56 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument"
2026/10/24 00:20:56 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:56 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabx: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabx: ok true
2026/10/24 00:20:56 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabx: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:20:56 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabx: abort write: context deadline exceeded (confirming from the record)
2026/10/24 00:20:56 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabx: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:20:56 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaabx (shop.example.com): aborted the move was cancelled
2026/10/24 00:20:56 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:56 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:56 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:56 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:56 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:56 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:56 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:56 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 4: invalid argument"
2026/10/24 00:20:56 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000002 size=841
2026/10/24 00:20:56 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:57 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaby: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaby: ok true
2026/10/24 00:20:57 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaby: staged round: 3 object(s) changed, 2479 bytes copied; 3 of 3 objects staged
2026/10/24 00:20:57 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaby: abort write: context deadline exceeded (confirming from the record)
2026/10/24 00:20:57 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaby: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:20:57 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaby (shop.example.com): aborted the move was cancelled
2026/10/24 00:20:57 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:57 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:57 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:57 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:57 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:57 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:57 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabz: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaabz: ok true
2026/10/24 00:20:57 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaabz: staged round: 3 object(s) changed, 2479 bytes copied; 3 of 3 objects staged
2026/10/24 00:20:57 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaabz (was ready)
shop.example.com: config updated
2026/10/24 00:20:57 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:57 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:57 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:57 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:57 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:57 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:57 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:57 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaca: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaca: ok true
2026/10/24 00:20:57 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaca: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:20:57 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaaca (was ready)
shop.example.com: config updated
2026/10/24 00:20:57 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:57 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:57 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:57 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:57 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:57 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:20:57 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:20:57 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:57 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:57 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:58 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacb: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacb: ok true
2026/10/24 00:20:58 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacb: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:20:58 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacb (was ready)
shop.example.com: config updated
2026/10/24 00:20:58 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:58 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:58 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:58 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:58 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:58 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:58 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:58 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:58 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:58 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:58 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:58 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:58 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:20:58 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:58 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:20:58 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:58 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacc: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacc: ok true
2026/10/24 00:20:58 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacc: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:20:58 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacc (was ready)
shop.example.com: config updated
2026/10/24 00:20:58 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:58 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:58 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:58 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:58 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:58 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:58 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:58 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:58 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:58 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:58 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:58 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:58 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:20:58 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:58 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:20:58 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:58 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacd: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacd: ok true
2026/10/24 00:20:58 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacd: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:20:58 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacd (was ready)
shop.example.com: config updated
2026/10/24 00:20:58 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:58 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:58 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:58 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:58 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:58 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:58 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:58 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:59 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:59 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:59 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:59 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:59 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:59 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:59 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument"
2026/10/24 00:20:59 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:59 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaace: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaace: ok true
2026/10/24 00:20:59 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaace: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:20:59 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaace (was ready)
shop.example.com: config updated
2026/10/24 00:20:59 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:59 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:59 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:59 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:59 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:59 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:59 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:59 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:59 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:59 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:59 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:59 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:59 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:59 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument"
2026/10/24 00:20:59 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:59 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacf: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacf: ok true
2026/10/24 00:20:59 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacf: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:20:59 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacf (was ready)
shop.example.com: config updated
2026/10/24 00:20:59 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:20:59 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:59 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:59 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:59 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:59 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:20:59 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:20:59 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:20:59 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:20:59 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:20:59 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:20:59 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:20:59 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:20:59 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:59 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:20:59 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000002 size=841
2026/10/24 00:20:59 console: materialized account bac (fixture) from the bucket
2026/10/24 00:20:59 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacg: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacg: ok true
2026/10/24 00:20:59 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacg: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:21:00 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacg (was ready)
shop.example.com: config updated
2026/10/24 00:21:00 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:00 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:00 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:00 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:00 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:00 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 4: invalid argument"
2026/10/24 00:21:00 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:00 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaach: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaach: ok true
2026/10/24 00:21:00 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaach: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged
2026/10/24 00:21:00 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaach (was ready)
shop.example.com: config updated
2026/10/24 00:21:00 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:00 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:00 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:00 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:00 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:00 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:00 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaci: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaci: ok true
2026/10/24 00:21:00 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaci: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged
2026/10/24 00:21:00 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaaci (was ready)
shop.example.com: config updated
2026/10/24 00:21:00 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:00 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:00 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:00 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:00 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:00 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:00 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:00 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:00 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:01 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacj: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacj: ok true
2026/10/24 00:21:01 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacj: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:21:01 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacj (was ready)
shop.example.com: config updated
2026/10/24 00:21:01 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:01 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:01 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:01 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:01 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:01 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:01 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:01 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:01 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:01 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:01 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:01 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:01 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:01 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:01 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:01 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:01 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaack: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaack: ok true
2026/10/24 00:21:01 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaack: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:21:01 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaack (was ready)
shop.example.com: config updated
2026/10/24 00:21:01 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:01 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:01 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:01 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:01 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:01 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:01 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:01 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:01 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:01 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:01 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:01 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:01 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:01 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:01 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:01 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacl: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacl: ok true
2026/10/24 00:21:01 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacl: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged
2026/10/24 00:21:02 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaacl: start write: faultStore: the reply was lost (confirming from the record)
2026/10/24 00:21:02 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacl (was ready)
shop.example.com: config updated
2026/10/24 00:21:02 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:02 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:02 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:02 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:02 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:02 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:02 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:02 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:02 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacm: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacm: ok true
2026/10/24 00:21:02 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacm: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:21:02 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaacm: start write: context deadline exceeded (confirming from the record)
2026/10/24 00:21:02 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacm (was ready)
shop.example.com: config updated
2026/10/24 00:21:02 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:02 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:02 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:02 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:02 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:02 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:02 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:02 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacn: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacn: ok true
2026/10/24 00:21:02 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacn: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:21:02 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaacn: start write: context deadline exceeded (confirming from the record)
2026/10/24 00:21:02 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacn (was ready)
shop.example.com: config updated
2026/10/24 00:21:02 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:02 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:02 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:02 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:02 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:02 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:02 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:02 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:02 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:02 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:03 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaco: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaco: ok true
2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaco: staged round: 5 object(s) changed, 4073 bytes copied; 5 of 5 objects staged
2026/10/24 00:21:03 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaaco (was ready)
shop.example.com: config updated
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:03 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003
2026/10/24 00:21:03 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaco: checkpointed the local database (position 2, replica 2)
2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaco: sealed at TXID 3 (6 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b)
2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaco: sealed
2026/10/24 00:21:03 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:03 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaaco (app "app" acct "bac")
2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaco: extracted the files and env
shop.example.com: config updated
2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaco: landed; every item is OK
2026/10/24 00:21:03 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:03 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:03 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:03 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:03 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:03 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:03 ERROR retention enforcement failed system=store db=shop.example.com.db error="enforce snapshot retention: remove ltx files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed"
2026/10/24 00:21:03 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:03 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacp: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacp: ok true
2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacp: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:21:03 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacp (was ready)
shop.example.com: config updated
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:03 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003
2026/10/24 00:21:03 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacp: checkpointed the local database (position 2, replica 2)
2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacp: sealed at TXID 3 (6 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b)
2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacp: sealed
2026/10/24 00:21:03 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:03 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaacp (app "app" acct "bac")
2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacp: extracted the files and env
shop.example.com: config updated
2026/10/24 00:21:03 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacp: landed; every item is OK
2026/10/24 00:21:03 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:03 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:03 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:03 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:03 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:03 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:03 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:03 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:04 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacq: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacq: ok true
2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacq: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:21:04 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacq (was ready)
shop.example.com: config updated
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:04 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003
2026/10/24 00:21:04 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:04 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument"
2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacq: checkpointed the local database (position 2, replica 2)
2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacq: sealed at TXID 3 (4 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b)
2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacq: sealed
2026/10/24 00:21:04 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:04 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaacq (app "app" acct "bac")
2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacq: extracted the files and env
shop.example.com: config updated
2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacq: landed; every item is OK
2026/10/24 00:21:04 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:04 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:04 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:04 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:04 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:04 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:04 ERROR retention enforcement failed system=store db=shop.example.com.db error="enforce snapshot retention: remove ltx files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed"
2026/10/24 00:21:04 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:04 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacr: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacr: ok true
2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacr: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:21:04 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacr (was ready)
shop.example.com: config updated
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:04 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003
2026/10/24 00:21:04 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacr: checkpointed the local database (position 2, replica 2)
2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacr: sealed at TXID 3 (5 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b)
2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacr: sealed
2026/10/24 00:21:04 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:04 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaacr (app "app" acct "bac")
2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacr: extracted the files and env
shop.example.com: config updated
2026/10/24 00:21:04 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacr: landed; every item is OK
2026/10/24 00:21:04 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:04 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:04 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:04 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:04 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:04 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:04 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:04 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:05 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacs: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacs: ok true
2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacs: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:21:05 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacs (was ready)
shop.example.com: config updated
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:05 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003
2026/10/24 00:21:05 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacs: checkpointed the local database (position 2, replica 2)
2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacs: sealed at TXID 3 (4 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b)
2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacs: sealed
2026/10/24 00:21:05 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:05 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaacs (app "app" acct "bac")
2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacs: extracted the files and env
shop.example.com: config updated
2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacs: landed; every item is OK
2026/10/24 00:21:05 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:05 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:05 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:05 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument"
2026/10/24 00:21:05 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:05 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:05 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:05 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:05 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaact: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaact: ok true
2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaact: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:21:05 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaact (was ready)
shop.example.com: config updated
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:05 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003
2026/10/24 00:21:05 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaact: checkpointed the local database (position 2, replica 2)
2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaact: sealed at TXID 3 (5 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b)
2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaact: sealed
2026/10/24 00:21:05 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:05 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaact (app "app" acct "bac")
2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaact: extracted the files and env
shop.example.com: config updated
2026/10/24 00:21:05 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaact: landed; every item is OK
2026/10/24 00:21:05 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:05 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:05 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:05 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:05 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:05 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:05 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:05 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:05 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:06 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacu: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacu: ok true
2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacu: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:21:06 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacu (was ready)
shop.example.com: config updated
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:06 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003
2026/10/24 00:21:06 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:06 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000003 size=880
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacu: checkpointed the local database (position 2, replica 2)
2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacu: sealed at TXID 3 (5 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b)
2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacu: sealed
2026/10/24 00:21:06 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:06 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaacu (app "app" acct "bac")
2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacu: extracted the files and env
shop.example.com: config updated
2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacu: landed; every item is OK
2026/10/24 00:21:06 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:06 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:06 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:06 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:06 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:06 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:06 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:06 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacv: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacv: ok true
2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacv: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:21:06 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacv (was ready)
shop.example.com: config updated
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:06 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003
2026/10/24 00:21:06 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacv: checkpointed the local database (position 2, replica 2)
2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacv: sealed at TXID 3 (5 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b)
2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacv: sealed
2026/10/24 00:21:06 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:06 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaacv (app "app" acct "bac")
2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacv: extracted the files and env
shop.example.com: config updated
2026/10/24 00:21:06 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacv: landed; every item is OK
2026/10/24 00:21:06 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:06 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:06 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:06 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:06 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:06 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:06 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:06 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:06 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:06 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:07 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacw: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacw: ok true
2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacw: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:21:07 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacw (was ready)
shop.example.com: config updated
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:07 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003
2026/10/24 00:21:07 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:07 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000003 size=880
2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacw: checkpointed the local database (position 2, replica 2)
2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacw: sealed at TXID 3 (6 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b)
2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacw: sealed
2026/10/24 00:21:07 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:07 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaacw (app "app" acct "bac")
2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacw: extracted the files and env
shop.example.com: config updated
2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacw: landed; every item is OK
2026/10/24 00:21:07 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:07 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:07 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:07 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:07 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:07 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:07 ERROR retention enforcement failed system=store db=shop.example.com.db error="enforce snapshot retention: remove ltx files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed"
2026/10/24 00:21:07 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:07 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:07 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacx: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacx: ok true
2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacx: staged round: 5 object(s) changed, 4073 bytes copied; 5 of 5 objects staged
2026/10/24 00:21:07 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacx (was ready)
shop.example.com: config updated
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:07 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003
2026/10/24 00:21:07 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:07 INFO compaction complete system=store db=shop.example.com.db level=2 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacx: checkpointed the local database (position 2, replica 2)
2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacx: sealed at TXID 3 (7 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b)
2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacx: sealed
2026/10/24 00:21:07 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:07 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaacx (app "app" acct "bac")
2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacx: extracted the files and env
shop.example.com: config updated
2026/10/24 00:21:07 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacx: landed; every item is OK
2026/10/24 00:21:07 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:07 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:07 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:07 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:07 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:07 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:07 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:07 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:07 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:08 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacy: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacy: ok true
2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacy: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:21:08 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacy (was ready)
shop.example.com: config updated
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:08 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003
2026/10/24 00:21:08 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:08 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000003.ltx: read upload data failed: write snapshot ltx: read database page 4: invalid argument"
2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacy: checkpointed the local database (position 2, replica 2)
2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacy: sealed at TXID 3 (4 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b)
2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacy: sealed
2026/10/24 00:21:08 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:08 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaacy (app "app" acct "bac")
2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacy: extracted the files and env
shop.example.com: config updated
2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacy: landed; every item is OK
2026/10/24 00:21:08 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:08 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:08 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:08 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:08 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:08 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:08 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:08 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument"
2026/10/24 00:21:08 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacz: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaacz: ok true
2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacz: staged round: 4 object(s) changed, 3232 bytes copied; 4 of 4 objects staged
2026/10/24 00:21:08 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaacz (was ready)
shop.example.com: config updated
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:08 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003
2026/10/24 00:21:08 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:08 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument"
2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacz: checkpointed the local database (position 2, replica 2)
2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacz: sealed at TXID 3 (5 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b)
2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacz: sealed
2026/10/24 00:21:08 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:08 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaacz (app "app" acct "bac")
2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacz: extracted the files and env
shop.example.com: config updated
2026/10/24 00:21:08 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaacz: landed; every item is OK
2026/10/24 00:21:08 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:08 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:08 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:08 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:08 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:08 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:08 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:08 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:08 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:08 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:09 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaada: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaada: ok true
2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaada: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:21:09 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaada (was ready)
shop.example.com: config updated
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:09 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:09 ERROR retention enforcement failed system=store db=shop.example.com.db error="enforce snapshot retention: remove ltx files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed"
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003
2026/10/24 00:21:09 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaada: checkpointed the local database (position 2, replica 2)
2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaada: sealed at TXID 3 (5 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b)
2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaada: sealed
2026/10/24 00:21:09 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaada: flip write: faultStore: the reply was lost (confirming from the record)
2026/10/24 00:21:09 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:09 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaada (app "app" acct "bac")
2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaada: extracted the files and env
shop.example.com: config updated
2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaada: landed; every item is OK
2026/10/24 00:21:09 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:09 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:09 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:09 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:09 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:09 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:09 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadb: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadb: ok true
2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadb: staged round: 3 object(s) changed, 2479 bytes copied; 3 of 3 objects staged
2026/10/24 00:21:09 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaadb (was ready)
shop.example.com: config updated
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:09 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003
2026/10/24 00:21:09 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadb: checkpointed the local database (position 2, replica 2)
2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadb: sealed at TXID 3 (4 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b)
2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadb: sealed
2026/10/24 00:21:09 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadb: flip write: context deadline exceeded (confirming from the record)
2026/10/24 00:21:09 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:09 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaadb (app "app" acct "bac")
2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadb: extracted the files and env
shop.example.com: config updated
2026/10/24 00:21:09 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadb: landed; every item is OK
2026/10/24 00:21:09 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:09 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:09 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:09 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:09 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:09 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:09 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:09 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:09 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument"
2026/10/24 00:21:09 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:10 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadc: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadc: ok true
2026/10/24 00:21:10 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadc: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:21:10 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaadc (was ready)
shop.example.com: config updated
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:10 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003
2026/10/24 00:21:10 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880
2026/10/24 00:21:10 INFO compaction complete system=store db=shop.example.com.db level=2 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:10 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000003 size=880
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:10 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadc: checkpointed the local database (position 2, replica 2)
2026/10/24 00:21:10 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadc: sealed at TXID 3 (6 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b)
2026/10/24 00:21:10 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadc: sealed
2026/10/24 00:21:10 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadc: flip write: context deadline exceeded (confirming from the record)
2026/10/24 00:21:10 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:10 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaadc (app "app" acct "bac")
2026/10/24 00:21:10 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadc: extracted the files and env
shop.example.com: config updated
2026/10/24 00:21:10 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadc: landed; every item is OK
2026/10/24 00:21:10 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:10 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:10 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:10 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:10 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument"
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:10 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:10 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:10 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:10 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadd: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadd: ok true
2026/10/24 00:21:10 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadd (shop.example.com): aborting: could not determine the replica's size on ie-box
2026/10/24 00:21:10 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadd (shop.example.com): aborted could not determine the replica's size on ie-box
2026/10/24 00:21:10 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:10 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:10 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:10 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:10 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:10 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:10 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:10 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:10 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaade: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaade: ok true
2026/10/24 00:21:10 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaade (shop.example.com): aborting: could not determine the replica's size on ie-box
2026/10/24 00:21:10 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaade (shop.example.com): aborted could not determine the replica's size on ie-box
2026/10/24 00:21:10 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:10 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:10 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:10 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:10 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:10 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:10 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:10 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadf: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadf: ok true
2026/10/24 00:21:10 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadf (shop.example.com): aborting: this account's files are too large to move this way
2026/10/24 00:21:10 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadf (shop.example.com): aborted this account's files are too large to move this way
2026/10/24 00:21:10 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:10 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:10 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:10 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:10 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:10 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:10 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadg: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadg: ok true
2026/10/24 00:21:10 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:10 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:10 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:10 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000001.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument"
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:10 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:10 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:10 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:10 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:10 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:10 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:11 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:11 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadh: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadh: ok true
2026/10/24 00:21:11 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadh: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:21:11 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadh: waiting: Waiting for ie-box in ie to confirm it has stopped shop.example.com. shop.example.com is offline.
2026/10/24 00:21:11 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadh: waiting: Waiting for ie-box in ie to confirm it has stopped shop.example.com. shop.example.com is offline.
2026/10/24 00:21:11 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadh: waiting: Waiting for ie-box in ie to confirm it has stopped shop.example.com. shop.example.com is offline.
2026/10/24 00:21:11 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:11 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:11 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:11 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:11 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:11 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument"
2026/10/24 00:21:11 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:11 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadi: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadi: ok true
2026/10/24 00:21:11 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadi: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:21:11 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:11 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:11 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:11 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:11 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:11 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:11 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:11 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:11 ERROR retention enforcement failed system=store db=shop.example.com.db error="enforce snapshot retention: remove ltx files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed"
2026/10/24 00:21:11 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:11 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadj: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadj: ok true
2026/10/24 00:21:11 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadj: staged round: 5 object(s) changed, 4073 bytes copied; 5 of 5 objects staged
2026/10/24 00:21:11 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadj (shop.example.com): sealed, the seal not read: the copy's bytes are unknown (no copy timing)
2026/10/24 00:21:11 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadj: waiting: this move is offline-sealed. run carlos move offline status --host shop.example.com to see what it waits for
2026/10/24 00:21:11 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadj: waiting: this move is offline-sealed. run carlos move offline status --host shop.example.com to see what it waits for
2026/10/24 00:21:11 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:11 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:11 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:11 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:11 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:11 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:11 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:11 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:11 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:11 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadk: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadk: ok true
2026/10/24 00:21:11 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadk: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:21:11 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:12 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:12 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:12 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument"
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:12 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:12 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:12 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:12 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:12 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadl: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadl: ok true
2026/10/24 00:21:12 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadl: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:21:12 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadl: waiting: this move is offline-landed. run carlos move offline status --host shop.example.com to see what it waits for
2026/10/24 00:21:12 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadl: waiting: this move is offline-landed. run carlos move offline status --host shop.example.com to see what it waits for
2026/10/24 00:21:12 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadl: waiting: this move is offline-landed. run carlos move offline status --host shop.example.com to see what it waits for
2026/10/24 00:21:12 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:12 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:12 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:12 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:12 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:12 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:12 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument"
2026/10/24 00:21:12 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:12 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadm: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadm: ok true
2026/10/24 00:21:12 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadm: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:21:12 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:12 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:12 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:12 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:12 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:12 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:12 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadn: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadn: ok true
2026/10/24 00:21:12 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadn: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged
2026/10/24 00:21:12 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:12 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:12 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:12 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:12 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:12 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:12 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:12 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:12 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:12 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaado: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaado: ok true
2026/10/24 00:21:12 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaado: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:21:13 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:13 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:13 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:13 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:13 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:13 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:13 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:13 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:13 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadp: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadp: ok true
2026/10/24 00:21:13 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadp: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:21:13 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadp (shop.example.com): an operator started the move; the job yields it
2026/10/24 00:21:13 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:13 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:13 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:13 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:13 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:13 ERROR enforce L0 time retention system=store db=shop.example.com.db error="remove expired l0 files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed"
2026/10/24 00:21:13 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000002 size=841
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:13 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:13 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadq: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadq: ok true
2026/10/24 00:21:13 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadq: staged round: 4 object(s) changed, 3320 bytes copied; 4 of 4 objects staged
2026/10/24 00:21:13 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadq (shop.example.com): an operator started the move; the job yields it
2026/10/24 00:21:13 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:13 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:13 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:13 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:13 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:13 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:13 ERROR retention enforcement failed system=store db=shop.example.com.db error="enforce snapshot retention: remove ltx files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed"
2026/10/24 00:21:13 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:13 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:13 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:13 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadr: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadr: ok true
2026/10/24 00:21:13 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadr: staged round: 5 object(s) changed, 4073 bytes copied; 5 of 5 objects staged
2026/10/24 00:21:13 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadr (shop.example.com): an operator started the move; the job yields it
2026/10/24 00:21:13 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:13 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:13 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:13 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:13 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:13 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:13 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:13 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:13 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:13 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaads: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaads: ok true
2026/10/24 00:21:13 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaads: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:21:14 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaads (shop.example.com): an operator started the move; the job yields it
2026/10/24 00:21:14 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:14 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:14 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:14 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:14 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:14 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument"
2026/10/24 00:21:14 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:14 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadt: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadt: ok true
2026/10/24 00:21:14 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadt: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged
2026/10/24 00:21:14 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadt (shop.example.com): an operator started the move; the job yields it
2026/10/24 00:21:14 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:14 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:14 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:14 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:14 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:14 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:14 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:14 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:14 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadu: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadu: ok true
2026/10/24 00:21:14 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadu: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:21:14 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadu: waiting: Waiting for ie-box in ie to confirm it has stopped shop.example.com. shop.example.com is offline.
2026/10/24 00:21:14 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadu: waiting: Waiting for ie-box in ie to confirm it has stopped shop.example.com. shop.example.com is offline.
2026/10/24 00:21:14 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadu: waiting: Waiting for ie-box in ie to confirm it has stopped shop.example.com. shop.example.com is offline.
2026/10/24 00:21:14 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:14 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:14 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:14 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:14 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:14 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:14 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:14 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:14 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadv: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadv: ok true
2026/10/24 00:21:14 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadv: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:21:14 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadv: waiting: Waiting for ie-box in ie to confirm it has stopped shop.example.com. shop.example.com is offline.
2026/10/24 00:21:14 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadv: waiting: Waiting for ie-box in ie to confirm it has stopped shop.example.com. shop.example.com is offline.
2026/10/24 00:21:14 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadv: waiting: Waiting for ie-box in ie to confirm it has stopped shop.example.com. shop.example.com is offline.
2026/10/24 00:21:14 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:14 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:14 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:14 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:14 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:14 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:14 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:14 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:14 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:14 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:15 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadw: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadw: ok true
2026/10/24 00:21:15 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadw: staged round: 4 object(s) changed, 3232 bytes copied; 4 of 4 objects staged
2026/10/24 00:21:15 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadw: waiting: this move is offline-sealed. run carlos move offline status --host shop.example.com to see what it waits for
2026/10/24 00:21:15 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadw: waiting: this move is offline-sealed. run carlos move offline status --host shop.example.com to see what it waits for
2026/10/24 00:21:15 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadw: waiting: this move is offline-sealed. run carlos move offline status --host shop.example.com to see what it waits for
2026/10/24 00:21:15 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaadw: waiting: this move is offline-sealed. run carlos move offline status --host shop.example.com to see what it waits for
2026/10/24 00:21:15 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:15 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:15 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:15 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:15 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:15 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:15 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:15 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:15 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadx: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadx: ok true
2026/10/24 00:21:15 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadx: staged round: 5 object(s) changed, 4073 bytes copied; 5 of 5 objects staged
2026/10/24 00:21:15 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:15 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:15 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:15 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:15 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:15 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:15 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:15 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaady: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaady: ok true
2026/10/24 00:21:15 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaady: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:21:15 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:15 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:15 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:15 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:15 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:15 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument"
2026/10/24 00:21:15 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:15 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadz: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaadz: ok true
2026/10/24 00:21:15 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaadz: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged
2026/10/24 00:21:15 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:15 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:15 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:15 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:15 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:15 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:15 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:15 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:15 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:16 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaea: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaea: ok true
2026/10/24 00:21:16 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaea: staged round: 3 object(s) changed, 2479 bytes copied; 3 of 3 objects staged
2026/10/24 00:21:16 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:16 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:16 ERROR enforce L0 time retention system=store db=shop.example.com.db error="remove expired l0 files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed"
2026/10/24 00:21:16 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:16 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:16 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:16 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:16 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:16 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaec (shop.example.com): aborted the move could not be started
2026/10/24 00:21:16 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:16 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:16 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:16 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:16 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaee: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:21:16 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaed (shop.example.com): aborted another move is in flight
2026/10/24 00:21:16 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:16 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:16 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:16 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaeg: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:21:16 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaef (shop.example.com): aborted another move is in flight
2026/10/24 00:21:16 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:16 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:16 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:16 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaeh: staged round: 4 object(s) changed, 3232 bytes copied; 4 of 4 objects staged
2026/10/24 00:21:16 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:16 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000001.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument"
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:16 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:16 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaei (shop.example.com): the record is offline-queued, behind the row's barrier; the row is left where it is
2026/10/24 00:21:16 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaei (shop.example.com): the record is offline-queued, behind the row's barrier; the row is left where it is
2026/10/24 00:21:16 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaei (shop.example.com): the record is offline-queued, behind the row's barrier; the row is left where it is
2026/10/24 00:21:16 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:16 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:16 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:16 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:16 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:16 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:16 INFO compaction complete system=store db=shop.example.com.db level=2 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:16 ERROR enforce L0 time retention system=store db=shop.example.com.db error="remove expired l0 files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed"
2026/10/24 00:21:16 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:16 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:16 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaej: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaej: ok true
2026/10/24 00:21:16 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaej: staged round: 5 object(s) changed, 4073 bytes copied; 5 of 5 objects staged
2026/10/24 00:21:17 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:17 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument"
2026/10/24 00:21:17 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:17 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaek (shop.example.com): aborted the instance was recreated
2026/10/24 00:21:17 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:17 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:17 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:17 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaael (shop.example.com): aborting: a run is open on this instance: run_1. wait for it to finish or cancel it, then move the instance
2026/10/24 00:21:17 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaael (shop.example.com): aborted a run is open on this instance: run_1. wait for it to finish or cancel it, then move the instance
2026/10/24 00:21:17 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:17 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:17 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:17 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaem (shop.example.com): aborting: shop.example.com is steered or runs under a unit. offline moves cannot carry either yet
2026/10/24 00:21:17 provision: skipping record apps/bac/app/instances/shop.example.com.json: invalid shape: steering "canary": want "" or "latency"
2026/10/24 00:21:17 provision: skipping record apps/bac/app/instances/shop.example.com.json: invalid shape: steering "canary": want "" or "latency"
2026/10/24 00:21:17 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaem (shop.example.com): aborted shop.example.com is steered or runs under a unit. offline moves cannot carry either yet
2026/10/24 00:21:17 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:17 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:17 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaen: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaen: ok true
2026/10/24 00:21:17 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaen: stage write: faultStore: the reply was lost (confirming from the record)
2026/10/24 00:21:17 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaen: stage write: faultStore: the reply was lost (confirming from the record)
2026/10/24 00:21:17 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaen: stage write: faultStore: the reply was lost (confirming from the record)
2026/10/24 00:21:17 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaen: stage write: faultStore: the reply was lost (confirming from the record)
2026/10/24 00:21:17 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaen (shop.example.com): faultStore: the reply was lost
2026/10/24 00:21:17 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:17 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:17 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:17 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:17 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaeo: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaeo: ok true
2026/10/24 00:21:17 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaeo: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:21:17 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:17 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:17 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:17 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:17 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:17 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:18 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaep: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaep: ok true
2026/10/24 00:21:18 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaep: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged
2026/10/24 00:21:18 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaep: abort write: provision: record apps/bac/app/instances/shop.example.com.json: incarnation 1 record_seq 4, expected 1 and 3: store: compare-and-swap conflict (confirming from the record)
2026/10/24 00:21:18 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaep: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:21:18 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaep (shop.example.com): aborted the move was cancelled
2026/10/24 00:21:18 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:18 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:18 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:18 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:18 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:18 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:18 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:18 ERROR retention enforcement failed system=store db=shop.example.com.db error="enforce snapshot retention: remove ltx files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed"
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:18 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:18 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaeq: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaeq: ok true
2026/10/24 00:21:18 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaeq: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:21:18 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:18 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:18 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:18 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:18 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:18 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:18 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:18 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:18 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaer: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaer: ok true
2026/10/24 00:21:18 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaer: staged round: 5 object(s) changed, 4073 bytes copied; 5 of 5 objects staged
2026/10/24 00:21:18 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:18 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:18 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:18 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:18 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:18 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:18 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:18 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 1: invalid argument"
2026/10/24 00:21:18 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:18 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaes: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaes: ok true
2026/10/24 00:21:18 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaes: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged
2026/10/24 00:21:18 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaes: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:21:19 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaes (shop.example.com): aborted the move was cancelled
2026/10/24 00:21:19 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:19 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:19 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:19 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:19 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:19 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:19 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument"
2026/10/24 00:21:19 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:19 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaet: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaet: ok true
2026/10/24 00:21:19 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaet: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:21:19 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaet: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:21:19 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaet (shop.example.com): aborted the move was cancelled
2026/10/24 00:21:19 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:19 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:19 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:19 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:19 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:19 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000002 size=841
2026/10/24 00:21:19 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:19 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaeu (shop.example.com): aborted the move was cancelled
2026/10/24 00:21:19 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:19 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:19 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:19 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:19 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:19 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:19 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:19 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaev (shop.example.com): aborted the move was cancelled
2026/10/24 00:21:19 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:19 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:19 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:19 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000001.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument"
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:19 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:19 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:19 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:19 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:19 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaew: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaew: ok true
2026/10/24 00:21:19 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaew (shop.example.com): aborted the move was cancelled
2026/10/24 00:21:19 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:19 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:19 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:19 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:19 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:19 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:19 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:19 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:19 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:19 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:19 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:19 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaex: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaex: ok true
2026/10/24 00:21:19 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaex: staged round: 5 object(s) changed, 4073 bytes copied; 5 of 5 objects staged
2026/10/24 00:21:19 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaex: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:21:19 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaex (shop.example.com): aborted the move was cancelled
2026/10/24 00:21:20 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:20 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:20 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:20 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:20 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:20 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:20 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:20 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:20 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:20 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:20 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:20 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:20 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:20 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument"
2026/10/24 00:21:20 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaey: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaey: ok true
2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaey: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged
2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaey: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:21:20 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaey (shop.example.com): aborted the move was cancelled
2026/10/24 00:21:20 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:20 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:20 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:20 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:20 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:20 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:20 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:20 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument"
2026/10/24 00:21:20 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:20 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:20 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:20 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:20 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:20 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:20 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:20 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaez: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaaez: ok true
2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaez: staged round: 4 object(s) changed, 3276 bytes copied; 4 of 4 objects staged
2026/10/24 00:21:20 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaaez (was ready)
shop.example.com: config updated
2026/10/24 00:21:20 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:20 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:20 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:20 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:20 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003
2026/10/24 00:21:20 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880
2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:20 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:20 INFO compaction complete system=store db=shop.example.com.db level=2 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:20 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000003.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument"
2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaez: checkpointed the local database (position 2, replica 2)
2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaez: sealed at TXID 3 (6 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b)
2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaez: sealed
2026/10/24 00:21:20 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:20 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaaez (app "app" acct "bac")
2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaez: extracted the files and env
shop.example.com: config updated
2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaez: landed; every item is OK
2026/10/24 00:21:20 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:20 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaez: wake fence lifted
2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaez: opened; the row is ready and adoption stages its build
2026/10/24 00:21:20 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:20 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:20 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:20 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaez: waiting for reconcile to remove the row
2026/10/24 00:21:20 provision: removed retired instance route shop.example.com (record gone)
2026/10/24 00:21:20 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaaez: tombstone lifted
2026/10/24 00:21:20 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaaez (shop.example.com): done
2026/10/24 00:21:20 moves: shop.example.com: reported the leftovers of move mv_mvjobaaaaaaaaaaaaaaaaaaaez for return request 1792801280914498284-console-e2e-job-f6668b83
2026/10/24 00:21:20 moves: shop.example.com: reported the leftovers of move mv_mvjobaaaaaaaaaaaaaaaaaaaez for return request 1792801280936195449-console-e2e-job-a79abc6f
2026/10/24 00:21:20 moves: shop.example.com: reported the leftovers of move mv_mvjobaaaaaaaaaaaaaaaaaaaez for return request 1792801280953331764-console-e2e-job-5c5b0e9c
2026/10/24 00:21:21 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:21 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:21 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:21 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:21 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:21 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:21 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafa: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaafa: ok true
2026/10/24 00:21:21 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafa: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged
2026/10/24 00:21:21 provision: held shop.example.com moving by mv_mvjobaaaaaaaaaaaaaaaaaaafa (was ready)
shop.example.com: config updated
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:21 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003
2026/10/24 00:21:21 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:21 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000003.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument"
2026/10/24 00:21:21 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafa: checkpointed the local database (position 2, replica 2)
2026/10/24 00:21:21 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafa: sealed at TXID 3 (3 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b)
2026/10/24 00:21:21 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafa: sealed
2026/10/24 00:21:21 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaafa (shop.example.com): sealed, the seal not read: the copy's bytes are unknown (no copy timing)
2026/10/24 00:21:21 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:21 provision: minted instance route shop.example.com held landing by mv_mvjobaaaaaaaaaaaaaaaaaaafa (app "app" acct "bac")
2026/10/24 00:21:21 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafa: extracted the files and env
shop.example.com: config updated
2026/10/24 00:21:21 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafa: landed; every item is OK
2026/10/24 00:21:21 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:21 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:21 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafa: wake fence lifted
2026/10/24 00:21:21 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafa: opened; the row is ready and adoption stages its build
2026/10/24 00:21:21 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:21 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:21 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:21 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:21 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafa: waiting for reconcile to remove the row
2026/10/24 00:21:21 provision: removed retired instance route shop.example.com (record gone)
2026/10/24 00:21:21 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafa: tombstone lifted
2026/10/24 00:21:21 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaafa (shop.example.com): done
2026/10/24 00:21:21 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:21 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:21 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:21 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:21 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:21 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:21 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:21 ERROR retention enforcement failed system=store db=shop.example.com.db error="enforce snapshot retention: remove ltx files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed"
2026/10/24 00:21:21 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:21 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:21 moves: shop.example.com: offline move mv_p5ayh5372ejinhnvjatvjy5eyu: lane probe of data-uk/moves/_lane/mv_p5ayh5372ejinhnvjatvjy5eyu: ok true
2026/10/24 00:21:21 moves: shop.example.com: offline move mv_p5ayh5372ejinhnvjatvjy5eyu: staged round: 5 object(s) changed, 4073 bytes copied; 5 of 5 objects staged
2026/10/24 00:21:21 moves: shop.example.com: offline move mv_p5ayh5372ejinhnvjatvjy5eyu: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:21:21 console: self-serve move mv_p5ayh5372ejinhnvjatvjy5eyu (shop.example.com): aborted the move was cancelled
2026/10/24 00:21:21 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:21 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:21 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:21 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:21 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:21 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:21 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:21 ERROR compaction failed system=store db=shop.example.com.db level=9 error="s3: upload to instances/shop.example.com/0009/0000000000000001-0000000000000002.ltx: read upload data failed: write snapshot ltx: read database page 2: invalid argument"
2026/10/24 00:21:22 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:22 moves: shop.example.com: offline move mv_otlxpjmww64k4wtx67snax3ose: lane probe of data-uk/moves/_lane/mv_otlxpjmww64k4wtx67snax3ose: ok true
2026/10/24 00:21:22 moves: shop.example.com: offline move mv_otlxpjmww64k4wtx67snax3ose: staged round: 3 object(s) changed, 2435 bytes copied; 3 of 3 objects staged
2026/10/24 00:21:22 console: self-serve move mv_otlxpjmww64k4wtx67snax3ose (shop.example.com): aborting: the target box already holds data for shop.example.com: /tmp/TestSelfServeAbortByFailedPrecheck2204483583/007/shop.example.com.env
2026/10/24 00:21:22 moves: shop.example.com: offline move mv_otlxpjmww64k4wtx67snax3ose: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:21:22 moves: parked /tmp/TestSelfServeAbortByFailedPrecheck2204483583/007/shop.example.com.env at /tmp/TestSelfServeAbortByFailedPrecheck2204483583/007/.offline-moves/shop.example.com/mv_otlxpjmww64k4wtx67snax3ose/target/shop.example.com.env
2026/10/24 00:21:22 console: self-serve move mv_otlxpjmww64k4wtx67snax3ose (shop.example.com): aborted the target box already holds data for shop.example.com: /tmp/TestSelfServeAbortByFailedPrecheck2204483583/007/shop.example.com.env
2026/10/24 00:21:22 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:22 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:22 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:22 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:22 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:22 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:22 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:22 ERROR compaction failed system=store db=shop.example.com.db level=9 error="invalid argument"
2026/10/24 00:21:22 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:22 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:22 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:22 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:22 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:22 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:22 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:22 moves: shop.example.com: offline move mv_enq4u76p6dfv6ue2yfyfhvx3uy: lane probe of data-uk/moves/_lane/mv_enq4u76p6dfv6ue2yfyfhvx3uy: ok true
2026/10/24 00:21:22 moves: shop.example.com: offline move mv_enq4u76p6dfv6ue2yfyfhvx3uy: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged
2026/10/24 00:21:22 provision: held shop.example.com moving by mv_enq4u76p6dfv6ue2yfyfhvx3uy (was ready)
shop.example.com: config updated
2026/10/24 00:21:22 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:22 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:22 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:22 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:22 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003
2026/10/24 00:21:22 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000003 size=880
2026/10/24 00:21:22 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880
2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:22 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000003 size=880
2026/10/24 00:21:22 moves: shop.example.com: offline move mv_enq4u76p6dfv6ue2yfyfhvx3uy: checkpointed the local database (position 2, replica 2)
2026/10/24 00:21:22 moves: shop.example.com: offline move mv_enq4u76p6dfv6ue2yfyfhvx3uy: sealed at TXID 3 (5 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b)
2026/10/24 00:21:22 moves: shop.example.com: offline move mv_enq4u76p6dfv6ue2yfyfhvx3uy: sealed
2026/10/24 00:21:22 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:22 provision: minted instance route shop.example.com held landing by mv_enq4u76p6dfv6ue2yfyfhvx3uy (app "app" acct "bac")
2026/10/24 00:21:22 moves: shop.example.com: offline move mv_enq4u76p6dfv6ue2yfyfhvx3uy: extracted the files and env
shop.example.com: config updated
2026/10/24 00:21:22 moves: shop.example.com: offline move mv_enq4u76p6dfv6ue2yfyfhvx3uy: landed; every item is OK
2026/10/24 00:21:22 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:22 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:22 moves: shop.example.com: offline move mv_enq4u76p6dfv6ue2yfyfhvx3uy: wake fence lifted
2026/10/24 00:21:22 moves: shop.example.com: offline move mv_enq4u76p6dfv6ue2yfyfhvx3uy: opened; the row is ready and adoption stages its build
2026/10/24 00:21:22 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:22 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:22 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:22 moves: shop.example.com: offline move mv_enq4u76p6dfv6ue2yfyfhvx3uy: waiting for reconcile to remove the row
2026/10/24 00:21:22 provision: removed retired instance route shop.example.com (record gone)
2026/10/24 00:21:22 moves: shop.example.com: offline move mv_enq4u76p6dfv6ue2yfyfhvx3uy: tombstone lifted
2026/10/24 00:21:22 console: self-serve move mv_enq4u76p6dfv6ue2yfyfhvx3uy (shop.example.com): done
2026/10/24 00:21:22 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:22 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:22 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:22 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:22 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:22 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:22 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:22 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:22 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:22 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:22 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:22 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:22 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:22 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:22 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:22 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
2026/10/24 00:21:23 console: self-serve move mv_wglxmbun23fhunzthfe3iicnq4: waiting: the box ie-box has not reported recently
--- FAIL: TestClaimsOffInstanceIDIsPublished (0.97s)
selfserve_operator_test.go:337: after 60 rounds, still not in staging
2026/10/24 00:21:23 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:23 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:23 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:23 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:23 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:23 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:23 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:23 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:23 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:23 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:23 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:23 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:23 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:23 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:23 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:23 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:23 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:23 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:23 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:23 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:23 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:23 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:23 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:23 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:23 moves: shop.example.com: offline move mv_oukeoqnajqhy2orte7e2hw5p7u: lane probe of data-uk/moves/_lane/mv_oukeoqnajqhy2orte7e2hw5p7u: ok true
2026/10/24 00:21:24 moves: shop.example.com: offline move mv_oukeoqnajqhy2orte7e2hw5p7u: staged round: 5 object(s) changed, 4073 bytes copied; 5 of 5 objects staged
2026/10/24 00:21:24 provision: held shop.example.com moving by mv_oukeoqnajqhy2orte7e2hw5p7u (was ready)
shop.example.com: config updated
2026/10/24 00:21:24 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaafc (c.shop.example.com): aborting: box x-box was replaced
2026/10/24 00:21:24 provision: minted instance route b.shop.example.com (app "app" acct "bac")
2026/10/24 00:21:24 provision: minted instance route c.shop.example.com (app "app" acct "bac")
2026/10/24 00:21:24 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:24 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:24 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:24 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:24 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000003
2026/10/24 00:21:24 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=880
2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/24 00:21:24 moves: shop.example.com: offline move mv_oukeoqnajqhy2orte7e2hw5p7u: checkpointed the local database (position 2, replica 2)
2026/10/24 00:21:24 moves: shop.example.com: offline move mv_oukeoqnajqhy2orte7e2hw5p7u: sealed at TXID 3 (7 staged objects, files sha256:d5d1e68c7c5625619df1b080c376d8a6f08442e5e03c6d5969c2d51af245789b)
2026/10/24 00:21:24 moves: shop.example.com: offline move mv_oukeoqnajqhy2orte7e2hw5p7u: sealed
2026/10/24 00:21:24 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaafc (c.shop.example.com): aborted box x-box was replaced
2026/10/24 00:21:24 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:24 provision: minted instance route shop.example.com held landing by mv_oukeoqnajqhy2orte7e2hw5p7u (app "app" acct "bac")
2026/10/24 00:21:24 moves: shop.example.com: offline move mv_oukeoqnajqhy2orte7e2hw5p7u: extracted the files and env
shop.example.com: config updated
2026/10/24 00:21:24 moves: shop.example.com: offline move mv_oukeoqnajqhy2orte7e2hw5p7u: landed; every item is OK
2026/10/24 00:21:24 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:24 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:24 moves: shop.example.com: offline move mv_oukeoqnajqhy2orte7e2hw5p7u: wake fence lifted
2026/10/24 00:21:24 moves: shop.example.com: offline move mv_oukeoqnajqhy2orte7e2hw5p7u: opened; the row is ready and adoption stages its build
2026/10/24 00:21:24 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:24 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:24 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:24 provision: not removing shop.example.com (record gone): participant of a running operation
2026/10/24 00:21:24 moves: shop.example.com: offline move mv_oukeoqnajqhy2orte7e2hw5p7u: waiting for reconcile to remove the row
2026/10/24 00:21:24 provision: removed retired instance route shop.example.com (record gone)
2026/10/24 00:21:24 moves: shop.example.com: offline move mv_oukeoqnajqhy2orte7e2hw5p7u: tombstone lifted
2026/10/24 00:21:24 console: self-serve move mv_oukeoqnajqhy2orte7e2hw5p7u (shop.example.com): done
2026/10/24 00:21:24 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:24 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:24 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:24 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:24 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:24 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:24 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:24 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000001 size=797
2026/10/24 00:21:24 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:24 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:24 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:24 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:24 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:24 INFO snapshot complete system=store db=shop.example.com.db txid=0000000000000002 size=841
2026/10/24 00:21:24 ERROR retention enforcement failed system=store db=shop.example.com.db error="enforce snapshot retention: remove ltx files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed"
2026/10/24 00:21:24 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:24 ERROR enforce L0 time retention system=store db=shop.example.com.db error="remove expired l0 files: s3: delete batch of 1 objects: operation error S3: DeleteObjects, https response error StatusCode: 405, RequestID: , HostID: , api error MethodNotAllowed: MethodNotAllowed"
2026/10/24 00:21:24 INFO compaction complete system=store db=shop.example.com.db level=1 txid.min=0000000000000001 txid.max=0000000000000001 size=797
2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:24 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:24 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaafd: queue write: provision: record apps/bac/app/instances/shop.example.com.json: incarnation 1 record_seq 1, expected 1 and 0: store: compare-and-swap conflict (confirming from the record)
2026/10/24 00:21:24 provision: converged shop.example.com steering "" -> "latency"
2026/10/24 00:21:24 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaafd (shop.example.com): aborting: shop.example.com is steered or runs under a unit. offline moves cannot carry either yet
2026/10/24 00:21:24 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaafd (shop.example.com): aborted shop.example.com is steered or runs under a unit. offline moves cannot carry either yet
2026/10/24 00:21:24 provision: minted instance route shop.example.com (app "app" acct "bac")
2026/10/24 00:21:24 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:24 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:24 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:24 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:24 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/24 00:21:24 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=797
2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/24 00:21:24 INFO starting compaction monitor system=store level=1 interval=1h0m0s
2026/10/24 00:21:24 INFO starting compaction monitor system=store level=3 interval=24h0m0s
2026/10/24 00:21:24 INFO starting compaction monitor system=store level=2 interval=6h0m0s
2026/10/24 00:21:24 INFO starting compaction monitor system=store level=9 interval=24h0m0s
2026/10/24 00:21:24 INFO starting L0 retention monitor system=store interval=1h0m0s retention=5m0s
2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/24 00:21:24 INFO ltx file uploaded system=store db=shop.example.com.db replica=s3 level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=841
2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:24 INFO replica sync system=store db=shop.example.com.db replica=s3 txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/24 00:21:24 console: materialized account bac (fixture) from the bucket
2026/10/24 00:21:24 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafe: lane probe of data-uk/moves/_lane/mv_mvjobaaaaaaaaaaaaaaaaaaafe: ok true
2026/10/24 00:21:24 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafe: staged round: 2 object(s) changed, 1638 bytes copied; 2 of 2 objects staged
2026/10/24 00:21:25 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaafe: start write: provision: record apps/bac/app/instances/shop.example.com.json: incarnation 1 record_seq 8, expected 1 and 7: store: compare-and-swap conflict (confirming from the record)
2026/10/24 00:21:25 provision: converged shop.example.com steering "" -> "latency"
2026/10/24 00:21:25 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaafe (shop.example.com): aborting: shop.example.com is steered or runs under a unit. offline moves cannot carry either yet
2026/10/24 00:21:25 moves: shop.example.com: offline move mv_mvjobaaaaaaaaaaaaaaaaaaafe: the staged copy in data-uk/moves/shop.example.com/ is deleted
2026/10/24 00:21:25 console: self-serve move mv_mvjobaaaaaaaaaaaaaaaaaaafe (shop.example.com): aborted shop.example.com is steered or runs under a unit. offline moves cannot carry either yet
FAIL
FAIL github.com/carlosframework/platform/internal/e2e/selfserve 40.806s
FAIL
=== e2e@3 (31 s)
ok github.com/carlosframework/platform/internal/e2e/selfserve 29.174s
=== e2e-vet (2 s)
FAILED: rest e2e@2
make: *** [Makefile:19: test] Error 1
amadan: step gate failed: exit status 2
amadan: job failed at step gate — its output is above, not at the tail of this log
Watching for changes…