amadan
design: self-serve offline moves, rev 1
A tenant's customer moves their own instance between regions through one owner-delegated door that runs the proven offline move as a console job: stage while serving, stop, seal, flip, open, health check, finish, cleanup; a holding page with an estimate while offline; automatic return to the old region on failure. No host claims, no Phase 1. Paul's rulings of 2026-10-21 recorded in section 1.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
1 file changed,
+205
−0
docs/superpowers/specs/2026-10-21-self-serve-offline-moves-design.md+205 −0
CI passed — run details
CI log
=== gate ===
.amadan/test-gate.sh
go: downloading github.com/aws/aws-sdk-go-v2 v1.44.0
go: downloading github.com/aws/aws-sdk-go-v2/service/costexplorer v1.67.5
go: downloading github.com/aws/aws-sdk-go-v2/service/s3 v1.105.0
go: downloading github.com/aws/smithy-go v1.28.1
go: downloading github.com/aws/aws-sdk-go-v2/service/cloudwatch v1.68.0
go: downloading golang.org/x/net v0.56.0
go: downloading github.com/superfly/ltx v0.5.1
go: downloading modernc.org/sqlite v1.55.0
go: downloading github.com/aws/aws-sdk-go-v2/config v1.32.6
go: downloading github.com/aws/aws-sdk-go-v2/service/sns v1.43.0
go: downloading golang.org/x/crypto v0.53.0
go: downloading github.com/aws/aws-sdk-go-v2/service/acm v1.45.0
go: downloading github.com/aws/aws-sdk-go-v2/service/ec2 v1.324.0
go: downloading github.com/aws/aws-sdk-go-v2/service/iam v1.58.2
go: downloading github.com/aws/aws-sdk-go-v2/service/resourcegroupstaggingapi v1.37.0
go: downloading github.com/aws/aws-sdk-go-v2/service/route53 v1.64.1
go: downloading github.com/aws/aws-sdk-go-v2/service/sts v1.45.6
go: downloading github.com/benbjohnson/litestream v0.5.14
go: downloading github.com/oschwald/maxminddb-golang v1.13.1
go: downloading golang.org/x/sys v0.46.0
go: downloading modernc.org/sqlite v1.55.0
go: downloading github.com/aws/aws-sdk-go-v2/service/ec2 v1.324.0
go: downloading github.com/aws/aws-sdk-go-v2/service/autoscaling v1.74.0
go: downloading github.com/benbjohnson/litestream v0.5.14
go: downloading github.com/aws/aws-sdk-go-v2/service/ssm v1.74.0
go: downloading github.com/aws/aws-sdk-go-v2/service/sesv2 v1.66.6
go: downloading github.com/aws/aws-sdk-go-v2/credentials v1.19.6
go: downloading github.com/aws/aws-sdk-go-v2/credentials v1.19.6
go: downloading github.com/aws/aws-sdk-go-v2/service/autoscaling v1.74.0
go: downloading github.com/pion/logging v0.2.4
go: downloading github.com/pion/stun/v3 v3.0.1
go: downloading github.com/pion/turn/v4 v4.1.4
go: downloading github.com/pierrec/lz4/v4 v4.1.22
go: downloading github.com/pion/stun/v3 v3.0.1
go: downloading github.com/pion/turn/v4 v4.1.4
go: downloading github.com/aws/aws-sdk-go-v2/service/sesv2 v1.66.6
go: downloading github.com/klauspost/compress v1.18.0
go: downloading github.com/pierrec/lz4/v4 v4.1.22
go: downloading github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.40
go: downloading github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.38
go: downloading github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.19
go: downloading github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.40
go: downloading github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.16
go: downloading github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4
go: downloading github.com/aws/aws-sdk-go-v2/service/signin v1.0.4
go: downloading github.com/aws/aws-sdk-go-v2/service/sso v1.30.8
go: downloading github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.12
go: downloading github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.40
go: downloading github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.14
go: downloading github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.23
go: downloading github.com/aws/aws-sdk-go-v2/service/sso v1.30.8
go: downloading github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.31
go: downloading github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.40
go: downloading github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.40
go: downloading github.com/pion/dtls/v3 v3.0.7
go: downloading github.com/pion/transport/v3 v3.0.8
go: downloading github.com/pion/transport/v3 v3.0.8
go: downloading github.com/pion/dtls/v3 v3.0.7
go: downloading github.com/prometheus/client_golang v1.17.0
go: downloading golang.org/x/sync v0.21.0
go: downloading github.com/prometheus/client_golang v1.17.0
go: downloading github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.20.18
go: downloading github.com/lmittmann/tint v1.1.3
go: downloading github.com/mattn/go-isatty v0.0.20
go: downloading github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.20.18
go: downloading github.com/lmittmann/tint v1.1.3
go: downloading github.com/prometheus/client_model v0.5.0
go: downloading google.golang.org/protobuf v1.36.10
go: downloading github.com/prometheus/procfs v0.12.0
go: downloading github.com/prometheus/common v0.45.0
go: downloading github.com/cespare/xxhash/v2 v2.3.0
go: downloading github.com/beorn7/perks v1.0.1
go: downloading github.com/prometheus/client_model v0.5.0
go: downloading github.com/prometheus/procfs v0.12.0
go: downloading github.com/beorn7/perks v1.0.1
go: downloading google.golang.org/protobuf v1.36.10
go: downloading github.com/prometheus/common v0.45.0
go: downloading github.com/cespare/xxhash/v2 v2.3.0
go: downloading github.com/pion/randutil v0.1.0
go: downloading github.com/pion/transport/v4 v4.0.1
go: downloading github.com/pion/randutil v0.1.0
go: downloading github.com/pion/transport/v4 v4.0.1
go: downloading github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0
go: downloading github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0
go: downloading github.com/wlynxg/anet v0.0.5
go: downloading github.com/wlynxg/anet v0.0.5
go: downloading golang.org/x/text v0.38.0
go: downloading golang.org/x/text v0.38.0
go: downloading modernc.org/libc v1.74.1
go: downloading modernc.org/libc v1.74.1
go: downloading github.com/google/uuid v1.6.0
go: downloading github.com/dustin/go-humanize v1.0.1
go: downloading modernc.org/mathutil v1.7.1
go: downloading modernc.org/memory v1.11.0
go: downloading modernc.org/memory v1.11.0
go: downloading github.com/google/uuid v1.6.0
go: downloading github.com/dustin/go-humanize v1.0.1
go: downloading modernc.org/mathutil v1.7.1
go: downloading github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec
go: downloading github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec
=== launchctx (4 s)
ok amadan.net/carlos/platform.git/launchctx 0.012s
? amadan.net/carlos/platform.git/launchctx/controlname [no test files]
? amadan.net/carlos/platform.git/launchctx/wireid [no test files]
=== vet (47 s)
go: downloading github.com/aws/aws-sdk-go-v2/service/ec2 v1.324.0
go: downloading github.com/aws/aws-sdk-go-v2/service/ssm v1.74.0
go: downloading modernc.org/sqlite v1.55.0
go: downloading github.com/klauspost/compress v1.18.0
go: downloading github.com/aws/aws-sdk-go-v2/credentials v1.19.6
go: downloading github.com/aws/aws-sdk-go-v2/service/autoscaling v1.74.0
go: downloading github.com/aws/aws-sdk-go-v2/service/sesv2 v1.66.6
go: downloading github.com/pion/stun/v3 v3.0.1
go: downloading github.com/pion/turn/v4 v4.1.4
go: downloading golang.org/x/tools v0.47.0
go: downloading github.com/pierrec/lz4/v4 v4.1.22
go: downloading github.com/aws/aws-sdk-go-v2/service/sso v1.30.8
go: downloading github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.40
go: downloading github.com/pion/dtls/v3 v3.0.7
go: downloading github.com/pion/transport/v3 v3.0.8
go: downloading github.com/prometheus/client_golang v1.17.0
go: downloading github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.20.18
go: downloading github.com/lmittmann/tint v1.1.3
go: downloading github.com/prometheus/client_model v0.5.0
go: downloading github.com/prometheus/common v0.45.0
go: downloading google.golang.org/protobuf v1.36.10
go: downloading github.com/prometheus/procfs v0.12.0
go: downloading github.com/cespare/xxhash/v2 v2.3.0
go: downloading github.com/beorn7/perks v1.0.1
go: downloading github.com/pion/transport/v4 v4.0.1
go: downloading github.com/pion/randutil v0.1.0
go: downloading github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0
go: downloading github.com/wlynxg/anet v0.0.5
go: downloading golang.org/x/text v0.38.0
go: downloading golang.org/x/mod v0.37.0
go: downloading modernc.org/libc v1.74.1
go: downloading modernc.org/memory v1.11.0
go: downloading github.com/dustin/go-humanize v1.0.1
go: downloading github.com/google/uuid v1.6.0
go: downloading modernc.org/mathutil v1.7.1
go: downloading github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec
=== gofmt (1 s)
gofmt -l .: clean
=== rest (193 s)
? github.com/carlosframework/platform/cmd/changelog [no test files]
ok github.com/carlosframework/platform/cmd/console 3.491s
ok github.com/carlosframework/platform/cmd/moveprobe 11.657s
ok github.com/carlosframework/platform/cmd/zddprobe 0.740s
ok github.com/carlosframework/platform/internal/activator 62.522s
ok github.com/carlosframework/platform/internal/agentsock 0.037s
ok github.com/carlosframework/platform/internal/alerting 0.010s
ok github.com/carlosframework/platform/internal/analytics 0.049s
ok github.com/carlosframework/platform/internal/appconfig 2.708s
ok github.com/carlosframework/platform/internal/apperr 0.142s
ok github.com/carlosframework/platform/internal/applog 0.206s
ok github.com/carlosframework/platform/internal/awsinv 0.004s
ok github.com/carlosframework/platform/internal/boxchoice 0.004s
ok github.com/carlosframework/platform/internal/boxid 0.579s
? github.com/carlosframework/platform/internal/boxid/boxidtest [no test files]
? github.com/carlosframework/platform/internal/boxid/internal/signtest [no test files]
ok github.com/carlosframework/platform/internal/boxvolumes 0.016s
ok github.com/carlosframework/platform/internal/caps 0.020s
ok github.com/carlosframework/platform/internal/certwarm 0.007s
ok github.com/carlosframework/platform/internal/cgroup 0.762s
ok github.com/carlosframework/platform/internal/changelog 0.419s
ok github.com/carlosframework/platform/internal/claimactivation 1.046s
ok github.com/carlosframework/platform/internal/clamd 0.064s
? github.com/carlosframework/platform/internal/cliflag [no test files]
? github.com/carlosframework/platform/internal/controlname [no test files]
? github.com/carlosframework/platform/internal/controlsock [no test files]
ok github.com/carlosframework/platform/internal/dbdoor 6.634s
ok github.com/carlosframework/platform/internal/dbdoor/dbdoortest 0.021s
ok github.com/carlosframework/platform/internal/dbverify 0.011s
ok github.com/carlosframework/platform/internal/dnsrec 0.005s
ok github.com/carlosframework/platform/internal/docsite 0.022s
ok github.com/carlosframework/platform/internal/domains 2.632s
ok github.com/carlosframework/platform/internal/doorkey 0.013s
ok github.com/carlosframework/platform/internal/economics 0.009s
ok github.com/carlosframework/platform/internal/edge 49.076s
? github.com/carlosframework/platform/internal/edgeproof [no test files]
ok github.com/carlosframework/platform/internal/edgesfile 0.012s
ok github.com/carlosframework/platform/internal/fleetapi 0.182s
ok github.com/carlosframework/platform/internal/gate 1.670s
ok github.com/carlosframework/platform/internal/genselect 0.017s
ok github.com/carlosframework/platform/internal/geo 0.018s
ok github.com/carlosframework/platform/internal/geoip 0.039s
ok github.com/carlosframework/platform/internal/historyreadback 0.228s
ok github.com/carlosframework/platform/internal/host 142.705s
ok github.com/carlosframework/platform/internal/hostactors 5.955s
ok github.com/carlosframework/platform/internal/hostactors/awsmembership 0.008s
ok github.com/carlosframework/platform/internal/hostbox 3.205s
ok github.com/carlosframework/platform/internal/hostclaim 0.763s
ok github.com/carlosframework/platform/internal/hostenv 0.006s
ok github.com/carlosframework/platform/internal/hostevidence 0.007s
ok github.com/carlosframework/platform/internal/hostexec 5.712s
ok github.com/carlosframework/platform/internal/hostfence 0.042s
ok github.com/carlosframework/platform/internal/hostfiles 10.644s
ok github.com/carlosframework/platform/internal/hostfloor 0.032s
ok github.com/carlosframework/platform/internal/hostguard 1.078s
ok github.com/carlosframework/platform/internal/hostops 2.078s
? github.com/carlosframework/platform/internal/hostops/fencerelease [no test files]
ok github.com/carlosframework/platform/internal/hostops/internal/fencerelease 0.005s
ok github.com/carlosframework/platform/internal/hostquarantine 0.004s
ok github.com/carlosframework/platform/internal/hostrelocate 0.545s
ok github.com/carlosframework/platform/internal/hostresolve 0.206s
ok github.com/carlosframework/platform/internal/hostwrite 0.002s
ok github.com/carlosframework/platform/internal/infraview 0.005s
ok github.com/carlosframework/platform/internal/instancectl 5.268s
ok github.com/carlosframework/platform/internal/instanceexec 2.885s
ok github.com/carlosframework/platform/internal/instancefs 0.020s
ok github.com/carlosframework/platform/internal/instancekey 0.020s
ok github.com/carlosframework/platform/internal/instanceupdates 0.021s
ok github.com/carlosframework/platform/internal/layout 1.872s
ok github.com/carlosframework/platform/internal/lbhealth 0.005s
ok github.com/carlosframework/platform/internal/lease 10.126s
ok github.com/carlosframework/platform/internal/lease/leasetest 0.561s
ok github.com/carlosframework/platform/internal/ledger 0.022s
ok github.com/carlosframework/platform/internal/mail 0.262s
ok github.com/carlosframework/platform/internal/maintenance 0.093s
ok github.com/carlosframework/platform/internal/maintenanceexec 5.569s
ok github.com/carlosframework/platform/internal/maintenanceguard 0.127s
ok github.com/carlosframework/platform/internal/metrics 0.019s
ok github.com/carlosframework/platform/internal/moves 30.275s
ok github.com/carlosframework/platform/internal/moves/archive 0.963s
ok github.com/carlosframework/platform/internal/objstore 0.291s
ok github.com/carlosframework/platform/internal/ops 0.192s
ok github.com/carlosframework/platform/internal/park 0.277s
ok github.com/carlosframework/platform/internal/plans 0.015s
ok github.com/carlosframework/platform/internal/privatefile 0.005s
ok github.com/carlosframework/platform/internal/provision 51.319s
ok github.com/carlosframework/platform/internal/publicrelease 0.011s
? github.com/carlosframework/platform/internal/regionname [no test files]
ok github.com/carlosframework/platform/internal/registry 5.730s
ok github.com/carlosframework/platform/internal/relay 7.197s
ok github.com/carlosframework/platform/internal/release 32.062s
ok github.com/carlosframework/platform/internal/rollout 4.199s
ok github.com/carlosframework/platform/internal/rolloutwords 0.004s
ok github.com/carlosframework/platform/internal/rootaws 0.007s
ok github.com/carlosframework/platform/internal/rootparticipation 0.066s
ok github.com/carlosframework/platform/internal/rootstate 0.027s
ok github.com/carlosframework/platform/internal/runs 0.604s
ok github.com/carlosframework/platform/internal/s3compat 0.005s
ok github.com/carlosframework/platform/internal/safefile 0.004s
ok github.com/carlosframework/platform/internal/schedule 2.880s
ok github.com/carlosframework/platform/internal/seeds 0.011s
ok github.com/carlosframework/platform/internal/sim/hostlife 12.459s
ok github.com/carlosframework/platform/internal/sim/litestreamfacts 4.067s
ok github.com/carlosframework/platform/internal/status 0.080s
ok github.com/carlosframework/platform/internal/steering 0.012s
ok github.com/carlosframework/platform/internal/stopevidence 0.299s
ok github.com/carlosframework/platform/internal/svcrole 0.002s
ok github.com/carlosframework/platform/internal/telemetry 0.004s
ok github.com/carlosframework/platform/internal/termfence 0.004s
ok github.com/carlosframework/platform/internal/tunnelclient 0.004s
ok github.com/carlosframework/platform/internal/unixdial 0.004s
ok github.com/carlosframework/platform/internal/updatewords 0.005s
ok github.com/carlosframework/platform/internal/usage 0.007s
ok github.com/carlosframework/platform/internal/version 0.002s
ok github.com/carlosframework/platform/internal/vet 1.680s
=== internal_console@0 (85 s)
ok github.com/carlosframework/platform/internal/console 83.862s
=== internal_console@1 (42 s)
ok github.com/carlosframework/platform/internal/console 40.860s
=== internal_console@2 (47 s)
ok github.com/carlosframework/platform/internal/console 45.222s
=== internal_console@3 (41 s)
ok github.com/carlosframework/platform/internal/console 40.048s
=== internal_console@4 (44 s)
ok github.com/carlosframework/platform/internal/console 42.329s
=== internal_console@5 (41 s)
ok github.com/carlosframework/platform/internal/console 39.434s
=== internal_console@6 (43 s)
ok github.com/carlosframework/platform/internal/console 42.042s
=== internal_console@7 (50 s)
ok github.com/carlosframework/platform/internal/console 48.677s
=== root@0 (38 s)
ok github.com/carlosframework/platform 36.842s
=== root@1 (47 s)
ok github.com/carlosframework/platform 45.524s
=== root@2 (46 s)
ok github.com/carlosframework/platform 44.274s
=== cross ===
--- shard 0 of 5 ---
cross-compiling darwin/arm64
--- shard 1 of 5 ---
cross-compiling darwin/amd64
--- shard 2 of 5 ---
cross-compiling linux/arm64
linux/arm64 carlos binary: 50069666 bytes (ceiling 52000000)
--- shard 3 of 5 ---
cross-compiling windows/amd64
--- shard 4 of 5 ---
cross-compiling windows/arm64
=== minio-contract ===
minio contract: arm64, minio RELEASE.2025-09-07T16-13-09Z, mc RELEASE.2025-08-13T08-35-41Z
minio: downloading https://github.com/minio/minio/releases/download/RELEASE.2025-09-07T16-13-09Z/minio.linux-arm64.RELEASE.2025-09-07T16-13-09Z
minio: downloaded, digest verified
mc: downloading https://github.com/minio/mc/releases/download/RELEASE.2025-08-13T08-35-41Z/mc.linux-arm64.RELEASE.2025-08-13T08-35-41Z
mc: downloaded, digest verified
minio healthy on 127.0.0.1:20999
Added `local` successfully.
Bucket created successfully `local/contract-bucket`.
Bucket created successfully `local/denied-bucket`.
Created policy `contract-only` successfully.
Added user `scopeduser` successfully.
Attached Policies: [contract-only]
To User: scopeduser
buckets and scoped user ready
=== RUN TestDeliveredContractMinio
=== RUN TestDeliveredContractMinio/put
=== RUN TestDeliveredContractMinio/get_round-trips_the_body
=== RUN TestDeliveredContractMinio/list_shows_the_key
=== RUN TestDeliveredContractMinio/head_and_list_agree_on_the_version
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_its_own_object
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict
=== RUN TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed
=== RUN TestDeliveredContractMinio/delete_removes_it
=== RUN TestDeliveredContractMinio/cross-bucket_denial
--- PASS: TestDeliveredContractMinio (0.02s)
--- PASS: TestDeliveredContractMinio/put (0.00s)
--- PASS: TestDeliveredContractMinio/get_round-trips_the_body (0.00s)
--- PASS: TestDeliveredContractMinio/list_shows_the_key (0.00s)
--- PASS: TestDeliveredContractMinio/head_and_list_agree_on_the_version (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route (0.01s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict (0.00s)
--- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed (0.00s)
--- PASS: TestDeliveredContractMinio/delete_removes_it (0.00s)
--- PASS: TestDeliveredContractMinio/cross-bucket_denial (0.00s)
PASS
ok github.com/carlosframework/platform/internal/objstore 0.025s
=== RUN TestT4StorageProbeMinio
2026/10/21 22:01:31 moves: T4: STORE: list: operation error S3: ListObjectsV2, https response error StatusCode: 403, RequestID: 18E0AA79A8328B78, HostID: 7c420053e0fbcd7f41653fdd92ffa2802c903d05303b1aa15adc2e8bb65d183f, api error SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.
--- PASS: TestT4StorageProbeMinio (0.01s)
PASS
ok github.com/carlosframework/platform/internal/host 0.022s
minio contract: PASS
diff --git a/docs/superpowers/specs/2026-10-21-self-serve-offline-moves-design.md b/docs/superpowers/specs/2026-10-21-self-serve-offline-moves-design.md| new file mode 100644 |
| index 000000000..537f6af80 |
| --- /dev/null |
| +++ b/docs/superpowers/specs/2026-10-21-self-serve-offline-moves-design.md |
| @@ -0,0 +1,205 @@ |
| +# Self-serve offline moves: a tenant's customer moves their own instance |
| + |
| +Rev 1, 2026-10-21. Status: design for review (Paul). Branch `design-self-serve-offline-moves`. |
| + |
| +Builds on the offline operator move (`docs/superpowers/plans/2026-10-20-offline-operator-move.md`, shipped in v0.28.5 to v0.28.7) and its runbook (`docs/superpowers/plans/2026-10-21-offline-move-runbook.md`). Supersedes nothing: Phase 1 (`docs/superpowers/specs/2026-10-02-move-any-app-design.md`) stays the live-move path for claims-on deployments, and the M1 to M18 rulings stand. M18 ("Tito waits; move mutations are operator-only") is answered here for one door, not lifted: §5 says what the customer's request may do and what it may not. |
| + |
| +## 1. Intent |
| + |
| +A Tito customer opens Account settings, Data residency, in Tito's dashboard, sees the regions our console lists as armed, picks one and clicks Move. They are told the account will be offline for a few minutes and how many, and they confirm; the customer has the final say, nobody at Tito or at ours is in the loop. The platform copies the account's data to the new region while it keeps serving, then takes it offline, finishes the copy, opens it in the new region and reopens it. From the moment it goes offline until it is back, the customer and their own visitors see a holding page: "Paul's account is temporarily offline. Please check back in approximately 3 minutes." If anything fails after the account has gone offline, the platform puts it back in the old region on its own, and the residency tab says so. Scheduling a move for later is nice to have; Tito's dashboard already offers a time, and the platform's start is "now" in this version. |
| + |
| +**Why now.** Tito's customer move has been off since 2026-10-02 (`TITO_REMOTE_PLACEMENT`): the pre-Phase-1 move lost the per-host key and config (#175 P1). The offline move is the one move proven on real customer data (three Tito instances, 2026-10-21, 2.5 to 3 minutes offline each) and it carries everything (§"The dependency list" of the offline plan). It needs no host claims, so it does not wait for the claim cohort, acceptance 10.1 to 10.6 or the F1 owner surface. Phase 1's live move is weeks away for Tito and refuses Tito's router outright (M18). |
| + |
| +**Paul's rulings in the conversation of 2026-10-21:** |
| +- The customer confirms the downtime themselves; nobody at Tito approves. |
| +- No size cap: the estimate is shown and the customer decides. |
| +- A failure after the account has gone offline puts it back in the old region automatically. |
| +- Scheduling is nice to have, not needed first. |
| +- Offline moves stay available to Tito as an operator-run option regardless. |
| + |
| +## 2. Goals, non-goals, anti-goals |
| + |
| +**Goals.** |
| +1. One request from Tito's router starts a move that runs to completion with no operator step: stage while serving, stop, seal, flip, verify, open, finish, clean up. |
| +2. The holding page while offline, with the account's name and a minutes estimate, on every edge that answers for the host. |
| +3. Automatic return to the old region on any failure after the stop, including a failure after open within a bounded health window. |
| +4. The estimate before the customer confirms, and a better one once the data is staged. |
| +5. Tito's existing router code (prepare, status, cut, abort; states scheduled, preparing, ready, frozen, cutting, done, failed, aborted) keeps working with at most a token change on Tito's side. |
| +6. Only an owner-delegated credential can start a move (§5). A member-minted instance token cannot. |
| +7. Several customers can move at once, bounded, without one move blocking every other (today: one offline-move lock per deployment). |
| +8. Every trap the first production moves hit (#339 and its siblings) is a refusal before the account goes offline, or is carried. |
| + |
| +**Non-goals.** |
| +- Live (seconds-of-cut) moves: Phase 1, claims-on only. |
| +- Object-store residency: the store bucket stays where it is (M3 is Phase 1's P8). |
| +- Moving a Tito controld account: it has no platform record and the door 404s; Tito's residency tab must refuse it first (Tito side, §9). |
| +- Moving into or out of a region with host claims or Phase 1 on: refused as today (MV-O20). |
| +- Scheduling on the platform side: Tito books the time; the platform starts when called. |
| +- Replica instances and seeded records: refused as the legacy door refuses them. |
| + |
| +**Anti-goals** (a failure even if it technically works). |
| +- A move that completes with data the app cannot read, or without its config or key (the kpmg outage class). The manifest and the landing checks are the guard, unchanged. |
| +- A customer left offline with nothing happening. Every wait in the job has a deadline, and every deadline ends in an abort before open or a return after it. |
| +- A credential that can move an instance without an account owner having said so. |
| +- Text on the holding page that came from the request. |
| + |
| +## 3. Constraints |
| + |
| +- No change to the offline move's data path or its stop guarantee (the three layers of the offline plan). The job drives the same transitions in the same order with the same evidence. New code is the job, the door's authorization, the pre-checks, the holding page, the estimate, the return and the cleanup. |
| +- Claims-off only: the console refuses the door when `CARLOS_HOST_CLAIM` or `CARLOS_MOVES_PHASE1` is on (MV-O20), as every offline door does. Boxes publish `offline-move-1` and `offline-stage-1` only in that state. |
| +- Off by default. A console switch `CARLOS_SELF_SERVE_MOVES=offline` turns the door on; absent, the legacy move door behaves exactly as today. The moves copy gate keeps the default off while any new copy row is pending Paul's review. |
| +- Root never runs the job. Everything the job does is console-side reads and record transitions plus the boxes' existing moves pass. The one new root-side action (§7.6, the source's leftovers) runs in root's existing hourly pass under its own census row. |
| +- Plan 14 census: every new or changed unit that reads root state takes a row in a PROPOSED overlay loaded last, ruled on #280. |
| +- Copy: every user-facing sentence (the holding page, the door's refusals, the status view's reasons, the CLI) is a copy row, reviewed before the default can turn on. |
| +- No tenant names in platform code, tests or copy (`shop.example.com`, "the account"). |
| + |
| +## 4. The customer's path, end to end |
| + |
| +1. Tito's residency tab lists the armed regions from our console's `/regions`, as today. The customer picks one and clicks Move. Tito shows the confirm page with the estimate it fetched from `GET .../instances/{host}/move?estimate=1` (§6.3): "about N minutes offline". The customer may pick a time (Tito's existing field) and confirms. |
| +2. At the time, Tito's router calls `POST .../instances/{host}/move {"region":"uk","display_name":"Paul's account","requested_by":"<tito user id>"}` with its move-capable token (§5). The console answers 202 with the status view, state `preparing`. |
| +3. **Preparing.** The job stages while the instance serves: the existing `stage` round, repeated until a round copies under the catch-up threshold (§6.2), then the pre-checks (§7.3). When both pass the state is `ready` and the view carries the refined estimate. Tito polls `GET .../move` as today. |
| +4. Tito freezes its own account writes (its existing `frozen` step) and calls `POST .../move/cut`. The console answers 202, state `cutting`. |
| +5. **Cutting.** The job runs `start` (every source box holds, fences, reports), `seal` (tombstone, final round), `flip` (the landing's ensures and checks), `open`, then the health check (§7.4), then `finish`. From `start` to the first 2xx on the target, every edge answers for the host with the holding page (§6.4). Expect 2.5 to 3 minutes for a 1.6 to 3.7 GB replica. |
| +6. **Done.** The state goes to `none`, `last_move` says `done` with the new region, and Tito's sweep marks its row done and unfreezes. The residency tab shows the new region. |
| +7. **Failure.** Before open, any refusal or deadline aborts: the record returns to the source, the tombstone is lifted, the source reopens, `last_move` says `aborted` with the reason, and Tito's status shows `failed` with that reason. After open, a failed health check within the window runs the return (§7.5): a second offline move back to the source region, run by the same job, with the same holding page. If the return itself cannot be started (§7.5 says when), the move is `done` in the new region with a `last_move.warning` naming the health failure, and operators are paged. |
| +8. **Cancel.** `DELETE .../move` aborts in `preparing` and `ready` (nothing was offline). In `cutting` it is refused: the customer confirmed, and a cut that has started either completes or returns on its own. |
| + |
| +## 5. Who may ask: the move-capable token |
| + |
| +M18 closed the move doors to instance-role tokens because a member-minted token could move an instance the member had no right to move. This design opens one door, the offline self-serve move, to a token that an account owner has explicitly made move-capable: |
| + |
| +- `carlos instance-token --moves` mints an instance token with `moves=1` in its `cli_tokens` row. Only a member whose role on the account is owner may mint it; the mint is audited ("minted a move-capable instance token for <app>"). Existing tokens have `moves=0` and cannot be upgraded; the owner mints a new one and Tito's router switches to it. |
| +- The self-serve door checks the token's row. A token without `moves=1` gets the move family's byte-identical 404, as a non-operator gets from the operator doors today. The status read stays open to every instance token (M18 kept it). |
| +- The request carries `requested_by`, an opaque identifier Tito supplies for the person who confirmed, and the console records it in the move's `requested_by` as `<token name>:<requested_by>` and in the move log. The platform does not verify Tito's user; Tito's confirm page is Tito's business. What the platform verifies is that an owner delegated moving to this credential. |
| +- Operators keep the operator doors (`carlos move offline ...`) unchanged. An operator-run move and a self-serve move of the same instance exclude each other through the record's one move. |
| + |
| +This is the F1 shape ("move-capable service authority explicitly owner-delegated") for one capability, without the rest of F1. My call; Paul to confirm it answers M18 (§10, Q1). |
| + |
| +## 6. The door, the states, the estimate and the holding page |
| + |
| +### 6.1 The door |
| + |
| +The legacy move doors keep their paths and roles. With `CARLOS_SELF_SERVE_MOVES=offline` on a claims-off, Phase-1-off console, their bodies run the self-serve offline move instead of the legacy online move: |
| + |
| +| call | today (legacy) | self-serve offline | |
| +|---|---|---| |
| +| `POST .../move {"region"}` | legacy prepare | offline stage + pre-checks, as a job; needs `moves=1` | |
| +| `GET .../move` | state none/preparing/ready/cutting | the same words, plus `kind: "offline"`, `estimate_seconds`, `offline_at`, `reason` | |
| +| `GET .../move?estimate=1` | n/a | the estimate for a move not yet requested (§6.3) | |
| +| `POST .../move/cut` | legacy cut | start, seal, flip, open, health, finish, as a job; needs `moves=1` | |
| +| `DELETE .../move` | legacy abort | abort in preparing or ready; 409 in cutting; needs `moves=1` | |
| + |
| +Refused before anything is written: a replica or a home with replicas, a seeded record, a run in flight, a move in flight (409, the existing sentences), the same region, a region without `offline-move-1` and `offline-stage-1` on every box, a target region whose boxes' `secrets_key_fp` differs from the console's (R9), a controld-shaped host with no record (404), and a token without `moves=1` (404). |
| + |
| +The states Tito reads are the legacy words: `none`, `preparing`, `ready`, `cutting`, with `last_move.outcome` `done`, `aborted` or `failed`. The record carries the offline states underneath (`offline-staging` through `offline-finishing`, plus `offline-reverting` and the return's own move), and the view maps them. `cutting` covers start through finish; `reason` says which step is running or what refused. |
| + |
| +### 6.2 Catch-up and readiness |
| + |
| +`preparing` repeats the stage round until a round copies less than 16 MiB or fewer than 8 objects (the legacy move's ready rule, which the seal's final round then finishes offline in seconds), with at least two rounds and at most 30 minutes; a busy instance that never settles goes `ready` at the deadline with the estimate reflecting the last round's delta. Then the pre-checks (§7.3). `ready` is held, re-staging every 10 minutes so the delta stays small until Tito cuts; a `ready` not cut within 24 hours aborts (nothing was offline) with the reason "the move was not confirmed in time". |
| + |
| +### 6.3 The estimate |
| + |
| +`estimate_seconds = base + replica_bytes / rate_replica + files_bytes / rate_files + delta_bytes / rate_delta`, rounded up to whole minutes, minimum 2. `base` is the fixed part the flagship measured: stop and tombstone about 20 s, flip about 50 s, open about 40 s, first wake about 20 s. The rates start at the flagship's measured values (the 2026-10-21 moves: 1.6 GB staged in 76 s, 3.7 GB in 142 s; seal of 1.6 GB without stage 6 min) and the console refines them from each finished move's recorded timings (a rolling median per region pair, kept in the console's database). Before a move is requested (`?estimate=1`) the replica size comes from the data bucket listing and the files size from the box's status report (`files_bytes`, a new field the box publishes per host), with `delta_bytes` assumed to be the replica's last-day growth; once staged, from the stage receipts. The view and the holding page use the same number; the holding page counts down from `offline_at`. |
| + |
| +### 6.4 The holding page |
| + |
| +From `start` to the first 2xx on the target, the host is held on every source box (not served) and has no ready row on the target. Today every edge answers a platform-domain host with the "not here yet" 503 retry page (`internal/edge/pending.go`) and a customer domain with 404. This design adds an offline answer: |
| + |
| +- The console publishes, in the record's move, `display_name` (as given, at most 80 characters, HTML-escaped at render) and `offline_at` and `estimate_seconds`. Every box's edges file lists a held host as `offline_move: {display_name, offline_at, estimate_seconds}` (the edges file already carries the box's routes; a held row is listed in this new field instead of dropped), and every edge's home map reads it. |
| +- An edge that finds a host in an `offline_move` entry, on any domain, answers 503 with `Retry-After: 30` and the holding page: "`<display_name>` is temporarily offline. Please check back in approximately N minutes." with N from the estimate and `offline_at`, a self-retrying page in the pending page's style, `noindex`, nothing from the request echoed. Without a `display_name` the sentence is "This account is temporarily offline. ...". The page's text is copy rows. |
| +- After open the target's row is ready, its edges file lists the host, and the holding entry goes; edges pick it up within their edges-file refresh (seconds to half a minute), the pending page's known window. The 503 at the source after open (runbook §12 point 6) is this same window and is accepted. |
| + |
| +## 7. The job |
| + |
| +A console worker pass (`StartSelfServeMovesPass`, the `RunClaimWorker` shape used by the other console passes), every 5 seconds, over every record whose move is an offline move with `self_serve: true`. Each tick advances each move by at most one transition, judged from the evidence the operator doors read today (the record, the edges files, the boxes' status reports, the reports under `control/offline-moves/`), through the same door bodies refactored into functions the operator doors and the job both call. The job writes nothing but the record and the move log. It is idempotent: a console restart resumes from the record's state. |
| + |
| +### 7.1 Locks and concurrency |
| + |
| +- One move per instance: the record's `move` field, as today. |
| +- The deployment-wide offline-move lock becomes a per-source-box and per-target-box limit: at most `CARLOS_SELF_SERVE_MOVES_PER_BOX` (default 2) moves whose source or target is a given box may be past `preparing` at once, and at most one may be between `start` and `open` on a given box (the stop and seal are per instance, but the box's disk and network are shared). A move that cannot take its slot waits in `ready` with the reason "waiting for another move on this box", not refused. The builder checks what today's deployment lock protects in `movesoffline_stage.go` and the stage door, and keeps whatever invariant it guards (left to the builder, §11). |
| +- An operator-run offline move counts against the same limits. |
| + |
| +### 7.2 Deadlines |
| + |
| +Every wait has one, and every deadline before open ends in abort: stop reports from every source box (5 min), seal (10 min), landing (10 min), open (3 min), the health check (§7.4, 3 min). `preparing` and `ready` have theirs in §6.2. An abort that itself stalls (a source box not confirming the return) is reported as `reverting` with the reason and pages operators after 10 minutes; nothing is lost, the source's copy was never removed. |
| + |
| +### 7.3 Pre-checks before anything goes offline |
| + |
| +Run at the end of `preparing` and again at `cut` before `start`. Each is a refusal (abort with the reason, nothing offline) unless marked carried: |
| + |
| +- Target region: every box publishes `offline-move-1` and `offline-stage-1`; no claim or Phase 1 switch; `secrets_key_fp` equals the console's (R9); the target region's staging grant on the source data bucket exists (the stage round having succeeded proves it). |
| +- Target box (the one that will land it, chosen as today): a **target-ready report** written by the box under `control/offline-moves/<host>/<id>/target/<box>.json` on its moves pass once the record is staging, saying: no route or timers for the host; `/opt/carlos/live/<host>` absent, or present and naming the build the record's channel resolves to (#339's stale link); none of `/data/instances/<host>`, `/data/.instances/<host>`, `/data/<host>`, `/data/programs/<host>.json`, `/data/<host>.env`, `/data/<host>.config.env`, `/data/<host>.db` present; `/data/.offline-moves/<host>` absent or holding only an earlier move's `target/`; free disk at least twice the replica's size; 0 current objects under `instances/<host>/` and `moves/<host>/` in the target data bucket and no `lock.json`. The box reports facts; the console decides. |
| +- Source: the instance's extra app directories beside its files directory (#339 point 4): carried when empty (the archive gains empty directory entries), refused when not, with the reason naming the directory. The source box's existing report gains the list. |
| +- Build: the channel's build is the one the source runs (`release.LiveVersion` equals the row's); a promote during the move fails the landing's `release` item as today, and the job aborts. |
| + |
| +### 7.4 The health check after open |
| + |
| +After `open`, the job waits for the first 2xx from the target for the host's probe path (the record's `probe`, else `/`), through the target box, within 3 minutes, and then one more 2xx 30 s later. Both pass: `finish`. Either fails: the return (§7.5). The console measures the offline time from `start` to the first 2xx and records it with the staged sizes for the estimate's rates (§6.3). |
| + |
| +### 7.5 The return |
| + |
| +The return is an offline move in the other direction, started by the job without a customer step, with the same holding page (the estimate for the return comes from the same rates). It needs: |
| + |
| +- the source region's staging grant on the target data bucket (the reverse of the lane the forward move used). The deployment's regions are configured in pairs with grants both ways (infra; §9), and `?estimate=1` and the door refuse a target whose reverse grant is absent, so a move never starts that cannot return; |
| +- the original box to pass the target-ready report (§7.3) for the return. The forward move's `finish` therefore removes the source's leftovers that block it (§7.6), and the return's own `finish` does the same on the box it left. |
| + |
| +The return's `last_move` is `returned` with the health failure as the reason; Tito's status shows `failed` with that reason and the account is where it started. If the return cannot start (no reverse grant, which the door should have refused; the original box not reporting ready within 5 minutes; a second move already holding its slot), the move stays `done` in the new region with `last_move.warning` and operators are paged (the existing alert sink). Fixing forward is then an operator's job, as the runbook says today. |
| + |
| +### 7.6 Cleanup: the source's leftovers |
| + |
| +Today a finished move leaves on the source box a root-owned `/opt/carlos/live/<host>` link and `/data/programs/<host>.json`, the parked database and directories under `/data/.offline-moves/<host>/`, and the source region's replica in its data bucket until an operator deletes it (runbook §10, §12). For self-serve: |
| + |
| +- Root's existing hourly pass (`carlos adopt`, the root pass) removes `/opt/carlos/live/<host>` and `/data/programs/<host>.json` for a host whose record names another region and whose `last_move` is `done` or `returned` older than 10 minutes and whose row is absent from the box's registry. This is the one new root action; it takes a census row at its sibling's class (the version sweep's) and never deletes a path that any live link or running process still names. |
| +- The agent removes the parked `/data/.offline-moves/<host>/<id>/` 7 days after `finish`. |
| +- The console's job deletes the source region's replica objects by exact key, keeping `lock.json` (the lifted tombstone), 24 hours after a `done` whose target has served since (the box's status report shows the host awake at least once). The deletion is logged key by key in the move log, as the runbook's cleanup was. |
| + |
| +### 7.7 Scheduling |
| + |
| +Tito books the time and calls prepare when it arrives (its sweep's `scheduled` step does this today). The platform's door takes an optional `start_at` and refuses one that is not "now" in this version ("scheduled starts are not available yet"), so the field exists for the version that honours it. |
| + |
| +## 8. Observability |
| + |
| +- The move log (the existing per-move log under `moves/<acct>/<app>/<host>/<id>/`) gains the job's lines: each transition, each refusal and its reason, the pre-check facts, the health probes, the timings, the cleanup keys. |
| +- The console's status view carries `reason` and `offline_at`; `carlos move offline status` shows them. |
| +- Paging through the existing alert sink: a return that could not start, an abort that stalls past 10 minutes, a job that finds a move it cannot advance for 30 minutes. |
| +- Member stream: each transition is an event on the app's platform log, as the legacy move's were. |
| + |
| +## 9. Rollout |
| + |
| +Platform: |
| +1. Build behind `CARLOS_SELF_SERVE_MOVES`, default off; copy rows pending until Paul's review; census overlay PROPOSED. |
| +2. Release and roll dev, then flagship, then Tito (roll order rule; boxes before consoles). |
| +3. Rehearse on dev with a Tito-shaped app (key file, store binding, config, custom domain) through Tito's own residency flow against the dev console: forward, cancel in ready, abort by a failed pre-check, a forced health failure and its return. |
| +4. Turn the switch on for Tito's console with Paul's go. |
| + |
| +Tito's side (titogo, from titogo-f6's notes of 2026-10-21): |
| +1. An account owner mints a move-capable token; the router switches `CARLOS_...` to it (one env var, env sync, a router restart). |
| +2. The residency tab shows the estimate from `?estimate=1` on the confirm page and refuses controld accounts (no platform record) before posting. |
| +3. Re-check the 2026-10-02 unsafe list on the region boxes: the edge door or `routes.home` for remote accounts, the secrets key on every region box (distributed 2026-10-03), the payments relay socket per region. |
| +4. `TITO_REMOTE_PLACEMENT=on`, env sync, restart the router. Publish the "Move your account" article. |
| + |
| +Infra: the staging grants both ways for every region pair the console arms (today IE to UK exists; UK to IE does not), as a region-host module output. |
| + |
| +## 10. Questions for Paul (none block the plan; defaults stated) |
| + |
| +- **Q1.** Does the move-capable token (§5) answer M18 for this door? Default: yes, as stated. |
| +- **Q2.** Cancel during `cutting` is refused (§4.8). Default: refused; the customer confirmed. |
| +- **Q3.** The 24-hour `ready` abort and the 30-minute staging bound (§6.2). Defaults as stated. |
| +- **Q4.** The source replica deletion 24 hours after done (§7.6) is automatic. Default: automatic, logged key by key. |
| + |
| +## 11. Left to the builder |
| + |
| +- What the deployment-wide offline-move lock protects today (`movesoffline_stage.go`, the stage door) and the smallest change that gives per-box limits without weakening it. |
| +- The exact split of the operator doors into functions the job calls (the doors keep their behaviour byte for byte). |
| +- The edges-file field for held hosts and its reader in the home map, with the pending page's retry mechanics reused for the holding page. |
| +- The target-ready report's writer in the box's moves pass and its census row. |
| +- The estimate's storage and the rolling median. |
| +- The token column (`moves`) migration and the mint flag. |
| +- Tests: the two-box end-to-end witness of the offline plan (Task 8) extended to the job: forward, cancel in ready, each pre-check refusal, each deadline, a health failure and its return, the holding page's answer on a third edge, two moves at once on one box. |
| + |
| +## 12. Risks |
| + |
| +- **The return has never been run.** The runbook says a move back "has never been run"; the return is the same machinery in the other direction and gets the same witness, but its first real run is on dev. |
| +- **The estimate is an estimate.** Rates differ by region pair and object count; the rolling median narrows it; the holding page says "approximately". |
| +- **Boxes that do not report.** A source box down at `start` stalls `seal`; the deadline aborts and the customer sees "failed, still in <region>" with the box named. Nothing is lost. |
| +- **Tito's freeze.** Tito freezes its own writes before cut; a Tito bug there is Tito's, and the platform's seal is final regardless (the tombstone, not Tito's freeze, is the stop). |