carlos / platform Public

console: the move door says the instance's last relocation is still finishing when the completion barrier refuses, and keeps its region sentence for every other pending chore (Paul's copy review)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Paul Campbell pushed by claude 1 week ago d383d06dbe6c40886332e2d620af0356585fd074 parent 48ad354

5 files changed, +54 −2

internal/console/apiinstancemove.go +2 −0
internal/console/apiinstancemove_relocation_test.go +44 −0
internal/hostclaim/ops.go +6 −0
internal/hostclaim/relocation_probestop.go +1 −1
internal/hostclaim/relocation_probestop_test.go +1 −1

CI failed — run details

CI log

=== gate ===
cd launchctx && go vet ./... && go test ./... -count=1
ok  	amadan.net/carlos/platform.git/launchctx	0.012s
?   	amadan.net/carlos/platform.git/launchctx/controlname	[no test files]
?   	amadan.net/carlos/platform.git/launchctx/wireid	[no test files]
go vet ./...
gofmt -l .: clean
go test ./... -count=1 -timeout 30m
using http://127.0.0.1:35191 as a@keymail.dev
shipped titesite abc1234
using http://127.0.0.1:39541 as a@keymail.dev
using http://127.0.0.1:45281 as a@keymail.dev
shipped titogo abc1234
using http://127.0.0.1:41293 as vicky@keymail.dev
titogo (staging):
KEY   VALUE
PORT  8080
--- FAIL: TestMovesStringsAreRecorded (2.49s)
    approved_moves_copy_test.go:1611: no row of the record carries internal/console/apiinstancemove.go:731 (jsonError) "the instance's last relocation is still finishing. try again in a few minutes"
2026/10/12 11:09:38 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:09:45 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:09:52 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:09:58 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:10:02 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:10:04 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:10:06 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:10:08 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:10:10 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:10:13 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:10:15 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:10:17 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:10:17 console: api instances get bab/titogo/probe-titogo.bab.oncarlos.test: read timeout
2026/10/12 11:10:17 console: api instances get bab/titogo/probe-titogo.bab.oncarlos.test: read timeout
2026/10/12 11:10:19 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:10:21 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:10:25 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:10:29 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:10:33 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:10:34 edges: box "uk-box" has an edges file under control/edges/ but no status snapshot at all, so no region outside the home is automatically eligible until it reports; said once
2026/10/12 11:10:34 edges: 1 edges file(s) under control/edges/ but no fresh box snapshot matched any of them, so automatic region placement sees the home alone (are edges and status in different buckets? see CARLOS_STATUS_BUCKET/CARLOS_STATUS_PREFIX)
2026/10/12 11:10:34 console: an incomplete read of the edges files or box snapshots (edges complete true, snapshots complete false), so no region outside the home is automatically eligible this request; said once
using http://127.0.0.1:40121 as paul@keymail.dev
account: ops (bab)
set 1 env var(s) for titogo (default)
using http://127.0.0.1:40121 as paul@keymail.dev
account: ops (bab)
set 1 secret(s) for titogo (default)
account: ops (bab)
account: ops (bab)
using http://127.0.0.1:40121 as paul@keymail.dev
account: ops (bab)
config sync requested for titogo the files land within seconds; programs carlos owns restart themselves, and anything else keeps its old values until you restart it
2026/10/12 11:10:59 INFO replica sync system=store db=source.db replica=file txid.replica=0000000000000000 txid.db=0000000000000001
2026/10/12 11:10:59 INFO ltx file uploaded system=store db=source.db replica=file level=0 minTXID=0000000000000001 maxTXID=0000000000000001 size=720
2026/10/12 11:10:59 INFO replica sync system=store db=source.db replica=file txid.replica=0000000000000001 txid.db=0000000000000001
2026/10/12 11:10:59 INFO replica sync system=store db=source.db replica=file txid.replica=0000000000000001 txid.db=0000000000000002
2026/10/12 11:10:59 INFO ltx file uploaded system=store db=source.db replica=file level=0 minTXID=0000000000000002 maxTXID=0000000000000002 size=370
2026/10/12 11:10:59 INFO replica sync system=store db=source.db replica=file txid.replica=0000000000000002 txid.db=0000000000000002
2026/10/12 11:11:01 INFO replica sync system=store db=source.db replica=file txid.replica=0000000000000002 txid.db=0000000000000003
2026/10/12 11:11:01 INFO ltx file uploaded system=store db=source.db replica=file level=0 minTXID=0000000000000003 maxTXID=0000000000000003 size=888
2026/10/12 11:11:01 INFO replica sync system=store db=source.db replica=file txid.replica=0000000000000003 txid.db=0000000000000003
2026/10/12 11:11:01 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
using http://127.0.0.1:43023 as paul@keymail.dev
using http://127.0.0.1:43023 as paul@keymail.dev
using http://127.0.0.1:43023 as paul@keymail.dev
Querying a copy of acme.go.tito.io's database, up to transaction 3, written 2026-10-12 11:11:01 UTC. Nothing you run here reaches acme.go.tito.io.
2026/10/12 11:11:01 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:11:01 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:11:03 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:11:05 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:11:07 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:11:09 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:11:11 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:11:13 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:11:15 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
using http://127.0.0.1:38843 as paul@keymail.dev
account: ops (bab)
attached tickets.example.com -> default.go.tito.io
DNS: pointing
using http://127.0.0.1:38843 as paul@keymail.dev
account: ops (bab)
attached tickets.example.com -> default.go.tito.io
DNS: pointing
using http://127.0.0.1:38843 as paul@keymail.dev
account: ops (bab)
attached tickets.example.com -> default.go.tito.io
DNS: pointing
using http://127.0.0.1:38843 as paul@keymail.dev
account: ops (bab)
attached tickets.example.com -> default.go.tito.io
DNS: pointing
using http://127.0.0.1:33665 as paul@keymail.dev
account: ops (bab)
attached tickets.example.com -> default.go.tito.io
DNS: pointing
using http://127.0.0.1:33665 as paul@keymail.dev
account: ops (bab)
attached carloku.example.com -> default.go.tito.io
2026/10/12 11:11:43 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:12:14 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:12:14 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:12:57 console: CARLOS_SECRETS_PUBKEY is unset, so no region outside the home can be chosen automatically
2026/10/12 11:13:02 domains: minted platform domain echoapp.bab.oncarlos.test (upstream app.local)
2026/10/12 11:13:02 domains: minted platform domain site.bab.oncarlos.test (upstream site.local)
2026/10/12 11:13:04 edge: static oldsite/stable: resolve channel: dirstore: apps/bab/oldsite/channels/stable.json: file does not exist — if this deployment predates account-scoped bucket keys its objects are still at apps/oldsite/, which the runtime no longer reads; move them once with: carlos accounts migrate --app oldsite --from legacy --to bab
2026/10/12 11:13:04 domains: minted platform domain blog.mdbbndjqx.oncarlos.test (upstream blog.kass.test)
2026/10/12 11:13:04 domains: minted platform domain blog.gdpkwhwrw.oncarlos.test (upstream blog.woodstar.test)
2026/10/12 11:13:04 domains: minted platform domain oldsite.bab.oncarlos.test (upstream old.test)
2026/10/12 11:13:04 edge: static oldsite/stable: resolve channel: dirstore: apps/bab/oldsite/channels/stable.json: file does not exist — if this deployment predates account-scoped bucket keys its objects are still at apps/oldsite/, which the runtime no longer reads; move them once with: carlos accounts migrate --app oldsite --from legacy --to bab
FAIL
FAIL	github.com/carlosframework/platform	338.706s
?   	github.com/carlosframework/platform/cmd/changelog	[no test files]
ok  	github.com/carlosframework/platform/cmd/console	3.041s
ok  	github.com/carlosframework/platform/cmd/moveprobe	93.257s
ok  	github.com/carlosframework/platform/internal/activator	210.314s
ok  	github.com/carlosframework/platform/internal/agentsock	0.016s
ok  	github.com/carlosframework/platform/internal/alerting	0.004s
ok  	github.com/carlosframework/platform/internal/analytics	0.018s
ok  	github.com/carlosframework/platform/internal/appconfig	2.306s
ok  	github.com/carlosframework/platform/internal/apperr	0.090s
ok  	github.com/carlosframework/platform/internal/applog	0.138s
ok  	github.com/carlosframework/platform/internal/awsinv	0.004s
ok  	github.com/carlosframework/platform/internal/boxchoice	0.005s
ok  	github.com/carlosframework/platform/internal/boxid	0.526s
?   	github.com/carlosframework/platform/internal/boxid/boxidtest	[no test files]
?   	github.com/carlosframework/platform/internal/boxid/internal/signtest	[no test files]
ok  	github.com/carlosframework/platform/internal/boxvolumes	0.009s
ok  	github.com/carlosframework/platform/internal/caps	0.013s
ok  	github.com/carlosframework/platform/internal/certwarm	0.006s
ok  	github.com/carlosframework/platform/internal/cgroup	0.189s
ok  	github.com/carlosframework/platform/internal/changelog	0.247s
ok  	github.com/carlosframework/platform/internal/clamd	0.055s
?   	github.com/carlosframework/platform/internal/cliflag	[no test files]
ok  	github.com/carlosframework/platform/internal/console	356.026s
?   	github.com/carlosframework/platform/internal/controlname	[no test files]
?   	github.com/carlosframework/platform/internal/controlsock	[no test files]
ok  	github.com/carlosframework/platform/internal/dbdoor	6.322s
ok  	github.com/carlosframework/platform/internal/dbdoor/dbdoortest	0.010s
ok  	github.com/carlosframework/platform/internal/dbverify	0.004s
ok  	github.com/carlosframework/platform/internal/dnsrec	0.003s
ok  	github.com/carlosframework/platform/internal/docsite	0.008s
ok  	github.com/carlosframework/platform/internal/domains	2.059s
ok  	github.com/carlosframework/platform/internal/doorkey	0.007s
ok  	github.com/carlosframework/platform/internal/economics	0.004s
ok  	github.com/carlosframework/platform/internal/edge	26.015s
?   	github.com/carlosframework/platform/internal/edgeproof	[no test files]
ok  	github.com/carlosframework/platform/internal/edgesfile	0.008s
ok  	github.com/carlosframework/platform/internal/fleetapi	0.135s
ok  	github.com/carlosframework/platform/internal/gate	1.658s
ok  	github.com/carlosframework/platform/internal/genselect	0.016s
ok  	github.com/carlosframework/platform/internal/geo	0.010s
ok  	github.com/carlosframework/platform/internal/geoip	0.046s
ok  	github.com/carlosframework/platform/internal/host	63.566s
ok  	github.com/carlosframework/platform/internal/hostactors	0.341s
ok  	github.com/carlosframework/platform/internal/hostbox	1.610s
ok  	github.com/carlosframework/platform/internal/hostclaim	0.464s
ok  	github.com/carlosframework/platform/internal/hostevidence	0.025s
ok  	github.com/carlosframework/platform/internal/hostexec	0.314s
ok  	github.com/carlosframework/platform/internal/hostfence	0.067s
ok  	github.com/carlosframework/platform/internal/hostfiles	10.454s
ok  	github.com/carlosframework/platform/internal/hostfloor	0.022s
ok  	github.com/carlosframework/platform/internal/hostguard	0.150s
ok  	github.com/carlosframework/platform/internal/hostops	1.417s
?   	github.com/carlosframework/platform/internal/hostops/fencerelease	[no test files]
ok  	github.com/carlosframework/platform/internal/hostops/internal/fencerelease	0.002s
ok  	github.com/carlosframework/platform/internal/hostquarantine	0.002s
ok  	github.com/carlosframework/platform/internal/hostrelocate	0.455s
ok  	github.com/carlosframework/platform/internal/hostresolve	0.138s
ok  	github.com/carlosframework/platform/internal/hostwrite	0.002s
ok  	github.com/carlosframework/platform/internal/infraview	0.002s
ok  	github.com/carlosframework/platform/internal/instancectl	1.704s
ok  	github.com/carlosframework/platform/internal/instanceexec	2.695s
ok  	github.com/carlosframework/platform/internal/instancefs	0.006s
ok  	github.com/carlosframework/platform/internal/instancekey	0.002s
ok  	github.com/carlosframework/platform/internal/instanceupdates	0.006s
ok  	github.com/carlosframework/platform/internal/layout	1.392s
ok  	github.com/carlosframework/platform/internal/lease	9.466s
ok  	github.com/carlosframework/platform/internal/lease/leasetest	0.006s
ok  	github.com/carlosframework/platform/internal/ledger	0.012s
ok  	github.com/carlosframework/platform/internal/mail	0.169s
ok  	github.com/carlosframework/platform/internal/maintenance	0.087s
ok  	github.com/carlosframework/platform/internal/maintenanceexec	5.550s
ok  	github.com/carlosframework/platform/internal/maintenanceguard	0.072s
ok  	github.com/carlosframework/platform/internal/metrics	0.005s
ok  	github.com/carlosframework/platform/internal/moves	21.242s
ok  	github.com/carlosframework/platform/internal/moves/archive	0.788s
ok  	github.com/carlosframework/platform/internal/objstore	0.172s
ok  	github.com/carlosframework/platform/internal/ops	0.153s
ok  	github.com/carlosframework/platform/internal/park	0.189s
ok  	github.com/carlosframework/platform/internal/privatefile	0.004s
ok  	github.com/carlosframework/platform/internal/provision	3.255s
ok  	github.com/carlosframework/platform/internal/publicrelease	0.008s
ok  	github.com/carlosframework/platform/internal/registry	3.748s
ok  	github.com/carlosframework/platform/internal/relay	7.218s
ok  	github.com/carlosframework/platform/internal/release	18.247s
ok  	github.com/carlosframework/platform/internal/rollout	1.524s
ok  	github.com/carlosframework/platform/internal/rolloutwords	0.004s
ok  	github.com/carlosframework/platform/internal/rootaws	0.007s
ok  	github.com/carlosframework/platform/internal/s3compat	0.005s
ok  	github.com/carlosframework/platform/internal/safefile	0.004s
ok  	github.com/carlosframework/platform/internal/schedule	2.258s
ok  	github.com/carlosframework/platform/internal/sim/hostlife	2.632s
ok  	github.com/carlosframework/platform/internal/sim/litestreamfacts	3.515s
ok  	github.com/carlosframework/platform/internal/status	0.041s
ok  	github.com/carlosframework/platform/internal/steering	0.006s
ok  	github.com/carlosframework/platform/internal/stopevidence	0.158s
ok  	github.com/carlosframework/platform/internal/svcrole	0.002s
ok  	github.com/carlosframework/platform/internal/telemetry	0.004s
ok  	github.com/carlosframework/platform/internal/termfence	0.004s
ok  	github.com/carlosframework/platform/internal/tunnelclient	0.004s
ok  	github.com/carlosframework/platform/internal/unixdial	0.003s
ok  	github.com/carlosframework/platform/internal/updatewords	0.002s
ok  	github.com/carlosframework/platform/internal/usage	0.005s
ok  	github.com/carlosframework/platform/internal/version	0.002s
ok  	github.com/carlosframework/platform/internal/vet	1.567s
FAIL
make: *** [Makefile:18: test] Error 1

amadan: step gate failed: exit status 2

amadan: job failed at step gate — its output is above, not at the tail of this log

diff --git a/internal/console/apiinstancemove.go b/internal/console/apiinstancemove.go

index e3e8e347e..2dd56bd27 100644

--- a/internal/console/apiinstancemove.go

+++ b/internal/console/apiinstancemove.go

@@ -727,6 +727,8 @@ func (s *Server) handleAPIInstanceMoveRequest(w http.ResponseWriter, r *http.Req

// admitted, with its recorded sentence. The error itself is only logged.

func moveClaimCASFailed(w http.ResponseWriter, acct *Account, app, host, to string, err error) {

switch {

+ case errors.Is(err, hostclaim.ErrRelocationFinishing):

+ jsonError(w, http.StatusConflict, "the instance's last relocation is still finishing. try again in a few minutes")

case errors.Is(err, hostclaim.ErrPendingChore):

jsonError(w, http.StatusConflict, "the instance has unfinished cleanup in "+to+". wait for it to finish")

case errors.Is(err, hostclaim.ErrConflict), errors.Is(err, hostclaim.ErrWrongState):

diff --git a/internal/console/apiinstancemove_relocation_test.go b/internal/console/apiinstancemove_relocation_test.go

new file mode 100644

index 000000000..12ce9fdd9

--- /dev/null

+++ b/internal/console/apiinstancemove_relocation_test.go

@@ -0,0 +1,44 @@

+package console

+import (

+ "encoding/json"

+ "fmt"

+ "net/http"

+ "net/http/httptest"

+ "testing"

+ "github.com/carlosframework/platform/internal/hostclaim"

+// The completion barrier's refusal gets its own sentence at the move door;

+// any other pending chore keeps the original one, which names the region.

+func TestMoveClaimCASFailedNamesAFinishingRelocation(t *testing.T) {

+ cases := []struct {

+ name string

+ err error

+ want string

+ }{

+ {"relocation finishing", fmt.Errorf("%w: %w: relocation rel-1 on box-b", hostclaim.ErrPendingChore, hostclaim.ErrRelocationFinishing),

+ "the instance's last relocation is still finishing. try again in a few minutes"},

+ {"other pending chore", fmt.Errorf("%w: park in uk", hostclaim.ErrPendingChore),

+ "the instance has unfinished cleanup in uk. wait for it to finish"},

+ }

+ for _, tc := range cases {

+ t.Run(tc.name, func(t *testing.T) {

+ w := httptest.NewRecorder()

+ moveClaimCASFailed(w, &Account{Sqid: "a"}, "shop", "shop-1", "uk", tc.err)

+ if w.Code != http.StatusConflict {

+ t.Fatalf("status %d, want 409", w.Code)

+ }

+ var body struct {

+ Error string `json:"error"`

+ }

+ if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil {

+ t.Fatalf("body %q: %v", w.Body.String(), err)

+ }

+ if body.Error != tc.want {

+ t.Fatalf("error %q, want %q", body.Error, tc.want)

+ }

+ })

+ }

diff --git a/internal/hostclaim/ops.go b/internal/hostclaim/ops.go

index b50bf0652..d645eb840 100644

--- a/internal/hostclaim/ops.go

+++ b/internal/hostclaim/ops.go

@@ -9,6 +9,12 @@ var (

ErrStaleOp = errors.New("hostclaim: the claim is under a different operation")

ErrWrongState = errors.New("hostclaim: transition not legal from the claim's state")

ErrPendingChore = errors.New("hostclaim: an obligation is pending in a location this operation writes")

+ // ErrRelocationFinishing is the completion barrier's refusal: the

+ // serving op journals a completed relocation that its box has not yet

+ // discharged. It is always returned wrapped with ErrPendingChore, so a

+ // caller that only knows ErrPendingChore refuses it the same way; the

+ // move door tells the two apart for its sentence.

+ ErrRelocationFinishing = errors.New("hostclaim: a completed relocation is not yet discharged")

)

func RequireOp(id string) func(*Claim) error {

diff --git a/internal/hostclaim/relocation_probestop.go b/internal/hostclaim/relocation_probestop.go

index a2fa9c528..05a8839e3 100644

--- a/internal/hostclaim/relocation_probestop.go

+++ b/internal/hostclaim/relocation_probestop.go

@@ -118,7 +118,7 @@ func completionBarrier(c *Claim, fencedSource string) error {

if !ok || done.Discharged || slices.Contains(c.Lost, done.Stop.Box) || (fencedSource != "" && fencedSource == done.Stop.Box) {

return nil

}

- return fmt.Errorf("%w: relocation %s is not yet discharged on %s", ErrPendingChore, done.Op, done.Stop.Box)

+ return fmt.Errorf("%w: %w: relocation %s on %s", ErrPendingChore, ErrRelocationFinishing, done.Op, done.Stop.Box)

}

// MarkCompletedDischarged records, on the serving op, that the box the

diff --git a/internal/hostclaim/relocation_probestop_test.go b/internal/hostclaim/relocation_probestop_test.go

index 914e4f792..04940d2f5 100644

--- a/internal/hostclaim/relocation_probestop_test.go

+++ b/internal/hostclaim/relocation_probestop_test.go

@@ -242,7 +242,7 @@ func TestMoveSuccessorWaitsForTheCompletedDischarge(t *testing.T) {

s, o := completedDrain(t)

move := BeginAdmittedMove("mv-1", "console", relTarget, relSpare, "uk", relObserved.Add(time.Hour),

MoveAdmission{AppID: "app-1", AdmissionID: "mv-1", SourceEnrollment: relEnrollment(relTarget), TargetEnrollment: relEnrollment(relSpare)})

- if _, err := Update(ctx, s, o, move); !errors.Is(err, ErrPendingChore) {

+ if _, err := Update(ctx, s, o, move); !errors.Is(err, ErrPendingChore) || !errors.Is(err, ErrRelocationFinishing) {

t.Fatalf("a move started before B's discharge: %v", err)

}

if _, err := Update(ctx, s, o, BeginDelete("del-1", "console", "etag-1", relObserved.Add(time.Hour), []string{"ie"}, []string{relTarget})); !errors.Is(err, ErrPendingChore) {