plan: self-serve moves 1, rev 4 (Astra plan review 3: R1 to R10)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
1 file changed,
+137
−100
docs/superpowers/plans/2026-10-22-self-serve-moves-1-lifetimes-and-order.md+137 −100
CI failed — run details
CI log
=== gate ===
.amadan/test-gate.sh
go: downloading github.com/aws/aws-sdk-go-v2/service/cloudwatch v1.68.0
go: downloading github.com/aws/aws-sdk-go-v2 v1.44.0
go: downloading github.com/aws/aws-sdk-go-v2/service/costexplorer v1.67.5
go: downloading github.com/aws/aws-sdk-go-v2/service/s3 v1.105.0
go: downloading github.com/aws/smithy-go v1.28.1
go: downloading github.com/superfly/ltx v0.5.1
go: downloading golang.org/x/net v0.56.0
go: downloading modernc.org/sqlite v1.55.0
go: downloading github.com/aws/aws-sdk-go-v2/config v1.32.6
go: downloading github.com/aws/aws-sdk-go-v2/service/sns v1.43.0
go: downloading golang.org/x/crypto v0.53.0
go: downloading github.com/benbjohnson/litestream v0.5.14
go: downloading github.com/aws/aws-sdk-go-v2/service/route53 v1.64.1
go: downloading github.com/oschwald/maxminddb-golang v1.13.1
go: downloading github.com/aws/aws-sdk-go-v2/service/acm v1.45.0
go: downloading github.com/aws/aws-sdk-go-v2/service/ec2 v1.324.0
go: downloading github.com/aws/aws-sdk-go-v2/service/iam v1.58.2
go: downloading github.com/aws/aws-sdk-go-v2/service/resourcegroupstaggingapi v1.37.0
go: downloading github.com/aws/aws-sdk-go-v2/service/sts v1.45.6
go: downloading github.com/aws/aws-sdk-go-v2/service/autoscaling v1.74.0
go: downloading golang.org/x/sys v0.46.0
go: downloading github.com/pion/logging v0.2.4
go: downloading github.com/pion/stun/v3 v3.0.1
go: downloading github.com/benbjohnson/litestream v0.5.14
go: downloading github.com/aws/aws-sdk-go-v2/service/ec2 v1.324.0
go: downloading modernc.org/sqlite v1.55.0
go: downloading github.com/aws/aws-sdk-go-v2/credentials v1.19.6
go: downloading github.com/aws/aws-sdk-go-v2/service/ssm v1.74.0
go: downloading github.com/aws/aws-sdk-go-v2/service/autoscaling v1.74.0
go: downloading github.com/pion/turn/v4 v4.1.4
go: downloading github.com/pion/turn/v4 v4.1.4
go: downloading github.com/klauspost/compress v1.18.0
go: downloading github.com/pierrec/lz4/v4 v4.1.22
go: downloading github.com/pion/dtls/v3 v3.0.7
go: downloading github.com/pion/transport/v3 v3.0.8
go: downloading github.com/pion/randutil v0.1.0
go: downloading github.com/pion/transport/v4 v4.0.1
go: downloading github.com/pierrec/lz4/v4 v4.1.22
go: downloading github.com/pion/dtls/v3 v3.0.7
go: downloading github.com/pion/transport/v3 v3.0.8
go: downloading github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.16
go: downloading github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4
go: downloading github.com/aws/aws-sdk-go-v2/service/signin v1.0.4
go: downloading github.com/aws/aws-sdk-go-v2/service/sso v1.30.8
go: downloading github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.12
go: downloading github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.40
go: downloading github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.38
go: downloading github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.38
go: downloading github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.19
go: downloading github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.40
go: downloading github.com/prometheus/client_golang v1.17.0
go: downloading github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.20.18
go: downloading github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.40
go: downloading github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.23
go: downloading github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.14
go: downloading github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.31
go: downloading github.com/wlynxg/anet v0.0.5
go: downloading github.com/lmittmann/tint v1.1.3
go: downloading github.com/mattn/go-isatty v0.0.20
go: downloading github.com/prometheus/client_golang v1.17.0
go: downloading github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.20.18
go: downloading github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.40
go: downloading github.com/wlynxg/anet v0.0.5
go: downloading github.com/beorn7/perks v1.0.1
go: downloading github.com/prometheus/common v0.45.0
go: downloading github.com/prometheus/procfs v0.12.0
go: downloading google.golang.org/protobuf v1.36.10
go: downloading github.com/prometheus/client_model v0.5.0
go: downloading github.com/cespare/xxhash/v2 v2.3.0
go: downloading github.com/prometheus/common v0.45.0
go: downloading google.golang.org/protobuf v1.36.10
go: downloading github.com/prometheus/procfs v0.12.0
go: downloading golang.org/x/text v0.38.0
go: downloading golang.org/x/text v0.38.0
go: downloading modernc.org/libc v1.74.1
go: downloading modernc.org/libc v1.74.1
go: downloading modernc.org/memory v1.11.0
go: downloading github.com/dustin/go-humanize v1.0.1
go: downloading modernc.org/mathutil v1.7.1
go: downloading github.com/google/uuid v1.6.0
go: downloading modernc.org/memory v1.11.0
go: downloading github.com/dustin/go-humanize v1.0.1
go: downloading github.com/google/uuid v1.6.0
go: downloading modernc.org/mathutil v1.7.1
go: downloading github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec
go: downloading github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec
=== launchctx (3 s)
ok amadan.net/carlos/platform.git/launchctx 0.011s
? amadan.net/carlos/platform.git/launchctx/controlname [no test files]
? amadan.net/carlos/platform.git/launchctx/wireid [no test files]
=== vet (75 s)
go: downloading github.com/aws/aws-sdk-go-v2/service/ec2 v1.324.0
go: downloading github.com/aws/aws-sdk-go-v2/service/autoscaling v1.74.0
go: downloading modernc.org/sqlite v1.55.0
go: downloading github.com/benbjohnson/litestream v0.5.14
go: downloading github.com/aws/aws-sdk-go-v2/service/sesv2 v1.66.6
go: downloading github.com/aws/aws-sdk-go-v2/service/ssm v1.74.0
go: downloading github.com/klauspost/compress v1.18.0
go: downloading golang.org/x/tools v0.47.0
go: downloading github.com/pierrec/lz4/v4 v4.1.22
go: downloading github.com/prometheus/client_golang v1.17.0
go: downloading golang.org/x/sync v0.21.0
go: downloading github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.20.18
go: downloading github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.23
go: downloading github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.31
go: downloading github.com/wlynxg/anet v0.0.5
go: downloading github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.14
go: downloading github.com/lmittmann/tint v1.1.3
go: downloading github.com/mattn/go-isatty v0.0.20
go: downloading github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.40
go: downloading github.com/prometheus/common v0.45.0
go: downloading github.com/cespare/xxhash/v2 v2.3.0
go: downloading github.com/prometheus/procfs v0.12.0
go: downloading google.golang.org/protobuf v1.36.10
go: downloading github.com/beorn7/perks v1.0.1
go: downloading github.com/prometheus/client_model v0.5.0
go: downloading github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0
go: downloading golang.org/x/text v0.38.0
go: downloading modernc.org/libc v1.74.1
go: downloading golang.org/x/mod v0.37.0
go: downloading github.com/google/uuid v1.6.0
go: downloading modernc.org/mathutil v1.7.1
go: downloading github.com/dustin/go-humanize v1.0.1
go: downloading modernc.org/memory v1.11.0
go: downloading github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec
=== gofmt (0 s)
gofmt -l .: clean
=== rest (226 s)
? github.com/carlosframework/platform/cmd/changelog [no test files]
ok github.com/carlosframework/platform/cmd/console 3.325s
ok github.com/carlosframework/platform/cmd/moveprobe 11.754s
ok github.com/carlosframework/platform/cmd/zddprobe 0.707s
ok github.com/carlosframework/platform/internal/activator 63.126s
ok github.com/carlosframework/platform/internal/agentsock 0.046s
ok github.com/carlosframework/platform/internal/alerting 0.007s
ok github.com/carlosframework/platform/internal/analytics 0.042s
ok github.com/carlosframework/platform/internal/appconfig 2.764s
ok github.com/carlosframework/platform/internal/apperr 0.121s
ok github.com/carlosframework/platform/internal/applog 0.202s
ok github.com/carlosframework/platform/internal/awsinv 0.008s
ok github.com/carlosframework/platform/internal/boxchoice 0.007s
ok github.com/carlosframework/platform/internal/boxid 0.594s
? github.com/carlosframework/platform/internal/boxid/boxidtest [no test files]
? github.com/carlosframework/platform/internal/boxid/internal/signtest [no test files]
ok github.com/carlosframework/platform/internal/boxvolumes 0.009s
ok github.com/carlosframework/platform/internal/caps 0.020s
ok github.com/carlosframework/platform/internal/certwarm 0.005s
ok github.com/carlosframework/platform/internal/cgroup 0.767s
ok github.com/carlosframework/platform/internal/changelog 0.417s
ok github.com/carlosframework/platform/internal/claimactivation 1.047s
ok github.com/carlosframework/platform/internal/clamd 0.061s
? github.com/carlosframework/platform/internal/cliflag [no test files]
? github.com/carlosframework/platform/internal/controlname [no test files]
? github.com/carlosframework/platform/internal/controlsock [no test files]
ok github.com/carlosframework/platform/internal/dbdoor 6.436s
ok github.com/carlosframework/platform/internal/dbdoor/dbdoortest 0.011s
ok github.com/carlosframework/platform/internal/dbverify 0.005s
ok github.com/carlosframework/platform/internal/dnsrec 0.005s
ok github.com/carlosframework/platform/internal/docsite 0.019s
ok github.com/carlosframework/platform/internal/domains 2.719s
ok github.com/carlosframework/platform/internal/doorkey 0.011s
ok github.com/carlosframework/platform/internal/economics 0.006s
ok github.com/carlosframework/platform/internal/edge 50.564s
? github.com/carlosframework/platform/internal/edgeproof [no test files]
ok github.com/carlosframework/platform/internal/edgesfile 0.009s
ok github.com/carlosframework/platform/internal/fleetapi 0.153s
ok github.com/carlosframework/platform/internal/gate 1.678s
ok github.com/carlosframework/platform/internal/genselect 0.020s
ok github.com/carlosframework/platform/internal/geo 0.035s
ok github.com/carlosframework/platform/internal/geoip 0.035s
ok github.com/carlosframework/platform/internal/historyreadback 0.187s
ok github.com/carlosframework/platform/internal/host 161.651s
ok github.com/carlosframework/platform/internal/hostactors 7.872s
ok github.com/carlosframework/platform/internal/hostactors/awsmembership 0.019s
ok github.com/carlosframework/platform/internal/hostbox 4.416s
ok github.com/carlosframework/platform/internal/hostclaim 1.374s
ok github.com/carlosframework/platform/internal/hostenv 0.006s
ok github.com/carlosframework/platform/internal/hostevidence 0.021s
ok github.com/carlosframework/platform/internal/hostexec 7.173s
ok github.com/carlosframework/platform/internal/hostfence 0.139s
ok github.com/carlosframework/platform/internal/hostfiles 10.904s
ok github.com/carlosframework/platform/internal/hostfloor 0.036s
ok github.com/carlosframework/platform/internal/hostguard 1.200s
ok github.com/carlosframework/platform/internal/hostops 3.035s
? github.com/carlosframework/platform/internal/hostops/fencerelease [no test files]
ok github.com/carlosframework/platform/internal/hostops/internal/fencerelease 0.006s
ok github.com/carlosframework/platform/internal/hostquarantine 0.003s
ok github.com/carlosframework/platform/internal/hostrelocate 1.107s
ok github.com/carlosframework/platform/internal/hostresolve 0.452s
ok github.com/carlosframework/platform/internal/hostwrite 0.017s
ok github.com/carlosframework/platform/internal/infraview 0.018s
ok github.com/carlosframework/platform/internal/instancectl 5.332s
ok github.com/carlosframework/platform/internal/instanceexec 3.076s
ok github.com/carlosframework/platform/internal/instancefs 0.029s
ok github.com/carlosframework/platform/internal/instancekey 0.017s
ok github.com/carlosframework/platform/internal/instanceupdates 0.016s
ok github.com/carlosframework/platform/internal/layout 2.381s
ok github.com/carlosframework/platform/internal/lbhealth 0.019s
ok github.com/carlosframework/platform/internal/lease 10.181s
ok github.com/carlosframework/platform/internal/lease/leasetest 0.921s
ok github.com/carlosframework/platform/internal/ledger 0.047s
ok github.com/carlosframework/platform/internal/mail 0.450s
ok github.com/carlosframework/platform/internal/maintenance 0.116s
ok github.com/carlosframework/platform/internal/maintenanceexec 5.766s
ok github.com/carlosframework/platform/internal/maintenanceguard 0.259s
ok github.com/carlosframework/platform/internal/metrics 0.027s
ok github.com/carlosframework/platform/internal/moves 36.780s
ok github.com/carlosframework/platform/internal/moves/archive 1.570s
ok github.com/carlosframework/platform/internal/objstore 0.461s
ok github.com/carlosframework/platform/internal/ops 0.276s
ok github.com/carlosframework/platform/internal/park 0.408s
ok github.com/carlosframework/platform/internal/plans 0.024s
ok github.com/carlosframework/platform/internal/privatefile 0.007s
ok github.com/carlosframework/platform/internal/provision 73.842s
ok github.com/carlosframework/platform/internal/publicrelease 0.009s
? github.com/carlosframework/platform/internal/regionname [no test files]
ok github.com/carlosframework/platform/internal/registry 6.569s
ok github.com/carlosframework/platform/internal/relay 7.232s
ok github.com/carlosframework/platform/internal/release 32.816s
--- FAIL: TestAMoveOntoTheInstalledGoodBuildWaitsForASignedRollback (5.03s)
--- FAIL: TestAMoveOntoTheInstalledGoodBuildWaitsForASignedRollback/socket_instance (5.02s)
holdran_test.go:796: timed out waiting for the move onto v1
FAIL
FAIL github.com/carlosframework/platform/internal/rollout 8.667s
ok github.com/carlosframework/platform/internal/rolloutwords 0.003s
ok github.com/carlosframework/platform/internal/rootaws 0.012s
ok github.com/carlosframework/platform/internal/rootparticipation 0.075s
ok github.com/carlosframework/platform/internal/rootstate 0.059s
ok github.com/carlosframework/platform/internal/runs 0.654s
ok github.com/carlosframework/platform/internal/s3compat 0.005s
ok github.com/carlosframework/platform/internal/safefile 0.007s
ok github.com/carlosframework/platform/internal/schedule 3.094s
ok github.com/carlosframework/platform/internal/seeds 0.028s
ok github.com/carlosframework/platform/internal/sim/hostlife 13.507s
ok github.com/carlosframework/platform/internal/sim/litestreamfacts 3.935s
ok github.com/carlosframework/platform/internal/status 0.132s
ok github.com/carlosframework/platform/internal/steering 0.011s
ok github.com/carlosframework/platform/internal/stopevidence 0.384s
ok github.com/carlosframework/platform/internal/svcrole 0.003s
ok github.com/carlosframework/platform/internal/telemetry 0.006s
ok github.com/carlosframework/platform/internal/termfence 0.005s
ok github.com/carlosframework/platform/internal/tunnelclient 0.006s
ok github.com/carlosframework/platform/internal/unixdial 0.005s
ok github.com/carlosframework/platform/internal/updatewords 0.006s
ok github.com/carlosframework/platform/internal/usage 0.007s
ok github.com/carlosframework/platform/internal/version 0.007s
ok github.com/carlosframework/platform/internal/vet 1.937s
FAIL
=== internal_console@0 (92 s)
ok github.com/carlosframework/platform/internal/console 90.346s
=== internal_console@1 (53 s)
ok github.com/carlosframework/platform/internal/console 51.068s
=== internal_console@2 (54 s)
ok github.com/carlosframework/platform/internal/console 51.983s
=== internal_console@3 (46 s)
ok github.com/carlosframework/platform/internal/console 44.724s
=== internal_console@4 (52 s)
ok github.com/carlosframework/platform/internal/console 50.953s
=== internal_console@5 (47 s)
ok github.com/carlosframework/platform/internal/console 45.466s
=== internal_console@6 (50 s)
ok github.com/carlosframework/platform/internal/console 48.881s
=== internal_console@7 (60 s)
ok github.com/carlosframework/platform/internal/console 58.389s
=== root@0 (49 s)
ok github.com/carlosframework/platform 47.152s
=== root@1 (56 s)
ok github.com/carlosframework/platform 54.497s
=== root@2 (53 s)
ok github.com/carlosframework/platform 51.251s
FAILED: rest
make: *** [Makefile:19: test] Error 1
amadan: step gate failed: exit status 2
amadan: job failed at step gate — its output is above, not at the tail of this log
diff --git a/docs/superpowers/plans/2026-10-22-self-serve-moves-1-lifetimes-and-order.md b/docs/superpowers/plans/2026-10-22-self-serve-moves-1-lifetimes-and-order.md| index ffbc9ae90..ebb266a97 100644 |
| --- a/docs/superpowers/plans/2026-10-22-self-serve-moves-1-lifetimes-and-order.md |
| +++ b/docs/superpowers/plans/2026-10-22-self-serve-moves-1-lifetimes-and-order.md |
| @@ -2,11 +2,11 @@ |
| > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. |
| -Rev 3, 2026-10-22. Rev 1 and rev 2 were reviewed by Astra (`after236/selfserve-plan1-review.md`, 16 findings; `after236/selfserve-plan1-review2.md`, 12 findings F1 to F12). Rev 3 answers F1 to F12: the slot carries its owning record key; every claims-off record removal (the import rule's ordinary delete, seed finishing, the detach and rollback callers) goes through one retirement function that classifies the record and the slot together before it writes; the create re-checks hostname ownership under the exclusion and reports a pending allocation as its own result so a caller never releases a claim over one; the consumers are named by function with concrete judged readers and a re-read on a lifetime mismatch; the row's pair moves only through guarded setters that name the lifetime they expect; the fold compares only entries of one lifetime; and the fixtures are corrected. |
| +Rev 4, 2026-10-22. Reviews: rev 1 (`after236/selfserve-plan1-review.md`, 16 findings), rev 2 (`selfserve-plan1-review2.md`, F1 to F12), rev 3 (`selfserve-plan1-review3.md`, R1 to R10). Rev 4 answers R1 to R10: the DELETE door's absent-record path goes through retirement with an "expect absent" precondition; a slot CAS conflict re-classifies and only positive successor evidence supersedes, with a complete error table after the allocation is durable; a legacy retirement's migration is bound to the caller's observed ETag; the retirement recovery table is complete with exact predicates; the slot cache returns the observation it validated and refreshes both objects before believing `stale`; the box's raw rereads are judged; the hold setters install the lifetime together with the hold on a legacy row and a held predecessor row is retired by its op before the successor is minted; the fold compares pairs only between unstamped entries and the claims-off writer alone publishes them; one authoritative transition table; the fixtures corrected. |
| **Goal:** Give every claims-off instance record a durable lifetime (the host incarnation slot) and every served row a serving order `(incarnation, serving_seq)` that boxes publish and edges honour, with crash-safe create, retirement and migration, so a later plan's holding pages and moves can never be out-ordered by a stale publication. |
| -**Architecture:** A per-hostname slot object in the deployment bucket (`control/host-incarnations/<host>.json`), created once and changed only by conditional writes, is the authority for a record's lifetime and names the record key that owns it; the console allocates, retires and migrates through persisted intents in SQLite under a per-hostname exclusion and recovers them at start. Records gain `incarnation`, `incarnation_migrating`, `move_seq` and a tombstone form; registry rows gain the pair; the edges file publishes it with each alias's target; the home map orders entries of one lifetime by it before the existing claim stamp. Retirement replaces the record with a tombstone before retiring the slot. Lazy migration marks a record by CAS before binding a slot to it, and the write primitive refuses every other writer while the marker stands. |
| +**Architecture:** A per-hostname slot object in the deployment bucket (`control/host-incarnations/<host>.json`), created once and changed only by conditional writes, is the authority for a record's lifetime and names the record key that owns it; the console allocates, retires and migrates through persisted intents in SQLite under a per-hostname exclusion and recovers them at start. Records gain `incarnation`, `incarnation_migrating`, `move_seq` and a tombstone form; registry rows gain the pair; the claims-off edges writer publishes it with each alias's target; the home map orders unstamped entries of one lifetime by it before the existing claim stamp rule. Retirement replaces the record with a tombstone before retiring the slot. Lazy migration marks a record by CAS before binding a slot to it, and the write primitive refuses every other writer while the marker stands. |
| **Tech Stack:** Go. `internal/provision` (records, sequence, reconcile, offline transitions), `internal/registry`, `internal/console` (doors, store, store gate, passes), `internal/host` (edges file, offline hold and release, the agent's config), `internal/edge` (home map). Tests use `release.NewDirStore`, real registries, the console's doors in-process, and the existing two-box offline witness `TestOfflineMoveE2E` (`internal/host/movesoffline_e2e_test.go`). |
| @@ -14,14 +14,14 @@ Rev 3, 2026-10-22. Rev 1 and rev 2 were reviewed by Astra (`after236/selfserve-p |
| ## Global Constraints |
| -- **Scope: claims-off, every record.** On the console the protocol runs only when `!s.cfg.HostClaim`; on a box only when `!hostwrite.Enabled(getenv)` (`CARLOS_HOST_CLAIM` unset). Where it runs it applies to **every** instance record, fieldless ones included (a fieldless record beside a slot is stale, Task 1). Claims-on records are created by `finishClaimCreate` and retired by the host-claim operations; this plan does not touch them, their consumers read no slot (the slot cache is nil there, pinned by `TestClaimsOnConsumersReadNoSlot`), and their rows keep the claim stamp order. Self-serve moves refuse on a claims-on console (MV-O20). |
| -- Every claims-off consumer treats a record whose lifetime verdict is `stale` (Task 1) as absent, and one whose verdict is `unreadable` as a record it cannot judge this pass: reconcile protects the row, a door answers 502, a listing reports the host unreadable. |
| -- The slot is never deleted and never advanced to abandon a create (spec §6.4). It never moves backwards: every slot write names the exact prior state it replaces. |
| -- One per-hostname exclusion (`withHostname`, Task 3) around create, retirement, migration and their recovery in the console process. A persisted pending operation for a hostname refuses any create or retirement of that hostname by another operation (409 "another change to this instance is still settling. try again in a minute") until recovery resolves it; unreadable evidence leaves it pending and keeps refusing. |
| +- **Scope: claims-off, every record.** On the console the protocol runs only when `!s.cfg.HostClaim`; on a box only when `!hostwrite.Enabled(getenv)` (`CARLOS_HOST_CLAIM` unset). Where it runs it applies to **every** instance record, fieldless ones included (a fieldless record beside a slot is stale, Task 1). Claims-on records are created by `finishClaimCreate` and retired by the host-claim operations; this plan does not touch them, their consumers read no slot (the slot cache is nil there, pinned by `TestClaimsOnConsumersReadNoSlot`), the claims-on edges writer publishes no pair, and their rows keep the claim stamp order (Task 8). Self-serve moves refuse on a claims-on console (MV-O20). |
| +- Every claims-off consumer treats a record whose lifetime verdict is `stale` (Task 1) as absent, one whose verdict is `corrupt` as today's unreadable body (reconcile protects; the delete rule may remove it conditionally), and one whose verdict is `unreadable` as a record it cannot judge this pass: reconcile protects the row, a door answers 502, a listing reports the host unreadable. |
| +- The slot is never deleted and never advanced to abandon a create (spec §6.4). It never moves backwards: every slot write names the exact prior state it replaces. A malformed slot body (unparseable, no owner, zero incarnation, a state other than `live`/`retired`) is `unreadable`, never `stale`. |
| +- One per-hostname exclusion (`withHostname`, Task 3) around create, retirement, migration and their recovery in the console process. A persisted pending operation for a hostname refuses any create or retirement of that hostname by another operation (409 "another change to this instance is still settling. try again in a minute") until recovery resolves it; unreadable evidence leaves it pending and keeps refusing. After the allocation, retirement or migration row is durable, no failure returns an ordinary error to the caller: the outcome is `errLifetimePending` (the intent stays pending; recovery owns it), a persisted terminal state, or success. |
| - Every record write stays on `provision.WriteRecord` with the exact bytes read and the sequence discipline of `record_sequence.go`; no new raw `Put` of a record key. Every record removal on a claims-off console is `retireRecordWithLifetime` (Task 4); `provision.DeleteRecord` gains no new caller and loses its claims-off instance callers. |
| - `release.DeleteIfMatch` never falls back to a plain delete; absence counts as success. |
| - No console write to a bucket outside the deployment bucket; no root-side change. |
| -- Census: every changed unit in `internal/host` or `internal/provision` takes a row in `internal/provision/testdata/plan14-root-reads-self-serve-moves-1.tsv`, loaded last by a new block at the end of `TestPlan14RootReadCensus` in `internal/provision/plan14_root_reads_test.go`, `rev3=PROPOSED-self-serve-moves-1`, with the `@fields` line naming the new record fields and the note `docs/superpowers/notes/census-proposal-self-serve-moves-1.md`. `TestPlan9B8RecordWriteCensus` takes its inventory from a new `plan14SelfServeMoves1WriteCensusInventory()` that wraps `plan14ClaimRosterBindingWriteCensusInventory()` and lists each added, changed and removed record-write site of this plan (the tombstone, the marker, the migration's two writes, the lifetime create, the transitions whose digests change, the removed `DeleteIfMatch` sites in `deleteRead` and `finishSeedOnce`). Both tests green. |
| +- Census: `TestPlan14RootReadCensus` and `TestPlan9B8RecordWriteCensus` are the authority on what they cover (every production package, source envelopes, tracked fields, key sites and primitive digests). The root-read overlay is `internal/provision/testdata/plan14-root-reads-self-serve-moves-1.tsv`, loaded last (before the comparison) by a new block at the end of the test in `internal/provision/plan14_root_reads_test.go`, `rev3=PROPOSED-self-serve-moves-1`, with an `@fields` line naming the new record fields and the registry columns, rows for every changed unit and envelope the test names (host, provision, registry, console, edge), reviewed dispositions for changed frozen units, and the note `docs/superpowers/notes/census-proposal-self-serve-moves-1.md`. The record-write inventory is `plan14SelfServeMoves1WriteCensusInventory()`, wrapping `plan14ClaimRosterBindingWriteCensusInventory()` with every added, changed and removed site the scanner reports (write sites, key sites, readers, primitive digests), each reviewed. Both tests green. |
| - The new SQLite tables are classified in `claimSQLTables` (`internal/console/store_gate.go`) so a claims-on store still opens. |
| - Copy: two new sentences are copy rows pending Paul's review: "the instance changed. read it again and try the delete once more" and "another change to this instance is still settling. try again in a minute". Existing sentences are reused where the plan says so. No tenant names: `shop.example.com`, account `bab`, app `shop`. |
| - Boxes roll before the console (the roll order rule); the migration pass is behind `CARLOS_LIFETIMES_MIGRATE=1`, default off, until every box of a deployment runs this code. |
| @@ -29,11 +29,11 @@ Rev 3, 2026-10-22. Rev 1 and rev 2 were reviewed by Astra (`after236/selfserve-p |
| ## Review Focus |
| -1. A console crash between the slot write and the record write, a second console start, then the late write: the replay and the late write must both be harmless (Task 3, `TestRecoveryReplaysCreateAndTheLateWriteIsHarmless`), and a delete arriving during that recovery is refused as settling (`TestDeleteDuringCreateRecoveryIsRefused`). |
| -2. A box-side offline transition issued against the pre-marker ETag during migration: its CAS fails and its retry succeeds after the field is set (Task 5, `TestAnIssuedWriteLosesToTheMarkerAndRetries`); a consumer whose record read straddles the marker must not retire the host (Task 6, `TestReconcileRereadsAcrossAMigration`). |
| -3. A legacy fieldless record beside a retired slot (an old console's late create): the strict listings, the console doors, reconcile and the offline keep view all treat it as absent, and the next create replaces it (Task 6 `TestEveryClaimsOffConsumerIgnoresAFieldlessRecordBesideASlot`, Task 3 `TestCreateReplacesStaleRecord`). |
| -4. A reconstructed registry row after a finished move and after an abort from landed: reconcile installs the record's pair on the minted row, `(1, 3)` in both cases (Task 7, `TestReconcileInstallsTheRecordsPairOnAMintedRow`). |
| -5. An existing row across a recreation and a migration, held and unheld: a held row never takes a new lifetime's pair, an unheld row of the old lifetime takes the new one once, and a setter from the old lifetime cannot move the successor's row (Task 7, `TestExistingRowsAcrossLifetimes`). |
| +1. A console crash between the slot write and the record write, a second console start, then the late write; and the mirror case where the original slot write lands after recovery read the predecessor: the replay and the late writes are harmless and the allocation is never abandoned (Task 3, `TestRecoveryReplaysCreateAndTheLateWriteIsHarmless`, `TestLateSlotWriteDoesNotAbandonTheAllocation`); a DELETE arriving while the record is absent and the allocation pending is refused as settling through the real door (`TestDeleteDuringCreateRecoveryIsRefused`). |
| +2. A box-side offline transition issued against the pre-marker ETag during migration: its CAS fails and its retry succeeds after the field is set (Task 5, `TestAnIssuedWriteLosesToTheMarkerAndRetries`); a consumer whose reads straddle the marker, in either order and with cached slot evidence, ends with the validated record and ETag, never a removal (Task 6, `TestJudgeAcrossAMigrationBothOrders`). |
| +3. A legacy fieldless record beside a retired slot (an old console's late create): the strict listings, the console doors, reconcile, the box's reassert and staging rereads and the offline keep view all treat it as absent, the edges file stops publishing its row after the next reconcile, and the next create replaces it (Task 6 `TestEveryClaimsOffConsumerIgnoresAFieldlessRecordBesideASlot`, Task 3 `TestCreateReplacesStaleRecord`). |
| +4. A reconstructed registry row after a finished move and after an abort from landed: reconcile installs `(1, 3)` on the minted row; an existing row that stayed through the finish keeps `(1, 2)` (Task 7, `TestReconcileInstallsTheRecordsPairOnAMintedRow`, `TestAnExistingRowKeepsOpenSequenceThroughFinish`). |
| +5. Existing rows across lifetimes: a legacy row held at incarnation 0 whose record migrates is released by the move's open or abort; a ready legacy row takes the lifetime and the hold in one write; a predecessor row held by a vanished move is retired by its op and the successor is minted ready at `(2, 0)` (Task 7, `TestExistingRowsAcrossLifetimes`). |
| (The alias repointed during a move and the holding entry's fold are plan 3's Review Focus.) |
| @@ -49,14 +49,14 @@ Rev 3, 2026-10-22. Rev 1 and rev 2 were reviewed by Astra (`after236/selfserve-p |
| **Interfaces:** |
| - Produces: |
| - `Record.Incarnation uint64 \`json:"incarnation,omitempty"\``, `Record.IncarnationMigrating string \`json:"incarnation_migrating,omitempty"\``, `Record.MoveSeq uint64 \`json:"move_seq,omitempty"\``, `Record.Retired bool \`json:"retired,omitempty"\``. None of the four joins `protectedFieldsEqual` (`maintenance.go:81`): ordinary updates do not touch them, the migration's writes do, and the fence (Task 5) stops everyone else while a marker stands. |
| - - `type HostIncarnation struct { Incarnation uint64 \`json:"incarnation"\`; State string \`json:"state"\`; Account string \`json:"account"\`; App string \`json:"app"\`; Intent string \`json:"intent,omitempty"\`; RecordETag string \`json:"record_etag,omitempty"\` }`; `const SlotLive = "live"`, `SlotRetired = "retired"`. `Account`/`App` are the record key the lifetime belongs to (the sqid and app, as `RecordKey` takes them); `RecordETag` is set only by a migration slot. |
| + - `type HostIncarnation struct { Incarnation uint64 \`json:"incarnation"\`; State string \`json:"state"\`; Account string \`json:"account"\`; App string \`json:"app"\`; Intent string \`json:"intent,omitempty"\`; RecordETag string \`json:"record_etag,omitempty"\` }`; `const SlotLive = "live"`, `SlotRetired = "retired"`. `Account`/`App` are the record key the lifetime belongs to; `RecordETag` is set only by a migration slot. `func (h HostIncarnation) Valid() bool` (incarnation > 0, state live or retired, account and app non-empty). |
| - `func HostIncarnationKey(host string) string` → `control/host-incarnations/<host>.json`; "" for a host failing `ValidHostShape`. |
| - - `func ReadHostIncarnation(ctx context.Context, s release.Store, host string) (*HostIncarnation, string, error)`: nil, "", nil when absent. |
| - - `func WriteHostIncarnation(ctx context.Context, s release.Store, host string, slot HostIncarnation, ifMatch string) (string, error)`: `ifMatch == ""` is create-only (`Store.Put` with "" is `If-None-Match: *` on S3 and refuses an existing file on `DirStore`), otherwise `If-Match`. |
| - - `type LifetimeVerdict int`; `const (LifetimeValid LifetimeVerdict = iota; LifetimeStale; LifetimeUnreadable)`. |
| - - `func LifetimeOf(rec *Record, recordETag string, slot *HostIncarnation) LifetimeVerdict` (pure; never `LifetimeUnreadable`, which only a failed slot read produces, Task 6). |
| + - `var ErrSlotMalformed`; `func ReadHostIncarnation(ctx context.Context, s release.Store, host string) (*HostIncarnation, string, error)`: nil, "", nil when absent; `ErrSlotMalformed` (wrapping the decode error) for a body that does not decode or fails `Valid`. |
| + - `func WriteHostIncarnation(ctx context.Context, s release.Store, host string, slot HostIncarnation, ifMatch string) (string, error)`: refuses an invalid slot; `ifMatch == ""` is create-only (`Store.Put` with "" is `If-None-Match: *` on S3 and refuses an existing file on `DirStore`), otherwise `If-Match`. |
| + - `type LifetimeVerdict int`; `const (LifetimeValid LifetimeVerdict = iota; LifetimeStale; LifetimeCorrupt; LifetimeUnreadable)`. |
| + - `func LifetimeOf(rec *Record, recordETag string, slot *HostIncarnation) LifetimeVerdict` (pure; never `LifetimeCorrupt` or `LifetimeUnreadable`, which only a failed decode or a failed read produce, Tasks 3 and 6). |
| - `func TombstoneBody(before []byte) ([]byte, error)`: decodes `before` as JSON fields, keeps exactly `schema`, `host`, `account`, `app`, `incarnation` and `record_seq` (the last two only when present in `before`), adds `retired: true`, drops everything else. The caller writes it with `WriteRecord(ctx, s, key, before, body, etag)`, so `record_seq` advances by one when `before` carries it. |
| - - `func GetRecordRaw(ctx context.Context, s release.Store, acct, app, host string) (*Record, []byte, string, error)`: the decoded record (tombstones included), its exact bytes and ETag; the path check of `GetRecordETag` applies (a body naming another key is an error). |
| + - `func GetRecordRaw(ctx context.Context, s release.Store, acct, app, host string) (*Record, []byte, string, error)`: the decoded record (tombstones included), its exact bytes and ETag; a body that does not decode or disagrees with its path returns `ErrRecordCorrupt` (new, wrapping the cause) **with the raw bytes and ETag** so a caller may act on the observation (the delete rule, Task 4). |
| - `GetRecordETag` (and so `GetRecord`) return `os.ErrNotExist` for a tombstone; `GetRecordRaw` is the only reader that sees one. |
| The verdict, exactly, with `owned := slot != nil && slot.Account == rec.Account && slot.App == rec.App`: |
| @@ -65,8 +65,8 @@ The verdict, exactly, with `owned := slot != nil && slot.Account == rec.Account |
| - `rec.IncarnationMigrating != ""`: slot nil → valid; slot `{1, live, owned, Intent == "migration " + marker, RecordETag == recordETag}` → valid; otherwise stale. |
| - `rec.Incarnation > 0`: slot non-nil, `owned`, `State == live`, `slot.Incarnation == rec.Incarnation` → valid; otherwise stale. |
| -- [ ] **Step 1: Write the failing tests** in `lifetime_test.go`: `TestLifetimeOf` (one case per line above, plus a slot owned by another app → stale, and a marked record whose ETag differs from the slot's → stale); `TestHostIncarnationWrites` against `release.NewDirStore(t.TempDir())` (create-only once; a second create-only fails `release.ErrCASConflict`; `If-Match` with the returned ETag succeeds; a stale ETag fails; absent reads nil); `TestTombstoneBodyKeepsTheSequence` (a body with `record_seq: 7` and many fields → exactly the seven keys, `record_seq 7`, `retired true`; `WriteRecord` of it against `before` lands `record_seq 8`; a body without `record_seq` → six keys and no sequence); `TestGetRecordHidesATombstone`. Extend `TestRecordWireShapeIsAdditive` for the four fields. |
| -- [ ] **Step 2: Run to verify they fail**: `go test ./internal/provision -run 'LifetimeOf|HostIncarnationWrites|TombstoneBody|GetRecordHidesATombstone|RecordWireShapeIsAdditive' -count=1`. |
| +- [ ] **Step 1: Write the failing tests** in `lifetime_test.go`: `TestLifetimeOf` (one case per line above, plus a slot owned by another app → stale, and a marked record whose ETag differs from the slot's → stale); `TestHostIncarnationWrites` against `release.NewDirStore(t.TempDir())` (create-only once; a second create-only fails `release.ErrCASConflict`; `If-Match` with the returned ETag succeeds; a stale ETag fails; absent reads nil; an invalid slot is refused on write; a malformed body reads `ErrSlotMalformed`); `TestTombstoneBodyKeepsTheSequence` (a body with `record_seq: 7` and many fields → exactly the seven keys, `record_seq 7`, `retired true`; `WriteRecord` of it against `before` lands `record_seq 8`; a body without `record_seq` → six keys and no sequence); `TestGetRecordHidesATombstone`; `TestGetRecordRawReportsCorruptWithTheETag`. Extend `TestRecordWireShapeIsAdditive` for the four fields. |
| +- [ ] **Step 2: Run to verify they fail**: `go test ./internal/provision -run 'LifetimeOf|HostIncarnationWrites|TombstoneBody|GetRecordHidesATombstone|GetRecordRawReportsCorrupt|RecordWireShapeIsAdditive' -count=1`. |
| - [ ] **Step 3: Implement.** |
| - [ ] **Step 4: Run to verify they pass**, plus `go test ./internal/provision -run 'Record' -count=1`. |
| - [ ] **Step 5: Commit** `provision: record lifetime fields, the owned host incarnation slot, the tombstone form and the lifetime verdict`. |
| @@ -81,12 +81,12 @@ The verdict, exactly, with `owned := slot != nil && slot.Account == rec.Account |
| **Interfaces:** |
| - Tables (`account` is the sqid): |
| - `host_allocations (op_id TEXT PRIMARY KEY, account TEXT, app TEXT, host TEXT, incarnation INTEGER, state TEXT, body BLOB, predecessor_etag TEXT, stale_etag TEXT, created_at INTEGER)` |
| - - `record_retirements (op_id TEXT PRIMARY KEY, account TEXT, app TEXT, host TEXT, incarnation INTEGER, record_etag TEXT, slot_etag TEXT, state TEXT, created_at INTEGER)` |
| - - `record_migrations (op_id TEXT PRIMARY KEY, account TEXT, app TEXT, host TEXT, marked_etag TEXT, state TEXT, created_at INTEGER)` |
| + - `record_retirements (op_id TEXT PRIMARY KEY, account TEXT, app TEXT, host TEXT, incarnation INTEGER, record_etag TEXT, slot_etag TEXT, slot_intent TEXT, state TEXT, created_at INTEGER)` |
| + - `record_migrations (op_id TEXT PRIMARY KEY, account TEXT, app TEXT, host TEXT, expect_etag TEXT, marked_etag TEXT, state TEXT, created_at INTEGER)` |
| - Classification in `claimSQLTables`: `{Sqid: "account", App: "app"}` for all three. |
| - States: `pending`, `committed`, `superseded`, `void`, `refused`. |
| - On `*Store`: `BeginAllocation(a Allocation) error`, `SetAllocationState(opID, state string) error`, `PendingAllocations() ([]Allocation, error)`, `LastIncarnation(host string) (uint64, error)` (the highest `incarnation` over every state but `void`; 0 when none); `BeginRetirement(r Retirement) error`, `SetRetirementState(opID, state string) error`, `PendingRetirements() ([]Retirement, error)`; `BeginMigration(m Migration) error`, `SetMigrationMarkedETag(opID, etag string) error`, `SetMigrationState(opID, state string) error`, `PendingMigrations() ([]Migration, error)`; `PendingForHost(host, exceptOpID string) (int, error)` (pending rows across the three tables, not counting `exceptOpID`). |
| -- `type Allocation struct { OpID, Account, App, Host string; Incarnation uint64; State string; Body []byte; PredecessorETag, StaleETag string }`, `type Retirement struct { OpID, Account, App, Host string; Incarnation uint64; RecordETag, SlotETag, State string }`, `type Migration struct { OpID, Account, App, Host, MarkedETag, State string }`. |
| +- `type Allocation struct { OpID, Account, App, Host string; Incarnation uint64; State string; Body []byte; PredecessorETag, StaleETag string }`, `type Retirement struct { OpID, Account, App, Host string; Incarnation uint64; RecordETag, SlotETag, SlotIntent, State string }`, `type Migration struct { OpID, Account, App, Host, ExpectETag, MarkedETag, State string }`. |
| - [ ] **Step 1: Write the failing tests**: `TestLifetimeTablesRoundTrip` (begin, list pending, set state and the marked ETag, list again, for all three); `TestLastIncarnationSkipsVoid` (pending 2 and void 3 → 2); `TestPendingForHostCountsAllThreeExceptOwn`; in the store-gate tests `TestClaimStoreOpensWithLifetimeTables`. |
| - [ ] **Step 2: Run to verify they fail.** |
| @@ -98,7 +98,7 @@ The verdict, exactly, with `owned := slot != nil && slot.Account == rec.Account |
| **Files:** |
| - Create: `internal/console/lifetimes.go` (the exclusion, admission and shared classification), `internal/console/lifetimes_create.go`, `internal/console/lifetimes_recovery.go` |
| -- Modify: `internal/console/apiinstances.go` (the claims-off branch at line 1664: `recordWriteErr = provision.CreateRecord(...)` becomes the lifetime create; the sibling undo at line 1684 stays on `retireInstanceRecord`, which Task 4 routes), `internal/console/apiinstanceseed.go` (line 341), `internal/console/apiinstanceimport.go` (line 172), `internal/console/domainattach.go` (the first-route create near line 237 and the two `rollbackAttach` calls that follow it), `internal/console/claim_workers.go` (register `"StartLifetimeRecoveryPass"`), `cmd/console/main.go` (start it beside the other passes) |
| +- Modify: `internal/console/server.go` (`NewServer`: `s.slots = provision.NewSlotCache()` when `!cfg.HostClaim`), `internal/console/apiinstances.go` (the claims-off branch at line 1664: `recordWriteErr = provision.CreateRecord(...)` becomes the lifetime create; the sibling undo at line 1684 stays on `retireInstanceRecord`, which Task 4 routes), `internal/console/apiinstanceseed.go` (line 341), `internal/console/apiinstanceimport.go` (line 172), `internal/console/domainattach.go` (the first-route create near line 237 and the two `rollbackAttach` calls that follow it), `internal/console/claim_workers.go` (register `"StartLifetimeRecoveryPass"`), `cmd/console/main.go` (start it beside the other passes) |
| - Test: `internal/console/lifetimes_create_test.go` |
| **Interfaces:** |
| @@ -107,80 +107,97 @@ The verdict, exactly, with `owned := slot != nil && slot.Account == rec.Account |
| - `func (s *Server) lifetimesOn() bool` (`!s.cfg.HostClaim && s.cfg.Releases != nil`). |
| - `func (s *Server) withHostname(host string, fn func() error) error` (a `sync.Map` of `*sync.Mutex`; not reentrant; every exported helper below takes it, every `...Locked` helper assumes it). |
| - `func (s *Server) admit(host, ownOpID string) error`: `errLifetimeSettling` when `PendingForHost(host, ownOpID) > 0`. |
| - - `func (s *Server) classifyLocked(ctx context.Context, acct, app, host string) (lifetimeClass, error)` where `type lifetimeClass struct { Rec *provision.Record; Raw []byte; RecordETag string; Slot *provision.HostIncarnation; SlotETag string; Verdict provision.LifetimeVerdict }` (`Verdict` is `LifetimeUnreadable` and the error non-nil when either read fails for a reason other than absence). |
| - - `func (s *Server) createRecordWithLifetime(ctx context.Context, rec provision.Record) (*provision.Record, error)`: the one function every claims-off creation path calls; returns the record as written (incarnation set, `RecordSeq` set). Errors: `release.ErrCASConflict` (a valid record under this key; the doors keep their 409), `errHostHeld` (a live slot owned by another record key, or `hostHolder` reports another holder; the doors answer the existing `hostHeldSentence`), `errLifetimeSettling` (409, the settling sentence), `errLifetimePending` (the allocation is persisted and the record write's outcome is unknown: recovery completes it; the door answers its existing "could not create" sentence and **does not** undo anything). |
| + - `func (s *Server) classifyLocked(ctx context.Context, acct, app, host string) (lifetimeClass, error)` where `type lifetimeClass struct { Rec *provision.Record; Raw []byte; RecordETag string; Slot *provision.HostIncarnation; SlotETag string; Verdict provision.LifetimeVerdict }`: `GetRecordRaw` and `ReadHostIncarnation`; a record read that fails other than absence or corruption, or a slot read that fails other than absence (`ErrSlotMalformed` included) → `Verdict: LifetimeUnreadable` and the error; a corrupt record → `Verdict: LifetimeCorrupt` with its raw bytes and ETag; otherwise `LifetimeOf`. After any lifetime write under the exclusion the caller runs `s.slots.Forget(host)` (Task 6). |
| + - `func (s *Server) createRecordWithLifetime(ctx context.Context, rec provision.Record) (*provision.Record, error)`: the one function every claims-off creation path calls; returns the record as written (incarnation set, `RecordSeq` set). Errors: `release.ErrCASConflict` (a valid record under this key; the doors keep their 409), `*hostHeldError` (`errors.As`; carries `*apexHold` for `hostHeldSentence`; a live slot owned by another record key is `&apexHold{Holder: slot.App, OtherAccount: slot.Account if ≠ rec.Account}`), `errLifetimeSettling` (409, the settling sentence), `errLifetimePending` (the allocation is persisted and the outcome of a later step is unknown: recovery completes it; the door answers its existing "could not create" sentence and **does not** undo anything), or a read error from step 2 (502, nothing persisted). |
| - `func (s *Server) recoverLifetimes(ctx context.Context) error`, `func (s *Server) StartLifetimeRecoveryPass(ctx context.Context)` (returns at once unless `lifetimesOn()`; once at start, then every 60 s). |
| The protocol, inside `withHostname(host)`: |
| 1. `admit(host, "")`. |
| -2. `classifyLocked`. Unreadable → the read error (502 at the doors). Valid record under this key → `ErrCASConflict`; the slot is untouched. A live slot owned by another key → `errHostHeld`. Then `s.hostHolder(ctx, acct, host, app)` again, under the lock (the pre-lock call at `apiinstances.go:1164` stays): a holder → `errHostHeld`. A stale record under this key (a tombstone, or a lifetime behind the slot) is remembered: `StaleETag`. |
| -3. `N = max(LastIncarnation(host), slot.Incarnation) + 1`; `rec.Incarnation = N`; `body = json.Marshal(rec)`; `BeginAllocation({OpID, Account, App, Host, N, pending, body, PredecessorETag: slot's ETag or "", StaleETag})`. |
| -4. `WriteHostIncarnation({N, live, rec.Account, rec.App, "create " + OpID}, PredecessorETag)`. A CAS conflict here (the slot moved after step 2) → `superseded`, and the caller gets `errLifetimeSettling` (the next request re-classifies). |
| -5. If `StaleETag != ""`: `DeleteIfMatch` the record on it; a conflict means the key changed → leave `pending` and return `errLifetimePending` (recovery re-classifies). |
| -6. `WriteRecord(ctx, s, key, []byte("{}"), body, "")`. Success → 7. `ErrCASConflict` → re-read raw: the body at N under this key → 7 (a replay landed); otherwise `pending` and `errLifetimePending`. Any other error → `pending` and `errLifetimePending`. |
| -7. `SetAllocationState(committed)`; return the written record. |
| +2. `classifyLocked`. Unreadable → the read error. Valid record under this key → `ErrCASConflict`; the slot is untouched. A live slot owned by another key → `*hostHeldError`. Then `s.hostHolder(ctx, acct, host, app)` with `acct` from `s.store.AccountBySqid(rec.Account)` (a lookup failure is a read error), under the lock (the pre-lock call at `apiinstances.go:1164` stays): a holder → `*hostHeldError`. A stale or corrupt record under this key is remembered: `StaleETag`. |
| +3. `N = max(LastIncarnation(host), slot.Incarnation) + 1`; `rec.Incarnation = N`; `body = json.Marshal(rec)`; `BeginAllocation({OpID, Account, App, Host, N, pending, body, PredecessorETag: slot's ETag or "", StaleETag})`. A SQLite failure here → the error (nothing durable). |
| +4. `WriteHostIncarnation({N, live, rec.Account, rec.App, "create " + OpID}, PredecessorETag)`. On `ErrCASConflict`: re-read the slot; `{N, live, this key, "create " + OpID}` (this allocation's own write landed) → continue at 5; a slot at an incarnation > N, or at N owned by another key, or at N with another intent (positive successor evidence) → `superseded`, `errLifetimeSettling`; anything else (unreadable, or the predecessor again) → `pending`, `errLifetimePending`. Any other error → `pending`, `errLifetimePending`. |
| +5. If `StaleETag != ""`: `DeleteIfMatch` the record on it; any error (conflict included) → `pending`, `errLifetimePending`. |
| +6. `WriteRecord(ctx, s, key, []byte("{}"), body, "")`. Success → 7. `ErrCASConflict` → re-read raw: a record at N under this key → 7 (a replay landed); otherwise `pending`, `errLifetimePending`. Any other error → `pending`, `errLifetimePending`. |
| +7. `SetAllocationState(committed)`; a SQLite failure → `errLifetimePending` (the record stands; recovery marks it committed). Return the written record. |
| Recovery (`recoverLifetimes`), per pending allocation, inside `withHostname` with `admit(host, OpID)` (its own operation does not refuse itself), after `classifyLocked`, the first matching branch: |
| - unreadable → leave pending; |
| -- slot at an incarnation past N, or at N with an intent other than `"create " + OpID` → `superseded` (an operator or a later lifetime moved it); |
| +- slot at an incarnation > N, or at N owned by another key, or at N and this key with an intent that is neither `"create " + OpID` nor the predecessor's (positive successor evidence) → `superseded`; |
| - slot `{N, live, this key, "create " + OpID}` and a record at N under this key → `committed`; |
| -- slot as above and a record under the key that is not at N → step 5 with the current ETag, then 6; |
| -- slot as above and no record → step 6; |
| -- slot absent, or at `PredecessorETag` → step 4; |
| -- anything else → `superseded`. |
| +- slot as above and a stale or corrupt record under the key → step 5 with the current ETag, then 6, 7; |
| +- slot as above and no record → step 6, 7; |
| +- slot absent (and `PredecessorETag == ""`), or the slot's ETag equals `PredecessorETag` → step 4 onward; |
| +- anything else → leave pending and log once per hour (an operator reads the row). |
| Then the retirements (Task 4) and the migrations (Task 5), in that order. |
| -Door mappings: the create door (`apiinstances.go`): `errHostHeld` → 409 `hostHeldSentence`; `errLifetimeSettling` → 409 settling; `errLifetimePending` → 502 "could not create the instance — try again" (the existing sentence), no undo. Seed (`apiinstanceseed.go:341`) and import (`apiinstanceimport.go:172`): the same three mappings onto their existing failure sentences; on `errLifetimePending` the door writes nothing further, and the record recovery completes carries the seed state the body had, which the existing seed sweep then owns (a begun seed with no upload times out as today). Attach (`domainattach.go`): `ErrCASConflict` keeps today's adoption branch; `errLifetimePending` → the ambiguous branch, but `rollbackAttach(ctx, acct, app, host, false, false)`: the claim is **kept** (it is this app's; the record recovery completes is this app's first-route record under its own claim, the case the door already adopts on retry), and the answer is the existing "could not declare the route for this domain — nothing was attached, try again"; `errLifetimeSettling` and `errHostHeld` → 409 with their sentences and `rollbackAttach(..., false, created)` as today's refusal branch does. |
| +Door mappings: the create door (`apiinstances.go`): `*hostHeldError` → 409 `hostHeldSentence(host, hold)`; `errLifetimeSettling` → 409 settling; `errLifetimePending` → 502 "could not create the instance — try again" (the existing sentence), no undo. Seed (`apiinstanceseed.go:341`) and import (`apiinstanceimport.go:172`): the same three mappings onto their existing failure sentences; on `errLifetimePending` the door writes nothing further, and the record recovery completes carries the seed state the body had, which the existing seed sweep then owns (a begun seed with no upload times out as today). Attach (`domainattach.go`): `ErrCASConflict` keeps today's adoption branch; `errLifetimePending` → `rollbackAttach(ctx, acct, app, host, false, false)`: the claim is **kept** (it is this app's; the record recovery completes is this app's first-route record under its own claim, the case the door already adopts on retry), and the answer is the existing "could not declare the route for this domain — nothing was attached, try again"; `errLifetimeSettling` and `*hostHeldError` → 409 with their sentences and `rollbackAttach(..., false, created)` as today's refusal branch does; a step-2 read error → the existing ambiguous branch (nothing was persisted, so `rollbackAttach(..., true, created)` is today's behaviour and its retirement finds nothing). |
| - [ ] **Step 1: Write the failing tests** against an in-process console with `release.NewDirStore` and `HostClaim` off: |
| - - `TestCreateAllocatesSlotThenRecord` (slot `{1, live, bab, shop, "create <id>"}`, record `Incarnation 1`, the returned record has it), `TestCreateAfterRetirementGetsNextIncarnation` (`{1, retired}` → `{2, live}`), `TestCreateReplacesStaleRecord` (a fieldless record beside `{1, retired}` is deleted on its ETag; incarnation 2 written), `TestCreateOfALiveInstanceIsTheExistingConflict` (slot unchanged, no allocation row), `TestCreateRefusesWhileAnOperationIsPending`, `TestCreateRefusesALiveSlotOfAnotherApp` (slot `{1, live, bab, other}` → `errHostHeld`, slot unchanged), `TestTwoAppsCreatingOneHostnameSerialise` (two goroutines, apps `shop` and `other`, one host: exactly one slot write, one record, the loser gets `errHostHeld` or `ErrCASConflict`, never a second lifetime). |
| + - `TestCreateAllocatesSlotThenRecord` (slot `{1, live, bab, shop, "create <id>"}`, record `Incarnation 1`, the returned record has it), `TestCreateAfterRetirementGetsNextIncarnation` (`{1, retired}` → `{2, live}`), `TestCreateReplacesStaleRecord` (a fieldless record beside `{1, retired}` is deleted on its ETag; incarnation 2 written), `TestCreateOfALiveInstanceIsTheExistingConflict` (slot unchanged, no allocation row), `TestCreateRefusesWhileAnOperationIsPending`, `TestCreateRefusesALiveSlotOfAnotherApp` (slot `{1, live, bab, other}` → `hostHeldSentence` names `other`, slot unchanged), `TestTwoAppsCreatingOneHostnameSerialise` (two goroutines, apps `shop` and `other`, one host: exactly one slot write, one record, the loser gets the held or the existing conflict, never a second lifetime). |
| + - `TestEveryPostAllocationFailureIsPending` (a failing store wrapper injects a non-CAS error at step 4, at step 5, at step 6, and a failing SQLite at step 7: each leaves the row `pending` and returns `errLifetimePending`; recovery then completes each). |
| - `TestRecoveryReplaysCreateAndTheLateWriteIsHarmless`: a pending allocation with the slot written and no record; recovery writes the record and commits; an injected late create-only write of the same body fails `ErrCASConflict` and the record's ETag is unchanged. |
| - - `TestDeleteDuringCreateRecoveryIsRefused` (a DELETE while the allocation is pending → 409 settling, nothing written). |
| - - `TestRecoveryWritesTheSlotWhenMissing`, `TestRecoveryContinuesFromAPredecessorSlot`, `TestRecoveryLeavesALandedCreateServing`, `TestRecoveryRemovesAStaleRecordThenWrites`, `TestRecoverySupersedesWhenSlotMovedPast`, `TestRecoveryLeavesPendingWhenUnreadable`, `TestRecoverySurvivesASecondCrash` (a crash injected after step 4 on the first recovery run), `TestRecoveryUsesOnlyPersistedState` (the SQLite store is closed and reopened between the crash and the recovery). |
| - - `TestAttachKeepsItsClaimOverAPendingAllocation` (failure injected after the slot write in the attach's first-route create: the claim stands, the answer is the ambiguous sentence, recovery completes the record, a second attach adopts it). |
| - - `TestSeedCreatePendingIsOwnedByTheSweep` (the same injection at the seed door: recovery completes the record with its seed state; the seed sweep's existing timeout path then rolls it back). |
| + - `TestLateSlotWriteDoesNotAbandonTheAllocation`: recovery reads the predecessor; the original slot write lands `{N, live, create A}` between the read and recovery's CAS; recovery's write loses, re-reads, recognises its own slot, writes the record, commits. The mirror: the original slot write lands after recovery's succeeded → the original fails CAS and nothing else happens. |
| + - `TestDeleteDuringCreateRecoveryIsRefused` (through the real DELETE door while the record is absent and the allocation pending → 409 settling, nothing written; Task 4's absent path). |
| + - `TestRecoveryWritesTheSlotWhenMissing`, `TestRecoveryContinuesFromAPredecessorSlot`, `TestRecoveryLeavesALandedCreateServing`, `TestRecoveryRemovesAStaleRecordThenWrites`, `TestRecoverySupersedesOnlyOnSuccessorEvidence` (slot past N → superseded; slot at N of another key → superseded; the predecessor still → replay, never superseded), `TestRecoveryLeavesPendingWhenUnreadable` (a failing slot read; a malformed slot body), `TestRecoverySurvivesASecondCrash` (a crash injected after step 4 on the first recovery run), `TestRecoveryUsesOnlyPersistedState` (the SQLite store is closed and reopened between the crash and the recovery). |
| + - `TestAttachKeepsItsClaimOverAPendingAllocation` (failure injected after the slot write, and separately a SQLite failure at step 7, in the attach's first-route create: the claim stands, the answer is the ambiguous sentence, recovery completes the record, a second attach adopts it). |
| + - `TestSeedCreatePendingIsOwnedByTheSweep`. |
| - [ ] **Step 2: Run to verify they fail.** |
| - [ ] **Step 3: Implement**; route the four claims-off creation paths through `createRecordWithLifetime`. |
| - [ ] **Step 4: Run** `go test ./internal/console -run 'Create|Lifetime|Recovery|Attach|Import|Seed' -count=1`. |
| -- [ ] **Step 5: Commit** `console: claims-off creates allocate an owned host incarnation through a persisted intent, with recovery`. |
| +- [ ] **Step 5: Commit** `console: claims-off creates allocate an owned host incarnation through a persisted intent, with recovery that never abandons it`. |
| ### Task 4: Retirement through a tombstone, from every claims-off removal |
| **Files:** |
| - Create: `internal/console/lifetimes_retire.go` |
| -- Modify: `internal/console/hostclaimops.go` (`retireHostRecord`, claims-off branch: `ref.Family == "instance"` → `retireRecordWithLifetime(ctx, ref.Account, ref.App, ref.Host, "")`; sidecars and everything else unchanged on `provision.RetireRecord`), `internal/console/apiinstanceimport.go` (`deleteRead`: `release.DeleteIfMatch(...)` becomes `retireRecordWithLifetime(ctx, acct.Sqid, app, host, etag)`; `ErrCASConflict` keeps the `again` re-decision; `errLifetimeSettling` → 409 settling; `errRetireRefused` → 409 its sentence), `internal/console/seedfinish.go` (`finishSeedOnce`'s final `DeleteIfMatch` becomes `retireRecordWithLifetime(ctx, sqid, app, host, etag)` with the ETag `deleteSeedReplica` returned; the seed resource teardown before it is unchanged; `ErrCASConflict` keeps `finishSeed`'s one retry), `internal/console/apiinstances.go` (`handleAPIInstanceDelete`: map `errRetireRefused` and `errLifetimeSettling` to 409 on the `retireInstanceRecord` branch), `internal/console/lifetimes_recovery.go` (the retirement branches) |
| +- Modify: `internal/console/hostclaimops.go` (`retireHostRecord`, claims-off branch: `ref.Family == "instance"` → `retireRecordWithLifetime(ctx, ref.Account, ref.App, ref.Host, "")`; sidecars and everything else unchanged on `provision.RetireRecord`), `internal/console/apiinstanceimport.go` (`readDeletedRecord` returns `(obs lifetimeClass, err error)` through `s.classifyObservation` (Task 6: judged, with the raw ETag and the corrupt case kept); `importDeleteRule`: the claims-off `os.ErrNotExist` branch becomes `retireRecordWithLifetime(ctx, acct.Sqid, app, host, retireExpectAbsent)` and answers finished on nil, 409 on settling; a `LifetimeStale` or `LifetimeCorrupt` observation → `deleteRead` with its ETag (the import logic needs a decoded, valid record and is skipped for these); `deleteRead`: `release.DeleteIfMatch(...)` becomes `retireRecordWithLifetime(ctx, acct.Sqid, app, host, etag)`; `ErrCASConflict` keeps the `again` re-decision; `errLifetimeSettling` → 409 settling; `errRetireRefused` → 409 its sentence), `internal/console/seedfinish.go` (`finishSeedOnce`'s final `DeleteIfMatch` becomes `retireRecordWithLifetime(ctx, sqid, app, host, etag)` with the ETag `deleteSeedReplica` returned; the seed resource teardown before it is unchanged; `ErrCASConflict` keeps `finishSeed`'s one retry), `internal/console/apiinstances.go` (`handleAPIInstanceDelete`: `errRetireRefused` and `errLifetimeSettling` → 409 on the `retireInstanceRecord` branch), `internal/console/domainattach.go` and `internal/console/apidomains.go` (the detach callers: `errLifetimeSettling` → 409 `claimDetachInProgressMsg`; `errRetireRefused` → 409 its sentence; `rollbackAttach` logs either and keeps the claim), `internal/console/lifetimes_recovery.go` (the retirement branches) |
| - Test: `internal/console/lifetimes_retire_test.go` |
| **Interfaces:** |
| - Consumes: Tasks 1 to 3; Task 5's `migrateHostLifetimeLocked` (build Task 5's function first or stub it behind the same signature). |
| -- Produces: `func (s *Server) retireRecordWithLifetime(ctx context.Context, acct, app, host, expectETag string) error`; `var errRetireRefused`. A nil return means the key is absent and the slot, if any, retired or someone else's: the callers' "deleted" and "already gone" answers. |
| +- Produces: `const retireExpectAbsent = "\x00absent"`; `func (s *Server) retireRecordWithLifetime(ctx context.Context, acct, app, host, expectETag string) error`; `var errRetireRefused`. A nil return means the key is absent (or was left untouched on purpose, below) and the slot, if any, is retired or someone else's: the callers' "deleted" and "already gone" answers. |
| -Inside `withHostname(host)`: `admit(host, "")`; `classifyLocked`; when `expectETag != ""` and the record's ETag differs → `release.ErrCASConflict` (the caller re-decides, as `deleteRead` does today). Then the first matching case: |
| +Inside `withHostname(host)`: `admit(host, "")`; `classifyLocked` (unreadable → the error, 502 at the doors). Preconditions: `expectETag == retireExpectAbsent` and a record is present → nil, nothing touched (the import rule's protection: a record created since the rule observed absence is not deleted); `expectETag` a real ETag and the record absent or at another ETag → `release.ErrCASConflict` (the caller re-decides, as `deleteRead` does today). Then the first matching case: |
| - (a) record absent, slot absent → nil. |
| -- (b) record absent, slot `{N, live}` owned by this key → an **orphan slot**: `BeginRetirement({..., N, RecordETag: "", SlotETag})`, then step (2) below, `committed`. (A crashed create would be a pending allocation and refused at `admit`; what remains is an old console's plain delete, and retiring the slot makes its late record stale.) |
| +- (b) record absent, slot `{N, live}` owned by this key → an **orphan slot**: `BeginRetirement({..., N, RecordETag: "", SlotETag: S, SlotIntent: slot.Intent})`, then step (2) below, `committed`. (A crashed create is a pending allocation and was refused at `admit`; what remains is an old console's plain delete, and retiring the slot makes its late record stale.) |
| - (c) record absent, slot owned by another key, or retired → nil. |
| -- (d) record present, verdict stale: a tombstone or a fieldless/behind record → `DeleteIfMatch` on its ETag (a conflict → `errLifetimeSettling`); the slot is untouched; nil. |
| -- (e) record present, fieldless, slot absent (legacy) → `migrateHostLifetimeLocked` (Task 5); if it does not reach `committed` → `errLifetimeSettling`; then re-classify and continue at (f). |
| -- (f) record valid at N under slot `{N, live, this key}` at ETag S → `BeginRetirement({OpID, Account, App, Host, N, E, S, pending})`, then: |
| - - (1) `WriteRecord(ctx, s, key, raw, TombstoneBody(raw), E)`. `ErrCASConflict` → re-classify: a non-tombstone record at N → `refused` (`errRetireRefused`); a tombstone at N → continue at (2); anything else → `void`, nil. |
| - - (2) `WriteHostIncarnation({N, retired, Account, App, "retire " + OpID}, S)`. `ErrCASConflict` → re-read the slot: `{N, retired}` → continue; otherwise `void` (a successor exists; the tombstone is left for (d) of a later call or for recovery). |
| - - (3) `DeleteIfMatch` the tombstone on its ETag (absence is success). |
| - - `committed`. |
| - |
| -Recovery, per pending retirement, inside `withHostname` with `admit(host, OpID)`, after `classifyLocked`, the first matching branch: unreadable → pending; slot past N, or at N and live with another intent and a non-tombstone record → `void`; record at `RecordETag` (unchanged) and slot at `SlotETag` → (1); tombstone at N and slot `{N, live}` at `SlotETag` → (2); tombstone at N and slot `{N, retired}` → (3); record absent and slot `{N, retired}` → `committed`; `RecordETag == ""` (an orphan slot retirement) and slot `{N, live}` at `SlotETag` → (2); slot `{N, live}` at `SlotETag` and a non-tombstone record at N whose ETag is not `RecordETag` → `refused`; anything else → `void`. |
| +- (d) record present, verdict stale (a tombstone, a fieldless or behind record) → `DeleteIfMatch` on its ETag (a conflict → `errLifetimeSettling`); the slot is untouched; nil. |
| +- (g) record present, verdict corrupt (today's "deleted the ordinary way" case) → `DeleteIfMatch` on its ETag (conflict → `errLifetimeSettling`); the slot is untouched; nil. |
| +- (e) record present, fieldless, slot absent (legacy) → `rec, raw, E, state, err := s.migrateHostLifetimeLocked(ctx, acct, app, host, RecordETag)` (Task 5; the marker CAS is bound to the caller's observed ETag): `ErrCASConflict` → `ErrCASConflict` to the caller (an ordinary change won; the caller re-decides); `state != committed` → `errLifetimeSettling`; committed → continue at (f) with the returned `rec, raw, E` and a fresh slot read (its ETag S). |
| +- (f) record valid at N under slot `{N, live, this key}` at ETag S → `BeginRetirement({OpID, Account, App, Host, N, E, S, SlotIntent: slot.Intent, pending})`, then: |
| + - (1) `WriteRecord(ctx, s, key, raw, TombstoneBody(raw), E)`. `ErrCASConflict` → re-classify: a non-tombstone record at N → `refused` (`errRetireRefused`); a tombstone at N → continue at (2); anything else → `void`, nil. Any other error → `pending`, `errLifetimePending`. |
| + - (2) `WriteHostIncarnation({N, retired, Account, App, "retire " + OpID}, S)`. `ErrCASConflict` → re-read the slot: `{N, retired}` → continue; a slot at > N or of another key → `void` (the tombstone is left for (d) of a later call or for recovery); anything else → `pending`, `errLifetimePending`. Any other error → `pending`, `errLifetimePending`. |
| + - (3) `DeleteIfMatch` the tombstone on its ETag (absence is success; any other error → `pending`, `errLifetimePending`). |
| + - `committed` (a SQLite failure → `errLifetimePending`). |
| + |
| +Recovery, per pending retirement, inside `withHostname` with `admit(host, OpID)`, after `classifyLocked`, the first matching branch, where `succ := slot != nil && (slot.Incarnation > N || !owned)` and `pred := slot != nil && slot.Incarnation == N && slot.State == live && owned && slot.Intent == SlotIntent` and `retired := slot != nil && slot.Incarnation == N && slot.State == retired && owned`: |
| +- unreadable → pending; |
| +- `succ` → `void`; |
| +- `retired` and record absent → `committed`; |
| +- `retired` and a tombstone at N → (3), `committed`; |
| +- `retired` and a non-tombstone record at N (a late write landed after the tombstone) → `committed`, and the record is deleted conditionally on its ETag (stale under a retired slot; a conflict leaves it for the next create); |
| +- `pred`, `RecordETag != ""`, a non-tombstone record present at `RecordETag` → (1), (2), (3); |
| +- `pred`, a tombstone at N → (2), (3); |
| +- `pred`, `RecordETag == ""`, record absent (an orphan slot retirement) → (2), `committed`; |
| +- `pred`, `RecordETag != ""`, a non-tombstone record at N with another ETag → `refused`; |
| +- `pred`, record absent, `RecordETag != ""` (someone else removed it) → (2), `committed`; |
| +- anything else → pending, logged once per hour. |
| - [ ] **Step 1: Write the failing tests** through the real doors (`DELETE /api/cli/.../instances/{host}` and the seed sweep) and the function: |
| - `TestDeleteDoorRetiresThroughTheTombstone` (the import rule's ordinary path: after a hook at (1) the key holds a tombstone, `GetRecord` says not found, `record_seq` advanced by one; at the end the slot is `{1, retired}` and the key absent; the door answers "deleted"); `TestSlotRetiresBeforeTheKeyIsAbsent` (a hook between (2) and (3): the slot is already retired while the tombstone stands, and a create-only write of a record at incarnation 1 is stale under it). |
| + - `TestDeleteDoorAbsentPathAdmitsAndRetiresOrphans` (record absent, slot `{1, live}`, no pending op → the door answers "deleted" and the slot is `{1, retired}`; a pending allocation → 409 settling; a record created between the rule's read and the retirement → untouched, "deleted" as today). |
| + - `TestDeleteDoorRemovesStaleAndCorruptRecordsConditionally` (a stale fieldless record beside a retired slot → removed, slot untouched; a body that does not decode → removed on its ETag as today). |
| - `TestSeedFinishRetiresThroughTheTombstone`. |
| - - `TestRetireRefusesWhenTheRecordChangedInItsLifetime` (a `SetRecordRegion` between the read and (1) → 409 with the sentence; record kept; state refused); `TestRetireVoidsAgainstASuccessor` (a stale record at 1 beside a live slot `{2}`: case (d), the slot stays `{2, live}`; and a retirement begun at `{1, live}` whose slot moves to `{2}` before (2) → void, slot untouched); `TestRetireNeverMovesTheSlotBackwards` (every case of a record at 1 beside slots `{2, live}` and `{2, retired}`: the slot's ETag is unchanged afterwards). |
| - - `TestOrphanSlotIsRetired` (record absent, slot `{1, live}`, no pending allocation → slot `{1, retired}`; a late fieldless create-only write then lands stale). |
| - - `TestLegacyRecordIsMigratedThenRetired` (no plain delete; the slot ends `{1, retired}`). |
| + - `TestRetireRefusesWhenTheRecordChangedInItsLifetime` (a `SetRecordRegion` between the read and (1) → 409 with the sentence; record kept; state refused); `TestRetireVoidsAgainstASuccessor`; `TestRetireNeverMovesTheSlotBackwards` (every case of a record at 1 beside slots `{2, live}` and `{2, retired}`: the slot's ETag is unchanged afterwards). |
| + - `TestOrphanSlotIsRetired`. |
| + - `TestLegacyRetirementIsBoundToTheCallersRead` (an ordinary update lands between `deleteRead`'s read and the marker CAS → the door answers the `again` path's re-decision, the updated record survives; with no interference the migration completes and the tombstone is written against the migrated record's ETag; no plain delete). |
| - `TestDelayedDeleteCannotEraseASuccessor`: block the conditional tombstone write (1) of a retirement with a hook; let recovery run after a restart (it finds the record at `RecordETag` and completes (1)–(3)); create the host again (`{2, live}`, record at 2); release the blocked write: it fails `ErrCASConflict`; the successor's record and slot are intact. |
| - - `TestRetirementRecoveryBranches` (one subtest per branch above; a second crash injected during recovery in two of them). |
| + - `TestRetirementRecoveryBranches` (one subtest per branch above with realistic predecessor intents `"create <x>"` and `"migration <x>"`, including the restored-record branch and its conditional cleanup; a second crash injected during recovery in two of them). |
| + - `TestDetachMapsLifetimeErrors`. |
| - [ ] **Step 2: Run to verify they fail.** |
| - [ ] **Step 3: Implement.** |
| -- [ ] **Step 4: Run** `go test ./internal/console -run 'Retire|Tombstone|Lifetime|InstanceDelete|SeedFinish|DeleteDoor' -count=1` and `go test ./internal/provision -run 'Retire' -count=1`. |
| -- [ ] **Step 5: Commit** `console: every claims-off record removal retires through a tombstone, classifying the record and its slot together`. |
| +- [ ] **Step 4: Run** `go test ./internal/console -run 'Retire|Tombstone|Lifetime|InstanceDelete|SeedFinish|DeleteDoor|Detach' -count=1` and `go test ./internal/provision -run 'Retire' -count=1`. |
| +- [ ] **Step 5: Commit** `console: every claims-off record removal retires through a tombstone, bound to the caller's read and classifying the record and its slot together`. |
| ### Task 5: Lazy migration with a marker fenced in the write primitive |
| @@ -190,89 +207,109 @@ Recovery, per pending retirement, inside `withHostname` with `admit(host, OpID)` |
| - Test: `internal/provision/record_sequence_test.go`, `internal/console/lifetimes_migrate_test.go` |
| **Interfaces:** |
| -- Produces: `var ErrRecordMigrating = fmt.Errorf("%w: the record is being migrated; read it again", release.ErrCASConflict)`; `func WithMigration(ctx context.Context, id string) context.Context`; `func (s *Server) migrateHostLifetimeLocked(ctx context.Context, acct, app, host string) (state string, err error)` (assumes `withHostname`; returns the final row state); `func (s *Server) StartIncarnationMigrationPass(ctx context.Context)` (behind `CARLOS_LIFETIMES_MIGRATE=1` and `lifetimesOn()`, every 5 minutes, at most 50 records per pass, oldest `created_at` first, each inside `withHostname` and `admit`, only fieldless unmarked records whose slot is absent). |
| +- Produces: `var ErrRecordMigrating = fmt.Errorf("%w: the record is being migrated; read it again", release.ErrCASConflict)`; `func WithMigration(ctx context.Context, id string) context.Context`; `func (s *Server) migrateHostLifetimeLocked(ctx context.Context, acct, app, host, expectETag string) (rec *provision.Record, raw []byte, etag string, state string, err error)` (assumes `withHostname`; `expectETag == ""` means "from the current record, retrying from an updated one"; a real ETag binds the marker CAS to it and a conflict returns `release.ErrCASConflict` with no retry; on `committed` the returned record, bytes and ETag are the migrated record as written by step (3)); `func (s *Server) StartIncarnationMigrationPass(ctx context.Context)` (behind `CARLOS_LIFETIMES_MIGRATE=1` and `lifetimesOn()`, every 5 minutes, at most 50 records per pass, oldest `created_at` first, each inside `withHostname` and `admit`, only fieldless unmarked records whose slot is absent, with `expectETag == ""`). |
| -The protocol (inside the exclusion): `admit(host, "")`; `BeginMigration({OpID, Account, App, Host, "", pending})`; (1) `classifyLocked`; the record must be fieldless, unmarked, slot absent (else `superseded`); `WriteRecord(WithMigration(ctx, OpID), s, key, raw, raw with IncarnationMigrating = OpID, E)`; on `ErrCASConflict` re-read and retry from the current record, at most 5 times, then leave pending; `SetMigrationMarkedETag(OpID, M)` with the returned ETag; (2) `WriteHostIncarnation({1, live, Account, App, "migration " + OpID, RecordETag: M}, "")` create-only, or `If-Match` when a slot `{1, live, this key, "migration " + OpID}` already exists (an earlier attempt's); (3) `WriteRecord(WithMigration(ctx, OpID), ..., Incarnation = 1, IncarnationMigrating = "", against M)`; `committed`. The slot's `RecordETag` stays M afterwards; the verdict of the finished record does not read it. |
| +The protocol (inside the exclusion): `admit(host, "")`; `BeginMigration({OpID, Account, App, Host, ExpectETag, "", pending})`; (1) `classifyLocked`; the record must be fieldless, unmarked, slot absent (else `superseded`, and for a bound call `ErrCASConflict` when the record's ETag differs from `ExpectETag`); `WriteRecord(WithMigration(ctx, OpID), s, key, raw, raw with IncarnationMigrating = OpID, E)`; on `ErrCASConflict`: bound → `superseded`, `ErrCASConflict`; unbound → re-read and retry from the current record, at most 5 times, then leave pending; `SetMigrationMarkedETag(OpID, M)` with the returned ETag (a SQLite failure → pending, `errLifetimePending`); (2) `WriteHostIncarnation({1, live, Account, App, "migration " + OpID, RecordETag: M}, "")` create-only, or `If-Match` when a slot `{1, live, this key, "migration " + OpID}` already exists (an earlier attempt's); a CAS conflict → re-read: this migration's slot → continue; otherwise `superseded`; (3) `WriteRecord(WithMigration(ctx, OpID), ..., Incarnation = 1, IncarnationMigrating = "", against M)` (a conflict → re-read: a record at `Incarnation 1` under this slot → continue; otherwise pending); `committed`; `s.slots.Forget(host)`. Every other error after `BeginMigration` → pending, `errLifetimePending`. |
| -Recovery, per pending migration, inside the exclusion with `admit(host, OpID)`: unreadable → pending; record carries `Incarnation 1` and slot `{1, live, this key, "migration " + OpID}` → `committed`; record marked with OpID, slot bound to the record's current ETag → (3); record marked with OpID and `MarkedETag == ""` → `SetMigrationMarkedETag(OpID, current ETag)` then (2); record marked with OpID, slot absent → (2); record fieldless and unmarked, slot absent → (1); record gone, another lifetime, a marker of another op, or a slot not of this migration → `superseded`. |
| +Recovery, per pending migration, inside the exclusion with `admit(host, OpID)`: unreadable → pending; record carries `Incarnation 1` and slot `{1, live, this key, "migration " + OpID}` → `committed`; record marked with OpID, slot bound to the record's current ETag → (3); record marked with OpID and `MarkedETag == ""` → `SetMigrationMarkedETag(OpID, current ETag)` then (2); record marked with OpID, slot absent → (2); record fieldless and unmarked, slot absent → (1) (unbound: a retirement's bound migration that did not reach the marker is simply retried by the caller, so its row is `superseded` here); record gone, another lifetime, a marker of another op, or a slot not of this migration → `superseded`. |
| - [ ] **Step 1: Write the failing tests**: |
| - `record_sequence_test.go`: `TestWriteRecordRefusesAMarkedRecordWithoutTheMigrationContext` (the error `Is` both `ErrRecordMigrating` and `release.ErrCASConflict`), `TestWriteRecordAllowsTheMigrationsOwnWrite`, `TestCreateOnlyWriteIgnoresTheMarker`, `TestClaimRawUpdateRecordSurfacesTheMarker`. |
| - - `TestMigrationMarksThenSlotsThenSets` (final: slot `{1, live, bab, shop, "migration <id>", RecordETag M}`, record `Incarnation 1`, no marker, committed). |
| - - `TestMigrationRestartsFromAnUpdateThatWon` (an update between the read and (1); the marker CAS fails; the pass re-reads and completes; the update's field survives). |
| + - `TestMigrationMarksThenSlotsThenSets` (final: slot `{1, live, bab, shop, "migration <id>", RecordETag M}`, record `Incarnation 1`, no marker, committed; the returned record and ETag equal a fresh read). |
| + - `TestMigrationRestartsFromAnUpdateThatWon` (unbound: an update between the read and (1); the marker CAS fails; the pass re-reads and completes; the update's field survives) and `TestBoundMigrationReturnsConflictToItsCaller` (bound: the same interference → `ErrCASConflict`, row `superseded`, record untouched). |
| - `TestAnIssuedWriteLosesToTheMarkerAndRetries` (a box-shaped `SetRecordMove` goroutine blocked on a hook with the pre-marker ETag: its `WriteRecord` fails `ErrCASConflict`, its re-read gets `ErrRecordMigrating`, it waits, and succeeds after (3)). |
| - - `TestWriterGivesUpAfterTheBound` (the marker never clears → `ErrRecordMigrating` after the shortened bound). |
| - - `TestMigrationRecoveryBranches` (one subtest per branch, including a restart after the marker write and before `MarkedETag` is persisted). |
| + - `TestWriterGivesUpAfterTheBound`. |
| + - `TestMigrationRecoveryBranches` (one subtest per branch, a restart after the marker write and before `MarkedETag` is persisted, and a second crash during recovery). |
| - `TestMigrationPassIsOffWithoutTheFlag`, `TestMigrationSkipsAFieldlessRecordBesideAnUnrelatedSlot`. |
| - [ ] **Step 2: Run to verify they fail.** |
| - [ ] **Step 3: Implement.** |
| - [ ] **Step 4: Run** `go test ./internal/provision -run 'WriteRecord|Migrat|Record|ClaimRawUpdate' -count=1` and `go test ./internal/console -run 'Migrat' -count=1`. |
| -- [ ] **Step 5: Commit** `lifetimes: lazy migration marks the record by CAS, fenced in the write primitive`. |
| +- [ ] **Step 5: Commit** `lifetimes: lazy migration marks the record by CAS, fenced in the write primitive, bound to a caller's read when it asks`. |
| ### Task 6: Consumers judge the lifetime |
| **Files:** |
| - Create: `internal/provision/lifetime_cache.go`, `internal/console/lifetimes_read.go` |
| -- Modify: `internal/provision/reconcile.go` (the decode loop at lines 855–941: `getEach` and `fetched` gain `etag`; after the path check and before `ValidHostShape`, the lifetime judgement; `ListAllRecordsStrict` gains a judged sibling and `readAllRecords` takes the cache), `internal/provision/provision.go` (`GetRecordETag`'s judged sibling; `ListRecordsStrict`, `ListRecordsComplete`, `ListRecords` gain judged siblings), `internal/host/agent.go` (`provisionTickConfig`: `Slots: provision.NewSlotCache()` when `!hostwrite.Enabled(getenv)`, nil otherwise), `internal/host/movespass.go` (line 1079) and `internal/host/movesoffline.go` (`refreshOfflineView`): the judged listing with the box's cache, `internal/console/*.go` (every `provision.GetRecord`, `GetRecordETag`, `ListRecords*` and `ListAllRecords*` call on instance records in the files the inventory below names goes through the `lifetimes_read.go` helpers) |
| +- Modify: `internal/provision/reconcile.go` (the decode loop at lines 855–941: `getEach` and `fetched` gain `etag`; after the path check and before `ValidHostShape`, the judgement, which may replace `rec`/`raw` with the observation it returns; `cfg.Slots.Begin()` at the top of the pass; `ListAllRecordsStrict` and `ListAllRecordsTolerant` gain judged siblings and `readAllRecords` takes the cache), `internal/provision/provision.go` (`GetRecordETag`'s judged sibling; `ListRecordsStrict`, `ListRecordsComplete`, `ListRecords` gain judged siblings), `internal/host/agent.go` (one `boxSlots *provision.SlotCache`, built once beside `provisionTickConfig` when `!hostwrite.Enabled(getenv)` and nil otherwise, handed to `ReconcileConfig.Slots`, `moveCfg.Slots` and the offline view), `internal/host/movespass.go` (line 1079: the judged strict listing), `internal/host/movesoffline.go` (`refreshOfflineView`: the judged listing; `offlineReassert` line 420: `ReadRecordJudged`), `internal/host/movesoffline_stage.go` (line 75 and the deletion gate at line 162: `ReadRecordJudged`), `internal/provision/reconcile.go` (`holdExisting` line 1716: `ReadRecordJudged` with `cfg.Slots`, and the pair for Task 7 from that fresh record), `internal/console/*.go` (every instance-record read in the inventory below goes through `lifetimes_read.go`) |
| - Test: `internal/provision/reconcile_lifetime_test.go`, `internal/provision/lifetime_cache_test.go`, `internal/console/lifetimes_read_test.go`, `internal/host/lifetime_gating_test.go` |
| **Interfaces:** |
| - Produces: |
| - - `type SlotCache struct{...}`; `func NewSlotCache() *SlotCache`; `func (c *SlotCache) Judge(ctx context.Context, s release.Store, rec *Record, recordETag string) (LifetimeVerdict, error)`: reads the slot (one `Get`; cached per host with the slot's ETag for 5 s; absence cached too; safe for concurrent use; a failed read → `LifetimeUnreadable, err`); then `LifetimeOf`. On a `stale` verdict where the slot's `Intent` begins `"migration "`, or the record is fieldless, it re-reads the record once (`GetRecordRaw`) and judges the fresh pair; still stale → stale. A nil `*SlotCache` judges everything valid (claims-on). |
| - - `func ReadRecordJudged(ctx context.Context, s release.Store, c *SlotCache, acct, app, host string) (*Record, string, error)`: `GetRecordRaw` then `Judge`; a tombstone or stale → `os.ErrNotExist`; unreadable → the slot read error. |
| - - `func ListRecordsStrictJudged(ctx context.Context, s release.Store, c *SlotCache, acct, app string, max int) (recs []Record, unreadable map[string]bool, err error)`, `ListRecordsCompleteJudged(ctx, s, c, acct, app, max) (recs []Record, truncated bool, skipped []string, err error)`, `ListRecordsJudged(ctx, s, c, acct, app, max) (recs []Record, truncated bool, err error)`, `ListAllRecordsStrictJudged(ctx context.Context, s release.Store, c *SlotCache) (recs []Record, unreadable map[string]bool, err error)`: each is its sibling with stale records dropped and an unreadable slot reported as that host unreadable (strict: the listing error, as a failed record read is today; complete: `skipped` names the host; the tolerant one: `complete = false`). |
| - - `ReconcileConfig.Slots *SlotCache`. In the decode loop: verdict stale → `continue` (not in `recs`, not protected: the removal phase retires the row as "record gone"); unreadable → `protect[host] = true` and `readFailed = true`, as a failed read is today. |
| - - Console: `func (s *Server) readRecord(ctx, acct, app, host) (*Record, string, error)` and `listRecords`, `listRecordsStrict`, `listRecordsComplete`, `listAllRecordsStrict` wrappers passing `s.slots` (`*provision.SlotCache`, nil when `s.cfg.HostClaim`). Inventory to route (every instance-record read in these files, by `grep -n 'provision.GetRecord\|provision.GetRecordETag\|provision.ListRecords\|provision.ListAllRecords'`): `apiinstancemove.go`, `apiinstances.go`, `seedfinish.go`, `channelmove.go`, `apiruns.go`, `apiinstancemove_offline.go`, `apiinstanceimport.go` (`readImportRecord`, `readDeletedRecord`), `updatescore.go`, `movesrecovery.go`, `domainattach.go`, `appsettings.go`, `apirunsfollow.go`, `apiretry.go`, `apidb.go`, `apex.go` (`hostHolder`), `apidomains.go`, `apiinstanceseed.go`, `apilegacyroutes.go`, `relayops.go`, `seedsweep.go`, `server.go`, `regiondrift.go`, `runsweep.go`, `seedmove.go`. A test pins the inventory: `TestNoConsoleReadBypassesTheJudgedReaders` walks `internal/console` (non-test) and fails on a direct `provision.GetRecord`, `GetRecordETag`, `ListRecords*` or `ListAllRecords*` call outside `lifetimes_read.go`, `lifetimes*.go` and the claims-on files it lists by name. |
| - |
| -- [ ] **Step 1: Write the failing tests**: `TestSlotCacheJudgesAndCaches` (two judgements, one slot read; absence cached; a changed slot seen after the window); `TestSlotCacheRereadsAcrossAMigration` (record read before the marker, slot read after it → valid, one re-read); `TestNilSlotCacheJudgesEverythingValid`; `TestReconcileRemovesARecordBehindItsSlot` (record at 1, slot `{2, live}`: no mint; an existing row retired as "record gone"); `TestReconcileRemovesATombstone` (the tombstone has no channel: it leaves before the channel check, so the row is retired, not protected); `TestReconcileKeepsALegacyRecordWithNoSlot`; `TestReconcileKeepsAMarkedRecordUnderItsMigrationSlot`; `TestReconcileProtectsWhenTheSlotIsUnreadable`; `TestReconcileRereadsAcrossAMigration` (hooks: the record body read before the marker, the slot after; the row survives); `TestEveryClaimsOffConsumerIgnoresAFieldlessRecordBesideASlot` (slot `{1, retired}`, fieldless record: `ReadRecordJudged` not found; the four judged listings omit it; the console GET answers 404; reconcile mints nothing; the offline keep view after `refreshOfflineView` does not keep a move it carries); `TestClaimsOnConsumersReadNoSlot` (`HostClaim` on: a counting store sees no `control/host-incarnations/` read from the doors, reconcile or the box listings); `TestProvisionTickConfigGatesTheSlotCache` (`CARLOS_HOST_CLAIM` set → `Slots == nil`; unset → non-nil); `TestNoConsoleReadBypassesTheJudgedReaders`. |
| + - `type Observation struct { Rec *Record; Raw []byte; ETag string; Slot *HostIncarnation; SlotETag string; Verdict LifetimeVerdict }`. |
| + - `type SlotCache struct{...}`; `func NewSlotCache() *SlotCache`; `func (c *SlotCache) Begin()` (drops every entry: a reconcile pass calls it first, so one pass reuses slot reads and no pass reuses another's); `func (c *SlotCache) Forget(host string)` (the console calls it after every lifetime write under the exclusion); `func (c *SlotCache) Judge(ctx context.Context, s release.Store, rec *Record, raw []byte, etag string) (Observation, error)`: round 1 judges `(rec, etag)` against the cached slot (an entry younger than 5 s; a miss reads the slot, caching its value or its absence); `valid` → return. Otherwise round 2: a fresh slot read bypassing the cache and a fresh `GetRecordRaw`; judge the fresh pair; `valid` → return the fresh observation (the caller uses **it**, not its first read); `stale` → return `stale` with the fresh observation; a corrupt fresh record → `corrupt`; absent → `stale` with `Rec == nil`; a read error in either round → `unreadable` with the error. The cache is refreshed with round 2's slot. A nil `*SlotCache` judges everything valid and returns the inputs unchanged. |
| + - `func ReadRecordJudged(ctx context.Context, s release.Store, c *SlotCache, acct, app, host string) (*Record, string, error)`: `GetRecordRaw` then `Judge`; stale or a tombstone → `os.ErrNotExist`; corrupt → `ErrRecordCorrupt`; unreadable → the error; valid → the observation's record and ETag. |
| + - `func ListRecordsStrictJudged(ctx, s, c, acct, app string, max int) (recs []Record, unreadable map[string]bool, err error)`, `ListRecordsCompleteJudged(ctx, s, c, acct, app, max) (recs []Record, truncated bool, skipped []string, err error)`, `ListRecordsJudged(ctx, s, c, acct, app, max) (recs []Record, truncated bool, err error)`, `ListAllRecordsStrictJudged(ctx, s, c) (recs []Record, unreadable map[string]bool, err error)`, `ListAllRecordsTolerantJudged(ctx, s, c) (recs []Record, complete bool, err error)`: each is its sibling with stale records dropped, records replaced by their validated observation, and an unreadable slot reported as that host unreadable (strict: the listing error, as a failed record read is today; complete: `skipped` names the host; tolerant: `complete = false`). |
| + - `ReconcileConfig.Slots *SlotCache`. In the decode loop: verdict stale → `continue` (not in `recs`, not protected: the removal phase retires the row as "record gone"); corrupt → `protect[host] = true` as today's malformed body; unreadable → `protect[host] = true` and `readFailed = true`, as a failed read is today; valid → the observation's record is the one judged by the shape checks that follow. |
| + - Console: `s.slots` (`NewServer`); `func (s *Server) classifyObservation(ctx, acct, app, host) (lifetimeClass, error)` (the unlocked read for doors: `GetRecordRaw`, `Judge`), `func (s *Server) readRecord(ctx, acct, app, host) (*Record, string, error)` and `listRecords`, `listRecordsStrict`, `listRecordsComplete`, `listAllRecordsStrict`, `listAllRecordsTolerant` wrappers passing `s.slots` (nil when `s.cfg.HostClaim`). Inventory to route (every instance-record read in these files): `apiinstancemove.go`, `apiinstances.go`, `seedfinish.go`, `channelmove.go`, `apiruns.go`, `apiinstancemove_offline.go`, `apiinstanceimport.go` (`readImportRecord`; `readDeletedRecord` through `classifyObservation`, Task 4), `updatescore.go`, `movesrecovery.go`, `domainattach.go`, `appsettings.go`, `apirunsfollow.go`, `apiretry.go`, `apidb.go`, `apex.go` (`hostHolder`), `apidomains.go`, `apiinstanceseed.go`, `apilegacyroutes.go`, `relayops.go`, `seedsweep.go`, `server.go`, `regiondrift.go`, `runsweep.go` (`ListAllRecordsTolerant`), `seedmove.go`. The walk test `TestNoConsoleReadBypassesTheJudgedReaders` fails on any of: `provision.GetRecord(`, `provision.GetRecordETag(`, `provision.GetRecordRaw(`, `provision.ListRecords`, `provision.ListAllRecords`, `provision.DecodeRecord(`, or `.Get(ctx, provision.RecordKey(` in `internal/console` non-test files outside `lifetimes*.go` and the claims-on files it lists by name. |
| + |
| +- [ ] **Step 1: Write the failing tests**: `TestSlotCacheJudgesAndCaches` (two judgements, one slot read; absence cached; `Begin` and `Forget` drop entries); `TestJudgeAcrossAMigrationBothOrders` (record read before the marker, slot after → round 2 returns the migrated record and its ETag, verdict valid; slot cached as absent, migration completes, the incarnation-1 record is read → round 2 refreshes the slot → valid; cached `{N, live}`, retirement completes, a delayed record at N is read → round 2 → stale); `TestJudgeInconsistentTwiceIsUnreadable` (a slot that changes between the two rounds and a record that keeps disagreeing → unreadable, not stale); `TestNilSlotCacheJudgesEverythingValid`; `TestReconcileRemovesARecordBehindItsSlot`; `TestReconcileRemovesATombstone` (the tombstone has no channel: it leaves before the channel check, so the row is retired, not protected); `TestReconcileKeepsALegacyRecordWithNoSlot`; `TestReconcileKeepsAMarkedRecordUnderItsMigrationSlot`; `TestReconcileProtectsWhenTheSlotIsUnreadable` (a failing read; a malformed body); `TestReconcileUsesTheValidatedObservation` (the fresh record's region, not the first read's, drives the pass); `TestEveryClaimsOffConsumerIgnoresAFieldlessRecordBesideASlot` (slot `{1, retired}`, fieldless record carrying an offline move: `ReadRecordJudged` not found; the five judged listings omit it; the console GET answers 404; reconcile mints nothing and retires its row; the edges file stops listing the host on the next write; `offlineReassert` and the staging deletion gate refuse to act for it; the offline keep view after `refreshOfflineView` does not keep it); `TestClaimsOnConsumersReadNoSlot` (`HostClaim` on the console, `CARLOS_HOST_CLAIM` on the box: a counting store sees no `control/host-incarnations/` read); `TestBoxSlotCacheIsSharedAndGated` (`CARLOS_HOST_CLAIM` set → every consumer's cache is nil; unset → one cache instance in reconcile, the moves worker and the offline view); `TestNoConsoleReadBypassesTheJudgedReaders`. |
| - [ ] **Step 2: Run to verify they fail.** |
| - [ ] **Step 3: Implement.** |
| - [ ] **Step 4: Run** the touched packages' targeted tests, then the two census tests and add the rows they name. |
| -- [ ] **Step 5: Commit** `lifetimes: reconcile, the box listings and every console read judge a record's lifetime; a mismatch is re-read before it is believed`. |
| +- [ ] **Step 5: Commit** `lifetimes: every claims-off read judges a record's lifetime and acts on the observation it validated`. |
| ### Task 7: `move_seq` on the record and the pair on rows |
| **Files:** |
| -- Modify: `internal/provision/offline_move.go` (`OfflineStart`, `OfflineStartStaged`, `OfflineOpenTransition`, `OfflineFinish` each set `rec.MoveSeq = rec.MoveSeq + 1` and `rec.Move.Seq` (nil-safe) to that value; `OfflineAbort` does so only from `OfflineStopping`, `OfflineSealing`, `OfflineSealed`, `OfflineLanding` or `OfflineLanded` (the source reopens), not from `OfflineStaging` (nothing was held); `OfflineStage`, `OfflineRestage`, `OfflineSeal`, `OfflineSealedBy`, `OfflineFlip`, `OfflineLandedBy`, `OfflineFinishingTransition` leave both alone), `internal/provision/provision.go` (`MoveState.Seq uint64 \`json:"seq,omitempty"\``), `internal/provision/offline_move_test.go` (`TestOfflineRecordDiffersOnlyWhereTheMoveSaysSo`: `move_seq` joins the allowlist and nothing else changes), `internal/registry/registry.go` (columns `incarnation INTEGER NOT NULL DEFAULT 0`, `serving_seq INTEGER NOT NULL DEFAULT 0` in the `ALTER TABLE` list; `Route.Incarnation uint64`, `Route.ServingSeq uint64`; `routeCols`, `routeColCount` (+2), `scanRoute`, `routeArgs`, `normalizeRoute`; `Upsert`'s conflict clause gains `incarnation=routes.incarnation, serving_seq=routes.serving_seq` (an ordinary upsert never moves the pair; the claim-ownership `WHERE` predicate is unchanged); `Fingerprint`'s column list gains both; `remote_dispatch.go` gains the new methods), `internal/provision/reconcile.go` (`RouteFor` copies `rec.Incarnation` and `rec.MoveSeq` into a minted row, so `mintHeld`'s mint carries them too; `ReconcileConfig.PairOf func(rec Record) registry.Pair` (nil → the zero pair, which the setters write as today's behaviour for Phase 1); `mintHeld` and `holdExisting` pass `cfg.PairOf(rec)` to the mint and to `SetHoldIfSeq`; after the converge of an existing provisioned row of this pass's app and account, `installLifetime(reg, existing, rec)`: when `existing.HoldOp == ""` and either (`existing.Incarnation == 0 && rec.Incarnation > 0`) or (`0 < existing.Incarnation < rec.Incarnation`), call `InstallLifetimeIf(host, Pair{existing.Incarnation, existing.ServingSeq}, Pair{rec.Incarnation, rec.MoveSeq})`; a held row, or one at the record's incarnation, is never touched here, and a held row of an older lifetime is logged once per pass), `internal/host/movesoffline.go` (`withOfflineHolds` sets `cfg.PairOf = func(rec) Pair{rec.Incarnation, rec.Move.Seq}` (nil-safe); `moveCfg` gains `ReleaseHoldSeq func(host, op string, pair registry.Pair) (bool, error)`, backed by a new `releaseHeldSeq(reg, written)` in `movesoffline.go`; the open path at `movesoffline.go:1077` and the abort path at `movesoffline_land.go:165` call it with `{rec.Incarnation, rec.Move.Seq}`; Phase 1's `releaseHeld` and `moveCfg.ReleaseHold` are untouched) |
| +- Modify: `internal/provision/offline_move.go` (the transition table below), `internal/provision/provision.go` (`MoveState.Seq uint64 \`json:"seq,omitempty"\``), `internal/provision/offline_move_test.go` (`TestOfflineRecordDiffersOnlyWhereTheMoveSaysSo`: `move_seq` joins the allowlist and nothing else changes; `TestOfflineAbort` keeps pinning that a repeated abort is a success without a write), `internal/registry/registry.go` (columns `incarnation INTEGER NOT NULL DEFAULT 0`, `serving_seq INTEGER NOT NULL DEFAULT 0` in the `ALTER TABLE` list; `Route.Incarnation uint64`, `Route.ServingSeq uint64`; `routeCols`, `routeColCount` (+2), `scanRoute`, `routeArgs`, `normalizeRoute`; `Upsert`'s conflict clause gains `incarnation=routes.incarnation, serving_seq=routes.serving_seq` (an ordinary upsert never moves the pair; the claim-ownership `WHERE` predicate is unchanged); `Fingerprint`'s column list gains both; `remote_dispatch.go` gains the new methods; `SetHold`, `SetHoldIf`, `HandHold`, `ReleaseHold`, `DeleteIfHold` are untouched), `internal/provision/reconcile.go` (`RouteFor` copies `rec.Incarnation` and `rec.MoveSeq` into a minted row, so `mintHeld`'s mint carries them too; `ReconcileConfig.PairOf func(rec Record) registry.Pair` (nil → `holdExisting` and the release paths call the existing `SetHoldIf`/`ReleaseHold`, untouched; non-nil → the `Seq` setters); `holdExisting` passes `cfg.PairOf(fresh)` from the judged fresh record to `SetHoldIfSeq`; the existing-row rule after the converge of a provisioned row of this pass's app and account: **(i) migration, unheld**: `existing.Incarnation == 0 && rec.Incarnation > 0 && existing.HoldOp == ""` → `InstallLifetimeIf(host, Pair{0, existing.ServingSeq}, Pair{rec.Incarnation, existing.ServingSeq})` (the row's own sequence is kept); **(ii) recreation, unheld**: `0 < existing.Incarnation < rec.Incarnation && existing.HoldOp == ""` → `InstallLifetimeIf(host, Pair{existing.Incarnation, existing.ServingSeq}, Pair{rec.Incarnation, rec.MoveSeq})`; **(iii) recreation, held**: `existing.Incarnation < rec.Incarnation && existing.HoldOp != ""` and `rec.Move == nil || rec.Move.ID != existing.HoldOp` (the hold's op is not this record's move) → under `WithHostHoldLock`, `cfg.DeleteIfHold(host, existing.HoldOp)` (the agent's `DeleteIfHoldWithTimers`), then the ordinary mint of the record on this pass; **(iv) same incarnation**: nothing; a held row whose op is the record's own move is left to the move), `internal/host/movesoffline.go` (`withOfflineHolds` sets `cfg.PairOf = func(rec) Pair{rec.Incarnation, rec.Move.Seq}` (nil-safe); `moveCfg` gains `ReleaseHoldSeq func(host, op string, pair registry.Pair) (bool, error)`, backed by a new `releaseHeldSeq(reg, written)` beside `releaseHeld`; the **source abort/revert** path at `movesoffline.go:1077` calls it with the fresh judged record's `{Incarnation, Move.Seq}`), `internal/host/movesoffline_land.go` (the **target open** path at line 165 likewise), `internal/host/movespass.go` (`armOffline`, line 371: `cfg.ReleaseHoldSeq = releaseHeldSeq(reg, holdStageRequester(dataDir))` beside `cfg.ReleaseHold`), `internal/host/movesoffline_test.go:207` and `internal/host/movesoffline_land_test.go:206` (the fixture overrides gain `cfg.ReleaseHoldSeq = releaseHeldSeq(b.reg, nil)`) |
| - Test: `internal/provision/offline_move_test.go`, `internal/registry/registry_test.go`, `internal/provision/reconcile_lifetime_test.go`, `internal/host/movesoffline_e2e_test.go` |
| +**The transition table (authoritative):** |
| + |
| +| transition | `move_seq` | note | |
| +|---|---|---| |
| +| `OfflineStart` (direct start) | +1 | the hold | |
| +| `OfflineStage`, `OfflineRestage` | unchanged | nothing held | |
| +| `OfflineStartStaged` | +1 | the hold | |
| +| `OfflineSeal`, `OfflineSealedBy`, `OfflineFlip`, `OfflineLandedBy` | unchanged | | |
| +| `OfflineOpenTransition` | +1 | the target serves | |
| +| `OfflineFinishingTransition` | unchanged | | |
| +| `OfflineAbort` from `offline-staging` | unchanged | never offline | |
| +| `OfflineAbort` from stopping, sealing, sealed, landing, landed | +1 | the source reopens | |
| +| `OfflineAbort` from `offline-reverting` | unchanged | `errRecordUnchanged`: success without a write, as today | |
| +| `OfflineFinish` with `StartedOffline()` | +1 | | |
| +| `OfflineFinish` without (an abort from staging) | unchanged | | |
| + |
| +Each `+1` sets `rec.MoveSeq++` and `rec.Move.Seq = rec.MoveSeq` (nil-safe) in the same mutation. |
| + |
| **Interfaces:** |
| -- Produces: `type registry.Pair struct { Incarnation, ServingSeq uint64 }`; `func (r *Registry) SetHoldIfSeq(host, status, op, expectOp string, pair Pair) (bool, error)` (`UPDATE routes SET status=?, hold_op=?, serving_seq=? WHERE host=? AND hold_op=? AND incarnation=?` with `pair.Incarnation` in the predicate, so a setter of another lifetime changes nothing; a zero `pair.Incarnation` matches only a row at 0, today's rows); `func (r *Registry) ReleaseHoldSeq(host, op string, pair Pair) (bool, error)` (`SET status='ready', hold_op='', serving_seq=? WHERE host=? AND hold_op=? AND incarnation=?`); `func (r *Registry) InstallLifetimeIf(host string, expect, next Pair) (bool, error)` (`SET incarnation=?, serving_seq=? WHERE host=? AND incarnation=? AND serving_seq=? AND hold_op=''`); `Route.Incarnation`, `Route.ServingSeq`; `MoveState.Seq`; `Record.MoveSeq`; `ReconcileConfig.PairOf`; `moveCfg.ReleaseHoldSeq`. |
| +- Produces: `type registry.Pair struct { Incarnation, ServingSeq uint64 }`; `func (r *Registry) SetHoldIfSeq(host, status, op, expectOp string, pair Pair) (bool, error)` (`SetHoldIf`'s validation of status, op and `expectOp` verbatim, then `UPDATE routes SET status=?, hold_op=?, incarnation=?, serving_seq=? WHERE host=? AND hold_op=? AND (incarnation=? OR incarnation=0)` with `pair.Incarnation` twice: a legacy row at 0 takes the lifetime with the hold in one write; a row of another lifetime changes nothing); `func (r *Registry) ReleaseHoldSeq(host, op string, pair Pair) (bool, error)` (`ReleaseHold`'s empty-op no-op, then `SET status='ready', hold_op='', incarnation=?, serving_seq=? WHERE host=? AND hold_op=? AND (incarnation=? OR incarnation=0)`); `func (r *Registry) InstallLifetimeIf(host string, expect, next Pair) (bool, error)` (`SET incarnation=?, serving_seq=? WHERE host=? AND incarnation=? AND serving_seq=? AND hold_op=''`); `Route.Incarnation`, `Route.ServingSeq`; `MoveState.Seq`; `Record.MoveSeq`; `ReconcileConfig.PairOf`; `moveCfg.ReleaseHoldSeq`. |
| - [ ] **Step 1: Write the failing tests**: |
| - - `TestOfflineTransitionsAdvanceMoveSeq`: direct path start 1, open 2, finish 3; staged path stage 0, start-staged 1, open 2, finish 3; abort after start 2, finish 3; abort from staging leaves 0 and finish then leaves 0; a second abort call on a reverting record is the existing refusal and changes nothing; stage, restage, seal, flip, landed, finishing leave it; a record at `MoveSeq 5` starts at 6; `Move.Seq` equals the record's value at each advancing transition. |
| - - `TestOfflineRecordDiffersOnlyWhereTheMoveSaysSo` passes with `move_seq` in its allowlist. |
| - - `TestRegistryPairColumns` (upsert with the pair; an ordinary upsert of another pair keeps the first; `Fingerprint` changes when only the pair changes); `TestSetHoldIfSeqNamesItsLifetime` (a setter at incarnation 1 against a row at 2 changes nothing; at 2 it writes status, op and seq together); `TestReleaseHoldSeqNamesItsLifetime`; `TestInstallLifetimeIfIsGuarded` (`(1, 9)` → `(2, 0)` with the right expectation; a wrong expectation or a held row → unchanged). |
| - - `TestRouteForCopiesThePair`; `TestReconcileInstallsTheRecordsPairOnAMintedRow` (after a finished move at `move_seq 3`, delete the row, reconcile → `(1, 3)`; after an abort from landed and finish, likewise `(1, 3)`); `TestExistingRowsAcrossLifetimes` (subtests: an unheld row `(1, 9)` with record `{2, 0}` → `(2, 0)`; an unheld row `(0, 0)` with a migrated record `{1, 2}` → `(1, 2)` once and never again while the record is at 1 (a later record at `{1, 5}` with the row unheld at `(1, 2)` stays `(1, 2)`); a held row `(1, 1)` by an old op with record `{2, 0}` → unchanged and logged; a held row `(0, 0)` with a migrated record `{1, 0}` → unchanged; a stale `ReleaseHoldSeq` for incarnation 1 against the successor's row at 2 → no change). |
| - - In `TestOfflineMoveE2E`: the records are created through the lifetime (slot `{1, live}`, `Incarnation 1`) with judging on; after `start` the source row is held with `(1, 1)`; after `open` the target row is ready with `(1, 2)`; a second trace aborting from landed leaves the source ready with `(1, 2)` and no target row. |
| + - `TestOfflineTransitionsAdvanceMoveSeq`: one subtest per row of the table (direct path 1, 2, 3; staged path 0, 1, 2, 3; abort after start 2 then finish 3; abort from staging 0 then finish 0; a repeated abort on a reverting record returns the record unchanged with no write; a record at `MoveSeq 5` starts at 6; `Move.Seq` equals the record's value at each advancing transition). |
| + - `TestOfflineRecordDiffersOnlyWhereTheMoveSaysSo` passes with `move_seq` in its allowlist; `TestOfflineAbort` unchanged. |
| + - `TestRegistryPairColumns` (upsert with the pair; an ordinary upsert of another pair keeps the first; `Fingerprint` changes when only the pair changes); `TestSetHoldIfSeqInstallsOnALegacyRowAndRefusesAnotherLifetime` (row at 0 → status, op and `(1, 1)` in one write; row at 2 with a setter at 1 → unchanged; `SetHoldIf`'s validation errors reproduced); `TestReleaseHoldSeqLikewise` (and the empty-op no-op); `TestInstallLifetimeIfIsGuarded`. |
| + - `TestRouteForCopiesThePair`; `TestReconcileInstallsTheRecordsPairOnAMintedRow` (after a finished move at `move_seq 3`, delete the row, reconcile → `(1, 3)`; after an abort from landed and finish, likewise `(1, 3)`); `TestAnExistingRowKeepsOpenSequenceThroughFinish` (the target row at `(1, 2)` after open stays `(1, 2)` after finish and after any number of passes). |
| + - `TestExistingRowsAcrossLifetimes` (subtests, each run to its end state: **ready legacy row, migrated record** `(0, 0)` + `{1, 0}` → `(1, 0)` once, and a later record at `{1, 5}` leaves it; **ready row, recreation** `(1, 9)` + `{2, 0}` → `(2, 0)`; **held legacy row, migration during a move**: a row held `(0, 1)` by op X, the record migrates to incarnation 1 with the move still X → the row is untouched by the installer, and the move's open (`ReleaseHoldSeq(host, X, {1, 2})`) releases it to ready `(1, 2)`; the abort path likewise; **hold applied across migration**: a ready row at `(0, 0)`, a migrated record `{1, 1}` asking for a hold → `holdExisting` writes status, op and `(1, 1)` together; **held predecessor row, recreation**: a row held `(1, 1)` by op X, a record at `{2, 0}` with no move → `DeleteIfHold(host, X)` then a ready mint at `(2, 0)` on the same pass; **a stale setter**: `ReleaseHoldSeq` for incarnation 1 against the successor's row at 2 → no change; **Phase 1 unchanged**: `PairOf == nil` with a row carrying `(1, 3)` from an earlier claims-off lifetime → `holdExisting` calls `SetHoldIf` and the release calls `ReleaseHold`, the pair untouched). |
| + - In `TestOfflineMoveE2E`: the records are created through the lifetime (slot `{1, live}`, `Incarnation 1`) with judging on; after `start` the source row is held with `(1, 1)`; after `open` the target row is ready with `(1, 2)`; a second trace aborting from landed leaves the source ready with `(1, 2)` and no target row; `armOffline`'s wiring (not a test override) is what releases. |
| - [ ] **Step 2: Run to verify they fail.** |
| - [ ] **Step 3: Implement.** |
| - [ ] **Step 4: Run** `go test ./internal/registry -run 'Pair|Hold|Lifetime|Fingerprint' -count=1`, `go test ./internal/provision -run 'MoveSeq|RouteFor|Offline|Reconcile|Lifetime' -count=1`, `go test ./internal/host -run 'OfflineMoveE2E|Offline' -count=1`, then both census tests. |
| -- [ ] **Step 5: Commit** `moves: the offline hold, open, abort and finish advance move_seq; rows carry (incarnation, serving_seq) and change it only by setters that name the lifetime`. |
| +- [ ] **Step 5: Commit** `moves: the offline hold, open, abort and finish advance move_seq; rows carry (incarnation, serving_seq), installed with the hold on legacy rows and never moved by another lifetime`. |
| -### Task 8: Publish the pair and order one lifetime's entries by it |
| +### Task 8: Publish the pair and order unstamped entries of one lifetime by it |
| **Files:** |
| -- Modify: `internal/host/edgesfile.go` (`edgeHost` gains `Incarnation uint64 \`json:"incarnation,omitempty"\``, `ServingSeq uint64 \`json:"serving_seq,omitempty"\``, `TargetHost string \`json:"target_host,omitempty"\``; a served route publishes its row's pair; the explicit alias entry at line 546 publishes `hosts[a.Host]`'s pair and `TargetHost: a.Host`; the wildcard dial entry at line 632, when `catchallServed[wd.Host]` names a `primary`, publishes `hosts[primary]`'s pair and `TargetHost: primary`; a certificate-only wildcard entry publishes none), `internal/edge/homemap.go` (`edgesHostView` and `homeEntry` gain `incarnation`, `servingSeq`, `targetHost`; `foldHomes` copies all three with the dial winner; `dialBeats` becomes: `!in.dial` → false; `!prev.dial` → true; `prev.alias != in.alias` → the real route wins (today); both real → higher `(incarnation, servingSeq)` wins, equal → `in.stamp > prev.stamp` (today); both alias with `prev.targetHost != "" && prev.targetHost == in.targetHost` → the same pair rule; both alias otherwise → `in.stamp > prev.stamp` (today, unchanged)) |
| +- Modify: `internal/host/edgesfile.go` (`edgeHost` gains `Incarnation uint64 \`json:"incarnation,omitempty"\``, `ServingSeq uint64 \`json:"serving_seq,omitempty"\``, `TargetHost string \`json:"target_host,omitempty"\``; **only the claims-off writer publishes the pair** (`edgesFileWriteClaims` with `claims == nil`): a served route publishes its row's pair; the explicit alias entry at line 546 publishes `hosts[a.Host]`'s pair and `TargetHost: a.Host`; the wildcard dial entry at line 632, when `catchallServed[wd.Host]` names a `primary`, publishes `hosts[primary]`'s pair and `TargetHost: primary`; a certificate-only wildcard entry publishes none; with `claims != nil` the pair fields stay zero whatever the row carries, and `TargetHost` is still published), `internal/edge/homemap.go` (`edgesHostView` and `homeEntry` gain `incarnation`, `servingSeq`, `targetHost`; `foldHomes` copies all three with the dial winner; `dialBeats` becomes: `!in.dial` → false; `!prev.dial` → true; `prev.alias != in.alias` → the real route wins (today); **if either entry carries a stamp (`stamp != 0`) → `in.stamp > prev.stamp` (today, the claim authority)**; both unstamped and both real → higher `(incarnation, servingSeq)` wins, equal → `in.stamp > prev.stamp` (false: today's first-wins); both unstamped aliases with `prev.targetHost != "" && prev.targetHost == in.targetHost` → the same pair rule; both unstamped aliases otherwise → today's rule) |
| - Test: `internal/host/edgesfile_test.go`, `internal/edge/homemap_test.go` |
| **Interfaces:** |
| -- Produces: the three fields on the wire and in the fold; the comparable rule above. The different-target alias rule (a ready route of another target over a hold) is plan 3's; plan 1 publishes `target_host` and never orders aliases of different or unknown targets by the pair. |
| +- Produces: the three fields on the wire and in the fold; the comparable rule above. A claims-on publication always carries a stamp (`applyEdgesClaims`), so a retained claims-off publication with a pair loses to it, and two claims-on publications fold exactly as today. The different-target alias rule is plan 3's. |
| -- [ ] **Step 1: Write the failing tests**: `TestEdgesFilePublishesThePairAndTargetHost` (a row `(2, 3)`; its explicit alias; a wildcard dial entry whose parent name differs from the primary; a certificate-only wildcard entry publishes no target and no pair); `TestFoldPrefersTheHigherPairOfOneLifetime` (`(1, 2)` over `(1, 1)`; `(2, 0)` over `(1, 9)`; `(0, 0)` loses to any pair; equal → the stamp rule); `TestFoldKeepsRealRouteOverAHigherPairAlias`; `TestFoldComparesAliasesOnlyForOneTarget` (aliases to A at `(5, 9)` and to B at `(1, 0)`: today's stamp rule decides, not the pair; aliases to A at `(1, 2)` and A at `(1, 1)`: the pair decides; an alias with no target never compares by pair); `TestFoldNeverDialsACertOnlyEntryForItsPair`; `TestFoldCopiesTheWinnersPairAndTarget`; `TestFoldLegacyPublicationsUnchanged` (fixtures without the fields fold as before; `TestHomeMapPicksTheSameHomeOnADuplicateClaim` still passes). |
| +- [ ] **Step 1: Write the failing tests**: `TestEdgesFilePublishesThePairAndTargetHost` (a row `(2, 3)`; its explicit alias; a wildcard dial entry whose parent name differs from the primary; a certificate-only wildcard entry publishes no target and no pair); `TestClaimsOnWriterPublishesNoPair` (`claims != nil`, a row carrying `(1, 3)` → the entry has a stamp and no pair fields); `TestFoldPrefersTheHigherPairOfOneLifetime` (`(1, 2)` over `(1, 1)`; `(2, 0)` over `(1, 9)`; `(0, 0)` loses to any pair; equal → today's rule); `TestFoldStampBeatsPair` (a retained unstamped `(1, 3)` against a stamped `(0, 0)` → the stamped entry; two stamped entries → the higher stamp as today, whatever pairs they carry); `TestFoldKeepsRealRouteOverAHigherPairAlias`; `TestFoldComparesAliasesOnlyForOneTarget`; `TestFoldNeverDialsACertOnlyEntryForItsPair`; `TestFoldCopiesTheWinnersPairAndTarget`; `TestFoldLegacyPublicationsUnchanged` (fixtures without the fields fold as before; `TestHomeMapPicksTheSameHomeOnADuplicateClaim` still passes). |
| - [ ] **Step 2: Run to verify they fail.** |
| - [ ] **Step 3: Implement.** |
| -- [ ] **Step 4: Run** `go test ./internal/host -run 'EdgesFile' -count=1`, `go test ./internal/edge -run 'Fold|HomeMap' -count=1`, then the census tests. |
| -- [ ] **Step 5: Commit** `edges: publish (incarnation, serving_seq) and target_host; the home map orders one lifetime's entries by the pair`. |
| +- [ ] **Step 4: Run** `go test ./internal/host -run 'EdgesFile|ClaimsOnWriter' -count=1`, `go test ./internal/edge -run 'Fold|HomeMap' -count=1`, then the census tests. |
| +- [ ] **Step 5: Commit** `edges: the claims-off writer publishes (incarnation, serving_seq) and target_host; the home map orders unstamped entries of one lifetime by the pair`. |
| ### Task 9: Lifetimes end to end |
| **Files:** |
| - Test: `internal/host/movesoffline_lifetime_test.go` (extending `TestOfflineMoveE2E`'s fixtures), `internal/console/lifetimes_e2e_test.go` |
| -These are §11's witnesses plan 1 owns: record and row transitions and the ordering of ready publications. Every witness that needs a holding publication (a hold winning a fold, a current hold beating a previous lifetime's route, a stale ready alias losing to its own hold, holding entries at the edge) is plan 3's and is not claimed here. |
| +These are §11's witnesses plan 1 owns: record and row transitions and the ordering of ready publications. Every witness that needs a holding publication is plan 3's. Where a fold input below is not something this plan's writer produces (a retained target publication at a hold's pair, a retained `(1, 9)`), the test builds it as a **synthetic fixture** and says so in its name. |
| -- [ ] **Step 1: Write the tests**: `TestLifetimeAcrossAForwardMove` (create `(1, 0)`; start → source row held `(1, 1)`; open → target row ready `(1, 2)`; finish → `(1, 3)`; a fold of the target's ready publication at `(1, 3)` against a retained source publication at `(1, 0)` prefers the target); `TestLifetimeAcrossAnAbort` (a second trace: start, flip to landed, abort → source reopens at `(1, 2)`, finish → `(1, 3)`; the source's publication at `(1, 3)` beats a retained target publication at `(1, 1)`); `TestReconstructedRowCarriesTheRecordsPair` (after a finished move at `move_seq 3`, delete the target's row, reconcile → `(1, 3)`, beats a retained `(1, 1)`); `TestReconstructedSourceAfterAbortWins` (abort from landed, finish, delete the source row, reconcile → `(1, 3)`, beats the target's retained `(1, 1)`); `TestRecreatedHostOutranksItsPreviousLifetime` (retire, create → `(2, 0)` beats a retained `(1, 9)`); `TestLateLegacyCreateIsInert` (slot `{1, retired}`; a fieldless record written by the test; reconcile mints nothing, the door 404s, the strict listings omit it, the next create replaces it); `TestDelayedCreateReplayMoveAndRetirementInterleave` (pending create replayed; a move completes; retirement crashes between the tombstone and the slot; the late original create fails on the tombstone; recovery retires the slot and removes the tombstone); `TestLandedCreateThenSecondRecoveryCrash` (a create whose record landed and serves, a crash before `committed`, recovery marks it committed with the record's ETag unchanged, including a second crash during that recovery). |
| +- [ ] **Step 1: Write the tests**: `TestLifetimeAcrossAForwardMove` (create `(1, 0)`; start → source row held `(1, 1)`; open → target row ready `(1, 2)`; finish → the target row stays `(1, 2)`, the record's `move_seq` is 3; a fold of the target's ready publication at `(1, 2)` against a retained source publication at `(1, 0)` prefers the target); `TestLifetimeAcrossAnAbort` (a second trace: start, flip to landed, abort → source reopens at `(1, 2)`, finish → the row stays `(1, 2)`, `move_seq` 3; the source's publication at `(1, 2)` beats a synthetic retained target publication at `(1, 1)`); `TestReconstructedRowCarriesTheRecordsPair` (after a finished move at `move_seq 3`, delete the target's row, reconcile → `(1, 3)`, beats a synthetic `(1, 1)`); `TestReconstructedSourceAfterAbortWins` (abort from landed, finish, delete the source row, reconcile → `(1, 3)`); `TestRecreatedHostOutranksItsPreviousLifetime` (retire, create → `(2, 0)` beats a synthetic retained `(1, 9)`); `TestLateLegacyCreateIsInert` (slot `{1, retired}`; a fieldless record written by the test; reconcile mints nothing, the door 404s, the strict listings omit it, the edges file does not list it, the next create replaces it); `TestDelayedCreateReplayMoveAndRetirementInterleave` (pending create replayed; a move completes; retirement crashes between the tombstone and the slot; the late original create fails on the tombstone; recovery retires the slot and removes the tombstone); `TestLandedCreateThenSecondRecoveryCrash` (a create whose record landed and serves, a crash before `committed`, recovery marks it committed with the record's ETag unchanged, including a second crash during that recovery). |
| - [ ] **Step 2: Run them; fix what they find within this plan's scope.** |
| - [ ] **Step 3: Run the census tests and both cross-builds** (`GOOS=darwin go build ./internal/host/`, `GOOS=windows go vet ./internal/host/`); write the census note. |
| - [ ] **Step 4: Commit** `lifetimes: end-to-end witnesses across a move, an abort, a reconstruction, a recreation and an interleaved retirement`; push. |
| @@ -281,7 +318,7 @@ These are §11's witnesses plan 1 owns: record and row transitions and the order |
| ## Self-review notes |
| -- §6.4 coverage: slot, owner key and tombstone form (T1, T4), create and recovery (T3), every claims-off removal and its recovery (T4), migration, its fence and recovery (T5), the verdict in every claims-off consumer with the re-read (T1, T6), `move_seq` and the row pair with guarded setters and the installation rule (T7), publication and the one-lifetime fold order (T8), witnesses (T9). Deferred by name: holding entries and their five-minute validity, the probe mode, the different-target alias rule and every holding-dependent witness (plan 3); the door, token, job, reservations and reports (plan 2); cleanup (plan 4). |
| -- Names used across tasks: `LifetimeOf`, `LifetimeVerdict` (`LifetimeValid`, `LifetimeStale`, `LifetimeUnreadable`), `ReadHostIncarnation`, `WriteHostIncarnation`, `HostIncarnation{Incarnation, State, Account, App, Intent, RecordETag}`, `TombstoneBody`, `GetRecordRaw`, `withHostname`, `admit`, `classifyLocked`, `createRecordWithLifetime`, `retireRecordWithLifetime`, `migrateHostLifetimeLocked`, `errLifetimeSettling`, `errLifetimePending`, `errHostHeld`, `errRetireRefused`, `ErrRecordMigrating`, `WithMigration`, `SlotCache.Judge`, `ReadRecordJudged`, `List*Judged`, `ReconcileConfig.Slots`, `ReconcileConfig.PairOf`, `registry.Pair`, `SetHoldIfSeq`, `ReleaseHoldSeq`, `InstallLifetimeIf`, `moveCfg.ReleaseHoldSeq`, `Route.Incarnation`, `Route.ServingSeq`, `MoveState.Seq`, `edgeHost.Incarnation/ServingSeq/TargetHost`. |
| -- Review Focus 1 to 5 are pinned by T3 (two tests), T5+T6, T6+T3, T7, T7. |
| -- Store primitives, as verified in the reviews: `Store.Put(..., "")` is create-only on both backends; `WriteRecord` with `before={}` and `etag=""` is the create and needs the real CAS base otherwise; `DeleteIfMatch` treats absence as success and has no unconditional fallback. |
| +- §6.4 coverage: slot, owner key and tombstone form (T1, T4), create and recovery (T3), every claims-off removal and its recovery (T4), migration, its fence and recovery (T5), the verdict in every claims-off consumer with the two-round judgement (T1, T6), `move_seq` and the row pair with lifetime-naming setters and the four-case existing-row rule (T7), publication by the claims-off writer and the unstamped one-lifetime fold order (T8), witnesses (T9). Deferred by name: holding entries and their five-minute validity, the probe mode, the different-target alias rule and every holding-dependent witness (plan 3); the door, token, job, reservations and reports (plan 2); cleanup (plan 4). |
| +- Names used across tasks: `LifetimeOf`, `LifetimeVerdict` (`LifetimeValid`, `LifetimeStale`, `LifetimeCorrupt`, `LifetimeUnreadable`), `HostIncarnation{Incarnation, State, Account, App, Intent, RecordETag}.Valid`, `ErrSlotMalformed`, `ErrRecordCorrupt`, `ReadHostIncarnation`, `WriteHostIncarnation`, `TombstoneBody`, `GetRecordRaw`, `withHostname`, `admit`, `classifyLocked`, `classifyObservation`, `lifetimeClass`, `createRecordWithLifetime`, `retireRecordWithLifetime`, `retireExpectAbsent`, `migrateHostLifetimeLocked`, `errLifetimeSettling`, `errLifetimePending`, `*hostHeldError`, `errRetireRefused`, `ErrRecordMigrating`, `WithMigration`, `Observation`, `SlotCache.Begin/Forget/Judge`, `ReadRecordJudged`, `List*Judged`, `ReconcileConfig.Slots`, `ReconcileConfig.PairOf`, `registry.Pair`, `SetHoldIfSeq`, `ReleaseHoldSeq`, `InstallLifetimeIf`, `moveCfg.ReleaseHoldSeq`, `releaseHeldSeq`, `Route.Incarnation`, `Route.ServingSeq`, `MoveState.Seq`, `edgeHost.Incarnation/ServingSeq/TargetHost`. |
| +- Review Focus 1 to 5 are pinned by T3 (three tests), T5+T6, T6+T3, T7 (two tests), T7. |
| +- Store primitives, as verified in the reviews: `Store.Put(..., "")` is create-only on both backends; `WriteRecord` with `before={}` and `etag=""` is the create and needs the real CAS base otherwise; `DeleteIfMatch` treats absence as success and has no unconditional fallback; `OfflineAbort` on a reverting record returns `errRecordUnchanged`, which `SetRecordMove` turns into a success without a write. |