amadan
console: the cutover door runs the box floor unless a validated marker was read (final review F1)
A watcher read failure while the marker is absent moves before to draining, which closes admission, but proves nothing about the marker. The door now skips the floor only on markerSeen (a validated body read, or the door's own write), never on the gate's state. Census: the door re-pinned in place and the internal/console envelope refreshed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
5 files changed,
+98
−5
docs/superpowers/notes/census-proposal-self-serve-moves-1.md+2 −0internal/console/lifetimes_cutover.go+14 −3internal/console/lifetimes_floor_test.go+72 −0internal/provision/self_serve_moves1_census_test.go+8 −1internal/provision/testdata/plan14-root-reads-self-serve-moves-1.tsv+2 −1
diff --git a/docs/superpowers/notes/census-proposal-self-serve-moves-1.md b/docs/superpowers/notes/census-proposal-self-serve-moves-1.md| index 1707412d9..ca3a4cbc8 100644 |
| --- a/docs/superpowers/notes/census-proposal-self-serve-moves-1.md |
| +++ b/docs/superpowers/notes/census-proposal-self-serve-moves-1.md |
| @@ -42,4 +42,6 @@ Task 9 adds the plan's end-to-end witnesses, in `internal/host/movesoffline_life |
| Task 9 fix round 1 corrects Task 3's create door. When a delayed create's SQLite commit is refused because recovery already committed its allocation, `Server.allocRecordLocked` now asks `Server.committedElsewhere`. That function reads the allocation row and the record through `readRecord` (judged) and writes nothing. When the row is committed and the record is valid at the allocation's incarnation, the door answers success; anything else stays pending. `Server.allocRecordLocked` stays a record-writer through `WriteRecord` and is re-pinned in place in the record-write inventory. The `internal/console` envelope is refreshed in place. No root read, class, legacy row or seal changes, and `TestNoUnjudgedRecordReads` stays green. |
| +Final review fix round. F1 changes the cutover door: it skips the box floor only once a validated marker body has been read or written by the door, never on the gate's state alone. A failed marker read still closes admission. `Server.handleAPILifetimesCutover` stays nonrecord and is re-pinned in place in the record-write inventory, and the `internal/console` envelope is refreshed in place. No root read, class, legacy row or seal changes. |
| + |
| Taken together, the rows marked `PROPOSED-self-serve-moves-1` are the census of one change. Claims-off instance records gain a lifetime: the host incarnation slot, written only through `WriteHostIncarnation` by the console's create, retirement and migration protocols. Every claims-off consumer judges a record against its slot before it acts on it. Every record write stays on `WriteRecord` with the sequence discipline. The per-host `move_seq` and each row's `(incarnation, serving_seq)` pair order the publications an edge folds. Each row is one of five kinds: a write site added or re-pinned in the record-write inventory (each through `WriteRecord`, a `DeleteIfMatch` on an observed ETag, or a Put of a control key that is not a record); a read made judged at its existing class; a console SQLite unit or a legacy fingerprint with a reviewed basis; the `@fields` schema row, which pins the registry fields Task 7 adds (`Route.Incarnation`, `Route.ServingSeq`, and the remote request's `Pair` and `ExpectPair`); or a source envelope refreshed over the new code. No row adds a raw record write, a root-read class or a task. The two registry read sites added in Task 8 are named in its paragraph. These rows are one proposal, for Paul to rule on once, together, after the plan's final review. Until then every row stays PROPOSED, and a green census means only that the rows match the code, not that they are accepted. |
diff --git a/internal/console/lifetimes_cutover.go b/internal/console/lifetimes_cutover.go| index db4af0ac1..fd7bd77ef 100644 |
| --- a/internal/console/lifetimes_cutover.go |
| +++ b/internal/console/lifetimes_cutover.go |
| @@ -258,6 +258,15 @@ func (g *cutoverGate) status() cutoverStatus { |
| return st |
| } |
| +// markerValidated reports whether a marker body was read and validated (or |
| +// written by the door). It is the only evidence that lets the door skip the |
| +// box floor: draining by a failed read is not. |
| +func (g *cutoverGate) markerValidated() bool { |
| + g.mu.Lock() |
| + defer g.mu.Unlock() |
| + return g.markerSeen |
| +} |
| + |
| func (g *cutoverGate) current() cutoverState { |
| g.mu.Lock() |
| defer g.mu.Unlock() |
| @@ -526,9 +535,11 @@ func (s *Server) handleAPILifetimesCutover(w http.ResponseWriter, r *http.Reques |
| } |
| g := s.cutover |
| now := g.clock() |
| - // The box floor (lifetimes_floor.go), before the marker is first |
| - // written; once it is, the door answers the status. |
| - if st := g.current(); st == cutoverBefore || st == cutoverUnknown { |
| + // The box floor (lifetimes_floor.go), until a validated marker body has |
| + // been read (or written by this door); then the door answers the status. |
| + // The state alone is no evidence: a failed marker read drains admission |
| + // without proving a marker exists, so draining still runs the floor. |
| + if !g.markerValidated() { |
| fctx, fcancel := context.WithTimeout(r.Context(), instanceStoreTimeout) |
| ref := s.lifetimesBoxFloor(fctx) |
| fcancel() |
diff --git a/internal/console/lifetimes_floor_test.go b/internal/console/lifetimes_floor_test.go| index f6b6edaf9..ca099f219 100644 |
| --- a/internal/console/lifetimes_floor_test.go |
| +++ b/internal/console/lifetimes_floor_test.go |
| @@ -127,3 +127,75 @@ func TestCutoverRefusesBoxesBelowTheFloor(t *testing.T) { |
| } |
| }) |
| } |
| + |
| +// A watcher read failure while the marker is absent closes admission |
| +// (before -> draining) but is no evidence that a marker exists: the door must |
| +// still run the box floor. Only a validated marker read skips it (final |
| +// review F1). |
| +func TestCutoverFloorRunsAfterAMarkerReadFailure(t *testing.T) { |
| + failOnce := func(t *testing.T, l lifetimesFx) { |
| + t.Helper() |
| + fault := &markerFaultStore{Store: l.fx.rel} |
| + fault.fail.Store(true) |
| + l.fx.srv.cfg.Releases = fault |
| + l.fx.srv.lifetimesWatchOnce(context.Background()) |
| + l.fx.srv.cfg.Releases = l.fx.rel // storage recovers |
| + if st := l.fx.srv.cutover.status(); st.State != "draining" || l.fx.srv.cutover.markerValidated() { |
| + t.Fatalf("after a failed marker read = %+v (validated %v), want draining with no validated marker", st, l.fx.srv.cutover.markerValidated()) |
| + } |
| + } |
| + for _, c := range []struct { |
| + name string |
| + arrange func(t *testing.T, l lifetimesFx) |
| + msg string |
| + }{ |
| + {"a box without the capability", func(t *testing.T, l lifetimesFx) { |
| + writeEdges(t, l.fx.rel, "annie", capLifetimes1) |
| + writeEdges(t, l.fx.rel, "bert") |
| + writeBoxStatus(t, l.fx.rel, "annie", time.Now()) |
| + writeBoxStatus(t, l.fx.rel, "bert", time.Now()) |
| + }, "box bert does not publish lifetimes-1; every box must run this release before the cutover"}, |
| + {"a missing status report", func(t *testing.T, l lifetimesFx) { |
| + writeEdges(t, l.fx.rel, "annie", capLifetimes1) |
| + }, "box annie has not reported in 10 minutes"}, |
| + } { |
| + t.Run(c.name, func(t *testing.T) { |
| + l := newLifetimesFx(t) |
| + c.arrange(t, l) |
| + failOnce(t, l) |
| + code, _, body := l.cutover(t, l.operator) |
| + if code != http.StatusConflict || cutoverErrorOf(body) != c.msg { |
| + t.Fatalf("cutover after a failed marker read = %d %s, want 409 %q", code, body, c.msg) |
| + } |
| + if _, _, err := l.fx.rel.Get(context.Background(), lifetimesCutoverKey); err == nil { |
| + t.Fatal("the refused cutover wrote the marker") |
| + } |
| + l.clock.Advance(2 * lifetimeSettle) |
| + l.fx.srv.lifetimesWatchOnce(context.Background()) |
| + if got := l.fx.srv.cutover.status().State; got != "before" { |
| + t.Fatalf("after the refusal and a clean read = %s, want before", got) |
| + } |
| + }) |
| + } |
| + t.Run("a validated marker skips the floor", func(t *testing.T) { |
| + l := newLifetimesFx(t) |
| + at := l.clock.Now() |
| + body, _ := json.Marshal(cutoverMarker{At: at, By: "paul@keymail.dev"}) |
| + if _, err := l.fx.rel.Put(context.Background(), lifetimesCutoverKey, body, ""); err != nil { |
| + t.Fatal(err) |
| + } |
| + l.fx.srv.lifetimesWatchOnce(context.Background()) |
| + if !l.fx.srv.cutover.markerValidated() { |
| + t.Fatal("the watcher read the marker and did not validate it") |
| + } |
| + etag := markerETag(t, l.fx.rel, lifetimesCutoverKey) |
| + writeEdges(t, l.fx.rel, "bert") // below the floor: the door must not look |
| + code, st, out := l.cutover(t, l.operator) |
| + if code != http.StatusOK || st.State != "draining" || !st.At.Equal(at) { |
| + t.Fatalf("cutover over a validated marker = %d %s, want 200 draining at %v", code, out, at) |
| + } |
| + if markerETag(t, l.fx.rel, lifetimesCutoverKey) != etag { |
| + t.Fatal("the door rewrote a validated marker") |
| + } |
| + }) |
| +} |
diff --git a/internal/provision/self_serve_moves1_census_test.go b/internal/provision/self_serve_moves1_census_test.go| index 2f124ffae..8033a1476 100644 |
| --- a/internal/provision/self_serve_moves1_census_test.go |
| +++ b/internal/provision/self_serve_moves1_census_test.go |
| @@ -225,6 +225,13 @@ package provision |
| // the record through readRecord (judged) and writes nothing; a row |
| // committed by recovery and a valid record at the allocation's |
| // incarnation answer the create as a success. |
| +// |
| +// Final review fix round (all PROPOSED): no write site added, moved or made |
| +// raw. |
| +// - F1: lifetimes_cutover.go:Server.handleAPILifetimesCutover: nonrecord |
| +// as before, re-pinned in place. The box floor is skipped only once a |
| +// validated marker body was read (or written by the door), never on the |
| +// gate's state alone; the marker Put is unchanged. |
| func plan14SelfServeMoves1WriteCensusInventory() []censusEntry { |
| removed := []struct{ file, unit string }{ |
| {"internal/provision/provision.go", "GetRecordETag"}, |
| @@ -282,7 +289,7 @@ func plan14SelfServeMoves1WriteCensusInventory() []censusEntry { |
| {"internal/provision/lifetime_judge.go", "readAllRecordsJudged", "key-site", "key:ParseRecordKey,lit:instances", "e56e43bb89e7a013445033b390cd57bf6eda16a539f5a398d1dd26597fe30ca4"}, |
| {"internal/provision/lifetime.go", "GetRecordRaw", "key-site", "key:RecordKey", "1e893cadc5dedac4687a6877457254c8e07c328a91b2c5923ef87fd3c41ff1fc"}, |
| {"internal/provision/lifetime.go", "WriteHostIncarnation", "nonrecord", "sel:Put", "c5f2719a53447c0477706600081a9846444071f39b85759a463d1ee757dc760e"}, |
| - {"internal/console/lifetimes_cutover.go", "Server.handleAPILifetimesCutover", "nonrecord", "sel:Put", "76f056056375ee90cbf7b168e032353f8c63d7238d5242af36160b7c6b1acf8a"}, |
| + {"internal/console/lifetimes_cutover.go", "Server.handleAPILifetimesCutover", "nonrecord", "sel:Put", "d420b4fd0bcaf52feb0e4271bd1c30a84c7008e83444c9726a581e3e350308f7"}, |
| {"internal/console/apiinstanceimport.go", "Server.runImportUpload", "record-writer", "key:SetRecordSeed", "f602216e89e9a93c330afb5153894621918ab390655897c1dc9bd2435f5cab80"}, |
| {"internal/console/apiinstances.go", "Server.handleAPIInstanceList", "key-site", "lit:instances", "8e290e7771e58b2c5d20029ef9a1e22564eafd83cb8a97506bb28218a2202d15"}, |
| {"internal/console/apiinstanceseed.go", "Server.runFrom", "record-writer", "key:SetRecordSeed", "0f0ed34b5a5c99cc7f8245f2e8d6284c300b4c7314b312af69f96d6d49f0d4c0"}, |
diff --git a/internal/provision/testdata/plan14-root-reads-self-serve-moves-1.tsv b/internal/provision/testdata/plan14-root-reads-self-serve-moves-1.tsv| index b47b2eb43..892349ddf 100644 |
| --- a/internal/provision/testdata/plan14-root-reads-self-serve-moves-1.tsv |
| +++ b/internal/provision/testdata/plan14-root-reads-self-serve-moves-1.tsv |
| @@ -45,7 +45,7 @@ internal/console/store_gate.go claimSQLCensus legacy accepted-legacy accepted-le |
| @remove internal/console/tempstore.go Store.CreateTemporaryAppInRegion |
| @removed internal/console/tempstore.go Store.CreateTemporaryAppInRegion H3 census-ruling-PROPOSED-self-serve-moves-1;rev3=PROPOSED-self-serve-moves-1;basis=constructor remains declared but has no direct read; app insertion and lifetime minting moved to insertAppRow, classified above; proposed removal only |
| # Task 2 envelopes, rev3=PROPOSED-self-serve-moves-1: internal/console covers the lifetime store, app constructors, lifetime projection and table-scoped gate labels; internal/provision (above) adds only ErrLifetimeExhausted beyond Task 1. Registry fields are unchanged. |
| -@coverage internal/console 33115e588cea10142e02514e086040a7d1d4505e654dab760bbff38e885c3e61 |
| +@coverage internal/console 7b2a80cf76ef16487ddb81ed28dfe079c11630b2c4d40227e848cb4fe6292715 |
| # Task 6a: rev3=PROPOSED-self-serve-moves-1; reviewed provision envelope now includes bounded record-then-slot observations, judged read/list/update gates and signed-counter exhaustion at slot read. No removal-loop, host or console change; registry fields and the legacy seal stand. |
| internal/provision/rootrecord.go IndexRecords authority B3,C rev3=PROPOSED-self-serve-moves-1;basis=retain every discovered candidate owner key per host for fresh lifetime judgement at Find; partial discovery still retries and no verdict or body is cached e44ac431865bf38baa4105f66c9cb23014cc3c7ffcd41d5385244cc76fc4c0b9 |
| internal/provision/rootrecord.go RecordIndex.Find authority B3,C rev3=PROPOSED-self-serve-moves-1;basis=observe each candidate through the context judge, refuse corruption and read failures, require one valid owner and validate its schema and shape; nil-judge duplicates retain permission refusal 760eed59cbd6a8ddab1098ce3a5f7a4272ecda81c17cc079dc135aba3a85a8f8 |
| @@ -193,3 +193,4 @@ internal/host/edgesfile.go edgesFileWriteClaims authority F,J1 rev3=PROPOSED-sel |
| @legacy 5ca6674843515dd4b3587ac65b3d8d380f9aa1bfa0a7d5b50b02f01ad18e90c2 internal/host/edgesfile.go edgesFileLoop legacy accepted-legacy accepted-legacy;rev3=PROPOSED-self-serve-moves-1;basis=the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos): census-ruling-moves-offline's body, which also takes the box's lifetimes switch (runAgent's boxJudge != nil) and hands it to every edgesFileWriteClaims pass; read site unchanged, offset moved; a report, not a decision; self-serve moves 1 Task 8 file:Read@2757 65c1d42c39f7f69035ccc6c2a6ce92f155f26d7c7af4b82b4633e215557615e7 |
| internal/host/edgesfile.go edgesFileLoop legacy accepted-legacy accepted-legacy;rev3=PROPOSED-self-serve-moves-1;basis=the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos): census-ruling-moves-offline's body, which also takes the box's lifetimes switch (runAgent's boxJudge != nil) and hands it to every edgesFileWriteClaims pass; read site unchanged, offset moved; a report, not a decision; self-serve moves 1 Task 8 file:Read@2757 65c1d42c39f7f69035ccc6c2a6ce92f155f26d7c7af4b82b4633e215557615e7 |
| # Task 9 fix round 1 (a delayed create whose replay already committed answers success), rev3=PROPOSED-self-serve-moves-1: allocRecordLocked asks committedElsewhere when its SQLite commit is refused (one SQLite read of its own allocation row, one judged record read through readRecord; no write). No root read, class, legacy row or seal changes; the internal/console envelope is refreshed in place above. |
| +# Final review fix round, rev3=PROPOSED-self-serve-moves-1: F1 makes the cutover door run the box floor unless a validated marker was read (no new read site; the floor reads are the door's as before). No root read, class, legacy row or seal changes; the internal/console envelope is refreshed in place above. |