amadan
console: the migration pass slows down on attempts, not candidates (final review F5)
A candidate skipped under its locks (admission refused by a pending operation, or no longer eligible) still counted, so one blocked record kept the full judged walk on the 60-second cadence. The pass now returns attempted. Census: migratePreLifetimeRecords re-pinned in place and the internal/console envelope refreshed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
5 files changed,
+40
−14
docs/superpowers/notes/census-proposal-self-serve-moves-1.md+1 −1internal/console/lifetimes_migrate.go+15 −11internal/console/lifetimes_migrate_test.go+18 −0internal/provision/self_serve_moves1_census_test.go+4 −1internal/provision/testdata/plan14-root-reads-self-serve-moves-1.tsv+2 −1
diff --git a/docs/superpowers/notes/census-proposal-self-serve-moves-1.md b/docs/superpowers/notes/census-proposal-self-serve-moves-1.md| index f271cc8ec..f8b10a9b2 100644 |
| --- a/docs/superpowers/notes/census-proposal-self-serve-moves-1.md |
| +++ b/docs/superpowers/notes/census-proposal-self-serve-moves-1.md |
| @@ -42,6 +42,6 @@ Task 9 adds the plan's end-to-end witnesses, in `internal/host/movesoffline_life |
| Task 9 fix round 1 corrects Task 3's create door. When a delayed create's SQLite commit is refused because recovery already committed its allocation, `Server.allocRecordLocked` now asks `Server.committedElsewhere`. That function reads the allocation row and the record through `readRecord` (judged) and writes nothing. When the row is committed and the record is valid at the allocation's incarnation, the door answers success; anything else stays pending. `Server.allocRecordLocked` stays a record-writer through `WriteRecord` and is re-pinned in place in the record-write inventory. The `internal/console` envelope is refreshed in place. No root read, class, legacy row or seal changes, and `TestNoUnjudgedRecordReads` stays green. |
| -Final review fix round. F1 changes the cutover door: it skips the box floor only once a validated marker body has been read or written by the door, never on the gate's state alone. A failed marker read still closes admission. `Server.handleAPILifetimesCutover` stays nonrecord and is re-pinned in place in the record-write inventory. F2 narrows the create's orphan rule: a live slot of another owner is retired only when no record exists under that owner's key, and a present body of any verdict refuses the create and is left untouched. `Server.planCreateLocked` is not a census unit, so only the envelope moves. F3 judges the catch-all label counts. `internal/domains` cannot import `internal/provision`, so it declares an `InstanceObserver` interface and its own context key, and `provision.WithJudge` records the same judge there (nil records none). `domains.InstanceLabelCountsJudged` counts only what the judge vouches for and reports a label with an unjudgeable record as unknown; with no observer it is `InstanceLabelCounts`. `ReconcileAliases` and `deriveInstanceAliases` (accepted legacy, rev3=J1) are re-frozen with reviewed bases: a stale body no longer makes a label shared, and an unknown label keeps its existing alias row and derives none, which adds four read sites on that row (`registry:Account`, `registry:Alias`, `registry:App` and the whole `RouteAlias`). That moves the seal. In the record-write inventory, `InstanceLabelCounts` leaves and `instanceHosts`, the key listing it now shares with the judged counter, takes its place as a key-site. `Server.catchallInstanceName` counts through the judged counter with the request's judge. The `internal/domains` and `internal/provision` envelopes are refreshed in place. The `internal/console` envelope is refreshed in place after each finding. No root read, class or task is added. |
| +Final review fix round. F1 changes the cutover door: it skips the box floor only once a validated marker body has been read or written by the door, never on the gate's state alone. A failed marker read still closes admission. `Server.handleAPILifetimesCutover` stays nonrecord and is re-pinned in place in the record-write inventory. F2 narrows the create's orphan rule: a live slot of another owner is retired only when no record exists under that owner's key, and a present body of any verdict refuses the create and is left untouched. `Server.planCreateLocked` is not a census unit, so only the envelope moves. F3 judges the catch-all label counts. `internal/domains` cannot import `internal/provision`, so it declares an `InstanceObserver` interface and its own context key, and `provision.WithJudge` records the same judge there (nil records none). `domains.InstanceLabelCountsJudged` counts only what the judge vouches for and reports a label with an unjudgeable record as unknown; with no observer it is `InstanceLabelCounts`. `ReconcileAliases` and `deriveInstanceAliases` (accepted legacy, rev3=J1) are re-frozen with reviewed bases: a stale body no longer makes a label shared, and an unknown label keeps its existing alias row and derives none, which adds four read sites on that row (`registry:Account`, `registry:Alias`, `registry:App` and the whole `RouteAlias`). That moves the seal. In the record-write inventory, `InstanceLabelCounts` leaves and `instanceHosts`, the key listing it now shares with the judged counter, takes its place as a key-site. `Server.catchallInstanceName` counts through the judged counter with the request's judge. The `internal/domains` and `internal/provision` envelopes are refreshed in place. F5 makes the migration pass report the migrations it attempted rather than the candidates it listed, so a candidate a pending operation blocks leaves the pass on its ten-minute idle cadence. `Server.migratePreLifetimeRecords` stays a key-site and is re-pinned in place. The `internal/console` envelope is refreshed in place after each finding. No root read, class or task is added. |
| Taken together, the rows marked `PROPOSED-self-serve-moves-1` are the census of one change. Claims-off instance records gain a lifetime: the host incarnation slot, written only through `WriteHostIncarnation` by the console's create, retirement and migration protocols. Every claims-off consumer judges a record against its slot before it acts on it. Every record write stays on `WriteRecord` with the sequence discipline. The per-host `move_seq` and each row's `(incarnation, serving_seq)` pair order the publications an edge folds. Each row is one of five kinds: a write site added or re-pinned in the record-write inventory (each through `WriteRecord`, a `DeleteIfMatch` on an observed ETag, or a Put of a control key that is not a record); a read made judged at its existing class; a console SQLite unit or a legacy fingerprint with a reviewed basis; the `@fields` schema row, which pins the registry fields Task 7 adds (`Route.Incarnation`, `Route.ServingSeq`, and the remote request's `Pair` and `ExpectPair`); or a source envelope refreshed over the new code. No row adds a raw record write, a root-read class or a task. The two registry read sites added in Task 8 are named in its paragraph. These rows are one proposal, for Paul to rule on once, together, after the plan's final review. Until then every row stays PROPOSED, and a green census means only that the rows match the code, not that they are accepted. |
diff --git a/internal/console/lifetimes_migrate.go b/internal/console/lifetimes_migrate.go| index f300f6151..100197505 100644 |
| --- a/internal/console/lifetimes_migrate.go |
| +++ b/internal/console/lifetimes_migrate.go |
| @@ -400,8 +400,8 @@ func (s *Server) StartIncarnationMigrationPass(ctx context.Context) { |
| // Task 0's worker lifecycle, paced: it never returns because of the |
| // gate's state, and each iteration admits for itself. |
| for { |
| - ran, candidates := s.incarnationMigrationPass(ctx) |
| - t := time.NewTimer(migrationPassInterval(ran, candidates)) |
| + ran, attempted := s.incarnationMigrationPass(ctx) |
| + t := time.NewTimer(migrationPassInterval(ran, attempted)) |
| select { |
| case <-ctx.Done(): |
| t.Stop() |
| @@ -412,9 +412,12 @@ func (s *Server) StartIncarnationMigrationPass(ctx context.Context) { |
| } |
| // migrationPassInterval is the wait before the next pass: ten minutes after |
| -// a pass that ran and found no candidate, 60 seconds otherwise. |
| -func migrationPassInterval(ran bool, candidates int) time.Duration { |
| - if ran && candidates == 0 { |
| +// a pass that ran and attempted no migration, 60 seconds otherwise. A |
| +// candidate the pass skipped (refused admission, or no longer eligible) is |
| +// not an attempt, so a candidate a pending operation blocks does not keep |
| +// the full judged walk on the short cadence. |
| +func migrationPassInterval(ran bool, attempted int) time.Duration { |
| + if ran && attempted == 0 { |
| return lifetimesMigrateIdleInterval |
| } |
| return lifetimesMigrateInterval |
| @@ -428,25 +431,26 @@ func (s *Server) incarnationMigrationIteration(ctx context.Context) bool { |
| } |
| // incarnationMigrationPass is incarnationMigrationIteration with the number |
| -// of candidates the pass found. |
| -func (s *Server) incarnationMigrationPass(ctx context.Context) (ran bool, candidates int) { |
| +// of migrations the pass attempted. |
| +func (s *Server) incarnationMigrationPass(ctx context.Context) (ran bool, attempted int) { |
| if !s.cfg.LifetimesMigrate { |
| return false, 0 |
| } |
| ran = s.lifetimesIteration(ctx, "incarnation migration", true, true, func(ctx context.Context) { |
| var err error |
| - if candidates, err = s.migratePreLifetimeRecords(ctx); err != nil { |
| + if attempted, err = s.migratePreLifetimeRecords(ctx); err != nil { |
| log.Printf("console: incarnation migration pass: %v", err) |
| } |
| }) |
| - return ran, candidates |
| + return ran, attempted |
| } |
| // migratePreLifetimeRecords is one pass: at most migrationPassLimit records |
| // with no lifetime yet (fieldless, or carrying a marker whose operation is |
| // gone), oldest first, each under s.lockApp then the hostname lock, |
| // admitted, and migrated unbound only while it still has no lifetime and no |
| -// slot for its hostname. It returns how many candidates it found. |
| +// slot for its hostname. It returns how many migrations it attempted: a |
| +// candidate skipped under its locks is not one. |
| func (s *Server) migratePreLifetimeRecords(ctx context.Context) (int, error) { |
| recs, _, err := provision.ListAllRecordsTolerantJudged(ctx, s.cfg.Releases, s.judge) |
| if err != nil { |
| @@ -486,5 +490,5 @@ func (s *Server) migratePreLifetimeRecords(ctx context.Context) (int, error) { |
| return nil |
| }) |
| } |
| - return len(candidates), nil |
| + return attempted, nil |
| } |
diff --git a/internal/console/lifetimes_migrate_test.go b/internal/console/lifetimes_migrate_test.go| index 3c0c7d1c0..567354937 100644 |
| --- a/internal/console/lifetimes_migrate_test.go |
| +++ b/internal/console/lifetimes_migrate_test.go |
| @@ -1058,6 +1058,24 @@ func TestMigrationPassSlowsDownWhenIdle(t *testing.T) { |
| if ran, n := l.fx.srv.incarnationMigrationPass(context.Background()); !ran || n != 0 || migrationPassInterval(ran, n) != lifetimesMigrateIdleInterval { |
| t.Fatalf("second pass = %v %d, want none and the idle cadence", ran, n) |
| } |
| + |
| + // A listed candidate that a durable pending operation blocks is not an |
| + // attempt: the pass is idle and slows down (final review F5). |
| + b := newLtOn(t) |
| + b.fx.srv.cfg.LifetimesMigrate = true |
| + etag := legacyBody(t, b.fx.rel, ltApp, ltOld, "stable") |
| + if err := b.fx.st.BeginAllocation(Allocation{OpID: "other", Account: "bab", App: ltApp, Host: ltOld, Incarnation: 1, Body: []byte(`{}`)}); err != nil { |
| + t.Fatal(err) |
| + } |
| + for i := 0; i < 2; i++ { |
| + ran, n := b.fx.srv.incarnationMigrationPass(context.Background()) |
| + if !ran || n != 0 || migrationPassInterval(ran, n) != lifetimesMigrateIdleInterval { |
| + t.Fatalf("pass %d over a blocked candidate = %v %d, want no attempt and the idle cadence", i+1, ran, n) |
| + } |
| + } |
| + if _, now := rawOf(t, b.fx.rel, ltApp, ltOld); now != etag { |
| + t.Fatal("the blocked candidate's record changed") |
| + } |
| } |
| func TestMigrationFenceReadsOneMarkerAcrossSpellings(t *testing.T) { |
diff --git a/internal/provision/self_serve_moves1_census_test.go b/internal/provision/self_serve_moves1_census_test.go| index 1db6e0dd8..6caf30e9a 100644 |
| --- a/internal/provision/self_serve_moves1_census_test.go |
| +++ b/internal/provision/self_serve_moves1_census_test.go |
| @@ -238,6 +238,9 @@ package provision |
| // InstanceLabelCountsJudged share. key-site as before: it lists keys and |
| // writes nothing. The judged counter observes each key through the |
| // context's judge (provision's ObserveInstance) and writes nothing. |
| +// - F5: lifetimes_migrate.go:Server.migratePreLifetimeRecords: key-site as |
| +// before, re-pinned in place; it returns the migrations it attempted |
| +// rather than the candidates it listed. No write site changes. |
| func plan14SelfServeMoves1WriteCensusInventory() []censusEntry { |
| removed := []struct{ file, unit string }{ |
| {"internal/provision/provision.go", "GetRecordETag"}, |
| @@ -314,7 +317,7 @@ func plan14SelfServeMoves1WriteCensusInventory() []censusEntry { |
| {"internal/console/lifetimes_migrate.go", "Server.migrationMarkLocked", "record-writer", "key:RecordKey,key:WriteRecord", "337a2818254cda934193630720afe01ece59cf7ba64b94ec5a214f38c02f4b8f"}, |
| {"internal/console/lifetimes_migrate.go", "Server.migrationFinishLocked", "record-writer", "key:RecordKey,key:WriteRecord", "6729c5e00f700ccce6432586baa32859da0d3d03079b9fff4c0ddf874fdf6fbb"}, |
| {"internal/console/lifetimes_migrate.go", "Server.migrationStepsLocked", "key-site", "key:RecordKey", "094a3316195e4144a71c35a2c25392d5f71f6d4b22aeb6803ed852b93ff4cca6"}, |
| - {"internal/console/lifetimes_migrate.go", "Server.migratePreLifetimeRecords", "key-site", "key:RecordKey", "815f90386e35dc4d99892b6593dd624262391986245c6ae8ba4980ee57ba62cf"}, |
| + {"internal/console/lifetimes_migrate.go", "Server.migratePreLifetimeRecords", "key-site", "key:RecordKey", "bd57e53f1d49b05d5cb4aba43fa3822f1a3e89dd74d014f42b354ee599843aea"}, |
| {"internal/console/lifetimes_migrate.go", "Server.quarantineCopy", "nonrecord", "sel:Put", "d4b7f4a8729b3e51789b5a53d8dd6cbc735f3a072314eafb02f92969a9818afb"}, |
| {"internal/console/apiinstanceimport.go", "Server.deleteRead", "key-site", "key:RecordKey", "1b9be43c3bff269739301b24da159af790a63bb4dbe255a8affa0eec12fdd1ab"}, |
| {"internal/console/apiinstanceimport.go", "Server.importDeleteRule", "record-writer", "key:SetRecordSeed", "173efd41a392c443cfd12036a15ff72bf61e5a32354bf5c692776d488e992834"}, |
diff --git a/internal/provision/testdata/plan14-root-reads-self-serve-moves-1.tsv b/internal/provision/testdata/plan14-root-reads-self-serve-moves-1.tsv| index 2cd3e0f3c..22c940803 100644 |
| --- a/internal/provision/testdata/plan14-root-reads-self-serve-moves-1.tsv |
| +++ b/internal/provision/testdata/plan14-root-reads-self-serve-moves-1.tsv |
| @@ -45,7 +45,7 @@ internal/console/store_gate.go claimSQLCensus legacy accepted-legacy accepted-le |
| @remove internal/console/tempstore.go Store.CreateTemporaryAppInRegion |
| @removed internal/console/tempstore.go Store.CreateTemporaryAppInRegion H3 census-ruling-PROPOSED-self-serve-moves-1;rev3=PROPOSED-self-serve-moves-1;basis=constructor remains declared but has no direct read; app insertion and lifetime minting moved to insertAppRow, classified above; proposed removal only |
| # Task 2 envelopes, rev3=PROPOSED-self-serve-moves-1: internal/console covers the lifetime store, app constructors, lifetime projection and table-scoped gate labels; internal/provision (above) adds only ErrLifetimeExhausted beyond Task 1. Registry fields are unchanged. |
| -@coverage internal/console 6a7239960c7ef46a0d16e3a0c01df6c74d68ea9f5c93a49bce7dcf45e9ec0b77 |
| +@coverage internal/console bfcd3b07e8252233f13df2a923e0af4012b710cf61e77b0a8f018bc3ba4b002a |
| # Task 6a: rev3=PROPOSED-self-serve-moves-1; reviewed provision envelope now includes bounded record-then-slot observations, judged read/list/update gates and signed-counter exhaustion at slot read. No removal-loop, host or console change; registry fields and the legacy seal stand. |
| internal/provision/rootrecord.go IndexRecords authority B3,C rev3=PROPOSED-self-serve-moves-1;basis=retain every discovered candidate owner key per host for fresh lifetime judgement at Find; partial discovery still retries and no verdict or body is cached e44ac431865bf38baa4105f66c9cb23014cc3c7ffcd41d5385244cc76fc4c0b9 |
| internal/provision/rootrecord.go RecordIndex.Find authority B3,C rev3=PROPOSED-self-serve-moves-1;basis=observe each candidate through the context judge, refuse corruption and read failures, require one valid owner and validate its schema and shape; nil-judge duplicates retain permission refusal 760eed59cbd6a8ddab1098ce3a5f7a4272ecda81c17cc079dc135aba3a85a8f8 |
| @@ -199,3 +199,4 @@ internal/host/edgesfile.go edgesFileLoop legacy accepted-legacy accepted-legacy; |
| internal/domains/aliases.go ReconcileAliases legacy accepted-legacy accepted-legacy;rev3=PROPOSED-self-serve-moves-1;basis=root-route; the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos), from its fast-converge and instances loops: rev3=J1's body, which now counts the catch-all's labels through domains.InstanceLabelCountsJudged with the pass context's observer (provision.WithLifetimes on a claims-off box; none on a claims-on box, which is today's key count); registry read sites unchanged, offsets moved registry:Account@15249,registry:Account@15262,registry:Alias@14864,registry:Alias@14904,registry:Alias@14974,registry:App@11952,registry:App@12152,registry:App@12308,registry:App@12509,registry:App@15275,registry:App@15284,registry:App@2026,registry:App@2222,registry:App@5823,registry:App@6022,registry:Host@15234,registry:Host@15523,registry:Host@15576,registry:Host@15648,registry:Host@16160,registry:Host@16252,registry:Host@16311,registry:Host@16392,registry:Host@406,registry:Prefix@16168,registry:Prefix@16266,registry:Prefix@16319,registry:Prefix@16400,registry:Status@8686,whole:Route@11921,whole:Route@11996,whole:Route@12143,whole:Route@12205,whole:Route@12348,whole:Route@12504,whole:Route@13400,whole:Route@1369,whole:Route@14469,whole:Route@14477,whole:Route@2066,whole:Route@2217,whole:Route@2273,whole:Route@359,whole:Route@388,whole:Route@399,whole:Route@417,whole:Route@5169,whole:Route@5867,whole:Route@6013,whole:Route@8016,whole:Route@8468,whole:Route@8630,whole:RouteAlias@14485,whole:RouteAlias@14839,whole:RoutePath@10213,whole:RoutePath@16053,whole:RoutePath@16095,whole:RoutePath@16209,whole:RoutePath@16239,whole:WildcardDomain@12720,whole:WildcardDomain@15164,whole:WildcardDomain@15213,whole:WildcardDomain@15376,whole:WildcardDomain@15482,whole:WildcardDomain@15511 f0cb08a90be8c8d6c97c24be3cf73720abb54cb8444793fa25ba982e9525e9f9 |
| @legacy a37ea03438815852c187e4b5e8d83641253f23e8cbd48898abd4e2c72820c65e internal/domains/aliases.go deriveInstanceAliases legacy accepted-legacy accepted-legacy;rev3=PROPOSED-self-serve-moves-1;basis=root-route; the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos): rev3=J1's body; a stale body under a retired lifetime no longer makes a label shared, and a label with a record that cannot be judged keeps its existing alias row of the catch-all's own account and app (reading that row's Account, App and Alias) and derives no new one; a report-only conservative keep, never a new route registry:Account@2163,registry:Account@2950,registry:Account@3913,registry:Alias@2225,registry:Alias@2293,registry:Alias@2337,registry:Alias@3006,registry:Alias@557,registry:App@1686,registry:App@2189,registry:App@2976,registry:App@3939,registry:Host@1843,registry:Host@3892,registry:Host@3902,registry:Host@4098,registry:Provisioned@1636,registry:Replica@1672,registry:Slug@1775,whole:Route@1620,whole:Route@1726,whole:Route@3502,whole:RouteAlias@2143,whole:RouteAlias@2920,whole:RouteAlias@3874,whole:RouteAlias@509,whole:RouteAlias@539,whole:RouteAlias@552,whole:RouteAlias@568 888a5a32dbbcd9619b0bf513c0dc746686ad7c4e5681bf82248022e8f13b2512 |
| internal/domains/aliases.go deriveInstanceAliases legacy accepted-legacy accepted-legacy;rev3=PROPOSED-self-serve-moves-1;basis=root-route; the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos): rev3=J1's body; a stale body under a retired lifetime no longer makes a label shared, and a label with a record that cannot be judged keeps its existing alias row of the catch-all's own account and app (reading that row's Account, App and Alias) and derives no new one; a report-only conservative keep, never a new route registry:Account@2163,registry:Account@2950,registry:Account@3913,registry:Alias@2225,registry:Alias@2293,registry:Alias@2337,registry:Alias@3006,registry:Alias@557,registry:App@1686,registry:App@2189,registry:App@2976,registry:App@3939,registry:Host@1843,registry:Host@3892,registry:Host@3902,registry:Host@4098,registry:Provisioned@1636,registry:Replica@1672,registry:Slug@1775,whole:Route@1620,whole:Route@1726,whole:Route@3502,whole:RouteAlias@2143,whole:RouteAlias@2920,whole:RouteAlias@3874,whole:RouteAlias@509,whole:RouteAlias@539,whole:RouteAlias@552,whole:RouteAlias@568 888a5a32dbbcd9619b0bf513c0dc746686ad7c4e5681bf82248022e8f13b2512 |
| +# Final review fix round F5, rev3=PROPOSED-self-serve-moves-1: the migration pass reports the migrations it attempted, not the candidates it listed, so a blocked candidate leaves it on the idle cadence; no read site added. The internal/console envelope is refreshed in place above; no root read, class, legacy row or seal changes. |