| 1 | # Census: parallel mirror rounds and the offline move's stage door |
| 2 | |
| 3 | Status: RULED. census-ruling-moves-offline-precopy, by the coordinator as census |
| 4 | owner for this branch, 2026-10-21: accepted as proposed. Every row carries |
| 5 | `rev3=census-ruling-moves-offline-precopy`. |
| 6 | |
| 7 | The branch is `moves-offline-precopy`, from main 3d444a1b. It was ruled at |
| 8 | 21cd69a4 (after Astra's round 1 fixes, 692d41cf), then re-ruled at 981294cf |
| 9 | (round 2, 0a6bb375) merged with main f550afa2 (see "Round 2 and the merge of |
| 10 | main" below). It does two things. It runs a mirror |
| 11 | round's copies and deletes side by side in `lease.MirrorPrefixKeep`. And it |
| 12 | adds `carlos move offline stage`, which pre-copies the replica while the |
| 13 | instance keeps serving, as a new record state, offline-staging. |
| 14 | |
| 15 | The ruling adds: |
| 16 | |
| 17 | - `internal/provision/testdata/plan14-root-reads-moves-offline-precopy.tsv`, |
| 18 | loaded last in `TestPlan14RootReadCensus`, after design-plans-and-caps, once |
| 19 | design-plans-and-caps' seal has been checked; |
| 20 | - `internal/provision/moves_offline_precopy_census_test.go`, the record-write |
| 21 | (9B8) overlay on design-plans-and-caps' inventory, which |
| 22 | `TestPlan9B8RecordWriteCensus` now reads; |
| 23 | - one line in `internal/console/testdata/claim-routes.txt`. |
| 24 | |
| 25 | No earlier overlay is edited. |
| 26 | |
| 27 | ## The footing |
| 28 | |
| 29 | Every unit here runs in the box agent (`/opt/carlos/carlos agent`, |
| 30 | carlos-edge.service, `User=carlos`) or the console (cmd/console serve, user |
| 31 | carlos-console). None makes a root decision. So each stays accepted legacy at |
| 32 | the disposition census-ruling-moves-offline gave it or its sibling: J1 |
| 33 | (root-route) for registry and route reads, J4 (root-input) for file reads. |
| 34 | |
| 35 | ## Root-read census |
| 36 | |
| 37 | | unit | change | executing identity | disposition | |
| 38 | |---|---|---|---| |
| 39 | | `internal/host/movesoffline.go` `movesWorker.offlineSourceRevert` | changed legacy: calls `offlineUnstage` before it reports reverted. Read sites unchanged; `whole:Route@701` moved to `@843` (the same `rt` read, shifted by the inserted lines) | the agent, on abort, through the armed moves worker | re-frozen, J1 | |
| 40 | | `internal/provision/offline_move.go` `offlineBegin` | new: OfflineStart's body moved unchanged, with the state as a parameter. Its one site, `file:Open@198`, is `rec.ActiveRun.Open()`, i.e. `RunState.Open()` (queued, picked or cancel), a method, not a file read | the console, through `SetRecordMove` | accepted legacy, J4 (as OfflineStart) | |
| 41 | | `internal/provision/offline_move.go` `OfflineStart` | retired: still declared, reads nothing. It calls `offlineBegin(…, OfflineStopping)` and is behaviour-identical to main: same refusals in the same order, same stamps, and the `StagedAt` stamp fires only for staging | the console | H3 census-ruling removal (the carlos-run `getJSON` form) | |
| 42 | |
| 43 | ### What offlineUnstage reads and writes |
| 44 | |
| 45 | It does nothing unless the record carries `staged_at` and this box is the |
| 46 | stager the stage door pinned on the record (`move.stager`). On that one box it |
| 47 | reads only the object store: |
| 48 | |
| 49 | - its own stager report (`OfflineReportKey`); |
| 50 | - this box's receipts object. |
| 51 | |
| 52 | It then: |
| 53 | |
| 54 | - deletes `moves/<host>/` in the staging bucket the stage door fixed on the |
| 55 | record (`move.stage_bucket`), through a mirror for `move.stage_region` |
| 56 | (round 2). It refuses a bare `moves/`, a missing bucket and this box's own |
| 57 | bucket, and in those cases acknowledges nothing. `offlineSealRevert` |
| 58 | already deletes the same prefix the same way; |
| 59 | - clears `move-receipts/<host>.json` in `cfg.OwnBucket`, the same bucket the |
| 60 | stage round writes them to (`tombstoneAt`); |
| 61 | - writes its stager report as reverted. |
| 62 | |
| 63 | It reads no local file. The host comes from the record. The stored bucket and |
| 64 | region come from the target box's edges file at stage time, written to the |
| 65 | record by the console through `SetRecordMove`. The operator names only a |
| 66 | region code, and the tenant controls none of these paths. |
| 67 | |
| 68 | ### Envelopes re-pinned (at 21cd69a4; superseded by the round 2 table below) |
| 69 | |
| 70 | None of these changes adds a read site. Every changed production file was |
| 71 | read for this ruling. |
| 72 | |
| 73 | | envelope | what changed | hash | |
| 74 | |---|---|---| |
| 75 | | `.` | move.go: the stage subcommand and its status and abort lines; no os, file, exec or env use | `44fc0f3e…8bdd` | |
| 76 | | `internal/console` | the stage door; `offlineBeginChecks`, shared by start and stage; the stager pin; the finish gate on the stager's reverted report; the status view | `6ae90250…92d6` | |
| 77 | | `internal/host` | `movesoffline_stage.go`, new: reads only the object store, and imports `os` only for `os.ErrNotExist`. `edgesfile.go` publishes `offline-stage-1`. offlineRound has a staging branch | `e9234db8…8f47` | |
| 78 | | `internal/lease` | mirror.go: `forEachParallel`; copies, then deletes, side by side; the receipts book read before the workers start | `980f38c4…62a14` | |
| 79 | | `internal/moves` | the stager role, the staged phase, the report's progress fields, `OfflineStager` | `33c7b507…e071` | |
| 80 | | `internal/provision` | `OfflineStage` (which needs a stager), `OfflineRestage`, `OfflineStartStaged`; `MoveState.StagedAt` and `MoveState.Stager` | `8c868695…4b64` | |
| 81 | |
| 82 | ## Record-write census (9B8) |
| 83 | |
| 84 | Two sites change, each keeping main's class. No site is added. |
| 85 | |
| 86 | | file:function | kinds | class | why | |
| 87 | |---|---|---|---| |
| 88 | | `internal/console/authcli.go:Server.registerCLIRoutes` | lit:instances,lit:sidecars | key-site (main's) | one route line for the stage door, beside the six offline doors; no write added. Digest `2aee959d…c459` | |
| 89 | | `internal/lease/mirror.go:MirrorPrefixKeep` | sel:CopyVersioned | versioned-transport (main's) | see below. Digest `dff4d03f…fb19` | |
| 90 | |
| 91 | MirrorPrefixKeep plans its skips in key order first. It then runs the copies |
| 92 | on up to `DefaultMirrorWorkers` (8) goroutines and, only once every copy has |
| 93 | landed, the deletes. Its writes are the same as before: |
| 94 | |
| 95 | - Copy or CopyVersioned into the destination prefix; |
| 96 | - Delete of destination keys the source no longer has; |
| 97 | - receipts through the book (`set`, `drop`, `save`). Every update is under one |
| 98 | mutex, and `receiptLedger.save` is still the one Put, unchanged. |
| 99 | |
| 100 | After round 1, the book is read once, before the workers start, whenever the |
| 101 | round has copies to make. So `set` and `drop` touch memory only, and no |
| 102 | request is made under the mutex. |
| 103 | |
| 104 | The stager pin is a new field on `MoveState`. It is written through the typed |
| 105 | `provision.SetRecordMove` (`offlineApply`), like every other move field, under |
| 106 | the same one-move lock. That is not a new write site. |
| 107 | |
| 108 | ## Claim route census |
| 109 | |
| 110 | `POST /api/cli/apps/{account}/{app}/instances/{host}/move/offline/stage` joins |
| 111 | `claim-routes.txt`. |
| 112 | |
| 113 | - **Access.** The door is operator-only: `offlineGate` calls |
| 114 | `moveOperatorGate`, and any token that is not an operator's gets a 404. |
| 115 | - **Claims consoles.** With host claims or Phase 1 on, it is refused with |
| 116 | MV-O20. |
| 117 | - **Tests.** Its claim-refusal fixture and its `svcroutes_test.go` exemption |
| 118 | match the six sibling offline doors exactly. |
| 119 | |
| 120 | ## Round 2 and the merge of main |
| 121 | |
| 122 | ### The merge (d5b020ba) |
| 123 | |
| 124 | Main (f550afa2) added two overlays, run-lifecycle-logs-325 and |
| 125 | design-plans-and-caps, and a 9B8 inventory, `plansCapsWriteCensusInventory`. |
| 126 | They touch none of this branch's units. The merge keeps both of main's |
| 127 | overlays and loads this branch's last, after design-plans-and-caps, whose seal |
| 128 | (`4f5e0ed6…df17`) is checked first. This branch's 9B8 overlay now sits on |
| 129 | `plansCapsWriteCensusInventory`. |
| 130 | |
| 131 | Three envelopes (`.`, internal/console, internal/host) and registerCLIRoutes |
| 132 | are changed by both sides. Because this branch's overlay loads last, it pins |
| 133 | them on the merged tree. |
| 134 | |
| 135 | ### Round 2 (0a6bb375): what changed, and the census reading |
| 136 | |
| 137 | - **The stage door fixes the staging destination on the record:** |
| 138 | `MoveState.StageBucket` and `StageRegion`, from the one target box's edges |
| 139 | file. The door refuses a source box's own bucket (MV-O191). |
| 140 | `provision.OfflineStage` takes a `StageTarget` and refuses an empty stager |
| 141 | or bucket. These are record fields written through the typed |
| 142 | `SetRecordMove` (`offlineBegin`, `offlineApply`), so there is no new write |
| 143 | site. |
| 144 | - **The stager re-reads the record** before a round, and once before its |
| 145 | delete phase (`stageDeleteGate`). `provision.GetRecord` is an object-store |
| 146 | read, not a local file, and the scan finds no root-read unit in |
| 147 | `movesoffline_stage.go`. A move that has left staging starts no round and |
| 148 | deletes nothing. |
| 149 | - **The stager acknowledges it has stopped** (`offlineStagerStopped`). It |
| 150 | writes its own stager report through `writeOfflineReport`, an existing |
| 151 | nonrecord site, and is called from `offlineSourceStep` in stopping through |
| 152 | landed. |
| 153 | - **offlineUnstage deletes from the stored destination:** `moves/<host>/` in |
| 154 | `move.stage_bucket`, through a mirror for `move.stage_region`. It refuses, |
| 155 | and acknowledges nothing, when no bucket is stored or the stored bucket is |
| 156 | this box's own. It reads only the object store. |
| 157 | - **offlineTakeSeal** gains one refusal before its mirror: a staged move |
| 158 | seals only into the stored bucket and region. This is a comparison of |
| 159 | record and edges-file fields. Its 16 read sites are unchanged, each shifted |
| 160 | by 487 bytes. Its Puts (nonrecord, under the move's `moves/@offline/` |
| 161 | prefix) are unchanged. |
| 162 | - **The console:** |
| 163 | - Start and seal refuse when the pinned stager is missing from the fleet |
| 164 | (MV-O190), and stage, start and seal refuse changed coordinates (MV-O192). |
| 165 | - Seal waits for a fresh acknowledgement from the stager (M-C10, or MV-O178 |
| 166 | when it is stale). |
| 167 | - These checks read only the fleet and the report evidence the doors |
| 168 | already read. |
| 169 | - **internal/edgesfile:** `PlacementView` gains `DataBucketRegion`, a JSON |
| 170 | field. No read site. |
| 171 | |
| 172 | ### Rows and sites changed by round 2 and the merge |
| 173 | |
| 174 | | census | unit or envelope | ruling | |
| 175 | |---|---|---| |
| 176 | | Plan 14 | `internal/host/movesoffline_seal.go` `movesWorker.offlineTakeSeal` | re-frozen at J1, census-ruling-moves-offline's disposition; digest `f606873f…cdaa` | |
| 177 | | Plan 14 | `movesWorker.offlineSourceRevert` | basis updated: offlineUnstage now uses the stored destination; digest and sites unchanged | |
| 178 | | Plan 14 | envelopes `.`, internal/console, internal/edgesfile, internal/host, internal/provision | re-pinned on the merged tree; internal/lease and internal/moves unchanged since round 1 | |
| 179 | | 9B8 | `movesWorker.offlineTakeSeal` | nonrecord (census-ruling-moves-offline's), sel:Put; digest `f606873f…cdaa` | |
| 180 | | 9B8 | `Server.registerCLIRoutes` | key-site, pinned on the merged tree; digest `a0aff01b…2e87` | |
| 181 | |
| 182 | | envelope | hash | |
| 183 | |---|---| |
| 184 | | `.` | `649bb6a5…eabd` | |
| 185 | | `internal/console` | `7bdd3caa…81c9` | |
| 186 | | `internal/edgesfile` | `71692438…811e` | |
| 187 | | `internal/host` | `d58c8e60…e219` | |
| 188 | | `internal/lease` | `980f38c4…62a14` | |
| 189 | | `internal/moves` | `33c7b507…e071` | |
| 190 | | `internal/provision` | `210b1740…34dc` | |
| 191 | |
| 192 | ## The seal |
| 193 | |
| 194 | Three accepted-legacy rows add frozen declarations (offlineSourceRevert, |
| 195 | offlineBegin and offlineTakeSeal), so the legacy seal moves from |
| 196 | `4f5e0ed6…df17` (main, after design-plans-and-caps) to |
| 197 | `940ed1bba8d911e0f57af83ecac253c77ba5e14dad590258e0ee03ec98ccb818`. A removal |
| 198 | is not in the seal. The test checks design-plans-and-caps' seal before it |
| 199 | loads this overlay. Each seal was read from a temporary `t.Logf` of |
| 200 | `rootReadLegacySeal(inv)`, which was then removed. |
| 201 | |
| 202 | Earlier seals of this overlay: |
| 203 | |
| 204 | - `965fda6e…c81`, on its own after scan-demand-revival; |
| 205 | - `8ef09f41…1a4d`, after the merge, before round 2. |
| 206 | |
| 207 | ## Ruling (the coordinator, 2026-10-21): census-ruling-moves-offline-precopy, ACCEPTED as proposed |
| 208 | |
| 209 | 1. offlineSourceRevert and offlineUnstage: ACCEPTED. Object-store reads only, |
| 210 | on the pinned stager only, deleting only in the staging destination fixed |
| 211 | on the record; there is no root read of an operator- or tenant-controlled |
| 212 | path. |
| 213 | 2. offlineBegin: ACCEPTED (J4, as OfflineStart). OfflineStart is retired by |
| 214 | the H3 removal, since it is behaviour-identical. |
| 215 | 3. offlineBegin's `file:Open@198` is `rec.ActiveRun.Open()`: ACCEPTED. |
| 216 | 4. The envelopes: ACCEPTED, re-pinned on the merged tree (981294cf with main |
| 217 | f550afa2), internal/edgesfile included. |
| 218 | 5. MirrorPrefixKeep: ACCEPTED at versioned-transport. Its record writes still |
| 219 | go through the book. |
| 220 | 6. registerCLIRoutes: ACCEPTED at key-site. |
| 221 | 7. The stage route: ACCEPTED. It is operator-only, refused with MV-O20 on |
| 222 | claims consoles, and has its fixture. |
| 223 | |
| 224 | 8. offlineTakeSeal (round 2): ACCEPTED, re-frozen at J1 and nonrecord in |
| 225 | 9B8. It adds one refusal and no read or write. |
| 226 | 9. The round 2 record fields and reads (`StageBucket`, `StageRegion`, the |
| 227 | record re-reads, the stop acknowledgement): ACCEPTED. The fields go through |
| 228 | `SetRecordMove`, the reads go to the object store, and the report write |
| 229 | goes through `writeOfflineReport`. |
| 230 | 10. The merge with main f550afa2: ACCEPTED. Both overlays are kept, this |
| 231 | branch's is loaded last, and the seal is recomputed. |
| 232 | |
| 233 | The first verification was at 25c0db26. It was re-verified at 21cd69a4 for the |
| 234 | round 1 fixes (the stager pin, cleanup by `staged_at`, the receipts preload), |
| 235 | then at 981294cf merged with main f550afa2 for the round 2 fixes (the stager |
| 236 | in the seal barrier, the fixed staging destination, the record re-reads). |
| 237 | |
amadan