| index d58fd4f54..a782df063 100644 |
| --- a/docs/superpowers/plans/2026-10-06-moves-phase1-1-copy-and-staging.md |
| +++ b/docs/superpowers/plans/2026-10-06-moves-phase1-1-copy-and-staging.md |
| @@ -10,7 +10,7 @@ |
| |
| **Spec:** `docs/superpowers/specs/2026-10-02-move-any-app-design.md` rev 7.22: §3, §4, §6.1, §6.2, §7.1, §7.2, §7.3, §8, §10, §11, §12 (door), §19.2 K2, K5, K12. Index: `2026-10-06-moves-phase1-0-index.md`. |
| |
| -**Plan revision:** rev 6 (after the round-5 Plan 1 review, `CODEX-PLAN1-REVIEW-R5.md`; rev 5 answered round 4; both dispositions in the index). **Build target: `origin/main` at d649d8d4 or later** (claim Plan 6, the box wiring for C1 to C4, merged 2026-10-06; inert while `CARLOS_HOST_CLAIM` is unset). Every API this plan names was checked against `origin/main` at 1c43c2ce and the line numbers re-checked at d649d8d4: `boxchoice.go`'s signatures and the absence of a production caller, `Claim.Validate`'s location grammar, the two system units' direct `carlos.prev` exec, the edge unit's `ProtectSystem=strict`, `edge.go`'s cache-before-activator order, `agentOwnedVars` and `startChild`'s unconditional `ListenControl`, `LoadMergedHost`'s single etag, `objstore.GetIn`, `leasetest`'s create-only `PUT`, `cgroup.Prepared`'s method set, `s3Mirror`'s `Copy`, `stageBlocked`'s error classes (`adopt.go:256`), the existing `LiveVersion` (`versions.go:290`), `decidePrev`'s `.prev version` call, `MirrorPrefix`'s two receipt branches and `GetRecord`'s `*Record`. |
| +**Plan revision:** rev 7 (after the round-6 Plan 1 review, `CODEX-PLAN1-REVIEW-R6.md`; revs 5 and 6 answered rounds 4 and 5; all dispositions in the index). **Build order:** 0, 1, 2, 3, 4, 5, 6, 7, **12**, 7b, 8, 9, 10, 11, 13, 14, 15, 16: Task 12 (the `hold_op` column) is built before Task 7b, whose `heldStageAuthority` reads it (round-6 finding 1); every other task follows its number. **Build target: `origin/main` at d649d8d4 or later** (claim Plan 6, the box wiring for C1 to C4, merged 2026-10-06; inert while `CARLOS_HOST_CLAIM` is unset). Every API this plan names was checked against `origin/main` at 1c43c2ce and the line numbers re-checked at d649d8d4: `boxchoice.go`'s signatures and the absence of a production caller, `Claim.Validate`'s location grammar, the two system units' direct `carlos.prev` exec, the edge unit's `ProtectSystem=strict`, `edge.go`'s cache-before-activator order, `agentOwnedVars` and `startChild`'s unconditional `ListenControl`, `LoadMergedHost`'s single etag, `objstore.GetIn`, `leasetest`'s create-only `PUT`, `cgroup.Prepared`'s method set, `s3Mirror`'s `Copy`, `stageBlocked`'s error classes (`adopt.go:256`), the existing `LiveVersion` (`versions.go:290`), `decidePrev`'s `.prev version` call, `MirrorPrefix`'s two receipt branches and `GetRecord`'s `*Record`. |
| |
| ## Global Constraints |
| |
| @@ -766,7 +766,7 @@ Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>" |
| |
| **Interfaces:** |
| - Consumes: `hostclaim.StartGuard`, `RequireOp`, `KindServing`, `KindMoving`, `Op`, `Touched`, `ErrWrongState`, `ErrMalformed`. |
| -- Produces: `hostclaim.BeginMove(opID, by, sourceBox, targetBox, targetRegion string, now time.Time) func(*Claim) error`; `hostclaim.Touch(opID string, regions, boxes []string) func(*Claim) error`; `hostclaim.RequireMoveOp(opID, kind, phase string) func(*Claim) error`; phase constants `PhaseMoving = ""`, `PhaseSealed = "sealed"`; `hostclaim.ReleaseMove(opID, execBox string) func(*Claim) error` (K8, in this plan so the interim abort can release: legal from `moving` with phase `""` or `"aborting"` when no obligation under the op remains, sets `Op = Op{Kind: KindServing, Exec: execBox, Journal: map[string]string{"released_move": opID}}`: the serving op remembers which move released it, **operation-bound completion evidence** for the interim abort's crash boundary (round-5 finding 10; `Claim.Validate` accepts an op journal on any kind, `claim.go:90`; if it does not for `serving`, widen it in this task); `hostclaim.RequireReleasedMove(opID string) func(*Claim) error` (nil when the claim is `serving` and `Op.Journal["released_move"] == opID`, else `ErrOpMismatch`), the replay rule Task 14 uses; `TestReleaseMoveLeavesEvidence` (after `ReleaseMove`, `RequireReleasedMove(id)` holds and `RequireReleasedMove(other)` does not; a plain serving claim satisfies neither); Plan 4 Task 2 widens it to `landing/committed`, `restoring/opened`, `reverting/opened`). Test `TestReleaseMoveFromMovingNeedsNoChores` beside the two below. |
| +- Produces: `hostclaim.BeginMove(opID, by, sourceBox, targetBox, targetRegion string, now time.Time) func(*Claim) error`; `hostclaim.Touch(opID string, regions, boxes []string) func(*Claim) error`; `hostclaim.RequireMoveOp(opID, kind, phase string) func(*Claim) error`; phase constants `PhaseMoving = ""`, `PhaseSealed = "sealed"`; `hostclaim.ReleaseMove(opID, execBox string) func(*Claim) error` (K8, in this plan so the interim abort can release: legal from `moving` with phase `""` or `"aborting"` when no obligation under the op remains, sets `Op = Op{Kind: KindServing, Exec: execBox, Journal: map[string]string{"released_move": opID}}`: the serving op remembers which move released it, **operation-bound completion evidence** for the interim abort's crash boundary (round-5 finding 10; `Claim.Validate` accepts an op journal on any kind, `claim.go:90`; if it does not for `serving`, widen it in this task); `hostclaim.RequireReleasedMove(opID string) func(*Claim) error` (nil when the claim is `serving` and `Op.Journal["released_move"] == opID`, else main's `ErrStaleOp`, `ops.go:9`), the replay rule Task 14 uses; `TestReleaseMoveLeavesEvidence` (after `ReleaseMove`, `RequireReleasedMove(id)` holds and `RequireReleasedMove(other)` does not; a plain serving claim satisfies neither); Plan 4 Task 2 widens it to `landing/committed`, `restoring/opened`, `reverting/opened`). Test `TestReleaseMoveFromMovingNeedsNoChores` beside the two below. |
| |
| - [ ] **Step 1: Write the failing test** |
| |
| @@ -953,7 +953,7 @@ Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>" |
| ### Task 4: The request door: operator-only, refusals, the claim CAS |
| |
| **Files:** |
| -- Create: `internal/boxid/boxidtest/boxidtest.go` (the exported form of `boxid_test.go`'s `newSigner`/`record`/`verifier`: `boxidtest.NewSigner(t) *Signer`, `(*Signer).Record(t, box, account, region, instance) (boxid.Record, []byte)`, `(*Signer).Certs() map[string]*x509.Certificate`; `boxid_test.go` is rewritten to use it, behaviour unchanged), `internal/console/boxselect.go` (`selectMoveBoxes`, `verifiedBoxes`), `internal/console/boxselect_test.go` |
| +- Create: `internal/boxid/boxidtest/boxidtest.go` (the exported form of `boxid_test.go`'s `newSigner`/`record`/`verifier`: `boxidtest.NewSigner(t) *Signer`, `(*Signer).Record(t, box, account, region, instance) (boxid.Record, []byte)`, `(*Signer).Certs() map[string]*x509.Certificate`; **`boxid_test.go` is left exactly as it is**: it is `package boxid`, and importing `boxidtest`, which imports `boxid`, would be an import cycle (round-6 finding 2); `boxidtest` is a 40-line exported twin of that file's signer, the duplication accepted), `internal/console/boxselect.go` (`selectMoveBoxes`, `verifiedBoxes`), `internal/console/boxselect_test.go` |
| - Modify: `internal/console/server.go` (`Config.AWSAccount`, `Server.boxCerts` loaded from `boxid.EmbeddedCerts()` at construction), `cmd/console/main.go` (`AWSAccount: os.Getenv("CARLOS_AWS_ACCOUNT")`) |
| - Modify: `internal/console/apiinstancemove.go` (whole `handleAPIInstanceMoveRequest`, `moveGate`, new `moveOperatorGate`), `internal/console/authcli.go:633-636` (routes) |
| - Test: `internal/console/apiinstancemove_test.go` (extend `TestAPIInstanceMoveVerbs`; add `TestMoveDoorRefusesWithoutClaimWrite`, `TestMoveMutationsAreOperatorOnly`) |
| @@ -1073,7 +1073,7 @@ func TestMoveDoorRefusesWithoutClaimWrite(t *testing.T) { |
| } |
| ``` |
| |
| -`operatorInstanceFixture` and `writeServingClaim` are test helpers added to `apiinstancemove_test.go`; `operatorInstanceFixture` calls `instanceFixture` then marks the token's address an operator with `fx.st.SetOperator(addr, true)` (`store.go:1952`, the call `alertwatch_test.go:52` and `account_test.go:250` already make), where `addr` is the address `instanceFixture` signed the token for. `fleetFixture(t, fx, boxes ...fleetBox)` (in `boxselect_test.go`, shared by the door tests and the selector test) makes the fleet production-shaped: it generates a secrets key pair and sets `fx.srv.cfg.SecretsPub`, sets `fx.srv.cfg.AWSAccount = "111122223333"` and `fx.srv.boxCerts = signer.Certs()` from `boxidtest.NewSigner(t)`, and for each box writes (1) the edges file `control/edges/<box>.json` with `v`, `box`, `box_id` (= box), `eip`, `region` (the AWS region), `placement`, `data_bucket`, `data_bucket_region`, `hibernation_configured: true`, `serves_shared_pool: true`, `accepts_inbound: true`, `secrets_key_fp` (the pair's fingerprint), `capabilities: ["region-ha-1"]`, `hosts: {}`, `moves_protocol: 1`, `moves_floor_min_release` (the published floor's, when `fleetBox.FloorArmed` is not set to false) and `prev_release: "none"` (overridable per box); (2) a fresh status snapshot through `status.WriteBox(ctx, fx.rel, status.Box{Label: box, GeneratedAt: now})` (`accountfleet_test.go:34` makes the same call), which is what `readLiveEdgesSnapshot` joins on; (3) the signed identity record `signer.Record(t, box, "111122223333", aws, box)` written with `boxid.Publish`. `fleetBox{Box, Placement, AWS, EIP, Bucket, PrevRelease string; Unverified, Stale bool}`: `Unverified` skips the identity record, `Stale` writes an old snapshot. |
| +`operatorInstanceFixture` and `writeServingClaim` are test helpers added to `apiinstancemove_test.go`; `operatorInstanceFixture` calls `instanceFixture` then marks the token's address an operator with `fx.st.SetOperator(addr, true)` (`store.go:1952`, the call `alertwatch_test.go:52` and `account_test.go:250` already make), where `addr` is the address `instanceFixture` signed the token for. `fleetFixture(t, fx, boxes ...fleetBox)` (in `boxselect_test.go`, shared by the door tests and the selector test) makes the fleet production-shaped: it generates a secrets key pair and sets `fx.srv.cfg.SecretsPub`, sets `fx.srv.cfg.AWSAccount = "111122223333"` and `fx.srv.boxCerts = signer.Certs()` from `boxidtest.NewSigner(t)`, and for each box writes (1) the edges file `control/edges/<box>.json` with `v`, `box`, `box_id` (= box), `eip`, `region` (the AWS region), `placement`, `data_bucket`, `data_bucket_region`, `hibernation_configured: true`, `serves_shared_pool: true`, `accepts_inbound: true`, `secrets_key_fp` (the pair's fingerprint), `capabilities: ["region-ha-1"]`, `hosts: {}`, `moves_protocol: 1`, `moves_floor_min_release` (the published floor's, unless `FloorUnarmed`), `prev_release: "none"` and `prev_allowed_for` (both overridable per box); (2) a fresh status snapshot through `status.WriteBox(ctx, fx.rel, status.Box{Label: box, GeneratedAt: now})` (`accountfleet_test.go:34` makes the same call), which is what `readLiveEdgesSnapshot` joins on; (3) the signed identity record `signer.Record(t, box, "111122223333", aws, box)` written with `boxid.Publish`. `fleetBox{Box, Placement, AWS, EIP, Bucket, PrevRelease, PrevAllowedFor string; Unverified, Stale, FloorUnarmed bool}`: `Unverified` skips the identity record, `Stale` writes an old snapshot, `FloorUnarmed` omits `moves_floor_min_release`. |
| |
| `TestAPIInstanceMoveVerbs` stays as it is (the legacy door, switch off). Add `TestAPIInstanceMoveVerbsPhase1` with `fx.srv.cfg.MovesEnabled = true`, `operatorInstanceFixture`, the `moves_protocol` field on its edges file and a serving claim written before the first request; it mirrors the legacy test's matrix with the new rules: a second request while in flight is 409; abort from `cutting/requested` is **202** (the guarded abort allows it, Task 2); after an abort completes the test releases the claim (`hostclaim.Update(ReleaseMove(id, "ie-box"))`, Task 3) before the next request, because a request against a `moving` claim is 409 by design. |
| |
| @@ -1316,8 +1316,8 @@ var errEdgesIncomplete = errors.New("console: the fleet's edges files could not |
| // when control/moves-protocol.json is published, moves_floor_min_release |
| // equal to its min_release (the box has armed its local floor files, so |
| // its units already select the recovery executable by the floor) and |
| -// prev_release either "none" or at or above min_release (the box's |
| -// carlos.prev is not below the floor). That is what makes the admission |
| +// either prev_release "none" or prev_allowed_for equal to min_release |
| +// (the box has judged its carlos.prev at or above the floor, Task 13). That is what makes the admission |
| // safe without any later asynchronous step on the boxes. An incomplete |
| // snapshot answers nothing: a box whose file could not be read may be |
| // below the floor or may be the second box of a region. |
| @@ -1347,10 +1347,11 @@ func (s *Server) selectMoveBoxes(ctx context.Context, to, from, app string) (mov |
| sel.FloorOK = false |
| } |
| if floor != nil { |
| - prevOK := f.PrevRelease == "none" |
| - if !prevOK && f.PrevRelease != "" { |
| - prevOK, _ = moves.ReleaseAtOrAbove(f.PrevRelease, floor.MinRelease) |
| - } |
| + // The box's own verdict (Task 13: carlos.prev.allowed names the |
| + // floor .prev was judged at or above, by ReleaseAtOrAbove), or no |
| + // .prev at all. The console does not re-derive the comparison |
| + // from prev_release: the box's units select by that verdict. |
| + prevOK := f.PrevRelease == "none" || f.PrevAllowedFor == floor.MinRelease |
| if f.MovesFloorMinRelease != floor.MinRelease || !prevOK { |
| sel.FloorOK = false |
| } |
| @@ -1377,7 +1378,7 @@ func (s *Server) selectMoveBoxes(ctx context.Context, to, from, app string) (mov |
| } |
| ``` |
| |
| -`edgesFilePlacementView` (`instancesplacement.go:38`, what `readEdgesFiles` decodes) gains `MovesProtocol int \`json:"moves_protocol"\`` (Task 13 adds the writer's side). Private storage is the instance's store binding (round-4 finding 13): `instanceStoreEnv(rec)` is `rec.Environment`, or `rec.Slug` when it has none, exactly the name the create door stamps (`apiinstances.go:1074-1079`), and `objstore.GetIn` (`objstore.go:236`) answers `nil` for no store; a read error is a 503, never a pass. The manifest's `PrivateStorage` policy is not consulted: it is mutable and create-time (`apiinstances.go:1960-1966`), so it says nothing about an existing instance. Task 14's round makes the same `GetIn` read before its first staging write and fails S1 with M-C17's sentence when a store has appeared since the request (`carlos store create --environment`). Tests: `TestMoveDoorRefusesABoundStoreNotAPolicy` (a record with a store under its slug and the policy off → M-C17 and no claim written; the policy on and no store → admitted; the store read failing → 503 and no claim written; the manifest read failing → 503). `TestSelectMoveBoxesComposesTheCreateDoorsInputs` (`boxselect_test.go`) builds a live fleet with `fleetFixture` (below) and checks: two verified boxes in `uk` → `Many == "uk"`; one verified box in `uk` and one unverifiable (its `boxid` record missing) → that one box is the target; no file in the source region → `Source == ""` and `FloorOK` false; a box whose file lacks `moves_protocol` → `FloorOK` false; a published floor and a box whose `prev_release` is below it, or whose `moves_floor_min_release` is absent → `FloorOK` false, `"none"` → true; an incomplete snapshot → `errEdgesIncomplete`. The door test adds: a claim whose `Op.Exec` is not the source region's eligible box → 409 and no claim written (round-5 finding 7). The fixtures are `boxid.Publish`ed records signed by `boxidtest.NewSigner(t)`, so `Verifier.Verify` runs for real. |
| +`edgesFilePlacementView` (`instancesplacement.go:38`, what `readEdgesFiles` decodes) gains its three reader fields **in this task**, since `selectMoveBoxes` reads them (round-6 finding 1): `MovesProtocol int \`json:"moves_protocol"\``, `MovesFloorMinRelease string \`json:"moves_floor_min_release"\``, `PrevRelease string \`json:"prev_release"\`` and `PrevAllowedFor string \`json:"prev_allowed_for"\``; Task 13 adds the writer's side. Private storage is the instance's store binding (round-4 finding 13): `instanceStoreEnv(rec)` is `rec.Environment`, or `rec.Slug` when it has none, exactly the name the create door stamps (`apiinstances.go:1074-1079`), and `objstore.GetIn` (`objstore.go:236`) answers `nil` for no store; a read error is a 503, never a pass. The manifest's `PrivateStorage` policy is not consulted: it is mutable and create-time (`apiinstances.go:1960-1966`), so it says nothing about an existing instance. Task 14's round makes the same `GetIn` read before its first staging write and fails S1 with M-C17's sentence when a store has appeared since the request (`carlos store create --environment`). Tests: `TestMoveDoorRefusesABoundStoreNotAPolicy` (a record with a store under its slug and the policy off → M-C17 and no claim written; the policy on and no store → admitted; the store read failing → 503 and no claim written; the manifest read failing → 503). `TestSelectMoveBoxesComposesTheCreateDoorsInputs` (`boxselect_test.go`) builds a live fleet with `fleetFixture` (below) and checks: two verified boxes in `uk` → `Many == "uk"`; one verified box in `uk` and one unverifiable (its `boxid` record missing) → that one box is the target; no file in the source region → `Source == ""` and `FloorOK` false; a box whose file lacks `moves_protocol` → `FloorOK` false; a published floor and a box whose `prev_allowed_for` names an older floor or is absent, or whose `moves_floor_min_release` is absent → `FloorOK` false; `prev_release: "none"` → true; an incomplete snapshot → `errEdgesIncomplete`. The door test adds: a claim whose `Op.Exec` is not the source region's eligible box → 409 and no claim written (round-5 finding 7). The fixtures are `boxid.Publish`ed records signed by `boxidtest.NewSigner(t)`, so `Verifier.Verify` runs for real. |
| |
| **The legacy door stays** (Task 0): today's bodies of `handleAPIInstanceMoveRequest`, `handleAPIInstanceMoveCut` and `handleAPIInstanceMoveAbort` are renamed `legacyHandleAPIInstanceMoveRequest` (and siblings) unchanged, and each new handler's first line is `if !s.cfg.MovesEnabled { s.legacyHandleAPIInstanceMove…(w, r); return }`; `handleAPIInstanceMoveResolve` answers 404 when the switch is off (the route did not exist before). The existing `TestAPIInstanceMoveVerbs` and `TestAPIInstanceMoveClassRails` run as they are with the switch off (no change in behaviour, which is the point); the new tests below set `fx.srv.cfg.MovesEnabled = true`. |
| |
| @@ -2438,7 +2439,7 @@ func Verify(sealed io.Reader, m Manifest, priv *ecdh.PrivateKey, b appconfig.Bin |
| } |
| ``` |
| |
| -(`Extract` declares `var modes []modeFix` before the loop and calls `restoreModes(modes)` on `io.EOF` before returning nil. `Verify`'s `h` hashes the decoded tar, trailer included, which is exactly what `Build`'s `capWriter` hashed, since `capWriter` sits between the `tar.Writer` and the zstd writer; `Verify` also checks `m.Sealed` over the sealed bytes it read and drains both streams to EOF so `Open`'s terminal authentication result is the verdict.) Tests added: `TestExtractPreservesModes` (a `0555` directory holding a `0400` file round-trips to exactly those modes; `PlainDigest` of the extracted tree equals the original) and `TestVerifyRefusesWhatTheManifestDoesNotPromise` (a correctly sealed archive with a manifest whose `Plain` is wrong, whose entry list lacks a file, whose entry sha256 differs, plus a hand-built sealed archive containing a symlink entry, one over the cap, one whose last sealed byte is flipped and one whose final chunk is cut off (the last two return `Open`'s authentication error, never nil): each returns the named error; the good archive verifies, and a manifest whose `sealed` differs from the bytes is refused) and `TestVerifyReleasesTheProducerOnEarlyMismatch` (a three-chunk archive and a manifest whose first entry's size is wrong: `Verify` returns the entry error and `runtime.NumGoroutine()` returns to its baseline within a second; `errVerifyAborted` is the package-level sentinel the deferred close uses). |
| +(`Extract` declares `var modes []modeFix` before the loop and calls `restoreModes(modes)` on `io.EOF` before returning nil. `Verify`'s `h` hashes the decoded tar, trailer included, which is exactly what `Build`'s `capWriter` hashed, since `capWriter` sits between the `tar.Writer` and the zstd writer; `Verify` also checks `m.Sealed` over the sealed bytes it read and drains both streams to EOF so `Open`'s terminal authentication result is the verdict.) Tests added: `TestExtractPreservesModes` (a `0555` directory holding a `0400` file round-trips to exactly those modes; `PlainDigest` of the extracted tree equals the original) and `TestVerifyRefusesWhatTheManifestDoesNotPromise` (a correctly sealed archive with a manifest whose `Plain` is wrong, whose entry list lacks a file, whose entry sha256 differs, plus a hand-built sealed archive containing a symlink entry, one over the cap, one whose last sealed byte is flipped, one whose final chunk is cut off (these two return `Open`'s authentication error, never nil), one with a byte appended after the final chunk and one that is two valid archives concatenated (both `ErrSealed`, "data after the final chunk": the sealed digest of the whole object would differ, round-6 finding 6): each returns the named error; the good archive verifies, and a manifest whose `sealed` differs from the bytes is refused) and `TestVerifyReleasesTheProducerOnEarlyMismatch` (a three-chunk archive and a manifest whose first entry's size is wrong: `Verify` returns the entry error and `runtime.NumGoroutine()` returns to its baseline within a second; `errVerifyAborted` is the package-level sentinel the deferred close uses). |
| |
| `seal.go`: |
| |
| @@ -2563,6 +2564,17 @@ func Open(sealed io.Reader, sealedKey string, priv *ecdh.PrivateKey, b appconfig |
| return err |
| } |
| if last { |
| + // The final chunk must end the stream: trailing ciphertext |
| + // would otherwise pass Verify against a manifest whose sealed |
| + // digest covers only this authenticated prefix (round-6 |
| + // finding 6). |
| + var one [1]byte |
| + if _, err := io.ReadFull(sealed, one[:]); err != io.EOF { |
| + if err == nil { |
| + return fmt.Errorf("%w: data after the final chunk", ErrSealed) |
| + } |
| + return err |
| + } |
| return nil |
| } |
| goto next |
| @@ -2601,9 +2613,9 @@ Moved here from Plan 2 (its Tasks 4 and 7) because T3, T7 and S2 need the real t |
| - Modify: `internal/activator/backend_exec.go` (`Strict`, `prepareStrict`, `NextSpawnStrict`, the strict spawn path at `spawnInCgroup`), `internal/activator/backend_exec_test.go`, `internal/appconfig/instances.go` (`LoadMergedHostVersions`), `internal/release/adopt.go` (`StageBlocked` exported), `internal/host/agent.go` (`carlos adopt --stage-only` calls `release.ServeHeldStageRequests` with `heldStageAuthority`), `internal/host/heldstage.go`, `internal/host/heldstage_test.go` |
| |
| **Interfaces:** |
| -- `appconfig.LoadMergedHostVersions(ctx, s release.Store, acct, app, envName, host string) (*Manifest, MergedVersions, error)` with `MergedVersions{Default, Environment, Overlay string; Shadowed []string}` (`Shadowed`: the bundle variables the instance overlay overrides, collected in `applyInstanceOverlay`'s loop; T3 reports them, round-5 finding 9): the same three reads `LoadMergedHost` and `LoadMerged` make (`instances.go:202`, `environments.go:169`), through `getIfExistsAtETag` (`store.go:65`) so the etag of each object actually merged is returned with the bytes; `LoadMergedHost` becomes a wrapper that discards all but the overlay etag (behaviour unchanged, its tests untouched). A later independent read could not establish which versions were rendered (round-4 finding 12), so this is the one loader `RenderHeld` and T3 use. |
| +- `appconfig.LoadMergedHostVersions(ctx, s release.Store, acct, app, envName, host string) (*Manifest, MergedVersions, error)` with `MergedVersions{Default, Environment, Overlay string}`: the same three reads `LoadMergedHost` and `LoadMerged` make (`instances.go:202`, `environments.go:169`), through `getIfExistsAtETag` (`store.go:65`) so the etag of each object actually merged is returned with the bytes; `LoadMergedHost` becomes a wrapper that discards all but the overlay etag (behaviour unchanged, its tests untouched). A later independent read could not establish which versions were rendered (round-4 finding 12), so this is the one loader `RenderHeld` and T3 use. |
| - `appconfig.RenderHeld(ctx, s release.Store, priv *ecdh.PrivateKey, dataDir string, rt registry.Route, op string, box string) (HeldRender, error)` with `HeldRender{Path, SHA256 string; BundleETags map[string]string; OverlayETag string}` (`BundleETags` = `{"default": v.Default, "environment": v.Environment}`, `OverlayETag` = `v.Overlay`): `LoadMergedHostVersions(ctx, s, acct, app, rt.Environment, rt.Host)`, rendered through the same file writer `syncExecHost` uses (the file-writing half extracted into `renderExecConfig(rt, m, key, priv, dataDir) (path string, changed bool, err error)` and called from both), written whether or not it changed, and a move-bound ack through `WriteConfigAck` with three additive `ConfigAck` fields: `MoveOp string \`json:"move_op,omitempty"\``, `FileSHA256 string \`json:"file_sha256,omitempty"\``, `BundleETags map[string]string \`json:"bundle_etags,omitempty"\``. `SyncExec` keeps skipping non-ready rows (`execRouteBase`), so the two never race on one file. |
| -- `release.MaterializeHeld(ctx, s Store, optDir string, rt registry.Route, version string) (*Manifest, error)`: `ensureStaged(ctx, s, optDir, RouteAccount(rt), rt.App, version, rt.Host)` (`adopt.go:1372`) then `PointLive(optDir, host, acct, version, app)` (`versions.go:267`, already exported), no restart, no bake-hold check: a held row runs exactly the pinned release. When this process cannot write `optDir` (classified by main's own `stageBlocked`, `adopt.go:256`: `EROFS`, `EPERM`, `EACCES`; `ProtectSystem=strict` answers a write with `EROFS`, which is **not** `fs.ErrPermission`, round-5 finding 5; exported in this task as `release.StageBlocked(err error) bool`; the edge unit writes only `/data` and `/run/carlos`, `carlos-edge.service:35-37`), it returns `release.ErrStageBlocked` wrapping the cause, so the caller relays instead of failing (Task 11's T7, round-4 finding 6). `release.HeldStageRequest{Account, App, Host, Version, Op, Box string}` and `release.ServeHeldStageRequests(ctx, s Store, optDir, dataDir string, authorize func(HeldStageRequest) error) error` is the root side of that relay (round-5 finding 2). For every `<dataDir>/moves/held-stage/<name>.json` it (1) decodes the request and **validates every component before any filesystem effect**: `Host == <name>` and `ValidHostName(Host)` (letters, digits, `.` and `-` only: no `/`, no `..`, no NUL), `ValidAccount(Account)`, `ValidApp(App)`, `ValidVersion(Version)` (`versions.go:69`), `Op != ""`, `Box != ""`; a request failing any of these is removed with a log line and never joined into a path; (2) calls `authorize(req)` and on error removes the request with a log line; (3) runs `MaterializeHeld` and writes `<dataDir>/moves/held-staged/<host>.json` (`{version, op, at}`), then removes the request. `authorize` is the host package's closure `heldStageAuthority(reg *registry.Registry, store release.Store, box string)` (`internal/host/heldstage.go`; `release` never imports `hostclaim`): it reads the registry row for `req.Host` and requires `hold_op == req.Op` (Task 12) and the route's account and app equal to the request's; reads the claim (`hostclaim.Read`) and requires `Op.ID == req.Op`, `Op.Kind` in `{KindMoving, KindLanding}`, `Op.Intent["target"] == req.Box` and `req.Box == box` (this box); reads the record and requires its pinned release (`rec.Move.Inputs.Release`, the version T7 is checking) equal to `req.Version`; anything else is `ErrStaleHeldStage`. So a request queued for operation A cannot run after A is aborted or replaced: the claim no longer names A, and `live/<host>` is never repointed to A's build. `carlos adopt --stage-only` (the root pass `carlos-stage.service` runs when `/data/stage-requested` changes, `agent.go:3606`) calls it after its release adoption, with `heldStageAuthority`. There is no `pub` parameter: `ensureStaged` (`adopt.go:1372`) is main's own staging step and verifies what the adopt pass verifies at that layer. `release.LiveVersion(optDir, host string) (string, error)` **already exists on main** (`versions.go:290`) and is reused: it reads the `live/<host>` symlink's target version (readable from the sandboxed agent), which is the proof T7 trusts; the ack file is only a hint. |
| +- `release.MaterializeHeld(ctx, s Store, optDir string, rt registry.Route, version string) (*Manifest, error)`: `ensureStaged(ctx, s, optDir, RouteAccount(rt), rt.App, version, rt.Host)` (`adopt.go:1372`) then `PointLive(optDir, host, acct, version, app)` (`versions.go:267`, already exported), no restart, no bake-hold check: a held row runs exactly the pinned release. When this process cannot write `optDir` (classified by main's own `stageBlocked`, `adopt.go:256`: `EROFS`, `EPERM`, `EACCES`; `ProtectSystem=strict` answers a write with `EROFS`, which is **not** `fs.ErrPermission`, round-5 finding 5; exported in this task as `release.StageBlocked(err error) bool`; the edge unit writes only `/data` and `/run/carlos`, `carlos-edge.service:35-37`), it returns `release.ErrStageBlocked` wrapping the cause, so the caller relays instead of failing (Task 11's T7, round-4 finding 6). `release.HeldStageRequest{Account, App, Host, Version, Op, Box string}` and `release.ServeHeldStageRequests(ctx, s Store, optDir, dataDir string, authorize func(HeldStageRequest) error) error` is the root side of that relay (round-5 finding 2). For every `<dataDir>/moves/held-stage/<name>.json` it (1) decodes the request and **validates every component before any filesystem effect**: `Host == <name>` and `ValidHostName(Host)` (letters, digits, `.` and `-` only: no `/`, no `..`, no NUL), `ValidAccount(Account)`, `ValidApp(App)`, `ValidVersion(Version)` (`versions.go:69`), `Op != ""`, `Box != ""`; a request failing any of these is removed with a log line and never joined into a path; (2) calls `authorize(req)` and on error removes the request with a log line; (3) runs `MaterializeHeld` and writes `<dataDir>/moves/held-staged/<host>.json` (`{version, op, at}`), then removes the request. `authorize` is the host package's closure `heldStageAuthority(reg *registry.Registry, store release.Store, box string)` (`internal/host/heldstage.go`; `release` never imports `hostclaim`): it reads the registry row for `req.Host` **when one exists** and requires `hold_op == req.Op` (Task 12) and the route's account and app equal to the request's (during `preparing` the target has no row yet: Plan 2's flip creates it, so an absent row is accepted and a row held by another op refuses; round-6 finding 3); reads the claim (`hostclaim.Read`) and requires `Op.ID == req.Op`, `Op.Kind` in `{KindMoving, KindLanding}`, `Op.Intent["target"] == req.Box` and `req.Box == box` (this box); and requires `req.Version` equal to the **source report's `Release`** (`moves.SourceReport.Release`, Task 11: the source's `release.LiveVersion(optDir, host)`, published every round; the producer of the pinned release's identity during `preparing`, which Plan 2's cut admission copies into `Move.Inputs.Release`), read from `SourceReportKey` in the deployment bucket; a missing report, an empty `Release` or any other mismatch is `ErrStaleHeldStage`. So a request queued for operation A cannot run after A is aborted or replaced: the claim no longer names A, and `live/<host>` is never repointed to A's build. `carlos adopt --stage-only` (the root pass `carlos-stage.service` runs when `/data/stage-requested` changes, `agent.go:3606`) calls it after its release adoption, with `heldStageAuthority`. There is no `pub` parameter: `ensureStaged` (`adopt.go:1372`) is main's own staging step and verifies what the adopt pass verifies at that layer. `release.LiveVersion(optDir, host string) (string, error)` **already exists on main** (`versions.go:290`) and is reused: it reads the `live/<host>` symlink's target version (readable from the sandboxed agent), which is the proof T7 trusts; the ack file is only a hint. |
| - `ExecBackend.Strict func(host string) bool` (when true the spawn uses `b.prepareStrict(host, limit)` then `Start(cmd)` on what it returns, keeping it for the process's life and running `KillAndEmpty(ctx)` then `Close()` on exit; a failure refuses the spawn with `ErrStrictRequired`). `prepareStrict func(host string, limit cgroup.Limit) (strictProc, error)` is an unexported seam with `strictProc interface{ Start(*exec.Cmd) error; KillAndEmpty(context.Context) error; Close() error }`, which `*cgroup.Prepared` satisfies on every build (`strict_linux.go:233-371`, `strict_other.go:15-18`); nil → `b.Cgroups.PrepareStrict`, and a nil `Cgroups` under `Strict` is `ErrStrictRequired`. `Cgroups` stays the concrete `*cgroup.Manager` (`backend_exec.go:100`): tests replace the seam, never the field (round-4 finding 2). `(b *ExecBackend) NextSpawnStrict(host string) moves.Check` (`moves.Check` is Task 1's; S2: OK when `Strict(host)` and cgroup v2 delegation is present, `cgroup.Manager.Root() != ""`, and `PrepareStrict` is supported on this build). The host package sets `eb.Strict` from the hold view (a host whose record has a move in flight and the switch on); off → never strict, today's behaviour. |
| |
| Tests: `RenderHeld` writes the file for a `landing` row `SyncExec` would skip and the ack carries `move_op`, the sha256 and the bundle etags, which equal the etags the dir store reports for the fixture's default and environment objects; a second call with no change still writes an ack. `MaterializeHeld` stages the pinned version into a temp `optDir` and points `live/<host>` at it, never at the channel head; with the `materializeHeldStage` seam (a package variable `= ensureStaged`, replaced by the test) failing with `&os.PathError{Err: syscall.EROFS}`, and separately `EACCES`, it returns `ErrStageBlocked` and writes nothing (the `chmod` fixture is gone: it tested discretionary permissions, not a read-only mount). `TestServeHeldStageRequestsRefusesStaleAndTraversal`: a request whose op the claim no longer names → removed, `live/<host>` untouched; a request whose `Host` is `../x`, or does not match its file name → removed before any path join (a `release.Store` fake records no reads); the happy path materializes, acks and removes; `LiveVersion` reads the result. `heldStageAuthority`'s own test covers each refused field. `TestSpawnUnderAMoveIsStrictOrRefused` (the `prepareStrict` seam returning an error → `Start` returns `ErrStrictRequired` and spawns nothing; the seam returning a recording `strictProc` → `Start` is called on it and `KillAndEmpty` then `Close` on exit; with `Strict` false the plain path runs as today). |
| @@ -2797,7 +2809,7 @@ Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>" |
| |
| **Interfaces:** |
| - Produces: `moves.ReportKeyFromSecrets(secretsKeyB64 string) ([]byte, error)` (HKDF-SHA256 over the raw private key bytes, info `"carlos-move-env-report-v1"`); `moves.EnvReport{V int; Vars map[string]string; Digest string}`; `moves.ReportEnv(key []byte, env map[string]string) EnvReport` (each value HMAC-SHA256 under key, hex; `Digest` the sha256 of the sorted `name=mac` lines); `moves.Class(name string) string` returning `"region" | "layout" | "advisory" | "box" | "equal"` (`LayoutVars` is `CARLOS_CONTROL_SOCKET` alone; `AdvisoryVars` is `CARLOS_PURGE`); `moves.CompareEnv(source, target EnvReport, flatLayout bool) []EnvDiff` with `EnvDiff{Name, Kind string}` (kinds: `missing`, `extra`, `differs`, `region-missing`, `region-same`, `layout-missing`, `layout-present`, `advisory-differs`; `(EnvDiff).Note() bool` is true for `advisory-differs` alone, the one kind T3 records without failing); `moves.RegionVar = "CARLOS_REGION"`, `LayoutVars = {"CARLOS_CONTROL_SOCKET", "CARLOS_PURGE"}`, `BoxVars = {"INVOCATION_ID", "JOURNAL_STREAM", "HOSTNAME", "PWD", "OLDPWD", "SHLVL", "_"}`. |
| -- `(b *ExecBackend) LaunchEnvironment(rt registry.Route) (map[string]string, error)`: `childEnv(instanceEnv(rt))` plus the agent-owned variables `Start` adds, decoded back into a map (the composed launch environment, codex R2-8). |
| +- `(b *ExecBackend) LaunchEnvironment(rt registry.Route) (env map[string]string, shadowed []string, err error)`: `childEnv(instanceEnv(rt))` plus the agent-owned variables `Start` adds, decoded back into a map, and `shadowed`, the sorted names present in both the rendered configuration file and the instance's own `<host>.env` (the override `mergeEnv(cfgEnv, ownEnv)` applies at `backend_exec.go:1183`: spec §7.3 T3's "a `<host>.env` variable that shadows a rendered one is reported", collected here because this is the only layer that sees both files, round-6 finding 7; `instanceEnv`'s two reads are extracted into `instanceEnvParts(rt) (cfgEnv, ownEnv map[string]string, err error)` so the composition is unchanged) (the composed launch environment, codex R2-8). |
| |
| - [ ] **Step 1: Write the failing tests** |
| |
| @@ -2868,10 +2880,17 @@ func TestLaunchEnvironmentComposesTheSpawnEnvironment(t *testing.T) { |
| os.WriteFile(filepath.Join(b.DataDir, rt.Host+".env"), []byte("MINE=1\nCARLOS_ADMIN_TOKEN=t\nCARLOS_PURGE=header\nCARLOS_REGION=fake\n"), 0o600) |
| t.Setenv("CARLOS_SECRETS_KEY", "never") |
| t.Setenv("UNRELATED", "yes") |
| - env, err := b.LaunchEnvironment(rt) |
| + // The rendered configuration file (the path instanceEnv reads first) |
| + // renders MINE=0; the instance's own .env overrides it, so MINE is |
| + // reported as shadowed (spec §7.3 T3). |
| + writeRenderedConfig(t, b, rt, "MINE=0\nFROM_CONFIG=c\n") // this file's helper for the config env file instanceEnv reads |
| + env, shadowed, err := b.LaunchEnvironment(rt) |
| if err != nil { |
| t.Fatal(err) |
| } |
| + if len(shadowed) != 1 || shadowed[0] != "MINE" || env["FROM_CONFIG"] != "c" { |
| + t.Fatalf("shadowed %v env %v", shadowed, env) |
| + } |
| if env["MINE"] != "1" || env["UNRELATED"] != "yes" || env["CARLOS_SECRETS_KEY"] != "" || env["CARLOS_ADMIN_TOKEN"] != "t" { |
| t.Fatalf("launch env: %v", env) |
| } |
| @@ -2884,7 +2903,7 @@ func TestLaunchEnvironmentComposesTheSpawnEnvironment(t *testing.T) { |
| t.Fatal("CARLOS_PURGE survived on a box without a cache") |
| } |
| b.Cache = edge.NewCache(16, 1<<20) |
| - if env, _ = b.LaunchEnvironment(rt); env["CARLOS_PURGE"] != "header" { |
| + if env, _, _ = b.LaunchEnvironment(rt); env["CARLOS_PURGE"] != "header" { |
| t.Fatalf("CARLOS_PURGE on a box with a cache: %v", env) |
| } |
| } |
| @@ -3076,18 +3095,26 @@ func CompareEnv(source, target EnvReport, flatLayout bool) []EnvDiff { |
| // b.Cache != nil exactly as agentOwnedVars does. The move's launch report |
| // (moves spec §7.3, codex R2-8) is computed over exactly this, so a |
| // target compares what would run, not a reconstruction of it. |
| -func (b *ExecBackend) LaunchEnvironment(rt registry.Route) (map[string]string, error) { |
| - fileEnv, err := b.instanceEnv(rt) |
| +func (b *ExecBackend) LaunchEnvironment(rt registry.Route) (map[string]string, []string, error) { |
| + cfgEnv, ownEnv, err := b.instanceEnvParts(rt) |
| if err != nil { |
| - return nil, err |
| + return nil, nil, err |
| } |
| + var shadowed []string |
| + for k := range ownEnv { |
| + if _, rendered := cfgEnv[k]; rendered { |
| + shadowed = append(shadowed, k) |
| + } |
| + } |
| + sort.Strings(shadowed) |
| + fileEnv := mergeEnv(cfgEnv, ownEnv) |
| out := map[string]string{} |
| for _, kv := range childEnv(b.agentOwnedVars(rt, fileEnv, true)) { |
| if i := strings.IndexByte(kv, '='); i > 0 { |
| out[kv[:i]] = kv[i+1:] |
| } |
| } |
| - return out, nil |
| + return out, shadowed, nil |
| } |
| ``` |
| |
| @@ -3186,11 +3213,11 @@ Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>" |
| |
| **Files:** |
| - Create: `internal/moves/readiness.go`, `internal/moves/readiness_test.go` |
| -- Create: `internal/host/movesreport.go`, `internal/host/movesreport_test.go` |
| +- (The host side, `internal/host/movesreport.go`, `movesreport_test.go` and `movesreport_minio_test.go` with the `.amadan/ci.d/30-minio-contract` amendment, is **created and committed in Task 14**, which has the harness; this task's commit is the pure package alone, round-6 finding 8.) |
| |
| **Interfaces:** |
| -- Produces (pure): `moves.PutReplace(ctx, s release.Store, key string, body []byte) error`: `release.Store.Put` with an empty `ifMatch` is create-only (`store.go:38`), so a recurring object (a report, the console record) is written by `Get` for the current etag (absent → `""`), then `Put(key, body, etag)`, retrying once on `release.ErrCASConflict`; every mutable object the series writes to the deployment bucket goes through it (reports here, the console record in Task 12, the hold view's receipts ledger when it lives in a `release.Store`); `moves.Check` (Task 1's); `moves.TargetReport{V int; Box, Region string; MoveID string; At time.Time; Checks map[string]Check; Inputs provision.MoveInputs; AttemptsAdmitted int; CutBudgetS int; Rates Rates; Notes []string}`; check ids `T1 … T12` as constants `CheckProtocol = "T1"` etc.; `moves.SourceReport{V int; Box string; MoveID string; At time.Time; Env EnvReport; S1, S2 Check; FilesPlain string; Archive *ArchiveRef}` with `ArchiveRef{ManifestKey, ManifestVersion, SealedKey, SealedVersion, Plain string}` (the versions the source's `PutVersionedAt` responses returned, Task 14's ledger `archive_versions`, published so T5 reads by exact version: round-4 finding 12); `moves.ReportFreshness = 10 * time.Minute`; `moves.Ready(roundDelta int64, bar int64, target *TargetReport, source *SourceReport, now time.Time) (ok bool, reason string)`; `(r TargetReport) Passing() (bool, string)`. |
| -- Produces (host): `(w *movesWorker) writeSourceReport(ctx, s, rec, env map[string]string, filesPlain string, archive *moves.ArchiveRef) error`; `(w *movesWorker) targetReportPass(ctx, s release.Store, recs []provision.Record) error` run by the target region's box for every record whose `Move.To` is its placement: computes T1 (own `moves_protocol` vs floor), T2 (own `secrets_key_fp`, `appconfig.KeyFingerprint` of `CARLOS_SECRETS_KEY`'s public half, equals the console's `SecretsKeyFP` in `moves.ConsoleKey` = `control/moves-console.json`, which Task 12 has the console write at startup; no such object yet on main), T5 (the archive manifest and sealed object the source report's `Archive` names, read by exact version through `lease.VersionedStore`'s `GetVersion`/`OpenVersion` in the target's own bucket; a report without `Archive` fails T5 "the source has published no archive yet": `archive.Verify(sealed, manifest, priv, binding)` (Task 7) checks the plaintext digest, every entry's path, size and sha256, the entry types and the cap; the report records the manifest's `plain`), T6 (`diskBytes(dataDir())`: free ≥ 2×replica + archive + 2 GiB and resulting use < 85%), T8 (one hosting box publishes the target placement), T9 (`hostclaim.Read`: `KindMoving`, id, holder incarnation = `move.json`'s; this region and box in `touched`), T10 (`PutIfAbsent`/`DeleteVersion` of `moves/_lane/<id>` in the source bucket through `lease.VersionedStore`), T11 (`moves.Estimate` with **measured** rates, never the worked example: `moves.Rates` come from `moves.Measurements{Cross, S3, Disk MeasuredRate}` with `MeasuredRate{MiBPerS float64; At time.Time; Bytes int64}`; the target measures regional S3 by putting and getting a 16 MiB object under `moves/<host>/<id>/probe/` in its own bucket once per report and the disk by writing and reading a 64 MiB file under `<DataDir>/.moves/` with `O_DIRECT` where available, fsync otherwise; the cross-region rate comes from the source's receipts ledger, which Task 14 extends with `LastRoundBytes` and `LastRoundSeconds` of its most recent copying round (copied through the target bucket's region, so it is the path the cut uses), published in the source report as `source.cross_rate`; a measurement older than one hour, or missing, fails T11 with "no fresh rate measurement for <which>"; `CARLOS_MOVE_RATES` exists only as a test override and is refused outside tests (`testing.Testing()`)), T12 (`claim.QuietAfter[region]` passed), T3 (compose the flat held row's environment: `appconfig.LoadMergedHostVersions` (Task 7b) for the record's environment rendered to a temp config file, its `MergedVersions` recorded as `Inputs.BundleETags` and `Inputs.OverlayETag` and its `Shadowed` names (bundle variables the instance overlay overrides) recorded in `TargetReport.Notes []string`, which `TestTargetReportPassFlagsEachCheck` asserts for an overlay that shadows one bundle variable (round-5 finding 9), the carried `<host>.env` from the newest archive, the agent-owned variables, this process's inherited variables; `moves.CompareEnv`), T4 (for each store binding in the composed environment per `objstore.EnvNames`: list one key, put/get/delete `.carlos-move-probe/<move id>` through the S3 client of `internal/objstore`'s existing probe helper if one exists, else a minimal AWS SDK client), T7 (`release.MaterializeHeld(ctx, s, optDir, rt, version)`, Task 7b; on a hardened box it returns `ErrStageBlocked`, and then the pass writes `<DataDir>/moves/held-stage/<host>.json` and touches the `stage-requested` marker through `stageRequester(dd)` (`agent.go:1330`, the existing relay to the root `carlos-stage.service`, whose `carlos adopt --stage-only` now calls `release.ServeHeldStageRequests`); T7 passes only once `release.LiveVersion(optDir, rt.Host)` equals the pinned version, and until then fails "waiting for the root stage pass"; the report records the version; round-4 finding 6); T8 and T9 bind to the named target: this box's id must equal `op.intent.target` (`selectMoveBoxes`'s choice, Task 4), else T9 fails "this box is not the move's target"; the report writer is the named target only. S2 (`ExecBackend.NextSpawnStrict(host)`, Task 7b); S1 wakes a hibernating source and reads the answer from its own process, never from the response cache (round-4 finding 11: `edge.go:1651-1663` serves a cached GET before the activator runs, so a cached 200 could certify a source that cannot wake): `(*edge.Edge).ServeUncached(w, r)` (Task 14 adds it: `ServeHTTP` with the request marked `uncachedProbe` in its context, the same marker pattern as `doorCallerFrom`, which makes `cacheable` false at `edge.go:1651` so neither `GetFor` nor `maybeStore` runs, while the hibernating branch still wakes through `e.act.ServeHTTP`) with `GET /<ready path>`, `Host: <host>`, a recording `ResponseWriter` and a 30 s timeout; `moveCfg.ReadyProbe` is set by `setMovesReadyProbe(e)` from `edge.RunWithActivatorReady`'s callback (`agent.go:613`, where the agent receives its `*edge.Edge`) and fails "edge not ready" before that. Direct socket dialing is not used: it cannot wake a sleeping instance. Writes `TargetReportKey` with `Inputs`, `AttemptsAdmitted` and the measurements. The host-side pass and its test are **written in Task 14** (they need its harness; D12); this task's deliverable is the pure package and `moves.PutReplace`. |
| +- Produces (pure): `moves.PutReplace(ctx, s release.Store, key string, body []byte) error`: `release.Store.Put` with an empty `ifMatch` is create-only (`store.go:38`), so a recurring object (a report, the console record) is written by `Get` for the current etag (absent → `""`), then `Put(key, body, etag)`, retrying once on `release.ErrCASConflict`; every mutable object the series writes to the deployment bucket goes through it (reports here, the console record in Task 12, the hold view's receipts ledger when it lives in a `release.Store`); `moves.Check` (Task 1's); `moves.TargetReport{V int; Box, Region string; MoveID string; At time.Time; Checks map[string]Check; Inputs provision.MoveInputs; AttemptsAdmitted int; CutBudgetS int; Rates Rates; Notes []string}`; check ids `T1 … T12` as constants `CheckProtocol = "T1"` etc.; `moves.SourceReport{V int; Box string; MoveID string; At time.Time; Env EnvReport; S1, S2 Check; FilesPlain string; Release string; Archive *ArchiveRef}` (`Release`: the source's live version, `release.LiveVersion(optDir, host)`, the pinned release T7 stages and `heldStageAuthority` checks, round-6 finding 3) with `ArchiveRef{ManifestKey, ManifestVersion, SealedKey, SealedVersion, Plain string}` (the versions the source's `PutVersionedAt` responses returned, Task 14's ledger `archive_versions`, published so T5 reads by exact version: round-4 finding 12); `moves.ReportFreshness = 10 * time.Minute`; `moves.Ready(roundDelta int64, bar int64, target *TargetReport, source *SourceReport, now time.Time) (ok bool, reason string)`; `(r TargetReport) Passing() (bool, string)`. |
| +- Produces (host): `(w *movesWorker) writeSourceReport(ctx, s, rec, env map[string]string, filesPlain string, archive *moves.ArchiveRef) error`; `(w *movesWorker) targetReportPass(ctx, s release.Store, recs []provision.Record) error` run by the target region's box for every record whose `Move.To` is its placement: computes T1 (own `moves_protocol` vs floor), T2 (own `secrets_key_fp`, `appconfig.KeyFingerprint` of `CARLOS_SECRETS_KEY`'s public half, equals the console's `SecretsKeyFP` in `moves.ConsoleKey` = `control/moves-console.json`, which Task 12 has the console write at startup; no such object yet on main), T5 (the archive manifest and sealed object the source report's `Archive` names, read by exact version through `lease.VersionedStore`'s `GetVersion`/`OpenVersion` in the target's own bucket; a report without `Archive` fails T5 "the source has published no archive yet": `archive.Verify(sealed, manifest, priv, binding)` (Task 7) checks the plaintext digest, every entry's path, size and sha256, the entry types and the cap; the report records the manifest's `plain`), T6 (`diskBytes(dataDir())`: free ≥ 2×replica + archive + 2 GiB and resulting use < 85%), T8 (one hosting box publishes the target placement), T9 (`hostclaim.Read`: `KindMoving`, id, holder incarnation = `move.json`'s; this region and box in `touched`), T10 (`PutIfAbsent`/`DeleteVersion` of `moves/_lane/<id>` in the source bucket through `lease.VersionedStore`), T11 (`moves.Estimate` with **measured** rates, never the worked example: `moves.Rates` come from `moves.Measurements{Cross, S3, Disk MeasuredRate}` with `MeasuredRate{MiBPerS float64; At time.Time; Bytes int64}`; the target measures regional S3 by putting and getting a 16 MiB object under `moves/<host>/<id>/probe/` in its own bucket once per report and the disk by writing and reading a 64 MiB file under `<DataDir>/.moves/` with `O_DIRECT` where available, fsync otherwise; the cross-region rate comes from the source's receipts ledger, which Task 14 extends with `LastRoundBytes` and `LastRoundSeconds` of its most recent copying round (copied through the target bucket's region, so it is the path the cut uses), published in the source report as `source.cross_rate`; a measurement older than one hour, or missing, fails T11 with "no fresh rate measurement for <which>"; `CARLOS_MOVE_RATES` exists only as a test override and is refused outside tests (`testing.Testing()`)), T12 (`claim.QuietAfter[region]` passed), T3 (compose the flat held row's environment: `appconfig.LoadMergedHostVersions` (Task 7b) for the record's environment rendered to a temp config file, its `MergedVersions` recorded as `Inputs.BundleETags` and `Inputs.OverlayETag`, and the `shadowed` names `LaunchEnvironment` returns (carried `<host>.env` variables overriding the rendered configuration, spec §7.3 T3, Task 9) recorded in `TargetReport.Notes []string`, which `TestTargetReportPassFlagsEachCheck` asserts for a carried `.env` that overrides one rendered variable with no instance overlay present (round-6 finding 7), the carried `<host>.env` from the newest archive, the agent-owned variables, this process's inherited variables; `moves.CompareEnv`), T4 (for each store binding in the composed environment per `objstore.EnvNames`: list one key, put/get/delete `.carlos-move-probe/<move id>` through the S3 client of `internal/objstore`'s existing probe helper if one exists, else a minimal AWS SDK client), T7 (`release.MaterializeHeld(ctx, s, optDir, rt, version)` with `version` the source report's `Release` (T7 fails "the source has not reported its release yet" without it), Task 7b; on a hardened box it returns `ErrStageBlocked`, and then the pass writes `<DataDir>/moves/held-stage/<host>.json` and touches the `stage-requested` marker through `stageRequester(dd)` (`agent.go:1330`, the existing relay to the root `carlos-stage.service`, whose `carlos adopt --stage-only` now calls `release.ServeHeldStageRequests`); T7 passes only once `release.LiveVersion(optDir, rt.Host)` equals the pinned version, and until then fails "waiting for the root stage pass"; the report records the version; round-4 finding 6); T8 and T9 bind to the named target: this box's id must equal `op.intent.target` (`selectMoveBoxes`'s choice, Task 4), else T9 fails "this box is not the move's target"; the report writer is the named target only. S2 (`ExecBackend.NextSpawnStrict(host)`, Task 7b); S1 wakes a hibernating source and reads the answer from its own process, never from the response cache (round-4 finding 11: `edge.go:1651-1663` serves a cached GET before the activator runs, so a cached 200 could certify a source that cannot wake): `(*edge.Edge).ServeUncached(w, r)` (Task 14 adds it: `ServeHTTP` with the request marked `uncachedProbe` in its context, the same marker pattern as `doorCallerFrom`, which makes `cacheable` false at `edge.go:1651` so neither `GetFor` nor `maybeStore` runs, while the hibernating branch still wakes through `e.act.ServeHTTP`) with `GET /<ready path>`, `Host: <host>`, a recording `ResponseWriter` and a 30 s timeout; `moveCfg.ReadyProbe` is set by `setMovesReadyProbe(e)` from `edge.RunWithActivatorReady`'s callback (`agent.go:613`, where the agent receives its `*edge.Edge`) and fails "edge not ready" before that. Direct socket dialing is not used: it cannot wake a sleeping instance. Writes `TargetReportKey` with `Inputs`, `AttemptsAdmitted` and the measurements. The host-side pass and its test are **written in Task 14** (they need its harness; D12); this task's deliverable is the pure package and `moves.PutReplace`. |
| |
| Tests (pure) `internal/moves/readiness_test.go`: `TestReadyNeedsFreshReports` below, and `TestReadyRejectsMismatchedInputs` (two fresh passing reports whose `target.Inputs.FilesPlain` and `source.FilesPlain` differ → not ready, the reason naming both digests; the same for `Inputs.EnvReport` against `source.Env.Digest`; equal inputs → ready): |
| |
| @@ -3293,6 +3320,7 @@ type SourceReport struct { |
| S1 Check `json:"s1"` |
| S2 Check `json:"s2"` |
| FilesPlain string `json:"files_plain"` |
| + Release string `json:"release"` |
| Archive *ArchiveRef `json:"archive,omitempty"` |
| } |
| |
| @@ -3378,7 +3406,7 @@ T4 against a real store (§14.1): add `TestT4StorageProbeMinio` in `internal/hos |
| - [ ] **Step 5: Commit** |
| |
| ```bash |
| -git add internal/moves/readiness.go internal/moves/readiness_test.go internal/host/movesreport.go internal/host/movesreport_test.go |
| +git add internal/moves/readiness.go internal/moves/readiness_test.go |
| git commit -m "moves: readiness reports T1 to T12, S1, S2 and the promotion rule (§7.3) |
| |
| Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>" |
| @@ -3494,24 +3522,26 @@ Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>" |
| |
| **Files:** |
| - Create: `internal/host/movesfloor.go`, `internal/host/movesfloor_test.go` |
| -- Modify: `internal/host/edgesfile.go` (fields and writer), `internal/host/edgesfile_test.go`, `internal/console/instancesplacement.go` (reader struct), `internal/host/systemapply.go` (`armFloor`, `floorAllows`, the `decidePrev` sidecar, the rollback hold rule), `internal/host/systemapply_test.go`, `internal/console/server.go` (publish the console's fingerprint and protocol at startup, switch on only), `internal/host/units/carlos-system-recover.service` and `internal/host/units/carlos-system-update.service` (the `.prev` selection by the floor, `prev_ok`), `internal/host/host_test.go` (the unit assertions and the shell-level selection test), `internal/host/movesfloor.go` and `internal/host/movesfloor_test.go` (the agent's hourly refresh of the local floor files and the edges-file fields' readers) |
| +- Modify: `internal/host/edgesfile.go` (fields and writer), `internal/host/edgesfile_test.go`, `internal/host/systemapply.go` (`armFloor`, `activationAllowed`, the `decidePrev` sidecar and verdict), `internal/host/systemapply_test.go`, `internal/console/server.go` (publish the console's fingerprint and protocol at startup, switch on only), `internal/host/units/carlos-system-recover.service` and `internal/host/units/carlos-system-update.service` (the `.prev` selection by the floor, `prev_ok`), `internal/host/host_test.go` (the unit assertions and the shell-level selection test), `internal/host/movesfloor.go` and `internal/host/movesfloor_test.go` (the agent's hourly refresh of the local floor files and the edges-file fields' readers) |
| |
| **Interfaces:** |
| -- Produces: `moves.ConsoleKey = "control/moves-console.json"`, `moves.ConsoleRecord{V int; Protocol int; SecretsKeyFP string; At time.Time}`; `moves.FloorFile = ".moves/floor.json"` and `moves.FloorMinReleaseFile = ".moves/floor.min-release"` (relative to `DataDir`); `moves.PrevVersionFile = "carlos.prev.version"` (beside `.prev` under `optDir`). `moves.ProtocolVersion`, `FloorKey`, `Floor`, `ReadFloor` and `ReleaseAtOrAbove` **are Task 1's** (`floor.go`; Task 4 reads them). |
| -- `edgesFile.MovesProtocol int \`json:"moves_protocol"\`` (not omitempty: absent means a box below any floor); `edgesFile.MovesFloorMinRelease string \`json:"moves_floor_min_release,omitempty"\`` (the local floor file's `min_release`; absent means the box has not armed a floor); `edgesFile.PrevRelease string \`json:"prev_release,omitempty"\`` (`carlos.prev.version`'s content; `"none"` when there is no `.prev`; absent means unknown, which the door treats as below the floor). |
| -- **The floor is enforced at executable selection, by the units themselves, and by the applier** (§11, "every path that activates an executable"; round-5 findings 3 and 4). Three durable files, all written synchronously by root, never deleted once written: (1) `<DataDir>/.moves/floor.json` and the one-line `<DataDir>/.moves/floor.min-release`, written by `carlos system apply` from `control/moves-protocol.json` **before it stages anything** (`install`'s first step, after `openSystemStore`; temp file, fsync, rename) and refreshed by the agent's hourly edges-file loop (the agent may write `/data`, `carlos-edge.service:37`; `<optDir>` it may not, round-4 finding 6); (2) `<optDir>/carlos.prev.version`, written by `decidePrev` (`systemapply.go:506-522`, which already runs `.prev version` to learn it) with that version, durably, **before the update journal is written**, so a `.prev` and its sidecar are never out of step at a recovery. The units then select the executable with the floor in hand. Both `ExecStart` lines become `/bin/sh -c 'prev_ok() { [ -s /data/.moves/floor.min-release ] || return 0; [ -s /opt/carlos/carlos.prev.version ] || return 1; f=$(cat /data/.moves/floor.min-release); p=$(cat /opt/carlos/carlos.prev.version); [ "$(printf "%s\n%s\n" "$f" "$p" | sort -V | head -n1)" = "$f" ]; }; if [ -f /opt/carlos/system/router/journal.json ] && [ -x /opt/carlos/carlos.prev ] && prev_ok; then exec /opt/carlos/carlos.prev system apply --recover; fi; exec /opt/carlos/carlos system apply --recover'` (`system apply` without `--recover` in the update unit; `sort -V` is GNU coreutils, on every Debian box; the `v` prefix sorts naturally; equal is at-or-above). No floor file → `prev_ok` is true and the line behaves exactly as today. A floor file with a `.prev` below it, or without a sidecar → the current `carlos system apply` runs, and its `floorAllows` refuses a below-floor activation itself: `install` for `req.Version` (after arming), `rollback` and `restoreFiles` for the sidecar's version, **reading only the local floor file** in `rollback` and `restoreFiles` (boot recovery runs before networking, `systemapply.go:109`). `host_test.go:1314-1330`'s unit assertions gain `prev_ok`, `floor.min-release` and `carlos.prev.version` for both units (red before the unit edit). |
| +- Produces: `moves.ConsoleKey = "control/moves-console.json"`, `moves.ConsoleRecord{V int; Protocol int; SecretsKeyFP string; At time.Time}`; `moves.FloorFile = ".moves/floor.json"` and `moves.FloorMinReleaseFile = ".moves/floor.min-release"` (relative to `DataDir`); `moves.PrevVersionFile = "carlos.prev.version"` and `moves.PrevAllowedFile = "carlos.prev.allowed"` (beside `.prev` under `optDir`). `moves.ProtocolVersion`, `FloorKey`, `Floor`, `ReadFloor` and `ReleaseAtOrAbove` **are Task 1's** (`floor.go`; Task 4 reads them). |
| +- `edgesFile.MovesProtocol int \`json:"moves_protocol"\`` (not omitempty: absent means a box below any floor); `edgesFile.MovesFloorMinRelease string \`json:"moves_floor_min_release,omitempty"\`` (the local floor file's `min_release`; absent means the box has not armed a floor); `edgesFile.PrevRelease string \`json:"prev_release,omitempty"\`` (`carlos.prev.version`'s content; `"none"` when there is no `.prev`; absent means unknown); `edgesFile.PrevAllowedFor string \`json:"prev_allowed_for,omitempty"\`` (`carlos.prev.allowed`'s content, the floor `.prev` was judged at or above; absent when there is no verdict, which the door treats as below the floor). |
| +- **The floor is enforced at executable selection, by the units themselves, and by the applier** (§11, "every path that activates an executable"; round-5 findings 3 and 4; round-6 findings 4 and 5). Durable files, all written synchronously by root, never deleted once written except where said: (1) `<DataDir>/.moves/floor.json` and the one-line `<DataDir>/.moves/floor.min-release`, written by `carlos system apply` from `control/moves-protocol.json` **before it stages anything** (`install`'s first step, after `openSystemStore`; temp file, fsync, rename) and refreshed by the agent's hourly edges-file loop (the agent may write `/data`, `carlos-edge.service:37`; `<optDir>` it may not, round-4 finding 6); (2) `<optDir>/carlos.prev.version`, written by `decidePrev` (`systemapply.go:506-522`, which already runs `.prev version` to learn it) with that version, durably, **before the update journal is written**; (3) the **verdict file** `<optDir>/carlos.prev.allowed`, whose content is the floor `min_release` that `.prev` was judged against: written when `moves.ReleaseAtOrAbove(prevVersion, floor.MinRelease)` holds **and** the registry holds no `hold_op` row (§11, "any downgrade of a participant in a held operation is refused": a `.prev` is a downgrade by definition), removed otherwise. The judge is `host.judgePrev(optDir, dataDir, reg)`: run by `decidePrev` at every apply (before the journal) and by every root `carlos adopt` pass (the hourly timer and `carlos-stage.service`), so a raised floor or a new hold is re-judged within the hour, or within seconds when the agent touches the `stage-requested` marker, which every `SetHold` caller does (Task 12's rule). The units then select the executable by **file equality alone**: both `ExecStart` lines become `/bin/sh -c 'if [ -f /opt/carlos/system/router/journal.json ] && [ -x /opt/carlos/carlos.prev ] && { ! [ -s /data/.moves/floor.min-release ] || cmp -s /data/.moves/floor.min-release /opt/carlos/carlos.prev.allowed; }; then exec /opt/carlos/carlos.prev system apply --recover; fi; exec /opt/carlos/carlos system apply --recover'` (`system apply` without `--recover` in the update unit). The line contains **no `%` and no `$`**: systemd expands `%` specifiers and `$VAR` references in `ExecStart` before the shell sees them (`systemd.unit(5)`, `systemd.service(5)`), which is what sank rev 6's `printf "%s\n%s\n" ... | sort -V` form (round-6 finding 4), and `sort -V` would also have ordered `v0.30.0-rc.1` above `v0.30.0`, unlike `ReleaseAtOrAbove`; the version comparison now happens once, in Go, when the verdict is written, and the shell only compares the verdict's floor with the current floor (`cmp`, diffutils, on every Debian box). No floor file → `.prev` as today. A floor file and no verdict, or a verdict for an older floor → the current `carlos system apply` runs, and its `activationAllowed` refuses a bad activation itself (below). `host_test.go:1314-1330`'s unit assertions gain `floor.min-release`, `carlos.prev.allowed` and `cmp -s` for both units (red before the unit edit). |
| + |
| + **The applier's own rule**, `(a *systemApplier) activationAllowed(version string) error`, covers every activating path with both halves of §11 (round-6 finding 5): below `Floor.MinRelease` (reading **only** `<DataDir>/.moves/floor.json`; boot recovery runs before networking, `systemapply.go:109`) is refused, and when `reg.HoldRows()` (Task 12) is non-empty, any `version` below the running release (`systemBinaryPath` `version`, the same `ops.run` `decidePrev` uses) is refused with "a participant in a held operation does not downgrade": called in `install` for `req.Version` (after `armFloor`), in `rollback` and in `restoreFiles` (which `recoverIfNeeded` calls directly, bypassing `rollback`) for the version `carlos.prev.version` records (a `.prev` without a sidecar is below any armed floor). |
| |
| - **The admission-to-enforcement boundary** (round-5 finding 3): the console admits a move only when every edges file of both regions publishes `moves_floor_min_release` equal to the published floor's `min_release` (so the box has written its local floor files and its units already select by them) and `prev_release` either `"none"` or at or above it (Task 4's `FloorOK`). So at the moment a move is committed, every participant's recovery already selects by the floor and its `.prev` is already at or above it; nothing asynchronous has to happen on a box after admission, and a crash right after the commit recovers correctly. A floor raised later is enforced at the next recovery by the same files once the hourly refresh has written them, and the door refuses new moves until every box's file shows the new value. |
| + **The admission-to-enforcement boundary** (round-5 finding 3): the console admits a move only when every edges file of both regions publishes `moves_floor_min_release` equal to the published floor's `min_release` (the box has written its local floor files, so its units already select by them) and either `prev_release == "none"` or `prev_allowed_for` equal to that same `min_release` (the box's `.prev` has been judged at or above the floor, by the same `ReleaseAtOrAbove` the console would use; Task 4's `FloorOK`). So at the moment a move is committed, every participant's recovery already selects by the floor and its `.prev` is already judged; nothing asynchronous has to happen on a box after admission, and a crash right after the commit recovers correctly. A floor raised later invalidates every verdict at once (the floor file changes, the verdict names the old floor) until root re-judges, and the door refuses new moves until every box's file shows the new value. |
| |
| - **Dormancy (D13; round-5 finding 4):** the floor is the one Phase 1 element not behind the switch, by design: it is a safety property armed by an operator action (publishing `control/moves-protocol.json`, create-only, before the first production move, §11), not by the switch. A deployment that never published the object has no floor files anywhere, `prev_ok` is true, `floorAllows` finds no local file and proceeds with one log line, and the registry is not consulted: `main` behaves exactly as today. Once armed, the floor stays armed if the switch is later turned off. `rollback` additionally refuses while the registry holds any `hold_op` row (§11, "any downgrade of a participant in a held operation"); no such row exists in a deployment where the switch was never on. The agent writes no marker and no `participants.json`: rev 5's `hold-prev` marker is withdrawn (its asynchronous publication left exactly the window the review named). |
| + **Dormancy (D13; round-5 finding 4):** the floor is the one Phase 1 element not behind the switch, by design: it is a safety property armed by an operator action (publishing `control/moves-protocol.json`, create-only, before the first production move, §11), not by the switch. A deployment that never published the object has no floor files anywhere, `prev_ok` is true, `activationAllowed` finds no local file and proceeds with one log line, and the registry is not consulted: `main` behaves exactly as today. Once armed, the floor stays armed if the switch is later turned off. `activationAllowed`'s downgrade half consults the registry only when a floor is armed; no `hold_op` row exists in a deployment where the switch was never on. The agent writes no marker and no `participants.json`: rev 5's `hold-prev` marker is withdrawn (its asynchronous publication left exactly the window the review named). |
| |
| - [ ] **Step 1: Write the failing tests** |
| |
| -(`TestFloorReadAndCompare` and `floor.go` are Task 1's.) In `internal/host/systemapply_test.go`, beside the existing applier tests: `TestSystemApplyArmsTheFloorBeforeStaging` (the store holds `min_release: "v0.30.0"`; `install` of a pointer at `v0.29.0` writes both local floor files first, then refuses with "moves protocol floor" and leaves the binary untouched); `TestBootRecoveryUsesTheArmedFloorOffline` (the store errors on every read; the local floor names `v0.30.0`; `carlos.prev.version` says `v0.29.0` → `restoreFiles` refuses; `v0.30.0` → proceeds; no sidecar → refuses); `TestDecidePrevWritesTheVersionSidecarBeforeTheJournal` (the fake `ops.run` answers `version`; the sidecar exists with `.prev`'s version before `journal.json` appears); `TestFloorIsDormantWithoutTheObject` (no object and no local files, a registry with a held row: `install`, `rollback` and `restoreFiles` all proceed; then an armed local file and the switch off: a below-floor `restoreFiles` still refuses); `TestRollbackRefusesUnderAHold` (a `hold_op` row refuses `rollback`; none does not). In `host_test.go`: `TestSystemUnitsSelectPrevByTheFloor` (both `ExecStart` scripts are extracted from the unit text and run under `sh -c` with `/opt/carlos` and `/data` rewritten to a temp root and stub `carlos`/`carlos.prev` scripts that record which ran: no floor file → `.prev`; floor `v0.30.0` and sidecar `v0.29.0` → current; sidecar `v0.30.0` → `.prev`; floor present and no sidecar → current). In `edgesfile_test.go`: `TestEdgesFilePublishesMovesProtocolFloorAndPrev` (`"moves_protocol":1`; `moves_floor_min_release` from the local file, absent without it; `prev_release` from the sidecar, `"none"` without a `.prev`). The door (Task 4) reads the live snapshot and `complete()` (every edges file and every status snapshot read; the floor question needs every file, and liveness is already required for eligibility). |
| +(`TestFloorReadAndCompare` and `floor.go` are Task 1's.) In `internal/host/systemapply_test.go`, beside the existing applier tests: `TestSystemApplyArmsTheFloorBeforeStaging` (the store holds `min_release: "v0.30.0"`; `install` of a pointer at `v0.29.0` writes both local floor files first, then refuses with "moves protocol floor" and leaves the binary untouched); `TestBootRecoveryUsesTheArmedFloorOffline` (the store errors on every read; the local floor names `v0.30.0`; `carlos.prev.version` says `v0.29.0` → `restoreFiles` refuses; `v0.30.0` → proceeds; no sidecar → refuses); `TestDecidePrevWritesTheSidecarAndVerdictBeforeTheJournal` (the fake `ops.run` answers `version`; the sidecar and the verdict exist before `journal.json` appears; the verdict names the floor; with `.prev` below the floor, or a `hold_op` row present, no verdict is written and an old one is removed); `TestHeldParticipantRefusesEveryDowngrade` (round-6 finding 5: floor `v0.30.0`, running `v0.32.0`, a `hold_op` row: `install` of `v0.31.0`, `rollback` and `restoreFiles` to a `.prev` at `v0.31.0` all refuse; without the row all three proceed); `TestFloorIsDormantWithoutTheObject` (no object and no local files, a registry with a held row: `install`, `rollback` and `restoreFiles` all proceed; then an armed local file and the switch off: a below-floor `restoreFiles` still refuses). In `host_test.go`: `TestSystemUnitsSelectPrevByTheFloor` (round-6 finding 4: first asserts both `ExecStart` lines contain neither `%` nor `$`, the two expansions systemd applies before the shell runs, then extracts the `sh -c` body from the unit text and runs it with `/opt/carlos` and `/data` rewritten to a temp root and stub `carlos`/`carlos.prev` scripts that record which ran: no floor file → `.prev`; floor `v0.30.0` and a verdict naming `v0.30.0` → `.prev`; a verdict naming `v0.29.0` → current; floor present and no verdict → current). In `edgesfile_test.go`: `TestEdgesFilePublishesMovesProtocolFloorAndPrev` (`"moves_protocol":1`; `moves_floor_min_release` from the local file, absent without it; `prev_release` from the sidecar, `"none"` without a `.prev`; `prev_allowed_for` from the verdict, absent without one). The door (Task 4) reads the live snapshot and `complete()` (every edges file and every status snapshot read; the floor question needs every file, and liveness is already required for eligibility). |
| |
| - [ ] **Step 2: Run tests to verify they fail**, **Step 3: implement**: |
| |
| -`internal/moves`: `ConsoleKey`, `ConsoleRecord` and the three file-name constants only (`floor.go` is Task 1's). Edges file: add the three fields in the writer at `edgesfile.go:722-736` (`MovesProtocol: moves.ProtocolVersion`; `MovesFloorMinRelease` read from `<DataDir>/.moves/floor.min-release` and `PrevRelease` from `<optDir>/carlos.prev.version`, or `"none"` when `.prev` is absent, both read at each write); `internal/console/instancesplacement.go`'s `edgesFilePlacementView` gains the same fields. Units: the two `ExecStart` lines above. Console startup, **only when `s.cfg.MovesEnabled`** (D13: off publishes nothing; `TestConsoleRecordNeedsTheSwitch`), where the edges-dependent config is logged (`server.go`): `moves.PutReplace(ctx, s.cfg.Releases, moves.ConsoleKey, json(ConsoleRecord{V:1, Protocol: moves.ProtocolVersion, SecretsKeyFP: fp, At: now}))` with `fp, _ := appconfig.PublicFingerprint(s.cfg.SecretsPub)` (empty when unset), best-effort, logged on failure; a second console start replaces the record (a create-only `Put` would fail every restart after the first). `systemapply.go`: `(a *systemApplier) armFloor(ctx) error` (first in `install`: `moves.ReadFloor` from `a.store`, the store `openSystemStore` returned, stored on the applier (add the field if absent); when present, both local files written durably; a store error with a local file present keeps the local file; absent everywhere → no floor) and `(a *systemApplier) floorAllows(version string) error` reading **only** `<DataDir>/.moves/floor.json` and returning `fmt.Errorf("moves protocol floor: %s is below %s", version, f.MinRelease)` when below (a `.prev` without a sidecar is below any armed floor); called in `install` for `req.Version` after `armFloor`, and in `rollback` and `restoreFiles` for the version `carlos.prev.version` records; a refusal in `restoreFiles` leaves the journal and returns the error (the box stays on the failed build, paged by the existing result path), per §11. `decidePrev` writes the sidecar durably with the version it already obtains from `.prev version`, before the journal. `rollback` refuses while `reg.HoldRows()` (Task 12) is non-empty. |
| +`internal/moves`: `ConsoleKey`, `ConsoleRecord` and the three file-name constants only (`floor.go` is Task 1's). Edges file: add the four fields in the writer at `edgesfile.go:722-736` (`MovesProtocol: moves.ProtocolVersion`; `MovesFloorMinRelease` read from `<DataDir>/.moves/floor.min-release`, `PrevRelease` from `<optDir>/carlos.prev.version` or `"none"` when `.prev` is absent, `PrevAllowedFor` from `<optDir>/carlos.prev.allowed`; all read at each write); the console's `edgesFilePlacementView` already carries the same fields (Task 4). Units: the two `ExecStart` lines above. Console startup, **only when `s.cfg.MovesEnabled`** (D13: off publishes nothing; `TestConsoleRecordNeedsTheSwitch`), where the edges-dependent config is logged (`server.go`): `moves.PutReplace(ctx, s.cfg.Releases, moves.ConsoleKey, json(ConsoleRecord{V:1, Protocol: moves.ProtocolVersion, SecretsKeyFP: fp, At: now}))` with `fp, _ := appconfig.PublicFingerprint(s.cfg.SecretsPub)` (empty when unset), best-effort, logged on failure; a second console start replaces the record (a create-only `Put` would fail every restart after the first). `systemapply.go`: `(a *systemApplier) armFloor(ctx) error` (first in `install`: `moves.ReadFloor` from `a.store`, the store `openSystemStore` returned, stored on the applier (add the field if absent); when present, both local files written durably; a store error with a local file present keeps the local file; absent everywhere → no floor) and `(a *systemApplier) activationAllowed(version string) error` as described above (the floor half from the local file alone, `fmt.Errorf("moves protocol floor: %s is below %s", version, f.MinRelease)`; the downgrade half from `reg.HoldRows()` and the running version); a refusal in `restoreFiles` leaves the journal and returns the error (the box stays on the failed build, paged by the existing result path), per §11. `decidePrev` writes the sidecar durably with the version it already obtains from `.prev version`, then runs `judgePrev`, before the journal. `movesfloor.go`: `judgePrev(optDir, dataDir string, reg *registry.Registry) error` and the agent's hourly floor refresh; `carlos adopt` (every mode, root) calls `judgePrev` after its pass. |
| |
| - [ ] **Step 4: Run**: `nice -n 10 go test ./internal/moves ./internal/host ./internal/console -count=1 -p 1` → PASS. |
| |
| @@ -3529,10 +3559,10 @@ Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>" |
| ### Task 14: Preparing and ready rounds on the new layout |
| |
| **Files:** |
| -- Modify: `internal/host/movespass.go` (`moveCfg`, `armMovesPass`, `runMoveRound`'s `MovePreparing`/`MoveReady` arm, `MoveAborting` arm's prefix), `internal/host/movespass_test.go` (`fakeMoveMirror` gains `CopyVersioned`/`PutVersioned`; `moveHarness` gains a `lease.VersionedStore` fake and a staging-lease fake), `internal/host/agent.go` (`setMovesReadyProbe` in the `RunWithActivatorReady` callback), `internal/edge/edge.go` (`ServeUncached` and the `uncachedProbe` context marker beside `doorCallerFrom`), `internal/edge/edge_test.go` (`TestServeUncachedBypassesTheCacheAndWakes`: the cache primed with a 200 for the path and a fake activator whose wake fails: `ServeHTTP` answers the cached 200, `ServeUncached` answers the wake failure and stores nothing) |
| +- Modify: `internal/host/movespass.go` (`moveCfg`, `armMovesPass`, `runMoveRound`'s `MovePreparing`/`MoveReady` arm, `MoveAborting` arm's prefix), `internal/host/movespass_test.go` (`fakeMoveMirror` gains `CopyVersioned`/`PutVersioned`; `moveHarness` gains a `lease.VersionedStore` fake and a staging-lease fake), `internal/host/agent.go` (`setMovesReadyProbe` in the `RunWithActivatorReady` callback), `internal/host/movesreport.go`, `internal/host/movesreport_test.go`, `internal/host/movesreport_minio_test.go` and `.amadan/ci.d/30-minio-contract` (Task 11's host side, built here), `internal/lease/mirror.go` (`MirrorPrefixKeep`), `internal/edge/edge.go` (`ServeUncached` and the `uncachedProbe` context marker beside `doorCallerFrom`), `internal/edge/edge_test.go` (`TestServeUncachedBypassesTheCacheAndWakes`: the cache primed with a 200 for the path and a fake activator whose wake fails: `ServeHTTP` answers the cached 200, `ServeUncached` answers the wake failure and stores nothing) |
| |
| **Interfaces:** |
| -- `moveCfg` gains: `Box string` (the box label `metricsBoxLabel` gives), `DeployBucket, DeployBucketRegion string` (from `CARLOS_DEPLOYMENT_BUCKET` and `CARLOS_AWS_REGION`/the IMDS region the edges loop resolves; **empty means Phase 1 rounds refuse** (round-5 finding 11: a 15-minute `lease.Acquire` can never satisfy the signing guard, and a per-participant bucket is not one shared coordination location): the round writes S1 "this deployment has no deployment bucket for the staging lease" and does nothing else; the harness supplies a staging-lease fake, so tests never need the bucket), `Versioned func(ctx, bucket, region string) (lease.VersionedStore, error)`, `StagingLease func(ctx, targetRegion, host string) (moves.SafeUntilGuard, func(context.Context) error, error)`, `SecretsKey string` (`CARLOS_SECRETS_KEY`), `SecretsPub *ecdh.PublicKey`, `LaunchEnv func(host string) (map[string]string, error)`, `Timers func(host string) ([]registry.Timer, error)`, `FilesDir func(host string) (dir, envPath string)` (flat: `<DataDir>/<host>/`, `<DataDir>/<host>.env`), `ReadyProbe func(ctx, host string) moves.Check` (S1 through `(*edge.Edge).ServeUncached`, Task 11's definition; set by `setMovesReadyProbe(e *edge.Edge)` from `RunWithActivatorReady`'s callback, `agent.go:613`; "edge not ready" before that), `StrictNext func(host string) moves.Check` (S2: `ExecBackend.NextSpawnStrict`, Task 7b). |
| +- `moveCfg` gains: `Box string` (the box label `metricsBoxLabel` gives), `DeployBucket, DeployBucketRegion string` (from `CARLOS_DEPLOYMENT_BUCKET` and `CARLOS_AWS_REGION`/the IMDS region the edges loop resolves; **empty means Phase 1 rounds refuse** (round-5 finding 11: a 15-minute `lease.Acquire` can never satisfy the signing guard, and a per-participant bucket is not one shared coordination location): the round writes S1 "this deployment has no deployment bucket for the staging lease" and does nothing else; the harness supplies a staging-lease fake, so tests never need the bucket), `Versioned func(ctx, bucket, region string) (lease.VersionedStore, error)`, `StagingLease func(ctx, targetRegion, host string) (moves.SafeUntilGuard, func(context.Context) error, error)`, `SecretsKey string` (`CARLOS_SECRETS_KEY`), `SecretsPub *ecdh.PublicKey`, `LaunchEnv func(host string) (env map[string]string, shadowed []string, err error)`, `Timers func(host string) ([]registry.Timer, error)`, `FilesDir func(host string) (dir, envPath string)` (flat: `<DataDir>/<host>/`, `<DataDir>/<host>.env`), `ReadyProbe func(ctx, host string) moves.Check` (S1 through `(*edge.Edge).ServeUncached`, Task 11's definition; set by `setMovesReadyProbe(e *edge.Edge)` from `RunWithActivatorReady`'s callback, `agent.go:613`; "edge not ready" before that), `StrictNext func(host string) moves.Check` (S2: `ExecBackend.NextSpawnStrict`, Task 7b). |
| - The switch (Task 0, D13): `moveCfg.Enabled` decides which worker runs. **Off:** today's `runMoveRound` (renamed `legacyMoveRound`, unchanged, with its tests unchanged) handles every record exactly as main does; an id'd record (none exists while the switch was never on) is skipped with one log line a minute. **On:** `phase1MoveRound` below handles id'd records; a legacy record is refused with one log line a minute (M-C: MV-N7's sentence in the log, not to a user). |
| - The interim abort (until Plan 5 Task 1 replaces it): for `aborting`, the source deletes its own `move-snapshots/<host>/<id>/` and receipts, then `hostclaim.Update(ReleaseMove(id, cfg.Box))` (Task 3), then `MoveGuardedAbortComplete(g, now)`; **the crash boundary between those two writes** (round-5 finding 10): a round that finds the record `aborting` and the claim already `serving` completes the record only when `hostclaim.RequireReleasedMove(id)` holds (the serving op's journal names this move as the one it released, Task 3); a serving claim without that evidence was released by someone else (the console pass, Plan 5) and the round leaves the record to it with one log line; `TestInterimAbortReleasesTheClaim` adds the crash case (stop the round after the claim release; run it again; the record completes; and a serving claim with no evidence does not complete it); the staging lane in the target bucket is left for Plan 5's target-owned park (the source deletes nothing there: M17 allows it, INV-8 does not want it). |
| - **Every transition is guarded by op identity** (Task 2's `MoveGuard{ID: rec.Move.ID, Incarnation: rec.Move.Incarnation}`, captured when the round read the record): `MoveGuardedPromoteReady`, `MoveGuardedCut`, `MoveGuardedAbortComplete`; a replacement move between the read and the CAS is `ErrMoveConflict` and the round ends (`TestRoundNeverAdvancesAReplacementMove`). |
| @@ -3540,14 +3570,14 @@ Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>" |
| - The round, per in-flight record this box owns, in order (§7.3, K5, K12), **after** taking the staging lease (`lease.AcquireTTL(..., moves.StagingLeaseTTL)`) and **then** re-reading the claim under it (§7.3: every round takes the lease first, then revalidates the claim's op id and phase): refuse a legacy record (log once a minute, `return nil`); refuse a nested row (`rt.DBPath != provision.DefaultDBPath(dataDir, host)`: "nested instances cannot move yet"); refuse a source with `<DataDir>/<host>-sidecar.env` present or any refused registry column set on its row (`Caps`, `Unit`, `Backing`, `Platform`, `Tunnel`, `FleetBox`, `Pool`, `Canary`, `Addr`, `Addrs`, `Health`, `Replica`, `Replicas`; §6.2, §7.1), writing the reason into the source report's `S1` so the operator sees it in `carlos move status`; refuse likewise (S1, M-C17's sentence) a source whose store binding exists: `objstore.GetIn(ctx, s, acct, app, provision.InstanceStoreEnv(rec))` non-nil, the door's check (Task 4) repeated every round before the first staging write, a read error ending the round; `hostclaim.RequireMoveOp(id, KindMoving, "")` on the claim read under the lease and holder incarnation = `rec.Move.Incarnation`; if the target region or the target box (`Op.Intent["target"]`) is not in `Touched`, `hostclaim.Update(... Touch(id, []string{to}, []string{targetBox}))` (K12 and T9 both hold from the first staging write; the target's `checking` re-adds its box idempotently); `move.json`: `stagingMirror.PutIfAbsentVersioned(move.json)` through the guarded staging mirror (Task 6; **every** write into the target bucket, `move.json`, `replica/`, `files/`, `timers/` and Task 11's `probe/` measurement object, goes through the one guarded path: `guardedMirror` implements `lease.MirrorStore`, `VersionedCopier` and `VersionedPutter` over `*lease.StagingMirror` by calling `Guarded.Before` and passing its `signAt` to `CopyVersionedAt`, `PutVersionedAt`, `PutIfAbsentVersionedAt` and `DeleteAt`, Task 6; nothing calls `lease.VersionedStore` against the target bucket); success records the returned version in the ledger; `lease.ErrExists` records `move_json: "present"` and the round goes on. The source cannot tell its own lost-response PUT from a foreign object (M17 forbids the read, and a ledger intent written first cannot tell them apart either: round-2 finding 7), so **the target owns that validation**: T9 reads `moves/<host>/<id>/move.json` in its own bucket and requires `move_id == rec.Move.ID`, `incarnation == holder.incarnation`, `host`, `from`, `to` all equal to the record's; a mismatch fails T9 with "the staging lane holds another move's move.json" and the move never becomes ready (the operator aborts; the target's park retires the lane); `checking` repeats the same check before any local write. The id is 128 random bits, so the case is a replayed request or a bug, never a collision in practice; `MirrorPrefixKeep(ReplicaSource(holder) → ReplicaPrefix, keep)` through the guarded staging mirror, whose caller retries a failed call only by calling again (a new check, a new signature); compute `archive.PlainDigest`; when it differs from the receipts' recorded plain, `archive.Build` + `archive.Seal` to a temp file, `PutVersioned` to `FilesPrefix+"files-<sealed>.sealed"` and `.json` (manifest), record both keys and both returned versions in the ledger's `archive_versions` and publish them as the source report's `Archive`; write the source report with `LaunchEnv` → `moves.ReportEnv`, S1 and S2; read the target report; `moves.Ready(round.DeltaBytes(), cfg.ReadyDelta, target, source, now)`; on ready from preparing, `MoveGuardedPromoteReady(g, now)` and `nudgeSerial`; in `ready`, keep mirroring. For `cut == "auto"` and `ready`, the source itself CASes `MoveGuardedCut(g)` (Task 2's signature: the cut stamps nothing at `requested`) (§7.4 "cut door or the source (auto-cut)"); Plan 2 picks it up. |
| - Receipts: `lease.Receipts{Bucket: cfg.OwnBucket, Key: moves.ReceiptsKey(host, id)}`; a per-op `moveLedger` JSON beside it at `move-receipts/<host>/<id>-moves.json` records `move_json_version`, `archive_plain`, `archive_sealed`, `archive_versions`. |
| |
| -Tests: the legacy tests (`TestMovesPassPrepareAndPromote`, every `TestMovesPassCut…`, `TestMovesPassAbort…`) stay as they are and run with the switch off; the new tests run the harness with `cfg.Enabled = true` and a `MoveStart`ed record: `TestPhase1PrepareAndPromote` (keys under `moves/<host>/<id>/replica/<level>/…`, `move.json` present; the round is not promoted until a passing fresh `target.json` exists in the harness store), `TestPhase1IsInertWithTheSwitchOff` (an id'd record and the switch off: no copy, no CAS, one log line), `TestStagingMoveJSONReplayAfterLostLedger` (Review Focus 2: the fake versioned store answers `ErrExists` and refuses every `GetVersion`/`OpenVersion` of the target lane with an access-denied error; the round proceeds and records `present`; the target-side `TestTargetReportPassFlagsEachCheck` has the matching case: a `move.json` naming another move id fails T9 with the lane sentence), `TestInterimAbortReleasesTheClaim`, `TestRoundNeverAdvancesAReplacementMove`, `TestReplicaSourceSelectsTheClaimsGeneration`, `TestEveryStagingWriteIsGuarded` (a `Guarded` whose `Now` is past the bound: the round issues no request of any class, `move.json`, replica copy, archive, timers or probe object), `TestPreparingRefusesLegacyNestedAndRefusedColumns`, `TestPreparingTouchesTheTargetRegionAndBoxFirst` (the claim's `touched.regions` gains `uk` and `touched.boxes` gains `uk-box` before the fake mirror sees a copy, from a claim that never touched either), `TestStagingWriterRetriesWithANewCheck` (the fake mirror fails one copy; the lane calls `Before` again before the retry; a `Now` past the bound makes the retry `ErrLeaseUnsafe` and the round ends without a second SDK call), `TestSourceReportIsWrittenEachRound` (two rounds, the second's report replaces the first: `PutReplace`), `TestArchiveIsRebuiltOnlyWhenPlainChanges`, `TestAutoCutMarksCuttingFromReady`. |
| +Tests: the legacy tests (`TestMovesPassPrepareAndPromote`, every `TestMovesPassCut…`, `TestMovesPassAbort…`) stay as they are and run with the switch off; the new tests run the harness with `cfg.Enabled = true` and a `MoveStart`ed record: `TestPhase1PrepareAndPromote` (keys under `moves/<host>/<id>/replica/<level>/…`, `move.json` present; the round is not promoted until a passing fresh `target.json` exists in the harness store), `TestPhase1IsInertWithTheSwitchOff` (an id'd record and the switch off: no copy, no CAS, one log line), `TestStagingMoveJSONReplayAfterLostLedger` (Review Focus 2: the fake versioned store answers `ErrExists` and refuses every `GetVersion`/`OpenVersion` of the target lane with an access-denied error; the round proceeds and records `present`; the target-side `TestTargetReportPassFlagsEachCheck` has the matching case: a `move.json` naming another move id fails T9 with the lane sentence), `TestInterimAbortReleasesTheClaim`, `TestRoundNeverAdvancesAReplacementMove`, `TestReplicaSourceSelectsTheClaimsGeneration`, `TestEveryStagingWriteIsGuarded` (a `Guarded` whose `Now` is past the bound: the round issues no request of any class, `move.json`, replica copy, archive, timers or probe object), `TestPreparingRefusesLegacyNestedAndRefusedColumns`, `TestPreparingTouchesTheTargetRegionAndBoxFirst` (the claim's `touched.regions` gains `uk` and `touched.boxes` gains `uk-box` before the fake mirror sees a copy, from a claim that never touched either), `TestStagingWriterRetriesWithANewCheck` (the fake mirror fails one copy; the lane calls `Before` again before the retry; a `Now` past the bound makes the retry `ErrLeaseUnsafe` and the round ends without a second SDK call), `TestSourceReportIsWrittenEachRound` (two rounds, the second's report replaces the first: `PutReplace`), `TestArchiveIsRebuiltOnlyWhenPlainChanges`, `TestAutoCutMarksCuttingFromReady`, `TestPreparingRelaysHeldStageOnAHardenedBox` (round-6 finding 3: from a `MoveStart`ed record with **no** target registry row, the target pass's `materializeHeldStage` seam fails with `EROFS`; the pass writes `moves/held-stage/<host>.json` naming the source report's `Release` and touches the `stage-requested` marker; `release.ServeHeldStageRequests` with `heldStageAuthority` (claim `moving`, target this box, no row, the report's `Release`) materializes into the temp `optDir`; the next pass finds `LiveVersion` equal and T7 passes; a request naming a version the source report does not, or an op the claim does not, is refused and `live/<host>` untouched). |
| |
| - [ ] **Step 1: Write the failing tests**, **Step 2: run**, **Step 3: implement**, **Step 4: run** `nice -n 10 go test ./internal/host -run 'TestMovesPass|TestStaging|TestPreparing|TestSourceReport|TestArchiveIsRebuilt|TestAutoCut' -count=1`. |
| |
| - [ ] **Step 5: Commit** |
| |
| ```bash |
| -git add internal/host internal/edge internal/moves |
| +git add internal/host internal/edge internal/moves internal/lease .amadan/ci.d/30-minio-contract |
| git commit -m "host: preparing and ready rounds on the per-operation staging layout, under the staging lease (§7.3) |
| |
| Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>" |