docs: region HA stage B plan rev 4 answers Astra round 3: O2' as (a) or (b') bound to the pinned writer and the delivered key, C10 by persisted binding with discharge before the hold is released, a bounded wait for the probe's stop, deleted env items installed as none
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 file changed,
+669
−143
docs/superpowers/plans/2026-10-10-region-ha-1b-next-stage-b.md+669 −143
CI failed — run details
diff --git a/docs/superpowers/plans/2026-10-10-region-ha-1b-next-stage-b.md b/docs/superpowers/plans/2026-10-10-region-ha-1b-next-stage-b.md| index f38d1e910..0f469f758 100644 |
| --- a/docs/superpowers/plans/2026-10-10-region-ha-1b-next-stage-b.md |
| +++ b/docs/superpowers/plans/2026-10-10-region-ha-1b-next-stage-b.md |
| @@ -1,4 +1,4 @@ |
| -# Region HA 1b-next Stage B: Relocation Wired Into the Claim Assembly Implementation Plan (rev 3) |
| +# Region HA 1b-next Stage B: Relocation Wired Into the Claim Assembly Implementation Plan (rev 4) |
| > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. |
| @@ -8,7 +8,301 @@ |
| **Tech Stack:** Go 1.26, module `github.com/carlosframework/platform`; `release.Store`; `hostclaim`, `hostactors`, `hostfence`, `hostguard`, `hostexec`, `hostbox`, `hostrelocate`, `hostfiles`, `genselect`, `moves`, `lease`, `activator`, `registry`, `appconfig`, `release`. |
| -**Spec:** `docs/superpowers/specs/2026-10-04-region-ha-box-placement-design.md` (rev 5.4 with §24): §5.3, §5.8, §6, §8. The 1b-next plan's rulings 1 to 27 stand; this plan's rulings continue the numbering at 28. Claim Plan 11 (`docs/superpowers/plans/2026-10-09-host-lifecycle-claim-11-region-ha-integration.md` on `origin/design-replica-ownership`, "Region HA integration and rollout", items 1 to 7) and the #236 answers (R1 to R10, C1 to C5, M1 to M3, J2, J3) are the contracts. Every signature below was read on this branch at 808c36a7, and rev 3's again at 7a41192c (main merged at dc333fb5); file:line cites are to this branch. |
| +**Spec:** `docs/superpowers/specs/2026-10-04-region-ha-box-placement-design.md` (rev 5.4 with §24): §5.3, §5.8, §6, §8. The 1b-next plan's rulings 1 to 27 stand; this plan's rulings continue the numbering at 28. Claim Plan 11 (`docs/superpowers/plans/2026-10-09-host-lifecycle-claim-11-region-ha-integration.md` on `origin/design-replica-ownership`, "Region HA integration and rollout", items 1 to 7) and the #236 answers (R1 to R10, C1 to C5, M1 to M3, J2, J3) are the contracts. Every signature below was read on this branch at 808c36a7, rev 3's again at 7a41192c, and rev 4's at 49277296 (main merged at dc333fb5); file:line cites are to this branch. |
| + |
| +--- |
| + |
| +## Rev 4: round 3 dispositions |
| + |
| +Astra's round 3 (`CODEX-PLAN-REGION-HA-STAGE-B-R3.md`) raised 3 P1s and 2 P2s and found the plan converging: N6 is judged sound and C11's fallback safe. Every cited path was read at 49277296, and all five findings reproduce. Counts: **accepted 5, accepted-changed 0, declined 0**. |
| + |
| +**Instance-key plan rev 3 was not committed when this revision was written.** Its branch `instance-key-build` stands at f3ac9ee3 (round 2 review), so O2′ here is written from the coordinator's description of its ruling R-B. That makes it a named dependency: **IK-R-B**, owned by the instance-key plan. The adoption record's fields, the platform item's name (`instance-key-adoption`) and the configured-env reader must match that plan's rev 3 when it lands. Until then `StorePinner.KeyEvidence` is nil, and only `instance.key` evidences a nonempty database's key (ruling 25 stands). |
| + |
| +All rev 4 code was assembled in a throwaway `git archive 49277296`, with rev 3's code restored from this plan; it is deleted. The checks run there: |
| + |
| +- `go build ./...` is clean. |
| +- `go vet` is clean on host, hostbox, hostclaim, hostrelocate, hostfiles, provision, registry and activator. |
| +- darwin, windows and linux/arm64 builds of `./internal/...` are clean, and `gofmt -l` is empty. |
| +- `hostrelocate`, `hostclaim`, `hostbox`, `hostfiles`, `registry` and `activator` pass in full. |
| +- `internal/host`'s claim, Plan 9, Plan 11, relocation, release, census, handover and participation tests pass, run with the sandbox off for their unix sockets. |
| + |
| +| # | Task | Finding | Disposition | Change | Test | |
| +|---|---|---|---|---|---| |
| +| 1 (P1) | 8b, 8c | Rev 3's O2′ accepted any non-empty env key with an adoption source, never comparing the record's digest to the key; `KeyConfig` read no env at all; the custom parser accepted `export CARLOS_INSTANCE_KEY=`, which the launch parser does not treat as that variable. | accepted | Ruling 65, which supersedes 55. O2′ has exactly two sources, per instance-key rev 3 R-B: (a) `instance.key`; or (b′) the pinned writer's own adoption record (host, incarnation, session, generation all equal to the pinned holder's) whose `key_sha256` is the digest of the exact key the target will deliver. That key is the pinned env item over the configured env, parsed and layered by the activator's own launch code (`activator.ParseEnv`, `MergeEnv`: named change N10, an unexported-to-exported wrapper; no custom parser). Source (c) is dropped. The digest is pinned (`FilesPin.KeySHA256`), and the install derives the key again from the restored env and the config it has just rendered, refusing on any difference. | `TestStorePinnerNeedsTheWritersAdoptionOfTheDeliveredKey` (written, passes; 15 cases). The cases: the nil-reader unused key; a platform key with no record; a mismatched adoption; another session's and another generation's record; a configured key overridden by the env (refuses), and the overriding env key adopted (passes); a configured key adopted; an unreadable env item (an error, never "unevidenced"); a duplicate assignment (the last wins, as at launch); an `export` line, which is not the variable; an empty key. Plus `TestRelocationInstallRefusesAKeyChangedSinceThePin` (Task 8c, specified: configuration changed between pin and install, so the install refuses). | |
| +| 2 (P1) | 9 | Completion released the hold and lifted the fence without closing the participation. An immediate request could be admitted (`writer.Admit` and `AfterAdmission` run before `persistPrelaunch` selects work, claimwakeassembly.go:311, :529), moving the holder off the probe session, after which C10's `completedPrior` refuses for good. | accepted | Ruling 66. `relocationReleaseHold` first calls `cfg.Discharge` = `productionClaimLocal.DischargeCompleted` (N1's third accessor). That is main's own `Prepare` on the serving observation, which closes the slot through C10's discharge and clears `.hostexec`, succeeding only when neither a slot nor a record is left. Only then does it release the hold and lift the activator fence. Until that point the held row keeps the edge from routing, and the fence keeps the activator from waking, so no serving admission is possible. | **Test 1**, `TestRelocationCompletionServesOnlyAfterDischarge` (harness, Task 9): the participation loop paused; a request immediately after the serving CAS is refused with the row held and nothing admitted; one pass discharges, then releases; the next request is admitted through N6 with no serving-shaped fence. Mutation: releasing the hold before the discharge fails it. Plus `TestRelocationReleaseHoldWaitsForDischarge` (unit, specified: a refusing `Discharge` leaves `ReleaseHold` and `LiftFence` uncalled). | |
| +| 3 (P1) | 9 | After a crash between completion and cleanup, `clearUntracked` passes the current serving observation to `cleanup` (claimparticipation_stale.go:31). C10 rejected it for not being a captured relocating claim, so cleanup ran a serving-shaped stop whose `BeginFence` replaces the probe's fence, and `BeginColdHandover` then refuses N6's receipt (hostguard/coldhandover.go:42). | accepted | Ruling 66. C10's text changes: the discharge is keyed by the persisted participation binding (the slot's `Config.Binding`, or after a restart the `.hostexec` record's `state.Binding`), not by any observation. It runs first in every cleanup path, so no ordinary cleanup, and no serving-shaped stop, runs for a completed relocation. `productionClaimLocal.cleanup` takes the binding (three call sites: the slot's `Cleanup`, `prepareWith`'s close, `clearUntracked`). | **Test 2**, `TestRelocationRestartAfterServingCASRecoversParticipation` (harness): the agent crashes after the serving CAS and before `ClearStale`. On restart the first participation pass runs `clearUntracked`, then C10 by binding, then `ClearStale`; the guard journal shows no new fence. The pass releases the hold, and the first serving request's N6 handover is accepted by `BeginColdHandover` with the original probe receipt. Main's `TestPlan9ParticipationCleanup*` pass with an empty binding (C10 falls through). | |
| +| 4 (P2) | 9 | With `probe` journaled and no `probe_stop`, the worker returned before any deadline; a lease lost after `RecordProbe` left a live target in `probing` indefinitely. | accepted | Ruling 67. `Probe.At` records when the probe passed (`RecordProbe` from the participant's clock). `ReleaseDeadline` after it with no stop receipt, the worker marks the op `stuck` for an operator, releasing nothing (denial, hold and session stay). A probe without `At` counts from `probing_since + ProbeDeadline`. | `TestWorkerBoundsAPassedProbeAwaitingItsStop` (written, passes): the lease lost right after `RecordProbe`, and a persistent stop refusal. Not stuck at the bound, stuck one second after; still relocating, exec unchanged, no `probe_stop`. `TestWorkerBoundsAnInstallThatNeverProbes` (ruling 60, written, passes). | |
| +| 5 (P2) | 8c | The install checked only map membership for `EnvItem`, then fetched a tombstone's (absent) object and failed, never removing a stale local env file. | accepted | Ruling 68. A deleted `EnvItem` is the same as none: the install removes any local env file and journals `env: none`. The repairing `EnsureLocal` goes through the same `Install`. | `TestRelocationInstallRemovesAStaleEnvForADeletedItem` (specified, Task 8c harness), through `Install` and through a repairing `EnsureLocal`, with a stale `<host>.env` left by an earlier install. | |
| + |
| +### C10's text on #236 (changed) |
| + |
| +> **C10 (rev 4).** A CC9 discharge of a region HA relocation participation by its persisted binding and exact completed probe evidence. In every `productionClaimLocal` cleanup path (the slot's close, `prepareWith`'s close of a retained slot, and `clearUntracked` after a restart), before any ordinary cleanup, read the claim. If it now serves on the binding's box, its op journals `completed_relocation` for exactly the binding's op, its holder is that record's stopped probe session (session, stamp, generation, writer box, incarnation equal to the binding's), and root's completed `writer-session-<S>` receipt with the recorded fence id, phase and digest is in this box's guard journal and verifies, then the participation is discharged: no stop is requested and `ClearStale` follows. Otherwise main's cleanup runs unchanged. Region HA guarantees the precondition: it releases the row's hold and the activator fence only after this discharge has cleared the slot and the record, so no serving admission can move the holder first. The proposed patch is in the plan (`claimparticipation_completed.go`, and `cleanup` taking the binding at its three call sites). |
| + |
| +### Rulings, rev 4 (65 to 69) |
| + |
| +65. **O2′ is exactly (a) or (b′)** (finding 1; supersedes 55; dependency IK-R-B). (a) The pinned manifest holds `instance.key`. (b′) The pinned manifest's `instance-key-adoption` platform item is a strictly decoded `AdoptionRecord`: v 1, host, incarnation, session and generation equal to the pinned holder's, and `key_sha256` equal to `sha256` of `EffectiveInstanceKey(configured env, pinned env item)`, which is `activator.MergeEnv(activator.ParseEnv(config), activator.ParseEnv(env))["CARLOS_INSTANCE_KEY"]`, the launch's own reading and layering. The configured env comes from `KeyEvidence.ConfiguredEnv`, rendered for the pinned release. Whichever source yields the key, it is accepted only through the writer's own adoption of it, and configuration is never evidence by itself. The digest is pinned as `FilesPin.KeySHA256`. After the held render the install computes the key again from the restored `<host>.env` and the rendered `<host>.config.env`, and refuses unless it has that digest. An unreadable env or adoption item is an error, never "no evidence". *Cost if wrong:* a host whose writer has not yet reported adoption (each new admission starts without a record) is `stuck` until its app restarts once, which is ruling 25's behaviour. |
| +66. **Participation is discharged before any serving admission is possible, by binding, in every cleanup path** (findings 2 and 3; C10's text changed). Completion's hold release waits for `DischargeCompleted`, and the activator fence lifts after it. C10 keys the discharge by the persisted binding and runs it before ordinary cleanup, so after a restart `clearUntracked` discharges the completed relocation and never asks a serving-shaped stop that would replace the probe's fence. *Cost if wrong:* if C10 is refused, the hold stays and the host is unserved on the target; the pass alerts once (the gate, as before). |
| +67. **A passed probe waiting for its stop is bounded** (finding 4). `ReleaseDeadline` after `Probe.At`, the op is `stuck` for an operator with everything held. *Cost if wrong:* an operator acts on a session whose stop root could not certify, which is the outcome Astra asks for. |
| +68. **A deleted `EnvItem` installs as none** (finding 5). Install and repair remove any stale local env file. *Cost if wrong:* none. |
| +69. **N10: the activator's env parser and layering are exported unchanged** (`ParseEnv`, `MergeEnv` in admintoken.go). They are the one implementation the pinner and the install use; the activator is not claim-owned code. *Cost if wrong:* none. |
| + |
| +### Rev 4 changes by task |
| + |
| +- **Task 8b.** Step 0 is O2′ (b′): `pin.go` and `pin_envkey_test.go` as below, with N10's activator export. `StorePinner.KeyEvidence` is `*KeyEvidence` (`ConfiguredEnv`), nil until IK-R-B lands. |
| +- **Task 8c.** `relocationinstall.go` (below) handles a deleted `EnvItem` as none, and checks the delivered key against `FilesPin.KeySHA256` after the render. |
| +- **Task 9.** |
| + - `relocationowners.go`'s `relocationReleaseHold` discharges first. |
| + - `RelocationAssemblyConfig.Discharge`, which Task 10's `relocationAssemblyConfig` takes as an argument and sets to `local.DischargeCompleted`. |
| + - `claimparticipation_relocation.go` gains `DischargeCompleted`. |
| + - C10's patch (below), now with the binding. |
| + - `hostclaim`'s `Probe.At`, and the worker's passed-probe bound. |
| + - Tests 1 and 2, specified in the harness. |
| +- **Tests edited:** `plan9_production_test.go`'s two direct `p.cleanup` calls pass `execlease.Binding{}`. |
| + |
| +### Rev 4 code |
| + |
| +N10, `internal/activator/admintoken.go`: |
| + |
| +```diff |
| +--- a/internal/activator/admintoken.go |
| ++++ b/internal/activator/admintoken.go |
| +@@ -255,3 +255,15 @@ |
| + } |
| + return m |
| + } |
| ++ |
| ++// ParseEnv is the launch path's EnvironmentFile parser, exported unchanged |
| ++// (region HA N10) so relocation reads <host>.env and <host>.config.env |
| ++// exactly as a launch does: KEY=VALUE lines, blank lines and # comments |
| ++// skipped, the last assignment of a key winning, and an "export KEY=..." |
| ++// line naming the key "export KEY", never KEY. |
| ++func ParseEnv(raw []byte) map[string]string { return parseEnvBytes(raw) } |
| ++ |
| ++// MergeEnv is the launch path's layering, exported unchanged (N10): the |
| ++// bucket-derived config env under the instance's own <host>.env, which |
| ++// wins every conflict. |
| ++func MergeEnv(base, overlay map[string]string) map[string]string { return mergeEnv(base, overlay) } |
| +``` |
| + |
| +C10 (rev 4, for the claim session; not committed by stage B): |
| + |
| +```go |
| +package host |
| + |
| +import ( |
| + "context" |
| + |
| + "github.com/carlosframework/platform/internal/hostclaim" |
| + execlease "github.com/carlosframework/platform/internal/hostexec" |
| +) |
| + |
| +// completedRelocationDischarge is claim ask C10's CC9 discharge, rev 2 of |
| +// its text (region HA rulings 56 and 66; Astra round 3 findings 2 and 3). |
| +// It is keyed by the persisted participation binding (the slot's, or after |
| +// a restart the .hostexec record's), never by a captured observation: |
| +// when the claim now serves on the binding's box, its holder is still the |
| +// stopped probe session the serving op records as completed for exactly |
| +// the binding's op and incarnation, and root's completed writer-session |
| +// receipt for it is in this box's guard journal and verifies, the session |
| +// is stopped with root's evidence and the participation is discharged. |
| +// Nothing is stopped again. It runs first in every cleanup path, the |
| +// slot's close and clearUntracked alike. done false falls through to |
| +// main's cleanup unchanged. Region HA never releases the row's hold or |
| +// the activator fence before this has succeeded (ruling 66), so no |
| +// serving admission can move the holder off the probe session first. |
| +func (a ClaimAssembly) completedRelocationDischarge(ctx context.Context, binding execlease.Binding) (done bool, err error) { |
| + if binding.Host == "" || binding.Op == "" || binding.Box == "" { |
| + return false, nil |
| + } |
| + now, err := hostclaim.Read(ctx, a.Claims, binding.Host) |
| + if err != nil || now.Absent || now.Claim == nil || now.Claim.Holder == nil { |
| + return false, nil |
| + } |
| + h := relocationCompletedHandover{Guard: a.Guard, Box: binding.Box} |
| + record, ok := h.completedPrior(now) |
| + if !ok || record.Op != binding.Op || now.Claim.Holder.Incarnation != binding.Incarnation { |
| + return false, nil |
| + } |
| + r, err := h.receipt(ctx, now) |
| + if err != nil { |
| + return true, err |
| + } |
| + return true, verifyRetirementReceipt(ctx, r) |
| +} |
| +``` |
| + |
| +```diff |
| +--- a/internal/host/claimparticipation.go |
| ++++ b/internal/host/claimparticipation.go |
| +@@ -124,7 +124,7 @@ |
| + // A previous failed cleanup remains denial until this fresh positive |
| + // proof succeeds. Its historical error must not veto CC9 forever. |
| + if !old.CleanupSucceeded { |
| +- if err = p.cleanup(ctx, old.Captured); err != nil { |
| ++ if err = p.cleanup(ctx, old.Captured, old.Config.Binding); err != nil { |
| + return err |
| + } |
| + } |
| +@@ -181,7 +181,7 @@ |
| + binding := execlease.Binding{Box: actor.Box, Enrollment: actor.Enrollment, Host: o.Host, Incarnation: c.Holder.Incarnation, Op: c.Op.ID, BootID: strings.TrimSpace(string(boot))} |
| + slot.Config = execlease.Config{Binding: binding, Store: p.Store, Guard: hostguard.Guard{DataDir: p.Resources.DataDir, Claims: hostguard.StoreReader{Store: p.Store}}, Directory: p.Directory, Clock: execlease.SystemClock{}, UpdateGuardDir: filepath.Join(p.Resources.OptDir, "system", "guard"), UpdateJournal: filepath.Join(p.Resources.OptDir, "system", "router", "journal.json"), |
| + CloseAdmission: func() { p.cancelWork(o.Host) }, CancelWork: func() { p.cancelWork(o.Host) }, Cleanup: func(ctx context.Context) error { |
| +- err := p.cleanup(ctx, o) |
| ++ err := p.cleanup(ctx, o, binding) |
| + slot.CleanupSucceeded = err == nil |
| + return err |
| + }} |
| +@@ -204,7 +204,7 @@ |
| + } |
| + } |
| + } |
| +-func (p *productionClaimLocal) cleanup(ctx context.Context, o hostclaim.Observed) error { |
| ++func (p *productionClaimLocal) cleanup(ctx context.Context, o hostclaim.Observed, binding execlease.Binding) error { |
| + p.mu.Lock() |
| + a := p.assembly |
| + local := p.stopIdentity |
| +@@ -219,6 +219,14 @@ |
| + return hostwrite.ErrUnavailable |
| + } |
| + p.cancelWork(o.Host) |
| ++ // Claim ask C10 (region HA rulings 56, 66): a relocation that completed |
| ++ // on this box is discharged by the persisted participation binding and |
| ++ // its exact completed probe evidence, before any ordinary cleanup, so |
| ++ // no serving-shaped stop ever replaces the probe's fence. The captured |
| ++ // observation is not consulted: after a restart it is the current one. |
| ++ if done, err := a.completedRelocationDischarge(ctx, binding); done { |
| ++ return err |
| ++ } |
| + // Cleanup uses the journal and producer set directly. It must never call |
| + // WorkSource.Select: Open is still waiting for this cleanup to finish. |
| + has, err := a.Launches.HasHost(ctx, o.Host) |
| +``` |
| + |
| +```diff |
| +--- a/internal/host/claimparticipation_stale.go |
| ++++ b/internal/host/claimparticipation_stale.go |
| +@@ -29,7 +29,7 @@ |
| + return execlease.ErrState |
| + } |
| + if o.Claim != nil && o.Claim.Holder != nil && state.Incarnation == o.Claim.Holder.Incarnation { |
| +- if err := p.cleanup(ctx, o); err != nil { |
| ++ if err := p.cleanup(ctx, o, state.Binding); err != nil { |
| + return err |
| + } |
| + return execlease.ClearStale(ctx, cfg) |
| +``` |
| + |
| +```diff |
| +--- a/internal/host/plan9_production_test.go |
| ++++ b/internal/host/plan9_production_test.go |
| +@@ -121,7 +121,7 @@ |
| + p := &productionClaimLocal{assembly: &controller.assembly, slots: map[string]*claimParticipation{recoveryHost: pending}} |
| + ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second) |
| + defer cancel() |
| +- if err := p.cleanup(ctx, o); err != nil { |
| ++ if err := p.cleanup(ctx, o, execlease.Binding{}); err != nil { |
| + t.Fatal(err) |
| + } |
| + select { |
| +@@ -169,7 +169,7 @@ |
| + controller.assembly.Identity = hostbox.IdentityProvider{} |
| + ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second) |
| + defer cancel() |
| +- if err := p.cleanup(ctx, o); err != nil { |
| ++ if err := p.cleanup(ctx, o, execlease.Binding{}); err != nil { |
| + t.Fatalf("withdrawal prevented cleanup: %v", err) |
| + } |
| + if len(f.receipts()) != 1 { |
| +``` |
| + |
| +`internal/hostrelocate/worker_bounds_rev4_test.go` (written, passes): |
| + |
| +```go |
| +package hostrelocate |
| + |
| +import ( |
| + "context" |
| + "strings" |
| + "testing" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/hostclaim" |
| +) |
| + |
| +// Astra round 3 finding 4, ruling 67: a passed probe whose stop receipt |
| +// never comes (the execution lease lost right after RecordProbe, or root |
| +// refusing the stop for ever) is bounded: ReleaseDeadline after the probe |
| +// passed the op is stuck for an operator; before it, nothing changes; at |
| +// no point is the participation, the hold or the session released. |
| +func TestWorkerBoundsAPassedProbeAwaitingItsStop(t *testing.T) { |
| + for _, name := range []string{"lease lost after RecordProbe", "root refuses the stop"} { |
| + t.Run(name, func(t *testing.T) { |
| + f := newFixture(t) |
| + ctx := context.Background() |
| + if _, err := f.w.Start(ctx, host, Request{Reason: hostclaim.ReasonDrain, Source: "src", Target: "dst"}); err != nil { |
| + t.Fatal(err) |
| + } |
| + f.ack("src") |
| + f.ack("dst") |
| + f.worker(nil) // releasing |
| + h := f.claim().Claim.Holder |
| + id := f.claim().Claim.Op.ID |
| + f.participant(hostclaim.JournalRelease(id, hostclaim.Release{ReleasedBy: "src", Enrollment: enrollment("src"), Generation: h.Generation, Session: h.Session, FinalTXID: 40, ManifestSeq: 7, Receipt: "r", ReceiptSHA256: strings.Repeat("d", 64)})) |
| + f.worker(nil) // pinning |
| + f.worker(nil) // pin |
| + f.worker(nil) // installing on dst |
| + f.installAndProbe("dst") |
| + o := f.claim() |
| + f.participant(hostclaim.RecordProbe(id, hostclaim.Probe{Box: "dst", Session: o.Claim.Holder.Session, At: f.now})) |
| + // Either way no probe_stop is ever journaled: the participant |
| + // cannot pass Ready, or EndProbe keeps failing. |
| + f.now = f.now.Add(ReleaseDeadline) |
| + if o := f.worker(ErrNotYet); o.Claim.Op.Journal["stuck"] != "" || o.Claim.Op.Kind != hostclaim.KindRelocating { |
| + t.Fatalf("stuck before the bound: %+v", o.Claim.Op.Journal) |
| + } |
| + f.now = f.now.Add(time.Second) |
| + o = f.worker(nil) |
| + if o.Claim.Op.Journal["stuck"] == "" || o.Claim.Op.Kind != hostclaim.KindRelocating || o.Claim.Op.Exec != "dst" || o.Claim.Op.Journal["probe_stop"] != "" { |
| + t.Fatalf("not stuck for an operator, or something released: %+v", o.Claim.Op) |
| + } |
| + }) |
| + } |
| +} |
| + |
| +// Ruling 60 (rev 3), tested here: an install that never enters probing |
| +// (its stamp retired, its participation denied) is stuck ProbeDeadline |
| +// after installed_at. |
| +func TestWorkerBoundsAnInstallThatNeverProbes(t *testing.T) { |
| + f := newFixture(t) |
| + ctx := context.Background() |
| + if _, err := f.w.Start(ctx, host, Request{Reason: hostclaim.ReasonDrain, Source: "src", Target: "dst"}); err != nil { |
| + t.Fatal(err) |
| + } |
| + f.ack("src") |
| + f.ack("dst") |
| + f.worker(nil) |
| + h := f.claim().Claim.Holder |
| + id := f.claim().Claim.Op.ID |
| + f.participant(hostclaim.JournalRelease(id, hostclaim.Release{ReleasedBy: "src", Enrollment: enrollment("src"), Generation: h.Generation, Session: h.Session, FinalTXID: 40, ManifestSeq: 7, Receipt: "r", ReceiptSHA256: strings.Repeat("d", 64)})) |
| + f.worker(nil) // pinning |
| + f.worker(nil) // pin |
| + f.worker(nil) // installing on dst |
| + f.worker(ErrNotYet) // P is not built yet |
| + p := f.claim().Claim.Op.Journal["prepared"] |
| + f.participant(func(c *hostclaim.Claim) error { c.Op.Journal["test.built"] = p; return nil }) |
| + if o := f.worker(nil); o.Claim.Op.Phase != hostclaim.PhaseInstalled || o.Claim.Op.Journal["installed_at"] == "" { |
| + t.Fatalf("installed with its start: %s %+v", o.Claim.Op.Phase, o.Claim.Op.Journal) |
| + } |
| + f.now = f.now.Add(ProbeDeadline) |
| + if o := f.worker(ErrNotYet); o.Claim.Op.Journal["stuck"] != "" { |
| + t.Fatal("stuck before the bound") |
| + } |
| + f.now = f.now.Add(time.Second) |
| + if o := f.worker(nil); o.Claim.Op.Journal["stuck"] == "" || o.Claim.Op.Phase != hostclaim.PhaseInstalled { |
| + t.Fatalf("not stuck: %+v", o.Claim.Op) |
| + } |
| +} |
| +``` |
| --- |
| @@ -56,7 +350,7 @@ All rev 3 code was assembled in a throwaway `git archive 7a41192c` under a uniqu |
| ## Rulings, rev 3 (55 to 64) |
| -55. **O2 is withdrawn; the pinner implements O2'** (finding 1; withdraws 54). Ruling 25's refusal stands unless the pinned manifest has the `instance.key` file, or `StorePinner.KeyEvidence` (O2' from the instance-key plan rev 2) reports the bucket-held config's key, or reports an adoption record *and* the pinned env carries a non-empty `CARLOS_INSTANCE_KEY`. The adoption record is the agent's durable record, written when the app's helper reports it used `$CARLOS_INSTANCE_KEY` for this incarnation. `KeyEvidence` is nil until that plan lands. The question names host, account, app, incarnation and the pinned release, so the evidence can be version-bound (migration evidence is O2''s to define). A reader error refuses. *Cost if wrong:* an app that really uses the env key but has no adoption record yet goes `stuck`, as ruling 25 already does. |
| +55. *Superseded in rev 4 by ruling 65 (Astra round 3 finding 1; instance-key rev 3 R-B: source (c) dropped, (b) bound to the pinned writer and the delivered key).* **O2 is withdrawn; the pinner implements O2'** (finding 1; withdraws 54). Ruling 25's refusal stands unless the pinned manifest has the `instance.key` file, or `StorePinner.KeyEvidence` (O2' from the instance-key plan rev 2) reports the bucket-held config's key, or reports an adoption record *and* the pinned env carries a non-empty `CARLOS_INSTANCE_KEY`. The adoption record is the agent's durable record, written when the app's helper reports it used `$CARLOS_INSTANCE_KEY` for this incarnation. `KeyEvidence` is nil until that plan lands. The question names host, account, app, incarnation and the pinned release, so the evidence can be version-bound (migration evidence is O2''s to define). A reader error refuses. *Cost if wrong:* an app that really uses the env key but has no adoption record yet goes `stuck`, as ruling 25 already does. |
| 56. **Completion hands over from the probe's existing receipt, and discharges its participation with it** (findings 5 and 6; C7 withdrawn, C10 raised). `CompleteRelocation` journals `completed_relocation` on the serving op: the op, the probe stop (box, session, fence id, receipt digest), the stamp, phase and generation of the stopped session. On the first serving cold wake on that box, N6 does three things. Before admission, `relocationCompletedHandover.Before` finds the exact receipt in this box's guard journal (`CompletedWriterStops` with matching fence id, obligation `writer-session-S`, phase and digest) and verifies it, in place of `Wake.Stop`, which would ask a second, serving-shaped fence that S's producers reject. After admission, `coldHandoverWith` posts that receipt's target as `Prior` to root's `/claim/handover`. Root's `ColdHandover.Run` takes `prior` as it is: it checks the successor's stamp and session above the prior's, same box and enrollment, the receipt and the prior's exact inventory, and never matches op ids. The participation slot opened at acknowledgement is discharged by the same evidence (C10). *Cost if wrong:* if C10 is refused, the completed target's slot denies serving; C10 is an activation gate, and the flags stay off. |
| 57. **The instance directory is made by main's `PrepareState`** (finding 3). The install reads the claim fresh after `MintHeld`, checks its authority, and calls `ClaimRoleAssembly.PrepareState(ctx, installed, work, {DBPath})` under the install work, then restores into the materialized directory. On a non-Linux build `PrepareState` refuses, so the install refuses there; relocation is Linux-only. *Cost if wrong:* none beyond main's protocol. |
| 58. **A later probe attempt is made only when root can launch it** (finding 4). The reader validates a later attempt at the current writer stamp. Main's `claimLaunchAuthority` still pins `install_stamp`, so until C11 `Preflight` answers `ErrLaterAttemptUnlaunchable` once P has opened, and the participant makes `probe-failed` at once. *Cost if wrong:* one probe attempt per target until C11; a failed P opening goes `probe-failed` sooner. |
| @@ -353,34 +647,34 @@ import ( |
| "context" |
| "github.com/carlosframework/platform/internal/hostclaim" |
| + execlease "github.com/carlosframework/platform/internal/hostexec" |
| ) |
| -// completedRelocationDischarge is claim ask C10's proposed CC9 discharge |
| -// (region HA ruling 56; Astra round 2 finding 5). The participation's |
| -// controller opened while the box acknowledged the relocation, so its |
| -// captured observation names an earlier writer; after the serving CAS, |
| -// stopCapturedSession(captured) can never match. When the claim now serves |
| -// on this box, the captured relocation is the one the serving op records |
| -// as completed here, and root's completed writer-session receipt for the |
| -// recorded probe session is in this box's guard journal, verified, the |
| -// session is stopped with root's evidence and the participation is |
| -// discharged: nothing is stopped again. done false falls through to main's |
| -// cleanup unchanged. |
| -func (a ClaimAssembly) completedRelocationDischarge(ctx context.Context, captured hostclaim.Observed) (done bool, err error) { |
| - if captured.Claim == nil || captured.Claim.Op.Kind != hostclaim.KindRelocating || captured.Claim.Holder == nil { |
| - return false, nil |
| - } |
| - local, err := a.localStopIdentity(ctx) |
| - if err != nil { |
| +// completedRelocationDischarge is claim ask C10's CC9 discharge, rev 2 of |
| +// its text (region HA rulings 56 and 66; Astra round 3 findings 2 and 3). |
| +// It is keyed by the persisted participation binding (the slot's, or after |
| +// a restart the .hostexec record's), never by a captured observation: |
| +// when the claim now serves on the binding's box, its holder is still the |
| +// stopped probe session the serving op records as completed for exactly |
| +// the binding's op and incarnation, and root's completed writer-session |
| +// receipt for it is in this box's guard journal and verifies, the session |
| +// is stopped with root's evidence and the participation is discharged. |
| +// Nothing is stopped again. It runs first in every cleanup path, the |
| +// slot's close and clearUntracked alike. done false falls through to |
| +// main's cleanup unchanged. Region HA never releases the row's hold or |
| +// the activator fence before this has succeeded (ruling 66), so no |
| +// serving admission can move the holder off the probe session first. |
| +func (a ClaimAssembly) completedRelocationDischarge(ctx context.Context, binding execlease.Binding) (done bool, err error) { |
| + if binding.Host == "" || binding.Op == "" || binding.Box == "" { |
| return false, nil |
| } |
| - now, err := hostclaim.Read(ctx, a.Claims, captured.Host) |
| - if err != nil || now.Absent { |
| + now, err := hostclaim.Read(ctx, a.Claims, binding.Host) |
| + if err != nil || now.Absent || now.Claim == nil || now.Claim.Holder == nil { |
| return false, nil |
| } |
| - h := relocationCompletedHandover{Guard: a.Guard, Box: local.Actor.Box} |
| + h := relocationCompletedHandover{Guard: a.Guard, Box: binding.Box} |
| record, ok := h.completedPrior(now) |
| - if !ok || record.Op != captured.Claim.Op.ID || now.Claim.Holder.Incarnation != captured.Claim.Holder.Incarnation { |
| + if !ok || record.Op != binding.Op || now.Claim.Holder.Incarnation != binding.Incarnation { |
| return false, nil |
| } |
| r, err := h.receipt(ctx, now) |
| @@ -394,14 +688,43 @@ func (a ClaimAssembly) completedRelocationDischarge(ctx context.Context, capture |
| ```diff |
| --- a/internal/host/claimparticipation.go |
| +++ b/internal/host/claimparticipation.go |
| -@@ -219,6 +219,12 @@ |
| +@@ -124,7 +124,7 @@ |
| + // A previous failed cleanup remains denial until this fresh positive |
| + // proof succeeds. Its historical error must not veto CC9 forever. |
| + if !old.CleanupSucceeded { |
| +- if err = p.cleanup(ctx, old.Captured); err != nil { |
| ++ if err = p.cleanup(ctx, old.Captured, old.Config.Binding); err != nil { |
| + return err |
| + } |
| + } |
| +@@ -181,7 +181,7 @@ |
| + binding := execlease.Binding{Box: actor.Box, Enrollment: actor.Enrollment, Host: o.Host, Incarnation: c.Holder.Incarnation, Op: c.Op.ID, BootID: strings.TrimSpace(string(boot))} |
| + slot.Config = execlease.Config{Binding: binding, Store: p.Store, Guard: hostguard.Guard{DataDir: p.Resources.DataDir, Claims: hostguard.StoreReader{Store: p.Store}}, Directory: p.Directory, Clock: execlease.SystemClock{}, UpdateGuardDir: filepath.Join(p.Resources.OptDir, "system", "guard"), UpdateJournal: filepath.Join(p.Resources.OptDir, "system", "router", "journal.json"), |
| + CloseAdmission: func() { p.cancelWork(o.Host) }, CancelWork: func() { p.cancelWork(o.Host) }, Cleanup: func(ctx context.Context) error { |
| +- err := p.cleanup(ctx, o) |
| ++ err := p.cleanup(ctx, o, binding) |
| + slot.CleanupSucceeded = err == nil |
| + return err |
| + }} |
| +@@ -204,7 +204,7 @@ |
| + } |
| + } |
| + } |
| +-func (p *productionClaimLocal) cleanup(ctx context.Context, o hostclaim.Observed) error { |
| ++func (p *productionClaimLocal) cleanup(ctx context.Context, o hostclaim.Observed, binding execlease.Binding) error { |
| + p.mu.Lock() |
| + a := p.assembly |
| + local := p.stopIdentity |
| +@@ -219,6 +219,14 @@ |
| return hostwrite.ErrUnavailable |
| } |
| p.cancelWork(o.Host) |
| -+ // Claim ask C10 (region HA ruling 56): a relocation that completed on |
| -+ // this box is discharged by its exact completed probe evidence, not by |
| -+ // stopping the session captured when the controller opened. |
| -+ if done, err := a.completedRelocationDischarge(ctx, o); done { |
| ++ // Claim ask C10 (region HA rulings 56, 66): a relocation that completed |
| ++ // on this box is discharged by the persisted participation binding and |
| ++ // its exact completed probe evidence, before any ordinary cleanup, so |
| ++ // no serving-shaped stop ever replaces the probe's fence. The captured |
| ++ // observation is not consulted: after a restart it is the current one. |
| ++ if done, err := a.completedRelocationDischarge(ctx, binding); done { |
| + return err |
| + } |
| // Cleanup uses the journal and producer set directly. It must never call |
| @@ -841,7 +1164,8 @@ Each is modelled as a dependency with a fallback. None is assumed. Rev 2 added C |
| | C5 | claim | The claim's kind in `AdmitWriter`'s op match | `RelocationWakeInputs.Select` refuses any kind but relocating, and the dispatcher routes by kind (claimowners.go:208). | none | |
| | **C6** | claim | A kind-scoped CC9 discharge for a relocation target the op excluded (`replaceTargetSlot` removes it from participants, relocation.go:1091) while its unopened install and held row remain. | **Rev 3, ruling 63: two safe outcomes.** If the slot was captured at the target's own install stamp, main's `CheckLocal` finds the complete installation and cleanup succeeds, so `ClearStale` removes the denial while the inert, held, never-reactivated installation waits for its `park-local`. If it was captured earlier, the census refuses the row and the denial stands until `park-local`. No writer can start there either way. The ported lifecycle property covers both. | rulings 42, 63 | |
| | ~~C7~~ | claim | **Withdrawn in rev 3 (ruling 56).** `ColdHandover.Run` accepts the probe stop's exact receipt as prior beside a validated cold serving successor without matching op ids; the agent supplies it through N6. Nothing is re-stopped or relabelled. | none | ruling 56 | |
| -| **C10** | claim | A CC9 discharge of a relocation participation by its exact completed probe evidence: the serving op's `completed_relocation` names the captured op, and root's completed `writer-session-S` receipt for it is in this box's guard journal and verifies (patch in the rev 3 section). | **None that serves: an activation gate.** Without it main's cleanup stops the stale captured observation and refuses, the slot and `.hostexec` record deny serving, and a completed target never serves. The flags stay off. | ruling 56, Task 9 | |
| +| **C10** | claim | **Text changed in rev 4** (see "C10's text on #236"): a CC9 discharge of a relocation participation keyed by the persisted binding and the exact completed probe evidence, first in every cleanup path, the slot's close and `clearUntracked` alike. | **None that serves: an activation gate.** Region HA releases the hold and the fence only after the discharge (ruling 66), so without C10 the target stays held and unserved, alerting once; nothing unsafe is admitted. | rulings 56, 66, Task 9 | |
| +| **IK-R-B** | instance-key plan rev 3 | O2′ (b′): the adoption record (`instance-key-adoption` platform item: host, incarnation, session, generation, `key_sha256`), written by the agent from its launch state when rastrillo's helper reports `$CARLOS_INSTANCE_KEY` resolved at a process start, published continuously, none for a new admission; and `KeyEvidence.ConfiguredEnv`, the configured env rendered for the pinned release. Not committed when rev 4 was written; field names to be matched on landing. | `KeyEvidence` nil: only `instance.key` evidences a nonempty database's key (ruling 25); such a host is `stuck` rather than relocated. | ruling 65, Task 8b | |
| | **C11** | claim | `claimLaunchAuthority` (claimlaunch_root.go:129-132) pins a relocating launch to `install_stamp` only for P's opening, and validates a later attempt at the admitted writer's stamp through root's attempt reader. | **One launchable attempt per target.** `Preflight` answers `ErrLaterAttemptUnlaunchable` once P has opened, and the participant makes `probe-failed` at once (no attempts spent on refusals). `RelocationAssemblyConfig.LaterProbeAttempts` is set when C11 lands. | ruling 58, Task 8d | |
| | C8 + **O1** | moves (platform-d8, #236) | The production `ClaimFilesManifest.Establish` provider and its `RegisterProductionClaimMoveSources` caller (none on this branch). O1: `Establish` publishes `<host>.env`'s exact bytes before it returns, and an error fails the wake. | **Relocation stays unbound** (ruling 53). `RelocationSource.Open` returns the zero owner set while `ClaimServingSources.Files` is nil, `claimRelocationC1` answers (false, nil), relocating is refused as before Plan 11, and the runtime is not built. | ruling 53, Tasks 8d and 10 | |
| | **C9** | claim | The claim's `Policy` admits relocating `install`, `held-config` and `held-release` for `op.exec` in `installed`, and in `probing` before P opens, with the hold absent or the op's own. Its `WithRefusals` then also run for them. | `relocationInstallPolicy` on main's WorkSource (ruling 46): the same `Guard`, `Prepare` and `Local` main uses, so the claim's execution, deadline and quarantine checks run on every `With`; the policy itself is region HA's, followed by region HA's `Policy.Relocation` owner. | ruling 46, Task 8c | |
| @@ -1008,6 +1332,8 @@ package host |
| import ( |
| "context" |
| + "errors" |
| + "os" |
| "github.com/carlosframework/platform/internal/hostbox" |
| "github.com/carlosframework/platform/internal/hostclaim" |
| @@ -1063,6 +1389,32 @@ func (p *productionClaimLocal) boundAssembly() *ClaimAssembly { |
| defer p.mu.Unlock() |
| return p.assembly |
| } |
| + |
| +// DischargeCompleted is ruling 66's gate before a completed relocation's |
| +// hold is released: main's own Prepare for the serving observation, which |
| +// closes the participation slot (its cleanup is C10's discharge, first) |
| +// and clears the .hostexec record, or, after a restart, clears the |
| +// untracked record through clearUntracked (the same discharge, keyed by |
| +// the persisted binding). It succeeds only once neither a slot nor a |
| +// record is left for the host. N1's third accessor; no new authority. |
| +func (p *productionClaimLocal) DischargeCompleted(ctx context.Context, o hostclaim.Observed) error { |
| + if o.Claim == nil || o.Claim.Holder == nil || o.Claim.Op.Kind != hostclaim.KindServing { |
| + return hostbox.ErrChanged |
| + } |
| + if err := p.Prepare(ctx, o); err != nil { |
| + return err |
| + } |
| + p.mu.Lock() |
| + slot := p.slots[o.Host] |
| + p.mu.Unlock() |
| + if slot != nil { |
| + return execlease.ErrInactive |
| + } |
| + if _, err := execlease.InspectLocal(p.Resources.DataDir, o.Host); !errors.Is(err, os.ErrNotExist) { |
| + return errors.Join(execlease.ErrInactive, err) |
| + } |
| + return nil |
| +} |
| ``` |
| - [ ] **Step 4: `RelocationEffects`, rewritten** |
| @@ -1587,7 +1939,11 @@ type RelocationAssemblyConfig struct { |
| // main: root then authorizes a relocating launch at the admitted |
| // writer's stamp, not only at install_stamp (ruling 58). |
| LaterProbeAttempts bool |
| - Now func() time.Time |
| + // Discharge is main's participation discharge for a completed |
| + // relocation (productionClaimLocal.DischargeCompleted, N1). The hold |
| + // and the activator fence are released only after it (ruling 66). |
| + Discharge func(context.Context, hostclaim.Observed) error |
| + Now func() time.Time |
| } |
| // RelocationEffects binds every provider to this bound assembly. ack is |
| @@ -1757,18 +2113,46 @@ The pinner implements O2' from the instance-key plan rev 2 (`instance-key-build` |
| ```diff |
| --- a/internal/hostrelocate/pin.go |
| +++ b/internal/hostrelocate/pin.go |
| -@@ -88,6 +88,10 @@ |
| +@@ -1,12 +1,14 @@ |
| + package hostrelocate |
| + |
| + import ( |
| ++ "bytes" |
| + "context" |
| + "crypto/sha256" |
| + "encoding/hex" |
| + "encoding/json" |
| + "errors" |
| + "fmt" |
| ++ "github.com/carlosframework/platform/internal/activator" |
| + "os" |
| + "sort" |
| + "strings" |
| +@@ -54,6 +56,12 @@ |
| + // reads: P is written from this copy, never from the source again. |
| + // Empty for a host that never published files. |
| + Retained string `json:"retained,omitempty"` |
| ++ // KeySHA256 is the digest of the instance key the target will deliver, |
| ++ // pinned when O2' (b') evidenced it (ruling 65): the install derives the |
| ++ // key again from the restored env and the config it renders, and |
| ++ // refuses unless it has this digest. Empty when instance.key is the |
| ++ // evidence, or no key is needed. |
| ++ KeySHA256 string `json:"key_sha256,omitempty"` |
| + } |
| + |
| + // RetainedManifestKey is where the pinner keeps the pinned manifest's |
| +@@ -88,6 +96,10 @@ |
| // Stage B wires it to the controller's registry. Nil is ErrUnsupported; |
| // an empty version refuses the pin, never a guess. |
| Release func(ctx context.Context, host string) (string, error) |
| -+ // KeyEvidence is O2's reader from the instance-key plan (region HA |
| -+ // ruling 55): nil until that plan lands, and then only the instance.key |
| -+ // file evidences a nonempty database's key. |
| -+ KeyEvidence func(ctx context.Context, q KeyQuestion) (KeySource, error) |
| ++ // KeyEvidence is O2' (b')'s configured-env reader from the instance-key |
| ++ // plan rev 3 (region HA ruling 65): nil until that plan lands, and then |
| ++ // only the instance.key file evidences a nonempty database's key. |
| ++ KeyEvidence *KeyEvidence |
| } |
| func (p StorePinner) Pin(ctx context.Context, obs hostclaim.Observed, now time.Time) (hostclaim.RecoveryPin, error) { |
| -@@ -142,7 +146,7 @@ |
| +@@ -142,7 +154,7 @@ |
| // that writer's instance.key: the pinned manifest must carry it (rev |
| // 18, round 17 finding 1). |
| needKey := c.Holder.Session != 0 && obj.DB != nil && obj.DB.MaxTXID > 0 |
| @@ -1777,7 +2161,7 @@ The pinner implements O2' from the instance-key plan rev 2 (`instance-key-build` |
| if err != nil { |
| return hostclaim.RecoveryPin{}, err |
| } |
| -@@ -221,7 +225,7 @@ |
| +@@ -221,7 +233,7 @@ |
| // stays restorable at g10. Every referenced object must lie under this |
| // holder's location (its incarnation) in this region's data bucket, at |
| // files/<generation>/objects/; any other reference refuses the pin. |
| @@ -1786,7 +2170,7 @@ The pinner implements O2' from the instance-key plan rev 2 (`instance-key-build` |
| chain := []string{h.Generation} |
| for i := len(h.Admissions) - 1; i >= 0; i-- { |
| chain = append(chain, h.Admissions[i].Generation, h.Admissions[i].Predecessor) |
| -@@ -247,16 +251,26 @@ |
| +@@ -247,18 +259,23 @@ |
| if err := json.Unmarshal(raw, &m); err != nil || m.V != hostfiles.ManifestVersion { |
| return FilesPin{}, nil, errors.Join(fmt.Errorf("%w: files manifest %s", ErrRefused, prefix), err) |
| } |
| @@ -1800,27 +2184,25 @@ The pinner implements O2' from the instance-key plan rev 2 (`instance-key-build` |
| if origins, err = referencedOrigins(h.Location, origins, m); err != nil { |
| return FilesPin{}, nil, err |
| } |
| ++ keySHA := "" |
| + if needKey && !liveEntry(m, InstanceKeyPath) { |
| + // Ruling 25 stands: a nonempty database is pinned only with |
| + // positive evidence of the key that sealed it. Without the |
| -+ // instance.key file, that is O2' from the instance-key plan: |
| -+ // the platform key in the pinned env plus the agent's durable |
| -+ // adoption record for this incarnation, or the bucket-held |
| -+ // config's key (region HA ruling 55). A platform key alone is |
| -+ // not evidence: the app may ignore it and seal with its own |
| -+ // instance.key, which this manifest lacks. |
| -+ ok, err := p.keyEvidenced(ctx, host, release, h, prefix, m, origins) |
| -+ if err != nil { |
| ++ // instance.key file that is O2' (b'), instance-key plan rev 3 |
| ++ // ruling R-B (region HA ruling 65): the pinned writer's own |
| ++ // adoption record for the exact key the target will deliver. |
| ++ var err error |
| ++ if keySHA, err = p.adoptedKey(ctx, host, release, h, prefix, m, origins); err != nil { |
| + return FilesPin{}, nil, err |
| + } |
| -+ if !ok { |
| -+ return FilesPin{}, nil, fmt.Errorf("%w: %s's pinned manifest has no %s and no evidence the app uses another key for a nonempty database", ErrFilesUnevidenced, prefix, InstanceKeyPath) |
| -+ } |
| + } |
| sum := sha256.Sum256(raw) |
| - return FilesPin{Prefix: prefix, Seq: m.Seq, SHA256: hex.EncodeToString(sum[:]), Origins: origins}, raw, nil |
| +- return FilesPin{Prefix: prefix, Seq: m.Seq, SHA256: hex.EncodeToString(sum[:]), Origins: origins}, raw, nil |
| ++ return FilesPin{Prefix: prefix, Seq: m.Seq, SHA256: hex.EncodeToString(sum[:]), Origins: origins, KeySHA256: keySHA}, raw, nil |
| } |
| -@@ -395,3 +409,72 @@ |
| + // No manifest anywhere in the chain. An empty files pin needs the |
| + // same kind of evidence an empty database pin does (ruling 15): no |
| +@@ -395,3 +412,118 @@ |
| } |
| return pinned, nil |
| } |
| @@ -1830,68 +2212,114 @@ The pinner implements O2' from the instance-key plan rev 2 (`instance-key-build` |
| +// published as the manifest's platform EnvItem. |
| +const InstanceKeyEnv = "CARLOS_INSTANCE_KEY" |
| + |
| -+// KeySource is O2' (instance-key plan rev 2): where a key other than the |
| -+// instance.key file is positively evidenced for one incarnation. |
| -+type KeySource int |
| ++// AdoptionItem is the manifest's platform item holding the adoption record |
| ++// (instance-key plan rev 3, ruling R-B), published continuously beside |
| ++// EnvItem. |
| ++const AdoptionItem = "instance-key-adoption" |
| + |
| -+const ( |
| -+ // KeyUnevidenced: no evidence. The pin refuses (ruling 25). |
| -+ KeyUnevidenced KeySource = iota |
| -+ // KeyAdopted: the agent's durable adoption record says the app's |
| -+ // helper reported using $CARLOS_INSTANCE_KEY for this incarnation. It |
| -+ // counts only with the platform key present in the pinned env. |
| -+ KeyAdopted |
| -+ // KeyConfig: the key is held in the bucket-held config for this |
| -+ // incarnation, outside the files set. |
| -+ KeyConfig |
| -+) |
| ++// AdoptionRecord is O2' (b'), as instance-key plan rev 3's agent writes it |
| ++// from its own launch state when rastrillo's helper reports, at a process |
| ++// start, that it resolved $CARLOS_INSTANCE_KEY: bound to one writer |
| ++// session of one installation, never carried to another. Every new writer |
| ++// admission starts with none. |
| ++type AdoptionRecord struct { |
| ++ V int `json:"v"` |
| ++ Host string `json:"host"` |
| ++ Incarnation string `json:"incarnation"` |
| ++ Session uint64 `json:"session"` |
| ++ Generation string `json:"generation"` |
| ++ KeySHA256 string `json:"key_sha256"` |
| ++} |
| + |
| -+// KeyQuestion is what the pinner asks O2's evidence reader. |
| ++// KeyQuestion is what the pinner asks for the configured env. |
| +type KeyQuestion struct { |
| + Host, Account, App, Incarnation string |
| + // Release is the pinned release the target will run. |
| + Release string |
| +} |
| + |
| -+// envHasInstanceKey reports whether m's platform env item has a non-empty |
| -+// CARLOS_INSTANCE_KEY. No env item is false; an env item named but |
| -+// unreadable is an error, never "no key". |
| -+func (p StorePinner) envHasInstanceKey(ctx context.Context, prefix string, m hostfiles.Manifest, origins []string) (bool, error) { |
| -+ if e, ok := m.Platform[hostfiles.EnvItem]; !ok || e.Deleted { |
| -+ return false, nil |
| ++// KeyEvidence is O2' (b')'s inputs beyond the manifest: the host's |
| ++// configured env (<host>.config.env bytes) as the target would render it |
| ++// now, the instance-key plan's reader. Nil KeyEvidence: only instance.key. |
| ++type KeyEvidence struct { |
| ++ ConfiguredEnv func(context.Context, KeyQuestion) ([]byte, error) |
| ++} |
| ++ |
| ++// EffectiveInstanceKey is the key a launch delivers: the activator's own |
| ++// parser and layering (activator.ParseEnv, MergeEnv, N10), <host>.env over |
| ++// <host>.config.env. "" is none. |
| ++func EffectiveInstanceKey(configEnv, hostEnv []byte) string { |
| ++ return activator.MergeEnv(activator.ParseEnv(configEnv), activator.ParseEnv(hostEnv))[InstanceKeyEnv] |
| ++} |
| ++ |
| ++// KeyDigest is sha256 of the key, hex, as the adoption record holds it. |
| ++func KeyDigest(key string) string { |
| ++ sum := sha256.Sum256([]byte(key)) |
| ++ return hex.EncodeToString(sum[:]) |
| ++} |
| ++ |
| ++// livePlatform reads a platform item that is present and not deleted; ok |
| ++// false when absent or a tombstone. A named item that cannot be read is an |
| ++// error, never "absent". |
| ++func (p StorePinner) livePlatform(ctx context.Context, prefix string, m hostfiles.Manifest, name string, origins []string) ([]byte, bool, error) { |
| ++ e, ok := m.Platform[name] |
| ++ if !ok || e.Deleted { |
| ++ return nil, false, nil |
| + } |
| -+ env, err := hostfiles.ReadPlatform(ctx, p.Data, prefix, m, hostfiles.EnvItem, origins) |
| ++ raw, err := hostfiles.ReadPlatform(ctx, p.Data, prefix, m, name, origins) |
| + if err != nil { |
| -+ return false, err |
| -+ } |
| -+ for _, line := range strings.Split(string(env), "\n") { |
| -+ line = strings.TrimPrefix(strings.TrimSpace(line), "export ") |
| -+ if v, ok := strings.CutPrefix(line, InstanceKeyEnv+"="); ok && strings.Trim(strings.TrimSpace(v), `"'`) != "" { |
| -+ return true, nil |
| -+ } |
| ++ return nil, false, err |
| + } |
| -+ return false, nil |
| ++ return raw, true, nil |
| +} |
| + |
| -+// keyEvidenced is O2' without the instance.key file: an adoption record |
| -+// with the platform key in the pinned env, or the bucket-held config's |
| -+// key. With no KeyEvidence reader (the instance-key plan not landed), |
| -+// nothing but the file is evidence. |
| -+func (p StorePinner) keyEvidenced(ctx context.Context, host, release string, h *hostclaim.Holder, prefix string, m hostfiles.Manifest, origins []string) (bool, error) { |
| ++// adoptedKey is O2' (b'): the pinned manifest's adoption record names the |
| ++// pinned writer (host, incarnation, session, generation) and the digest of |
| ++// the exact key the target will deliver, which is the pinned env item's |
| ++// key over the configured env's (the launch's layering). It returns that |
| ++// digest, pinned as FilesPin.KeySHA256 for the install to verify again. |
| ++func (p StorePinner) adoptedKey(ctx context.Context, host, release string, h *hostclaim.Holder, prefix string, m hostfiles.Manifest, origins []string) (string, error) { |
| ++ refuse := func(why string) (string, error) { |
| ++ return "", fmt.Errorf("%w: %s's pinned manifest has no %s, and %s, for a nonempty database", ErrFilesUnevidenced, prefix, InstanceKeyPath, why) |
| ++ } |
| + if p.KeyEvidence == nil { |
| -+ return false, nil |
| ++ return refuse("no adoption evidence is wired (instance-key plan rev 3)") |
| ++ } |
| ++ raw, ok, err := p.livePlatform(ctx, prefix, m, AdoptionItem, origins) |
| ++ if err != nil { |
| ++ return "", err |
| ++ } |
| ++ if !ok { |
| ++ return refuse("no adoption record") |
| ++ } |
| ++ var rec AdoptionRecord |
| ++ d := json.NewDecoder(bytes.NewReader(raw)) |
| ++ d.DisallowUnknownFields() |
| ++ if err := d.Decode(&rec); err != nil || rec.V != 1 { |
| ++ return "", errors.Join(fmt.Errorf("%w: adoption record in %s", ErrRefused, prefix), err) |
| ++ } |
| ++ if rec.Host != host || rec.Incarnation != h.Incarnation || rec.Session != h.Session || rec.Generation != h.Generation { |
| ++ return refuse("the adoption record is not the pinned writer's") |
| + } |
| -+ src, err := p.KeyEvidence(ctx, KeyQuestion{Host: host, Account: h.Account, App: h.App, Incarnation: h.Incarnation, Release: release}) |
| ++ env, _, err := p.livePlatform(ctx, prefix, m, hostfiles.EnvItem, origins) |
| + if err != nil { |
| -+ return false, err |
| ++ return "", err |
| + } |
| -+ switch src { |
| -+ case KeyConfig: |
| -+ return true, nil |
| -+ case KeyAdopted: |
| -+ return p.envHasInstanceKey(ctx, prefix, m, origins) |
| ++ var config []byte |
| ++ if p.KeyEvidence.ConfiguredEnv != nil { |
| ++ if config, err = p.KeyEvidence.ConfiguredEnv(ctx, KeyQuestion{Host: host, Account: h.Account, App: h.App, Incarnation: h.Incarnation, Release: release}); err != nil { |
| ++ return "", err |
| ++ } |
| ++ } |
| ++ key := EffectiveInstanceKey(config, env) |
| ++ if key == "" { |
| ++ return refuse("the target would deliver no instance key") |
| ++ } |
| ++ if digest := KeyDigest(key); digest != rec.KeySHA256 { |
| ++ return refuse("the adoption record is for another key than the target would deliver") |
| ++ } else { |
| ++ return digest, nil |
| + } |
| -+ return false, nil |
| +} |
| ``` |
| @@ -1904,6 +2332,7 @@ import ( |
| "context" |
| "crypto/sha256" |
| "encoding/hex" |
| + "encoding/json" |
| "errors" |
| "fmt" |
| "testing" |
| @@ -1913,57 +2342,92 @@ import ( |
| "github.com/carlosframework/platform/internal/release" |
| ) |
| -// O2' (instance-key plan rev 2, region HA ruling 55): with no instance.key |
| -// file, a nonempty database is pinned only with positive evidence that the |
| -// app uses another key. The platform key alone is not evidence: Astra |
| -// round 2's case, an unused platform key and an unpublished app key, refuses. |
| -func TestStorePinnerNeedsPositiveEvidenceForAnEnvKey(t *testing.T) { |
| +// O2' (b') (instance-key plan rev 3, ruling R-B; region HA ruling 65): |
| +// without the instance.key file, a nonempty database pins only when the |
| +// pinned writer's own adoption record names the digest of the exact key |
| +// the target will deliver: the pinned env item over the configured env, |
| +// parsed and layered by the activator's own launch code. |
| +func TestStorePinnerNeedsTheWritersAdoptionOfTheDeliveredKey(t *testing.T) { |
| ctx := context.Background() |
| at := time.Date(2026, 10, 7, 9, 0, 0, 0, time.UTC) |
| - const withKey = "CARLOS_ADMIN_TOKEN=t\nCARLOS_INSTANCE_KEY=k1\n" |
| + adopt := func(key string) *AdoptionRecord { |
| + return &AdoptionRecord{V: 1, Host: host, Incarnation: "inc", Session: 3, Generation: "g3", KeySHA256: KeyDigest(key)} |
| + } |
| + config := func(env string) *KeyEvidence { |
| + return &KeyEvidence{ConfiguredEnv: func(context.Context, KeyQuestion) ([]byte, error) { return []byte(env), nil }} |
| + } |
| for _, tc := range []struct { |
| - name string |
| - env string // "" is no env item |
| - evidence func(context.Context, KeyQuestion) (KeySource, error) |
| - want error |
| + name string |
| + env string // "" is no env item |
| + unreadable bool // the env item is named but its object is missing |
| + record *AdoptionRecord |
| + evidence *KeyEvidence |
| + wantErr error |
| + wantKey string // the pinned key's digest source, on success |
| }{ |
| - {"an unused platform key and an unpublished app key", withKey, nil, ErrFilesUnevidenced}, |
| - {"a platform key, the reader says unevidenced", withKey, func(context.Context, KeyQuestion) (KeySource, error) { return KeyUnevidenced, nil }, ErrFilesUnevidenced}, |
| - {"an adoption record and the platform key", withKey, func(context.Context, KeyQuestion) (KeySource, error) { return KeyAdopted, nil }, nil}, |
| - {"an adoption record without the platform key", "CARLOS_ADMIN_TOKEN=t\n", func(context.Context, KeyQuestion) (KeySource, error) { return KeyAdopted, nil }, ErrFilesUnevidenced}, |
| - {"an adoption record, an empty key", "CARLOS_INSTANCE_KEY=\n", func(context.Context, KeyQuestion) (KeySource, error) { return KeyAdopted, nil }, ErrFilesUnevidenced}, |
| - {"the bucket-held config's key", "", func(context.Context, KeyQuestion) (KeySource, error) { return KeyConfig, nil }, nil}, |
| - {"the reader fails", withKey, func(context.Context, KeyQuestion) (KeySource, error) { return KeyUnevidenced, errReaderDown }, errReaderDown}, |
| + {name: "an unused platform key and an unpublished app key (no reader)", env: "CARLOS_INSTANCE_KEY=k1\n", record: nil, evidence: nil, wantErr: ErrFilesUnevidenced}, |
| + {name: "the platform key, no adoption record", env: "CARLOS_INSTANCE_KEY=k1\n", evidence: config(""), wantErr: ErrFilesUnevidenced}, |
| + {name: "the writer adopted the env key", env: "CARLOS_ADMIN_TOKEN=t\nCARLOS_INSTANCE_KEY=k1\n", record: adopt("k1"), evidence: config(""), wantKey: "k1"}, |
| + {name: "a mismatched adoption", env: "CARLOS_INSTANCE_KEY=k1\n", record: adopt("k2"), evidence: config(""), wantErr: ErrFilesUnevidenced}, |
| + {name: "another session's adoption", env: "CARLOS_INSTANCE_KEY=k1\n", record: &AdoptionRecord{V: 1, Host: host, Incarnation: "inc", Session: 2, Generation: "g3", KeySHA256: KeyDigest("k1")}, evidence: config(""), wantErr: ErrFilesUnevidenced}, |
| + {name: "another generation's adoption", env: "CARLOS_INSTANCE_KEY=k1\n", record: &AdoptionRecord{V: 1, Host: host, Incarnation: "inc", Session: 3, Generation: "g2", KeySHA256: KeyDigest("k1")}, evidence: config(""), wantErr: ErrFilesUnevidenced}, |
| + {name: "a configured key overridden by the env", env: "CARLOS_INSTANCE_KEY=k2\n", record: adopt("c1"), evidence: config("CARLOS_INSTANCE_KEY=c1\n"), wantErr: ErrFilesUnevidenced}, |
| + {name: "the env key that overrides the configured one, adopted", env: "CARLOS_INSTANCE_KEY=k2\n", record: adopt("k2"), evidence: config("CARLOS_INSTANCE_KEY=c1\n"), wantKey: "k2"}, |
| + {name: "a configured key, adopted, no env key", env: "CARLOS_ADMIN_TOKEN=t\n", record: adopt("c1"), evidence: config("CARLOS_INSTANCE_KEY=c1\n"), wantKey: "c1"}, |
| + {name: "an unreadable env item", env: "CARLOS_INSTANCE_KEY=k1\n", unreadable: true, record: adopt("k1"), evidence: config("")}, |
| + {name: "a duplicate assignment: the last wins, as at launch", env: "CARLOS_INSTANCE_KEY=a\nCARLOS_INSTANCE_KEY=b\n", record: adopt("a"), evidence: config(""), wantErr: ErrFilesUnevidenced}, |
| + {name: "a duplicate assignment adopted as launched", env: "CARLOS_INSTANCE_KEY=a\nCARLOS_INSTANCE_KEY=b\n", record: adopt("b"), evidence: config(""), wantKey: "b"}, |
| + {name: "an export line is not the variable at launch", env: "export CARLOS_INSTANCE_KEY=k1\n", record: adopt("k1"), evidence: config(""), wantErr: ErrFilesUnevidenced}, |
| + {name: "an export line, the configured key delivered and adopted", env: "export CARLOS_INSTANCE_KEY=k1\n", record: adopt("c1"), evidence: config("CARLOS_INSTANCE_KEY=c1\n"), wantKey: "c1"}, |
| + {name: "an empty key", env: "CARLOS_INSTANCE_KEY=\n", record: adopt(""), evidence: config(""), wantErr: ErrFilesUnevidenced}, |
| } { |
| t.Run(tc.name, func(t *testing.T) { |
| claims, data := release.NewDirStore(t.TempDir()), release.NewDirStore(t.TempDir()) |
| m := hostfiles.Manifest{V: hostfiles.ManifestVersion, Seq: 4, Entries: map[string]hostfiles.Entry{}, Platform: map[string]hostfiles.Entry{}} |
| - if tc.env != "" { |
| - sum := sha256.Sum256([]byte(tc.env)) |
| + put := func(item string, body []byte, store bool) { |
| + sum := sha256.Sum256(body) |
| sha := hex.EncodeToString(sum[:]) |
| key := pinLocation + "/files/g3/objects/" + fmt.Sprintf("%016d-%s", 4, sha) |
| - if _, err := data.Put(ctx, key, []byte(tc.env), ""); err != nil { |
| - t.Fatal(err) |
| + if store { |
| + if _, err := data.Put(ctx, key, body, ""); err != nil { |
| + t.Fatal(err) |
| + } |
| } |
| - m.Platform[hostfiles.EnvItem] = hostfiles.Entry{Object: key, SHA256: sha, Size: int64(len(tc.env)), Mode: 0o600, Seq: 4} |
| + m.Platform[item] = hostfiles.Entry{Object: key, SHA256: sha, Size: int64(len(body)), Mode: 0o600, Seq: 4} |
| } |
| - putManifest(t, data, "g3", m) |
| - var asked KeyQuestion |
| - p := StorePinner{Claims: claims, Data: data, Release: routeRelease, Bucket: "data", Region: "ie", SelectDB: selectAt(41)} |
| - if tc.evidence != nil { |
| - p.KeyEvidence = func(ctx context.Context, q KeyQuestion) (KeySource, error) { asked = q; return tc.evidence(ctx, q) } |
| + if tc.env != "" { |
| + put(hostfiles.EnvItem, []byte(tc.env), !tc.unreadable) |
| } |
| - if _, err := p.Pin(ctx, pinClaim(), at); !errors.Is(err, tc.want) { |
| - t.Fatalf("got %v, want %v", err, tc.want) |
| + if tc.record != nil { |
| + raw, _ := json.Marshal(tc.record) |
| + put(AdoptionItem, raw, true) |
| + } |
| + putManifest(t, data, "g3", m) |
| + o := pinClaim() |
| + o.Claim.Holder.Incarnation = "inc" |
| + p := StorePinner{Claims: claims, Data: data, Release: routeRelease, Bucket: "data", Region: "ie", SelectDB: selectAt(41), KeyEvidence: tc.evidence} |
| + ref, err := p.Pin(ctx, o, at) |
| + switch { |
| + case tc.unreadable: |
| + if err == nil || errors.Is(err, ErrFilesUnevidenced) { |
| + t.Fatalf("an unreadable env item is an error, never no evidence: %v", err) |
| + } |
| + return |
| + case tc.wantErr != nil: |
| + if !errors.Is(err, tc.wantErr) { |
| + t.Fatalf("got %v, want %v", err, tc.wantErr) |
| + } |
| + return |
| + case err != nil: |
| + t.Fatal(err) |
| } |
| - if tc.evidence != nil && (asked.Host != host || asked.Release != "v1.4.2") { |
| - t.Fatalf("asked %+v", asked) |
| + obj, err := ReadPin(ctx, claims, ref) |
| + if err != nil || obj.Files.KeySHA256 != KeyDigest(tc.wantKey) { |
| + t.Fatalf("pinned key digest %q, want %s's: %v", obj.Files.KeySHA256, tc.wantKey, err) |
| } |
| }) |
| } |
| } |
| - |
| -var errReaderDown = errors.New("adoption records unreadable") |
| ``` |
| Run: `nice -n 19 go test ./internal/hostrelocate/ -run TestStorePinner -count=1`. Expected: PASS (rev 3's scratch build ran it with every existing pinner test). |
| @@ -2695,7 +3159,10 @@ func (a ClaimAssembly) relocationInstall(cfg RelocationAssemblyConfig) func(cont |
| return err |
| } |
| journal.Steps["files"] = strconv.FormatInt(m.Seq, 10) |
| - if _, ok := m.Platform[hostfiles.EnvItem]; ok { |
| + // A deleted EnvItem is the same as none (Astra round 3 finding |
| + // 5): there is no object to fetch, and any env file here is not |
| + // the host's, so it goes. Otherwise the exact bytes (O3). |
| + if e, ok := m.Platform[hostfiles.EnvItem]; ok && !e.Deleted { |
| env, err := hostfiles.ReadPlatform(ctx, cfg.Data, prefix, m, hostfiles.EnvItem, origins) |
| if err != nil { |
| return err |
| @@ -2705,7 +3172,9 @@ func (a ClaimAssembly) relocationInstall(cfg RelocationAssemblyConfig) func(cont |
| } |
| journal.Steps["env"] = "written" |
| } else if err := os.Remove(paths.EnvPath); err != nil && !errors.Is(err, os.ErrNotExist) { |
| - return err // ENOENT in P's manifest means the host has none |
| + return err |
| + } else { |
| + journal.Steps["env"] = "none" |
| } |
| ws, ok, err := hostfiles.LoadWake(ctx, cfg.Data, prefix, m, origins) |
| if err != nil { |
| @@ -2736,6 +3205,24 @@ func (a ClaimAssembly) relocationInstall(cfg RelocationAssemblyConfig) func(cont |
| return hostclaim.LocalInstall{}, err |
| } |
| journal.Steps["render"] = rendered.SHA256 |
| + // O2' (b') at install (ruling 65): the key this install will |
| + // deliver, the restored env over the config just rendered, must be |
| + // the one pinned by digest; a configuration change since the pin |
| + // makes the install refuse rather than deliver another key. |
| + if pin.Files.KeySHA256 != "" { |
| + configEnv, err := os.ReadFile(rendered.Path) |
| + if err != nil { |
| + return hostclaim.LocalInstall{}, err |
| + } |
| + hostEnv, err := os.ReadFile(paths.EnvPath) |
| + if err != nil && !errors.Is(err, os.ErrNotExist) { |
| + return hostclaim.LocalInstall{}, err |
| + } |
| + if key := hostrelocate.EffectiveInstanceKey(configEnv, hostEnv); key == "" || hostrelocate.KeyDigest(key) != pin.Files.KeySHA256 { |
| + return hostclaim.LocalInstall{}, fmt.Errorf("%w: the instance key this install would deliver is not the pinned key", hostrelocate.ErrRefused) |
| + } |
| + journal.Steps["key"] = pin.Files.KeySHA256 |
| + } |
| // The pinned release (R9): install_release, never the route's |
| // version, through the claim-aware held stage (never MaterializeHeld, |
| // which refuses with CARLOS_HOST_CLAIM on). The edge unit cannot |
| @@ -3229,10 +3716,12 @@ In `internal/hostrelocate/participant.go` rename `certifiedFor` to `CertifiedFor |
| if err != nil || !passed { |
| return obs, errors.Join(ErrNotYet, err) |
| } |
| -@@ -201,10 +243,44 @@ |
| +@@ -200,11 +242,45 @@ |
| + if err != nil || fresh.Absent { |
| return obs, errors.Join(ErrNotYet, err) |
| } |
| - return p.update(ctx, fresh, hostclaim.RecordProbe(id, hostclaim.Probe{Box: p.Box, Session: session})) |
| +- return p.update(ctx, fresh, hostclaim.RecordProbe(id, hostclaim.Probe{Box: p.Box, Session: session})) |
| ++ return p.update(ctx, fresh, hostclaim.RecordProbe(id, hostclaim.Probe{Box: p.Box, Session: session, At: p.Now()})) |
| + case executor && c.Op.Phase == hostclaim.PhaseRelocatingProbing && c.Op.Journal["probe_stop"] == "": |
| + // Ruling 49: the passed probe's session is stopped, with root's |
| + // receipt, before the serving CAS ends it. |
| @@ -4497,7 +4986,7 @@ func (a ClaimAssembly) relocationStopProbe(cfg RelocationAssemblyConfig) func(co |
| // hold: an excluded box's row is its park-local's. |
| func (a ClaimAssembly) relocationReleaseHold(cfg RelocationAssemblyConfig) func(context.Context, string, string) error { |
| return func(ctx context.Context, host, op string) error { |
| - if cfg.Registry == nil || cfg.Activator == nil { |
| + if cfg.Registry == nil || cfg.Activator == nil || cfg.Discharge == nil { |
| return ErrClaimStopUnavailable |
| } |
| local, err := a.localStopIdentity(ctx) |
| @@ -4511,6 +5000,17 @@ func (a ClaimAssembly) relocationReleaseHold(cfg RelocationAssemblyConfig) func( |
| if obs.Absent || obs.Claim.Holder == nil || obs.Claim.Op.Kind != hostclaim.KindServing || obs.Claim.Op.Exec != local.Actor.Box || obs.Claim.Holder.WriterBox != local.Actor.Box { |
| return nil |
| } |
| + // Ruling 66: no serving admission may become possible before the |
| + // participation is discharged, or the admission moves the holder |
| + // off the probe session and C10's evidence no longer matches. The |
| + // hold and the fence go only after the discharge succeeded; a |
| + // refusal keeps both and is retried next pass. |
| + if done, ok := hostclaim.ReadCompletedRelocation(obs.Claim); !ok || done.Op != op || done.Stop.Session != obs.Claim.Holder.Session { |
| + return fmt.Errorf("%w: %s's serving holder is not %s's stopped probe session", hostrelocate.ErrRefused, host, op) |
| + } |
| + if err := cfg.Discharge(ctx, obs); err != nil { |
| + return err |
| + } |
| if _, err := cfg.Registry.ReleaseHold(host, op); err != nil { |
| return err |
| } |
| @@ -4751,7 +5251,18 @@ func ToInstalledAt(opID string, a PreparedAuthority, now time.Time) func(*Claim) |
| ```diff |
| --- a/internal/hostclaim/relocation.go |
| +++ b/internal/hostclaim/relocation.go |
| -@@ -1009,6 +1009,11 @@ |
| +@@ -165,6 +165,10 @@ |
| + type Probe struct { |
| + Box string `json:"box"` |
| + Session uint64 `json:"session"` |
| ++ // At is when the probe passed (rev 4, ruling 67): the start of the |
| ++ // bound on obtaining its stop receipt. Zero in a probe journaled before |
| ++ // rev 4; the worker then counts from probing_since. |
| ++ At time.Time `json:"at,omitzero"` |
| + } |
| + |
| + // Target names a box for the relocation's target slot. |
| +@@ -1009,6 +1013,11 @@ |
| if !readJournal(c.Op, "probe", &p) || p.Box != c.Op.Exec || p.Session != c.Holder.Session || c.Holder.WriterBox != p.Box { |
| return fmt.Errorf("%w: no passing probe of the current session", ErrRelocation) |
| } |
| @@ -4763,7 +5274,7 @@ func ToInstalledAt(opID string, a PreparedAuthority, now time.Time) func(*Claim) |
| target := c.Op.Exec |
| var owed []string |
| readJournal(c.Op, "installed_boxes", &owed) |
| -@@ -1052,7 +1057,21 @@ |
| +@@ -1052,7 +1061,21 @@ |
| c.Obligations = append(c.Obligations, Obligation{ID: newID(), Op: opID, Kind: "park-generation", Region: region, Generation: g, Retirement: newID(), Owner: "region:" + region}) |
| } |
| } |
| @@ -4793,7 +5304,7 @@ The worker completes only after the stop: |
| ```diff |
| --- a/internal/hostrelocate/worker.go |
| +++ b/internal/hostrelocate/worker.go |
| -@@ -394,13 +394,16 @@ |
| +@@ -394,13 +394,31 @@ |
| if err != nil { |
| return obs, err |
| } |
| @@ -4806,12 +5317,27 @@ The worker completes only after the stop: |
| return next, err |
| } |
| + if c.Op.Journal["probe"] != "" { |
| -+ return obs, ErrNotYet // the target stops its probe session first (ruling 49) |
| ++ // The target stops its probe session first (ruling 49). Ruling |
| ++ // 67: bounded. If root's stop receipt cannot be had (the |
| ++ // execution lease lost after RecordProbe, so the participant |
| ++ // never passes Ready; a stop root keeps refusing), the op is |
| ++ // stuck ReleaseDeadline after the probe passed, for an operator. |
| ++ // Nothing is released: the participation's denial, the row's |
| ++ // hold and the session stay as they are. |
| ++ var p hostclaim.Probe |
| ++ passed := since.Add(ProbeDeadline) |
| ++ if json.Unmarshal([]byte(c.Op.Journal["probe"]), &p) == nil && !p.At.IsZero() { |
| ++ passed = p.At |
| ++ } |
| ++ if w.Now().Sub(passed) > ReleaseDeadline { |
| ++ return w.update(ctx, obs, hostclaim.MarkStuck(id, fmt.Sprintf("%s passed its probe but no root stop receipt for session %d within %s: an operator must obtain the stop", c.Op.Exec, p.Session, ReleaseDeadline))) |
| ++ } |
| ++ return obs, ErrNotYet |
| + } |
| if w.Now().Sub(since) > ProbeDeadline { |
| return w.update(ctx, obs, hostclaim.ToProbeFailed(id, w.Now())) |
| } |
| -@@ -410,9 +413,19 @@ |
| +@@ -410,9 +428,19 @@ |
| if err != nil || !done || !acked(c) { |
| return obs, errors.Join(ErrNotYet, err) |
| } |
| @@ -4979,15 +5505,15 @@ func relocationLeases(s relocationStores) func(context.Context, string) (hostrel |
| // relocationAssemblyConfig fills Task 8a's config from the bound assembly |
| // and the stores. |
| -func relocationAssemblyConfig(a *ClaimAssembly, providers ClaimProviders, s relocationStores, reg *registry.Registry, act RelocationActivator, d hostactors.Directory) (RelocationAssemblyConfig, error) { |
| +func relocationAssemblyConfig(a *ClaimAssembly, providers ClaimProviders, s relocationStores, reg *registry.Registry, act RelocationActivator, d hostactors.Directory, discharge func(context.Context, hostclaim.Observed) error) (RelocationAssemblyConfig, error) { |
| src, ok := providers.Selector.(hostbox.WorkSource) |
| - if !ok || a == nil || reg == nil || act == nil { |
| + if !ok || a == nil || reg == nil || act == nil || discharge == nil { |
| return RelocationAssemblyConfig{}, hostwrite.ErrUnavailable |
| } |
| dd := a.Guard.DataDir |
| return RelocationAssemblyConfig{Directory: d, Data: s.Files, Versions: s.Versions, Bucket: s.Bucket, Region: s.Region, LeaseRegion: s.LeaseRegion, |
| Registry: reg, DataDir: dd, WorkDir: filepath.Join(dd, "relocate", "build"), ReleaseDir: filepath.Join(dd, "relocate", "release"), |
| - Activator: act, Source: src, Now: time.Now}, nil |
| + Activator: act, Source: src, Discharge: discharge, Now: time.Now}, nil |
| } |
| // channelRelease is ruling 30: the host's channel pointer in the control |
| @@ -5197,7 +5723,7 @@ Before `CARLOS_REGION_HA` is set on any box with a second box in its region, on |
| 14. **W4.** Root's relocating attempt reader is live: `carlos-instance-control` answers the relocating capability, and a probe's first launch is authorized from `<DataDir>/relocate/ops/<op>/attempts.json`. |
| 15. **Census.** After a relocation completes on dev, `find /data -path '*<host>*'` on the target lists only the host's instance directory and the census-opaque `relocate/` (N9), and on the source nothing outside them after its `park-local`; within a pass of completion `relocate/` holds nothing of that op. |
| 16. **C11.** Until it lands each target gets one launchable probe attempt; set `LaterProbeAttempts` only after it does. |
| -17. **O2'.** The instance-key plan's `KeyEvidence` reader is wired into `StorePinner`; until then only `instance.key` evidences a nonempty database's key. |
| +17. **O2′ (IK-R-B).** The instance-key plan rev 3's adoption record is published by the C8 provider and `KeyEvidence.ConfiguredEnv` is wired into `StorePinner`; until then only `instance.key` evidences a nonempty database's key. On dev: a relocation of an app on `carlos.InstanceKey` pins with `FilesPin.KeySHA256`, and changing its configured key between pin and install makes the install refuse. |
| --- |