amadan
plan: claim Plan 11 rev 2 (review r1: root verifies the reservation against the authoritative attempt, rechecks local authority before launch, validates shape before admission, bounded namespace wait)
Co-Authored-By: Codex <noreply@openai.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 file changed,
+2241
−34
docs/superpowers/plans/2026-10-09-host-lifecycle-claim-11-region-ha-integration.md+2241 −34
CI passed — run details
diff --git a/docs/superpowers/plans/2026-10-09-host-lifecycle-claim-11-region-ha-integration.md b/docs/superpowers/plans/2026-10-09-host-lifecycle-claim-11-region-ha-integration.md| index ef10f4881..fa382dd32 100644 |
| --- a/docs/superpowers/plans/2026-10-09-host-lifecycle-claim-11-region-ha-integration.md |
| +++ b/docs/superpowers/plans/2026-10-09-host-lifecycle-claim-11-region-ha-integration.md |
| @@ -1,6 +1,8 @@ |
| # Host lifecycle claim, Plan 11: region HA integration |
| -Implementation plan, 2026-10-09, rev 1. This implements the claim session's accepted answers in amadan discussion #236: R1-R5, the narrowed R8, R10, `relocation-release` and `enrolled_at`. Claim spec rev 6.31 is normative. This plan authorizes no production changes. |
| +Implementation plan, 2026-10-09, rev 2. This implements the claim session's accepted answers in amadan discussion #236: R1-R5, the narrowed R8, R10, `relocation-release` and `enrolled_at`. Claim spec rev 6.31 is normative. This plan authorizes no production changes. |
| + |
| +**Rev 2 — review round 1:** answers both P1 and both P2 findings in Task D's R10 root launch, with executable code and failing-first regressions. The published region HA signatures are unchanged. The root launch protocol capability becomes `isolated-unit-v2`, and internal root/kernel/unit interfaces are extended; agent and root must be upgraded together. The mandatory Task D amendment runs after G. Kinds without a root-side authoritative attempt verifier are refused; production operation launches remain gated on those verifiers. |
| ## Interfaces for region HA |
| @@ -92,7 +94,7 @@ Save this document's `apply.py` block in the scratch directory, then run the boo |
| CLAIM11_REPO=/home/paulca/amadan.net/carlos/platform/.claude/worktrees/replica-ownership |
| CLAIM11_PLAN="$CLAIM11_REPO/docs/superpowers/plans/2026-10-09-host-lifecycle-claim-11-region-ha-integration.md" |
| CLAIM11_SCRATCH=$(mktemp -d /home/paulca/.cache/tmp/claude-1001/plan11.XXXXXX) |
| -CLAIM11_SOCKET_TMP=$(mktemp -d /home/paulca/.cache/tmp/p11.XXXXXX) |
| +CLAIM11_SOCKET_TMP=$(mktemp -d /home/paulca/.cache/tmp/p.XXX) |
| export TMPDIR="$CLAIM11_SOCKET_TMP" |
| # Leave the default cache when writable. The brief permits this shared fallback. |
| if ! test -w "$(go env GOCACHE)"; then |
| @@ -5313,9 +5315,11 @@ Expected: the focused package set passes. |
| ## Task D: Carry the per-role reservation through isolated root launch |
| -Invariant: Every supported claim-enabled role executes through the existing isolated root protocol under its exact durable reservation; an unsupported shape or stale tuple cannot launch. |
| +Invariant: Root launches tenant code only for a reservation it has itself verified against the authoritative per-kind attempt (serving retry or operation attempt, including executable and input identity), under a trusted local participation state, deadline and regional quiet period rechecked under the host guard immediately before launch, and only for a launch shape fully validated before any admission or attempt reservation. Kinds without a verifier are refused. |
| + |
| +Rev 2 completes this task with the mandatory Task D amendment after G. The post-start wait stays on the same invocation and process, has a deadline, and consumes no additional retry while a valid namespace/exec setup completes. |
| -Extend the isolated-exec protocol additively. Root revalidates the holder, installation stamp, writer session, operation and executor, plus the per-role reservation and authenticated caller. Keep the existing systemd namespace isolation and terminal/cgroup checks. Do not reinterpret an ordinary start as a claim start, or fall back to an agent child if the extension is unavailable. Preflight runs before counting a probe attempt. |
| +Extend the isolated-exec protocol additively. Root revalidates the holder, installation stamp, writer session, operation and executor, plus the per-role reservation and authenticated caller. Keep the existing systemd namespace isolation and terminal/cgroup checks. Do not reinterpret an ordinary start as a claim start, or fall back to an agent child if the extension is unavailable. Preflight runs before any admission or serving/operation attempt reservation. Apply the amendment before treating R10 as implemented. |
| ### Failing tests |
| @@ -10821,52 +10825,2255 @@ nice -n 10 go test -p 1 -timeout 5m ./internal/hostbox ./internal/host ./interna |
| Expected: the focused package set passes. |
| -## Region HA integration and rollout |
| +## Task D amendment (Rev 2): verify authority at the root launch boundary |
| -Keep all flags off while landing the code. Registration alone does not activate relocation. The rollout still needs Plan 9's verified identity, complete owners, activation census and floor guard. Region HA additionally needs `CARLOS_REGION_HA`; this implementation shares Group D with `CARLOS_MOVES_PHASE1`, so it requires that group to be complete too. Missing owners, an older root protocol, unsupported role shapes and incomplete stop evidence all refuse. |
| +Apply this mandatory amendment **after Task G**, before final verification or activation. It corrects Task D's R10 implementation and uses Task G's durable `WritersClosed` latch and settled-intention handling. Keeping it here preserves the exact replay order of the existing A–G patches. Task D's initial implementation alone is not the completed launch boundary. |
| -The claim-enabled protocol in Task D carries a main role in main's isolated run unit, using the nested instance layout and a supported memory tier. Seed, sidecar and other layouts remain refused where the reservation cannot describe them. The existing flag-off `/start`, `/seed`, process backend and unit templates retain their existing path. Staged builds must also be root-owned and not writable by the agent or tenant. Verify the deployed stager satisfies that rule before activation. Do not remove those refusals to make a probe pass. Extend the reservation and root verification first if another shape is required. |
| +Review round 1 is `/home/paulca/.cache/tmp/claude-1001/own/findings-plan11-r1.md`. These are the four dispositions: |
| -Before region HA stage B is armed, rebase its adapters onto these interfaces and test these joins: |
| +| Finding | Implementation | Failing-first coverage | |
| +| --- | --- | --- | |
| +| P1: invented reservation IDs, invocations and inputs | Root reads the serving claim's attempting `WakeRetry`, or a root-installed reader of the operation kind's authoritative attempt. It compares every attempt/owner field, recomputes the unchanged `ClaimLaunchDigest` preimage, and independently hashes the pinned executable. Missing per-kind readers refuse. | `TestPlan11Rev2RootRefusesInventedAttempt`, `TestPlan11Rev2DigestAndExecutable`, `TestPlan11Rev2RootHashesThePinnedExecutable`, `TestPlan11Rev2OperationVerifierAndLocalState` | |
| +| P1: root bypasses participation, deadline and quiet-after | Root calls the trusted `LocalAuthority`, reads local operation participation, binds its process to the authenticated peer, and runs `hostexec.CheckLocal`. Serving requires no residual operation state. The final callback rereads the claim, checks `WritersClosed`, then revalidates the actor, attempt, local state, boot deadline and regional quiet period while still holding the host guard, after unit preparation and immediately before `systemctl start`. | `TestPlan11Rev2RootQuietPeriod`, `TestPlan11Rev2OperationVerifierAndLocalState`, `TestPlan11Rev2RechecksAfterUnitPreparation`, `TestPlan11Rev2UnitFinalCheckAndSnapshot` | |
| +| P2: incomplete preflight consumes attempts | Serving's read-only snapshot checks the declared seed and staged co-spawned sidecar before token creation, and carries its real required roles. Prerequisites validate roles, paths, tier, executable/input identity shape, arguments, environment and exact socket/database destinations. Preflight and launch share `IsolatedArgv`. Operation prerequisites also query root's per-kind verifier capability. | `TestPlan11Rev2ShapeBeforeReservation`, `TestRev2Shape*`, `TestPlan11Rev2KindPreflightRefusesUnverifiedOperation` | |
| +| P2: `Type=simple` returns before namespace/exec readiness | Unit start waits within its existing budget for the new invocation's PID. Root then polls for at most five seconds, bounded by request cancellation, for the same invocation and process start to expose both the expected namespace binding and running executable. It never calls Reserve or Start again during that wait. Timeout, exit, replacement, identity mismatch and cancellation refuse; cleanup settles only after proving the invocation gone. A replacement invocation is never stopped by the old reservation. | `TestPlan11Rev2SlowValidStart`, `TestPlan11Rev2WaitsForSameInvocation`, `TestPlan11Rev2UnitSameInvocation` | |
| -1. Replace its duplicate `AutomaticLoss`, `LocalDischarge`, loss-ID helper and `Claim.EvidenceRefs` with Task A's canonical definitions. Preserve its fence-consuming transitions, D4 retention and strict decode tests. |
| -2. Register the complete relocation owner set once. Route the probe through `OperationAttempts`; do not apply the old `Extend` hook or introduce a separate probe lifecycle. Keep the same admitted session through `relocating/probing -> relocating/serving`. The built region branch currently commits directly to kind `serving`; that kind change ends this session and requires a separate serving wake. It must not be treated as an implicit continuation. |
| -3. Reconcile its prepared-generation and `released` coverage admission with the base's generic admission. This plan permits only the accepted cold probe shape. It does not import region HA's transition producer or grant a new coverage authority. An adapter cannot bypass the gate with an optional policy callback. |
| -4. For a drain, call `ClaimAssembly.WritersStopped`, register the files publisher under `ClaimProducerFiles`, retain replication and file publication, and use root's observation. Drop the region branch's duplicate descriptor observer and consume the claim-owned `(TID, Device, Inode)` descriptor shape. Confirm with the live Litestream witness that descriptors opened lazily after `Replicate` do not leave the narrowing hint incomplete. An incomplete hint refuses the observation. After release publication, queue the stop-only obligation and request the full root stop. Pass its immutable key and digest to `RelocationReleaseReceipts.Complete`, then perform the region-owned executor transition. Never use a writers-only witness as a full stop receipt. |
| -5. A release without its exact root receipt remains pending, including if the source terminates before producing it. Plan 11 grants no fence-only discharge for this new obligation. That receipt-only rule is the accepted contract; the region-owned termination transition must account for it before activation. It must not silently drop the obligation or route it through generic chore completion. |
| -6. Preserve the source's files live set until the authoritative record commits. Moves Plan 5 must replace the stand-in view and prove the push, landing-install and rollback-return park exclusions at each destructive step. The model's passing J1 tests do not implement those production exclusions. |
| -7. Keep the R6 no-LTX refusal for every positive writer session. Keep volume-based automatic discharge refused for legacy roster entries with unknown enrollment time. Publish no invented timestamp or empty generation. |
| +The authoritative attempt's input digest includes config/release identity, executable path and SHA-256, arguments and environment. A request's `LaunchSpec` supplies only the digest preimage. Root grants no authority from it. Root builds the trampoline inputs from that verified preimage and stores a private, root-owned snapshot under `/run/carlos-control/exe/.claim/<iid>/launch.json`. A dedicated runtime drop-in adds a read-only bind over the tenant's `launch.run.json`, preserving the template's other namespace bindings. The mutable tenant file cannot substitute inputs between verification and exec. Stop and crash recovery remove only this snapshot and `60-claim-launch.conf`, after proving the unit empty. A failed final check clears its unstarted snapshot. Snapshot readiness and teardown still need the disposable systemd rollout test below. |
| -The release floor must include claim-level `evidence_refs` and its nested `AutomaticLoss` fields, roster `enrolled_at`, the `relocation-release` kind, and the launch journal and protocol additions, including `writers_closed` and `close-writers`. Publish it only after every enrolled actor and console can strictly decode these fields, the signed census proves the same executable set, and the existing guard transaction has armed the new floor. Do not substitute the branch's commit hash for a release tag or floor. Old binaries must not participate after publication. Rollback must respect the published floor. |
| +The production unit template binds `/data`; both production root construction and preflight refuse a different data root. Tests may supply isolated scratch roots directly to the lower-level unit and kernel fixtures. An operation's lease controller ID differs from the agent's launch controller ID: root verifies the lease's binding and lifetime locks through `CheckLocal`, and its PID, process start and kernel boot against the authenticated agent. It does not invent equality between the two controller IDs. |
| -Run the region HA two-box model and its joint review after the stage B adapters are rewritten. Then exercise isolated launch, cancellation, root crash recovery, writers-only stop and full stop in a designated disposable systemd/cgroup environment. This authoring session used local mocks and kernel fixtures; it did not create or modify production infrastructure. |
| +**Interface compatibility:** the region HA interfaces listed at the top, `OperationAttempts`, `ClaimLaunchDigest`, and `RootRoleLauncher` retain their signatures and digest contract. R10's capability changes to `isolated-unit-v2`; the request adds `spec` and a capability-only `kind`. Internal root/kernel/unit providers gain mandatory checks, `StartChecked`, and same-invocation observation. An old protocol cannot satisfy preflight. Production wires the serving verifier only. Landing and relocating remain refused at prerequisites and at root until root construction installs readers of their actual durable operation journals. The agent-side `CurrentAttempt` cache and `ClaimRelocationOwners` registration do not supply those readers. This is an explicit stage B activation dependency, not an interface change or an implicit permission to launch probes. |
| -## Final verification |
| +### Failing-first regressions against Rev 1 |
| -After all tasks, run the following in the disposable export. Run commands serially; no background test jobs. |
| +These tests compile against the completed A–G overlay without the amendment. Each must fail for the named behavior, not for a cache or socket setup error. |
| -```bash |
| -nice -n 10 go test -p 1 -timeout 5m ./internal/hostactors ./internal/hostclaim ./internal/hostbox ./internal/hostguard ./internal/stopevidence ./internal/instancectl -count=1 |
| -nice -n 10 go test -p 1 -timeout 5m ./internal/host ./internal/activator -run 'TestPlan11|TestStopWriters|TestWriters|TestVerifyWriters|TestVerifyNoProcess|TestNoProcess|TestRootRole|TestRootClaimProcess|TestClaimLaunch|TestClaimUnitStop|TestClaimStop|TestClaimProducer|TestPlan9D|TestClaimIsolation' -count=1 |
| -nice -n 10 go vet ./internal/hostactors ./internal/hostclaim ./internal/hostbox ./internal/hostguard ./internal/stopevidence ./internal/instancectl ./internal/host ./internal/activator |
| -nice -n 10 go build ./internal/host ./internal/activator ./internal/instancectl |
| -GOOS=darwin GOARCH=arm64 nice -n 10 go build ./internal/host ./internal/activator ./internal/instancectl |
| -GOOS=windows GOARCH=amd64 nice -n 10 go build ./internal/host ./internal/activator ./internal/instancectl |
| +<!-- apply: D2R tests --> |
| +```diff |
| +--- /dev/null |
| ++++ b/internal/host/claimlaunch_rev2_test.go |
| +@@ -0,0 +1,84 @@ |
| ++package host |
| ++ |
| ++import ( |
| ++ "github.com/carlosframework/platform/internal/cgroup" |
| ++ "github.com/carlosframework/platform/internal/hostbox" |
| ++ "github.com/carlosframework/platform/internal/hostclaim" |
| ++ "github.com/carlosframework/platform/internal/hostwrite" |
| ++ "github.com/carlosframework/platform/internal/instancectl" |
| ++ "github.com/carlosframework/platform/internal/provision" |
| ++ "strings" |
| ++ "testing" |
| ++ "time" |
| ++) |
| ++ |
| ++func TestPlan11Rev2RootRefusesInventedAttempt(t *testing.T) { |
| ++ for _, field := range []string{"id", "invocation", "inputs", "missing", "settled"} { |
| ++ t.Run(field, func(t *testing.T) { |
| ++ f := newLaunchFixture(t, servingOp()) |
| ++ switch field { |
| ++ case "id": |
| ++ f.req.Reservation.ID = "invented:main" |
| ++ case "invocation": |
| ++ f.req.Reservation.InvocationID = "invented" |
| ++ case "inputs": |
| ++ f.req.Reservation.Inputs = "invented" |
| ++ case "missing": |
| ++ f.claims.set(func(c *hostclaim.Claim) { c.Holder.WakeRetry = nil }) |
| ++ case "settled": |
| ++ f.claims.set(func(c *hostclaim.Claim) { c.Holder.WakeRetry = &hostclaim.WakeRetry{EpisodeID: "old", State: "open"} }) |
| ++ } |
| ++ if _, err := f.root.Serve(t.Context(), f.peer, f.req); err == nil { |
| ++ t.Fatal("root launched without its authoritative attempt") |
| ++ } |
| ++ f.noReservation(t, field) |
| ++ }) |
| ++ } |
| ++} |
| ++ |
| ++func TestPlan11Rev2RootQuietPeriod(t *testing.T) { |
| ++ f := newLaunchFixture(t, servingOp()) |
| ++ f.claims.set(func(c *hostclaim.Claim) { c.QuietAfter = map[string]time.Time{"eu": time.Now().Add(time.Hour)} }) |
| ++ if _, err := f.root.Serve(t.Context(), f.peer, f.req); err == nil { |
| ++ t.Fatal("root launched during the regional quiet period") |
| ++ } |
| ++ f.noReservation(t, "quiet") |
| ++} |
| ++ |
| ++type rev2SlowKernel struct { |
| ++ *launchKernel |
| ++ calls int |
| ++} |
| ++ |
| ++func (k *rev2SlowKernel) Observe(s instancectl.ClaimUnitStarted) (ClaimLaunchObserved, error) { |
| ++ k.calls++ |
| ++ if k.calls < 4 { |
| ++ return ClaimLaunchObserved{ProcessStart: k.observed.ProcessStart, Group: k.observed.Group}, nil |
| ++ } |
| ++ return k.launchKernel.Observe(s) |
| ++} |
| ++ |
| ++func TestPlan11Rev2SlowValidStart(t *testing.T) { |
| ++ f := newLaunchFixture(t, servingOp()) |
| ++ k := &rev2SlowKernel{launchKernel: f.kernel} |
| ++ f.root.Kernel = k |
| ++ if _, err := f.root.Serve(t.Context(), f.peer, f.req); err != nil { |
| ++ t.Fatalf("slow valid start was killed: %v", err) |
| ++ } |
| ++ starts, stops := f.units.counts() |
| ++ if starts != 1 || stops != 0 || k.calls != 4 { |
| ++ t.Fatalf("starts=%d stops=%d observations=%d", starts, stops, k.calls) |
| ++ } |
| ++} |
| ++ |
| ++func TestPlan11Rev2ShapeBeforeReservation(t *testing.T) { |
| ++ host := "shop.example" |
| ++ a := &ClaimRoleAssembly{Root: fakeLaunchRoot{protocol: hostbox.RoleLaunchProtocol}, ManagedRoots: map[string]string{"data": "/data"}} |
| ++ in := ClaimLaunchInputs{Roles: []string{"main"}, DBPath: provision.NestedDBPath("/data", host), Limit: cgroup.TierDefault, Executable: "/opt/carlos/versions/v1/app", ExecutableSHA256: strings.Repeat("a", 64), Identity: hostwrite.Identity{Config: "config", Release: "release"}, Env: []string{"CARLOS_ADMIN_TOKEN=test"}} |
| ++ for _, args := range [][]string{{"--db", in.DBPath}, {"--socket", "/run/other.sock", "--db", in.DBPath}, {"--socket", provision.NestedSocket(host), "--db", in.DBPath, "--exec", "/bin/sh"}} { |
| ++ in.Args = args |
| ++ if err := a.PreflightLaunch(host, in); err == nil { |
| ++ t.Fatalf("malformed shape passed prerequisites: %v", args) |
| ++ } |
| ++ } |
| ++} |
| ``` |
| -Check `gofmt -l` on every changed Go file. Before a later implementation commit lands, run the repository's full build, vet, test and formatting gate and obtain green amadan CI on that tip. This plan's focused overlay verification does not replace that landing gate. |
| - |
| -### Authoring results |
| +```bash |
| +python3 "$CLAIM11_SCRATCH/apply.py" "$CLAIM11_PLAN" D2R tests |
| +nice -n 10 go test -p 1 -timeout 5m ./internal/host -run '^TestPlan11Rev2' -count=1 |
| +``` |
| -The plan was replayed from a fresh export of the pinned base. All fourteen test and implementation patch blocks applied in order, and the resulting changed Go files matched the tested combined overlay byte for byte. No git index or ref was changed. |
| +Expected red: invented/missing/settled attempts launch; a regional quiet period is ignored; a slow valid start is killed; malformed arguments pass prerequisites. All four behavioral failures were observed against Rev 1. |
| -The supporting packages `hostactors`, `hostclaim`, `hostbox`, `hostguard`, `stopevidence` and `instancectl` passed their full package tests. The focused host and activator set passed, including Plan 9 dispatch, writers-only stop, descriptor and no-process observations, root launch, receipt validation and disabled-flag regressions. Vet passed for all eight packages. Builds of host, activator and instancectl passed on Linux and cross-compiled for darwin/arm64 and windows/amd64. Formatting was checked on all changed files. |
| +### Additional tests and authoritative fixtures |
| -Red runs first exposed the missing record, enrollment, dispatch, writer-stop, observation and launch interfaces. Integration regressions then demonstrated mutable policy authority, cached witness aliasing, an omitted isolated cgroup, hidden root observation interfaces, a stale console enrollment time, refusal of a valid session-zero release, recovery settling an unobserved unit, and a probe launch before the executor became the admitted writer. Each corresponding guard was fixed and its focused tests passed. The final plan places every test block before its implementation. |
| +Update the successful fixtures to carry a real durable serving retry, or a separate operation reader plus an active local lease with held lifetime/live locks. Retry fixtures must advance the authoritative attempt too; changing only the request is now refused. The following adds final-check races, local-state refusals, input substitutions, snapshot binding/cleanup, per-kind preflight, kernel hashing and bounded readiness tests. |
| -Some early delegated socket runs failed because of sandbox restrictions or long Unix socket paths; the parent reran the focused combined set under its short disk-backed TMPDIR and it passed. The long `TestClaimCompletedProducersBoundStopAndRetry` test exceeded an early delegated timeout; it is outside the final focused set. No production systemd unit, deployment, bucket, roster or floor was changed. Real unit isolation and the joint two-box stage B activation remain rollout checks, not claimed authoring results. |
| +<!-- apply: D2 tests --> |
| +```diff |
| +--- /dev/null |
| ++++ b/internal/activator/claim_shape_rev2_shape_test.go |
| +@@ -0,0 +1,49 @@ |
| ++package activator |
| ++ |
| ++import ( |
| ++ "errors" |
| ++ "testing" |
| ++ |
| ++ "github.com/carlosframework/platform/internal/hostbox" |
| ++ "github.com/carlosframework/platform/internal/registry" |
| ++) |
| ++ |
| ++func TestRev2ShapeIsolatedArgv(t *testing.T) { |
| ++ const launch, run = "/data/instances/i1", "/run/carlos/instances/i1" |
| ++ sock, db := run+"/app.sock", launch+"/app.db" |
| ++ args := func(extra ...string) []string { return append([]string{"--socket", sock, "--db", db}, extra...) } |
| ++ env := []string{"CARLOS_ADMIN_TOKEN=t", "A=1"} |
| ++ argv, s, d, err := IsolatedArgv("h.example", launch, run, args(), env) |
| ++ if err != nil || s != sock || d != db || len(argv) != 5 || argv[0] != "/opt/carlos/live/h.example" && argv[0] == "" { |
| ++ t.Fatalf("valid: %v %q %q %v", err, s, d, argv) |
| ++ } |
| ++ for name, tc := range map[string]struct{ args, env []string }{ |
| ++ "odd": {args("--x"), env}, |
| ++ "extra": {args("--x", "y"), env}, |
| ++ "dup socket": {args("--socket", sock), env}, |
| ++ "dup db": {args("--db", db), env}, |
| ++ "NUL": {[]string{"--socket", sock + "\x00", "--db", db}, env}, |
| ++ "wrong dir": {[]string{"--socket", "/tmp/app.sock", "--db", db}, env}, |
| ++ "unclean": {[]string{"--socket", run + "/../i1/app.sock", "--db", db}, env}, |
| ++ "no env": {args(), nil}, |
| ++ "no token": {args(), []string{"A=1"}}, |
| ++ "empty token": {args(), []string{"CARLOS_ADMIN_TOKEN="}}, |
| ++ "dup env": {args(), append([]string{"A=2"}, env...)}, |
| ++ "bad env": {args(), append([]string{"X"}, env...)}, |
| ++ "NUL env": {args(), append([]string{"X=\x00"}, env...)}, |
| ++ } { |
| ++ if _, _, _, err := IsolatedArgv("h.example", launch, run, tc.args, tc.env); !errors.Is(err, hostbox.ErrRoleLaunchShape) { |
| ++ t.Fatalf("%s: %v", name, err) |
| ++ } |
| ++ } |
| ++} |
| ++ |
| ++func TestRev2ShapeClaimLaunchRoles(t *testing.T) { |
| ++ b := &ExecBackend{DataDir: t.TempDir()} |
| ++ if got := b.ClaimLaunchRoles(registry.Route{Host: "h.example"}); len(got) != 1 || got[0] != "main" { |
| ++ t.Fatalf("roles=%v", got) |
| ++ } |
| ++ if got := b.ClaimLaunchRoles(registry.Route{Host: "h.example", Seed: `["init"]`}); len(got) != 2 || got[1] != "seed" { |
| ++ t.Fatalf("roles=%v", got) |
| ++ } |
| ++} |
| +--- a/internal/host/claimlaunch_kernel_linux_test.go |
| ++++ b/internal/host/claimlaunch_kernel_linux_test.go |
| +@@ -3,6 +3,8 @@ |
| + package host |
| + |
| + import ( |
| ++ "crypto/sha256" |
| ++ "encoding/hex" |
| + "os" |
| + "path/filepath" |
| + "strings" |
| +@@ -112,3 +114,31 @@ |
| + t.Fatal("process outside the unit group accepted") |
| + } |
| + } |
| ++ |
| ++func TestPlan11Rev2RootHashesThePinnedExecutable(t *testing.T) { |
| ++ root := t.TempDir() |
| ++ path := filepath.Join(root, "versions", "v1", "app") |
| ++ if err := os.MkdirAll(filepath.Dir(path), 0755); err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ if err := os.Chmod(root, 0755); err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ if err := os.WriteFile(path, []byte("app"), 0555); err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ k := linuxClaimLaunchKernel{OptRoot: root, Owner: uint32(os.Getuid())} |
| ++ sum := sha256.Sum256([]byte("app")) |
| ++ digest := hex.EncodeToString(sum[:]) |
| ++ id := plan11LaunchFileID(t, path) |
| ++ if err := k.VerifyDigest(t.Context(), path, id, digest); err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ if err := k.VerifyDigest(t.Context(), path, id, strings.Repeat("0", 64)); err == nil { |
| ++ t.Fatal("root accepted forged executable digest") |
| ++ } |
| ++ id.Inode++ |
| ++ if err := k.VerifyDigest(t.Context(), path, id, digest); err == nil { |
| ++ t.Fatal("root accepted another inode") |
| ++ } |
| ++} |
| +--- /dev/null |
| ++++ b/internal/host/claimlaunch_rev2_linux_test.go |
| +@@ -0,0 +1,288 @@ |
| ++//go:build linux |
| ++ |
| ++package host |
| ++ |
| ++import ( |
| ++ "context" |
| ++ "encoding/json" |
| ++ "errors" |
| ++ "net" |
| ++ "os" |
| ++ "path/filepath" |
| ++ "strings" |
| ++ "testing" |
| ++ "time" |
| ++ |
| ++ "github.com/carlosframework/platform/internal/hostbox" |
| ++ "github.com/carlosframework/platform/internal/hostclaim" |
| ++ execlease "github.com/carlosframework/platform/internal/hostexec" |
| ++ "github.com/carlosframework/platform/internal/hostwrite" |
| ++ "github.com/carlosframework/platform/internal/instancectl" |
| ++ "github.com/carlosframework/platform/internal/provision" |
| ++ "golang.org/x/sys/unix" |
| ++) |
| ++ |
| ++type rev2Local struct{ err error } |
| ++ |
| ++func (l *rev2Local) Check(context.Context, hostbox.Actor, *hostclaim.Claim, hostbox.Selection) error { |
| ++ return l.err |
| ++} |
| ++ |
| ++type rev2Clock struct { |
| ++ wall time.Time |
| ++ boot int64 |
| ++} |
| ++ |
| ++func (c *rev2Clock) Now() (time.Time, int64, error) { return c.wall, c.boot, nil } |
| ++func (c *rev2Clock) After(d time.Duration) <-chan time.Time { return time.After(d) } |
| ++ |
| ++type rev2Attempts struct { |
| ++ attempt hostclaim.WakeRetryAttempt |
| ++ op hostclaim.Op |
| ++ stamp, session uint64 |
| ++} |
| ++ |
| ++func (a *rev2Attempts) ReadAttempt(_ context.Context, o hostclaim.Observed) (hostclaim.WakeRetryAttempt, error) { |
| ++ if o.Claim.Op.ID != a.op.ID || o.Claim.Op.Kind != a.op.Kind || o.Claim.Op.Phase != a.op.Phase || o.Claim.Holder.Stamp != a.stamp || o.Claim.Holder.Session != a.session { |
| ++ return hostclaim.WakeRetryAttempt{}, hostbox.ErrChanged |
| ++ } |
| ++ return a.attempt, nil |
| ++} |
| ++ |
| ++func configureRev2Launch(t *testing.T, f *launchFixture) { |
| ++ t.Helper() |
| ++ f.root.DataRoot = f.root.Guard.DataDir |
| ++ f.root.Local = &rev2Local{} |
| ++ f.root.Clock = &rev2Clock{wall: time.Now(), boot: 100} |
| ++ f.root.ReadyBudget, f.root.ReadyPoll = 40*time.Millisecond, time.Millisecond |
| ++ f.root.ExecLockDir = t.TempDir() |
| ++ f.kernel.digest = strings.Repeat("a", 64) |
| ++ f.kernel.observed.Running = f.kernel.observed.Executable |
| ++ f.req.Spec = &hostbox.LaunchSpec{Identity: hostwrite.Identity{Config: "config", Release: "release"}, Executable: f.kernel.path, Digest: f.kernel.digest, Args: []string{"--socket", filepath.Join(provision.InstanceRunDir(f.target.Host), "app.sock"), "--db", provision.NestedDBPath(f.root.DataRoot, f.target.Host)}, Env: []string{"CARLOS_ADMIN_TOKEN=test"}} |
| ++ digest, err := f.req.Spec.Hash() |
| ++ if err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ f.req.Reservation.Inputs = digest |
| ++ a := hostclaim.WakeRetryAttempt{ID: "attempt-1", InvocationID: "inv-1", LaunchInputs: digest, Box: f.target.Box, Enrollment: f.target.Enrollment, BootID: f.peer.BootID, ControllerID: f.peer.ControllerID, Session: f.target.Session} |
| ++ if f.claims.obs.Claim.Op.Kind == hostclaim.KindServing { |
| ++ f.claims.set(func(c *hostclaim.Claim) { |
| ++ c.Holder.WakeRetry = &hostclaim.WakeRetry{EpisodeID: "episode", State: "attempting", AttemptsReserved: 1, Attempt: &a} |
| ++ }) |
| ++ } else { |
| ++ f.root.Attempts = map[string]claimLaunchAttemptReader{f.claims.obs.Claim.Op.Kind: &rev2Attempts{attempt: a, op: f.claims.obs.Claim.Op, stamp: f.target.Stamp, session: f.target.Session}} |
| ++ writeRev2Local(t, f, "active", 1000) |
| ++ for _, suffix := range []string{".lock", ".live"} { |
| ++ file, err := os.OpenFile(filepath.Join(f.root.ExecLockDir, f.target.Host+suffix), os.O_CREATE|os.O_RDWR, 0660) |
| ++ if err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ if err := file.Chmod(0660); err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ if err := unix.Flock(int(file.Fd()), unix.LOCK_EX|unix.LOCK_NB); err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ t.Cleanup(func() { file.Close() }) |
| ++ } |
| ++ } |
| ++} |
| ++func writeRev2Local(t *testing.T, f *launchFixture, state string, deadline int64) { |
| ++ t.Helper() |
| ++ s := execlease.State{V: 1, Binding: execlease.Binding{Host: f.target.Host, Incarnation: f.target.Incarnation, Op: f.target.Op, Box: f.target.Box, Enrollment: f.target.Enrollment, BootID: f.peer.BootID, ControllerID: "lease-controller"}, Process: execlease.ControllerProcess{PID: f.peer.PID, Start: f.peer.ProcessStart, KernelBootID: f.peer.BootID}, State: state, LeaseETag: "lease-etag", Expires: time.Now().Add(time.Hour), SafeUntil: deadline} |
| ++ dir := filepath.Join(f.root.Guard.DataDir, ".hostexec") |
| ++ if err := os.MkdirAll(dir, 0700); err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ raw, err := json.Marshal(s) |
| ++ if err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ if err := os.WriteFile(filepath.Join(dir, f.target.Host+".json"), raw, 0600); err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++} |
| ++func rev2Operation() hostclaim.Op { |
| ++ return hostclaim.Op{Kind: hostclaim.KindRelocating, ID: "operation", Phase: "probing", Exec: "box-1", Journal: map[string]string{"install_stamp": "11"}} |
| ++} |
| ++ |
| ++func TestPlan11Rev2OperationVerifierAndLocalState(t *testing.T) { |
| ++ for _, name := range []string{"valid", "no verifier", "wrong attempt", "wrong input", "expired", "pending", "lost", "admission closed", "released live lock", "missing state"} { |
| ++ t.Run(name, func(t *testing.T) { |
| ++ f := newLaunchFixture(t, rev2Operation()) |
| ++ switch name { |
| ++ case "no verifier": |
| ++ f.root.Attempts = nil |
| ++ case "wrong attempt": |
| ++ f.root.Attempts[hostclaim.KindRelocating].(*rev2Attempts).attempt.InvocationID = "other" |
| ++ case "wrong input": |
| ++ f.req.Spec.Env = []string{"CARLOS_ADMIN_TOKEN=other"} |
| ++ case "expired": |
| ++ writeRev2Local(t, f, "active", 100) |
| ++ case "pending": |
| ++ writeRev2Local(t, f, "pending", 1000) |
| ++ case "lost": |
| ++ writeRev2Local(t, f, "lost", 1000) |
| ++ case "admission closed": |
| ++ f.root.Local.(*rev2Local).err = execlease.ErrLost |
| ++ case "released live lock": |
| ++ if err := os.Remove(filepath.Join(f.root.ExecLockDir, f.target.Host+".live")); err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ case "missing state": |
| ++ if err := os.Remove(filepath.Join(f.root.Guard.DataDir, ".hostexec", f.target.Host+".json")); err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ } |
| ++ _, err := f.root.Serve(t.Context(), f.peer, f.req) |
| ++ if name == "valid" { |
| ++ if err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ return |
| ++ } |
| ++ if err == nil { |
| ++ t.Fatal("unsafe operation launch") |
| ++ } |
| ++ f.noReservation(t, name) |
| ++ }) |
| ++ } |
| ++} |
| ++ |
| ++func TestPlan11Rev2RechecksAfterUnitPreparation(t *testing.T) { |
| ++ for _, name := range []string{"deadline", "quiet", "closed", "reservation", "roster"} { |
| ++ t.Run(name, func(t *testing.T) { |
| ++ f := newLaunchFixture(t, rev2Operation()) |
| ++ f.units.beforeCheck = func() { |
| ++ switch name { |
| ++ case "deadline": |
| ++ f.root.Clock.(*rev2Clock).boot = 1000 |
| ++ case "quiet": |
| ++ f.claims.set(func(c *hostclaim.Claim) { c.QuietAfter = map[string]time.Time{"eu": time.Now().Add(time.Hour)} }) |
| ++ case "closed": |
| ++ f.root.Local.(*rev2Local).err = execlease.ErrLost |
| ++ case "reservation": |
| ++ f.root.Attempts[hostclaim.KindRelocating].(*rev2Attempts).attempt.ID = "replaced" |
| ++ case "roster": |
| ++ f.actor.Eligible = false |
| ++ } |
| ++ } |
| ++ if _, err := f.root.Serve(t.Context(), f.peer, f.req); err == nil { |
| ++ t.Fatal("stale final check") |
| ++ } |
| ++ if starts, _ := f.units.counts(); starts != 0 { |
| ++ t.Fatal("tenant code started") |
| ++ } |
| ++ got, err := f.journal.UnitReservation(t.Context(), f.target, f.req.Reservation.ID) |
| ++ if err != nil || got.State != hostbox.UnitSettled { |
| ++ t.Fatalf("unstarted intention not settled: %+v %v", got, err) |
| ++ } |
| ++ }) |
| ++ } |
| ++} |
| ++ |
| ++func TestPlan11Rev2WaitsForSameInvocation(t *testing.T) { |
| ++ for _, name := range []string{"slow valid", "timeout", "replacement invocation", "reused pid", "wrong executable", "cancel"} { |
| ++ t.Run(name, func(t *testing.T) { |
| ++ f := newLaunchFixture(t, servingOp()) |
| ++ calls := 0 |
| ++ ctx, cancel := context.WithCancel(t.Context()) |
| ++ defer cancel() |
| ++ f.kernel.observe = func(instancectl.ClaimUnitStarted) (ClaimLaunchObserved, error) { |
| ++ calls++ |
| ++ got := f.kernel.observed |
| ++ if name == "replacement invocation" && calls == 1 { |
| ++ f.units.sameErr = instancectl.ErrClaimUnit |
| ++ } |
| ++ if name == "reused pid" && calls > 1 { |
| ++ got.ProcessStart = "reused" |
| ++ } |
| ++ if name == "cancel" { |
| ++ cancel() |
| ++ } |
| ++ if calls < 4 || name == "timeout" { |
| ++ return ClaimLaunchObserved{ProcessStart: got.ProcessStart, Group: got.Group}, errClaimLaunchPending |
| ++ } |
| ++ if name == "wrong executable" { |
| ++ got.Running.Inode++ |
| ++ } |
| ++ return got, nil |
| ++ } |
| ++ _, err := f.root.Serve(ctx, f.peer, f.req) |
| ++ starts, stops := f.units.counts() |
| ++ if starts != 1 || f.claims.obs.Claim.Holder.WakeRetry.AttemptsReserved != 1 { |
| ++ t.Fatalf("readiness consumed an attempt: %d", starts) |
| ++ } |
| ++ if name == "slow valid" { |
| ++ if err != nil || stops != 0 || calls != 4 { |
| ++ t.Fatalf("valid start killed: %v calls=%d stops=%d", err, calls, stops) |
| ++ } |
| ++ return |
| ++ } |
| ++ wantStops := 1 |
| ++ if name == "replacement invocation" { |
| ++ wantStops = 0 |
| ++ } |
| ++ if err == nil || stops != wantStops { |
| ++ t.Fatalf("unverified start granted or replacement stopped: %v stops=%d", err, stops) |
| ++ } |
| ++ if name == "timeout" && !errors.Is(err, context.DeadlineExceeded) { |
| ++ t.Fatal(err) |
| ++ } |
| ++ }) |
| ++ } |
| ++} |
| ++ |
| ++func TestPlan11Rev2DigestAndExecutable(t *testing.T) { |
| ++ f := newLaunchFixture(t, servingOp()) |
| ++ s := f.req.Spec |
| ++ want, err := ClaimLaunchDigest(ClaimLaunchInputs{Identity: s.Identity, Executable: s.Executable, ExecutableSHA256: s.Digest, Args: s.Args, Env: s.Env}) |
| ++ got, e := s.Hash() |
| ++ if err != nil || e != nil || got != want { |
| ++ t.Fatalf("digest contract drift: %q %q %v %v", got, want, err, e) |
| ++ } |
| ++ for _, name := range []string{"config", "release", "executable", "sha", "args", "env", "inode"} { |
| ++ t.Run(name, func(t *testing.T) { |
| ++ f := newLaunchFixture(t, servingOp()) |
| ++ s := *f.req.Spec |
| ++ f.req.Spec = &s |
| ++ switch name { |
| ++ case "config": |
| ++ s.Identity.Config = "other" |
| ++ case "release": |
| ++ s.Identity.Release = "other" |
| ++ case "executable": |
| ++ s.Executable = "/opt/other" |
| ++ case "sha": |
| ++ s.Digest = strings.Repeat("b", 64) |
| ++ case "args": |
| ++ s.Args = append([]string(nil), s.Args...) |
| ++ s.Args[1] += "x" |
| ++ case "env": |
| ++ s.Env = []string{"CARLOS_ADMIN_TOKEN=other"} |
| ++ case "inode": |
| ++ f.req.Reservation.Executable.Inode++ |
| ++ } |
| ++ if _, err := f.root.Serve(t.Context(), f.peer, f.req); err == nil { |
| ++ t.Fatal("input substitution launched") |
| ++ } |
| ++ f.noReservation(t, name) |
| ++ }) |
| ++ } |
| ++} |
| ++ |
| ++func TestPlan11Rev2KindPreflightRefusesUnverifiedOperation(t *testing.T) { |
| ++ f := newLaunchFixture(t, servingOp()) |
| ++ f.root.Authenticate = func(context.Context, net.Conn) (StopPeer, error) { return f.peer, nil } |
| ++ client := NewClaimRoleRootClient(serveClaimRoot(t, ClaimRootWithLaunch{Launch: f.root})) |
| ++ roles := &ClaimRoleAssembly{Root: client} |
| ++ for _, kind := range []string{hostclaim.KindLanding, hostclaim.KindRelocating, "unknown"} { |
| ++ if err := roles.PreflightKind(t.Context(), kind); err == nil { |
| ++ t.Fatalf("unverified %s passed prerequisites", kind) |
| ++ } |
| ++ } |
| ++ f.noReservation(t, "unsupported kinds") |
| ++ if err := roles.PreflightKind(t.Context(), hostclaim.KindServing); err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++} |
| +--- /dev/null |
| ++++ b/internal/host/claimlaunch_rev2_other_test.go |
| +@@ -0,0 +1,7 @@ |
| ++//go:build !linux |
| ++ |
| ++package host |
| ++ |
| ++import "testing" |
| ++ |
| ++func configureRev2Launch(t *testing.T, _ *launchFixture) { t.Skip("root launch requires Linux") } |
| +--- a/internal/host/claimlaunch_root_test.go |
| ++++ b/internal/host/claimlaunch_root_test.go |
| +@@ -46,11 +46,13 @@ |
| + } |
| + |
| + type launchUnits struct { |
| +- mu sync.Mutex |
| +- starts []instancectl.ClaimUnit |
| +- stops []string |
| +- startErr error |
| +- goneErr error |
| ++ mu sync.Mutex |
| ++ starts []instancectl.ClaimUnit |
| ++ stops []string |
| ++ startErr error |
| ++ beforeCheck func() |
| ++ sameErr error |
| ++ goneErr error |
| + // goneAfterStartErr answers Gone only once a start was attempted. |
| + goneAfterStartErr error |
| + stopErr error |
| +@@ -95,6 +97,8 @@ |
| + verifyErr error |
| + observed ClaimLaunchObserved |
| + observeErr error |
| ++ observe func(instancectl.ClaimUnitStarted) (ClaimLaunchObserved, error) |
| ++ digest string |
| + } |
| + |
| + func (k *launchKernel) Verify(path string, want hostbox.LaunchFile) error { |
| +@@ -108,6 +112,9 @@ |
| + } |
| + |
| + func (k *launchKernel) Observe(instancectl.ClaimUnitStarted) (ClaimLaunchObserved, error) { |
| ++ if k.observe != nil { |
| ++ return k.observe(instancectl.ClaimUnitStarted{}) |
| ++ } |
| + return k.observed, k.observeErr |
| + } |
| + |
| +@@ -153,7 +160,9 @@ |
| + root := &ClaimLaunchRoot{Guard: guard, Journal: journal, Units: units, Kernel: kernel, Actor: func(context.Context) (hostbox.Actor, error) { return *actor, nil }} |
| + reservation := hostbox.LaunchReservation{ID: "attempt-1:main", Role: "main", InvocationID: "inv-1", Inputs: "inputs-1", Target: target, Group: hostbox.LaunchGroup{RelativePath: instancectl.ClaimUnitControlGroup(host)}, Executable: exe} |
| + req := hostbox.RoleLaunchRequest{V: hostbox.RoleLaunchV, Op: hostbox.RoleLaunchStart, Reservation: &reservation, Executable: kernel.path, MemoryMax: cgroup.TierDefault.MemoryMax} |
| +- return &launchFixture{root: root, claims: claims, units: units, kernel: kernel, peer: peer, req: req, journal: journal, target: target, actor: actor} |
| ++ f := &launchFixture{root: root, claims: claims, units: units, kernel: kernel, peer: peer, req: req, journal: journal, target: target, actor: actor} |
| ++ configureRev2Launch(t, f) |
| ++ return f |
| + } |
| + |
| + func servingOp() hostclaim.Op { return hostclaim.Op{Kind: hostclaim.KindServing, Exec: "box-1"} } |
| +@@ -386,11 +395,14 @@ |
| + t.Fatalf("capability=%+v %v", resp, err) |
| + } |
| + for name, strip := range map[string]func(*ClaimLaunchRoot){ |
| +- "units": func(r *ClaimLaunchRoot) { r.Units = nil }, |
| +- "kernel": func(r *ClaimLaunchRoot) { r.Kernel = nil }, |
| +- "journal": func(r *ClaimLaunchRoot) { r.Journal = nil }, |
| +- "actor": func(r *ClaimLaunchRoot) { r.Actor = nil }, |
| +- "guard": func(r *ClaimLaunchRoot) { r.Guard.Claims = nil }, |
| ++ "local": func(r *ClaimLaunchRoot) { r.Local = nil }, |
| ++ "clock": func(r *ClaimLaunchRoot) { r.Clock = nil }, |
| ++ "data root": func(r *ClaimLaunchRoot) { r.DataRoot = "" }, |
| ++ "units": func(r *ClaimLaunchRoot) { r.Units = nil }, |
| ++ "kernel": func(r *ClaimLaunchRoot) { r.Kernel = nil }, |
| ++ "journal": func(r *ClaimLaunchRoot) { r.Journal = nil }, |
| ++ "actor": func(r *ClaimLaunchRoot) { r.Actor = nil }, |
| ++ "guard": func(r *ClaimLaunchRoot) { r.Guard.Claims = nil }, |
| + } { |
| + g := newLaunchFixture(t, servingOp()) |
| + strip(g.root) |
| +@@ -466,3 +478,20 @@ |
| + } |
| + |
| + var _ = strconv.Itoa |
| ++ |
| ++func (u *launchUnits) StartChecked(ctx context.Context, in instancectl.ClaimUnit, _ instancectl.Launch, check func() error) (instancectl.ClaimUnitStarted, error) { |
| ++ if u.beforeCheck != nil { |
| ++ u.beforeCheck() |
| ++ } |
| ++ if err := check(); err != nil { |
| ++ return instancectl.ClaimUnitStarted{}, err |
| ++ } |
| ++ return u.Start(ctx, in) |
| ++} |
| ++func (u *launchUnits) Same(context.Context, instancectl.ClaimUnitStarted) error { return u.sameErr } |
| ++func (k *launchKernel) VerifyDigest(_ context.Context, path string, want hostbox.LaunchFile, digest string) error { |
| ++ if digest != k.digest { |
| ++ return hostbox.ErrChanged |
| ++ } |
| ++ return k.Verify(path, want) |
| ++} |
| +--- a/internal/host/claimlaunch_transport_test.go |
| ++++ b/internal/host/claimlaunch_transport_test.go |
| +@@ -122,6 +122,7 @@ |
| + } |
| + f.kernel.path, f.kernel.want = exe, hostbox.LaunchFile{Device: uint64(st.Dev), Inode: st.Ino} |
| + f.kernel.observed.Executable = f.kernel.want |
| ++ f.kernel.observed.Running = f.kernel.want |
| + f.kernel.observed.ProcessStart = procSelfStart(t) |
| + f.units.started.MainPID = os.Getpid() |
| + sock := serveClaimRoot(t, ClaimRootWithLaunch{Launch: f.root}) |
| +@@ -129,15 +130,21 @@ |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| +- launchDir, runDir := filepath.Join(base, "data"), filepath.Join(base, "run") |
| ++ f.root.DataRoot, f.root.runRoot = base, filepath.Join(base, "run") |
| ++ launchDir, runDir := provision.InstanceDir(base, f.target.Host), filepath.Join(f.root.runRoot, provision.InstanceID(f.target.Host)) |
| + if err := os.MkdirAll(launchDir, 0o700); err != nil { |
| + t.Fatal(err) |
| + } |
| + launch := activator.RoleLaunch{Work: work, Target: f.target, Root: NewClaimRoleRootClient(sock), LaunchDir: launchDir, RunDir: runDir, LogSocket: filepath.Join(base, "l.sock"), MemoryMax: cgroup.TierDefault.MemoryMax, Executable: func(context.Context, *exec.Cmd) (*os.File, error) { return os.Open(exe) }} |
| + attempt := hostclaim.WakeRetryAttempt{ID: "attempt-1", InvocationID: "inv-1", LaunchInputs: "inputs-1", Session: 13, Box: "box-1", Enrollment: "enrollment-1", BootID: "boot-1", ControllerID: "controller-1"} |
| + build := func() *exec.Cmd { |
| +- return exec.Command("/opt/carlos/live/shop.example", "--socket", filepath.Join(runDir, "app.sock"), "--db", filepath.Join(launchDir, "app.db")) |
| +- } |
| ++ cmd := exec.Command("/opt/carlos/live/shop.example", "--socket", filepath.Join(runDir, "app.sock"), "--db", filepath.Join(launchDir, "app.db")) |
| ++ cmd.Env = []string{"CARLOS_ADMIN_TOKEN=test"} |
| ++ return cmd |
| ++ } |
| ++ launch.Spec = &hostbox.LaunchSpec{Identity: f.req.Spec.Identity, Executable: exe, Digest: f.kernel.digest, Args: build().Args[1:], Env: build().Env} |
| ++ attempt.LaunchInputs, _ = launch.Spec.Hash() |
| ++ f.claims.set(func(c *hostclaim.Claim) { a := attempt; c.Holder.WakeRetry.Attempt = &a }) |
| + cmd, _, settle, _, err := launch.Start(t.Context(), "main", attempt, build, func() error { return nil }) |
| + if err != nil { |
| + t.Fatal(err) |
| +@@ -146,7 +153,7 @@ |
| + t.Fatalf("pid=%d", cmd.Process.Pid) |
| + } |
| + got, err := f.journal.UnitReservation(t.Context(), f.target, "attempt-1:main") |
| +- if err != nil || got.State != hostbox.UnitStarted || got.InvocationID != "inv-1" || got.Inputs != "inputs-1" { |
| ++ if err != nil || got.State != hostbox.UnitStarted || got.InvocationID != "inv-1" || got.Inputs != attempt.LaunchInputs { |
| + t.Fatalf("root journal=%+v %v", got, err) |
| + } |
| + if err := settle(t.Context()); err != nil { |
| +@@ -246,10 +253,11 @@ |
| + t.Fatalf("launch root=%+v", l) |
| + } |
| + for name, bad := range map[string]func(*ClaimRootHTTP, *map[string]string){ |
| +- "no journal": func(r *ClaimRootHTTP, _ *map[string]string) { r.Launches = nil }, |
| +- "no authenticate": func(r *ClaimRootHTTP, _ *map[string]string) { r.Stop.Authenticate = nil }, |
| +- "no stop kernel": func(r *ClaimRootHTTP, _ *map[string]string) { r.Stop.Coordinator.Kernel = nil }, |
| +- "no opt root": func(_ *ClaimRootHTTP, m *map[string]string) { *m = map[string]string{"data": "/data"} }, |
| ++ "no journal": func(r *ClaimRootHTTP, _ *map[string]string) { r.Launches = nil }, |
| ++ "no authenticate": func(r *ClaimRootHTTP, _ *map[string]string) { r.Stop.Authenticate = nil }, |
| ++ "no stop kernel": func(r *ClaimRootHTTP, _ *map[string]string) { r.Stop.Coordinator.Kernel = nil }, |
| ++ "unbound data root": func(_ *ClaimRootHTTP, m *map[string]string) { (*m)["data"] = "/elsewhere" }, |
| ++ "no opt root": func(_ *ClaimRootHTTP, m *map[string]string) { *m = map[string]string{"data": "/data"} }, |
| + } { |
| + r := root |
| + m := map[string]string{"opt": "/opt/carlos", "data": "/data"} |
| +@@ -267,3 +275,10 @@ |
| + } |
| + |
| + func iidOf(host string) string { return provision.InstanceID(host) } |
| ++ |
| ++func (f fakeLaunchRoot) KindCapability(context.Context, string) error { |
| ++ if f.protocol != hostbox.RoleLaunchProtocol { |
| ++ return activator.ErrClaimIsolationUnavailable |
| ++ } |
| ++ return f.err |
| ++} |
| +--- /dev/null |
| ++++ b/internal/host/claimshape_rev2_shape_test.go |
| +@@ -0,0 +1,207 @@ |
| ++//go:build linux |
| ++ |
| ++package host |
| ++ |
| ++import ( |
| ++ "context" |
| ++ "errors" |
| ++ "github.com/carlosframework/platform/internal/hostwrite" |
| ++ "os" |
| ++ "path/filepath" |
| ++ "reflect" |
| ++ "strings" |
| ++ "testing" |
| ++ |
| ++ "github.com/carlosframework/platform/internal/activator" |
| ++ "github.com/carlosframework/platform/internal/cgroup" |
| ++ "github.com/carlosframework/platform/internal/hostbox" |
| ++ "github.com/carlosframework/platform/internal/hostclaim" |
| ++ "github.com/carlosframework/platform/internal/provision" |
| ++ "github.com/carlosframework/platform/internal/registry" |
| ++) |
| ++ |
| ++const shapeData = "/data" |
| ++ |
| ++func shapeAssembly() *ClaimRoleAssembly { |
| ++ return &ClaimRoleAssembly{Root: fakeLaunchRoot{protocol: hostbox.RoleLaunchProtocol}, ManagedRoots: map[string]string{"data": shapeData}} |
| ++} |
| ++ |
| ++func shapeInputs(host string) ClaimLaunchInputs { |
| ++ db := filepath.Join(provision.InstanceDir(shapeData, host), host+".db") |
| ++ return ClaimLaunchInputs{Executable: "/opt/carlos/versions/v1/app", ExecutableSHA256: strings.Repeat("a", 64), Identity: hostwrite.Identity{Config: "config", Release: "release"}, Roles: []string{"main"}, DBPath: db, Limit: cgroup.TierDefault, |
| ++ Args: []string{"--socket", provision.NestedSocket(host), "--db", db}, |
| ++ Env: []string{"CARLOS_ADMIN_TOKEN=secret", "PATH=/usr/bin"}} |
| ++} |
| ++ |
| ++func TestRev2ShapePreflightLaunchAdmitsValidMain(t *testing.T) { |
| ++ if err := shapeAssembly().PreflightLaunch("shop.example", shapeInputs("shop.example")); err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++} |
| ++ |
| ++func TestRev2ShapePreflightLaunchRefusesMalformedLaunch(t *testing.T) { |
| ++ host := "shop.example" |
| ++ good := shapeInputs(host) |
| ++ db, sock := good.DBPath, provision.NestedSocket(host) |
| ++ other := filepath.Join(provision.InstanceDir(shapeData, host), "other.db") |
| ++ for name, mutate := range map[string]func(*ClaimLaunchInputs){ |
| ++ "seed role": func(in *ClaimLaunchInputs) { in.Roles = []string{"main", "seed"} }, |
| ++ "sidecar role": func(in *ClaimLaunchInputs) { in.Roles = []string{"main", "sidecar"} }, |
| ++ "no roles": func(in *ClaimLaunchInputs) { in.Roles = nil }, |
| ++ "no args": func(in *ClaimLaunchInputs) { in.Args = nil }, |
| ++ "extra flag": func(in *ClaimLaunchInputs) { in.Args = append(in.Args, "--verbose", "1") }, |
| ++ "extra positional": func(in *ClaimLaunchInputs) { in.Args = append(in.Args, "serve") }, |
| ++ "duplicate socket": func(in *ClaimLaunchInputs) { in.Args = append(in.Args, "--socket", sock) }, |
| ++ "duplicate db": func(in *ClaimLaunchInputs) { in.Args = append(in.Args, "--db", db) }, |
| ++ "missing socket": func(in *ClaimLaunchInputs) { in.Args = []string{"--db", db} }, |
| ++ "missing db": func(in *ClaimLaunchInputs) { in.Args = []string{"--socket", sock} }, |
| ++ "NUL in arg": func(in *ClaimLaunchInputs) { in.Args = []string{"--socket", sock + "\x00x", "--db", db} }, |
| ++ "db mismatch": func(in *ClaimLaunchInputs) { in.Args = []string{"--socket", sock, "--db", other} }, |
| ++ "db flat": func(in *ClaimLaunchInputs) { in.Args = []string{"--socket", sock, "--db", "/data/shop.example.db"} }, |
| ++ "socket flat": func(in *ClaimLaunchInputs) { |
| ++ in.Args = []string{"--socket", provision.SocketDir + "/shop.example.sock", "--db", db} |
| ++ }, |
| ++ "socket other unit": func(in *ClaimLaunchInputs) { |
| ++ in.Args = []string{"--socket", provision.NestedSocket("other.example"), "--db", db} |
| ++ }, |
| ++ "socket not app": func(in *ClaimLaunchInputs) { |
| ++ in.Args = []string{"--socket", filepath.Join(filepath.Dir(sock), "x.sock"), "--db", db} |
| ++ }, |
| ++ "no env": func(in *ClaimLaunchInputs) { in.Env = nil }, |
| ++ "no admin token": func(in *ClaimLaunchInputs) { in.Env = []string{"PATH=/usr/bin"} }, |
| ++ "empty admin token": func(in *ClaimLaunchInputs) { in.Env = []string{"CARLOS_ADMIN_TOKEN=", "PATH=/usr/bin"} }, |
| ++ "malformed env": func(in *ClaimLaunchInputs) { in.Env = append(in.Env, "NOEQUALS") }, |
| ++ "empty env key": func(in *ClaimLaunchInputs) { in.Env = append(in.Env, "=v") }, |
| ++ "duplicate env": func(in *ClaimLaunchInputs) { in.Env = append(in.Env, "PATH=/bin") }, |
| ++ "NUL in env": func(in *ClaimLaunchInputs) { in.Env = append(in.Env, "A=b\x00c") }, |
| ++ "bad tier": func(in *ClaimLaunchInputs) { in.Limit = cgroup.Limit{MemoryMax: 7} }, |
| ++ } { |
| ++ t.Run(name, func(t *testing.T) { |
| ++ in := good |
| ++ in.Args, in.Env = append([]string(nil), good.Args...), append([]string(nil), good.Env...) |
| ++ mutate(&in) |
| ++ err := shapeAssembly().PreflightLaunch(host, in) |
| ++ if !errors.Is(err, hostbox.ErrRoleLaunchShape) && !errors.Is(err, hostbox.ErrUnsupported) { |
| ++ t.Fatalf("accepted: %v", err) |
| ++ } |
| ++ }) |
| ++ } |
| ++} |
| ++ |
| ++// Select reports the roles the real start would spawn, not a hardcoded main. |
| ++func TestRev2ShapeServingSelectReportsLaunchRoles(t *testing.T) { |
| ++ for name, tc := range map[string]struct { |
| ++ roles []string |
| ++ want []string |
| ++ }{ |
| ++ "main": {[]string{"main"}, []string{"main"}}, |
| ++ "seed": {[]string{"main", "seed"}, []string{"main", "seed"}}, |
| ++ "sidecar": {[]string{"main", "sidecar"}, []string{"main", "sidecar"}}, |
| ++ } { |
| ++ t.Run(name, func(t *testing.T) { |
| ++ f := plan9D5Fixture(t) |
| ++ f.launch.Roles = tc.roles |
| ++ in, err := f.owners.Select(t.Context(), plan9D5Route, plan9D5Observed(hostclaim.Holder{Stamp: 3}), activator.StartPolicy{MaxStarts: 5}) |
| ++ if err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ if !reflect.DeepEqual(in.Roles, tc.want) { |
| ++ t.Fatalf("roles=%v want %v", in.Roles, tc.want) |
| ++ } |
| ++ }) |
| ++ } |
| ++} |
| ++ |
| ++// The production launch source refuses a route that would spawn a seed or a |
| ++// co-spawned sidecar before the admin token is minted. |
| ++func TestRev2ShapeLaunchSourceRefusesSeedAndSidecarBeforeTokenMint(t *testing.T) { |
| ++ for name, setup := range map[string]func(*adminTokenFixture){ |
| ++ "seed": func(f *adminTokenFixture) { f.rt.Seed = `["init"]` }, |
| ++ "sidecar": func(f *adminTokenFixture) { |
| ++ if err := os.WriteFile(filepath.Join(f.data, adminTokenHost+"-sidecar.env"), []byte("A=1\n"), 0o600); err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ }, |
| ++ } { |
| ++ t.Run(name, func(t *testing.T) { |
| ++ f := newAdminTokenFixture(t, "box-1") |
| ++ setup(f) |
| ++ _, err := claimServingLaunchSource(f.backend, f.tokens)(t.Context(), f.rt) |
| ++ if !errors.Is(err, hostbox.ErrUnsupported) { |
| ++ t.Fatalf("err=%v", err) |
| ++ } |
| ++ if _, err := os.Stat(f.envPath()); !errors.Is(err, os.ErrNotExist) { |
| ++ t.Fatalf("token minted for refused shape: %v", err) |
| ++ } |
| ++ }) |
| ++ } |
| ++} |
| ++ |
| ++func TestRev2ShapeLaunchSourceReportsMainOnly(t *testing.T) { |
| ++ f := newAdminTokenFixture(t, "box-1") |
| ++ l, err := claimServingLaunchSource(f.backend, f.tokens)(t.Context(), f.rt) |
| ++ if err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ if !reflect.DeepEqual(l.Roles, []string{"main"}) { |
| ++ t.Fatalf("roles=%v", l.Roles) |
| ++ } |
| ++} |
| ++ |
| ++type shapeMutatedInputs struct { |
| ++ recoveryInputs |
| ++ mutate func(*ClaimLaunchInputs) |
| ++} |
| ++ |
| ++func (s shapeMutatedInputs) Select(ctx context.Context, rt registry.Route, o hostclaim.Observed, p activator.StartPolicy) (ClaimLaunchInputs, error) { |
| ++ in, err := s.recoveryInputs.Select(ctx, rt, o, p) |
| ++ s.mutate(&in) |
| ++ return in, err |
| ++} |
| ++ |
| ++// A malformed launch is refused in Prerequisites before any retry reservation, |
| ++// guard record, stop request, lease or journal entry. |
| ++func TestRev2ShapeMalformedLaunchRefusedBeforeStateChanges(t *testing.T) { |
| ++ for name, mutate := range map[string]func(*ClaimLaunchInputs){ |
| ++ "extra arg": func(in *ClaimLaunchInputs) { in.Args = append(in.Args, "--x", "y") }, |
| ++ "duplicate flag": func(in *ClaimLaunchInputs) { in.Args = append(in.Args, "--db", in.DBPath) }, |
| ++ "db mismatch": func(in *ClaimLaunchInputs) { in.Args[3] = in.DBPath + "-other" }, |
| ++ "bad socket": func(in *ClaimLaunchInputs) { in.Args[1] = "/tmp/app.sock" }, |
| ++ "no env": func(in *ClaimLaunchInputs) { in.Env = nil }, |
| ++ "seed role": func(in *ClaimLaunchInputs) { in.Roles = []string{"main", "seed"} }, |
| ++ } { |
| ++ t.Run(name, func(t *testing.T) { |
| ++ f := newRecoveryFixture(t) |
| ++ c := f.controller("boot-1", "controller-1", 4101, "41") |
| ++ c.wake.Roles = &ClaimRoleAssembly{Root: fakeLaunchRoot{protocol: hostbox.RoleLaunchProtocol}, ManagedRoots: map[string]string{"data": shapeData}, Manager: c.wake.Manager, Journal: f.journal} |
| ++ route := registry.Route{Host: recoveryHost, Kind: "instance", Hibernate: true, |
| ++ DBPath: provision.NestedDBPath(f.guard.DataDir, recoveryHost), Socket: provision.NestedSocket(recoveryHost)} |
| ++ c.wake.Inputs = shapeMutatedInputs{recoveryInputs{f}, func(in *ClaimLaunchInputs) { |
| ++ base := shapeInputs(recoveryHost) |
| ++ base.DBPath = provision.NestedDBPath(shapeData, recoveryHost) |
| ++ base.Args = []string{"--socket", provision.NestedSocket(recoveryHost), "--db", base.DBPath} |
| ++ in.DBPath, in.Args, in.Env, in.Limit = base.DBPath, base.Args, base.Env, cgroup.TierDefault |
| ++ mutate(in) |
| ++ }} |
| ++ before := f.read() |
| ++ guardBefore, err := os.ReadDir(filepath.Join(f.guard.DataDir, "hostguard")) |
| ++ if err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ err = c.runtime.Prerequisites(t.Context(), route, before, recoveryPolicy) |
| ++ if !errors.Is(err, hostbox.ErrRoleLaunchShape) && !errors.Is(err, hostbox.ErrUnsupported) { |
| ++ t.Fatalf("accepted: %v", err) |
| ++ } |
| ++ if after := f.read(); after.ETag != before.ETag || after.Claim.Holder.WakeRetry != nil { |
| ++ t.Fatalf("refusal changed claim or spent retry: %+v", after) |
| ++ } |
| ++ guardAfter, err := os.ReadDir(filepath.Join(f.guard.DataDir, "hostguard")) |
| ++ if err != nil || !reflect.DeepEqual(guardBefore, guardAfter) || len(c.wake.pending) != 0 || len(f.requests) != 0 { |
| ++ t.Fatalf("refusal reached guard/stop/lease: err=%v pending=%d requests=%d", err, len(c.wake.pending), len(f.requests)) |
| ++ } |
| ++ if _, err := f.journal.Read(t.Context(), recoveryTuple(before, c.local)); err == nil { |
| ++ t.Fatal("refusal journaled a launch") |
| ++ } |
| ++ }) |
| ++ } |
| ++} |
| +--- a/internal/host/plan11_launch_retry_linux_test.go |
| ++++ b/internal/host/plan11_launch_retry_linux_test.go |
| +@@ -16,6 +16,7 @@ |
| + "github.com/carlosframework/platform/internal/hostclaim" |
| + "github.com/carlosframework/platform/internal/hostwrite" |
| + "github.com/carlosframework/platform/internal/instancectl" |
| ++ "github.com/carlosframework/platform/internal/provision" |
| + "golang.org/x/sys/unix" |
| + ) |
| + |
| +@@ -54,6 +55,7 @@ |
| + f.kernel.path = exe |
| + f.kernel.want = hostbox.LaunchFile{Device: uint64(st.Dev), Inode: st.Ino} |
| + f.kernel.observed.Executable = f.kernel.want |
| ++ f.kernel.observed.Running = f.kernel.want |
| + f.units.started.MainPID = os.Getpid() |
| + f.kernel.observed.ProcessStart, err = claimConstructionProcessStart() |
| + if err != nil { |
| +@@ -67,16 +69,26 @@ |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| +- launch := activator.RoleLaunch{Root: plan11LaunchClient{f}, Target: f.target, Journal: f.journal, Work: work, LaunchDir: dir, RunDir: filepath.Join(dir, "run"), LogSocket: filepath.Join(dir, "log.sock"), MemoryMax: cgroup.TierDefault.MemoryMax, ParentPath: "carlos/claim-parent"} |
| ++ f.root.DataRoot, f.root.runRoot = dir, filepath.Join(dir, "run") |
| ++ launchDir := provision.InstanceDir(dir, f.target.Host) |
| ++ if err := os.MkdirAll(launchDir, 0700); err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ launch := activator.RoleLaunch{Root: plan11LaunchClient{f}, Target: f.target, Journal: f.journal, Work: work, LaunchDir: launchDir, RunDir: filepath.Join(f.root.runRoot, provision.InstanceID(f.target.Host)), LogSocket: filepath.Join(dir, "log.sock"), MemoryMax: cgroup.TierDefault.MemoryMax, ParentPath: "carlos/claim-parent"} |
| + launch.Executable = func(context.Context, *exec.Cmd) (*os.File, error) { return os.Open(exe) } |
| + launch.RegisterProducer = func(ctx context.Context, role string) (activator.RoleProducer, error) { |
| + ctx, finish, err := producers.Register(ctx, f.target, role) |
| + return activator.RoleProducer{Context: ctx, Finish: finish}, err |
| + } |
| + build := func() *exec.Cmd { |
| +- return exec.Command(exe, "--socket", filepath.Join(launch.RunDir, "app.sock"), "--db", filepath.Join(dir, "data.db")) |
| ++ cmd := exec.Command(exe, "--socket", filepath.Join(launch.RunDir, "app.sock"), "--db", filepath.Join(launchDir, "data.db")) |
| ++ cmd.Env = []string{"CARLOS_ADMIN_TOKEN=test"} |
| ++ return cmd |
| + } |
| + attempt := hostclaim.WakeRetryAttempt{ID: "first", InvocationID: "inv-first", LaunchInputs: "inputs", Session: f.target.Session, Box: f.target.Box, Enrollment: f.target.Enrollment, BootID: f.target.BootID, ControllerID: f.target.ControllerID} |
| ++ launch.Spec = &hostbox.LaunchSpec{Identity: f.req.Spec.Identity, Executable: exe, Digest: f.kernel.digest, Args: build().Args[1:], Env: build().Env} |
| ++ attempt.LaunchInputs, _ = launch.Spec.Hash() |
| ++ f.claims.set(func(c *hostclaim.Claim) { a := attempt; c.Holder.WakeRetry.Attempt = &a }) |
| + f.units.startErr = errors.New("first unit start failed") |
| + if _, _, _, _, err := launch.Start(t.Context(), "main", attempt, build, func() error { return nil }); err == nil { |
| + t.Fatal("failed start reported success") |
| +@@ -84,6 +96,11 @@ |
| + f.units.startErr = nil |
| + attempt.ID = "second" |
| + attempt.InvocationID = "inv-second" |
| ++ f.claims.set(func(c *hostclaim.Claim) { |
| ++ a := attempt |
| ++ c.Holder.WakeRetry.Attempt = &a |
| ++ c.Holder.WakeRetry.AttemptsReserved = 2 |
| ++ }) |
| + _, _, settle, producer, err := launch.Start(t.Context(), "main", attempt, build, func() error { return nil }) |
| + if err != nil { |
| + t.Fatalf("retry after settled failure: %v", err) |
| +@@ -141,6 +158,14 @@ |
| + } |
| + q := f.req |
| + q.Reservation = &next |
| ++ f.claims.set(func(c *hostclaim.Claim) { |
| ++ a := *c.Holder.WakeRetry.Attempt |
| ++ a.ID = "next" |
| ++ a.InvocationID = next.InvocationID |
| ++ a.ControllerID = next.Target.ControllerID |
| ++ c.Holder.WakeRetry.Attempt = &a |
| ++ c.Holder.WakeRetry.AttemptsReserved = 2 |
| ++ }) |
| + if _, err := f.root.Serve(t.Context(), nextPeer, q); err != nil { |
| + t.Fatal(err) |
| + } |
| +--- /dev/null |
| ++++ b/internal/instancectl/claimlaunch_rev2_test.go |
| +@@ -0,0 +1,92 @@ |
| ++package instancectl |
| ++ |
| ++import ( |
| ++ "context" |
| ++ "encoding/json" |
| ++ "errors" |
| ++ "os" |
| ++ "path/filepath" |
| ++ "strings" |
| ++ "testing" |
| ++) |
| ++ |
| ++func TestPlan11Rev2UnitFinalCheckAndSnapshot(t *testing.T) { |
| ++ for _, deny := range []bool{false, true} { |
| ++ t.Run(map[bool]string{true: "denied", false: "allowed"}[deny], func(t *testing.T) { |
| ++ u, ctl, in, _ := claimUnitFixture(t) |
| ++ u.UnitRoot, u.DataRoot = t.TempDir(), t.TempDir() |
| ++ if err := os.Chmod(u.UnitRoot, 0700); err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ launch := Launch{Schema: LaunchSchema, Host: in.Host, Argv: []string{LiveDir + "/" + in.Host, "--socket", "/run/app.sock", "--db", "/data/app.db"}, Env: []string{"TOKEN=secret"}} |
| ++ checks := 0 |
| ++ _, err := u.StartChecked(t.Context(), in, launch, func() error { |
| ++ checks++ |
| ++ source, dropin, e := u.launchPaths(in.Host) |
| ++ if e != nil { |
| ++ t.Fatal(e) |
| ++ } |
| ++ body, e := os.ReadFile(filepath.Join(source, "launch.json")) |
| ++ if e != nil { |
| ++ t.Fatal(e) |
| ++ } |
| ++ var got Launch |
| ++ if e = json.Unmarshal(body, &got); e != nil || strings.Join(got.Env, ",") != "TOKEN=secret" { |
| ++ t.Fatalf("snapshot %s %v", body, e) |
| ++ } |
| ++ body, e = os.ReadFile(filepath.Join(dropin, "60-claim-launch.conf")) |
| ++ if e != nil || !strings.Contains(string(body), "BindReadOnlyPaths="+filepath.Join(source, "launch.json")) { |
| ++ t.Fatalf("binding: %s %v", body, e) |
| ++ } |
| ++ if strings.Contains(ctl.joined(), "start ") { |
| ++ t.Fatal("check ran after start") |
| ++ } |
| ++ if deny { |
| ++ return errors.New("expired") |
| ++ } |
| ++ return nil |
| ++ }) |
| ++ if checks != 1 { |
| ++ t.Fatalf("checks=%d err=%v", checks, err) |
| ++ } |
| ++ if deny { |
| ++ if err == nil || strings.Contains(ctl.joined(), "start ") { |
| ++ t.Fatal("denial launched") |
| ++ } |
| ++ return |
| ++ } |
| ++ if err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ if err = u.Stop(context.Background(), in.Host); err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ source, dropin, _ := u.launchPaths(in.Host) |
| ++ for _, p := range []string{filepath.Join(source, "launch.json"), filepath.Join(dropin, "60-claim-launch.conf")} { |
| ++ if _, e := os.Stat(p); !errors.Is(e, os.ErrNotExist) { |
| ++ t.Fatalf("snapshot outlived stopped unit: %s %v", p, e) |
| ++ } |
| ++ } |
| ++ }) |
| ++ } |
| ++} |
| ++ |
| ++func TestPlan11Rev2UnitSameInvocation(t *testing.T) { |
| ++ u, ctl, in, _ := claimUnitFixture(t) |
| ++ s, err := u.Start(t.Context(), in) |
| ++ if err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ if err = u.Same(t.Context(), s); err != nil { |
| ++ t.Fatal(err) |
| ++ } |
| ++ ctl.invocation = "replacement" |
| ++ if err = u.Same(t.Context(), s); err == nil { |
| ++ t.Fatal("replacement accepted") |
| ++ } |
| ++ ctl.invocation = s.InvocationID |
| ++ ctl.pid++ |
| ++ if err = u.Same(t.Context(), s); err == nil { |
| ++ t.Fatal("replacement pid accepted") |
| ++ } |
| ++} |
| +``` |
| + |
| +```bash |
| +python3 "$CLAIM11_SCRATCH/apply.py" "$CLAIM11_PLAN" D2 tests |
| +nice -n 10 go test -p 1 -timeout 5m ./internal/host ./internal/activator ./internal/instancectl -run 'TestPlan11Rev2|TestRev2Shape' -count=1 |
| +``` |
| + |
| +Expected red: the added fixtures and tests require the new root providers, input preimage and unit checks. The behavioral red run above establishes the defects independently of these missing interfaces. |
| + |
| +### Implementation |
| + |
| +<!-- apply: D2 implementation --> |
| +```diff |
| +--- /dev/null |
| ++++ b/internal/activator/claim_shape.go |
| +@@ -0,0 +1,86 @@ |
| ++package activator |
| ++ |
| ++import ( |
| ++ "path/filepath" |
| ++ "strings" |
| ++ |
| ++ "github.com/carlosframework/platform/internal/hostbox" |
| ++ "github.com/carlosframework/platform/internal/instancectl" |
| ++ "github.com/carlosframework/platform/internal/registry" |
| ++) |
| ++ |
| ++// ClaimLaunchRoles are the processes the real start of rt spawns: main |
| ++// always, the route's first-boot seed when it has one, and the legacy |
| ++// co-spawned sidecar when its env file is staged. The isolated claim launch |
| ++// carries main alone. |
| ++func (b *ExecBackend) ClaimLaunchRoles(rt registry.Route) []string { |
| ++ roles := []string{"main"} |
| ++ if rt.Seed != "" { |
| ++ roles = append(roles, "seed") |
| ++ } |
| ++ if b.HasSidecar(rt) { |
| ++ roles = append(roles, "sidecar") |
| ++ } |
| ++ return roles |
| ++} |
| ++ |
| ++// claimLaunchShape refuses, read-only, a route whose real start is more than |
| ++// the isolated main launch. |
| ++func (b *ExecBackend) claimLaunchShape(rt registry.Route) error { |
| ++ if roles := b.ClaimLaunchRoles(rt); len(roles) != 1 { |
| ++ return hostbox.ErrUnsupported |
| ++ } |
| ++ return nil |
| ++} |
| ++ |
| ++func isolatedCleanAbs(p string) bool { |
| ++ return filepath.IsAbs(p) && filepath.Clean(p) == p && !strings.ContainsRune(p, 0) |
| ++} |
| ++ |
| ++// IsolatedArgv validates the complete isolated launch shape and returns the |
| ++// trampoline's argv: the unit's view of the host's live build, then exactly |
| ++// --socket in the run directory and --db in the instance directory, each |
| ++// once. args excludes argv[0]. env must be well-formed (KEY=value, no NUL, |
| ++// no duplicate key) and carry the instance's admin token. It is the one |
| ++// validator for startIsolated and the claim preflight. |
| ++func IsolatedArgv(host, launchDir, runDir string, args, env []string) (argv []string, socket, db string, err error) { |
| ++ if len(args)%2 != 0 || !isolatedCleanAbs(launchDir) || !isolatedCleanAbs(runDir) { |
| ++ return nil, "", "", hostbox.ErrRoleLaunchShape |
| ++ } |
| ++ for i := 0; i < len(args); i += 2 { |
| ++ value := args[i+1] |
| ++ switch { |
| ++ case args[i] == "--socket" && socket == "" && isolatedCleanAbs(value) && filepath.Dir(value) == runDir: |
| ++ socket = value |
| ++ case args[i] == "--db" && db == "" && isolatedCleanAbs(value) && filepath.Dir(value) == launchDir: |
| ++ db = value |
| ++ default: |
| ++ return nil, "", "", hostbox.ErrRoleLaunchShape |
| ++ } |
| ++ } |
| ++ if socket == "" || db == "" { |
| ++ return nil, "", "", hostbox.ErrRoleLaunchShape |
| ++ } |
| ++ seen := make(map[string]bool, len(env)) |
| ++ for _, kv := range env { |
| ++ k, _, ok := strings.Cut(kv, "=") |
| ++ if !ok || k == "" || strings.ContainsRune(kv, 0) || seen[k] { |
| ++ return nil, "", "", hostbox.ErrRoleLaunchShape |
| ++ } |
| ++ seen[k] = true |
| ++ } |
| ++ if !seen["CARLOS_ADMIN_TOKEN"] || !envHas(env, "CARLOS_ADMIN_TOKEN=") { |
| ++ return nil, "", "", hostbox.ErrRoleLaunchShape |
| ++ } |
| ++ return append([]string{filepath.Join(instancectl.LiveDir, host)}, args...), socket, db, nil |
| ++} |
| ++ |
| ++// envHas reports a non-empty value for the given "KEY=" prefix. |
| ++func envHas(env []string, prefix string) bool { |
| ++ for _, kv := range env { |
| ++ if v, ok := strings.CutPrefix(kv, prefix); ok && v != "" { |
| ++ return true |
| ++ } |
| ++ } |
| ++ return false |
| ++} |
| +--- a/internal/activator/claim_snapshot.go |
| ++++ b/internal/activator/claim_snapshot.go |
| +@@ -20,6 +20,7 @@ |
| + Identity hostwrite.Identity |
| + Executable, SHA256, Version string |
| + Args, Env []string |
| ++ Roles []string |
| + Limit cgroup.Limit |
| + } |
| + |
| +@@ -42,6 +43,9 @@ |
| + } |
| + if !claimLaunchRoute(rt) { |
| + return hostwrite.ErrUnavailable |
| ++ } |
| ++ if err := b.claimLaunchShape(rt); err != nil { |
| ++ return err |
| + } |
| + vars, err := b.instanceEnv(rt) |
| + if err != nil { |
| +@@ -69,6 +73,9 @@ |
| + } |
| + if !claimLaunchRoute(rt) { |
| + return out, hostwrite.ErrUnavailable |
| ++ } |
| ++ if err := b.claimLaunchShape(rt); err != nil { |
| ++ return out, err |
| + } |
| + vars, err := b.instanceEnv(rt) |
| + if err != nil { |
| +@@ -101,6 +108,7 @@ |
| + if err != nil || out.Version == "" { |
| + return out, hostwrite.ErrUnavailable |
| + } |
| ++ out.Roles = b.ClaimLaunchRoles(rt) |
| + out.Args = []string{"--socket", rt.Socket, "--db", rt.DBPath} |
| + out.Env = childEnv(b.agentOwnedVars(rt, "", vars, false)) |
| + raw, err := json.Marshal(out.Env) |
| +--- a/internal/activator/rolelaunch.go |
| ++++ b/internal/activator/rolelaunch.go |
| +@@ -31,6 +31,7 @@ |
| + } |
| + |
| + type RoleLaunch struct { |
| ++ Spec *hostbox.LaunchSpec |
| + // Root carries a reservation to root's isolated launcher. Nil keeps |
| + // every claimed launch refused. |
| + Root RootRoleLauncher |
| +--- a/internal/activator/rolelaunch_root_linux.go |
| ++++ b/internal/activator/rolelaunch_root_linux.go |
| +@@ -97,7 +97,7 @@ |
| + if err != nil { |
| + return fail(err) |
| + } |
| +- request := hostbox.RoleLaunchRequest{V: hostbox.RoleLaunchV, Op: hostbox.RoleLaunchStart, Reservation: &reservation, Executable: executable.Name(), MemoryMax: l.MemoryMax} |
| ++ request := hostbox.RoleLaunchRequest{Spec: l.Spec, V: hostbox.RoleLaunchV, Op: hostbox.RoleLaunchStart, Reservation: &reservation, Executable: executable.Name(), MemoryMax: l.MemoryMax} |
| + stopRequest := hostbox.RoleLaunchRequest{V: hostbox.RoleLaunchV, Op: hostbox.RoleLaunchStop, Reservation: &reservation} |
| + if err = request.Validate(); err != nil { |
| + return fail(err) |
| +@@ -165,26 +165,8 @@ |
| + if len(cmd.Args) == 0 { |
| + return nil, "", hostbox.ErrRoleLaunchShape |
| + } |
| +- args := cmd.Args[1:] |
| +- socket, db := "", "" |
| +- if len(args)%2 != 0 { |
| +- return nil, "", hostbox.ErrRoleLaunchShape |
| +- } |
| +- for i := 0; i < len(args); i += 2 { |
| +- value := args[i+1] |
| +- switch { |
| +- case args[i] == "--socket" && socket == "" && cleanAbs(value) && filepath.Dir(value) == runDir: |
| +- socket = value |
| +- case args[i] == "--db" && db == "" && cleanAbs(value) && filepath.Dir(value) == launchDir: |
| +- db = value |
| +- default: |
| +- return nil, "", hostbox.ErrRoleLaunchShape |
| +- } |
| +- } |
| +- if socket == "" || db == "" { |
| +- return nil, "", hostbox.ErrRoleLaunchShape |
| +- } |
| +- return append([]string{filepath.Join(instancectl.LiveDir, host)}, args...), socket, nil |
| ++ argv, socket, _, err := IsolatedArgv(host, launchDir, runDir, cmd.Args[1:], cmd.Env) |
| ++ return argv, socket, err |
| + } |
| + |
| + // PeerInRole verifies that conn's peer runs inside the unit root started |
| +--- /dev/null |
| ++++ b/internal/host/claimlaunch_authority.go |
| +@@ -0,0 +1,183 @@ |
| ++package host |
| ++ |
| ++import ( |
| ++ "context" |
| ++ "errors" |
| ++ "os" |
| ++ "path/filepath" |
| ++ "strconv" |
| ++ "time" |
| ++ |
| ++ "github.com/carlosframework/platform/internal/activator" |
| ++ "github.com/carlosframework/platform/internal/cgroup" |
| ++ "github.com/carlosframework/platform/internal/hostbox" |
| ++ "github.com/carlosframework/platform/internal/hostclaim" |
| ++ execlease "github.com/carlosframework/platform/internal/hostexec" |
| ++ "github.com/carlosframework/platform/internal/hostguard" |
| ++ "github.com/carlosframework/platform/internal/instancectl" |
| ++ "github.com/carlosframework/platform/internal/provision" |
| ++) |
| ++ |
| ++// Installed only by root construction. An operation adapter must read its |
| ++// durable per-kind journal and validate the exact observed op, phase, holder |
| ++// generation, stamp and session. Agent CurrentAttempt caches are not readers. |
| ++// A missing adapter refuses that kind. No operation adapter ships in rev 2. |
| ++type claimLaunchAttemptReader interface { |
| ++ ReadAttempt(context.Context, hostclaim.Observed) (hostclaim.WakeRetryAttempt, error) |
| ++} |
| ++ |
| ++func (h *ClaimLaunchRoot) authoritativeAttempt(ctx context.Context, c *hostclaim.Claim) (hostclaim.WakeRetryAttempt, error) { |
| ++ if c.Op.Kind == hostclaim.KindServing { |
| ++ retry := c.Holder.WakeRetry |
| ++ if retry == nil || retry.State != "attempting" || retry.EpisodeID == "" || retry.AttemptsReserved < 1 || retry.AttemptsReserved > hostclaim.WakeRetryLimit || retry.Attempt == nil || !retry.NotBefore.IsZero() { |
| ++ return hostclaim.WakeRetryAttempt{}, hostbox.ErrChanged |
| ++ } |
| ++ return *retry.Attempt, nil |
| ++ } |
| ++ switch c.Op.Kind { |
| ++ case hostclaim.KindLanding, hostclaim.KindRelocating: |
| ++ if reader := h.Attempts[c.Op.Kind]; reader != nil { |
| ++ return reader.ReadAttempt(ctx, hostclaim.Observed{Host: c.Host, Claim: c}) |
| ++ } |
| ++ } |
| ++ return hostclaim.WakeRetryAttempt{}, hostbox.ErrUnsupported |
| ++} |
| ++ |
| ++func (h *ClaimLaunchRoot) launchSpec(q hostbox.RoleLaunchRequest) (instancectl.Launch, error) { |
| ++ if q.Spec == nil || q.Reservation == nil { |
| ++ return instancectl.Launch{}, hostbox.ErrRoleLaunchShape |
| ++ } |
| ++ t := q.Reservation.Target |
| ++ paths, err := claimIsolatedPaths(map[string]string{"data": h.DataRoot}, t.Host, ClaimLaunchInputs{DBPath: provision.NestedDBPath(h.DataRoot, t.Host), Limit: cgroup.Limit{MemoryMax: q.MemoryMax}}) |
| ++ if err != nil { |
| ++ return instancectl.Launch{}, err |
| ++ } |
| ++ if h.runRoot != "" { |
| ++ paths.RunDir = filepath.Join(h.runRoot, provision.InstanceID(t.Host)) |
| ++ } |
| ++ argv, _, _, err := activator.IsolatedArgv(t.Host, paths.LaunchDir, paths.RunDir, q.Spec.Args, q.Spec.Env) |
| ++ if err != nil { |
| ++ return instancectl.Launch{}, err |
| ++ } |
| ++ return instancectl.Launch{Schema: instancectl.LaunchSchema, Host: t.Host, Argv: argv, Env: append([]string(nil), q.Spec.Env...)}, nil |
| ++} |
| ++ |
| ++// authorize is always called under the host guard, including inside |
| ++// StartChecked after namespace/tier preparation and immediately before start. |
| ++func (h *ClaimLaunchRoot) authorize(ctx context.Context, peer StopPeer, q hostbox.RoleLaunchRequest, c *hostclaim.Claim) error { |
| ++ t, r := q.Reservation.Target, *q.Reservation |
| ++ if err := ctx.Err(); err != nil { |
| ++ return err |
| ++ } |
| ++ actor, err := h.Actor(ctx) |
| ++ if err != nil { |
| ++ return err |
| ++ } |
| ++ if !actor.Eligible || actor.Quarantined || actor.Box != t.Box || actor.Enrollment != t.Enrollment || actor.Region != t.Region || strconv.FormatUint(actor.Revision, 10) != t.DirectoryRevision { |
| ++ return hostbox.ErrChanged |
| ++ } |
| ++ if c == nil || c.Host != t.Host { |
| ++ return hostbox.ErrChanged |
| ++ } |
| ++ if err := claimLaunchAuthority(c, t, actor.Box); err != nil { |
| ++ return err |
| ++ } |
| ++ a, err := h.authoritativeAttempt(ctx, c) |
| ++ if err != nil { |
| ++ return err |
| ++ } |
| ++ if a.ID == "" || a.InvocationID == "" || a.LaunchInputs == "" || r.ID != a.ID+":main" || r.InvocationID != a.InvocationID || r.Inputs != a.LaunchInputs || a.Session != t.Session || a.Box != t.Box || a.Enrollment != t.Enrollment || a.BootID != peer.BootID || a.ControllerID != peer.ControllerID { |
| ++ return hostbox.ErrChanged |
| ++ } |
| ++ if q.Spec == nil || q.Spec.Identity.Config == "" || q.Spec.Identity.Release == "" || !claimHexDigest(q.Spec.Digest) { |
| ++ return hostbox.ErrRoleLaunchShape |
| ++ } |
| ++ digest, err := q.Spec.Hash() |
| ++ if err != nil || digest != a.LaunchInputs { |
| ++ return hostbox.ErrChanged |
| ++ } |
| ++ if _, err := h.launchSpec(q); err != nil { |
| ++ return err |
| ++ } |
| ++ if err := h.Kernel.VerifyDigest(ctx, q.Executable, r.Executable, q.Spec.Digest); err != nil { |
| ++ return err |
| ++ } |
| ++ selection := hostbox.Selection{Host: t.Host, Action: claimSpawnAction(c), Box: t.Box, Region: t.Region, Captured: hostguard.Captured{Incarnation: t.Incarnation, Stamp: t.Stamp}, Kind: c.Op.Kind, Op: c.Op.ID, Phase: c.Op.Phase, Generation: c.Holder.Generation, Session: t.Session} |
| ++ if err := h.Local.Check(ctx, actor, c, selection); err != nil { |
| ++ return err |
| ++ } |
| ++ // Serving has no execution lease; a leftover operation state denies it. |
| ++ state, localErr := execlease.InspectLocal(h.Guard.DataDir, t.Host) |
| ++ if c.Op.Kind == hostclaim.KindServing { |
| ++ if !errors.Is(localErr, os.ErrNotExist) { |
| ++ return errors.Join(execlease.ErrLost, localErr) |
| ++ } |
| ++ } else { |
| ++ if localErr != nil { |
| ++ return localErr |
| ++ } |
| ++ // The operation lease has its own controller ID. Match its process to |
| ++ // the authenticated agent, and its binding to the current claim/boot. |
| ++ if state.Host != t.Host || state.Op != c.Op.ID || state.Incarnation != t.Incarnation || state.Box != t.Box || state.Enrollment != t.Enrollment || state.BootID != peer.BootID || state.Process.PID != peer.PID || state.Process.Start != peer.ProcessStart || state.Process.KernelBootID != peer.BootID { |
| ++ return execlease.ErrLost |
| ++ } |
| ++ if err := execlease.CheckLocal(h.Guard.DataDir, state.Binding, h.Clock, h.ExecLockDir); err != nil { |
| ++ return err |
| ++ } |
| ++ } |
| ++ now, boot, err := h.Clock.Now() |
| ++ if err != nil { |
| ++ return err |
| ++ } |
| ++ if c.Op.Kind != hostclaim.KindServing && boot >= state.SafeUntil { |
| ++ return execlease.ErrLost |
| ++ } |
| ++ if until, ok := c.QuietAfter[t.Region]; ok && !now.After(until) { |
| ++ return hostbox.ErrChanged |
| ++ } |
| ++ return nil |
| ++} |
| ++ |
| ++// waitReady observes one start, never reserves or starts another attempt. |
| ++// Type=simple returns before namespace setup and the trampoline exec. |
| ++func (h *ClaimLaunchRoot) waitReady(ctx context.Context, s instancectl.ClaimUnitStarted, want hostbox.LaunchFile) (ClaimLaunchObserved, error) { |
| ++ budget, poll := h.ReadyBudget, h.ReadyPoll |
| ++ if budget <= 0 { |
| ++ budget = 5 * time.Second |
| ++ } |
| ++ if poll <= 0 { |
| ++ poll = 25 * time.Millisecond |
| ++ } |
| ++ ctx, cancel := context.WithTimeout(ctx, budget) |
| ++ defer cancel() |
| ++ var pinned string |
| ++ for { |
| ++ if err := ctx.Err(); err != nil { |
| ++ return ClaimLaunchObserved{}, err |
| ++ } |
| ++ if err := h.Units.Same(ctx, s); err != nil { |
| ++ return ClaimLaunchObserved{}, err |
| ++ } |
| ++ got, err := h.Kernel.Observe(s) |
| ++ if got.ProcessStart == "" || (pinned != "" && pinned != got.ProcessStart) { |
| ++ return ClaimLaunchObserved{}, hostbox.ErrChanged |
| ++ } |
| ++ pinned = got.ProcessStart |
| ++ if err != nil && !errors.Is(err, errClaimLaunchPending) { |
| ++ return ClaimLaunchObserved{}, err |
| ++ } |
| ++ if err == nil && got.Executable == want && got.Running == want { |
| ++ if err := h.Units.Same(ctx, s); err != nil { |
| ++ return ClaimLaunchObserved{}, err |
| ++ } |
| ++ return got, nil |
| ++ } |
| ++ select { |
| ++ case <-ctx.Done(): |
| ++ return ClaimLaunchObserved{}, ctx.Err() |
| ++ case <-time.After(poll): |
| ++ } |
| ++ } |
| ++} |
| ++ |
| ++var errClaimLaunchPending = errors.New("claim launch: namespace or exec pending") |
| +--- a/internal/host/claimlaunch_kernel_linux.go |
| ++++ b/internal/host/claimlaunch_kernel_linux.go |
| +@@ -3,7 +3,10 @@ |
| + package host |
| + |
| + import ( |
| ++ "context" |
| ++ "errors" |
| + "os" |
| ++ "os/exec" |
| + "path/filepath" |
| + "strconv" |
| + "strings" |
| +@@ -120,13 +123,43 @@ |
| + if bound == "" { |
| + bound = "opt/carlos/app" |
| + } |
| +- var app unix.Stat_t |
| +- if err = unix.Stat(filepath.Join(k.ProcRoot, strconv.Itoa(s.MainPID), "root", bound), &app); err != nil { |
| +- return ClaimLaunchObserved{}, err |
| +- } |
| ++ var app, running unix.Stat_t |
| ++ observed := ClaimLaunchObserved{ProcessStart: start, Group: hostbox.LaunchFile{Device: uint64(group.Dev), Inode: group.Ino}} |
| ++ boundErr := unix.Stat(filepath.Join(k.ProcRoot, strconv.Itoa(s.MainPID), "root", bound), &app) |
| ++ execErr := unix.Stat(filepath.Join(k.ProcRoot, strconv.Itoa(s.MainPID), "exe"), &running) |
| + again, err := claimLaunchProcessStart(k.ProcRoot, s.MainPID) |
| + if err != nil || again != start { |
| + return ClaimLaunchObserved{}, hostbox.ErrChanged |
| + } |
| +- return ClaimLaunchObserved{ProcessStart: start, Group: hostbox.LaunchFile{Device: uint64(group.Dev), Inode: group.Ino}, Executable: hostbox.LaunchFile{Device: uint64(app.Dev), Inode: app.Ino}}, nil |
| ++ if boundErr != nil { |
| ++ if errors.Is(boundErr, os.ErrNotExist) { |
| ++ return observed, errClaimLaunchPending |
| ++ } |
| ++ return observed, boundErr |
| ++ } |
| ++ if execErr != nil { |
| ++ return observed, execErr |
| ++ } |
| ++ observed.Executable = hostbox.LaunchFile{Device: uint64(app.Dev), Inode: app.Ino} |
| ++ observed.Running = hostbox.LaunchFile{Device: uint64(running.Dev), Inode: running.Ino} |
| ++ return observed, nil |
| + } |
| ++ |
| ++func (k linuxClaimLaunchKernel) VerifyDigest(ctx context.Context, path string, want hostbox.LaunchFile, digest string) error { |
| ++ if err := k.Verify(path, want); err != nil { |
| ++ return err |
| ++ } |
| ++ f, err := verifiedClaimExecutable(ctx, exec.Command(path), path, digest) |
| ++ if err != nil { |
| ++ return err |
| ++ } |
| ++ defer f.Close() |
| ++ var st unix.Stat_t |
| ++ if err := unix.Fstat(int(f.Fd()), &st); err != nil { |
| ++ return err |
| ++ } |
| ++ if uint64(st.Dev) != want.Device || st.Ino != want.Inode { |
| ++ return hostbox.ErrChanged |
| ++ } |
| ++ return nil |
| ++} |
| +--- a/internal/host/claimlaunch_kernel_other.go |
| ++++ b/internal/host/claimlaunch_kernel_other.go |
| +@@ -3,6 +3,7 @@ |
| + package host |
| + |
| + import ( |
| ++ "context" |
| + "github.com/carlosframework/platform/internal/hostbox" |
| + "github.com/carlosframework/platform/internal/instancectl" |
| + ) |
| +@@ -18,3 +19,7 @@ |
| + func (linuxClaimLaunchKernel) Observe(instancectl.ClaimUnitStarted) (ClaimLaunchObserved, error) { |
| + return ClaimLaunchObserved{}, hostbox.ErrUnsupported |
| + } |
| ++ |
| ++func (linuxClaimLaunchKernel) VerifyDigest(context.Context, string, hostbox.LaunchFile, string) error { |
| ++ return hostbox.ErrUnsupported |
| ++} |
| +--- a/internal/host/claimlaunch_production.go |
| ++++ b/internal/host/claimlaunch_production.go |
| +@@ -8,6 +8,7 @@ |
| + |
| + "github.com/carlosframework/platform/internal/controlsock" |
| + "github.com/carlosframework/platform/internal/hostbox" |
| ++ execlease "github.com/carlosframework/platform/internal/hostexec" |
| + "github.com/carlosframework/platform/internal/instancectl" |
| + ) |
| + |
| +@@ -49,13 +50,21 @@ |
| + return nil, &ClaimDependencyError{Contracts: []string{"claim Plan 11 isolated root launch: production claim assembly"}} |
| + } |
| + a := entry.Assembly |
| +- return claimLaunchFor(root, a.Identity.Verify, a.ManagedRoots) |
| ++ wrapped, err := claimLaunchFor(root, a.Identity.Verify, a.ManagedRoots) |
| ++ if err != nil { |
| ++ return nil, err |
| ++ } |
| ++ if a.Local == nil { |
| ++ return nil, ErrClaimLaunchUnavailable |
| ++ } |
| ++ wrapped.Launch.Local = a.Local |
| ++ return wrapped, nil |
| + } |
| + |
| + func claimLaunchFor(root ClaimRootHTTP, actor func(context.Context) (hostbox.Actor, error), roots map[string]string) (ClaimRootWithLaunch, error) { |
| + coordinator := root.Stop.Coordinator |
| + if root.Launches == nil || root.Stop.Authenticate == nil || coordinator.Kernel == nil || coordinator.Guard.DataDir == "" || coordinator.Guard.Claims == nil || actor == nil || |
| +- !claimLaunchAbs(roots["opt"]) || !claimLaunchAbs(roots["data"]) { |
| ++ !claimLaunchAbs(roots["opt"]) || roots["data"] != "/data" { |
| + return ClaimRootWithLaunch{}, ErrClaimStopUnavailable |
| + } |
| + units := instancectl.ClaimUnits{ |
| +@@ -64,10 +73,11 @@ |
| + return string(out), err |
| + }, |
| + CgroupRoot: "/sys/fs/cgroup", |
| +- ExeRoot: filepath.Join(controlsock.Dir, "exe"), |
| +- Room: instancectl.DiskRoom(filepath.Join(roots["data"], "instances")), |
| ++ UnitRoot: "/run/systemd/system", DataRoot: roots["data"], |
| ++ ExeRoot: filepath.Join(controlsock.Dir, "exe"), |
| ++ Room: instancectl.DiskRoom(filepath.Join(roots["data"], "instances")), |
| + } |
| +- launch := &ClaimLaunchRoot{Authenticate: root.Stop.Authenticate, Guard: coordinator.Guard, Actor: actor, Journal: root.Launches, Units: units, |
| ++ launch := &ClaimLaunchRoot{DataRoot: roots["data"], Clock: execlease.SystemClock{}, Authenticate: root.Stop.Authenticate, Guard: coordinator.Guard, Actor: actor, Journal: root.Launches, Units: units, |
| + Kernel: linuxClaimLaunchKernel{OptRoot: roots["opt"], CgroupRoot: "/sys/fs/cgroup", ProcRoot: "/proc"}} |
| + root.Stop.Coordinator.Kernel = claimUnitStopKernel{Inner: coordinator.Kernel, Journal: root.Launches, Units: units} |
| + return ClaimRootWithLaunch{Root: root, Launch: launch}, nil |
| +--- a/internal/host/claimlaunch_root.go |
| ++++ b/internal/host/claimlaunch_root.go |
| +@@ -15,6 +15,7 @@ |
| + "github.com/carlosframework/platform/internal/cgroup" |
| + "github.com/carlosframework/platform/internal/hostbox" |
| + "github.com/carlosframework/platform/internal/hostclaim" |
| ++ execlease "github.com/carlosframework/platform/internal/hostexec" |
| + "github.com/carlosframework/platform/internal/hostguard" |
| + "github.com/carlosframework/platform/internal/instancectl" |
| + ) |
| +@@ -26,7 +27,8 @@ |
| + // ClaimLaunchUnits starts, stops and observes the host's isolated run unit. |
| + // Production is instancectl.ClaimUnits. |
| + type ClaimLaunchUnits interface { |
| +- Start(context.Context, instancectl.ClaimUnit) (instancectl.ClaimUnitStarted, error) |
| ++ StartChecked(context.Context, instancectl.ClaimUnit, instancectl.Launch, func() error) (instancectl.ClaimUnitStarted, error) |
| ++ Same(context.Context, instancectl.ClaimUnitStarted) error |
| + Stop(context.Context, string) error |
| + Gone(context.Context, string, string) error |
| + } |
| +@@ -36,6 +38,7 @@ |
| + // Verify accepts only a root-owned, immutable build under the release |
| + // tree with exactly the pinned identity. |
| + Verify(string, hostbox.LaunchFile) error |
| ++ VerifyDigest(context.Context, string, hostbox.LaunchFile, string) error |
| + // Observe reads the started unit's main process: its start time, the |
| + // identity of the cgroup it is a member of, and the build bound at the |
| + // unit's /opt/carlos/app. |
| +@@ -45,6 +48,7 @@ |
| + type ClaimLaunchObserved struct { |
| + ProcessStart string |
| + Group, Executable hostbox.LaunchFile |
| ++ Running hostbox.LaunchFile |
| + } |
| + |
| + // ClaimLaunchRoot is root's side of the reservation-bearing isolated launch |
| +@@ -53,16 +57,22 @@ |
| + // roster identity, its local launch journal and its own view of the kernel. |
| + // A request only names which reservation to run. |
| + type ClaimLaunchRoot struct { |
| +- Authenticate func(context.Context, net.Conn) (StopPeer, error) |
| +- Guard hostguard.Guard |
| +- Actor func(context.Context) (hostbox.Actor, error) |
| +- Journal *hostbox.LaunchJournal |
| +- Units ClaimLaunchUnits |
| +- Kernel ClaimLaunchKernel |
| ++ Local hostbox.LocalAuthority |
| ++ Clock execlease.Clock |
| ++ DataRoot, ExecLockDir string |
| ++ runRoot string |
| ++ Attempts map[string]claimLaunchAttemptReader |
| ++ ReadyBudget, ReadyPoll time.Duration |
| ++ Authenticate func(context.Context, net.Conn) (StopPeer, error) |
| ++ Guard hostguard.Guard |
| ++ Actor func(context.Context) (hostbox.Actor, error) |
| ++ Journal *hostbox.LaunchJournal |
| ++ Units ClaimLaunchUnits |
| ++ Kernel ClaimLaunchKernel |
| + } |
| + |
| + func (h *ClaimLaunchRoot) complete() bool { |
| +- return h != nil && h.Guard.DataDir != "" && h.Guard.Claims != nil && h.Actor != nil && h.Journal != nil && h.Units != nil && h.Kernel != nil |
| ++ return h != nil && h.Guard.DataDir != "" && h.Guard.Claims != nil && h.Actor != nil && h.Local != nil && h.Clock != nil && h.DataRoot != "" && h.Journal != nil && h.Units != nil && h.Kernel != nil |
| + } |
| + |
| + // Serve answers one validated request for an authenticated peer. |
| +@@ -78,6 +88,12 @@ |
| + } |
| + out := hostbox.RoleLaunchResponse{V: hostbox.RoleLaunchV, Protocol: hostbox.RoleLaunchProtocol} |
| + switch q.Op { |
| ++ case hostbox.RoleLaunchCapability: |
| ++ if q.Kind != "" && q.Kind != hostclaim.KindServing { |
| ++ if (q.Kind != hostclaim.KindLanding && q.Kind != hostclaim.KindRelocating) || h.Attempts[q.Kind] == nil { |
| ++ return hostbox.RoleLaunchResponse{}, hostbox.ErrUnsupported |
| ++ } |
| ++ } |
| + case hostbox.RoleLaunchStart: |
| + grant, err := h.start(ctx, peer, q) |
| + if err != nil { |
| +@@ -155,7 +171,7 @@ |
| + return hostbox.RoleLaunchGrant{}, err |
| + } |
| + var grant hostbox.RoleLaunchGrant |
| +- check := func(c *hostclaim.Claim) error { return claimLaunchAuthority(c, t, actor.Box) } |
| ++ check := func(c *hostclaim.Claim) error { return h.authorize(ctx, peer, q, c) } |
| + // The fence, the claim check, the journal intention, the unit start and |
| + // the started record all happen under the host guard a root fence also |
| + // takes, so no fence or second launch interleaves with them. |
| +@@ -197,11 +213,33 @@ |
| + } |
| + cleanup, cancel := context.WithTimeout(context.WithoutCancel(ctx), 2*time.Minute) |
| + defer cancel() |
| +- started, err := h.Units.Start(ctx, instancectl.ClaimUnit{Host: t.Host, Executable: q.Executable, MemoryMax: q.MemoryMax}) |
| ++ launch, err := h.launchSpec(q) |
| + if err != nil { |
| + return hostbox.RoleLaunchGrant{}, errors.Join(err, h.settleGone(cleanup, t, r.ID, "")) |
| + } |
| +- observed, err := h.Kernel.Observe(started) |
| ++ started, err := h.Units.StartChecked(ctx, instancectl.ClaimUnit{Host: t.Host, Executable: q.Executable, MemoryMax: q.MemoryMax}, launch, func() error { |
| ++ // Unit preparation can block. Read the claim, actor and local deadline |
| ++ // again at the final systemctl boundary, still under this host guard. |
| ++ fresh, err := h.Guard.Claims.Read(ctx, t.Host) |
| ++ if err != nil { |
| ++ return err |
| ++ } |
| ++ if fresh.Absent || fresh.Host != t.Host { |
| ++ return hostbox.ErrChanged |
| ++ } |
| ++ in, err := h.Journal.Read(ctx, t) |
| ++ if err != nil { |
| ++ return err |
| ++ } |
| ++ if in.WritersClosed { |
| ++ return hostbox.ErrChanged |
| ++ } |
| ++ return h.authorize(ctx, peer, q, fresh.Claim) |
| ++ }) |
| ++ if err != nil { |
| ++ return hostbox.RoleLaunchGrant{}, errors.Join(err, h.settleGone(cleanup, t, r.ID, "")) |
| ++ } |
| ++ observed, err := h.waitReady(ctx, started, r.Executable) |
| + if err == nil && (started.ControlGroup != r.Group.RelativePath || observed.Executable != r.Executable || observed.ProcessStart == "" || observed.Group.Device == 0 || observed.Group.Inode == 0) { |
| + err = hostbox.ErrChanged |
| + } |
| +@@ -210,9 +248,16 @@ |
| + err = h.Journal.StartedUnit(ctx, r, group, started.MainPID, observed.ProcessStart) |
| + } |
| + if err != nil { |
| +- stopErr := h.Units.Stop(cleanup, t.Host) |
| +- if stopErr == nil { |
| ++ // A replacement detected while waiting is never stopped by the old |
| ++ // reservation. Settle only if root can prove the old invocation gone. |
| ++ var stopErr error |
| ++ if sameErr := h.Units.Same(cleanup, started); sameErr != nil { |
| + stopErr = h.settleGone(cleanup, t, r.ID, started.InvocationID) |
| ++ } else { |
| ++ stopErr = h.Units.Stop(cleanup, t.Host) |
| ++ if stopErr == nil { |
| ++ stopErr = h.settleGone(cleanup, t, r.ID, started.InvocationID) |
| ++ } |
| + } |
| + return hostbox.RoleLaunchGrant{}, errors.Join(err, stopErr) |
| + } |
| +--- a/internal/host/claimlaunch_transport.go |
| ++++ b/internal/host/claimlaunch_transport.go |
| +@@ -113,3 +113,14 @@ |
| + _, err := c.call(ctx, q) |
| + return err |
| + } |
| ++ |
| ++func (c *ClaimRoleRootClient) KindCapability(ctx context.Context, kind string) error { |
| ++ out, err := c.call(ctx, hostbox.RoleLaunchRequest{V: hostbox.RoleLaunchV, Op: hostbox.RoleLaunchCapability, Kind: kind}) |
| ++ if err != nil { |
| ++ return err |
| ++ } |
| ++ if out.Protocol != hostbox.RoleLaunchProtocol { |
| ++ return hostbox.ErrChanged |
| ++ } |
| ++ return nil |
| ++} |
| +--- a/internal/host/claimowners.go |
| ++++ b/internal/host/claimowners.go |
| +@@ -101,6 +101,9 @@ |
| + Args, Env []string |
| + Version string |
| + Limit cgroup.Limit |
| ++ // Roles are the processes the real start spawns: "main", plus "seed" |
| ++ // and "sidecar" when the route carries them. Nil means main alone. |
| ++ Roles []string |
| + } |
| + |
| + // ClaimMarkerStore writes generation.json conditionally (moves §5.1, claim |
| +@@ -385,7 +388,11 @@ |
| + if !claimFreshGeneration(h, generation) { |
| + return ClaimLaunchInputs{}, hostwrite.ErrUnavailable |
| + } |
| +- in := ClaimLaunchInputs{Roles: []string{"main"}, Identity: l.Identity, Executable: l.Executable, ExecutableSHA256: l.ExecutableSHA256, Args: append([]string(nil), l.Args...), Env: append([]string(nil), l.Env...), DBPath: rt.DBPath, Version: l.Version, Limit: l.Limit, |
| ++ roles := append([]string(nil), l.Roles...) |
| ++ if len(roles) == 0 { |
| ++ roles = []string{"main"} |
| ++ } |
| ++ in := ClaimLaunchInputs{Roles: roles, Identity: l.Identity, Executable: l.Executable, ExecutableSHA256: l.ExecutableSHA256, Args: append([]string(nil), l.Args...), Env: append([]string(nil), l.Env...), DBPath: rt.DBPath, Version: l.Version, Limit: l.Limit, |
| + Admission: hostclaim.Admission{ObservedSession: h.Session, ObservedGeneration: h.Generation, Box: s.Box, Mode: "cold", NewGeneration: generation, Region: s.Region}} |
| + if claimEmptyBootstrap(h) { |
| + in.Empty = true |
| +--- a/internal/host/claimproduction.go |
| ++++ b/internal/host/claimproduction.go |
| +@@ -412,6 +412,6 @@ |
| + return ClaimServingLaunch{}, err |
| + } |
| + snapshot, err := backend.SnapshotClaimLaunch(ctx, rt) |
| +- return ClaimServingLaunch{Identity: snapshot.Identity, Executable: snapshot.Executable, ExecutableSHA256: snapshot.SHA256, Args: snapshot.Args, Env: snapshot.Env, Version: snapshot.Version, Limit: snapshot.Limit}, err |
| +- } |
| +-} |
| ++ return ClaimServingLaunch{Identity: snapshot.Identity, Executable: snapshot.Executable, ExecutableSHA256: snapshot.SHA256, Args: snapshot.Args, Env: snapshot.Env, Roles: snapshot.Roles, Version: snapshot.Version, Limit: snapshot.Limit}, err |
| ++ } |
| ++} |
| +--- a/internal/host/claimroleassembly.go |
| ++++ b/internal/host/claimroleassembly.go |
| +@@ -77,8 +77,28 @@ |
| + if a == nil || a.Root == nil { |
| + return activator.ErrClaimIsolationUnavailable |
| + } |
| +- _, err := claimIsolatedPaths(a.ManagedRoots, host, in) |
| +- return err |
| ++ if a.ManagedRoots["data"] != "/data" { |
| ++ return hostbox.ErrRoleLaunchShape |
| ++ } |
| ++ if len(in.Roles) != 1 || in.Roles[0] != "main" { |
| ++ return hostbox.ErrUnsupported |
| ++ } |
| ++ if !filepath.IsAbs(in.Executable) || filepath.Clean(in.Executable) != in.Executable || !claimHexDigest(in.ExecutableSHA256) || in.Identity.Config == "" || in.Identity.Release == "" { |
| ++ return hostbox.ErrRoleLaunchShape |
| ++ } |
| ++ paths, err := claimIsolatedPaths(a.ManagedRoots, host, in) |
| ++ if err != nil { |
| ++ return err |
| ++ } |
| ++ // The same validator startIsolated runs on the built command. |
| ++ _, socket, db, err := activator.IsolatedArgv(host, paths.LaunchDir, paths.RunDir, in.Args, in.Env) |
| ++ if err != nil { |
| ++ return err |
| ++ } |
| ++ if db != in.DBPath || socket != provision.NestedSocket(host) { |
| ++ return hostbox.ErrRoleLaunchShape |
| ++ } |
| ++ return nil |
| + } |
| + |
| + // claimIsolatedLaunch is where main's isolated run unit finds a host: its |
| +@@ -103,3 +123,18 @@ |
| + } |
| + return claimIsolatedLaunch{LaunchDir: dir, RunDir: provision.InstanceRunDir(host), LogSocket: provision.LogSocket(host, "run"), MemoryMax: in.Limit.MemoryMax}, nil |
| + } |
| ++ |
| ++// PreflightKind checks root's installed verifier before any attempt/admission. |
| ++// The existing RootRoleLauncher signatures remain unchanged. |
| ++func (a *ClaimRoleAssembly) PreflightKind(ctx context.Context, kind string) error { |
| ++ if kind == hostclaim.KindServing { |
| ++ return nil |
| ++ } |
| ++ client, ok := a.Root.(interface { |
| ++ KindCapability(context.Context, string) error |
| ++ }) |
| ++ if !ok { |
| ++ return activator.ErrClaimIsolationUnavailable |
| ++ } |
| ++ return client.KindCapability(ctx, kind) |
| ++} |
| +--- a/internal/host/claimroleassembly_linux.go |
| ++++ b/internal/host/claimroleassembly_linux.go |
| +@@ -135,6 +135,7 @@ |
| + return nil, hostwrite.ErrUnavailable |
| + } |
| + launch := *pre |
| ++ launch.Spec = &hostbox.LaunchSpec{Identity: in.Identity, Executable: in.Executable, Digest: in.ExecutableSHA256, Args: append([]string(nil), in.Args...), Env: append([]string(nil), in.Env...)} |
| + launch.Target, launch.Work, launch.Identity = target, work, in.Identity |
| + var mu sync.Mutex |
| + launch.Initialize = func(ctx context.Context) error { |
| +--- a/internal/host/claimwakeassembly.go |
| ++++ b/internal/host/claimwakeassembly.go |
| +@@ -213,10 +213,16 @@ |
| + if err := w.Roles.Preflight(call, ClaimLaunchInputs{}); err != nil { |
| + return err |
| + } |
| ++ if o.Claim == nil { |
| ++ return hostbox.ErrChanged |
| ++ } |
| ++ if err := w.Roles.PreflightKind(call, o.Claim.Op.Kind); err != nil { |
| ++ return err |
| ++ } |
| + if err := w.admitsOperation(o, p, writer.Box); err != nil { |
| + return err |
| + } |
| +- if rt.Backing != "" || rt.Unit != "" || rt.IsSidecar() || rt.Tunnel != "" { |
| ++ if rt.Seed != "" || rt.Backing != "" || rt.Unit != "" || rt.IsSidecar() || rt.Tunnel != "" { |
| + return hostbox.ErrUnsupported |
| + } |
| + in, err := w.Inputs.Select(call, rt, o, p) |
| +--- /dev/null |
| ++++ b/internal/hostbox/launchspec.go |
| +@@ -0,0 +1,25 @@ |
| ++package hostbox |
| ++ |
| ++import ( |
| ++ "crypto/sha256" |
| ++ "encoding/hex" |
| ++ "encoding/json" |
| ++ "github.com/carlosframework/platform/internal/hostwrite" |
| ++) |
| ++ |
| ++// LaunchSpec is a digest preimage, never authority. Root compares its digest |
| ++// with the authoritative attempt and hashes the executable independently. |
| ++type LaunchSpec struct { |
| ++ Identity hostwrite.Identity |
| ++ Executable, Digest string |
| ++ Args, Env []string |
| ++} |
| ++ |
| ++func (s LaunchSpec) Hash() (string, error) { |
| ++ raw, err := json.Marshal(s) |
| ++ if err != nil { |
| ++ return "", err |
| ++ } |
| ++ sum := sha256.Sum256(raw) |
| ++ return hex.EncodeToString(sum[:]), nil |
| ++} |
| +--- a/internal/hostbox/rolelaunch.go |
| ++++ b/internal/hostbox/rolelaunch.go |
| +@@ -21,7 +21,7 @@ |
| + // The request names which reservation to run; nothing in it is authority. |
| + const ( |
| + RoleLaunchV = 1 |
| +- RoleLaunchProtocol = "isolated-unit-v1" |
| ++ RoleLaunchProtocol = "isolated-unit-v2" |
| + RoleLaunchLimit = 32 << 10 |
| + |
| + RoleLaunchCapability = "capability" |
| +@@ -47,6 +47,8 @@ |
| + // agent asks root to run, the build it pinned and the slice tier; stop names |
| + // the same reservation; capability carries nothing. |
| + type RoleLaunchRequest struct { |
| ++ Kind string `json:"kind,omitempty"` |
| ++ Spec *LaunchSpec `json:"spec,omitempty"` |
| + V int `json:"v"` |
| + Op string `json:"op"` |
| + Reservation *LaunchReservation `json:"reservation,omitempty"` |
| +@@ -96,15 +98,15 @@ |
| + } |
| + switch q.Op { |
| + case RoleLaunchCapability: |
| +- if q.Reservation != nil || q.Executable != "" || q.MemoryMax != 0 { |
| ++ if q.Spec != nil || q.Reservation != nil || q.Executable != "" || q.MemoryMax != 0 { |
| + return ErrRoleLaunchShape |
| + } |
| + case RoleLaunchStart: |
| +- if q.Reservation == nil || !unitReservationShape(*q.Reservation) || !filepath.IsAbs(q.Executable) || filepath.Clean(q.Executable) != q.Executable || q.MemoryMax <= 0 { |
| ++ if q.Kind != "" || q.Reservation == nil || !unitReservationShape(*q.Reservation) || !filepath.IsAbs(q.Executable) || filepath.Clean(q.Executable) != q.Executable || q.MemoryMax <= 0 { |
| + return ErrRoleLaunchShape |
| + } |
| + case RoleLaunchStop: |
| +- if q.Reservation == nil || !unitReservationShape(*q.Reservation) || q.Executable != "" || q.MemoryMax != 0 { |
| ++ if q.Kind != "" || q.Spec != nil || q.Reservation == nil || !unitReservationShape(*q.Reservation) || q.Executable != "" || q.MemoryMax != 0 { |
| + return ErrRoleLaunchShape |
| + } |
| + default: |
| +--- /dev/null |
| ++++ b/internal/instancectl/claimlaunch_snapshot.go |
| +@@ -0,0 +1,90 @@ |
| ++package instancectl |
| ++ |
| ++import ( |
| ++ "context" |
| ++ "encoding/json" |
| ++ "errors" |
| ++ "fmt" |
| ++ "os" |
| ++ "path/filepath" |
| ++ "strings" |
| ++ |
| ++ "github.com/carlosframework/platform/internal/instancefs" |
| ++ "github.com/carlosframework/platform/internal/provision" |
| ++) |
| ++ |
| ++// Only root configures these roots; requests cannot choose a source or target. |
| ++// The private source hides secrets on the host. Inside the isolated namespace |
| ++// the verified file is readable but cannot be replaced by the tenant/agent. |
| ++func (u ClaimUnits) launchPaths(host string) (string, string, error) { |
| ++ for _, root := range []string{u.UnitRoot, u.ExeRoot, u.DataRoot} { |
| ++ if !filepath.IsAbs(root) || filepath.Clean(root) != root || strings.ContainsAny(root, " \t\r\n\x00:%") { |
| ++ return "", "", ErrClaimUnit |
| ++ } |
| ++ } |
| ++ if !provision.ValidHostShape(host) { |
| ++ return "", "", ErrClaimUnit |
| ++ } |
| ++ return filepath.Join(u.ExeRoot, ".claim", provision.InstanceID(host)), filepath.Join(u.UnitRoot, UnitName(RoleRun, host)+".d"), nil |
| ++} |
| ++ |
| ++func (u ClaimUnits) freezeLaunch(ctx context.Context, host string, launch Launch) error { |
| ++ source, dropin, err := u.launchPaths(host) |
| ++ if err != nil { |
| ++ return err |
| ++ } |
| ++ if launch.Host != host || launch.Schema != LaunchSchema { |
| ++ return ErrClaimUnit |
| ++ } |
| ++ // These are root-owned trees, opened without symlinks or permissive modes. |
| ++ for _, dir := range []string{u.ExeRoot, filepath.Dir(source), source, u.UnitRoot, dropin} { |
| ++ if err := claimRootDir(dir); err != nil { |
| ++ return fmt.Errorf("root directory %s: %w", dir, err) |
| ++ } |
| ++ } |
| ++ if err := os.Chmod(filepath.Dir(source), 0700); err != nil { |
| ++ return err |
| ++ } |
| ++ raw, err := json.Marshal(launch) |
| ++ if err != nil { |
| ++ return err |
| ++ } |
| ++ if err := instancefs.WriteAtomic(source, "launch.json", raw, 0444); err != nil { |
| ++ return err |
| ++ } |
| ++ if err := os.Chmod(filepath.Join(source, "launch.json"), 0444); err != nil { |
| ++ return err |
| ++ } |
| ++ destination := filepath.Join(provision.InstanceDir(u.DataRoot, host), LaunchFileName(RoleRun)) |
| ++ body := "[Service]\nBindReadOnlyPaths=" + filepath.Join(source, "launch.json") + ":" + destination + "\n" |
| ++ if err := instancefs.WriteAtomic(dropin, "60-claim-launch.conf", []byte(body), 0644); err != nil { |
| ++ return err |
| ++ } |
| ++ _, err = u.Systemctl(ctx, "daemon-reload") |
| ++ return err |
| ++} |
| ++ |
| ++func (u ClaimUnits) clearLaunch(ctx context.Context, host string) error { |
| ++ // Ordinary Start users do not configure the snapshot roots. |
| ++ if u.UnitRoot == "" { |
| ++ return nil |
| ++ } |
| ++ source, dropin, err := u.launchPaths(host) |
| ++ if err != nil { |
| ++ return err |
| ++ } |
| ++ // Each caller first proved the unit empty (or has not issued start). |
| ++ err = os.Remove(filepath.Join(dropin, "60-claim-launch.conf")) |
| ++ if err != nil && !errors.Is(err, os.ErrNotExist) { |
| ++ return err |
| ++ } |
| ++ if err == nil { |
| ++ if _, err = u.Systemctl(ctx, "daemon-reload"); err != nil { |
| ++ return err |
| ++ } |
| ++ } |
| ++ if err := os.Remove(filepath.Join(source, "launch.json")); err != nil && !errors.Is(err, os.ErrNotExist) { |
| ++ return err |
| ++ } |
| ++ return nil |
| ++} |
| +--- /dev/null |
| ++++ b/internal/instancectl/claimlaunch_snapshot_linux.go |
| +@@ -0,0 +1,28 @@ |
| ++//go:build linux |
| ++ |
| ++package instancectl |
| ++ |
| ++import ( |
| ++ "errors" |
| ++ "golang.org/x/sys/unix" |
| ++ "os" |
| ++) |
| ++ |
| ++func claimRootDir(dir string) error { |
| ++ if err := os.Mkdir(dir, 0700); err != nil && !errors.Is(err, os.ErrExist) { |
| ++ return err |
| ++ } |
| ++ fd, err := unix.Openat2(unix.AT_FDCWD, dir, &unix.OpenHow{Flags: unix.O_DIRECTORY | unix.O_RDONLY | unix.O_CLOEXEC, Resolve: unix.RESOLVE_NO_SYMLINKS | unix.RESOLVE_NO_MAGICLINKS}) |
| ++ if err != nil { |
| ++ return err |
| ++ } |
| ++ defer unix.Close(fd) |
| ++ var st unix.Stat_t |
| ++ if err := unix.Fstat(fd, &st); err != nil { |
| ++ return err |
| ++ } |
| ++ if st.Uid != uint32(os.Geteuid()) || st.Mode&0022 != 0 { |
| ++ return ErrClaimUnit |
| ++ } |
| ++ return nil |
| ++} |
| +--- /dev/null |
| ++++ b/internal/instancectl/claimlaunch_snapshot_other.go |
| +@@ -0,0 +1,5 @@ |
| ++//go:build !linux |
| ++ |
| ++package instancectl |
| ++ |
| ++func claimRootDir(string) error { return ErrClaimUnit } |
| +--- a/internal/instancectl/claimunit.go |
| ++++ b/internal/instancectl/claimunit.go |
| +@@ -26,6 +26,8 @@ |
| + Systemctl func(context.Context, ...string) (string, error) |
| + CgroupRoot string |
| + ExeRoot string |
| ++ UnitRoot string |
| ++ DataRoot string |
| + // Room is the disk floor check; nil skips it, as a nil Statfs does. |
| + Room func() error |
| + WaitBudget time.Duration |
| +@@ -112,6 +114,20 @@ |
| + // unit is idle, and stops any invocation it then cannot vouch for: one that |
| + // systemd did not newly create, or that runs outside the host's run cgroup. |
| + func (u ClaimUnits) Start(ctx context.Context, in ClaimUnit) (ClaimUnitStarted, error) { |
| ++ return u.start(ctx, in, nil, nil) |
| ++} |
| ++ |
| ++// StartChecked runs the root check after all preparation, immediately before |
| ++// systemctl start. It binds root's verified input snapshot over the mutable |
| ++// tenant launch file. The ordinary Start path keeps its existing inputs. |
| ++func (u ClaimUnits) StartChecked(ctx context.Context, in ClaimUnit, launch Launch, check func() error) (ClaimUnitStarted, error) { |
| ++ if check == nil { |
| ++ return ClaimUnitStarted{}, ErrClaimUnit |
| ++ } |
| ++ return u.start(ctx, in, &launch, check) |
| ++} |
| ++ |
| ++func (u ClaimUnits) start(ctx context.Context, in ClaimUnit, launch *Launch, check func() error) (ClaimUnitStarted, error) { |
| + tier, ok := claimTier(in.MemoryMax) |
| + if u.Systemctl == nil || u.CgroupRoot == "" || u.ExeRoot == "" || !ok || !provision.ValidHostShape(in.Host) || |
| + !filepath.IsAbs(in.Executable) || filepath.Clean(in.Executable) != in.Executable || in.Executable == SystemBinary || strings.HasPrefix(in.Executable, LiveDir+"/") { |
| +@@ -144,19 +160,48 @@ |
| + defer cancel() |
| + return ClaimUnitStarted{}, errors.Join(cause, u.Stop(cleanup, in.Host)) |
| + } |
| ++ if launch != nil { |
| ++ if err := u.freezeLaunch(ctx, in.Host, *launch); err != nil { |
| ++ return ClaimUnitStarted{}, err |
| ++ } |
| ++ } |
| ++ if check != nil { |
| ++ if err := check(); err != nil { |
| ++ return ClaimUnitStarted{}, errors.Join(err, u.clearLaunch(ctx, in.Host)) |
| ++ } |
| ++ } |
| + if out, err := u.Systemctl(ctx, "start", unit); err != nil { |
| + return refuse(fmt.Errorf("start %s: %s: %w", unit, strings.TrimSpace(out), err)) |
| + } |
| +- after, err := u.show(ctx, unit, "InvocationID", "MainPID", "ControlGroup", "ActiveState") |
| +- if err != nil { |
| +- return refuse(err) |
| +- } |
| +- pid, err := strconv.Atoi(after["MainPID"]) |
| +- if err != nil || pid <= 0 || after["InvocationID"] == "" || after["InvocationID"] == before["InvocationID"] || |
| +- after["ControlGroup"] != "/"+ClaimUnitControlGroup(in.Host) || after["ActiveState"] != "active" { |
| +- return refuse(ErrClaimUnit) |
| +- } |
| +- return ClaimUnitStarted{Unit: unit, InvocationID: after["InvocationID"], ControlGroup: ClaimUnitControlGroup(in.Host), MainPID: pid}, nil |
| ++ ready, cancel := context.WithTimeout(ctx, u.budget()) |
| ++ defer cancel() |
| ++ var invocation string |
| ++ for { |
| ++ after, err := u.show(ready, unit, "InvocationID", "MainPID", "ControlGroup", "ActiveState") |
| ++ if err != nil { |
| ++ return refuse(err) |
| ++ } |
| ++ id := after["InvocationID"] |
| ++ if idleState(after["ActiveState"]) || (id != "" && id == before["InvocationID"]) || (invocation != "" && id != invocation) { |
| ++ return refuse(ErrClaimUnit) |
| ++ } |
| ++ if id != "" { |
| ++ invocation = id |
| ++ } |
| ++ pid, _ := strconv.Atoi(after["MainPID"]) |
| ++ if id != "" && pid > 0 && after["ActiveState"] == "active" { |
| ++ if after["ControlGroup"] != "/"+ClaimUnitControlGroup(in.Host) { |
| ++ return refuse(ErrClaimUnit) |
| ++ } |
| ++ return ClaimUnitStarted{Unit: unit, InvocationID: id, ControlGroup: ClaimUnitControlGroup(in.Host), MainPID: pid}, nil |
| ++ } |
| ++ select { |
| ++ case <-ready.Done(): |
| ++ return refuse(ready.Err()) |
| ++ case <-time.After(u.poll()): |
| ++ } |
| ++ } |
| ++ |
| + } |
| + |
| + // Stop stops host's run unit and returns once systemd reports it inactive or |
| +@@ -182,7 +227,10 @@ |
| + case <-time.After(u.poll()): |
| + } |
| + } |
| +- return WaitCgroupEmpty(ctx, u.CgroupRoot, SliceName(host), unit, u.poll()) |
| ++ if err := WaitCgroupEmpty(ctx, u.CgroupRoot, SliceName(host), unit, u.poll()); err != nil { |
| ++ return err |
| ++ } |
| ++ return u.clearLaunch(ctx, host) |
| + } |
| + |
| + // Gone proves an invocation of host's run unit has ended: systemd reports a |
| +@@ -202,7 +250,10 @@ |
| + if !idleState(state["ActiveState"]) { |
| + return ErrClaimUnitBusy |
| + } |
| +- return u.cgroupEmpty(host) |
| ++ if err := u.cgroupEmpty(host); err != nil { |
| ++ return err |
| ++ } |
| ++ return u.clearLaunch(ctx, host) |
| + } |
| + |
| + // DiskRoom is the start floor /start applies (#214 item 1), for the claim |
| +@@ -219,3 +270,18 @@ |
| + return nil |
| + } |
| + } |
| ++ |
| ++// Same refuses replacement invocations and PID changes throughout readiness. |
| ++func (u ClaimUnits) Same(ctx context.Context, s ClaimUnitStarted) error { |
| ++ if u.Systemctl == nil || s.Unit == "" || s.InvocationID == "" || s.MainPID <= 0 { |
| ++ return ErrClaimUnit |
| ++ } |
| ++ state, err := u.show(ctx, s.Unit, "InvocationID", "MainPID", "ControlGroup", "ActiveState") |
| ++ if err != nil { |
| ++ return err |
| ++ } |
| ++ if state["InvocationID"] != s.InvocationID || state["MainPID"] != strconv.Itoa(s.MainPID) || state["ControlGroup"] != "/"+s.ControlGroup || state["ActiveState"] != "active" { |
| ++ return ErrClaimUnit |
| ++ } |
| ++ return nil |
| ++} |
| +``` |
| + |
| +```bash |
| +python3 "$CLAIM11_SCRATCH/apply.py" "$CLAIM11_PLAN" D2 implementation |
| +nice -n 10 go test -p 1 -timeout 5m ./internal/hostbox ./internal/instancectl ./internal/activator ./internal/host -run 'TestPlan11|TestRev2Shape|TestRoleLaunch|TestUnitReservation|TestRootRole|TestRootClaimProcess|TestClaimLaunch|TestClaimUnit|TestClaimIsolation|TestClaimRole' -count=1 |
| +``` |
| + |
| +Expected: all amendment and existing launch/retry/drain tests pass. A slow valid start makes one unit start, obtains one grant, makes no stop, and leaves `AttemptsReserved` at one. |
| + |
| +## Region HA integration and rollout |
| + |
| +Keep all flags off while landing the code. Registration alone does not activate relocation. The rollout still needs Plan 9's verified identity, complete owners, activation census and floor guard. Region HA additionally needs `CARLOS_REGION_HA`; this implementation shares Group D with `CARLOS_MOVES_PHASE1`, so it requires that group to be complete too. Missing owners, an older root protocol, unsupported role shapes and incomplete stop evidence all refuse. |
| + |
| +The claim-enabled protocol in Task D carries a main role in main's isolated run unit, using the nested instance layout and a supported memory tier. Seed, sidecar and other layouts remain refused where the reservation cannot describe them. The existing flag-off `/start`, `/seed`, process backend and unit templates retain their existing path. Staged builds must also be root-owned and not writable by the agent or tenant. Verify the deployed stager satisfies that rule before activation. Do not remove those refusals to make a probe pass. Extend the reservation and root verification first if another shape is required. |
| + |
| +Before region HA stage B is armed, install and test root-owned authoritative attempt readers for each enabled operation kind, including the prepared executable/input digest and the exact operation/session/install identity. Until then, the new per-kind preflight refuses those launches before admission or attempt reservation. Keep the existing region HA adapter signatures. Also prove the root-owned input snapshot bind, same-invocation namespace/exec wait, final deadline check and snapshot teardown against the deployed unit template in the disposable environment. |
| + |
| +Rebase the stage B adapters onto these interfaces and test these joins: |
| + |
| +1. Replace its duplicate `AutomaticLoss`, `LocalDischarge`, loss-ID helper and `Claim.EvidenceRefs` with Task A's canonical definitions. Preserve its fence-consuming transitions, D4 retention and strict decode tests. |
| +2. Register the complete relocation owner set once. Route the probe through `OperationAttempts`; do not apply the old `Extend` hook or introduce a separate probe lifecycle. Keep the same admitted session through `relocating/probing -> relocating/serving`. The built region branch currently commits directly to kind `serving`; that kind change ends this session and requires a separate serving wake. It must not be treated as an implicit continuation. |
| +3. Reconcile its prepared-generation and `released` coverage admission with the base's generic admission. This plan permits only the accepted cold probe shape. It does not import region HA's transition producer or grant a new coverage authority. An adapter cannot bypass the gate with an optional policy callback. |
| +4. For a drain, call `ClaimAssembly.WritersStopped`, register the files publisher under `ClaimProducerFiles`, retain replication and file publication, and use root's observation. Drop the region branch's duplicate descriptor observer and consume the claim-owned `(TID, Device, Inode)` descriptor shape. Confirm with the live Litestream witness that descriptors opened lazily after `Replicate` do not leave the narrowing hint incomplete. An incomplete hint refuses the observation. After release publication, queue the stop-only obligation and request the full root stop. Pass its immutable key and digest to `RelocationReleaseReceipts.Complete`, then perform the region-owned executor transition. Never use a writers-only witness as a full stop receipt. |
| +5. A release without its exact root receipt remains pending, including if the source terminates before producing it. Plan 11 grants no fence-only discharge for this new obligation. That receipt-only rule is the accepted contract; the region-owned termination transition must account for it before activation. It must not silently drop the obligation or route it through generic chore completion. |
| +6. Preserve the source's files live set until the authoritative record commits. Moves Plan 5 must replace the stand-in view and prove the push, landing-install and rollback-return park exclusions at each destructive step. The model's passing J1 tests do not implement those production exclusions. |
| +7. Keep the R6 no-LTX refusal for every positive writer session. Keep volume-based automatic discharge refused for legacy roster entries with unknown enrollment time. Publish no invented timestamp or empty generation. |
| + |
| +The release floor must include claim-level `evidence_refs` and its nested `AutomaticLoss` fields, roster `enrolled_at`, the `relocation-release` kind, and the launch journal and protocol additions, including `writers_closed`, `close-writers`, the `spec` preimage, capability-only `kind`, and `isolated-unit-v2`. Publish it only after every enrolled actor and console can strictly decode these fields, the signed census proves the same executable set, and the existing guard transaction has armed the new floor. Do not substitute the branch's commit hash for a release tag or floor. Old binaries must not participate after publication. Rollback must respect the published floor. |
| + |
| +Run the region HA two-box model and its joint review after the stage B adapters are rewritten. Then exercise isolated launch, cancellation, root crash recovery, writers-only stop and full stop in a designated disposable systemd/cgroup environment. This authoring session used local mocks and kernel fixtures; it did not create or modify production infrastructure. |
| + |
| +## Final verification |
| + |
| +After all tasks, run the following in the disposable export. Run commands serially; no background test jobs. |
| + |
| +```bash |
| +nice -n 10 go test -p 1 -timeout 5m ./internal/hostactors ./internal/hostclaim ./internal/hostbox ./internal/hostguard ./internal/stopevidence ./internal/instancectl -count=1 |
| +nice -n 10 go test -p 1 -timeout 5m ./internal/host ./internal/activator -run 'TestPlan11|TestRev2Shape|TestStopWriters|TestWriters|TestVerifyWriters|TestVerifyNoProcess|TestNoProcess|TestRootRole|TestRootClaimProcess|TestClaimLaunch|TestClaimUnitStop|TestClaimStop|TestClaimProducer|TestPlan9D|TestClaimIsolation' -count=1 |
| +nice -n 10 go vet ./internal/hostactors ./internal/hostclaim ./internal/hostbox ./internal/hostguard ./internal/stopevidence ./internal/instancectl ./internal/host ./internal/activator |
| +nice -n 10 go build ./internal/host ./internal/activator ./internal/instancectl |
| +GOOS=darwin GOARCH=arm64 nice -n 10 go build ./internal/host ./internal/activator ./internal/instancectl |
| +GOOS=windows GOARCH=amd64 nice -n 10 go build ./internal/host ./internal/activator ./internal/instancectl |
| +``` |
| + |
| +Check `gofmt -l` on every changed Go file. Before a later implementation commit lands, run the repository's full build, vet, test and formatting gate and obtain green amadan CI on that tip. This plan's focused overlay verification does not replace that landing gate. |
| + |
| +### Rev 1 authoring results |
| + |
| +The plan was replayed from a fresh export of the pinned base. All fourteen test and implementation patch blocks applied in order, and the resulting changed Go files matched the tested combined overlay byte for byte. No git index or ref was changed. |
| + |
| +The supporting packages `hostactors`, `hostclaim`, `hostbox`, `hostguard`, `stopevidence` and `instancectl` passed their full package tests. The focused host and activator set passed, including Plan 9 dispatch, writers-only stop, descriptor and no-process observations, root launch, receipt validation and disabled-flag regressions. Vet passed for all eight packages. Builds of host, activator and instancectl passed on Linux and cross-compiled for darwin/arm64 and windows/amd64. Formatting was checked on all changed files. |
| + |
| +Red runs first exposed the missing record, enrollment, dispatch, writer-stop, observation and launch interfaces. Integration regressions then demonstrated mutable policy authority, cached witness aliasing, an omitted isolated cgroup, hidden root observation interfaces, a stale console enrollment time, refusal of a valid session-zero release, recovery settling an unobserved unit, and a probe launch before the executor became the admitted writer. Each corresponding guard was fixed and its focused tests passed. The final plan places every test block before its implementation. |
| + |
| +Some early delegated socket runs failed because of sandbox restrictions or long Unix socket paths; the parent reran the focused combined set under its short disk-backed TMPDIR and it passed. The long `TestClaimCompletedProducersBoundStopAndRetry` test exceeded an early delegated timeout; it is outside the final focused set. No production systemd unit, deployment, bucket, roster or floor was changed. Real unit isolation and the joint two-box stage B activation remain rollout checks, not claimed authoring results. |
| + |
| +### Rev 2 authoring results |
| + |
| +The four R10 behavioral regressions were run against the completed Rev 1 overlay first. They failed with the expected behavioral assertions, including the premature kill of a slow valid start. The amendment's successful root fixtures now contain authoritative attempts. Its additional tests cover both serving and operation tuples, missing verifiers, input and executable substitution, pending/lost/expired participation, loss of the live lock, state changes during unit preparation, namespace/exec delay, timeout, cancellation, PID reuse and invocation replacement, and snapshot binding/cleanup. |
| + |
| +All seventeen patch blocks replayed from the pinned base with zero fuzz. The resulting Go files matched the tested combined overlay byte for byte, and formatting was clean. The full supporting-package tests passed for `hostactors`, `hostclaim`, `hostbox`, `hostguard`, `stopevidence` and `instancectl`; the expanded focused host/activator set passed, including the amendment, launch, retry/drain and Plan 9 dispatch regressions. The focused amendment/launch set also passed from the fresh replay. Vet passed for all eight packages. Host, activator and instancectl built on Linux and cross-compiled for darwin/arm64 and windows/amd64. |
| + |
| +The default Go cache was read-only in the sandbox, so verification used the explicitly permitted shared cache `/home/paulca/.cache/tmp/claude-1001/gocache`. It was not deleted. An initial broad host run exceeded the Unix socket path limit; the bootstrap now uses a shorter disk-backed temporary directory, and the same tests passed there. These setup errors were not counted as regression evidence. No git index, ref or worktree registration was written. |
| + |
| +Opus reviewed the root authority and snapshot implementation and found no P1 or P2. Its data-root observation is handled by refusing production roots other than the template's `/data`, with a construction regression. The parent checked the final callback ordering, per-kind preflight refusal and replacement-invocation cleanup. Sonnet supplied the preflight shape changes; the parent integrated and verified them. No live systemd mount or production experiment was run. The real namespace/snapshot and region HA stage B checks remain activation gates. |
| -## Self-review |
| +## Rev 1 self-review |
| The record/dispatch review found no authority bypass in signed receipt validation or refusal-only policy. Its console re-enrollment finding is fixed by assigning a new root time to the new enrollment. Its proposed fence-only discharge conflicts with the accepted receipt-only obligation contract, so this plan preserves the refusal and names the resulting availability dependency in rollout. Generic obligation validation already rejects empty operation and retirement identities. |