amadan

carlos / platform Public

Plan 14 integrated: lifecycle E2E fixes eight more production defects (not for merge)

Missing-journal cleanup trap; recursive launch-settings host lock; never-launched bootstrap recovery; cancellation provenance and retirement; repeated-stop predecessor ambiguity; empty-recovery predecessor mismatch; reboot lock group denial; recovery receipt versus original launch. E2E passes create, two cold wakes, handover, reboot, third wake and crash recovery.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Paul Campbell pushed by claude f32a66a898ffa08327afbe859f727b2bdf3e2479 parent 4437d70
35 files changed, +1220 −64
  • internal/host/claimconstruction.go +4 −0
  • internal/host/claimempty_recovery.go +134 −0
  • internal/host/claimempty_recovery_test.go +100 −0
  • internal/host/claimfilesseed_guard.go +9 −0
  • internal/host/claimlaunch_root.go +10 −0
  • internal/host/claimowners.go +37 −17
  • internal/host/claimservingpark.go +17 −7
  • internal/host/claimservingpark_test.go +23 −4
  • internal/host/claimservingretire.go +1 −1
  • internal/host/claimstop.go +8 −1
  • internal/host/claimstop_kernel_linux.go +16 −2
  • internal/host/claimstop_kernel_other.go +1 −1
  • internal/host/claimstop_production.go +1 −1
  • internal/host/claimstop_recovery.go +26 −1
  • internal/host/claimstop_unjournaled_linux_test.go +193 −0
  • internal/host/claimstop_writers_linux.go +107 −2
  • internal/host/movesfiles.go +2 −1
  • internal/host/plan14_lifecycle_crash_root_linux_test.go +5 −0
  • internal/host/plan14_lifecycle_reboot_root_linux_test.go +5 −1
  • internal/host/plan14_lifecycle_root_linux_test.go +120 −9
  • internal/host/plan14_lifecycle_s3_root_linux_test.go +94 −0
  • internal/host/plan14_lifecycle_trace_root_linux_test.go +79 −0
  • internal/host/rootrestart_requests_source.go +18 −1
  • internal/host/rootupdates.go +14 −1
  • internal/host/rootupdates_launch_test.go +63 −0
  • internal/host/rootupdates_runtime.go +1 −1
  • internal/hostbox/launchjournal.go +2 −1
  • internal/hostbox/roleintention.go +1 −1
  • internal/hostbox/rolelaunch.go +2 −1
  • internal/hostbox/rolelaunch_test.go +18 −1
  • internal/hostguard/coldhandover.go +12 −0
  • internal/hostguard/coldpredecessor_root_test.go +44 −0
  • internal/hostguard/flock_unix.go +5 −0
  • internal/provision/plan14_p3_census_test.go +1 −1
  • internal/provision/testdata/plan14-root-reads-p3.tsv +47 −8

CI passed — run details

CI log 
=== gate ===
cd launchctx && go vet ./... && go test ./... -count=1
ok  	amadan.net/carlos/platform.git/launchctx	0.012s
?   	amadan.net/carlos/platform.git/launchctx/controlname	[no test files]
?   	amadan.net/carlos/platform.git/launchctx/wireid	[no test files]
go vet ./...
gofmt -l .: clean
go test ./... -count=1 -timeout 30m
ok  	github.com/carlosframework/platform	310.048s
?   	github.com/carlosframework/platform/cmd/changelog	[no test files]
ok  	github.com/carlosframework/platform/cmd/console	2.624s
ok  	github.com/carlosframework/platform/cmd/moveprobe	92.230s
ok  	github.com/carlosframework/platform/cmd/zddprobe	0.706s
ok  	github.com/carlosframework/platform/internal/activator	248.992s
ok  	github.com/carlosframework/platform/internal/agentsock	0.015s
ok  	github.com/carlosframework/platform/internal/alerting	0.003s
ok  	github.com/carlosframework/platform/internal/analytics	0.018s
ok  	github.com/carlosframework/platform/internal/appconfig	2.480s
ok  	github.com/carlosframework/platform/internal/apperr	0.071s
ok  	github.com/carlosframework/platform/internal/applog	0.124s
ok  	github.com/carlosframework/platform/internal/awsinv	0.003s
ok  	github.com/carlosframework/platform/internal/boxchoice	0.007s
ok  	github.com/carlosframework/platform/internal/boxid	0.806s
?   	github.com/carlosframework/platform/internal/boxid/boxidtest	[no test files]
?   	github.com/carlosframework/platform/internal/boxid/internal/signtest	[no test files]
ok  	github.com/carlosframework/platform/internal/boxvolumes	0.008s
ok  	github.com/carlosframework/platform/internal/caps	0.015s
ok  	github.com/carlosframework/platform/internal/certwarm	0.002s
ok  	github.com/carlosframework/platform/internal/cgroup	0.704s
ok  	github.com/carlosframework/platform/internal/changelog	0.309s
ok  	github.com/carlosframework/platform/internal/clamd	0.055s
?   	github.com/carlosframework/platform/internal/cliflag	[no test files]
ok  	github.com/carlosframework/platform/internal/console	395.179s
?   	github.com/carlosframework/platform/internal/controlname	[no test files]
?   	github.com/carlosframework/platform/internal/controlsock	[no test files]
ok  	github.com/carlosframework/platform/internal/dbdoor	5.987s
ok  	github.com/carlosframework/platform/internal/dbdoor/dbdoortest	0.010s
ok  	github.com/carlosframework/platform/internal/dbverify	0.005s
ok  	github.com/carlosframework/platform/internal/dnsrec	0.003s
ok  	github.com/carlosframework/platform/internal/docsite	0.005s
ok  	github.com/carlosframework/platform/internal/domains	2.356s
ok  	github.com/carlosframework/platform/internal/doorkey	0.006s
ok  	github.com/carlosframework/platform/internal/economics	0.005s
ok  	github.com/carlosframework/platform/internal/edge	26.703s
?   	github.com/carlosframework/platform/internal/edgeproof	[no test files]
ok  	github.com/carlosframework/platform/internal/edgesfile	0.008s
ok  	github.com/carlosframework/platform/internal/fleetapi	0.135s
ok  	github.com/carlosframework/platform/internal/gate	1.656s
ok  	github.com/carlosframework/platform/internal/genselect	0.015s
ok  	github.com/carlosframework/platform/internal/geo	0.015s
ok  	github.com/carlosframework/platform/internal/geoip	0.030s
ok  	github.com/carlosframework/platform/internal/historyreadback	0.157s
ok  	github.com/carlosframework/platform/internal/host	178.754s
ok  	github.com/carlosframework/platform/internal/hostactors	0.365s
ok  	github.com/carlosframework/platform/internal/hostbox	2.661s
ok  	github.com/carlosframework/platform/internal/hostclaim	0.532s
ok  	github.com/carlosframework/platform/internal/hostevidence	0.005s
ok  	github.com/carlosframework/platform/internal/hostexec	4.864s
ok  	github.com/carlosframework/platform/internal/hostfence	0.035s
ok  	github.com/carlosframework/platform/internal/hostfiles	10.497s
ok  	github.com/carlosframework/platform/internal/hostfloor	0.014s
ok  	github.com/carlosframework/platform/internal/hostguard	0.989s
ok  	github.com/carlosframework/platform/internal/hostops	1.623s
?   	github.com/carlosframework/platform/internal/hostops/fencerelease	[no test files]
ok  	github.com/carlosframework/platform/internal/hostops/internal/fencerelease	0.002s
ok  	github.com/carlosframework/platform/internal/hostquarantine	0.002s
ok  	github.com/carlosframework/platform/internal/hostrelocate	0.463s
ok  	github.com/carlosframework/platform/internal/hostresolve	0.133s
ok  	github.com/carlosframework/platform/internal/hostwrite	0.002s
ok  	github.com/carlosframework/platform/internal/infraview	0.002s
ok  	github.com/carlosframework/platform/internal/instancectl	5.222s
ok  	github.com/carlosframework/platform/internal/instanceexec	2.730s
ok  	github.com/carlosframework/platform/internal/instancefs	0.005s
ok  	github.com/carlosframework/platform/internal/instancekey	0.017s
ok  	github.com/carlosframework/platform/internal/instanceupdates	0.019s
ok  	github.com/carlosframework/platform/internal/layout	1.563s
ok  	github.com/carlosframework/platform/internal/lease	9.811s
ok  	github.com/carlosframework/platform/internal/lease/leasetest	0.019s
ok  	github.com/carlosframework/platform/internal/ledger	0.024s
ok  	github.com/carlosframework/platform/internal/mail	0.164s
ok  	github.com/carlosframework/platform/internal/maintenance	0.079s
ok  	github.com/carlosframework/platform/internal/maintenanceexec	5.528s
ok  	github.com/carlosframework/platform/internal/maintenanceguard	0.068s
ok  	github.com/carlosframework/platform/internal/metrics	0.005s
ok  	github.com/carlosframework/platform/internal/moves	21.214s
ok  	github.com/carlosframework/platform/internal/moves/archive	0.804s
ok  	github.com/carlosframework/platform/internal/objstore	0.174s
ok  	github.com/carlosframework/platform/internal/ops	0.180s
ok  	github.com/carlosframework/platform/internal/park	0.212s
ok  	github.com/carlosframework/platform/internal/privatefile	0.004s
ok  	github.com/carlosframework/platform/internal/provision	33.428s
ok  	github.com/carlosframework/platform/internal/publicrelease	0.006s
?   	github.com/carlosframework/platform/internal/regionname	[no test files]
ok  	github.com/carlosframework/platform/internal/registry	5.456s
ok  	github.com/carlosframework/platform/internal/relay	7.235s
ok  	github.com/carlosframework/platform/internal/release	21.875s
ok  	github.com/carlosframework/platform/internal/rollout	3.768s
ok  	github.com/carlosframework/platform/internal/rolloutwords	0.003s
ok  	github.com/carlosframework/platform/internal/rootaws	0.009s
ok  	github.com/carlosframework/platform/internal/rootparticipation	0.058s
ok  	github.com/carlosframework/platform/internal/rootstate	0.027s
ok  	github.com/carlosframework/platform/internal/runs	0.580s
ok  	github.com/carlosframework/platform/internal/s3compat	0.005s
ok  	github.com/carlosframework/platform/internal/safefile	0.004s
ok  	github.com/carlosframework/platform/internal/schedule	2.527s
ok  	github.com/carlosframework/platform/internal/seeds	0.011s
ok  	github.com/carlosframework/platform/internal/sim/hostlife	9.792s
ok  	github.com/carlosframework/platform/internal/sim/litestreamfacts	3.914s
ok  	github.com/carlosframework/platform/internal/status	0.041s
ok  	github.com/carlosframework/platform/internal/steering	0.006s
ok  	github.com/carlosframework/platform/internal/stopevidence	0.231s
ok  	github.com/carlosframework/platform/internal/svcrole	0.002s
ok  	github.com/carlosframework/platform/internal/telemetry	0.004s
ok  	github.com/carlosframework/platform/internal/termfence	0.004s
ok  	github.com/carlosframework/platform/internal/tunnelclient	0.004s
ok  	github.com/carlosframework/platform/internal/unixdial	0.003s
ok  	github.com/carlosframework/platform/internal/updatewords	0.006s
ok  	github.com/carlosframework/platform/internal/usage	0.004s
ok  	github.com/carlosframework/platform/internal/version	0.002s
ok  	github.com/carlosframework/platform/internal/vet	1.583s
=== cross ===
--- shard 0 of 5 ---
cross-compiling darwin/arm64
--- shard 1 of 5 ---
cross-compiling darwin/amd64
--- shard 2 of 5 ---
cross-compiling linux/arm64
linux/arm64 carlos binary: 48496802 bytes (ceiling 52000000)
--- shard 3 of 5 ---
cross-compiling windows/amd64
--- shard 4 of 5 ---
cross-compiling windows/arm64
=== minio-contract ===
minio contract: arm64, minio RELEASE.2025-09-07T16-13-09Z, mc RELEASE.2025-08-13T08-35-41Z
  minio: cached, digest verified
  mc: cached, digest verified
  minio healthy on 127.0.0.1:20999
Added `local` successfully.
Bucket created successfully `local/contract-bucket`.
Bucket created successfully `local/denied-bucket`.
Created policy `contract-only` successfully.
Added user `scopeduser` successfully.
Attached Policies: [contract-only]
To User: scopeduser
  buckets and scoped user ready
=== RUN   TestDeliveredContractMinio
=== RUN   TestDeliveredContractMinio/put
=== RUN   TestDeliveredContractMinio/get_round-trips_the_body
=== RUN   TestDeliveredContractMinio/list_shows_the_key
=== RUN   TestDeliveredContractMinio/head_and_list_agree_on_the_version
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed
=== RUN   TestDeliveredContractMinio/delete_removes_it
=== RUN   TestDeliveredContractMinio/cross-bucket_denial
--- PASS: TestDeliveredContractMinio (0.02s)
    --- PASS: TestDeliveredContractMinio/put (0.00s)
    --- PASS: TestDeliveredContractMinio/get_round-trips_the_body (0.00s)
    --- PASS: TestDeliveredContractMinio/list_shows_the_key (0.00s)
    --- PASS: TestDeliveredContractMinio/head_and_list_agree_on_the_version (0.00s)
    --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload (0.00s)
    --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route (0.01s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed (0.00s)
    --- PASS: TestDeliveredContractMinio/delete_removes_it (0.00s)
    --- PASS: TestDeliveredContractMinio/cross-bucket_denial (0.00s)
PASS
ok  	github.com/carlosframework/platform/internal/objstore	0.024s
=== RUN   TestT4StorageProbeMinio
2026/10/16 18:37:54 moves: T4: STORE: list: operation error S3: ListObjectsV2, https response error StatusCode: 403, RequestID: 18DF1676810E3491, HostID: 7c420053e0fbcd7f41653fdd92ffa2802c903d05303b1aa15adc2e8bb65d183f, api error SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.
--- PASS: TestT4StorageProbeMinio (0.01s)
PASS
ok  	github.com/carlosframework/platform/internal/host	0.022s
minio contract: PASS
diff --git a/internal/host/claimconstruction.go b/internal/host/claimconstruction.go
index 4ae846a22..f824633b6 100644
--- a/internal/host/claimconstruction.go
+++ b/internal/host/claimconstruction.go
@@ -298,6 +298,10 @@ func OpenProductionClaimAssembly(ctx context.Context, store release.Store, r Cla
var groupD claimGroupD
if servingEnabled {
serving = &claimServingOwners{HomeRegion: actor.HomeRegion, Data: hostbox.RequestStore{Store: owners.Data, Check: func(ctx context.Context) error { _, err := owners.Identity.Verify(ctx); return err }}, Bucket: actor.Bucket, Region: park.PlacementRegion(actor.Region), Box: actor.Box, NewID: newID, Now: time.Now, Sources: owners.Serving}
+ serving.emptyRecovery = rootEmptyRecovery(guard)
+ if files, ok := serving.Sources.Files.(*movesFiles); ok {
+ files.emptyRecovery = serving.emptyRecovery
+ }
dispatch := ClaimOwnerDispatch{Serving: serving.owners()}
if movesC1 {
groupD, err = claimGroupDWiring(claimEntryEnv(ctx), serving.owners(), owners.Moves, owners.Relocation)
diff --git a/internal/host/claimempty_recovery.go b/internal/host/claimempty_recovery.go
new file mode 100644
index 000000000..9eb4e8677
--- /dev/null
+++ b/internal/host/claimempty_recovery.go
@@ -0,0 +1,134 @@
+package host
+
+import (
+ "context"
+ "encoding/json"
+ "reflect"
+ "slices"
+ "strings"
+
+ lss3 "github.com/benbjohnson/litestream/s3"
+ "github.com/carlosframework/platform/internal/hostbox"
+ "github.com/carlosframework/platform/internal/hostclaim"
+ "github.com/carlosframework/platform/internal/hostfiles"
+ "github.com/carlosframework/platform/internal/hostguard"
+ "github.com/carlosframework/platform/internal/instancekey"
+)
+
+type claimEmptyRecovery func(context.Context, hostclaim.Observed, []hostclaim.AdmissionRecord) (bool, error)
+
+func rootEmptyRecovery(g hostguard.Guard) claimEmptyRecovery {
+ return func(ctx context.Context, o hostclaim.Observed, admissions []hostclaim.AdmissionRecord) (bool, error) {
+ if o.Claim == nil || o.Claim.Holder == nil || len(admissions) == 0 {
+ return false, nil
+ }
+ h := o.Claim.Holder
+ if len(admissions) > len(h.Admissions) || len(admissions) < len(h.Admissions)-1 || !reflect.DeepEqual(admissions, h.Admissions[:len(admissions)]) {
+ return false, hostbox.ErrChanged
+ }
+ captured := hostguard.Captured{Incarnation: h.Incarnation, Stamp: h.Stamp}
+ var view hostguard.FenceView
+ var err error
+ if g.Root != nil {
+ view, err = g.Root.View(ctx, o.Host, captured)
+ } else if g.FenceView != nil {
+ view, err = g.FenceView(ctx, o.Host, captured)
+ } else {
+ return false, nil
+ }
+ if err != nil {
+ return false, err
+ }
+ if view.Installation.Host != o.Host || view.Installation.Incarnation != h.Incarnation || view.Installation.Stamp != h.Stamp {
+ return false, hostbox.ErrChanged
+ }
+ predecessor := ""
+ for _, a := range admissions {
+ if a.Mode != "cold" || a.Covered || a.Box != view.Installation.Box || a.Predecessor != predecessor || a.Generation == "" {
+ return false, nil
+ }
+ found := false
+ for _, r := range view.Receipts {
+ if r.Target.Host != o.Host || r.Target.Incarnation != h.Incarnation || r.Target.Session != a.Session || r.Target.Box != a.Box || r.Target.Enrollment != view.Installation.Enrollment || r.Target.Op != "" {
+ continue
+ }
+ capturedReceipt := hostguard.Captured{Incarnation: r.Target.Incarnation, Stamp: r.Target.Stamp}
+ if r.Target.Stamp != h.Stamp && !slices.Contains(view.Retired, capturedReceipt) {
+ continue
+ }
+ var receipt StopReceipt
+ if json.Unmarshal(r.Bytes, &receipt) != nil {
+ return false, hostbox.ErrChanged
+ }
+ if receipt.Kind == StopReceiptStopped && receipt.Host == r.Target.Host && receipt.Incarnation == r.Target.Incarnation && receipt.InstallationStamp == r.Target.Stamp && receipt.WriterSession == r.Target.Session && receipt.FenceID == r.Target.FenceID && slices.Contains(receipt.CgroupIdentities, "unlaunched:"+digestStop(r.Target)) {
+ found = true
+ break
+ }
+ }
+ if !found {
+ return false, nil
+ }
+ predecessor = a.Generation
+ }
+ return true, nil
+ }
+}
+
+func (s *claimServingOwners) emptyBootstrap(ctx context.Context, o hostclaim.Observed) (bool, error) {
+ if claimEmptyBootstrap(o.Claim.Holder) {
+ return true, nil
+ }
+ if s.emptyRecovery == nil {
+ return false, nil
+ }
+ yes, err := s.emptyRecovery(ctx, o, o.Claim.Holder.Admissions)
+ if err != nil || !yes {
+ return false, err
+ }
+ return true, s.emptyLocation(ctx, o.Claim.Holder.Location, o.Claim.Holder.Admissions...)
+}
+
+func (s *claimServingOwners) emptyLocation(ctx context.Context, location string, admissions ...hostclaim.AdmissionRecord) error {
+ versions, err := s.Data.ListVersions(ctx, s.Bucket, strings.TrimSuffix(location, "/")+"/")
+ if err != nil {
+ return err
+ }
+ leaseKey := strings.TrimSuffix(location, "/") + "/" + lss3.DefaultLeasePath
+ allowed := map[string]bool{leaseKey: true}
+ for _, a := range admissions {
+ prefix := movesFilesPrefix(location, a.Generation)
+ key := hostfiles.ManifestKey(prefix)
+ for _, v := range versions {
+ if v.Key != key {
+ continue
+ }
+ if v.DeleteMarker || v.Size > 1<<20 {
+ return hostbox.ErrChanged
+ }
+ raw, err := s.Data.GetVersion(ctx, s.Bucket, key, v.VersionID)
+ if err != nil {
+ return err
+ }
+ var m hostfiles.Manifest
+ if json.Unmarshal(raw, &m) != nil || m.V != hostfiles.ManifestVersion || len(m.Entries) != 0 {
+ return hostbox.ErrChanged
+ }
+ for name, e := range m.Platform {
+ if name == instancekey.AdoptionItem && e.Deleted {
+ continue
+ }
+ if name != hostfiles.EnvItem || e.Deleted || e.Torn || e.Skipped != "" || !strings.HasPrefix(e.Object, prefix+"objects/") || !claimHexDigest(e.SHA256) {
+ return hostbox.ErrChanged
+ }
+ allowed[e.Object] = true
+ }
+ allowed[key] = true
+ }
+ }
+ for _, v := range versions {
+ if !allowed[v.Key] || v.Key != leaseKey && v.DeleteMarker {
+ return hostbox.ErrChanged
+ }
+ }
+ return nil
+}
diff --git a/internal/host/claimempty_recovery_test.go b/internal/host/claimempty_recovery_test.go
new file mode 100644
index 000000000..10de4bf5b
--- /dev/null
+++ b/internal/host/claimempty_recovery_test.go
@@ -0,0 +1,100 @@
+package host
+
+import (
+ "context"
+ "encoding/json"
+ "errors"
+ "github.com/carlosframework/platform/internal/hostfiles"
+ "strings"
+ "testing"
+
+ "github.com/carlosframework/platform/internal/hostclaim"
+ "github.com/carlosframework/platform/internal/hostguard"
+ "github.com/carlosframework/platform/internal/lease/leasetest"
+)
+
+func TestPlan14UnlaunchedBootstrapRecovery(t *testing.T) {
+ for _, mode := range []string{"empty", "lease-only", "platform-only", "tenant-files", "unreferenced-file", "published", "deleted-publication", "missing-receipt", "wrong-stamp", "wrong-session", "ordinary-stop", "covered", "foreign-box", "view-error"} {
+ t.Run(mode, func(t *testing.T) {
+ a := hostclaim.AdmissionRecord{Session: 1, Box: "box", Mode: "cold", Region: "ie", Generation: "first"}
+ h := &hostclaim.Holder{Incarnation: "inc", Stamp: 1, Session: 1, Generation: "first", WriterBox: "box", Location: "location", Admissions: []hostclaim.AdmissionRecord{a}}
+ o := hostclaim.Observed{Host: "shop.example", Claim: &hostclaim.Claim{Holder: h}}
+ target := hostguard.FenceTarget{Host: o.Host, Incarnation: h.Incarnation, Stamp: h.Stamp, Session: 1, Box: "box", Enrollment: "enrollment", FenceID: "fence"}
+ if mode == "wrong-stamp" {
+ target.Stamp++
+ }
+ if mode == "wrong-session" {
+ target.Session++
+ }
+ receipt := StopReceipt{Kind: StopReceiptStopped, Host: target.Host, Incarnation: target.Incarnation, InstallationStamp: target.Stamp, WriterSession: target.Session, FenceID: target.FenceID, CgroupIdentities: []string{"unlaunched:" + digestStop(target)}}
+ if mode == "ordinary-stop" {
+ receipt.CgroupIdentities = []string{"empty-cgroup"}
+ }
+ if mode == "covered" {
+ h.Admissions[0].Covered = true
+ }
+ if mode == "foreign-box" {
+ h.Admissions[0].Box = "other"
+ }
+ raw, _ := json.Marshal(receipt)
+ view := hostguard.FenceView{Installation: hostguard.RootInstallation{Host: o.Host, Incarnation: h.Incarnation, Stamp: h.Stamp, Box: "box", Enrollment: "enrollment"}, Receipts: []hostguard.FenceReceipt{{Target: target, Bytes: raw}}}
+ if mode == "missing-receipt" {
+ view.Receipts = nil
+ }
+ g := hostguard.Guard{FenceView: func(context.Context, string, hostguard.Captured) (hostguard.FenceView, error) {
+ if mode == "view-error" {
+ return hostguard.FenceView{}, errors.New("unavailable")
+ }
+ return view, nil
+ }}
+ data := leasetest.NewVersionedBucket()
+ data.AddBucket("data")
+ if mode == "lease-only" {
+ data.Put("data", "location/lock.json", []byte("lease"))
+ }
+ if mode == "published" || mode == "deleted-publication" {
+ data.Put("data", "location/generations/first/manifest.json", []byte("published"))
+ }
+ if mode == "deleted-publication" {
+ data.BareDelete("data", "location/generations/first/manifest.json")
+ }
+ if mode == "platform-only" || mode == "tenant-files" || mode == "unreferenced-file" {
+ prefix := movesFilesPrefix(h.Location, a.Generation)
+ object := prefix + "objects/0001-" + strings.Repeat("a", 64)
+ data.Put("data", object, []byte("env"))
+ m := hostfiles.Manifest{V: hostfiles.ManifestVersion, Entries: map[string]hostfiles.Entry{}, Platform: map[string]hostfiles.Entry{hostfiles.EnvItem: {Object: object, SHA256: strings.Repeat("a", 64), Size: 3}}}
+ if mode == "tenant-files" {
+ m.Entries["tenant.txt"] = m.Platform[hostfiles.EnvItem]
+ }
+ if mode == "unreferenced-file" {
+ data.Put("data", prefix+"objects/unreferenced", []byte("foreign"))
+ }
+ raw, _ := json.Marshal(m)
+ data.Put("data", hostfiles.ManifestKey(prefix), raw)
+ }
+ s := &claimServingOwners{Data: data, Bucket: "data", emptyRecovery: rootEmptyRecovery(g)}
+ yes, err := s.emptyBootstrap(t.Context(), o)
+ want := mode == "empty" || mode == "lease-only" || mode == "platform-only"
+ if (yes && err == nil) != want {
+ t.Fatalf("recovery=%v err=%v", yes, err)
+ }
+ if want {
+ h.Admissions = append(h.Admissions, hostclaim.AdmissionRecord{Session: 2, Box: "box", Mode: "cold", Region: "ie", Generation: "second", Predecessor: "first"})
+ if yes, err := s.emptyBootstrap(t.Context(), o); yes && err == nil {
+ t.Fatal("unproved second admission accepted")
+ }
+ h.Stamp = 2
+ view.Installation.Stamp = 2
+ view.Retired = []hostguard.Captured{{Incarnation: h.Incarnation, Stamp: 1}}
+ if yes, err := s.emptyRecovery(t.Context(), o, h.Admissions[:1]); !yes || err != nil {
+ t.Fatal("proved predecessor refused after handover", err)
+ }
+ view.Retired = nil
+ if yes, err := s.emptyRecovery(t.Context(), o, h.Admissions[:1]); yes && err == nil {
+ t.Fatal("unretained installation accepted")
+ }
+
+ }
+ })
+ }
+}
diff --git a/internal/host/claimfilesseed_guard.go b/internal/host/claimfilesseed_guard.go
index bf9d76dc8..fef68caae 100644
--- a/internal/host/claimfilesseed_guard.go
+++ b/internal/host/claimfilesseed_guard.go
@@ -26,6 +26,15 @@ func (f *movesFiles) checkServingFilesEvidence(ctx context.Context, o hostclaim.
if claimEmptyBootstrap(h) {
return nil
}
+ if f.emptyRecovery != nil {
+ yes, err := f.emptyRecovery(ctx, o, h.Admissions)
+ if err != nil {
+ return err
+ }
+ if yes {
+ return nil
+ }
+ }
bounded, cancel := context.WithTimeout(ctx, hostbox.RequestLimit)
defer cancel()
chain := movesFilesChain(h, f.HomeRegion, f.Region)
diff --git a/internal/host/claimlaunch_root.go b/internal/host/claimlaunch_root.go
index 354b7a57d..866ababad 100644
--- a/internal/host/claimlaunch_root.go
+++ b/internal/host/claimlaunch_root.go
@@ -601,6 +601,16 @@ func (k claimUnitStopKernel) VerifyStopped(ctx context.Context, t hostguard.Fenc
return evidence, nil
}
+func (k claimUnitStopKernel) VerifyUnjournaled(ctx context.Context, t hostguard.FenceTarget, inv StopInventory) (StopKernelEvidence, error) {
+ kernel, ok := k.Inner.(interface {
+ VerifyUnjournaled(context.Context, hostguard.FenceTarget, StopInventory) (StopKernelEvidence, error)
+ })
+ if !ok {
+ return StopKernelEvidence{}, ErrClaimStopUnavailable
+ }
+ return kernel.VerifyUnjournaled(ctx, t, inv)
+}
+
func (k claimUnitStopKernel) RecoverStopped(ctx context.Context, t hostguard.FenceTarget, in StopInventory) (StopKernelEvidence, DrainAcknowledgement, error) {
recovery, ok := k.Inner.(stopRecoveryKernel)
if !ok {
diff --git a/internal/host/claimowners.go b/internal/host/claimowners.go
index 29d7edee0..dc1c17b7c 100644
--- a/internal/host/claimowners.go
+++ b/internal/host/claimowners.go
@@ -12,7 +12,6 @@ import (
"sync"
"time"
- lss3 "github.com/benbjohnson/litestream/s3"
"github.com/carlosframework/platform/internal/activator"
"github.com/carlosframework/platform/internal/cgroup"
"github.com/carlosframework/platform/internal/genselect"
@@ -320,6 +319,7 @@ func (d ClaimOwnerDispatch) PublicationEvidence(ctx context.Context, o hostclaim
// acknowledged (INV-11), and publication evidence binds the exact marker
// version this process wrote and the exact root version it named.
type claimServingOwners struct {
+ emptyRecovery claimEmptyRecovery
HomeRegion string
Data genselect.Reader
Bucket, Region, Box string
@@ -438,7 +438,14 @@ func (s *claimServingOwners) Select(ctx context.Context, rt registry.Route, o ho
}
in := ClaimLaunchInputs{Unit: l.Unit, Roles: roles, Identity: l.Identity, Executable: l.Executable, ExecutableSHA256: l.ExecutableSHA256, Args: append([]string(nil), l.Args...), Env: append([]string(nil), l.Env...), DBPath: rt.DBPath, Version: l.Version, Staged: l.Staged, Limit: l.Limit,
Admission: hostclaim.Admission{ObservedSession: h.Session, ObservedGeneration: h.Generation, Box: s.Box, Mode: "cold", NewGeneration: generation, Region: s.Region}}
- if claimEmptyBootstrap(h) {
+ empty, err := s.emptyBootstrap(ctx, o)
+ if err != nil {
+ return ClaimLaunchInputs{}, err
+ }
+ if empty {
+ if len(h.Admissions) > 0 {
+ in.Admission.PredecessorRegion = h.Admissions[len(h.Admissions)-1].Region
+ }
in.Empty = true
return in, nil
}
@@ -487,24 +494,29 @@ func (s *claimServingOwners) EmptyAuthority(ctx context.Context, o hostclaim.Obs
return err
}
h := o.Claim.Holder
- if !in.Empty || in.Baseline != nil || len(h.Admissions) != 1 || h.WriterBox != s.Box {
- return hostbox.ErrChanged
- }
- a := h.Admissions[0]
- if a.Session != h.Session || a.Box != s.Box || a.Mode != "cold" || a.Generation != h.Generation || a.Generation != in.Admission.NewGeneration || a.Predecessor != "" || a.PredecessorRegion != "" || a.Region != s.Region {
+ if !in.Empty || in.Baseline != nil || len(h.Admissions) == 0 || h.WriterBox != s.Box {
return hostbox.ErrChanged
}
- versions, err := s.Data.ListVersions(ctx, s.Bucket, strings.TrimSuffix(h.Location, "/")+"/")
- if err != nil {
- return err
- }
- leaseKey := strings.TrimSuffix(h.Location, "/") + "/" + lss3.DefaultLeasePath
- for _, version := range versions {
- if version.Key != leaseKey {
+ n := len(h.Admissions)
+ a := h.Admissions[n-1]
+ predecessor, region := "", ""
+ if n > 1 {
+ if s.emptyRecovery == nil {
return hostbox.ErrChanged
}
+ yes, err := s.emptyRecovery(ctx, o, h.Admissions[:n-1])
+ if err != nil {
+ return err
+ }
+ if !yes {
+ return hostbox.ErrChanged
+ }
+ predecessor, region = h.Admissions[n-2].Generation, h.Admissions[n-2].Region
}
- return nil
+ if a.Session != h.Session || a.Box != s.Box || a.Mode != "cold" || a.Generation != h.Generation || a.Generation != in.Admission.NewGeneration || a.Predecessor != predecessor || a.PredecessorRegion != region || a.Region != s.Region {
+ return hostbox.ErrChanged
+ }
+ return s.emptyLocation(ctx, h.Location, h.Admissions[:n-1]...)
}
func claimPublishedKey(host string, session uint64) string {
@@ -633,8 +645,16 @@ func (s *claimServingOwners) admissionEvidence(ctx context.Context, o hostclaim.
in.Mode != "cold" || in.PredecessorCertified || in.KeepStamp || in.Coverage != "" || len(in.AdditionalWritesLost) != 0 || in.LossID != "" || !claimFreshGeneration(h, in.NewGeneration) {
return hostbox.ErrChanged
}
- if claimEmptyBootstrap(h) {
- if baseline != nil || in.RemoteMaxTXID != 0 || in.PredecessorRegion != "" {
+ empty, err := s.emptyBootstrap(ctx, o)
+ if err != nil {
+ return err
+ }
+ if empty {
+ region := ""
+ if len(h.Admissions) > 0 {
+ region = h.Admissions[len(h.Admissions)-1].Region
+ }
+ if baseline != nil || in.RemoteMaxTXID != 0 || in.PredecessorRegion != region {
return hostbox.ErrChanged
}
return nil
diff --git a/internal/host/claimservingpark.go b/internal/host/claimservingpark.go
index a1469ae54..dd65332a0 100644
--- a/internal/host/claimservingpark.go
+++ b/internal/host/claimservingpark.go
@@ -33,26 +33,36 @@ func servingGenerationParkDisposition(ctx context.Context, journal *hostbox.Laun
if t.Host != prior.Host || t.Box != prior.Box || t.Region != prior.Region || t.Incarnation != prior.Incarnation || t.Stamp != prior.Stamp || t.Session != prior.Session || t.Op != "" || t.Phase != "" || t.ObligationID != "" || verifyRetirementReceipt(ctx, receipt) != nil {
return servingParkRefuse, ErrClaimStopReceipt
}
- // Artifact-free retirement never replaces or refreshes an existing park.
- if !databasePresent {
- continue
- }
+ }
+ if databasePresent {
+ t := receipts[0].Target
records, err := journal.RootSessionLaunches(ctx, prior.Host, prior.Incarnation, prior.Stamp, prior.Session, prior.Box, t.Enrollment)
if err != nil {
return servingParkRefuse, err
}
- if len(records) != 1 || !hostbox.SameLaunchOwner(records[0].Target, t) {
+ if len(records) != 1 {
+ return servingParkRefuse, hostbox.ErrChanged
+ }
+ record := records[0]
+ matched := false
+ for _, receipt := range receipts {
+ matched = matched || hostbox.SameLaunchOwner(record.Target, receipt.Target)
+ }
+ if !matched {
return servingParkRefuse, hostbox.ErrChanged
}
seen := make(map[string]bool)
- for _, reservation := range records[0].Reservations {
+ for _, reservation := range record.Reservations {
if reservation.Role != "main" {
continue
}
- if reservation.ID == "" || seen[reservation.ID] || !hostbox.SameLaunchOwner(reservation.Target, t) {
+ if reservation.ID == "" || seen[reservation.ID] || !hostbox.SameLaunchOwner(reservation.Target, record.Target) {
return servingParkRefuse, hostbox.ErrChanged
}
seen[reservation.ID] = true
+ if reservation.NeverIntended && record.WritersClosed && reservation.State == hostbox.UnitSettled && reservation.PID == 0 && reservation.ProcessStart == "" && reservation.Group.InvocationID == "" && reservation.Group.Device == 0 && reservation.Group.Inode == 0 {
+ continue
+ }
if !hostbox.IsUnitReservation(reservation) || reservation.State != hostbox.UnitSettled || reservation.PID <= 0 || reservation.ProcessStart == "" || reservation.Group.InvocationID == "" || reservation.Group.Device == 0 || reservation.Group.Inode == 0 || reservation.Executable.Device == 0 || reservation.Executable.Inode == 0 {
return servingParkRefuse, hostbox.ErrChanged
}
diff --git a/internal/host/claimservingpark_test.go b/internal/host/claimservingpark_test.go
index e7bfa2733..5c469bfcf 100644
--- a/internal/host/claimservingpark_test.go
+++ b/internal/host/claimservingpark_test.go
@@ -4,6 +4,7 @@ package host
import (
"context"
+ "encoding/json"
"testing"
"github.com/carlosframework/platform/internal/hostbox"
@@ -12,7 +13,7 @@ import (
)
func TestServingParkRequiresRootObservedTenantStart(t *testing.T) {
- for _, state := range []string{"prelaunch", "intended", "started", "settled-unobserved", "settled-started", "tombstoned", "missing", "remote"} {
+ for _, state := range []string{"prelaunch", "intended", "started", "settled-unobserved", "settled-started", "tombstoned", "cancelled", "missing", "remote"} {
for _, database := range []bool{true, false} {
t.Run(state+map[bool]string{true: "/database", false: "/no-database"}[database], func(t *testing.T) {
f, _, _, in := servingRetirementFixture(t)
@@ -30,7 +31,7 @@ func TestServingParkRequiresRootObservedTenantStart(t *testing.T) {
if err := f.journal.IntendUnit(t.Context(), reservation); err != nil {
t.Fatal(err)
}
- case "tombstoned":
+ case "tombstoned", "cancelled":
if err := f.journal.TombstoneUnit(t.Context(), reservation); err != nil {
t.Fatal(err)
}
@@ -46,6 +47,11 @@ func TestServingParkRequiresRootObservedTenantStart(t *testing.T) {
t.Fatal(err)
}
}
+ if state == "cancelled" {
+ if err := f.journal.CloseWriters(t.Context(), target); err != nil {
+ t.Fatal(err)
+ }
+ }
journal := f.journal
remoteCalls := 0
if state == "remote" {
@@ -58,11 +64,24 @@ func TestServingParkRequiresRootObservedTenantStart(t *testing.T) {
if err != nil {
t.Fatal(err)
}
- got, err := servingGenerationParkDisposition(t.Context(), journal, prior, []hostguard.FenceReceipt{receipt}, database)
+ receipts := []hostguard.FenceReceipt{receipt}
+ if state == "cancelled" {
+ recovered := receipt
+ recovered.Target.ControllerID = "recovery-controller"
+ recovered.Target.FenceID = "recovery-fence"
+ var proof StopReceipt
+ if err := json.Unmarshal(receipt.Bytes, &proof); err != nil {
+ t.Fatal(err)
+ }
+ proof.ControllerID, proof.FenceID = recovered.Target.ControllerID, recovered.Target.FenceID
+ recovered.Bytes = mustMarshalStopReceipt(proof)
+ receipts = append([]hostguard.FenceReceipt{recovered}, receipts...)
+ }
+ got, err := servingGenerationParkDisposition(t.Context(), journal, prior, receipts, database)
want := servingParkRefuse
if !database {
want = servingParkArtifactFree
- } else if state == "prelaunch" {
+ } else if state == "prelaunch" || state == "cancelled" {
want = servingParkDiscard
} else if state == "settled-started" {
want = servingParkRetain
diff --git a/internal/host/claimservingretire.go b/internal/host/claimservingretire.go
index 123179a42..3cf1becc3 100644
--- a/internal/host/claimservingretire.go
+++ b/internal/host/claimservingretire.go
@@ -211,7 +211,7 @@ func (r *claimServingArtifactRetirement) BeforeLegacyInstall(ctx context.Context
if stamp != 0 {
return r.Recover(ctx, admitted, in)
}
- prior := servingLegacyPredecessor{Session: in.Admission.ObservedSession, Generation: in.Admission.ObservedGeneration, Empty: in.Empty}
+ prior := servingLegacyPredecessor{Session: in.Admission.ObservedSession, Generation: in.Admission.ObservedGeneration, Empty: in.Empty && in.Admission.ObservedSession == 0}
if (prior.Session == 0) != (prior.Generation == "") || prior.Session >= current.Session || (prior.Empty && prior.Session != 0) {
return hostbox.ErrChanged
}
diff --git a/internal/host/claimstop.go b/internal/host/claimstop.go
index d04ba5fe7..e61c6893a 100644
--- a/internal/host/claimstop.go
+++ b/internal/host/claimstop.go
@@ -5,6 +5,7 @@ import (
"context"
"crypto/ed25519"
"crypto/sha256"
+ "database/sql"
"encoding/hex"
"encoding/json"
"errors"
@@ -296,7 +297,7 @@ func (c ClaimStopCoordinator) FenceAndStop(ctx context.Context, req StopRequest)
// Root reads the tuple's own launch record and proves its controller
// dead, or the boot gone, before it kills, verifies and settles. A
// missing record or a live controller refuses; the fence stays pending.
- kernel, ack, err = recovery.RecoverStopped(ctx, target, inv)
+ kernel, ack, err = c.recoverKernel(ctx, recovery, target, inv)
if err != nil {
return StopResult{}, err
}
@@ -304,6 +305,9 @@ func (c ClaimStopCoordinator) FenceAndStop(ctx context.Context, req StopRequest)
(req.BootID == peer.BootID && req.ControllerID == peer.ControllerID && ack.ProcessStart == peer.ProcessStart) {
return StopResult{}, ErrClaimStopIdentity
}
+ } else if absent, absentErr := c.verifyUnjournaled(ctx, target, inv, sql.ErrNoRows); absentErr == nil {
+ kernel = absent
+ ack = DrainClaimAcknowledgement(fenceID, req.ControllerID, peer.ProcessStart, nil, hostbox.ProducerCompletion{})
} else {
ack, err = c.Drainer.DrainClaim(ctx, req, fenceID)
if err != nil {
@@ -313,6 +317,9 @@ func (c ClaimStopCoordinator) FenceAndStop(ctx context.Context, req StopRequest)
return StopResult{}, ErrClaimStopIdentity
}
kernel, err = c.Kernel.VerifyStopped(ctx, target, inv, ack)
+ if err != nil {
+ kernel, err = c.verifyUnjournaled(ctx, target, inv, err)
+ }
if err != nil {
return StopResult{}, err
}
diff --git a/internal/host/claimstop_kernel_linux.go b/internal/host/claimstop_kernel_linux.go
index a6a23d532..cd1b1762a 100644
--- a/internal/host/claimstop_kernel_linux.go
+++ b/internal/host/claimstop_kernel_linux.go
@@ -21,6 +21,7 @@ import (
)
type LinuxStopKernel struct {
+ rootAuthority bool
Journal hostbox.StopKernelJournalReader
ManagedRoots map[string]string
CgroupRoot, ProcRoot string
@@ -79,6 +80,15 @@ func (k LinuxStopKernel) VerifyStopped(ctx context.Context, t hostguard.FenceTar
if _, err := scanStopCgroups(ctx, k.CgroupRoot, j.Cgroups); err != nil {
return StopKernelEvidence{}, err
}
+ if raw, ok := k.Journal.(stopLaunchHosts); ok && k.rootAuthority {
+ records, err := raw.HostLaunches(ctx, t.Host)
+ if err != nil {
+ return StopKernelEvidence{}, err
+ }
+ if stopNeverIntended(records, t) {
+ groups = append(groups, "unlaunched:"+digestStop(t))
+ }
+ }
units := []string{}
return StopKernelEvidence{ObservedBootID: t.BootID, InventorySHA256: digestStop(inv), LocalJournalSHA256: digestStop(j), CgroupIdentities: groups, UnitInvocationIDs: units}, nil
}
@@ -271,8 +281,12 @@ func scanStopDescriptors(ctx context.Context, proc string, files []hostbox.StopF
return nil
}
-func newProductionStopKernel(j hostbox.StopKernelJournalReader, roots map[string]string) StopKernelVerifier {
- return LinuxStopKernel{Journal: j, ManagedRoots: roots, CgroupRoot: "/sys/fs/cgroup", ProcRoot: "/proc", AgentCgroup: agentUnitCgroup}
+func newProductionStopKernel(j hostbox.StopKernelJournalReader, roots map[string]string, rootAuthority ...bool) StopKernelVerifier {
+ agent := agentUnitCgroup
+ if len(rootAuthority) == 1 && rootAuthority[0] {
+ agent = rootStopAgentCgroup
+ }
+ return LinuxStopKernel{rootAuthority: len(rootAuthority) == 1 && rootAuthority[0], Journal: j, ManagedRoots: roots, CgroupRoot: "/sys/fs/cgroup", ProcRoot: "/proc", AgentCgroup: agent}
}
func (k LinuxStopKernel) RecoverStopped(ctx context.Context, t hostguard.FenceTarget, in StopInventory) (StopKernelEvidence, DrainAcknowledgement, error) {
diff --git a/internal/host/claimstop_kernel_other.go b/internal/host/claimstop_kernel_other.go
index fe9253c25..ad97162e7 100644
--- a/internal/host/claimstop_kernel_other.go
+++ b/internal/host/claimstop_kernel_other.go
@@ -4,6 +4,6 @@ package host
import "github.com/carlosframework/platform/internal/hostbox"
-func newProductionStopKernel(hostbox.StopKernelJournalReader, map[string]string) StopKernelVerifier {
+func newProductionStopKernel(hostbox.StopKernelJournalReader, map[string]string, ...bool) StopKernelVerifier {
return nil
}
diff --git a/internal/host/claimstop_production.go b/internal/host/claimstop_production.go
index 0cf3c582d..883ce1896 100644
--- a/internal/host/claimstop_production.go
+++ b/internal/host/claimstop_production.go
@@ -45,7 +45,7 @@ func NewProductionClaimStopHandler(c ClaimStopCoordinator, j hostbox.StopKernelJ
}
return hex.EncodeToString(b[:])
}
- c.Kernel = newProductionStopKernel(j, roots[0])
+ c.Kernel = newProductionStopKernel(j, roots[0], c.Guard.Root != nil)
authenticate := func(ctx context.Context, conn net.Conn) (StopPeer, error) {
ctx, cancel := context.WithTimeout(ctx, 3*time.Second)
defer cancel()
diff --git a/internal/host/claimstop_recovery.go b/internal/host/claimstop_recovery.go
index bca0d2ff3..3d73d8b0a 100644
--- a/internal/host/claimstop_recovery.go
+++ b/internal/host/claimstop_recovery.go
@@ -3,6 +3,7 @@ package host
import (
"context"
"crypto/ed25519"
+ "database/sql"
"encoding/json"
"errors"
"time"
@@ -20,6 +21,30 @@ type exactStopInventory interface {
ReadExactStopInventory(context.Context, hostguard.FenceTarget) (StopInventory, error)
}
+func (c ClaimStopCoordinator) verifyUnjournaled(ctx context.Context, target hostguard.FenceTarget, in StopInventory, cause error) (StopKernelEvidence, error) {
+ kernel, ok := c.Kernel.(interface {
+ VerifyUnjournaled(context.Context, hostguard.FenceTarget, StopInventory) (StopKernelEvidence, error)
+ })
+ if !ok || c.Guard.Root == nil || !errors.Is(cause, sql.ErrNoRows) {
+ return StopKernelEvidence{}, cause
+ }
+ if err := c.Guard.ResumeFence(ctx, target); err != nil {
+ return StopKernelEvidence{}, err
+ }
+ return kernel.VerifyUnjournaled(ctx, target, in)
+}
+
+func (c ClaimStopCoordinator) recoverKernel(ctx context.Context, kernel stopRecoveryKernel, target hostguard.FenceTarget, in StopInventory) (StopKernelEvidence, DrainAcknowledgement, error) {
+ evidence, ack, err := kernel.RecoverStopped(ctx, target, in)
+ if err != nil {
+ evidence, err = c.verifyUnjournaled(ctx, target, in, err)
+ if err == nil {
+ ack = DrainClaimAcknowledgement(target.FenceID, target.ControllerID, "unlaunched", nil, hostbox.ProducerCompletion{})
+ }
+ }
+ return evidence, ack, err
+}
+
func (c ClaimStopCoordinator) RecoverStopped(ctx context.Context, target hostguard.FenceTarget) (StopResult, error) {
kernel, ok := c.Kernel.(stopRecoveryKernel)
if !ok || c.Now == nil || c.NewID == nil || len(c.EvidenceSigner) != ed25519.PrivateKeySize || c.RequestLimit != hostbox.RequestLimit || !stopOperationTuple(target.Op, target.Phase, target.ObligationID) {
@@ -39,7 +64,7 @@ func (c ClaimStopCoordinator) RecoverStopped(ctx context.Context, target hostgua
if in.Host != target.Host || in.Incarnation != target.Incarnation || in.InstallationStamp != target.Stamp || in.Box != target.Box || in.Enrollment != target.Enrollment || in.DirectoryRevision != target.DirectoryRevision {
return StopResult{}, ErrClaimStopInventory
}
- evidence, ack, err := kernel.RecoverStopped(ctx, target, in)
+ evidence, ack, err := c.recoverKernel(ctx, kernel, target, in)
if err != nil {
return StopResult{}, err
}
diff --git a/internal/host/claimstop_unjournaled_linux_test.go b/internal/host/claimstop_unjournaled_linux_test.go
new file mode 100644
index 000000000..545cb1cf1
--- /dev/null
+++ b/internal/host/claimstop_unjournaled_linux_test.go
@@ -0,0 +1,193 @@
+//go:build linux
+
+package host
+
+import (
+ "context"
+ "database/sql"
+ "errors"
+ "os"
+ "path/filepath"
+ "strings"
+ "testing"
+
+ "github.com/carlosframework/platform/internal/cgroup"
+ "github.com/carlosframework/platform/internal/hostbox"
+ "github.com/carlosframework/platform/internal/hostguard"
+ "github.com/carlosframework/platform/internal/instancectl"
+)
+
+type changingLaunchHosts struct {
+ records []hostbox.LaunchInventory
+ next []hostbox.LaunchInventory
+ calls int
+}
+
+func (j *changingLaunchHosts) HostLaunches(context.Context, string) ([]hostbox.LaunchInventory, error) {
+ j.calls++
+ if j.calls > 1 && j.next != nil {
+ return j.next, nil
+ }
+ return j.records, nil
+}
+
+func TestPlan14UnjournaledStopAbsence(t *testing.T) {
+ for _, mode := range []string{"absent", "predecessor", "journal", "changed-journal", "descriptor", "containment", "isolated", "inventory"} {
+ t.Run(mode, func(t *testing.T) {
+ f := newWritersKernelFixture(t)
+ f.dropAgentHolds(t)
+ f.target.FenceID = "fence"
+ j := &changingLaunchHosts{}
+ switch mode {
+ case "predecessor":
+ target := f.target
+ target.Stamp--
+ j.records = []hostbox.LaunchInventory{{Target: target}}
+ case "journal":
+ j.records = []hostbox.LaunchInventory{{Target: f.target}}
+ case "changed-journal":
+ j.next = []hostbox.LaunchInventory{{Target: f.target}}
+ case "descriptor":
+ f.holdDB(t, 30, 3)
+ case "containment", "isolated":
+ dir := filepath.Join(f.agentCgroup, cgroup.ExecName(claimContainmentName(f.target)))
+ if mode == "isolated" {
+ // Use the same canonical host slice as the production observer.
+ dir = filepath.Join(f.cgroupRoot, filepath.Dir(instancectl.ClaimUnitControlGroup(f.target.Host)))
+ }
+ if err := os.MkdirAll(dir, 0700); err != nil {
+ t.Fatal(err)
+ }
+ if err := os.WriteFile(filepath.Join(dir, "cgroup.events"), []byte("populated 1\n"), 0600); err != nil {
+ t.Fatal(err)
+ }
+ case "inventory":
+ f.inv.Entries[0].Inode++
+ }
+ _, err := verifyNoLaunchAt(t.Context(), f.roots(), j, f.target, f.inv)
+ wantPass := mode == "absent" || mode == "predecessor"
+ if (err == nil) != wantPass {
+ t.Fatalf("%s: %v", mode, err)
+ }
+ })
+ }
+}
+
+type unjournaledKernel struct {
+ stopKernelFunc
+ calls int
+ evidence StopKernelEvidence
+}
+
+func (k *unjournaledKernel) VerifyUnjournaled(context.Context, hostguard.FenceTarget, StopInventory) (StopKernelEvidence, error) {
+ k.calls++
+ return k.evidence, nil
+}
+func TestPlan14UnjournaledStopRequiresRootFence(t *testing.T) {
+ c, _, _ := claimStopFixture(t)
+ k := &unjournaledKernel{}
+ c.Kernel = k
+ for _, cause := range []error{sql.ErrNoRows, os.ErrPermission, hostbox.ErrTypedInventory} {
+ if _, err := c.verifyUnjournaled(t.Context(), hostguard.FenceTarget{}, StopInventory{}, cause); !errors.Is(err, cause) || k.calls != 0 {
+ t.Fatal("legacy fallback", err)
+ }
+ }
+ c.Guard.Root = hostguard.NewRootAuthority(&rollbackMemory{files: map[string][]byte{}})
+ if _, err := c.verifyUnjournaled(t.Context(), hostguard.FenceTarget{Host: "shop.example", Incarnation: "inc-1", Stamp: 11}, StopInventory{}, sql.ErrNoRows); err == nil || k.calls != 0 {
+ t.Fatal("missing root fence accepted", err)
+ }
+}
+
+func (k *unjournaledKernel) RecoverStopped(context.Context, hostguard.FenceTarget, StopInventory) (StopKernelEvidence, DrainAcknowledgement, error) {
+ return StopKernelEvidence{}, DrainAcknowledgement{}, sql.ErrNoRows
+}
+
+func TestPlan14UnjournaledStopRecoveryReceipt(t *testing.T) {
+ c, req, _ := claimStopFixture(t)
+ c.Guard.Root = hostguard.NewRootAuthority(&rollbackMemory{files: map[string][]byte{}})
+ in := hostguard.RootInstallation{Host: req.Host, Incarnation: req.Incarnation, Stamp: req.InstallationStamp, Account: "bab", App: "app", Record: "record", Box: req.Box, Enrollment: req.Enrollment, Migration: 1}
+ if err := c.Guard.Root.RetainMigrationDeny(t.Context(), in, req.WriterSession); err != nil {
+ t.Fatal(err)
+ }
+ k := &unjournaledKernel{stopKernelFunc: func(context.Context, hostguard.FenceTarget, StopInventory, DrainAcknowledgement) (StopKernelEvidence, error) {
+ return StopKernelEvidence{}, sql.ErrNoRows
+ }}
+ c.Kernel = k
+ c.Inventory = exactEvidenceInventory{c.Inventory}
+ c.Drainer = stopDrainFunc(func(context.Context, StopRequest, string) (DrainAcknowledgement, error) {
+ t.Fatal("unjournaled stop tried to close nonexistent writers")
+ return DrainAcknowledgement{}, sql.ErrNoRows
+ })
+ // Fail the evidence check after exercising the fallback, leaving the durable fence.
+ if _, err := c.FenceAndStop(t.Context(), req); !errors.Is(err, ErrClaimStopReceipt) || k.calls != 1 {
+ t.Fatal("initial fallback", err, k.calls)
+ }
+ targets, err := c.Guard.PendingStopFences(t.Context())
+ if err != nil || len(targets) != 1 {
+ t.Fatal("pending fence", err, targets)
+ }
+ c.NewID = func() string { return "recovered" }
+ k.evidence = StopKernelEvidence{ObservedBootID: "boot-2", InventorySHA256: strings.Repeat("b", 64), LocalJournalSHA256: strings.Repeat("c", 64), CgroupIdentities: []string{"absent"}}
+ if _, err = c.RecoverStopped(t.Context(), targets[0]); err != nil || k.calls != 2 {
+ t.Fatal("recovery fallback", err, k.calls)
+ }
+ if pending, err := c.Guard.PendingStopFences(t.Context()); err != nil || len(pending) != 0 {
+ t.Fatal("recovery left a pending fence", err, pending)
+ }
+ if _, err := c.Guard.FenceReceiptFor(t.Context(), targets[0]); err != nil {
+ t.Fatal("recovery receipt", err)
+ }
+}
+
+func TestPlan14StoppedAgentContainment(t *testing.T) {
+ for _, tc := range []struct{ name, raw, want string }{
+ {"running", "Id=carlos-edge.service\nControlGroup=/carlos.slice/carlos-edge.service\nSlice=carlos.slice\nActiveState=active", "/sys/fs/cgroup/carlos.slice/carlos-edge.service"},
+ {"stopped", "Id=carlos-edge.service\nControlGroup=\nSlice=carlos.slice\nActiveState=inactive", "/sys/fs/cgroup/carlos.slice/carlos-edge.service"},
+ {"failed", "Id=carlos-edge.service\nControlGroup=\nSlice=carlos.slice\nActiveState=failed", "/sys/fs/cgroup/carlos.slice/carlos-edge.service"},
+ {"nested slice", "Id=carlos-edge.service\nControlGroup=\nSlice=tenant-carlos.slice\nActiveState=inactive", "/sys/fs/cgroup/tenant.slice/tenant-carlos.slice/carlos-edge.service"},
+ {"active without group", "Id=carlos-edge.service\nControlGroup=\nSlice=carlos.slice\nActiveState=active", ""},
+ {"wrong unit", "Id=other.service\nControlGroup=\nSlice=carlos.slice\nActiveState=inactive", ""},
+ {"missing slice", "Id=carlos-edge.service\nControlGroup=\nActiveState=inactive", ""},
+ {"traversal", "Id=carlos-edge.service\nControlGroup=\nSlice=../carlos.slice\nActiveState=inactive", ""},
+ {"duplicate", "Id=carlos-edge.service\nId=carlos-edge.service\nControlGroup=\nSlice=carlos.slice\nActiveState=inactive", ""},
+ } {
+ t.Run(tc.name, func(t *testing.T) {
+ got, err := rootStopAgentGroup([]byte(tc.raw))
+ if (err == nil) != (tc.want != "") || got != tc.want {
+ t.Fatal(got, err)
+ }
+ })
+ }
+}
+
+func TestPlan14NeverIntendedStopProof(t *testing.T) {
+ target := hostguard.FenceTarget{Host: "shop.example", Incarnation: "inc", Stamp: 1}
+ for _, mode := range []string{"absent", "tombstone", "intended", "started", "legacy", "open-writers", "other-record"} {
+ t.Run(mode, func(t *testing.T) {
+ in := hostbox.LaunchInventory{Target: target, WritersClosed: true, Groups: []hostbox.LaunchGroup{{RelativePath: "agent/parent"}}}
+ r := hostbox.LaunchReservation{NeverIntended: true, State: hostbox.UnitSettled, Group: hostbox.LaunchGroup{RelativePath: "host.service"}}
+ if mode != "absent" {
+ in.Reservations = []hostbox.LaunchReservation{r}
+ }
+ switch mode {
+ case "intended":
+ in.Reservations[0].NeverIntended = false
+ case "started":
+ in.Reservations[0].PID = 1
+ case "legacy":
+ in.Reservations[0].Group.RelativePath = "role-child"
+ case "open-writers":
+ in.WritersClosed = false
+ }
+ records := []hostbox.LaunchInventory{in}
+ if mode == "other-record" {
+ other := in
+ other.WritersClosed = false
+ records = append(records, other)
+ }
+ if got := stopNeverIntended(records, target); got != (mode == "absent" || mode == "tombstone") {
+ t.Fatal("unlaunched proof", mode, got)
+ }
+ })
+ }
+}
diff --git a/internal/host/claimstop_writers_linux.go b/internal/host/claimstop_writers_linux.go
index 4f0e47a9d..a9334e2b5 100644
--- a/internal/host/claimstop_writers_linux.go
+++ b/internal/host/claimstop_writers_linux.go
@@ -147,6 +147,34 @@ func verifyNoProcessAt(ctx context.Context, roots stopKernelRoots, launches stop
if launches == nil || t.Session != 0 || !filepath.IsAbs(roots.AgentCgroup) {
return StopKernelEvidence{}, ErrClaimStopUnavailable
}
+ return verifyNoLaunchAt(ctx, roots, launches, t, inv)
+}
+
+func (k LinuxStopKernel) VerifyUnjournaled(ctx context.Context, t hostguard.FenceTarget, inv StopInventory) (StopKernelEvidence, error) {
+ roots, err := k.realRoots()
+ if err != nil {
+ return StopKernelEvidence{}, err
+ }
+ launches, ok := k.Journal.(stopLaunchHosts)
+ if !ok || k.AgentCgroup == nil || t.Session == 0 || t.FenceID == "" {
+ return StopKernelEvidence{}, ErrClaimStopUnavailable
+ }
+ if roots.AgentCgroup, err = k.AgentCgroup(ctx); err != nil {
+ return StopKernelEvidence{}, err
+ }
+ boot, err := os.ReadFile(filepath.Join(roots.Proc, "sys/kernel/random/boot_id"))
+ if err != nil || strings.TrimSpace(string(boot)) == "" {
+ return StopKernelEvidence{}, errors.Join(ErrClaimStopIdentity, err)
+ }
+ evidence, err := verifyNoLaunchAt(ctx, roots, launches, t, inv)
+ evidence.ObservedBootID = strings.TrimSpace(string(boot))
+ if err == nil {
+ evidence.CgroupIdentities = append(evidence.CgroupIdentities, "unlaunched:"+digestStop(t))
+ }
+ return evidence, err
+}
+
+func verifyNoLaunchAt(ctx context.Context, roots stopKernelRoots, launches stopLaunchHosts, t hostguard.FenceTarget, inv StopInventory) (StopKernelEvidence, error) {
records, err := launches.HostLaunches(ctx, t.Host)
if err != nil {
return StopKernelEvidence{}, err
@@ -159,7 +187,11 @@ func verifyNoProcessAt(ctx context.Context, roots stopKernelRoots, launches stop
return StopKernelEvidence{}, ErrClaimStopInventory
}
}
- groups, err := noProcessGroups(ctx, roots, t.Host, records...)
+ containments := records
+ if t.Session != 0 {
+ containments = append(append([]hostbox.LaunchInventory(nil), records...), hostbox.LaunchInventory{Target: t})
+ }
+ groups, err := noProcessGroups(ctx, roots, t.Host, containments...)
if err != nil {
return StopKernelEvidence{}, err
}
@@ -182,7 +214,7 @@ func verifyNoProcessAt(ctx context.Context, roots stopKernelRoots, launches stop
if err != nil || !reflect.DeepEqual(records, again) {
return StopKernelEvidence{}, errors.Join(ErrClaimStopInventory, err)
}
- if again, err := noProcessGroups(ctx, roots, t.Host, records...); err != nil || !reflect.DeepEqual(again, groups) {
+ if again, err := noProcessGroups(ctx, roots, t.Host, containments...); err != nil || !reflect.DeepEqual(again, groups) {
return StopKernelEvidence{}, errors.Join(ErrClaimStopInventory, err)
}
journal := digestStop(records)
@@ -397,3 +429,76 @@ func noProcessGroups(ctx context.Context, roots stopKernelRoots, host string, re
}
return out, nil
}
+
+func rootStopAgentCgroup(ctx context.Context) (string, error) {
+ ctx, cancel := context.WithTimeout(ctx, 3*time.Second)
+ defer cancel()
+ raw, err := exec.CommandContext(ctx, "systemctl", "show", "-p", "ControlGroup,Id,Slice,ActiveState", "carlos-edge.service").Output()
+ if err != nil {
+ return "", err
+ }
+ return rootStopAgentGroup(raw)
+}
+
+func rootStopAgentGroup(raw []byte) (string, error) {
+ fields := map[string]string{}
+ for _, line := range strings.Split(strings.TrimSpace(string(raw)), "\n") {
+ key, value, ok := strings.Cut(line, "=")
+ if !ok {
+ return "", ErrClaimStopUnavailable
+ }
+ if _, exists := fields[key]; exists {
+ return "", ErrClaimStopUnavailable
+ }
+ fields[key] = value
+ }
+ if fields["Id"] != "carlos-edge.service" {
+ return "", ErrClaimStopUnavailable
+ }
+ rel := strings.TrimPrefix(fields["ControlGroup"], "/")
+ if rel == "" {
+ if fields["ActiveState"] != "inactive" && fields["ActiveState"] != "failed" {
+ return "", ErrClaimStopUnavailable
+ }
+ slice := fields["Slice"]
+ if !strings.HasSuffix(slice, ".slice") || !filepath.IsLocal(slice) || filepath.Base(slice) != slice {
+ return "", ErrClaimStopUnavailable
+ }
+ if slice != "-.slice" {
+ parts := strings.Split(strings.TrimSuffix(slice, ".slice"), "-")
+ for i, part := range parts {
+ if part == "" {
+ return "", ErrClaimStopUnavailable
+ }
+ rel = filepath.Join(rel, strings.Join(parts[:i+1], "-")+".slice")
+ }
+ }
+ rel = filepath.Join(rel, fields["Id"])
+ }
+ if !filepath.IsLocal(rel) || filepath.Clean(rel) != rel {
+ return "", ErrClaimStopUnavailable
+ }
+ return filepath.Join("/sys/fs/cgroup", rel), nil
+}
+
+func stopNeverIntended(records []hostbox.LaunchInventory, t hostguard.FenceTarget) bool {
+ found := false
+ for _, in := range records {
+ if in.Target.Host != t.Host {
+ return false
+ }
+ if in.Target.Incarnation != t.Incarnation || in.Target.Stamp != t.Stamp {
+ continue
+ }
+ if !in.WritersClosed || len(in.Groups) != 1 {
+ return false
+ }
+ found = true
+ for _, r := range in.Reservations {
+ if !hostbox.IsUnitReservation(r) || r.State != hostbox.UnitSettled || !r.NeverIntended || r.PID != 0 || r.ProcessStart != "" || r.Group.Device != 0 || r.Group.Inode != 0 || r.Group.InvocationID != "" {
+ return false
+ }
+ }
+ }
+ return found
+}
diff --git a/internal/host/movesfiles.go b/internal/host/movesfiles.go
index faf0ac08a..1dc698f76 100644
--- a/internal/host/movesfiles.go
+++ b/internal/host/movesfiles.go
@@ -55,7 +55,8 @@ import (
// Absence of a manifest, of a pin, or of a reader is never evidence: with
// Evidence nil, a chain without a manifest refuses every wake.
type movesFiles struct {
- Seed *filesSeedOperation
+ emptyRecovery claimEmptyRecovery
+ Seed *filesSeedOperation
// Files is this box's data bucket as a release.Store: current reads
// with the object's etag, create-only and compare-and-swap writes
// (the S3 store's Put with If-None-Match and If-Match). Every write
diff --git a/internal/host/plan14_lifecycle_crash_root_linux_test.go b/internal/host/plan14_lifecycle_crash_root_linux_test.go
index 78aa901ba..9190931e9 100644
--- a/internal/host/plan14_lifecycle_crash_root_linux_test.go
+++ b/internal/host/plan14_lifecycle_crash_root_linux_test.go
@@ -7,6 +7,7 @@ import (
"errors"
"os"
"os/exec"
+ "syscall"
"testing"
"time"
@@ -104,6 +105,10 @@ func lifecycleStartControl(t *testing.T, endpoint string) func() {
}
t.Cleanup(func() {
if cmd != nil {
+ if t.Failed() {
+ cmd.Process.Signal(syscall.SIGQUIT)
+ time.Sleep(100 * time.Millisecond)
+ }
cmd.Process.Kill()
cmd.Wait()
}
diff --git a/internal/host/plan14_lifecycle_reboot_root_linux_test.go b/internal/host/plan14_lifecycle_reboot_root_linux_test.go
index fabbc9fcf..286879248 100644
--- a/internal/host/plan14_lifecycle_reboot_root_linux_test.go
+++ b/internal/host/plan14_lifecycle_reboot_root_linux_test.go
@@ -49,7 +49,11 @@ func lifecycleRestoreBucket(t *testing.T, model *leasetest.VersionedBucket) {
if saved.Boot == "" || saved.Boot == strings.TrimSpace(string(boot)) {
t.Fatal("lifecycle runner did not reboot")
}
- sort.Slice(saved.Objects, func(i, j int) bool { return saved.Objects[i].Version.VersionID < saved.Objects[j].Version.VersionID })
+ sort.Slice(saved.Objects, func(i, j int) bool {
+ a, _ := strconv.Atoi(strings.TrimPrefix(saved.Objects[i].Version.VersionID, "v"))
+ b, _ := strconv.Atoi(strings.TrimPrefix(saved.Objects[j].Version.VersionID, "v"))
+ return a < b
+ })
next := 1
for _, object := range saved.Objects {
n, err := strconv.Atoi(strings.TrimPrefix(object.Version.VersionID, "v"))
diff --git a/internal/host/plan14_lifecycle_root_linux_test.go b/internal/host/plan14_lifecycle_root_linux_test.go
index 2ea82b32d..a1b22b8ad 100644
--- a/internal/host/plan14_lifecycle_root_linux_test.go
+++ b/internal/host/plan14_lifecycle_root_linux_test.go
@@ -155,7 +155,7 @@ func (p lifecycleOwners) OpenClaimOwners(ctx context.Context, store release.Stor
func lifecycleEnvironment(t *testing.T, endpoint string) {
t.Helper()
- for k, v := range map[string]string{"CARLOS_HOST_CLAIM": "1", "CARLOS_HOST_CLAIM_ORIGINS": "1", "CARLOS_MOVES_PHASE1": "", "CARLOS_REGION_HA": "", "CARLOS_DATA_DIR": "/data", "CARLOS_OPT_DIR": "/opt/carlos", "CARLOS_DEPLOYMENT_BUCKET": "", "CARLOS_DEPLOYMENT_DIR": lifecycleHome + "/deployment", "CARLOS_REGION": "ie", "AWS_ACCESS_KEY_ID": "test-key", "AWS_SECRET_ACCESS_KEY": "test-secret", "AWS_SESSION_TOKEN": "", "AWS_EC2_METADATA_DISABLED": "true", "AWS_CONFIG_FILE": "/nonexistent", "AWS_SHARED_CREDENTIALS_FILE": "/nonexistent", "CARLOS_S3_ENDPOINT": endpoint, "CARLOS_RELEASE_PUBKEY": base64.StdEncoding.EncodeToString(ed25519.NewKeyFromSeed(bytes.Repeat([]byte{1}, ed25519.SeedSize)).Public().(ed25519.PublicKey))} {
+ for k, v := range map[string]string{"CARLOS_HOST_CLAIM": "1", "CARLOS_HOST_CLAIM_ORIGINS": "1", "CARLOS_MOVES_PHASE1": "", "CARLOS_REGION_HA": "", "CARLOS_DATA_DIR": "/data", "CARLOS_OPT_DIR": "/opt/carlos", "CARLOS_DEPLOYMENT_BUCKET": "", "CARLOS_DEPLOYMENT_DIR": lifecycleHome + "/deployment", "CARLOS_REGION": "ie", "AWS_ACCESS_KEY_ID": "test-key", "AWS_SECRET_ACCESS_KEY": "test-secret", "AWS_SESSION_TOKEN": "", "AWS_EC2_METADATA_DISABLED": "true", "AWS_CONFIG_FILE": "/nonexistent", "AWS_SHARED_CREDENTIALS_FILE": "/nonexistent", "CARLOS_S3_ENDPOINT": endpoint, "AWS_ENDPOINT_URL_S3": endpoint, "CARLOS_RELEASE_PUBKEY": base64.StdEncoding.EncodeToString(ed25519.NewKeyFromSeed(bytes.Repeat([]byte{1}, ed25519.SeedSize)).Public().(ed25519.PublicKey))} {
t.Setenv(k, v)
}
}
@@ -228,7 +228,7 @@ func lifecycleRun(t *testing.T) {
lifecycleRestoreBucket(t, model)
}
handler, _ := leasetest.NewS3Server(model)
- server := httptest.NewServer(handler)
+ server := httptest.NewServer(lifecycleS3(handler, model))
t.Cleanup(server.Close)
lifecycleEnvironment(t, server.URL)
store := release.NewDirStore(lifecycleHome + "/deployment")
@@ -495,6 +495,21 @@ func lifecycleAgent(t *testing.T) {
t.Logf("completed cold wake %d", wake+1)
}
if os.Getenv("PLAN14_LIFECYCLE_PHASE") == "2" || os.Getenv("PLAN14_LIFECYCLE_PHASE") == "crash" {
+ parkDeadline := time.Now().Add(time.Minute)
+ for {
+ obs, err := hostclaim.Read(ctx, store, lifecycleHost)
+ lifecycleCheck(t, "pre-delete observation", err)
+ if len(obs.Claim.Obligations) == 0 {
+ break
+ }
+ if time.Now().After(parkDeadline) {
+ t.Fatal("pre-delete obligations did not drain", obs.Claim.Obligations)
+ }
+ if err := providers.Regional.(hostbox.RegionalWorker).Run(ctx); err != nil {
+ t.Log("pre-delete parks", err)
+ }
+ time.Sleep(100 * time.Millisecond)
+ }
lifecycleCheck(t, "request deletion", os.WriteFile("/run/carlos/lifecycle-delete", []byte("ready"), 0600))
deadline := time.Now().Add(time.Minute)
for time.Now().Before(deadline) {
@@ -522,7 +537,24 @@ func lifecycleWake(t *testing.T, ctx context.Context, runtime *activator.ClaimRu
observed, err := runtime.Read(ctx, lifecycleHost)
lifecycleCheck(t, "read for wake", err)
policy := activator.StartPolicy{MaxStarts: 5, Window: 30 * time.Second}
- lifecycleCheck(t, "wake prerequisites", runtime.Prerequisites(ctx, route, observed, policy))
+ prerequisiteDeadline := time.Now().Add(2 * time.Minute)
+ for {
+ e := runtime.Prerequisites(ctx, route, observed, policy)
+ if e == nil {
+ break
+ }
+ fresh, readErr := runtime.Read(ctx, lifecycleHost)
+ retry := errors.Is(e, hostbox.ErrChanged) && readErr == nil && fresh.ETag != observed.ETag
+ if errors.Is(e, hostclaim.ErrWrongState) && readErr == nil && fresh.Claim.Holder.WakeRetry != nil && fresh.Claim.Holder.WakeRetry.State == "open" {
+ retry = true
+ }
+ if !retry || time.Now().After(prerequisiteDeadline) {
+ t.Logf("prerequisite state: holder=%+v route=%+v", observed.Claim.Holder, route)
+ lifecycleCheck(t, "wake prerequisites", e)
+ }
+ observed = fresh
+ time.Sleep(100 * time.Millisecond)
+ }
admitted, err := runtime.Admit(ctx, route, observed, policy)
lifecycleCheck(t, "cold admission", err)
start, err := runtime.Prepare(ctx, route, admitted, policy)
@@ -533,8 +565,24 @@ func lifecycleWake(t *testing.T, ctx context.Context, runtime *activator.ClaimRu
launch, err := start.PrepareLaunch(ctx)
lifecycleCheck(t, "launch preparation", err)
cmd, _, settle, _, err := launch.Start(ctx, "main", start.CurrentAttempt(), start.Build, func() error { return ctx.Err() })
+ if settle != nil {
+ defer func() {
+ cleanup, cancel := context.WithTimeout(context.Background(), 30*time.Second)
+ defer cancel()
+ for {
+ stopErr := settle(cleanup)
+ if stopErr == nil {
+ return
+ }
+ if cleanup.Err() != nil {
+ t.Error("settle failed launch", stopErr)
+ return
+ }
+ time.Sleep(100 * time.Millisecond)
+ }
+ }()
+ }
lifecycleCheck(t, "root launch", err)
- defer settle(context.Background())
lifecycleCheck(t, "tenant ready", start.Ready(ctx, cmd))
prefix, err := hostbox.GenerationPrefix(*admitted.Claim.Holder)
lifecycleCheck(t, "generation prefix", err)
@@ -558,11 +606,11 @@ func lifecycleWake(t *testing.T, ctx context.Context, runtime *activator.ClaimRu
if response.StatusCode != 200 || string(raw) != "lifecycle tenant" {
t.Fatalf("tenant response %d %q", response.StatusCode, raw)
}
- lifecycleCheck(t, "stop tenant", settle(ctx))
+ lifecycleCheck(t, "stop tenant", lifecycleRetry(ctx, settle))
lifecycleCheck(t, "stop replication", rep.Close(ctx))
current, err = runtime.Read(ctx, lifecycleHost)
lifecycleCheck(t, "read stopped session", err)
- lifecycleCheck(t, "stop session", wake.Stop(ctx, current))
+ lifecycleCheck(t, "stop session", lifecycleRetry(ctx, func(ctx context.Context) error { return wake.Stop(ctx, current) }))
}
func lifecycleInstallUnits(t *testing.T) {
@@ -570,7 +618,7 @@ func lifecycleInstallUnits(t *testing.T) {
for _, name := range []string{"carlos.slice", "carlos-instances.slice", "data-instances.mount", "run-carlos-instances.mount", "run-carlos-logs.mount", "carlos-instance@.service"} {
raw, err := unitsFS.ReadFile("units/" + name)
lifecycleCheck(t, "unit template", err)
- path := filepath.Join("/run/systemd/system", name)
+ path := filepath.Join("/etc/systemd/system", name)
f, err := os.OpenFile(path, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0644)
lifecycleCheck(t, "exclusive unit", err)
_, err = f.Write(raw)
@@ -733,10 +781,14 @@ func TestPlan14LifecycleControlProcess(t *testing.T) {
wrapped, err := WithProductionClaimLaunch(ctx, root)
lifecycleCheck(t, "launch constructor", err)
rootHTTP := wrapped.(ClaimRootWithLaunch)
+ rootHTTP.Launch.Kernel = lifecycleKernelTrace{rootHTTP.Launch.Kernel, t}
+ rootHTTP.Launch.Units = lifecycleUnitsTrace{rootHTTP.Launch.Units, t}
begin := rootHTTP.Launch.generations.beginUpdate
rootHTTP.Launch.generations.beginUpdate = func(ctx context.Context, s *rootGenerationSource, b release.GenerationBinding) (func(string, error, func() error) error, func() error, error) {
lifecycleCrash(t, "reservation")
- return begin(ctx, s, b)
+ finish, close, err := begin(ctx, s, b)
+ t.Log("generation update opener", err)
+ return finish, close, err
}
t.Cleanup(func() { rootHTTP.Root.executionRuntime.Close(context.Background()) })
listener, err := net.Listen("unix", controlsock.Path)
@@ -745,7 +797,7 @@ func TestPlan14LifecycleControlProcess(t *testing.T) {
lifecycleCheck(t, "root socket group", os.Chown(controlsock.Path, 0, gid))
controlCtx, cancelControl := context.WithCancel(ctx)
defer cancelControl()
- control := &instancectl.Server{ClaimStop: rootHTTP, PeerOK: func(c net.Conn) error { _, err := rootHTTP.Root.Stop.Authenticate(ctx, c); return err }, Logf: t.Logf}
+ control := &instancectl.Server{ClaimStop: lifecycleControlTrace{rootHTTP, t}, PeerOK: func(c net.Conn) error { _, err := rootHTTP.Root.Stop.Authenticate(ctx, c); return err }, Logf: t.Logf}
done := make(chan error, 1)
go func() { done <- control.Serve(controlCtx, listener) }()
defer func() { cancelControl(); lifecycleCheck(t, "control shutdown", <-done) }()
@@ -757,6 +809,49 @@ func TestPlan14LifecycleControlProcess(t *testing.T) {
}
func lifecycleTraceWake(t *testing.T, w *ClaimWakeAssembly) {
+ if old := w.AfterAdmission; old != nil {
+ w.AfterAdmission = func(ctx context.Context, p, o hostclaim.Observed) error {
+ e := old(ctx, p, o)
+ t.Log("after admission", e)
+ return e
+ }
+ }
+ if old := w.Prelaunch; old != nil {
+ w.Prelaunch = func(ctx context.Context, o hostclaim.Observed, work hostwrite.Work, in ClaimLaunchInputs) (*activator.RoleLaunch, error) {
+ l, e := old(ctx, o, work, in)
+ t.Log("prelaunch", e)
+ return l, e
+ }
+ }
+
+ if dispatch, ok := w.Inputs.(ClaimOwnerDispatch); ok {
+ if serving, ok := dispatch.Serving.Inputs.(*claimServingOwners); ok && serving.emptyRecovery != nil {
+ check := serving.emptyRecovery
+ serving.emptyRecovery = func(ctx context.Context, o hostclaim.Observed, a []hostclaim.AdmissionRecord) (bool, error) {
+ yes, err := check(ctx, o, a)
+ t.Logf("empty recovery holder=%+v admissions=%+v yes=%v err=%v", o.Claim.Holder, a, yes, err)
+ return yes, err
+ }
+ if f, ok := serving.Sources.Files.(*movesFiles); ok {
+ f.emptyRecovery = serving.emptyRecovery
+ }
+ }
+ }
+ if old := w.ServingPrerequisites; old != nil {
+ w.ServingPrerequisites = func(ctx context.Context, rt registry.Route, o hostclaim.Observed) error {
+ e := old(ctx, rt, o)
+ t.Log("files prerequisites", e)
+ return e
+ }
+ }
+ if old := w.recoveryPrior; old != nil {
+ w.recoveryPrior = func(ctx context.Context, o hostclaim.Observed) (hostclaim.Observed, bool, error) {
+ p, b, e := old(ctx, o)
+ t.Logf("recovery prior=%+v pending=%v error=%v", p.Claim, b, e)
+ return p, b, e
+ }
+ }
+
recover := w.RecoverArtifacts
if recover != nil {
w.RecoverArtifacts = func(ctx context.Context, o hostclaim.Observed, in ClaimLaunchInputs) error {
@@ -790,3 +885,19 @@ func lifecycleTraceWake(t *testing.T, w *ClaimWakeAssembly) {
}
}
}
+
+func lifecycleRetry(ctx context.Context, fn func(context.Context) error) error {
+ bounded, cancel := context.WithTimeout(ctx, 30*time.Second)
+ defer cancel()
+ for {
+ err := fn(bounded)
+ if err == nil {
+ return nil
+ }
+ select {
+ case <-bounded.Done():
+ return errors.Join(err, bounded.Err())
+ case <-time.After(100 * time.Millisecond):
+ }
+ }
+}
diff --git a/internal/host/plan14_lifecycle_s3_root_linux_test.go b/internal/host/plan14_lifecycle_s3_root_linux_test.go
new file mode 100644
index 000000000..363f58df8
--- /dev/null
+++ b/internal/host/plan14_lifecycle_s3_root_linux_test.go
@@ -0,0 +1,94 @@
+//go:build linux && rootrunner
+
+package host
+
+import (
+ "context"
+ "encoding/xml"
+ "fmt"
+ "net/http"
+ "net/http/httptest"
+ "sort"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/aws/aws-sdk-go-v2/aws"
+ "github.com/aws/aws-sdk-go-v2/service/s3"
+ "github.com/carlosframework/platform/internal/lease/leasetest"
+)
+
+func TestPlan14LifecycleS3(t *testing.T) {
+ model := leasetest.NewVersionedBucket()
+ model.AddBucket("data")
+ handler, _ := leasetest.NewS3Server(model)
+ server := httptest.NewServer(lifecycleS3(handler, model))
+ defer server.Close()
+ lifecycleEnvironment(t, server.URL)
+ markers, err := newProductionClaimMarkers(t.Context(), "data", "eu-west-1")
+ if err != nil {
+ t.Fatal(err)
+ }
+ key := "instances/lifecycle/generations/one/generation.json"
+ id, err := markers.PutMarker(t.Context(), "data", key, []byte("marker"), "", func(context.Context) error { return nil })
+ if err != nil || id == "" {
+ t.Fatal(id, err)
+ }
+ listed, err := markers.(productionClaimMarkers).Client.ListObjectsV2(t.Context(), &s3.ListObjectsV2Input{Bucket: aws.String("data"), Prefix: aws.String("instances/")})
+ if err != nil || len(listed.Contents) != 1 || aws.ToString(listed.Contents[0].Key) != key {
+ t.Fatal(listed, err)
+ }
+}
+
+func lifecycleS3(next http.Handler, model *leasetest.VersionedBucket) http.Handler {
+ return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ q := r.URL.Query()
+ if r.Method != http.MethodGet || strings.Trim(r.URL.Path, "/") != "data" || q.Get("list-type") != "2" {
+ next.ServeHTTP(w, r)
+ return
+ }
+ versions, err := model.ListVersions(r.Context(), "data", q.Get("prefix"))
+ if err != nil {
+ http.Error(w, err.Error(), 500)
+ return
+ }
+ type object struct {
+ Key, LastModified, ETag string
+ Size int64
+ }
+ type common struct{ Prefix string }
+ result := struct {
+ XMLName xml.Name `xml:"ListBucketResult"`
+ Name, Prefix, Delimiter string
+ MaxKeys, KeyCount int
+ IsTruncated bool
+ Contents []object
+ CommonPrefixes []common
+ }{Name: "data", Prefix: q.Get("prefix"), Delimiter: q.Get("delimiter"), MaxKeys: 1000}
+ prefixes := map[string]bool{}
+ for _, v := range versions {
+ if !v.IsLatest || v.DeleteMarker {
+ continue
+ }
+ if result.Delimiter != "" {
+ rest := strings.TrimPrefix(v.Key, result.Prefix)
+ if at := strings.Index(rest, result.Delimiter); at >= 0 {
+ prefixes[result.Prefix+rest[:at+len(result.Delimiter)]] = true
+ continue
+ }
+ }
+ result.Contents = append(result.Contents, object{v.Key, v.LastModified.UTC().Format(time.RFC3339Nano), fmt.Sprintf("%q", v.ETag), v.Size})
+ }
+ for p := range prefixes {
+ result.CommonPrefixes = append(result.CommonPrefixes, common{p})
+ }
+ sort.Slice(result.CommonPrefixes, func(i, j int) bool { return result.CommonPrefixes[i].Prefix < result.CommonPrefixes[j].Prefix })
+ result.KeyCount = len(result.Contents) + len(result.CommonPrefixes)
+ if result.KeyCount > result.MaxKeys {
+ http.Error(w, "fixture listing exceeds one page", 500)
+ return
+ }
+ w.Header().Set("Content-Type", "application/xml")
+ xml.NewEncoder(w).Encode(result)
+ })
+}
diff --git a/internal/host/plan14_lifecycle_trace_root_linux_test.go b/internal/host/plan14_lifecycle_trace_root_linux_test.go
new file mode 100644
index 000000000..844a43fb2
--- /dev/null
+++ b/internal/host/plan14_lifecycle_trace_root_linux_test.go
@@ -0,0 +1,79 @@
+//go:build linux && rootrunner
+
+package host
+
+import (
+ "bytes"
+ "context"
+ "encoding/json"
+ "io"
+ "net/http"
+ "os/exec"
+ "testing"
+
+ "github.com/carlosframework/platform/internal/hostbox"
+ "github.com/carlosframework/platform/internal/instancectl"
+)
+
+type lifecycleControlTrace struct {
+ ClaimRootWithLaunch
+ t *testing.T
+}
+
+func (h lifecycleControlTrace) ServeHTTP(w http.ResponseWriter, r *http.Request) {
+ var q hostbox.RoleLaunchRequest
+ if r.URL.Path == "/claim/launch" {
+ raw, _ := io.ReadAll(r.Body)
+ r.Body = io.NopCloser(bytes.NewReader(raw))
+ json.Unmarshal(raw, &q)
+ }
+ h.ClaimRootWithLaunch.ServeHTTP(w, r)
+ if q.Op == hostbox.RoleLaunchStop && q.Reservation != nil {
+ in, err := h.Launch.Journal.Read(r.Context(), q.Reservation.Target)
+ h.t.Logf("stop record err=%v request=%+v record=%+v containment=%v", err, *q.Reservation, in, validatePersistedUnitContainment(r.Context(), in, h.Launch.Units))
+ }
+}
+
+type lifecycleKernelTrace struct {
+ ClaimLaunchKernel
+ t *testing.T
+}
+
+func (k lifecycleKernelTrace) Observe(s instancectl.ClaimUnitStarted) (ClaimLaunchObserved, error) {
+ got, err := k.ClaimLaunchKernel.Observe(s)
+ k.t.Logf("launch observation: started=%+v observed=%+v error=%v", s, got, err)
+ return got, err
+}
+func (k lifecycleKernelTrace) VerifyDigest(ctx context.Context, path string, id hostbox.LaunchFile, digest string) error {
+ err := k.ClaimLaunchKernel.VerifyDigest(ctx, path, id, digest)
+ k.t.Log("launch digest", err)
+ return err
+}
+
+type lifecycleUnitsTrace struct {
+ ClaimLaunchUnits
+ t *testing.T
+}
+
+func (u lifecycleUnitsTrace) Placement(host string) (string, string, error) {
+ return claimLaunchPlacement(u.ClaimLaunchUnits, host)
+}
+func (u lifecycleUnitsTrace) StopInvocation(ctx context.Context, host, inv string) error {
+ err := claimStopInvocation(ctx, u.ClaimLaunchUnits, host, inv)
+ u.t.Log("stop invocation", inv, err)
+ return err
+}
+func (u lifecycleUnitsTrace) Gone(ctx context.Context, host, inv string) error {
+ err := u.ClaimLaunchUnits.Gone(ctx, host, inv)
+ u.t.Log("gone", inv, err)
+ return err
+}
+func (u lifecycleUnitsTrace) StartReserved(ctx context.Context, in instancectl.ClaimUnit, l instancectl.Launch, p instancectl.ClaimExecPermit, check func() error) (instancectl.ClaimUnitStarted, error) {
+ got, err := u.ClaimLaunchUnits.StartReserved(ctx, in, l, p, func() error { e := check(); u.t.Log("unit start revalidation", e); return e })
+ u.t.Log("start reserved", got, err)
+ if err != nil {
+ raw, _ := exec.Command("journalctl", "-u", instancectl.UnitName(instancectl.RoleRun, in.Host), "--no-pager", "-n", "60").CombinedOutput()
+ u.t.Log(string(raw))
+ }
+ return got, err
+}
diff --git a/internal/host/rootrestart_requests_source.go b/internal/host/rootrestart_requests_source.go
index ebe1edd5f..52c62e09f 100644
--- a/internal/host/rootrestart_requests_source.go
+++ b/internal/host/rootrestart_requests_source.go
@@ -35,8 +35,25 @@ func (s rootRestartRequestSource) installation(ctx context.Context, host string)
if err != nil {
return hostguard.RootInstallation{}, err
}
+ return s.checkInstallation(v.Installation)
+}
+
+func (s rootRestartRequestSource) launchInstallation(ctx context.Context, host string, b release.GenerationBinding) (hostguard.RootInstallation, error) {
+ if host != b.Binding.Host || host != b.Target.Host {
+ return hostguard.RootInstallation{}, os.ErrPermission
+ }
+ in, err := hostguard.NewRootAuthority(s.state).Installation(ctx, host, hostguard.Captured{Incarnation: b.Target.Incarnation, Stamp: b.Target.Stamp})
+ if err != nil {
+ return in, err
+ }
+ if rootHistoryInstallation(in) != b.Installation || in.Account != b.Binding.Account || in.App != b.Binding.App || in.Record != b.Record || in.Box != b.Target.Box || in.Enrollment != b.Target.Enrollment {
+ return in, os.ErrPermission
+ }
+ return s.checkInstallation(in)
+}
+
+func (s rootRestartRequestSource) checkInstallation(in hostguard.RootInstallation) (hostguard.RootInstallation, error) {
c, err := s.census()
- in := v.Installation
if err != nil {
return in, err
}
diff --git a/internal/host/rootupdates.go b/internal/host/rootupdates.go
index b60136f51..63494428a 100644
--- a/internal/host/rootupdates.go
+++ b/internal/host/rootupdates.go
@@ -14,6 +14,7 @@ import (
"time"
"github.com/carlosframework/platform/internal/hostclaim"
+ "github.com/carlosframework/platform/internal/hostguard"
"github.com/carlosframework/platform/internal/provision"
"github.com/carlosframework/platform/internal/registry"
"github.com/carlosframework/platform/internal/release"
@@ -40,6 +41,7 @@ type rootServedBuild struct {
type rootUpdateSource struct {
enforceFailures bool
+ heldGeneration *release.GenerationBinding
legacy []byte
ready func(registry.Route) error
@@ -75,6 +77,10 @@ func rootFailureLegacy(data string) ([]byte, error) {
}
func productionRootUpdates(dd string, probes ...func() (rootparticipation.Ledger, bool, error)) release.RootUpdateOpener {
+ return productionRootUpdatesHeld(dd, nil, probes...)
+}
+
+func productionRootUpdatesHeld(dd string, held *release.GenerationBinding, probes ...func() (rootparticipation.Ledger, bool, error)) release.RootUpdateOpener {
return func(ctx context.Context, store release.Store, pub ed25519.PublicKey) (*release.RootUpdateInputs, error) {
probe := rootparticipation.OpenForUpdate
if len(probes) > 1 || len(probes) == 1 && probes[0] == nil {
@@ -123,13 +129,20 @@ func productionRootUpdates(dd string, probes ...func() (rootparticipation.Ledger
observer := productionProofObserver(release.OptDir())
observer.resolve = binding.resolve
p := rootUpdateSource{legacy: legacy, ready: func(rt registry.Route) error { return rootUpdateReady(ctx, rt) }, publish: publishRootRetry, source: rootRestartRequestSource{state: fences, store: store, census: func() (rootparticipation.Census, error) { return c, nil }}, binding: binding, state: state, book: book, store: store, pub: pub, data: dd, observe: func(rt registry.Route) (liveBuild, error) { return observer.observe(rt, 0) }}
+ p.heldGeneration = held
return &release.RootUpdateInputs{Resolve: p.resolve, Close: func() error { return errors.Join(fences.Close(), state.Close()) }}, nil
}
}
func (p rootUpdateSource) resolve(ctx context.Context, host string) (release.RootUpdateInput, error) {
var out release.RootUpdateInput
- in, err := p.source.installation(ctx, host)
+ var in hostguard.RootInstallation
+ var err error
+ if p.heldGeneration != nil {
+ in, err = p.source.launchInstallation(ctx, host, *p.heldGeneration)
+ } else {
+ in, err = p.source.installation(ctx, host)
+ }
if err != nil {
return out, err
}
diff --git a/internal/host/rootupdates_launch_test.go b/internal/host/rootupdates_launch_test.go
new file mode 100644
index 000000000..73eff4a7a
--- /dev/null
+++ b/internal/host/rootupdates_launch_test.go
@@ -0,0 +1,63 @@
+package host
+
+import (
+ "context"
+ "os"
+ "testing"
+
+ "github.com/carlosframework/platform/internal/hostguard"
+ "github.com/carlosframework/platform/internal/release"
+ "github.com/carlosframework/platform/internal/rootparticipation"
+ "github.com/carlosframework/platform/internal/rootstate"
+)
+
+type heldHostStore struct {
+ rootstate.Store
+ locks int
+}
+
+func (s *heldHostStore) Lock(context.Context, string) (func(), error) {
+ s.locks++
+ return nil, os.ErrPermission
+}
+
+func TestPlan14LaunchSettingsUnderHostGuard(t *testing.T) {
+ disk := &rollbackMemory{files: map[string][]byte{}}
+ in := hostguard.RootInstallation{Host: "shop.example", Incarnation: "inc", Stamp: 2, Account: "bab", App: "app", Record: "record", Box: "box", Enrollment: "en", Migration: 1, InventoryRevision: 1}
+ if err := hostguard.NewRootAuthority(disk).RetainMigrationDeny(t.Context(), in, 1); err != nil {
+ t.Fatal(err)
+ }
+ held := &heldHostStore{Store: disk}
+ source := rootRestartRequestSource{state: held, census: func() (rootparticipation.Census, error) {
+ return rootparticipation.Census{Complete: true, Box: "box", Enrollment: "en", Migration: 1}, nil
+ }}
+ b := release.GenerationBinding{Binding: release.HistoryBinding{Host: in.Host, Account: in.Account, App: in.App}, Target: release.GenerationOwner{Host: in.Host, Incarnation: in.Incarnation, Stamp: in.Stamp, Box: in.Box, Enrollment: in.Enrollment}, Installation: rootHistoryInstallation(in), Record: in.Record}
+ if got, err := source.launchInstallation(t.Context(), in.Host, b); err != nil || got.Host != in.Host || held.locks != 0 {
+ t.Fatal("nested lock or wrong installation", err, held.locks)
+ }
+ for _, field := range []string{"host", "stamp", "installation", "account", "record", "box", "enrollment"} {
+ bad := b
+ switch field {
+ case "host":
+ bad.Target.Host = "other.example"
+ case "stamp":
+ bad.Target.Stamp++
+ case "installation":
+ bad.Installation = "foreign"
+ case "account":
+ bad.Binding.Account = "foreign"
+ case "record":
+ bad.Record = "foreign"
+ case "box":
+ bad.Target.Box = "foreign"
+ case "enrollment":
+ bad.Target.Enrollment = "foreign"
+ }
+ if _, err := source.launchInstallation(t.Context(), in.Host, bad); err == nil {
+ t.Fatal("accepted changed", field)
+ }
+ }
+ if _, err := source.installation(t.Context(), in.Host); err == nil || held.locks == 0 {
+ t.Fatal("ordinary lookup stopped taking the host lock", err)
+ }
+}
diff --git a/internal/host/rootupdates_runtime.go b/internal/host/rootupdates_runtime.go
index b335f2509..98c530817 100644
--- a/internal/host/rootupdates_runtime.go
+++ b/internal/host/rootupdates_runtime.go
@@ -196,7 +196,7 @@ func beginRootGenerationUpdate(ctx context.Context, s *rootGenerationSource, b r
if err != nil {
return nil, nil, err
}
- in, err := productionRootUpdates(dataDir())(ctx, s.store, pub)
+ in, err := productionRootUpdatesHeld(dataDir(), &b)(ctx, s.store, pub)
if err != nil {
return nil, nil, err
}
diff --git a/internal/hostbox/launchjournal.go b/internal/hostbox/launchjournal.go
index 217a9eeaf..4cd4a06c8 100644
--- a/internal/hostbox/launchjournal.go
+++ b/internal/hostbox/launchjournal.go
@@ -20,6 +20,7 @@ type LaunchGroup struct {
InvocationID string
}
type LaunchReservation struct {
+ NeverIntended bool `json:",omitempty"`
ID, Role, InvocationID, Inputs, State, ProcessStart string
Target hostguard.FenceTarget
Group LaunchGroup
@@ -252,7 +253,7 @@ func (j *LaunchJournal) Reserve(ctx context.Context, r LaunchReservation) error
return err
}
- if r.ID == "" || r.Role == "" || r.InvocationID == "" || r.Inputs == "" || r.Group.RelativePath == "" || r.Group.Device == 0 || r.Group.Inode == 0 || r.Executable.Device == 0 || r.Executable.Inode == 0 || r.State != "" || r.PID != 0 || r.Group.InvocationID != r.InvocationID {
+ if r.NeverIntended || r.ID == "" || r.Role == "" || r.InvocationID == "" || r.Inputs == "" || r.Group.RelativePath == "" || r.Group.Device == 0 || r.Group.Inode == 0 || r.Executable.Device == 0 || r.Executable.Inode == 0 || r.State != "" || r.PID != 0 || r.Group.InvocationID != r.InvocationID {
return ErrUnsupported
}
return j.mutate(ctx, r.Target, func(in *LaunchInventory) error {
diff --git a/internal/hostbox/roleintention.go b/internal/hostbox/roleintention.go
index b606f1d20..62c261b08 100644
--- a/internal/hostbox/roleintention.go
+++ b/internal/hostbox/roleintention.go
@@ -18,7 +18,7 @@ func (j *LaunchJournal) IntendRole(ctx context.Context, r LaunchReservation) err
return err
}
- if r.ID == "" || r.Role == "" || r.InvocationID == "" || r.Inputs == "" || r.State != "" || r.PID != 0 || r.ProcessStart != "" || r.Group.Device != 0 || r.Group.Inode != 0 || r.Group.InvocationID != r.InvocationID || r.Executable.Device == 0 || r.Executable.Inode == 0 || r.Parent.Device == 0 || r.Parent.Inode == 0 || !managedName(r.Parent.RelativePath) || path.Dir(r.Group.RelativePath) != r.Parent.RelativePath || !strings.HasPrefix(path.Base(r.Group.RelativePath), "role-") || !managedName(r.Group.RelativePath) {
+ if r.NeverIntended || r.ID == "" || r.Role == "" || r.InvocationID == "" || r.Inputs == "" || r.State != "" || r.PID != 0 || r.ProcessStart != "" || r.Group.Device != 0 || r.Group.Inode != 0 || r.Group.InvocationID != r.InvocationID || r.Executable.Device == 0 || r.Executable.Inode == 0 || r.Parent.Device == 0 || r.Parent.Inode == 0 || !managedName(r.Parent.RelativePath) || path.Dir(r.Group.RelativePath) != r.Parent.RelativePath || !strings.HasPrefix(path.Base(r.Group.RelativePath), "role-") || !managedName(r.Group.RelativePath) {
return ErrUnsupported
}
return j.mutate(ctx, r.Target, func(in *LaunchInventory) error {
diff --git a/internal/hostbox/rolelaunch.go b/internal/hostbox/rolelaunch.go
index b9615be8c..9f3f6293c 100644
--- a/internal/hostbox/rolelaunch.go
+++ b/internal/hostbox/rolelaunch.go
@@ -95,7 +95,7 @@ func UnitGroupPath(p string) bool {
// a pinned build and a complete owner tuple, with the unit's cgroup named but
// not identified, and nothing root alone records.
func unitReservationShape(r LaunchReservation) bool {
- return r.ID != "" && r.Role == "main" && r.InvocationID != "" && r.Inputs != "" && r.State == "" && r.PID == 0 && r.ProcessStart == "" &&
+ return !r.NeverIntended && r.ID != "" && r.Role == "main" && r.InvocationID != "" && r.Inputs != "" && r.State == "" && r.PID == 0 && r.ProcessStart == "" &&
r.Parent == (LaunchGroup{}) && r.Executable.Device != 0 && r.Executable.Inode != 0 && validLaunchOwner(r.Target) && component(r.Target.Host) &&
UnitGroupPath(r.Group.RelativePath) && r.Group.Device == 0 && r.Group.Inode == 0 && r.Group.InvocationID == ""
}
@@ -232,6 +232,7 @@ func (j *LaunchJournal) TombstoneUnit(ctx context.Context, r LaunchReservation)
}
}
r.State = UnitSettled
+ r.NeverIntended = true
in.Reservations = append(in.Reservations, r)
return nil
})
diff --git a/internal/hostbox/rolelaunch_test.go b/internal/hostbox/rolelaunch_test.go
index 89b5b3324..ad9d7fcde 100644
--- a/internal/hostbox/rolelaunch_test.go
+++ b/internal/hostbox/rolelaunch_test.go
@@ -181,7 +181,7 @@ func TestUnitTombstoneBurnsAnUnseenReservation(t *testing.T) {
t.Fatalf("start after its stop accepted: %v", err)
}
got, err := j.UnitReservation(ctx, r.Target, r.ID)
- if err != nil || got.State != UnitSettled || got.PID != 0 {
+ if err != nil || got.State != UnitSettled || got.PID != 0 || !got.NeverIntended {
t.Fatalf("tombstone=%+v %v", got, err)
}
}
@@ -208,3 +208,20 @@ func TestUnitJournalRefusesRemoteAndMissingRecords(t *testing.T) {
t.Fatal("intention without the prelaunch record")
}
}
+
+func TestPlan14NeverIntendedIsRootOnly(t *testing.T) {
+ j, r := roleLaunchFixture(t)
+ r.NeverIntended = true
+ if unitReservationShape(r) {
+ t.Fatal("agent set root cancellation proof")
+ }
+ if err := j.TombstoneUnit(t.Context(), r); err == nil {
+ t.Fatal("forged cancellation proof accepted")
+ }
+ if err := j.IntendRole(t.Context(), r); err == nil {
+ t.Fatal("forged legacy intention accepted")
+ }
+ if err := j.Reserve(t.Context(), r); err == nil {
+ t.Fatal("forged legacy reservation accepted")
+ }
+}
diff --git a/internal/hostguard/coldhandover.go b/internal/hostguard/coldhandover.go
index 8919d1265..b0716d536 100644
--- a/internal/hostguard/coldhandover.go
+++ b/internal/hostguard/coldhandover.go
@@ -118,6 +118,18 @@ func (g Guard) CompletedColdPredecessor(ctx context.Context, host string, captur
if err != nil {
return err
}
+ if r.root && r.ActiveFence != nil {
+ t := *r.ActiveFence
+ if t.Host == host && t.Incarnation == captured.Incarnation && t.Stamp == captured.Stamp && t.Session == session && t.Op == op && t.Phase == phase {
+ for _, receipt := range r.FenceJournal {
+ if receipt.Target == t {
+ found = receipt
+ return nil
+ }
+ }
+ return ErrNoFenceReceipt
+ }
+ }
for _, receipt := range r.FenceJournal {
t := receipt.Target
if t.Host == host && t.Incarnation == captured.Incarnation && t.Stamp == captured.Stamp && t.Session == session && t.Op == op && t.Phase == phase {
diff --git a/internal/hostguard/coldpredecessor_root_test.go b/internal/hostguard/coldpredecessor_root_test.go
new file mode 100644
index 000000000..7c94b5dbc
--- /dev/null
+++ b/internal/hostguard/coldpredecessor_root_test.go
@@ -0,0 +1,44 @@
+//go:build unix
+
+package hostguard
+
+import (
+ "context"
+ "testing"
+ "time"
+)
+
+func TestPlan14ColdPredecessorAfterRepeatedRootStop(t *testing.T) {
+ f, target, captured := fenceFixture(t)
+ f.g.Root = NewRootAuthority(&rootTestStore{dir: t.TempDir()})
+ f.obs.Claim.Holder.Account, f.obs.Claim.Holder.App = "bab", "app"
+ in := RootInstallation{Host: target.Host, Incarnation: target.Incarnation, Stamp: target.Stamp, Account: "bab", App: "app", Record: "record", Box: target.Box, Enrollment: target.Enrollment, Migration: 1}
+ if err := f.g.EstablishRoot(t.Context(), in, allow); err != nil {
+ t.Fatal(err)
+ }
+ verify := func(context.Context, FenceTarget, []byte, time.Time) error { return nil }
+ for i := 0; i < 2; i++ {
+ if i == 1 {
+ target.ControllerID = "replacement"
+ target.FenceID = "replacement-fence"
+ }
+ if _, err := f.g.BeginFence(t.Context(), target); err != nil {
+ t.Fatal(err)
+ }
+ if i == 1 {
+ if _, err := f.g.CompletedColdPredecessor(t.Context(), target.Host, captured, target.Session, target.Op, target.Phase); err == nil {
+ t.Fatal("pending stop accepted")
+ }
+ }
+ if err := f.g.CompleteFence(t.Context(), target, []byte(target.FenceID), time.Now(), verify); err != nil {
+ t.Fatal(err)
+ }
+ }
+ r, err := f.g.CompletedColdPredecessor(t.Context(), target.Host, captured, target.Session, target.Op, target.Phase)
+ if err != nil || r.Target != target {
+ t.Fatal("current completed fence not selected", r.Target, err)
+ }
+ if _, err := f.g.CompletedColdPredecessor(t.Context(), target.Host, captured, target.Session+1, target.Op, target.Phase); err == nil {
+ t.Fatal("foreign session accepted")
+ }
+}
diff --git a/internal/hostguard/flock_unix.go b/internal/hostguard/flock_unix.go
index 78a251a77..4ff5d4ece 100644
--- a/internal/hostguard/flock_unix.go
+++ b/internal/hostguard/flock_unix.go
@@ -58,6 +58,11 @@ func acquire(ctx context.Context, dir, host string, verifyOwner ...bool) (func()
if unsafe(st) {
return fail(ErrUnsafeFile)
}
+ if created && len(verifyOwner) == 1 && verifyOwner[0] && os.Geteuid() == 0 {
+ if err := unix.Fchown(fd, -1, int(parent.Gid)); err != nil {
+ return fail(err)
+ }
+ }
if created || st.Mode&07777 != 0660 {
if err := unix.Fchmod(fd, 0660); err != nil {
return fail(err)
diff --git a/internal/provision/plan14_p3_census_test.go b/internal/provision/plan14_p3_census_test.go
index 710c429a1..72edd0b0b 100644
--- a/internal/provision/plan14_p3_census_test.go
+++ b/internal/provision/plan14_p3_census_test.go
@@ -51,7 +51,7 @@ func plan14P3WriteCensusInventory() []censusEntry {
censusEntry{"internal/host/proofnamed.go", "proofBindingSource.record", "key-site", "key:RecordKey", "97f1501b9543e36c6a231170425c5c79c7c22489500aadf119e064abc1abde93"},
censusEntry{"internal/host/rootinventory.go", "ClaimRootHTTP.commitRootInventoryPending", "key-site", "key:RecordKey", "18fcea255a124e5c970c32d943c960836602dd12d9f00669583d8b4bd72fe7e6"},
censusEntry{"internal/host/rootgenerations_launch.go", "rootGenerationSource.resolve", "key-site", "key:RecordKey", "3cd7c8132e9c451c983392f4d1cc676ddc14b761c90dec3e631d54ae800f9951"},
- censusEntry{"internal/host/rootupdates.go", "rootUpdateSource.resolve", "key-site", "key:RecordKey", "8b7160d5b5cc1510eea7bbb3d9febe310df695337cfbc2b5e2ada7e9857ea83f"},
+ censusEntry{"internal/host/rootupdates.go", "rootUpdateSource.resolve", "key-site", "key:RecordKey", "6c5c447d2abd518a941979dcd1748ee028f3b555ce077fad849c02be0c77c2fa"},
censusEntry{"internal/host/rootmovesfloor.go", "rootMovesFloors.refresh", "key-site", "key:writeRecord", "cb70a1eefbb4f8f243090270f23141c9c9a773ee278af002c79a6c3ef6b7c3b7"},
censusEntry{"internal/host/rootmovesfloor.go", "rootMovesFloors.persist", "key-site", "key:writeRecord", "2c1e0843c5865a31c0fab011d7d4825963bbb5a301727f37f3b1b6c3519eba5e"},
censusEntry{"internal/host/rootmovesfloor.go", "rootMovesFloors.writeRecord", "key-site", "key:writeRecord", "a8a51bdd73f8ab1447edf2241d19a41ba1bcb0084ef045d370e5282aca7e9e66"},
diff --git a/internal/provision/testdata/plan14-root-reads-p3.tsv b/internal/provision/testdata/plan14-root-reads-p3.tsv
index 2908e8c96..a8a64020c 100644
--- a/internal/provision/testdata/plan14-root-reads-p3.tsv
+++ b/internal/provision/testdata/plan14-root-reads-p3.tsv
@@ -2,7 +2,7 @@
# Earlier overlays and frozen legacy fingerprints are unchanged.
internal/console/boxselect.go Server.selectMoveBoxes authority F,J4 rev3=PROPOSED-p3;identity=carlos console serve, console.service uid carlos-console;basis=applies accepted/generation refusal only to advertised root-floor participants; legacy selection retains main exactly d51a6a29a190dd36e896567adf4e0e0e1c8c9a08ff909cfbedd7732c2321ae78
internal/console/seedfloor.go Server.seedRegionReady authority F,J4 rev3=PROPOSED-p3;identity=carlos console serve uid carlos-console;basis=requires accepted seeds generation only from root-floor participants; mixed fleet nonparticipants retain exact legacy readiness d116ff2f72a1386a3a4120c69fea671cd3d43af264a20a1b8e98b5c61f11efb2
-internal/host/claimconstruction.go OpenProductionClaimAssembly authority G,J4 rev3=PROPOSED-p3;run=13;identity=agent uid carlos; adopt and instance-control uid 0;basis=checks registry and independently read canonical records before construction; root also checks retained unit state and settles retained pending inventory before full launch-inventory verification file:Read@10442,file:Read@4684,file:ReadFile@4228,registry:Host@10469,whole:Route@10557,whole:Route@11306 190aa844e443768c8dc4557289e89e5d8106a638a8dec759bbe0b313802a4c3c
+internal/host/claimconstruction.go OpenProductionClaimAssembly authority C,E3,J4 rev3=PROPOSED-p3;run=20;identity=root controller and service UID using authenticated root projection;basis=empty recovery requires retained root unlaunched stop receipts for every prior cold admission of the exact installation and local box; checks all object versions, allowing only lease and preparatory platform files under the root proof; normal baseline and no-census behavior retained file:Read@10604,file:Read@4684,file:ReadFile@4228,registry:Host@10631,whole:Route@10719,whole:Route@11468 1e572a7ebaad420e9ac5bd7d5af2bb5b6e300362c909a1965beaf5bbbd17142f
internal/host/claimjournal_migration.go RunSystemJournalMigrate authority G,J4 rev3=PROPOSED-p3;identity=carlos system journal-migrate, offline operator CLI, uid 0;basis=explicit offline root operator entry requests full G2 bootstrap after verified claim owner setup 4f204d8a97620bfb22430a94e236d6c5b14a5265b35f68e21fb2ad34903ae0f3
internal/host/claimjournal_migration.go migrateClaimJournalsWith authority G,J4 rev3=PROPOSED-p3;run=13;identity=offline migration uid 0;basis=checks canonical cohort eligibility before opening migration owners and journals b66afddd13e0f346ddecb481e88e58d3ad4a40b703f9f8199bdb6641fda6f54e
internal/host/guardarm.go RunSystemGuardArm authority F,J4 rev3=PROPOSED-p3;identity=carlos system guard-arm, offline CLI uid 0;basis=root guard-arm checks retained census floor before installing guard; no census retains main file:OpenFile@565,file:ReadFile@1142,file:ReadFile@1756,file:ReadFile@5797,file:ReadFile@865,file:Stat@3448 3eb11b09adc3b122f555513cc35aac8124f89149e72db995351cde384b2cf809
@@ -212,9 +212,9 @@ internal/host/rootgenerations_launch.go productionRootGenerations authority E2,E
internal/host/rootgenerations_launch.go rootGenerationSource.resolve authority E2 E20: read the atomic root installation under the existing host exclusion without recursive lock acquisition; retain fresh claim, canonical record, signed artifact and final reservation checks. Fixtures: TestPlan14GenerationLaunchDoesNotReenterRootLock, TestPlan14GenerationLaunchRefusals, TestPlan14RootDispatchRejectsRegistryMutation. PROPOSED E47: root-owned immutable unit/shape binding, reserved launch placement and exact invocation stop; retained generation binding carries the canonical unit. Default unit paths remain unchanged. Production named-route wiring and unit/env/timer retirement are not complete.;rev3=PROPOSED-p3;run=8;identity=root claim launch uid 0;basis=captures persistent executable and tree proofs only after canonical installation and signed artifact verification under exclusion;rev3=PROPOSED-p3;run=11;identity=root claim, migration or adoption controller uid 0;basis=merges E FileIdentity callers and P3 authority effects; retains the recorded family checks and both tracks behavioral fixtures file:Open@2501 3cd7c8132e9c451c983392f4d1cc676ddc14b761c90dec3e631d54ae800f9951
internal/host/rootgenerations_launch.go type rootGenerationSource authority E2,E3 rev3=PROPOSED-p3;identity=root claim/adoption controller uid 0;basis=binds production runtime authority and persistent file capture to the root launch source;rev3=PROPOSED-p3;run=11;identity=root claim, migration or adoption controller uid 0;basis=merges E FileIdentity callers and P3 authority effects; retains the recorded family checks and both tracks behavioral fixtures f291da85d463e50322a32fed1208629c6bf991112c4e8f614d9bf30bdb0338f5
internal/host/rootupdates.go initializeMigrationFailures authority G,E3 rev3=PROPOSED-p3;run=8;identity=offline journal-migrate uid 0;basis=requires complete root census before creating migration-bound root failure book; missing legacy history supplies no known refusal file:Open@254 5f7f652988f3def733e1b525260a52fac3568242960468e5457975ced1cd41d8
-internal/host/rootupdates.go productionRootUpdates authority E3 rev3=PROPOSED-p3;run=10;identity=root claim controller and adopt uid 0;basis=keeps canonical root inputs and retry; leaves sticky failure enforcement disabled; legacy bytes only identify an optional exact retry mirror and cannot block opening or inject fresh denials file:Open@1124,file:Open@765,file:Read@1447,whole:Route@1760,whole:Route@2081 ce81dce48e37f4055efeb3fd8d3562057e504ec5cd74ccbef42c0b8fcd6f20f0
+internal/host/rootupdates.go productionRootUpdates authority E3 rev3=PROPOSED-p3;run=20;identity=root update/adopt uid 0;basis=ordinary callers retain independent host locking through the shared opener b75a0937953dae3f2bbe1f9d9b7598eca6aa3add1f9c70d564a21f5412217f8d
internal/host/rootupdates.go rootFailureLegacy authority E3 rev3=PROPOSED-p3;run=10;identity=root adoption and offline migration uid 0;basis=bounded nonblocking regular-file read supplies optional exact retry mirror identity or one-time migration deny evidence; refuses symlinks and special files file:OpenFile@65,file:ReadAll@265 efae62d1b996f14baa592e39e0cc22e0f5c8cde5cd94f429498d41039f5a70e4
-internal/host/rootupdates.go rootUpdateSource.resolve authority E3,J1,J3 rev3=PROPOSED-p3;run=16;identity=root adopt/update/control uid 0;basis=canonical deployment settings stored per exact root installation with nested claim paths for readiness; legacy settings reused only for the same identity; successor completion state is empty while the shared root failure book retains denials file:Read@1374,file:Read@1447,registry:Account@1975,registry:Account@2860,registry:App@2013,registry:App@2905,registry:BuildState@1209,registry:GoodDemand@1175,registry:GoodDigest@1160,registry:GoodVersion@1144,registry:Host@1947,registry:Host@2825,registry:ServingVersion@1190,registry:Unit@2556,whole:Route@1953,whole:Route@1981,whole:Route@2019,whole:Route@2188,whole:Route@2195,whole:Route@2615,whole:Route@2838,whole:Route@2873,whole:Route@2918,whole:Route@3686,whole:Route@3694,whole:Route@5322,whole:Route@5329 8b7160d5b5cc1510eea7bbb3d9febe310df695337cfbc2b5e2ada7e9857ea83f
+internal/host/rootupdates.go rootUpdateSource.resolve authority E3 rev3=PROPOSED-p3;run=20;identity=root launch/update uid 0;basis=launch uses exact retained installation and census validation without reacquiring its held host lock; ordinary updates still lock independently; existing settings and failure authority retained file:Read@1537,file:Read@1610,registry:Account@2138,registry:Account@3023,registry:App@2176,registry:App@3068,registry:BuildState@1372,registry:GoodDemand@1338,registry:GoodDigest@1323,registry:GoodVersion@1307,registry:Host@2110,registry:Host@2988,registry:ServingVersion@1353,registry:Unit@2719,whole:Route@2116,whole:Route@2144,whole:Route@2182,whole:Route@2351,whole:Route@2358,whole:Route@2778,whole:Route@3001,whole:Route@3036,whole:Route@3081,whole:Route@3849,whole:Route@3857,whole:Route@5485,whole:Route@5492 6c5c447d2abd518a941979dcd1748ee028f3b555ce077fad849c02be0c77c2fa
internal/host/updatesroot.go updatesPassOption authority E3 rev3=PROPOSED-p3;run=8;identity=root adopt uid 0; unchanged agent opener uid carlos;basis=installs census-only root input provider while preserving the legacy failure-book opener for nonparticipating passes 4f66774059952bda531435d68487fa032a95b8d1c8ca33ec8b71b6639fa117e3
internal/release/adopt.go PassFilter authority B3,E3,J1,E1,E2 Signed named-unit recovery is independent of release and health history. Root rechecks canonical source, unit, process, signed target and trusted live pointer; captures attempt authority before publication; verifies readiness and resumes without repeated effects. Fixtures: TestPlan14SignedActiveFixIgnoresHistory, TestPlan14SignedActiveFixAuthority, TestPlan14SignedFixSupersedesOlderAttempt, TestPlan14ActiveRecoveryProcessCrash, TestPlan14SignedRestartWithoutHealthHistory, TestPlan14CapturedRollbackWithoutHealthHistory, TestPlan14SignedRestartRequiresTrustedLiveLink. E7 applies merge ruling 4: unbound no-census routes use the legacy pass and restart; ordinary candidates make no binding store call. Fixtures: TestR2UnguardedERecoveryUpdate, TestR2UnguardedERecoveryIncidentRecipe, TestPlan14ERecoveryBindingRefusalKeepsPromotedFix. E19: both destructive readers use the root reservation snapshot under existing release exclusion. Exact artifact and containing-tree identities are retained; missing completeness refuses destruction. Production installation/launch/kernel completeness and verified retirement remain unfinished. Fixtures: TestPlan14GenerationDeletionCannotUnpin, TestPlan14GenerationSnapshotCompleteness, TestPlan14GenerationSnapshotPermitsUnreservedRestage, TestPlan14GenerationSnapshotPinsContainingTree, TestPlan14GenerationSnapshotEmptyIsUnknown, TestPlan14GenerationConsumerMode.;rev3=PROPOSED-p3;run=8;identity=carlos adopt uid 0; legacy agent pass uid carlos;basis=resolves protected route inputs before filters, recovery and dispatch on census boxes and bypasses service-owned failure opener only there registry:App@26795,registry:App@29252,registry:App@29744,registry:App@29981,registry:App@31388,registry:App@6590,registry:App@9615,registry:App@9756,registry:Hibernate@29627,registry:Host@1061,registry:Host@1426,registry:Host@29797,registry:Host@30713,registry:Host@31441,registry:Host@3263,registry:Host@3330,registry:Host@3426,registry:Host@3461,registry:UpdateSettings@9493,whole:Route@1046,whole:Route@1164,whole:Route@1188,whole:Route@1461,whole:Route@1622,whole:Route@26790,whole:Route@2726,whole:Route@2768,whole:Route@2877,whole:Route@2901,whole:Route@29247,whole:Route@2932,whole:Route@29505,whole:Route@29527,whole:Route@29574,whole:Route@29592,whole:Route@29602,whole:Route@29645,whole:Route@29666,whole:Route@29679,whole:Route@29739,whole:Route@29765,whole:Route@29839,whole:Route@29878,whole:Route@2989,whole:Route@29894,whole:Route@29922,whole:Route@29976,whole:Route@30002,whole:Route@3075,whole:Route@30755,whole:Route@30809,whole:Route@30825,whole:Route@3097,whole:Route@31332,whole:Route@31383,whole:Route@31409,whole:Route@31483,whole:Route@31518,whole:Route@31534,whole:Route@3197,whole:Route@4280,whole:Route@4927,whole:Route@6512,whole:Route@6534,whole:Route@6585,whole:Route@6611,whole:Route@761,whole:Route@803,whole:Route@9444,whole:Route@950,whole:Route@9558,whole:Route@9603,whole:Route@9751,whole:Route@988 270763804170539d06afd58252d86e1a8ff692ab4d9355be3a2127e9efc9d4a2
internal/release/adopt.go adoptHost authority B3,E1,E2,E3,E4,J1 E16: root first-registration creation evidence under release exclusion; exact installation identity is required for first-program adoption and is consumed before an attempt. Installed-program history comes from the verified claim launch. Fixtures: TestPlan14FreshInstallRegistration, TestPlan14FreshInstallProgramGate, TestPlan14FreshInstallCanonicalIdentity, TestPlan14FreshInstallLaunchObservation, TestPlan14InstalledLaunchProgramEvidence, TestPlan14FreshInstallProcessCrash, TestPlan14FreshInstallVisibleSync, TestClaimControlUnitInstallers. E22: root generation pins guard existing-tree artifact and manifest writes in adoption and moves-held staging/publication. Held snapshot is read under release exclusion before the leaf held-stage lock; failures remain restrictive for writes. Claims-off source returns no root pins. Fresh wholly absent version trees retain signed staging. Fixtures: TestPlan14GenerationSiblingWrites, TestPlan14GenerationVerifiedSiblingWrites, TestPlan14GenerationHeldStage (production source assembly, both flag values without census, lock order, pin/error/fresh cases). Production completeness remains unavailable; no new completeness grant.;rev3=PROPOSED-p3;run=8;identity=carlos adopt uid 0; legacy agent pass uid carlos;basis=applies root settings to canonically resolved routes including named units while legacy settings eligibility remains unchanged;rev3=PROPOSED-p3;run=16;basis=P3-R11 approved punctuation edit only; authority and reads unchanged registry:App@10936,registry:App@11096,registry:App@12125,registry:App@13051,registry:App@14630,registry:App@19789,registry:App@25265,registry:App@3144,registry:App@33024,registry:App@6758,registry:App@8059,registry:App@8253,registry:Hibernate@33885,registry:Host@10816,registry:Host@10849,registry:Host@10949,registry:Host@11109,registry:Host@11211,registry:Host@11243,registry:Host@11308,registry:Host@11758,registry:Host@11891,registry:Host@12338,registry:Host@12447,registry:Host@12649,registry:Host@13141,registry:Host@13927,registry:Host@14402,registry:Host@14616,registry:Host@14958,registry:Host@15548,registry:Host@19279,registry:Host@19387,registry:Host@19757,registry:Host@20577,registry:Host@20735,registry:Host@2105,registry:Host@21197,registry:Host@21249,registry:Host@21396,registry:Host@21483,registry:Host@22589,registry:Host@22833,registry:Host@2294,registry:Host@23493,registry:Host@2375,registry:Host@23878,registry:Host@24019,registry:Host@25233,registry:Host@25360,registry:Host@26702,registry:Host@27352,registry:Host@30585,registry:Host@31203,registry:Host@31528,registry:Host@3157,registry:Host@31602,registry:Host@31849,registry:Host@32000,registry:Host@33196,registry:Host@33363,registry:Host@33470,registry:Host@33698,registry:Host@33841,registry:Host@34586,registry:Host@34712,registry:Host@34800,registry:Host@4469,registry:Host@4583,registry:Host@5546,registry:Host@5651,registry:Host@6870,registry:Host@6918,registry:Host@7035,registry:Host@7067,registry:Host@7945,registry:Host@8113,registry:Host@8239,registry:Host@8358,registry:Host@8836,registry:Host@9223,registry:Host@9449,registry:Host@9545,registry:Host@9636,registry:Kind@554,registry:Unit@31748,registry:Unit@579,whole:Route@10073,whole:Route@10931,whole:Route@11091,whole:Route@12115,whole:Route@13041,whole:Route@14598,whole:Route@15163,whole:Route@15524,whole:Route@16747,whole:Route@17050,whole:Route@17206,whole:Route@19536,whole:Route@19680,whole:Route@19779,whole:Route@2206,whole:Route@22318,whole:Route@22717,whole:Route@25255,whole:Route@30156,whole:Route@3139,whole:Route@33172,whole:Route@3595,whole:Route@4347,whole:Route@6753 2b2c351c0ecc567e5eaf4146cf036ee073b47e2d6ae123198f4d970d0e4ccb96
@@ -257,7 +257,7 @@ internal/host/rootinventory.go ClaimRootHTTP.commitRootInventoryPending authorit
internal/host/rootinventory.go ClaimRootHTTP.verifyInventoryPath authority C,J1 rev3=PROPOSED-p3;identity=root claim/adoption controller uid 0;basis=uses checked persistent evidence for protected installation entries; explicit legacy fixtures preserve their same-boot check;rev3=PROPOSED-p3;run=11;identity=root claim, migration or adoption controller uid 0;basis=merges E FileIdentity callers and P3 authority effects; retains the recorded family checks and both tracks behavioral fixtures file:Lstat@315 2ee5519ed6e8ff27bcfdb5db7748a71f4f331a271d84d7688835798792824a78
internal/host/rootinventory_identity.go ClaimRootHTTP.bindInventoryPaths authority C,E2,J1 rev3=PROPOSED-p3;identity=root claim/adoption controller uid 0;basis=discards all caller-supplied persistent evidence and captures or revalidates it independently against the retained root inventory;rev3=PROPOSED-p3;run=11;identity=root claim, migration or adoption controller uid 0;basis=discards caller proof, captures new objects independently, retains immutable prior proof, and verifies cross-boot ancestry through the shared inventory verifier;rev3=PROPOSED-p3;run=11;identity=root claim, migration or adoption controller uid 0;basis=requires the proposed boot to equal the independently observed boot for new materialization, discards caller filesystem proof, captures new objects independently, retains immutable prior proof, and verifies cross-boot ancestry through the shared inventory verifier 73c94e6482dafa576ac607ba4a54a83921febf0f2092361e88a259a884db75d7
internal/host/rootinventory_state_linux.go ClaimRoleAssembly.prepareStateInventory authority C,J1 rev3=PROPOSED-p3;identity=carlos agent claim lifecycle uid carlos; final authorization remains in root;basis=uses root-supplied persistent state identity for remote inventory reuse while keeping agent proposals non-authoritative;rev3=PROPOSED-p3;run=11;identity=carlos agent uid carlos; root validates final inventory and launch;basis=merges E FileIdentity callers and P3 authority effects; retains the recorded family checks and both tracks behavioral fixtures file:Fstatat@1313,file:Fstatat@2304,file:Open@799,file:Openat@1086 c5355a26fa146eb0ddb4cc7c1e536de5781e171d52363a0be38ffb2074651a53
-internal/host/rootupdates_runtime.go beginRootGenerationUpdate authority E2,E3 rev3=PROPOSED-p3;run=13;identity=root launch controller uid 0;basis=resolves canonical runtime update identity; sticky failure veto is disabled in production under the narrowed activation gate b060f3f53e87b9ea7bc78a6cd5d3435db63b07a83c3b0a38618d2481b000605e
+internal/host/rootupdates_runtime.go beginRootGenerationUpdate authority E2,E3 rev3=PROPOSED-p3;run=20;identity=root launch uid 0;basis=passes already reserved generation binding to settings resolver while holding release and host exclusion; avoids recursive host lock acquisition c67ae6552b1232728d3cdcbef857b19ff81bce04bd05e79b8a65b11c67efc7c8
internal/host/rootupdates_runtime.go publishRootRetry authority E3 rev3=PROPOSED-p3;identity=root claim/adoption controller uid 0;basis=writes root-owned public retry status only from the independently verified root book file:Open@84 e3c680d5a7d3c423dc183319bb3d0518bbea6cdc752c7bd6b540d57f884b414e
internal/host/rootupdates_runtime.go rootRetryStatus hint E3 rev3=PROPOSED-p3;identity=carlos agent status probe uid carlos;basis=agent reads root-owned public retry status; an exact legacy-book acknowledgement may clear only the corresponding local mirror, never root authority file:Lstat@89,file:Open@183,file:Read@721 7db6b16062bd095634a03c58cfdcd3dde2d49c990a34fe37d260d0908a16aa5f
internal/host/rootupdates_runtime.go rootUpdateReady authority E3 rev3=PROPOSED-p3;identity=root claim/adoption controller uid 0;basis=probes the canonical deployment endpoint before root records runtime served/request success whole:Route@95 fe93218ee43c1346b148e4c9f6f55731d8865acde4f72ec85f60c5732a6a7d76
@@ -358,13 +358,52 @@ internal/host/claimproduction.go claimServingLaunchSource authority E2 rev3=PROP
internal/host/claimtoken_inventory_linux.go claimAdminTokens.prepareState authority C rev3=PROPOSED-p3;run=19;identity=agent uid carlos, independently validated by root journal uid 0;basis=uses current claim, exact installation and deployment-record expectation, root fence view, and the existing two-phase state-directory inventory registration before credential minting file:Read@376,registry:DBPath@753,registry:DBPath@94,registry:Host@150,registry:Host@406,registry:Host@714 e5e33367df92021bf920625050d828016c078a525b6deb5f28163f37d51f762a
internal/host/claimtoken_inventory_other.go claimAdminTokens.prepareState authority C rev3=PROPOSED-p3;run=19;identity=non-Linux caller;basis=refusal-only stub; cannot register state or mint authority candidate:Route@56 73d3f3886940f05753d25b761d2274b6b953dce650f34795ad03a1d12af77c26
internal/hostbox/servinghandover.go ServingHandover.validate authority C,J4 rev3=PROPOSED-p3;run=19;identity=root instance-control uid 0;basis=allows route-only initial handover only from verified session zero; later sessions still require exactly one verified state-directory entry; all receipt, inventory and claim checks remain file:Exec@258,registry:ClaimIncarnation@1906,registry:ClaimOwner@1379,registry:ClaimOwner@1980,registry:ClaimOwnership@1874,registry:ClaimRowID@2015,registry:ClaimStamp@1949,registry:Name@1860,registry:Registration@2052,registry:Route@1033,registry:Route@1379,registry:Timers@1806,registry:Timers@2282,whole:ClaimOwnership@1880,whole:ColdRows@1035,whole:ColdRows@1381,whole:ColdRows@1808,whole:ColdRows@2284,whole:Route@1042,whole:Route@1388,whole:Timer@1815,whole:Timer@2291 d8015871bf0cdfb7dfb9715510d1cd649d05aeed8ecded9c8cfb2fae0d6fc946
-internal/host/claimowners.go claimServingOwners.EmptyAuthority authority C,J4 rev3=PROPOSED-p3;run=19;identity=root or agent;basis=first-writer proof rejects all object versions except the exact Litestream lease key at the holder location; leased admission no longer rejects itself 0068e5e3ccc1f15b87a0143393907391f2089729c042f28e389ee972449141ae
+internal/host/claimowners.go claimServingOwners.EmptyAuthority authority C,E3,J4 rev3=PROPOSED-p3;run=20;identity=root controller and service UID using authenticated root projection;basis=empty recovery requires retained root unlaunched stop receipts for every prior cold admission of the exact installation and local box; checks all object versions, allowing only lease and preparatory platform files under the root proof; normal baseline and no-census behavior retained 4167cdb7b73f4d9fea5ae9dbb23b8885f9c6f254c8344332b056ef0b63fb2811
internal/host/claimstop_recovery.go ClaimStopHTTP.RecoverPending authority C,J4 rev3=PROPOSED-p3;run=19;identity=root instance-control;basis=recovers only pending stop fences; persisted handover barriers remain for the handover journal recovery 0376cfcfebdebf8a689d264c47b559a6e5e8deef64768d4e75bebef909be19a1
internal/hostguard/recoveryfence.go Guard.PendingFences authority C,J2,J4,J1 rev3=PROPOSED-p3;run=19;identity=root;basis=retains complete pending-fence enumeration including handover barriers bc4b4a5d58f470fd0e7b641acc61f2ac4082c021e729534ec4dd00ae423cd03c
internal/hostguard/recoveryfence.go Guard.PendingStopFences authority C,J4 rev3=PROPOSED-p3;run=19;identity=root;basis=stop-specific enumeration excludes only an exact barrier persisted in the root record f48f8b28b289556e3be77fb76a485f0d4c0a013ec12cb90197071cd2f5ddb87b
internal/hostguard/recoveryfence.go Guard.pendingFences authority C,J2,J4,J1 rev3=PROPOSED-p3;run=19;identity=root;basis=reads root-owned records under host lock and keeps exact barrier classification separate from ordinary stop recovery; legacy behavior retained 20be9535fd54587a54e3027f5388581ae5a64e5fcfced5a6031ffc9a4fc76a3b
internal/host/claimroleassembly_linux.go verifiedClaimExecutable authority C,G,J4 rev3=PROPOSED-p3;run=19;identity=agent uid carlos;basis=pins root-owned executable bytes and keeps the verified descriptor while supplying its absolute path to the root launcher file:Fstat@1122,file:Fstat@579,file:OpenFile@376,file:Read@949 7a7e9af520e4157e60ae93d12c3268a629a9b0bd1e83b263cdd27103cf2e227c
internal/host/claimroleassembly_linux.go claimExecutableName authority C,G,J4 rev3=PROPOSED-p3;run=19;identity=agent uid carlos;basis=duplicates the existing descriptor with CLOEXEC and its validated absolute name; never reopens a potentially replaced path e887880731a16c9e7934aef33a08169b34e7236f6d3b85a195602d35f8b69107
-@coverage internal/host 4f93a7d806207d2d87c068a362c2bcb89259885ea3bd6270cae5cfeb4366592e
-@coverage internal/hostbox 5e2d3da17aef4ad41c5451e482540e83563624b70b8b5f1fbb270ac6e4c5b468
-@coverage internal/hostguard fbfa2f109431ed36c25662b30e56bf8cf16cccb06b10a7e8f41df408b86cae31
+@coverage internal/host 470c8d923787bc0cd3c55f4e263a89ca206d992e4f282d86386fd663d6eb43e2
+@coverage internal/hostbox 1a416db4f7a98faf6a39675a84e516013e13d79d6999fd1c921248b1f82e1557
+@coverage internal/hostguard 1b3341e3f1a3b56c46b1a85ec5e12cb08e0c26128b1b5c865a562ab5aa0994fb
+internal/host/claimstop.go ClaimStopCoordinator.FenceAndStop authority C,J4 rev3=PROPOSED-p3;run=20;identity=root instance-control uid 0;basis=authenticated session stop retains root fence before independent unjournaled absence check; proves no journaled installation, containment or open files before bypassing nonexistent writer closure; normal drain remains for recorded launches; no-census unchanged file:Read@1610 58147f3abbf278037fc660392aeec0b6e76b667e3fe8435845ebaedc62d2ead1
+internal/host/claimstop_recovery.go ClaimStopCoordinator.RecoverStopped authority C,J4 rev3=PROPOSED-p3;run=20;identity=root instance-control uid 0;basis=pending exact root fence and exact installation inventory permit recovery through independent absence verification when no launch journal was created a91b528d9f9259612c767bf0a94e0ff7664294668f29f584a681a1a0f656ad90
+internal/host/claimstop_recovery.go ClaimStopCoordinator.verifyUnjournaled authority C,J4 rev3=PROPOSED-p3;run=20;identity=root instance-control uid 0;basis=requires root authority, exact resumed fence and sql.ErrNoRows before allowing the independent kernel absence witness; no-census unchanged 0fa59df5d2e838b944c0b36ae60598f958b1057b978cb8c123de7edace143d7a
+internal/host/claimstop_recovery.go ClaimStopCoordinator.recoverKernel authority C,J4 rev3=PROPOSED-p3;run=20;identity=root instance-control uid 0;basis=only verified unjournaled absence supplies empty-producer recovery evidence; ordinary journaled recovery unchanged e04ffee4dfcd61ec8066e53d6cdbe51da920af0f1f75db6c23357122278f8c08
+internal/host/claimstop_writers_linux.go verifyNoProcessAt authority C,J4 rev3=PROPOSED-p3;run=20;identity=root instance-control uid 0;basis=session-zero entry retains its checks and delegates unchanged observation to shared helper 1c404c09d26c9c5afab607984ebf6fc6f4a592eecca4f9dd1e19daeada963802
+internal/host/claimstop_writers_linux.go LinuxStopKernel.VerifyUnjournaled authority C,J4 rev3=PROPOSED-p3;run=20;identity=root instance-control uid 0;basis=real root proc and cgroup mounts, systemd agent containment and observed boot bind the unjournaled kernel witness; requires nonzero session and durable fence identity file:ReadFile@510 1c136916f6179bab634f671036718d919fff64c447032717dedf53427ec442e4
+internal/host/claimstop_writers_linux.go verifyNoLaunchAt authority C,J4 rev3=PROPOSED-p3;run=20;identity=root instance-control uid 0;basis=rejects any root launch history for the installation; scans prior and proposed delegated containment, isolated host slice and root-derived installation descriptors; repeats journal and containment observation c51e82ce02eb33c8c209803db655bff4a2e819a24c04787cadd9303f8075fd67
+internal/host/claimlaunch_root.go claimUnitStopKernel.VerifyUnjournaled authority C,J4 rev3=PROPOSED-p3;run=20;identity=root instance-control uid 0;basis=forwards the non-destructive absence check to the real kernel without trying to settle nonexistent unit reservations 01aad0207c58d2e2ed50ed726fee6a33f5f1136c3fe15615a778a4e3b731988b
+internal/host/claimstop_kernel_linux.go newProductionStopKernel authority C,J4 rev3=PROPOSED-p3;run=20;identity=root instance-control uid 0;basis=root authority selects recovery-capable systemd containment reader; nonparticipants keep exact previous reader 3f98eed355fb9818a0f2f33ad588f0473315d80c322be568465aafdcdc50dfee
+internal/host/claimstop_kernel_other.go newProductionStopKernel authority C,J4 rev3=PROPOSED-p3;run=20;identity=non-Linux root caller;basis=unsupported platform stub returns no verifier 76915e738b8b76c59407623ffdb19872452f2c93e840c033abceec635973b167
+internal/host/claimstop_production.go NewProductionClaimStopHandler authority C,J4 rev3=PROPOSED-p3;run=20;identity=root instance-control uid 0;basis=root authority selects stopped-agent containment observation without changing peer authentication or nonparticipating construction file:CommandContext@1106,file:Read@786 44cc78066b3cfd94821fa7f2ff80e47c851c0b83f57ffd97eca94f81d49c9ed6
+internal/host/claimstop_writers_linux.go rootStopAgentCgroup authority C,J4 rev3=PROPOSED-p3;run=20;identity=root instance-control uid 0;basis=reads systemd unit identity, active state, slice and cgroup to observe recorded containment even after the edge unit stopped file:CommandContext@149 ca8dcb4f62ccd2f0a96bf9b53350f4e32836fd71352d33a7427defe12e171174
+internal/host/claimstop_writers_linux.go rootStopAgentGroup authority C,J4 rev3=PROPOSED-p3;run=20;identity=root instance-control uid 0;basis=requires exact edge unit and either a valid live cgroup or a stopped/failed unit with a valid root-configured slice; rejects ambiguous fields and traversal f75315926ad8c4c58ef36986b75f6b0cc646b53dca9f53e41ee680ce5a94990b
+internal/host/rootupdates.go productionRootUpdatesHeld authority E3 rev3=PROPOSED-p3;run=20;identity=root launch/update uid 0;basis=constructs canonical update inputs; only the launch path supplies its exact root generation binding under existing release and host exclusion file:Open@1161,file:Open@802,file:Read@1484,whole:Route@1797,whole:Route@2118 5f3cbf8f79715e44257e1539658c25c50a6596a64a6b8916dd38f776f2faa2b6
+internal/host/rootupdates.go type rootUpdateSource authority E3 rev3=PROPOSED-p3;run=20;identity=root launch/update uid 0;basis=carries optional exact generation binding solely for launch settings resolution under host exclusion c098d3dee0212a06536177561d3bd72c868d762ea88c99be99e42aa9d2751727
+internal/host/rootrestart_requests_source.go rootRestartRequestSource.installation authority E1 rev3=PROPOSED-p3;run=20;identity=root update/restart uid 0;basis=ordinary installation lookup keeps host capture and locked view then common complete-census validation 6ac80df03bfde1f6249ef0197774323def84ae5c0067dad670802eeb9a33eacb
+internal/host/rootrestart_requests_source.go rootRestartRequestSource.launchInstallation authority E2,E3 rev3=PROPOSED-p3;run=20;identity=root launch uid 0;basis=under caller-held host lock reads exact root installation and rechecks generation binding host, account, app, record, stamp, box, enrollment and installation hash before census validation 3175a18762e6a1ccf8273bff68b8a73d01390ef5674e5931060aafc24d2250fc
+internal/host/rootrestart_requests_source.go rootRestartRequestSource.checkInstallation authority E1,E3 rev3=PROPOSED-p3;run=20;identity=root update/restart/launch uid 0;basis=requires complete nonempty installation with matching box, enrollment, migration and inventory revision in complete census c5b96d796114cf804163ed391957be76fda5dd08b10f2f297ad4d27957e61896
+internal/host/claimempty_recovery.go type claimEmptyRecovery authority C,E3,J4 rev3=PROPOSED-p3;run=20;identity=root controller and service UID using authenticated root projection;basis=empty recovery requires retained root unlaunched stop receipts for every prior cold admission of the exact installation and local box; checks all object versions, allowing only lease and preparatory platform files under the root proof; normal baseline and no-census behavior retained d63aa0641dd723b4f35992d4333a17870f4822cae9851ba89a3049131850c727
+internal/host/claimempty_recovery.go rootEmptyRecovery authority C,E3,J4 rev3=PROPOSED-p3;run=20;identity=root controller and service UID using authenticated root projection;basis=empty recovery requires retained root unlaunched stop receipts for every prior cold admission of the exact installation and local box; checks all object versions, allowing only lease and preparatory platform files under the root proof; normal baseline and no-census behavior retained ba817c5296f8afc18777cde478b1dfb87f84636178df929c6d8a1cf010836f10
+internal/host/claimempty_recovery.go claimServingOwners.emptyBootstrap authority C,E3,J4 rev3=PROPOSED-p3;run=20;identity=root controller and service UID using authenticated root projection;basis=empty recovery requires retained root unlaunched stop receipts for every prior cold admission of the exact installation and local box; checks all object versions, allowing only lease and preparatory platform files under the root proof; normal baseline and no-census behavior retained c9b2eef681d6555bc75a055486f161c3fb3512d2f69963ba6f2afc74c456ab5c
+internal/host/claimempty_recovery.go claimServingOwners.emptyLocation authority C,J4 rev3=PROPOSED-p3;run=20;identity=root controller and service using authenticated proof;basis=checks all object versions; only after all admissions have root no-launch proof allows preparatory platform-only files manifests and their named env objects; refuses tenant entries, database objects, unknown objects and deleted file versions fd9212fe323b2ac7f28b5885db65cfed25e6298f8a26699bd555967f5a275382
+internal/host/claimowners.go type claimServingOwners authority C,E3,J4 rev3=PROPOSED-p3;run=20;identity=root controller and service UID using authenticated root projection;basis=empty recovery requires retained root unlaunched stop receipts for every prior cold admission of the exact installation and local box; checks all object versions, allowing only lease and preparatory platform files under the root proof; normal baseline and no-census behavior retained cacabc7b0196545f1cc388c4b4d22c2785c3c3b811a52bd5d7fa89e6af951d05
+internal/host/claimowners.go claimServingOwners.Select authority C,E3,J4 rev3=PROPOSED-p3;run=20;identity=root controller and service UID using authenticated root projection;basis=empty recovery requires retained root unlaunched stop receipts for every prior cold admission of the exact installation and local box; checks all object versions, allowing only lease and preparatory platform files under the root proof; normal baseline and no-census behavior retained registry:DBPath@1081,registry:DBPath@461,registry:Host@424,whole:Route@555 586f62b80a544716c0e8873916457f7d67f02254db659fd8dd053be836e194e3
+internal/host/claimowners.go claimServingOwners.admissionEvidence authority C,E3,J4 rev3=PROPOSED-p3;run=20;identity=root controller and service UID using authenticated root projection;basis=empty recovery requires retained root unlaunched stop receipts for every prior cold admission of the exact installation and local box; checks all object versions, allowing only lease and preparatory platform files under the root proof; normal baseline and no-census behavior retained dd40914d4910247bd6616fd4b9fdfc86011c8df690bd52d5d8537cc2596e5492
+internal/host/movesfiles.go type movesFiles authority C,E3,J4 rev3=PROPOSED-p3;run=20;identity=root controller and service UID using authenticated root projection;basis=empty recovery requires retained root unlaunched stop receipts for every prior cold admission of the exact installation and local box; checks all object versions, allowing only lease and preparatory platform files under the root proof; normal baseline and no-census behavior retained 95045569641618c3acc27ee1d55087bbb9b855dd99147c648bdfd01dc8ff1fee
+internal/host/claimfilesseed_guard.go movesFiles.checkServingFilesEvidence authority C,E3,J4 rev3=PROPOSED-p3;run=20;identity=root controller and service UID using authenticated root projection;basis=empty recovery requires retained root unlaunched stop receipts for every prior cold admission of the exact installation and local box; checks all object versions, allowing only lease and preparatory platform files under the root proof; normal baseline and no-census behavior retained whole:Route@265 bafb35881d8da7eef7dfbf97687932bc8b134de21a8d8b6bb3803a07323ac55c
+internal/hostguard/coldhandover.go Guard.CompletedColdPredecessor authority C,J4 rev3=PROPOSED-p3;run=20;identity=root and authenticated root-view consumer;basis=selects root current completed fence when multiple controller receipts exist for one session; pending current fence refuses; legacy ambiguity behavior retained 662a4f20ec0bd817b57e256e582f8fee9e3784b1bf9764d111ca509ec596ec90
+internal/hostbox/launchjournal.go type LaunchReservation authority C,J4 rev3=PROPOSED-p3;run=20;identity=root launch journal;basis=additive NeverIntended cancellation evidence is root-owned; all inbound reservation constructors refuse the bit 6706ea23539c9f5c89c66f4656b197689ddbc020c08f04aae652a74ccfc9f9da
+internal/hostbox/launchjournal.go LaunchJournal.Reserve authority C,J4 rev3=PROPOSED-p3;run=20;identity=root journal and legacy agent;basis=refuses caller-supplied root cancellation evidence d7360f3407eb6bc302eac0fb27159419e372b38ad02779a7d350b79f8e0f66a7
+internal/hostbox/roleintention.go LaunchJournal.IntendRole authority C,J4 rev3=PROPOSED-p3;run=20;identity=root journal and legacy agent;basis=refuses caller-supplied root cancellation evidence 4499b3ea59f8982c585e3bb46de9766d0ee0926f55843e33ac010b15780b94da
+internal/hostbox/rolelaunch.go unitReservationShape authority C,J4 rev3=PROPOSED-p3;run=20;identity=root launch uid 0;basis=refuses caller-supplied NeverIntended evidence on both start and stop b505ac110c9d78f86813a9fb679ea5ec010e228a98ea2f14d5a8ddad2084a00c
+internal/hostbox/rolelaunch.go LaunchJournal.TombstoneUnit authority C,J4 rev3=PROPOSED-p3;run=20;identity=root launch uid 0;basis=sets never-intended evidence only when exact reservation is absent and is durably spent before any later start f86a9c0de3d5a5edbd296ff0cbfad4e60d9a997ce4278a826e1ee9717242b7d6
+internal/host/claimstop_kernel_linux.go type LinuxStopKernel authority C,J4 rev3=PROPOSED-p3;run=20;identity=root stop uid 0;basis=rootAuthority restricts added never-intended evidence to root-owned journals ae158c8eede6bc837534f03f89cd00770c910cbe2ebb45ef51fc38bed75f4fde
+internal/host/claimstop_kernel_linux.go LinuxStopKernel.VerifyStopped authority C,J4 rev3=PROPOSED-p3;run=20;identity=root stop uid 0;basis=after closure and independent absence checks additionally proves every root launch record for installation had no intended tenant execution 9665066696894745669b778ec525b240771dfdc974cd2e80e76460809a77a9ac
+internal/host/claimstop_writers_linux.go stopNeverIntended authority C,J4 rev3=PROPOSED-p3;run=20;identity=root stop uid 0;basis=requires all installation records writer-closed, one parent group, and only root tombstones with no process or unit identity f5165bb027430b7cdac2592b11fbaeffcee008f9e884868d7bc904b25189f38c
+internal/host/claimservingretire.go claimServingArtifactRetirement.BeforeLegacyInstall authority C,J4 rev3=PROPOSED-p3;run=20;identity=service caller of root retirement;basis=separates empty database recovery from session-zero predecessor; root still verifies ordinary prior admission stop receipts daf192253b905a23e3f49ced325e9e431edad01676fb22a6577a56dd493010d2
+internal/hostguard/flock_unix.go acquire authority C,J4 rev3=PROPOSED-p3;run=20;identity=root and service host-lock callers;basis=a root census caller creating the shared lock preserves its directory group so service can reopen it after reboot; existing owner validation and no-census path unchanged file:Fstat@1356,file:Fstat@398,file:Fstatat@665,file:Openat@1024,file:Openat@897 a5e19f0fb712194d07aecbfa72f7aa91fe90b7bbc16ea18e33dadd025b5f625f
+internal/host/claimservingpark.go servingGenerationParkDisposition authority C,J4 rev3=PROPOSED-p3;run=20;identity=root retirement uid 0;basis=validates every root stop receipt and matches the original launch controller to at least one; root never-intended tombstone with closed writers and absent process identity permits discard; ambiguous intentions refuse c1e013c260fef86a61945f7c38bc1352f339adbd3c8fb8b92b3785ea6b96de59