carlos / platform Public

Claim activation tools, run 10: canonical baseline (not for merge)

offline-begin admits only the canonical healthy serving baseline from one code table; restore returns to it; resume always reruns a whole-record completion gate. 61 restore-checkpoint reboots on production units.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Paul Campbell pushed by claude agent, operated by paul@keymail.dev b177d9141f36ce9610618bd7812e5146377d4f6a parent 28814d7
14 files changed, +410 −634
  • internal/host/activation_adoption_linux.go +14 −0
  • internal/host/activation_adoption_other.go +4 −0
  • internal/host/activation_baseline_linux_test.go +57 −0
  • internal/host/activation_baseline_rootrunner_linux_test.go +100 −87
  • internal/host/activation_invariants_rootrunner_linux_test.go +5 −5
  • internal/host/activation_matrix_rootrunner_linux_test.go +52 −424
  • internal/host/activation_offline_linux.go +101 −47
  • internal/host/activation_offline_rootrunner_linux_test.go +17 −8
  • internal/host/activation_production_reboot_rootrunner_linux_test.go +12 −39
  • internal/host/activation_production_rootrunner_linux_test.go +22 −9
  • internal/host/activation_restore_order_rootrunner_linux_test.go +5 −2
  • internal/host/activation_seed_cli_rootrunner_linux_test.go +1 −2
  • internal/host/systemapply.go +6 −0
  • internal/provision/testdata/plan14-root-reads-activation-tools.tsv +14 −11

CI passed — run details

CI log
=== gate ===
cd launchctx && go vet ./... && go test ./... -count=1
ok  	amadan.net/carlos/platform.git/launchctx	0.011s
?   	amadan.net/carlos/platform.git/launchctx/controlname	[no test files]
?   	amadan.net/carlos/platform.git/launchctx/wireid	[no test files]
go vet ./...
gofmt -l .: clean
go test ./... -count=1 -timeout 30m
ok  	github.com/carlosframework/platform	296.102s
?   	github.com/carlosframework/platform/cmd/changelog	[no test files]
ok  	github.com/carlosframework/platform/cmd/console	2.489s
ok  	github.com/carlosframework/platform/cmd/moveprobe	92.210s
ok  	github.com/carlosframework/platform/cmd/zddprobe	0.700s
ok  	github.com/carlosframework/platform/internal/activator	252.319s
ok  	github.com/carlosframework/platform/internal/agentsock	0.019s
ok  	github.com/carlosframework/platform/internal/alerting	0.004s
ok  	github.com/carlosframework/platform/internal/analytics	0.019s
ok  	github.com/carlosframework/platform/internal/appconfig	2.407s
ok  	github.com/carlosframework/platform/internal/apperr	0.081s
ok  	github.com/carlosframework/platform/internal/applog	0.142s
ok  	github.com/carlosframework/platform/internal/awsinv	0.005s
ok  	github.com/carlosframework/platform/internal/boxchoice	0.005s
ok  	github.com/carlosframework/platform/internal/boxid	0.751s
?   	github.com/carlosframework/platform/internal/boxid/boxidtest	[no test files]
?   	github.com/carlosframework/platform/internal/boxid/internal/signtest	[no test files]
ok  	github.com/carlosframework/platform/internal/boxvolumes	0.009s
ok  	github.com/carlosframework/platform/internal/caps	0.014s
ok  	github.com/carlosframework/platform/internal/certwarm	0.002s
ok  	github.com/carlosframework/platform/internal/cgroup	0.695s
ok  	github.com/carlosframework/platform/internal/changelog	0.306s
ok  	github.com/carlosframework/platform/internal/claimactivation	0.026s
ok  	github.com/carlosframework/platform/internal/clamd	0.055s
?   	github.com/carlosframework/platform/internal/cliflag	[no test files]
ok  	github.com/carlosframework/platform/internal/console	390.996s
?   	github.com/carlosframework/platform/internal/controlname	[no test files]
?   	github.com/carlosframework/platform/internal/controlsock	[no test files]
ok  	github.com/carlosframework/platform/internal/dbdoor	6.077s
ok  	github.com/carlosframework/platform/internal/dbdoor/dbdoortest	0.014s
ok  	github.com/carlosframework/platform/internal/dbverify	0.004s
ok  	github.com/carlosframework/platform/internal/dnsrec	0.008s
ok  	github.com/carlosframework/platform/internal/docsite	0.007s
ok  	github.com/carlosframework/platform/internal/domains	2.352s
ok  	github.com/carlosframework/platform/internal/doorkey	0.007s
ok  	github.com/carlosframework/platform/internal/economics	0.004s
ok  	github.com/carlosframework/platform/internal/edge	26.717s
?   	github.com/carlosframework/platform/internal/edgeproof	[no test files]
ok  	github.com/carlosframework/platform/internal/edgesfile	0.008s
ok  	github.com/carlosframework/platform/internal/fleetapi	0.135s
ok  	github.com/carlosframework/platform/internal/gate	1.657s
ok  	github.com/carlosframework/platform/internal/genselect	0.019s
ok  	github.com/carlosframework/platform/internal/geo	0.017s
ok  	github.com/carlosframework/platform/internal/geoip	0.034s
ok  	github.com/carlosframework/platform/internal/historyreadback	0.157s
ok  	github.com/carlosframework/platform/internal/host	173.984s
ok  	github.com/carlosframework/platform/internal/hostactors	0.373s
ok  	github.com/carlosframework/platform/internal/hostbox	2.613s
ok  	github.com/carlosframework/platform/internal/hostclaim	0.523s
ok  	github.com/carlosframework/platform/internal/hostevidence	0.014s
ok  	github.com/carlosframework/platform/internal/hostexec	4.752s
ok  	github.com/carlosframework/platform/internal/hostfence	0.031s
ok  	github.com/carlosframework/platform/internal/hostfiles	10.497s
ok  	github.com/carlosframework/platform/internal/hostfloor	0.015s
ok  	github.com/carlosframework/platform/internal/hostguard	1.036s
ok  	github.com/carlosframework/platform/internal/hostops	1.570s
?   	github.com/carlosframework/platform/internal/hostops/fencerelease	[no test files]
ok  	github.com/carlosframework/platform/internal/hostops/internal/fencerelease	0.015s
ok  	github.com/carlosframework/platform/internal/hostquarantine	0.013s
ok  	github.com/carlosframework/platform/internal/hostrelocate	0.468s
ok  	github.com/carlosframework/platform/internal/hostresolve	0.139s
ok  	github.com/carlosframework/platform/internal/hostwrite	0.014s
ok  	github.com/carlosframework/platform/internal/infraview	0.013s
ok  	github.com/carlosframework/platform/internal/instancectl	5.239s
ok  	github.com/carlosframework/platform/internal/instanceexec	2.764s
ok  	github.com/carlosframework/platform/internal/instancefs	0.013s
ok  	github.com/carlosframework/platform/internal/instancekey	0.015s
ok  	github.com/carlosframework/platform/internal/instanceupdates	0.019s
ok  	github.com/carlosframework/platform/internal/layout	1.520s
ok  	github.com/carlosframework/platform/internal/lease	9.811s
ok  	github.com/carlosframework/platform/internal/lease/leasetest	0.006s
ok  	github.com/carlosframework/platform/internal/ledger	0.011s
ok  	github.com/carlosframework/platform/internal/mail	0.189s
ok  	github.com/carlosframework/platform/internal/maintenance	0.085s
ok  	github.com/carlosframework/platform/internal/maintenanceexec	5.503s
ok  	github.com/carlosframework/platform/internal/maintenanceguard	0.066s
ok  	github.com/carlosframework/platform/internal/metrics	0.005s
ok  	github.com/carlosframework/platform/internal/moves	21.032s
ok  	github.com/carlosframework/platform/internal/moves/archive	0.772s
ok  	github.com/carlosframework/platform/internal/objstore	0.192s
ok  	github.com/carlosframework/platform/internal/ops	0.171s
ok  	github.com/carlosframework/platform/internal/park	0.219s
ok  	github.com/carlosframework/platform/internal/privatefile	0.007s
ok  	github.com/carlosframework/platform/internal/provision	28.970s
ok  	github.com/carlosframework/platform/internal/publicrelease	0.008s
?   	github.com/carlosframework/platform/internal/regionname	[no test files]
ok  	github.com/carlosframework/platform/internal/registry	5.464s
ok  	github.com/carlosframework/platform/internal/relay	7.228s
ok  	github.com/carlosframework/platform/internal/release	22.263s
ok  	github.com/carlosframework/platform/internal/rollout	3.661s
ok  	github.com/carlosframework/platform/internal/rolloutwords	0.008s
ok  	github.com/carlosframework/platform/internal/rootaws	0.007s
ok  	github.com/carlosframework/platform/internal/rootparticipation	0.062s
ok  	github.com/carlosframework/platform/internal/rootstate	0.026s
ok  	github.com/carlosframework/platform/internal/runs	0.576s
ok  	github.com/carlosframework/platform/internal/s3compat	0.005s
ok  	github.com/carlosframework/platform/internal/safefile	0.004s
ok  	github.com/carlosframework/platform/internal/schedule	2.478s
ok  	github.com/carlosframework/platform/internal/seeds	0.011s
ok  	github.com/carlosframework/platform/internal/sim/hostlife	9.048s
ok  	github.com/carlosframework/platform/internal/sim/litestreamfacts	3.597s
ok  	github.com/carlosframework/platform/internal/status	0.042s
ok  	github.com/carlosframework/platform/internal/steering	0.006s
ok  	github.com/carlosframework/platform/internal/stopevidence	0.306s
ok  	github.com/carlosframework/platform/internal/svcrole	0.002s
ok  	github.com/carlosframework/platform/internal/telemetry	0.004s
ok  	github.com/carlosframework/platform/internal/termfence	0.004s
ok  	github.com/carlosframework/platform/internal/tunnelclient	0.004s
ok  	github.com/carlosframework/platform/internal/unixdial	0.003s
ok  	github.com/carlosframework/platform/internal/updatewords	0.004s
ok  	github.com/carlosframework/platform/internal/usage	0.004s
ok  	github.com/carlosframework/platform/internal/version	0.003s
ok  	github.com/carlosframework/platform/internal/vet	1.548s
=== cross ===
--- shard 0 of 5 ---
cross-compiling darwin/arm64
--- shard 1 of 5 ---
cross-compiling darwin/amd64
--- shard 2 of 5 ---
cross-compiling linux/arm64
linux/arm64 carlos binary: 48693410 bytes (ceiling 52000000)
--- shard 3 of 5 ---
cross-compiling windows/amd64
--- shard 4 of 5 ---
cross-compiling windows/arm64
=== minio-contract ===
minio contract: arm64, minio RELEASE.2025-09-07T16-13-09Z, mc RELEASE.2025-08-13T08-35-41Z
  minio: cached, digest verified
  mc: cached, digest verified
  minio healthy on 127.0.0.1:20999
Added `local` successfully.
Bucket created successfully `local/contract-bucket`.
Bucket created successfully `local/denied-bucket`.
Created policy `contract-only` successfully.
Added user `scopeduser` successfully.
Attached Policies: [contract-only]
To User: scopeduser
  buckets and scoped user ready
=== RUN   TestDeliveredContractMinio
=== RUN   TestDeliveredContractMinio/put
=== RUN   TestDeliveredContractMinio/get_round-trips_the_body
=== RUN   TestDeliveredContractMinio/list_shows_the_key
=== RUN   TestDeliveredContractMinio/head_and_list_agree_on_the_version
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed
=== RUN   TestDeliveredContractMinio/delete_removes_it
=== RUN   TestDeliveredContractMinio/cross-bucket_denial
--- PASS: TestDeliveredContractMinio (0.02s)
    --- PASS: TestDeliveredContractMinio/put (0.00s)
    --- PASS: TestDeliveredContractMinio/get_round-trips_the_body (0.00s)
    --- PASS: TestDeliveredContractMinio/list_shows_the_key (0.00s)
    --- PASS: TestDeliveredContractMinio/head_and_list_agree_on_the_version (0.00s)
    --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload (0.00s)
    --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route (0.01s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed (0.00s)
    --- PASS: TestDeliveredContractMinio/delete_removes_it (0.00s)
    --- PASS: TestDeliveredContractMinio/cross-bucket_denial (0.00s)
PASS
ok  	github.com/carlosframework/platform/internal/objstore	0.022s
=== RUN   TestT4StorageProbeMinio
2026/10/17 01:46:40 moves: T4: STORE: list: operation error S3: ListObjectsV2, https response error StatusCode: 403, RequestID: 18DF2DDC3E25CDB7, HostID: 7c420053e0fbcd7f41653fdd92ffa2802c903d05303b1aa15adc2e8bb65d183f, api error SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.
--- PASS: TestT4StorageProbeMinio (0.01s)
PASS
ok  	github.com/carlosframework/platform/internal/host	0.021s
minio contract: PASS
diff --git a/internal/host/activation_adoption_linux.go b/internal/host/activation_adoption_linux.go
index 5dfd18ec8..f3059ef75 100644
--- a/internal/host/activation_adoption_linux.go
+++ b/internal/host/activation_adoption_linux.go
@@ -8,6 +8,8 @@ import (
"path/filepath"
)
+var errActivationRecoveryHeld = errors.New("activation recovery held")
+
// An explicit offline transaction excludes automatic unit repair until closure.
// A process lock alone would release that exclusion on interruption or reboot.
func activationAdoptionAllowed(etc string) error {
@@ -35,6 +37,18 @@ func activationTransactionAllowed(etc string, recovery bool) error {
if err != nil {
return err
}
+ if recovery && r.Phase == "complete" {
+ // Edge startup requires this service. While closure owns the units, only
+ // an empty recovery may succeed; Run handles this sentinel before effects.
+ tx.updateRoot, err = activationConfiguredUpdateRoot(filepath.Join(etc, "carlos", "host.env"), "")
+ if err != nil {
+ return err
+ }
+ if err = tx.updateSettled(); err != nil {
+ return err
+ }
+ return errActivationRecoveryHeld
+ }
if r.Phase != "done" {
return errActivation
}
diff --git a/internal/host/activation_adoption_other.go b/internal/host/activation_adoption_other.go
index 7c83f2fc7..0f37e7f71 100644
--- a/internal/host/activation_adoption_other.go
+++ b/internal/host/activation_adoption_other.go
@@ -2,6 +2,10 @@
package host
+import "errors"
+
+var errActivationRecoveryHeld = errors.New("activation recovery held")
+
func activationAdoptionAllowed(string) error { return nil }
func activationTransactionAllowed(string, bool) error { return nil }
diff --git a/internal/host/activation_baseline_linux_test.go b/internal/host/activation_baseline_linux_test.go
new file mode 100644
index 000000000..1269efff9
--- /dev/null
+++ b/internal/host/activation_baseline_linux_test.go
@@ -0,0 +1,57 @@
+//go:build linux
+
+package host
+
+import (
+ "strings"
+ "testing"
+)
+
+func TestActivationCanonicalBaseline(t *testing.T) {
+ for _, name := range activationOfflineUnits() {
+ want := "inactive"
+ if name == activationEdgeUnit {
+ want = "active"
+ }
+ for _, trigger := range activationTriggers {
+ if name == trigger {
+ want = "active"
+ }
+ }
+ for _, active := range []string{"inactive", "active", "failed", "activating"} {
+ item := activationOfflineItem{Name: name, LoadState: "loaded", ActiveState: active, Active: active == "active"}
+ if got := activationSupportedBaseline(item); got != (active == want) {
+ t.Errorf("%s loaded/%s admitted=%t; canonical=%s", name, active, got, want)
+ }
+ }
+ item := activationOfflineItem{Name: name, LoadState: "masked", ActiveState: "inactive", Masked: true}
+ wantMask := name != activationEdgeUnit && name != "carlos-system-recover.service"
+ if got := activationSupportedBaseline(item); got != wantMask {
+ t.Errorf("%s operator mask admitted=%t; want=%t", name, got, wantMask)
+ }
+ }
+}
+
+func TestActivationCanonicalProductionUnits(t *testing.T) {
+ names := activationOfflineUnits()
+ if len(names) != len(activationCanonicalBaseline) {
+ t.Fatal("canonical set differs from migration set")
+ }
+ for i, row := range activationCanonicalBaseline {
+ if names[i] != row.Name {
+ t.Fatal("canonical order", names[i], row.Name)
+ }
+ raw, err := unitsFS.ReadFile("units/" + row.Name)
+ if err != nil {
+ t.Fatal(err)
+ }
+ text := string(raw)
+ if strings.HasSuffix(row.Name, ".service") && row.Name != activationEdgeUnit {
+ if !strings.Contains(text, "Type=oneshot\n") || strings.Contains(text, "RemainAfterExit=yes") || row.Active != "inactive" {
+ t.Fatal("oneshot baseline drift", row.Name)
+ }
+ } else if row.Active != "active" || !strings.Contains(text, "WantedBy=") {
+ t.Fatal("serving baseline drift", row.Name)
+ }
+ }
+}
diff --git a/internal/host/activation_baseline_rootrunner_linux_test.go b/internal/host/activation_baseline_rootrunner_linux_test.go
index ea4f5092c..ab8e34161 100644
--- a/internal/host/activation_baseline_rootrunner_linux_test.go
+++ b/internal/host/activation_baseline_rootrunner_linux_test.go
@@ -125,21 +125,13 @@ func TestRootActivationBaselineAdmission(t *testing.T) {
}
func TestRootActivationProductionBaselineCombinations(t *testing.T) {
- for _, edge := range []bool{false, true} {
- for _, timer := range []bool{false, true} {
- t.Run(fmt.Sprintf("edge=%t/timer=%t", edge, timer), func(t *testing.T) {
- tx := setupActivationProduction(t)
- defer cleanupActivationProduction(t, tx)
- if !edge {
- servingRunnerSystemctl(t, "stop", activationEdgeUnit)
- }
- if timer {
- servingRunnerSystemctl(t, "start", "carlos-adopt.timer")
- servingRunnerSystemctl(t, "stop", "carlos-adopt.service")
- servingRunnerSystemctl(t, "reset-failed", "carlos-adopt.service")
- }
- // Inactive operator masks are supported on both a trigger and a service.
+ for _, masks := range []bool{false, true} {
+ t.Run(fmt.Sprint("operator-masks=", masks), func(t *testing.T) {
+ tx := setupActivationProduction(t)
+ defer cleanupActivationProduction(t, tx)
+ if masks {
for _, name := range []string{"carlos-stage.path", "carlos-stage.service"} {
+ servingRunnerSystemctl(t, "stop", name)
if err := os.Remove(filepath.Join(tx.units, name)); err != nil {
t.Fatal(err)
}
@@ -148,53 +140,95 @@ func TestRootActivationProductionBaselineCombinations(t *testing.T) {
}
}
servingRunnerSystemctl(t, "daemon-reload")
- if err := tx.begin(t.Context(), true); err != nil {
- t.Fatal("dry admission", err)
- }
- if err := tx.begin(t.Context(), false); err != nil {
- t.Fatal("admission", err)
- }
- r := offlineRecord(t, tx)
- if r.Items[0].Active != timer || r.Items[len(r.Items)-1].Active != edge {
- t.Fatal("wrong recorded baseline")
- }
- // A live edge after durable closure must still be stopped when the
- // admitted edge was inactive. Keep the same recorded unit inode.
- if !edge {
- tx.checkpoint = func(point string) error {
- if point == "edge-reload" {
- servingRunnerSystemctl(t, "start", activationEdgeUnit)
- }
- return nil
+ }
+ if err := tx.begin(t.Context(), true); err != nil {
+ t.Fatal(err)
+ }
+ if err := tx.begin(t.Context(), false); err != nil {
+ t.Fatal(err)
+ }
+ if err := tx.restore(t.Context(), false); err != nil {
+ t.Fatal(err)
+ }
+ r := offlineRecord(t, tx)
+ if r.Phase != "done" || r.EdgePending || r.TriggersPending {
+ t.Fatal("unfinished", r)
+ }
+ if diff := tx.closeDifferences(t.Context(), r); len(diff) != 0 {
+ t.Fatal(diff)
+ }
+ assertActivationProductionEdge(t)
+ })
+ }
+}
+
+func TestRootActivationProductionWholeRecordGate(t *testing.T) {
+ for _, persistent := range []bool{false, true} {
+ t.Run(fmt.Sprint("persistent=", persistent), func(t *testing.T) {
+ tx := setupActivationProduction(t)
+ defer cleanupActivationProduction(t, tx)
+ if err := tx.begin(t.Context(), false); err != nil {
+ t.Fatal(err)
+ }
+ checks := 0
+ tx.checkpoint = func(point string) error {
+ if point == "verify-completion" {
+ checks++
+ if persistent || checks == 1 {
+ servingRunnerSystemctl(t, "stop", activationEdgeUnit)
}
}
- if err := tx.restore(t.Context(), false); err != nil {
- t.Fatal("restore", err)
- }
- r = offlineRecord(t, tx)
- if r.Phase != "done" || r.EdgePending || r.TriggersPending {
- t.Fatal("unfinished", r.Phase)
- }
- for _, item := range r.Items {
- load, active, err := tx.status(t.Context(), item.Name)
- wantLoad, wantActive := activationRecordedState(item)
- if err != nil || load != wantLoad || active != wantActive {
- t.Errorf("%s: %s/%s want %s/%s: %v", item.Name, load, active, wantLoad, wantActive, err)
- }
+ return nil
+ }
+ err := tx.restore(t.Context(), false)
+ r := offlineRecord(t, tx)
+ if persistent {
+ if err == nil || !strings.Contains(err.Error(), activationEdgeUnit) || r.Phase != "complete" || checks != 3 {
+ t.Fatalf("closed on drift: phase=%s checks=%d err=%v", r.Phase, checks, err)
}
- t.Logf("production admission-to-completion edge=%t timer=%t", edge, timer)
- })
- }
+ } else if err != nil || r.Phase != "done" || checks != 2 {
+ t.Fatalf("did not converge: %s %d %v", r.Phase, checks, err)
+ }
+ // Both flags are clear here. Every continuation must still verify all units.
+ tx.checkpoint = nil
+ servingRunnerSystemctl(t, "stop", activationEdgeUnit)
+ if err := tx.restore(t.Context(), false); err != nil {
+ t.Fatal("resume", err)
+ }
+ if diff := tx.closeDifferences(t.Context(), offlineRecord(t, tx)); len(diff) != 0 {
+ t.Fatal(diff)
+ }
+ })
}
}
func TestRootActivationProductionUnsupportedBaselines(t *testing.T) {
- for _, scenario := range []string{"failed-edge", "masked-active-trigger", "activating-unit"} {
+ for _, scenario := range []string{"failed-edge", "masked-active-trigger", "activating-unit", "masked-recovery", "inactive-edge", "inactive-timer", "active-oneshot"} {
t.Run(scenario, func(t *testing.T) {
tx := setupActivationProduction(t)
defer cleanupActivationProduction(t, tx)
name, load, active := activationEdgeUnit, "loaded", "failed"
switch scenario {
+ case "masked-recovery":
+ name, load, active = "carlos-system-recover.service", "masked", "inactive"
+ if err := os.Remove(filepath.Join(tx.units, name)); err != nil {
+ t.Fatal(err)
+ }
+ if err := os.Symlink("/dev/null", filepath.Join(tx.units, name)); err != nil {
+ t.Fatal(err)
+ }
+ servingRunnerSystemctl(t, "daemon-reload")
+ case "inactive-edge", "inactive-timer":
+ active = "inactive"
+ if scenario == "inactive-timer" {
+ name = "carlos-adopt.timer"
+ }
+ servingRunnerSystemctl(t, "stop", name)
+ case "active-oneshot":
+ name, active = "carlos-config-sync.service", "active"
+ activationProductionWrite(t, filepath.Join(tx.units, name+".d/95-active.conf"), []byte("[Service]\nExecStart=\nExecStart=/bin/true\nRemainAfterExit=yes\n"), 0644)
+ servingRunnerSystemctl(t, "daemon-reload")
+ servingRunnerSystemctl(t, "start", name)
case "failed-edge":
activationProductionWrite(t, filepath.Join(tx.units, activationEdgeUnit+".d/95-failure.conf"), []byte("[Service]\nRestart=no\n"), 0644)
servingRunnerSystemctl(t, "daemon-reload")
@@ -232,43 +266,22 @@ func TestRootActivationProductionUnsupportedBaselines(t *testing.T) {
}
func TestRootActivationExactEdgeVerification(t *testing.T) {
- for _, want := range []string{"active", "inactive"} {
- t.Run(want, func(t *testing.T) {
- tx, _ := offlineFixture(t)
- real := tx.systemctl
- active := want
- tx.systemctl = func(ctx context.Context, args ...string) ([]byte, error) {
- if len(args) > 1 && args[1] == activationEdgeUnit {
- switch args[0] {
- case "start":
- active = "active"
- case "stop":
- active = "inactive"
- case "show":
- raw, err := real(ctx, args...)
- return []byte(strings.ReplaceAll(string(raw), "ActiveState=inactive", "ActiveState="+active)), err
- }
- }
- return real(ctx, args...)
- }
- if err := tx.begin(t.Context(), false); err != nil {
- t.Fatal(err)
- }
- if err := tx.restore(t.Context(), false); err != nil {
- t.Fatal(err)
- }
- r := offlineRecord(t, tx)
- if diff := tx.closeDifferences(t.Context(), r); len(diff) != 0 {
- t.Fatal("exact restored state rejected", diff)
- }
- active = "active"
- if want == "active" {
- active = "inactive"
- }
- diff := strings.Join(tx.closeDifferences(t.Context(), r), "\n")
- if !strings.Contains(diff, activationEdgeUnit+": load=loaded active="+active+"; want load=loaded active="+want) {
- t.Fatal("edge target substituted", diff)
- }
- })
+ tx, _ := offlineFixture(t)
+ if err := tx.begin(t.Context(), false); err != nil {
+ t.Fatal(err)
+ }
+ if err := tx.restore(t.Context(), false); err != nil {
+ t.Fatal(err)
+ }
+ r := offlineRecord(t, tx)
+ if diff := tx.closeDifferences(t.Context(), r); len(diff) != 0 {
+ t.Fatal(diff)
+ }
+ if _, err := tx.systemctl(t.Context(), "stop", activationEdgeUnit); err != nil {
+ t.Fatal(err)
+ }
+ diff := strings.Join(tx.closeDifferences(t.Context(), r), "\n")
+ if !strings.Contains(diff, activationEdgeUnit+": load=loaded active=inactive; want load=loaded active=active") {
+ t.Fatal("edge omitted", diff)
}
}
diff --git a/internal/host/activation_invariants_rootrunner_linux_test.go b/internal/host/activation_invariants_rootrunner_linux_test.go
index e97388655..744cf9381 100644
--- a/internal/host/activation_invariants_rootrunner_linux_test.go
+++ b/internal/host/activation_invariants_rootrunner_linux_test.go
@@ -106,7 +106,7 @@ func TestRootActivationCloseInvariant(t *testing.T) {
}
}
-func TestRootActivationTriggerAfterClose(t *testing.T) {
+func TestRootActivationTriggerBeforeFinalGate(t *testing.T) {
tx, _ := offlineFixture(t)
base := tx.systemctl
active := true
@@ -114,8 +114,8 @@ func TestRootActivationTriggerAfterClose(t *testing.T) {
if args[1%len(args)] == "carlos-adopt.timer" {
switch args[0] {
case "start":
- if offlineRecord(t, tx).Phase != "done" {
- return nil, errors.New("timer started before done")
+ if offlineRecord(t, tx).Phase != "complete" {
+ return nil, errors.New("timer started before complete")
}
active = true
case "stop":
@@ -260,7 +260,7 @@ func TestRootActivationDoneUpdateRace(t *testing.T) {
}()
tx.checkpoint = func(point string) error {
r := offlineRecord(t, tx)
- if point == "journal-done" && !r.EdgePending && r.TriggersPending {
+ if point == "edge-clear" && !r.EdgePending && r.TriggersPending {
var err error
held, err = tx.settled(t.Context(), false)
return err
@@ -272,7 +272,7 @@ func TestRootActivationDoneUpdateRace(t *testing.T) {
t.Fatal("update race admitted", err)
}
r := offlineRecord(t, tx)
- if r.Phase != "done" || r.EdgePending || !r.TriggersPending {
+ if r.Phase != "complete" || r.EdgePending || !r.TriggersPending {
t.Fatal("lost continuation", r)
}
held()
diff --git a/internal/host/activation_matrix_rootrunner_linux_test.go b/internal/host/activation_matrix_rootrunner_linux_test.go
index 796cc0450..e083cca9d 100644
--- a/internal/host/activation_matrix_rootrunner_linux_test.go
+++ b/internal/host/activation_matrix_rootrunner_linux_test.go
@@ -5,31 +5,21 @@ package host
import (
"bytes"
"context"
- "crypto/ed25519"
"encoding/json"
"errors"
- "fmt"
"os"
"path/filepath"
- "reflect"
"strings"
"testing"
"time"
-
- "github.com/carlosframework/platform/internal/hostactors"
- "github.com/carlosframework/platform/internal/hostfloor"
- "github.com/carlosframework/platform/internal/release"
)
const activationMatrixDir = "/var/lib/carlos-activation-matrix"
type activationMatrixCursor struct {
- Cut int
- Timer bool
- Resume bool
- Boot string
- Point string
- Pending bool
+ Cut int
+ Resume bool
+ Boot, Point string
}
func activationMatrixTransaction() activationOffline {
@@ -42,20 +32,7 @@ func activationMatrixPending(t *testing.T, tx activationOffline, operation func(
if _, err := os.Lstat(path); !os.IsNotExist(err) {
t.Fatal("pre-existing update journal", err)
}
- // A valid interrupted install, with the fixture's real previous executable.
- raw, err := json.Marshal(systemJournal{Phase: systemPhaseInstalled, Target: "v2.0.0", PrevBuild: "v1.0.0", StartedAt: time.Now()})
- if err != nil {
- t.Fatal(err)
- }
- activationProductionWrite(t, path, raw, 0600)
- activationMatrixRefusePending(t, tx, operation)
- if err := os.Remove(path); err != nil {
- t.Fatal(err)
- }
-}
-
-func activationMatrixRefusePending(t *testing.T, tx activationOffline, operation func(context.Context, bool) error) {
- t.Helper()
+ activationProductionWrite(t, path, []byte("pending"), 0600)
before, _ := os.ReadFile(filepath.Join(tx.state, "transaction.json"))
err := operation(t.Context(), false)
if err == nil || !strings.Contains(err.Error(), "settle the update first") {
@@ -65,81 +42,49 @@ func activationMatrixRefusePending(t *testing.T, tx activationOffline, operation
if !bytes.Equal(before, after) {
t.Fatal("refusal changed transaction")
}
+ if err := os.Remove(path); err != nil {
+ t.Fatal(err)
+ }
}
-func activationMatrixBaseline(t *testing.T, tx activationOffline, timer bool) {
+func activationMatrixBaseline(t *testing.T, tx activationOffline) {
t.Helper()
- servingRunnerSystemctl(t, "stop", activationEdgeUnit, "carlos-adopt.timer")
servingRunnerSystemctl(t, append([]string{"stop"}, rootMigrationOfflineUnits...)...)
servingRunnerSystemctl(t, "reset-failed")
if err := os.RemoveAll(tx.state); err != nil {
t.Fatal(err)
}
- if timer {
- servingRunnerSystemctl(t, "start", "carlos-adopt.timer")
- // The production OnBootSec timer can run immediately. Wait for the
- // real adoption process to settle before recording the baseline.
- for n := 0; n < 100; n++ {
- raw, err := tx.systemctl(t.Context(), "show", "carlos-adopt.service", "--property=ActiveState", "--value")
- if err != nil {
- t.Fatal(err)
- }
- if strings.TrimSpace(string(raw)) != "activating" {
- break
- }
- time.Sleep(20 * time.Millisecond)
- }
- servingRunnerSystemctl(t, "stop", "carlos-adopt.service")
- servingRunnerSystemctl(t, "reset-failed", "carlos-adopt.service")
- }
+ servingRunnerSystemctl(t, "start", activationEdgeUnit)
+ servingRunnerSystemctl(t, append([]string{"start"}, activationTriggers...)...)
+ servingRunnerSystemctl(t, "stop", "carlos-adopt.service")
+ servingRunnerSystemctl(t, "reset-failed", "carlos-adopt.service")
activationMatrixPending(t, tx, tx.begin)
if err := tx.begin(t.Context(), false); err != nil {
t.Fatal(err)
}
- r := offlineRecord(t, tx)
- if r.Items[0].Active != timer {
- t.Fatal("wrong timer baseline", r.Items[0])
- }
}
func activationMatrixCheck(t *testing.T, tx activationOffline, point string) {
t.Helper()
r := offlineRecord(t, tx)
- if r.Phase != "done" {
+ if r.Phase != "complete" && r.Phase != "done" {
if target, _ := os.Readlink(filepath.Join(tx.units, activationEdgeUnit)); target != "/dev/null" {
- t.Fatal("edge released while open", point, r.Phase)
- }
- for _, item := range r.Items {
- if err := tx.checkItem(item); err != nil {
- t.Fatal("foreign inode while open", point, item.Name, err)
- }
- }
- }
- if point == "journal-done" && r.EdgePending {
- if differences := tx.heldDifferences(t.Context(), r); len(differences) != 0 {
- t.Fatal("closed on mismatch", differences)
+ t.Fatal("edge released before complete", point, r.Phase)
}
}
- if strings.HasPrefix(point, "start-") && strings.HasSuffix(point, ".timer") && r.Phase != "done" {
- t.Fatal("timer activated before closure")
- }
- if r.Phase == "done" && (point == "stage-"+activationEdgeUnit || point == "replace-"+activationEdgeUnit || strings.HasPrefix(point, "start-") && (strings.HasSuffix(point, ".timer") || strings.HasSuffix(point, ".path"))) {
- lock, err := os.OpenFile(filepath.Join(tx.updateDir(), systemLockFile), os.O_RDWR, 0)
- if err != nil {
- t.Fatal(err)
+ if point == "journal-done" {
+ if r.EdgePending || r.TriggersPending {
+ t.Fatal("pending work at done")
}
- err = flockExclusiveNonblock(lock)
- lock.Close()
- if err == nil {
- t.Fatal("continuation effect without updater exclusion", point)
+ if diff := tx.closeDifferences(t.Context(), r); len(diff) != 0 {
+ t.Fatal("whole-record mismatch at done", diff)
}
}
}
func activationMatrixInterrupt(t *testing.T, tx activationOffline, cut int) (string, bool) {
t.Helper()
- n := 0
- point := ""
+ n, point := 0, ""
stop := errors.New("matrix interruption")
tx.checkpoint = func(name string) error {
activationMatrixCheck(t, tx, name)
@@ -159,113 +104,43 @@ func activationMatrixInterrupt(t *testing.T, tx activationOffline, cut int) (str
func activationMatrixResume(t *testing.T, tx activationOffline, point string) {
t.Helper()
- before := offlineRecord(t, tx)
- foreign := map[string]activationOfflineItem{}
- if before.Phase == "done" {
- for _, item := range before.Items {
- path := filepath.Join(tx.units, item.Name)
- if sameActivationFile(path, filepath.Join(tx.state, item.Name+".original")) || sameActivationFile(path, filepath.Join(tx.state, item.Name+".mask")) {
- continue
- }
- snapshot, err := activationUnitSnapshot(path, item.Name)
- if err != nil {
- t.Fatal(err)
- }
- foreign[item.Name] = snapshot
- }
- real := tx.systemctl
- tx.systemctl = func(ctx context.Context, args ...string) ([]byte, error) {
- if len(args) > 1 && (args[0] == "start" || args[0] == "stop" || args[0] == "reset-failed") {
- if _, exists := foreign[args[1]]; exists {
- t.Fatal("closed continuation reclaimed service state", args)
- }
- }
- return real(ctx, args...)
- }
- }
- if before.Phase != "done" {
- activationMatrixPending(t, tx, tx.restore)
- }
- tx.checkpoint = func(name string) error {
- for unit := range foreign {
- if name == "stage-"+unit || name == "replace-"+unit {
- t.Fatal("closed continuation replaced released unit", unit, name)
- }
- }
- activationMatrixCheck(t, tx, name)
- return nil
- }
deadline := time.Now().Add(20 * time.Second)
for {
- saved, err := os.ReadFile(filepath.Join(tx.state, "transaction.json"))
- if err != nil {
- t.Fatal(err)
- }
- err = tx.restore(t.Context(), false)
+ err := tx.restore(t.Context(), false)
if err == nil {
break
}
if !strings.HasPrefix(err.Error(), "claim activation: settle the update first:") || time.Now().After(deadline) {
t.Fatal("resume", point, err)
}
- after, e := os.ReadFile(filepath.Join(tx.state, "transaction.json"))
- if e != nil || !bytes.Equal(saved, after) {
- t.Fatal("busy recovery refusal changed transaction", e)
- }
- t.Log("boot recovery still settling; unchanged refusal", err)
time.Sleep(50 * time.Millisecond)
}
r := offlineRecord(t, tx)
if r.Phase != "done" || r.EdgePending || r.TriggersPending {
- t.Fatal("unfinished continuation", r.Phase, r.EdgePending, r.TriggersPending)
+ t.Fatal("unfinished continuation", r)
}
- for name, snapshot := range foreign {
- path := filepath.Join(tx.units, name)
- got, err := activationUnitSnapshot(path, name)
- if err != nil || sameActivationFile(path, filepath.Join(tx.state, name+".original")) || sameActivationFile(path, filepath.Join(tx.state, name+".mask")) {
- t.Fatal("closed continuation reclaimed released unit", name, err)
- }
- if !reflect.DeepEqual(got, snapshot) {
- t.Log("ordinary writer changed released unit; continuation made no replacement", name)
- }
- }
- // Once done is durable, timer-driven adoption is ordinary operation. The
- // close checkpoint, not a later live service observation, checks equality.
- for _, item := range r.Items[:len(activationTriggers)] {
- if item.Active && sameActivationFile(filepath.Join(tx.units, item.Name), filepath.Join(tx.state, item.Name+".original")) {
- _, active, err := tx.status(t.Context(), item.Name)
- if err != nil || active != "active" {
- t.Fatal("active trigger not restored", item.Name, active, err)
- }
- }
+ if diff := tx.closeDifferences(t.Context(), r); len(diff) != 0 {
+ t.Fatal("whole-record completion", point, diff)
}
+ assertActivationProductionEdge(t)
}
func TestRootActivationProductionInvariantMatrix(t *testing.T) {
tx := setupActivationProduction(t)
defer cleanupActivationProduction(t, tx)
- activationMatrixTimer(t)
- for _, timer := range []bool{false, true} {
- complete := false
- for cut := 1; cut <= 120; cut++ {
- activationMatrixBaseline(t, tx, timer)
- point, interrupted := activationMatrixInterrupt(t, tx, cut)
- activationMatrixResume(t, tx, point)
- t.Logf("matrix reboot=false timer=%t pending={open:refused,closed:ordinary-recovery,absent:restored} cut=%d point=%s", timer, cut, point)
- if !interrupted {
- complete = true
- break
- }
- }
- if !complete {
- t.Fatal("matrix did not enumerate all boundaries")
+ for cut := 1; cut <= 120; cut++ {
+ activationMatrixBaseline(t, tx)
+ point, interrupted := activationMatrixInterrupt(t, tx, cut)
+ activationMatrixResume(t, tx, point)
+ t.Logf("canonical matrix reboot=false cut=%d point=%s", cut, point)
+ if !interrupted {
+ return
}
}
+ t.Fatal("matrix did not enumerate all boundaries")
}
-// The designated runner invokes this test from a persistent boot unit. Each
-// successful invocation prepares exactly one actual reboot; cursor and boot ID
-// are durable. Pending admission is a refused branch, so it has no checkpoints.
+// The runner's persistent unit executes one cut per boot, using real boot IDs.
func TestRootActivationProductionInvariantBootStep(t *testing.T) {
var c activationMatrixCursor
cursor := activationMatrixDir + "/cursor.json"
@@ -273,8 +148,6 @@ func TestRootActivationProductionInvariantBootStep(t *testing.T) {
tx := activationMatrixTransaction()
if os.IsNotExist(err) {
tx = setupActivationProduction(t)
- activationMatrixTimer(t)
- activationMatrixOwnTmpfiles(t)
c.Cut = 1
} else if err != nil || json.Unmarshal(raw, &c) != nil {
t.Fatal("cursor", err)
@@ -287,284 +160,39 @@ func TestRootActivationProductionInvariantBootStep(t *testing.T) {
if c.Boot == string(boot) {
t.Fatal("no actual reboot")
}
- if !c.Pending {
- if _, err := os.Lstat(filepath.Join(tx.updateDir(), "journal.json")); !os.IsNotExist(err) {
- t.Fatal("absent-journal reboot resumed with a journal", err)
- }
- }
- activationMatrixCheck(t, tx, "boot-resume")
- if c.Pending && offlineRecord(t, tx).Phase == "done" {
- activationMatrixRecovered(t, tx)
- } else if c.Pending {
- activationMatrixRefusePending(t, tx, tx.restore)
- if err := os.Remove(filepath.Join(tx.updateDir(), "journal.json")); err != nil {
- t.Fatal(err)
+ if c.Point == "edge-clear" || c.Point == "trigger-complete" {
+ if offlineRecord(t, tx).EdgePending {
+ t.Fatal("edge flag was not durably clear")
}
+ servingRunnerSystemctl(t, "stop", activationEdgeUnit)
+ t.Log("post-boot edge drift with EdgePending=false", c.Point)
}
activationMatrixResume(t, tx, c.Point)
- t.Logf("matrix reboot=true timer=%t pending={open:refused,closed:ordinary-recovery,absent:restored} cut=%d point=%s boot_pending=%t boot=%s", c.Timer, c.Cut, c.Point, c.Pending, strings.TrimSpace(string(boot)))
+ t.Logf("canonical matrix reboot=true cut=%d point=%s boot=%s", c.Cut, c.Point, strings.TrimSpace(string(boot)))
c.Cut++
}
if c.Cut > 120 {
t.Fatal("unbounded checkpoint matrix")
}
- for {
- activationMatrixBaseline(t, tx, c.Timer)
- point, interrupted := activationMatrixInterrupt(t, tx, c.Cut)
- if !interrupted {
- activationMatrixResume(t, tx, point)
- t.Logf("matrix completion-control timer=%t boot_pending=%t cut=%d interrupted=false", c.Timer, os.Getenv("ACTIVATION_MATRIX_PENDING") != "absent", c.Cut)
- if c.Timer {
- cleanupActivationProduction(t, tx)
- if err := os.Remove("/etc/tmpfiles.d/carlos.conf"); err != nil && !os.IsNotExist(err) {
- t.Fatal(err)
- }
- activationProductionWrite(t, activationMatrixDir+"/control", []byte("done\n"), 0600)
- return
- }
- c.Timer = true
- c.Cut = 1
- continue
- }
- c.Pending = os.Getenv("ACTIVATION_MATRIX_PENDING") != "absent"
- if !c.Pending {
- if _, err := os.Lstat(filepath.Join(tx.updateDir(), "journal.json")); !os.IsNotExist(err) {
- t.Fatal("absent-journal reboot prepared with a journal", err)
- }
- }
- if c.Pending && offlineRecord(t, tx).Phase == "done" {
- activationMatrixClosedJournal(t, tx)
- } else if c.Pending {
- raw, err := json.Marshal(systemJournal{Phase: systemPhaseInstalled, Target: "v2.0.0", PrevBuild: "v1.0.0", StartedAt: time.Now()})
- if err != nil {
- t.Fatal(err)
- }
- if err := writeFileDurable(filepath.Join(tx.updateDir(), "journal.json"), raw, 0600); err != nil {
- t.Fatal(err)
- }
- }
- c.Point = point
- c.Resume = true
- c.Boot = string(boot)
- raw, err := json.Marshal(c)
- if err != nil {
- t.Fatal(err)
- }
- if err := writeFileDurable(cursor, raw, 0600); err != nil {
- t.Fatal(err)
- }
- if err := writeFileDurable(activationMatrixDir+"/control", []byte("reboot\n"), 0600); err != nil {
- t.Fatal(err)
- }
- t.Log(fmt.Sprintf("actual reboot prepared: timer=%t cut=%d point=%s", c.Timer, c.Cut, c.Point))
+ activationMatrixBaseline(t, tx)
+ point, interrupted := activationMatrixInterrupt(t, tx, c.Cut)
+ if !interrupted {
+ activationMatrixResume(t, tx, point)
+ t.Logf("canonical matrix completion-control cut=%d", c.Cut)
+ cleanupActivationProduction(t, tx)
+ activationProductionWrite(t, activationMatrixDir+"/control", []byte("done\n"), 0600)
return
}
-}
-
-// Keep the production timer, service, executable and dependencies; make its
-// boot deadline overdue on every boot so activation really can fire it.
-func activationMatrixTimer(t *testing.T) {
- activationProductionWrite(t, "/etc/systemd/system/carlos-adopt.timer.d/10-runner-deadline.conf", []byte("[Timer]\nOnBootSec=\nOnBootSec=1us\nRandomizedDelaySec=0\n"), 0644)
- servingRunnerSystemctl(t, "daemon-reload")
-}
-
-// Exercise recovery's real unit replacement at every closed continuation cut.
-// The staged rollback set contains the production units this fixture owns.
-func activationMatrixClosedJournal(t *testing.T, tx activationOffline) {
- t.Helper()
- if offlineRecord(t, tx).Phase != "done" {
- t.Fatal("pending recovery fixture before closure")
- }
- prev := filepath.Join(tx.updateDir(), "staging", "prev")
- tmp, err := os.ReadFile("/etc/tmpfiles.d/carlos-activation-runner.conf")
- if err != nil {
- t.Fatal(err)
- }
- activationProductionWrite(t, filepath.Join(prev, "etc/tmpfiles.d/carlos.conf"), tmp, 0644)
- for _, name := range activationProductionNames() {
- raw, err := unitsFS.ReadFile("units/" + name)
- if err != nil {
- t.Fatal(err)
- }
- activationProductionWrite(t, filepath.Join(prev, "etc/systemd/system", name), raw, 0644)
- }
- keyBytes, err := os.ReadFile(offlineRebootState + "/signing-key")
- if err != nil {
- t.Fatal(err)
- }
- key := ed25519.PrivateKey(keyBytes)
- disk := hostfloor.Disk{Dir: "/opt/carlos/system/guard", Bucket: "activation-seed-fixture", PublicKey: key.Public().(ed25519.PublicKey)}
- _, floor, err := disk.Load()
- if err != nil {
- t.Fatal(err)
- }
- id := fmt.Sprintf("%032x", time.Now().UnixNano())
- enrollment := strings.Repeat("a", 32)
- hash := hostfloor.Digest(activationCarlosImage)
- actor := "box:seed-fixture"
- box := hostactors.Box{Enrollment: enrollment, Namespace: "test", BoxID: "seed-fixture", Label: "fixture", Region: "ie", Pool: "default", Cloud: hostactors.Cloud{Provider: "aws", Account: "fixture", Region: "eu-west-1", InstanceID: "i-fixture"}, EvidenceSHA256: strings.Repeat("b", 64), State: "active"}
- reservation := hostactors.Reservation{ID: id, FloorSHA256: floor.Digest, Actors: map[string]string{actor: enrollment}, Releases: map[string]hostactors.Report{actor: {Enrollment: enrollment}}, Hashes: map[string][]string{actor: {hash, hash, hash}}}
- roster := hostactors.Roster{V: 1, DeploymentBucket: disk.Bucket, Revision: 1, Boxes: map[string]hostactors.Box{"seed-fixture": box}, Consoles: map[string]hostactors.Console{}, Holds: map[string]hostactors.Reservation{}, Activations: map[string]hostactors.Reservation{id: reservation}}
- if err := roster.Validate(disk.Bucket); err != nil {
- t.Fatal(err)
- }
- wire, err := release.SignRootEnvelope(release.RootSigner(key), roster)
- if err != nil {
- t.Fatal(err)
- }
- guard, err := json.Marshal(guardTransaction{Roster: wire, ID: id, Actor: actor, Enrollment: enrollment})
- if err != nil {
- t.Fatal(err)
- }
- if err := hostfloor.Durable(filepath.Join(disk.Dir, "activation.json"), guard, 0644); err != nil {
- t.Fatal(err)
- }
- if err := os.MkdirAll("/data/.hostexec", 0755); err != nil {
- t.Fatal(err)
- }
- if err := disk.Begin(t.Context(), id, "/data"); err != nil {
- t.Fatal(err)
- }
-
- raw, err := json.Marshal(bootstrapPlan{Enabled: []string{"carlos-edge.service", "carlos-system-recover.service", "carlos-adopt.timer"}})
+ c.Point, c.Resume, c.Boot = point, true, string(boot)
+ raw, err = json.Marshal(c)
if err != nil {
t.Fatal(err)
}
- activationProductionWrite(t, filepath.Join(prev, "plan.json"), raw, 0600)
- activationProductionWrite(t, prevVersionPath("/opt/carlos"), []byte("v1.0.0\n"), 0600)
- raw, err = json.Marshal(systemJournal{Phase: systemPhaseInstalled, Target: "v2.0.0", PrevBuild: "v1.0.0", StartedAt: time.Now()})
- if err != nil {
+ if err := writeFileDurable(cursor, raw, 0600); err != nil {
t.Fatal(err)
}
- if err := writeFileDurable(filepath.Join(tx.updateDir(), "journal.json"), raw, 0600); err != nil {
+ if err := writeFileDurable(activationMatrixDir+"/control", []byte("reboot\n"), 0600); err != nil {
t.Fatal(err)
}
-}
-
-func activationMatrixRecovered(t *testing.T, tx activationOffline) {
- t.Helper()
- deadline := time.Now().Add(20 * time.Second)
- for {
- servingRunnerSystemctl(t, "start", "carlos-system-recover.service")
- _, err := os.Lstat(filepath.Join(tx.updateDir(), "journal.json"))
- if os.IsNotExist(err) {
- break
- }
- if err != nil || time.Now().After(deadline) {
- t.Fatal("recovery left pending journal", err)
- }
- // The overdue adoption timer can briefly hold the shared update lock.
- // A successful deferred start is not evidence that recovery completed.
- r := offlineRecord(t, tx)
- if r.EdgePending || r.TriggersPending {
- activationMatrixRefusePending(t, tx, tx.restore)
- }
- t.Log("recovery deferred; journal retained; retrying within deadline")
- time.Sleep(100 * time.Millisecond)
- }
- raw, err := os.ReadFile(filepath.Join(tx.updateDir(), "result.json"))
- if err != nil || !bytes.Contains(raw, []byte(systemOutcomeRolledBack)) {
- t.Fatal("recovery did not record rollback", string(raw), err)
- }
- if err := tx.updateSettled(); err != nil {
- t.Fatal("recovery did not settle update state", err)
- }
- guardRaw, err := os.ReadFile("/opt/carlos/system/guard/activation.json")
- var guard guardTransaction
- if err != nil || json.Unmarshal(guardRaw, &guard) != nil {
- t.Fatal("recovered guard transaction", err)
- }
- completed, err := os.ReadFile("/opt/carlos/system/guard/completed.json")
- if err != nil || string(completed) != guard.ID {
- t.Fatal("recovery did not complete this reservation", err)
- }
- for _, item := range offlineRecord(t, tx).Items {
- if sameActivationFile(filepath.Join(tx.units, item.Name), filepath.Join(tx.state, item.Name+".original")) {
- t.Fatal("recovery did not replace production unit", item.Name)
- }
- }
-}
-
-func TestRootActivationProductionDonePendingMatrix(t *testing.T) {
- for _, order := range []string{"pending-first", "recovery-first"} {
- t.Run(order, func(t *testing.T) {
- tx := setupActivationProduction(t)
- defer cleanupActivationProduction(t, tx)
- activationMatrixOwnTmpfiles(t)
- defer os.Remove("/etc/tmpfiles.d/carlos.conf")
- activationMatrixTimer(t)
- for _, timer := range []bool{false, true} {
- activationMatrixBaseline(t, tx, timer)
- var cuts []int
- count := 0
- tx.checkpoint = func(point string) error {
- count++
- activationMatrixCheck(t, tx, point)
- if offlineRecord(t, tx).Phase == "done" {
- cuts = append(cuts, count)
- }
- return nil
- }
- if err := tx.restore(t.Context(), false); err != nil {
- t.Fatal(err)
- }
- tx.checkpoint = nil
- if len(cuts) == 0 {
- t.Fatal("no closed continuation checkpoints")
- }
- complete := false
- for cut := cuts[0]; cut <= 120; cut++ {
- activationMatrixBaseline(t, tx, timer)
- point, interrupted := activationMatrixInterrupt(t, tx, cut)
- r := offlineRecord(t, tx)
- if r.Phase != "done" {
- t.Fatal("closed cut changed", cut, point, r.Phase)
- }
- if !interrupted && (r.EdgePending || r.TriggersPending) {
- t.Fatal("uninterrupted control left continuation pending")
- }
- activationMatrixClosedJournal(t, tx)
- if order == "pending-first" {
- if r.EdgePending || r.TriggersPending {
- activationMatrixRefusePending(t, tx, tx.restore)
- } else {
- saved, err := os.ReadFile(filepath.Join(tx.state, "transaction.json"))
- if err != nil {
- t.Fatal(err)
- }
- noop := tx
- noop.systemctl = func(context.Context, ...string) ([]byte, error) {
- t.Fatal("finished transaction called systemctl")
- return nil, nil
- }
- if err := noop.restore(t.Context(), false); err != nil {
- t.Fatal("finished transaction resumed work", err)
- }
- after, err := os.ReadFile(filepath.Join(tx.state, "transaction.json"))
- if err != nil || !bytes.Equal(saved, after) {
- t.Fatal("finished transaction changed", err)
- }
- }
- }
- activationMatrixRecovered(t, tx)
- activationMatrixResume(t, tx, point)
- t.Logf("post-done reboot=false timer=%t pending=recovered order=%s cut=%d point=%s interrupted=%t", timer, order, cut, point, interrupted)
- if !interrupted {
- complete = true
- break
- }
- }
- if !complete {
- t.Fatal("post-done matrix did not reach uninterrupted completion")
- }
- }
- })
- }
-}
-
-func activationMatrixOwnTmpfiles(t *testing.T) {
- t.Helper()
- if _, err := os.Lstat("/etc/tmpfiles.d/carlos.conf"); !os.IsNotExist(err) {
- t.Fatal("pre-existing tmpfiles configuration", err)
- }
+ t.Logf("actual canonical reboot prepared cut=%d point=%s", c.Cut, point)
}
diff --git a/internal/host/activation_offline_linux.go b/internal/host/activation_offline_linux.go
index 2acef2f38..a6f068bef 100644
--- a/internal/host/activation_offline_linux.go
+++ b/internal/host/activation_offline_linux.go
@@ -195,15 +195,40 @@ func (t activationOffline) read(state *os.File) (activationOfflineRecord, error)
}
return r, nil
}
+
+// units/*.service are non-remaining oneshots; the timer and paths are armed
+// on a healthy box. guardarm.go makes recovery a required edge dependency.
+// Existing inactive operator masks are the only exceptions in this table.
+var activationCanonicalBaseline = []struct {
+ Name, Load, Active string
+ OperatorMask bool
+}{
+ {"carlos-adopt.timer", "loaded", "active", true},
+ {"carlos-adopt.path", "loaded", "active", true},
+ {"carlos-stage.path", "loaded", "active", true},
+ {"carlos-restart.path", "loaded", "active", true},
+ {"carlos-system-update.path", "loaded", "active", true},
+ {"carlos-adopt.service", "loaded", "inactive", true},
+ {"carlos-stage.service", "loaded", "inactive", true},
+ {"carlos-config-sync.service", "loaded", "inactive", true},
+ {"carlos-restart.service", "loaded", "inactive", true},
+ {"carlos-system-update.service", "loaded", "inactive", true},
+ {"carlos-system-recover.service", "loaded", "inactive", false},
+ {"carlos-edge.service", "loaded", "active", false},
+}
+
func activationSupportedBaseline(item activationOfflineItem) bool {
load, active := activationRecordedState(item)
- if active != "active" && active != "inactive" {
- return false
- }
- if item.Masked {
- return item.Name != activationEdgeUnit && load == "masked" && active == "inactive"
+ for _, canonical := range activationCanonicalBaseline {
+ if canonical.Name != item.Name {
+ continue
+ }
+ if item.Masked {
+ return canonical.OperatorMask && load == "masked" && active == "inactive" && !item.Active
+ }
+ return load == canonical.Load && active == canonical.Active && item.Active == (active == "active")
}
- return load == "loaded"
+ return false
}
func (t activationOffline) baseline(ctx context.Context) ([]activationOfflineItem, error) {
@@ -214,12 +239,16 @@ func (t activationOffline) baseline(ctx context.Context) ([]activationOfflineIte
defer units.Close()
var items []activationOfflineItem
var unsupported []string
- for _, name := range activationOfflineUnits() {
+ for _, canonical := range activationCanonicalBaseline {
+ name := canonical.Name
item, fileErr := activationUnitSnapshot(filepath.Join(t.units, name), name)
load, active, stateErr := t.status(ctx, name)
item.LoadState, item.ActiveState, item.Active = load, active, active == "active"
if fileErr != nil || stateErr != nil || !activationSupportedBaseline(item) {
- detail := fmt.Sprintf("%s: load=%s active=%s file_mask=%t", name, load, active, item.Masked)
+ detail := fmt.Sprintf("%s: load=%s active=%s file_mask=%t; want load=%s active=%s file_mask=false", name, load, active, item.Masked, canonical.Load, canonical.Active)
+ if canonical.OperatorMask {
+ detail += "; operator_mask=masked/inactive"
+ }
if err := errors.Join(fileErr, stateErr); err != nil {
detail += ": " + err.Error()
}
@@ -487,14 +516,19 @@ func (t activationOffline) restore(ctx context.Context, dry bool) error {
if r.Phase == "prepared" || r.Phase == "seed-prepared" {
return errActivation
}
- if r.Phase == "done" && !r.EdgePending && !r.TriggersPending {
- return nil
- }
release, err := t.settled(ctx, dry)
if err != nil {
return err
}
defer func() { release() }()
+ if r.Phase == "done" {
+ r.Phase = "complete"
+ if !dry {
+ if err = t.save(state, r); err != nil {
+ return err
+ }
+ }
+ }
if r.Phase != "done" {
for _, item := range r.Items {
if err = t.checkItem(item); err != nil {
@@ -562,9 +596,8 @@ func (t activationOffline) restore(ctx context.Context, dry bool) error {
return err
}
}
- // Restore-issued trigger starts wait for durable closure. An enabled
- // timer can start on reboot once its file is restored; resume stops it.
- // Edge and active triggers have separate exact-state checks after done.
+ // Verify the temporary holds before releasing the edge and triggers.
+ // complete retains writer exclusion until the whole-record gate passes.
var differences []string
const rounds = 3
for round := 0; round < rounds; round++ {
@@ -594,57 +627,78 @@ func (t activationOffline) restore(ctx context.Context, dry bool) error {
if len(differences) != 0 {
return fmt.Errorf("claim activation: restore remains complete after %d verification rounds:\n%s", rounds, strings.Join(differences, "\n"))
}
- r.Phase, r.EdgePending, r.TriggersPending = "done", true, true
+ r.Phase, r.EdgePending, r.TriggersPending = "complete", true, true
if err = t.save(state, r); err != nil {
return err
}
}
- // After closure, another root writer may have replaced any unit. Only an
- // owned mask can still authorize replacement; retain updater exclusion
- // across that last rename, then release it before the edge's recovery job.
- edgeOwned := t.checkItem(edge) == nil
- if r.EdgePending {
- if sameActivationFile(filepath.Join(t.units, edge.Name), filepath.Join(t.state, edge.Name+".mask")) || edgeOwned {
- if err = t.replace(units, state, edge, true); err != nil {
- return err
- }
- if _, err = t.systemctl(ctx, "daemon-reload"); err != nil {
- return err
- }
- if err = t.point("edge-reload"); err != nil {
- return err
- }
- }
+ // complete still owns every original. Pending flags record progress only;
+ // they never suppress the final comparison on resume.
+ if err = t.replace(units, state, edge, true); err != nil {
+ return err
+ }
+ if _, err = t.systemctl(ctx, "daemon-reload"); err != nil {
+ return err
+ }
+ if err = t.point("edge-reload"); err != nil {
+ return err
}
release()
release = func() {}
- // The pending edge still owes its exact recorded state, including inactive.
- // A later writer's replacement is outside this transaction's authority.
- if r.EdgePending && edgeOwned && t.checkItem(edge) == nil {
- if err = t.reconcile(ctx, edge); err != nil {
- return err
- }
+ if err = t.reconcile(ctx, edge); err != nil {
+ return err
}
r.EdgePending = false
if err = t.save(state, r); err != nil {
return err
}
- if r.TriggersPending {
- nextRelease, e := t.settled(ctx, false)
- if e != nil {
- return e
+ if err = t.point("edge-clear"); err != nil {
+ return err
+ }
+ nextRelease, e := t.settled(ctx, false)
+ if e != nil {
+ return e
+ }
+ release = nextRelease
+ for _, item := range r.Items[:len(activationTriggers)] {
+ if err = t.reconcile(ctx, item); err != nil {
+ return err
}
- release = nextRelease
- for _, item := range r.Items[:len(activationTriggers)] {
- if t.checkItem(item) == nil {
+ }
+ r.TriggersPending = false
+ if err = t.save(state, r); err != nil {
+ return err
+ }
+ if err = t.point("trigger-complete"); err != nil {
+ return err
+ }
+ const rounds = 3
+ var differences []string
+ for round := 0; round < rounds; round++ {
+ if round > 0 {
+ // Reconcile only retained originals, and leave the edge until last.
+ for _, item := range r.Items {
+ if t.checkItem(item) != nil {
+ continue
+ }
if err = t.reconcile(ctx, item); err != nil {
- return err
+ var interrupted activationInterruption
+ if errors.As(err, &interrupted) {
+ return err
+ }
}
}
}
- r.TriggersPending = false
+ if err = t.point("verify-completion"); err != nil {
+ return err
+ }
+ differences = t.closeDifferences(ctx, r)
+ if len(differences) == 0 {
+ r.Phase = "done"
+ return t.save(state, r)
+ }
}
- return t.save(state, r)
+ return fmt.Errorf("claim activation: restore remains complete after %d verification rounds:\n%s", rounds, strings.Join(differences, "\n"))
}
type activationOfflinePreview struct {
diff --git a/internal/host/activation_offline_rootrunner_linux_test.go b/internal/host/activation_offline_rootrunner_linux_test.go
index e3148ba4a..1a74a9c70 100644
--- a/internal/host/activation_offline_rootrunner_linux_test.go
+++ b/internal/host/activation_offline_rootrunner_linux_test.go
@@ -25,13 +25,27 @@ func offlineFixture(t *testing.T) (activationOffline, string) {
}
os.Remove(filepath.Join(units, "carlos-stage.service"))
os.Symlink("/dev/null", filepath.Join(units, "carlos-stage.service"))
+ active := map[string]bool{activationEdgeUnit: true}
+ for _, name := range activationTriggers {
+ active[name] = true
+ }
ctl := func(ctx context.Context, args ...string) ([]byte, error) {
+ if args[0] == "start" {
+ active[args[1]] = true
+ }
+ if args[0] == "stop" {
+ active[args[1]] = false
+ }
if args[0] == "show" {
load := "loaded"
if target, _ := os.Readlink(filepath.Join(units, args[1])); target == "/dev/null" {
load = "masked"
}
- return []byte("LoadState=" + load + "\nActiveState=inactive\n"), nil
+ state := "inactive"
+ if active[args[1]] {
+ state = "active"
+ }
+ return []byte("LoadState=" + load + "\nActiveState=" + state + "\n"), nil
}
return nil, nil
}
@@ -184,15 +198,10 @@ func TestRootActivationModelEdgeLast(t *testing.T) {
}
}
}
- if args[1] == "carlos-edge.service" && offlineRecord(t, tx).Phase != "done" {
+ if args[1] == "carlos-edge.service" && offlineRecord(t, tx).Phase != "complete" {
t.Fatal("edge started before durable completion")
}
- if args[1] != "carlos-edge.service" && offlineRecord(t, tx).Phase != "done" {
- target, _ := os.Readlink(filepath.Join(tx.units, "carlos-edge.service"))
- if target != "/dev/null" {
- t.Fatal("edge unmasked before triggers restored")
- }
- }
+
started[args[1]] = true
}
if args[0] == "show" && started[args[1]] {
diff --git a/internal/host/activation_production_reboot_rootrunner_linux_test.go b/internal/host/activation_production_reboot_rootrunner_linux_test.go
index 215d4debc..81daeaab4 100644
--- a/internal/host/activation_production_reboot_rootrunner_linux_test.go
+++ b/internal/host/activation_production_reboot_rootrunner_linux_test.go
@@ -4,7 +4,6 @@ package host
import (
"bytes"
- "context"
"errors"
"os"
"path/filepath"
@@ -14,9 +13,7 @@ import (
func activationProductionRebootPhase1(t *testing.T, phase string) {
tx := setupActivationProduction(t)
- if phase == "complete" {
- servingRunnerSystemctl(t, "stop", "carlos-edge.service")
- }
+
stop := errors.New("reboot checkpoint")
point := "journal-" + phase
tx.checkpoint = func(name string) error {
@@ -39,27 +36,21 @@ func activationProductionRebootPhase1(t *testing.T, phase string) {
if r.Phase != phase {
t.Fatal("phase", r.Phase, phase)
}
- if phase != "prepared" {
+ if phase != "prepared" && phase != "done" {
load, active, err := tx.status(t.Context(), activationEdgeUnit)
if err != nil || load != "masked" || active != "inactive" {
t.Fatal("edge escaped before release", load, active, err)
}
}
if phase == "done" {
- if !r.EdgePending {
- t.Fatal("missing ordinary edge continuation")
+ if r.EdgePending || r.TriggersPending {
+ t.Fatal("pending work at done")
}
- for _, item := range r.Items[:len(r.Items)-1] {
- _, active, err := tx.status(t.Context(), item.Name)
- want := "inactive"
- if item.Active && !item.Masked {
- want = "active"
- }
- if err != nil || active != want {
- t.Fatal("closed before reconciliation", item.Name, active, want, err)
- }
+ if diff := tx.closeDifferences(t.Context(), r); len(diff) != 0 {
+ t.Fatal(diff)
}
}
+
boot, err := os.ReadFile("/proc/sys/kernel/random/boot_id")
if err != nil {
t.Fatal(err)
@@ -84,7 +75,7 @@ func activationProductionRebootPhase2(t *testing.T, phase string) {
if r.Phase != phase {
t.Fatal("durable phase changed", r.Phase, phase)
}
- if phase != "prepared" {
+ if phase != "prepared" && phase != "done" {
load, active, err := tx.status(t.Context(), activationEdgeUnit)
if err != nil || load != "masked" || active != "inactive" {
t.Fatal("edge escaped on reboot", load, active, err)
@@ -115,24 +106,10 @@ func activationProductionRebootPhase2(t *testing.T, phase string) {
}
servingRunnerSystemctl(t, "start", "carlos-adopt.timer")
}
- if phase == "done" {
- // Closure releases ordinary service writers. Pending triggers and the
- // edge still owe their exact state while their files remain owned.
- original := tx.systemctl
- tx.systemctl = func(ctx context.Context, args ...string) ([]byte, error) {
- if args[0] == "start" || args[0] == "stop" {
- for _, name := range rootMigrationOfflineUnits {
- if args[1] == name {
- t.Fatalf("closed transaction changed %s", args[1])
- }
- }
- }
- return original(ctx, args...)
- }
- }
+
tx.checkpoint = func(point string) error {
if point == "replace-"+activationEdgeUnit || point == "start-"+activationEdgeUnit {
- if offlineRecord(t, tx).Phase != "done" {
+ if offlineRecord(t, tx).Phase != "complete" {
t.Fatal("edge released before done", point)
}
}
@@ -141,12 +118,8 @@ func activationProductionRebootPhase2(t *testing.T, phase string) {
if err := tx.restore(t.Context(), false); err != nil {
t.Fatal("resume after actual reboot", phase, err)
}
- if phase != "done" {
- assertActivationProductionRecorded(t, tx, true)
- }
- if phase != "complete" {
- assertActivationProductionEdge(t)
- }
+ assertActivationProductionRecorded(t, tx, true)
+ assertActivationProductionEdge(t)
if got := offlineRecord(t, tx); got.Phase != "done" || got.EdgePending {
t.Fatal("unfinished", got.Phase, got.EdgePending)
}
diff --git a/internal/host/activation_production_rootrunner_linux_test.go b/internal/host/activation_production_rootrunner_linux_test.go
index e5aae5ced..dea8fa1ef 100644
--- a/internal/host/activation_production_rootrunner_linux_test.go
+++ b/internal/host/activation_production_rootrunner_linux_test.go
@@ -221,6 +221,9 @@ func setupActivationProduction(t *testing.T) activationOffline {
servingRunnerSystemctl(t, "daemon-reload")
servingRunnerSystemctl(t, "enable", "carlos-edge.service", "carlos-system-recover.service", "carlos-adopt.timer")
servingRunnerSystemctl(t, "start", "carlos-edge.service")
+ servingRunnerSystemctl(t, append([]string{"enable", "--now"}, activationTriggers...)...)
+ servingRunnerSystemctl(t, "stop", "carlos-adopt.service")
+ servingRunnerSystemctl(t, "reset-failed", "carlos-adopt.service")
assertActivationProductionEdge(t)
return tx
}
@@ -233,7 +236,7 @@ func TestRootActivationProductionRestore(t *testing.T) {
}
tx.checkpoint = func(point string) error {
if point == "replace-carlos-edge.service" || point == "start-carlos-edge.service" {
- if phase := offlineRecord(t, tx).Phase; phase != "done" {
+ if phase := offlineRecord(t, tx).Phase; phase != "complete" {
t.Fatalf("edge released before closed: %s at %s", phase, point)
}
}
@@ -248,7 +251,6 @@ func TestRootActivationProductionRestore(t *testing.T) {
func TestRootActivationProductionCompleteResume(t *testing.T) {
tx := setupActivationProduction(t)
defer cleanupActivationProduction(t, tx)
- servingRunnerSystemctl(t, "stop", "carlos-edge.service")
if err := tx.begin(t.Context(), false); err != nil {
t.Fatal(err)
}
@@ -316,11 +318,11 @@ func TestRootActivationProductionEdgeFailureClosed(t *testing.T) {
if err := tx.restore(t.Context(), false); err == nil {
t.Fatal("real failed edge start succeeded")
}
- if r := offlineRecord(t, tx); r.Phase != "done" || !r.EdgePending {
+ if r := offlineRecord(t, tx); r.Phase != "complete" || !r.EdgePending {
t.Fatal("edge failure reopened transaction", r.Phase, r.EdgePending)
}
- if err := activationAdoptionAllowed("/etc"); err != nil {
- t.Fatal("failed ordinary edge held adoption", err)
+ if err := activationAdoptionAllowed("/etc"); err == nil {
+ t.Fatal("failed edge released adoption", err)
}
if err := os.Remove(bad); err != nil {
t.Fatal(err)
@@ -356,13 +358,24 @@ func TestRootActivationProductionRecoveryGate(t *testing.T) {
if err := (&systemApplier{etcDir: "/etc"}).Run(t.Context(), false); !errors.Is(err, errActivation) {
t.Fatal("complete allowed update", err)
}
- if _, err := tx.systemctl(t.Context(), "start", "carlos-system-recover.service"); err == nil {
- t.Fatal("complete allowed recovery")
+ if _, err := tx.systemctl(t.Context(), "start", "carlos-system-recover.service"); err != nil {
+ t.Fatal("settled recovery no-op refused", err)
}
raw, err := tx.systemctl(t.Context(), "show", "carlos-system-recover.service", "--property=Result", "--value")
- if err != nil || strings.TrimSpace(string(raw)) != "exit-code" {
+ if err != nil || strings.TrimSpace(string(raw)) != "success" {
t.Fatal("complete did not exclude real recovery", string(raw), err)
}
+ journal := filepath.Join(tx.updateDir(), "journal.json")
+ activationProductionWrite(t, journal, []byte("pending"), 0600)
+ if err := (&systemApplier{etcDir: "/etc"}).Run(t.Context(), true); err == nil {
+ t.Fatal("pending recovery passed the no-op gate")
+ }
+ if raw, err := os.ReadFile(journal); err != nil || string(raw) != "pending" {
+ t.Fatal("pending recovery changed evidence", err)
+ }
+ if err := os.Remove(journal); err != nil {
+ t.Fatal(err)
+ }
tx.checkpoint = nil
if err := tx.restore(t.Context(), false); err != nil {
t.Fatal(err)
@@ -383,7 +396,7 @@ func TestRootActivationProductionRestoreBoundaries(t *testing.T) {
tx.checkpoint = func(name string) error {
count++
r := offlineRecord(t, tx)
- if r.Phase != "done" {
+ if r.Phase != "done" && r.Phase != "complete" {
target, _ := os.Readlink(filepath.Join(tx.units, activationEdgeUnit))
if target != "/dev/null" {
t.Fatal("edge released before closure", name, r.Phase)
diff --git a/internal/host/activation_restore_order_rootrunner_linux_test.go b/internal/host/activation_restore_order_rootrunner_linux_test.go
index c12e93cc1..a2325960a 100644
--- a/internal/host/activation_restore_order_rootrunner_linux_test.go
+++ b/internal/host/activation_restore_order_rootrunner_linux_test.go
@@ -86,7 +86,7 @@ func TestRootActivationModelRestoreEveryBoundaryBoot(t *testing.T) {
if !active["carlos-edge.service"] {
t.Fatal("edge not restored")
}
- if r.Phase != "done" && active["carlos-adopt.timer"] {
+ if !active["carlos-adopt.timer"] {
t.Fatalf("inactive timer not reconciled after %s", point)
}
})
@@ -109,6 +109,9 @@ func TestRootActivationModelRepairExclusion(t *testing.T) {
tx.units = units
tx.state = filepath.Join(etc, "carlos", "claim-activation", "offline")
active := map[string]bool{"carlos-adopt.timer": true, "carlos-edge.service": true}
+ for _, name := range activationTriggers {
+ active[name] = true
+ }
tx.systemctl = func(ctx context.Context, args ...string) ([]byte, error) {
switch args[0] {
case "stop":
@@ -205,7 +208,7 @@ func TestRootActivationAdoptionGate(t *testing.T) {
t.Fatal("gate phase", phase, err)
}
err = activationTransactionAllowed(etc, true)
- if (err == nil) != (phase == "done") {
+ if (err == nil) != (phase == "done") || (errors.Is(err, errActivationRecoveryHeld) != (phase == "complete")) {
t.Fatal("recovery gate phase", phase, err)
}
}
diff --git a/internal/host/activation_seed_cli_rootrunner_linux_test.go b/internal/host/activation_seed_cli_rootrunner_linux_test.go
index 2bc6c2bd5..5e940f8d8 100644
--- a/internal/host/activation_seed_cli_rootrunner_linux_test.go
+++ b/internal/host/activation_seed_cli_rootrunner_linux_test.go
@@ -34,7 +34,7 @@ exit 0
func setupActivationSeedCLI(t *testing.T) activationOffline {
t.Helper()
tx := setupActivationProduction(t)
- servingRunnerSystemctl(t, "stop", "carlos-edge.service")
+ activationSeedCLI(t, "offline-begin")
servingRunnerSystemctl(t, "start", "carlos-instance-control.service")
raw, err := os.ReadFile("/etc/carlos/host.env")
if err != nil {
@@ -57,7 +57,6 @@ func setupActivationSeedCLI(t *testing.T) activationOffline {
}
activationProductionWrite(t, "/opt/carlos/system/guard/identities/v1.0.1.json", identity, 0644)
activationProductionWrite(t, "/opt/carlos/carlos", []byte(activationSeedCLIPayload), 0755)
- activationSeedCLI(t, "offline-begin")
return tx
}
diff --git a/internal/host/systemapply.go b/internal/host/systemapply.go
index fbef3d9fc..47ed9deeb 100644
--- a/internal/host/systemapply.go
+++ b/internal/host/systemapply.go
@@ -274,6 +274,9 @@ func (a *systemApplier) file(name string) string { return filepath.Join(a.rootDi
// would only start it again with nothing left to do.
func (a *systemApplier) Run(ctx context.Context, recover bool) (retErr error) {
if err := activationTransactionAllowed(a.etcDir, recover); err != nil {
+ if errors.Is(err, errActivationRecoveryHeld) {
+ return nil
+ }
return err
}
defer func() {
@@ -310,6 +313,9 @@ func (a *systemApplier) Run(ctx context.Context, recover bool) (retErr error) {
return nil
}
if err := activationTransactionAllowed(a.etcDir, recover); err != nil {
+ if errors.Is(err, errActivationRecoveryHeld) {
+ return nil
+ }
return err
}
if a.floorHook != nil {
diff --git a/internal/provision/testdata/plan14-root-reads-activation-tools.tsv b/internal/provision/testdata/plan14-root-reads-activation-tools.tsv
index f7b831651..f0668007f 100644
--- a/internal/provision/testdata/plan14-root-reads-activation-tools.tsv
+++ b/internal/provision/testdata/plan14-root-reads-activation-tools.tsv
@@ -1,7 +1,7 @@
-# Activation-tools runs 1–9: proposals only; loaded after all existing overlays.
+# Activation-tools runs 1–10: proposals only; loaded after all existing overlays.
@coverage . efeeda08e816fc2c7e3aa451b06cce95ab5778af79682290cf98a60d8b9a841a
@coverage internal/claimactivation 625d82dce14bae92f619077ca92a8c9066a9446545dfcc96a6f818e89e20d4c2
-@coverage internal/host d620d371d9c5ae6ab5301881099f9f7bd394152653bbdf20c6513df2bd4ef68d
+@coverage internal/host aef98870e366deb9774a74d8fb8f85ed2b94e8ea425f63c36b6a3517959665cd
@coverage internal/hostfence 552570510fb62d2a8822a3f24fa76aeb1437c0a9516966dc5ce15ef58e3fb4c6
internal/claimactivation/runner.go Runner.Run authority F,G,J4 rev3=PROPOSED-activation-tools;identity=explicit operator CLI uid 0;basis=trusted operator phases bind expected signed roster revision, exact deployment and actors; verify each write; dry-run overlay never calls backing Put file:Read@310,file:Read@5986,file:Read@6318 683059aedf27656d4b49c88102089b2bb2f2ce7fcc108ff8545e52a86a0f8385
internal/claimactivation/runner.go checkedStore.Delete authority F,G,J4 rev3=PROPOSED-activation-tools;identity=explicit operator CLI uid 0;basis=trusted operator phases bind expected signed roster revision, exact deployment and actors; verify each write; dry-run overlay never calls backing Put 6bc9103a30cfd53cea59d611ccacb7dcc0ca0ea620b403497ef543f5bdaa0217
@@ -22,15 +22,15 @@ internal/host/activation_install_linux.go installActivationConsole authority G,J
internal/host/activation_install_linux.go installActivationControl authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit operator CLI uid 0;basis=explicit root operator command only; protected ancestry, exact inputs, retained mask ownership and durable readback; no automatic claims-off call file:ReadFile@616 16e3c76c1d1d06fc429725cfeaee21b17d6a37620da287b40482b12e2f6b20a5
internal/host/activation_install_linux.go installActivationEvidenceKey authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit operator CLI uid 0;basis=explicit root operator command only; protected ancestry, exact inputs, retained mask ownership and durable readback; no automatic claims-off call 74f311fcb31aec95f5760ae9781186cc44a6f650858349c379e82f3a1fc6fd10
internal/host/activation_install_linux.go installActivationFenceCredentials authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit operator CLI uid 0;basis=explicit root operator command only; protected ancestry, exact inputs, retained mask ownership and durable readback; no automatic claims-off call 9e18de6df5fb9d3e6898967c9ed91e72d19faf5fd6bff7331ee654d76dfe7075
-internal/host/activation_offline_linux.go activationOffline.begin authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit operator CLI uid 0;basis=run9 supported-baseline admission before recording; exact recorded state verification; temporary holds checked separately; existing updater exclusion and file ownership retained e1a8254f6cd071d09fc7e1ca4bdf4b2db35ae625c1dbeb1ef3c0599234ffa4e5
+internal/host/activation_offline_linux.go activationOffline.begin authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=canonical healthy admission; complete excludes writers; settled recovery succeeds without effects for edge dependency; final whole-record verification precedes done on every resume e1a8254f6cd071d09fc7e1ca4bdf4b2db35ae625c1dbeb1ef3c0599234ffa4e5
internal/host/activation_offline_linux.go activationOffline.checkItem authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=settled update lock and journal precondition; all writers excluded until done; bounded full close verification with held triggers and masked edge; pending trigger and edge continuation after durable done; absent transaction retains legacy behavior 55e4f594ebd4666d8b5eff5eb995a163a6f9ef3838b1d6db5f705a90b93a16e8
internal/host/activation_offline_linux.go activationOffline.dirs authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=settled update lock and journal precondition; all writers excluded until done; bounded full close verification with held triggers and masked edge; pending trigger and edge continuation after durable done; absent transaction retains legacy behavior file:Stat@384 cdbba773717e890be772460b0f8235b4f7f0e158a11977740a2c963aa3a1c923
internal/host/activation_offline_linux.go activationOffline.point authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=settled update lock and journal precondition; all writers excluded until done; bounded full close verification with held triggers and masked edge; pending trigger and edge continuation after durable done; absent transaction retains legacy behavior c28de19acebdc98b1b836151fbba47103ccb431f40d79984fee55a0b921ad0e0
-internal/host/activation_offline_linux.go activationOffline.prepare authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit operator CLI uid 0;basis=run9 supported-baseline admission before recording; exact recorded state verification; temporary holds checked separately; existing updater exclusion and file ownership retained file:Lstat@757,file:Readlink@789 88a9d196707cf63be2d1802a0de9efaac9fe9c0bea756340be878873b124c6c6
+internal/host/activation_offline_linux.go activationOffline.prepare authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=canonical healthy admission; complete excludes writers; settled recovery succeeds without effects for edge dependency; final whole-record verification precedes done on every resume file:Lstat@757,file:Readlink@789 88a9d196707cf63be2d1802a0de9efaac9fe9c0bea756340be878873b124c6c6
internal/host/activation_offline_linux.go activationOffline.preview authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=settled update lock and journal precondition; all writers excluded until done; bounded full close verification with held triggers and masked edge; pending trigger and edge continuation after durable done; absent transaction retains legacy behavior a092bb6fb96f64e406a266cef59a4516145a7f58935ff8d1bbc42a96a653aab1
internal/host/activation_offline_linux.go activationOffline.read authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=settled update lock and journal precondition; all writers excluded until done; bounded full close verification with held triggers and masked edge; pending trigger and edge continuation after durable done; absent transaction retains legacy behavior 6ffaa998c2441b9a2d7889275b526bace11feb269df6856527b199a40cfa5bec
internal/host/activation_offline_linux.go activationOffline.replace authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=settled update lock and journal precondition; all writers excluded until done; bounded full close verification with held triggers and masked edge; pending trigger and edge continuation after durable done; absent transaction retains legacy behavior 2a64167ac7053a5c73fe64aa9fea7e4f8db3b1409c7e34fa78c1b842328ccdbb
-internal/host/activation_offline_linux.go activationOffline.restore authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit operator CLI uid 0;basis=run9 supported-baseline admission before recording; exact recorded state verification; temporary holds checked separately; existing updater exclusion and file ownership retained 1ff1a1548000683e6785ca9b42b871bf0586f0171b6eae603d3b561287562386
+internal/host/activation_offline_linux.go activationOffline.restore authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=canonical healthy admission; complete excludes writers; settled recovery succeeds without effects for edge dependency; final whole-record verification precedes done on every resume ee2dd65962263ff367b1f72140fd2c5fc5bffd4bfbd6d6d127c94bb68b5e2036
internal/host/activation_offline_linux.go activationOffline.save authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=settled update lock and journal precondition; all writers excluded until done; bounded full close verification with held triggers and masked edge; pending trigger and edge continuation after durable done; absent transaction retains legacy behavior 3082a1585eb287687e05756f495aae7de7e11aab6e96bbee920c50b4807ab9b7
internal/host/activation_offline_linux.go activationOffline.status authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=settled update lock and journal precondition; all writers excluded until done; bounded full close verification with held triggers and masked edge; pending trigger and edge continuation after durable done; absent transaction retains legacy behavior 616c3058401766ada7c16395412aaca50b4d68be8d3a2e57ae969a53e21292ff
internal/host/activation_offline_linux.go activationOfflineUnits authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=settled update lock and journal precondition; all writers excluded until done; bounded full close verification with held triggers and masked edge; pending trigger and edge continuation after durable done; absent transaction retains legacy behavior 372c65c44288c766e9b2aa0ece8cbc55c95fb1ccae7e9bae80a5bfcda2550b4b
@@ -65,18 +65,21 @@ internal/host/activation_adoption_other.go activationAdoptionAllowed authority G
internal/host/activation_offline_linux.go var activationEdgeUnit authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=settled update lock and journal precondition; all writers excluded until done; bounded full close verification with held triggers and masked edge; pending trigger and edge continuation after durable done; absent transaction retains legacy behavior 42c79dbb523cf08c19f23d2e4f1754902524276b86f58f0d3d5bfb6117d3fcc6
internal/host/agent.go runAdoptUnderRootRelease authority B3,E1,E2,E3 rev3=PROPOSED-activation-tools;identity=root adoption or explicit offline operator;basis=only an explicit protected offline transaction excludes adoption and automatic repair until done; absent transaction preserves existing behavior; complete is durable before edge release; prior authority constraints retained. rev3=PROPOSED-p3;run=16;identity=root adopt/update/control uid 0;basis=no census selects legacy proof/history and update effects; proof observation cannot fetch signed recovery manifests on that path; participating adoption retains root release exclusion and independent generation pins whole:Route@11179,whole:Route@18368,whole:Route@18404,whole:Route@8481 2f755c7ed82fcac622a118d1e4ad0b5d746de2b97e0456e125114e12351cd0c8
internal/host/systemunits.go ensureSystemUnits authority F,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=settled update lock and journal precondition; all writers excluded until done; bounded full close verification with held triggers and masked edge; pending trigger and edge continuation after durable done; absent transaction retains legacy behavior file:OpenFile@375,file:ReadFile@3129,file:ReadFile@3262,file:ReadFile@4176,file:Stat@3038 eda788ec9bfb20192d503aa2ce07f707871de3bba111830f00bc2be0b5c8daa2
-internal/host/systemapply.go systemApplier.Run authority D,F,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=settled update lock and journal precondition; all writers excluded until done; bounded full close verification with held triggers and masked edge; pending trigger and edge continuation after durable done; absent transaction retains legacy behavior file:OpenFile@572 3e714f6843366bdc621c416c6be2a55f1d3ed5de53f1ee12948f73fe855f6996
-internal/host/activation_adoption_linux.go activationTransactionAllowed authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=all automatic writers excluded through complete; only durable done releases ordinary recovery and repair; absent transaction retains legacy behavior file:Lstat@157,file:Stat@377 fb9878a1a768b7faf9244702c330a14e062fe4e6e6b3b0753ae13034384dcf4a
+internal/host/systemapply.go systemApplier.Run authority D,F,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=canonical healthy admission; complete excludes writers; settled recovery succeeds without effects for edge dependency; final whole-record verification precedes done on every resume file:OpenFile@639 6e9f3d6e5af32111448d6e3a48052e411d2e7e38e42475c71fd76cf8d40cf068
+internal/host/activation_adoption_linux.go activationTransactionAllowed authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=canonical healthy admission; complete excludes writers; settled recovery succeeds without effects for edge dependency; final whole-record verification precedes done on every resume file:Lstat@157,file:Stat@377 538b2498363c32a7e54514fa1639ea53acaac1b032ff6adf1bfbd38c7c461c3d
internal/host/activation_adoption_other.go activationTransactionAllowed authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=all automatic writers excluded through complete; only durable done releases ordinary recovery and repair; absent transaction retains legacy behavior 8c451f30ca31ccca7618f0fd6126ed7d31267c74213c0adb6475e6d504c80723
internal/host/activation_settled_linux.go activationOffline.settled authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=settled update lock and journal precondition; all writers excluded until done; bounded full close verification with held triggers and masked edge; pending trigger and edge continuation after durable done; absent transaction retains legacy behavior file:Openat@663,file:Stat@1011 176b2ba6cd9eb6af75ded2a7e27c4f6f8cd6ff1107f5949b1c40f884e651644a
internal/host/activation_settled_linux.go activationOffline.updateSettled authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=settled update lock and journal precondition; all writers excluded until done; bounded full close verification with held triggers and masked edge; pending trigger and edge continuation after durable done; absent transaction retains legacy behavior file:Lstat@317 38adccbf4e50966a54a966af12c3afffc8c02dbcdf43825423bb60ee92eb9c80
-internal/host/activation_offline_linux.go activationOffline.closeDifferences authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit operator CLI uid 0;basis=run9 supported-baseline admission before recording; exact recorded state verification; temporary holds checked separately; existing updater exclusion and file ownership retained ac9fd3519c2d80a41acd20a0a1d8992d0a17c8cd1c57bf2910cba0d03fb2f559
-internal/host/activation_offline_linux.go activationOffline.reconcile authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit operator CLI uid 0;basis=run9 supported-baseline admission before recording; exact recorded state verification; temporary holds checked separately; existing updater exclusion and file ownership retained 65f7b70c7f5f5e9dec0c5b038f675767eff1d2af3b8ccd78fcc71dc0fb4d5e22
+internal/host/activation_offline_linux.go activationOffline.closeDifferences authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=canonical healthy admission; complete excludes writers; settled recovery succeeds without effects for edge dependency; final whole-record verification precedes done on every resume ac9fd3519c2d80a41acd20a0a1d8992d0a17c8cd1c57bf2910cba0d03fb2f559
+internal/host/activation_offline_linux.go activationOffline.reconcile authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=canonical healthy admission; complete excludes writers; settled recovery succeeds without effects for edge dependency; final whole-record verification precedes done on every resume 65f7b70c7f5f5e9dec0c5b038f675767eff1d2af3b8ccd78fcc71dc0fb4d5e22
internal/host/activation_settled_linux.go activationOffline.updateDir authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=settled update lock and journal precondition; all writers excluded until done; bounded full close verification with held triggers and masked edge; pending trigger and edge continuation after durable done; absent transaction retains legacy behavior d729025e9d11aa6007ec09318e5e4b84cd14ffda17059d305bbbffe982740fb1
internal/host/activation_settled_linux.go activationOffline.updateUnitsSettled authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator;basis=system update and recovery units must be inactive or failed on admission and resume; transient and unknown states refuse before transaction changes f220fdee955f9a1f27d00f2f7678f143aaf15ac0819e3ec5e208b951b8939c1f
internal/host/activation_offline_linux.go activationRecordedState authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator;basis=exact recorded load and active state; boolean fallback only for older records 99ed5f22fad14f6a8dc572ec7166a1a04415f7067dd1cb4e624b2ad5797947f9
internal/host/activation_settled_linux.go activationConfiguredUpdateRoot authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator;basis=protected host.env binds admission to actual updater lock and journals; conflicting shell override refused 8f9e59c225857ef0601d1f6800bc7e0938f0800589dfaf614c6df93b854dcee1
-internal/host/activation_offline_linux.go activationSupportedBaseline authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit operator CLI uid 0;basis=run9 supported-baseline admission and distinct pre-close hold checks; no automatic claims-off call 038727f4346d66fbda3476733324109ec6ff512fe6fbc6c56cf00c220cf07e12
-internal/host/activation_offline_linux.go activationOffline.baseline authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit operator CLI uid 0;basis=run9 supported-baseline admission and distinct pre-close hold checks; no automatic claims-off call cbcfb2acce3afd20caa3ffa5e89734183342066294e3d196384a85498db9124a
+internal/host/activation_offline_linux.go activationSupportedBaseline authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=canonical healthy admission; complete excludes writers; settled recovery succeeds without effects for edge dependency; final whole-record verification precedes done on every resume 53ffcf02a01db552c8322cccf7808d009e6236524ae66617f4c333abde524614
+internal/host/activation_offline_linux.go activationOffline.baseline authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=canonical healthy admission; complete excludes writers; settled recovery succeeds without effects for edge dependency; final whole-record verification precedes done on every resume 345295b67287a0737a9a2fd557381f23f482e666f930904c2b1855630a1823c1
internal/host/activation_offline_linux.go activationOffline.hold authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit operator CLI uid 0;basis=run9 supported-baseline admission and distinct pre-close hold checks; no automatic claims-off call 75d1d23fc72bdf4697cc29a6381a43320e95ba8697212b179d130372149c910d
internal/host/activation_offline_linux.go activationOffline.heldDifferences authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit operator CLI uid 0;basis=run9 supported-baseline admission and distinct pre-close hold checks; no automatic claims-off call a6cb61a00217d89009ec2436cdce3858d4a75a08bccd84420c92ebf8aab2cf3b
+internal/host/activation_adoption_linux.go var errActivationRecoveryHeld authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=canonical healthy admission; complete excludes writers; settled recovery succeeds without effects for edge dependency; final whole-record verification precedes done on every resume 7fe82a76cf22b8d3f5c8adf31ccb47a2ade429bfb944c51fdc77685f2fe763d1
+internal/host/activation_adoption_other.go var errActivationRecoveryHeld authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=canonical healthy admission; complete excludes writers; settled recovery succeeds without effects for edge dependency; final whole-record verification precedes done on every resume 7fe82a76cf22b8d3f5c8adf31ccb47a2ade429bfb944c51fdc77685f2fe763d1
+internal/host/activation_offline_linux.go var activationCanonicalBaseline authority G,J4 rev3=PROPOSED-activation-tools;identity=explicit root operator or root updater;basis=canonical healthy admission; complete excludes writers; settled recovery succeeds without effects for edge dependency; final whole-record verification precedes done on every resume 2b53e28b44dd1d1910a00059411b1cb2d59c904c483880e95360d84fbf5315b1