amadan
claim-step8-readiness: backfill phases, leo's migration census, journal-migrate --window
Dev cohort-1 step 8 readiness (Paul's approval 2026-10-19; controller
rulings R1-R5):
- R1 (S1): carlos system claims run gains backfill-collect (create-only
signed collecting plan freezing every bucket instance record) and
backfill-adopt (If-Match advance through hostops Backfill.Publish, which
now takes each host's box and region from the one signed census report
naming it; read back through Backfill.VerifyAdopting). The migration
census reports an ordinary legacy route row as the box's route
installation of its frozen record, so adoption can name the box.
- R2 (S2): the migration census classifies leo's real tree: the window's
/dev/null masks (journal-migrate --window, the twelve window units, held
originals only), the edge's certificate cache and the box's own files,
a reported route installation's paths and live link, and a frozen
record's retained leftovers. Refusals name the path.
- R3: journal-migrate's public floor directories and install-control's
retry directory are made with mkdirAllExact, so umask 077 leaves 0755.
- Found by the witness: journal-migrate failed on a box with no legacy
journal directory; an absent directory now imports nothing.
Witness: TestRootStep8WindowWitness{DevRoster,SophieReported} rebuild
leo's tree from its metadata snapshot and run 8.1's masks, both backfill
phases, journal-migrate and the console's adoption pass. They show two
remaining STOPs: backfill-adopt needs a census report from sophie (S3),
and root refuses the adopted legacy host before any seed admission (S4).
Census overlay plan14-root-reads-claim-step8-readiness.tsv and the 9B8
addendum are PROPOSED (docs/superpowers/notes/census-proposal-claim-step8-readiness.md).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
31 files changed,
+2839
−7
docs/site/cli.md+4 −0docs/superpowers/notes/census-proposal-claim-step8-readiness.md+44 −0internal/claimactivation/backfill.go+232 −0internal/claimactivation/backfill_test.go+299 −0internal/claimactivation/runner.go+17 −0internal/host/claimcensus.go+38 −1internal/host/claimcensus_migration_linux.go+224 −0internal/host/claimcensus_migration_linux_test.go+332 −0internal/host/claimconstruction.go+4 −0internal/host/claimjournal_migration.go+10 −1internal/host/claimjournal_migration_other.go+4 −0internal/host/claimstep8_ruling6_test.go+88 −0internal/host/claimstep8_witness_rootrunner_linux_test.go+506 −0internal/host/leo_snapshot_linux_test.go+184 −0internal/host/rootmigration_census.go+48 −0internal/host/rootmigration_floordirs_linux_test.go+48 −0internal/host/rootmigration_production_linux.go+29 −2internal/host/rootmigration_routes_linux_test.go+107 −0internal/host/rootupdates_runtime.go+5 −0internal/host/testdata/leo-step8/leo-routes.tsv+2 −0internal/host/testdata/leo-step8/leo-tree-snapshot.tsv+251 −0internal/host/testdata/leo-step8/leo-units-post-8.1.tsv+28 −0internal/hostbox/journalmigration_absent_linux_test.go+43 −0internal/hostbox/journalmigration_linux.go+10 −0internal/hostbox/migration_census_linux.go+14 −2internal/hostops/backfill_owners_test.go+158 −0internal/hostops/backfill_publish.go+47 −0internal/provision/claim_step8_readiness_census_test.go+34 −0internal/provision/plan14_root_reads_test.go+10 −0internal/provision/plan9_b8_census_test.go+1 −1internal/provision/testdata/plan14-root-reads-claim-step8-readiness.tsv+18 −0
CI failed — run details
CI log
=== gate ===
cd launchctx && go vet ./... && go test ./... -count=1
ok amadan.net/carlos/platform.git/launchctx 0.012s
? amadan.net/carlos/platform.git/launchctx/controlname [no test files]
? amadan.net/carlos/platform.git/launchctx/wireid [no test files]
go vet ./...
gofmt -l .: clean
go test ./... -count=1 -timeout 30m
ok github.com/carlosframework/platform 307.297s
? github.com/carlosframework/platform/cmd/changelog [no test files]
ok github.com/carlosframework/platform/cmd/console 2.913s
ok github.com/carlosframework/platform/cmd/moveprobe 93.255s
ok github.com/carlosframework/platform/cmd/zddprobe 0.731s
ok github.com/carlosframework/platform/internal/activator 252.235s
ok github.com/carlosframework/platform/internal/agentsock 0.021s
ok github.com/carlosframework/platform/internal/alerting 0.005s
ok github.com/carlosframework/platform/internal/analytics 0.032s
ok github.com/carlosframework/platform/internal/appconfig 2.493s
ok github.com/carlosframework/platform/internal/apperr 0.119s
ok github.com/carlosframework/platform/internal/applog 0.192s
ok github.com/carlosframework/platform/internal/awsinv 0.006s
ok github.com/carlosframework/platform/internal/boxchoice 0.004s
ok github.com/carlosframework/platform/internal/boxid 0.640s
? github.com/carlosframework/platform/internal/boxid/boxidtest [no test files]
? github.com/carlosframework/platform/internal/boxid/internal/signtest [no test files]
ok github.com/carlosframework/platform/internal/boxvolumes 0.010s
ok github.com/carlosframework/platform/internal/caps 0.014s
ok github.com/carlosframework/platform/internal/certwarm 0.003s
ok github.com/carlosframework/platform/internal/cgroup 0.725s
ok github.com/carlosframework/platform/internal/changelog 0.363s
ok github.com/carlosframework/platform/internal/claimactivation 0.978s
ok github.com/carlosframework/platform/internal/clamd 0.061s
? github.com/carlosframework/platform/internal/cliflag [no test files]
ok github.com/carlosframework/platform/internal/console 403.600s
? github.com/carlosframework/platform/internal/controlname [no test files]
? github.com/carlosframework/platform/internal/controlsock [no test files]
ok github.com/carlosframework/platform/internal/dbdoor 6.476s
ok github.com/carlosframework/platform/internal/dbdoor/dbdoortest 0.027s
ok github.com/carlosframework/platform/internal/dbverify 0.012s
ok github.com/carlosframework/platform/internal/dnsrec 0.015s
ok github.com/carlosframework/platform/internal/docsite 0.015s
ok github.com/carlosframework/platform/internal/domains 2.460s
ok github.com/carlosframework/platform/internal/doorkey 0.023s
ok github.com/carlosframework/platform/internal/economics 0.014s
ok github.com/carlosframework/platform/internal/edge 45.236s
? github.com/carlosframework/platform/internal/edgeproof [no test files]
ok github.com/carlosframework/platform/internal/edgesfile 0.018s
ok github.com/carlosframework/platform/internal/fleetapi 0.177s
ok github.com/carlosframework/platform/internal/gate 1.691s
ok github.com/carlosframework/platform/internal/genselect 0.035s
ok github.com/carlosframework/platform/internal/geo 0.030s
ok github.com/carlosframework/platform/internal/geoip 0.073s
ok github.com/carlosframework/platform/internal/historyreadback 0.211s
ok github.com/carlosframework/platform/internal/host 179.082s
ok github.com/carlosframework/platform/internal/hostactors 6.832s
ok github.com/carlosframework/platform/internal/hostactors/awsmembership 0.005s
ok github.com/carlosframework/platform/internal/hostbox 2.757s
ok github.com/carlosframework/platform/internal/hostclaim 0.588s
ok github.com/carlosframework/platform/internal/hostenv 0.005s
ok github.com/carlosframework/platform/internal/hostevidence 0.005s
ok github.com/carlosframework/platform/internal/hostexec 5.006s
ok github.com/carlosframework/platform/internal/hostfence 0.046s
ok github.com/carlosframework/platform/internal/hostfiles 10.477s
ok github.com/carlosframework/platform/internal/hostfloor 0.015s
ok github.com/carlosframework/platform/internal/hostguard 1.049s
ok github.com/carlosframework/platform/internal/hostops 1.627s
? github.com/carlosframework/platform/internal/hostops/fencerelease [no test files]
ok github.com/carlosframework/platform/internal/hostops/internal/fencerelease 0.005s
ok github.com/carlosframework/platform/internal/hostquarantine 0.002s
ok github.com/carlosframework/platform/internal/hostrelocate 0.481s
ok github.com/carlosframework/platform/internal/hostresolve 0.153s
ok github.com/carlosframework/platform/internal/hostwrite 0.003s
ok github.com/carlosframework/platform/internal/infraview 0.002s
ok github.com/carlosframework/platform/internal/instancectl 5.235s
ok github.com/carlosframework/platform/internal/instanceexec 2.759s
ok github.com/carlosframework/platform/internal/instancefs 0.005s
ok github.com/carlosframework/platform/internal/instancekey 0.004s
ok github.com/carlosframework/platform/internal/instanceupdates 0.012s
2026/10/19 22:05:42 activator: h.example.com hibernated
--- FAIL: TestRawInstancesDirectoryIsNamedOnlyHere (0.35s)
rawdir_test.go:58: ../../internal/host/claimcensus_migration_linux.go:214 names the raw .instances directory; only internal/layout (and bootstrap's tmpfiles lines) may
FAIL
FAIL github.com/carlosframework/platform/internal/layout 1.577s
ok github.com/carlosframework/platform/internal/lbhealth 0.002s
ok github.com/carlosframework/platform/internal/lease 9.860s
ok github.com/carlosframework/platform/internal/lease/leasetest 0.007s
ok github.com/carlosframework/platform/internal/ledger 0.022s
ok github.com/carlosframework/platform/internal/mail 0.173s
ok github.com/carlosframework/platform/internal/maintenance 0.106s
ok github.com/carlosframework/platform/internal/maintenanceexec 5.532s
ok github.com/carlosframework/platform/internal/maintenanceguard 0.091s
ok github.com/carlosframework/platform/internal/metrics 0.008s
ok github.com/carlosframework/platform/internal/moves 21.084s
ok github.com/carlosframework/platform/internal/moves/archive 0.868s
ok github.com/carlosframework/platform/internal/objstore 0.200s
ok github.com/carlosframework/platform/internal/ops 0.174s
ok github.com/carlosframework/platform/internal/park 0.234s
ok github.com/carlosframework/platform/internal/privatefile 0.006s
ok github.com/carlosframework/platform/internal/provision 36.844s
ok github.com/carlosframework/platform/internal/publicrelease 0.009s
? github.com/carlosframework/platform/internal/regionname [no test files]
ok github.com/carlosframework/platform/internal/registry 5.520s
ok github.com/carlosframework/platform/internal/relay 7.236s
ok github.com/carlosframework/platform/internal/release 24.413s
ok github.com/carlosframework/platform/internal/rollout 3.841s
ok github.com/carlosframework/platform/internal/rolloutwords 0.011s
ok github.com/carlosframework/platform/internal/rootaws 0.007s
ok github.com/carlosframework/platform/internal/rootparticipation 0.058s
ok github.com/carlosframework/platform/internal/rootstate 0.027s
ok github.com/carlosframework/platform/internal/runs 0.576s
ok github.com/carlosframework/platform/internal/s3compat 0.008s
ok github.com/carlosframework/platform/internal/safefile 0.004s
ok github.com/carlosframework/platform/internal/schedule 2.496s
ok github.com/carlosframework/platform/internal/seeds 0.011s
ok github.com/carlosframework/platform/internal/sim/hostlife 9.896s
ok github.com/carlosframework/platform/internal/sim/litestreamfacts 4.264s
ok github.com/carlosframework/platform/internal/status 0.062s
ok github.com/carlosframework/platform/internal/steering 0.006s
ok github.com/carlosframework/platform/internal/stopevidence 0.287s
ok github.com/carlosframework/platform/internal/svcrole 0.002s
ok github.com/carlosframework/platform/internal/telemetry 0.004s
ok github.com/carlosframework/platform/internal/termfence 0.004s
ok github.com/carlosframework/platform/internal/tunnelclient 0.005s
ok github.com/carlosframework/platform/internal/unixdial 0.002s
ok github.com/carlosframework/platform/internal/updatewords 0.004s
ok github.com/carlosframework/platform/internal/usage 0.004s
ok github.com/carlosframework/platform/internal/version 0.002s
ok github.com/carlosframework/platform/internal/vet 1.576s
FAIL
make: *** [Makefile:18: test] Error 1
amadan: step gate failed: exit status 2
amadan: job failed at step gate — its output is above, not at the tail of this log
diff --git a/docs/site/cli.md b/docs/site/cli.md| index 22601a4d6..2943d2fb6 100644 |
| --- a/docs/site/cli.md |
| +++ b/docs/site/cli.md |
| @@ -2160,6 +2160,8 @@ The member command remains `carlos ops host resolve`. |
| Two `run` phases make the files `carlos system guard-arm` takes. `sign-policy` signs the armed policy for a floor and writes nothing to the bucket. `publish-identity` publishes a release's signed identity. It takes the router's hash from the release's signed checksum list, and the console's from the console app's signed release, chosen by its build. The console must be built with the release tag as its version, or the phase refuses it. Build it stripped, with -s -w, as releases are built. Each phase writes its file to `--out`, a new path, readable by root only. Pass these files to `guard-arm` exactly as written. A JSON tool would change the signed bytes. |
| +Two more `run` phases make the backfill that `carlos system journal-migrate` and claim adoption read. Each takes the plan's `backfill.id`, and prints the signed plan as read back. `backfill-collect` freezes every instance record the bucket lists in a new signed plan. While that plan is in place, claim-mode record writes are refused. `backfill-adopt` runs once every roster box has published its census for the plan. It gives each host to the box that reported it, and a host no box reported is never adopted. |
| + |
| `install-console` copies the console build that the published identity names into `/opt/carlos/system/console/console` and `console.prev`, owned by root. It refuses a source whose bytes don't match the identity, and it doesn't touch `console.service`. Pass that path to `guard-arm --console`. |
| After starting the guarded console, run `console-report` on its box as root. It checks the signed roster, guard, systemd process and running executable, then records that process in the existing private box report. It refuses stale box evidence and leaves the report’s timestamp unchanged. Run it again after a console restart and immediately before floor publication; the next agent report may replace the process entry. |
| @@ -2168,6 +2170,8 @@ After starting the guarded console, run `console-report` on its box as root. It |
| Before starting claim-enabled services on a box with legacy journals, stop the edge and agent. Run `carlos system journal-migrate` as root with `CARLOS_HOST_CLAIM=1` and the deployment environment loaded. The command imports journals only when signed root stop evidence verifies them. It leaves the original files in place. After migration, root ignores changed legacy files and logs an alert. Keep the migration marker. |
| +The migration census reads the backfill plan `backfill-collect` wrote. It reports each ordinary instance route on the box as that box's installation, so `backfill-adopt` can give the host to this box. If the window masks units by moving each unit file aside and linking `/dev/null` in its place, pass `--window DIR`, the window's directory, with the originals in `DIR/held`. The census then accepts those masks and no others. |
| + |
| #### Deployment-wide object scanning |
| ```sh |
diff --git a/docs/superpowers/notes/census-proposal-claim-step8-readiness.md b/docs/superpowers/notes/census-proposal-claim-step8-readiness.md| new file mode 100644 |
| index 000000000..aed77e8f5 |
| --- /dev/null |
| +++ b/docs/superpowers/notes/census-proposal-claim-step8-readiness.md |
| @@ -0,0 +1,44 @@ |
| +# Census proposal: claim-step8-readiness |
| + |
| +Status: PROPOSED to platform-1e as census owner. Overlay `internal/provision/testdata/plan14-root-reads-claim-step8-readiness.tsv`, loaded last (after `claim-guard-umask`), every row `rev3=PROPOSED-claim-step8-readiness`. The 9B8 addendum is `internal/provision/claim_step8_readiness_census_test.go`. No accepted-legacy row is added, so the seal stands. |
| + |
| +## What changed |
| + |
| +Dev cohort-1 step 8 readiness (Paul's approval of 2026-10-19; controller rulings R1 to R5): |
| + |
| +- **R1 (S1).** `carlos system claims run` gains `backfill-collect` and `backfill-adopt` on the trusted signer. Collect writes the signed collecting plan at `control/hosts-backfill.json`, create-only, freezing every instance record the bucket lists (`hostops.Apps`, `hostops.Records`, the census's own listing). Adopt advances it to adopting under If-Match through the existing `hostops.Backfill.Publish`, which now takes each host's box and region from the one box whose signed census report names it, then reads the plan back through `Backfill.VerifyAdopting`, the adopters' own acceptance. The migration census reports an ordinary legacy route row as the box's route installation of its frozen record, so adoption can name this box. |
| +- **R2 (S2).** The migration census classifies leo's real tree: the window's `/dev/null` masks (only with `journal-migrate --window`, only for the twelve window units, only with each original held), the edge's certificate cache and the box's own files (in the classes every claim census shares), a reported route installation's instance directories, program file and live link, and a frozen record's retained leftovers with no row and no claim. A refusal names the path. |
| +- **R3.** journal-migrate's public floor directories and install-control's retry directory are made with `mkdirAllExact` (#317), so a umask 077 shell leaves them 0755. |
| +- **Found by the witness.** A box with no legacy journal directory (leo) failed the import: the copy named each missing file while the identity said `absent`. `MigrateLegacy` now treats both as nothing to import. |
| + |
| +Ruling 6: every new or widened entry is reached only from claim-mode code (`TestStep8CensusChangesReachOnlyClaimMode` pins the callers): journal-migrate refuses without `CARLOS_HOST_CLAIM`, the root migration runs only inside it, the shared census classes are read only by the claim assembly, root's claim inventory startup and the migration, and the retry directory is prepared only with a census or claim units on. The backfill phases run only from `claims run` on the signer. |
| + |
| +## Units (root-read census) |
| + |
| +| Unit | Change | Class | Executing identity | |
| +|---|---|---|---| |
| +| `internal/claimactivation/runner.go` `Runner.Run` | two phases and their one allowed key | authority F,G,J4 (as ruled for claim-step6-artifacts) | `carlos system claims run`, explicit operator CLI, uid 0 on the trusted signing host | |
| +| `internal/claimactivation/backfill.go` `backfillReady` | new | authority F,G,J4 | the same signer CLI. Reads the published floor; refusal only | |
| +| `internal/host/claimjournal_migration.go` `RunSystemJournalMigrate` | `--window`, checked before any write; declaration digest only | authority G,J4 (as ruled in p3) | `carlos system journal-migrate`, offline operator CLI, uid 0 | |
| +| `internal/host/rootmigration_census.go` `rootMigrationCollector.collect` | reports legacy route rows | authority G,J4 (as ruled in p3) | the same, uid 0 | |
| +| `internal/host/rootmigration_census.go` `legacyRouteRow` | new | authority G,J4 | the same, uid 0. Reads an agent-written registry row while the agent is stopped and masked. A match only nominates this box for its frozen record's adoption, which the adopter checks again against the signed census digests and the committed domain claim | |
| +| `internal/host/rootmigration_production_linux.go` `migrateRootClaimAuthority` | wiring: routes, the migration classes, the floor directories | authority G,J4,E3 (as ruled in p3) | offline root migration, uid 0 | |
| +| `internal/host/claimcensus_migration_linux.go` `claimMigrationSharedPaths` | new | authority G,J4 | offline root migration, uid 0. Refusal-only classification | |
| +| `internal/host/claimcensus_migration_linux.go` `verifyMigrationWindow`, `rootPrivateEntry` | new | authority G,J4 | `carlos system journal-migrate`, uid 0. Lstat and Readlink of the window and the window units; refusal only | |
| +| `internal/host/rootupdates_runtime.go` `prepareRootRetryUnitDirectory` | `mkdirAllExact` before the public open | authority E3 (as ruled in p3) | root claim and adoption controllers, and `install-control`, uid 0 | |
| +| `internal/hostbox/migration_census_linux.go` `CheckMigrationRoots` | a refusal names its root and path | authority G,J4 (as ruled in p3) | offline root migration, uid 0 | |
| +| `internal/hostbox/journalmigration_linux.go` `JournalService.MigrateLegacy` | an absent legacy directory imports nothing | authority C,J2,J4 (as ruled, root-input) | offline root migration, uid 0 | |
| + |
| +Envelopes re-pinned: `internal/claimactivation`, `internal/host`, `internal/hostbox`, `internal/hostops`. New helpers with no read site of their own (`claimBoxDataCensus`, `hostInstallationPaths`, `prepareRootFloorDirs`, `verifyJournalMigrateWindow`, `listBackfillHosts`, `readSignedPlan`, `backfillCollect`, `backfillAdopt`, `Backfill.VerifyAdopting`) are envelope only. |
| + |
| +## 9B8 |
| + |
| +| Unit | Class | Why | |
| +|---|---|---| |
| +| `internal/hostops/backfill_publish.go` `Backfill.Publish` | nonrecord (re-reviewed) | its one Put, of the backfill manifest under its etag, is unchanged; it now copies box and region from the census before it | |
| +| `internal/claimactivation/backfill.go` `Runner.backfillCollect` | nonrecord (new) | its one Put is the create-only manifest through the runner's checkedStore, which admits only `control/hosts-backfill.json` for the phase; it reads records and writes none | |
| +| `internal/host/claimcensus_migration_linux.go` `hostInstallationPaths` | key-site (new) | the `instances` literal names a host's nested data directory for the census; no record key | |
| + |
| +## Rows carried over |
| + |
| +Every re-pinned row keeps its class and task exactly; only the bases add what changed. |
diff --git a/internal/claimactivation/backfill.go b/internal/claimactivation/backfill.go| new file mode 100644 |
| index 000000000..c53239f3f |
| --- /dev/null |
| +++ b/internal/claimactivation/backfill.go |
| @@ -0,0 +1,232 @@ |
| +package claimactivation |
| + |
| +import ( |
| + "context" |
| + "errors" |
| + "os" |
| + "reflect" |
| + "sort" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/hostactors" |
| + "github.com/carlosframework/platform/internal/hostfloor" |
| + "github.com/carlosframework/platform/internal/hostops" |
| + "github.com/carlosframework/platform/internal/release" |
| +) |
| + |
| +// BackfillInput is backfill-collect's and backfill-adopt's plan: the |
| +// reservation id the boxes' census reports are published under |
| +// (control/host-census/<id>/<box>.json). Nothing else is typed: the hosts |
| +// come from the bucket's own record listing, and each host's box and region |
| +// from the boxes' signed census reports. |
| +type BackfillInput struct { |
| + ID string `json:"id"` |
| +} |
| + |
| +// BackfillHost is one frozen host, as read back from the signed plan. |
| +type BackfillHost struct { |
| + Host string `json:"host"` |
| + hostops.BackfillHost |
| +} |
| + |
| +// BackfillSummary is what the backfill phases read back from the signed |
| +// plan after writing it (or, on a dry run, what they would write). |
| +type BackfillSummary struct { |
| + ID string `json:"id"` |
| + Phase string `json:"phase"` |
| + DirectoryRevision uint64 `json:"directory_revision"` |
| + Hosts []BackfillHost `json:"hosts"` |
| + // Reports is each box's census report digest, adopting only. |
| + Reports map[string]string `json:"reports,omitempty"` |
| + // Unowned lists the hosts no box reported. They keep no box and are |
| + // never adopted. Adopting only. |
| + Unowned []string `json:"unowned,omitempty"` |
| +} |
| + |
| +var ( |
| + // ErrBackfillState is the refusal when the roster or the floor does not |
| + // allow a backfill yet. |
| + ErrBackfillState = errors.New("backfill refused: the roster must be published with a home region and a verified floor") |
| + // ErrBackfillExists is backfill-collect's refusal over an existing plan. |
| + ErrBackfillExists = errors.New("backfill refused: a backfill plan already exists") |
| + // ErrBackfillRecords is backfill-collect's refusal when a record cannot |
| + // be frozen: one without an acquisition, or records that changed while |
| + // the plan was written. |
| + ErrBackfillRecords = errors.New("backfill refused: every instance record must carry an acquisition and stay unchanged while the plan is written") |
| + // ErrBackfillPlan is backfill-adopt's refusal when the plan is not the |
| + // named reservation, still collecting, at this roster revision. |
| + ErrBackfillPlan = errors.New("backfill refused: the plan must be the named reservation, still collecting, at this roster revision") |
| +) |
| + |
| +// backfillReady is both phases' refusal up front: a published roster with a |
| +// home region, whose floor digest is the published floor's. |
| +func backfillReady(ctx context.Context, d hostactors.Directory, o hostactors.Observed) error { |
| + if o.Roster.Publication != "published" || o.Roster.HomeRegion == "" || o.Roster.FloorSHA256 == "" { |
| + return ErrBackfillState |
| + } |
| + floor, err := hostfloor.Read(ctx, d.Store, d.PublicKey, d.Bucket) |
| + if err != nil { |
| + return errors.Join(ErrBackfillState, err) |
| + } |
| + if floor.Digest != o.Roster.FloorSHA256 { |
| + return ErrBackfillState |
| + } |
| + return nil |
| +} |
| + |
| +// listBackfillHosts enumerates every record in the deployment through the |
| +// same listing the box census and the adopters read (hostops.Apps and |
| +// hostops.Records). |
| +func listBackfillHosts(ctx context.Context, s release.Store) (map[string]hostops.BackfillHost, error) { |
| + apps, err := hostops.Apps(ctx, s) |
| + if err != nil { |
| + return nil, err |
| + } |
| + hosts := map[string]hostops.BackfillHost{} |
| + for _, app := range apps { |
| + records, err := hostops.Records(ctx, s, app.Account, app.Name) |
| + if err != nil { |
| + return nil, err |
| + } |
| + for _, r := range records { |
| + if _, dup := hosts[r.Holder.Host]; dup || r.Holder.Host == "" || r.Holder.Key == "" || r.ETag == "" { |
| + return nil, ErrBackfillRecords |
| + } |
| + if r.Holder.Acquisition == "" { |
| + return nil, ErrBackfillRecords |
| + } |
| + hosts[r.Holder.Host] = hostops.BackfillHost{Key: r.Holder.Key, ETag: r.ETag, Acquisition: r.Holder.Acquisition} |
| + } |
| + } |
| + return hosts, nil |
| +} |
| + |
| +func backfillSummary(p hostops.BackfillPlan) BackfillSummary { |
| + out := BackfillSummary{ID: p.ID, Phase: p.Phase, DirectoryRevision: p.DirectoryRevision, Hosts: []BackfillHost{}, Reports: p.Reports} |
| + for host, h := range p.Hosts { |
| + out.Hosts = append(out.Hosts, BackfillHost{Host: host, BackfillHost: h}) |
| + if p.Phase == "adopting" && h.Box == "" { |
| + out.Unowned = append(out.Unowned, host) |
| + } |
| + } |
| + sort.Slice(out.Hosts, func(i, j int) bool { return out.Hosts[i].Host < out.Hosts[j].Host }) |
| + sort.Strings(out.Unowned) |
| + return out |
| +} |
| + |
| +// readSignedPlan reads the backfill plan and verifies its root signature. |
| +func readSignedPlan(ctx context.Context, d hostactors.Directory) (hostops.BackfillPlan, string, error) { |
| + var p hostops.BackfillPlan |
| + raw, tag, err := d.Store.Get(ctx, hostops.BackfillKey) |
| + if err != nil { |
| + return p, "", err |
| + } |
| + if tag == "" || release.VerifyRootEnvelope(d.PublicKey, raw, &p) != nil || p.V != 1 { |
| + return p, "", ErrBackfillPlan |
| + } |
| + return p, tag, nil |
| +} |
| + |
| +// backfillCollect writes the signed collecting plan, create-only. The plan |
| +// freezes every instance record in the deployment by key and etag: the box |
| +// census refuses any record it does not name, and claim-mode record writers |
| +// refuse while it exists. It is read back, verified, and the records are |
| +// listed again afterwards: a record that moved while the plan was written |
| +// is a refusal, because the box census would refuse the plan anyway. |
| +func (r Runner) backfillCollect(ctx context.Context, d hostactors.Directory, o hostactors.Observed, in BackfillInput, result *Result) error { |
| + if !validBackfillID(in.ID) { |
| + return ErrActivation |
| + } |
| + if err := backfillReady(ctx, d, o); err != nil { |
| + return err |
| + } |
| + if _, _, err := d.Store.Get(ctx, hostops.BackfillKey); err == nil { |
| + return ErrBackfillExists |
| + } else if !errors.Is(err, os.ErrNotExist) { |
| + return err |
| + } |
| + hosts, err := listBackfillHosts(ctx, d.Store) |
| + if err != nil { |
| + return err |
| + } |
| + p := hostops.BackfillPlan{V: 1, ID: in.ID, Phase: "collecting", DirectoryRevision: o.Roster.Revision, CreatedAt: time.Now().UTC(), Hosts: hosts} |
| + body, err := release.SignRootEnvelope(r.Signer, p) |
| + if err != nil { |
| + return err |
| + } |
| + if _, err = d.Store.Put(ctx, hostops.BackfillKey, body, ""); err != nil { |
| + return err |
| + } |
| + got, _, err := readSignedPlan(ctx, d) |
| + if err != nil { |
| + return err |
| + } |
| + if got.Phase != "collecting" || got.ID != in.ID || got.DirectoryRevision != o.Roster.Revision || !reflect.DeepEqual(got.Hosts, hosts) { |
| + return ErrBackfillPlan |
| + } |
| + again, err := listBackfillHosts(ctx, d.Store) |
| + if err != nil { |
| + return err |
| + } |
| + if !reflect.DeepEqual(again, hosts) { |
| + return ErrBackfillRecords |
| + } |
| + summary := backfillSummary(got) |
| + result.Backfill = &summary |
| + return nil |
| +} |
| + |
| +// backfillAdopt advances the named collecting plan to adopting through |
| +// hostops' Backfill.Publish, under the plan's own etag (If-Match). Publish |
| +// requires every roster box's signed census report for the reservation, |
| +// freezes their digests, takes each host's box and region from them, and |
| +// refuses unfinished seed state. The adopters' own acceptance |
| +// (VerifyAdopting) is then run on the result as readback. |
| +func (r Runner) backfillAdopt(ctx context.Context, d hostactors.Directory, o hostactors.Observed, in BackfillInput, result *Result) error { |
| + if !validBackfillID(in.ID) { |
| + return ErrActivation |
| + } |
| + if err := backfillReady(ctx, d, o); err != nil { |
| + return err |
| + } |
| + p, tag, err := readSignedPlan(ctx, d) |
| + if err != nil { |
| + return err |
| + } |
| + if p.ID != in.ID || p.Phase != "collecting" || p.DirectoryRevision != o.Roster.Revision { |
| + return ErrBackfillPlan |
| + } |
| + b := &hostops.Backfill{Store: d.Store, Directory: d} |
| + if err := b.Publish(ctx, r.Signer, tag); err != nil { |
| + return err |
| + } |
| + got, err := b.VerifyAdopting(ctx) |
| + if err != nil { |
| + return err |
| + } |
| + if got.ID != in.ID || got.DirectoryRevision != o.Roster.Revision || len(got.Hosts) != len(p.Hosts) { |
| + return ErrBackfillPlan |
| + } |
| + for host, h := range p.Hosts { |
| + n, ok := got.Hosts[host] |
| + if !ok || n.Key != h.Key || n.ETag != h.ETag || n.Acquisition != h.Acquisition { |
| + return ErrBackfillPlan |
| + } |
| + } |
| + summary := backfillSummary(got) |
| + result.Backfill = &summary |
| + return nil |
| +} |
| + |
| +// validBackfillID keeps the reservation id one safe path segment. |
| +func validBackfillID(id string) bool { |
| + if id == "" || len(id) > 64 || id == "." || id == ".." { |
| + return false |
| + } |
| + for _, c := range id { |
| + if !(c >= 'a' && c <= 'z' || c >= '0' && c <= '9' || c == '-' || c == '_' || c == '.') { |
| + return false |
| + } |
| + } |
| + return true |
| +} |
diff --git a/internal/claimactivation/backfill_test.go b/internal/claimactivation/backfill_test.go| new file mode 100644 |
| index 000000000..c2d662e72 |
| --- /dev/null |
| +++ b/internal/claimactivation/backfill_test.go |
| @@ -0,0 +1,299 @@ |
| +package claimactivation |
| + |
| +import ( |
| + "context" |
| + "crypto/ed25519" |
| + "crypto/rand" |
| + "errors" |
| + "os" |
| + "strings" |
| + "testing" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/hostactors" |
| + "github.com/carlosframework/platform/internal/hostbox" |
| + "github.com/carlosframework/platform/internal/hostfloor" |
| + "github.com/carlosframework/platform/internal/hostops" |
| + "github.com/carlosframework/platform/internal/provision" |
| + "github.com/carlosframework/platform/internal/release" |
| +) |
| + |
| +// backfillDeployment is a dev-shaped deployment at step 8: a published |
| +// roster at home region ie with a verified floor, leo in pool cohort1 with |
| +// its census key bound, sophie in pool console (with or without a census |
| +// key), and three instance records. |
| +type backfillDeployment struct { |
| + r Runner |
| + keys map[string]ed25519.PrivateKey |
| + records map[string]hostops.Record |
| +} |
| + |
| +func newBackfillDeployment(t *testing.T, sophieKey bool) *backfillDeployment { |
| + t.Helper() |
| + return newBackfillDeploymentAt(t, sophieKey, true) |
| +} |
| + |
| +func newBackfillDeploymentAt(t *testing.T, sophieKey, published bool) *backfillDeployment { |
| + t.Helper() |
| + ctx := t.Context() |
| + b := &backfillDeployment{r: runnerFixture(t), keys: map[string]ed25519.PrivateKey{}, records: map[string]hostops.Record{}} |
| + d := b.r.Directory |
| + if err := d.Initialize(ctx, b.r.Signer); err != nil { |
| + t.Fatal(err) |
| + } |
| + o, err := d.Read(ctx) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := hostfloor.Publish(ctx, d.Store, b.r.Signer, "bucket", hostfloor.Floor{Min: 1, MinRelease: "v1.0.0"}); err != nil { |
| + t.Fatal(err) |
| + } |
| + f, err := hostfloor.Read(ctx, d.Store, d.PublicKey, "bucket") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + next := hostactors.Clone(o.Roster) |
| + next.RequestLimitNS = int64(hostbox.RequestLimit) |
| + next.Regions = map[string]hostactors.Region{"ie": {Account: "123456789012", AWSRegion: "eu-west-1", Namespaces: []string{"platform"}, Pools: []string{"cohort1", "console"}}} |
| + next.HomeRegion = "ie" |
| + if published { |
| + next.Publication = "published" |
| + next.FloorSHA256 = f.Digest |
| + } |
| + for i, box := range []struct{ ref, pool string }{{"leo-climbing", "cohort1"}, {"sophie-knitting", "console"}} { |
| + row := hostactors.Box{Enrollment: strings.Repeat(string(rune('a'+i)), 32), Namespace: "platform", BoxID: box.ref, Label: box.ref, Region: "ie", Pool: box.pool, |
| + Cloud: hostactors.Cloud{Provider: "aws", Account: "123456789012", Region: "eu-west-1", InstanceID: "i-" + box.ref}, EvidenceSHA256: strings.Repeat("b", 64), State: "active", LossIDs: []string{}} |
| + if box.ref == "leo-climbing" || sophieKey { |
| + pub, key, _ := ed25519.GenerateKey(rand.Reader) |
| + row.CensusPublicKey, b.keys[box.ref] = pub, key |
| + } |
| + next.Boxes[box.ref] = row |
| + } |
| + if _, err := d.Replace(ctx, b.r.Signer, o, next); err != nil { |
| + t.Fatal(err) |
| + } |
| + for _, rec := range []provision.Record{ |
| + {Schema: 1, Account: "bab", App: "claimtrial", Host: "claimtrial.example.com", Slug: "claimtrial", Channel: "edge", Pool: "cohort1", Acquisition: "acq-claimtrial"}, |
| + {Schema: 1, Account: "bab", App: "rollcheck", Host: "rollcheck.example.com", Slug: "rollcheck", Channel: "edge", Acquisition: "acq-rollcheck"}, |
| + {Schema: 1, Account: "bab", App: "wakecheck", Host: "wakecheck.example.com", Slug: "wakecheck", Channel: "edge", Acquisition: "acq-wakecheck"}, |
| + } { |
| + b.record(t, rec) |
| + } |
| + return b |
| +} |
| + |
| +func (b *backfillDeployment) record(t *testing.T, rec provision.Record) { |
| + t.Helper() |
| + r, err := hostops.PrepareInstance(rec, time.Date(2026, 10, 18, 9, 0, 0, 0, time.UTC), "ie") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if _, err := b.r.Directory.Store.Put(t.Context(), r.Holder.Key, r.Body, ""); err != nil { |
| + t.Fatal(err) |
| + } |
| + if r, err = hostops.ReadRecord(t.Context(), b.r.Directory.Store, r.Holder.Key); err != nil { |
| + t.Fatal(err) |
| + } |
| + b.records[rec.Host] = r |
| +} |
| + |
| +func (b *backfillDeployment) revision(t *testing.T) uint64 { |
| + t.Helper() |
| + o, err := b.r.Directory.Read(t.Context()) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + return o.Roster.Revision |
| +} |
| + |
| +func (b *backfillDeployment) plan(t *testing.T, phase string) Plan { |
| + return Plan{Phase: phase, Bucket: "bucket", ExpectedRevision: b.revision(t), Backfill: &BackfillInput{ID: "cohort1-20261021"}} |
| +} |
| + |
| +// report publishes box's signed census report for the reservation, naming |
| +// hosts as its route installations: what journal-migrate's census publishes. |
| +func (b *backfillDeployment) report(t *testing.T, box string, hosts ...string) { |
| + t.Helper() |
| + o, err := b.r.Directory.Read(t.Context()) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + report := hostops.CensusReport{Box: box, Enrollment: o.Roster.Boxes[box].Enrollment, BootID: "boot", Reservation: "cohort1-20261021", DirectoryRevision: o.Roster.Revision, |
| + CapturedAt: time.Now().UTC(), Complete: true, Installations: []hostops.CensusInstallation{}, Seeds: &hostops.CensusSeeds{Journaled: []string{}, Gated: []string{}}} |
| + for _, host := range hosts { |
| + r := b.records[host] |
| + inc, err := r.Incarnation() |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + report.Installations = append(report.Installations, hostops.CensusInstallation{Host: host, Account: r.Holder.Account, App: r.Holder.App, Family: r.Holder.Family, Key: r.Holder.Key, Incarnation: inc, Route: true}) |
| + } |
| + if err := hostops.PublishCensusReport(t.Context(), b.r.Directory.Store, b.r.Directory, report, b.keys[box]); err != nil { |
| + t.Fatal(err) |
| + } |
| +} |
| + |
| +func readPlan(t *testing.T, r Runner) hostops.BackfillPlan { |
| + t.Helper() |
| + p, _, err := readSignedPlan(t.Context(), r.Directory) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + return p |
| +} |
| + |
| +// backfill-collect freezes every record the bucket lists, create-only, and |
| +// prints the plan on a dry run without writing it. backfill-adopt, once each |
| +// roster box has published its census, advances it under If-Match, takes |
| +// the boxes from the census, and leaves unreported hosts without one. |
| +func TestBackfillPhases(t *testing.T) { |
| + ctx := t.Context() |
| + b := newBackfillDeployment(t, true) |
| + dry, err := b.r.Run(ctx, b.plan(t, "backfill-collect"), true) |
| + if err != nil || len(dry.Writes) != 1 || dry.Writes[0].Key != hostops.BackfillKey || dry.Writes[0].BeforeETag != "" || dry.Backfill == nil || len(dry.Backfill.Hosts) != 3 { |
| + t.Fatalf("collect dry run = %+v, %v", dry, err) |
| + } |
| + if _, _, err := b.r.Directory.Store.Get(ctx, hostops.BackfillKey); !errors.Is(err, os.ErrNotExist) { |
| + t.Fatal("dry run wrote the plan", err) |
| + } |
| + got, err := b.r.Run(ctx, b.plan(t, "backfill-collect"), false) |
| + if err != nil || got.Backfill.Phase != "collecting" || len(got.Backfill.Hosts) != 3 { |
| + t.Fatalf("collect = %+v, %v", got, err) |
| + } |
| + p := readPlan(t, b.r) |
| + for host, r := range b.records { |
| + h := p.Hosts[host] |
| + if h.Key != r.Holder.Key || h.ETag != r.ETag || h.Acquisition != r.Holder.Acquisition || h.Box != "" || h.Region != "" { |
| + t.Fatalf("frozen %s = %+v", host, h) |
| + } |
| + } |
| + if p.Phase != "collecting" || p.DirectoryRevision != b.revision(t) || p.ID != "cohort1-20261021" { |
| + t.Fatalf("plan = %+v", p) |
| + } |
| + if _, err := b.r.Run(ctx, b.plan(t, "backfill-collect"), false); !errors.Is(err, ErrBackfillExists) { |
| + t.Fatalf("second collect = %v", err) |
| + } |
| + // Adopt waits for every roster box's census report. |
| + if _, err := b.r.Run(ctx, b.plan(t, "backfill-adopt"), false); err == nil { |
| + t.Fatal("adopted without census reports") |
| + } |
| + b.report(t, "leo-climbing", "claimtrial.example.com") |
| + if _, err := b.r.Run(ctx, b.plan(t, "backfill-adopt"), false); err == nil { |
| + t.Fatal("adopted without sophie's report") |
| + } |
| + b.report(t, "sophie-knitting", "wakecheck.example.com") |
| + wrong := b.plan(t, "backfill-adopt") |
| + wrong.Backfill.ID = "other" |
| + if _, err := b.r.Run(ctx, wrong, false); !errors.Is(err, ErrBackfillPlan) { |
| + t.Fatalf("other reservation = %v", err) |
| + } |
| + dry, err = b.r.Run(ctx, b.plan(t, "backfill-adopt"), true) |
| + if err != nil || len(dry.Writes) != 1 || dry.Backfill.Phase != "adopting" { |
| + t.Fatalf("adopt dry run = %+v, %v", dry, err) |
| + } |
| + if readPlan(t, b.r).Phase != "collecting" { |
| + t.Fatal("dry run advanced the plan") |
| + } |
| + got, err = b.r.Run(ctx, b.plan(t, "backfill-adopt"), false) |
| + if err != nil || got.Backfill.Phase != "adopting" || len(got.Backfill.Reports) != 2 || len(got.Backfill.Unowned) != 1 || got.Backfill.Unowned[0] != "rollcheck.example.com" { |
| + t.Fatalf("adopt = %+v, %v", got.Backfill, err) |
| + } |
| + if got.Writes[0].BeforeETag == "" { |
| + t.Fatal("adopt wrote without If-Match") |
| + } |
| + p = readPlan(t, b.r) |
| + if h := p.Hosts["claimtrial.example.com"]; h.Box != "leo-climbing" || h.Region != "ie" { |
| + t.Fatalf("claimtrial = %+v", h) |
| + } |
| + if h := p.Hosts["wakecheck.example.com"]; h.Box != "sophie-knitting" { |
| + t.Fatalf("wakecheck = %+v", h) |
| + } |
| + if _, err := b.r.Run(ctx, b.plan(t, "backfill-adopt"), false); !errors.Is(err, ErrBackfillPlan) { |
| + t.Fatalf("second adopt = %v", err) |
| + } |
| +} |
| + |
| +// The dev roster at step 8: sophie is a roster box with no census key, so |
| +// no census report of hers can verify, and backfill-adopt refuses. The plan |
| +// stays collecting. |
| +func TestBackfillAdoptRefusesABoxWithoutCensusKey(t *testing.T) { |
| + ctx := t.Context() |
| + b := newBackfillDeployment(t, false) |
| + if _, err := b.r.Run(ctx, b.plan(t, "backfill-collect"), false); err != nil { |
| + t.Fatal(err) |
| + } |
| + b.report(t, "leo-climbing", "claimtrial.example.com") |
| + if _, err := b.r.Run(ctx, b.plan(t, "backfill-adopt"), false); err == nil { |
| + t.Fatal("adopted with a roster box that cannot report") |
| + } |
| + if readPlan(t, b.r).Phase != "collecting" { |
| + t.Fatal("plan moved") |
| + } |
| +} |
| + |
| +// Refusals up front: a roster that is not published, or has no home region |
| +// or no verified floor; a reservation id that is not one safe segment; a |
| +// record with no acquisition. |
| +func TestBackfillCollectRefusesUpFront(t *testing.T) { |
| + ctx := t.Context() |
| + if _, err := newBackfillDeploymentAt(t, true, false).r.Run(ctx, Plan{Phase: "backfill-collect", Bucket: "bucket", ExpectedRevision: 2, Backfill: &BackfillInput{ID: "cohort1-20261021"}}, true); !errors.Is(err, ErrBackfillState) { |
| + t.Fatalf("collect on an unpublished roster = %v", err) |
| + } |
| + gone := newBackfillDeployment(t, true) |
| + for _, key := range []string{hostfloor.Key, hostfloor.AuthKey} { |
| + if err := gone.r.Directory.Store.Delete(ctx, key); err != nil { |
| + t.Fatal(err) |
| + } |
| + } |
| + if _, err := gone.r.Run(ctx, gone.plan(t, "backfill-collect"), true); !errors.Is(err, ErrBackfillState) { |
| + t.Fatalf("collect with no verified floor = %v", err) |
| + } |
| + b := newBackfillDeployment(t, true) |
| + for _, id := range []string{"", "a/b", "..", "UPPER", strings.Repeat("a", 65)} { |
| + p := b.plan(t, "backfill-collect") |
| + p.Backfill.ID = id |
| + if _, err := b.r.Run(ctx, p, true); !errors.Is(err, ErrActivation) { |
| + t.Fatalf("id %q = %v", id, err) |
| + } |
| + } |
| + p := b.plan(t, "backfill-collect") |
| + p.Backfill = nil |
| + if _, err := b.r.Run(ctx, p, true); !errors.Is(err, ErrActivation) { |
| + t.Fatalf("no backfill input = %v", err) |
| + } |
| + b.record(t, provision.Record{Schema: 1, Account: "bab", App: "legacy", Host: "legacy.example.com", Slug: "legacy", Channel: "edge"}) |
| + if _, err := b.r.Run(ctx, b.plan(t, "backfill-collect"), true); !errors.Is(err, ErrBackfillRecords) { |
| + t.Fatalf("record without acquisition = %v", err) |
| + } |
| +} |
| + |
| +// A record that moves while the plan is being written leaves a plan the box |
| +// census would refuse; collect reports it rather than succeeding. |
| +type movingRecord struct { |
| + release.Store |
| + key string |
| + body []byte |
| + done bool |
| +} |
| + |
| +func (s *movingRecord) Put(ctx context.Context, key string, raw []byte, before string) (string, error) { |
| + tag, err := s.Store.Put(ctx, key, raw, before) |
| + if err == nil && key == hostops.BackfillKey && !s.done { |
| + s.done = true |
| + _, old, _ := s.Store.Get(ctx, s.key) |
| + if _, e := s.Store.Put(ctx, s.key, s.body, old); e != nil { |
| + return "", e |
| + } |
| + } |
| + return tag, err |
| +} |
| + |
| +func TestBackfillCollectRefusesARecordThatMoved(t *testing.T) { |
| + b := newBackfillDeployment(t, true) |
| + r := b.records["claimtrial.example.com"] |
| + body := append([]byte(nil), r.Body...) |
| + b.r.Directory.Store = &movingRecord{Store: b.r.Directory.Store, key: r.Holder.Key, body: append(body, '\n')} |
| + if _, err := b.r.Run(t.Context(), b.plan(t, "backfill-collect"), false); !errors.Is(err, ErrBackfillRecords) { |
| + t.Fatalf("collect over a moving record = %v", err) |
| + } |
| +} |
diff --git a/internal/claimactivation/runner.go b/internal/claimactivation/runner.go| index d89f734fe..32fc38a05 100644 |
| --- a/internal/claimactivation/runner.go |
| +++ b/internal/claimactivation/runner.go |
| @@ -16,6 +16,7 @@ import ( |
| "github.com/carlosframework/platform/internal/hostactors" |
| "github.com/carlosframework/platform/internal/hostbox" |
| "github.com/carlosframework/platform/internal/hostfloor" |
| + "github.com/carlosframework/platform/internal/hostops" |
| "github.com/carlosframework/platform/internal/release" |
| ) |
| @@ -33,6 +34,7 @@ type Plan struct { |
| Binding *hostactors.CensusKeyBinding `json:"binding,omitempty"` |
| Proof []byte `json:"proof,omitempty"` |
| Identity *IdentityInput `json:"identity,omitempty"` |
| + Backfill *BackfillInput `json:"backfill,omitempty"` |
| } |
| // IdentityInput is publish-identity's plan: a release tag, its moves |
| @@ -90,6 +92,9 @@ type Result struct { |
| Release *release.SystemRelease `json:"release,omitempty"` |
| OutputSHA256 string `json:"output_sha256,omitempty"` |
| Output []byte `json:"-"` |
| + // backfill-collect and backfill-adopt only: the signed plan as read |
| + // back after the write (on a dry run, the plan that would be written). |
| + Backfill *BackfillSummary `json:"backfill,omitempty"` |
| } |
| type Runner struct { |
| Directory hostactors.Directory |
| @@ -162,6 +167,8 @@ func (r Runner) Run(ctx context.Context, p Plan, dry bool) (Result, error) { |
| } |
| case "sign-policy": |
| // Writes nothing: the policy goes to the operator's file. |
| + case "backfill-collect", "backfill-adopt": |
| + allowed[hostops.BackfillKey] = true |
| } |
| store := &checkedStore{allowed: allowed, Store: d.Store, pub: d.PublicKey, bucket: d.Bucket, expected: o.ETag, dry: dry, overlay: map[string]overlayObject{}, writes: &result.Writes} |
| d.Store = store |
| @@ -284,6 +291,16 @@ func (r Runner) Run(ctx context.Context, p Plan, dry bool) (Result, error) { |
| return result, ErrActivation |
| } |
| err = r.publishIdentity(ctx, d, *p.Identity, dry, &result) |
| + case "backfill-collect": |
| + if p.Backfill == nil { |
| + return result, ErrActivation |
| + } |
| + err = r.backfillCollect(ctx, d, o, *p.Backfill, &result) |
| + case "backfill-adopt": |
| + if p.Backfill == nil { |
| + return result, ErrActivation |
| + } |
| + err = r.backfillAdopt(ctx, d, o, *p.Backfill, &result) |
| default: |
| return result, ErrActivation |
| } |
diff --git a/internal/host/claimcensus.go b/internal/host/claimcensus.go| index 0b79ddd6b..06c2abd47 100644 |
| --- a/internal/host/claimcensus.go |
| +++ b/internal/host/claimcensus.go |
| @@ -11,6 +11,7 @@ import ( |
| "github.com/carlosframework/platform/internal/hostbox" |
| "github.com/carlosframework/platform/internal/hostwrite" |
| "github.com/carlosframework/platform/internal/release" |
| + "github.com/carlosframework/platform/internal/rollout" |
| ) |
| // claimGlobalSharedPaths is the empty-installation census's classification |
| @@ -63,7 +64,43 @@ func claimDataCensus(dataDir string) []string { |
| // it, so the fallback stays, as moves asked: .moves is a shared tree, |
| // not opaque: regular files and directories only, and anything naming |
| // the host being censused still refuses. |
| - return []string{".hostclaims", ".hostexec", "hostguard", "relocate", stage, staged, hostbox.CensusTree(movesDirName), hostbox.CensusFile(reg), hostbox.CensusFile(reg + "-wal"), hostbox.CensusFile(reg + "-shm")} |
| + entries := []string{".hostclaims", ".hostexec", "hostguard", "relocate", stage, staged, hostbox.CensusTree(movesDirName), hostbox.CensusFile(reg), hostbox.CensusFile(reg + "-wal"), hostbox.CensusFile(reg + "-shm")} |
| + return append(entries, claimBoxDataCensus(dataDir)...) |
| +} |
| + |
| +// claimBoxDataCensus is the box's own state under the data root, found on a |
| +// serving box (leo, 2026-10-19) and owned by no installation (step 8 |
| +// readiness, R2 (b) and (c)). Each entry is the exact path its writer |
| +// uses, a regular file only, except the edge's certificate cache, which is a |
| +// shared tree of directories and regular files. Nothing here names a host, |
| +// and the per-host census still refuses any path that names the host it |
| +// censuses, so a certificate for that host still refuses its install. |
| +func claimBoxDataCensus(dataDir string) []string { |
| + rel := func(path string) string { return claimCensusRel(dataDir, path) } |
| + files := []string{ |
| + // The edge's autocert cache: certificates and keys, the ACME account |
| + // key (edge.go's -acme-cache default; bootstrap makes the directory). |
| + hostbox.CensusTree(rel(filepath.Join(dataDir, "acme"))), |
| + } |
| + for _, path := range []string{ |
| + filepath.Join(dataDir, stageRequestMarker), // the agent's root stage request |
| + filepath.Join(dataDir, systemRequestMarker), // the agent's self-update request |
| + systemRequestPath(dataDir), // the update the agent asked for |
| + restartMarksPath(dataDir), // root's restart memory |
| + restartMarksPath(dataDir) + ".lock", // and its lock (lockRestartMarks) |
| + adoptPassLockPath(dataDir), // every root adoption pass's lock |
| + filepath.Join(dataDir, rollout.FailureFile), // the swap failure record |
| + release.FirstSeenPath(dataDir), // channel first-seen observations |
| + release.GenerationsPath(dataDir), // zero-downtime generations |
| + release.GenerationsSeqPath(release.GenerationsPath(dataDir)), |
| + // The edge's GeoIP mirror pointer (edge.go newGeoMirror's directory, |
| + // geoipmirror.go's "current") and the metrics loop's alert state. |
| + filepath.Join(dataDir, "carlos", "geoip", "current"), |
| + filepath.Join(dataDir, "monitor", "alerts.json"), |
| + } { |
| + files = append(files, hostbox.CensusFile(rel(path))) |
| + } |
| + return files |
| } |
| func claimOptCensus(optDir string) []string { |
diff --git a/internal/host/claimcensus_migration_linux.go b/internal/host/claimcensus_migration_linux.go| new file mode 100644 |
| index 000000000..8b37f43b3 |
| --- /dev/null |
| +++ b/internal/host/claimcensus_migration_linux.go |
| @@ -0,0 +1,224 @@ |
| +//go:build linux |
| + |
| +package host |
| + |
| +import ( |
| + "errors" |
| + "fmt" |
| + "os" |
| + "path/filepath" |
| + "slices" |
| + "syscall" |
| + |
| + "github.com/carlosframework/platform/internal/hostbox" |
| + "github.com/carlosframework/platform/internal/provision" |
| + "github.com/carlosframework/platform/internal/release" |
| +) |
| + |
| +// claimWindowUnits are the twelve units the attended offline window masks |
| +// persistently before journal-migrate (dev cohort-1 runbook 8.1): the five |
| +// triggers, the six migration services and the edge. Each mask is the |
| +// unit file moved into <window>/held and a /dev/null link in its place. |
| +var claimWindowUnits = []string{ |
| + "carlos-adopt.timer", "carlos-adopt.path", "carlos-stage.path", "carlos-restart.path", "carlos-system-update.path", |
| + "carlos-adopt.service", "carlos-stage.service", "carlos-config-sync.service", "carlos-restart.service", "carlos-system-update.service", "carlos-system-recover.service", |
| + "carlos-edge.service", |
| +} |
| + |
| +var ( |
| + // errMigrationWindow refuses a --window that is not the attended |
| + // window's private directory, or a mask without its held unit. |
| + errMigrationWindow = errors.New("the window directory must be root's own private directory outside the unit and data paths, with every masked unit's original in its held directory") |
| +) |
| + |
| +// verifyMigrationWindow returns the window units that window masks: each |
| +// <units>/<unit> that is a symlink whose raw target is exactly /dev/null, |
| +// with its original unit file held in <window>/held/<unit>, a root-owned |
| +// regular file with one link. The window and held directories must be |
| +// root's, closed to group and other, outside every systemd unit path and |
| +// the census roots. Any other symlink at a window unit's path, or a mask |
| +// without its held original, refuses. An empty window masks nothing: the |
| +// census then refuses masks, as it always has. |
| +func verifyMigrationWindow(window string, r ClaimRuntimeResources) (map[string]bool, error) { |
| + masked := map[string]bool{} |
| + if window == "" { |
| + return masked, nil |
| + } |
| + if !filepath.IsAbs(window) || filepath.Clean(window) != window { |
| + return nil, errMigrationWindow |
| + } |
| + for _, root := range []string{"/etc/systemd", "/run/systemd", "/usr/lib/systemd", "/lib/systemd", r.UnitsDir, r.DataDir, r.OptDir} { |
| + if root == "" { |
| + continue |
| + } |
| + if rel, err := filepath.Rel(root, window); err == nil && filepath.IsLocal(rel) || window == root { |
| + return nil, errMigrationWindow |
| + } |
| + } |
| + held := filepath.Join(window, "held") |
| + for _, dir := range []string{window, held} { |
| + if err := rootPrivateEntry(dir, true); err != nil { |
| + return nil, err |
| + } |
| + } |
| + for _, unit := range claimWindowUnits { |
| + path := filepath.Join(r.UnitsDir, unit) |
| + info, err := os.Lstat(path) |
| + if errors.Is(err, os.ErrNotExist) { |
| + continue |
| + } |
| + if err != nil { |
| + return nil, err |
| + } |
| + if info.Mode()&os.ModeSymlink == 0 { |
| + continue |
| + } |
| + target, err := os.Readlink(path) |
| + if err != nil { |
| + return nil, err |
| + } |
| + if target != "/dev/null" { |
| + return nil, errMigrationWindow |
| + } |
| + if err := rootPrivateEntry(filepath.Join(held, unit), false); err != nil { |
| + return nil, err |
| + } |
| + masked[unit] = true |
| + } |
| + return masked, nil |
| +} |
| + |
| +// rootPrivateEntry requires path to be root's: a directory closed to group |
| +// and other, or a regular file with one link. Never a symlink. |
| +func rootPrivateEntry(path string, dir bool) error { |
| + info, err := os.Lstat(path) |
| + if err != nil { |
| + return fmt.Errorf("%w: %v", errMigrationWindow, err) |
| + } |
| + st, ok := info.Sys().(*syscall.Stat_t) |
| + if !ok || st.Uid != 0 { |
| + return errMigrationWindow |
| + } |
| + if dir { |
| + if !info.IsDir() || info.Mode().Perm()&0o077 != 0 { |
| + return errMigrationWindow |
| + } |
| + return nil |
| + } |
| + if !info.Mode().IsRegular() || st.Nlink != 1 { |
| + return errMigrationWindow |
| + } |
| + return nil |
| +} |
| + |
| +// claimMigrationSharedPaths is the migration census's classification: the |
| +// shared classes every claim census uses (claimGlobalSharedPaths), plus three |
| +// that exist only at migration, each exact. |
| +// |
| +// - The window's masks: each window unit masked through --window is a |
| +// symlink to exactly /dev/null, in place of the regular unit file. |
| +// - A legacy route installation the census reports (Route, not Local): |
| +// its nested instance directory and its raw directory, holding only |
| +// directories and regular files; its program history file; and its live |
| +// link, whose target is exactly the version path its row names. |
| +// - A retained database: a frozen instance record's leftovers on a box |
| +// with no route row and no claim for it (a removed route). The same |
| +// paths, with a live link into the record's own app under one version. |
| +// Retained data is never reported, so never adopted, and the census |
| +// deletes nothing. |
| +func claimMigrationSharedPaths(r ClaimRuntimeResources, masked map[string]bool, s rootMigrationSnapshot) (map[string][]string, error) { |
| + shared := claimGlobalSharedPaths(r) |
| + units := shared["units"][:0:0] |
| + for _, entry := range shared["units"] { |
| + replaced := false |
| + for unit := range masked { |
| + if entry == hostbox.CensusFile(unit) { |
| + units = append(units, hostbox.CensusLink(unit, "/dev/null")) |
| + replaced = true |
| + } |
| + } |
| + if !replaced { |
| + units = append(units, entry) |
| + } |
| + } |
| + for unit := range masked { |
| + if !slices.Contains(units, hostbox.CensusLink(unit, "/dev/null")) { |
| + // A masked unit the platform does not install is not the window's. |
| + return nil, errMigrationWindow |
| + } |
| + } |
| + shared["units"] = units |
| + if r.Registry == nil { |
| + return nil, errRootMigrationBusy |
| + } |
| + claimed := map[string]bool{} |
| + for _, item := range s.Items { |
| + claimed[item.Installation.Host] = true |
| + } |
| + for _, in := range s.Report.Installations { |
| + if !in.Route || in.Local { |
| + continue |
| + } |
| + row, err := r.Registry.Get(in.Host) |
| + if err != nil { |
| + return nil, err |
| + } |
| + if row == nil || row.Account != in.Account || row.App != in.App || !release.ValidVersion(row.Version) { |
| + return nil, errRootMigrationBusy |
| + } |
| + data, opt := hostInstallationPaths(r, in.Host) |
| + shared["data"] = append(shared["data"], data...) |
| + shared["opt"] = append(shared["opt"], hostbox.CensusLink(opt, release.VersionPath(r.OptDir, row.Account, row.Version, row.App))) |
| + claimed[in.Host] = true |
| + } |
| + for _, rec := range s.Records { |
| + if claimed[rec.Host] { |
| + continue |
| + } |
| + row, err := r.Registry.Get(rec.Host) |
| + if err != nil { |
| + return nil, err |
| + } |
| + if row != nil { |
| + return nil, errRootMigrationBusy |
| + } |
| + data, opt := hostInstallationPaths(r, rec.Host) |
| + shared["data"] = append(shared["data"], data...) |
| + target, err := os.Readlink(filepath.Join(r.OptDir, opt)) |
| + if errors.Is(err, os.ErrNotExist) { |
| + continue |
| + } |
| + if err != nil { |
| + return nil, err |
| + } |
| + version := filepath.Base(filepath.Dir(target)) |
| + if rec.Account != "" && rec.Account != release.OpsAccount { |
| + version = filepath.Base(filepath.Dir(filepath.Dir(target))) |
| + } |
| + if !release.ValidVersion(version) || target != release.VersionPath(r.OptDir, rec.Account, version, rec.App) { |
| + return nil, errRootMigrationBusy |
| + } |
| + shared["opt"] = append(shared["opt"], hostbox.CensusLink(opt, target)) |
| + } |
| + return shared, nil |
| +} |
| + |
| +// hostInstallationPaths are one host's paths under the nested layout, |
| +// relative to their census roots: the instance directory and its raw |
| +// directory (shared trees of directories and regular files), the program |
| +// history file, and the live link under <opt>. |
| +func hostInstallationPaths(r ClaimRuntimeResources, host string) ([]string, string) { |
| + id := provision.InstanceID(host) |
| + return []string{ |
| + hostbox.CensusTree(claimCensusRel(r.DataDir, filepath.Join(r.DataDir, "instances", id))), |
| + hostbox.CensusTree(claimCensusRel(r.DataDir, filepath.Join(r.DataDir, ".instances", id))), |
| + // release's programPath: the program history the bake keeps per host. |
| + hostbox.CensusFile(claimCensusRel(r.DataDir, filepath.Join(r.DataDir, "programs", host+".json"))), |
| + }, claimCensusRel(r.OptDir, release.LiveLink(r.OptDir, host)) |
| +} |
| + |
| +// verifyJournalMigrateWindow is journal-migrate's up-front check of |
| +// --window, before the migration writes anything. |
| +func verifyJournalMigrateWindow(r ClaimRuntimeResources) (map[string]bool, error) { |
| + return verifyMigrationWindow(r.MigrationWindow, r) |
| +} |
diff --git a/internal/host/claimcensus_migration_linux_test.go b/internal/host/claimcensus_migration_linux_test.go| new file mode 100644 |
| index 000000000..7676d16f2 |
| --- /dev/null |
| +++ b/internal/host/claimcensus_migration_linux_test.go |
| @@ -0,0 +1,332 @@ |
| +//go:build linux |
| + |
| +package host |
| + |
| +import ( |
| + "encoding/json" |
| + "errors" |
| + "os" |
| + "path/filepath" |
| + "strings" |
| + "testing" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/hostbox" |
| + "github.com/carlosframework/platform/internal/hostclaim" |
| + "github.com/carlosframework/platform/internal/hostops" |
| + "github.com/carlosframework/platform/internal/provision" |
| + "github.com/carlosframework/platform/internal/registry" |
| +) |
| + |
| +const ( |
| + leoClaimtrial = "claimtrial.dev.carlosframework.com" |
| + leoWakecheck = "wakecheck.dev.carlosframework.com" |
| + leoRollcheck = "rollcheck-0261.dev.carlosframework.com" |
| +) |
| + |
| +// leoRecords are dev's three bucket records at step 8 (bab/claimtrial, |
| +// bab/rollcheck-0261, bab/wakecheck), made by the production record |
| +// constructor. Only claimtrial is routed on leo. |
| +func leoRecords(t *testing.T) []hostops.Record { |
| + t.Helper() |
| + var out []hostops.Record |
| + for _, rec := range []provision.Record{ |
| + {Schema: 1, Account: "bab", App: "claimtrial", Host: leoClaimtrial, Slug: "claimtrial", Channel: "edge", Pool: "cohort1", Acquisition: "acq-claimtrial"}, |
| + {Schema: 1, Account: "bab", App: "rollcheck-0261", Host: leoRollcheck, Slug: "rollcheck-0261", Channel: "edge", Acquisition: "acq-rollcheck"}, |
| + {Schema: 1, Account: "bab", App: "wakecheck", Host: leoWakecheck, Slug: "wakecheck", Channel: "edge", Acquisition: "acq-wakecheck"}, |
| + } { |
| + r, err := hostops.PrepareInstance(rec, time.Date(2026, 10, 18, 9, 0, 0, 0, time.UTC), "ie") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + out = append(out, r) |
| + } |
| + return out |
| +} |
| + |
| +// leoRoute is leo's one registry row (leo-routes.tsv): claimtrial, an |
| +// instance with no backing and no unit, hibernating, nested, running the |
| +// version its live link names. |
| +func leoRoute(dataDir string) registry.Route { |
| + return registry.Route{Host: leoClaimtrial, Kind: "instance", App: "claimtrial", Account: "bab", Slug: "claimtrial", Socket: provision.NestedSocket(leoClaimtrial), |
| + DBPath: provision.NestedDBPath(dataDir, leoClaimtrial), Version: "wc4", Hibernate: true, Provisioned: true, Pool: "cohort1", Channel: "edge", Status: "ready"} |
| +} |
| + |
| +type leoCensusTree struct { |
| + r ClaimRuntimeResources |
| + roots map[string]string |
| + snap rootMigrationSnapshot |
| +} |
| + |
| +// newLeoCensusTree rebuilds leo's /data and /opt/carlos, and the carlos part |
| +// of /etc/systemd/system, from the snapshot with step 8.1's masks applied. |
| +// Vendor unit links are left out: the census trusts only root-owned links |
| +// into the vendor unit directories, which an unprivileged test cannot make. |
| +// The root-runner witness builds them. |
| +func newLeoCensusTree(t *testing.T) *leoCensusTree { |
| + t.Helper() |
| + base := t.TempDir() |
| + roots := map[string]string{"data": filepath.Join(base, "data"), "opt": filepath.Join(base, "opt"), "units": filepath.Join(base, "units")} |
| + for _, dir := range roots { |
| + if err := os.Mkdir(dir, 0o755); err != nil { |
| + t.Fatal(err) |
| + } |
| + } |
| + buildLeoTree(t, leoTreeRows(t, true), leoTreeOptions{ |
| + Roots: map[string]string{"/data": roots["data"], "/opt/carlos": roots["opt"], "/etc/systemd/system": roots["units"]}, |
| + Skip: func(r leoRow) bool { |
| + return r.Root == "/etc/systemd/system" && r.Type == "l" && (strings.HasPrefix(r.Target, "/usr/lib/") || strings.HasPrefix(r.Target, "/lib/")) |
| + }, |
| + }) |
| + reg, err := registry.Open(filepath.Join(t.TempDir(), "registry.db")) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + t.Cleanup(func() { reg.Close() }) |
| + if err := reg.Upsert(leoRoute(roots["data"])); err != nil { |
| + t.Fatal(err) |
| + } |
| + c := &leoCensusTree{roots: roots, r: ClaimRuntimeResources{Mode: "instance-control", Registry: reg, DataDir: roots["data"], OptDir: roots["opt"], UnitsDir: roots["units"]}} |
| + for _, r := range leoRecords(t) { |
| + var rec provision.Record |
| + if err := json.Unmarshal(r.Body, &rec); err != nil { |
| + t.Fatal(err) |
| + } |
| + c.snap.Records = append(c.snap.Records, rec) |
| + if r.Holder.Host == leoClaimtrial { |
| + inc, _ := r.Incarnation() |
| + c.snap.Report.Installations = append(c.snap.Report.Installations, hostops.CensusInstallation{Host: leoClaimtrial, Account: "bab", App: "claimtrial", Family: "instance", Key: r.Holder.Key, Incarnation: inc, Route: true}) |
| + } |
| + } |
| + return c |
| +} |
| + |
| +func allWindowUnits() map[string]bool { |
| + m := map[string]bool{} |
| + for _, u := range claimWindowUnits { |
| + m[u] = true |
| + } |
| + return m |
| +} |
| + |
| +func (c *leoCensusTree) check(t *testing.T, masked map[string]bool) error { |
| + t.Helper() |
| + shared, err := claimMigrationSharedPaths(c.r, masked, c.snap) |
| + if err != nil { |
| + return err |
| + } |
| + return hostbox.CheckMigrationRoots(t.Context(), c.roots, shared, nil) |
| +} |
| + |
| +// R2: the migration census accepts leo's real tree under the window's masks, |
| +// and only with every class in place. Each class covers these snapshot rows: |
| +// |
| +// (a) the window's masks: the twelve /etc/systemd/system rows of |
| +// leo-units-post-8.1.tsv (carlos-adopt.timer ... carlos-edge.service); |
| +// (b) /data/acme: 17 certificate and key files (platform, tunnel, proof, |
| +// proof2, website, helloworld, appsdomain-check, rollcheck-0261, |
| +// claimtrial, wakecheck, the ACME account key); |
| +// (c) box files: stage-requested, system-update-requested, |
| +// system/router/requested.json, restart-marks.json and its .lock, |
| +// adopt-pass.lock, swap-failures.json, channel-first-seen.json, |
| +// zdd/generations.json and .seq, carlos/geoip/current, |
| +// monitor/alerts.json; |
| +// (d) claimtrial's route installation: instances/ and .instances/ |
| +// claimtrial.dev.carlosframework.com (6 files each), |
| +// programs/claimtrial…json and /opt/carlos/live/claimtrial… (into |
| +// versions/wc4/claimtrial, the version its row names); |
| +// (e) wakecheck's retained database (route removed, record frozen): |
| +// instances/ and .instances/wakecheck.dev.carlosframework.com (8 files |
| +// each), programs/wakecheck…json and /opt/carlos/live/wakecheck… (into |
| +// versions/wc4/wakecheck). |
| +func TestMigrationCensusClassifiesLeosTree(t *testing.T) { |
| + c := newLeoCensusTree(t) |
| + if err := c.check(t, allWindowUnits()); err != nil { |
| + t.Fatalf("leo's tree under the window's masks: %v", err) |
| + } |
| + // Without the window, the masks refuse, as they always have. |
| + if err := c.check(t, nil); !errors.Is(err, hostbox.ErrTypedInventory) || !strings.Contains(err.Error(), "units entry") { |
| + t.Fatalf("masks without --window = %v", err) |
| + } |
| + // Without claimtrial's reported route, its files refuse. |
| + reported := c.snap.Report.Installations |
| + c.snap.Report.Installations = nil |
| + if err := c.check(t, allWindowUnits()); !errors.Is(err, errRootMigrationBusy) && !errors.Is(err, hostbox.ErrTypedInventory) { |
| + t.Fatalf("claimtrial unreported = %v", err) |
| + } |
| + c.snap.Report.Installations = reported |
| + // Without wakecheck's frozen record, its leftovers refuse. |
| + records := c.snap.Records |
| + c.snap.Records = nil |
| + for _, rec := range records { |
| + if rec.Host != leoWakecheck { |
| + c.snap.Records = append(c.snap.Records, rec) |
| + } |
| + } |
| + if err := c.check(t, allWindowUnits()); !errors.Is(err, hostbox.ErrTypedInventory) || !strings.Contains(err.Error(), "wakecheck") { |
| + t.Fatalf("wakecheck without its record = %v", err) |
| + } |
| +} |
| + |
| +// Each class is exact: one entry outside it refuses, naming the entry. |
| +func TestMigrationCensusRefusesOutsideLeosClasses(t *testing.T) { |
| + for name, mutate := range map[string]func(*testing.T, *leoCensusTree){ |
| + "a mask on a unit the window does not mask": func(t *testing.T, c *leoCensusTree) { |
| + leoReplaceWithLink(t, filepath.Join(c.roots["units"], "carlos-relay.service"), "/dev/null") |
| + }, |
| + "a window unit linked elsewhere": func(t *testing.T, c *leoCensusTree) { |
| + leoReplaceWithLink(t, filepath.Join(c.roots["units"], "carlos-edge.service"), "/etc/systemd/system/carlos-relay.service") |
| + }, |
| + "a symlink in claimtrial's instance directory": func(t *testing.T, c *leoCensusTree) { |
| + leoLink(t, "/etc/passwd", filepath.Join(c.roots["data"], ".instances", leoClaimtrial, "escape")) |
| + }, |
| + "a symlink in the certificate cache": func(t *testing.T, c *leoCensusTree) { |
| + leoLink(t, "/etc/passwd", filepath.Join(c.roots["data"], "acme", "escape")) |
| + }, |
| + "an unknown box file": func(t *testing.T, c *leoCensusTree) { |
| + leoWrite(t, filepath.Join(c.roots["data"], "new-marker")) |
| + }, |
| + "a box file replaced by a link": func(t *testing.T, c *leoCensusTree) { |
| + leoReplaceWithLink(t, filepath.Join(c.roots["data"], "adopt-pass.lock"), "/etc/passwd") |
| + }, |
| + "claimtrial's live link on another version": func(t *testing.T, c *leoCensusTree) { |
| + leoReplaceWithLink(t, filepath.Join(c.roots["opt"], "live", leoClaimtrial), filepath.Join(c.roots["opt"], "versions", "wc5", "claimtrial")) |
| + }, |
| + "wakecheck's live link into another app": func(t *testing.T, c *leoCensusTree) { |
| + leoReplaceWithLink(t, filepath.Join(c.roots["opt"], "live", leoWakecheck), filepath.Join(c.roots["opt"], "versions", "wc4", "claimtrial")) |
| + }, |
| + "a live link for a host with no record": func(t *testing.T, c *leoCensusTree) { |
| + leoLink(t, filepath.Join(c.roots["opt"], "versions", "wc4", "claimtrial"), filepath.Join(c.roots["opt"], "live", "other.example.com")) |
| + }, |
| + "a program file for a host with no record": func(t *testing.T, c *leoCensusTree) { |
| + leoWrite(t, filepath.Join(c.roots["data"], "programs", "other.example.com.json")) |
| + }, |
| + "an instance directory for a host with no record": func(t *testing.T, c *leoCensusTree) { |
| + dir := filepath.Join(c.roots["data"], ".instances", "other.example.com") |
| + if err := os.Mkdir(dir, 0o700); err != nil { |
| + t.Fatal(err) |
| + } |
| + leoWrite(t, filepath.Join(dir, "other.example.com.db")) |
| + }, |
| + } { |
| + t.Run(name, func(t *testing.T) { |
| + c := newLeoCensusTree(t) |
| + mutate(t, c) |
| + err := c.check(t, allWindowUnits()) |
| + if !errors.Is(err, hostbox.ErrTypedInventory) && !errors.Is(err, errRootMigrationBusy) { |
| + t.Fatalf("census = %v, want a refusal", err) |
| + } |
| + t.Logf("refused: %v", err) |
| + }) |
| + } |
| +} |
| + |
| +// A route row the census cannot report refuses before any walk: claimtrial |
| +// pointing at another version, a row whose record is not frozen. |
| +func TestMigrationCensusRefusesAMovedRoute(t *testing.T) { |
| + c := newLeoCensusTree(t) |
| + row := leoRoute(c.r.DataDir) |
| + row.Version = "" |
| + if err := c.r.Registry.Upsert(row); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := c.check(t, allWindowUnits()); !errors.Is(err, errRootMigrationBusy) { |
| + t.Fatalf("row with no version = %v", err) |
| + } |
| + c = newLeoCensusTree(t) |
| + other := leoRoute(c.r.DataDir) |
| + other.Host, other.DBPath = leoWakecheck, provision.NestedDBPath(c.r.DataDir, leoWakecheck) |
| + other.App = "wakecheck" |
| + if err := c.r.Registry.Upsert(other); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := c.check(t, allWindowUnits()); !errors.Is(err, errRootMigrationBusy) { |
| + t.Fatalf("an unreported row beside retained data = %v", err) |
| + } |
| +} |
| + |
| +// --window refuses anything but root's own private directory outside the |
| +// unit and data paths. (The accepting path needs root-owned files: the |
| +// root-runner witness runs it.) |
| +func TestMigrationWindowRefusals(t *testing.T) { |
| + c := newLeoCensusTree(t) |
| + if m, err := verifyMigrationWindow("", c.r); err != nil || len(m) != 0 { |
| + t.Fatalf("no window = %v, %v", m, err) |
| + } |
| + own := filepath.Join(t.TempDir(), "window") |
| + if err := os.MkdirAll(filepath.Join(own, "held"), 0o700); err != nil { |
| + t.Fatal(err) |
| + } |
| + for _, w := range []string{"relative/window", own + "/", "/etc/systemd/system/window", filepath.Join(c.r.UnitsDir, "w"), filepath.Join(c.r.DataDir, "w"), c.r.OptDir, own} { |
| + if _, err := verifyMigrationWindow(w, c.r); !errors.Is(err, errMigrationWindow) { |
| + t.Fatalf("window %q = %v", w, err) |
| + } |
| + } |
| +} |
| + |
| +// Ruling 6's other half: the claims-on per-host census, which root runs |
| +// before a claim's first inventory, still refuses claimtrial on leo once it |
| +// is adopted. Its legacy route row and files are not an empty installation, |
| +// and the window's masks are not unit files. Nothing on main installs an |
| +// adopted legacy host (populated backfill), so the files seed cannot admit |
| +// it: step 8 STOP S4. |
| +func TestPerHostCensusRefusesAdoptedLegacyHost(t *testing.T) { |
| + c := newLeoCensusTree(t) |
| + journal, err := hostbox.OpenLaunchJournal(filepath.Join(t.TempDir(), "journals.db")) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + t.Cleanup(func() { journal.Close() }) |
| + service, err := hostbox.NewJournalService(journal) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + census := hostbox.EmptyInstallationCensus{Inventory: service.Inventory, Registry: c.r.Registry, Launches: journal, Box: "leo-climbing", Enrollment: strings.Repeat("a", 32), DirectoryRevision: "9", |
| + Roots: c.roots, SharedPaths: claimGlobalSharedPaths(c.r)} |
| + var record hostops.Record |
| + for _, r := range leoRecords(t) { |
| + if r.Holder.Host == leoClaimtrial { |
| + record = r |
| + } |
| + } |
| + n := record.Holder |
| + claim := &hostclaim.Claim{Host: leoClaimtrial} |
| + if err := hostclaim.Adopt(n, "leo-climbing", nil, []string{"sophie-knitting"}, "op", func() string { return "id" })(claim); err != nil { |
| + t.Fatal(err) |
| + } |
| + obs := hostclaim.Observed{Host: leoClaimtrial, Claim: claim, ETag: "etag"} |
| + inc, _ := record.Incarnation() |
| + err = census.CheckLocal(t.Context(), obs, hostbox.RecordSelection{Host: leoClaimtrial, Key: record.Holder.Key, Incarnation: inc, Family: "instance"}) |
| + if !errors.Is(err, hostbox.ErrTypedInventory) { |
| + t.Fatalf("per-host census for the adopted legacy host = %v, want ErrTypedInventory", err) |
| + } |
| + // With the legacy row gone, it still refuses claimtrial's own files. |
| + if err := c.r.Registry.Delete(leoClaimtrial); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err = census.CheckLocal(t.Context(), obs, hostbox.RecordSelection{Host: leoClaimtrial, Key: record.Holder.Key, Incarnation: inc, Family: "instance"}); !errors.Is(err, hostbox.ErrTypedInventory) { |
| + t.Fatalf("per-host census over claimtrial's files = %v", err) |
| + } |
| +} |
| + |
| +func leoReplaceWithLink(t *testing.T, path, target string) { |
| + t.Helper() |
| + if err := os.Remove(path); err != nil { |
| + t.Fatal(err) |
| + } |
| + leoLink(t, target, path) |
| +} |
| + |
| +func leoLink(t *testing.T, target, path string) { |
| + t.Helper() |
| + if err := os.Symlink(target, path); err != nil { |
| + t.Fatal(err) |
| + } |
| +} |
| + |
| +func leoWrite(t *testing.T, path string) { |
| + t.Helper() |
| + if err := os.WriteFile(path, nil, 0o644); err != nil { |
| + t.Fatal(err) |
| + } |
| +} |
diff --git a/internal/host/claimconstruction.go b/internal/host/claimconstruction.go| index f824633b6..e9c6f08ee 100644 |
| --- a/internal/host/claimconstruction.go |
| +++ b/internal/host/claimconstruction.go |
| @@ -42,6 +42,10 @@ type ClaimRuntimeResources struct { |
| Registry *registry.Registry |
| Manager *cgroup.Manager |
| DataDir, OptDir, UnitsDir string |
| + // MigrationWindow is journal-migrate's --window: the attended offline |
| + // window's directory, whose held unit files let the migration census |
| + // accept that window's own masks. Empty everywhere else. |
| + MigrationWindow string |
| } |
| // ClaimOwnerBindings are the provider's typed owners. Serving feeds the |
diff --git a/internal/host/claimjournal_migration.go b/internal/host/claimjournal_migration.go| index edd9659db..7b73bfdf9 100644 |
| --- a/internal/host/claimjournal_migration.go |
| +++ b/internal/host/claimjournal_migration.go |
| @@ -33,6 +33,7 @@ func checkClaimJournalMigration(ctx context.Context, journals *hostbox.JournalSe |
| // import, so the edge's runtime mask is lifted on the way out. |
| func RunSystemJournalMigrate(args []string) { |
| fs := flag.NewFlagSet("system journal-migrate", flag.ExitOnError) |
| + window := fs.String("window", "", journalMigrateWindowFlag) |
| _ = fs.Parse(args) |
| if os.Geteuid() != 0 || fs.NArg() != 0 { |
| log.Fatal(hostwrite.ErrUnavailable) |
| @@ -55,12 +56,20 @@ func RunSystemJournalMigrate(args []string) { |
| if provider == nil { |
| provider = ProductionClaimOwnerProvider{} |
| } |
| - r := ClaimRuntimeResources{Mode: "instance-control", Registry: reg, DataDir: dataDir(), OptDir: release.OptDir(), UnitsDir: "/etc/systemd/system"} |
| + r := ClaimRuntimeResources{Mode: "instance-control", Registry: reg, DataDir: dataDir(), OptDir: release.OptDir(), UnitsDir: "/etc/systemd/system", MigrationWindow: *window} |
| + // The window is checked before anything is written; the census checks |
| + // its masks again when it walks the unit directory. |
| + if _, err = verifyJournalMigrateWindow(r); err != nil { |
| + log.Fatal(err) |
| + } |
| if err = migrateClaimJournalsWith(ctx, claimStore(ctx), r, provider, true); err != nil { |
| log.Fatal(err) |
| } |
| } |
| +// journalMigrateWindowFlag is --window's help. |
| +const journalMigrateWindowFlag = "the attended window's directory: its held directory keeps the original of each window unit the window masked, so the migration census accepts those masks" |
| + |
| func migrateClaimJournalsWith(ctx context.Context, store release.Store, r ClaimRuntimeResources, provider ClaimOwnerProvider, census bool) (resultErr error) { |
| if store == nil || provider == nil || r.Mode != "instance-control" || !filepath.IsAbs(r.DataDir) { |
| return hostwrite.ErrUnavailable |
diff --git a/internal/host/claimjournal_migration_other.go b/internal/host/claimjournal_migration_other.go| index 2847bfe24..fb18236e7 100644 |
| --- a/internal/host/claimjournal_migration_other.go |
| +++ b/internal/host/claimjournal_migration_other.go |
| @@ -10,3 +10,7 @@ import ( |
| func quiesceLegacyClaimJournals(context.Context) (func(), error) { return nil, hostbox.ErrUnsupported } |
| func recoverLegacyClaimEdgeMask(context.Context) error { return hostbox.ErrUnsupported } |
| + |
| +func verifyJournalMigrateWindow(ClaimRuntimeResources) (map[string]bool, error) { |
| + return nil, hostbox.ErrUnsupported |
| +} |
diff --git a/internal/host/claimstep8_ruling6_test.go b/internal/host/claimstep8_ruling6_test.go| new file mode 100644 |
| index 000000000..69812bb94 |
| --- /dev/null |
| +++ b/internal/host/claimstep8_ruling6_test.go |
| @@ -0,0 +1,88 @@ |
| +package host |
| + |
| +import ( |
| + "go/ast" |
| + "go/parser" |
| + "go/token" |
| + "os" |
| + "slices" |
| + "sort" |
| + "strings" |
| + "testing" |
| +) |
| + |
| +// Ruling 6 for step 8 readiness: nothing changes an effect on a claims-off |
| +// or no-census box. Every new or widened census entry point is reached only |
| +// from claim-mode code. journal-migrate refuses without CARLOS_HOST_CLAIM |
| +// (RunSystemJournalMigrate), the root migration runs only under it, the |
| +// claim assembly and root inventory startup exist only with claims on, and |
| +// the retry directory is prepared only with a census or claim units on. This |
| +// pins the callers, so a new claims-off caller fails here first. |
| +func TestStep8CensusChangesReachOnlyClaimMode(t *testing.T) { |
| + callers := map[string][]string{} |
| + fset := token.NewFileSet() |
| + entries, err := os.ReadDir(".") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + for _, e := range entries { |
| + name := e.Name() |
| + if !strings.HasSuffix(name, ".go") || strings.HasSuffix(name, "_test.go") { |
| + continue |
| + } |
| + f, err := parser.ParseFile(fset, name, nil, 0) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + for _, decl := range f.Decls { |
| + fn, ok := decl.(*ast.FuncDecl) |
| + if !ok || fn.Body == nil { |
| + continue |
| + } |
| + ast.Inspect(fn.Body, func(n ast.Node) bool { |
| + switch x := n.(type) { |
| + case *ast.CallExpr: |
| + if id, ok := x.Fun.(*ast.Ident); ok { |
| + callers[id.Name] = append(callers[id.Name], fn.Name.Name) |
| + } |
| + case *ast.CompositeLit: |
| + // The collector's registry listing: routes: r.Registry.List. |
| + if typ, ok := x.Type.(*ast.Ident); ok && typ.Name == "rootMigrationCollector" { |
| + for _, el := range x.Elts { |
| + if kv, ok := el.(*ast.KeyValueExpr); ok { |
| + if k, ok := kv.Key.(*ast.Ident); ok && k.Name == "routes" { |
| + callers["collector.routes"] = append(callers["collector.routes"], fn.Name.Name) |
| + } |
| + } |
| + } |
| + } |
| + } |
| + return true |
| + }) |
| + } |
| + } |
| + want := map[string][]string{ |
| + "claimMigrationSharedPaths": {"migrateRootClaimAuthority"}, |
| + "verifyMigrationWindow": {"migrateRootClaimAuthority", "verifyJournalMigrateWindow"}, |
| + "verifyJournalMigrateWindow": {"RunSystemJournalMigrate"}, |
| + "prepareRootFloorDirs": {"migrateRootClaimAuthority"}, |
| + "claimBoxDataCensus": {"claimDataCensus"}, |
| + "claimDataCensus": {"claimGlobalSharedPaths"}, |
| + "claimGlobalSharedPaths": {"OpenProductionClaimAssembly", "claimMigrationSharedPaths", "recoverRootInventoryStartup"}, |
| + "migrateRootClaimAuthority": {"migrateClaimJournalsWith"}, |
| + "prepareRootRetryUnitDirectory": {"installActivationControl", "readClaimUnitSettings"}, |
| + "collector.routes": {"migrateRootClaimAuthority"}, |
| + } |
| + for name, w := range want { |
| + got := slices.Compact(sortedCopy(callers[name])) |
| + if !slices.Equal(got, w) { |
| + t.Errorf("%s is called from %v, want %v", name, got, w) |
| + } |
| + } |
| +} |
| + |
| +func sortedCopy(s []string) []string { |
| + out := append([]string(nil), s...) |
| + sort.Strings(out) |
| + return out |
| +} |
diff --git a/internal/host/claimstep8_witness_rootrunner_linux_test.go b/internal/host/claimstep8_witness_rootrunner_linux_test.go| new file mode 100644 |
| index 000000000..22ef5f120 |
| --- /dev/null |
| +++ b/internal/host/claimstep8_witness_rootrunner_linux_test.go |
| @@ -0,0 +1,506 @@ |
| +//go:build linux && rootrunner |
| + |
| +package host |
| + |
| +import ( |
| + "bytes" |
| + "context" |
| + "crypto/ed25519" |
| + "encoding/base64" |
| + "encoding/json" |
| + "errors" |
| + "fmt" |
| + "os" |
| + "os/exec" |
| + "path/filepath" |
| + "slices" |
| + "strconv" |
| + "strings" |
| + "syscall" |
| + "testing" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/claimactivation" |
| + "github.com/carlosframework/platform/internal/domains" |
| + "github.com/carlosframework/platform/internal/genselect" |
| + "github.com/carlosframework/platform/internal/hostactors" |
| + "github.com/carlosframework/platform/internal/hostbox" |
| + "github.com/carlosframework/platform/internal/hostclaim" |
| + "github.com/carlosframework/platform/internal/hostfloor" |
| + "github.com/carlosframework/platform/internal/hostops" |
| + "github.com/carlosframework/platform/internal/hostwrite" |
| + "github.com/carlosframework/platform/internal/moves" |
| + "github.com/carlosframework/platform/internal/park" |
| + "github.com/carlosframework/platform/internal/registry" |
| + "github.com/carlosframework/platform/internal/release" |
| + "github.com/carlosframework/platform/internal/rootparticipation" |
| + "github.com/carlosframework/platform/internal/rootstate" |
| +) |
| + |
| +// Step 8 readiness witness (R4). Leo's real tree, rebuilt from its metadata |
| +// snapshot, goes through the window in order: |
| +// |
| +// 1. 8.1's masks, applied with the runbook's own commands to leo's unit set |
| +// installed in this runner's /etc/systemd/system, and checked by |
| +// systemd and by install-control's verifyOfflineRootMigration; |
| +// 2. backfill-collect (claims run, production runner); |
| +// 3. journal-migrate without --window: the census refuses the masks (S2 as |
| +// found), and leaves the migration resumable; |
| +// 4. journal-migrate with --window, under umask 077 (R3): census and |
| +// migration complete, the floor directories come out 0755, and leo's |
| +// signed census report names claimtrial as its route installation; |
| +// 5. backfill-adopt: on the dev roster it refuses, because sophie is a |
| +// roster box with no census key (STOP S3). With sophie's report standing |
| +// in, it publishes, claimtrial goes to leo, and the others stay unowned; |
| +// 6. the console's adoption pass (production ReconcileClaims) gives |
| +// claimtrial a serving claim on leo; the files seed selects it; |
| +// 7. root's first step for an adopted host with no installation, the |
| +// per-host empty census, refuses claimtrial's legacy row and files |
| +// (STOP S4): nothing on main installs an adopted legacy host. |
| +// |
| +// Scaffold writes (state from before the window, never claim state on the |
| +// window's path): the roster at revision 9 and the floor, signed by a |
| +// fixture release key (steps 5 to 7 made them); the three bucket records |
| +// and their committed domain claims (the bucket as it is); leo's identity |
| +// and census key (fixture owner bindings, as every accepted root migration |
| +// witness uses); the root floor history and guard binary (guard-arm, |
| +// step 6); claimtrial's registry row (leo-routes.tsv); and, in the second |
| +// test only, sophie's census report, which nothing on dev can produce. |
| +func TestRootStep8WindowWitnessDevRoster(t *testing.T) { rootStep8Witness(t, false) } |
| + |
| +func TestRootStep8WindowWitnessSophieReported(t *testing.T) { rootStep8Witness(t, true) } |
| + |
| +const ( |
| + step8Window = "/etc/carlos/claim-window-cohort1-20261021T0900Z" |
| + step8Bucket = "deployment" |
| + step8Leo = "leo-climbing" |
| + step8Sophie = "sophie-knitting" |
| + step8ID = "cohort1-20261021" |
| +) |
| + |
| +func rootStep8Witness(t *testing.T, sophieReports bool) { |
| + if os.Geteuid() != 0 { |
| + t.Fatal("designated runner must execute as root") |
| + } |
| + ctx := t.Context() |
| + uid, gid := generationRunnerRegistryUser(t) |
| + for _, path := range []string{"/var/lib/carlos-claims", "/var/lib/carlos-release", "/var/lib/carlos-swap-status", "/etc/carlos"} { |
| + if _, err := os.Lstat(path); !errors.Is(err, os.ErrNotExist) { |
| + t.Fatal("runner already holds", path, err) |
| + } |
| + t.Cleanup(func() { |
| + if err := os.RemoveAll(path); err != nil { |
| + t.Error(err) |
| + } |
| + }) |
| + } |
| + for _, path := range []string{"/run/carlos", "/run/carlos/hostguard"} { |
| + if err := os.Mkdir(path, 0755); err == nil { |
| + t.Cleanup(func() { os.RemoveAll(path) }) |
| + } else if !os.IsExist(err) { |
| + t.Fatal(err) |
| + } |
| + } |
| + |
| + // Leo's /data and /opt/carlos from the snapshot, owned as on leo. |
| + scratch := generationRootRunnerDir(t) |
| + data, opt := filepath.Join(scratch, "data"), filepath.Join(scratch, "opt") |
| + for _, dir := range []string{data, opt} { |
| + if err := os.Mkdir(dir, 0755); err != nil { |
| + t.Fatal(err) |
| + } |
| + } |
| + owner := func(path, who string) error { |
| + switch who { |
| + case "root:root": |
| + return os.Lchown(path, 0, 0) |
| + case "carlos:carlos": |
| + return os.Lchown(path, uid, gid) |
| + case "root:carlos": |
| + return os.Lchown(path, 0, gid) |
| + } |
| + return fmt.Errorf("unexpected owner %q at %s", who, path) |
| + } |
| + buildLeoTree(t, leoTreeRows(t, false), leoTreeOptions{ |
| + Roots: map[string]string{"/data": data, "/opt/carlos": opt}, |
| + Owner: owner, |
| + Content: func(root, path string) []byte { |
| + if root == "/data" && path == "swap-failures.json" { |
| + return []byte(`{"schema":1,"records":null}`) |
| + } |
| + return nil |
| + }, |
| + Skip: func(r leoRow) bool { |
| + // Rebuilt by the scaffold: the registry, root's guard and |
| + // update state, the platform binary and its fallback. |
| + if r.Root == "/data" && strings.HasPrefix(r.Path, "registry.db") { |
| + return true |
| + } |
| + return r.Root == "/opt/carlos" && (r.Path == "system" || strings.HasPrefix(r.Path, "system/") || r.Path == "carlos" || strings.HasPrefix(r.Path, "carlos.prev")) |
| + }, |
| + }) |
| + |
| + // Leo's unit set in this runner's /etc/systemd/system, then 8.1's masks. |
| + installLeoUnits(t) |
| + runStep8Shell(t, ` |
| +WINDOW='`+step8Window+`' |
| +TRIGGERS=(carlos-adopt.timer carlos-adopt.path carlos-stage.path carlos-restart.path carlos-system-update.path) |
| +SERVICES=(carlos-adopt.service carlos-stage.service carlos-config-sync.service carlos-restart.service carlos-system-update.service carlos-system-recover.service) |
| +UNITS=("${TRIGGERS[@]}" "${SERVICES[@]}" carlos-edge.service) |
| +mkdir -p -m 0755 /etc/carlos |
| +mkdir -m 0700 -- "$WINDOW" |
| +mkdir -m 0700 -- "$WINDOW/held" |
| +systemctl daemon-reload |
| +systemctl stop "${TRIGGERS[@]}" |
| +systemctl stop "${SERVICES[@]}" |
| +systemctl stop carlos-edge.service |
| +systemctl disable carlos-edge.service |
| +for unit in "${UNITS[@]}"; do |
| + mv -T -- "/etc/systemd/system/$unit" "$WINDOW/held/$unit" |
| + ln -s -- /dev/null "/etc/systemd/system/$unit" |
| +done |
| +systemctl daemon-reload |
| +systemctl show "${UNITS[@]}" -p Id -p LoadState -p ActiveState |
| +`) |
| + for _, unit := range claimWindowUnits { |
| + out, err := exec.Command("systemctl", "show", unit, "-p", "LoadState", "--value").CombinedOutput() |
| + if err != nil || strings.TrimSpace(string(out)) != "masked" { |
| + t.Fatalf("8.1: %s is %q (%v)", unit, out, err) |
| + } |
| + } |
| + if _, err := os.Lstat("/etc/systemd/system/multi-user.target.wants/carlos-edge.service"); !errors.Is(err, os.ErrNotExist) { |
| + t.Fatal("8.1: the edge is still wanted", err) |
| + } |
| + if err := verifyOfflineRootMigration(ctx); err != nil { |
| + t.Fatal("8.1: install-control's offline check", err) |
| + } |
| + |
| + // The bucket and roster as steps 5 to 7 left them (scaffold). |
| + store := release.NewDirStore(filepath.Join(scratch, "bucket")) |
| + key := ed25519.NewKeyFromSeed(bytes.Repeat([]byte{1}, ed25519.SeedSize)) |
| + signer := release.RootSigner(key) |
| + directory := hostactors.Directory{Store: store, PublicKey: key.Public().(ed25519.PublicKey), Bucket: step8Bucket} |
| + leoKey := ed25519.NewKeyFromSeed(bytes.Repeat([]byte{2}, ed25519.SeedSize)) |
| + sophieKey := ed25519.NewKeyFromSeed(bytes.Repeat([]byte{3}, ed25519.SeedSize)) |
| + enrollment := strings.Repeat("a", 32) |
| + identity := handoverRecoveryIdentity(t, hostbox.Actor{Box: step8Leo, Enrollment: enrollment, Revision: 9, Region: "ie", ControlBucket: step8Bucket}) |
| + actor, err := identity.Verify(ctx) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := hostfloor.Publish(ctx, store, signer, step8Bucket, hostfloor.Floor{Min: 1, MinRelease: "v2.0.0"}); err != nil { |
| + t.Fatal(err) |
| + } |
| + floor, err := hostfloor.Read(ctx, store, directory.PublicKey, step8Bucket) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + roster := hostactors.Roster{V: 1, DeploymentBucket: step8Bucket, Revision: 9, RequestLimitNS: int64(hostbox.RequestLimit), HomeRegion: "ie", Publication: "published", FloorSHA256: floor.Digest, |
| + Regions: map[string]hostactors.Region{"ie": {Account: actor.Cloud.Account, AWSRegion: "eu-west-1", Namespaces: []string{"platform"}, Pools: []string{"cohort1", "console"}}}, |
| + Boxes: map[string]hostactors.Box{}, |
| + Consoles: map[string]hostactors.Console{}, Holds: map[string]hostactors.Reservation{}, Activations: map[string]hostactors.Reservation{}} |
| + roster.Boxes[step8Leo] = hostactors.Box{BoxID: step8Leo, Enrollment: enrollment, Namespace: "platform", Label: step8Leo, Region: "ie", Pool: "cohort1", State: "active", LossIDs: []string{}, |
| + Cloud: hostactors.Cloud{Provider: "aws", Account: actor.Cloud.Account, Region: actor.Cloud.Region, InstanceID: actor.Cloud.InstanceID}, EvidenceSHA256: strings.Repeat("c", 64), CensusPublicKey: leoKey.Public().(ed25519.PublicKey)} |
| + sophie := hostactors.Box{BoxID: step8Sophie, Enrollment: strings.Repeat("b", 32), Namespace: "platform", Label: step8Sophie, Region: "ie", Pool: "console", State: "active", LossIDs: []string{}, |
| + Cloud: hostactors.Cloud{Provider: "aws", Account: actor.Cloud.Account, Region: actor.Cloud.Region, InstanceID: "i-0def"}, EvidenceSHA256: strings.Repeat("d", 64)} |
| + if sophieReports { |
| + sophie.CensusPublicKey = sophieKey.Public().(ed25519.PublicKey) |
| + } |
| + roster.Boxes[step8Sophie] = sophie |
| + body, err := release.SignRootEnvelope(signer, roster) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if _, err := store.Put(ctx, hostactors.Key, body, ""); err != nil { |
| + t.Fatal(err) |
| + } |
| + records := map[string]hostops.Record{} |
| + for _, r := range leoRecords(t) { |
| + if _, err := store.Put(ctx, r.Holder.Key, r.Body, ""); err != nil { |
| + t.Fatal(err) |
| + } |
| + if r, err = hostops.ReadRecord(ctx, store, r.Holder.Key); err != nil { |
| + t.Fatal(err) |
| + } |
| + records[r.Holder.Host] = r |
| + inc, _ := r.Incarnation() |
| + raw, _ := json.Marshal(domains.Claim{Schema: 1, Host: r.Holder.Host, Account: r.Holder.Account, App: r.Holder.App, Acquisition: r.Holder.Acquisition, RecordIncarnation: inc, OwnershipState: "committed"}) |
| + if _, err := store.Put(ctx, domains.ClaimKey(r.Holder.Host), raw, ""); err != nil { |
| + t.Fatal(err) |
| + } |
| + } |
| + // Leo's registry: its one route (leo-routes.tsv), owned as on leo. |
| + reg, err := registry.Open(filepath.Join(data, "registry.db")) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + t.Cleanup(func() { reg.Close() }) |
| + if err := reg.Upsert(leoRoute(data)); err != nil { |
| + t.Fatal(err) |
| + } |
| + for _, suffix := range []string{"", "-wal", "-shm"} { |
| + if err := os.Lchown(filepath.Join(data, "registry.db"+suffix), uid, gid); err != nil && !os.IsNotExist(err) { |
| + t.Fatal(err) |
| + } |
| + } |
| + // Root's floor history and guard binary, as guard-arm left them. |
| + t.Setenv("CARLOS_OPT_DIR", opt) |
| + t.Setenv("CARLOS_DATA_DIR", data) |
| + t.Setenv("CARLOS_DEPLOYMENT_BUCKET", step8Bucket) |
| + t.Setenv("CARLOS_RELEASE_PUBKEY", base64.StdEncoding.EncodeToString(directory.PublicKey)) |
| + t.Setenv("CARLOS_HOST_CLAIM", "1") |
| + t.Setenv("CARLOS_HOST_CLAIM_ORIGINS", "1") |
| + t.Setenv("CARLOS_POOLS", "cohort1") |
| + t.Setenv("TMPDIR", scratch) |
| + floorID := rootFloorIdentity{Deployment: step8Bucket, Box: actor.Box, Enrollment: actor.Enrollment} |
| + floors, err := openRootMovesFloors(opt, floorID, true) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := floors.reconstruct(ctx, rootFloorReconstruction{Identity: floorID, Floor: &moves.Floor{Min: 1, MinRelease: "v2.0.0"}, Source: "enrolled root history", Digest: floorTestDigest, Authorization: floorTestAuthorization(t, floors)}); err != nil { |
| + t.Fatal(err) |
| + } |
| + floors.close() |
| + rootFloorRunnerBinary(t, opt, systemBinaryPath(opt), "v2.0.0", key) |
| + rootFloorRunnerBinary(t, opt, filepath.Join(systemGuardDir(opt), "carlos"), "v2.0.0", key) |
| + if _, err := os.Lstat(rootSeedsFloorDir(opt)); !errors.Is(err, os.ErrNotExist) { |
| + t.Fatal("the seeds floor directory exists before journal-migrate", err) |
| + } |
| + unused := fix4UnusedDependencies{} |
| + owners := fix4Owners{ClaimOwnerBindings{ |
| + Identity: identity, RetirementAuthority: &hostops.ProductionAuthority{Store: store, Directory: directory, Now: time.Now}, EvidenceKey: leoKey, |
| + Local: unused, Controllers: unused, Data: unused, Inputs: unused, Evidence: unused, |
| + InputPolicy: func(context.Context, *hostclaim.Claim, hostbox.Selection, hostwrite.Identity) error { return nil }, |
| + MovesPolicy: func(context.Context, *hostclaim.Claim, hostbox.Selection, hostwrite.Identity) error { return nil }, |
| + LeaseSet: func(context.Context, string, string) ([]string, error) { return nil, nil }, |
| + View: func(context.Context, hostclaim.Observed) (park.View, error) { return park.View{}, nil }, |
| + Authorize: func(context.Context, hostclaim.Observed, hostclaim.Obligation) error { return nil }, |
| + AdmissionEvidence: func(context.Context, hostclaim.Observed, hostclaim.Admission, *genselect.Plan) error { return nil }, |
| + PublicationEvidence: func(context.Context, hostclaim.Observed) error { return nil }, |
| + Close: func(context.Context) error { return nil }, |
| + }} |
| + |
| + // 2. backfill-collect, dry run then apply. |
| + runner := claimactivation.Runner{Directory: directory, Signer: signer} |
| + collect := claimactivation.Plan{Phase: "backfill-collect", Bucket: step8Bucket, ExpectedRevision: 9, Backfill: &claimactivation.BackfillInput{ID: step8ID}} |
| + if res, err := runner.Run(ctx, collect, true); err != nil || len(res.Backfill.Hosts) != 3 { |
| + t.Fatalf("backfill-collect dry run: %+v %v", res.Backfill, err) |
| + } |
| + res, err := runner.Run(ctx, collect, false) |
| + if err != nil { |
| + t.Fatal("backfill-collect", err) |
| + } |
| + t.Logf("backfill-collect: %d hosts frozen, phase %s, revision %d", len(res.Backfill.Hosts), res.Backfill.Phase, res.Backfill.DirectoryRevision) |
| + |
| + r := ClaimRuntimeResources{Mode: "instance-control", Registry: reg, DataDir: data, OptDir: opt, UnitsDir: "/etc/systemd/system"} |
| + // 3. journal-migrate as S2 found it: no --window, the masks refuse. |
| + err = migrateClaimJournalsWith(ctx, store, r, owners, true) |
| + if !errors.Is(err, hostbox.ErrTypedInventory) || !strings.Contains(err.Error(), "units entry") { |
| + t.Fatalf("journal-migrate without --window = %v, want the census to refuse a mask", err) |
| + } |
| + t.Logf("journal-migrate without --window refused: %v", err) |
| + // 4. journal-migrate --window, as the window's root shell runs it, but |
| + // under umask 077 (R3). |
| + r.MigrationWindow = step8Window |
| + masked, err := verifyJournalMigrateWindow(r) |
| + if err != nil || len(masked) != len(claimWindowUnits) { |
| + t.Fatalf("--window: %v %v", masked, err) |
| + } |
| + old := syscall.Umask(0o077) |
| + err = migrateClaimJournalsWith(ctx, store, r, owners, true) |
| + syscall.Umask(old) |
| + if err != nil { |
| + t.Fatal("journal-migrate --window", err) |
| + } |
| + state, err := rootstate.Open(rootMigrationDir, false, false) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + raw, err := state.Read("migration.json") |
| + state.Close() |
| + var saved rootMigrationRecord |
| + if err != nil || rootparticipation.Decode(raw, &saved) != nil || saved.Phase != "complete" { |
| + t.Fatalf("migration.json: %s %v", raw, err) |
| + } |
| + for path, want := range map[string]os.FileMode{"/var/lib/carlos-claims": 0o700, rootFloorDir(opt): 0o755, rootSeedsFloorDir(opt): 0o755, systemGuardDir(opt): 0o755} { |
| + st, err := os.Stat(path) |
| + if err != nil || st.Mode().Perm() != want || st.Sys().(*syscall.Stat_t).Uid != 0 { |
| + t.Fatalf("mode check after journal-migrate: %s %v %v", path, st.Mode(), err) |
| + } |
| + } |
| + reportRaw, _, err := store.Get(ctx, "control/host-census/"+step8ID+"/"+step8Leo+".json") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + var envelope hostops.CensusEnvelope |
| + if err := json.Unmarshal(reportRaw, &envelope); err != nil { |
| + t.Fatal(err) |
| + } |
| + inc, _ := records[leoClaimtrial].Incarnation() |
| + if len(envelope.Report.Installations) != 1 || envelope.Report.Installations[0] != (hostops.CensusInstallation{Host: leoClaimtrial, Account: "bab", App: "claimtrial", Family: "instance", Key: records[leoClaimtrial].Holder.Key, Incarnation: inc, Route: true}) { |
| + t.Fatalf("leo's census report: %+v", envelope.Report.Installations) |
| + } |
| + t.Logf("journal-migrate --window: complete; leo's report names %s as a route installation", leoClaimtrial) |
| + |
| + // 5. backfill-adopt. |
| + adopt := claimactivation.Plan{Phase: "backfill-adopt", Bucket: step8Bucket, ExpectedRevision: 9, Backfill: &claimactivation.BackfillInput{ID: step8ID}} |
| + if !sophieReports { |
| + _, err := runner.Run(ctx, adopt, false) |
| + if !errors.Is(err, os.ErrNotExist) { |
| + t.Fatalf("backfill-adopt on the dev roster = %v, want sophie's missing census report refused", err) |
| + } |
| + var plan hostops.BackfillPlan |
| + if raw, _, err := store.Get(ctx, hostops.BackfillKey); err != nil || release.VerifyRootEnvelope(directory.PublicKey, raw, &plan) != nil || plan.Phase != "collecting" { |
| + t.Fatalf("the plan after a refused adopt: %+v %v", plan, err) |
| + } |
| + t.Logf("STOP S3: backfill-adopt refused on the dev roster: %v", err) |
| + return |
| + } |
| + sophieReport := hostops.CensusReport{Box: step8Sophie, Enrollment: sophie.Enrollment, BootID: "sophie-boot", Reservation: step8ID, DirectoryRevision: 9, CapturedAt: time.Now().UTC(), Complete: true, |
| + Installations: []hostops.CensusInstallation{}, Seeds: &hostops.CensusSeeds{Journaled: []string{}, Gated: []string{}}} |
| + if err := hostops.PublishCensusReport(ctx, store, directory, sophieReport, sophieKey); err != nil { |
| + t.Fatal(err) |
| + } |
| + if res, err = runner.Run(ctx, adopt, true); err != nil { |
| + t.Fatal("backfill-adopt dry run", err) |
| + } |
| + if res, err = runner.Run(ctx, adopt, false); err != nil { |
| + t.Fatal("backfill-adopt", err) |
| + } |
| + for _, h := range res.Backfill.Hosts { |
| + want := "" |
| + if h.Host == leoClaimtrial { |
| + want = step8Leo |
| + } |
| + if h.Box != want { |
| + t.Fatalf("adopting plan: %s on %q", h.Host, h.Box) |
| + } |
| + } |
| + if !slices.Equal(res.Backfill.Unowned, []string{leoRollcheck, leoWakecheck}) { |
| + t.Fatalf("unowned: %v", res.Backfill.Unowned) |
| + } |
| + t.Logf("backfill-adopt: adopting; %s on %s; unowned %v", leoClaimtrial, step8Leo, res.Backfill.Unowned) |
| + |
| + // 6. The console's adoption pass, as production builds it. |
| + serial := 0 |
| + controller := hostops.NewBackfillController(store, directory, time.Now, func() string { serial++; return "witness-op-" + strconv.Itoa(serial) }) |
| + err = controller.ReconcileClaims(ctx) |
| + if !errors.Is(err, hostops.ErrIdentity) { |
| + t.Fatalf("adoption pass = %v, want the unowned hosts refused", err) |
| + } |
| + observed, err := hostclaim.Read(ctx, store, leoClaimtrial) |
| + if err != nil || observed.Absent || observed.Claim.Op.Kind != hostclaim.KindServing || observed.Claim.Op.Exec != step8Leo || observed.Claim.Holder.WriterBox != step8Leo { |
| + t.Fatalf("claimtrial's claim: %+v %v", observed.Claim, err) |
| + } |
| + candidates, err := filesSeedCandidates(ctx, store, step8Leo, "") |
| + if err != nil || len(candidates) != 1 || candidates[0].Host != leoClaimtrial { |
| + t.Fatalf("files seed candidates: %+v %v", candidates, err) |
| + } |
| + t.Logf("adoption: %s serving on %s (stamp %d, obligations %d); the files seed selects it", leoClaimtrial, step8Leo, observed.Claim.Holder.Stamp, len(observed.Claim.Obligations)) |
| + |
| + // 7. Root's first step for a claim with no installation on this box |
| + // (rootexecution_authority's ErrNoRecord path): the per-host empty |
| + // census, built as root's startup builds it, over the migrated box. |
| + launches, err := openClaimRootJournal() |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + defer launches.Close() |
| + journals, err := hostbox.NewJournalService(launches) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + roots := map[string]string{"data": r.DataDir, "opt": r.OptDir, "units": r.UnitsDir} |
| + census := &hostbox.EmptyInstallationCensus{Inventory: journals.Inventory, Registry: r.Registry, Launches: launches, Box: actor.Box, Enrollment: actor.Enrollment, DirectoryRevision: "9", Roots: roots, SharedPaths: claimGlobalSharedPaths(r)} |
| + h := ClaimRootHTTP{Journals: journals, Launches: launches, Roots: roots, Reconcile: hostbox.ReconcileAdapter{Registry: r.Registry, Inventory: journals.Inventory, Census: census, Box: actor.Box, Enrollment: actor.Enrollment}} |
| + err = h.rootEmptyCensus().CheckLocal(ctx, observed, hostbox.RecordSelection{Host: leoClaimtrial, Key: records[leoClaimtrial].Holder.Key, ETag: records[leoClaimtrial].ETag, Incarnation: inc, Family: "instance"}) |
| + if !errors.Is(err, hostbox.ErrTypedInventory) { |
| + t.Fatalf("root's per-host census for the adopted legacy host = %v, want a refusal", err) |
| + } |
| + t.Logf("STOP S4: root refuses the adopted legacy host before any seed admission: %v", err) |
| +} |
| + |
| +// installLeoUnits writes leo's carlos units, drop-ins and enablement links |
| +// into this runner's /etc/systemd/system: unit files with the embedded |
| +// bytes leo's were installed from, drop-ins with a minimal body, and the |
| +// links exactly as the snapshot names them. The runner's own vendor links |
| +// stand in for leo's. Everything written here is removed afterwards. |
| +func installLeoUnits(t *testing.T) { |
| + t.Helper() |
| + const units = "/etc/systemd/system" |
| + var made []string |
| + t.Cleanup(func() { |
| + for i := len(made) - 1; i >= 0; i-- { |
| + if err := os.Remove(made[i]); err != nil && !os.IsNotExist(err) { |
| + t.Error(err) |
| + } |
| + } |
| + for _, unit := range claimWindowUnits { |
| + // The window's masks and held files. |
| + if target, err := os.Readlink(filepath.Join(units, unit)); err == nil && target == "/dev/null" { |
| + os.Remove(filepath.Join(units, unit)) |
| + } |
| + } |
| + exec.Command("systemctl", "daemon-reload").Run() |
| + }) |
| + carlos := func(name string) bool { |
| + base := filepath.Base(name) |
| + return strings.HasPrefix(base, "carlos") || base == "data-instances.mount" || strings.HasPrefix(base, "run-carlos-") |
| + } |
| + for _, r := range leoTreeRows(t, false) { |
| + if r.Root != units || r.Path == "." { |
| + continue |
| + } |
| + path := filepath.Join(units, r.Path) |
| + switch r.Type { |
| + case "d": |
| + if !carlos(r.Path) { |
| + if err := os.Mkdir(path, 0755); err == nil { |
| + made = append(made, path) |
| + } else if !os.IsExist(err) { |
| + t.Fatal(err) |
| + } |
| + continue |
| + } |
| + if err := os.Mkdir(path, r.Mode); err != nil { |
| + t.Fatal("runner already holds", path, err) |
| + } |
| + made = append(made, path) |
| + case "f": |
| + if !carlos(r.Path) { |
| + continue |
| + } |
| + body, err := unitsFS.ReadFile("units/" + filepath.Base(r.Path)) |
| + if err != nil { |
| + body = []byte("[Service]\n") |
| + } |
| + if _, err := os.Lstat(path); !errors.Is(err, os.ErrNotExist) { |
| + t.Fatal("runner already holds", path, err) |
| + } |
| + if err := os.WriteFile(path, body, r.Mode); err != nil { |
| + t.Fatal(err) |
| + } |
| + made = append(made, path) |
| + case "l": |
| + if !carlos(r.Path) || !strings.HasPrefix(r.Target, units+"/") { |
| + continue |
| + } |
| + if err := os.Symlink(r.Target, path); err != nil { |
| + t.Fatal("runner already holds", path, err) |
| + } |
| + made = append(made, path) |
| + } |
| + } |
| + // Directories are removed after what is in them. |
| + slices.SortStableFunc(made, func(a, b string) int { return strings.Count(a, "/") - strings.Count(b, "/") }) |
| +} |
| + |
| +func runStep8Shell(t *testing.T, script string) { |
| + t.Helper() |
| + cmd := exec.Command("bash", "-c", "set -eu\numask 077\n"+script) |
| + out, err := cmd.CombinedOutput() |
| + if err != nil { |
| + t.Fatalf("runbook block: %v\n%s", err, out) |
| + } |
| + t.Logf("runbook block:\n%s", out) |
| +} |
diff --git a/internal/host/leo_snapshot_linux_test.go b/internal/host/leo_snapshot_linux_test.go| new file mode 100644 |
| index 000000000..880f25fc4 |
| --- /dev/null |
| +++ b/internal/host/leo_snapshot_linux_test.go |
| @@ -0,0 +1,184 @@ |
| +//go:build linux |
| + |
| +package host |
| + |
| +import ( |
| + _ "embed" |
| + "os" |
| + "path/filepath" |
| + "strconv" |
| + "strings" |
| + "testing" |
| +) |
| + |
| +// leo-climbing's real tree as metadata only (dev cohort-1, step 8 readiness): |
| +// every path under /data, /opt/carlos, /etc/systemd/system and /etc/carlos |
| +// on 2026-10-19 at 21:07Z, read-only over SSM with find -xdev -printf |
| +// '<root>\t%y\t%u:%g\t%m\t%s\t%n\t%P\t%l\n'. No file contents. The /etc/carlos |
| +// names that could be secrets are redacted. leo-units-post-8.1.tsv is what |
| +// the window's step 8.1 changes; leo-routes.tsv is leo's one registry route. |
| +var ( |
| + //go:embed testdata/leo-step8/leo-tree-snapshot.tsv |
| + leoTreeSnapshot string |
| + //go:embed testdata/leo-step8/leo-units-post-8.1.tsv |
| + leoUnitsPostWindow string |
| + //go:embed testdata/leo-step8/leo-routes.tsv |
| + leoRoutes string |
| +) |
| + |
| +type leoRow struct { |
| + Root, Type, Owner string |
| + Mode os.FileMode |
| + Size int64 |
| + Path, Target string |
| + State string // post-8.1 rows only |
| +} |
| + |
| +func parseLeoRows(t *testing.T, raw string) []leoRow { |
| + t.Helper() |
| + var rows []leoRow |
| + for i, line := range strings.Split(strings.TrimRight(raw, "\n"), "\n") { |
| + if i == 0 { |
| + continue |
| + } |
| + f := strings.Split(line, "\t") |
| + if len(f) < 8 { |
| + t.Fatalf("snapshot line %d: %q", i+1, line) |
| + } |
| + mode, err := strconv.ParseUint(f[3], 8, 32) |
| + if err != nil { |
| + t.Fatalf("snapshot line %d mode: %v", i+1, err) |
| + } |
| + size, err := strconv.ParseInt(f[4], 10, 64) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + row := leoRow{Root: f[0], Type: f[1], Owner: f[2], Mode: unixMode(uint32(mode)), Size: size, Path: f[6], Target: f[7]} |
| + if len(f) > 8 { |
| + row.State = f[8] |
| + } |
| + rows = append(rows, row) |
| + } |
| + return rows |
| +} |
| + |
| +func unixMode(m uint32) os.FileMode { |
| + mode := os.FileMode(m & 0o777) |
| + if m&0o4000 != 0 { |
| + mode |= os.ModeSetuid |
| + } |
| + if m&0o2000 != 0 { |
| + mode |= os.ModeSetgid |
| + } |
| + if m&0o1000 != 0 { |
| + mode |= os.ModeSticky |
| + } |
| + return mode |
| +} |
| + |
| +// leoTreeRows is the snapshot with step 8.1 applied: the twelve window units |
| +// replaced by /dev/null links and the edge's wants link removed. The held |
| +// originals (under /etc/carlos) are left out: the window directory is not a |
| +// census root, and a witness makes its own. |
| +func leoTreeRows(t *testing.T, window bool) []leoRow { |
| + t.Helper() |
| + rows := parseLeoRows(t, leoTreeSnapshot) |
| + if !window { |
| + return rows |
| + } |
| + post := map[string]leoRow{} |
| + for _, r := range parseLeoRows(t, leoUnitsPostWindow) { |
| + if r.Root == "/etc/systemd/system" { |
| + post[r.Path] = r |
| + } |
| + } |
| + var out []leoRow |
| + for _, r := range rows { |
| + if p, ok := post[r.Path]; ok && r.Root == "/etc/systemd/system" { |
| + if p.State == "expected-post-8.1-absent" { |
| + continue |
| + } |
| + out = append(out, p) |
| + continue |
| + } |
| + out = append(out, r) |
| + } |
| + return out |
| +} |
| + |
| +// leoTreeOptions maps snapshot roots to where a test builds them, and |
| +// rewrites absolute link targets that point into a rebuilt root. |
| +type leoTreeOptions struct { |
| + Roots map[string]string // snapshot root -> directory |
| + // Content gives the synthetic bytes for a file a reader needs; nil |
| + // writes an empty file. |
| + Content func(root, path string) []byte |
| + // Owner, when set, applies a row's owner (root runner only). |
| + Owner func(path, owner string) error |
| + // Skip leaves a row out (and, for a directory, nothing below it is |
| + // expected to be skipped by the caller). |
| + Skip func(r leoRow) bool |
| +} |
| + |
| +func buildLeoTree(t *testing.T, rows []leoRow, o leoTreeOptions) { |
| + t.Helper() |
| + type dirMode struct { |
| + path string |
| + mode os.FileMode |
| + } |
| + var dirs []dirMode |
| + for _, r := range rows { |
| + base, ok := o.Roots[r.Root] |
| + if !ok || (o.Skip != nil && o.Skip(r)) { |
| + continue |
| + } |
| + path := base |
| + if r.Path != "." { |
| + path = filepath.Join(base, r.Path) |
| + } |
| + switch r.Type { |
| + case "d": |
| + if r.Path != "." { |
| + if err := os.Mkdir(path, 0o700); err != nil && !os.IsExist(err) { |
| + t.Fatal(err) |
| + } |
| + } |
| + dirs = append(dirs, dirMode{path, r.Mode}) |
| + case "f": |
| + var body []byte |
| + if o.Content != nil { |
| + body = o.Content(r.Root, r.Path) |
| + } |
| + if err := os.WriteFile(path, body, 0o600); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := os.Chmod(path, r.Mode); err != nil { |
| + t.Fatal(err) |
| + } |
| + case "l": |
| + target := r.Target |
| + for root, dir := range o.Roots { |
| + if target == root || strings.HasPrefix(target, root+"/") { |
| + target = dir + strings.TrimPrefix(target, root) |
| + } |
| + } |
| + if err := os.Symlink(target, path); err != nil { |
| + t.Fatal(err) |
| + } |
| + default: |
| + t.Fatalf("snapshot type %q at %s", r.Type, r.Path) |
| + } |
| + if o.Owner != nil { |
| + if err := o.Owner(path, r.Owner); err != nil { |
| + t.Fatal(err) |
| + } |
| + } |
| + } |
| + // Directory modes last, deepest first, so a 0700 or 0750 parent does |
| + // not stop the build of what is below it. |
| + for i := len(dirs) - 1; i >= 0; i-- { |
| + if err := os.Chmod(dirs[i].path, dirs[i].mode); err != nil { |
| + t.Fatal(err) |
| + } |
| + } |
| +} |
diff --git a/internal/host/rootmigration_census.go b/internal/host/rootmigration_census.go| index 135e98823..70f7f6aa4 100644 |
| --- a/internal/host/rootmigration_census.go |
| +++ b/internal/host/rootmigration_census.go |
| @@ -3,6 +3,7 @@ package host |
| import ( |
| "context" |
| "encoding/json" |
| + "path/filepath" |
| "sort" |
| "strings" |
| "time" |
| @@ -13,6 +14,7 @@ import ( |
| "github.com/carlosframework/platform/internal/hostguard" |
| "github.com/carlosframework/platform/internal/hostops" |
| "github.com/carlosframework/platform/internal/provision" |
| + "github.com/carlosframework/platform/internal/registry" |
| "github.com/carlosframework/platform/internal/release" |
| ) |
| @@ -27,6 +29,24 @@ type rootMigrationCollector struct { |
| launches func(context.Context) ([]hostbox.LaunchInventory, error) |
| reservations func(context.Context) ([]release.GenerationReservation, error) |
| boot func() (string, error) |
| + // routes is the box's registry listing and dataDir its data root. A |
| + // legacy route row, an ordinary instance route no claim owns yet, is |
| + // reported as the box's route installation of its frozen record, so the |
| + // backfill can adopt it to this box. Nil reports none (the census then |
| + // refuses the row's files, as before). |
| + routes func() ([]registry.Route, error) |
| + dataDir string |
| +} |
| + |
| +// legacyRouteRow is the one route shape the migration census reports as a |
| +// legacy installation: an ordinary nested-layout instance row, unowned by |
| +// any claim, of the frozen record's account and app. These are the routes |
| +// the files seed can serve; every other row refuses the census. |
| +func legacyRouteRow(row registry.Route, r hostops.Record, dataDir string) bool { |
| + return row.Kind == "instance" && r.Holder.Family == "instance" && row.ClaimOwnership == (registry.ClaimOwnership{}) && |
| + !row.Replica && row.Seed == "" && row.SeedSource == "" && row.Backing == "" && row.Unit == "" && !row.IsSidecar() && row.Tunnel == "" && |
| + row.Account == r.Holder.Account && row.App == r.Holder.App && filepath.IsAbs(dataDir) && row.DBPath == provision.NestedDBPath(dataDir, row.Host) && |
| + release.ValidVersion(row.Version) |
| } |
| func (c rootMigrationCollector) collect(ctx context.Context) (rootMigrationSnapshot, error) { |
| @@ -213,6 +233,34 @@ func (c rootMigrationCollector) collect(ctx context.Context) (rootMigrationSnaps |
| if len(local) != 0 { |
| return s, errRootMigrationBusy |
| } |
| + if c.routes != nil { |
| + rows, err := c.routes() |
| + if err != nil { |
| + return s, err |
| + } |
| + reported := map[string]int{} |
| + for i, in := range s.Report.Installations { |
| + reported[in.Host] = i |
| + } |
| + for _, row := range rows { |
| + if _, ok := reported[row.Host]; ok { |
| + // A claimed installation of this box: the claim's own |
| + // inventory accounts for its row. |
| + continue |
| + } |
| + r, ok := records[row.Host] |
| + if !ok || !legacyRouteRow(row, r, c.dataDir) { |
| + return s, errRootMigrationBusy |
| + } |
| + inc, err := r.Incarnation() |
| + if err != nil { |
| + return s, errRootMigrationBusy |
| + } |
| + reported[row.Host] = len(s.Report.Installations) |
| + s.Report.Installations = append(s.Report.Installations, hostops.CensusInstallation{Host: row.Host, Account: r.Holder.Account, App: r.Holder.App, Family: r.Holder.Family, Key: r.Holder.Key, Incarnation: inc, Route: true}) |
| + } |
| + sort.Slice(s.Report.Installations, func(i, j int) bool { return s.Report.Installations[i].Host < s.Report.Installations[j].Host }) |
| + } |
| fresh, ftag, err := c.store.Get(ctx, hostops.BackfillKey) |
| if err != nil || ftag != tag || string(fresh) != string(raw) { |
| return s, errRootMigrationBusy |
diff --git a/internal/host/rootmigration_floordirs_linux_test.go b/internal/host/rootmigration_floordirs_linux_test.go| new file mode 100644 |
| index 000000000..97122b7c3 |
| --- /dev/null |
| +++ b/internal/host/rootmigration_floordirs_linux_test.go |
| @@ -0,0 +1,48 @@ |
| +//go:build linux |
| + |
| +package host |
| + |
| +import ( |
| + "os" |
| + "path/filepath" |
| + "syscall" |
| + "testing" |
| +) |
| + |
| +// journal-migrate runs from the window's umask 077 root shell. Its ready |
| +// step's public floor directories must still come out 0755, and a pair an |
| +// earlier run left 0700 is repaired rather than refused forever. |
| +func TestRootFloorDirsIgnoreUmask(t *testing.T) { |
| + old := syscall.Umask(0o077) |
| + defer syscall.Umask(old) |
| + for _, earlier := range []bool{false, true} { |
| + opt := t.TempDir() |
| + guard := filepath.Join(opt, "system", "guard") |
| + if err := os.MkdirAll(guard, 0o755); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := os.Chmod(filepath.Join(opt, "system"), 0o755); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := os.Chmod(guard, 0o755); err != nil { |
| + t.Fatal(err) |
| + } |
| + if earlier { |
| + if err := os.Mkdir(rootFloorDir(opt), 0o700); err != nil { |
| + t.Fatal(err) |
| + } |
| + } |
| + if err := prepareRootFloorDirs(opt); err != nil { |
| + t.Fatal(err) |
| + } |
| + for _, dir := range []string{rootFloorDir(opt), rootSeedsFloorDir(opt), guard} { |
| + st, err := os.Stat(dir) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if st.Mode().Perm() != 0o755 { |
| + t.Fatalf("earlier=%v: %s is %o, want 755", earlier, dir, st.Mode().Perm()) |
| + } |
| + } |
| + } |
| +} |
diff --git a/internal/host/rootmigration_production_linux.go b/internal/host/rootmigration_production_linux.go| index 3dc7e87d4..61d4d4736 100644 |
| --- a/internal/host/rootmigration_production_linux.go |
| +++ b/internal/host/rootmigration_production_linux.go |
| @@ -55,7 +55,7 @@ func migrateRootClaimAuthority(ctx context.Context, store release.Store, r Claim |
| collector := rootMigrationCollector{retirements: productionGenerationRetirements, suspects: func(ctx context.Context) ([]string, error) { return rootMigrationLegacySuspects(ctx, r.DataDir) }, store: store, directory: owners.RetirementAuthority.Directory, identity: id, seeds: r.Registry, installations: root.MigrationInstallations, launches: launches.GenerationLaunches, boot: func() (string, error) { |
| raw, err := os.ReadFile("/proc/sys/kernel/random/boot_id") |
| return strings.TrimSpace(string(raw)), err |
| - }, reservations: func(ctx context.Context) ([]release.GenerationReservation, error) { |
| + }, routes: r.Registry.List, dataDir: r.DataDir, reservations: func(ctx context.Context) ([]release.GenerationReservation, error) { |
| state, err := rootstate.Open(release.RootGenerationsDir, false, false) |
| if errors.Is(err, os.ErrNotExist) { |
| return nil, nil |
| @@ -153,7 +153,15 @@ func migrateRootClaimAuthority(ctx context.Context, store release.Store, r Claim |
| } |
| all = append(all, in) |
| } |
| - if err := hostbox.CheckMigrationRoots(ctx, roots, claimGlobalSharedPaths(r), all); err != nil { |
| + masked, err := verifyMigrationWindow(r.MigrationWindow, r) |
| + if err != nil { |
| + return err |
| + } |
| + shared, err := claimMigrationSharedPaths(r, masked, s) |
| + if err != nil { |
| + return err |
| + } |
| + if err := hostbox.CheckMigrationRoots(ctx, roots, shared, all); err != nil { |
| return err |
| } |
| if err := migrationExecutionEmpty(execution); err != nil { |
| @@ -168,6 +176,9 @@ func migrateRootClaimAuthority(ctx context.Context, store release.Store, r Claim |
| } |
| generations.Close() |
| floorID := rootFloorIdentity{Deployment: id.Deployment, Box: id.Box, Enrollment: id.Enrollment} |
| + if err := prepareRootFloorDirs(r.OptDir); err != nil { |
| + return err |
| + } |
| for _, dir := range []string{rootFloorDir(r.OptDir), rootSeedsFloorDir(r.OptDir)} { |
| f, err := openRootProtocolFloors(dir, floorID, true) |
| if err != nil { |
| @@ -186,6 +197,22 @@ func migrateRootClaimAuthority(ctx context.Context, store release.Store, r Claim |
| }) |
| } |
| +// prepareRootFloorDirs makes the two public floor directories the |
| +// migration's ready step opens, whatever the operator shell's umask (#317's |
| +// class): rootstate creates a public directory with Mkdirat(0755) and never |
| +// chmods, so under umask 077 it would come out 0700, rootstate would refuse |
| +// it, and every rerun would refuse it again. The carlos agent reads the |
| +// floor projections there, so both must be 0755. mkdirAllExact also repairs |
| +// a directory an earlier run left 0700. |
| +func prepareRootFloorDirs(opt string) error { |
| + for _, dir := range []string{rootFloorDir(opt), rootSeedsFloorDir(opt)} { |
| + if err := mkdirAllExact(dir, 0755); err != nil { |
| + return err |
| + } |
| + } |
| + return nil |
| +} |
| + |
| func migrationExecutionEmpty(state rootstate.Store) error { |
| names, err := state.Names() |
| if err != nil { |
diff --git a/internal/host/rootmigration_routes_linux_test.go b/internal/host/rootmigration_routes_linux_test.go| new file mode 100644 |
| index 000000000..40b211d13 |
| --- /dev/null |
| +++ b/internal/host/rootmigration_routes_linux_test.go |
| @@ -0,0 +1,107 @@ |
| +//go:build linux |
| + |
| +package host |
| + |
| +import ( |
| + "bytes" |
| + "context" |
| + "crypto/ed25519" |
| + "errors" |
| + "path/filepath" |
| + "testing" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/hostbox" |
| + "github.com/carlosframework/platform/internal/hostguard" |
| + "github.com/carlosframework/platform/internal/hostops" |
| + "github.com/carlosframework/platform/internal/registry" |
| + "github.com/carlosframework/platform/internal/release" |
| +) |
| + |
| +// S1: the migration census reports a legacy route row as the box's route |
| +// installation of its frozen record, so backfill-adopt can give the host |
| +// to this box. Every other row shape refuses the census. |
| +func TestRootMigrationCensusReportsLegacyRoutes(t *testing.T) { |
| + store, _, owners, _ := fix4JournalFixture(t) |
| + actor, err := owners.Identity.Verify(t.Context()) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + key := ed25519.NewKeyFromSeed(bytes.Repeat([]byte{1}, ed25519.SeedSize)) |
| + plan := hostops.BackfillPlan{V: 1, ID: "migration", Phase: "collecting", DirectoryRevision: 1, CreatedAt: time.Now(), Hosts: map[string]hostops.BackfillHost{}} |
| + records := map[string]hostops.Record{} |
| + for _, r := range leoRecords(t) { |
| + if _, err := store.Put(t.Context(), r.Holder.Key, r.Body, ""); err != nil { |
| + t.Fatal(err) |
| + } |
| + r, err = hostops.ReadRecord(t.Context(), store, r.Holder.Key) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + records[r.Holder.Host] = r |
| + plan.Hosts[r.Holder.Host] = hostops.BackfillHost{Key: r.Holder.Key, ETag: r.ETag, Acquisition: r.Holder.Acquisition} |
| + } |
| + raw, err := release.SignRootEnvelope(release.RootSigner(key), plan) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if _, err := store.Put(t.Context(), hostops.BackfillKey, raw, ""); err != nil { |
| + t.Fatal(err) |
| + } |
| + dataDir := t.TempDir() |
| + var rows []registry.Route |
| + c := rootMigrationCollector{store: store, directory: owners.RetirementAuthority.Directory, identity: rootMigrationIdentity{Deployment: actor.ControlBucket, Box: actor.Box, Enrollment: actor.Enrollment, Generation: 1}, |
| + seeds: &migrationSeeds{}, installations: func(context.Context) ([]hostguard.FenceView, error) { return nil, nil }, launches: func(context.Context) ([]hostbox.LaunchInventory, error) { return nil, nil }, |
| + reservations: func(context.Context) ([]release.GenerationReservation, error) { return nil, nil }, boot: func() (string, error) { return "boot", nil }, |
| + routes: func() ([]registry.Route, error) { return rows, nil }, dataDir: dataDir} |
| + rows = []registry.Route{leoRoute(dataDir)} |
| + s, err := c.collect(t.Context()) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := s.preflight(c.identity); err != nil { |
| + t.Fatal(err) |
| + } |
| + inc, _ := records[leoClaimtrial].Incarnation() |
| + want := hostops.CensusInstallation{Host: leoClaimtrial, Account: "bab", App: "claimtrial", Family: "instance", Key: records[leoClaimtrial].Holder.Key, Incarnation: inc, Route: true} |
| + if len(s.Report.Installations) != 1 || s.Report.Installations[0] != want || len(s.Items) != 0 { |
| + t.Fatalf("report = %+v, items %+v", s.Report.Installations, s.Items) |
| + } |
| + if err := hostops.PublishCensusReport(t.Context(), store, c.directory, s.Report, owners.EvidenceKey); err != nil { |
| + t.Fatal(err) |
| + } |
| + // No registry listing reports nothing, as before. |
| + without := c |
| + without.routes = nil |
| + if s, err := without.collect(t.Context()); err != nil || len(s.Report.Installations) != 0 { |
| + t.Fatalf("no listing = %+v, %v", s.Report.Installations, err) |
| + } |
| + for name, mutate := range map[string]func(*registry.Route){ |
| + "unit": func(r *registry.Route) { r.Unit = "app.service" }, |
| + "backing": func(r *registry.Route) { r.Backing = "unit" }, |
| + "replica": func(r *registry.Route) { r.Replica = true }, |
| + "seed": func(r *registry.Route) { r.Seed = "seed" }, |
| + "tunnel": func(r *registry.Route) { r.Tunnel = "t" }, |
| + "sidecar": func(r *registry.Route) { r.Kind = "sidecar" }, |
| + "service": func(r *registry.Route) { r.Kind = "service" }, |
| + "flat layout": func(r *registry.Route) { r.DBPath = filepath.Join(dataDir, r.Host+".db") }, |
| + "claimed row": func(r *registry.Route) { |
| + r.ClaimIncarnation, r.ClaimStamp, r.ClaimOwner, r.ClaimRowID = "inc", 1, "owner", "row" |
| + }, |
| + "another app": func(r *registry.Route) { r.App = "wakecheck" }, |
| + "another account": func(r *registry.Route) { r.Account = "zzz" }, |
| + "no version": func(r *registry.Route) { r.Version = "" }, |
| + "no record": func(r *registry.Route) { |
| + r.Host, r.DBPath = "other.example.com", filepath.Join(dataDir, "instances", "other.example.com", "other.example.com.db") |
| + }, |
| + } { |
| + t.Run(name, func(t *testing.T) { |
| + row := leoRoute(dataDir) |
| + mutate(&row) |
| + rows = []registry.Route{row} |
| + if _, err := c.collect(t.Context()); !errors.Is(err, errRootMigrationBusy) { |
| + t.Fatalf("collect = %v, want the census refused", err) |
| + } |
| + }) |
| + } |
| +} |
diff --git a/internal/host/rootupdates_runtime.go b/internal/host/rootupdates_runtime.go| index 98c530817..903ed3d11 100644 |
| --- a/internal/host/rootupdates_runtime.go |
| +++ b/internal/host/rootupdates_runtime.go |
| @@ -283,6 +283,11 @@ func prepareRootRetryUnitDirectory() (bool, error) { |
| if !census.Complete || census.NeverEnrolled { |
| return false, os.ErrPermission |
| } |
| + // install-control runs from the window's root shell: make the agent's |
| + // read directory 0755 whatever that shell's umask (#317's class). |
| + if err := mkdirAllExact(rootRetryProjection, 0755); err != nil { |
| + return false, err |
| + } |
| state, err := rootstate.Open(rootRetryProjection, true, true) |
| if err != nil { |
| return false, err |
diff --git a/internal/host/testdata/leo-step8/leo-routes.tsv b/internal/host/testdata/leo-step8/leo-routes.tsv| new file mode 100644 |
| index 000000000..444328262 |
| --- /dev/null |
| +++ b/internal/host/testdata/leo-step8/leo-routes.tsv |
| @@ -0,0 +1,2 @@ |
| +host app kind backing unit hibernate layout |
| +claimtrial.dev.carlosframework.com claimtrial instance - - 1 nested |
diff --git a/internal/host/testdata/leo-step8/leo-tree-snapshot.tsv b/internal/host/testdata/leo-step8/leo-tree-snapshot.tsv| new file mode 100644 |
| index 000000000..5bcd2eddc |
| --- /dev/null |
| +++ b/internal/host/testdata/leo-step8/leo-tree-snapshot.tsv |
| @@ -0,0 +1,251 @@ |
| +root type owner mode size nlink path link_target |
| +/data d carlos:carlos 750 4096 16 . |
| +/data d carlos:carlos 2770 4096 2 hostguard |
| +/data f carlos:carlos 644 21 1 stage-requested |
| +/data d carlos:carlos 755 4096 3 carlos |
| +/data d carlos:carlos 755 4096 2 carlos/geoip |
| +/data f carlos:carlos 644 2 1 carlos/geoip/current |
| +/data f root:root 644 0 1 restart-marks.json.lock |
| +/data d carlos:carlos 755 4096 2 zdd |
| +/data f carlos:carlos 644 67 1 zdd/generations.json |
| +/data f carlos:carlos 644 4 1 zdd/generations.seq |
| +/data f root:root 644 198 1 restart-marks.json |
| +/data d root:root 755 4096 2 programs |
| +/data f root:root 644 37 1 programs/wakecheck.dev.carlosframework.com.json |
| +/data f root:root 644 37 1 programs/claimtrial.dev.carlosframework.com.json |
| +/data d carlos:carlos 750 4096 2 relay |
| +/data d carlos:carlos 750 4096 2 acme |
| +/data f carlos:carlos 600 5084 1 acme/platform.dev.carlosframework.com |
| +/data f carlos:carlos 600 7375 1 acme/tunnel.dev.carlosframework.com+rsa |
| +/data f carlos:carlos 600 5064 1 acme/proof2.dev.oncarlos.com |
| +/data f carlos:carlos 600 5064 1 acme/website.dev.oncarlos.com |
| +/data f carlos:carlos 600 5060 1 acme/proof.dev.oncarlos.com |
| +/data f carlos:carlos 600 7379 1 acme/platform.dev.carlosframework.com+rsa |
| +/data f carlos:carlos 600 5080 1 acme/tunnel.dev.carlosframework.com |
| +/data f carlos:carlos 600 7346 1 acme/proof2.dev.oncarlos.com+rsa |
| +/data f carlos:carlos 600 7379 1 acme/wakecheck.dev.carlosframework.com+rsa |
| +/data f carlos:carlos 600 5076 1 acme/helloworld.dev.oncarlos.com |
| +/data f carlos:carlos 600 5100 1 acme/appsdomain-check.bab.dev.oncarlos.com |
| +/data f carlos:carlos 600 5100 1 acme/rollcheck-0261.dev.carlosframework.com |
| +/data f carlos:carlos 600 227 1 acme/acme_account+key |
| +/data f carlos:carlos 600 5092 1 acme/claimtrial.dev.carlosframework.com |
| +/data f carlos:carlos 600 5088 1 acme/wakecheck.dev.carlosframework.com |
| +/data f carlos:carlos 600 5076 1 acme/proof.dev.carlosframework.com |
| +/data f carlos:carlos 600 7367 1 acme/proof.dev.carlosframework.com+rsa |
| +/data f carlos:carlos 644 4120032 1 registry.db-wal |
| +/data d carlos:carlos 755 4096 3 system |
| +/data d carlos:carlos 755 4096 2 system/router |
| +/data f carlos:carlos 644 132 1 system/router/requested.json |
| +/data d carlos:carlos 2770 4096 2 .parked |
| +/data d carlos:carlos 700 4096 5 instances |
| +/data d carlos:carlos 700 4096 2 instances/_probe |
| +/data d carlos:carlos 700 4096 3 instances/claimtrial.dev.carlosframework.com |
| +/data f carlos:carlos 644 24752 1 instances/claimtrial.dev.carlosframework.com/claimtrial.dev.carlosframework.com.db-wal |
| +/data f carlos:carlos 600 149 1 instances/claimtrial.dev.carlosframework.com/claimtrial.dev.carlosframework.com.env |
| +/data f carlos:carlos 644 24576 1 instances/claimtrial.dev.carlosframework.com/claimtrial.dev.carlosframework.com.db |
| +/data d carlos:carlos 700 4096 3 instances/claimtrial.dev.carlosframework.com/.activator-claimtrial.dev.carlosframework.com.db-litestream |
| +/data d carlos:carlos 700 4096 3 instances/claimtrial.dev.carlosframework.com/.activator-claimtrial.dev.carlosframework.com.db-litestream/ltx |
| +/data d carlos:carlos 700 4096 2 instances/claimtrial.dev.carlosframework.com/.activator-claimtrial.dev.carlosframework.com.db-litestream/ltx/0 |
| +/data f carlos:carlos 644 206 1 instances/claimtrial.dev.carlosframework.com/.activator-claimtrial.dev.carlosframework.com.db-litestream/ltx/0/0000000000000076-0000000000000076.ltx |
| +/data f carlos:carlos 644 805 1 instances/claimtrial.dev.carlosframework.com/.activator-claimtrial.dev.carlosframework.com.db-litestream/ltx/0/0000000000000075-0000000000000075.ltx |
| +/data f carlos:carlos 644 32768 1 instances/claimtrial.dev.carlosframework.com/claimtrial.dev.carlosframework.com.db-shm |
| +/data d carlos:carlos 700 4096 3 instances/wakecheck.dev.carlosframework.com |
| +/data f carlos:carlos 600 1136 1 instances/wakecheck.dev.carlosframework.com/launch.run.json |
| +/data f carlos:carlos 600 149 1 instances/wakecheck.dev.carlosframework.com/wakecheck.dev.carlosframework.com.env |
| +/data f carlos:carlos 644 32768 1 instances/wakecheck.dev.carlosframework.com/wakecheck.dev.carlosframework.com.db-shm |
| +/data f carlos:carlos 644 16384 1 instances/wakecheck.dev.carlosframework.com/wakecheck.dev.carlosframework.com.db |
| +/data f carlos:carlos 644 12392 1 instances/wakecheck.dev.carlosframework.com/wakecheck.dev.carlosframework.com.db-wal |
| +/data d carlos:carlos 700 4096 3 instances/wakecheck.dev.carlosframework.com/.activator-wakecheck.dev.carlosframework.com.db-litestream |
| +/data d carlos:carlos 700 4096 3 instances/wakecheck.dev.carlosframework.com/.activator-wakecheck.dev.carlosframework.com.db-litestream/ltx |
| +/data d carlos:carlos 700 4096 2 instances/wakecheck.dev.carlosframework.com/.activator-wakecheck.dev.carlosframework.com.db-litestream/ltx/0 |
| +/data f carlos:carlos 644 2945 1 instances/wakecheck.dev.carlosframework.com/.activator-wakecheck.dev.carlosframework.com.db-litestream/ltx/0/0000000000000164-0000000000000164.ltx |
| +/data f carlos:carlos 644 206 1 instances/wakecheck.dev.carlosframework.com/.activator-wakecheck.dev.carlosframework.com.db-litestream/ltx/0/0000000000000163-0000000000000163.ltx |
| +/data f carlos:carlos 644 206 1 instances/wakecheck.dev.carlosframework.com/.activator-wakecheck.dev.carlosframework.com.db-litestream/ltx/0/0000000000000165-0000000000000165.ltx |
| +/data f carlos:carlos 644 498 1 channel-first-seen.json |
| +/data f carlos:carlos 644 102400 1 registry.db |
| +/data d carlos:carlos 700 4096 5 .instances |
| +/data d carlos:carlos 700 4096 2 .instances/_probe |
| +/data d carlos:carlos 700 4096 3 .instances/claimtrial.dev.carlosframework.com |
| +/data f carlos:carlos 644 24752 1 .instances/claimtrial.dev.carlosframework.com/claimtrial.dev.carlosframework.com.db-wal |
| +/data f carlos:carlos 600 149 1 .instances/claimtrial.dev.carlosframework.com/claimtrial.dev.carlosframework.com.env |
| +/data f carlos:carlos 644 24576 1 .instances/claimtrial.dev.carlosframework.com/claimtrial.dev.carlosframework.com.db |
| +/data d carlos:carlos 700 4096 3 .instances/claimtrial.dev.carlosframework.com/.activator-claimtrial.dev.carlosframework.com.db-litestream |
| +/data d carlos:carlos 700 4096 3 .instances/claimtrial.dev.carlosframework.com/.activator-claimtrial.dev.carlosframework.com.db-litestream/ltx |
| +/data d carlos:carlos 700 4096 2 .instances/claimtrial.dev.carlosframework.com/.activator-claimtrial.dev.carlosframework.com.db-litestream/ltx/0 |
| +/data f carlos:carlos 644 206 1 .instances/claimtrial.dev.carlosframework.com/.activator-claimtrial.dev.carlosframework.com.db-litestream/ltx/0/0000000000000076-0000000000000076.ltx |
| +/data f carlos:carlos 644 805 1 .instances/claimtrial.dev.carlosframework.com/.activator-claimtrial.dev.carlosframework.com.db-litestream/ltx/0/0000000000000075-0000000000000075.ltx |
| +/data f carlos:carlos 644 32768 1 .instances/claimtrial.dev.carlosframework.com/claimtrial.dev.carlosframework.com.db-shm |
| +/data d carlos:carlos 700 4096 3 .instances/wakecheck.dev.carlosframework.com |
| +/data f carlos:carlos 600 1136 1 .instances/wakecheck.dev.carlosframework.com/launch.run.json |
| +/data f carlos:carlos 600 149 1 .instances/wakecheck.dev.carlosframework.com/wakecheck.dev.carlosframework.com.env |
| +/data f carlos:carlos 644 32768 1 .instances/wakecheck.dev.carlosframework.com/wakecheck.dev.carlosframework.com.db-shm |
| +/data f carlos:carlos 644 16384 1 .instances/wakecheck.dev.carlosframework.com/wakecheck.dev.carlosframework.com.db |
| +/data f carlos:carlos 644 12392 1 .instances/wakecheck.dev.carlosframework.com/wakecheck.dev.carlosframework.com.db-wal |
| +/data d carlos:carlos 700 4096 3 .instances/wakecheck.dev.carlosframework.com/.activator-wakecheck.dev.carlosframework.com.db-litestream |
| +/data d carlos:carlos 700 4096 3 .instances/wakecheck.dev.carlosframework.com/.activator-wakecheck.dev.carlosframework.com.db-litestream/ltx |
| +/data d carlos:carlos 700 4096 2 .instances/wakecheck.dev.carlosframework.com/.activator-wakecheck.dev.carlosframework.com.db-litestream/ltx/0 |
| +/data f carlos:carlos 644 2945 1 .instances/wakecheck.dev.carlosframework.com/.activator-wakecheck.dev.carlosframework.com.db-litestream/ltx/0/0000000000000164-0000000000000164.ltx |
| +/data f carlos:carlos 644 206 1 .instances/wakecheck.dev.carlosframework.com/.activator-wakecheck.dev.carlosframework.com.db-litestream/ltx/0/0000000000000163-0000000000000163.ltx |
| +/data f carlos:carlos 644 206 1 .instances/wakecheck.dev.carlosframework.com/.activator-wakecheck.dev.carlosframework.com.db-litestream/ltx/0/0000000000000165-0000000000000165.ltx |
| +/data f carlos:carlos 644 31 1 system-update-requested |
| +/data f carlos:carlos 644 32768 1 registry.db-shm |
| +/data d carlos:carlos 755 4096 2 monitor |
| +/data f carlos:carlos 644 102 1 monitor/alerts.json |
| +/data f root:root 644 0 1 adopt-pass.lock |
| +/data d carlos:carlos 2770 4096 2 .hostexec |
| +/data d carlos:carlos 755 4096 2 .moves |
| +/data f root:root 644 0 1 .moves/floor.lock |
| +/data f carlos:carlos 644 34 1 .moves/floor.json |
| +/data f carlos:carlos 644 8 1 .moves/floor.min-release |
| +/data f carlos:carlos 644 108 1 swap-failures.json |
| +/data d carlos:carlos 700 4096 2 layout |
| +/opt/carlos d root:root 755 4096 5 . |
| +/opt/carlos f root:root 755 49217698 1 carlos |
| +/opt/carlos f root:root 644 8 1 carlos.prev.allowed |
| +/opt/carlos d root:root 755 4096 3 versions |
| +/opt/carlos d root:root 755 4096 2 versions/wc4 |
| +/opt/carlos f root:root 755 9437346 1 versions/wc4/claimtrial |
| +/opt/carlos f root:root 755 9437346 1 versions/wc4/wakecheck |
| +/opt/carlos f root:root 644 419 1 versions/wc4/wakecheck.manifest.json |
| +/opt/carlos f root:root 644 424 1 versions/wc4/claimtrial.manifest.json |
| +/opt/carlos d root:root 755 4096 4 system |
| +/opt/carlos d root:root 755 4096 3 system/guard |
| +/opt/carlos f root:root 755 49217698 1 system/guard/carlos |
| +/opt/carlos f root:root 644 507 1 system/guard/policy.json |
| +/opt/carlos d root:root 755 4096 2 system/guard/identities |
| +/opt/carlos f root:root 644 1076 1 system/guard/identities/v0.28.4.json |
| +/opt/carlos d root:root 755 4096 2 system/router |
| +/opt/carlos f root:root 644 156 1 system/router/result.json |
| +/opt/carlos f root:root 644 92 1 system/router/applied.json |
| +/opt/carlos f root:root 644 39 1 system/router/seen.json |
| +/opt/carlos f root:root 600 0 1 system/router/lock |
| +/opt/carlos f root:root 644 1076 1 system/router/channel-identity.json |
| +/opt/carlos f root:root 600 0 1 system/router/prev.lock |
| +/opt/carlos f root:root 644 8 1 carlos.prev.version |
| +/opt/carlos f root:root 755 49217698 1 carlos.prev |
| +/opt/carlos d root:root 755 4096 2 live |
| +/opt/carlos l root:root 777 35 1 live/claimtrial.dev.carlosframework.com /opt/carlos/versions/wc4/claimtrial |
| +/opt/carlos l root:root 777 34 1 live/wakecheck.dev.carlosframework.com /opt/carlos/versions/wc4/wakecheck |
| +/etc/systemd/system d root:root 755 4096 21 . |
| +/etc/systemd/system f root:root 644 873 1 run-carlos-logs.mount |
| +/etc/systemd/system d root:root 755 4096 2 sshd.service.wants |
| +/etc/systemd/system l root:root 777 43 1 sshd.service.wants/sshd-keygen.service /usr/lib/systemd/system/sshd-keygen.service |
| +/etc/systemd/system d root:root 755 4096 2 network-online.target.wants |
| +/etc/systemd/system l root:root 777 60 1 network-online.target.wants/systemd-networkd-wait-online.service /usr/lib/systemd/system/systemd-networkd-wait-online.service |
| +/etc/systemd/system f root:root 644 2701 1 carlos-restore-verify.service |
| +/etc/systemd/system d root:root 755 4096 2 sysinit.target.wants |
| +/etc/systemd/system l root:root 777 49 1 sysinit.target.wants/systemd-timesyncd.service /usr/lib/systemd/system/systemd-timesyncd.service |
| +/etc/systemd/system l root:root 777 46 1 sysinit.target.wants/systemd-pstore.service /usr/lib/systemd/system/systemd-pstore.service |
| +/etc/systemd/system l root:root 777 48 1 sysinit.target.wants/systemd-resolved.service /usr/lib/systemd/system/systemd-resolved.service |
| +/etc/systemd/system l root:root 777 57 1 sysinit.target.wants/systemd-network-generator.service /usr/lib/systemd/system/systemd-network-generator.service |
| +/etc/systemd/system l root:root 777 40 1 sysinit.target.wants/apparmor.service /usr/lib/systemd/system/apparmor.service |
| +/etc/systemd/system f root:root 644 1719 1 data-instances.mount |
| +/etc/systemd/system d root:root 755 4096 2 getty.target.wants |
| +/etc/systemd/system l root:root 777 38 1 getty.target.wants/getty@tty1.service /usr/lib/systemd/system/getty@.service |
| +/etc/systemd/system f root:root 644 3682 1 carlos-system-recover.service |
| +/etc/systemd/system d root:root 755 4096 2 hybrid-sleep.target.wants |
| +/etc/systemd/system l root:root 777 43 1 hybrid-sleep.target.wants/grub-common.service /usr/lib/systemd/system/grub-common.service |
| +/etc/systemd/system d root:root 755 4096 2 suspend.target.wants |
| +/etc/systemd/system l root:root 777 43 1 suspend.target.wants/grub-common.service /usr/lib/systemd/system/grub-common.service |
| +/etc/systemd/system d root:root 755 4096 2 sockets.target.wants |
| +/etc/systemd/system l root:root 777 36 1 sockets.target.wants/uuidd.socket /usr/lib/systemd/system/uuidd.socket |
| +/etc/systemd/system l root:root 777 47 1 sockets.target.wants/systemd-networkd.socket /usr/lib/systemd/system/systemd-networkd.socket |
| +/etc/systemd/system l root:root 777 50 1 sockets.target.wants/carlos-instance-control.socket /etc/systemd/system/carlos-instance-control.socket |
| +/etc/systemd/system f root:root 644 9561 1 carlos-instance@.service |
| +/etc/systemd/system f root:root 644 7826 1 carlos-instance-gen@.service |
| +/etc/systemd/system l root:root 777 48 1 dbus-org.freedesktop.network1.service /usr/lib/systemd/system/systemd-networkd.service |
| +/etc/systemd/system l root:root 777 35 1 sshd.service /usr/lib/systemd/system/ssh.service |
| +/etc/systemd/system d root:root 755 4096 2 carlos-system-update.service.d |
| +/etc/systemd/system f root:root 644 87 1 carlos-system-update.service.d/90-host-floor.conf |
| +/etc/systemd/system f root:root 644 626 1 carlos-system-join.path |
| +/etc/systemd/system f root:root 644 1924 1 carlos-app@.socket |
| +/etc/systemd/system d root:root 755 4096 2 timers.target.wants |
| +/etc/systemd/system l root:root 777 32 1 timers.target.wants/fstrim.timer /lib/systemd/system/fstrim.timer |
| +/etc/systemd/system l root:root 777 40 1 timers.target.wants/dpkg-db-backup.timer /lib/systemd/system/dpkg-db-backup.timer |
| +/etc/systemd/system l root:root 777 43 1 timers.target.wants/apt-daily-upgrade.timer /lib/systemd/system/apt-daily-upgrade.timer |
| +/etc/systemd/system l root:root 777 41 1 timers.target.wants/e2scrub_all.timer /usr/lib/systemd/system/e2scrub_all.timer |
| +/etc/systemd/system l root:root 777 36 1 timers.target.wants/man-db.timer /usr/lib/systemd/system/man-db.timer |
| +/etc/systemd/system l root:root 777 38 1 timers.target.wants/carlos-adopt.timer /etc/systemd/system/carlos-adopt.timer |
| +/etc/systemd/system l root:root 777 35 1 timers.target.wants/apt-daily.timer /lib/systemd/system/apt-daily.timer |
| +/etc/systemd/system f root:root 644 1930 1 carlos-config-sync.service |
| +/etc/systemd/system f root:root 644 616 1 carlos-instance-control.socket |
| +/etc/systemd/system f root:root 644 9020 1 carlos-instance-cmd@.service |
| +/etc/systemd/system f root:root 644 2849 1 carlos-hibernate-verify.service |
| +/etc/systemd/system f root:root 644 2310 1 carlos-stage.service |
| +/etc/systemd/system l root:root 777 49 1 dbus-org.freedesktop.timesync1.service /usr/lib/systemd/system/systemd-timesyncd.service |
| +/etc/systemd/system f root:root 644 1294 1 carlos-relay.service |
| +/etc/systemd/system f root:root 644 1295 1 carlos-stage.path |
| +/etc/systemd/system f root:root 644 1436 1 carlos-system-join.service |
| +/etc/systemd/system f root:root 644 2275 1 carlos-instance-control.service |
| +/etc/systemd/system f root:root 644 9611 1 carlos-instance-seed@.service |
| +/etc/systemd/system f root:root 644 475 1 carlos-hibernate-verify.timer |
| +/etc/systemd/system f root:root 644 3229 1 carlos-app@.service |
| +/etc/systemd/system d root:root 755 4096 2 hibernate.target.wants |
| +/etc/systemd/system l root:root 777 43 1 hibernate.target.wants/grub-common.service /usr/lib/systemd/system/grub-common.service |
| +/etc/systemd/system f root:root 644 881 1 carlos.slice |
| +/etc/systemd/system d root:root 755 4096 2 sshd@.service.wants |
| +/etc/systemd/system l root:root 777 43 1 sshd@.service.wants/sshd-keygen.service /usr/lib/systemd/system/sshd-keygen.service |
| +/etc/systemd/system f root:root 644 3973 1 carlos-system-update.service |
| +/etc/systemd/system d root:root 755 4096 2 carlos-system-recover.service.d |
| +/etc/systemd/system f root:root 644 89 1 carlos-system-recover.service.d/90-host-floor.conf |
| +/etc/systemd/system f root:root 644 482 1 carlos-instances.slice |
| +/etc/systemd/system d root:root 755 4096 2 cloud-config.target.wants |
| +/etc/systemd/system l root:root 777 50 1 cloud-config.target.wants/cloud-init-hotplugd.socket /usr/lib/systemd/system/cloud-init-hotplugd.socket |
| +/etc/systemd/system f root:root 644 1364 1 carlos-adopt.service |
| +/etc/systemd/system f root:root 644 7134 1 carlos-instance-probe.service |
| +/etc/systemd/system f root:root 644 2233 1 carlos-restart.service |
| +/etc/systemd/system f root:root 644 1339 1 carlos-restart.path |
| +/etc/systemd/system f root:root 644 9583 1 carlos-instance-sidecar@.service |
| +/etc/systemd/system d root:root 755 4096 2 cloud-init.target.wants |
| +/etc/systemd/system l root:root 777 47 1 cloud-init.target.wants/cloud-init-main.service /usr/lib/systemd/system/cloud-init-main.service |
| +/etc/systemd/system l root:root 777 50 1 cloud-init.target.wants/cloud-init-network.service /usr/lib/systemd/system/cloud-init-network.service |
| +/etc/systemd/system l root:root 777 44 1 cloud-init.target.wants/cloud-config.service /usr/lib/systemd/system/cloud-config.service |
| +/etc/systemd/system l root:root 777 48 1 cloud-init.target.wants/cloud-init-local.service /usr/lib/systemd/system/cloud-init-local.service |
| +/etc/systemd/system l root:root 777 43 1 cloud-init.target.wants/cloud-final.service /usr/lib/systemd/system/cloud-final.service |
| +/etc/systemd/system f root:root 644 5218 1 carlos-edge.service |
| +/etc/systemd/system f root:root 644 1323 1 carlos-litestream.service |
| +/etc/systemd/system l root:root 777 48 1 dbus-org.freedesktop.resolve1.service /usr/lib/systemd/system/systemd-resolved.service |
| +/etc/systemd/system f root:root 644 419 1 carlos-adopt.timer |
| +/etc/systemd/system d root:root 755 4096 2 suspend-then-hibernate.target.wants |
| +/etc/systemd/system l root:root 777 43 1 suspend-then-hibernate.target.wants/grub-common.service /usr/lib/systemd/system/grub-common.service |
| +/etc/systemd/system d root:root 755 4096 2 multi-user.target.wants |
| +/etc/systemd/system l root:root 777 41 1 multi-user.target.wants/run-carlos-logs.mount /etc/systemd/system/run-carlos-logs.mount |
| +/etc/systemd/system l root:root 777 40 1 multi-user.target.wants/data-instances.mount /etc/systemd/system/data-instances.mount |
| +/etc/systemd/system l root:root 777 49 1 multi-user.target.wants/carlos-system-recover.service /etc/systemd/system/carlos-system-recover.service |
| +/etc/systemd/system l root:root 777 44 1 multi-user.target.wants/e2scrub_reap.service /usr/lib/systemd/system/e2scrub_reap.service |
| +/etc/systemd/system l root:root 777 40 1 multi-user.target.wants/remote-fs.target /usr/lib/systemd/system/remote-fs.target |
| +/etc/systemd/system l root:root 777 43 1 multi-user.target.wants/grub-common.service /usr/lib/systemd/system/grub-common.service |
| +/etc/systemd/system l root:root 777 43 1 multi-user.target.wants/carlos-system-join.path /etc/systemd/system/carlos-system-join.path |
| +/etc/systemd/system l root:root 777 48 1 multi-user.target.wants/amazon-ssm-agent.service /usr/lib/systemd/system/amazon-ssm-agent.service |
| +/etc/systemd/system l root:root 777 48 1 multi-user.target.wants/systemd-networkd.service /usr/lib/systemd/system/systemd-networkd.service |
| +/etc/systemd/system l root:root 777 40 1 multi-user.target.wants/carlos-relay.service /etc/systemd/system/carlos-relay.service |
| +/etc/systemd/system l root:root 777 37 1 multi-user.target.wants/carlos-stage.path /etc/systemd/system/carlos-stage.path |
| +/etc/systemd/system l root:root 777 35 1 multi-user.target.wants/ssh.service /usr/lib/systemd/system/ssh.service |
| +/etc/systemd/system l root:root 777 51 1 multi-user.target.wants/unattended-upgrades.service /usr/lib/systemd/system/unattended-upgrades.service |
| +/etc/systemd/system l root:root 777 39 1 multi-user.target.wants/carlos-restart.path /etc/systemd/system/carlos-restart.path |
| +/etc/systemd/system l root:root 777 39 1 multi-user.target.wants/carlos-edge.service /etc/systemd/system/carlos-edge.service |
| +/etc/systemd/system l root:root 777 45 1 multi-user.target.wants/carlos-system-update.path /etc/systemd/system/carlos-system-update.path |
| +/etc/systemd/system l root:root 777 46 1 multi-user.target.wants/run-carlos-instances.mount /etc/systemd/system/run-carlos-instances.mount |
| +/etc/systemd/system l root:root 777 37 1 multi-user.target.wants/carlos-adopt.path /etc/systemd/system/carlos-adopt.path |
| +/etc/systemd/system f root:root 644 473 1 carlos-system-update.path |
| +/etc/systemd/system f root:root 644 881 1 run-carlos-instances.mount |
| +/etc/systemd/system d root:root 755 4096 2 ssh.socket.wants |
| +/etc/systemd/system l root:root 777 43 1 ssh.socket.wants/sshd-keygen.service /usr/lib/systemd/system/sshd-keygen.service |
| +/etc/systemd/system f root:root 644 302 1 carlos-restore-verify.timer |
| +/etc/systemd/system f root:root 644 941 1 carlos-adopt.path |
| +/etc/systemd/system d root:root 755 4096 2 ssh.service.wants |
| +/etc/systemd/system l root:root 777 43 1 ssh.service.wants/sshd-keygen.service /usr/lib/systemd/system/sshd-keygen.service |
| +/etc/systemd/system d root:root 755 4096 2 carlos-edge.service.d |
| +/etc/systemd/system f root:root 644 168 1 carlos-edge.service.d/90-host-floor.conf |
| +/etc/carlos d root:root 755 4096 4 . |
| +/etc/carlos d root:root 755 4096 2 apps |
| +/etc/carlos f root:root 600 350 1 host.env |
| +/etc/carlos f root:root 600 0 1 .claims-activation.lock |
| +/etc/carlos f root:root 600 329 1 host.env.bak-pool-20261018 |
| +/etc/carlos f root:carlos 640 5 1 <redacted-1> |
| +/etc/carlos f root:root 644 106 1 <redacted-2> |
| +/etc/carlos d root:root 700 4096 2 <redacted-3> |
| +/etc/carlos f root:root 600 219 1 <redacted-3>/<redacted-4> |
| +/etc/carlos f root:root 600 0 1 <redacted-3>/<redacted-5> |
| +/etc/carlos f root:root 600 83 1 <redacted-3>/<redacted-6> |
| +/etc/carlos f root:root 600 45 1 <redacted-7> |
diff --git a/internal/host/testdata/leo-step8/leo-units-post-8.1.tsv b/internal/host/testdata/leo-step8/leo-units-post-8.1.tsv| new file mode 100644 |
| index 000000000..cc6ca097e |
| --- /dev/null |
| +++ b/internal/host/testdata/leo-step8/leo-units-post-8.1.tsv |
| @@ -0,0 +1,28 @@ |
| +root type owner mode size nlink path link_target state change |
| +/etc/carlos d root:root 700 4096 3 claim-window-cohort1-20261021T0900Z expected-post-8.1 new: mkdir -m 0700 $WINDOW (window ID proposed in the package; moves with the date) |
| +/etc/carlos d root:root 700 4096 2 claim-window-cohort1-20261021T0900Z/held expected-post-8.1 new: mkdir -m 0700 $WINDOW/held |
| +/etc/systemd/system l root:root 777 9 1 carlos-adopt.timer /dev/null expected-post-8.1 mask: /dev/null link replaces the unit file (ln -s -- /dev/null) |
| +/etc/carlos f root:root 644 419 1 claim-window-cohort1-20261021T0900Z/held/carlos-adopt.timer expected-post-8.1 moved aside: mv -T from /etc/systemd/system/carlos-adopt.timer (same filesystem, same inode and metadata) |
| +/etc/systemd/system l root:root 777 9 1 carlos-adopt.path /dev/null expected-post-8.1 mask: /dev/null link replaces the unit file (ln -s -- /dev/null) |
| +/etc/carlos f root:root 644 941 1 claim-window-cohort1-20261021T0900Z/held/carlos-adopt.path expected-post-8.1 moved aside: mv -T from /etc/systemd/system/carlos-adopt.path (same filesystem, same inode and metadata) |
| +/etc/systemd/system l root:root 777 9 1 carlos-stage.path /dev/null expected-post-8.1 mask: /dev/null link replaces the unit file (ln -s -- /dev/null) |
| +/etc/carlos f root:root 644 1295 1 claim-window-cohort1-20261021T0900Z/held/carlos-stage.path expected-post-8.1 moved aside: mv -T from /etc/systemd/system/carlos-stage.path (same filesystem, same inode and metadata) |
| +/etc/systemd/system l root:root 777 9 1 carlos-restart.path /dev/null expected-post-8.1 mask: /dev/null link replaces the unit file (ln -s -- /dev/null) |
| +/etc/carlos f root:root 644 1339 1 claim-window-cohort1-20261021T0900Z/held/carlos-restart.path expected-post-8.1 moved aside: mv -T from /etc/systemd/system/carlos-restart.path (same filesystem, same inode and metadata) |
| +/etc/systemd/system l root:root 777 9 1 carlos-system-update.path /dev/null expected-post-8.1 mask: /dev/null link replaces the unit file (ln -s -- /dev/null) |
| +/etc/carlos f root:root 644 473 1 claim-window-cohort1-20261021T0900Z/held/carlos-system-update.path expected-post-8.1 moved aside: mv -T from /etc/systemd/system/carlos-system-update.path (same filesystem, same inode and metadata) |
| +/etc/systemd/system l root:root 777 9 1 carlos-adopt.service /dev/null expected-post-8.1 mask: /dev/null link replaces the unit file (ln -s -- /dev/null) |
| +/etc/carlos f root:root 644 1364 1 claim-window-cohort1-20261021T0900Z/held/carlos-adopt.service expected-post-8.1 moved aside: mv -T from /etc/systemd/system/carlos-adopt.service (same filesystem, same inode and metadata) |
| +/etc/systemd/system l root:root 777 9 1 carlos-stage.service /dev/null expected-post-8.1 mask: /dev/null link replaces the unit file (ln -s -- /dev/null) |
| +/etc/carlos f root:root 644 2310 1 claim-window-cohort1-20261021T0900Z/held/carlos-stage.service expected-post-8.1 moved aside: mv -T from /etc/systemd/system/carlos-stage.service (same filesystem, same inode and metadata) |
| +/etc/systemd/system l root:root 777 9 1 carlos-config-sync.service /dev/null expected-post-8.1 mask: /dev/null link replaces the unit file (ln -s -- /dev/null) |
| +/etc/carlos f root:root 644 1930 1 claim-window-cohort1-20261021T0900Z/held/carlos-config-sync.service expected-post-8.1 moved aside: mv -T from /etc/systemd/system/carlos-config-sync.service (same filesystem, same inode and metadata) |
| +/etc/systemd/system l root:root 777 9 1 carlos-restart.service /dev/null expected-post-8.1 mask: /dev/null link replaces the unit file (ln -s -- /dev/null) |
| +/etc/carlos f root:root 644 2233 1 claim-window-cohort1-20261021T0900Z/held/carlos-restart.service expected-post-8.1 moved aside: mv -T from /etc/systemd/system/carlos-restart.service (same filesystem, same inode and metadata) |
| +/etc/systemd/system l root:root 777 9 1 carlos-system-update.service /dev/null expected-post-8.1 mask: /dev/null link replaces the unit file (ln -s -- /dev/null) |
| +/etc/carlos f root:root 644 3973 1 claim-window-cohort1-20261021T0900Z/held/carlos-system-update.service expected-post-8.1 moved aside: mv -T from /etc/systemd/system/carlos-system-update.service (same filesystem, same inode and metadata) |
| +/etc/systemd/system l root:root 777 9 1 carlos-system-recover.service /dev/null expected-post-8.1 mask: /dev/null link replaces the unit file (ln -s -- /dev/null) |
| +/etc/carlos f root:root 644 3682 1 claim-window-cohort1-20261021T0900Z/held/carlos-system-recover.service expected-post-8.1 moved aside: mv -T from /etc/systemd/system/carlos-system-recover.service (same filesystem, same inode and metadata) |
| +/etc/systemd/system l root:root 777 9 1 carlos-edge.service /dev/null expected-post-8.1 mask: /dev/null link replaces the unit file (ln -s -- /dev/null) |
| +/etc/carlos f root:root 644 5218 1 claim-window-cohort1-20261021T0900Z/held/carlos-edge.service expected-post-8.1 moved aside: mv -T from /etc/systemd/system/carlos-edge.service (same filesystem, same inode and metadata) |
| +/etc/systemd/system l root:root 777 39 1 multi-user.target.wants/carlos-edge.service /etc/systemd/system/carlos-edge.service expected-post-8.1-absent removed by systemctl disable carlos-edge.service (WantedBy=multi-user.target only) |
diff --git a/internal/hostbox/journalmigration_absent_linux_test.go b/internal/hostbox/journalmigration_absent_linux_test.go| new file mode 100644 |
| index 000000000..8e4b9db21 |
| --- /dev/null |
| +++ b/internal/hostbox/journalmigration_absent_linux_test.go |
| @@ -0,0 +1,43 @@ |
| +//go:build linux |
| + |
| +package hostbox |
| + |
| +import ( |
| + "context" |
| + "os" |
| + "path/filepath" |
| + "testing" |
| +) |
| + |
| +// Step 8 readiness: leo never ran the legacy claim writer, so it has no |
| +// legacy journal directory at all. The import is empty and completes, |
| +// exactly as for an empty directory, and a rerun is a no-op. A directory |
| +// that appears after the marker is still checked against it. |
| +func TestMigrateLegacyWithNoLegacyDirectory(t *testing.T) { |
| + service, _, _ := journalFixture(t) |
| + legacy := filepath.Join(t.TempDir(), ".hostclaims") |
| + called := 0 |
| + recover := func(context.Context) error { called++; return nil } |
| + if err := service.MigrateLegacy(t.Context(), legacy, "box", "enrollment", nil, nil, recover); err != nil { |
| + t.Fatalf("absent legacy directory: %v", err) |
| + } |
| + source, files, _, marked, err := service.journalMigrationMarker(t.Context()) |
| + if err != nil || !marked || source != legacy || files != "absent" { |
| + t.Fatalf("marker = %q %q %v %v", source, files, marked, err) |
| + } |
| + if err := service.MigrateLegacy(t.Context(), legacy, "box", "enrollment", nil, nil, recover); err != nil || called != 2 { |
| + t.Fatalf("rerun: %v (recoveries %d)", err, called) |
| + } |
| + // Legacy journals that appear after the marker are not imported silently. |
| + if err := os.Mkdir(legacy, 0o700); err != nil { |
| + t.Fatal(err) |
| + } |
| + for _, name := range []string{"launches.db", "inventory.db"} { |
| + if err := os.WriteFile(filepath.Join(legacy, name), []byte("late legacy journal"), 0o600); err != nil { |
| + t.Fatal(err) |
| + } |
| + } |
| + if err := service.MigrateLegacy(t.Context(), legacy, "box", "enrollment", nil, nil); err == nil { |
| + t.Fatal("legacy journals that appeared after the marker were accepted") |
| + } |
| +} |
diff --git a/internal/hostbox/journalmigration_linux.go b/internal/hostbox/journalmigration_linux.go| index f7967b148..4561bcca7 100644 |
| --- a/internal/hostbox/journalmigration_linux.go |
| +++ b/internal/hostbox/journalmigration_linux.go |
| @@ -198,6 +198,12 @@ func (s *JournalService) MigrateLegacy(ctx context.Context, legacy, box, enrollm |
| if err != nil { |
| return err |
| } |
| + // A box that never ran the legacy claim writer has no legacy directory |
| + // at all (leo, 2026-10-19). The copy names each missing file instead of |
| + // "absent"; both say there is nothing to import. |
| + if files == "absent" && copied == legacyNoJournals { |
| + copied = files |
| + } |
| // The rows came from exactly the bytes the identity names: the digests |
| // are of the copies themselves, not of a later re-read. |
| if copied != files { |
| @@ -387,6 +393,10 @@ func legacyJournalFiles(legacy string) (string, int, error) { |
| return strings.Join(parts, "/"), present, nil |
| } |
| +// legacyNoJournals is readLegacyJournals' identity when neither legacy |
| +// journal nor its write-ahead log exists. |
| +const legacyNoJournals = "launches.db:-/launches.db-wal:-/inventory.db:-/inventory.db-wal:-" |
| + |
| // legacyRowsDigest is the order-independent digest of the canonical rows. |
| func legacyRowsDigest(launches []LaunchInventory, inventories map[string]StopInventory) string { |
| var rows []string |
diff --git a/internal/hostbox/migration_census_linux.go b/internal/hostbox/migration_census_linux.go| index 61201d49e..f047ac80b 100644 |
| --- a/internal/hostbox/migration_census_linux.go |
| +++ b/internal/hostbox/migration_census_linux.go |
| @@ -2,11 +2,19 @@ package hostbox |
| import ( |
| "context" |
| + "fmt" |
| "io/fs" |
| "os" |
| "path/filepath" |
| ) |
| +// migrationRefusal names the root and path the migration census could not |
| +// classify, so the operator sees which entry stopped it. It still wraps the |
| +// census's own error. |
| +func migrationRefusal(err error, root, rel string) error { |
| + return fmt.Errorf("%w: migration census: %s entry %q is not classified", err, root, rel) |
| +} |
| + |
| // CheckMigrationRoots accounts for every managed entry under offline exclusion. |
| // Only independently retained inventory may classify an installation subtree. |
| func CheckMigrationRoots(ctx context.Context, roots map[string]string, shared map[string][]string, inventories []StopInventory) error { |
| @@ -79,7 +87,11 @@ func CheckMigrationRoots(ctx context.Context, roots map[string]string, shared ma |
| return nil |
| } |
| if rule, ok := rules[rel]; ok { |
| - return rule.check(path, d) |
| + err := rule.check(path, d) |
| + if err != nil && err != filepath.SkipDir { |
| + return migrationRefusal(err, name, rel) |
| + } |
| + return err |
| } |
| if name == "units" && censusVendorUnitLink(root, path) { |
| return nil |
| @@ -90,7 +102,7 @@ func CheckMigrationRoots(ctx context.Context, roots map[string]string, shared ma |
| if d.IsDir() { |
| return nil |
| } |
| - return ErrTypedInventory |
| + return migrationRefusal(ErrTypedInventory, name, rel) |
| }); err != nil { |
| return err |
| } |
diff --git a/internal/hostops/backfill_owners_test.go b/internal/hostops/backfill_owners_test.go| new file mode 100644 |
| index 000000000..400be8115 |
| --- /dev/null |
| +++ b/internal/hostops/backfill_owners_test.go |
| @@ -0,0 +1,158 @@ |
| +package hostops |
| + |
| +import ( |
| + "encoding/json" |
| + "errors" |
| + "fmt" |
| + "testing" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/domains" |
| + "github.com/carlosframework/platform/internal/hostclaim" |
| + "github.com/carlosframework/platform/internal/release" |
| +) |
| + |
| +// publishOwned writes a collecting manifest that names no box or region |
| +// (backfill-collect's plan), one signed report per box listing the hosts |
| +// owners gives it, and runs Publish. |
| +func (a *activation) publishOwned(owners map[string][]string, named map[string]BackfillHost) (string, error) { |
| + a.t.Helper() |
| + p := BackfillPlan{V: 1, ID: "backfill", Phase: "collecting", DirectoryRevision: 1, CreatedAt: a.f.now, Hosts: map[string]BackfillHost{}} |
| + byHost := map[string]Record{} |
| + for _, r := range a.records { |
| + byHost[r.Holder.Host] = r |
| + h := BackfillHost{Key: r.Holder.Key, ETag: r.ETag, Acquisition: "acquisition"} |
| + if n, ok := named[r.Holder.Host]; ok { |
| + h.Box, h.Region = n.Box, n.Region |
| + } |
| + p.Hosts[r.Holder.Host] = h |
| + } |
| + raw, err := release.SignRootEnvelope(a.sg, p) |
| + if err != nil { |
| + a.t.Fatal(err) |
| + } |
| + tag, err := a.f.store.Put(a.ctx, BackfillKey, raw, "") |
| + if err != nil { |
| + a.t.Fatal(err) |
| + } |
| + for ref, box := range a.roster.Boxes { |
| + report := CensusReport{Box: ref, Enrollment: box.Enrollment, BootID: "boot", Reservation: p.ID, DirectoryRevision: 1, |
| + CapturedAt: a.f.now, Complete: true, Installations: []CensusInstallation{}, Seeds: &CensusSeeds{Journaled: []string{}, Gated: []string{}}} |
| + for _, host := range owners[ref] { |
| + r := byHost[host] |
| + inc, err := r.Incarnation() |
| + if err != nil { |
| + a.t.Fatal(err) |
| + } |
| + report.Installations = append(report.Installations, CensusInstallation{Host: host, Account: r.Holder.Account, App: r.Holder.App, Family: r.Holder.Family, Key: r.Holder.Key, Incarnation: inc, Route: true}) |
| + } |
| + if err := PublishCensusReport(a.ctx, a.f.store, a.d, report, a.boxKeys[ref]); err != nil { |
| + a.t.Fatalf("publish %s's census report: %v", ref, err) |
| + } |
| + } |
| + b := &Backfill{Store: a.f.store, Directory: a.d} |
| + return tag, b.Publish(a.ctx, a.sg, tag) |
| +} |
| + |
| +func (a *activation) plan() BackfillPlan { |
| + a.t.Helper() |
| + b := &Backfill{Store: a.f.store, Directory: a.d} |
| + p, _, err := b.readPlan(a.ctx) |
| + if err != nil { |
| + a.t.Fatal(err) |
| + } |
| + return p |
| +} |
| + |
| +// Publish takes each host's box and region from the one box whose signed |
| +// report names it. A host no box reports keeps no box, so it is never |
| +// adopted. The adopters' own check then accepts the published plan, and the |
| +// console's adoption pass gives the reported host its claim on that box. |
| +func TestPublishTakesOwnersFromTheCensus(t *testing.T) { |
| + a := newActivation(t) |
| + a.record("legacy.shop.example.com", nil) |
| + a.record("stale.shop.example.com", nil) |
| + if _, err := a.publishOwned(map[string][]string{"box-a": {"legacy.shop.example.com"}}, nil); err != nil { |
| + t.Fatalf("Publish: %v", err) |
| + } |
| + p := a.plan() |
| + if p.Phase != "adopting" { |
| + t.Fatalf("phase %q", p.Phase) |
| + } |
| + if h := p.Hosts["legacy.shop.example.com"]; h.Box != "box-a" || h.Region != "ie" { |
| + t.Fatalf("reported host = %+v, want box-a in ie", h) |
| + } |
| + if h := p.Hosts["stale.shop.example.com"]; h.Box != "" || h.Region != "" { |
| + t.Fatalf("unreported host = %+v, want no box", h) |
| + } |
| + b := &Backfill{Store: a.f.store, Directory: a.d} |
| + if _, err := b.VerifyAdopting(a.ctx); err != nil { |
| + t.Fatalf("VerifyAdopting: %v", err) |
| + } |
| + // The console's adoption pass, as production builds it. |
| + for _, r := range a.records { |
| + inc, err := r.Incarnation() |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + raw, _ := json.Marshal(domains.Claim{Schema: 1, Host: r.Holder.Host, Account: r.Holder.Account, App: r.Holder.App, Acquisition: "acquisition", RecordIncarnation: inc, OwnershipState: "committed"}) |
| + if _, err := a.f.store.Put(a.ctx, domains.ClaimKey(r.Holder.Host), raw, ""); err != nil { |
| + t.Fatal(err) |
| + } |
| + } |
| + serial := 0 |
| + c := NewBackfillController(a.f.store, a.d, func() time.Time { return a.f.now }, func() string { serial++; return fmt.Sprintf("op-%d", serial) }) |
| + err := c.ReconcileClaims(a.ctx) |
| + if !errors.Is(err, ErrIdentity) { |
| + t.Fatalf("ReconcileClaims = %v; want the unreported host refused", err) |
| + } |
| + obs, err := hostclaim.Read(a.ctx, a.f.store, "legacy.shop.example.com") |
| + if err != nil || obs.Absent || obs.Claim.Op.Exec != "box-a" || obs.Claim.Holder.WriterBox != "box-a" || obs.Claim.Op.Kind != hostclaim.KindServing { |
| + t.Fatalf("reported host's claim = %+v, %v; want serving on box-a", obs.Claim, err) |
| + } |
| + if obs, err := hostclaim.Read(a.ctx, a.f.store, "stale.shop.example.com"); err != nil || !obs.Absent { |
| + t.Fatalf("unreported host gained a claim: %+v %v", obs.Claim, err) |
| + } |
| + // A census report changed after Publish no longer matches its digest. |
| + key, _ := reportKey("backfill", "box-b") |
| + old, tag, err := a.f.store.Get(a.ctx, key) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if _, err := a.f.store.Put(a.ctx, key, append(old, ' '), tag); err != nil { |
| + t.Fatal(err) |
| + } |
| + if _, err := b.VerifyAdopting(a.ctx); !errors.Is(err, ErrIdentity) { |
| + t.Fatalf("VerifyAdopting over a changed report = %v", err) |
| + } |
| +} |
| + |
| +// Two boxes reporting one host, or a census that disagrees with a box or |
| +// region the collecting plan named, refuse and leave the plan collecting. |
| +func TestPublishRefusesConflictingOwners(t *testing.T) { |
| + for name, c := range map[string]struct { |
| + owners map[string][]string |
| + named map[string]BackfillHost |
| + }{ |
| + "two boxes": {map[string][]string{"box-a": {"legacy.shop.example.com"}, "box-b": {"legacy.shop.example.com"}}, nil}, |
| + "other box": {map[string][]string{"box-a": {"legacy.shop.example.com"}}, map[string]BackfillHost{"legacy.shop.example.com": {Box: "box-b"}}}, |
| + "other region": {map[string][]string{"box-a": {"legacy.shop.example.com"}}, map[string]BackfillHost{"legacy.shop.example.com": {Region: "uk"}}}, |
| + } { |
| + t.Run(name, func(t *testing.T) { |
| + a := newActivation(t) |
| + a.record("legacy.shop.example.com", nil) |
| + if _, err := a.publishOwned(c.owners, c.named); !errors.Is(err, ErrIdentity) { |
| + t.Fatalf("Publish = %v, want ErrIdentity", err) |
| + } |
| + if got := a.phase(); got != "collecting" { |
| + t.Fatalf("phase %q after a refusal", got) |
| + } |
| + }) |
| + } |
| + // A plan that already names the census's own box and region publishes. |
| + a := newActivation(t) |
| + a.record("legacy.shop.example.com", nil) |
| + if _, err := a.publishOwned(map[string][]string{"box-a": {"legacy.shop.example.com"}}, map[string]BackfillHost{"legacy.shop.example.com": {Box: "box-a", Region: "ie"}}); err != nil { |
| + t.Fatal(err) |
| + } |
| +} |
diff --git a/internal/hostops/backfill_publish.go b/internal/hostops/backfill_publish.go| index c14b30295..547d12e2f 100644 |
| --- a/internal/hostops/backfill_publish.go |
| +++ b/internal/hostops/backfill_publish.go |
| @@ -64,6 +64,30 @@ func (b *Backfill) Publish(ctx context.Context, signer release.Signer, collectin |
| if !ok || selected.Key != r.Holder.Key || selected.ETag != r.ETag || selected.Acquisition == "" || r.Holder.Acquisition != selected.Acquisition { |
| return ErrIdentity |
| } |
| + // Each host's box and region come from the signed census, never from |
| + // a typed list: the one box whose report names the host, and that |
| + // box's enrolled region. Adoption checks the same ownership again. |
| + // A host no box reports keeps no box, so it is never adopted. A box |
| + // or region the collecting plan already named must agree. |
| + var owners []string |
| + for ref, report := range census.Reports { |
| + for _, i := range report.Installations { |
| + if i.Host == r.Holder.Host { |
| + owners = append(owners, ref) |
| + } |
| + } |
| + } |
| + if len(owners) > 1 { |
| + return ErrIdentity |
| + } |
| + if len(owners) == 1 { |
| + enrolled := census.Roster.Roster.Boxes[owners[0]] |
| + if (selected.Box != "" && selected.Box != owners[0]) || (selected.Region != "" && selected.Region != enrolled.Region) || enrolled.Region == "" { |
| + return ErrIdentity |
| + } |
| + selected.Box, selected.Region = owners[0], enrolled.Region |
| + p.Hosts[r.Holder.Host] = selected |
| + } |
| } |
| if len(p.Hosts) != len(census.Records) { |
| return ErrIdentity |
| @@ -77,6 +101,29 @@ func (b *Backfill) Publish(ctx context.Context, signer release.Signer, collectin |
| return err |
| } |
| +// VerifyAdopting is the adopters' own acceptance of a published plan, run |
| +// by the signer as readback after Publish: the signed plan reads adopting, |
| +// and the complete census it froze verifies against its report digests, as |
| +// Adoption and AuthorizeBackfillApp require. It writes nothing. |
| +func (b *Backfill) VerifyAdopting(ctx context.Context) (BackfillPlan, error) { |
| + p, tag, err := b.read(ctx) |
| + if err != nil { |
| + return p, err |
| + } |
| + a := ProductionAuthority{Store: b.Store, Directory: b.Directory} |
| + if _, err = a.census(ctx, "", "", p.ID, p.DirectoryRevision, p.Reports); err != nil { |
| + return p, err |
| + } |
| + _, fresh, err := b.read(ctx) |
| + if err != nil { |
| + return p, err |
| + } |
| + if fresh != tag { |
| + return p, ErrPending |
| + } |
| + return p, nil |
| +} |
| + |
| // PublishCensusReport is called by the guarded box census collector. Each box |
| // owns its private signing key; the root-signed roster binds its public key to |
| // verified enrollment. The console has only that public key. |
diff --git a/internal/provision/claim_step8_readiness_census_test.go b/internal/provision/claim_step8_readiness_census_test.go| new file mode 100644 |
| index 000000000..63a782644 |
| --- /dev/null |
| +++ b/internal/provision/claim_step8_readiness_census_test.go |
| @@ -0,0 +1,34 @@ |
| +package provision |
| + |
| +// plan14ClaimStep8ReadinessWriteCensusInventory is the record-write |
| +// inventory for the step 8 readiness branch, PROPOSED to platform-1e as |
| +// census owner (rev3=PROPOSED-claim-step8-readiness; |
| +// docs/superpowers/notes/census-proposal-claim-step8-readiness.md). |
| +// |
| +// - Backfill.Publish, re-reviewed: before its one Put of the backfill |
| +// manifest it now copies each host's box and region from the one box |
| +// whose signed census report names the host (or refuses two). The Put, |
| +// its key and its If-Match are unchanged. Still nonrecord. |
| +// - Runner.backfillCollect, new: its one Put is the create-only collecting |
| +// manifest at control/hosts-backfill.json, through the runner's |
| +// checkedStore, whose allow-list admits only that key for the phase. It |
| +// reads records (hostops.Apps, hostops.Records) and writes none. Hence |
| +// nonrecord. |
| +// - hostInstallationPaths, new: the "instances" literal names a host's |
| +// nested data directory under the box's data root, for the migration |
| +// census's classification. It builds no record key. Hence key-site. |
| +func plan14ClaimStep8ReadinessWriteCensusInventory() []censusEntry { |
| + entries := plan14ClaimStep6ArtifactsWriteCensusInventory() |
| + kept := make([]censusEntry, 0, len(entries)+3) |
| + for _, e := range entries { |
| + if e.File == "internal/hostops/backfill_publish.go" && e.Unit == "Backfill.Publish" { |
| + continue |
| + } |
| + kept = append(kept, e) |
| + } |
| + return append(kept, |
| + censusEntry{"internal/hostops/backfill_publish.go", "Backfill.Publish", "nonrecord", "sel:Put", "92754dcc02c07fecaab58ef014bfb381c49ee5d90d5f5ca256bbfe275087d8b9"}, |
| + censusEntry{"internal/claimactivation/backfill.go", "Runner.backfillCollect", "nonrecord", "sel:Put", "ac53546891b2c6f982c0316595f86ff6b550250a3a4edcbc16d51d84fd18b06a"}, |
| + censusEntry{"internal/host/claimcensus_migration_linux.go", "hostInstallationPaths", "key-site", "lit:instances", "e697ca40c82911d63a5f8b401f61fee7e9e20552a2530ed358e55c4b8d827370"}, |
| + ) |
| +} |
diff --git a/internal/provision/plan14_root_reads_test.go b/internal/provision/plan14_root_reads_test.go| index 8e68b7dbb..1ccb80993 100644 |
| --- a/internal/provision/plan14_root_reads_test.go |
| +++ b/internal/provision/plan14_root_reads_test.go |
| @@ -934,6 +934,16 @@ func TestPlan14RootReadCensus(t *testing.T) { |
| t.Fatal(err) |
| } |
| seal = "53b3cbb6f46d657ce32f49dad84dae79316c80da949ccf37273833eef5c5b2d7" |
| + // Census overlay claim-step8-readiness (dev cohort-1 step 8: the backfill |
| + // phases, leo's migration census classes and window masks, the floor and |
| + // retry directories' umask, and the absent legacy journal directory), |
| + // loaded last. PROPOSED to platform-1e as census owner |
| + // (docs/superpowers/notes/census-proposal-claim-step8-readiness.md), |
| + // marked rev3=PROPOSED-claim-step8-readiness. It adds no accepted-legacy |
| + // row, so the seal stands. |
| + if err = loadRootReadInventory(filepath.Join(root, "internal/provision/testdata/plan14-root-reads-claim-step8-readiness.tsv"), &inv, true); err != nil { |
| + t.Fatal(err) |
| + } |
| errs := append(compareRootReads(got, inv), checkRootReadLegacy(inv, seal)...) |
| errs = append(errs, checkRootReadDeclarations(got, inv)...) |
| if len(errs) > 0 { |
diff --git a/internal/provision/plan9_b8_census_test.go b/internal/provision/plan9_b8_census_test.go| index de6143aaf..bb9c60dfa 100644 |
| --- a/internal/provision/plan9_b8_census_test.go |
| +++ b/internal/provision/plan9_b8_census_test.go |
| @@ -366,7 +366,7 @@ func TestPlan9B8RecordWriteCensus(t *testing.T) { |
| if len(got) < 100 { |
| t.Fatalf("census found only %d units under %s; the walk is broken", len(got), root) |
| } |
| - inventory := plan14ClaimStep6ArtifactsWriteCensusInventory() |
| + inventory := plan14ClaimStep8ReadinessWriteCensusInventory() |
| problems := append(censusDiff(inventory, got), censusPolicy(inventory)...) |
| if len(problems) > 0 { |
| t.Fatalf("record-write census rejected %d site(s); update the explicit inventory only after reviewing the source:\n %s", len(problems), strings.Join(problems, "\n ")) |
diff --git a/internal/provision/testdata/plan14-root-reads-claim-step8-readiness.tsv b/internal/provision/testdata/plan14-root-reads-claim-step8-readiness.tsv| new file mode 100644 |
| index 000000000..2534d5041 |
| --- /dev/null |
| +++ b/internal/provision/testdata/plan14-root-reads-claim-step8-readiness.tsv |
| @@ -0,0 +1,18 @@ |
| +# Census overlay claim-step8-readiness (dev cohort-1 step 8 readiness: the backfill phases, leo's migration census classes, journal-migrate's window masks and umask, and the absent legacy directory), loaded LAST, after claim-guard-umask. Every row is PROPOSED to platform-1e as census owner; the proposal is docs/superpowers/notes/census-proposal-claim-step8-readiness.md. |
| +# Existing units keep their class and task exactly; their bases add what changed. New units: backfillReady (signer), legacyRouteRow, claimMigrationSharedPaths, verifyMigrationWindow and rootPrivateEntry (journal-migrate). It adds no accepted-legacy row, so the seal stands. |
| +@coverage internal/claimactivation 798e0c4e47cef053d27f7e0c476e2b1fa1e4ce62905a7886a2fb20fb26a2e623 |
| +@coverage internal/host 7af2df2472fe86811814416b977dbd5d782fd813636eaeafd143ef7025f228ad |
| +@coverage internal/hostbox 46f97aa29603b9fcd9ee1f32fea80669c00df36601437eb5871b0cd83507e305 |
| +@coverage internal/hostops 29aeb7672fc3fec094a15a3b378821f32832ca41858afeb5a25dfe841ee4d239 |
| +internal/claimactivation/runner.go Runner.Run authority F,G,J4 rev3=PROPOSED-claim-step8-readiness;identity=explicit operator CLI uid 0 on the trusted signing host;basis=unchanged phases as ruled for claim-step6-artifacts; adds backfill-collect and backfill-adopt, whose only writable key is control/hosts-backfill.json, each write verified and read back, the dry-run overlay never calling backing Put file:Read@310,file:Read@6305,file:Read@7208 de4a7b1cbde996a1ea3ed4af10e9e15c97aa1118554f786d53a3b565df1e49b7 |
| +internal/claimactivation/backfill.go backfillReady authority F,G,J4 rev3=PROPOSED-claim-step8-readiness;identity=explicit operator CLI uid 0 on the trusted signing host;basis=refusal up front only: the signed roster must be published with a home region, and the published floor must verify under the release key to the roster's floor digest; it reads and grants nothing else file:Read@240 ebbb04dde23cff468ca19227aa30ba6f3de738d3de173c809b95c5bb438d5885 |
| +internal/host/claimjournal_migration.go RunSystemJournalMigrate authority G,J4 rev3=PROPOSED-claim-step8-readiness;identity=carlos system journal-migrate, offline operator CLI, uid 0;basis=explicit offline root operator entry requests full G2 bootstrap after verified claim owner setup; --window is checked before anything is written and only lets the census accept the named window's own masks 2e9b7b851f6aba29dcdcd607d950ab5ae26c6a4291becfa35ff91e05d443cd8c |
| +internal/host/rootmigration_census.go rootMigrationCollector.collect authority G,J4 rev3=PROPOSED-claim-step8-readiness;identity=carlos system journal-migrate, offline operator CLI, uid 0;basis=signed collecting manifest and actor directory bound complete records/claims census; registry Seeds are refusal-only; with the agent stopped and masked, a legacy route row is reported as a route installation only when legacyRouteRow matches it to its frozen record, which only nominates this box for that record's adoption (checked again by the adopter against the signed census and the committed domain claim) and installs nothing; any other row refuses file:Exec@4684,file:Read@2868,file:Read@689,registry:Host@6224,registry:Host@6376,registry:Host@6578,registry:Host@6710,whole:Route@6193,whole:Route@6415 d1348e630d27e62d90643b9d73ee4ac0a1f11af720410430f23d113aa69f390d |
| +internal/host/rootmigration_census.go legacyRouteRow authority G,J4 rev3=PROPOSED-claim-step8-readiness;identity=carlos system journal-migrate, offline operator CLI, uid 0;basis=pure shape check of an agent-written registry row under offline exclusion: an ordinary nested instance row, unowned by any claim, of the frozen signed record's account and app, with a valid version; a match only nominates the box for adoption and grants nothing on it registry:Account@342,registry:App@377,registry:Backing@261,registry:ClaimOwnership@148,registry:DBPath@431,registry:Host@477,registry:Kind@89,registry:Replica@204,registry:Seed@219,registry:SeedSource@237,registry:Tunnel@320,registry:Unit@282,registry:Version@513,whole:ClaimOwnership@152 71bf89aad3a4357a87a11a74ed8afc861dedf2060dab10cd8ccd5450913bcbf6 |
| +internal/host/rootmigration_production_linux.go migrateRootClaimAuthority authority G,J4,E3 rev3=PROPOSED-claim-step8-readiness;run=15;identity=offline root migration uid 0;basis=requires masked inactive root mutator units and no running legacy root command under mandatory release exclusion; ordinary nonparticipants acquire no migration lock; the census classifies with claimMigrationSharedPaths after verifyMigrationWindow, and the public floor directories are made 0755 by mkdirAllExact whatever the operator's umask file:Open@1814,file:Open@374,file:Open@615,file:Open@7287,file:Open@739,file:Read@2766,file:ReadFile@1571,whole:Route@3079 004467e63a878593d7a4836d5f41e1a6555f034233971592f2f49078a38ddb5e |
| +internal/host/claimcensus_migration_linux.go claimMigrationSharedPaths authority G,J4 rev3=PROPOSED-claim-step8-readiness;identity=offline root migration uid 0;basis=refusal-only classification for the migration census: a reported route installation's paths and live link at exactly its row's version path, a frozen record's retained leftovers with no row and no claim (never reported, so never adopted, and never deleted), and window masks to exactly /dev/null; anything else stays unclassified and refuses file:Readlink@1795,registry:Account@1078,registry:Account@1399,registry:App@1107,registry:App@1425,registry:Version@1150,registry:Version@1412,whole:Route@1064,whole:Route@1624 9c3958b4041cfb3ebe1a45bd7fd677f4c9ee4cffc5f4682ae682a50cc5e80705 |
| +internal/host/claimcensus_migration_linux.go verifyMigrationWindow authority G,J4 rev3=PROPOSED-claim-step8-readiness;identity=carlos system journal-migrate, offline operator CLI, uid 0;basis=refusal-only: --window must be root's private directory outside unit, data and opt paths; a window unit that is a symlink must point exactly at /dev/null with its original held as a root-owned single-link regular file file:Lstat@841,file:Readlink@1023 921f8cde74c5747229ac0f2672f80ceeae7f3872203f43c22a52cecbb3da79d9 |
| +internal/host/claimcensus_migration_linux.go rootPrivateEntry authority G,J4 rev3=PROPOSED-claim-step8-readiness;identity=carlos system journal-migrate, offline operator CLI, uid 0;basis=Lstat only: root-owned, a directory closed to group and other or a single-link regular file, never a symlink; refusal only file:Lstat@67 13cd93f6a737106cdec1efdbcc31384f121a886421791ea7afb8fab07adca931 |
| +internal/host/rootupdates_runtime.go prepareRootRetryUnitDirectory authority E3 rev3=PROPOSED-claim-step8-readiness;identity=root claim/adoption controller uid 0;basis=root unit installer requires an independently opened complete census before creating the public retry directory or enabling its controller write path; the directory is made 0755 by mkdirAllExact whatever the operator's umask file:Lstat@67,file:Open@832 ab65b707653a835a94f11776baca68ae2525d6a72b1ec89d8b0d7a338abfc591 |
| +internal/hostbox/migration_census_linux.go CheckMigrationRoots authority G,J4 rev3=PROPOSED-claim-step8-readiness;run=18;identity=offline root migration uid 0;basis=accounts for trusted vendor systemd links separately from retained installation artifacts without exempting unknown files or CARLOS unit aliases; a refusal now names the root and path it could not classify file:Lstat@932,file:WalkDir@1202 dbe07156eaf030cbcc8578beff09cc021c0ef5d6ec8b86954cd6db0a0c837302 |
| +internal/hostbox/journalmigration_linux.go JournalService.MigrateLegacy authority C,J2,J4 rev3=PROPOSED-claim-step8-readiness;root-input; an absent legacy directory imports nothing, exactly as an empty one does, and its marker records absent file:QueryRowContext@3944,file:QueryRowContext@5658,file:ReadDir@914 c51829af08eaf56acc99c8e83cda448cc9453d868ed3c16b0d21cd617076984d |