amadan

carlos / platform Public

Merge origin/main 70e57449 into files seed + WIP checkpoint 32 (not for merge)

Census overlay re-pinned after main's chain; retained unverifiable parks no longer block later retirement or wakes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Paul Campbell pushed by claude 7bf183eea3a2e72cb642f7086c98a59cc759736d parent f086305parent 70e5744
20 files changed, +1133 −81
  • approved_moves_copy_test.go +1 −0
  • docs/superpowers/plans/2026-10-06-moves-phase1-copy.md +14 −0
  • docs/superpowers/plans/2026-10-06-moves-phase1-rehearsal.md +10 −2
  • internal/host/claimfilesseed_reboot_rootrunner_linux_test.go +49 −14
  • internal/host/claimfilesseed_renumber_rootrunner_linux_test.go +7 −0
  • internal/host/claimservingpark_records.go +13 −5
  • internal/host/claimservingquarantine_linux_test.go +15 −9
  • internal/host/claimservingretained_linux_test.go +117 −0
  • internal/host/movesfiles.go +85 −28
  • internal/host/movesfiles_test.go +11 −5
  • internal/host/movesfilescap_test.go +539 −0
  • internal/host/movesphase1.go +106 −3
  • internal/moves/archive/archive.go +17 −0
  • internal/moves/archive/archive_test.go +90 −6
  • internal/moves/archive/limits.go +15 −0
  • internal/provision/plan14_legacy_identity_test.go +3 −3
  • internal/provision/plan14_root_reads_test.go +19 −1
  • internal/provision/plan9_b8_census_test.go +3 −1
  • internal/provision/testdata/plan14-root-reads-moves-overcap-cut-236.tsv +15 −0
  • internal/provision/testdata/plan14-root-reads-seed.tsv +4 −4

CI passed — run details

CI log 
=== gate ===
cd launchctx && go vet ./... && go test ./... -count=1
ok  	amadan.net/carlos/platform.git/launchctx	0.011s
?   	amadan.net/carlos/platform.git/launchctx/controlname	[no test files]
?   	amadan.net/carlos/platform.git/launchctx/wireid	[no test files]
go vet ./...
gofmt -l .: clean
go test ./... -count=1 -timeout 30m
ok  	github.com/carlosframework/platform	284.932s
?   	github.com/carlosframework/platform/cmd/changelog	[no test files]
ok  	github.com/carlosframework/platform/cmd/console	2.714s
ok  	github.com/carlosframework/platform/cmd/moveprobe	92.296s
ok  	github.com/carlosframework/platform/cmd/zddprobe	0.710s
ok  	github.com/carlosframework/platform/internal/activator	250.207s
ok  	github.com/carlosframework/platform/internal/agentsock	0.015s
ok  	github.com/carlosframework/platform/internal/alerting	0.003s
ok  	github.com/carlosframework/platform/internal/analytics	0.021s
ok  	github.com/carlosframework/platform/internal/appconfig	2.478s
ok  	github.com/carlosframework/platform/internal/apperr	0.073s
ok  	github.com/carlosframework/platform/internal/applog	0.139s
ok  	github.com/carlosframework/platform/internal/awsinv	0.003s
ok  	github.com/carlosframework/platform/internal/boxchoice	0.004s
ok  	github.com/carlosframework/platform/internal/boxid	0.893s
?   	github.com/carlosframework/platform/internal/boxid/boxidtest	[no test files]
?   	github.com/carlosframework/platform/internal/boxid/internal/signtest	[no test files]
ok  	github.com/carlosframework/platform/internal/boxvolumes	0.008s
ok  	github.com/carlosframework/platform/internal/caps	0.011s
ok  	github.com/carlosframework/platform/internal/certwarm	0.002s
ok  	github.com/carlosframework/platform/internal/cgroup	0.716s
ok  	github.com/carlosframework/platform/internal/changelog	0.305s
ok  	github.com/carlosframework/platform/internal/clamd	0.054s
?   	github.com/carlosframework/platform/internal/cliflag	[no test files]
ok  	github.com/carlosframework/platform/internal/console	388.752s
?   	github.com/carlosframework/platform/internal/controlname	[no test files]
?   	github.com/carlosframework/platform/internal/controlsock	[no test files]
ok  	github.com/carlosframework/platform/internal/dbdoor	5.847s
ok  	github.com/carlosframework/platform/internal/dbdoor/dbdoortest	0.008s
ok  	github.com/carlosframework/platform/internal/dbverify	0.004s
ok  	github.com/carlosframework/platform/internal/dnsrec	0.002s
ok  	github.com/carlosframework/platform/internal/docsite	0.005s
ok  	github.com/carlosframework/platform/internal/domains	2.322s
ok  	github.com/carlosframework/platform/internal/doorkey	0.006s
ok  	github.com/carlosframework/platform/internal/economics	0.004s
ok  	github.com/carlosframework/platform/internal/edge	26.539s
?   	github.com/carlosframework/platform/internal/edgeproof	[no test files]
ok  	github.com/carlosframework/platform/internal/edgesfile	0.007s
ok  	github.com/carlosframework/platform/internal/fleetapi	0.130s
ok  	github.com/carlosframework/platform/internal/gate	1.655s
ok  	github.com/carlosframework/platform/internal/genselect	0.014s
ok  	github.com/carlosframework/platform/internal/geo	0.021s
ok  	github.com/carlosframework/platform/internal/geoip	0.029s
ok  	github.com/carlosframework/platform/internal/host	159.864s
ok  	github.com/carlosframework/platform/internal/hostactors	0.310s
ok  	github.com/carlosframework/platform/internal/hostbox	2.453s
ok  	github.com/carlosframework/platform/internal/hostclaim	0.494s
ok  	github.com/carlosframework/platform/internal/hostevidence	0.014s
ok  	github.com/carlosframework/platform/internal/hostexec	4.959s
ok  	github.com/carlosframework/platform/internal/hostfence	0.029s
ok  	github.com/carlosframework/platform/internal/hostfiles	10.512s
ok  	github.com/carlosframework/platform/internal/hostfloor	0.014s
ok  	github.com/carlosframework/platform/internal/hostguard	1.007s
ok  	github.com/carlosframework/platform/internal/hostops	1.555s
?   	github.com/carlosframework/platform/internal/hostops/fencerelease	[no test files]
ok  	github.com/carlosframework/platform/internal/hostops/internal/fencerelease	0.013s
ok  	github.com/carlosframework/platform/internal/hostquarantine	0.014s
ok  	github.com/carlosframework/platform/internal/hostrelocate	0.447s
ok  	github.com/carlosframework/platform/internal/hostresolve	0.141s
ok  	github.com/carlosframework/platform/internal/hostwrite	0.013s
ok  	github.com/carlosframework/platform/internal/infraview	0.016s
ok  	github.com/carlosframework/platform/internal/instancectl	4.170s
ok  	github.com/carlosframework/platform/internal/instanceexec	2.726s
ok  	github.com/carlosframework/platform/internal/instancefs	0.018s
ok  	github.com/carlosframework/platform/internal/instancekey	0.015s
ok  	github.com/carlosframework/platform/internal/instanceupdates	0.022s
ok  	github.com/carlosframework/platform/internal/layout	1.525s
ok  	github.com/carlosframework/platform/internal/lease	9.808s
ok  	github.com/carlosframework/platform/internal/lease/leasetest	0.018s
ok  	github.com/carlosframework/platform/internal/ledger	0.010s
ok  	github.com/carlosframework/platform/internal/mail	0.163s
ok  	github.com/carlosframework/platform/internal/maintenance	0.083s
ok  	github.com/carlosframework/platform/internal/maintenanceexec	5.509s
ok  	github.com/carlosframework/platform/internal/maintenanceguard	0.070s
ok  	github.com/carlosframework/platform/internal/metrics	0.007s
ok  	github.com/carlosframework/platform/internal/moves	21.003s
ok  	github.com/carlosframework/platform/internal/moves/archive	0.788s
ok  	github.com/carlosframework/platform/internal/objstore	0.177s
ok  	github.com/carlosframework/platform/internal/ops	0.178s
ok  	github.com/carlosframework/platform/internal/park	0.191s
ok  	github.com/carlosframework/platform/internal/privatefile	0.008s
ok  	github.com/carlosframework/platform/internal/provision	17.221s
ok  	github.com/carlosframework/platform/internal/publicrelease	0.007s
?   	github.com/carlosframework/platform/internal/regionname	[no test files]
ok  	github.com/carlosframework/platform/internal/registry	4.927s
ok  	github.com/carlosframework/platform/internal/relay	7.236s
ok  	github.com/carlosframework/platform/internal/release	19.572s
ok  	github.com/carlosframework/platform/internal/rollout	3.820s
ok  	github.com/carlosframework/platform/internal/rolloutwords	0.003s
ok  	github.com/carlosframework/platform/internal/rootaws	0.006s
ok  	github.com/carlosframework/platform/internal/rootparticipation	0.007s
ok  	github.com/carlosframework/platform/internal/rootstate	0.025s
ok  	github.com/carlosframework/platform/internal/runs	0.579s
ok  	github.com/carlosframework/platform/internal/s3compat	0.005s
ok  	github.com/carlosframework/platform/internal/safefile	0.004s
ok  	github.com/carlosframework/platform/internal/schedule	2.504s
ok  	github.com/carlosframework/platform/internal/seeds	0.012s
ok  	github.com/carlosframework/platform/internal/sim/hostlife	9.625s
ok  	github.com/carlosframework/platform/internal/sim/litestreamfacts	3.653s
ok  	github.com/carlosframework/platform/internal/status	0.037s
ok  	github.com/carlosframework/platform/internal/steering	0.006s
ok  	github.com/carlosframework/platform/internal/stopevidence	0.181s
ok  	github.com/carlosframework/platform/internal/svcrole	0.002s
ok  	github.com/carlosframework/platform/internal/telemetry	0.004s
ok  	github.com/carlosframework/platform/internal/termfence	0.003s
ok  	github.com/carlosframework/platform/internal/tunnelclient	0.004s
ok  	github.com/carlosframework/platform/internal/unixdial	0.002s
ok  	github.com/carlosframework/platform/internal/updatewords	0.004s
ok  	github.com/carlosframework/platform/internal/usage	0.004s
ok  	github.com/carlosframework/platform/internal/version	0.001s
ok  	github.com/carlosframework/platform/internal/vet	1.565s
=== cross ===
--- shard 0 of 5 ---
cross-compiling darwin/arm64
--- shard 1 of 5 ---
cross-compiling darwin/amd64
--- shard 2 of 5 ---
cross-compiling linux/arm64
linux/arm64 carlos binary: 47055010 bytes (ceiling 52000000)
--- shard 3 of 5 ---
cross-compiling windows/amd64
--- shard 4 of 5 ---
cross-compiling windows/arm64
=== minio-contract ===
minio contract: arm64, minio RELEASE.2025-09-07T16-13-09Z, mc RELEASE.2025-08-13T08-35-41Z
  minio: cached, digest verified
  mc: cached, digest verified
  minio healthy on 127.0.0.1:20999
Added `local` successfully.
Bucket created successfully `local/contract-bucket`.
Bucket created successfully `local/denied-bucket`.
Created policy `contract-only` successfully.
Added user `scopeduser` successfully.
Attached Policies: [contract-only]
To User: scopeduser
  buckets and scoped user ready
=== RUN   TestDeliveredContractMinio
=== RUN   TestDeliveredContractMinio/put
=== RUN   TestDeliveredContractMinio/get_round-trips_the_body
=== RUN   TestDeliveredContractMinio/list_shows_the_key
=== RUN   TestDeliveredContractMinio/head_and_list_agree_on_the_version
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed
=== RUN   TestDeliveredContractMinio/delete_removes_it
=== RUN   TestDeliveredContractMinio/cross-bucket_denial
--- PASS: TestDeliveredContractMinio (0.02s)
    --- PASS: TestDeliveredContractMinio/put (0.00s)
    --- PASS: TestDeliveredContractMinio/get_round-trips_the_body (0.00s)
    --- PASS: TestDeliveredContractMinio/list_shows_the_key (0.00s)
    --- PASS: TestDeliveredContractMinio/head_and_list_agree_on_the_version (0.00s)
    --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload (0.00s)
    --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route (0.01s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed (0.00s)
    --- PASS: TestDeliveredContractMinio/delete_removes_it (0.00s)
    --- PASS: TestDeliveredContractMinio/cross-bucket_denial (0.00s)
PASS
ok  	github.com/carlosframework/platform/internal/objstore	0.022s
=== RUN   TestT4StorageProbeMinio
2026/10/16 00:36:18 moves: T4: STORE: list: operation error S3: ListObjectsV2, https response error StatusCode: 403, RequestID: 18DEDB709D4FEF87, HostID: 7c420053e0fbcd7f41653fdd92ffa2802c903d05303b1aa15adc2e8bb65d183f, api error SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.
--- PASS: TestT4StorageProbeMinio (0.01s)
PASS
ok  	github.com/carlosframework/platform/internal/host	0.020s
minio contract: PASS
diff --git a/approved_moves_copy_test.go b/approved_moves_copy_test.go
index 5793d69b2..2b6af98a8 100644
--- a/approved_moves_copy_test.go
+++ b/approved_moves_copy_test.go
@@ -89,6 +89,7 @@ var movesCopyIDs = []string{
"MV-N440", "MV-N441", "MV-N442", "MV-N443",
"MV-N450", "MV-N451", "MV-N454", "MV-N455", "MV-N456",
"MV-N500",
+ "MV-N501", "MV-N502",
}
// copyRow is one row of the record. The File cell is the file the sentence is
diff --git a/docs/superpowers/plans/2026-10-06-moves-phase1-copy.md b/docs/superpowers/plans/2026-10-06-moves-phase1-copy.md
index e44720f29..f01e150c8 100644
--- a/docs/superpowers/plans/2026-10-06-moves-phase1-copy.md
+++ b/docs/superpowers/plans/2026-10-06-moves-phase1-copy.md
@@ -448,3 +448,17 @@ Paul's copy review of 2026-10-12 (region HA stage B, final review minor 9). The
| MV-N500 | internal/console/apiinstancemove.go | the instance's last relocation is still finishing. try again in a few minutes | approved |
The five rows added after Plan 12 (MV-N450, MV-N451, MV-N454, MV-N455 and MV-N456) were approved by Paul on 2026-10-13 with no edits.
+
+## Files manifest past the path cap
+
+Paul's ruling of 2026-10-15 on amadan #236 (Astra's P2 on a8cebad). The files readers refuse, terminally, a manifest with an entry path longer than 4095 bytes (`hostfiles.ErrTooDeep`), and a manifest written before that cap can still hold one. A recovery after the cut that selected it could not restore the source, so the source's round refuses the move before any staging write, stop or cut, and writes MV-N501 into S1, where `carlos move status` shows it. MV-N501 is the first number past MV-N500.
+
+| id | file | text | status |
+|---|---|---|---|
+| MV-N501 | internal/host/movesphase1.go | <host> has a files manifest with a path longer than 4095 bytes, so it cannot move. an operator must replace the manifest | approved |
+
+Fix round 1 (Astra's second P2 on #236). The same cap applies to the files directory itself. The archive format carries a path of up to 4096 bytes (MV-N81), but the target stages every file into a prepared files manifest after the cut, and that refuses a path longer than 4095 bytes, so the landing would fail. The source's archive now refuses a path of exactly 4096 bytes at S1 with MV-N502, before the cut. A longer path still gets MV-N81. MV-N501 names a manifest, not a file, so it does not fit here.
+
+| id | file | text | status |
+|---|---|---|---|
+| MV-N502 | internal/host/movesphase1.go | <host> has a file whose path is longer than 4095 bytes, so it cannot move. shorten the path or remove the file | approved |
diff --git a/docs/superpowers/plans/2026-10-06-moves-phase1-rehearsal.md b/docs/superpowers/plans/2026-10-06-moves-phase1-rehearsal.md
index 8ca66882b..f095dfd5b 100644
--- a/docs/superpowers/plans/2026-10-06-moves-phase1-rehearsal.md
+++ b/docs/superpowers/plans/2026-10-06-moves-phase1-rehearsal.md
@@ -548,8 +548,16 @@ reached the stopped position). The target box logs both:
`eu-west-2`, box `carlos-dev-uk`, data bucket
`carlos-dev-uk-640012953448` (platform-infrastructure#42). Before Run 1:
-- [ ] Bootstrap `carlos-dev-uk`.
-- [ ] Copy the SSM parameters a dev box reads at boot to `eu-west-2`.
+- [x] Bootstrap `carlos-dev-uk`. Done 2026-10-15: box `dennis-stenciling`
+ on v0.27.3, with `CARLOS_SWAP=overlap` and `CARLOS_REGION=uk`.
+- [x] SSM parameters: nothing to copy. No box reads SSM at boot or at
+ runtime. Never copy `/carlos/release-key` to a region whose host role can
+ read `/carlos/*`. In `eu-west-2` the `carlos-dev-uk` role can.
+- [x] `carlos-dev-uk` needs the dev console's `CARLOS_SECRETS_KEY` before
+ Run 1 (T2 and config sync). Done 2026-10-15: `dennis-stenciling` got the
+ dev key through SSM `eu-west-2` `/carlos/secrets-key`. Its
+ `secrets_key_fp` is `sha256:bb067582…f5fe`, the same as sophie's. The key
+ is backed up in 1Password as "CARLOS_SECRETS_KEY dev".
- [ ] `CARLOS_REGION_HA=true` on `carlos-dev-ie` and `carlos-dev-uk`, and
`CARLOS_AWS_ACCOUNT` on the dev console.
- [ ] Keep `carlos-dev-ie-b` ineligible: no `CARLOS_REGION_HA` in its
diff --git a/internal/host/claimfilesseed_reboot_rootrunner_linux_test.go b/internal/host/claimfilesseed_reboot_rootrunner_linux_test.go
index 37c8a6460..5035026bc 100644
--- a/internal/host/claimfilesseed_reboot_rootrunner_linux_test.go
+++ b/internal/host/claimfilesseed_reboot_rootrunner_linux_test.go
@@ -3,6 +3,7 @@
package host
import (
+ "bytes"
"context"
"crypto/ed25519"
"encoding/json"
@@ -105,20 +106,7 @@ func saveFilesSeedReboot(t *testing.T, binary string, hosts []*seedLiveHost) {
t.Fatal(err)
}
if r.Authorization.Successor.Host == host && r.Parked && !r.Closed {
- if strings.Contains(t.Name(), "Renumber") {
- // Loop devices have no independent volume serial. Expire the
- // retained park before reboot; FS-R7 forbids its deletion after
- // renumber. The ordinary EBS pair retains its parks across boot.
- f.f.now = r.ReapAfter.Add(time.Second)
- if err := f.p.ledger.withNamespace(t.Context(), func() error { return f.p.ledger.reapLocked(t.Context(), r.Authorization.ID) }); err != nil {
- t.Fatal("expire loop park before reboot", err)
- }
- r, err = f.p.ledger.readHistorical(r.Authorization.ID)
- if err != nil || !r.Closed {
- t.Fatal("expired loop park", err)
- }
- t.Log("loop park expired before reboot", host)
- }
+
sh.Records = append(sh.Records, r)
}
}
@@ -210,6 +198,24 @@ func TestRootRebootFilesSeedTwoHostPhase2(t *testing.T) {
}
}
}
+ retained := map[string][]byte{}
+ if strings.Contains(t.Name(), "Renumber") {
+ for _, host := range saved.Hosts {
+ for _, r := range host.Records {
+ for _, file := range r.Source.Files {
+ if !file.Present {
+ continue
+ }
+ path := filepath.Join("/data/instances/.serving-parked", r.Authorization.ID, file.Name)
+ body, err := os.ReadFile(path)
+ if err != nil {
+ t.Fatal(err)
+ }
+ retained[path] = body
+ }
+ }
+ }
+ }
var hosts []*seedLiveHost
for _, savedHost := range saved.Hosts {
h := reopenSeedLiveHost(t, savedHost, witness, state)
@@ -245,6 +251,35 @@ func TestRootRebootFilesSeedTwoHostPhase2(t *testing.T) {
h.probe(t)
h.stop()
}
+ for path, want := range retained {
+ got, err := os.ReadFile(path)
+ if err != nil || !bytes.Equal(got, want) {
+ t.Fatalf("retained park changed after two wakes: %s %v", path, err)
+ }
+ }
+ if len(retained) != 0 {
+ ids, more, err := ledger.recoveryIDs(t.Context(), "", servingParkRecoveryLimit)
+ if err != nil || more {
+ t.Fatal("retained census", err)
+ }
+ count := 0
+ for _, host := range saved.Hosts {
+ for _, old := range host.Records {
+ found := false
+ for _, id := range ids {
+ if id == old.Authorization.ID {
+ found = true
+ }
+ }
+ r, err := ledger.readHistorical(old.Authorization.ID)
+ if !found || err != nil || r.Closed || !r.ReapAfter.Equal(old.ReapAfter) {
+ t.Fatal("lost retained record", r, err)
+ }
+ count++
+ }
+ }
+ t.Logf("%d unverifiable parks retained after two live wakes per host; %d records counted", count, len(ids))
+ }
}
func reopenSeedLiveHost(t *testing.T, saved seedRebootHost, witness filesSeedLiveWitness, state rootstate.Store) *seedLiveHost {
diff --git a/internal/host/claimfilesseed_renumber_rootrunner_linux_test.go b/internal/host/claimfilesseed_renumber_rootrunner_linux_test.go
index d6a54c6bd..3249a63e3 100644
--- a/internal/host/claimfilesseed_renumber_rootrunner_linux_test.go
+++ b/internal/host/claimfilesseed_renumber_rootrunner_linux_test.go
@@ -102,3 +102,10 @@ func TestRootServingHandoverDirectoryProof(t *testing.T) {
servingRootRunner(t)
TestServingHandoverRebootDirectoryProof(t)
}
+
+func TestRootServingParkRetainedDoesNotBlockRetirement(t *testing.T) {
+ servingRootRunner(t)
+ TestServingParkRetainedDoesNotBlockRetirement(t)
+ TestServingParkRetainedReplacementWriteFailures(t)
+ TestServingParkQuarantineRetainedParkAllowsRetirement(t)
+}
diff --git a/internal/host/claimservingpark_records.go b/internal/host/claimservingpark_records.go
index 929b52ea1..7a92c13e4 100644
--- a/internal/host/claimservingpark_records.go
+++ b/internal/host/claimservingpark_records.go
@@ -8,6 +8,7 @@ import (
"encoding/json"
"errors"
"io"
+ "log"
"os"
"reflect"
"time"
@@ -315,7 +316,7 @@ func (l *servingParkLedger) slot(a servingParkAuthorization) (servingParkSlot, e
return s, hostbox.ErrChanged
}
if saved.ID != "" {
- r, err := l.read(saved.ID)
+ r, err := l.readHistorical(saved.ID)
if err != nil || r.Disposition != servingParkRetain || r.Authorization.Successor.Box != s.Box || r.Authorization.Successor.Host != s.Host {
return s, hostbox.ErrChanged
}
@@ -327,7 +328,7 @@ func (l *servingParkLedger) slot(a servingParkAuthorization) (servingParkSlot, e
}
}
if saved.Last != "" {
- r, err := l.read(saved.Last)
+ r, err := l.readHistorical(saved.Last)
if err != nil || r.Authorization.Successor.Box != s.Box || r.Authorization.Successor.Host != s.Host {
return s, hostbox.ErrChanged
}
@@ -423,7 +424,7 @@ func (l *servingParkLedger) parkWithEvidenceLocked(ctx context.Context, a servin
// intent from a restored source in the same incarnation. The outer root
// guard independently requires the current incarnation and successor.
if slot.Last != "" {
- last, e := l.read(slot.Last)
+ last, e := l.readHistorical(slot.Last)
if e != nil {
return zero, e
}
@@ -666,7 +667,7 @@ func (l *servingParkLedger) replaceLocked(ctx context.Context, id string) error
return err
}
defer unlock()
- old, err := l.read(id)
+ old, err := l.readHistorical(id)
if err != nil || !old.Complete {
return hostbox.ErrChanged
}
@@ -677,7 +678,14 @@ func (l *servingParkLedger) replaceLocked(ctx context.Context, id string) error
if old.Deleting {
cause = old.Cause
}
- return l.deleteLocked(ctx, &old, cause)
+ current, err := l.read(id)
+ if err == nil {
+ err = l.deleteLocked(ctx, &current, cause)
+ }
+ if err != nil {
+ log.Printf("Host %q: serving database retirement %q retained: %s.", old.Authorization.Successor.Host, id, err)
+ }
+ return ctx.Err()
}
func sameServingRootStops(a, b []hostguard.FenceReceipt) bool {
diff --git a/internal/host/claimservingquarantine_linux_test.go b/internal/host/claimservingquarantine_linux_test.go
index 7cab7c29d..a0524cb6e 100644
--- a/internal/host/claimservingquarantine_linux_test.go
+++ b/internal/host/claimservingquarantine_linux_test.go
@@ -546,7 +546,7 @@ func TestServingParkQuarantineRefusesUnsafeAuthority(t *testing.T) {
}
}
-func TestServingParkQuarantinePreservesBlockedSlot(t *testing.T) {
+func TestServingParkQuarantineRetainedParkAllowsRetirement(t *testing.T) {
f := newServingParkDiskFixture(t)
f.put("", "old served database")
a, stops := f.evidence(10, true, "missing")
@@ -576,20 +576,26 @@ func TestServingParkQuarantinePreservesBlockedSlot(t *testing.T) {
quarantinePreserved(t, f, q)
f.put("", "new served database")
next, receipts := f.evidence(11, false, "started")
- if _, err := f.park(next, receipts); err == nil {
- t.Fatal("unresolved slot replaced")
+ parked, err := f.park(next, receipts)
+ if err != nil {
+ t.Fatal("retained quarantine blocked retirement", err)
}
- slot, err := f.ledger.slot(a)
- if err != nil || slot.ID != a.ID {
- t.Fatal("blocked slot changed", err)
+ f.complete(next)
+ f.retained(next, "new served database", parked.ReapAfter)
+ old, err := f.ledger.readHistorical(a.ID)
+ if err != nil || old.Closed || !old.ReapAfter.Equal(r.ReapAfter) {
+ t.Fatal("lost retained quarantine record", old, err)
}
body, err := os.ReadFile(filepath.Join(path+"-saved", a.ID, a.Successor.Host+".db"))
if err != nil || string(body) != "old served database" {
t.Fatal("old park changed", err)
}
- body, err = os.ReadFile(provision.NestedDBPath(f.data, f.base.Host))
- if err != nil || string(body) != "new served database" {
- t.Fatal("blocked replacement changed live source", err)
+ body, err = os.ReadFile(filepath.Join(path+"-saved", a.ID, "unknown"))
+ if err != nil || string(body) != "retain" {
+ t.Fatal("unknown retained child changed", err)
+ }
+ if _, err = os.Stat(provision.NestedDBPath(f.data, f.base.Host)); !errors.Is(err, os.ErrNotExist) {
+ t.Fatal("new retirement left its source", err)
}
f.base.Host, f.target.Host = "other.example", "other.example"
if err := os.MkdirAll(filepath.Dir(provision.NestedDBPath(f.data, f.base.Host)), 0700); err != nil {
diff --git a/internal/host/claimservingretained_linux_test.go b/internal/host/claimservingretained_linux_test.go
new file mode 100644
index 000000000..ccdc0d332
--- /dev/null
+++ b/internal/host/claimservingretained_linux_test.go
@@ -0,0 +1,117 @@
+//go:build linux
+
+package host
+
+import (
+ "bytes"
+ "fmt"
+ "log"
+ "os"
+ "testing"
+
+ "github.com/carlosframework/platform/internal/instancectl"
+)
+
+func TestServingParkRetainedDoesNotBlockRetirement(t *testing.T) {
+ for _, attack := range []string{"unreadable-volume", "changed-volume", "hardlink", "unreadable-binding"} {
+ t.Run(attack, func(t *testing.T) {
+ f, old := servingRebootRecordFixture(t, "complete")
+ servingRenumberHistory(t, f, &old)
+ if attack == "unreadable-volume" {
+ f.fs.volume = func(uint64) string { return "" }
+ }
+ if attack == "changed-volume" {
+ f.fs.volume = func(uint64) string { return "vol11111111111111111" }
+ }
+ root := &servingParkRoot{ledger: f.ledger}
+ if err := root.commitServingReboot(t.Context(), old.Authorization.ID); err != nil {
+ t.Fatal(err)
+ }
+ if attack == "hardlink" {
+ if err := os.Link(f.path(old.Authorization, ""), f.path(old.Authorization, "")+".extra"); err != nil {
+ t.Fatal(err)
+ }
+ }
+ if attack == "unreadable-binding" {
+ if err := f.disk.Write("current-retirement-"+old.Authorization.ID+".json", []byte("invalid")); err != nil {
+ t.Fatal(err)
+ }
+ }
+ var report bytes.Buffer
+ output := log.Writer()
+ log.SetOutput(&report)
+ defer log.SetOutput(output)
+ for n := uint64(2); n <= 3; n++ {
+ body := fmt.Sprintf("tenant writes %d", n)
+ f.put("", body)
+ next, stops := f.evidence(n, false, "started")
+ parked, err := f.park(next, stops)
+ if err != nil {
+ t.Fatalf("retained park blocked retirement %d: %v", n, err)
+ }
+ f.complete(next)
+ f.retained(next, body, parked.ReapAfter)
+ }
+ saved, err := f.ledger.readHistorical(old.Authorization.ID)
+ if err != nil || saved.Closed || !saved.ReapAfter.Equal(old.ReapAfter) {
+ t.Fatal("lost retained record", saved, err)
+ }
+ for suffix, want := range map[string]string{"": "last database", "-wal": "last WAL", "-shm": "last SHM"} {
+ got, err := os.ReadFile(f.path(old.Authorization, suffix))
+ if err != nil || string(got) != want {
+ t.Fatalf("retained %s: %q %v", suffix, got, err)
+ }
+ }
+ ids, more, err := f.ledger.recoveryIDs(t.Context(), "", servingParkRecoveryLimit)
+ if err != nil || more || len(ids) != 3 {
+ t.Fatal("retained record missing from census", ids, more, err)
+ }
+ if !bytes.Contains(report.Bytes(), []byte(old.Authorization.ID)) {
+ t.Fatal("retained park not reported", report.String())
+ }
+ if err := f.ledger.maintainParks(t.Context(), func() error { return instancectl.ErrDiskFull }); err == nil {
+ t.Fatal("pressure granted deletion of retained park")
+ }
+ if _, err := os.Stat(f.path(old.Authorization, "")); err != nil {
+ t.Fatal("pressure lost retained park", err)
+ }
+ })
+ }
+}
+
+func TestServingParkRetainedReplacementWriteFailures(t *testing.T) {
+ for _, before := range []bool{false, true} {
+ for boundary := 1; boundary <= 18; boundary++ {
+ t.Run(fmt.Sprintf("before-%t-write-%d", before, boundary), func(t *testing.T) {
+ f, old := servingRebootRecordFixture(t, "complete")
+ servingRenumberHistory(t, f, &old)
+ f.fs.volume = func(uint64) string { return "" }
+ root := &servingParkRoot{ledger: f.ledger}
+ if err := root.commitServingReboot(t.Context(), old.Authorization.ID); err != nil {
+ t.Fatal(err)
+ }
+ next, stops := f.evidence(2, false, "started")
+ faults := &servingParkFaultStore{Store: f.disk, failAt: boundary, before: before}
+ f.ledger.state, f.fs.state = faults, faults
+ if _, err := f.park(next, stops); err == nil {
+ _ = f.ledger.withNamespace(t.Context(), func() error { return f.ledger.completeLocked(t.Context(), next.ID, next.Successor) })
+ }
+ f.ledger.state, f.fs.state = f.disk, f.disk
+ parked, err := f.park(next, stops)
+ if err != nil {
+ t.Fatal("retirement retry", err)
+ }
+ f.complete(next)
+ f.retained(next, "successor database", parked.ReapAfter)
+ got, err := os.ReadFile(f.path(old.Authorization, ""))
+ if err != nil || string(got) != "last database" {
+ t.Fatalf("retained bytes after retry: %q %v", got, err)
+ }
+ saved, err := f.ledger.readHistorical(old.Authorization.ID)
+ if err != nil || saved.Closed || !saved.ReapAfter.Equal(old.ReapAfter) {
+ t.Fatal("retained record after retry", saved, err)
+ }
+ })
+ }
+ }
+}
diff --git a/internal/host/movesfiles.go b/internal/host/movesfiles.go
index 58ff91b25..faf0ac08a 100644
--- a/internal/host/movesfiles.go
+++ b/internal/host/movesfiles.go
@@ -250,17 +250,29 @@ func buildPrepared(ctx context.Context, s release.Store, location, generation, o
if err != nil {
return preparedManifest{}, err
}
- err = filepath.WalkDir(dir, func(path string, d fs.DirEntry, err error) error {
+ // The tree is read through an os.Root over dir, every name relative to
+ // it (#236 fix round 1): a path within the files tree's cap (4095 bytes,
+ // which the archive the install extracted holds every path to) is past
+ // PATH_MAX once the data directory is prefixed, so an absolute walk
+ // fails with ENAMETOOLONG after the cut. The root also keeps every read
+ // inside dir.
+ //
+ // Census ruling moves-overcap-cut-236 (platform-1e, 2026-10-15;
+ // binding): no read here falls back to a path-based open. A root that
+ // cannot be opened, or a name it cannot open (readFileBounded), refuses
+ // the build; nothing retries by absolute path.
+ root, err := os.OpenRoot(dir)
+ if err != nil {
+ return preparedManifest{}, fmt.Errorf("%w: the files directory %s the install placed: %v", errLandingChanged, dir, err)
+ }
+ defer root.Close()
+ err = fs.WalkDir(root.FS(), ".", func(rel string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
- if path == dir || d.IsDir() || !d.Type().IsRegular() {
+ if rel == "." || d.IsDir() || !d.Type().IsRegular() {
return nil
}
- rel, err := filepath.Rel(dir, path)
- if err != nil {
- return err
- }
info, err := d.Info()
if err != nil {
return err
@@ -270,16 +282,16 @@ func buildPrepared(ctx context.Context, s release.Store, location, generation, o
// its stat or by growing while it is read, is recorded as skipped
// and never held whole in memory.
if info.Size() > hostfiles.MaxFileBytes {
- return p.StageSkipped(filepath.ToSlash(rel), movesFilesTooLarge(info.Size()))
+ return p.StageSkipped(rel, movesFilesTooLarge(info.Size()))
}
- body, tooLarge, err := readFileBounded(path, hostfiles.MaxFileBytes)
+ body, tooLarge, err := readFileBounded(root, rel, hostfiles.MaxFileBytes)
if err != nil {
return err
}
if tooLarge {
- return p.StageSkipped(filepath.ToSlash(rel), movesFilesTooLarge(int64(len(body))))
+ return p.StageSkipped(rel, movesFilesTooLarge(int64(len(body))))
}
- return p.Stage(ctx, filepath.ToSlash(rel), body, uint32(info.Mode().Perm()))
+ return p.Stage(ctx, rel, body, uint32(info.Mode().Perm()))
})
if err != nil {
return preparedManifest{}, fmt.Errorf("prepared manifest of %s: %w", dir, err)
@@ -313,12 +325,15 @@ func movesFilesTooLarge(size int64) string {
return fmt.Sprintf("over the %d-byte limit (%d bytes or more)", hostfiles.MaxFileBytes, size)
}
-// readFileBounded reads path whole when it holds at most limit bytes. It
-// reads at most limit+1 bytes however large the file is or becomes while
-// it is read; tooLarge reports that the file exceeded the limit, and the
-// bytes returned with it are then not the whole file. A test seam.
-var readFileBounded = func(path string, limit int64) ([]byte, bool, error) {
- f, err := os.Open(path)
+// readFileBounded reads rel, relative to root, whole when it holds at most
+// limit bytes. It reads at most limit+1 bytes however large the file is or
+// becomes while it is read; tooLarge reports that the file exceeded the
+// limit, and the bytes returned with it are then not the whole file. A
+// test seam.
+var readFileBounded = func(root *os.Root, rel string, limit int64) ([]byte, bool, error) {
+ // Through the root only (census ruling moves-overcap-cut-236, binding):
+ // an open that fails is the error, never a retry by path.
+ f, err := root.Open(rel)
if err != nil {
return nil, false, err
}
@@ -406,7 +421,14 @@ func movesFilesChainFrom(h *hostclaim.Holder, start, home, region string) []stri
// names it: "" when the key has no current object. A delete marker or an
// unversioned current object is an error.
func (f *movesFiles) manifestVersion(ctx context.Context, key string) (string, error) {
- vs, err := f.Data.ListVersions(ctx, f.Bucket, key)
+ return movesManifestVersion(ctx, f.Data, f.Bucket, key)
+}
+
+// movesManifestVersion is manifestVersion over any versioned reader of the
+// data bucket: the cut's preflight reads with the moves worker's own
+// (sourceFilesRefusal), the establishment with the claim's.
+func movesManifestVersion(ctx context.Context, data lease.VersionedStore, bucket, key string) (string, error) {
+ vs, err := data.ListVersions(ctx, bucket, key)
if err != nil {
return "", err
}
@@ -430,30 +452,58 @@ func (f *movesFiles) manifestVersion(ctx context.Context, key string) (string, e
// one in its place). found is false when no generation of the chain has a
// manifest.
func (f *movesFiles) newestManifest(ctx context.Context, o hostclaim.Observed, in hostclaim.Admission, location string, chain []string) (hostfiles.Manifest, []string, bool, error) {
+ m, key, version, found, err := movesFilesSelect(ctx, f.Data, f.Bucket, location, chain)
+ if err != nil || !found {
+ return hostfiles.Manifest{}, nil, false, err
+ }
+ allowed, err := f.movesFilesAllowed(ctx, o, in, location, in.NewGeneration, m)
+ if err != nil {
+ return hostfiles.Manifest{}, nil, false, fmt.Errorf("host: the files manifest %s@%s: %w", key, version, err)
+ }
+ return m, allowed, true, nil
+}
+
+// movesFilesSelect is newestManifest's selection, apart from its
+// validation: the first generation of chain whose manifest key has a
+// current version, read at exactly that version, with the key and the
+// version. found is false when no generation of chain has a manifest. The
+// establishment (newestManifest, the src_gen bridge included) and the
+// cut's preflight (sourceFilesRefusal) both select through it, so the cut
+// checks the manifest a recovery would restore, not a guess at it (#236).
+func movesFilesSelect(ctx context.Context, data lease.VersionedStore, bucket, location string, chain []string) (m hostfiles.Manifest, key, version string, found bool, err error) {
for _, g := range chain {
prefix := movesFilesPrefix(location, g)
key := hostfiles.ManifestKey(prefix)
- version, err := f.manifestVersion(ctx, key)
+ version, err := movesManifestVersion(ctx, data, bucket, key)
if err != nil {
- return hostfiles.Manifest{}, nil, false, err
+ return hostfiles.Manifest{}, "", "", false, err
}
if version == "" {
continue
}
- m, found, err := hostfiles.Load(ctx, versionStore{data: f.Data, bucket: f.Bucket, key: key, version: version}, prefix)
+ m, found, err := hostfiles.Load(ctx, versionStore{data: data, bucket: bucket, key: key, version: version}, prefix)
if err != nil {
- return hostfiles.Manifest{}, nil, false, err
+ return hostfiles.Manifest{}, "", "", false, err
}
if !found {
- return hostfiles.Manifest{}, nil, false, fmt.Errorf("host: the files manifest %s@%s is not readable", key, version)
- }
- allowed, err := f.movesFilesAllowed(ctx, o, in, location, in.NewGeneration, m)
- if err != nil {
- return hostfiles.Manifest{}, nil, false, fmt.Errorf("host: the files manifest %s@%s: %w", key, version, err)
+ return hostfiles.Manifest{}, "", "", false, fmt.Errorf("host: the files manifest %s@%s is not readable", key, version)
}
- return m, allowed, true, nil
+ return m, key, version, true, nil
}
- return hostfiles.Manifest{}, nil, false, nil
+ return hostfiles.Manifest{}, "", "", false, nil
+}
+
+// movesFilesRestorable is the reader's path cap as moves applies it
+// (hostfiles.RestorableWithinCap, the rule checkManifest applies to every
+// entry, live, tombstoned or skipped): a manifest that fails it is
+// hostfiles.ErrTooDeep, and no establishment restores it. The recovery's
+// establishment (movesFilesAllowed) and the cut's preflight
+// (sourceFilesRefusal) both call it, so a move whose recovery could not
+// restore the source's files never starts its cut (#236; Paul's ruling of
+// 2026-10-15). If region HA relaxes the readers' rule, RestorableWithinCap
+// follows it, and both sides here with it.
+func movesFilesRestorable(m hostfiles.Manifest) error {
+ return hostfiles.RestorableWithinCap(m)
}
// movesOriginProof proves that generations of the holder's incarnation
@@ -533,6 +583,13 @@ func (f *movesFiles) proveOrigins(ctx context.Context, o hostclaim.Observed, in
// before; only unreferenced hints and unreferenced chain generations, which
// no entry depends on, are no longer asked about.
func (f *movesFiles) movesFilesAllowed(ctx context.Context, o hostclaim.Observed, in hostclaim.Admission, location, admitted string, m hostfiles.Manifest) ([]string, error) {
+ // The path cap first, by the rule the cut's preflight applies
+ // (movesFilesRestorable): ValidateManifest below refuses the same
+ // manifest with the same error, so this only refuses it before any
+ // origin proof, and keeps the two sides on one function.
+ if err := movesFilesRestorable(m); err != nil {
+ return nil, err
+ }
named, err := movesFilesOrigins(location, "", m)
if err != nil {
return nil, err
diff --git a/internal/host/movesfiles_test.go b/internal/host/movesfiles_test.go
index 91ff1f834..b2b3ea3ba 100644
--- a/internal/host/movesfiles_test.go
+++ b/internal/host/movesfiles_test.go
@@ -8,6 +8,7 @@ import (
"errors"
"fmt"
"os"
+ "path"
"path/filepath"
"reflect"
"sort"
@@ -474,11 +475,11 @@ func TestPreparedBuildBoundsEveryRead(t *testing.T) {
x.writeEnv("K=1\n")
reads := 0
orig := readFileBounded
- readFileBounded = func(path string, limit int64) ([]byte, bool, error) {
- if filepath.Base(path) == "big.bin" {
+ readFileBounded = func(root *os.Root, rel string, limit int64) ([]byte, bool, error) {
+ if path.Base(rel) == "big.bin" {
reads++
}
- return orig(path, limit)
+ return orig(root, rel, limit)
}
defer func() { readFileBounded = orig }()
if _, err := x.f.BuildPrepared(ctx, filesLoc, "g1", "g1", 7, dir, x.envPath(), true, hostfiles.WakeState{V: 1, Host: filesHost, Timers: []hostfiles.WakeTimer{}}); err != nil {
@@ -501,11 +502,16 @@ func TestPreparedBuildBoundsEveryRead(t *testing.T) {
if err := os.WriteFile(small, []byte("0123456789"), 0o600); err != nil {
t.Fatal(err)
}
- body, tooLarge, err := orig(small, 4)
+ xroot, err := os.OpenRoot(x.dir)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer xroot.Close()
+ body, tooLarge, err := orig(xroot, "small", 4)
if err != nil || !tooLarge || len(body) > 5 {
t.Fatalf("over the limit: %q %v %v", body, tooLarge, err)
}
- body, tooLarge, err = orig(small, 10)
+ body, tooLarge, err = orig(xroot, "small", 10)
if err != nil || tooLarge || string(body) != "0123456789" {
t.Fatalf("at the limit: %q %v %v", body, tooLarge, err)
}
diff --git a/internal/host/movesfilescap_test.go b/internal/host/movesfilescap_test.go
new file mode 100644
index 000000000..4380028a7
--- /dev/null
+++ b/internal/host/movesfilescap_test.go
@@ -0,0 +1,539 @@
+package host
+
+import (
+ "bytes"
+ "context"
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io/fs"
+ "os"
+ "path"
+ "path/filepath"
+ "reflect"
+ "slices"
+ "strings"
+ "sync"
+ "testing"
+ "time"
+
+ "github.com/carlosframework/platform/internal/hostbox"
+ "github.com/carlosframework/platform/internal/hostclaim"
+ "github.com/carlosframework/platform/internal/hostfiles"
+ "github.com/carlosframework/platform/internal/lease"
+ "github.com/carlosframework/platform/internal/moves"
+ "github.com/carlosframework/platform/internal/moves/archive"
+ "github.com/carlosframework/platform/internal/provision"
+)
+
+// The cut's files preflight (#236; Paul's ruling of 2026-10-15): a source
+// whose files manifest a recovery could not restore, because an entry path
+// is past the readers' cap, is refused at S1 before anything is staged,
+// stopped or cut.
+
+// capPath is a clean relative path of exactly n bytes: 200-byte segments
+// joined by "/", as hostfiles' own tests build them.
+func capPath(n int) string {
+ var b strings.Builder
+ for b.Len() < n {
+ if b.Len() > 0 {
+ b.WriteByte('/')
+ }
+ b.WriteString(strings.Repeat("x", min(200, n-b.Len())))
+ }
+ return b.String()
+}
+
+// capCase is one manifest shape: its one entry's kind (live, tombstone or
+// skipped) at a path of the given length.
+type capCase struct {
+ kind string
+ n int
+}
+
+// capCases are the shapes every test here runs: an ordinary manifest, a
+// live entry at the cap, and each kind of entry one byte past it.
+var capCases = map[string]capCase{
+ "an ordinary manifest": {"live", len("a.txt")},
+ "a live entry at the cap": {"live", hostfiles.MaxPathBytes},
+ "a live entry past it": {"live", hostfiles.MaxPathBytes + 1},
+ "a tombstone past it": {"tombstone", hostfiles.MaxPathBytes + 1},
+ "a skipped entry past it": {"skipped", hostfiles.MaxPathBytes + 1},
+}
+
+func (c capCase) path() string {
+ if c.n == len("a.txt") {
+ return "a.txt"
+ }
+ return capPath(c.n)
+}
+
+// manifest is the case's manifest for generation gen at location loc: a
+// live entry names an object under gen's files prefix, as the publisher
+// writes it.
+func (c capCase) manifest(loc, gen string) hostfiles.Manifest {
+ var e hostfiles.Entry
+ switch c.kind {
+ case "live":
+ sum := sha([]byte("A"))
+ e = hostfiles.Entry{Object: movesFilesPrefix(loc, gen) + fmt.Sprintf("objects/%016d-%s", 1, sum), SHA256: sum, Size: 1, Mode: 0o600, Seq: 1}
+ case "tombstone":
+ e = hostfiles.Entry{Seq: 1, Deleted: true}
+ case "skipped":
+ e = hostfiles.Entry{Seq: 1, Skipped: "too-large"}
+ }
+ return hostfiles.Manifest{V: hostfiles.ManifestVersion, Seq: 1, Entries: map[string]hostfiles.Entry{c.path(): e}, Platform: map[string]hostfiles.Entry{}}
+}
+
+// readerRefuses is what the files readers do with m today: ValidateManifest
+// (checkManifest, the establishment's reader) refuses it as ErrTooDeep. The
+// tests hold the cut to this verdict rather than to a fixed answer per
+// kind: today the readers cap every entry, tombstones and skipped entries
+// included, so the cut refuses all three past the cap. If region HA
+// relaxes the readers for tombstones and skipped entries (its part of
+// #236), RestorableWithinCap follows, and the cut with it.
+func readerRefuses(m hostfiles.Manifest, loc, gen string) bool {
+ return errors.Is(hostfiles.ValidateManifest(m, []string{movesFilesPrefix(loc, gen)}), hostfiles.ErrTooDeep)
+}
+
+// putP1Manifest stores m as the current version of gen's files manifest in
+// the Phase 1 harness's data bucket, written directly (a manifest from
+// before the cap), never through the staging mirror.
+func putP1Manifest(t *testing.T, h *phase1Harness, loc, gen string, m hostfiles.Manifest) {
+ t.Helper()
+ raw, err := json.Marshal(m)
+ if err != nil {
+ t.Fatal(err)
+ }
+ key := hostfiles.ManifestKey(movesFilesPrefix(loc, gen))
+ h.f.store("data-ie", key, fakeMoveObject{lease.MirrorObject{ETag: "manifest-" + gen, Size: int64(len(raw))}, string(raw)})
+ h.f.setVersion("data-ie", key, "ver-manifest-"+gen)
+}
+
+// A source whose files manifest has an entry past the cap is refused at S1
+// with MV-N501, and nothing is staged: no staging request, no touch of the
+// target in the claim, the move still preparing. A manifest the readers
+// accept passes, and the round stages as before.
+func TestPreparingRefusesAFilesManifestPastTheCap(t *testing.T) {
+ for name, c := range capCases {
+ t.Run(name, func(t *testing.T) {
+ h := newPhase1Harness(t)
+ h.start(t, "manual")
+ loc := "instances/" + p1Host + "/" + h.inc
+ m := c.manifest(loc, "cur")
+ putP1Manifest(t, h, loc, "cur", m)
+ touched := h.claim(t).Touched
+ h.pass(t)
+ sr := h.sourceReport(t)
+ if sr == nil {
+ t.Fatal("no source report")
+ }
+ refuses := readerRefuses(m, loc, "cur")
+ if c.kind == "live" && refuses != (c.n > hostfiles.MaxPathBytes) {
+ t.Fatalf("the reader's verdict on a live entry of %d bytes is %v", c.n, refuses)
+ }
+ if !refuses {
+ if !sr.S1.OK || len(h.f.stagingCalls()) == 0 {
+ t.Fatalf("a manifest the readers accept: S1 %+v, %d staging calls", sr.S1, len(h.f.stagingCalls()))
+ }
+ return
+ }
+ if sr.S1.OK || sr.S1.Reason != s1FilesPastCap(p1Host) {
+ t.Fatalf("S1 = %+v, want %q", sr.S1, s1FilesPastCap(p1Host))
+ }
+ if calls := h.f.stagingCalls(); len(calls) != 0 {
+ t.Fatalf("staged despite the refusal: %+v", calls)
+ }
+ if got := h.claim(t).Touched; !reflect.DeepEqual(got, touched) {
+ t.Fatalf("the claim was touched: %+v, was %+v", got, touched)
+ }
+ if st := h.read(t, p1Host).Move.State; st != provision.MovePreparing {
+ t.Fatalf("the move is %s", st)
+ }
+ })
+ }
+}
+
+// A files manifest that cannot be read ends the round as the other read
+// errors do: no source report, nothing staged, the reason in the log.
+func TestPreparingEndsTheRoundWhenTheFilesManifestCannotBeRead(t *testing.T) {
+ h := newPhase1Harness(t)
+ h.start(t, "manual")
+ h.vs.deny = func(op, _, key string) error {
+ if op == "list-versions" && strings.Contains(key, "/files/g-") {
+ return errors.New("injected 403")
+ }
+ return nil
+ }
+ logs := captureLog(t)
+ h.pass(t)
+ if sr := h.sourceReport(t); sr != nil {
+ t.Fatalf("a report was written: %+v", sr)
+ }
+ if calls := h.f.stagingCalls(); len(calls) != 0 {
+ t.Fatalf("staged: %+v", calls)
+ }
+ if !strings.Contains(logs(), "read the files manifest") {
+ t.Fatalf("no log of the read:\n%s", logs())
+ }
+}
+
+// stallingVersioned is the harness's versioned reader with the files
+// manifest's download stalled: GetVersion on a manifest key blocks until
+// its context ends, as an S3 body that stops arriving does, and answers
+// the context's error.
+type stallingVersioned struct {
+ *fakeVersioned
+ stalled chan struct{} // closed when the first stalled read starts
+ once sync.Once
+}
+
+func (v *stallingVersioned) GetVersion(ctx context.Context, bucket, key, versionID string) ([]byte, error) {
+ if strings.Contains(key, "/files/g-") && strings.HasSuffix(key, "/manifest.json") {
+ v.once.Do(func() { close(v.stalled) })
+ <-ctx.Done()
+ return nil, ctx.Err()
+ }
+ return v.fakeVersioned.GetVersion(ctx, bucket, key, versionID)
+}
+
+// Fix round 1, P2 1: the preflight's reads are bounded
+// (movesFilesPreflightTimeout). A stalled manifest download is cancelled at
+// the bound, even under a pass with no deadline of its own; the round ends
+// with no report and nothing staged; the staging lease is released and the
+// pass mutex is free, so the next pass runs.
+func TestPreparingBoundsAStalledFilesManifestRead(t *testing.T) {
+ const bound = 50 * time.Millisecond
+ orig := movesFilesPreflightBound
+ movesFilesPreflightBound = bound
+ defer func() { movesFilesPreflightBound = orig }()
+ if movesFilesPreflightTimeout <= bound || movesFilesPreflightTimeout > hostbox.RequestLimit {
+ t.Fatalf("the production bound %v is not a short one inside the claim's request limit", movesFilesPreflightTimeout)
+ }
+
+ h := newPhase1Harness(t)
+ h.start(t, "manual")
+ loc := "instances/" + p1Host + "/" + h.inc
+ putP1Manifest(t, h, loc, "cur", capCases["an ordinary manifest"].manifest(loc, "cur"))
+ sv := &stallingVersioned{fakeVersioned: h.vs, stalled: make(chan struct{})}
+ h.cfg.Versioned = func(context.Context, string, string) (lease.VersionedStore, error) { return sv, nil }
+ logs := captureLog(t)
+
+ done := make(chan error, 1)
+ start := time.Now()
+ go func() { done <- movesPass(context.Background(), h.s, h.w()) }()
+ select {
+ case <-done:
+ case <-time.After(10 * time.Second):
+ t.Fatal("the pass is still blocked on the stalled read")
+ }
+ if elapsed := time.Since(start); elapsed < bound {
+ t.Fatalf("the pass ended after %v, before the bound %v", elapsed, bound)
+ }
+ select {
+ case <-sv.stalled:
+ default:
+ t.Fatal("the manifest download was never asked for")
+ }
+ if sr := h.sourceReport(t); sr != nil {
+ t.Fatalf("a report was written: %+v", sr)
+ }
+ if calls := h.f.stagingCalls(); len(calls) != 0 {
+ t.Fatalf("staged: %+v", calls)
+ }
+ if !strings.Contains(logs(), context.DeadlineExceeded.Error()) {
+ t.Fatalf("no log of the cancelled read:\n%s", logs())
+ }
+ h.lease.mu.Lock()
+ acquired, released := h.lease.acquired, h.lease.released
+ h.lease.mu.Unlock()
+ if acquired == 0 || acquired != released {
+ t.Fatalf("staging lease acquired %d, released %d", acquired, released)
+ }
+ if !h.w().pass.TryLock() {
+ t.Fatal("the pass mutex is still held")
+ }
+ h.w().pass.Unlock()
+
+ // The download recovers: the next pass reports and stages.
+ h.cfg.Versioned = func(context.Context, string, string) (lease.VersionedStore, error) { return h.vs, nil }
+ h.pass(t)
+ if sr := h.sourceReport(t); sr == nil || !sr.S1.OK || len(h.f.stagingCalls()) == 0 {
+ t.Fatalf("the next pass: %+v, %d staging calls", sr, len(h.f.stagingCalls()))
+ }
+}
+
+// Fix round 1, P2 2: the files directory itself is held to the tree's cap.
+// A file whose path inside it is 4096 bytes (the archive format's own
+// bound, which used to pass) is refused at S1 with MV-N502 before anything
+// is staged; one of 4095 bytes passes and its archive is staged. The
+// files directory is under the data directory, so both absolute paths are
+// past PATH_MAX: the archive reads through an os.Root.
+func TestPreparingRefusesAFilePathPastTheTreeCap(t *testing.T) {
+ for _, n := range []int{hostfiles.MaxPathBytes + 1, hostfiles.MaxPathBytes} {
+ t.Run(fmt.Sprint(n), func(t *testing.T) {
+ h := newPhase1Harness(t)
+ h.start(t, "manual")
+ root, err := os.OpenRoot(filepath.Join(h.cfg.DataDir, p1Host))
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer root.Close()
+ leaf := capPath(n)
+ if err := root.MkdirAll(path.Dir(leaf), 0o755); err != nil {
+ t.Fatal(err)
+ }
+ if err := root.WriteFile(leaf, []byte("deep"), 0o644); err != nil {
+ t.Fatal(err)
+ }
+ h.pass(t)
+ sr := h.sourceReport(t)
+ if sr == nil {
+ t.Fatal("no source report")
+ }
+ if n > hostfiles.MaxPathBytes {
+ if sr.S1.OK || sr.S1.Reason != s1FilePastCap(p1Host) {
+ t.Fatalf("S1 = %+v, want %q", sr.S1, s1FilePastCap(p1Host))
+ }
+ if calls := h.f.stagingCalls(); slices.ContainsFunc(calls, func(c stagingCall) bool { return strings.HasPrefix(c.Key, moves.FilesPrefix(p1Host, p1ID)) }) {
+ t.Fatalf("an archive was staged: %+v", calls)
+ }
+ return
+ }
+ if !sr.S1.OK || sr.Archive == nil {
+ t.Fatalf("a path at the cap: S1 %+v, archive %+v", sr.S1, sr.Archive)
+ }
+ })
+ }
+}
+
+// Fix round 1, P2 2(b): a path at the tree's cap goes end to end. The
+// source's archive carries it, the target's Extract places it, and the
+// prepared files manifest build stages it (hostfiles.Prepared.Stage), with
+// the extracted tree's absolute paths past PATH_MAX: a 4095-byte file, a
+// deep tree of files near it, and an ordinary one. The build reads through
+// an os.Root over the files directory, every name relative to it.
+func TestATreeCapPathReachesThePreparedManifest(t *testing.T) {
+ ctx := context.Background()
+ x := newFilesFixture(t)
+ src := filepath.Join(t.TempDir(), "h")
+ if err := os.Mkdir(src, 0o755); err != nil {
+ t.Fatal(err)
+ }
+ root, err := os.OpenRoot(src)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer root.Close()
+ files := map[string]string{"a.txt": "A"}
+ files[capPath(hostfiles.MaxPathBytes)] = "at the cap"
+ for _, n := range []int{hostfiles.MaxPathBytes - 1, 4000, 3900} {
+ files[capPath(n)+"-"+fmt.Sprint(n)] = fmt.Sprint(n) // under one deep tree
+ }
+ for rel, body := range files {
+ if len(rel) > hostfiles.MaxPathBytes {
+ rel = rel[:hostfiles.MaxPathBytes]
+ }
+ if err := root.MkdirAll(path.Dir(rel), 0o755); err != nil {
+ t.Fatal(err)
+ }
+ if err := root.WriteFile(rel, []byte(body), 0o644); err != nil {
+ t.Fatal(err)
+ }
+ }
+ var tarz bytes.Buffer
+ if _, _, _, err := archive.Build(src, "", &tarz); err != nil {
+ t.Fatalf("the source's archive: %v", err)
+ }
+ dst := filepath.Join(x.dir, filesHost)
+ if len(dst)+1+hostfiles.MaxPathBytes <= 4096 {
+ t.Fatalf("%s does not put the tree past PATH_MAX", dst)
+ }
+ if err := archive.Extract(bytes.NewReader(tarz.Bytes()), dst, x.envPath()); err != nil {
+ t.Fatalf("the target's extract: %v", err)
+ }
+ if _, err := x.f.BuildPrepared(ctx, filesLoc, "g1", "g1", 7, dst, x.envPath(), false, hostfiles.WakeState{V: 1, Host: filesHost, Timers: []hostfiles.WakeTimer{}}); err != nil {
+ t.Fatalf("the prepared manifest: %v", err)
+ }
+ m, found := x.manifest("g1")
+ if !found {
+ t.Fatal("no prepared manifest")
+ }
+ staged := 0
+ err = fs.WalkDir(root.FS(), ".", func(rel string, d fs.DirEntry, err error) error {
+ if err != nil || d.IsDir() {
+ return err
+ }
+ body, err := root.ReadFile(rel)
+ if err != nil {
+ return err
+ }
+ if e, ok := m.Entries[rel]; !ok || e.SHA256 != sha(body) || e.Skipped != "" {
+ t.Errorf("%d-byte path: entry %+v, %v", len(rel), e, ok)
+ }
+ staged++
+ return nil
+ })
+ if err != nil {
+ t.Fatal(err)
+ }
+ if staged != len(files) || len(m.Entries) != len(files) {
+ t.Fatalf("%d files, %d walked, %d entries", len(files), staged, len(m.Entries))
+ }
+ if err := movesFilesRestorable(m); err != nil {
+ t.Fatalf("the prepared manifest is not restorable: %v", err)
+ }
+}
+
+// The cut's requested round runs the same source round (G13): a manifest
+// past the cap that appeared after readiness sends the cut back to ready
+// before anything holds, fences or stops, with MV-N501 in S1, and the move
+// is not pinned again while it stands. Once an operator replaces the
+// manifest, a passing round pins it again and the cut admits.
+func TestCutRefusesAFilesManifestPastTheCap(t *testing.T) {
+ h := newCutHarness(t)
+ h.ready(t)
+ h.ev.stopped = false // stop at stopping, to look at the admission
+ past := capCases["a live entry past it"]
+ putP1Manifest(t, h.phase1Harness, cutRoot, "cur", past.manifest(cutRoot, "cur"))
+ h.cut(t)
+ h.pass(t)
+ m := h.move(t)
+ if m.State != provision.MoveReady || m.CutPhase != "" || m.Inputs != nil || m.CutAt != nil {
+ t.Fatalf("not back to ready: %+v", m)
+ }
+ if h.ev.calls != 0 || len(h.holds) != 0 || h.fences != 0 {
+ t.Fatalf("something stopped: %d evidence calls, holds %v, %d fences", h.ev.calls, h.holds, h.fences)
+ }
+ if sr := h.sourceReport(t); sr == nil || sr.S1.OK || sr.S1.Reason != s1FilesPastCap(p1Host) {
+ t.Fatalf("S1 = %+v, want %q", sr, s1FilesPastCap(p1Host))
+ }
+ h.targetPass(t, h.target())
+ h.pass(t)
+ if m := h.move(t); m.State != provision.MoveReady || m.Inputs != nil {
+ t.Fatalf("pinned again while the manifest stands: %+v", m)
+ }
+
+ // The operator replaces the manifest.
+ putP1Manifest(t, h.phase1Harness, cutRoot, "cur", capCases["an ordinary manifest"].manifest(cutRoot, "cur"))
+ // A refused report carries no archive, so the source reports once, the
+ // target checks that report, and the next round pins.
+ h.pass(t)
+ h.targetPass(t, h.target())
+ h.pass(t)
+ if m := h.move(t); m.State != provision.MoveReady || m.Inputs == nil {
+ t.Fatalf("not pinned again after the replacement: %+v", m)
+ }
+ h.cut(t)
+ h.pass(t)
+ if m := h.move(t); m.State != provision.MoveCutting || m.CutPhase != provision.CutStopping {
+ t.Fatalf("the cut did not admit after the replacement: %+v", m)
+ }
+}
+
+// The cut's preflight and the recovery's establishment agree on every
+// manifest shape, so they cannot drift: the source's round before the flip
+// (sourceFilesRefusal over the holder at src) refuses exactly the
+// manifests a recovery's Establish refuses as ErrTooDeep, through the
+// src_gen bridge of a clean return and through a revert's own lineage, and
+// both agree with the readers (readerRefuses). Both call
+// movesFilesRestorable; a manifest the cut passes, the recovery
+// establishes from.
+func TestCutPreflightAndRecoveryAgreeOnTheCap(t *testing.T) {
+ ctx := context.Background()
+ recoveries := map[string]func(o *hostclaim.Observed){
+ "a clean return": func(*hostclaim.Observed) {},
+ "a revert": func(o *hostclaim.Observed) {
+ o.Claim.Op.Kind = hostclaim.KindReverting
+ delete(o.Claim.Op.Intent, "rollback")
+ o.Claim.Holder.Admissions = []hostclaim.AdmissionRecord{{Generation: "src", Region: "uk", Predecessor: "@root", PredecessorRegion: "uk", Mode: "cold", Session: 3, Box: "uk-box"},
+ {Generation: "r1", Region: "uk", Predecessor: "src", PredecessorRegion: "uk", Mode: "cold", Session: 3, Box: "uk-box"}}
+ },
+ }
+ for name, c := range capCases {
+ for rname, shape := range recoveries {
+ t.Run(name+", "+rname, func(t *testing.T) {
+ x := newFilesFixture(t)
+ x.writeEnv("K=1\n")
+ m := c.manifest(filesLoc, "src")
+ raw, err := json.Marshal(m)
+ if err != nil {
+ t.Fatal(err)
+ }
+ x.bucket.Put("data", hostfiles.ManifestKey(movesFilesPrefix(filesLoc, "src")), raw)
+
+ // The cut's side: the source's round before the flip, its
+ // holder at src.
+ w := &movesWorker{cfg: moveCfg{Region: "uk", OwnBucket: "data", OwnBucketRegion: "eu-west-2",
+ Versioned: func(context.Context, string, string) (lease.VersionedStore, error) { return x.bucket, nil }}}
+ pre := servingObserved("src", "src:uk:@root:uk")
+ why, err := w.sourceFilesRefusal(ctx, provision.Record{Host: filesHost}, pre.Claim.Holder)
+ if err != nil {
+ t.Fatalf("the preflight: %v", err)
+ }
+ if why != "" && why != s1FilesPastCap(filesHost) {
+ t.Fatalf("the preflight's reason %q", why)
+ }
+
+ // The recovery's side: the first recovery admission after
+ // the cut, whose src_gen is the holder's generation then.
+ o := recoveryObserved(hostclaim.KindRestoring, "src", "mv1", "r1")
+ shape(&o)
+ o = x.observe(o, "src", "r1")
+ est := x.f.Establish(ctx, o, launchInputs("r1"), x)
+ if est != nil && !errors.Is(est, hostfiles.ErrTooDeep) {
+ t.Fatalf("the establishment failed for another reason: %v", est)
+ }
+
+ cut, recovery, reader := why != "", est != nil, readerRefuses(m, filesLoc, "src")
+ if cut != recovery || cut != reader {
+ t.Fatalf("the cut refuses %v, the recovery refuses %v (%v), the readers refuse %v", cut, recovery, est, reader)
+ }
+ })
+ }
+ }
+}
+
+// The establishment applies the cap before any origin proof: a manifest
+// past it is ErrTooDeep with no Proven call, by the same function the
+// cut's preflight calls.
+func TestEstablishRefusesPastTheCapBeforeTheProof(t *testing.T) {
+ ctx := context.Background()
+ x := newFilesFixture(t)
+ x.writeEnv("K=1\n")
+ m := capCases["a live entry past it"].manifest(filesLoc, "src")
+ if err := movesFilesRestorable(m); !errors.Is(err, hostfiles.ErrTooDeep) {
+ t.Fatalf("movesFilesRestorable: %v", err)
+ }
+ proof := &countingProof{proven: map[string]bool{"src": true, "r1": true}}
+ x.f.Proof = proof
+ if _, err := x.f.movesFilesAllowed(ctx, servingObserved("src"), launchInputs("r1").Admission, filesLoc, "r1", m); !errors.Is(err, hostfiles.ErrTooDeep) {
+ t.Fatalf("movesFilesAllowed: %v", err)
+ }
+ if len(proof.calls) != 0 {
+ t.Fatalf("the proof was asked %v", proof.calls)
+ }
+}
+
+// The preflight reads by exact version through the data bucket's versioned
+// reader; a holder with no location, or a chain with no manifest, has
+// nothing to check.
+func TestSourceFilesRefusalNeedsAManifestToRefuse(t *testing.T) {
+ ctx := context.Background()
+ x := newFilesFixture(t)
+ w := &movesWorker{cfg: moveCfg{Region: "uk", OwnBucket: "data", OwnBucketRegion: "eu-west-2",
+ Versioned: func(context.Context, string, string) (lease.VersionedStore, error) { return x.bucket, nil }}}
+ pre := servingObserved("src", "src:uk:@root:uk")
+ if why, err := w.sourceFilesRefusal(ctx, provision.Record{Host: filesHost}, pre.Claim.Holder); why != "" || err != nil {
+ t.Fatalf("no manifest: %q, %v", why, err)
+ }
+ noLoc := *pre.Claim.Holder
+ noLoc.Location = ""
+ if why, err := w.sourceFilesRefusal(ctx, provision.Record{Host: filesHost}, &noLoc); why != "" || err != nil {
+ t.Fatalf("no location: %q, %v", why, err)
+ }
+ w.cfg.Versioned = nil
+ if _, err := w.sourceFilesRefusal(ctx, provision.Record{Host: filesHost}, pre.Claim.Holder); err == nil {
+ t.Fatal("no versioned reader passed")
+ }
+}
diff --git a/internal/host/movesphase1.go b/internal/host/movesphase1.go
index 36a6b2a2b..33f1f8c3c 100644
--- a/internal/host/movesphase1.go
+++ b/internal/host/movesphase1.go
@@ -17,6 +17,7 @@ import (
"github.com/carlosframework/platform/internal/appconfig"
"github.com/carlosframework/platform/internal/hostclaim"
+ "github.com/carlosframework/platform/internal/hostfiles"
"github.com/carlosframework/platform/internal/lease"
"github.com/carlosframework/platform/internal/moves"
"github.com/carlosframework/platform/internal/moves/archive"
@@ -77,6 +78,26 @@ func s1OverCap(host string) string {
return host + " keeps more than 256 MiB of files beside its database. Keep large files in object storage."
}
+// s1FilesPastCap is MV-N501 (#236): the files manifest a recovery of the
+// host would restore holds an entry path past the readers' cap, so no
+// recovery after the cut could restore it (sourceFilesRefusal).
+func s1FilesPastCap(host string) string {
+ return host + " has a files manifest with a path longer than 4095 bytes, so it cannot move. an operator must replace the manifest"
+}
+
+// s1FilePastCap is MV-N502 (#236 fix round 1): the files directory itself
+// holds a path longer than the files tree's cap (archive.ErrPastTreeCap,
+// one byte under the archive format's MV-N81 bound). The archive could
+// carry it, but the target's prepared files manifest could not, so the
+// landing would fail after the cut; the source refuses here instead.
+func s1FilePastCap(host string) string {
+ return host + " has a file whose path is longer than 4095 bytes, so it cannot move. shorten the path or remove the file"
+}
+
+// The rows say 4095, so the sentences do; this fails to compile if
+// hostfiles.MaxPathBytes is ever changed without the rows.
+const _ = uint(hostfiles.MaxPathBytes-4095) + uint(4095-hostfiles.MaxPathBytes)
+
// s1TooManyEntries and s1LongPath are MV-N79 and MV-N81, the archive's own
// rows, written here from data (the sentinel, the path the error carries)
// rather than taken from an error's text. s1ArchiveFailed is every archive
@@ -309,7 +330,7 @@ func (w *movesWorker) sourceRound(ctx context.Context, s release.Store, rec prov
// The box-step refusals (§7.1: every box step re-checks), and the
// one-box rule the door applied (D15, ruling G13): a region that has
// gained a second box since is refused here, every round.
- if why, err := w.sourceRefusal(ctx, s, rec); err != nil {
+ if why, err := w.sourceRefusal(ctx, s, rec, obs.Claim.Holder); err != nil {
return nil, nil, err
} else if why != "" {
return report(&srcRound{s1: &moves.Check{Reason: why}})
@@ -532,8 +553,9 @@ func (w *movesWorker) refreshCrossRate(ctx context.Context, gm *guardedMirror, s
// sourceRefusal is the box-step re-check (§6.2, §7.1, F14): the reason this
// source cannot move, or "". A registry or bucket read that fails is an
-// error and ends the round.
-func (w *movesWorker) sourceRefusal(ctx context.Context, s release.Store, rec provision.Record) (string, error) {
+// error and ends the round. holder is the claim's, as the round read it
+// under the staging lease.
+func (w *movesWorker) sourceRefusal(ctx context.Context, s release.Store, rec provision.Record, holder *hostclaim.Holder) (string, error) {
cfg := w.cfg
if cfg.Route == nil {
return "", errors.New("no registry to re-check the source against")
@@ -565,6 +587,85 @@ func (w *movesWorker) sourceRefusal(ctx context.Context, s release.Store, rec pr
if cfg.SecretsPub == nil || cfg.SecretsKey == "" {
return s1NoSecretsKey, nil
}
+ return w.sourceFilesRefusal(ctx, rec, holder)
+}
+
+// movesFilesPreflightTimeout bounds sourceFilesRefusal's reads together:
+// creating the versioned reader, listing the manifest key's versions for
+// each generation of the chain, and downloading the selected manifest. A
+// files manifest is one JSON object, small next to what a round copies, so
+// a healthy read takes well under a second; 30 seconds absorbs a slow
+// bucket, is a quarter of the claim's own request limit
+// (hostbox.RequestLimit), and is far inside the 45-minute staging lease the
+// round holds (moves.StagingLeaseTTL). Past it the round ends with an
+// error and no report, and the next pass tries again.
+const movesFilesPreflightTimeout = 30 * time.Second
+
+// movesFilesPreflightBound is the bound in force: movesFilesPreflightTimeout,
+// a variable only so a test can watch a stalled read end without waiting
+// the full 30 seconds.
+var movesFilesPreflightBound = movesFilesPreflightTimeout
+
+// sourceFilesRefusal is the cut's files preflight (#236; Paul's ruling of
+// 2026-10-15): the files manifest a recovery of this host would restore
+// must pass the readers' path cap, or the move never reaches its cut. A
+// manifest written before the cap (a8cebad) can hold a path past it, and
+// every reader refuses that manifest terminally: a recovery that selected
+// it would fail its establishment and leave the source offline after the
+// cut. It runs in every preparing round and in the cut's requested round
+// (G13), before any staging write, stop or cut.
+//
+// The manifest is the one the establishment selects (movesFilesSelect):
+// the newest manifest of the holder's admitted chain in this region. That
+// is the one a revert restores (its holder never leaves the source), and
+// the one a clean return's src_gen bridge restores, since the flip
+// journals the holder's generation as src_gen and every later write of a
+// manifest is held to the cap by Commit. A dirty return restores from a
+// prepared manifest its install builds through the capped writers, so
+// nothing published before the cap reaches it. The rule is the
+// establishment's own (movesFilesRestorable). A host with no files
+// manifest has nothing to check here. A read that fails ends the round.
+//
+// It is deny-only (census ruling moves-overcap-cut-236, platform-1e,
+// 2026-10-15; binding): it can only refuse a move, never allow anything.
+// Passing it grants nothing, and it never skips or short-circuits the
+// establishment's own check of the same manifest, which reads through the
+// claim's identity-bound store (hostbox.RequestStore) and applies
+// movesFilesRestorable and ValidateManifest again. Its reads go through
+// cfg.Versioned, the moves worker's own reader, and are bounded by
+// movesFilesPreflightTimeout. A change that lets it authorize anything
+// needs a new census ruling.
+func (w *movesWorker) sourceFilesRefusal(ctx context.Context, rec provision.Record, holder *hostclaim.Holder) (string, error) {
+ cfg := w.cfg
+ if holder == nil || holder.Location == "" {
+ return "", nil // no location, no files manifest (a legacy holder)
+ }
+ if cfg.Versioned == nil {
+ return "", errors.New("no versioned reader to check the files manifest with")
+ }
+ // The round runs under the staging lease and the pass mutex every moves
+ // pass shares, and the instances loop calls it with no deadline: a
+ // stalled listing or download must end the round, not hold both.
+ ctx, cancel := context.WithTimeout(ctx, movesFilesPreflightBound)
+ defer cancel()
+ vs, err := cfg.Versioned(ctx, cfg.OwnBucket, cfg.OwnBucketRegion)
+ if err != nil {
+ return "", fmt.Errorf("the data bucket, to check the files manifest: %w", err)
+ }
+ m, key, version, found, err := movesFilesSelect(ctx, vs, cfg.OwnBucket, holder.Location, movesFilesChain(holder, cfg.ParkHomeRegion, cfg.parkRegion()))
+ if err != nil {
+ return "", fmt.Errorf("read the files manifest: %w", err)
+ }
+ if !found {
+ return "", nil
+ }
+ if err := movesFilesRestorable(m); err != nil {
+ if !errors.Is(err, hostfiles.ErrTooDeep) {
+ return "", fmt.Errorf("the files manifest %s@%s: %w", key, version, err)
+ }
+ log.Printf("moves: %s: the files manifest %s@%s cannot be restored (%v); an operator must replace it", rec.Host, key, version, err)
+ return s1FilesPastCap(rec.Host), nil
+ }
return "", nil
}
@@ -810,6 +911,8 @@ func archiveFailure(host string, err error) (why string, retry bool, ferr error)
return s1SpecialFile(host, pe.Path), false, nil // M-C14, hard links too
case errors.Is(err, archive.ErrOverCap):
return s1OverCap(host), false, nil // M-C15
+ case errors.Is(err, archive.ErrPastTreeCap):
+ return s1FilePastCap(host), false, nil // MV-N502
case errors.Is(err, archive.ErrTooManyEntries):
return s1TooManyEntries, false, nil // MV-N79
case errors.Is(err, archive.ErrBadPath) && hasPath:
diff --git a/internal/moves/archive/archive.go b/internal/moves/archive/archive.go
index 648743084..1c3287872 100644
--- a/internal/moves/archive/archive.go
+++ b/internal/moves/archive/archive.go
@@ -42,6 +42,9 @@ var (
ErrBadPath = errors.New("archive: refused path")
ErrTooLarge = errors.New("archive: the decoded archive is over its size limit")
ErrNotEmpty = errors.New("archive: the destination files directory is not empty")
+ // ErrPastTreeCap is Build's refusal of a path longer than TreePathLen
+ // (hostfiles' cap) but within MaxPathLen; the source says MV-N502.
+ ErrPastTreeCap = errors.New("archive: a path is past the files tree's length cap")
)
// PathError is a Build refusal that names a file: Err is the sentinel
@@ -83,6 +86,14 @@ func changedError(path string) error {
// MaxPathLen is ever changed without the row.
const _ = uint(MaxPathLen-4096) + uint(4096-MaxPathLen)
+// treePathError is a path past TreePathLen but within MaxPathLen: the
+// archive format could carry it, the target's files manifest could not.
+// The operator's sentence is the caller's (MV-N502, from the sentinel);
+// this text reaches the log only.
+func treePathError(path string) error {
+ return &PathError{Err: ErrPastTreeCap, Path: path, msg: fmt.Sprintf("archive: refused path: %s is longer than the files tree's %d-byte cap", path, TreePathLen)}
+}
+
func longPathError(path string) error {
return &PathError{Err: ErrBadPath, Path: path, msg: fmt.Sprintf("archive: refused path: %s is longer than 4096 bytes", path)}
}
@@ -157,6 +168,9 @@ func walk(root *os.Root, envPath string) ([]item, error) {
}
return nil
}
+ // Through the root only (census ruling moves-overcap-cut-236,
+ // binding): a read that fails refuses the walk, never a retry by
+ // path.
info, err := root.Lstat(p)
if err != nil {
return err
@@ -167,6 +181,9 @@ func walk(root *os.Root, envPath string) ([]item, error) {
if len(p) > MaxPathLen {
return longPathError(p)
}
+ if len(p) > TreePathLen {
+ return treePathError(p)
+ }
return add(item{name: FilesPrefix + p, rel: p, root: root, info: info})
})
if err != nil {
diff --git a/internal/moves/archive/archive_test.go b/internal/moves/archive/archive_test.go
index 4cf281c17..9cf54e7d7 100644
--- a/internal/moves/archive/archive_test.go
+++ b/internal/moves/archive/archive_test.go
@@ -267,6 +267,7 @@ func refusedStreams() map[string]refusedStreamCase {
"nested late dir": {[]rawEntry{dirEntry("files/a/"), reg("files/a/b/c", "x"), dirEntry("files/a/b/")}, ErrBadPath},
"env as dir": {[]rawEntry{dirEntry("env/")}, ErrBadPath},
"too long": {[]rawEntry{reg("files/"+strings.Repeat("a", MaxPathLen+1), "x")}, ErrBadPath},
+ "past tree cap": {[]rawEntry{reg("files/"+strings.Repeat("a", TreePathLen+1), "x")}, ErrBadPath},
"symlink": {[]rawEntry{{hdr: tar.Header{Name: "files/l", Typeflag: tar.TypeSymlink, Linkname: "/etc/passwd"}}}, ErrSpecialFile},
"hard link": {[]rawEntry{reg("files/a", "x"), {hdr: tar.Header{Name: "files/l", Typeflag: tar.TypeLink, Linkname: "files/a"}}}, ErrSpecialFile},
"char device": {[]rawEntry{{hdr: tar.Header{Name: "files/d", Typeflag: tar.TypeChar, Devmajor: 1, Devminor: 3}}}, ErrSpecialFile},
@@ -468,9 +469,10 @@ func TestThousandsOfSmallFilesArchive(t *testing.T) {
// path Build admits, so every archive Build writes is one Extract accepts.
func TestTheLongestPathFitsTheEntryOverhead(t *testing.T) {
dir := filepath.Join(t.TempDir(), "h")
- // Components of 200 bytes nested to a relative path of MaxPathLen-ish.
+ // Components of 200 bytes nested to a relative path of TreePathLen-ish:
+ // the longest Build admits is the files tree's cap (#236 fix round 1).
rel := ""
- for len(rel)+201 <= MaxPathLen-10 {
+ for len(rel)+201 <= TreePathLen-10 {
if rel != "" {
rel += "/"
}
@@ -479,8 +481,8 @@ func TestTheLongestPathFitsTheEntryOverhead(t *testing.T) {
if err := os.Mkdir(dir, 0o755); err != nil {
t.Fatal(err)
}
- leaf := rel + "/" + strings.Repeat("f", MaxPathLen-len(rel)-1)
- if len(leaf) != MaxPathLen {
+ leaf := rel + "/" + strings.Repeat("f", TreePathLen-len(rel)-1)
+ if len(leaf) != TreePathLen {
t.Fatalf("leaf %d", len(leaf))
}
// Created through a Root (openat), as Build reads it: the absolute
@@ -503,15 +505,97 @@ func TestTheLongestPathFitsTheEntryOverhead(t *testing.T) {
if per := (b.tarBytes - 2*512 - b.content) / int64(b.count); per > entryOverhead {
t.Fatalf("per-entry overhead %d over the bound %d", per, entryOverhead)
}
- // One byte longer is refused by Build, naming the path.
+ // One byte longer (MaxPathLen) is past the tree's cap: refused by Build
+ // with ErrPastTreeCap, naming the path.
if err := root.Rename(leaf, leaf+"x"); err != nil {
t.Fatal(err)
}
- if _, err := build(dir, "", io.Discard); !errors.Is(err, ErrBadPath) {
+ var pe *PathError
+ if _, err := build(dir, "", io.Discard); !errors.Is(err, ErrPastTreeCap) || errors.Is(err, ErrBadPath) || !errors.As(err, &pe) || pe.Path != leaf+"x" {
+ t.Fatalf("a path of MaxPathLen: %v", err)
+ }
+ // Past MaxPathLen it is the format's own refusal (MV-N81), as before.
+ if err := root.Rename(leaf+"x", leaf+"xy"); err != nil {
+ t.Fatal(err)
+ }
+ if _, err := build(dir, "", io.Discard); !errors.Is(err, ErrBadPath) || !errors.As(err, &pe) || pe.Path != leaf+"xy" {
t.Fatalf("over-long path: %v", err)
}
}
+// #236 fix round 1: a path of exactly the files tree's cap (TreePathLen,
+// hostfiles.MaxPathBytes) goes through Build, Seal, Verify and Extract, into
+// a destination whose absolute path is past PATH_MAX once the path is
+// added; one byte longer never leaves the source, and a stream that carries
+// it (an older source's) is refused by Verify and Extract (the "past tree
+// cap" hostile stream).
+func TestATreeCapPathRoundTrips(t *testing.T) {
+ // TreePathLen is hostfiles.MaxPathBytes by definition; MaxPathLen is
+ // one byte more, which the next line fails to compile without.
+ const _ = uint(MaxPathLen-1-TreePathLen) + uint(TreePathLen-(MaxPathLen-1))
+ dir := filepath.Join(t.TempDir(), "h")
+ if err := os.Mkdir(dir, 0o755); err != nil {
+ t.Fatal(err)
+ }
+ leaf := treeCapPath(TreePathLen)
+ root, err := os.OpenRoot(dir)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer root.Close()
+ if err := root.MkdirAll(filepath.Dir(leaf), 0o755); err != nil {
+ t.Fatal(err)
+ }
+ if err := root.WriteFile(leaf, []byte("deep"), 0o644); err != nil {
+ t.Fatal(err)
+ }
+ var tarz bytes.Buffer
+ plain, entries, n, err := Build(dir, "", &tarz)
+ if err != nil {
+ t.Fatal(err)
+ }
+ pub, priv, b := keys(t)
+ var sealed bytes.Buffer
+ key, sealedSHA, err := Seal(bytes.NewReader(tarz.Bytes()), pub, b, &sealed)
+ if err != nil {
+ t.Fatal(err)
+ }
+ m := Manifest{V: ManifestVersion, Op: "mv_x", Incarnation: "i", Plain: plain, Sealed: sealedSHA, Bytes: n, Entries: entries, SealedDataKey: key, Chunk: ChunkSize}
+ if err := Verify(bytes.NewReader(sealed.Bytes()), m, priv, b); err != nil {
+ t.Fatalf("Verify of a tree-cap path: %v", err)
+ }
+ _, files, env := extractTarget(t)
+ if len(files)+1+len(leaf) <= 4096 {
+ t.Fatalf("the destination %s does not put the path past PATH_MAX", files)
+ }
+ if err := Extract(bytes.NewReader(tarz.Bytes()), files, env); err != nil {
+ t.Fatalf("Extract of a tree-cap path: %v", err)
+ }
+ out, err := os.OpenRoot(files)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer out.Close()
+ if got, err := out.ReadFile(leaf); err != nil || string(got) != "deep" {
+ t.Fatalf("extracted %q, %v", got, err)
+ }
+}
+
+// treeCapPath is a relative path of exactly n bytes, 200-byte directory
+// components and a leaf.
+func treeCapPath(n int) string {
+ var b strings.Builder
+ for b.Len()+201 <= n-10 {
+ if b.Len() > 0 {
+ b.WriteByte('/')
+ }
+ b.WriteString(strings.Repeat("d", 200))
+ }
+ b.WriteByte('/')
+ b.WriteString(strings.Repeat("f", n-b.Len()))
+ return b.String()
+}
+
// Build, Verify and Extract share one entry-count limit, and Extract and
// Verify bound the decoded stream, so a zstd bomb that passed the sealing
// is still bounded on the target.
diff --git a/internal/moves/archive/limits.go b/internal/moves/archive/limits.go
index d61e0b29a..44f6bf90c 100644
--- a/internal/moves/archive/limits.go
+++ b/internal/moves/archive/limits.go
@@ -8,6 +8,7 @@ import (
"path/filepath"
"strings"
+ "github.com/carlosframework/platform/internal/hostfiles"
"github.com/klauspost/compress/zstd"
)
@@ -26,6 +27,17 @@ const (
MaxPathLen = 4096
)
+// TreePathLen is the files tree's own cap, hostfiles' (MaxPathBytes, 4095
+// bytes): the most a path inside the files directory may hold and still be
+// staged into the prepared files manifest the target builds after the cut
+// (hostfiles.Prepared.Stage refuses anything longer). It is one byte under
+// MaxPathLen, the archive format's bound that MV-N81 states, so Build
+// refuses a path of exactly MaxPathLen with ErrPastTreeCap (a refusal at
+// S1, before the cut), and every reader refuses one past TreePathLen
+// (checkName), so an archive an older source built with such a path never
+// passes the target's verification (#236 fix round 1).
+const TreePathLen = hostfiles.MaxPathBytes
+
// entryOverhead is the most tar bytes one entry costs beyond its content:
// a 512-byte header, a PAX header (512 bytes plus the path record, at most
// len("files/")+MaxPathLen+1 bytes and a length prefix, rounded up to 4608)
@@ -89,6 +101,9 @@ func checkName(name string, dir bool) (rel string, isEnv bool, err error) {
if !ok || rel == "" {
return "", false, bad("is outside files/ and env")
}
+ if len(rel) > TreePathLen {
+ return "", false, bad("is past the files tree's length cap")
+ }
if rel == MarkerName || strings.HasPrefix(rel, MarkerName+"/") {
return "", false, bad("is the protocol marker or inside it")
}
diff --git a/internal/provision/plan14_legacy_identity_test.go b/internal/provision/plan14_legacy_identity_test.go
index 578aa44f4..f0e4d3cbf 100644
--- a/internal/provision/plan14_legacy_identity_test.go
+++ b/internal/provision/plan14_legacy_identity_test.go
@@ -603,12 +603,12 @@ func TestPlan14SeededInstancesOverlayPreservesLegacySeal(t *testing.T) {
func TestPlan14SeedOverlayPreservesLegacySeal(t *testing.T) {
root := censusRoot(t)
inv := rootReadInventory{Entries: map[string]classifiedRootRead{}, Coverage: map[string]string{}}
- for i, name := range []string{"plan14-root-reads.tsv", "plan14-root-reads-moves.tsv", "plan14-root-reads-merge.tsv", "plan14-root-reads-fix1.tsv", "plan14-root-reads-fix2.tsv", "plan14-root-reads-merge3.tsv", "plan14-root-reads-fix4.tsv", "plan14-root-reads-fix5.tsv", "plan14-root-reads-fix6.tsv", "plan14-root-reads-zddr3.tsv", "plan14-root-reads-hotfix281.tsv", "plan14-root-reads-zddr4.tsv", "plan14-root-reads-reserved285.tsv", "plan14-root-reads-292.tsv", "plan14-root-reads-census-ruling-275.tsv", "plan14-root-reads-census-ruling-owed.tsv", "plan14-root-reads-carlos-run.tsv", "plan14-root-reads-seeded-instances.tsv", "plan14-root-reads-hostfiles-cap-236.tsv"} {
+ for i, name := range []string{"plan14-root-reads.tsv", "plan14-root-reads-moves.tsv", "plan14-root-reads-merge.tsv", "plan14-root-reads-fix1.tsv", "plan14-root-reads-fix2.tsv", "plan14-root-reads-merge3.tsv", "plan14-root-reads-fix4.tsv", "plan14-root-reads-fix5.tsv", "plan14-root-reads-fix6.tsv", "plan14-root-reads-zddr3.tsv", "plan14-root-reads-hotfix281.tsv", "plan14-root-reads-zddr4.tsv", "plan14-root-reads-reserved285.tsv", "plan14-root-reads-292.tsv", "plan14-root-reads-census-ruling-275.tsv", "plan14-root-reads-census-ruling-owed.tsv", "plan14-root-reads-carlos-run.tsv", "plan14-root-reads-seeded-instances.tsv", "plan14-root-reads-hostfiles-cap-236.tsv", "plan14-root-reads-moves-overcap-cut-236.tsv"} {
if err := loadRootReadInventory(root+"/internal/provision/testdata/"+name, &inv, i > 0); err != nil {
t.Fatal(err)
}
}
- const seal = "10e024eccd584f40b64e959a105c551302585e3e14e55104fc250dae5960898a"
+ const seal = "fc38db001ebd8825709a8ca9c41cc0e8f165c1b1967e5967347dc9bd3556afec"
if errs := checkRootReadLegacy(inv, seal); len(errs) != 0 {
t.Fatal(errs)
}
@@ -619,7 +619,7 @@ func TestPlan14SeedOverlayPreservesLegacySeal(t *testing.T) {
if len(inv.Legacy) != before+7 {
t.Fatal("FS-R6, FS-R10 and FS-R11 preserve exactly seven additional legacy fingerprints")
}
- if errs := checkRootReadLegacy(inv, "586fc489beaebe0f6fbce9f5199d0e80379f2bd00e3e3e716a273d6fe2e67d36"); len(errs) != 0 {
+ if errs := checkRootReadLegacy(inv, "624e96c0f0fd397d09575e807f7347f86cbcb109b4b713eb8369f0b64622e483"); len(errs) != 0 {
t.Fatal(errs)
}
for _, key := range []string{"internal/activator/servingretry.go:ServingRetries.Wrap", "internal/activator/servingretry.go:ServingRetries.settle", "internal/park/local.go:Local.RetireSuperseded"} {
diff --git a/internal/provision/plan14_root_reads_test.go b/internal/provision/plan14_root_reads_test.go
index 6892f03ae..6c82ddccb 100644
--- a/internal/provision/plan14_root_reads_test.go
+++ b/internal/provision/plan14_root_reads_test.go
@@ -744,11 +744,29 @@ func TestPlan14RootReadCensus(t *testing.T) {
if errs := checkRootReadLegacy(inv, seal); len(errs) != 0 {
t.Fatal(errs)
}
+ // Census overlay moves-overcap-cut-236 (#236, moves' part of Paul's
+ // ruling of 2026-10-15), loaded last: the moves source round's files
+ // manifest preflight, under census ruling moves-overcap-cut-236
+ // (platform-1e, 2026-10-15). sourceRefusal and sourceRound stay
+ // accepted-legacy with the ruling's lines, on the binding condition that
+ // the preflight is deny-only (the overlay's header). The ruling's
+ // extension (fix round 1) covers archive.walk, buildPrepared and
+ // readFileBounded for the files tree's cap, on the binding condition that
+ // no read falls back to a path-based open when a root open fails. The
+ // seal moves; the internal/host and internal/moves/archive envelopes are
+ // re-pinned.
+ if err = loadRootReadInventory(filepath.Join(root, "internal/provision/testdata/plan14-root-reads-moves-overcap-cut-236.tsv"), &inv, true); err != nil {
+ t.Fatal(err)
+ }
+ seal = "fc38db001ebd8825709a8ca9c41cc0e8f165c1b1967e5967347dc9bd3556afec"
+ if errs := checkRootReadLegacy(inv, seal); len(errs) != 0 {
+ t.Fatal(errs)
+ }
if err = loadRootReadInventory(filepath.Join(root, "internal/provision/testdata/plan14-root-reads-seed.tsv"), &inv, true); err != nil {
t.Fatal(err)
}
// FS-R6 preserves three legacy readers; FS-R10 re-digests the test-only S3 server. Other proposals remain unapproved.
- seal = "586fc489beaebe0f6fbce9f5199d0e80379f2bd00e3e3e716a273d6fe2e67d36"
+ seal = "624e96c0f0fd397d09575e807f7347f86cbcb109b4b713eb8369f0b64622e483"
errs := append(compareRootReads(got, inv), checkRootReadLegacy(inv, seal)...)
errs = append(errs, checkRootReadDeclarations(got, inv)...)
if len(errs) > 0 {
diff --git a/internal/provision/plan9_b8_census_test.go b/internal/provision/plan9_b8_census_test.go
index 8e9c8eafb..ba5efc8dd 100644
--- a/internal/provision/plan9_b8_census_test.go
+++ b/internal/provision/plan9_b8_census_test.go
@@ -782,7 +782,9 @@ var plan9B8CensusInventory = []censusEntry{
{"internal/host/movesphase1.go", "writeMoveLedger", "nonrecord", "sel:Put", "597a70ac06ceb4e9eb52bb9f61b28a4937e703fa4a0916f64bd95d9594f0556e"},
// settings-4: added the build-state lane (mirrorBuildState: holder location's
// build-state.json to the move lane's state/ prefix in the data bucket); no record key.
- {"internal/host/movesphase1.go", "movesWorker.sourceRound", "nonrecord", "sel:PutIfAbsentVersioned", "9f87605eaeb58cb034c2147a2f54e02b542d1cf8930b6fe3a1b78868cc8e5345"},
+ // moves-overcap-cut (#236): sourceRefusal is passed the claim holder for the
+ // files manifest preflight; the write sites are unchanged.
+ {"internal/host/movesphase1.go", "movesWorker.sourceRound", "nonrecord", "sel:PutIfAbsentVersioned", "aacb3d17ad302c969e05ccfc0e091c4c6a2b0d6cc3791fa5f717ee7ad905a9e7"},
{"internal/host/movesphase1.go", "movesWorker.refreshCrossRate", "nonrecord", "sel:CopyVersioned", "7e03e680bd61be26dddcf168cacd2427bb535571dfd326c4a3130ebe65ea1968"},
{"internal/host/movesphase1.go", "movesWorker.writeSourceReport", "nonrecord", "sel:PutReplace", "370407cca2c1b91f6e5e7f135238f04c0dce9d7986e6e1ee9bebbf88cad0b9e0"},
{"internal/host/movesphase1.go", "movesWorker.decideReady", "record-writer", "key:SetRecordMove", "d29c22eafacfb894b96cd0c40ec447b6ab02f46ea087201685627861f992085d"},
diff --git a/internal/provision/testdata/plan14-root-reads-moves-overcap-cut-236.tsv b/internal/provision/testdata/plan14-root-reads-moves-overcap-cut-236.tsv
new file mode 100644
index 000000000..1ff0c6c27
--- /dev/null
+++ b/internal/provision/testdata/plan14-root-reads-moves-overcap-cut-236.tsv
@@ -0,0 +1,15 @@
+# Plan 14 census overlay moves-overcap-cut-236 (#236, moves' part of Paul's ruling of 2026-10-15; loaded after hostfiles-cap-236): the moves source round refuses, at S1, a source whose files manifest a recovery could not restore because an entry path is past the readers' cap (hostfiles.RestorableWithinCap), and (fix round 1) a source whose files directory holds a path past that cap.
+# Census ruling moves-overcap-cut-236 (platform-1e, 2026-10-15), #236: movesWorker.sourceRefusal and movesWorker.sourceRound stay accepted-legacy with main's dispositions. sourceRound passes the claim holder it read under the staging lease to sourceRefusal, which ends in sourceFilesRefusal: a read of the holder's files manifest from this box's own data bucket by exact version through cfg.Versioned, bounded by movesFilesPreflightTimeout. CONDITION (binding): that check is deny-only. It never allows anything, and never skips or short-circuits the establishment's own check on the claim's identity-bound store. If it ever authorizes something, it needs a new ruling.
+# Extension of census ruling moves-overcap-cut-236 (platform-1e, 2026-10-15), #236 fix round 1: archive.walk (Build refuses a path past the files tree's cap, archive.TreePathLen = hostfiles.MaxPathBytes), buildPrepared (the prepared files manifest build reads the extracted tree through an os.Root over the files directory, names relative to it, so a path within the cap is not past PATH_MAX; the new site is that OpenRoot) and var readFileBounded (it opens through that root) stay accepted-legacy with main's dispositions and the ruling's suffix. CONDITION (binding): no read falls back to a path-based open when a root open fails; it refuses instead.
+@legacy 98a854a936ae2bf5be2f8c07f401caadb37731407d67b73fabf2015826c8bfc1 internal/host/movesphase1.go movesWorker.sourceRefusal legacy accepted-legacy accepted-legacy;rev3=J1;basis=root-route;census-ruling-moves-overcap-cut-236 whole:Route@364,whole:Route@476 690fda4d0209035cffc5b2d066a93e04e7a9a584f55e1fd629395187f1c56484
+internal/host/movesphase1.go movesWorker.sourceRefusal legacy accepted-legacy accepted-legacy;rev3=J1;basis=root-route;census-ruling-moves-overcap-cut-236 whole:Route@364,whole:Route@476 690fda4d0209035cffc5b2d066a93e04e7a9a584f55e1fd629395187f1c56484
+@legacy 973f36144cc7c6bcda570a12c6d8bb2b35a69392db98413a93c0633fe15e4ace internal/host/movesphase1.go movesWorker.sourceRound legacy accepted-legacy accepted-legacy;rev3=J4;basis=root-input;census-ruling-moves-overcap-cut-236 file:Exec@1981,file:Exec@2099,file:Read@1427 aacb3d17ad302c969e05ccfc0e091c4c6a2b0d6cc3791fa5f717ee7ad905a9e7
+internal/host/movesphase1.go movesWorker.sourceRound legacy accepted-legacy accepted-legacy;rev3=J4;basis=root-input;census-ruling-moves-overcap-cut-236 file:Exec@1981,file:Exec@2099,file:Read@1427 aacb3d17ad302c969e05ccfc0e091c4c6a2b0d6cc3791fa5f717ee7ad905a9e7
+@legacy 826af0218f3ea5b1cf04fa64f3546303ab5a7c261f1f5462264f994134c9b998 internal/moves/archive/archive.go walk legacy accepted-legacy accepted-legacy;rev3=J4;basis=root-input;census-ruling-moves-overcap-cut-236 file:Lstat@1168,file:Lstat@747,file:WalkDir@242 6ee298c79079c2bb5a1dfb15f0af4c0ca41b9ef7b4cd349894deec47d21d5e54
+internal/moves/archive/archive.go walk legacy accepted-legacy accepted-legacy;rev3=J4;basis=root-input;census-ruling-moves-overcap-cut-236 file:Lstat@1168,file:Lstat@747,file:WalkDir@242 6ee298c79079c2bb5a1dfb15f0af4c0ca41b9ef7b4cd349894deec47d21d5e54
+@legacy 825ac4c6b6df65127e2334d47521cc5d302246f376dad2ae26a6bc6d599d17b0 internal/host/movesfiles.go buildPrepared legacy accepted-legacy accepted-legacy;rev3=J4;basis=root-input;census-ruling-moves-overcap-cut-236 file:Lstat@613,file:OpenRoot@2066,file:ReadFile@808,file:WalkDir@2252 16939eca2043b765376bcf44653902d46f51af192ff36d8ef0fbb2b5b098b59e
+internal/host/movesfiles.go buildPrepared legacy accepted-legacy accepted-legacy;rev3=J4;basis=root-input;census-ruling-moves-overcap-cut-236 file:Lstat@613,file:OpenRoot@2066,file:ReadFile@808,file:WalkDir@2252 16939eca2043b765376bcf44653902d46f51af192ff36d8ef0fbb2b5b098b59e
+@legacy 68bb7ee7e82af29cc8438c3fc3f575f11c86a71e53806af37bef1a5c76082832 internal/host/movesfiles.go var readFileBounded legacy accepted-legacy accepted-legacy;rev3=J4;basis=root-input;census-ruling-moves-overcap-cut-236 file:Open@236,file:ReadAll@327 9a6274fadde8c8905c83791f982e195eb894209d2dfed4ba7ce2929ee0f66cc9
+internal/host/movesfiles.go var readFileBounded legacy accepted-legacy accepted-legacy;rev3=J4;basis=root-input;census-ruling-moves-overcap-cut-236 file:Open@236,file:ReadAll@327 9a6274fadde8c8905c83791f982e195eb894209d2dfed4ba7ce2929ee0f66cc9
+@coverage internal/host 590dd6508e927969a8e54ea4ae074abd1772e106c1a2cedcc37b5c2f5ae3e962
+@coverage internal/moves/archive 4d329d65e065ce06cd6108529a9d58bac7aad481289d8b4e18c23ddcd2ed8cac
diff --git a/internal/provision/testdata/plan14-root-reads-seed.tsv b/internal/provision/testdata/plan14-root-reads-seed.tsv
index 57ceceeb6..a3dc77643 100644
--- a/internal/provision/testdata/plan14-root-reads-seed.tsv
+++ b/internal/provision/testdata/plan14-root-reads-seed.tsv
@@ -186,12 +186,12 @@ internal/host/claimservingpark_records.go servingParkLedger.completeLocked autho
internal/host/claimservingpark_records.go servingParkLedger.continueLocked authority C,J4 rev3=PROPOSED-seed; namespace/host-protected slot and record locks; pending original retirement, absent source and retained parked identities before durable bounded continuation 87f297a924f7f153624304233218d8b5548a016fc7dbf0f7cb8dedb789324009
internal/host/claimservingpark_records.go servingParkLedger.deleteLocked authority C,J4 rev3=PROPOSED-seed; complete retained record or proved discard disposition and persisted cause/progress before filesystem deletion 1545950c564d51d9c29b0d76daca1ddec5a159481698b11accd4dd7411157b37
internal/host/claimservingpark_records.go servingParkLedger.parkLocked authority C,J4 rev3=PROPOSED-seed; generation or test-fixture legacy entry delegates to immutable evidence-aware ledger; no independent source authority 1f16bea822e6c63df15c8259c8972a4084cdf364769f10396f396424e0fe848c
-internal/host/claimservingpark_records.go servingParkLedger.parkWithEvidenceLocked authority C,J4 rev3=PROPOSED-seed; root-authenticated source, prior slot and immutable intent; a missing same-incarnation retirement must be newer than the last successor; preserve park ownership and replay refusal after collection; old-format replay preserves original consumption and source evidence without inferred handles file:Prepare@3772,file:Read@1774 c0cdd64d93a729abd8cf2a48606579516824c35181f977631de4a1a395a0b8c7
+internal/host/claimservingpark_records.go servingParkLedger.parkWithEvidenceLocked authority C,J4 rev3=PROPOSED-seed; run32; root; last immutable retirement supplies only successor ordering; current source capture, stop evidence, pending intent, slot writes and rename authorization unchanged file:Prepare@3782,file:Read@1784 429fccd009db1aea9bc42dd7a5489d1bedd2a2eabab08bd3cbdab056b49761a6
internal/host/claimservingpark_records.go servingParkLedger.read authority C,J4 rev3=PROPOSED-seed; root-private retirement bytes and separate mount projection; forward reads and inverse progress writes validate both representations f029f0fb8d31bdccaf86a5e336ffc1643eb2487fc7adb87852a9d21ad287e732
internal/host/claimservingpark_records.go servingParkLedger.readHistorical authority C,J4 rev3=PROPOSED-seed; root-private retirement bytes and separate mount projection; forward reads and inverse progress writes validate both representations file:Read@259 7af371e0f68db2dbf035c4dd75c9f8312ea3733d1bbfbdd4139088b2e1805bb3
-internal/host/claimservingpark_records.go servingParkLedger.replaceLocked authority C,J4 rev3=PROPOSED-seed; completed incumbent record under its record lock; resumes its recorded deletion cause before slot replacement 3e67316e959c9a901c32a2a61fda4e6ea5e92fe51bdb7fdb5f5c53d9d5960aad
+internal/host/claimservingpark_records.go servingParkLedger.replaceLocked authority C,J4 rev3=PROPOSED-seed; run32; root; validates completed incumbent from immutable private record; rebinds and applies existing identity/volume checks before deletion; failed deletion retains its record and artifacts, reports the existing approved warning, and permits the new slot after cancellation check; no deletion authority added 39322abbd7ed543d0aa3a82619a4d6a2507a1deb83cd1fe1dd00e33737465e23
internal/host/claimservingpark_records.go servingParkLedger.save authority C,J4 rev3=PROPOSED-seed; root-private retirement bytes and separate mount projection; forward reads and inverse progress writes validate both representations 187f2c360d38e10a9f22b20f4c02a021c66b9308cad404e536ef5d0f0a860674
-internal/host/claimservingpark_records.go servingParkLedger.slot authority C,J4 rev3=PROPOSED-seed; private box/host slot and pending retirement identities across incarnation changes; last-retirement identity supports discarded/artifact-free completion replay file:Read@179 239b806b2e2bfdd568f5d6b749d299f0bc7b118d675e872429432591a89b7b68
+internal/host/claimservingpark_records.go servingParkLedger.slot authority C,J4 rev3=PROPOSED-seed; run32; root; immutable private ID and Last records validate host, box and retained disposition without requiring a usable boot binding; pending retirement still requires current binding; these reads authorize slot identity only, never filesystem mutation file:Read@179 aa8be9edd8e4515abf4e8f567420e7f7f642d0cc24e729c450542853e7446144
internal/host/claimservingpark_records.go servingParkLedger.withNamespace authority C,J4 rev3=PROPOSED-seed; private rootstate namespace lock and cancellation before invoking root authority; composed with root host authority through the typed journal service b7cfae44fdb7de9ef8130f5581710ae3a1a526bb059b4f616650172490f5c8ac
internal/host/claimservingpark_records.go servingParkLedger.write authority C,J4 rev3=PROPOSED-seed; serialized private record sent only to the supplied rootstate store 0635d2c27871e67079119a0f2e3034e817b765d0fe3a47e7d9bdc66fb2779571
internal/host/claimservingpark_records.go servingParkRecord.activeAuthorization authority C,J4 rev3=PROPOSED-seed; last persisted continuation authorization or immutable original authorization; no source or deadline changes fe168fe52317e45ce70c3e97a9bc2c52d2409328ab4844a8d2a34fc4babba36d
@@ -325,7 +325,7 @@ system.go runSystem authority B,C,D,F,J4 rev3=PROPOSED-seed; command arguments,
system.go systemVerb authority B,C,D,F,J4 rev3=PROPOSED-seed; command arguments, claim flags, service identity and root delegation capability before constructing agent-mode resources; one-shot skips Configure, Start, duties and listeners; close resources only after private runtime joins ee25513ed72f9b8338d95682c4c76b4ced26ec1f060528fc5804da9afbfbfd83
@coverage . bf3d70b6a2c5d769a3c580c9d5511231499b788d9c08a542895de8381e5a39c9
@coverage internal/activator 9bc3f0f89cc3902c34917f1c6b2f86a778cec0bed499e5ea783fac9d93c95a15
-@coverage internal/host 0b8d526b4e8b1504dafba96903efca8946b2d0bdb8c3ce35bf3b94b9131736e3
+@coverage internal/host 6753f47be8c2d263dd1fd116fb112a39a6184fe8400b26046a74f9462c15e1af
@coverage internal/hostbox 86337ee9d1f7a4bdd15e331cac712c55d39da4cc26a7c013573a7a9535fe3975
@coverage internal/hostfiles a38dbcd9789ba4acd237e88c834f2b2bfca9847b0d43c32b0116dc8996d7a142
@coverage internal/hostguard a721142f4f3ac5fa17b6b1e2ccfc66ca809cfdcbf2733fdc00fbd0efaa90aeb6