amadan

carlos / platform Public

WIP files seed checkpoint 9 (not for merge)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Paul Campbell pushed by claude 7169220313cc2e5aa620b3f16cf012f06b73a80f parent 4039860
9 files changed, +842 −22
  • internal/host/claimretire_transport.go +4 −1
  • internal/host/claimservingpark_linux.go +106 −3
  • internal/host/claimservingpark_records.go +11 −0
  • internal/host/claimservingpark_recovery.go +151 −0
  • internal/host/claimservingpark_recovery_linux.go +110 −0
  • internal/host/claimservingpark_recovery_linux_test.go +391 −0
  • internal/host/claimservingpark_rootrunner_linux_test.go +25 −2
  • internal/host/claimservingparkroot_linux.go +22 −9
  • internal/provision/testdata/plan14-root-reads-seed.tsv +22 −7

CI passed — run details

CI log 
=== gate ===
cd launchctx && go vet ./... && go test ./... -count=1
ok  	amadan.net/carlos/platform.git/launchctx	0.011s
?   	amadan.net/carlos/platform.git/launchctx/controlname	[no test files]
?   	amadan.net/carlos/platform.git/launchctx/wireid	[no test files]
go vet ./...
gofmt -l .: clean
go test ./... -count=1 -timeout 30m
ok  	github.com/carlosframework/platform	346.763s
?   	github.com/carlosframework/platform/cmd/changelog	[no test files]
ok  	github.com/carlosframework/platform/cmd/console	3.120s
ok  	github.com/carlosframework/platform/cmd/moveprobe	93.123s
ok  	github.com/carlosframework/platform/cmd/zddprobe	0.705s
ok  	github.com/carlosframework/platform/internal/activator	234.385s
ok  	github.com/carlosframework/platform/internal/agentsock	0.019s
ok  	github.com/carlosframework/platform/internal/alerting	0.007s
ok  	github.com/carlosframework/platform/internal/analytics	0.036s
ok  	github.com/carlosframework/platform/internal/appconfig	2.318s
ok  	github.com/carlosframework/platform/internal/apperr	0.092s
ok  	github.com/carlosframework/platform/internal/applog	0.145s
ok  	github.com/carlosframework/platform/internal/awsinv	0.010s
ok  	github.com/carlosframework/platform/internal/boxchoice	0.010s
ok  	github.com/carlosframework/platform/internal/boxid	0.625s
?   	github.com/carlosframework/platform/internal/boxid/boxidtest	[no test files]
?   	github.com/carlosframework/platform/internal/boxid/internal/signtest	[no test files]
ok  	github.com/carlosframework/platform/internal/boxvolumes	0.011s
ok  	github.com/carlosframework/platform/internal/caps	0.024s
ok  	github.com/carlosframework/platform/internal/certwarm	0.002s
ok  	github.com/carlosframework/platform/internal/cgroup	0.698s
ok  	github.com/carlosframework/platform/internal/changelog	0.298s
ok  	github.com/carlosframework/platform/internal/clamd	0.058s
?   	github.com/carlosframework/platform/internal/cliflag	[no test files]
ok  	github.com/carlosframework/platform/internal/console	364.970s
?   	github.com/carlosframework/platform/internal/controlname	[no test files]
?   	github.com/carlosframework/platform/internal/controlsock	[no test files]
ok  	github.com/carlosframework/platform/internal/dbdoor	5.639s
ok  	github.com/carlosframework/platform/internal/dbdoor/dbdoortest	0.018s
ok  	github.com/carlosframework/platform/internal/dbverify	0.006s
ok  	github.com/carlosframework/platform/internal/dnsrec	0.006s
ok  	github.com/carlosframework/platform/internal/docsite	0.011s
ok  	github.com/carlosframework/platform/internal/domains	2.120s
ok  	github.com/carlosframework/platform/internal/doorkey	0.012s
ok  	github.com/carlosframework/platform/internal/economics	0.006s
ok  	github.com/carlosframework/platform/internal/edge	26.001s
?   	github.com/carlosframework/platform/internal/edgeproof	[no test files]
ok  	github.com/carlosframework/platform/internal/edgesfile	0.014s
ok  	github.com/carlosframework/platform/internal/fleetapi	0.135s
ok  	github.com/carlosframework/platform/internal/gate	1.665s
ok  	github.com/carlosframework/platform/internal/genselect	0.030s
ok  	github.com/carlosframework/platform/internal/geo	0.027s
ok  	github.com/carlosframework/platform/internal/geoip	0.038s
ok  	github.com/carlosframework/platform/internal/host	138.304s
ok  	github.com/carlosframework/platform/internal/hostactors	0.343s
ok  	github.com/carlosframework/platform/internal/hostbox	2.138s
ok  	github.com/carlosframework/platform/internal/hostclaim	0.507s
ok  	github.com/carlosframework/platform/internal/hostevidence	0.015s
ok  	github.com/carlosframework/platform/internal/hostexec	4.845s
ok  	github.com/carlosframework/platform/internal/hostfence	0.042s
ok  	github.com/carlosframework/platform/internal/hostfiles	10.403s
ok  	github.com/carlosframework/platform/internal/hostfloor	0.015s
ok  	github.com/carlosframework/platform/internal/hostguard	0.982s
ok  	github.com/carlosframework/platform/internal/hostops	1.450s
?   	github.com/carlosframework/platform/internal/hostops/fencerelease	[no test files]
ok  	github.com/carlosframework/platform/internal/hostops/internal/fencerelease	0.015s
ok  	github.com/carlosframework/platform/internal/hostquarantine	0.015s
ok  	github.com/carlosframework/platform/internal/hostrelocate	0.468s
ok  	github.com/carlosframework/platform/internal/hostresolve	0.139s
ok  	github.com/carlosframework/platform/internal/hostwrite	0.015s
ok  	github.com/carlosframework/platform/internal/infraview	0.015s
ok  	github.com/carlosframework/platform/internal/instancectl	1.816s
ok  	github.com/carlosframework/platform/internal/instanceexec	2.710s
ok  	github.com/carlosframework/platform/internal/instancefs	0.021s
ok  	github.com/carlosframework/platform/internal/instancekey	0.015s
ok  	github.com/carlosframework/platform/internal/instanceupdates	0.021s
ok  	github.com/carlosframework/platform/internal/layout	1.421s
ok  	github.com/carlosframework/platform/internal/lease	9.480s
ok  	github.com/carlosframework/platform/internal/lease/leasetest	0.021s
ok  	github.com/carlosframework/platform/internal/ledger	0.024s
ok  	github.com/carlosframework/platform/internal/mail	0.149s
ok  	github.com/carlosframework/platform/internal/maintenance	0.085s
ok  	github.com/carlosframework/platform/internal/maintenanceexec	5.497s
ok  	github.com/carlosframework/platform/internal/maintenanceguard	0.075s
ok  	github.com/carlosframework/platform/internal/metrics	0.005s
ok  	github.com/carlosframework/platform/internal/moves	20.945s
ok  	github.com/carlosframework/platform/internal/moves/archive	0.761s
ok  	github.com/carlosframework/platform/internal/objstore	0.167s
ok  	github.com/carlosframework/platform/internal/ops	0.169s
ok  	github.com/carlosframework/platform/internal/park	0.199s
ok  	github.com/carlosframework/platform/internal/privatefile	0.003s
ok  	github.com/carlosframework/platform/internal/provision	11.125s
ok  	github.com/carlosframework/platform/internal/publicrelease	0.008s
?   	github.com/carlosframework/platform/internal/regionname	[no test files]
ok  	github.com/carlosframework/platform/internal/registry	4.018s
ok  	github.com/carlosframework/platform/internal/relay	7.233s
ok  	github.com/carlosframework/platform/internal/release	18.966s
ok  	github.com/carlosframework/platform/internal/rollout	3.655s
ok  	github.com/carlosframework/platform/internal/rolloutwords	0.003s
ok  	github.com/carlosframework/platform/internal/rootaws	0.007s
ok  	github.com/carlosframework/platform/internal/rootparticipation	0.007s
ok  	github.com/carlosframework/platform/internal/rootstate	0.028s
ok  	github.com/carlosframework/platform/internal/s3compat	0.005s
ok  	github.com/carlosframework/platform/internal/safefile	0.004s
ok  	github.com/carlosframework/platform/internal/schedule	2.272s
ok  	github.com/carlosframework/platform/internal/sim/hostlife	8.951s
ok  	github.com/carlosframework/platform/internal/sim/litestreamfacts	3.751s
ok  	github.com/carlosframework/platform/internal/status	0.043s
ok  	github.com/carlosframework/platform/internal/steering	0.006s
ok  	github.com/carlosframework/platform/internal/stopevidence	0.325s
ok  	github.com/carlosframework/platform/internal/svcrole	0.002s
ok  	github.com/carlosframework/platform/internal/telemetry	0.004s
ok  	github.com/carlosframework/platform/internal/termfence	0.004s
ok  	github.com/carlosframework/platform/internal/tunnelclient	0.004s
ok  	github.com/carlosframework/platform/internal/unixdial	0.002s
ok  	github.com/carlosframework/platform/internal/updatewords	0.007s
ok  	github.com/carlosframework/platform/internal/usage	0.005s
ok  	github.com/carlosframework/platform/internal/version	0.002s
ok  	github.com/carlosframework/platform/internal/vet	1.543s
=== cross ===
--- shard 0 of 5 ---
cross-compiling darwin/arm64
--- shard 1 of 5 ---
cross-compiling darwin/amd64
--- shard 2 of 5 ---
cross-compiling linux/arm64
--- shard 3 of 5 ---
cross-compiling windows/amd64
--- shard 4 of 5 ---
cross-compiling windows/arm64
=== minio-contract ===
minio contract: arm64, minio RELEASE.2025-09-07T16-13-09Z, mc RELEASE.2025-08-13T08-35-41Z
  minio: cached, digest verified
  mc: cached, digest verified
  minio healthy on 127.0.0.1:20999
Added `local` successfully.
Bucket created successfully `local/contract-bucket`.
Bucket created successfully `local/denied-bucket`.
Created policy `contract-only` successfully.
Added user `scopeduser` successfully.
Attached Policies: [contract-only]
To User: scopeduser
  buckets and scoped user ready
=== RUN   TestDeliveredContractMinio
=== RUN   TestDeliveredContractMinio/put
=== RUN   TestDeliveredContractMinio/get_round-trips_the_body
=== RUN   TestDeliveredContractMinio/list_shows_the_key
=== RUN   TestDeliveredContractMinio/head_and_list_agree_on_the_version
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed
=== RUN   TestDeliveredContractMinio/delete_removes_it
=== RUN   TestDeliveredContractMinio/cross-bucket_denial
--- PASS: TestDeliveredContractMinio (0.02s)
    --- PASS: TestDeliveredContractMinio/put (0.00s)
    --- PASS: TestDeliveredContractMinio/get_round-trips_the_body (0.00s)
    --- PASS: TestDeliveredContractMinio/list_shows_the_key (0.00s)
    --- PASS: TestDeliveredContractMinio/head_and_list_agree_on_the_version (0.00s)
    --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload (0.00s)
    --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route (0.01s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed (0.00s)
    --- PASS: TestDeliveredContractMinio/delete_removes_it (0.00s)
    --- PASS: TestDeliveredContractMinio/cross-bucket_denial (0.00s)
PASS
ok  	github.com/carlosframework/platform/internal/objstore	0.023s
=== RUN   TestT4StorageProbeMinio
2026/10/14 17:37:49 moves: T4: STORE: list: operation error S3: ListObjectsV2, https response error StatusCode: 403, RequestID: 18DE7605F2FA35C3, HostID: 7c420053e0fbcd7f41653fdd92ffa2802c903d05303b1aa15adc2e8bb65d183f, api error SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.
--- PASS: TestT4StorageProbeMinio (0.01s)
PASS
ok  	github.com/carlosframework/platform/internal/host	0.022s
minio contract: PASS

Intent

full ledger: feat/claim-files-backfill

diff --git a/internal/host/claimretire_transport.go b/internal/host/claimretire_transport.go
index 4c174187a..3495b03bd 100644
--- a/internal/host/claimretire_transport.go
+++ b/internal/host/claimretire_transport.go
@@ -34,7 +34,10 @@ type ClaimRootHTTP struct {
Roots map[string]string
}
-func (h ClaimRootHTTP) RecoverPending(ctx context.Context) error {
+func (h ClaimRootHTTP) RecoverPending(ctx context.Context) (recoveryErr error) {
+ // Serving intents retain their own authority checks even when another
+ // startup recovery failed. Attempt them before the service accepts work.
+ defer func() { recoveryErr = errors.Join(recoveryErr, h.servingParks.recoverPending(ctx)) }()
cold := hostbox.ColdHandover{Guard: h.Stop.Coordinator.Guard, Claims: h.Stop.Coordinator.Evidence, Registry: h.Reconcile.Registry, Inventory: h.Reconcile.Inventory, Launches: h.Launches, Roots: h.Roots, JournalDir: filepath.Join(h.Reconcile.Retirement.JournalDir, "cold"), VerifyReceipt: verifyRetirementReceipt}
if err := cold.Recover(ctx); err != nil {
diff --git a/internal/host/claimservingpark_linux.go b/internal/host/claimservingpark_linux.go
index ddb47bd02..78f07a7a2 100644
--- a/internal/host/claimservingpark_linux.go
+++ b/internal/host/claimservingpark_linux.go
@@ -403,6 +403,9 @@ func (f *servingParkFS) Move(r *servingParkRecord, save func() error) error {
}
func (f *servingParkFS) Delete(r *servingParkRecord, save func() error) error {
+ if r.EventRemoveIntent {
+ return f.removeEvent(r, save)
+ }
root, event, err := f.event(r)
if err != nil {
return err
@@ -483,12 +486,30 @@ func (f *servingParkFS) Delete(r *servingParkRecord, save func() error) error {
return e
}
}
- // Keep the recorded empty event until tombstone cleanup is implemented.
- // It owns no database blocks and cannot be reused by another retirement.
- return nil
+ return f.removeEvent(r, save)
}
func (f *servingParkFS) CheckParked(r *servingParkRecord) error {
+ if r.EventRemoved {
+ if err := f.removedEvent(r); err != nil {
+ return err
+ }
+ source, parent, err := f.source(r.Authorization)
+ if err != nil {
+ return err
+ }
+ defer source.Close()
+ if parent != r.Source.Parent {
+ return hostbox.ErrChanged
+ }
+ for _, item := range r.Source.Files {
+ actual, err := servingParkStat(source, item.Name)
+ if err != nil || actual.Present {
+ return hostbox.ErrChanged
+ }
+ }
+ return nil
+ }
root, event, err := f.event(r)
if err != nil {
return err
@@ -533,3 +554,85 @@ func (f *servingParkFS) CheckParked(r *servingParkRecord) error {
}
return nil
}
+
+// removeEvent removes only a recorded empty event. The durable intent makes
+// absence after a lost unlink reply replayable; it never licenses a replacement.
+func (f *servingParkFS) removeEvent(r *servingParkRecord, save func() error) error {
+ if !r.Deleting || !r.Prepared || !r.Parked {
+ return hostbox.ErrChanged
+ }
+ for _, item := range r.Source.Files {
+ if item.Present && !item.Deleted {
+ return hostbox.ErrChanged
+ }
+ }
+ root, n, err := f.namespace()
+ if err != nil {
+ return err
+ }
+ defer root.Close()
+ if n.Parent != r.Source.Parent {
+ return hostbox.ErrChanged
+ }
+ event, id, err := f.privateDir(root, r.Authorization.ID)
+ if errors.Is(err, unix.ENOENT) {
+ if !r.EventRemoveIntent {
+ return hostbox.ErrChanged
+ }
+ } else if err != nil {
+ return err
+ } else {
+ defer event.Close()
+ if r.EventRemoved || id != r.Event || id.Device != n.Root.Device || id.Mount != n.Root.Mount {
+ return hostbox.ErrChanged
+ }
+ names, e := event.Readdirnames(1)
+ if e != nil && !errors.Is(e, io.EOF) {
+ return e
+ }
+ if len(names) != 0 {
+ return hostbox.ErrChanged
+ }
+ if !r.EventRemoveIntent {
+ r.EventRemoveIntent = true
+ if err = save(); err != nil {
+ return err
+ }
+ }
+ // Root owns the parent and namespace lock, including during recovery.
+ if err = f.unlink(int(root.Fd()), r.Authorization.ID, unix.AT_REMOVEDIR); err != nil {
+ return err
+ }
+ }
+ if err = f.sync(int(root.Fd())); err != nil {
+ return err
+ }
+ if !r.EventRemoved {
+ r.EventRemoved = true
+ return save()
+ }
+ return nil
+}
+
+func (f *servingParkFS) removedEvent(r *servingParkRecord) error {
+ if !r.EventRemoveIntent || !r.EventRemoved {
+ return hostbox.ErrChanged
+ }
+ root, n, err := f.namespace()
+ if err != nil {
+ return err
+ }
+ defer root.Close()
+ if n.Parent != r.Source.Parent {
+ return hostbox.ErrChanged
+ }
+ d, _, err := f.privateDir(root, r.Authorization.ID)
+ if err == nil {
+ d.Close()
+ return hostbox.ErrChanged
+ }
+ if !errors.Is(err, unix.ENOENT) {
+ return err
+ }
+ return nil
+}
diff --git a/internal/host/claimservingpark_records.go b/internal/host/claimservingpark_records.go
index 0f36b8bca..57a0cdc4c 100644
--- a/internal/host/claimservingpark_records.go
+++ b/internal/host/claimservingpark_records.go
@@ -44,6 +44,7 @@ type servingParkSource struct {
type servingParkRecord struct {
LegacyEvidence *servingLegacyEvidence `json:",omitempty"`
Continuations []servingParkContinuation `json:",omitempty"`
+ EventRemoveIntent, EventRemoved bool `json:",omitempty"`
Version int
Authorization servingParkAuthorization
Source servingParkSource
@@ -211,6 +212,16 @@ func validateServingParkRecord(r servingParkRecord) error {
if r.Deleting && !r.Parked {
return hostbox.ErrChanged
}
+ if r.EventRemoved && !r.EventRemoveIntent || r.EventRemoveIntent && !r.Deleting {
+ return hostbox.ErrChanged
+ }
+ if r.EventRemoveIntent {
+ for _, item := range r.Source.Files {
+ if item.Present && !item.Deleted {
+ return hostbox.ErrChanged
+ }
+ }
+ }
if r.Closed && !r.Deleting {
return hostbox.ErrChanged
}
diff --git a/internal/host/claimservingpark_recovery.go b/internal/host/claimservingpark_recovery.go
new file mode 100644
index 000000000..c3e0c0eb5
--- /dev/null
+++ b/internal/host/claimservingpark_recovery.go
@@ -0,0 +1,151 @@
+package host
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "sort"
+ "strings"
+
+ "github.com/carlosframework/platform/internal/hostbox"
+)
+
+const servingParkRecoveryLimit = 128
+
+var errServingParkRetagPending = errors.New("the earlier park is incomplete; the next wake or root service restart completes it before a new park")
+
+type servingParkRecoveryResult struct {
+ Host, ID string
+ Err error
+}
+
+type servingParkRecoveryPage struct {
+ Results []servingParkRecoveryResult
+ Next string
+ More bool
+}
+
+// The cursor bounds record processing and lets a later pass reach records
+// beyond a blocked host. Store.Names still enumerates the private directory.
+func (l *servingParkLedger) recoveryIDs(ctx context.Context, after string, limit int) ([]string, bool, error) {
+ if ctx == nil || l == nil || l.state == nil || limit < 1 || limit > servingParkRecoveryLimit {
+ return nil, false, hostbox.ErrUnsupported
+ }
+ if err := ctx.Err(); err != nil {
+ return nil, false, err
+ }
+ names, err := l.state.Names()
+ if err != nil {
+ return nil, false, err
+ }
+ var ids []string
+ for _, name := range names {
+ if strings.HasPrefix(name, "retirement-") && strings.HasSuffix(name, ".json") {
+ id := strings.TrimSuffix(strings.TrimPrefix(name, "retirement-"), ".json")
+ if id > after {
+ ids = append(ids, id)
+ }
+ }
+ }
+ sort.Strings(ids)
+ more := len(ids) > limit
+ if more {
+ ids = ids[:limit]
+ }
+ return ids, more, ctx.Err()
+}
+
+// Completed deletion needs no live claim or agent. Its original durable cause
+// and recorded identities remain the authority after a host is removed.
+func (l *servingParkLedger) recoverCompleted(ctx context.Context, id string) error {
+ return l.withNamespace(ctx, func() error {
+ r, err := l.read(id)
+ if err != nil {
+ return err
+ }
+ unlock, err := l.state.Lock(ctx, "slot-"+servingParkKey(r.Authorization.Successor.Box, r.Authorization.Successor.Host)+".lock")
+ if err != nil {
+ return err
+ }
+ defer unlock()
+ unlockRecord, err := l.state.Lock(ctx, "retirement-"+id+".lock")
+ if err != nil {
+ return err
+ }
+ defer unlockRecord()
+ if !r.Complete {
+ return hostbox.ErrChanged
+ }
+ if r.Deleting {
+ if !r.Closed {
+ err = l.deleteLocked(ctx, &r, r.Cause)
+ } else {
+ // Older records can be closed with their empty event retained.
+ err = l.fs.Delete(&r, func() error {
+ if err := ctx.Err(); err != nil {
+ return err
+ }
+ return l.save(r)
+ })
+ }
+ if err != nil {
+ return err
+ }
+ }
+ // Completion may be durable while clearing Pending lost its reply.
+ // Last and ID are replay references and survive this cleanup.
+ slot, err := l.slot(r.Authorization)
+ if err != nil {
+ return err
+ }
+ if slot.Pending == id {
+ if err = ctx.Err(); err != nil {
+ return err
+ }
+ slot.Pending = ""
+ return l.write("slot-"+servingParkKey(slot.Box, slot.Host)+".json", slot)
+ }
+ return nil
+ })
+}
+
+func (p *servingParkRuntime) recoverPending(ctx context.Context) error {
+ if p == nil {
+ return nil
+ }
+ call, done, err := p.begin(ctx)
+ if err != nil {
+ return err
+ }
+ defer done()
+ recovery, ok := p.authority.(interface {
+ recoverServingParks(context.Context, string, int) (servingParkRecoveryPage, error)
+ })
+ if !ok {
+ return hostbox.ErrUnsupported
+ }
+ page, err := recovery.recoverServingParks(call, "", servingParkRecoveryLimit)
+ var failures []error
+ if err != nil {
+ failures = append(failures, hostbox.ErrChanged)
+ }
+ for _, result := range page.Results {
+ if result.Err != nil {
+ // Draft retirement.reap_failed and reason.park_pending from the index.
+ reason := "a local park operation failed; inspect the park listing and root service journal"
+ if errors.Is(result.Err, errServingParkRetagPending) {
+ reason = errServingParkRetagPending.Error()
+ } else if errors.Is(result.Err, hostbox.ErrChanged) || errors.Is(result.Err, ErrClaimStopReceipt) {
+ reason = "the parked artifact or its parent no longer matches the root-recorded identity"
+ }
+ if result.Host == "unknown" {
+ reason = "the private retirement record could not be validated; retain the data for investigation"
+ }
+ failures = append(failures, fmt.Errorf("Host %q: serving database retirement %q retained: %s.", result.Host, result.ID, reason))
+ }
+ }
+ if page.More {
+ failures = append(failures, hostbox.ErrChanged)
+ }
+ return errors.Join(failures...)
+}
diff --git a/internal/host/claimservingpark_recovery_linux.go b/internal/host/claimservingpark_recovery_linux.go
new file mode 100644
index 000000000..df0e58392
--- /dev/null
+++ b/internal/host/claimservingpark_recovery_linux.go
@@ -0,0 +1,110 @@
+//go:build linux
+
+package host
+
+import (
+ "context"
+ "reflect"
+
+ "github.com/carlosframework/platform/internal/hostbox"
+ "github.com/carlosframework/platform/internal/hostclaim"
+ "github.com/carlosframework/platform/internal/hostguard"
+)
+
+// Startup resumes only an existing immutable intent for its active successor.
+// A newer successor needs the ordinary request's stopped-continuation checks.
+// Incomplete paired retagging remains the independent claim writer's work.
+func (p *servingParkRoot) recoverServingPark(ctx context.Context, saved servingParkRecord) error {
+ active := saved.activeAuthorization()
+ return p.withRootInstallation(ctx, active.Successor, ctx.Err, func(o hostclaim.Observed, view hostguard.FenceView, actor hostbox.Actor, dir servingArtifactDirectory, tags uint64) error {
+ r, err := p.ledger.read(saved.Authorization.ID)
+ if err != nil || !reflect.DeepEqual(saved, r) || r.Complete || active.Directory != dir || (tags != active.TagsFrom && tags != active.Successor.Stamp) {
+ return hostbox.ErrChanged
+ }
+ if err = validateServingRootHistory(ctx, o, view, actor, r); err != nil {
+ return err
+ }
+ // Any main-role intent is ambiguous at startup, even if a start reply
+ // was lost. Never resume a rename beneath a potentially live successor.
+ if err = p.ledger.legacyUnlaunched(ctx, active.Successor, actor.Enrollment); err != nil {
+ return err
+ }
+ if r.Authorization.Legacy {
+ consumed, e := p.ledger.legacyConsumption(active.Successor)
+ if e != nil || consumed == nil || !sameServingLegacyConsumption(*consumed, r) {
+ return hostbox.ErrChanged
+ }
+ if err = p.validateLegacyRootEvidence(ctx, o, view, actor, r.Authorization, r.LegacyEvidence, r.Receipts); err != nil {
+ return err
+ }
+ } else {
+ prior, e := servingPredecessor(o.Claim, active.Successor, r.Authorization.Predecessor.Stamp, r.Authorization.Predecessor.Generation)
+ if e != nil || prior != r.Authorization.Predecessor {
+ return hostbox.ErrChanged
+ }
+ receipts, e := servingRootStops(ctx, view, prior, actor)
+ if e != nil || !sameServingRootStops(receipts, r.Receipts) {
+ return ErrClaimStopReceipt
+ }
+ disposition, e := servingGenerationParkDisposition(ctx, p.ledger.journal, prior, receipts, r.Source.Files[0].Present)
+ if e != nil || disposition != r.Disposition {
+ return hostbox.ErrChanged
+ }
+ }
+ r, err = p.ledger.parkWithEvidenceLocked(ctx, r.Authorization, r.Receipts, r.LegacyEvidence)
+ if err != nil {
+ return err
+ }
+ if tags != active.Successor.Stamp {
+ return errServingParkRetagPending
+ }
+ return p.ledger.completeLocked(ctx, r.Authorization.ID, active.Successor)
+ })
+}
+
+func sameServingLegacyConsumption(c servingLegacyConsumption, r servingParkRecord) bool {
+ if c.Authorization != r.Authorization || !sameServingRootStops(c.Receipts, r.Receipts) || !sameServingLegacyEvidence(c.LegacyEvidence, r.LegacyEvidence) || c.Source.Directory != r.Source.Directory || c.Source.Parent != r.Source.Parent {
+ return false
+ }
+ for i, item := range c.Source.Files {
+ other := r.Source.Files[i]
+ item.Moved, item.DeleteIntent, item.Deleted = false, false, false
+ other.Moved, other.DeleteIntent, other.Deleted = false, false, false
+ if item != other {
+ return false
+ }
+ }
+ return true
+}
+
+func (p *servingParkRoot) recoverServingParks(ctx context.Context, after string, limit int) (servingParkRecoveryPage, error) {
+ var out servingParkRecoveryPage
+ if p == nil || p.ledger == nil {
+ return out, hostbox.ErrUnsupported
+ }
+ ids, more, err := p.ledger.recoveryIDs(ctx, after, limit)
+ if err != nil {
+ return out, err
+ }
+ out.More = more
+ for _, id := range ids {
+ if err = ctx.Err(); err != nil {
+ out.More = true
+ return out, err
+ }
+ r, readErr := p.ledger.read(id)
+ result := servingParkRecoveryResult{Host: "unknown", ID: id, Err: readErr}
+ if readErr == nil {
+ result.Host = r.Authorization.Successor.Host
+ switch {
+ case !r.Complete:
+ result.Err = p.recoverServingPark(ctx, r)
+ default:
+ result.Err = p.ledger.recoverCompleted(ctx, id)
+ }
+ }
+ out.Results = append(out.Results, result)
+ out.Next = id
+ }
+ return out, nil
+}
diff --git a/internal/host/claimservingpark_recovery_linux_test.go b/internal/host/claimservingpark_recovery_linux_test.go
new file mode 100644
index 000000000..14503a6e6
--- /dev/null
+++ b/internal/host/claimservingpark_recovery_linux_test.go
@@ -0,0 +1,391 @@
+//go:build linux
+
+package host
+
+import (
+ "context"
+ "errors"
+ "net"
+ "os"
+ "path/filepath"
+ "reflect"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/carlosframework/platform/internal/hostbox"
+ "github.com/carlosframework/platform/internal/hostclaim"
+ "golang.org/x/sys/unix"
+)
+
+func recoveryPage(t *testing.T, p *servingParkRoot) servingParkRecoveryPage {
+ t.Helper()
+ page, err := p.recoverServingParks(t.Context(), "", servingParkRecoveryLimit)
+ if err != nil || page.More || len(page.Results) != 1 {
+ t.Fatalf("recovery page: %+v %v", page, err)
+ }
+ return page
+}
+
+func TestServingParkStartupResumesRecordedIntent(t *testing.T) {
+ for _, boundary := range []string{"first-rename", "last-rename", "parked", "retagged", "complete-slot-pending"} {
+ t.Run(boundary, func(t *testing.T) {
+ f := newServingRootFixture(t, true)
+ fs := f.p.ledger.fs.(*servingParkFS)
+ count := 0
+ fs.rename = func(a int, b string, c int, d string, flags uint) error {
+ err := unix.Renameat2(a, b, c, d, flags)
+ count++
+ if err == nil && (boundary == "first-rename" && count == 1 || boundary == "last-rename" && count == 4) {
+ return unix.EIO
+ }
+ return err
+ }
+ a, err := f.park()
+ if strings.HasSuffix(boundary, "rename") {
+ if !errors.Is(err, unix.EIO) {
+ t.Fatalf("lost rename reply: %v", err)
+ }
+ a.ID = servingRetirementID(f.prior, f.current, false)
+ } else if err != nil {
+ t.Fatal(err)
+ }
+ before, err := f.p.ledger.read(a.ID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if boundary == "retagged" || boundary == "complete-slot-pending" {
+ f.tags(f.current.Stamp)
+ }
+ if boundary == "complete-slot-pending" {
+ before.Complete = true
+ if err = f.p.ledger.save(before); err != nil {
+ t.Fatal(err)
+ }
+ }
+ claimBefore := f.f.read()
+ // New authority instance, same durable stores. Startup has no agent
+ // connection and must not invoke a transport peer callback.
+ restarted := *f.p
+ ledger := *f.p.ledger
+ restarted.ledger = &ledger
+ newfs := newServingParkFS(fs.data, fs.state)
+ newfs.owner, newfs.mount = fs.owner, fs.mount
+ ledger.fs = newfs
+ restarted.root.Stop.Authenticate = func(context.Context, net.Conn) (StopPeer, error) { panic("startup authenticated an agent") }
+ page := recoveryPage(t, &restarted)
+ wantComplete := boundary == "retagged" || boundary == "complete-slot-pending"
+ if wantComplete && page.Results[0].Err != nil || !wantComplete && !errors.Is(page.Results[0].Err, errServingParkRetagPending) {
+ t.Fatalf("unexpected recovery: %+v", page.Results)
+ }
+ after, err := ledger.read(a.ID)
+ if err != nil || !after.Parked || after.Complete != wantComplete || after.Authorization != before.Authorization || !sameServingRootStops(after.Receipts, before.Receipts) {
+ t.Fatalf("changed intent or incomplete rename: %+v %v", after, err)
+ }
+ if !before.ParkedAt.IsZero() && (!after.ParkedAt.Equal(before.ParkedAt) || !after.ReapAfter.Equal(before.ReapAfter)) {
+ t.Fatal("startup refreshed retention")
+ }
+ if !reflect.DeepEqual(f.f.read(), claimBefore) {
+ t.Fatal("startup changed claim or retry state")
+ }
+ for suffix, want := range map[string]string{"": "served bytes", ".gen": "G", "-wal": "WAL", "-shm": "SHM"} {
+ body, err := os.ReadFile(filepath.Join(f.f.guard.DataDir, "instances", ".serving-parked", a.ID, f.current.Host+".db"+suffix))
+ if err != nil || string(body) != want {
+ t.Fatalf("retained bytes %s: %q %v", suffix, body, err)
+ }
+ }
+ if !wantComplete {
+ f.tags(f.current.Stamp)
+ if page = recoveryPage(t, &restarted); page.Results[0].Err != nil {
+ t.Fatal(page.Results[0].Err)
+ }
+ }
+ slot, err := ledger.slot(after.Authorization)
+ if err != nil || slot.Pending != "" || slot.Last != a.ID || slot.ID != a.ID {
+ t.Fatalf("completion replay references: %+v %v", slot, err)
+ }
+ })
+ }
+}
+
+func TestServingParkStartupRefusesChangedAuthority(t *testing.T) {
+ for _, change := range []string{"enrollment", "revision", "claim", "tags", "directory", "receipt", "launch-intent", "census", "cancel"} {
+ t.Run(change, func(t *testing.T) {
+ f := newServingRootFixture(t, true)
+ a, err := f.park()
+ if err != nil {
+ t.Fatal(err)
+ }
+ f.tags(f.current.Stamp)
+ before, _ := f.disk.Read("retirement-" + a.ID + ".json")
+ ctx := t.Context()
+ switch change {
+ case "enrollment":
+ f.actor.Enrollment = "new-enrollment"
+ case "revision":
+ f.actor.Revision++
+ case "claim":
+ _, err = hostclaim.Update(ctx, f.f.claims, f.f.read(), func(c *hostclaim.Claim) error { c.Holder.WriterBox = "other-box"; return nil })
+ case "tags":
+ f.tags(f.current.Stamp + 1)
+ case "directory":
+ err = os.Rename(filepath.Dir(f.db), filepath.Dir(f.db)+"-saved")
+ if err == nil {
+ err = os.Mkdir(filepath.Dir(f.db), 0700)
+ }
+ case "receipt":
+ f.mutateRoot(func(m map[string]any) { m["State"].(map[string]any)["fence_journal"] = []any{} })
+ case "launch-intent":
+ target := f.receipt.Target
+ target.Stamp, target.Session = f.current.Stamp, f.current.Session
+ err = f.f.journal.Prelaunch(ctx, hostbox.LaunchInventory{Target: target, ControllerPID: 77, ProcessStart: "edge-start", NoState: true, Groups: []hostbox.LaunchGroup{{RelativePath: "carlos/parent", Device: 1, Inode: 2}}})
+ if err == nil {
+ err = f.f.journal.IntendUnit(ctx, hostbox.LaunchReservation{ID: "new:main", Role: "main", InvocationID: "new", Inputs: "digest", Target: target, Group: hostbox.LaunchGroup{RelativePath: "system.slice/carlos-app@shop.service"}, Executable: hostbox.LaunchFile{Device: 1, Inode: 3}})
+ }
+ case "census":
+ f.p.root.RequireNewWork = func(context.Context) error { return hostbox.ErrChanged }
+ case "cancel":
+ var cancel context.CancelFunc
+ ctx, cancel = context.WithCancel(ctx)
+ cancel()
+ }
+ if err != nil {
+ t.Fatal(err)
+ }
+ page, err := f.p.recoverServingParks(ctx, "", 128)
+ if err == nil && (len(page.Results) != 1 || page.Results[0].Err == nil) {
+ t.Fatalf("accepted changed %s: %+v", change, page)
+ }
+ after, _ := f.disk.Read("retirement-" + a.ID + ".json")
+ if string(after) != string(before) {
+ t.Fatal("refusal changed private retirement")
+ }
+ })
+ }
+}
+
+func TestServingParkStartupDeletionAndEmptyEvent(t *testing.T) {
+ for _, boundary := range []string{"unlink", "rmdir", "closed-old-event", "replacement", "unknown-child", "missing-no-intent"} {
+ t.Run(boundary, func(t *testing.T) {
+ f := newServingParkDiskFixture(t)
+ f.put("", "retain")
+ a, receipts := f.evidence(10, true, "missing")
+ _, err := f.park(a, receipts)
+ if err != nil {
+ t.Fatal(err)
+ }
+ f.complete(a)
+ f.now = f.now.Add(7 * 24 * time.Hour)
+ event := filepath.Dir(f.path(a, ""))
+ held, err := os.Open(event)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer held.Close()
+ f.fs.unlink = func(fd int, name string, flags int) error {
+ if boundary == "closed-old-event" && flags == unix.AT_REMOVEDIR {
+ return unix.EIO
+ }
+ err := unix.Unlinkat(fd, name, flags)
+ if err == nil && (boundary == "unlink" && flags == 0 || boundary != "unlink" && flags == unix.AT_REMOVEDIR) {
+ return unix.EIO
+ }
+ return err
+ }
+ err = f.ledger.withNamespace(t.Context(), func() error { return f.ledger.reapLocked(t.Context(), a.ID) })
+ if !errors.Is(err, unix.EIO) {
+ t.Fatalf("missing fault: %v", err)
+ }
+ r, err := f.ledger.read(a.ID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if boundary == "closed-old-event" {
+ r.Closed, r.EventRemoveIntent = true, false
+ if err = f.ledger.save(r); err != nil {
+ t.Fatal(err)
+ }
+ }
+ if boundary == "replacement" || boundary == "unknown-child" {
+ if err = os.Mkdir(event, 0700); err != nil {
+ t.Fatal(err)
+ }
+ if boundary == "unknown-child" {
+ err = os.WriteFile(filepath.Join(event, "victim"), []byte("keep"), 0600)
+ }
+ if err != nil {
+ t.Fatal(err)
+ }
+ }
+ if boundary == "missing-no-intent" {
+ r.EventRemoveIntent = false
+ if err = f.ledger.save(r); err != nil {
+ t.Fatal(err)
+ }
+ }
+ f.fs.unlink = unix.Unlinkat
+ // No live claim/root host installation is needed for a deletion
+ // already authorized by a complete private retirement.
+ err = f.ledger.recoverCompleted(t.Context(), a.ID)
+ refuse := boundary == "replacement" || boundary == "unknown-child" || boundary == "missing-no-intent"
+ if (err != nil) != refuse {
+ t.Fatalf("recovery: %v, refuse=%v", err, refuse)
+ }
+ if refuse {
+ if boundary == "replacement" || boundary == "unknown-child" {
+ if _, err = os.Lstat(event); err != nil {
+ t.Fatal("refusal removed replacement", err)
+ }
+ }
+ if boundary == "unknown-child" {
+ body, err := os.ReadFile(filepath.Join(event, "victim"))
+ if err != nil || string(body) != "keep" {
+ t.Fatal("refusal removed unknown child", err)
+ }
+ }
+ return
+ }
+ r, err = f.ledger.read(a.ID)
+ if err != nil || !r.Closed || !r.EventRemoved {
+ t.Fatalf("unfinished cleanup: %+v %v", r, err)
+ }
+ if _, err = os.Lstat(event); !errors.Is(err, os.ErrNotExist) {
+ t.Fatal("empty event survived", err)
+ }
+ slot, err := f.ledger.slot(a)
+ if err != nil || slot.ID != a.ID || slot.Last != a.ID {
+ t.Fatal("lost replay reference", err)
+ }
+ if err = f.ledger.recoverCompleted(t.Context(), a.ID); err != nil {
+ t.Fatal("cleanup replay", err)
+ }
+ })
+ }
+}
+
+func TestServingParkStartupBoundedScanContinuesAfterCorruptRecord(t *testing.T) {
+ f := newServingRootFixture(t, true)
+ a, err := f.park()
+ if err != nil {
+ t.Fatal(err)
+ }
+ f.tags(f.current.Stamp)
+ if err = f.disk.Write("retirement-0000.json", []byte(`{}`)); err != nil {
+ t.Fatal(err)
+ }
+ page, err := f.p.recoverServingParks(t.Context(), "", 1)
+ if err != nil || !page.More || len(page.Results) != 1 || page.Results[0].Host != "unknown" || page.Results[0].Err == nil {
+ t.Fatalf("first page: %+v %v", page, err)
+ }
+ page, err = f.p.recoverServingParks(t.Context(), page.Next, 1)
+ if err != nil || page.More || len(page.Results) != 1 || page.Results[0].ID != a.ID || page.Results[0].Err != nil {
+ t.Fatalf("second page: %+v %v", page, err)
+ }
+}
+
+func TestServingParkStartupLegacyKeepsConsumption(t *testing.T) {
+ for _, empty := range []bool{false, true} {
+ t.Run(map[bool]string{false: "legacy", true: "empty"}[empty], func(t *testing.T) {
+ f, _ := newServingLegacyRootFixture(t, empty)
+ a, err := f.p.parkLegacyServingDatabase(t.Context(), f.conn, f.current, servingLegacyPredecessor{Empty: empty})
+ if err != nil {
+ t.Fatal(err)
+ }
+ key := servingLegacyKey("legacy", f.current)
+ consumedBefore, err := f.disk.Read(key)
+ if err != nil {
+ t.Fatal(err)
+ }
+ stopBefore, err := f.disk.Read(servingLegacyKey("legacy-stop", f.current))
+ if err != nil {
+ t.Fatal(err)
+ }
+ before, err := f.p.ledger.read(a.ID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ f.tags(f.current.Stamp)
+ page := recoveryPage(t, f.p)
+ if page.Results[0].Err != nil {
+ t.Fatal(page.Results[0].Err)
+ }
+ after, err := f.p.ledger.read(a.ID)
+ if err != nil || !after.Complete || after.Authorization != before.Authorization || !after.ReapAfter.Equal(before.ReapAfter) {
+ t.Fatalf("legacy recovery: %+v %v", after, err)
+ }
+ consumedAfter, _ := f.disk.Read(key)
+ stopAfter, _ := f.disk.Read(servingLegacyKey("legacy-stop", f.current))
+ if string(consumedBefore) != string(consumedAfter) || string(stopBefore) != string(stopAfter) {
+ t.Fatal("startup changed legacy evidence")
+ }
+ })
+ }
+}
+
+func TestServingParkStartupRuntimeJoinsBeforeClose(t *testing.T) {
+ f := newServingRootFixture(t, true)
+ if _, err := f.park(); err != nil {
+ t.Fatal(err)
+ }
+ f.tags(f.current.Stamp)
+ entered, release := make(chan struct{}), make(chan struct{})
+ f.p.root.RequireNewWork = func(ctx context.Context) error {
+ close(entered)
+ <-ctx.Done()
+ <-release
+ return ctx.Err()
+ }
+ closed := &servingCloseStore{Store: f.disk}
+ ctx, cancel := context.WithCancel(context.Background())
+ defer cancel()
+ p := &servingParkRuntime{authority: f.p, state: closed, ctx: ctx, cancel: cancel, idle: make(chan struct{})}
+ done := make(chan error, 1)
+ go func() { done <- p.recoverPending(t.Context()) }()
+ select {
+ case <-entered:
+ case <-time.After(5 * time.Second):
+ t.Fatal("recovery did not enter authority")
+ }
+ short, stop := context.WithTimeout(t.Context(), 20*time.Millisecond)
+ defer stop()
+ if err := p.Close(short); !errors.Is(err, context.DeadlineExceeded) || closed.closed.Load() {
+ t.Fatal("closed underneath recovery", err)
+ }
+ close(release)
+ if err := <-done; err == nil {
+ t.Fatal("cancelled recovery succeeded")
+ }
+ if err := p.Close(t.Context()); err != nil || !closed.closed.Load() {
+ t.Fatal("recovery did not join", err)
+ }
+ if err := p.recoverPending(t.Context()); err == nil {
+ t.Fatal("closed runtime recovered")
+ }
+}
+
+func TestServingParkStartupRunsAfterOtherRecoveryFailure(t *testing.T) {
+ f := newServingRootFixture(t, true)
+ a, err := f.park()
+ if err != nil {
+ t.Fatal(err)
+ }
+ f.tags(f.current.Stamp)
+ ctx, cancel := context.WithCancel(t.Context())
+ defer cancel()
+ p := &servingParkRuntime{authority: f.p, state: f.disk, ctx: ctx, cancel: cancel, idle: make(chan struct{})}
+ defer p.Close(t.Context())
+ h := f.p.root
+ h.servingParks = p
+ bad := filepath.Join(t.TempDir(), "not-a-directory")
+ if err = os.WriteFile(bad, nil, 0600); err != nil {
+ t.Fatal(err)
+ }
+ h.Reconcile.Retirement.JournalDir = bad
+ if err = h.RecoverPending(t.Context()); err == nil {
+ t.Fatal("other recovery error was swallowed")
+ }
+ r, err := f.p.ledger.read(a.ID)
+ if err != nil || !r.Complete {
+ t.Fatalf("serving recovery skipped: %+v %v", r, err)
+ }
+}
diff --git a/internal/host/claimservingpark_rootrunner_linux_test.go b/internal/host/claimservingpark_rootrunner_linux_test.go
index 139330d79..eee0c1a9e 100644
--- a/internal/host/claimservingpark_rootrunner_linux_test.go
+++ b/internal/host/claimservingpark_rootrunner_linux_test.go
@@ -134,6 +134,11 @@ func servingRunnerPark(t *testing.T, data string, state rootstate.Store) serving
if err = fs.CheckParked(&r); err != nil {
t.Fatal(err)
}
+ r.Parked, r.ParkedAt = true, time.Now().UTC()
+ r.ReapAfter = r.ParkedAt.Add(7 * 24 * time.Hour)
+ if err = save(); err != nil {
+ t.Fatal(err)
+ }
return r
}
@@ -164,8 +169,26 @@ func TestRootServingParkUnitPayload(t *testing.T) {
fs := newServingParkFS(data, state)
r.Deleting, r.Cause = true, "expiry"
save := func() error { return (&servingParkLedger{state: state}).write("fixture.json", r) }
- if err := fs.Delete(&r, save); err != nil {
- t.Fatal(err)
+ fs.unlink = func(fd int, name string, flags int) error {
+ err := unix.Unlinkat(fd, name, flags)
+ if err == nil && flags == unix.AT_REMOVEDIR {
+ return unix.EIO
+ }
+ return err
+ }
+ if err := fs.Delete(&r, save); !errors.Is(err, unix.EIO) {
+ t.Fatalf("lost empty-directory removal reply: %v", err)
+ }
+ body, err := state.Read("fixture.json")
+ if err != nil || servingParkDecode(body, &r) != nil || !r.EventRemoveIntent || r.EventRemoved {
+ t.Fatal("missing durable event removal intent", err)
+ }
+ fs = newServingParkFS(data, state)
+ if err = fs.Delete(&r, save); err != nil || !r.EventRemoved {
+ t.Fatal("installed unit did not recover empty event removal", err)
+ }
+ if _, err = os.Lstat(filepath.Join(data, "instances", ".serving-parked", r.Authorization.ID)); !errors.Is(err, os.ErrNotExist) {
+ t.Fatal("recorded empty event survived", err)
}
for _, item := range r.Source.Files {
if item.Present && !item.Deleted {
diff --git a/internal/host/claimservingparkroot_linux.go b/internal/host/claimservingparkroot_linux.go
index 2c10fcd26..7004e9eea 100644
--- a/internal/host/claimservingparkroot_linux.go
+++ b/internal/host/claimservingparkroot_linux.go
@@ -35,7 +35,26 @@ func newServingParkAuthority(h ClaimRootHTTP, state rootstate.Store, empty servi
}
func (p *servingParkRoot) withAuthority(ctx context.Context, conn net.Conn, current servingArtifactIdentity, fn func(hostclaim.Observed, hostguard.FenceView, hostbox.Actor, servingArtifactDirectory, uint64) error) error {
- if ctx == nil || conn == nil || p == nil || p.ledger == nil || p.root.Stop.Authenticate == nil || p.root.RequireNewWork == nil || p.root.InventoryActor == nil || p.root.Journals == nil || p.root.Journals.Inventory == nil || p.root.Journals.Inventory.Remote() || p.root.Stop.Coordinator.Guard.Root == nil || fn == nil {
+ if conn == nil || fn == nil {
+ return hostbox.ErrUnsupported
+ }
+ return p.withRootInstallation(ctx, current, func() error {
+ // Waiting for root locks must not preserve a stale peer authentication.
+ peer, err := p.root.Stop.Authenticate(ctx, conn)
+ if err != nil {
+ return err
+ }
+ if peer.UID == 0 || peer.PID <= 0 || peer.ProcessStart == "" || peer.BootID == "" || peer.ControllerID == "" {
+ return ErrClaimStopIdentity
+ }
+ return nil
+ }, fn)
+}
+
+// withRootInstallation is also used by root startup to replay private intents.
+// It grants no new admission, enrollment, stop receipt or claim tag.
+func (p *servingParkRoot) withRootInstallation(ctx context.Context, current servingArtifactIdentity, check func() error, fn func(hostclaim.Observed, hostguard.FenceView, hostbox.Actor, servingArtifactDirectory, uint64) error) error {
+ if ctx == nil || check == nil || p == nil || p.ledger == nil || p.root.Stop.Authenticate == nil || p.root.RequireNewWork == nil || p.root.InventoryActor == nil || p.root.Journals == nil || p.root.Journals.Inventory == nil || p.root.Journals.Inventory.Remote() || p.root.Stop.Coordinator.Guard.Root == nil || fn == nil {
return hostbox.ErrUnsupported
}
data := p.root.Roots["data"]
@@ -44,16 +63,10 @@ func (p *servingParkRoot) withAuthority(ctx context.Context, conn net.Conn, curr
}
return p.ledger.withNamespace(ctx, func() error {
return p.root.Stop.Coordinator.Guard.WithServingRetirement(ctx, current.Host, hostguard.Captured{Incarnation: current.Incarnation, Stamp: current.Stamp}, func(o hostclaim.Observed, view hostguard.FenceView) error {
- // Reauthenticate after waiting for locks: an earlier connection check
- // cannot establish that the edge main process is still this caller.
- peer, err := p.root.Stop.Authenticate(ctx, conn)
- if err != nil {
+ if err := check(); err != nil {
return err
}
- if peer.UID == 0 || peer.PID <= 0 || peer.ProcessStart == "" || peer.BootID == "" || peer.ControllerID == "" {
- return ErrClaimStopIdentity
- }
- if err = p.root.RequireNewWork(ctx); err != nil {
+ if err := p.root.RequireNewWork(ctx); err != nil {
return err
}
actor, err := p.root.InventoryActor(ctx)
diff --git a/internal/provision/testdata/plan14-root-reads-seed.tsv b/internal/provision/testdata/plan14-root-reads-seed.tsv
index d48ae235b..169040101 100644
--- a/internal/provision/testdata/plan14-root-reads-seed.tsv
+++ b/internal/provision/testdata/plan14-root-reads-seed.tsv
@@ -1,4 +1,4 @@
-# Files seed checkpoint 8 overlay; pending Paul’s census ruling.
+# Files seed checkpoint 9 overlay; pending Paul’s census ruling.
# Generation, legacy and empty-bootstrap root authority have test callers only; production binding remains refused.
# Load after zddr4 and its explicit seal check. No legacy fingerprints are added or removed.
# Every classification below is rev3=PROPOSED-seed, not an approved disposition.
@@ -35,8 +35,8 @@ internal/host/claimservingpark_linux.go newServingParkFS authority C,J4 rev3=PRO
internal/host/claimservingpark_linux.go sameServingParkFile authority C,J4 rev3=PROPOSED-seed; recorded basename, presence, mount, device, inode, regular type, single link and allocated-byte identity d5d684356e793dcfd55fe1ec9c609f2320e0155e052caca42475348075e8fd68
internal/host/claimservingpark_linux.go servingParkDirID authority C,J4 rev3=PROPOSED-seed; descriptor statx directory type, device, inode and mount identity file:Statx@102 3480dad6d65fedcc17e678df38cf13352ad3c4a8a9d0a09a395973d92b98094e
internal/host/claimservingpark_linux.go servingParkFS.Capture authority C,J4 rev3=PROPOSED-seed; canonical nested source directory, four fixed basenames, source mount and bounded generation-marker bytes through the captured descriptor file:Fstat@920,file:Openat@694,file:ReadAll@1138 c3a9348ddd9548f3fcfdcd45abc42b6a758a6399ed34e92481785e26c110763d
-internal/host/claimservingpark_linux.go servingParkFS.CheckParked authority C,J4 rev3=PROPOSED-seed; all recorded event identities, unexpected children and source absence before root completion file:Readdirnames@822 144c738d31a4b6495d742310dc3ac171bd0d70bff902395a8cc77119191ccd22
-internal/host/claimservingpark_linux.go servingParkFS.Delete authority C,J4 rev3=PROPOSED-seed; root-recorded event and artifact identities; validates the whole set before journaled per-name unlink, fsync and progress; no recursive deletion file:Readdirnames@341 8837ae9e8947479346f6dddeb8ea8f13290ad56d9d2b96f315e203f5c2e1646a
+internal/host/claimservingpark_linux.go servingParkFS.CheckParked authority C,J4 rev3=PROPOSED-seed; all recorded event identities, unexpected children and source absence before root completion file:Readdirnames@1266 ac675e110e1493ab1ec69359696d462228f1e1265923c2c0d471111851a2c645
+internal/host/claimservingpark_linux.go servingParkFS.Delete authority C,J4 rev3=PROPOSED-seed; recorded event and artifact identities; validates complete child set before journaled unlink, then removes only the recorded empty event with durable intent and progress file:Readdirnames@402 9242121cefd1767725b160092a248a8f60d952e5093b6270ea71dfc377b5300c
internal/host/claimservingpark_linux.go servingParkFS.Move authority C,J4 rev3=PROPOSED-seed; recorded source/destination inodes and explicit absences; no-replace same-mount rename, post-rename identity and both parent fsyncs before progress 6e43ae58b6a02d167135fabdde7ade06502511c5237b313e6ca6c2a8e9a2c8d0
internal/host/claimservingpark_linux.go servingParkFS.Prepare authority C,J4 rev3=PROPOSED-seed; recorded private namespace and empty event before persisting event identity; existing unrecorded namespace is refused file:Prepare@24,file:Readdirnames@890 2480401a7da95a24a7557be0b8adff67394689abb418e36d0fafecdb8c1d827b
internal/host/claimservingpark_linux.go servingParkFS.event authority C,J4 rev3=PROPOSED-seed; recorded namespace parent and private event mount/device/inode identities 8e2ce5bf2f5931e702b847d664bd5cff5fbf90c71ed74b5a2d52ca06d9e0da7c
@@ -49,7 +49,7 @@ internal/host/claimservingpark_linux.go servingParkStat authority C,J4 rev3=PROP
internal/host/claimservingpark_other.go newServingParkFS authority C,J4 rev3=PROPOSED-seed; non-Linux refusal or construction of refusal-only filesystem; no filesystem authority 69baa4c747e687578e73b8369f2df9987fe6fb5836b78796cc41c776ba12c802
internal/host/claimservingpark_other.go servingParkFS.Capture authority C,J4 rev3=PROPOSED-seed; non-Linux refusal or construction of refusal-only filesystem; no filesystem authority a5560b338356792386a5a1b2e27cb08ca0d5aa024cd03b567a78dfd986ce7871
internal/host/claimservingpark_other.go servingParkFS.CheckParked authority C,J4 rev3=PROPOSED-seed; non-Linux refusal or construction of refusal-only filesystem; no filesystem authority 3a8cb4d8052bbdea8cf7a300944de72c68012e6efa252c293fc1dc06a3a2150e
-internal/host/claimservingpark_other.go servingParkFS.Delete authority C,J4 rev3=PROPOSED-seed; non-Linux refusal or construction of refusal-only filesystem; no filesystem authority 8c169e8428b913d2f05b502ce971be49d5febaad95d8ae811aae21c81f3434ce
+internal/host/claimservingpark_other.go servingParkFS.Delete authority C,J4 rev3=PROPOSED-seed; recorded event and artifact identities; validates complete child set before journaled unlink, then removes only the recorded empty event with durable intent and progress 8c169e8428b913d2f05b502ce971be49d5febaad95d8ae811aae21c81f3434ce
internal/host/claimservingpark_other.go servingParkFS.Move authority C,J4 rev3=PROPOSED-seed; non-Linux refusal or construction of refusal-only filesystem; no filesystem authority 9d215b168da646166104db5624727038f793d2ae0c40eb71f23883f12b6b4b13
internal/host/claimservingpark_other.go servingParkFS.Prepare authority C,J4 rev3=PROPOSED-seed; non-Linux refusal or construction of refusal-only filesystem; no filesystem authority file:Prepare@22 645770372357707f635351a377e943e78f1c160222385e41f42b80d57adedc29
internal/host/claimservingpark_records.go servingParkDecode authority C,J4 rev3=PROPOSED-seed; strict private-record JSON with unknown-field and trailing-value refusal 189c7944ea9f5d269d39ed6a869b6bd739e3033195f06cc1f2c9f23a23180184
@@ -64,10 +64,10 @@ internal/host/claimservingpark_records.go servingParkLedger.slot authority C,J4
internal/host/claimservingpark_records.go servingParkLedger.withNamespace authority C,J4 rev3=PROPOSED-seed; private rootstate namespace lock and cancellation before invoking root authority; composed with root host authority through the typed journal service b7cfae44fdb7de9ef8130f5581710ae3a1a526bb059b4f616650172490f5c8ac
internal/host/claimservingpark_records.go servingParkLedger.write authority C,J4 rev3=PROPOSED-seed; serialized private record sent only to the supplied rootstate store 0635d2c27871e67079119a0f2e3034e817b765d0fe3a47e7d9bdc66fb2779571
internal/host/claimservingpark_records.go type servingParkFilesystem authority C,J4 rev3=PROPOSED-seed; private filesystem effect interface over root-recorded retirement identities; no wire or public request fields file:Prepare@180 b93f0e2af16b66aee1ff71294e215eeba9e661fe696c917c982d9ebc2dc682c7
-internal/host/claimservingpark_records.go validateServingParkRecord authority C,J4 rev3=PROPOSED-seed; version, immutable tuples, exact serving receipt bytes, source identities, per-file progress, disposition, completion and retention deadline invariants; bounded immutable continuation history and exact predecessor stops 7164d44093a2bf9efd5d8798d8bf6674f5b399bca163d098b3f2c9b100e9ea87
+internal/host/claimservingpark_records.go validateServingParkRecord authority C,J4 rev3=PROPOSED-seed; version, immutable tuples, exact serving receipt bytes, source identities, per-file progress, disposition, completion and retention deadline invariants; bounded immutable continuation history and exact predecessor stops a4715a0b91f5573ceaef4af9d2f4492924f7ab079cf11a41f2e99b6b0db2a87d
internal/host/claimservingparksource_linux.go servingParkStateDirectory authority C,J4 rev3=PROPOSED-seed; root installation/inventory snapshots and authoritative serving observation; exact account/app/record, revision/enrollment, canonical nested path, ownership and descriptor identity; root acquires these inputs under the host lock through the typed journal service ec4a121330c5c8f27343f21af0bbe458cced885d4754e96c0d87b12d9b4a8f35
internal/hostguard/servingretirement.go Guard.WithServingRetirement authority C,J4 rev3=PROPOSED-seed; fresh serving claim and root installation/fence snapshot under root and shared host locks; refuses pending, quarantined, foreign, retired or already stopped successors; permits an older completed predecessor 3d105c50be7e613777055f561321efadf77fe823d8dc3263a60f2c1211b71108
-internal/host/claimservingparkroot_linux.go servingParkRoot.withAuthority authority C,J4 rev3=PROPOSED-seed; independent active-edge peer authentication, census gate, roster, local root inventory and fresh claim recheck under namespace and root host authority; no registry path or caller receipt file:Read@2432 6d0c84a5a9c5403f166db4f200504a3ccd75ebcd7ddac1736e3ac046237a2882
+internal/host/claimservingparkroot_linux.go servingParkRoot.withAuthority authority C,J4 rev3=PROPOSED-seed; live edge peer reauthentication under namespace and host locks before census, inventory and final claim checks; root request entry refuses nil connections d3582f0760b4459721b3e6040cda738b105c1da19cc5d0746d04fea26e313b5c
internal/host/claimservingparkroot_linux.go servingRootStops authority C,J4 rev3=PROPOSED-seed; root fence snapshot and known installation stamps; exact local serving receipt bytes, enrollment, revision and unique fence identities d16eb077adfb8f0f8c9e63d1609250dc3470ec424bdde63b57c47d09500b2bce
internal/host/claimservingparkroot_linux.go servingParkRoot.parkServingDatabase authority C,J4 rev3=PROPOSED-seed; fresh paired tags, root slot/record and retained admissions; recorded legacy work dispatches to separately locked fresh legacy authority 61c29d5b37168ab053499553eaf0052fcee134a601c86acb3195fad416d28247
internal/host/claimservingparkroot_linux.go servingParkRoot.completeServingDatabasePark authority C,J4 rev3=PROPOSED-seed; fresh paired enrollment, same current writer, root source and original/continuation stop history; legacy additionally authenticates persistent no-process or admitted predecessor evidence 8ca9bdd3c1209c6d5257bc07a7d12d0bab900e9c2911d0698077eb2116e5123c
@@ -75,7 +75,7 @@ internal/host/claimservingparkroot_linux.go validateServingRootHistory authority
internal/host/claimservingpark_records.go servingParkRecord.activeAuthorization authority C,J4 rev3=PROPOSED-seed; last persisted continuation authorization or immutable original authorization; no source or deadline changes fe168fe52317e45ce70c3e97a9bc2c52d2409328ab4844a8d2a34fc4babba36d
internal/host/claimservingpark_records.go servingParkLedger.continueLocked authority C,J4 rev3=PROPOSED-seed; namespace/host-protected slot and record locks; pending original retirement, absent source and retained parked identities before durable bounded continuation 87f297a924f7f153624304233218d8b5548a016fc7dbf0f7cb8dedb789324009
internal/host/claimservingpark_records.go servingParkLedger.replaceLocked authority C,J4 rev3=PROPOSED-seed; completed incumbent record under its record lock; resumes its recorded deletion cause before slot replacement 3e67316e959c9a901c32a2a61fda4e6ea5e92fe51bdb7fdb5f5c53d9d5960aad
-@coverage internal/host 2d65820f0366a1ee92490ef9af8888cfda33292f5a5367d35d353e320f031a85
+@coverage internal/host c10497dab1ac3aef334df090b73e4d20fcc67d43a31dd4d457ead6f4f61de83f
@coverage internal/hostbox 4b68d7fc25b29963dcf1b7fe92b047fd2229515d83e5cf424c0835d6e690859d
@coverage internal/hostguard 94912485e1e5fad4bad56985dd48ede776cfcb5fe18e0f4a59953584ddab06c0
internal/host/claimservinglegacy.go servingLegacyKey authority C,J4 rev3=PROPOSED-seed; box, host and incarnation derive private stop and consumption keys; never a caller path 93c10e40202ab23986b54a366edd7abfab9f8d942736305b5732e8741e96f3ab
@@ -122,3 +122,18 @@ internal/hostbox/servingpark_transport.go validateServingParkRequest authority C
internal/host/claimservingpark_transport.go var openClaimServingParkAuthority authority C,J4 rev3=PROPOSED-seed; descriptor-relative private rootstate at /var/lib/carlos-claims/authority/serving-parks with creation enabled and public reads disabled file:Open@78 7a1db088a7240e6a65ba0bed96fc5f973a69f70c2273c4475c486a6e0359fb23
internal/host/claimwakestop.go ClaimAssembly.collectCapturedStop authority C,G,J4 rev3=PROPOSED-seed; fresh claim and verified local stop identity; session-zero legacy holder may name only the local writer box, with no admissions; root independently proves and retains no-process evidence file:Exec@2994,file:Read@605 d4cd80663dfae5f9737a74e319d3423f4ef853e1843666b02b4cd2341e869d72
internal/hostbox/restore.go loadPinned authority C,J2,J4 rev3=PROPOSED-seed; exact selected LTX versions and original trailer/pin proof; validated bytes normalized for Litestream while retaining original database checksum for final comparison file:ReadAll@662 1e2b22c7c501d31cd1f2cb3be0fe419fa7e25afde87c6ecb43ca85edb7dbc3a4
+internal/host/claimretire_transport.go ClaimRootHTTP.RecoverPending authority C,J4 rev3=PROPOSED-seed; root startup recovery order; attempts bounded serving retirement recovery even when another root recovery returns an error 0e6c92aae15cac7055607fb24d0e9044223c74933767b2859ed105e49974e006
+internal/host/claimservingparkroot_linux.go servingParkRoot.withRootInstallation authority C,J4 rev3=PROPOSED-seed; fresh claim, root installation and directory inventory, eligible enrollment, paired tags, census gate and post-input claim reread under namespace then host guard; startup grants no new identity file:Read@2113 8060d3a2628f86257c3600c986242e0d4957ee918214a89655af828dd3ddf93b
+internal/host/claimservingpark_linux.go servingParkFS.removeEvent authority C,J4 rev3=PROPOSED-seed; recorded empty event directory identity and absence; journaled removal intent before descriptor-relative rmdir, root parent fsync and durable progress; no recursion file:Readdirnames@789 b493696d2f16425ddc2714184b407acbaf13ef62c93f28c3d4ac8dc7012607c9
+internal/host/claimservingpark_linux.go servingParkFS.removedEvent authority C,J4 rev3=PROPOSED-seed; recorded namespace parent and absence of previously removed event; replacement names refuse completion 792218fc7bfb4ac7475e7c694f72489812db8591493218f8ee1c9545cb813566
+internal/host/claimservingpark_records.go type servingParkRecord authority C,J4 rev3=PROPOSED-seed; private versioned retirement shape; additive empty-event deletion intent and completion bits checked alongside artifact deletion progress bfa0992f8494ebae3850471f2377e5d1e08a0a6bdd825bca6a80458d40a2abbe
+internal/host/claimservingpark_recovery.go const servingParkRecoveryLimit authority C,J4 rev3=PROPOSED-seed; fixed root-side processing limit of 128 retirement records per recovery page 6b949581334f55a7bae1b4579e56ac74810fc3042fdf568fbfa8e6b3c5a2ef11
+internal/host/claimservingpark_recovery.go var errServingParkRetagPending authority C,J4 rev3=PROPOSED-seed; fixed pending-retag refusal; startup cannot write the independent claim writer tags 8fa87ebef4defae4fc1733528a76e0cc260711c2f5a664308e5368d69286d45b
+internal/host/claimservingpark_recovery.go type servingParkRecoveryPage authority C,J4 rev3=PROPOSED-seed; private bounded recovery results, cursor and incomplete indicator; never a transport selector c2fb4445ed50b5237b72e8a5fff9b1b2f6c9dd1397473eeb2c78a6b8c2804b27
+internal/host/claimservingpark_recovery.go type servingParkRecoveryResult authority C,J4 rev3=PROPOSED-seed; root-derived host and record ID with local recovery outcome 4dd13c07e97470a6c5aa4fa98d326de8947422018f0d30d5c5af32d753cfcc39
+internal/host/claimservingpark_recovery.go servingParkLedger.recoveryIDs authority C,J4 rev3=PROPOSED-seed; private rootstate names; sorted bounded retirement page and continuation cursor; malformed record IDs retained for reporting 5b450cd0fb6ae6694bca2f9f75fe972c0d65681981a4caddde7e5c7269108782
+internal/host/claimservingpark_recovery.go servingParkLedger.recoverCompleted authority C,J4 rev3=PROPOSED-seed; completed private deletion intent, original cause, slot and recorded filesystem identities under namespace/slot/record locks; preserves Last and ID replay references without live claim authority 2c7f9aed9d06a42f39ff1fd18bfcea919e5e1a17dcb46cd4274b3ecc68fce98b
+internal/host/claimservingpark_recovery.go servingParkRuntime.recoverPending authority C,J4 rev3=PROPOSED-seed; service lifetime cancellation and join; bounded startup results rendered only with indexed host/record/fixed-reason text 0aa3ef7edd0ac2c12b4221907ccd5927e7e3e03a4c2c0b77b669bc10cc49c241
+internal/host/claimservingpark_recovery_linux.go sameServingLegacyConsumption authority C,J4 rev3=PROPOSED-seed; immutable legacy consumption and retained retirement source identities; excludes only per-artifact journal progress from comparison 1524b2aae315e05734a666f80596bc341a007098a84fedeaf1e32451663d4fd4
+internal/host/claimservingpark_recovery_linux.go servingParkRoot.recoverServingPark authority C,J4 rev3=PROPOSED-seed; retained intent, exact current successor, independently rechecked installation, enrollment, receipt and local launch history; resumes original park and completes only existing paired tags; no retry settlement 0839c879792469648ceb62edce21319050351cf4db0a57c1a5220282afb7808f
+internal/host/claimservingpark_recovery_linux.go servingParkRoot.recoverServingParks authority C,J4 rev3=PROPOSED-seed; private record page and per-record validation; continues past corrupt or refused hosts and reports incomplete enumeration; completed deletion separated from live host authority c2f1d9da88c4aaea97783a6d3f3e6b78856f20402e55900cc96d88e8c88ba323