amadan

carlos / platform Public

hotfix #281 fix round 1: drop the early listener; only the activator start moves off the edge's path

Both reviews found the early listener weakened the update gate and opened
a wrong-answer window (TLS before certificates, an empty home map). The
edge now starts exactly as in v0.27.0 (edge.go and door.go unchanged);
only the activator start (sweep, recovery, Start, key pass) runs in the
background, with instance admission held until Start (30 s, then the
wake-timeout 503). Shutdown order restored via a startup child context.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Paul Campbell pushed by claude 494ba40155f0aed940f89139089fe19765745185 parent d92b20c
13 files changed, +244 −332
  • internal/activator/activator.go +4 −3
  • internal/activator/startup_test.go +23 −0
  • internal/edge/door.go +5 −16
  • internal/edge/early.go +0 −120
  • internal/edge/early_test.go +0 −62
  • internal/edge/edge.go +32 −73
  • internal/edge/startup_test.go +86 −0
  • internal/host/agent.go +23 −26
  • internal/host/instancekeys_test.go +17 −8
  • internal/host/startup.go +0 −9
  • internal/host/startup_test.go +49 −5
  • internal/provision/plan14_root_reads_test.go +1 −1
  • internal/provision/testdata/plan14-root-reads-hotfix281.tsv +4 −9

CI passed — run details

CI log 
=== gate ===
cd launchctx && go vet ./... && go test ./... -count=1
ok  	amadan.net/carlos/platform.git/launchctx	0.012s
?   	amadan.net/carlos/platform.git/launchctx/controlname	[no test files]
?   	amadan.net/carlos/platform.git/launchctx/wireid	[no test files]
go vet ./...
gofmt -l .: clean
go test ./... -count=1 -timeout 30m
ok  	github.com/carlosframework/platform	350.983s
?   	github.com/carlosframework/platform/cmd/changelog	[no test files]
ok  	github.com/carlosframework/platform/cmd/console	3.103s
ok  	github.com/carlosframework/platform/cmd/moveprobe	92.221s
ok  	github.com/carlosframework/platform/cmd/zddprobe	0.384s
ok  	github.com/carlosframework/platform/internal/activator	225.778s
ok  	github.com/carlosframework/platform/internal/agentsock	0.014s
ok  	github.com/carlosframework/platform/internal/alerting	0.004s
ok  	github.com/carlosframework/platform/internal/analytics	0.018s
ok  	github.com/carlosframework/platform/internal/appconfig	2.361s
ok  	github.com/carlosframework/platform/internal/apperr	0.072s
ok  	github.com/carlosframework/platform/internal/applog	0.131s
ok  	github.com/carlosframework/platform/internal/awsinv	0.003s
ok  	github.com/carlosframework/platform/internal/boxchoice	0.004s
ok  	github.com/carlosframework/platform/internal/boxid	0.902s
?   	github.com/carlosframework/platform/internal/boxid/boxidtest	[no test files]
?   	github.com/carlosframework/platform/internal/boxid/internal/signtest	[no test files]
ok  	github.com/carlosframework/platform/internal/boxvolumes	0.008s
ok  	github.com/carlosframework/platform/internal/caps	0.011s
ok  	github.com/carlosframework/platform/internal/certwarm	0.002s
ok  	github.com/carlosframework/platform/internal/cgroup	0.698s
ok  	github.com/carlosframework/platform/internal/changelog	0.297s
ok  	github.com/carlosframework/platform/internal/clamd	0.054s
?   	github.com/carlosframework/platform/internal/cliflag	[no test files]
ok  	github.com/carlosframework/platform/internal/console	365.693s
?   	github.com/carlosframework/platform/internal/controlname	[no test files]
?   	github.com/carlosframework/platform/internal/controlsock	[no test files]
ok  	github.com/carlosframework/platform/internal/dbdoor	5.895s
ok  	github.com/carlosframework/platform/internal/dbdoor/dbdoortest	0.009s
ok  	github.com/carlosframework/platform/internal/dbverify	0.004s
ok  	github.com/carlosframework/platform/internal/dnsrec	0.002s
ok  	github.com/carlosframework/platform/internal/docsite	0.005s
ok  	github.com/carlosframework/platform/internal/domains	2.135s
ok  	github.com/carlosframework/platform/internal/doorkey	0.006s
ok  	github.com/carlosframework/platform/internal/economics	0.004s
ok  	github.com/carlosframework/platform/internal/edge	26.386s
?   	github.com/carlosframework/platform/internal/edgeproof	[no test files]
ok  	github.com/carlosframework/platform/internal/edgesfile	0.007s
ok  	github.com/carlosframework/platform/internal/fleetapi	0.124s
ok  	github.com/carlosframework/platform/internal/gate	1.654s
ok  	github.com/carlosframework/platform/internal/genselect	0.015s
ok  	github.com/carlosframework/platform/internal/geo	0.018s
ok  	github.com/carlosframework/platform/internal/geoip	0.030s
ok  	github.com/carlosframework/platform/internal/host	98.585s
ok  	github.com/carlosframework/platform/internal/hostactors	0.523s
ok  	github.com/carlosframework/platform/internal/hostbox	2.153s
ok  	github.com/carlosframework/platform/internal/hostclaim	0.532s
ok  	github.com/carlosframework/platform/internal/hostevidence	0.015s
ok  	github.com/carlosframework/platform/internal/hostexec	4.943s
ok  	github.com/carlosframework/platform/internal/hostfence	0.037s
ok  	github.com/carlosframework/platform/internal/hostfiles	10.592s
ok  	github.com/carlosframework/platform/internal/hostfloor	0.016s
ok  	github.com/carlosframework/platform/internal/hostguard	1.007s
ok  	github.com/carlosframework/platform/internal/hostops	1.463s
?   	github.com/carlosframework/platform/internal/hostops/fencerelease	[no test files]
ok  	github.com/carlosframework/platform/internal/hostops/internal/fencerelease	0.015s
ok  	github.com/carlosframework/platform/internal/hostquarantine	0.015s
ok  	github.com/carlosframework/platform/internal/hostrelocate	0.451s
ok  	github.com/carlosframework/platform/internal/hostresolve	0.154s
ok  	github.com/carlosframework/platform/internal/hostwrite	0.015s
ok  	github.com/carlosframework/platform/internal/infraview	0.019s
ok  	github.com/carlosframework/platform/internal/instancectl	1.817s
ok  	github.com/carlosframework/platform/internal/instanceexec	2.720s
ok  	github.com/carlosframework/platform/internal/instancefs	0.020s
ok  	github.com/carlosframework/platform/internal/instancekey	0.015s
ok  	github.com/carlosframework/platform/internal/instanceupdates	0.024s
ok  	github.com/carlosframework/platform/internal/layout	1.448s
ok  	github.com/carlosframework/platform/internal/lease	9.509s
ok  	github.com/carlosframework/platform/internal/lease/leasetest	0.021s
ok  	github.com/carlosframework/platform/internal/ledger	0.010s
ok  	github.com/carlosframework/platform/internal/mail	0.165s
ok  	github.com/carlosframework/platform/internal/maintenance	0.087s
ok  	github.com/carlosframework/platform/internal/maintenanceexec	5.525s
ok  	github.com/carlosframework/platform/internal/maintenanceguard	0.087s
ok  	github.com/carlosframework/platform/internal/metrics	0.008s
ok  	github.com/carlosframework/platform/internal/moves	21.062s
ok  	github.com/carlosframework/platform/internal/moves/archive	0.769s
ok  	github.com/carlosframework/platform/internal/objstore	0.174s
ok  	github.com/carlosframework/platform/internal/ops	0.163s
ok  	github.com/carlosframework/platform/internal/park	0.202s
ok  	github.com/carlosframework/platform/internal/privatefile	0.005s
ok  	github.com/carlosframework/platform/internal/provision	13.630s
ok  	github.com/carlosframework/platform/internal/publicrelease	0.008s
?   	github.com/carlosframework/platform/internal/regionname	[no test files]
ok  	github.com/carlosframework/platform/internal/registry	4.079s
ok  	github.com/carlosframework/platform/internal/relay	7.268s
ok  	github.com/carlosframework/platform/internal/release	20.303s
ok  	github.com/carlosframework/platform/internal/rollout	3.796s
ok  	github.com/carlosframework/platform/internal/rolloutwords	0.003s
ok  	github.com/carlosframework/platform/internal/rootaws	0.007s
ok  	github.com/carlosframework/platform/internal/rootparticipation	0.008s
ok  	github.com/carlosframework/platform/internal/rootstate	0.025s
ok  	github.com/carlosframework/platform/internal/s3compat	0.005s
ok  	github.com/carlosframework/platform/internal/safefile	0.004s
ok  	github.com/carlosframework/platform/internal/schedule	2.278s
ok  	github.com/carlosframework/platform/internal/sim/hostlife	10.001s
ok  	github.com/carlosframework/platform/internal/sim/litestreamfacts	3.634s
ok  	github.com/carlosframework/platform/internal/status	0.038s
ok  	github.com/carlosframework/platform/internal/steering	0.006s
ok  	github.com/carlosframework/platform/internal/stopevidence	0.241s
ok  	github.com/carlosframework/platform/internal/svcrole	0.002s
ok  	github.com/carlosframework/platform/internal/telemetry	0.004s
ok  	github.com/carlosframework/platform/internal/termfence	0.004s
ok  	github.com/carlosframework/platform/internal/tunnelclient	0.004s
ok  	github.com/carlosframework/platform/internal/unixdial	0.002s
ok  	github.com/carlosframework/platform/internal/updatewords	0.005s
ok  	github.com/carlosframework/platform/internal/usage	0.005s
ok  	github.com/carlosframework/platform/internal/version	0.002s
ok  	github.com/carlosframework/platform/internal/vet	1.593s
=== cross ===
--- shard 0 of 5 ---
cross-compiling darwin/arm64
--- shard 1 of 5 ---
cross-compiling darwin/amd64
--- shard 2 of 5 ---
cross-compiling linux/arm64
--- shard 3 of 5 ---
cross-compiling windows/amd64
--- shard 4 of 5 ---
cross-compiling windows/arm64
=== minio-contract ===
minio contract: arm64, minio RELEASE.2025-09-07T16-13-09Z, mc RELEASE.2025-08-13T08-35-41Z
  minio: downloading https://github.com/minio/minio/releases/download/RELEASE.2025-09-07T16-13-09Z/minio.linux-arm64.RELEASE.2025-09-07T16-13-09Z
  minio: downloaded, digest verified
  mc: downloading https://github.com/minio/mc/releases/download/RELEASE.2025-08-13T08-35-41Z/mc.linux-arm64.RELEASE.2025-08-13T08-35-41Z
  mc: downloaded, digest verified
  minio healthy on 127.0.0.1:20999
Added `local` successfully.
Bucket created successfully `local/contract-bucket`.
Bucket created successfully `local/denied-bucket`.
Created policy `contract-only` successfully.
Added user `scopeduser` successfully.
Attached Policies: [contract-only]
To User: scopeduser
  buckets and scoped user ready
=== RUN   TestDeliveredContractMinio
=== RUN   TestDeliveredContractMinio/put
=== RUN   TestDeliveredContractMinio/get_round-trips_the_body
=== RUN   TestDeliveredContractMinio/list_shows_the_key
=== RUN   TestDeliveredContractMinio/head_and_list_agree_on_the_version
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict
=== RUN   TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed
=== RUN   TestDeliveredContractMinio/delete_removes_it
=== RUN   TestDeliveredContractMinio/cross-bucket_denial
--- PASS: TestDeliveredContractMinio (0.02s)
    --- PASS: TestDeliveredContractMinio/put (0.00s)
    --- PASS: TestDeliveredContractMinio/get_round-trips_the_body (0.00s)
    --- PASS: TestDeliveredContractMinio/list_shows_the_key (0.00s)
    --- PASS: TestDeliveredContractMinio/head_and_list_agree_on_the_version (0.00s)
    --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/not_by_PutObjectTagging (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_its_own_object/and_not_by_x-amz-tagging_on_upload (0.00s)
    --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route (0.01s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CreateMultipartUpload (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/not_by_CopyObject_with_a_REPLACE_tagging_directive (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/but_multipart_itself_is_allowed (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_copy_itself_is_allowed (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/a_POST-policy_forged_tag_with_a_matching_version_is_still_not_a_verdict (0.00s)
        --- PASS: TestDeliveredContractMinio/the_app_cannot_tag_by_any_other_route/and_an_ordinary_tagged-with-something-else_upload_is_allowed (0.00s)
    --- PASS: TestDeliveredContractMinio/delete_removes_it (0.00s)
    --- PASS: TestDeliveredContractMinio/cross-bucket_denial (0.00s)
PASS
ok  	github.com/carlosframework/platform/internal/objstore	0.024s
=== RUN   TestT4StorageProbeMinio
2026/10/13 20:43:42 moves: T4: STORE: list: operation error S3: ListObjectsV2, https response error StatusCode: 403, RequestID: 18DE31962A461770, HostID: 7c420053e0fbcd7f41653fdd92ffa2802c903d05303b1aa15adc2e8bb65d183f, api error SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.
--- PASS: TestT4StorageProbeMinio (0.01s)
PASS
ok  	github.com/carlosframework/platform/internal/host	0.022s
minio contract: PASS

Intent

Humanclaude~paul@keymail.dev

<task-notification> <task-id>bgtu3i3pb</task-id> <tool-use-id>toolu_016Dd95E2RQQZ21WTbkSpsE4</tool-use-id> <output-file>/home/paulca/.cache/tmp/claude-1001/-home-paulca-amadan-net-carlos-platform/6387394d-0b76-4de7-bbb4-d725fe24b08c/tasks/bgtu3i3pb.output</output-file> <status>completed</status> <summary>Background command "Wait for hotfix fix round 1" completed (exit code 0)</summary> </task-notification>

full ledger: hotfix-startup-sweep-281

diff --git a/internal/activator/activator.go b/internal/activator/activator.go
index baadd327a..5c2f1780e 100644
--- a/internal/activator/activator.go
+++ b/internal/activator/activator.go
@@ -1007,6 +1007,7 @@ type configAdmitter interface {
const configRefusedRetryAfter = 30
var errWakeTimeout = errors.New("timed out waiting for instance wake")
+var errStartupStopped = errors.New("activator stopped during startup")
// Service is the activator (Eleven's activatorSvc). It structurally satisfies
// the edge package's Activator interface — the edge never imports this
@@ -1281,14 +1282,14 @@ func (a *Service) awaitStartup(ctx context.Context, deadline <-chan time.Time) e
}
select {
case <-a.Startup:
- if err := a.lifeCtx.Err(); err != nil {
- return err
+ if a.lifeCtx.Err() != nil {
+ return errStartupStopped
}
return ctx.Err()
case <-ctx.Done():
return ctx.Err()
case <-a.lifeCtx.Done():
- return context.Canceled
+ return errStartupStopped
case <-deadline:
return errWakeTimeout
}
diff --git a/internal/activator/startup_test.go b/internal/activator/startup_test.go
index a4ddf36bc..91563b3ca 100644
--- a/internal/activator/startup_test.go
+++ b/internal/activator/startup_test.go
@@ -52,3 +52,26 @@ func TestStartupBlocksWorkerAndSidecarAdmission(t *testing.T) {
t.Fatal("startup admission created a wake")
}
}
+
+func TestStartupShutdownAnswersServiceUnavailable(t *testing.T) {
+ for _, ready := range []bool{false, true} {
+ t.Run(map[bool]string{false: "held", true: "released"}[ready], func(t *testing.T) {
+ a := New(Config{}, &fakeBackend{}, nil)
+ started := make(chan struct{})
+ a.Startup = started
+ if ready {
+ close(started)
+ }
+ a.Shutdown(t.Context())
+ rt := registry.Route{Host: "shop.example.com", Kind: "instance", Hibernate: true}
+ w := httptest.NewRecorder()
+ a.ServeHTTP(w, httptest.NewRequest(http.MethodGet, "http://shop.example.com/", nil), rt, http.NotFoundHandler())
+ if w.Code != http.StatusServiceUnavailable {
+ t.Fatalf("service shutdown answered %d: %q", w.Code, w.Body.String())
+ }
+ if a.Owns(rt.Host) {
+ t.Fatal("shutdown request created a wake")
+ }
+ })
+ }
+}
diff --git a/internal/edge/door.go b/internal/edge/door.go
index fd3c6e0c2..c9438f5bb 100644
--- a/internal/edge/door.go
+++ b/internal/edge/door.go
@@ -236,25 +236,14 @@ func (e *Edge) armDoor(ctx context.Context) error {
return why
}
keys := newDoorKeyMirror(e.staticStore, e.staticPub)
- refresh := func() {
- rctx, cancel := context.WithTimeout(ctx, doorKeyTimeout)
- if err := keys.Refresh(rctx); err != nil {
- log.Printf("edge: first door-key refresh failed: %v. Every door key is refused until a later poll lands", err)
- }
- cancel()
- }
- if e.early == nil {
- refresh()
+ rctx, cancel := context.WithTimeout(ctx, doorKeyTimeout)
+ if err := keys.Refresh(rctx); err != nil {
+ log.Printf("edge: first door-key refresh failed: %v. Every door key is refused until a later poll lands", err)
}
-
+ cancel()
e.doorKeys = keys
stop := make(chan struct{})
- go func() {
- if e.early != nil {
- refresh()
- }
- keys.run(stop)
- }()
+ go keys.run(stop)
closeSrv, err := e.openDoor(e.doorPath)
if err != nil {
close(stop)
diff --git a/internal/edge/early.go b/internal/edge/early.go
deleted file mode 100644
index 05aa55e1c..000000000
--- a/internal/edge/early.go
+++ /dev/null
@@ -1,120 +0,0 @@
-package edge
-
-import (
- "context"
- "crypto/tls"
- "log"
- "net"
- "net/http"
- "sync"
- "time"
-)
-
-// EarlyListeners binds before agent setup, which can scan installations and
-// contact other services. Only the local update probe bypasses setup.
-type EarlyListeners struct {
- http, https *http.Server
- httpReady, httpsReady chan struct{}
- httpHandler, httpsHandler http.Handler
- tlsConfig *tls.Config
- once sync.Once
-}
-
-func ListenEarly(args []string) (*EarlyListeners, error) {
- f := parseRunFlags(args)
- return listenEarly(*f.httpAddr, *f.httpsAddr, *f.dev)
-}
-
-func listenEarly(httpAddr, httpsAddr string, dev bool) (*EarlyListeners, error) {
- l := &EarlyListeners{httpReady: make(chan struct{}), httpsReady: make(chan struct{})}
- httpListener, err := net.Listen("tcp", httpAddr)
- if err != nil {
- return nil, err
- }
- var httpsListener net.Listener
- if !dev {
- httpsListener, err = net.Listen("tcp", httpsAddr)
- if err != nil {
- httpListener.Close()
- return nil, err
- }
- }
- l.http = &http.Server{Addr: httpListener.Addr().String(), Handler: http.HandlerFunc(l.serveHTTP), ReadHeaderTimeout: 10 * time.Second}
- go func() {
- if err := l.http.Serve(httpListener); err != nil && err != http.ErrServerClosed {
- log.Fatalf("edge: http listener %s: %v", httpAddr, err)
- }
- }()
- if httpsListener != nil {
- l.https = &http.Server{
- Addr: httpsListener.Addr().String(),
- Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
- l.httpsHandler.ServeHTTP(w, r)
- }),
- ReadHeaderTimeout: 10 * time.Second,
- TLSConfig: &tls.Config{GetConfigForClient: func(hello *tls.ClientHelloInfo) (*tls.Config, error) {
- ctx, cancel := context.WithTimeout(hello.Context(), 30*time.Second)
- defer cancel()
- select {
- case <-l.httpsReady:
- return l.tlsConfig, nil
- case <-ctx.Done():
- return nil, ctx.Err()
- }
- }},
- }
- go func() {
- if err := l.https.ServeTLS(httpsListener, "", ""); err != nil && err != http.ErrServerClosed {
- log.Fatalf("edge: https listener %s: %v", httpsAddr, err)
- }
- }()
- }
- return l, nil
-}
-
-func (l *EarlyListeners) Option() RunOption { return func(e *Edge) { e.early = l } }
-
-func (l *EarlyListeners) serveHTTP(w http.ResponseWriter, r *http.Request) {
- select {
- case <-l.httpReady:
- l.httpHandler.ServeHTTP(w, r)
- return
- default:
- }
- host, _, err := net.SplitHostPort(r.Host)
- if err != nil {
- host = r.Host
- }
- peer, _, _ := net.SplitHostPort(r.RemoteAddr)
- if net.ParseIP(host).IsLoopback() && net.ParseIP(peer).IsLoopback() && r.URL.Path == "/" && r.Method == http.MethodGet {
- http.NotFound(w, r)
- return
- }
- ctx, cancel := context.WithTimeout(r.Context(), 30*time.Second)
- defer cancel()
- select {
- case <-l.httpReady:
- l.httpHandler.ServeHTTP(w, r)
- case <-ctx.Done():
- http.Error(w, http.StatusText(http.StatusServiceUnavailable), http.StatusServiceUnavailable)
- }
-}
-
-func (l *EarlyListeners) publish(h, s http.Handler, cfg *tls.Config) {
- l.httpHandler, l.httpsHandler, l.tlsConfig = h, s, cfg
- close(l.httpReady)
- close(l.httpsReady)
-}
-
-func (l *EarlyListeners) Close() {
- l.once.Do(func() {
- ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
- defer cancel()
- if l.https != nil {
- _ = l.https.Shutdown(ctx)
- _ = l.https.Close()
- }
- _ = l.http.Shutdown(ctx)
- _ = l.http.Close()
- })
-}
diff --git a/internal/edge/early_test.go b/internal/edge/early_test.go
deleted file mode 100644
index bd94f0a2c..000000000
--- a/internal/edge/early_test.go
+++ /dev/null
@@ -1,62 +0,0 @@
-package edge
-
-import (
- "io"
- "net/http"
- "net/http/httptest"
- "testing"
- "time"
-)
-
-func TestEarlyListenersAnswerProbeBeforeSetupAndPublishHTTPAndTLS(t *testing.T) {
- l, err := listenEarly("127.0.0.1:0", "127.0.0.1:0", false)
- if err != nil {
- t.Fatal(err)
- }
- defer l.Close()
- client := &http.Client{Timeout: time.Second}
- resp, err := client.Get("http://" + l.http.Addr + "/")
- if err != nil {
- t.Fatal(err)
- }
- resp.Body.Close()
- if resp.StatusCode != http.StatusNotFound {
- t.Fatalf("gate: %d", resp.StatusCode)
- }
-
- result := make(chan int, 1)
- go func() {
- req, _ := http.NewRequest(http.MethodGet, "http://"+l.http.Addr+"/", nil)
- req.Host = "console.example.com"
- resp, err := client.Do(req)
- if err != nil {
- result <- 0
- return
- }
- resp.Body.Close()
- result <- resp.StatusCode
- }()
- select {
- case status := <-result:
- t.Fatalf("normal request bypassed setup: %d", status)
- case <-time.After(25 * time.Millisecond):
- }
- cert := httptest.NewTLSServer(http.NotFoundHandler())
- defer cert.Close()
- h := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(http.StatusNoContent) })
- l.publish(h, h, cert.TLS.Clone())
- if status := <-result; status != http.StatusNoContent {
- t.Fatalf("held HTTP: %d", status)
- }
- tlsClient := cert.Client()
- tlsClient.Timeout = time.Second
- resp, err = tlsClient.Get("https://" + l.https.Addr + "/")
- if err != nil {
- t.Fatal(err)
- }
- io.Copy(io.Discard, resp.Body)
- resp.Body.Close()
- if resp.StatusCode != http.StatusNoContent {
- t.Fatalf("TLS: %d", resp.StatusCode)
- }
-}
diff --git a/internal/edge/edge.go b/internal/edge/edge.go
index 6a82106c8..ce690d055 100644
--- a/internal/edge/edge.go
+++ b/internal/edge/edge.go
@@ -92,7 +92,6 @@ type Activator interface {
// its backend. The host→backend map is swapped atomically, so reloads and
// zero-downtime deploys never drop a live connection.
type Edge struct {
- early *EarlyListeners
reg *registry.Registry
upstreams atomic.Pointer[map[string]upstream]
// act, when non-nil, is the hibernation activator: hibernate-flagged
@@ -3047,26 +3046,6 @@ func RunWithActivator(args []string, act Activator, cache *Cache) {
RunWithActivatorReady(args, act, cache, nil)
}
-type runFlags struct {
- registryPath, acmeCache, httpsAddr, httpAddr, fallbackURL *string
- dev *bool
-}
-
-func parseRunFlags(args []string) runFlags {
- dataDir := envOr("CARLOS_DATA_DIR", "/data")
- fs := flag.NewFlagSet("edge", flag.ExitOnError)
- registryPath := fs.String("registry", filepath.Join(dataDir, "registry.db"), "registry SQLite path (default $CARLOS_DATA_DIR/registry.db)")
- acmeCache := fs.String("acme-cache", filepath.Join(dataDir, "acme"), "autocert certificate cache dir (default $CARLOS_DATA_DIR/acme)")
- httpsAddr := fs.String("https", ":443", "HTTPS listen address")
- httpAddr := fs.String("http", ":80", "HTTP listen address (ACME HTTP-01 challenge + redirect to https; in --dev mode, the only listener)")
- fallbackURL := fs.String("fallback", "", "redirect requests for unknown hosts here, e.g. an apex marketing site; empty keeps the plain 404")
- dev := fs.Bool("dev", false, "plain-HTTP mode: serve routes on --http with no TLS/ACME (local development only)")
- fs.Usage = cliflag.Usage(fs, "")
- _ = fs.Parse(args)
-
- return runFlags{registryPath, acmeCache, httpsAddr, httpAddr, fallbackURL, dev}
-}
-
// RunWithActivatorReady is RunWithActivator plus a ready hook: ready (when
// non-nil) is called with the fully configured Edge — every EnableXxx done,
// tunnel hub armed if configured — immediately before the process blocks on
@@ -3081,9 +3060,16 @@ func parseRunFlags(args []string) runFlags {
// The ready hook is the wrong home for those: it fires once, at the end, for
// state a caller wants to REACH.
func RunWithActivatorReady(args []string, act Activator, cache *Cache, ready func(*Edge), opts ...RunOption) {
- f := parseRunFlags(args)
- registryPath, acmeCache, httpsAddr, httpAddr, fallbackURL, dev := f.registryPath, f.acmeCache, f.httpsAddr, f.httpAddr, f.fallbackURL, f.dev
dataDir := envOr("CARLOS_DATA_DIR", "/data")
+ fs := flag.NewFlagSet("edge", flag.ExitOnError)
+ registryPath := fs.String("registry", filepath.Join(dataDir, "registry.db"), "registry SQLite path (default $CARLOS_DATA_DIR/registry.db)")
+ acmeCache := fs.String("acme-cache", filepath.Join(dataDir, "acme"), "autocert certificate cache dir (default $CARLOS_DATA_DIR/acme)")
+ httpsAddr := fs.String("https", ":443", "HTTPS listen address")
+ httpAddr := fs.String("http", ":80", "HTTP listen address (ACME HTTP-01 challenge + redirect to https; in --dev mode, the only listener)")
+ fallbackURL := fs.String("fallback", "", "redirect requests for unknown hosts here, e.g. an apex marketing site; empty keeps the plain 404")
+ dev := fs.Bool("dev", false, "plain-HTTP mode: serve routes on --http with no TLS/ACME (local development only)")
+ fs.Usage = cliflag.Usage(fs, "")
+ _ = fs.Parse(args)
reg, err := registry.Open(*registryPath)
if err != nil {
@@ -3158,8 +3144,8 @@ func RunWithActivatorReady(args []string, act Activator, cache *Cache, ready fun
stop := make(chan os.Signal, 1)
signal.Notify(stop, syscall.SIGTERM, syscall.SIGINT)
- // The route table and handlers are configured here. An early listener
- // still holds normal requests until publish below.
+ // The Edge is fully configured here — every EnableXxx applied and the
+ // route table rebuilt behind them — and nothing is listening yet.
if ready != nil {
ready(e)
}
@@ -3171,12 +3157,6 @@ func RunWithActivatorReady(args []string, act Activator, cache *Cache, ready fun
// ReadHeaderTimeout guards against Slowloris-style slow-header
// connection exhaustion. No ReadTimeout/WriteTimeout — those would
// break long-lived proxied WebSockets.
- if e.early != nil {
- e.early.publish(e, nil, nil)
- <-stop
- e.early.Close()
- return
- }
srv := &http.Server{Addr: *httpAddr, Handler: e, ReadHeaderTimeout: 10 * time.Second}
go func() {
log.Printf("edge: DEV MODE — plain HTTP, serving %d route(s) on %s (registry %s)",
@@ -3214,45 +3194,37 @@ func RunWithActivatorReady(args []string, act Activator, cache *Cache, ready fun
// control/certs/, which is what makes the gate safe to shut. nil when
// there is no deployment store — armCertIssuance has then cleared the
// gate too, and this box issues and serves entirely for itself.
- var afterListening []func()
- boot := func(f func()) {
- if e.early != nil {
- afterListening = append(afterListening, f)
- } else {
- f()
- }
- }
mirror := e.newCertMirror(localCerts)
if mirror != nil {
- // Legacy agents refresh after publishing the handlers and can serve
- // locally cached certificates meanwhile. Other callers keep their
- // synchronous first refresh.
- boot(func() {
- ctx, cancel := context.WithTimeout(context.Background(), certMirrorBootTimeout)
- if err := mirror.Refresh(ctx); err != nil {
- log.Printf("edge: first cert mirror refresh failed: %v — serving from this box's own cache until a later poll lands", err)
- }
- cancel()
- })
+ // One synchronous pass before anything listens: a box that has just
+ // booted must not spend its first 30 seconds handshaking with nothing,
+ // and the wildcard manager's first pass (below) reads what this lands
+ // in the local cache.
+ ctx, cancel := context.WithTimeout(context.Background(), certMirrorBootTimeout)
+ if err := mirror.Refresh(ctx); err != nil {
+ log.Printf("edge: first cert mirror refresh failed: %v — serving from this box's own cache until a later poll lands", err)
+ }
+ cancel()
}
// The home map and the mesh keys (homemap.go), armed beside the cert
- // mirror and on the same terms. Publish the handle before the HTTP
- // handler; the first refresh is asynchronous for legacy agents.
+ // mirror and on the same terms: nil when there is no deployment store to
+ // mirror from, one synchronous refresh before anything listens so the
+ // first request does not meet an empty map, then the 30s loop. Stored
+ // before either listener binds, so ServeHTTP never sees a half-armed edge.
if hm := e.newHomeMap(); hm != nil {
- boot(func() {
- ctx, cancel := context.WithTimeout(context.Background(), homeMapBootTimeout)
- if err := hm.Refresh(ctx); err != nil {
- log.Printf("edge: first home map refresh failed: %v — hosts homed on another box are unreachable from here until a later poll lands", err)
- }
- cancel()
- })
+ ctx, cancel := context.WithTimeout(context.Background(), homeMapBootTimeout)
+ if err := hm.Refresh(ctx); err != nil {
+ log.Printf("edge: first home map refresh failed: %v — hosts homed on another box are unreachable from here until a later poll lands", err)
+ }
+ cancel()
e.homes.Store(hm)
go hm.run(make(chan struct{})) // process-lifetime; never stopped, like the cert mirror's
}
// The edge-local door (door.go, #175 P7), armed right after the home map
- // it dials through: its own key mirror, its loop, then the socket. Off, with the reason in the journal, when there
+ // it dials through: its own key mirror, one synchronous refresh, its
+ // loop, then the socket. Off, with the reason in the journal, when there
// is no store, no release key or no public address. Closed in the
// shutdown below, ahead of the public listeners.
_ = e.armDoor(context.Background())
@@ -3292,7 +3264,7 @@ func RunWithActivatorReady(args []string, act Activator, cache *Cache, ready fun
// A remote box's edges file landing (or leaving) can add or drop a
// parent (#300): re-derive on the home map's poll, not the 12h tick.
e.homes.Load().setHostsWake(w.refresh)
- boot(w.refresh)
+ w.refresh()
go w.run()
log.Printf("edge: wildcard certs on (static %v, apps domain %q, delegation zone %q)",
splitDomains(os.Getenv("CARLOS_WILDCARD_DOMAINS")), os.Getenv("PLATFORM_APPS_DOMAIN"), w.delegationZone)
@@ -3348,19 +3320,6 @@ func RunWithActivatorReady(args []string, act Activator, cache *Cache, ready fun
acmeAndFallback = mirror.challengeHandler(acmeAndFallback)
}
- if e.early != nil {
- e.early.publish(acmeAndFallback, e, tlsCfg)
- for _, f := range afterListening {
- go f()
- }
- <-stop
- if e.closeDoor != nil {
- e.closeDoor()
- }
- e.early.Close()
- return
- }
-
// ReadHeaderTimeout guards the public front door against Slowloris-style
// slow-header connection exhaustion. No ReadTimeout/WriteTimeout — those
// would break long-lived proxied WebSockets.
diff --git a/internal/edge/startup_test.go b/internal/edge/startup_test.go
new file mode 100644
index 000000000..81fb78612
--- /dev/null
+++ b/internal/edge/startup_test.go
@@ -0,0 +1,86 @@
+package edge
+
+import (
+ "bufio"
+ "bytes"
+ "context"
+ "io"
+ "net"
+ "net/http"
+ "os"
+ "os/exec"
+ "path/filepath"
+ "strings"
+ "testing"
+ "time"
+)
+
+func TestStartupGateFailsWhenHandlerSetupFails(t *testing.T) {
+ if os.Getenv("CARLOS_TEST_SETUP_FAILURE") == "1" {
+ RunWithActivatorReady([]string{"-dev", "-registry", filepath.Join(os.Getenv("CARLOS_DATA_DIR"), "registry.db"), "-http", os.Getenv("CARLOS_TEST_HTTP_ADDR")}, nil, nil, nil, func(*Edge) {
+ _, _ = os.Stdout.WriteString("configuring\n")
+ _, _ = io.CopyN(io.Discard, os.Stdin, 1)
+ })
+ os.Exit(0)
+ }
+ listener, err := net.Listen("tcp", "127.0.0.1:0")
+ if err != nil {
+ t.Fatal(err)
+ }
+ addr := listener.Addr().String()
+ listener.Close()
+ exe, err := os.Executable()
+ if err != nil {
+ t.Fatal(err)
+ }
+ ctx, cancel := context.WithTimeout(t.Context(), 10*time.Second)
+ defer cancel()
+ cmd := exec.CommandContext(ctx, exe, "-test.run=^TestStartupGateFailsWhenHandlerSetupFails$")
+ dd := t.TempDir()
+ cmd.Dir = dd
+ cmd.Env = []string{
+ "HOME=" + dd, "TMPDIR=" + os.TempDir(),
+ "CARLOS_DATA_DIR=" + dd, "CARLOS_TEST_SETUP_FAILURE=1", "CARLOS_TEST_HTTP_ADDR=" + addr,
+ "CARLOS_TUNNEL_HOST=tunnel.example.com", "CARLOS_TUNNEL_CLIENTS=invalid",
+ }
+ var stderr bytes.Buffer
+ cmd.Stderr = &stderr
+ stdout, err := cmd.StdoutPipe()
+ if err != nil {
+ t.Fatal(err)
+ }
+ stdin, err := cmd.StdinPipe()
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := cmd.Start(); err != nil {
+ t.Fatal(err)
+ }
+ defer func() { cancel(); _ = cmd.Wait() }()
+ line, err := bufio.NewReader(stdout).ReadString('\n')
+ if err != nil || line != "configuring\n" {
+ t.Fatalf("setup barrier: %q, %v", line, err)
+ }
+ client := &http.Client{Transport: &http.Transport{Proxy: nil}, Timeout: 100 * time.Millisecond}
+ defer client.CloseIdleConnections()
+ probeMustFail := func() {
+ t.Helper()
+ resp, err := client.Get("http://" + addr + "/")
+ if err == nil {
+ resp.Body.Close()
+ t.Fatalf("update probe succeeded before handler setup: %d", resp.StatusCode)
+ }
+ }
+ probeMustFail()
+ if _, err := stdin.Write([]byte("x")); err != nil {
+ t.Fatal(err)
+ }
+ stdin.Close()
+ if err := cmd.Wait(); err == nil {
+ t.Fatal("invalid tunnel configuration did not fail setup")
+ }
+ if !strings.Contains(stderr.String(), "edge: tunnels:") {
+ t.Fatalf("did not reach handler setup failure: %s", stderr.String())
+ }
+ probeMustFail()
+}
diff --git a/internal/host/agent.go b/internal/host/agent.go
index 8620001c5..350bb75a8 100644
--- a/internal/host/agent.go
+++ b/internal/host/agent.go
@@ -213,15 +213,9 @@ func RunAgent(args []string) { runAgent(args, productionClaimFactory) }
func runAgent(args []string, buildClaims ClaimFactory) {
agentCtx, cancelAgent := context.WithCancel(context.Background())
defer cancelAgent()
- var listeners *edge.EarlyListeners
- if !hostwrite.Enabled(os.Getenv) {
- var err error
- listeners, err = edge.ListenEarly(args)
- if err != nil {
- log.Fatal(err)
- }
- defer listeners.Close()
- }
+ // Startup can stop before children drain; rollout keeps agentCtx until Close.
+ startupCtx, cancelStartup := context.WithCancel(agentCtx)
+ defer cancelStartup()
claimCtx, err := claimEntryContext(agentCtx, os.Getenv, openDeployStore)
if err != nil {
log.Fatalf("agent: host claims: %v", err)
@@ -549,20 +543,20 @@ func runAgent(args []string, buildClaims ClaimFactory) {
// box, Start alone. The order and the branch are activatorStart's.
svc.Startup = started
startupDone = activatorStart{
- Configure: func(ctx context.Context) {
+ Configure: func(_ context.Context) {
armAlways(svc, reg, dd) // flag on or off
var rolloutSink activator.EventSink
if logSink != nil { // a typed nil in the interface would not be nil
rolloutSink = logSink
}
- armRolloutAndRelay(ctx, os.Getenv, svc, reg, dd, rolloutSink, openDeployStore)
+ armRolloutAndRelay(agentCtx, os.Getenv, svc, reg, dd, rolloutSink, openDeployStore)
// The residence-move machinery (issue #331), armed with the
// activator: the staging-fallback restore needs a same-region
// mirror client, and the move worker needs the service's quiesce
// answer. Both are best-effort — a mirror client that cannot
// build logs and leaves the fallback dark, exactly like every
// other optional duty here.
- if m, err := lease.NewS3Mirror(ctx, cfg.Region, cfg.Region); err != nil {
+ if m, err := lease.NewS3Mirror(agentCtx, cfg.Region, cfg.Region); err != nil {
log.Printf("agent: staging-fallback mirror client: %v (restore fallback disarmed)", err)
} else {
eb.Mirror = m
@@ -595,7 +589,7 @@ func runAgent(args []string, buildClaims ClaimFactory) {
Slot: svc.PrepareScheduledToken,
Publish: armedLayoutPass.Store,
Start: svc.Start,
- }.Run(agentCtx)
+ }.Run(startupCtx)
act = svc
// The agent socket: the one channel a sidecar dials back over
@@ -629,7 +623,7 @@ func runAgent(args []string, buildClaims ClaimFactory) {
go func() {
select {
case <-started:
- case <-agentCtx.Done():
+ case <-startupCtx.Done():
return
}
routes, err := reg.List()
@@ -654,15 +648,19 @@ func runAgent(args []string, buildClaims ClaimFactory) {
}
}()
var stopHousekeeper func()
+ if !claims.Writes.Enabled {
+ // Certificate issuance must not wait for instance recovery.
+ stopHousekeeper = startHousekeeper()
+ }
dutiesDone := make(chan struct{})
runDuties := func() {
defer close(dutiesDone)
select {
case <-started:
- case <-agentCtx.Done():
+ case <-startupCtx.Done():
return
}
- if agentCtx.Err() != nil {
+ if startupCtx.Err() != nil {
return
}
@@ -718,7 +716,7 @@ func runAgent(args []string, buildClaims ClaimFactory) {
log.Fatalf("agent: host claims: %v", err)
}
if sd != nil {
- sd.runner.Start(agentCtx) // runs until the process ends, like every loop below
+ sd.runner.Start(context.Background()) // runs until the process ends, like every loop below
}
serialClaims := make(chan struct{}, 1)
regionalReport := func(err error) { log.Printf("agent: regional claims: %v", err) }
@@ -755,7 +753,9 @@ func runAgent(args []string, buildClaims ClaimFactory) {
// minute away) — see housekeeper.go. Not a `go` of its own: it publishes
// the process's housekeeper and spawns its loop itself. Its stand-down
// runs in the post-drain block at the end of this function.
- stopHousekeeper = startHousekeeper()
+ if claims.Writes.Enabled {
+ stopHousekeeper = startHousekeeper()
+ }
go retentionLoop(reg, collectBuildStates)
go registrySnapshotLoop(reg)
go edgesFileLoop(reg)
@@ -791,7 +791,7 @@ func runAgent(args []string, buildClaims ClaimFactory) {
// The same serving loop as `carlos edge`, with the activator injected.
// Caching applies here too: cached hits still avoid the activator, which
// is the entire point of checking the cache before the hibernation branch.
- // The ready hook fires synchronously before publishing the handler, so
+ // The ready hook fires synchronously before listening, so
// tunnelLoop is handed off to its own goroutine rather than called here —
// calling it directly would block the front door forever.
edgeOpts := append([]edge.RunOption{
@@ -825,9 +825,6 @@ func runAgent(args []string, buildClaims ClaimFactory) {
if tunnelEnvComplete(os.Getenv) {
edgeOpts = append(edgeOpts, edge.WithGeoTunnelClient())
}
- if listeners != nil {
- edgeOpts = append(edgeOpts, listeners.Option())
- }
edge.RunWithActivatorReady(args, act, cache, func(e *edge.Edge) {
go tunnelLoop(e)
// The edge owns the health probers, so its state table is only
@@ -862,15 +859,15 @@ func runAgent(args []string, buildClaims ClaimFactory) {
go usageLoop(m.Drain, am)
}, edgeOpts...)
- cancelAgent()
+ cancelStartup()
joinCtx, joinCancel := context.WithTimeout(context.Background(), 5*time.Second)
defer joinCancel()
joinAgentStartup(joinCtx, startupDone)
- dutiesJoined := joinAgentStartup(joinCtx, dutiesDone)
+ joinAgentStartup(joinCtx, dutiesDone)
// The edge has drained; take the children down too (they're this process's
// children — orphaning them would strand their leases until TTL expiry).
- if svc != nil && startupFinished(started) {
+ if svc != nil {
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
svc.Shutdown(ctx)
@@ -902,7 +899,7 @@ func runAgent(args []string, buildClaims ClaimFactory) {
// roll of a single-box deployment. Stopping cancels every duty in flight
// and releases the lock, so the next box (or this one, a second later)
// takes over immediately.
- if dutiesJoined && stopHousekeeper != nil {
+ if stopHousekeeper != nil {
stopHousekeeper()
}
}
diff --git a/internal/host/instancekeys_test.go b/internal/host/instancekeys_test.go
index 111cfe54c..8babc8ff4 100644
--- a/internal/host/instancekeys_test.go
+++ b/internal/host/instancekeys_test.go
@@ -153,20 +153,22 @@ func TestActivatorStartBackfillsOnlyALegacyBoxAfterRecoveryAndStart(t *testing.T
listErr error
want []string
}{
- {"legacy box", false, nil, []string{"sweep", "recover", "publish", "start", "backfill"}},
- {"legacy box, routes unreadable", false, errors.New("db locked"), []string{"sweep", "recover", "publish", "start"}},
- {"claim box", true, nil, []string{"start"}},
+ {"legacy box", false, nil, []string{"configure", "sweep", "recover", "publish", "start", "ready", "backfill"}},
+ {"legacy box, routes unreadable", false, errors.New("db locked"), []string{"configure", "sweep", "recover", "publish", "start", "ready"}},
+ {"claim box", true, nil, []string{"configure", "start", "ready"}},
} {
t.Run(tc.name, func(t *testing.T) {
var mu sync.Mutex
var events []string
note := func(e string) { mu.Lock(); events = append(events, e); mu.Unlock() }
done := activatorStart{
- Claims: tc.claims,
- Sweep: func(context.Context) error { note("sweep"); return nil },
- Recover: func(context.Context, bool) *layout.Pass { note("recover"); return nil },
- Routes: func() ([]registry.Route, error) { return []registry.Route{ready}, tc.listErr },
- Route: routeOf(ready),
+ Configure: func(context.Context) { note("configure") },
+ Ready: func() { note("ready") },
+ Claims: tc.claims,
+ Sweep: func(context.Context) error { note("sweep"); return nil },
+ Recover: func(context.Context, bool) *layout.Pass { note("recover"); return nil },
+ Routes: func() ([]registry.Route, error) { return []registry.Route{ready}, tc.listErr },
+ Route: routeOf(ready),
Keys: keyBackfillFunc(func(context.Context, registry.Route) (activator.InstanceKeyOutcome, error) {
note("backfill")
return activator.InstanceKeyMinted, nil
@@ -175,6 +177,13 @@ func TestActivatorStartBackfillsOnlyALegacyBoxAfterRecoveryAndStart(t *testing.T
Publish: func(*layout.Pass) { note("publish") },
Start: func() { note("start") },
}.Run(context.Background())
+ if tc.claims {
+ select {
+ case <-done:
+ default:
+ t.Fatal("claim-box startup became asynchronous")
+ }
+ }
select {
case <-done:
case <-time.After(10 * time.Second):
diff --git a/internal/host/startup.go b/internal/host/startup.go
index 60ec2ee66..d1de4d560 100644
--- a/internal/host/startup.go
+++ b/internal/host/startup.go
@@ -13,12 +13,3 @@ func joinAgentStartup(ctx context.Context, done <-chan struct{}) bool {
return false
}
}
-
-func startupFinished(done <-chan struct{}) bool {
- select {
- case <-done:
- return true
- default:
- return false
- }
-}
diff --git a/internal/host/startup_test.go b/internal/host/startup_test.go
index 63875ea44..bb07584b0 100644
--- a/internal/host/startup_test.go
+++ b/internal/host/startup_test.go
@@ -21,14 +21,15 @@ import (
type startupBackend struct {
fakeWakeBackend
recovered *atomic.Bool
+ started *atomic.Bool
attempts atomic.Int32
gate func() error
}
func (b *startupBackend) Acquire(ctx context.Context, rt registry.Route) (activator.Lease, error) {
b.attempts.Add(1)
- if !b.recovered.Load() {
- return nil, errors.New("wake preceded recovery")
+ if !b.recovered.Load() || !b.started.Load() {
+ return nil, errors.New("wake preceded recovery or Start")
}
if b.gate != nil {
if err := b.gate(); err != nil {
@@ -69,8 +70,8 @@ func TestStartupServesWhileSweepAndRecoveryWait(t *testing.T) {
if err := reg.Upsert(registry.Route{Host: "console.example.com", Addr: console.Listener.Addr().String()}); err != nil {
t.Fatal(err)
}
- var recovered atomic.Bool
- backend := &startupBackend{fakeWakeBackend: fakeWakeBackend{inst: inst}, recovered: &recovered}
+ var recovered, startedService atomic.Bool
+ backend := &startupBackend{fakeWakeBackend: fakeWakeBackend{inst: inst}, recovered: &recovered, started: &startedService}
control := instancectl.NewClient(filepath.Join(dd, "unreachable.sock"))
if fail {
backend.gate = nestedGate(func() error { return nil }, func(string) (bool, error) { return true, nil }, control.Ready)
@@ -110,7 +111,7 @@ func TestStartupServesWhileSweepAndRecoveryWait(t *testing.T) {
recovered.Store(true)
return nil
},
- Publish: func(*layout.Pass) {}, Start: svc.Start, Ready: func() { close(ready) },
+ Publish: func(*layout.Pass) {}, Start: func() { svc.Start(); startedService.Store(true) }, Ready: func() { close(ready) },
Routes: func() ([]registry.Route, error) { return nil, nil },
}.Run(ctx)
defer func() { cancel(); <-done }()
@@ -239,3 +240,46 @@ func TestStartupJoinsKeyPassOnAgentCancellation(t *testing.T) {
t.Fatal("key pass outlived agent cancellation")
}
}
+
+func TestStartupCancellationDuringRecoveryPreservesAgentLifetime(t *testing.T) {
+ agentCtx, cancelAgent := context.WithCancel(t.Context())
+ defer cancelAgent()
+ startupCtx, cancelStartup := context.WithCancel(agentCtx)
+ defer cancelStartup()
+ recovering := make(chan struct{})
+ svc := activator.New(activator.Config{}, &fakeWakeBackend{}, nil)
+ ready := make(chan struct{})
+ svc.Startup = ready
+ done := activatorStart{
+ Configure: func(context.Context) {
+ if agentCtx.Err() != nil {
+ t.Error("agent lifetime ended before configuration")
+ }
+ },
+ Sweep: func(context.Context) error { return nil },
+ Recover: func(ctx context.Context, _ bool) *layout.Pass {
+ close(recovering)
+ <-ctx.Done()
+ return nil
+ },
+ Publish: func(*layout.Pass) {},
+ Start: func() { t.Error("Start after recovery cancellation") },
+ Ready: func() { close(ready) },
+ }.Run(startupCtx)
+ <-recovering
+ cancelStartup()
+ join, stop := context.WithTimeout(t.Context(), time.Second)
+ defer stop()
+ if !joinAgentStartup(join, done) {
+ t.Fatal("recovery did not join")
+ }
+ svc.Shutdown(join)
+ if agentCtx.Err() != nil {
+ t.Fatal("startup cancellation stopped rollout's agent lifetime before children drained")
+ }
+ select {
+ case <-ready:
+ t.Fatal("cancelled recovery released admission")
+ default:
+ }
+}
diff --git a/internal/provision/plan14_root_reads_test.go b/internal/provision/plan14_root_reads_test.go
index 5c8cb566f..4cea48bd2 100644
--- a/internal/provision/plan14_root_reads_test.go
+++ b/internal/provision/plan14_root_reads_test.go
@@ -657,7 +657,7 @@ func TestPlan14RootReadCensus(t *testing.T) {
if err = loadRootReadInventory(filepath.Join(root, "internal/provision/testdata/plan14-root-reads-hotfix281.tsv"), &inv, true); err != nil {
t.Fatal(err)
}
- seal = "48e04b1e94edbd7c05ed68b0e6f8f060934365ff7fcc8c91bbbd183da69be0c2"
+ seal = "4b12ba7e94ff1fbc7f2313dbcfb79c5ac1f0455f248f214a8257c08805c1a70f"
errs := append(compareRootReads(got, inv), checkRootReadLegacy(inv, seal)...)
errs = append(errs, checkRootReadDeclarations(got, inv)...)
if len(errs) > 0 {
diff --git a/internal/provision/testdata/plan14-root-reads-hotfix281.tsv b/internal/provision/testdata/plan14-root-reads-hotfix281.tsv
index 48803e471..a46eb8992 100644
--- a/internal/provision/testdata/plan14-root-reads-hotfix281.tsv
+++ b/internal/provision/testdata/plan14-root-reads-hotfix281.tsv
@@ -1,4 +1,4 @@
-# Hotfix 281: reviewed read-preserving startup changes. Earlier overlays remain unchanged.
+# Hotfix 281: changed units only; read-preserving startup changes under census-ruling-hotfix281.
@legacy f28afc22c409e85b3c186db8947c41e30c541bbd007b1b374e56e2c98cc0a74c internal/activator/activator.go Service.BeginSidecarRequest legacy accepted-legacy accepted-legacy;rev3=census-ruling-hotfix281;basis=The same host reads follow a bounded startup wait; no new source or authority. registry:Host@259,registry:Host@333 2f3da0a2d25f3c75ed9b89dd361e28a00410843fe0f80cc3995e8f1f1a1ae994
internal/activator/activator.go Service.BeginSidecarRequest legacy accepted-legacy accepted-legacy;rev3=census-ruling-hotfix281;basis=The same host reads follow a bounded startup wait; no new source or authority. registry:Host@259,registry:Host@333 2f3da0a2d25f3c75ed9b89dd361e28a00410843fe0f80cc3995e8f1f1a1ae994
@legacy ed600af90f562c710f917d1d0a5ebc0fae5c9f0851a69f23395a18bb83901ff3 internal/activator/activator.go Service.WakeWorker legacy accepted-legacy accepted-legacy;rev3=census-ruling-hotfix281;basis=The same host and route reads follow a bounded startup wait; no new source or authority. registry:Host@1242,registry:Host@1652,registry:Host@834,registry:Host@864,whole:Route@1225 bda1e33466a92d540eeb724284aa431550a323e7dbb432cc7b8872498929bec9
@@ -9,15 +9,10 @@ internal/activator/activator.go Service.begin legacy accepted-legacy accepted-le
internal/activator/activator.go Service.serve legacy accepted-legacy accepted-legacy;rev3=census-ruling-hotfix281;basis=Startup repeats the existing Kind and Hibernate checks; error logging moved to serveWakeError. The same route fields and sources are read. registry:Hibernate@151,registry:Hibernate@788,registry:Host@397,registry:Kind@126,registry:Kind@762,whole:Route@1907,whole:Route@2876,whole:Route@2936,whole:Route@347 44becc931aec04495ff7daae2154fbbf500c4e6d4df65b328a790e33cfa3a0ac
@legacy bf4ca1db43d8023db08187eb3afdcfbd4599b3f6966c81274b60e2752b6c5fac internal/activator/activator.go Service.serveWakeError legacy accepted-legacy accepted-legacy;rev3=census-ruling-hotfix281;basis=Extracted unchanged error mapping and Host log read from Service.serve; no new source or authority. registry:Host@1364 e4ad730f8b463cf112c59bd061c7384ff4afe626d2bedd7f90ccd3df791333ba
internal/activator/activator.go Service.serveWakeError legacy accepted-legacy accepted-legacy;rev3=census-ruling-hotfix281;basis=Extracted unchanged error mapping and Host log read from Service.serve; no new source or authority. registry:Host@1364 e4ad730f8b463cf112c59bd061c7384ff4afe626d2bedd7f90ccd3df791333ba
-@legacy 910d75544b6bbcfffefa1958ca9170599ddec521faacf0ab1ac4d71fe6e345dd internal/edge/edge.go RunWithActivatorReady legacy accepted-legacy accepted-legacy;rev3=census-ruling-hotfix281;basis=Flag parsing moved to parseRunFlags; the same registry and data-directory sources are used. Initial mirrors run asynchronously only with early listeners. data-literal@297,data-literal@316,file:Open@339 f061377870ded091f7c0de63f17733e48edd9c9a063025898521497ad13bdc42
-internal/edge/edge.go RunWithActivatorReady legacy accepted-legacy accepted-legacy;rev3=census-ruling-hotfix281;basis=Flag parsing moved to parseRunFlags; the same registry and data-directory sources are used. Initial mirrors run asynchronously only with early listeners. data-literal@297,data-literal@316,file:Open@339 f061377870ded091f7c0de63f17733e48edd9c9a063025898521497ad13bdc42
-@legacy df3dc08211aba4445f61ccaf6fda66ff445ef32a31bffbeec8536a84c11f238d internal/edge/edge.go parseRunFlags legacy accepted-legacy accepted-legacy;rev3=census-ruling-hotfix281;basis=Extracted unchanged flags and data-directory defaults from RunWithActivatorReady; no new source or authority. data-literal@218,data-literal@351,data-literal@63,data-literal@82 f0109557c4e84d4b52c3dc73ef31a7da34ecbbe57fbddb2f3ae6358557acb5e2
-internal/edge/edge.go parseRunFlags legacy accepted-legacy accepted-legacy;rev3=census-ruling-hotfix281;basis=Extracted unchanged flags and data-directory defaults from RunWithActivatorReady; no new source or authority. data-literal@218,data-literal@351,data-literal@63,data-literal@82 f0109557c4e84d4b52c3dc73ef31a7da34ecbbe57fbddb2f3ae6358557acb5e2
-internal/host/agent.go runAgent authority B,C,D,F rev3=census-ruling-hotfix281;basis=Startup ordering and lifetime changed; registry, route, data-directory and backend inputs are unchanged. Existing B,C,D,F authority classification is retained. file:Exec@13711,file:Open@768,registry:App@10102,whole:Route@10094,whole:Route@21421 8fb88c131c968692ad2bf2967e9968aff19dc7caa195e0a56df67b2fc4eb85ce
+internal/host/agent.go runAgent authority B,C,D,F rev3=census-ruling-hotfix281;basis=Only activator startup and dependent duties run asynchronously; shutdown cancels their child context before the join, preserving the agent lifetime through child and rollout drain. Registry, route, data-directory and backend inputs are unchanged; B,C,D,F authority is retained. file:Exec@13674,file:Open@731,registry:App@10065,whole:Route@10057,whole:Route@21396 5d8ca6215a4f7eddc85fd05a6696494d22922e8c23eab4d4eb91605377d2277c
@legacy daa120f77654cd1238b494d92b7a4db9ee115f4f56945243b26a7ebe02e584db internal/host/instancekeys.go backfillInstanceKeys legacy accepted-legacy accepted-legacy;rev3=census-ruling-hotfix281;basis=Cancellation stops the loop early; route reads, per-host preparation and key sources are unchanged. registry:Host@553,registry:Host@701,registry:Host@867,whole:Route@251,whole:Route@320,whole:Route@399 b4d7ba9a5dc32cb04c214adc69184d91bc65d2487c661150fed899f1deba676d
internal/host/instancekeys.go backfillInstanceKeys legacy accepted-legacy accepted-legacy;rev3=census-ruling-hotfix281;basis=Cancellation stops the loop early; route reads, per-host preparation and key sources are unchanged. registry:Host@553,registry:Host@701,registry:Host@867,whole:Route@251,whole:Route@320,whole:Route@399 b4d7ba9a5dc32cb04c214adc69184d91bc65d2487c661150fed899f1deba676d
@legacy 8f772c7c43dec1102a01dbe4a945b8eb0c751f0e2f6efe05dc437e13b21daede internal/host/layoutleg.go activatorStart.Run legacy accepted-legacy accepted-legacy;rev3=census-ruling-hotfix281;basis=Legacy sweep, recovery and key pass run under the agent lifetime in the background. The same route list and callbacks are used; claim boxes retain Start alone. whole:Route@803 e93d0c33f6ac39f9c644b7bfa2463c30a9980dfd0e23a515ba87f356e8899c57
internal/host/layoutleg.go activatorStart.Run legacy accepted-legacy accepted-legacy;rev3=census-ruling-hotfix281;basis=Legacy sweep, recovery and key pass run under the agent lifetime in the background. The same route list and callbacks are used; claim boxes retain Start alone. whole:Route@803 e93d0c33f6ac39f9c644b7bfa2463c30a9980dfd0e23a515ba87f356e8899c57
-@coverage internal/activator ccf2519fb636a45e909e302662a619d14a032a5e191df87971b413fecb87aabb
-@coverage internal/edge d5d3002fd6f1f920aeb983234d7ca8c4f83ad353fdeae76e37fc375ea8def822
-@coverage internal/host 2aaede4445eb11839322b809d81e5b34eb1f610debf57db03a4e7027de95185a
+@coverage internal/activator af5b4506d7b3146462790223d541f6caca55b0606055734733e79accaad9440a
+@coverage internal/host ee4e46a4efcbea38cbc81fcdcab7ac4dc5eac2eeecb9ed879aecc4968315cf74