carlos / platform Public

config sync: keep a waiting host until its config is actually written (fix round 1, finding 3)

Astra's review, MEDIUM. syncExecSerialized cleared every waiting host after
any pass that returned nil, but SyncExec logs and leaves a route as it is on
an unresolved bundle or overlay and still returns nil. A transient bundle
read on the first tick after the directory appeared dropped the host, and it
waited for the hourly pass again.

SyncExec now names each in-scope host done or not once it has gone through
every route (WithHostOutcome): done when the file is as the pass resolved it
(written, up to date, removed, rightly absent). The agent keeps a waiting
host until a pass reports it done, or until a complete pass no longer has it
in scope (row gone or no longer bundled). A host a pass leaves undone with
its directory there is retried on the next tick, then backs off (1, 2, 4 ...
ticks, at most 16, about a minute), so a bundle that will not read does not
run a full pass every 3 s.

Test: a transient read failure on the first tick after the directory
appears; the host stays waited for and gets its config on the next tick; a
bundle that keeps failing is retried on the backoff. With the old clearing
rule it fails.

Census: SyncExecWithCallbacks and pendingDirs.appeared re-pinned, envelopes
internal/appconfig and internal/host, seal edc04f73.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

claude 4 days ago 7cd5db4d41bb4a9ba844024b20194763b7e92426 parent 4e24d5a

6 files changed, +219 −32

docs/superpowers/notes/census-proposal-moves-offline-fixes.md +3 −1
internal/appconfig/syncexec.go +27 −0
internal/host/agent.go +64 −17
internal/host/configdirpending_test.go +117 −7
internal/provision/plan14_root_reads_test.go +1 −1
internal/provision/testdata/plan14-root-reads-moves-offline-fixes.tsv +7 −6

CI cancelled — superseded by a newer push — run details

CI log

=== gate ===
cd launchctx && go vet ./... && go test ./... -count=1
ok  	amadan.net/carlos/platform.git/launchctx	0.012s
?   	amadan.net/carlos/platform.git/launchctx/controlname	[no test files]
?   	amadan.net/carlos/platform.git/launchctx/wireid	[no test files]
go vet ./...
gofmt -l .: clean
go test ./... -count=1 -timeout 30m
ok  	github.com/carlosframework/platform	331.092s
?   	github.com/carlosframework/platform/cmd/changelog	[no test files]
ok  	github.com/carlosframework/platform/cmd/console	2.866s
ok  	github.com/carlosframework/platform/cmd/moveprobe	92.344s
ok  	github.com/carlosframework/platform/cmd/zddprobe	0.726s
ok  	github.com/carlosframework/platform/internal/activator	251.693s
ok  	github.com/carlosframework/platform/internal/agentsock	0.018s
ok  	github.com/carlosframework/platform/internal/alerting	0.004s
ok  	github.com/carlosframework/platform/internal/analytics	0.019s
ok  	github.com/carlosframework/platform/internal/appconfig	2.526s
ok  	github.com/carlosframework/platform/internal/apperr	0.074s
ok  	github.com/carlosframework/platform/internal/applog	0.164s
ok  	github.com/carlosframework/platform/internal/awsinv	0.004s
ok  	github.com/carlosframework/platform/internal/boxchoice	0.004s
ok  	github.com/carlosframework/platform/internal/boxid	0.803s
?   	github.com/carlosframework/platform/internal/boxid/boxidtest	[no test files]
?   	github.com/carlosframework/platform/internal/boxid/internal/signtest	[no test files]
ok  	github.com/carlosframework/platform/internal/boxvolumes	0.008s
ok  	github.com/carlosframework/platform/internal/caps	0.017s
ok  	github.com/carlosframework/platform/internal/certwarm	0.003s
ok  	github.com/carlosframework/platform/internal/cgroup	0.714s
ok  	github.com/carlosframework/platform/internal/changelog	0.347s
ok  	github.com/carlosframework/platform/internal/claimactivation	0.863s
ok  	github.com/carlosframework/platform/internal/clamd	0.055s
?   	github.com/carlosframework/platform/internal/cliflag	[no test files]

amadan: job cancelled — a newer push superseded it

diff --git a/docs/superpowers/notes/census-proposal-moves-offline-fixes.md b/docs/superpowers/notes/census-proposal-moves-offline-fixes.md

index 67ba505c7..0891f7ba9 100644

--- a/docs/superpowers/notes/census-proposal-moves-offline-fixes.md

+++ b/docs/superpowers/notes/census-proposal-moves-offline-fixes.md

@@ -66,13 +66,15 @@ Envelopes re-pinned: `internal/host`, `internal/schedule`.

| 1 | `internal/schedule/runner.go` `Runner.dueRows` | new: held hosts left out by the query when `Skip` is set, and hosts `Skip` names paged past before the limit (one row read per host in the batch) | the agent's schedule runner | accepted legacy, J1 (as `Runner.Tick`) |

## The seal

The accepted-legacy rows add frozen declarations, so the legacy seal moves,

from `393e7f45…bec29` (main, after moves-offline and armed-box-updates) to

-`0d4e20c578532f1ca600de8569e1a46036bd959c14efbe879fb8bca47dcf4568`. The test

+`edc04f7349acf2b9ea98ffc8a1d3b71516a6ddb95505abfc2e38e49fd802ccc2` (after fix round 1). The test

checks main's seal before it loads this overlay, so a change on main's side

still shows on its own.

diff --git a/internal/appconfig/syncexec.go b/internal/appconfig/syncexec.go

index b02382b4d..493627a7a 100644

--- a/internal/appconfig/syncexec.go

+++ b/internal/appconfig/syncexec.go

@@ -439,11 +439,15 @@ func SyncExecWithCallbacks(ctx context.Context, reg *registry.Registry, s releas

// the errors.As branch below.

said := map[appKey]bool{}

+ // done is WithHostOutcome's answer per in-scope host: false until the

+ // host's file is as this pass resolved it.

+ done := map[string]bool{}

for _, rt := range routes {

in, addrShape, sidecarShape := inScope(rt)

if !in {

continue

}

+ done[rt.Host] = false

// A host the layout pass holds has its files between two

// directories, and either path this pass could pick would be the

// wrong one by the time the pass finishes. Nothing is written or

@@ -652,6 +656,14 @@ func SyncExecWithCallbacks(ctx context.Context, reg *registry.Registry, s releas

if err != nil && hostWrites.Enabled {

return err

}

+ if err == nil {

+ done[rt.Host] = true

+ }

+ if o.outcome != nil {

+ for host, ok := range done {

+ o.outcome(host, ok)

+ }

}

// Stale-secrets notice (issue #18 design ruling, ratified 2026-08-03

// review): an addr-shaped route whose app is DEFINITIVELY not opted in

@@ -741,6 +753,7 @@ type SyncOption func(*syncOptions)

type syncOptions struct {

fenced func(host string) bool

dirAbsent func(host, dir string)

+ outcome func(host string, done bool)

hostWrites hostwrite.Config

hostWriteCount int

}

@@ -776,6 +789,20 @@ func SkipFenced(fenced func(host string) bool) SyncOption {

return func(o *syncOptions) { o.fenced = fenced }

}

+// WithHostOutcome hands SyncExec a callback it calls, once a pass has gone

+// through every route without stopping, for each in-scope route: done is

+// true when the route's file is as the pass resolved it (written, already up

+// to date, removed, or rightly left absent), and false when the pass left it

+// as it was for any reason (an unresolved bundle or overlay, a guard, a

+// layout migration in flight, a write that failed, an instance directory not

+// made yet). A route out of scope is not named. The agent keeps a host whose

+// instance directory it waits for until the host is done (fix round 1,

+// Astra's finding 3: a pass that skipped a host on a transient read failure

+// returned nil, and the agent took that as delivered).

+func WithHostOutcome(outcome func(host string, done bool)) SyncOption {

+ return func(o *syncOptions) { o.outcome = outcome }

// execRouteBase is the lifecycle-managed-instance shape every exec route

// kind shares: the unit path's syncable() exclusions, narrowed to

// non-unit backing (unit routes belong to the root adopt path). Neither

diff --git a/internal/host/agent.go b/internal/host/agent.go

index 451941db1..98caa3e48 100644

--- a/internal/host/agent.go

+++ b/internal/host/agent.go

@@ -1376,9 +1376,11 @@ func syncExecSerialized(ctx context.Context, reg *registry.Registry, store relea

configSyncMu.Lock()

defer configSyncMu.Unlock()

pending := map[string]string{}

+ outcomes := map[string]bool{}

opts := []appconfig.SyncOption{

appconfig.SkipFenced(func(host string) bool { return layout.JournalPresent(dd, host) }),

appconfig.WhenInstanceDirAbsent(func(host, dir string) { pending[host] = dir }),

+ appconfig.WithHostOutcome(func(host string, done bool) { outcomes[host] = done }),

}

for _, cfg := range writes {

opts = append(opts, appconfig.WithHostWrite(cfg))

@@ -1392,7 +1394,7 @@ func syncExecSerialized(ctx context.Context, reg *registry.Registry, store relea

} else {

err = appconfig.SyncExec(ctx, reg, store, priv, dd, bounce(svc), os.Getenv("PLATFORM_APPS_DOMAIN"), configAckFunc(ctx, store, "agent"), opts...)

}

- configDirsPending.record(pending, err == nil)

+ configDirsPending.record(pending, outcomes, err == nil)

return err

}

@@ -1408,35 +1410,80 @@ var configDirsPending pendingDirs

type pendingDirs struct {

mu sync.Mutex

- dirs map[string]string

+ dirs map[string]*pendingDir

}

-// record keeps what one pass found. Every pass sweeps every route, so a pass

-// that completed replaces the set; one that failed part way only adds to it,

-// since a host it never reached may still be waiting.

-func (p *pendingDirs) record(found map[string]string, complete bool) {

+// pendingDir is one host the config pass waits for: its instance directory,

+// and, once the directory is there, how many passes since have left the host

+// undone and how many ticks to let go by before the next retry.

+type pendingDir struct {

+ dir string

+ failures int

+ waitTicks int

+// pendingRetryCap bounds the retry backoff: at most 16 generation-loop ticks

+// (about a minute) between passes for a host whose directory is there but

+// whose config a pass keeps leaving undone (a bundle that will not read).

+const pendingRetryCap = 16

+// record keeps what one pass found: absent, the hosts whose instance

+// directory was not there; outcomes, every in-scope host the pass named,

+// done or not (appconfig.WithHostOutcome). A host stays waited for until a

+// pass reports it done (fix round 1, Astra's finding 3: a pass that left it

+// undone on a read failure still returned nil), or until a complete pass no

+// longer has it in scope (its row is gone or no longer takes a bundle). A

+// pass that failed part way names fewer hosts than it might have, so it only

+// adds. A host a pass leaves undone with its directory there backs off,

+// doubling from the next tick up to pendingRetryCap ticks.

+func (p *pendingDirs) record(absent map[string]string, outcomes map[string]bool, complete bool) {

p.mu.Lock()

defer p.mu.Unlock()

- if complete || p.dirs == nil {

- p.dirs = map[string]string{}

+ if p.dirs == nil {

+ p.dirs = map[string]*pendingDir{}

}

- for host, dir := range found {

- p.dirs[host] = dir

+ for host, e := range p.dirs {

+ if _, again := absent[host]; again {

+ continue

+ }

+ done, named := outcomes[host]

+ switch {

+ case named && done, !named && complete:

+ delete(p.dirs, host)

+ case named:

+ e.failures++

+ e.waitTicks = min(1<<(e.failures-1), pendingRetryCap) - 1

+ }

+ for host, dir := range absent {

+ if e, ok := p.dirs[host]; ok {

+ e.dir = dir

+ continue

+ }

+ p.dirs[host] = &pendingDir{dir: dir}

}

-// appeared names a host whose instance directory the last pass waited for

-// and which exists now; ok is false when there is none. Lstat only: the

-// directory is the activator's, and nothing here follows or opens it.

+// appeared names a host whose instance directory a pass waited for, which

+// exists now, and whose retry is due; ok is false when there is none. Each

+// call is one tick of the backoff. Lstat only: the directory is the

+// activator's, and nothing here follows or opens it.

func (p *pendingDirs) appeared() (host string, ok bool) {

p.mu.Lock()

defer p.mu.Unlock()

- for h, dir := range p.dirs {

- if _, err := os.Lstat(dir); err == nil {

- return h, true

+ for h, e := range p.dirs {

+ if _, err := os.Lstat(e.dir); err != nil {

+ continue

+ }

+ if e.waitTicks > 0 {

+ e.waitTicks--

+ continue

+ }

+ if !ok {

+ host, ok = h, true

}

- return "", false

+ return host, ok

}

// configAckFunc builds the callback config sync's two passes (the root

diff --git a/internal/host/configdirpending_test.go b/internal/host/configdirpending_test.go

index dc3d1639d..748f18a2b 100644

--- a/internal/host/configdirpending_test.go

+++ b/internal/host/configdirpending_test.go

@@ -2,6 +2,7 @@ package host

import (

"context"

+ "errors"

"os"

"path/filepath"

"testing"

@@ -24,8 +25,8 @@ import (

// nothing while the directory is absent, and on the first tick after the

// first wake has made it, writes the file.

func TestConfigGenTickWritesANestedRowsConfigOnceItsDirAppears(t *testing.T) {

- configDirsPending.record(nil, true)

- t.Cleanup(func() { configDirsPending.record(nil, true) })

+ configDirsPending = pendingDirs{}

+ t.Cleanup(func() { configDirsPending = pendingDirs{} })

ctx := context.Background()

s := release.NewDirStore(t.TempDir())

reg := restartTestRegistry(t)

@@ -109,20 +110,129 @@ func TestConfigGenTickWritesANestedRowsConfigOnceItsDirAppears(t *testing.T) {

}

// A pass that fails part way does not forget a host an earlier pass was

-// waiting for; a complete pass replaces the set.

+// waiting for; a complete pass that no longer has the host in scope drops it.

func TestConfigDirsPendingKeepsWhatAFailedPassDidNotReach(t *testing.T) {

var p pendingDirs

dir := filepath.Join(t.TempDir(), "instances", "shop.example.com")

- p.record(map[string]string{"shop.example.com": dir}, true)

- p.record(map[string]string{}, false)

+ p.record(map[string]string{"shop.example.com": dir}, nil, true)

+ p.record(map[string]string{}, nil, false)

if err := os.MkdirAll(dir, 0o700); err != nil {

t.Fatal(err)

}

if h, ok := p.appeared(); !ok || h != "shop.example.com" {

t.Fatalf("appeared = %q %v after a failed pass", h, ok)

}

- p.record(map[string]string{}, true)

+ p.record(map[string]string{}, map[string]bool{}, true)

if h, ok := p.appeared(); ok {

- t.Fatalf("appeared = %q after a complete pass found nothing", h)

+ t.Fatalf("appeared = %q after a complete pass with the host out of scope", h)

}

+// blipStore fails reads of one key while fail is set: a bucket blip on

+// one app's bundle.

+type blipStore struct {

+ release.Store

+ key string

+ fail bool

+func (f *blipStore) Get(ctx context.Context, key string) ([]byte, string, error) {

+ if f.fail && key == f.key {

+ return nil, "", errors.New("transient: connection reset")

+ }

+ return f.Store.Get(ctx, key)

+// Fix round 1, Astra's finding 3: the first tick after the first wake makes

+// the directory meets a transient failure reading the app's bundle. The pass

+// returns nil, leaving the file unwritten; the host stays waited for, and the

+// next tick writes its config, with no config change and no hourly pass. A

+// bundle that keeps failing is retried on a backoff, not every tick.

+func TestConfigGenTickRetriesAHostAPassLeftUndone(t *testing.T) {

+ configDirsPending = pendingDirs{}

+ t.Cleanup(func() { configDirsPending = pendingDirs{} })

+ ctx := context.Background()

+ base := release.NewDirStore(t.TempDir())

+ s := &blipStore{Store: base, key: appconfig.Key("bab", "shop")}

+ reg := restartTestRegistry(t)

+ dd := t.TempDir()

+ privText, _, err := appconfig.GenerateKey()

+ if err != nil {

+ t.Fatal(err)

+ }

+ priv, err := appconfig.ParsePrivateKey(privText, "test")

+ if err != nil {

+ t.Fatal(err)

+ }

+ if err := appconfig.SetEnv(ctx, base, "bab", "shop", map[string]string{"STORE_BUCKET": "store-shop"}); err != nil {

+ t.Fatal(err)

+ }

+ host := "shop.example.com"

+ if err := reg.Upsert(registry.Route{Host: host, Kind: "instance", App: "shop", Account: "bab", Hibernate: true,

+ Status: "ready", DBPath: provision.NestedDBPath(dd, host), Socket: provision.NestedSocket(host)}); err != nil {

+ t.Fatal(err)

+ }

+ cfgPath := filepath.Join(provision.InstanceDir(dd, host), host+".config.env")

+ cache := newGenCache()

+ passes := 0

+ tick := func() {

+ t.Helper()

+ routes, err := reg.List()

+ if err != nil {

+ t.Fatal(err)

+ }

+ if _, due := configDirsPending.peek(); due {

+ passes++

+ }

+ configGenTick(ctx, routes, reg, s, priv, dd, nil, hostwrite.Config{}, cache)

+ }

+ if err := syncExecSerialized(ctx, reg, s, priv, dd, nil, hostwrite.Config{}); err != nil {

+ t.Fatal(err)

+ }

+ tick() // seeds the cache

+ if err := os.MkdirAll(provision.InstanceDir(dd, host), 0o700); err != nil {

+ t.Fatal(err)

+ }

+ s.fail = true

+ tick()

+ if _, err := os.Lstat(cfgPath); !os.IsNotExist(err) {

+ t.Fatalf("a bundle that did not read was written (err %v)", err)

+ }

+ if _, ok := configDirsPending.peek(); !ok {

+ t.Fatal("the pass that left the host undone dropped it")

+ }

+ s.fail = false

+ tick()

+ if b, err := os.ReadFile(cfgPath); err != nil || string(b) != "STORE_BUCKET=store-shop\n" {

+ t.Fatalf("the next tick's bundle = %q, %v", b, err)

+ }

+ if _, ok := configDirsPending.peek(); ok {

+ t.Fatal("the written host is still waited for")

+ }

+ // A bundle that keeps failing: retries back off (1, 2, 4 ... ticks apart).

+ if err := os.Remove(cfgPath); err != nil {

+ t.Fatal(err)

+ }

+ configDirsPending.record(map[string]string{host: provision.InstanceDir(dd, host)}, nil, false)

+ s.fail = true

+ passes = 0

+ for range 20 {

+ tick()

+ }

+ if passes == 0 || passes > 6 {

+ t.Fatalf("%d retries in 20 ticks of a failing bundle, want a backoff (1 to 6)", passes)

+ }

+// peek is appeared without the tick: whether a retry is due now.

+func (p *pendingDirs) peek() (string, bool) {

+ p.mu.Lock()

+ defer p.mu.Unlock()

+ for h, e := range p.dirs {

+ if _, err := os.Lstat(e.dir); err == nil && e.waitTicks == 0 {

+ return h, true

+ }

+ return "", false

diff --git a/internal/provision/plan14_root_reads_test.go b/internal/provision/plan14_root_reads_test.go

index 080727663..fdbf3fe99 100644

--- a/internal/provision/plan14_root_reads_test.go

+++ b/internal/provision/plan14_root_reads_test.go

@@ -996,7 +996,7 @@ func TestPlan14RootReadCensus(t *testing.T) {

if err = loadRootReadInventory(filepath.Join(root, "internal/provision/testdata/plan14-root-reads-moves-offline-fixes.tsv"), &inv, true); err != nil {

t.Fatal(err)

}

- seal = "d3b7eb063b214441776cc5381cf44dba91aabce4466e990e685f8fc29c26e9a5"

+ seal = "edc04f7349acf2b9ea98ffc8a1d3b71516a6ddb95505abfc2e38e49fd802ccc2"

errs := append(compareRootReads(got, inv), checkRootReadLegacy(inv, seal)...)

errs = append(errs, checkRootReadDeclarations(got, inv)...)

if len(errs) > 0 {

diff --git a/internal/provision/testdata/plan14-root-reads-moves-offline-fixes.tsv b/internal/provision/testdata/plan14-root-reads-moves-offline-fixes.tsv

index f59af42a9..45ff1f6b3 100644

--- a/internal/provision/testdata/plan14-root-reads-moves-offline-fixes.tsv

+++ b/internal/provision/testdata/plan14-root-reads-moves-offline-fixes.tsv

@@ -5,20 +5,21 @@

# Defect 8 (the source's scheduler wrote the moved host's env file after open): Runner.Tick and newScheduleDeps re-frozen at their dispositions, scheduleSkip new (J1), Runner.fire re-pinned at its class and task (hint, E2: overlay e's reclassification); internal/host and internal/schedule re-pinned. The seal moves.

# Fix round 1, finding 1 (Astra: skipped timers could starve other hosts): Runner.Tick re-pinned at its re-frozen disposition (its due query moved into dueRows); Runner.dueRows new, accepted legacy (J1, as Runner.Tick): held hosts excluded in the query and hosts Skip names paged past before the limit; internal/schedule re-pinned. The seal moves.

# Fix round 1, finding 2 (Astra: the report refresh extended the tombstone): envelope only. The refresh now calls movesWorker.offlineSealerRefresh, which reads the seal, the lease (offlineLock) and the source reports and writes only the sealer's report, never the lease; it has no root read of its own. internal/host re-pinned; the seal stands.

-@coverage internal/appconfig 2c32778cc777f026f1dd7b6404b11e3fabf1b6bd4912799d6ef26bfb59d42a07

+# Fix round 1, finding 3 (Astra: a skipped config read cleared the pending retry): SyncExecWithCallbacks re-pinned (it names each in-scope host done or not, WithHostOutcome), pendingDirs.appeared re-pinned (the same Lstat, now with a retry backoff); internal/appconfig and internal/host re-pinned. The seal moves.

+@coverage internal/appconfig d2258ea641530870dc3b7600cda01896b0d647229a3b70b83addcd30b15210aa

@coverage internal/console 7b015747c5ef8a3ee7473ad63f7f99200ce14c63c1b7a47945c212ad9239756b

-@coverage internal/host 9bdce80d912f31b185c3dfd74667bf8fa1caab32ac700b72d03d0a6dae45c28e

+@coverage internal/host 242917d80f343a8a2ca1536c5b1775a9bf8f445ce643cc22fde9fa55f12a627c

@coverage internal/schedule a76efa679b9089eff4f6dae8e6f0adc5d92de628e66e16d4a00c81ba70e086ff

-@legacy 783751049991bdd17a140bba6ce9a3049a4075b1170e4882c7207b20bed24b41 internal/appconfig/syncexec.go SyncExecWithCallbacks legacy accepted-legacy accepted-legacy;rev3=PROPOSED-moves-offline-fixes;basis=route-and-input (J1,J4, main's disposition); the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos), through syncExecSerialized: main's body, with a nested row whose write found no instance directory named and skipped (errInstanceDirAbsent, WhenInstanceDirAbsent) instead of failing the pass; no new read site file:Stat@28119,registry:App@10156,registry:App@10371,registry:App@15749,registry:App@28004,registry:Environment@10164,registry:Environment@10379,registry:Environment@15757,registry:Environment@28012,registry:Host@1031,registry:Host@1460,registry:Host@15584,registry:Host@15682,registry:Host@16514,registry:Host@16865,registry:Host@17800,registry:Host@18341,registry:Host@21268,registry:Host@21485,registry:Host@22043,registry:Host@22233,registry:Host@22242,registry:Host@22261,registry:Host@25162,registry:Host@25779,registry:Host@25831,registry:Host@28513,registry:Host@831,registry:Host@902,registry:Host@996,whole:Route@10049,whole:Route@10110,whole:Route@10151,whole:Route@10268,whole:Route@10302,whole:Route@10366,whole:Route@1412,whole:Route@1441,whole:Route@15193,whole:Route@15243,whole:Route@15744,whole:Route@24758,whole:Route@25172,whole:Route@25876,whole:Route@27903,whole:Route@27936,whole:Route@27999,whole:Route@28100,whole:Route@28750,whole:Route@28823,whole:Route@693,whole:Route@721,whole:Route@750,whole:Route@776,whole:Route@9984 8d58864244e2a823103fab97653d48b7d586843eee5fcb5848194447c2f0210f

-internal/appconfig/syncexec.go SyncExecWithCallbacks legacy accepted-legacy accepted-legacy;rev3=PROPOSED-moves-offline-fixes;basis=route-and-input (J1,J4, main's disposition); the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos), through syncExecSerialized: main's body, with a nested row whose write found no instance directory named and skipped (errInstanceDirAbsent, WhenInstanceDirAbsent) instead of failing the pass; no new read site file:Stat@28119,registry:App@10156,registry:App@10371,registry:App@15749,registry:App@28004,registry:Environment@10164,registry:Environment@10379,registry:Environment@15757,registry:Environment@28012,registry:Host@1031,registry:Host@1460,registry:Host@15584,registry:Host@15682,registry:Host@16514,registry:Host@16865,registry:Host@17800,registry:Host@18341,registry:Host@21268,registry:Host@21485,registry:Host@22043,registry:Host@22233,registry:Host@22242,registry:Host@22261,registry:Host@25162,registry:Host@25779,registry:Host@25831,registry:Host@28513,registry:Host@831,registry:Host@902,registry:Host@996,whole:Route@10049,whole:Route@10110,whole:Route@10151,whole:Route@10268,whole:Route@10302,whole:Route@10366,whole:Route@1412,whole:Route@1441,whole:Route@15193,whole:Route@15243,whole:Route@15744,whole:Route@24758,whole:Route@25172,whole:Route@25876,whole:Route@27903,whole:Route@27936,whole:Route@27999,whole:Route@28100,whole:Route@28750,whole:Route@28823,whole:Route@693,whole:Route@721,whole:Route@750,whole:Route@776,whole:Route@9984 8d58864244e2a823103fab97653d48b7d586843eee5fcb5848194447c2f0210f

+@legacy 8941e875e49f6b5d23c8153f793ac36c78f20776ff9e064534302e19ca7ca62b internal/appconfig/syncexec.go SyncExecWithCallbacks legacy accepted-legacy accepted-legacy;rev3=PROPOSED-moves-offline-fixes;basis=route-and-input (J1,J4, main's disposition); the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos), through syncExecSerialized: main's body, with a nested row whose write found no instance directory named and skipped (errInstanceDirAbsent, WhenInstanceDirAbsent) instead of failing the pass, and each in-scope host reported done or not done (WithHostOutcome) once the pass has gone through every route; no new read site file:Stat@28417,registry:App@10156,registry:App@10371,registry:App@15917,registry:App@28302,registry:Environment@10164,registry:Environment@10379,registry:Environment@15925,registry:Environment@28310,registry:Host@1031,registry:Host@1460,registry:Host@15425,registry:Host@15752,registry:Host@15850,registry:Host@16682,registry:Host@17033,registry:Host@17968,registry:Host@18509,registry:Host@21436,registry:Host@21653,registry:Host@22211,registry:Host@22401,registry:Host@22410,registry:Host@22429,registry:Host@25330,registry:Host@25947,registry:Host@25999,registry:Host@26155,registry:Host@28811,registry:Host@831,registry:Host@902,registry:Host@996,whole:Route@10049,whole:Route@10110,whole:Route@10151,whole:Route@10268,whole:Route@10302,whole:Route@10366,whole:Route@1412,whole:Route@1441,whole:Route@15337,whole:Route@15387,whole:Route@15912,whole:Route@24926,whole:Route@25340,whole:Route@26044,whole:Route@28201,whole:Route@28234,whole:Route@28297,whole:Route@28398,whole:Route@29048,whole:Route@29121,whole:Route@693,whole:Route@721,whole:Route@750,whole:Route@776,whole:Route@9984 b523044621e10b4a702d5e1b8c5ecd843b645cf72b327487ed424f96956a3a7e

+internal/appconfig/syncexec.go SyncExecWithCallbacks legacy accepted-legacy accepted-legacy;rev3=PROPOSED-moves-offline-fixes;basis=route-and-input (J1,J4, main's disposition); the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos), through syncExecSerialized: main's body, with a nested row whose write found no instance directory named and skipped (errInstanceDirAbsent, WhenInstanceDirAbsent) instead of failing the pass, and each in-scope host reported done or not done (WithHostOutcome) once the pass has gone through every route; no new read site file:Stat@28417,registry:App@10156,registry:App@10371,registry:App@15917,registry:App@28302,registry:Environment@10164,registry:Environment@10379,registry:Environment@15925,registry:Environment@28310,registry:Host@1031,registry:Host@1460,registry:Host@15425,registry:Host@15752,registry:Host@15850,registry:Host@16682,registry:Host@17033,registry:Host@17968,registry:Host@18509,registry:Host@21436,registry:Host@21653,registry:Host@22211,registry:Host@22401,registry:Host@22410,registry:Host@22429,registry:Host@25330,registry:Host@25947,registry:Host@25999,registry:Host@26155,registry:Host@28811,registry:Host@831,registry:Host@902,registry:Host@996,whole:Route@10049,whole:Route@10110,whole:Route@10151,whole:Route@10268,whole:Route@10302,whole:Route@10366,whole:Route@1412,whole:Route@1441,whole:Route@15337,whole:Route@15387,whole:Route@15912,whole:Route@24926,whole:Route@25340,whole:Route@26044,whole:Route@28201,whole:Route@28234,whole:Route@28297,whole:Route@28398,whole:Route@29048,whole:Route@29121,whole:Route@693,whole:Route@721,whole:Route@750,whole:Route@776,whole:Route@9984 b523044621e10b4a702d5e1b8c5ecd843b645cf72b327487ed424f96956a3a7e

@legacy 356c576628a285021657e989a015ef938bb1da3498d8146a734903a23fea1e4a internal/appconfig/syncexec.go renderExecConfigBefore legacy accepted-legacy accepted-legacy;rev3=PROPOSED-moves-offline-fixes;basis=route-and-input (J1,J4, main's disposition); the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos), from config sync and the moves worker's held render: main's body with one Lstat of a nested row's instance directory after its write failed, to tell a directory not made yet from any other failure; it follows no link and changes no outcome but the error's kind file:Lstat@3893,file:ReadFile@951,registry:Environment@3190,registry:Host@1070,registry:Host@2586,registry:Host@4088,registry:Host@4320,registry:Host@4576,registry:Host@4669,registry:Host@876,whole:Route@2522,whole:Route@257,whole:Route@3715,whole:Route@680 a60a9a2e2158f3a8dadf6fcf46810930dad89b67d1acc23fb57c324a7d9ac1f1

internal/appconfig/syncexec.go renderExecConfigBefore legacy accepted-legacy accepted-legacy;rev3=PROPOSED-moves-offline-fixes;basis=route-and-input (J1,J4, main's disposition); the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos), from config sync and the moves worker's held render: main's body with one Lstat of a nested row's instance directory after its write failed, to tell a directory not made yet from any other failure; it follows no link and changes no outcome but the error's kind file:Lstat@3893,file:ReadFile@951,registry:Environment@3190,registry:Host@1070,registry:Host@2586,registry:Host@4088,registry:Host@4320,registry:Host@4576,registry:Host@4669,registry:Host@876,whole:Route@2522,whole:Route@257,whole:Route@3715,whole:Route@680 a60a9a2e2158f3a8dadf6fcf46810930dad89b67d1acc23fb57c324a7d9ac1f1

@legacy 547dba02dd41e9e318da6368d9bf493f94152a4d5e56737765199cc2f298a302 internal/host/agent.go configGenLoop legacy accepted-legacy accepted-legacy;rev3=PROPOSED-moves-offline-fixes;basis=root-route (J1, main's disposition); the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos): main's loop, its tick's body moved into configGenTick; the registry listing stays here whole:Route@1265 a0b36186f7c8abdd818c81c2651a75b202078ad7f197d93b6157bcab97604188

internal/host/agent.go configGenLoop legacy accepted-legacy accepted-legacy;rev3=PROPOSED-moves-offline-fixes;basis=root-route (J1, main's disposition); the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos): main's loop, its tick's body moved into configGenTick; the registry listing stays here whole:Route@1265 a0b36186f7c8abdd818c81c2651a75b202078ad7f197d93b6157bcab97604188

@legacy 2bf1eb09b85c6c8fa62feb2173628e757913e153a656247a3423ef9a753783b0 internal/host/agent.go configGenTick legacy accepted-legacy accepted-legacy;rev3=PROPOSED-moves-offline-fixes;basis=root-route (J1, as configGenLoop); the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos): new, configGenLoop's tick over the routes it listed; it also runs the agent's own config pass once an instance directory the last pass waited for exists whole:Route@268 751870d636c3ea7b3670a7e00c71d085e94ae9b013545e319047a4609f514943

internal/host/agent.go configGenTick legacy accepted-legacy accepted-legacy;rev3=PROPOSED-moves-offline-fixes;basis=root-route (J1, as configGenLoop); the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos): new, configGenLoop's tick over the routes it listed; it also runs the agent's own config pass once an instance directory the last pass waited for exists whole:Route@268 751870d636c3ea7b3670a7e00c71d085e94ae9b013545e319047a4609f514943

-@legacy 237af935c475d587a83b4e7a33d21e1f3fd32f36333a8440b9f3b44b21b8509e internal/host/agent.go pendingDirs.appeared legacy accepted-legacy accepted-legacy;rev3=PROPOSED-moves-offline-fixes;basis=root-input (J4, as the moves units' isDir and lexists); the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos): new, an Lstat that never follows a link, of the instance directories the last config pass could not write into; it decides only whether the agent runs its own config pass again file:Lstat@137 bc87759cf499661d1058712b03230924e1669a8a1710b88e4053c8b4f7bd3aa3

-internal/host/agent.go pendingDirs.appeared legacy accepted-legacy accepted-legacy;rev3=PROPOSED-moves-offline-fixes;basis=root-input (J4, as the moves units' isDir and lexists); the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos): new, an Lstat that never follows a link, of the instance directories the last config pass could not write into; it decides only whether the agent runs its own config pass again file:Lstat@137 bc87759cf499661d1058712b03230924e1669a8a1710b88e4053c8b4f7bd3aa3

+@legacy 681979f76e87cb6dcd82e4b0843e0a720382366629ccc1d46d393b3ed78746c4 internal/host/agent.go pendingDirs.appeared legacy accepted-legacy accepted-legacy;rev3=PROPOSED-moves-offline-fixes;basis=root-input (J4, as the moves units' isDir and lexists); the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos): new, an Lstat that never follows a link, of the instance directories config passes could not write into, with a retry backoff for a host a pass left undone; it decides only whether the agent runs its own config pass again file:Lstat@135 691589305b19ebe4e7311bb853ae68003676786ce0684e6e7133382d26c982c7

+internal/host/agent.go pendingDirs.appeared legacy accepted-legacy accepted-legacy;rev3=PROPOSED-moves-offline-fixes;basis=root-input (J4, as the moves units' isDir and lexists); the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos): new, an Lstat that never follows a link, of the instance directories config passes could not write into, with a retry backoff for a host a pass left undone; it decides only whether the agent runs its own config pass again file:Lstat@135 691589305b19ebe4e7311bb853ae68003676786ce0684e6e7133382d26c982c7

@legacy fb3a335703bd3d6008ff82252d64aa9f41676650a874eb786291d1d201e57910 internal/host/schedule_wiring.go newScheduleDeps legacy accepted-legacy accepted-legacy;rev3=PROPOSED-moves-offline-fixes;basis=root-route (J1, main's disposition); the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos), from armSchedule: main's body with the runner's Skip set to scheduleSkip; no new read site registry:App@1479,registry:App@388,registry:Host@1546,registry:Host@396,whole:Route@1474,whole:Route@383 aa93bed2781d1005cf864ebaaf6ac4597d6f547c45d6a1063f10c4a483ba46c3

internal/host/schedule_wiring.go newScheduleDeps legacy accepted-legacy accepted-legacy;rev3=PROPOSED-moves-offline-fixes;basis=root-route (J1, main's disposition); the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos), from armSchedule: main's body with the runner's Skip set to scheduleSkip; no new read site registry:App@1479,registry:App@388,registry:Host@1546,registry:Host@396,whole:Route@1474,whole:Route@383 aa93bed2781d1005cf864ebaaf6ac4597d6f547c45d6a1063f10c4a483ba46c3

@legacy 48af22e2b896b940598175e9c4988e2837b14a72e88b7ee1ba8346098f209606 internal/host/schedule_wiring.go scheduleSkip legacy accepted-legacy accepted-legacy;rev3=PROPOSED-moves-offline-fixes;basis=root-route (J1, as the runner's own hold checks); the agent (/opt/carlos/carlos agent, carlos-edge.service, user carlos), from its schedule runner: new, reads a row's status (any hold) and host (the armed offline view: the record names another region) to tell the runner to leave the host alone; it only withholds a tick, never grants one registry:Host@155,registry:Status@64 4f471727429b5b79bfa909693a09e588051b9f490eafd8480ee9893581362d64