amadan
host: grant-drift converge in RunAdopt, three advisories, and FleetDBs learns the app's databases
RunAdopt gains spec §4d's step 2, between config sync and release adoption: capabilityPass converges every adoptable unit route from the STAGED SIBLING MANIFEST OF ITS LIVE VERSION. adoptHost's converge is what a VERSION change moves through; this is what a GRANT change moves through — and the repair path for a drop-in deleted by hand, a socket unit nobody enabled, a daemon that crashed into a stopped state, a first adoption whose restart failed (there was no previous version to roll back to, so its drop-ins stand and this pass re-verifies them), and any shape stranded mid-write. Both entry points call the same caps.Converge and running both in one pass is a no-op the second time: same bytes, no write, no reload, no restart. DeferRestart stays FALSE here, per caps.Plan's contract: restarting the live binary to pick up a new unit shape is the entire point of this pass, and nothing here follows up with a caps.Restart. The three states of a manifest read stay distinct. An absent sibling — never staged, or a write that failed — is leave-alone with one line, never a removal: a bucket hiccup must not strip :25 off a running mail server. This pass deliberately does not fetch; adoptHost's back-fill is what heals the sibling, and this picks it up next hour. A sibling that exists but does not parse gets no plan and a line naming it. A manifest that RESOLVED and declares nothing carries a non-nil empty Set, so a capability dropped from a manifest is actually removed. An unparseable grants column refuses that route's converge rather than defaulting to no grants — "no grants" is a real instruction that drops :25. buildPlans is the pure half, so every word an operator reads in the hourly log is table-testable without disk, systemd or network. Two of the three advisories fall out of it (declared-but-ungranted, naming the exact `carlos route -grant` command; capabilities on an exec-backed route, the guided path through the §5 migration). The third needs the box's firewall and passwd database and lives with the other probing advisory: a host_role grant whose carlos-<app> uid is absent from the IMDS nftables gate. The platform PRINTS that rule and never writes it — the firewall is operator territory — and "the uid is missing" and "I could not read the gate" are two different claims with two different lines. parseNftSkuids is scoped to rules naming 169.254.169.254, because counting a box's unrelated skuid rules would report a gate that allows uids it does not: the fail-open direction, on the exact starvation F1 was silent about. ops.FleetDBs finally learns a unit tenant's database instead of depending on an operator having typed -db at `carlos add`: data.replicate="platform" with dbs stamps DBPath = <state dir>/<dbs[0] sorted>, only when it differs, with the extras named rather than quietly dropped (DBPath is single-valued). An app that replicates its own WAL NEVER has its DBPath cleared — two replicators on one WAL corrupt the replica, and which one stops is an operator decision, so it is advised. release.capabilityRoute becomes exported CapabilityRoute: the two converge entry points must agree to the character on which routes may have root-owned unit configuration written for them, and a second copy of that predicate is exactly the drift that would let one pass materialize what the other refuses. An empty route list is never authority to do anything (appconfig.Sync's guard, same reason), and a per-route refusal logs and continues — one misconfigured route must not stop the rest of the box's drift being repaired.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
6 files changed,
+1213
−4
internal/host/adopt_advice.go+161 −0internal/host/adopt_advice_test.go+138 −0internal/host/agent.go+27 −0internal/host/caps_pass.go+293 −0internal/host/caps_pass_test.go+584 −0internal/release/adopt.go+10 −4
diff --git a/internal/host/adopt_advice.go b/internal/host/adopt_advice.go| index 5357713c9..6c3d9d090 100644 |
| --- a/internal/host/adopt_advice.go |
| +++ b/internal/host/adopt_advice.go |
| @@ -5,11 +5,16 @@ |
| package host |
| import ( |
| + "bytes" |
| "log" |
| "os/exec" |
| + "os/user" |
| + "strconv" |
| "strings" |
| + "github.com/carlosframework/platform/internal/caps" |
| "github.com/carlosframework/platform/internal/registry" |
| + "github.com/carlosframework/platform/internal/release" |
| ) |
| // warnBackingMismatch reports hosts whose route claims exec backing |
| @@ -82,3 +87,159 @@ func adviseBackingMismatches(routes []registry.Route) { |
| log.Printf("adopt: route %s claims exec backing but carlos-app@%s is ACTIVE — stamp backing='unit' (see platform issue #14), then remove the stray /data/%s.config.env the exec loop materialized — the unit reads /etc/carlos/apps instead", h, h, h) |
| } |
| } |
| + |
| +// imdsAddr is the link-local metadata address every cloud instance role is |
| +// reached through, and the thing the box's nftables gate is keyed on. A rule |
| +// that does not name it is not part of that gate (a box can have plenty of |
| +// other skuid rules), so it is the scope of the parse below as well as a |
| +// string in the printed command. |
| +const imdsAddr = "169.254.169.254" |
| + |
| +// adviseHostRoleGateMissing: the app holds the host_role grant, so it runs as |
| +// the static user carlos-<app> — but that uid is not in this box's IMDS |
| +// nftables gate, and the observed failure mode of that gap is SILENCE, not an |
| +// error (keymail's rehearsal finding F1: the restore came up empty and |
| +// served). So it is named every pass for as long as the gap exists. |
| +// |
| +// The platform NEVER writes the firewall rule. The firewall is operator |
| +// territory; this prints the exact command and stops there (spec §4b, §5a). |
| +// |
| +// Two seams, two distinct claims, deliberately not folded together: |
| +// |
| +// - uidOf answers whether the static user EXISTS on this box at all. It is |
| +// also the quiet gate on the whole advisory: converge creates that user |
| +// when it first materializes host_role, so a user that does not resolve |
| +// means nothing is running under it yet and there is nothing for the gate |
| +// to be missing. Advising then would be an hourly line about a uid that |
| +// does not exist. |
| +// - gateUIDs answers (uids, ok). ok == false is "I could not read a gate", |
| +// which is NOT the same claim as "the uid is absent from the gate" and |
| +// gets its own line, because acting on the wrong one of those two sends |
| +// an operator to edit a firewall that is fine. |
| +// |
| +// A route whose grants column does not parse is skipped in silence here: |
| +// buildPlans already says so once, naming the row and the fix, and repeating |
| +// it in a second voice would double every line of an already-broken row. |
| +func adviseHostRoleGateMissing(routes []registry.Route, uidOf func(user string) (int, bool), gateUIDs func() (map[int]bool, bool)) { |
| + var cands []registry.Route |
| + for _, rt := range routes { |
| + // Only a route that can actually materialize host_role: the same |
| + // predicate the converge passes use. A grant on a route that cannot |
| + // materialize is `carlos route -grant`'s problem (grantRail refuses |
| + // it), not the firewall's. |
| + if !release.CapabilityRoute(rt) { |
| + continue |
| + } |
| + g, err := caps.ParseGrants(rt.Caps) |
| + if err != nil || !g.HostRole { |
| + continue |
| + } |
| + cands = append(cands, rt) |
| + } |
| + if len(cands) == 0 { |
| + return // no candidates, no `nft` invocation at all |
| + } |
| + uids, ok := gateUIDs() // ONE read of the ruleset for the whole pass |
| + for _, rt := range cands { |
| + u := caps.AppUser(rt.App) |
| + uid, exists := uidOf(u) |
| + if !exists { |
| + continue |
| + } |
| + if !ok { |
| + log.Printf("adopt: %s holds the host_role grant on %s but this box's IMDS gate could not be read (nft absent, unreadable, or carrying no rule naming %s) — confirm by hand that the uid is allowed (see the app-capabilities design §4b)", u, rt.Host, imdsAddr) |
| + continue |
| + } |
| + if !uids[uid] { |
| + log.Printf("adopt: %s (uid %d) holds the host_role grant on %s but is not in this box's IMDS gate — its AWS clients will be starved SILENTLY; add it: nft add rule inet filter output ip daddr %s skuid %d accept (and add the uid to harden-host.sh so a re-run does not strip it)", u, uid, rt.Host, imdsAddr, uid) |
| + } |
| + } |
| +} |
| + |
| +// lookupUID resolves a system user to its numeric uid — the same account the |
| +// converge pass creates for a host_role app, looked up the same way |
| +// caps.Ops.ensureUser proves its own useradd resolved. |
| +func lookupUID(name string) (int, bool) { |
| + u, err := user.Lookup(name) |
| + if err != nil { |
| + return 0, false |
| + } |
| + uid, err := strconv.Atoi(u.Uid) |
| + if err != nil { |
| + return 0, false |
| + } |
| + return uid, true |
| +} |
| + |
| +// nftGateUIDs is the real read of the box's IMDS gate: the OS glue half of |
| +// the parseNftSkuids split, exactly as unitsAreActive is to parseIsActiveBatch. |
| +// |
| +// `-n` is load-bearing, not tidiness: it makes nft print uids numerically, so |
| +// the parse never has to resolve a printed account NAME back to a number and |
| +// can never mistake one for the other. |
| +// |
| +// ok is false when nft cannot run (absent binary, not root, no ruleset) AND |
| +// when the ruleset carries no rule naming the metadata address at all — a box |
| +// with no IMDS gate is one this advisory cannot make the "uid is missing" |
| +// claim about, and saying "could not read a gate" of it is the honest answer |
| +// rather than telling an operator to add a rule to a firewall that gates |
| +// nothing. |
| +func nftGateUIDs() (map[int]bool, bool) { |
| + out, err := exec.Command("nft", "-n", "list", "ruleset").Output() |
| + if err != nil || !bytes.Contains(out, []byte(imdsAddr)) { |
| + return nil, false |
| + } |
| + return parseNftSkuids(out), true |
| +} |
| + |
| +// parseNftSkuids reads the uids allowed to reach the metadata address out of |
| +// `nft -n list ruleset` output. |
| +// |
| +// Scoped to lines naming imdsAddr on purpose: a box has other skuid rules |
| +// (SSH, a backup agent), and counting those would report a gate that allows |
| +// uids it does not allow — the fail-OPEN direction, where this advisory goes |
| +// quiet about a starvation it should be shouting about. |
| +// |
| +// Both spellings nft prints are read: `skuid 998` and the set form |
| +// `skuid { 998, 999 }`. Negated and range matches (`skuid != 0`, |
| +// `skuid 100-200`) are deliberately not interpreted — neither is an |
| +// allowlist of one uid, and guessing at one would be the fail-open direction |
| +// again; an unparsed rule simply leaves the uid looking absent, which at |
| +// worst produces one advisory line an operator can dismiss. |
| +func parseNftSkuids(out []byte) map[int]bool { |
| + uids := map[int]bool{} |
| + for _, line := range strings.Split(string(out), "\n") { |
| + if !strings.Contains(line, imdsAddr) { |
| + continue |
| + } |
| + fields := strings.Fields(line) |
| + for i := 0; i < len(fields); i++ { |
| + if fields[i] != "skuid" || i+1 >= len(fields) { |
| + continue |
| + } |
| + if fields[i+1] == "{" { |
| + j := i + 2 |
| + for ; j < len(fields) && fields[j] != "}"; j++ { |
| + addUID(uids, fields[j]) |
| + } |
| + i = j |
| + continue |
| + } |
| + addUID(uids, fields[i+1]) |
| + i++ |
| + } |
| + } |
| + return uids |
| +} |
| + |
| +// addUID records one uid token, tolerating the punctuation nft's set syntax |
| +// carries ("998," inside braces) and ignoring anything that is not a plain |
| +// non-negative number. |
| +func addUID(uids map[int]bool, tok string) { |
| + tok = strings.Trim(tok, `",;{}`) |
| + n, err := strconv.Atoi(tok) |
| + if err != nil || n < 0 { |
| + return |
| + } |
| + uids[n] = true |
| +} |
diff --git a/internal/host/adopt_advice_test.go b/internal/host/adopt_advice_test.go| index 9caadd2da..fc5faebd3 100644 |
| --- a/internal/host/adopt_advice_test.go |
| +++ b/internal/host/adopt_advice_test.go |
| @@ -2,6 +2,7 @@ package host |
| import ( |
| "reflect" |
| + "strings" |
| "testing" |
| "github.com/carlosframework/platform/internal/registry" |
| @@ -84,3 +85,140 @@ func TestParseIsActiveBatch(t *testing.T) { |
| }) |
| } |
| } |
| + |
| +// ---- the IMDS gate advisory ------------------------------------------------ |
| + |
| +// parseNftSkuids reads the uids allowed to reach the metadata address. It is |
| +// scoped to rules that NAME that address: a box's other skuid rules (SSH, a |
| +// backup agent) are not the IMDS gate, and counting them would report a gate |
| +// that allows uids it does not — the fail-open direction, where this advisory |
| +// goes quiet about exactly the silent starvation it exists to shout about. |
| +func TestParseNftSkuids(t *testing.T) { |
| + realistic := `table inet filter { |
| + chain output { |
| + type filter hook output priority filter; policy accept; |
| + ip daddr 169.254.169.254 skuid 0 accept |
| + ip daddr 169.254.169.254 skuid 998 accept |
| + ip daddr 169.254.169.254 counter drop |
| + tcp dport 22 skuid 1000 accept |
| + } |
| +}` |
| + cases := []struct { |
| + name string |
| + out string |
| + want map[int]bool |
| + }{ |
| + {"two IMDS rules, and an unrelated skuid rule ignored", realistic, map[int]bool{0: true, 998: true}}, |
| + {"set form", "ip daddr 169.254.169.254 skuid { 998, 999 } accept\n", map[int]bool{998: true, 999: true}}, |
| + {"no IMDS rule at all", "tcp dport 22 skuid 1000 accept\n", map[int]bool{}}, |
| + {"garbage", "not a ruleset at all", map[int]bool{}}, |
| + {"empty", "", map[int]bool{}}, |
| + {"skuid with no value", "ip daddr 169.254.169.254 skuid\n", map[int]bool{}}, |
| + {"negated match is not an allowlist", "ip daddr 169.254.169.254 skuid != 0 drop\n", map[int]bool{}}, |
| + } |
| + for _, c := range cases { |
| + t.Run(c.name, func(t *testing.T) { |
| + got := parseNftSkuids([]byte(c.out)) |
| + if !reflect.DeepEqual(got, c.want) { |
| + t.Errorf("parseNftSkuids() = %v, want %v", got, c.want) |
| + } |
| + }) |
| + } |
| +} |
| + |
| +// "the uid is missing from the gate" and "I could not read the gate" are |
| +// different claims, and acting on the wrong one sends an operator to edit a |
| +// firewall that is fine. Each gets its own line, and only the first prints a |
| +// command — the platform never writes the rule, it prints it (spec §4b). |
| +func TestAdviseHostRoleGateMissingSaysWhichClaimItIsMaking(t *testing.T) { |
| + granted := registry.Route{Host: "keymail.dev", Kind: "instance", App: "keymail", Backing: registry.BackingUnit, Status: "ready", Caps: "host-role,port:25"} |
| + uidOf := func(string) (int, bool) { return 998, true } |
| + |
| + t.Run("uid absent from a readable gate", func(t *testing.T) { |
| + logged := captureLog(t) |
| + adviseHostRoleGateMissing([]registry.Route{granted}, uidOf, |
| + func() (map[int]bool, bool) { return map[int]bool{0: true}, true }) |
| + out := logged() |
| + for _, w := range []string{"carlos-keymail (uid 998)", "keymail.dev", "starved SILENTLY", |
| + "nft add rule inet filter output ip daddr 169.254.169.254 skuid 998 accept", "harden-host.sh"} { |
| + if !strings.Contains(out, w) { |
| + t.Errorf("log = %q, want it to contain %q", out, w) |
| + } |
| + } |
| + }) |
| + |
| + t.Run("gate unreadable", func(t *testing.T) { |
| + logged := captureLog(t) |
| + adviseHostRoleGateMissing([]registry.Route{granted}, uidOf, |
| + func() (map[int]bool, bool) { return nil, false }) |
| + out := logged() |
| + if !strings.Contains(out, "could not be read") { |
| + t.Errorf("log = %q, want the unreadable-gate claim", out) |
| + } |
| + if strings.Contains(out, "nft add rule") { |
| + t.Errorf("log = %q, want NO firewall command when the gate could not be read", out) |
| + } |
| + }) |
| + |
| + t.Run("uid present in the gate says nothing", func(t *testing.T) { |
| + logged := captureLog(t) |
| + adviseHostRoleGateMissing([]registry.Route{granted}, uidOf, |
| + func() (map[int]bool, bool) { return map[int]bool{998: true}, true }) |
| + if out := logged(); out != "" { |
| + t.Errorf("log = %q, want silence on a correctly gated box", out) |
| + } |
| + }) |
| +} |
| + |
| +// The advisory is quiet where it would be wrong or useless: a route with no |
| +// host_role grant, a route that cannot materialize capabilities at all, and a |
| +// user that does not exist yet (converge creates it when host_role first |
| +// materializes, so nothing is running under that uid to be starved). None of |
| +// those may even reach for `nft`. |
| +func TestAdviseHostRoleGateMissingIsQuietWithNothingToSay(t *testing.T) { |
| + cases := []struct { |
| + name string |
| + routes []registry.Route |
| + uidOf func(string) (int, bool) |
| + }{ |
| + { |
| + name: "no host_role grant", |
| + routes: []registry.Route{{Host: "a", Kind: "instance", App: "keymail", Backing: registry.BackingUnit, Status: "ready", Caps: "port:25"}}, |
| + }, |
| + { |
| + name: "exec-backed route cannot materialize host_role", |
| + routes: []registry.Route{{Host: "b", Kind: "instance", App: "keymail", Status: "ready", Caps: "host-role"}}, |
| + }, |
| + { |
| + name: "unparseable grants column is buildPlans' line to print, not this one's", |
| + routes: []registry.Route{{Host: "c", Kind: "instance", App: "keymail", Backing: registry.BackingUnit, Status: "ready", Caps: "port:025"}}, |
| + }, |
| + { |
| + name: "the static user does not exist yet", |
| + routes: []registry.Route{{Host: "d", Kind: "instance", App: "keymail", Backing: registry.BackingUnit, Status: "ready", Caps: "host-role"}}, |
| + uidOf: func(string) (int, bool) { return 0, false }, |
| + }, |
| + } |
| + for _, c := range cases { |
| + t.Run(c.name, func(t *testing.T) { |
| + logged := captureLog(t) |
| + uidOf := c.uidOf |
| + if uidOf == nil { |
| + uidOf = func(string) (int, bool) { |
| + t.Error("looked a user up with no candidate route") |
| + return 0, false |
| + } |
| + } |
| + gate := func() (map[int]bool, bool) { |
| + if c.uidOf == nil { |
| + t.Error("read the box's firewall with no candidate route") |
| + } |
| + return nil, true |
| + } |
| + adviseHostRoleGateMissing(c.routes, uidOf, gate) |
| + if out := logged(); out != "" { |
| + t.Errorf("log = %q, want silence", out) |
| + } |
| + }) |
| + } |
| +} |
diff --git a/internal/host/agent.go b/internal/host/agent.go| index 13cf0c62d..b584ec5d0 100644 |
| --- a/internal/host/agent.go |
| +++ b/internal/host/agent.go |
| @@ -793,6 +793,26 @@ func RunAdopt(args []string) { |
| } |
| } |
| + // Step 2 (spec §4d): the grant-drift capability converge, BETWEEN config |
| + // sync and release adoption. Before release.Pass because it repairs the |
| + // shape of what is ALREADY live — a revoked grant, a deleted drop-in, a |
| + // socket unit nobody enabled — and a route that release.Pass then moves |
| + // to a new version converges again inside adoptHost, which is a no-op |
| + // when nothing about the shape changed: same bytes, no write, no reload, |
| + // no restart. |
| + // |
| + // The zero caps.Ops is the production one (real /etc/systemd/system, real |
| + // systemctl, real useradd), for the same reason WithCapabilities below is |
| + // only ever set here: this verb is root out of carlos-adopt.service, and |
| + // the in-process loops in this file cannot write unit configuration at |
| + // all. |
| + // |
| + // Never fatal. A failure here is the registry not listing; adoption of |
| + // releases is a separate job and must still run. |
| + if err := capabilityPass(reg, release.OptDir(), caps.Ops{}); err != nil { |
| + log.Printf("adopt: capability pass: %v", err) |
| + } |
| + |
| // NagExpiredCanaries(os.Stdout) is set HERE ONLY (2026-08-03 review) — the |
| // canary-TTL nag (spec §5) belongs to the hourly root pass |
| // (carlos-adopt.service, this verb's ExecStart), the same split PR #9's |
| @@ -812,7 +832,14 @@ func RunAdopt(args []string) { |
| log.Fatalf("adopt: %v", err) |
| } |
| + // Step 4: the advisories. The two capability advisories that fall out of |
| + // READING manifests (declared-but-ungranted, and capabilities on an |
| + // exec-backed route) are printed by capabilityPass above, where the |
| + // manifests are already in hand; this one needs the box's firewall and |
| + // its passwd database instead, so it lives here with the other |
| + // systemd-probing advisory. |
| if routes, err := reg.List(); err == nil { |
| adviseBackingMismatches(routes) |
| + adviseHostRoleGateMissing(routes, lookupUID, nftGateUIDs) |
| } |
| } |
diff --git a/internal/host/caps_pass.go b/internal/host/caps_pass.go| new file mode 100644 |
| index 000000000..3ce63f461 |
| --- /dev/null |
| +++ b/internal/host/caps_pass.go |
| @@ -0,0 +1,293 @@ |
| +// The grant-drift capability pass: RunAdopt's step 2 (spec §4d). |
| +// |
| +// This is the survey half of the capability design. adoptHost's converge |
| +// (release.WithCapabilities) is what a VERSION change moves through; this is |
| +// what a GRANT change moves through — and, because it converges every |
| +// adoptable unit route from the staged manifest of the version that is |
| +// ACTUALLY LIVE, it is also the repair path for everything else that can |
| +// leave a box's unit shape disagreeing with what the platform believes: |
| +// |
| +// - a grant added or revoked with `carlos route -grant/-revoke`; |
| +// - a drop-in deleted by hand, or lost with an /etc restore; |
| +// - a socket unit that is not enabled, or a daemon that is not running; |
| +// - a FIRST adoption whose restart failed: the symlink rolled back to |
| +// nothing (there was no previous version), so the drop-ins this pass |
| +// re-verifies are the live version's and stand as converged; |
| +// - any stranded shape left by a pass that died between two writes. |
| +// |
| +// Both entry points call the same caps.Converge, and running both in one |
| +// pass is idempotent: the second sees byte-identical content, writes nothing, |
| +// reloads nothing and restarts nothing. |
| +package host |
| + |
| +import ( |
| + "fmt" |
| + "log" |
| + "path/filepath" |
| + "sort" |
| + |
| + "github.com/carlosframework/platform/internal/caps" |
| + "github.com/carlosframework/platform/internal/registry" |
| + "github.com/carlosframework/platform/internal/release" |
| +) |
| + |
| +// capabilityPass converges every adoptable unit route on this box and stamps |
| +// what it learned about their databases onto the registry. |
| +// |
| +// Pure enough to test: the registry, the opt dir and the caps.Ops seams are |
| +// all arguments, so its tests need no systemd, no root and no /etc. |
| +// |
| +// Its error return is for FOUNDATION failures only (the registry would not |
| +// list). A per-route refusal — an unreadable manifest, an unparseable grants |
| +// column, a port another route already claims — logs and continues: this pass |
| +// surveys a whole box, and one misconfigured route must not stop every other |
| +// route's drift from being repaired. |
| +func capabilityPass(reg *registry.Registry, optDir string, o caps.Ops) error { |
| + routes, err := reg.List() |
| + if err != nil { |
| + return err |
| + } |
| + // An empty route list is never authority to do anything, exactly as in |
| + // appconfig.Sync: registry.Open silently creates an empty database when |
| + // handed a wrong path or an unmounted /data, and a pass that believed it |
| + // would see every route on the box as removed. Nothing here deletes on |
| + // absence today — a drop-in is only ever removed for a route this pass |
| + // still sees — but the guard is cheap and keeps that true by construction. |
| + if len(routes) == 0 { |
| + return nil |
| + } |
| + |
| + plans, advice := buildPlans(routes, stagedManifestOfLiveVersion(optDir)) |
| + for _, line := range advice { |
| + log.Print(line) |
| + } |
| + |
| + // Box-wide exclusivity across every plan at once, BEFORE anything is |
| + // written: two routes granted the same privileged port, or the same |
| + // runtime_dir. Converge repeats the check against what is on DISK; this |
| + // one is the only half that can see the other plans. |
| + refused := caps.CheckClaims(plans) |
| + for _, p := range plans { |
| + if err := refused[p.Host]; err != nil { |
| + log.Printf("adopt: %s: %v — nothing written for it this pass", p.Host, err) |
| + continue |
| + } |
| + // Converge has already logged both a refusal's operator fix and a |
| + // failure's cause, in its own house style, so nothing is re-logged |
| + // here. DeferRestart stays FALSE (caps.Plan's contract): restarting |
| + // the live binary to pick up a new unit shape is the entire point of |
| + // this pass, and nothing here follows up with a Restart. |
| + _, _ = caps.Converge(o, p) |
| + } |
| + |
| + stampDBPaths(reg, routes, plans, o) |
| + return nil |
| +} |
| + |
| +// stagedManifestOfLiveVersion is the production loader: the staged sibling |
| +// manifest of the version a route's live symlink CURRENTLY names. |
| +// |
| +// The live version, never the newest staged one — a box that has staged a |
| +// version it has not adopted (a bake window, a poisoned build, a rollback) |
| +// must keep the unit shape of what it is actually running. |
| +// |
| +// (nil, nil) — no live symlink, or no sibling beside the binary — is the |
| +// leave-alone state, and it is deliberately NOT healed here: this pass does |
| +// not fetch from the bucket. release.adoptHost's back-fill re-stages the live |
| +// version's sibling on every pass, so a sibling that failed to write once |
| +// heals there and this pass picks it up next hour. |
| +func stagedManifestOfLiveVersion(optDir string) func(registry.Route) (*release.Manifest, error) { |
| + return func(rt registry.Route) (*release.Manifest, error) { |
| + ver, err := release.LiveVersion(optDir, rt.Host) |
| + if err != nil { |
| + return nil, err |
| + } |
| + if ver == "" { |
| + return nil, nil |
| + } |
| + return release.ReadStagedManifest(optDir, release.RouteAccount(rt), ver, rt.App) |
| + } |
| +} |
| + |
| +// buildPlans is the pure half: routes plus their live manifests in, converge |
| +// plans plus the advisory lines that fall out of reading them out. Extracted |
| +// so every word an operator reads in the hourly log is table-testable without |
| +// touching disk, systemd or the network. |
| +// |
| +// Three states, and they are not interchangeable (spec §4d): |
| +// |
| +// - load returns (nil, nil) — no live version, or no staged sibling for it: |
| +// a plan with a NIL Set, which converge reads as "leave every drop-in |
| +// exactly as it is" and logs once. Not a removal: a bucket hiccup must |
| +// not strip :25 off a running mail server. |
| +// - load returns an error (a sibling that exists but does not parse): the |
| +// route gets NO plan at all — the same leave-alone outcome — plus one |
| +// line naming what could not be read. |
| +// - load returns a manifest: a plan whose Set is non-nil even when the |
| +// manifest declares nothing, so that a capability DROPPED from a manifest |
| +// is actually removed. Manifest.Capabilities is a nil pointer for both a |
| +// legacy manifest and a fetch that never happened, so it is never passed |
| +// through verbatim. |
| +// |
| +// An unparseable grants column refuses that route's converge and says so, |
| +// rather than defaulting to no grants: "no grants" is a real instruction that |
| +// drops :25, and a column this binary cannot read is not evidence that an |
| +// operator asked for that. |
| +func buildPlans(routes []registry.Route, load func(rt registry.Route) (*release.Manifest, error)) ([]caps.Plan, []string) { |
| + var plans []caps.Plan |
| + var advice []string |
| + for _, rt := range routes { |
| + unitRoute := release.CapabilityRoute(rt) |
| + // The exec-backed candidate set is warnBackingMismatch's, exactly: |
| + // an instance route of a known app that is not a platform alias and |
| + // is not unit-backed. It materializes nothing (release.CapabilityRoute |
| + // excludes it) and is told so once per pass. |
| + execCandidate := !unitRoute && rt.Kind == "instance" && rt.App != "" && !rt.Platform && rt.Backing != registry.BackingUnit |
| + if !unitRoute && !execCandidate { |
| + continue |
| + } |
| + |
| + m, err := load(rt) |
| + if err != nil { |
| + advice = append(advice, fmt.Sprintf("adopt: %s: the staged manifest of its live version could not be read (%v) — its capability drop-ins are left exactly as they are until it can be", rt.Host, err)) |
| + continue |
| + } |
| + |
| + if execCandidate { |
| + if m != nil && m.Capabilities != nil { |
| + advice = append(advice, fmt.Sprintf("adopt: route %s declares capabilities but is exec-backed — nothing is materialized; the row has to be stamped backing='unit' before any capability can be written (see the app-capabilities design §5 for the stop → copy → start migration)", rt.Host)) |
| + } |
| + continue |
| + } |
| + |
| + g, err := caps.ParseGrants(rt.Caps) |
| + if err != nil { |
| + advice = append(advice, fmt.Sprintf("adopt: %s: %v — refusing to converge this route rather than guess which grants were meant; fix the row with `carlos route -host %s -grant/-revoke` from a known-good state", rt.Host, err, rt.Host)) |
| + continue |
| + } |
| + |
| + p := caps.Plan{Host: rt.Host, App: rt.App, Grants: g, Hibernate: rt.Hibernate} |
| + if m == nil { |
| + // Set stays nil: leave-alone, and converge logs the one line. |
| + plans = append(plans, p) |
| + continue |
| + } |
| + p.Set = m.Capabilities |
| + if p.Set == nil { |
| + p.Set = &caps.Set{V: caps.Version} |
| + } |
| + plans = append(plans, p) |
| + advice = append(advice, ungrantedAdvice(rt, p.Set, g)...) |
| + advice = append(advice, dataAdvice(rt, p.Set)...) |
| + } |
| + return plans, advice |
| +} |
| + |
| +// ungrantedAdvice: one line per (route, operator-consented capability) that |
| +// the app DECLARES and this box has not GRANTED. Not a failure and not an |
| +// adoption failure — the drop-in simply does not carry it — so the line names |
| +// the consequence and the exact command that fixes it. |
| +// |
| +// Deliberately silent for the self-service capabilities: `command`, `daemon`, |
| +// `children`, `runtime_dir` and `data` need no grant, so there is nothing for |
| +// an operator to do about them and a line would be noise. On a healthy fleet |
| +// (every declared grant granted) this advisory produces nothing at all. |
| +func ungrantedAdvice(rt registry.Route, s *caps.Set, g caps.Grants) []string { |
| + var out []string |
| + ports := append([]caps.Port(nil), s.Ports...) |
| + sort.Slice(ports, func(i, j int) bool { return ports[i].Port < ports[j].Port }) |
| + for _, decl := range ports { |
| + if g.Ports[decl.Port] { |
| + continue |
| + } |
| + out = append(out, fmt.Sprintf("adopt: %s declares capability %s but this box has not granted it — nothing is bound on :%d; grant it with: carlos route -host %s -grant %s", |
| + rt.Host, caps.PortToken(decl.Port), decl.Port, rt.Host, caps.PortToken(decl.Port))) |
| + } |
| + if s.HostRole && !g.HostRole { |
| + out = append(out, fmt.Sprintf("adopt: %s declares capability %s but this box has not granted it — the unit keeps DynamicUser and cannot use the box's instance role; grant it with: carlos route -host %s -grant %s", |
| + rt.Host, caps.HostRoleToken, rt.Host, caps.HostRoleToken)) |
| + } |
| + return out |
| +} |
| + |
| +// dataAdvice is what the DBPath stamp cannot say by stamping (spec §6). |
| +// |
| +// Route.DBPath is single-valued and data.dbs is a list, so only the first |
| +// name (sorted) is replicated and the rest are named rather than quietly |
| +// dropped. And a route whose app streams its own WAL keeps whatever DBPath an |
| +// operator set — the platform never clears it: two replicators on one WAL |
| +// fight over checkpointing and corrupt the replica, and deciding which one to |
| +// switch off is the operator's call, not an hourly timer's. |
| +func dataAdvice(rt registry.Route, s *caps.Set) []string { |
| + if s.Data == nil { |
| + return nil |
| + } |
| + var out []string |
| + if s.Data.Replicate == caps.DataReplicatePlatform && len(s.Data.DBs) > 1 { |
| + for _, extra := range sortedDBs(s.Data.DBs)[1:] { |
| + out = append(out, fmt.Sprintf("adopt: %s declares data.dbs %q, but a route replicates exactly one database (DBPath is single-valued) — %q is NOT replicated by the box-wide litestream unit", |
| + rt.Host, extra, extra)) |
| + } |
| + } |
| + if s.Data.Replicate == caps.DataReplicateApp && rt.DBPath != "" { |
| + out = append(out, fmt.Sprintf("adopt: %s declares data.replicate=%q — the app streams its own WAL — but its route still names db_path %s, so the box-wide litestream unit replicates it too; two replicators on one WAL fight over checkpointing and corrupt the replica. The platform will not clear the column for you: which replicator stops is an operator decision", |
| + rt.Host, caps.DataReplicateApp, rt.DBPath)) |
| + } |
| + return out |
| +} |
| + |
| +// stampDBPaths is how ops.FleetDBs learns a unit tenant's database, instead |
| +// of depending on an operator having typed `-db` at `carlos add` — which the |
| +// current code documents as "silently replicated by nobody" (spec §6). |
| +// |
| +// Only `data.replicate: "platform"` with a non-empty dbs list stamps, only |
| +// the first name in sorted order, and only when the value would actually |
| +// change: this runs hourly, and a write per pass would be a registry |
| +// heartbeat nobody asked for. |
| +// |
| +// It is deliberately NOT gated on the route's converge succeeding. The state |
| +// directory is the unchanged template's own StateDirectory=carlos-app/%i, so |
| +// the path is right whether or not a drop-in was written this pass, and a |
| +// route refused for an unrelated reason (a port another route already claims) |
| +// should still have its database replicated. FleetDBs stats every path and |
| +// skips what is not there, so a stamp that precedes the file is harmless. |
| +// |
| +// The state root comes from the same Ops the converge used, so a test's temp |
| +// dir is stamped rather than a production path a test never wrote to. |
| +func stampDBPaths(reg *registry.Registry, routes []registry.Route, plans []caps.Plan, o caps.Ops) { |
| + root := o.StateRoot |
| + if root == "" { |
| + root = caps.DefaultStateRoot |
| + } |
| + cur := make(map[string]string, len(routes)) |
| + for _, rt := range routes { |
| + cur[rt.Host] = rt.DBPath |
| + } |
| + for _, p := range plans { |
| + if p.Set == nil || p.Set.Data == nil { |
| + continue |
| + } |
| + d := p.Set.Data |
| + if d.Replicate != caps.DataReplicatePlatform || len(d.DBs) == 0 { |
| + continue |
| + } |
| + want := filepath.Join(root, p.Host, sortedDBs(d.DBs)[0]) |
| + if cur[p.Host] == want { |
| + continue |
| + } |
| + if err := reg.SetDBPath(p.Host, want); err != nil { |
| + log.Printf("adopt: %s: recording db_path %s: %v", p.Host, want, err) |
| + continue |
| + } |
| + log.Printf("adopt: %s: db_path recorded as %s from the app's data.dbs — the box-wide litestream unit replicates it from the next `carlos ops litestream` run", p.Host, want) |
| + } |
| +} |
| + |
| +// sortedDBs is data.dbs in a deterministic order, on a COPY: the slice |
| +// belongs to a decoded manifest that other readers share, and an in-place |
| +// sort here would reorder what they see. |
| +func sortedDBs(dbs []string) []string { |
| + out := append([]string(nil), dbs...) |
| + sort.Strings(out) |
| + return out |
| +} |
diff --git a/internal/host/caps_pass_test.go b/internal/host/caps_pass_test.go| new file mode 100644 |
| index 000000000..51cfe8947 |
| --- /dev/null |
| +++ b/internal/host/caps_pass_test.go |
| @@ -0,0 +1,584 @@ |
| +package host |
| + |
| +import ( |
| + "bytes" |
| + "encoding/json" |
| + "fmt" |
| + "log" |
| + "os" |
| + "path/filepath" |
| + "strings" |
| + "testing" |
| + |
| + "github.com/carlosframework/platform/internal/caps" |
| + "github.com/carlosframework/platform/internal/registry" |
| + "github.com/carlosframework/platform/internal/release" |
| +) |
| + |
| +// ---- fixtures -------------------------------------------------------------- |
| + |
| +// passOps is a caps.Ops whose every OS contact is a temp dir or a recorder: |
| +// no systemd, no /etc, no useradd, no root. Query answers "yes" to |
| +// is-enabled/is-active so a converged box's pass has no drift to repair and |
| +// the "zero calls when nothing changed" assertion means what it says. |
| +type passRec struct { |
| + calls [][]string |
| + users []string |
| + logs []string |
| +} |
| + |
| +func passOps(t *testing.T) (caps.Ops, *passRec) { |
| + t.Helper() |
| + rec := &passRec{} |
| + return caps.Ops{ |
| + EtcDir: t.TempDir(), |
| + StateRoot: t.TempDir(), |
| + RunDir: t.TempDir(), |
| + Systemctl: func(args ...string) error { |
| + rec.calls = append(rec.calls, args) |
| + return nil |
| + }, |
| + Query: func(args ...string) bool { return true }, |
| + EnsureUser: func(name string) error { rec.users = append(rec.users, name); return nil }, |
| + Logf: func(f string, a ...any) { rec.logs = append(rec.logs, strings.TrimSpace(fmt.Sprintf(f, a...))) }, |
| + }, rec |
| +} |
| + |
| +// stageVersion writes a staged binary plus its sibling manifest for ver, and |
| +// returns nothing: every caller then points the live symlink itself, because |
| +// WHICH version is live is the thing most of these tests are about. |
| +func stageVersion(t *testing.T, optDir, app, ver string, set *caps.Set) { |
| + t.Helper() |
| + m := release.Manifest{ |
| + Schema: 1, |
| + App: app, |
| + Version: ver, |
| + Kind: "binary", |
| + Artifacts: map[string]release.Artifact{ |
| + "linux-arm64": {File: app + "-" + ver, SHA256: strings.Repeat("a", 64), Size: 1}, |
| + }, |
| + Capabilities: set, |
| + } |
| + raw, err := json.MarshalIndent(m, "", " ") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + bin := release.VersionPath(optDir, release.OpsAccount, ver, app) |
| + if err := os.MkdirAll(filepath.Dir(bin), 0o755); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := os.WriteFile(bin, []byte("#!/bin/true\n"), 0o755); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := os.WriteFile(release.StagedManifestPath(optDir, release.OpsAccount, ver, app), raw, 0o644); err != nil { |
| + t.Fatal(err) |
| + } |
| +} |
| + |
| +func openReg(t *testing.T, routes ...registry.Route) *registry.Registry { |
| + t.Helper() |
| + reg, err := registry.Open(filepath.Join(t.TempDir(), "registry.db")) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + for _, rt := range routes { |
| + if err := reg.Upsert(rt); err != nil { |
| + t.Fatal(err) |
| + } |
| + } |
| + t.Cleanup(func() { reg.Close() }) |
| + return reg |
| +} |
| + |
| +// unitRoute is the shape every capability route has: a ready, non-platform, |
| +// unit-backed instance of a known app. |
| +func unitRoute(host, app string) registry.Route { |
| + return registry.Route{Host: host, Kind: "instance", App: app, Backing: registry.BackingUnit, Status: "ready", Socket: "/run/carlos/apps/" + host + ".sock"} |
| +} |
| + |
| +func readDropIns(t *testing.T, o caps.Ops, host string) (svc, sock string) { |
| + t.Helper() |
| + svcPath, sockPath := caps.DropInPaths(o.EtcDir, host) |
| + b, err := os.ReadFile(svcPath) |
| + if err == nil { |
| + svc = string(b) |
| + } |
| + b, err = os.ReadFile(sockPath) |
| + if err == nil { |
| + sock = string(b) |
| + } |
| + return svc, sock |
| +} |
| + |
| +// captureLog redirects the standard logger — where every advisory in this |
| +// package lands — for the duration of the returned func. |
| +func captureLog(t *testing.T) func() string { |
| + t.Helper() |
| + var buf bytes.Buffer |
| + old := log.Writer() |
| + flags := log.Flags() |
| + log.SetOutput(&buf) |
| + log.SetFlags(0) |
| + t.Cleanup(func() { log.SetOutput(old); log.SetFlags(flags) }) |
| + return buf.String |
| +} |
| + |
| +// ---- capabilityPass -------------------------------------------------------- |
| + |
| +// The pass converges from the staged manifest of the version that is LIVE, |
| +// never the newest version staged on the box. A box that has staged a build |
| +// it has not adopted (a bake window, a poisoned build, a rollback) must keep |
| +// running the unit shape of what it is actually running — the opposite would |
| +// bind a port for a binary that is not there. |
| +func TestCapabilityPassConvergesFromTheStagedManifestOfTheLiveVersion(t *testing.T) { |
| + optDir := t.TempDir() |
| + o, _ := passOps(t) |
| + stageVersion(t, optDir, "keymail", "aaaaaaa", &caps.Set{V: 1, Daemon: true, Ports: []caps.Port{{Port: 25, Name: "smtp"}}}) |
| + stageVersion(t, optDir, "keymail", "bbbbbbb", &caps.Set{V: 1, Daemon: true, Ports: []caps.Port{{Port: 587}}}) |
| + if err := release.PointLive(optDir, "keymail.dev", release.OpsAccount, "aaaaaaa", "keymail"); err != nil { |
| + t.Fatal(err) |
| + } |
| + rt := unitRoute("keymail.dev", "keymail") |
| + rt.Caps = "port:25,port:587" // both granted; only the live manifest declares 25 |
| + reg := openReg(t, rt) |
| + |
| + if err := capabilityPass(reg, optDir, o); err != nil { |
| + t.Fatalf("capabilityPass: %v", err) |
| + } |
| + _, sock := readDropIns(t, o, "keymail.dev") |
| + if !strings.Contains(sock, "ListenStream=25") { |
| + t.Errorf("socket drop-in = %q, want the LIVE version's ListenStream=25", sock) |
| + } |
| + if strings.Contains(sock, "ListenStream=587") { |
| + t.Errorf("socket drop-in = %q, want nothing from the newer staged version", sock) |
| + } |
| +} |
| + |
| +// A grant added after the version went live is exactly what this pass exists |
| +// for: nothing about the release changed, so adoptHost's converge never runs |
| +// — this one has to bind the port. |
| +func TestCapabilityPassMaterializesAGrantAddedAfterTheVersionWentLive(t *testing.T) { |
| + optDir := t.TempDir() |
| + o, rec := passOps(t) |
| + stageVersion(t, optDir, "keymail", "aaaaaaa", &caps.Set{V: 1, Daemon: true, Ports: []caps.Port{{Port: 25}}}) |
| + if err := release.PointLive(optDir, "keymail.dev", release.OpsAccount, "aaaaaaa", "keymail"); err != nil { |
| + t.Fatal(err) |
| + } |
| + rt := unitRoute("keymail.dev", "keymail") |
| + reg := openReg(t, rt) |
| + |
| + // Ungranted first: nothing binds. |
| + if err := capabilityPass(reg, optDir, o); err != nil { |
| + t.Fatal(err) |
| + } |
| + if _, sock := readDropIns(t, o, "keymail.dev"); sock != "" { |
| + t.Fatalf("socket drop-in written without a grant: %q", sock) |
| + } |
| + // Then the operator grants it. |
| + if err := reg.SetCaps("keymail.dev", "", "port:25"); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := capabilityPass(reg, optDir, o); err != nil { |
| + t.Fatal(err) |
| + } |
| + if _, sock := readDropIns(t, o, "keymail.dev"); !strings.Contains(sock, "ListenStream=25") { |
| + t.Errorf("socket drop-in after the grant = %q, want ListenStream=25", sock) |
| + } |
| + // The listener has to be re-bound, not merely re-written: DeferRestart is |
| + // FALSE on this path, so the socket sequence runs here. |
| + if !recordedVerb(rec, "restart", caps.SocketUnit("keymail.dev")) && !recordedVerb(rec, "start", caps.SocketUnit("keymail.dev")) { |
| + t.Errorf("systemctl calls = %v, want the socket unit re-bound", rec.calls) |
| + } |
| +} |
| + |
| +func recordedVerb(rec *passRec, verb, unit string) bool { |
| + for _, c := range rec.calls { |
| + if len(c) == 2 && c[0] == verb && c[1] == unit { |
| + return true |
| + } |
| + } |
| + return false |
| +} |
| + |
| +// Two converges of the same shape in one hour — this pass, then adoptHost's |
| +// on a version change, or simply two passes — must be a no-op the second |
| +// time: byte-identical content, no write, no daemon-reload, no restart. An |
| +// hourly pass that cycled a mail server every hour would be worse than no |
| +// pass at all. |
| +func TestCapabilityPassIsQuietWhenNothingChanged(t *testing.T) { |
| + optDir := t.TempDir() |
| + o, rec := passOps(t) |
| + stageVersion(t, optDir, "keymail", "aaaaaaa", &caps.Set{V: 1, Daemon: true, HostRole: true, Ports: []caps.Port{{Port: 25}}}) |
| + if err := release.PointLive(optDir, "keymail.dev", release.OpsAccount, "aaaaaaa", "keymail"); err != nil { |
| + t.Fatal(err) |
| + } |
| + rt := unitRoute("keymail.dev", "keymail") |
| + rt.Caps = "host-role,port:25" |
| + reg := openReg(t, rt) |
| + |
| + if err := capabilityPass(reg, optDir, o); err != nil { |
| + t.Fatal(err) |
| + } |
| + if len(rec.calls) == 0 { |
| + t.Fatal("first pass made no systemctl calls at all — it converged nothing") |
| + } |
| + |
| + rec.calls, rec.logs, rec.users = nil, nil, nil |
| + logged := captureLog(t) |
| + if err := capabilityPass(reg, optDir, o); err != nil { |
| + t.Fatal(err) |
| + } |
| + if len(rec.calls) != 0 { |
| + t.Errorf("second pass systemctl calls = %v, want none", rec.calls) |
| + } |
| + if len(rec.logs) != 0 { |
| + t.Errorf("second pass converge log = %v, want silence", rec.logs) |
| + } |
| + if out := logged(); out != "" { |
| + t.Errorf("second pass advisories = %q, want silence on a converged box", out) |
| + } |
| +} |
| + |
| +// A sibling manifest that is absent (never staged, or a write that failed) is |
| +// the leave-alone state, NOT a removal: a bucket hiccup must not strip :25 |
| +// off a running mail server. The drop-ins on disk stay byte-identical and the |
| +// pass says so once. |
| +func TestCapabilityPassLeavesDropInsAloneWhenTheSiblingIsMissing(t *testing.T) { |
| + optDir := t.TempDir() |
| + o, rec := passOps(t) |
| + stageVersion(t, optDir, "keymail", "aaaaaaa", &caps.Set{V: 1, Daemon: true, Ports: []caps.Port{{Port: 25}}}) |
| + if err := release.PointLive(optDir, "keymail.dev", release.OpsAccount, "aaaaaaa", "keymail"); err != nil { |
| + t.Fatal(err) |
| + } |
| + rt := unitRoute("keymail.dev", "keymail") |
| + rt.Caps = "port:25" |
| + reg := openReg(t, rt) |
| + if err := capabilityPass(reg, optDir, o); err != nil { |
| + t.Fatal(err) |
| + } |
| + svcBefore, sockBefore := readDropIns(t, o, "keymail.dev") |
| + if sockBefore == "" { |
| + t.Fatal("nothing converged, so this test would prove nothing") |
| + } |
| + |
| + // The sibling disappears — the exact state a failed StageManifest leaves. |
| + if err := os.Remove(release.StagedManifestPath(optDir, release.OpsAccount, "aaaaaaa", "keymail")); err != nil { |
| + t.Fatal(err) |
| + } |
| + rec.calls, rec.logs = nil, nil |
| + if err := capabilityPass(reg, optDir, o); err != nil { |
| + t.Fatal(err) |
| + } |
| + svcAfter, sockAfter := readDropIns(t, o, "keymail.dev") |
| + if svcAfter != svcBefore || sockAfter != sockBefore { |
| + t.Errorf("drop-ins changed with no manifest:\n svc %q -> %q\n sock %q -> %q", svcBefore, svcAfter, sockBefore, sockAfter) |
| + } |
| + if len(rec.calls) != 0 { |
| + t.Errorf("systemctl calls = %v, want none — leave-alone changes no unit state", rec.calls) |
| + } |
| + if !strings.Contains(strings.Join(rec.logs, "\n"), "no capability manifest resolved") { |
| + t.Errorf("logs = %v, want the leave-alone line", rec.logs) |
| + } |
| +} |
| + |
| +// ops.FleetDBs learns a unit tenant's database from the app's own |
| +// declaration, instead of depending on an operator having typed -db at |
| +// `carlos add`. dbs[0] SORTED, under the route's own state dir. |
| +func TestCapabilityPassStampsDBPathFromDataDBs(t *testing.T) { |
| + optDir := t.TempDir() |
| + o, _ := passOps(t) |
| + stageVersion(t, optDir, "console", "aaaaaaa", &caps.Set{ |
| + V: 1, Daemon: true, |
| + Data: &caps.Data{Replicate: caps.DataReplicatePlatform, DBs: []string{"zebra.db", "console.db"}}, |
| + }) |
| + if err := release.PointLive(optDir, "console.example.com", release.OpsAccount, "aaaaaaa", "console"); err != nil { |
| + t.Fatal(err) |
| + } |
| + reg := openReg(t, unitRoute("console.example.com", "console")) |
| + if err := capabilityPass(reg, optDir, o); err != nil { |
| + t.Fatal(err) |
| + } |
| + got, err := reg.Get("console.example.com") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + want := filepath.Join(o.StateRoot, "console.example.com", "console.db") |
| + if got.DBPath != want { |
| + t.Errorf("DBPath = %q, want %q (dbs[0] sorted, under the state dir)", got.DBPath, want) |
| + } |
| +} |
| + |
| +// The stamp writes only when the value would change: this pass runs hourly, |
| +// and a registry write per pass is a heartbeat nobody asked for. |
| +func TestCapabilityPassDoesNotRestampAnUnchangedDBPath(t *testing.T) { |
| + optDir := t.TempDir() |
| + o, _ := passOps(t) |
| + stageVersion(t, optDir, "console", "aaaaaaa", &caps.Set{ |
| + V: 1, Data: &caps.Data{Replicate: caps.DataReplicatePlatform, DBs: []string{"console.db"}}, |
| + }) |
| + if err := release.PointLive(optDir, "console.example.com", release.OpsAccount, "aaaaaaa", "console"); err != nil { |
| + t.Fatal(err) |
| + } |
| + rt := unitRoute("console.example.com", "console") |
| + rt.DBPath = filepath.Join(o.StateRoot, "console.example.com", "console.db") |
| + reg := openReg(t, rt) |
| + |
| + logged := captureLog(t) |
| + if err := capabilityPass(reg, optDir, o); err != nil { |
| + t.Fatal(err) |
| + } |
| + if strings.Contains(logged(), "db_path recorded") { |
| + t.Errorf("log = %q, want no stamp line for an unchanged db_path", logged()) |
| + } |
| +} |
| + |
| +// replicate:"app" means the app streams its own WAL. The platform never |
| +// clears an existing DBPath to match: two replicators on one WAL fight over |
| +// checkpointing and corrupt the replica, and deciding which one stops is an |
| +// operator's call, not an hourly timer's. It advises instead. |
| +func TestCapabilityPassNeverClearsDBPathForAppReplication(t *testing.T) { |
| + optDir := t.TempDir() |
| + o, _ := passOps(t) |
| + stageVersion(t, optDir, "keymail", "aaaaaaa", &caps.Set{ |
| + V: 1, Daemon: true, Data: &caps.Data{Replicate: caps.DataReplicateApp}, |
| + }) |
| + if err := release.PointLive(optDir, "keymail.dev", release.OpsAccount, "aaaaaaa", "keymail"); err != nil { |
| + t.Fatal(err) |
| + } |
| + rt := unitRoute("keymail.dev", "keymail") |
| + rt.DBPath = "/data/keymail.dev.db" |
| + reg := openReg(t, rt) |
| + |
| + logged := captureLog(t) |
| + if err := capabilityPass(reg, optDir, o); err != nil { |
| + t.Fatal(err) |
| + } |
| + got, err := reg.Get("keymail.dev") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if got.DBPath != "/data/keymail.dev.db" { |
| + t.Errorf("DBPath = %q, want it untouched", got.DBPath) |
| + } |
| + if !strings.Contains(logged(), "two replicators on one WAL") { |
| + t.Errorf("log = %q, want the two-replicators advisory", logged()) |
| + } |
| +} |
| + |
| +// registry.Open silently creates an empty database when handed a wrong path |
| +// or an unmounted /data. A pass against that must do nothing at all — the |
| +// same guard, for the same reason, as appconfig.Sync's. |
| +func TestCapabilityPassWithNoRoutesTouchesNothing(t *testing.T) { |
| + o, rec := passOps(t) |
| + reg := openReg(t) |
| + if err := capabilityPass(reg, t.TempDir(), o); err != nil { |
| + t.Fatal(err) |
| + } |
| + ents, err := os.ReadDir(o.EtcDir) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if len(ents) != 0 { |
| + t.Errorf("EtcDir = %v, want empty", ents) |
| + } |
| + if len(rec.calls) != 0 { |
| + t.Errorf("systemctl calls = %v, want none", rec.calls) |
| + } |
| +} |
| + |
| +// ---- buildPlans (the pure half) -------------------------------------------- |
| + |
| +// loader turns a fixed host -> manifest map into buildPlans' seam. |
| +func loader(m map[string]*release.Manifest, errs map[string]error) func(registry.Route) (*release.Manifest, error) { |
| + return func(rt registry.Route) (*release.Manifest, error) { |
| + if err := errs[rt.Host]; err != nil { |
| + return nil, err |
| + } |
| + return m[rt.Host], nil |
| + } |
| +} |
| + |
| +func manifest(app string, set *caps.Set) *release.Manifest { |
| + return &release.Manifest{Schema: 1, App: app, Version: "aaaaaaa", Kind: "binary", Capabilities: set} |
| +} |
| + |
| +// The declared-but-ungranted advisory: one line per (route, operator-consented |
| +// capability), naming the consequence and a command an operator can paste. |
| +// The self-service capabilities never produce a line — there is nothing to |
| +// grant, so a line would be hourly noise. |
| +func TestBuildPlansAdvisesUngrantedDeclaredCapabilities(t *testing.T) { |
| + cases := []struct { |
| + name string |
| + set *caps.Set |
| + grants string |
| + want []string |
| + wantNot []string |
| + }{ |
| + { |
| + name: "ungranted port names the exact grant command", |
| + set: &caps.Set{V: 1, Ports: []caps.Port{{Port: 25, Name: "smtp"}}}, |
| + want: []string{"declares capability port:25", "nothing is bound on :25", "carlos route -host keymail.dev -grant port:25"}, |
| + }, |
| + { |
| + name: "granted port says nothing", |
| + set: &caps.Set{V: 1, Ports: []caps.Port{{Port: 25}}}, |
| + grants: "port:25", |
| + want: nil, |
| + }, |
| + { |
| + name: "ungranted host_role names the transfer of authority", |
| + set: &caps.Set{V: 1, HostRole: true}, |
| + want: []string{"declares capability host-role", "carlos route -host keymail.dev -grant host-role"}, |
| + }, |
| + { |
| + name: "one line per capability, in port order", |
| + set: &caps.Set{V: 1, HostRole: true, Ports: []caps.Port{{Port: 465}, {Port: 25}}}, |
| + grants: "", |
| + want: []string{"-grant port:25", "-grant port:465", "-grant host-role"}, |
| + }, |
| + { |
| + name: "self-service capabilities are never advised", |
| + set: &caps.Set{V: 1, Daemon: true, Children: true, Command: "relay", RuntimeDir: "keymail"}, |
| + wantNot: []string{"has not granted"}, |
| + }, |
| + } |
| + for _, c := range cases { |
| + t.Run(c.name, func(t *testing.T) { |
| + rt := unitRoute("keymail.dev", "keymail") |
| + rt.Caps = c.grants |
| + _, advice := buildPlans([]registry.Route{rt}, loader(map[string]*release.Manifest{"keymail.dev": manifest("keymail", c.set)}, nil)) |
| + joined := strings.Join(advice, "\n") |
| + for _, w := range c.want { |
| + if !strings.Contains(joined, w) { |
| + t.Errorf("advice =\n%s\nwant a line containing %q", joined, w) |
| + } |
| + } |
| + for _, w := range c.wantNot { |
| + if strings.Contains(joined, w) { |
| + t.Errorf("advice =\n%s\nwant NOTHING containing %q", joined, w) |
| + } |
| + } |
| + if len(c.want) == 0 && len(c.wantNot) == 0 && len(advice) != 0 { |
| + t.Errorf("advice = %v, want none", advice) |
| + } |
| + }) |
| + } |
| +} |
| + |
| +// An exec-backed route whose release declares capabilities materializes |
| +// nothing (release.CapabilityRoute excludes it) and is told so — the guided |
| +// path through the §5 migration, and the line an operator sees during the |
| +// reversible dress rehearsal. It gets no plan, so nothing can be written for |
| +// it by accident. |
| +func TestBuildPlansAdvisesExecBackedCapabilities(t *testing.T) { |
| + execRT := registry.Route{Host: "console.example.com", Kind: "instance", App: "console", Status: "ready"} |
| + quietRT := registry.Route{Host: "blog.example.com", Kind: "instance", App: "blog", Status: "ready"} |
| + aliasRT := registry.Route{Host: "alias.example.com", Kind: "instance", App: "console", Platform: true, Status: "ready"} |
| + set := &caps.Set{V: 1, Daemon: true, Data: &caps.Data{Replicate: caps.DataReplicatePlatform, DBs: []string{"console.db"}}} |
| + plans, advice := buildPlans( |
| + []registry.Route{execRT, quietRT, aliasRT}, |
| + loader(map[string]*release.Manifest{ |
| + "console.example.com": manifest("console", set), |
| + "blog.example.com": manifest("blog", nil), |
| + "alias.example.com": manifest("console", set), |
| + }, nil), |
| + ) |
| + if len(plans) != 0 { |
| + t.Errorf("plans = %v, want none — nothing may be materialized for an exec-backed or alias route", plans) |
| + } |
| + if len(advice) != 1 { |
| + t.Fatalf("advice = %v, want exactly one line (the exec-backed route only)", advice) |
| + } |
| + for _, w := range []string{"console.example.com", "exec-backed", "backing='unit'", "§5"} { |
| + if !strings.Contains(advice[0], w) { |
| + t.Errorf("advice = %q, want it to contain %q", advice[0], w) |
| + } |
| + } |
| +} |
| + |
| +// A caps column this binary cannot read refuses that route's converge and |
| +// says so. Defaulting to "no grants" would be a revoke nobody typed — and a |
| +// revoke of port:25 drops a mail server's listener. |
| +func TestBuildPlansRefusesAnUnparseableGrantsColumn(t *testing.T) { |
| + rt := unitRoute("keymail.dev", "keymail") |
| + rt.Caps = "port:025,host-role" // a hand edit: PortToken never writes "025" |
| + plans, advice := buildPlans([]registry.Route{rt}, |
| + loader(map[string]*release.Manifest{"keymail.dev": manifest("keymail", &caps.Set{V: 1, Ports: []caps.Port{{Port: 25}}})}, nil)) |
| + if len(plans) != 0 { |
| + t.Errorf("plans = %v, want none — an unreadable grants column converges nothing", plans) |
| + } |
| + if len(advice) != 1 || !strings.Contains(advice[0], "keymail.dev") || !strings.Contains(advice[0], "carlos route -host keymail.dev -grant/-revoke") { |
| + t.Errorf("advice = %v, want one line naming the row and the fix", advice) |
| + } |
| +} |
| + |
| +// The three states of a manifest read are not interchangeable (spec §4d): |
| +// unreadable -> no plan at all (leave-alone, plus a line naming the failure); |
| +// absent -> a plan with a NIL Set (converge's own leave-alone, logged there); |
| +// resolved-but-declaring-nothing -> a NON-NIL empty Set, which is what makes |
| +// converge REMOVE a drop-in for a capability the app has dropped. |
| +func TestBuildPlansDistinguishesTheThreeManifestStates(t *testing.T) { |
| + unreadable := unitRoute("bad.example.com", "keymail") |
| + absent := unitRoute("absent.example.com", "keymail") |
| + empty := unitRoute("empty.example.com", "keymail") |
| + plans, advice := buildPlans( |
| + []registry.Route{unreadable, absent, empty}, |
| + loader( |
| + map[string]*release.Manifest{"empty.example.com": manifest("keymail", nil)}, |
| + map[string]error{"bad.example.com": os.ErrInvalid}, |
| + ), |
| + ) |
| + if len(plans) != 2 { |
| + t.Fatalf("plans = %v, want 2 (the unreadable route gets none)", plans) |
| + } |
| + if plans[0].Host != "absent.example.com" || plans[0].Set != nil { |
| + t.Errorf("plan[0] = %+v, want absent.example.com with a NIL Set (leave-alone)", plans[0]) |
| + } |
| + if plans[1].Host != "empty.example.com" || plans[1].Set == nil { |
| + t.Errorf("plan[1] = %+v, want empty.example.com with a NON-NIL Set (removal semantics)", plans[1]) |
| + } |
| + if len(advice) != 1 || !strings.Contains(advice[0], "bad.example.com") { |
| + t.Errorf("advice = %v, want one line naming the unreadable manifest", advice) |
| + } |
| +} |
| + |
| +// DeferRestart belongs to the version-change converge alone. Restarting the |
| +// live binary to pick up a new unit shape is the entire point of THIS pass, |
| +// and nothing here follows up with a caps.Restart — a deferred plan would |
| +// leave a daemon stopped until the next version change. |
| +func TestBuildPlansNeverDefersARestart(t *testing.T) { |
| + rt := unitRoute("keymail.dev", "keymail") |
| + plans, _ := buildPlans([]registry.Route{rt}, |
| + loader(map[string]*release.Manifest{"keymail.dev": manifest("keymail", &caps.Set{V: 1, Daemon: true})}, nil)) |
| + if len(plans) != 1 { |
| + t.Fatalf("plans = %v, want 1", plans) |
| + } |
| + if plans[0].DeferRestart { |
| + t.Error("DeferRestart is true — this pass has no follow-up restart, so a daemon would stay down") |
| + } |
| +} |
| + |
| +// data.dbs is a list and Route.DBPath is single-valued. The extras are named |
| +// rather than quietly dropped, so nobody believes a database is replicated |
| +// when it is not. |
| +func TestBuildPlansAdvisesUnreplicatedExtraDBs(t *testing.T) { |
| + rt := unitRoute("console.example.com", "console") |
| + _, advice := buildPlans([]registry.Route{rt}, |
| + loader(map[string]*release.Manifest{"console.example.com": manifest("console", &caps.Set{ |
| + V: 1, Data: &caps.Data{Replicate: caps.DataReplicatePlatform, DBs: []string{"console.db", "sessions.db", "audit.db"}}, |
| + })}, nil)) |
| + joined := strings.Join(advice, "\n") |
| + // Sorted, so audit.db is the one that gets the DBPath stamp and the other |
| + // two are the ones an operator has to hear about by name. |
| + for _, w := range []string{`"console.db" is NOT replicated`, `"sessions.db" is NOT replicated`} { |
| + if !strings.Contains(joined, w) { |
| + t.Errorf("advice =\n%s\nwant a line containing %q", joined, w) |
| + } |
| + } |
| + if strings.Contains(joined, `"audit.db" is NOT replicated`) { |
| + t.Errorf("advice =\n%s\nwant the FIRST db (audit.db, sorted) treated as replicated", joined) |
| + } |
| + if len(advice) != 2 { |
| + t.Errorf("advice = %v, want exactly one line per unreplicated extra", advice) |
| + } |
| +} |
diff --git a/internal/release/adopt.go b/internal/release/adopt.go| index 1eba14e32..e305885b6 100644 |
| --- a/internal/release/adopt.go |
| +++ b/internal/release/adopt.go |
| @@ -283,7 +283,7 @@ func adoptable(rt registry.Route) bool { |
| (rt.Status == "" || rt.Status == "ready") |
| } |
| -// capabilityRoute reports whether this route may materialize capabilities: |
| +// CapabilityRoute reports whether this route may materialize capabilities: |
| // a lifecycle-managed instance of a known app, unit-backed, non-platform, |
| // ready (spec §4e). It is adoptable() plus the unit backing, deliberately — |
| // a route this pass would not adopt is not one whose unit shape this pass |
| @@ -297,7 +297,13 @@ func adoptable(rt registry.Route) bool { |
| // materialises nothing here and is told so once per pass by RunAdopt's |
| // advisory, not by this function — that is the guided path through the §5 |
| // migration. |
| -func capabilityRoute(rt registry.Route) bool { |
| +// |
| +// Exported for host.capabilityPass (the hourly grant-drift pass): the two |
| +// converge entry points must agree, to the character, on which routes may |
| +// have a root-owned unit drop-in written for them. A second copy of this |
| +// predicate in another package is exactly the drift that would let one pass |
| +// materialize a shape the other refuses. |
| +func CapabilityRoute(rt registry.Route) bool { |
| return adoptable(rt) && rt.Backing == registry.BackingUnit |
| } |
| @@ -412,7 +418,7 @@ func adoptHost(ctx context.Context, reg *registry.Registry, s Store, optDir stri |
| plan caps.Plan |
| converged bool |
| ) |
| - if cfg.capsOps != nil && capabilityRoute(rt) { |
| + if cfg.capsOps != nil && CapabilityRoute(rt) { |
| // restart != nil is what makes the deferral safe: it says somebody is |
| // going to start this daemon after the symlink moves. With no restart |
| // callback at all (a caller with no proc manager) converge keeps |
| @@ -493,7 +499,7 @@ func adoptHost(ctx context.Context, reg *registry.Registry, s Store, optDir stri |
| // mark a window, not to diff it. It costs nothing — m is already in hand — |
| // and it errs toward one extra line, once per version change per host, |
| // never per tick. |
| - if cfg.capsOps == nil && capabilityRoute(rt) && m != nil && m.Capabilities != nil { |
| + if cfg.capsOps == nil && CapabilityRoute(rt) && m != nil && m.Capabilities != nil { |
| fmt.Printf(" %s: %s declares capabilities and this pass does not write unit configuration — it runs against the drop-ins the previous version left until the root adopt pass (carlos-adopt.service, hourly) converges them\n", rt.Host, ver) |
| } |