| new file mode 100644 |
| index 000000000..1988c88a1 |
| --- /dev/null |
| +++ b/docs/superpowers/plans/2026-10-06-console-release-defences-plan-3-d3.md |
| @@ -0,0 +1,3599 @@ |
| +# Console release defences, Plan 3 of 4: D3 (the flagship console adopts only builds proven on dev) |
| + |
| +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. |
| + |
| +**Goal:** A deployment that lists an app in its proof policy adopts a new |
| +build of that app only with a signed proof that the proving deployment (dev) |
| +ran those exact bytes healthily, or when the box has itself run that build |
| +healthily before. Dev mints the proof on its own; an operator carries it. |
| +A deployment with no policy file sees none of this. |
| + |
| +**Architecture:** `internal/release` gains the proof record (`proof.go`), the |
| +policy file reader (`proofpolicy.go`), the healthy history (`healthy.go`) and |
| +a long-lived `ProofGate` (`proofgate.go`) that `adoptHost` consults twice: a |
| +cheap gate before `ensureStaged` and a staged-bytes check before `PointLive`. |
| +`internal/host` establishes a running unit's build from its bytes |
| +(`proofobserve.go`), records healthy builds from D1's settle and from a |
| +long-running live build, mints on the proving box in a root pass |
| +(`proofroot.go`), and on the agent re-asks for that root pass every 2 minutes |
| +until the proof is in the bucket (`proofask.go`). A `carlos proof` verb mints |
| +the key and carries proofs; `carlos status --live` shows a host that is waiting. |
| + |
| +**Tech Stack:** Go stdlib (`crypto/ed25519`, `crypto/sha256`, `encoding/json`), |
| +the existing release, host, status and CLI packages. |
| + |
| +**Spec:** `docs/superpowers/specs/2026-10-06-console-release-defences-design.md` |
| +(rev 5): §4 (D3) and §7's D3 tests. **Depends on Plan 1** (`release.Refusal`, |
| +`WriteRefusal`, `ReadRefusal`, `ListRefusals`; `unitProps`, `unitSystem`, |
| +`systemdUnits`, `parseShow`, `settleUnit`, `routeReady`, `checkedUnitRestart` |
| +and the test helpers `fakeUnits` and `liveTo` in `internal/host`) and on |
| +**Plan 2** (its D2 block in `adoptHost`, which starts |
| +`newProg := FileProgram(...)`, is the anchor this plan's binding check sits |
| +after). Land Plans 1 and 2 first. |
| + |
| +## Global Constraints |
| + |
| +- The proof is `{"schema":1,"app","platform","sha256","version","prover","active_since","proven_at","sig"}`; `sig` is base64 ed25519 over the canonical JSON of every other field (spec §4.2). |
| +- A proof binds `app`, `platform` and `sha256`; `version` is informational only (spec §4.2). |
| +- Every sha256 in D3 is hashed from bytes: minting hashes `/proc/<MainPID>/exe`, the requiring box hashes the staged file (cache hit included), the healthy history hashes the executable that passed (spec §4.2, codex 3). |
| +- The proving key is a fresh ed25519 key, separate from both release keys, at `/etc/carlos/proving.key`, root 0600; `carlos proof keygen` writes the private half and prints the public half (spec §4.2-§4.3). |
| +- The policy file is `/etc/carlos/proof-policy.json`, root-owned 0644, shape `{"schema":1,"prove":{"apps","prover","after","key_file"},"require":[{"app","prover","key"}]}` (spec §4.5). |
| +- Absent policy file: D3 off, no proof read or written, no bucket GET under `proofs/` (spec §4.5-§4.6). |
| +- Unreadable or unparseable policy file: D3 on and waiting for every named-unit route, logged every pass (spec §4.5). Unknown keys count as unparseable, so a typo cannot open the gate. A root pass is a fresh process and prints it every pass; the agent's 2 s loop prints it at most once a minute so the journal is not flooded. |
| +- Agent and root both re-read the policy file at the start of every pass (spec §4.5, codex 6). |
| +- Mint conditions, all at once: binding verified, unit `active` with `ActiveEnterTimestamp` at least `after` ago and one `InvocationID` throughout, S = sha256 of `/proc/<MainPID>/exe` equal to the live link's bytes, readiness passes now, no refusal record names S, no proof for S in the bucket (spec §4.3). |
| +- `after` is the `prove` block's value; 15m when there is no `prove` block or no policy file (spec §4.3, §4.5). |
| +- The agent re-requests a mint every 2 minutes, through the stage-request relay, until it sees the proof (spec §4.3, codex 5). |
| +- Proof key, in both buckets: `apps/<acct>/<app>/proofs/<sha256>/<prover>.json` (spec §4.4; see Task 1's layout note for §4.3). |
| +- The dev box role may write it: `infra/modules/region-host/main.tf:166-170` grants `s3:PutObject` on `${deployment_bucket_arn}/*`, and the only bucket Deny (`infra/modules/deployment-buckets/main.tf:69-70`) names `control/boxes/` alone (spec §4.3's "the plan confirms"). |
| +- A missing proof is not a refusal: print `<host>: <ver> (sha256 <S>) is not proven on <prover>; staying on <cur>` and keep looking (spec §4.5). |
| +- A staged hash that is neither the proof's nor in the healthy history is a refusal record naming both hashes (spec §4.5 check 2). |
| +- A missing proof is cached per (route, sha256) for 30 s (spec §4.5). |
| +- Healthy history: added by D1's settle passing, and by a root pass finding a named-unit route's live build meeting the first three mint conditions (spec §4.5). |
| +- A waiting non-canary route stays in `fastMembers`, rejoining the sweep each time its 30 s miss runs out, which is what bounds its cost (spec §4.5, codex 14). |
| +- While the requirement is off on a box with healthy history for an app, the hourly root pass logs that it is off (spec §4.5). No bucket object or CLI flag can switch it off. |
| +- Gate with `GOFLAGS=-buildvcs=false`; in this sandbox set `GOCACHE=$TMPDIR/gocache` per command. |
| + |
| +## Review Focus |
| + |
| +1. **A typo in flagship's policy file** (a missing brace, or `"requires"` for `"require"`). Expect: the gate stays closed for every named-unit route and the pass says why; nothing adopts. Test in Task 4 (`TestProofPolicyUnparseableKeepsGateClosed`). |
| +2. **A reused version name or a stale staged cache**: the manifest's sha256 has a valid proof, but the staged file's bytes differ. Expect: refused at the binding check with a refusal record naming both hashes, never pointed live. Test in Task 4 (`TestProofBindingRefusesStagedBytesThatDiffer`). |
| +3. **Rolling flagship back to a build it ran this morning.** Expect: adopted at once from the healthy history, with no proof and no wait on dev. Test in Task 4 (`TestProofHealthyHistoryAdmitsWithoutProof`); the history entry itself comes from Task 6 (`TestProofRootRecordsTheLiveBuildAsHealthy`). |
| +4. **A deployment with no policy file** (Tito, self-hosters, every customer app). Expect: adoption byte-identical to before, and not one bucket GET under `proofs/`. Test in Task 4 (`TestProofNoPolicyMakesNoProofRead`). |
| +5. **Dev's mint request is dropped by the relay, or the mint write fails.** Expect: the agent asks again every 2 minutes and stops once the proof is in the bucket. Test in Task 7 (`TestMintAskerReasksUntilTheProofAppears`). |
| + |
| +--- |
| + |
| +### Task 1: The proof record |
| + |
| +**Files:** |
| +- Create: `internal/release/proof.go` |
| +- Test: `internal/release/proof_test.go` |
| + |
| +**Interfaces:** |
| +- Consumes: `GenerateKey`, `ParseSigningKey`, `ParsePublicKey`, `ValidApp`, `ValidVersion`, `appPrefix`, `putImmutable`, `ErrCASConflict` (internal/release/release.go, store.go). |
| +- Produces: |
| + - `type Proof struct { Schema int; App, Platform, SHA256, Version, Prover string; ActiveSince, ProvenAt time.Time; Sig string }` (JSON keys as the spec) |
| + - `func SignProof(priv ed25519.PrivateKey, p Proof) (*Proof, error)` |
| + - `func ParseProof(b []byte) (*Proof, error)` |
| + - `func (p *Proof) Verify(pub ed25519.PublicKey) error` |
| + - `func (p *Proof) Check(pub ed25519.PublicKey, app, platform, sha, prover string) error` |
| + - `func ValidProver(s string) bool` |
| + - `func AdoptPlatform() string` |
| + - `func FileSHA256(path string) (string, error)` |
| + - `func ProofKey(acct, app, sha, prover string) string`, `func ProofsPrefix(acct, app, sha string) string`, `func ProofPendingKey(acct, app, sha, prover string) string` |
| + - `func GetProof(ctx context.Context, s Store, acct, app, sha, prover string) (*Proof, error)` (absent: `os.ErrNotExist`-wrapped) |
| + - `func PutProof(ctx context.Context, s Store, acct string, p *Proof) error` (create-only; the same bytes again is a no-op) |
| + - `type ProofPending struct { Schema int; App, Platform, SHA256, Version, Prover string; LiveSince time.Time; After string }` |
| + - `func PutProofPending(ctx context.Context, s Store, acct string, p ProofPending) error` (create-only; an existing record is kept) |
| + - `func GetProofPending(ctx context.Context, s Store, acct, app, sha, prover string) (*ProofPending, error)` |
| + |
| +**Layout note (for the reviewer):** spec §4.3 has dev write |
| +`proofs/<app>/<sha256>.json`, and §4.4 has `proof put` write |
| +`apps/<acct>/<app>/proofs/<sha256>/<prover>.json` on the target. This plan uses |
| +the §4.4 key in both buckets. `proof get` and `proof put` then move a file |
| +between identical keys, the minting box's "no proof for S yet" check and the |
| +requiring box's lookup are one function, and both sit under the app's prefix, |
| +which `PurgeApp` already sweeps. |
| + |
| +**Pending note:** `proof get`'s failure text names how long the build has been |
| +live on dev (spec §4.4). Nothing else records that, so dev's agent writes a |
| +`<prover>.pending.json` beside where the proof will go, once per build, when it |
| +first sees a listed build live (Task 7). It is advisory: no box ever reads it. |
| + |
| +- [ ] **Step 1: Write the failing tests** |
| + |
| +```go |
| +package release |
| + |
| +import ( |
| + "context" |
| + "crypto/ed25519" |
| + "encoding/json" |
| + "errors" |
| + "os" |
| + "path/filepath" |
| + "strings" |
| + "testing" |
| + "time" |
| +) |
| + |
| +func proverKey(t *testing.T) (ed25519.PrivateKey, ed25519.PublicKey, string) { |
| + t.Helper() |
| + seed, pubB64, err := GenerateKey() |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + priv, err := ParseSigningKey(seed, "test") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + pub, err := ParsePublicKey(pubB64, "test") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + return priv, pub, pubB64 |
| +} |
| + |
| +var testSHA = strings.Repeat("ab", 32) |
| + |
| +func sampleProof() Proof { |
| + return Proof{App: "console", Platform: "linux-arm64", SHA256: testSHA, Version: "f4478b15", Prover: "dev", |
| + ActiveSince: time.Date(2026, 10, 6, 9, 0, 0, 0, time.UTC), |
| + ProvenAt: time.Date(2026, 10, 6, 9, 15, 0, 0, time.UTC)} |
| +} |
| + |
| +func TestProofSignVerifyRoundTrip(t *testing.T) { |
| + priv, pub, _ := proverKey(t) |
| + p, err := SignProof(priv, sampleProof()) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + b, err := json.MarshalIndent(p, "", " ") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + got, err := ParseProof(b) |
| + if err != nil { |
| + t.Fatalf("ParseProof: %v", err) |
| + } |
| + if got.Schema != 1 { |
| + t.Fatalf("schema = %d", got.Schema) |
| + } |
| + if err := got.Check(pub, "console", "linux-arm64", testSHA, "dev"); err != nil { |
| + t.Fatalf("Check: %v", err) |
| + } |
| + // The signature is over the canonical form, not the file's bytes, so a |
| + // proof re-encoded in transit still verifies. |
| + compact, err := json.Marshal(got) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + again, err := ParseProof(compact) |
| + if err != nil || again.Verify(pub) != nil { |
| + t.Fatalf("a compact re-encoding does not verify: %v", err) |
| + } |
| +} |
| + |
| +func TestProofEditedFieldDoesNotVerify(t *testing.T) { |
| + priv, pub, _ := proverKey(t) |
| + p, err := SignProof(priv, sampleProof()) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + for name, edit := range map[string]func(*Proof){ |
| + "sha256": func(p *Proof) { p.SHA256 = strings.Repeat("cd", 32) }, |
| + "app": func(p *Proof) { p.App = "other" }, |
| + "platform": func(p *Proof) { p.Platform = "linux-amd64" }, |
| + "version": func(p *Proof) { p.Version = "v9" }, |
| + "prover": func(p *Proof) { p.Prover = "staging" }, |
| + "proven_at": func(p *Proof) { p.ProvenAt = p.ProvenAt.Add(time.Second) }, |
| + } { |
| + q := *p |
| + edit(&q) |
| + if err := q.Verify(pub); err == nil { |
| + t.Fatalf("%s edited and the proof still verifies", name) |
| + } |
| + } |
| + _, other, _ := proverKey(t) |
| + if err := p.Verify(other); err == nil { |
| + t.Fatal("the proof verifies against another prover's key") |
| + } |
| +} |
| + |
| +func TestProofCheckNamesTheMismatch(t *testing.T) { |
| + priv, pub, _ := proverKey(t) |
| + p, err := SignProof(priv, sampleProof()) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + for _, c := range []struct{ app, plat, sha, prover, want string }{ |
| + {"other", "linux-arm64", testSHA, "dev", "app"}, |
| + {"console", "linux-amd64", testSHA, "dev", "platform"}, |
| + {"console", "linux-arm64", strings.Repeat("cd", 32), "dev", "sha256"}, |
| + {"console", "linux-arm64", testSHA, "staging", "prover"}, |
| + } { |
| + err := p.Check(pub, c.app, c.plat, c.sha, c.prover) |
| + if err == nil || !strings.Contains(err.Error(), c.want) { |
| + t.Fatalf("Check(%s,%s,%s,%s) = %v, want an error naming %s", c.app, c.plat, c.sha, c.prover, err, c.want) |
| + } |
| + } |
| +} |
| + |
| +func TestParseProofRejectsBadShape(t *testing.T) { |
| + priv, _, _ := proverKey(t) |
| + good, err := SignProof(priv, sampleProof()) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + for name, edit := range map[string]func(map[string]any){ |
| + "schema": func(m map[string]any) { m["schema"] = 2 }, |
| + "sha256": func(m map[string]any) { m["sha256"] = "../../etc" }, |
| + "prover": func(m map[string]any) { m["prover"] = "Dev Box" }, |
| + "app": func(m map[string]any) { m["app"] = "" }, |
| + "sig": func(m map[string]any) { m["sig"] = "not base64!" }, |
| + "proven_at": func(m map[string]any) { delete(m, "proven_at") }, |
| + } { |
| + b, _ := json.Marshal(good) |
| + var m map[string]any |
| + if err := json.Unmarshal(b, &m); err != nil { |
| + t.Fatal(err) |
| + } |
| + edit(m) |
| + b, _ = json.Marshal(m) |
| + if _, err := ParseProof(b); err == nil { |
| + t.Fatalf("%s: a malformed proof parsed", name) |
| + } |
| + } |
| + if _, err := ParseProof([]byte("{")); err == nil { |
| + t.Fatal("torn JSON parsed") |
| + } |
| +} |
| + |
| +func TestProofStoreRoundTrip(t *testing.T) { |
| + s := NewDirStore(t.TempDir()) |
| + ctx := context.Background() |
| + priv, pub, _ := proverKey(t) |
| + p, err := SignProof(priv, sampleProof()) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if got, want := ProofKey("bab", "console", testSHA, "dev"), "apps/bab/console/proofs/"+testSHA+"/dev.json"; got != want { |
| + t.Fatalf("ProofKey = %s, want %s", got, want) |
| + } |
| + if _, err := GetProof(ctx, s, "bab", "console", testSHA, "dev"); !errors.Is(err, os.ErrNotExist) { |
| + t.Fatalf("absent proof = %v, want os.ErrNotExist", err) |
| + } |
| + if err := PutProof(ctx, s, "bab", p); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := PutProof(ctx, s, "bab", p); err != nil { |
| + t.Fatalf("carrying the same proof twice: %v", err) |
| + } |
| + got, err := GetProof(ctx, s, "bab", "console", testSHA, "dev") |
| + if err != nil || got.Verify(pub) != nil { |
| + t.Fatalf("GetProof = %+v, %v", got, err) |
| + } |
| + if _, err := GetProof(ctx, s, "bab", "console", "../x", "dev"); err == nil { |
| + t.Fatal("a path in the sha256 reached the store") |
| + } |
| +} |
| + |
| +func TestProofPendingIsCreateOnly(t *testing.T) { |
| + s := NewDirStore(t.TempDir()) |
| + ctx := context.Background() |
| + first := time.Date(2026, 10, 6, 9, 30, 30, 0, time.UTC) |
| + rec := ProofPending{App: "console", Platform: "linux-arm64", SHA256: testSHA, Version: "v2", Prover: "dev", LiveSince: first, After: "15m"} |
| + if err := PutProofPending(ctx, s, "bab", rec); err != nil { |
| + t.Fatal(err) |
| + } |
| + rec.LiveSince = first.Add(time.Hour) |
| + if err := PutProofPending(ctx, s, "bab", rec); err != nil { |
| + t.Fatalf("a second sighting: %v", err) |
| + } |
| + got, err := GetProofPending(ctx, s, "bab", "console", testSHA, "dev") |
| + if err != nil || !got.LiveSince.Equal(first) { |
| + t.Fatalf("pending = %+v, %v; want the first sighting kept", got, err) |
| + } |
| +} |
| + |
| +func TestFileSHA256(t *testing.T) { |
| + p := filepath.Join(t.TempDir(), "f") |
| + if err := os.WriteFile(p, []byte("bad build"), 0o644); err != nil { |
| + t.Fatal(err) |
| + } |
| + got, err := FileSHA256(p) |
| + if err != nil || got != etagOf([]byte("bad build")) { |
| + t.Fatalf("FileSHA256 = %q, %v", got, err) |
| + } |
| +} |
| +``` |
| + |
| +`etagOf` (internal/release/store.go:303) is the hex sha256 `Ship` stamps into |
| +manifests, so `FileSHA256` and a manifest's `sha256` compare directly. |
| + |
| +- [ ] **Step 2: Run them to verify they fail** |
| + |
| +Run: `GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/release -run 'Proof|FileSHA256' -count=1` |
| +Expected: FAIL, `undefined: SignProof`. |
| + |
| +- [ ] **Step 3: Implement `proof.go`** |
| + |
| +```go |
| +// Dev proofs — D3 of the console release defences (spec |
| +// 2026-10-06-console-release-defences-design.md §4.2-§4.4). |
| +// |
| +// A proving deployment (dev) signs a Proof once a build has run healthily on |
| +// its console for a bake period. An operator carries it to a requiring |
| +// deployment's bucket, and a requiring box adopts a listed app's build only |
| +// with a proof for those exact bytes from a prover it trusts (proofgate.go). |
| +// A proof binds app, platform and the artifact's sha256. version is |
| +// informational, because version names get reused after pruning. |
| +// |
| +// The signature covers the canonical JSON of every field except sig: |
| +// json.Marshal of proofBody, whose field order the struct fixes and whose |
| +// times are UTC. Verify re-marshals the parsed fields, so a proof whose bytes |
| +// were reformatted in transit still verifies and one whose fields were edited |
| +// does not. The proving key is its own ed25519 key, never a release key. |
| +package release |
| + |
| +import ( |
| + "context" |
| + "crypto/ed25519" |
| + "crypto/sha256" |
| + "encoding/base64" |
| + "encoding/hex" |
| + "encoding/json" |
| + "errors" |
| + "fmt" |
| + "io" |
| + "os" |
| + "regexp" |
| + "time" |
| +) |
| + |
| +const proofSchema = 1 |
| + |
| +type Proof struct { |
| + Schema int `json:"schema"` |
| + App string `json:"app"` |
| + Platform string `json:"platform"` |
| + SHA256 string `json:"sha256"` |
| + Version string `json:"version"` |
| + Prover string `json:"prover"` |
| + ActiveSince time.Time `json:"active_since"` |
| + ProvenAt time.Time `json:"proven_at"` |
| + Sig string `json:"sig"` |
| +} |
| + |
| +// proofBody is the signed form: every field of Proof except Sig, in order. |
| +type proofBody struct { |
| + Schema int `json:"schema"` |
| + App string `json:"app"` |
| + Platform string `json:"platform"` |
| + SHA256 string `json:"sha256"` |
| + Version string `json:"version"` |
| + Prover string `json:"prover"` |
| + ActiveSince time.Time `json:"active_since"` |
| + ProvenAt time.Time `json:"proven_at"` |
| +} |
| + |
| +func (p *Proof) canonical() ([]byte, error) { |
| + return json.Marshal(proofBody{Schema: p.Schema, App: p.App, Platform: p.Platform, SHA256: p.SHA256, |
| + Version: p.Version, Prover: p.Prover, ActiveSince: p.ActiveSince.UTC(), ProvenAt: p.ProvenAt.UTC()}) |
| +} |
| + |
| +var ( |
| + proverRe = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{0,31}$`) |
| + sha256Re = regexp.MustCompile(`^[0-9a-f]{64}$`) |
| + platformRe = regexp.MustCompile(`^[a-z0-9]+-[a-z0-9]+$`) |
| +) |
| + |
| +// ValidProver is a prover name: lower case, digits and dashes, at most 32. |
| +func ValidProver(s string) bool { return proverRe.MatchString(s) } |
| + |
| +// AdoptPlatform is the artifact key this box installs (adoptPlatform), for |
| +// callers outside the package that mint or describe proofs. |
| +func AdoptPlatform() string { return adoptPlatform } |
| + |
| +func (p *Proof) shape() error { |
| + switch { |
| + case p.Schema != proofSchema: |
| + return fmt.Errorf("proof: schema %d, want %d", p.Schema, proofSchema) |
| + case !ValidApp(p.App): |
| + return fmt.Errorf("proof: invalid app %q", p.App) |
| + case !platformRe.MatchString(p.Platform): |
| + return fmt.Errorf("proof: invalid platform %q", p.Platform) |
| + case !sha256Re.MatchString(p.SHA256): |
| + return fmt.Errorf("proof: sha256 %q is not 64 lower-case hex digits", p.SHA256) |
| + case !ValidVersion(p.Version): |
| + return fmt.Errorf("proof: invalid version %q", p.Version) |
| + case !ValidProver(p.Prover): |
| + return fmt.Errorf("proof: invalid prover %q", p.Prover) |
| + case p.ActiveSince.IsZero() || p.ProvenAt.IsZero(): |
| + return fmt.Errorf("proof: active_since and proven_at are both required") |
| + } |
| + return nil |
| +} |
| + |
| +// SignProof stamps the schema and signs p with the proving key. |
| +func SignProof(priv ed25519.PrivateKey, p Proof) (*Proof, error) { |
| + if len(priv) != ed25519.PrivateKeySize { |
| + return nil, fmt.Errorf("proof: no usable proving key") |
| + } |
| + p.Schema, p.Sig = proofSchema, "" |
| + if err := p.shape(); err != nil { |
| + return nil, err |
| + } |
| + body, err := p.canonical() |
| + if err != nil { |
| + return nil, err |
| + } |
| + p.Sig = base64.StdEncoding.EncodeToString(ed25519.Sign(priv, body)) |
| + return &p, nil |
| +} |
| + |
| +// ParseProof checks shape only. Whether it verifies is the requiring box's |
| +// question (Check), because only the box holds the prover's public key. |
| +func ParseProof(b []byte) (*Proof, error) { |
| + var p Proof |
| + if err := json.Unmarshal(b, &p); err != nil { |
| + return nil, fmt.Errorf("proof: %v", err) |
| + } |
| + if err := p.shape(); err != nil { |
| + return nil, err |
| + } |
| + if sig, err := base64.StdEncoding.DecodeString(p.Sig); err != nil || len(sig) != ed25519.SignatureSize { |
| + return nil, fmt.Errorf("proof: sig is not a base64 ed25519 signature") |
| + } |
| + return &p, nil |
| +} |
| + |
| +func (p *Proof) Verify(pub ed25519.PublicKey) error { |
| + if len(pub) != ed25519.PublicKeySize { |
| + return fmt.Errorf("proof: no usable prover public key") |
| + } |
| + sig, err := base64.StdEncoding.DecodeString(p.Sig) |
| + if err != nil { |
| + return fmt.Errorf("proof: sig: %v", err) |
| + } |
| + body, err := p.canonical() |
| + if err != nil { |
| + return err |
| + } |
| + if !ed25519.Verify(pub, body, sig) { |
| + return fmt.Errorf("proof signature does not verify against the configured key") |
| + } |
| + return nil |
| +} |
| + |
| +// Check is Verify plus the binding: this proof is for exactly app, platform |
| +// and sha, from prover. |
| +func (p *Proof) Check(pub ed25519.PublicKey, app, platform, sha, prover string) error { |
| + if err := p.Verify(pub); err != nil { |
| + return err |
| + } |
| + switch { |
| + case p.App != app: |
| + return fmt.Errorf("proof is for app %s, not %s", p.App, app) |
| + case p.Platform != platform: |
| + return fmt.Errorf("proof is for platform %s, not %s", p.Platform, platform) |
| + case p.SHA256 != sha: |
| + return fmt.Errorf("proof is for sha256 %s, not %s", p.SHA256, sha) |
| + case p.Prover != prover: |
| + return fmt.Errorf("proof names prover %s, not %s", p.Prover, prover) |
| + } |
| + return nil |
| +} |
| + |
| +// FileSHA256 hashes the bytes at path (following a symlink), the one way D3 |
| +// learns a sha256 (spec §4.2, codex 3). |
| +func FileSHA256(path string) (string, error) { |
| + f, err := os.Open(path) |
| + if err != nil { |
| + return "", err |
| + } |
| + defer f.Close() |
| + h := sha256.New() |
| + if _, err := io.Copy(h, f); err != nil { |
| + return "", err |
| + } |
| + return hex.EncodeToString(h.Sum(nil)), nil |
| +} |
| + |
| +func ProofsPrefix(acct, app, sha string) string { return appPrefix(acct, app) + "proofs/" + sha + "/" } |
| +func ProofKey(acct, app, sha, prover string) string { |
| + return ProofsPrefix(acct, app, sha) + prover + ".json" |
| +} |
| +func ProofPendingKey(acct, app, sha, prover string) string { |
| + return ProofsPrefix(acct, app, sha) + prover + ".pending.json" |
| +} |
| + |
| +func GetProof(ctx context.Context, s Store, acct, app, sha, prover string) (*Proof, error) { |
| + if !sha256Re.MatchString(sha) || !ValidProver(prover) || !ValidApp(app) { |
| + return nil, fmt.Errorf("proof: invalid key %s/%s/%s", app, sha, prover) |
| + } |
| + raw, _, err := s.Get(ctx, ProofKey(acct, app, sha, prover)) |
| + if err != nil { |
| + return nil, err |
| + } |
| + return ParseProof(raw) |
| +} |
| + |
| +func PutProof(ctx context.Context, s Store, acct string, p *Proof) error { |
| + if err := p.shape(); err != nil { |
| + return err |
| + } |
| + b, err := json.MarshalIndent(p, "", " ") |
| + if err != nil { |
| + return err |
| + } |
| + return putImmutable(ctx, s, ProofKey(acct, p.App, p.SHA256, p.Prover), append(b, '\n')) |
| +} |
| + |
| +// ProofPending is the proving box's note that a listed build is live and |
| +// baking, so `carlos proof get` can say how long it has to go. Advisory: no |
| +// box reads it. |
| +type ProofPending struct { |
| + Schema int `json:"schema"` |
| + App string `json:"app"` |
| + Platform string `json:"platform"` |
| + SHA256 string `json:"sha256"` |
| + Version string `json:"version"` |
| + Prover string `json:"prover"` |
| + LiveSince time.Time `json:"live_since"` |
| + After string `json:"after"` |
| +} |
| + |
| +func PutProofPending(ctx context.Context, s Store, acct string, p ProofPending) error { |
| + if !ValidApp(p.App) || !sha256Re.MatchString(p.SHA256) || !ValidProver(p.Prover) { |
| + return fmt.Errorf("proof pending: invalid record %s/%s/%s", p.App, p.SHA256, p.Prover) |
| + } |
| + p.Schema = proofSchema |
| + b, err := json.MarshalIndent(p, "", " ") |
| + if err != nil { |
| + return err |
| + } |
| + _, err = s.Put(ctx, ProofPendingKey(acct, p.App, p.SHA256, p.Prover), append(b, '\n'), "") |
| + if errors.Is(err, ErrCASConflict) { |
| + return nil // the first sighting stands |
| + } |
| + return err |
| +} |
| + |
| +func GetProofPending(ctx context.Context, s Store, acct, app, sha, prover string) (*ProofPending, error) { |
| + if !sha256Re.MatchString(sha) || !ValidProver(prover) || !ValidApp(app) { |
| + return nil, fmt.Errorf("proof pending: invalid key %s/%s/%s", app, sha, prover) |
| + } |
| + raw, _, err := s.Get(ctx, ProofPendingKey(acct, app, sha, prover)) |
| + if err != nil { |
| + return nil, err |
| + } |
| + var p ProofPending |
| + if err := json.Unmarshal(raw, &p); err != nil { |
| + return nil, fmt.Errorf("proof pending: %v", err) |
| + } |
| + return &p, nil |
| +} |
| +``` |
| + |
| +- [ ] **Step 4: Run the tests to verify they pass** |
| + |
| +Run: `GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/release -run 'Proof|FileSHA256' -count=1` |
| +Expected: PASS. |
| + |
| +- [ ] **Step 5: Commit** |
| + |
| +```bash |
| +git add internal/release/proof.go internal/release/proof_test.go |
| +git commit -m "release: signed dev proofs, their bucket keys and the pending note (console defences D3)" |
| +``` |
| + |
| +--- |
| + |
| +### Task 2: The proof policy file |
| + |
| +**Files:** |
| +- Create: `internal/release/proofpolicy.go` |
| +- Test: `internal/release/proofpolicy_test.go` |
| + |
| +**Interfaces:** |
| +- Consumes: Task 1's `ValidProver`; `ValidApp`, `ParsePublicKey`. |
| +- Produces: |
| + - `const ProofPolicyPath = "/etc/carlos/proof-policy.json"`, `const DefaultProveAfter = 15 * time.Minute` |
| + - `type ProofPolicy struct { Schema int; Prove *ProveBlock; Require []RequireEntry }` |
| + - `type ProveBlock struct { Apps []string; Prover, After, KeyFile string }` (JSON `apps`, `prover`, `after`, `key_file`) |
| + - `type RequireEntry struct { App, Prover, Key string }` with `func (e RequireEntry) PublicKey() ed25519.PublicKey` |
| + - `func LoadProofPolicy(path string) (*ProofPolicy, error)`: `(nil, nil)` when absent, `(nil, err)` when unreadable or invalid |
| + - `func ParseProofPolicy(b []byte) (*ProofPolicy, error)` |
| + - nil-safe: `func (p *ProofPolicy) Requires(app string) []RequireEntry`, `func (p *ProofPolicy) Proves(app string) bool`, `func (p *ProofPolicy) ProveAfter() time.Duration` |
| + |
| +- [ ] **Step 1: Write the failing tests** |
| + |
| +```go |
| +package release |
| + |
| +import ( |
| + "fmt" |
| + "os" |
| + "path/filepath" |
| + "testing" |
| + "time" |
| +) |
| + |
| +func TestProofPolicyAbsentIsOff(t *testing.T) { |
| + p, err := LoadProofPolicy(filepath.Join(t.TempDir(), "proof-policy.json")) |
| + if p != nil || err != nil { |
| + t.Fatalf("absent = %+v, %v; want nil, nil (D3 off)", p, err) |
| + } |
| + if p.Requires("console") != nil || p.Proves("console") || p.ProveAfter() != DefaultProveAfter { |
| + t.Fatal("a nil policy is not inert") |
| + } |
| +} |
| + |
| +func TestProofPolicyParses(t *testing.T) { |
| + _, _, pubB64 := proverKey(t) |
| + path := filepath.Join(t.TempDir(), "proof-policy.json") |
| + body := fmt.Sprintf(`{"schema":1, |
| + "prove":{"apps":["console"],"prover":"dev","after":"20m","key_file":"/etc/carlos/proving.key"}, |
| + "require":[{"app":"console","prover":"dev","key":%q}]}`, pubB64) |
| + if err := os.WriteFile(path, []byte(body), 0o644); err != nil { |
| + t.Fatal(err) |
| + } |
| + p, err := LoadProofPolicy(path) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if !p.Proves("console") || p.Proves("other") || p.ProveAfter() != 20*time.Minute { |
| + t.Fatalf("prove block = %+v", p.Prove) |
| + } |
| + req := p.Requires("console") |
| + if len(req) != 1 || req[0].Prover != "dev" || len(req[0].PublicKey()) == 0 { |
| + t.Fatalf("require = %+v", req) |
| + } |
| + if p.Requires("other") != nil { |
| + t.Fatal("an unlisted app is required") |
| + } |
| +} |
| + |
| +func TestProofPolicyRequireOnlyUsesTheDefaultAfter(t *testing.T) { |
| + _, _, pubB64 := proverKey(t) |
| + p, err := ParseProofPolicy([]byte(fmt.Sprintf(`{"schema":1,"require":[{"app":"console","prover":"dev","key":%q}]}`, pubB64))) |
| + if err != nil || p.ProveAfter() != DefaultProveAfter { |
| + t.Fatalf("p = %+v, %v", p, err) |
| + } |
| +} |
| + |
| +// A typo must never open the gate: every one of these is an error, which the |
| +// gate turns into "on and waiting". |
| +func TestProofPolicyUnparseableIsAnError(t *testing.T) { |
| + _, _, pubB64 := proverKey(t) |
| + for name, body := range map[string]string{ |
| + "empty": ``, |
| + "torn": `{"schema":1,"require":[`, |
| + "unknown key": fmt.Sprintf(`{"schema":1,"requires":[{"app":"console","prover":"dev","key":%q}]}`, pubB64), |
| + "schema 2": `{"schema":2}`, |
| + "bad key": `{"schema":1,"require":[{"app":"console","prover":"dev","key":"nope"}]}`, |
| + "bad prover": fmt.Sprintf(`{"schema":1,"require":[{"app":"console","prover":"Dev","key":%q}]}`, pubB64), |
| + "bad after": `{"schema":1,"prove":{"apps":["console"],"prover":"dev","after":"soon","key_file":"/k"}}`, |
| + "relative key": `{"schema":1,"prove":{"apps":["console"],"prover":"dev","after":"15m","key_file":"proving.key"}}`, |
| + "no apps": `{"schema":1,"prove":{"apps":[],"prover":"dev","after":"15m","key_file":"/k"}}`, |
| + "trailing junk": `{"schema":1} {}`, |
| + } { |
| + if _, err := ParseProofPolicy([]byte(body)); err == nil { |
| + t.Fatalf("%s: parsed, want an error", name) |
| + } |
| + } |
| +} |
| +``` |
| + |
| +- [ ] **Step 2: Run them to verify they fail** |
| + |
| +Run: `GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/release -run 'ProofPolicy' -count=1` |
| +Expected: FAIL, `undefined: LoadProofPolicy`. |
| + |
| +- [ ] **Step 3: Implement `proofpolicy.go`** |
| + |
| +```go |
| +// The proof policy file — where D3 is switched on (spec |
| +// 2026-10-06-console-release-defences-design.md §4.5, codex 6). |
| +// |
| +// It is not host.env. host.env is root 0600 and reaches the agent only when |
| +// carlos-edge.service starts, while every root pass starts fresh, so an edit |
| +// there would reach one process and not the other. This file is root-owned |
| +// 0644 and holds no secrets: app names, the prover name, public keys and a |
| +// duration. The agent and root both read it at the start of every pass. |
| +// |
| +// Absent: D3 is off. Anything else that does not parse, including an unknown |
| +// key, is an error, and the gate reads an error as "on and waiting for every |
| +// named-unit route": the box cannot tell which apps were listed, and a typo |
| +// must not open the gate. |
| +package release |
| + |
| +import ( |
| + "bytes" |
| + "crypto/ed25519" |
| + "encoding/json" |
| + "errors" |
| + "fmt" |
| + "os" |
| + "path/filepath" |
| + "time" |
| +) |
| + |
| +const ( |
| + ProofPolicyPath = "/etc/carlos/proof-policy.json" |
| + DefaultProveAfter = 15 * time.Minute |
| +) |
| + |
| +type ProofPolicy struct { |
| + Schema int `json:"schema"` |
| + Prove *ProveBlock `json:"prove,omitempty"` |
| + Require []RequireEntry `json:"require,omitempty"` |
| +} |
| + |
| +// ProveBlock makes this box a prover: it mints proofs for Apps once a build |
| +// has run healthily for After, signed with the root-only key at KeyFile. |
| +type ProveBlock struct { |
| + Apps []string `json:"apps"` |
| + Prover string `json:"prover"` |
| + After string `json:"after"` |
| + KeyFile string `json:"key_file"` |
| + after time.Duration |
| +} |
| + |
| +// RequireEntry makes App on this box adopt only builds Prover has proven. |
| +type RequireEntry struct { |
| + App string `json:"app"` |
| + Prover string `json:"prover"` |
| + Key string `json:"key"` |
| + pub ed25519.PublicKey |
| +} |
| + |
| +func (e RequireEntry) PublicKey() ed25519.PublicKey { return e.pub } |
| + |
| +func LoadProofPolicy(path string) (*ProofPolicy, error) { |
| + b, err := os.ReadFile(path) |
| + if errors.Is(err, os.ErrNotExist) { |
| + return nil, nil |
| + } |
| + if err != nil { |
| + return nil, fmt.Errorf("proof policy %s: %w", path, err) |
| + } |
| + p, err := ParseProofPolicy(b) |
| + if err != nil { |
| + return nil, fmt.Errorf("proof policy %s: %w", path, err) |
| + } |
| + return p, nil |
| +} |
| + |
| +func ParseProofPolicy(b []byte) (*ProofPolicy, error) { |
| + dec := json.NewDecoder(bytes.NewReader(b)) |
| + dec.DisallowUnknownFields() |
| + var p ProofPolicy |
| + if err := dec.Decode(&p); err != nil { |
| + return nil, fmt.Errorf("does not parse: %v", err) |
| + } |
| + if dec.More() { |
| + return nil, fmt.Errorf("does not parse: data after the policy object") |
| + } |
| + if p.Schema != 1 { |
| + return nil, fmt.Errorf("schema %d, want 1", p.Schema) |
| + } |
| + if pr := p.Prove; pr != nil { |
| + if len(pr.Apps) == 0 { |
| + return nil, fmt.Errorf("prove: apps is empty") |
| + } |
| + for _, a := range pr.Apps { |
| + if !ValidApp(a) { |
| + return nil, fmt.Errorf("prove: invalid app %q", a) |
| + } |
| + } |
| + if !ValidProver(pr.Prover) { |
| + return nil, fmt.Errorf("prove: invalid prover %q", pr.Prover) |
| + } |
| + d, err := time.ParseDuration(pr.After) |
| + if err != nil || d <= 0 { |
| + return nil, fmt.Errorf("prove: after %q is not a positive duration", pr.After) |
| + } |
| + pr.after = d |
| + if !filepath.IsAbs(pr.KeyFile) { |
| + return nil, fmt.Errorf("prove: key_file %q is not an absolute path", pr.KeyFile) |
| + } |
| + } |
| + for i := range p.Require { |
| + e := &p.Require[i] |
| + if !ValidApp(e.App) { |
| + return nil, fmt.Errorf("require[%d]: invalid app %q", i, e.App) |
| + } |
| + if !ValidProver(e.Prover) { |
| + return nil, fmt.Errorf("require[%d]: invalid prover %q", i, e.Prover) |
| + } |
| + k, err := ParsePublicKey(e.Key, fmt.Sprintf("require[%d].key", i)) |
| + if err != nil { |
| + return nil, err |
| + } |
| + e.pub = k |
| + } |
| + return &p, nil |
| +} |
| + |
| +func (p *ProofPolicy) Requires(app string) []RequireEntry { |
| + if p == nil { |
| + return nil |
| + } |
| + var out []RequireEntry |
| + for _, e := range p.Require { |
| + if e.App == app { |
| + out = append(out, e) |
| + } |
| + } |
| + return out |
| +} |
| + |
| +func (p *ProofPolicy) Proves(app string) bool { |
| + if p == nil || p.Prove == nil { |
| + return false |
| + } |
| + for _, a := range p.Prove.Apps { |
| + if a == app { |
| + return true |
| + } |
| + } |
| + return false |
| +} |
| + |
| +// ProveAfter is how long a build must run before it is proven, and before a |
| +// root pass records it as healthy (spec §4.5): the prove block's after, else |
| +// DefaultProveAfter. |
| +func (p *ProofPolicy) ProveAfter() time.Duration { |
| + if p == nil || p.Prove == nil { |
| + return DefaultProveAfter |
| + } |
| + return p.Prove.after |
| +} |
| +``` |
| + |
| +- [ ] **Step 4: Run the tests to verify they pass** |
| + |
| +Run: `GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/release -run 'ProofPolicy' -count=1` |
| +Expected: PASS. |
| + |
| +- [ ] **Step 5: Commit** |
| + |
| +```bash |
| +git add internal/release/proofpolicy.go internal/release/proofpolicy_test.go |
| +git commit -m "release: the proof policy file, closed on any parse error" |
| +``` |
| + |
| +--- |
| + |
| +### Task 3: The healthy history |
| + |
| +**Files:** |
| +- Create: `internal/release/healthy.go` |
| +- Test: `internal/release/healthy_test.go` |
| + |
| +**Interfaces:** |
| +- Consumes: Task 1's `sha256Re`; `ValidAccount`, `ValidApp`. |
| +- Produces: |
| + - `type HealthyBuild struct { SHA256, Version, How string; At time.Time }` (JSON `sha256`, `version`, `how`, `at`; How is `"settle"` or `"live"`) |
| + - `func ReadHealthy(bakeDir, acct, app string) []HealthyBuild` (newest first; nil when absent or corrupt) |
| + - `func IsHealthy(bakeDir, acct, app, sha string) bool` |
| + - `func RecordHealthy(bakeDir, acct, app string, b HealthyBuild) error` |
| + |
| +- [ ] **Step 1: Write the failing tests** |
| + |
| +```go |
| +package release |
| + |
| +import ( |
| + "fmt" |
| + "os" |
| + "path/filepath" |
| + "testing" |
| + "time" |
| +) |
| + |
| +func TestHealthyRecordAndRead(t *testing.T) { |
| + bake := t.TempDir() |
| + at := time.Date(2026, 10, 6, 9, 0, 0, 0, time.UTC) |
| + b := HealthyBuild{SHA256: testSHA, Version: "f4478b15", How: "settle", At: at} |
| + if IsHealthy(bake, "bab", "console", testSHA) { |
| + t.Fatal("healthy before anything was recorded") |
| + } |
| + if err := RecordHealthy(bake, "bab", "console", b); err != nil { |
| + t.Fatal(err) |
| + } |
| + b.How, b.At = "live", at.Add(time.Hour) |
| + if err := RecordHealthy(bake, "bab", "console", b); err != nil { |
| + t.Fatal(err) |
| + } |
| + got := ReadHealthy(bake, "bab", "console") |
| + if len(got) != 1 || got[0].How != "settle" || !got[0].At.Equal(at) { |
| + t.Fatalf("history = %+v, want one entry, the first record kept", got) |
| + } |
| + if !IsHealthy(bake, "bab", "console", testSHA) || IsHealthy(bake, "bab", "other", testSHA) || IsHealthy(bake, "bab", "console", "") { |
| + t.Fatal("IsHealthy answers for the wrong app or an empty sha256") |
| + } |
| +} |
| + |
| +func TestHealthyKeepsTheNewestFifty(t *testing.T) { |
| + bake := t.TempDir() |
| + for i := 0; i < 55; i++ { |
| + if err := RecordHealthy(bake, "bab", "console", HealthyBuild{SHA256: fmt.Sprintf("%064x", i), How: "live"}); err != nil { |
| + t.Fatal(err) |
| + } |
| + } |
| + got := ReadHealthy(bake, "bab", "console") |
| + if len(got) != 50 || got[0].SHA256 != fmt.Sprintf("%064x", 54) || IsHealthy(bake, "bab", "console", fmt.Sprintf("%064x", 0)) { |
| + t.Fatalf("len = %d, first = %s; want the newest fifty", len(got), got[0].SHA256) |
| + } |
| +} |
| + |
| +func TestHealthyCorruptReadsEmpty(t *testing.T) { |
| + bake := t.TempDir() |
| + p := filepath.Join(bake, "healthy", "bab", "console.json") |
| + if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := os.WriteFile(p, []byte(`{"schema":1,"builds":[`), 0o644); err != nil { |
| + t.Fatal(err) |
| + } |
| + if got := ReadHealthy(bake, "bab", "console"); got != nil { |
| + t.Fatalf("corrupt history = %+v, want nil", got) |
| + } |
| + if err := RecordHealthy(bake, "bab", "console", HealthyBuild{SHA256: "nope"}); err == nil { |
| + t.Fatal("a non-hex sha256 was recorded") |
| + } |
| +} |
| +``` |
| + |
| +- [ ] **Step 2: Run them to verify they fail** |
| + |
| +Run: `GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/release -run 'Healthy' -count=1` |
| +Expected: FAIL, `undefined: RecordHealthy`. |
| + |
| +- [ ] **Step 3: Implement `healthy.go`** |
| + |
| +```go |
| +// The healthy history — builds this box has run well (spec |
| +// 2026-10-06-console-release-defences-design.md §4.5). A sha256 in an app's |
| +// history is adopted with no proof, which is what lets a requiring box roll |
| +// back to a build it ran before without waiting on the prover. |
| +// |
| +// Root writes it from two places in internal/host: when D1's settle check |
| +// passes for a restarted named unit (How "settle"), and when a root pass finds |
| +// a named-unit route's live build bound, active in one invocation for the |
| +// policy's after, hashed from its running executable and ready (How "live"). |
| +// Every sha256 here was hashed from bytes, never copied from a manifest. It |
| +// lives in the bake directory, which the sandboxed agent can read. |
| +package release |
| + |
| +import ( |
| + "encoding/json" |
| + "fmt" |
| + "log" |
| + "os" |
| + "path/filepath" |
| + "time" |
| +) |
| + |
| +const healthyMax = 50 |
| + |
| +type HealthyBuild struct { |
| + SHA256 string `json:"sha256"` |
| + Version string `json:"version,omitempty"` |
| + How string `json:"how"` |
| + At time.Time `json:"at"` |
| +} |
| + |
| +type healthyFile struct { |
| + Schema int `json:"schema"` |
| + Builds []HealthyBuild `json:"builds"` |
| +} |
| + |
| +func healthyPath(bakeDir, acct, app string) (string, error) { |
| + if bakeDir == "" || !ValidAccount(acct) || !ValidApp(app) { |
| + return "", fmt.Errorf("healthy history: invalid %q/%q", acct, app) |
| + } |
| + return filepath.Join(bakeDir, "healthy", acct, app+".json"), nil |
| +} |
| + |
| +func ReadHealthy(bakeDir, acct, app string) []HealthyBuild { |
| + p, err := healthyPath(bakeDir, acct, app) |
| + if err != nil { |
| + return nil |
| + } |
| + b, err := os.ReadFile(p) |
| + if err != nil { |
| + return nil |
| + } |
| + var f healthyFile |
| + if err := json.Unmarshal(b, &f); err != nil || f.Schema != 1 { |
| + log.Printf("healthy history: %s does not parse (%v), treating it as empty", p, err) |
| + return nil |
| + } |
| + return f.Builds |
| +} |
| + |
| +func IsHealthy(bakeDir, acct, app, sha string) bool { |
| + if sha == "" { |
| + return false |
| + } |
| + for _, b := range ReadHealthy(bakeDir, acct, app) { |
| + if b.SHA256 == sha { |
| + return true |
| + } |
| + } |
| + return false |
| +} |
| + |
| +// RecordHealthy adds b, newest first, keeping the first record of a sha256 |
| +// and the newest healthyMax builds. Written atomically and world-readable, |
| +// because the reader is the sandboxed agent. |
| +func RecordHealthy(bakeDir, acct, app string, b HealthyBuild) error { |
| + p, err := healthyPath(bakeDir, acct, app) |
| + if err != nil { |
| + return err |
| + } |
| + if !sha256Re.MatchString(b.SHA256) { |
| + return fmt.Errorf("healthy history: sha256 %q is not 64 lower-case hex digits", b.SHA256) |
| + } |
| + have := ReadHealthy(bakeDir, acct, app) |
| + for _, h := range have { |
| + if h.SHA256 == b.SHA256 { |
| + return nil |
| + } |
| + } |
| + builds := append([]HealthyBuild{b}, have...) |
| + if len(builds) > healthyMax { |
| + builds = builds[:healthyMax] |
| + } |
| + body, err := json.MarshalIndent(healthyFile{Schema: 1, Builds: builds}, "", " ") |
| + if err != nil { |
| + return err |
| + } |
| + if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil { |
| + return err |
| + } |
| + tmp, err := os.CreateTemp(filepath.Dir(p), ".healthy-*") |
| + if err != nil { |
| + return err |
| + } |
| + defer os.Remove(tmp.Name()) |
| + if _, err := tmp.Write(append(body, '\n')); err != nil { |
| + tmp.Close() |
| + return err |
| + } |
| + if err := tmp.Chmod(0o644); err != nil { |
| + tmp.Close() |
| + return err |
| + } |
| + if err := tmp.Close(); err != nil { |
| + return err |
| + } |
| + return os.Rename(tmp.Name(), p) |
| +} |
| +``` |
| + |
| +- [ ] **Step 4: Run the tests to verify they pass** |
| + |
| +Run: `GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/release -run 'Healthy' -count=1` |
| +Expected: PASS. |
| + |
| +- [ ] **Step 5: Commit** |
| + |
| +```bash |
| +git add internal/release/healthy.go internal/release/healthy_test.go |
| +git commit -m "release: the healthy history a requiring box adopts from without a proof" |
| +``` |
| + |
| +--- |
| + |
| +### Task 4: The proof gate in `adoptHost` |
| + |
| +**Files:** |
| +- Create: `internal/release/proofgate.go` |
| +- Modify: `internal/release/adopt.go`: `passConfig` (34) gains `proof`; new `WithProofGate` below `WithStartUnit` (246); `PassFilter` (303) calls `BeginPass` after its options loop; `adoptHost` (901) clears a wait on the `cur == ver` return (947), runs `admit` immediately after Plan 1's refusal block (before `ensureStaged`, 964), and runs `bind` immediately after Plan 2's D2 block (before the capability converge comment, 976) |
| +- Test: `internal/release/proofgate_test.go` |
| + |
| +**Interfaces:** |
| +- Consumes: Tasks 1-3; Plan 1's `WriteRefusal`, `Refusal`, `ReadRefusal`; `ReadStagedManifest` (versions.go:243), `GetManifest`, `VersionPath`, `RouteAccount`. |
| +- Produces: |
| + - `const ProofMissTTL = 30 * time.Second` |
| + - `type ProofWait struct { Host, App, Version, SHA256, Prover, Reason string; Since time.Time }` |
| + - `func NewProofGate(policyPath, bakeDir string) *ProofGate` |
| + - `func (g *ProofGate) WithClock(now func() time.Time) *ProofGate` |
| + - `func (g *ProofGate) BeginPass()` |
| + - `func (g *ProofGate) Waits() []ProofWait` (sorted by host) |
| + - `func (g *ProofGate) WaitingDue(host string) bool`: host is waiting and its 30 s miss has expired |
| + - `func WithProofGate(g *ProofGate) PassOption` |
| + - unexported: `admit(ctx, s, optDir, rt, ptr, cur) (proofVerdict, bool)`, `bind(optDir, rt, ptr, v, now) bool`, `clear(host)`. All `*ProofGate` methods are nil-safe, so `cfg.proof` nil is today's behaviour. |
| + |
| +**Manifest note (for the reviewer):** spec §4.5 places the cheap gate "after |
| +reading the new manifest and before `ensureStaged`", but `adoptHost` reads no |
| +manifest before `ensureStaged` today (ensureStaged reads it). `admit` reads the |
| +staged sibling when one exists and otherwise `GetManifest`, and caches the |
| +sha256 per (account, app, version, `PromotedAt`), which is immutable for one |
| +promotion (Plan 1 Task 1's keying note). While a route waits, its cost is one |
| +proof GET per 30 s. |
| + |
| +- [ ] **Step 1: Write the failing tests** |
| + |
| +```go |
| +package release |
| + |
| +import ( |
| + "bytes" |
| + "context" |
| + "crypto/ed25519" |
| + "encoding/json" |
| + "fmt" |
| + "os" |
| + "path/filepath" |
| + "strings" |
| + "sync" |
| + "testing" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/registry" |
| +) |
| + |
| +const proofHost = "console.example.com" |
| + |
| +// proofEnv is startUnitFixture (live "old", channel canary/x promoted to |
| +// "new" whose bytes are "bad build") plus a prover key, a policy path, a bake |
| +// directory and one gate shared by every pass, as the agent shares one. |
| +type proofEnv struct { |
| + ctx context.Context |
| + reg *registry.Registry |
| + store Store |
| + pub ed25519.PublicKey |
| + opt, bake string |
| + policy string |
| + prover ed25519.PrivateKey |
| + proverPub string |
| + newSHA string |
| + clock time.Time |
| + gate *ProofGate |
| +} |
| + |
| +func newProofEnv(t *testing.T) *proofEnv { |
| + t.Helper() |
| + ctx, reg, s, pub, opt := startUnitFixture(t, proofHost) |
| + prover, _, pubB64 := proverKey(t) |
| + e := &proofEnv{ctx: ctx, reg: reg, store: s, pub: pub, opt: opt, bake: t.TempDir(), |
| + policy: filepath.Join(t.TempDir(), "proof-policy.json"), prover: prover, proverPub: pubB64, |
| + newSHA: etagOf([]byte("bad build")), clock: time.Date(2026, 10, 6, 10, 0, 0, 0, time.UTC)} |
| + e.gate = NewProofGate(e.policy, e.bake).WithClock(func() time.Time { return e.clock }) |
| + return e |
| +} |
| + |
| +func (e *proofEnv) writePolicy(t *testing.T, body string) { |
| + t.Helper() |
| + if err := os.WriteFile(e.policy, []byte(body), 0o644); err != nil { |
| + t.Fatal(err) |
| + } |
| +} |
| + |
| +func (e *proofEnv) requireDev(t *testing.T) { |
| + e.writePolicy(t, fmt.Sprintf(`{"schema":1,"require":[{"app":"console","prover":"dev","key":%q}]}`, e.proverPub)) |
| +} |
| + |
| +func (e *proofEnv) pass(t *testing.T, s Store) string { |
| + t.Helper() |
| + var out bytes.Buffer |
| + withCapturedStdout(t, &out, func() { |
| + if err := Pass(e.ctx, e.reg, s, e.pub, e.opt, func(string) error { return nil }, |
| + BakeDir(e.bake), WithProofGate(e.gate)); err != nil { |
| + t.Fatalf("Pass: %v", err) |
| + } |
| + }) |
| + return out.String() |
| +} |
| + |
| +func (e *proofEnv) live() string { v, _ := LiveVersion(e.opt, proofHost); return v } |
| + |
| +func (e *proofEnv) signed(t *testing.T, key ed25519.PrivateKey, app, sha, prover string) *Proof { |
| + t.Helper() |
| + p, err := SignProof(key, Proof{App: app, Platform: adoptPlatform, SHA256: sha, Version: "new", Prover: prover, |
| + ActiveSince: e.clock.Add(-time.Hour), ProvenAt: e.clock}) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + return p |
| +} |
| + |
| +// putAt writes p at the dev key for sha, whatever p itself says. |
| +func (e *proofEnv) putAt(t *testing.T, sha string, p *Proof) { |
| + t.Helper() |
| + b, err := json.MarshalIndent(p, "", " ") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if _, err := e.store.Put(e.ctx, ProofKey("bab", "console", sha, "dev"), b, ""); err != nil { |
| + t.Fatal(err) |
| + } |
| +} |
| + |
| +type proofReadCountingStore struct { |
| + Store |
| + mu sync.Mutex |
| + gets []string |
| +} |
| + |
| +func (c *proofReadCountingStore) Get(ctx context.Context, key string) ([]byte, string, error) { |
| + c.mu.Lock() |
| + c.gets = append(c.gets, key) |
| + c.mu.Unlock() |
| + return c.Store.Get(ctx, key) |
| +} |
| + |
| +func (c *proofReadCountingStore) proofGets() int { |
| + c.mu.Lock() |
| + defer c.mu.Unlock() |
| + n := 0 |
| + for _, k := range c.gets { |
| + if strings.Contains(k, "/proofs/") { |
| + n++ |
| + } |
| + } |
| + return n |
| +} |
| + |
| +func TestProofNoPolicyMakesNoProofRead(t *testing.T) { |
| + e := newProofEnv(t) |
| + cs := &proofReadCountingStore{Store: e.store} |
| + e.pass(t, cs) |
| + if e.live() != "new" { |
| + t.Fatalf("live = %q, want new: with no policy file adoption is unchanged", e.live()) |
| + } |
| + if n := cs.proofGets(); n != 0 { |
| + t.Fatalf("%d proof GETs with no policy file, want 0 (spec §4.6)", n) |
| + } |
| +} |
| + |
| +func TestProofMissingStaysAndIsNotARefusal(t *testing.T) { |
| + e := newProofEnv(t) |
| + e.requireDev(t) |
| + out := e.pass(t, e.store) |
| + if e.live() != "old" { |
| + t.Fatalf("live = %q, want old", e.live()) |
| + } |
| + if !strings.Contains(out, "is not proven on dev; staying on old") { |
| + t.Fatalf("printed %q, want the not-proven line", out) |
| + } |
| + if r, _ := ReadRefusal(e.bake, proofHost); r != nil { |
| + t.Fatalf("a missing proof wrote a refusal: %+v", r) |
| + } |
| + w := e.gate.Waits() |
| + if len(w) != 1 || w[0].Host != proofHost || w[0].SHA256 != e.newSHA || w[0].Prover != "dev" { |
| + t.Fatalf("waits = %+v", w) |
| + } |
| +} |
| + |
| +func TestProofValidAdopts(t *testing.T) { |
| + e := newProofEnv(t) |
| + e.requireDev(t) |
| + e.putAt(t, e.newSHA, e.signed(t, e.prover, "console", e.newSHA, "dev")) |
| + e.pass(t, e.store) |
| + if e.live() != "new" { |
| + t.Fatalf("live = %q, want new", e.live()) |
| + } |
| + if w := e.gate.Waits(); len(w) != 0 { |
| + t.Fatalf("waits = %+v after adopting", w) |
| + } |
| +} |
| + |
| +func TestProofRejectsBadProofs(t *testing.T) { |
| + for name, mk := range map[string]func(t *testing.T, e *proofEnv) *Proof{ |
| + "bad signature": func(t *testing.T, e *proofEnv) *Proof { |
| + other, _, _ := proverKey(t) |
| + return e.signed(t, other, "console", e.newSHA, "dev") |
| + }, |
| + "wrong sha256": func(t *testing.T, e *proofEnv) *Proof { |
| + return e.signed(t, e.prover, "console", etagOf([]byte("other bytes")), "dev") |
| + }, |
| + "wrong app": func(t *testing.T, e *proofEnv) *Proof { |
| + return e.signed(t, e.prover, "other", e.newSHA, "dev") |
| + }, |
| + "wrong prover": func(t *testing.T, e *proofEnv) *Proof { |
| + return e.signed(t, e.prover, "console", e.newSHA, "staging") |
| + }, |
| + } { |
| + t.Run(name, func(t *testing.T) { |
| + e := newProofEnv(t) |
| + e.requireDev(t) |
| + e.putAt(t, e.newSHA, mk(t, e)) |
| + out := e.pass(t, e.store) |
| + if e.live() != "old" || !strings.Contains(out, "is not proven on dev") { |
| + t.Fatalf("live = %q, printed %q; want old and the not-proven line", e.live(), out) |
| + } |
| + }) |
| + } |
| +} |
| + |
| +// Review focus 3: rolling back to a build this box ran never waits on dev. |
| +func TestProofHealthyHistoryAdmitsWithoutProof(t *testing.T) { |
| + e := newProofEnv(t) |
| + e.requireDev(t) |
| + if err := RecordHealthy(e.bake, "bab", "console", HealthyBuild{SHA256: e.newSHA, Version: "new", How: "settle", At: e.clock}); err != nil { |
| + t.Fatal(err) |
| + } |
| + cs := &proofReadCountingStore{Store: e.store} |
| + e.pass(t, cs) |
| + if e.live() != "new" { |
| + t.Fatalf("live = %q, want new from the healthy history", e.live()) |
| + } |
| + if n := cs.proofGets(); n != 0 { |
| + t.Fatalf("%d proof GETs for a build in the healthy history, want 0", n) |
| + } |
| +} |
| + |
| +// Review focus 2: the manifest's sha256 is proven, but the staged file (a |
| +// cache hit, which ensureStaged does not re-hash) holds other bytes. |
| +func TestProofBindingRefusesStagedBytesThatDiffer(t *testing.T) { |
| + e := newProofEnv(t) |
| + e.requireDev(t) |
| + stale := []byte("not the shipped bytes") |
| + if err := StageArtifact(e.opt, "bab", "new", "console", stale); err != nil { |
| + t.Fatal(err) |
| + } |
| + e.putAt(t, e.newSHA, e.signed(t, e.prover, "console", e.newSHA, "dev")) |
| + out := e.pass(t, e.store) |
| + if e.live() != "old" || !strings.Contains(out, "not pointed live") { |
| + t.Fatalf("live = %q, printed %q; want old and the binding refusal", e.live(), out) |
| + } |
| + r, _ := ReadRefusal(e.bake, proofHost) |
| + if r == nil || !strings.Contains(r.Reason, etagOf(stale)) || !strings.Contains(r.Reason, e.newSHA) { |
| + t.Fatalf("refusal = %+v, want one naming both hashes", r) |
| + } |
| +} |
| + |
| +func TestProofMissIsCachedThirtySeconds(t *testing.T) { |
| + e := newProofEnv(t) |
| + e.requireDev(t) |
| + cs := &proofReadCountingStore{Store: e.store} |
| + e.pass(t, cs) |
| + e.clock = e.clock.Add(10 * time.Second) |
| + e.pass(t, cs) |
| + if n := cs.proofGets(); n != 1 { |
| + t.Fatalf("%d proof GETs inside 30s, want 1", n) |
| + } |
| + if e.gate.WaitingDue(proofHost) { |
| + t.Fatal("due inside its 30s miss") |
| + } |
| + e.clock = e.clock.Add(21 * time.Second) |
| + if !e.gate.WaitingDue(proofHost) { |
| + t.Fatal("not due after 31s") |
| + } |
| + e.pass(t, cs) |
| + if n := cs.proofGets(); n != 2 { |
| + t.Fatalf("%d proof GETs after 31s, want 2", n) |
| + } |
| +} |
| + |
| +// Review focus 1: a typo keeps the gate closed for a named-unit route. |
| +func TestProofPolicyUnparseableKeepsGateClosed(t *testing.T) { |
| + e := newProofEnv(t) |
| + if err := e.reg.SetUnit(proofHost, "console.service"); err != nil { |
| + t.Fatal(err) |
| + } |
| + e.writePolicy(t, `{"schema":1,"require":[`) |
| + out := e.pass(t, e.store) |
| + if e.live() != "old" { |
| + t.Fatalf("live = %q, want old: an unparseable policy must not open the gate", e.live()) |
| + } |
| + if !strings.Contains(out, "does not parse") { |
| + t.Fatalf("printed %q, want the policy error", out) |
| + } |
| + if w := e.gate.Waits(); len(w) != 1 || !strings.Contains(w[0].Reason, "does not parse") { |
| + t.Fatalf("waits = %+v", w) |
| + } |
| +} |
| + |
| +// The agent's gate lives across passes: a policy edit is seen on the next |
| +// pass without restarting anything. |
| +func TestProofPolicyChangeSeenNextPass(t *testing.T) { |
| + e := newProofEnv(t) |
| + e.requireDev(t) |
| + e.pass(t, e.store) |
| + if e.live() != "old" { |
| + t.Fatalf("live = %q, want old while required", e.live()) |
| + } |
| + e.writePolicy(t, `{"schema":1}`) |
| + e.pass(t, e.store) |
| + if e.live() != "new" { |
| + t.Fatalf("live = %q, want new once the require entry is gone", e.live()) |
| + } |
| +} |
| +``` |
| + |
| +- [ ] **Step 2: Run them to verify they fail** |
| + |
| +Run: `GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/release -run 'TestProof' -count=1` |
| +Expected: FAIL, `undefined: NewProofGate`. |
| + |
| +- [ ] **Step 3: Implement `proofgate.go`** |
| + |
| +```go |
| +// The dev-proof gate — the requiring half of D3 (spec |
| +// 2026-10-06-console-release-defences-design.md §4.5). |
| +// |
| +// adoptHost asks it twice. admit runs before ensureStaged, so an unproven |
| +// build is neither downloaded nor relayed to a root pass: it takes the |
| +// manifest's sha256 for this platform and admits the build when that sha256 |
| +// is in this box's healthy history, or a configured prover's proof for it |
| +// verifies. bind runs after ensureStaged and before PointLive and hashes the |
| +// staged file, because a manifest can name bytes the box is not running (a |
| +// reused version name, a stale cache): the staged hash must be the proof's or |
| +// in the healthy history, or the build is refused with a refusal record. |
| +// |
| +// A missing proof is not a refusal. The box keeps looking, so the build adopts |
| +// soon after its proof arrives. A miss is cached per (route, sha256) for |
| +// ProofMissTTL, and the gate keeps a waiting list that the agent's fast loop |
| +// (fastMembers) and `carlos status --live` read. |
| +// |
| +// One gate lives for the life of its process and is shared by every pass in |
| +// it. BeginPass re-reads the policy file, so an edit reaches the agent and |
| +// root alike without a restart. An absent file is D3 off: nothing is read or |
| +// required. A file that does not parse is D3 on and waiting for every |
| +// named-unit route, because the box cannot tell which apps were listed. |
| +package release |
| + |
| +import ( |
| + "context" |
| + "errors" |
| + "fmt" |
| + "os" |
| + "sort" |
| + "strings" |
| + "sync" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/registry" |
| +) |
| + |
| +const ( |
| + ProofMissTTL = 30 * time.Second |
| + // proofPolicyLogEvery bounds how often a broken policy file is printed |
| + // by a long-lived process. A root pass is a fresh process and prints it |
| + // every pass; the agent's 2-second loop prints it once a minute. |
| + proofPolicyLogEvery = time.Minute |
| +) |
| + |
| +// ProofWait is a host held on its current build because its channel head is |
| +// not proven (or the policy file does not parse). |
| +type ProofWait struct { |
| + Host, App, Version, SHA256, Prover, Reason string |
| + Since time.Time |
| +} |
| + |
| +type promotionKey struct { |
| + acct, app, version string |
| + promotedAt int64 |
| +} |
| + |
| +type missKey struct{ host, sha string } |
| + |
| +type ProofGate struct { |
| + policyPath string |
| + bakeDir string |
| + now func() time.Time |
| + |
| + mu sync.Mutex |
| + policy *ProofPolicy |
| + policyErr error |
| + lastPolicyErr string |
| + lastPolicyLog time.Time |
| + shas map[promotionKey]string |
| + misses map[missKey]time.Time |
| + waits map[string]ProofWait |
| +} |
| + |
| +func NewProofGate(policyPath, bakeDir string) *ProofGate { |
| + return &ProofGate{policyPath: policyPath, bakeDir: bakeDir, now: time.Now, |
| + shas: map[promotionKey]string{}, misses: map[missKey]time.Time{}, waits: map[string]ProofWait{}} |
| +} |
| + |
| +// WithClock replaces the gate's clock (tests). |
| +func (g *ProofGate) WithClock(now func() time.Time) *ProofGate { |
| + g.now = now |
| + return g |
| +} |
| + |
| +// BeginPass re-reads the policy file. PassFilter calls it once per pass. |
| +func (g *ProofGate) BeginPass() { |
| + if g == nil { |
| + return |
| + } |
| + pol, err := LoadProofPolicy(g.policyPath) |
| + now := g.now() |
| + say := false |
| + g.mu.Lock() |
| + g.policy, g.policyErr = pol, err |
| + if err != nil { |
| + msg := err.Error() |
| + say = msg != g.lastPolicyErr || now.Sub(g.lastPolicyLog) >= proofPolicyLogEvery |
| + if say { |
| + g.lastPolicyErr, g.lastPolicyLog = msg, now |
| + } |
| + } else { |
| + g.lastPolicyErr = "" |
| + } |
| + g.mu.Unlock() |
| + if say { |
| + fmt.Printf(" %v; every named-unit route waits until it is fixed\n", err) |
| + } |
| +} |
| + |
| +func (g *ProofGate) state() (*ProofPolicy, error) { |
| + g.mu.Lock() |
| + defer g.mu.Unlock() |
| + return g.policy, g.policyErr |
| +} |
| + |
| +func (g *ProofGate) Waits() []ProofWait { |
| + if g == nil { |
| + return nil |
| + } |
| + g.mu.Lock() |
| + out := make([]ProofWait, 0, len(g.waits)) |
| + for _, w := range g.waits { |
| + out = append(out, w) |
| + } |
| + g.mu.Unlock() |
| + sort.Slice(out, func(i, j int) bool { return out[i].Host < out[j].Host }) |
| + return out |
| +} |
| + |
| +// WaitingDue reports whether host is waiting and its cached miss has run |
| +// out, which is when the fast loop should look at it again. |
| +func (g *ProofGate) WaitingDue(host string) bool { |
| + if g == nil { |
| + return false |
| + } |
| + now := g.now() |
| + g.mu.Lock() |
| + defer g.mu.Unlock() |
| + w, ok := g.waits[host] |
| + if !ok { |
| + return false |
| + } |
| + until, cached := g.misses[missKey{host, w.SHA256}] |
| + return !cached || !now.Before(until) |
| +} |
| + |
| +func (g *ProofGate) clear(host string) { |
| + if g == nil { |
| + return |
| + } |
| + g.mu.Lock() |
| + delete(g.waits, host) |
| + g.mu.Unlock() |
| +} |
| + |
| +// hold records host as waiting, caches the miss, and prints the line once |
| +// per change of version, sha256 or reason. |
| +func (g *ProofGate) hold(rt registry.Route, ver, sha, prover, reason, cur string) { |
| + if cur == "" { |
| + cur = "nothing" |
| + } |
| + now := g.now() |
| + g.mu.Lock() |
| + prev, had := g.waits[rt.Host] |
| + same := had && prev.Version == ver && prev.SHA256 == sha |
| + w := ProofWait{Host: rt.Host, App: rt.App, Version: ver, SHA256: sha, Prover: prover, Reason: reason, Since: now} |
| + if same { |
| + w.Since = prev.Since |
| + } |
| + g.waits[rt.Host] = w |
| + g.misses[missKey{rt.Host, sha}] = now.Add(ProofMissTTL) |
| + g.mu.Unlock() |
| + if same && prev.Reason == reason { |
| + return |
| + } |
| + if sha == "" { |
| + fmt.Printf(" %s: %s is waiting: %s; staying on %s\n", rt.Host, ver, reason, cur) |
| + return |
| + } |
| + fmt.Printf(" %s: %s (sha256 %s) is not proven on %s; staying on %s (%s)\n", rt.Host, ver, sha, prover, cur, reason) |
| +} |
| + |
| +// proofVerdict is admit's answer, carried to bind. |
| +type proofVerdict struct { |
| + required bool // the route's app is listed in a require entry |
| + sha string // the manifest's sha256 for this platform |
| + proof *Proof // the verified proof; nil when admitted by the healthy history |
| +} |
| + |
| +// admit is the cheap gate (spec §4.5 check 1). false means adoptHost returns. |
| +func (g *ProofGate) admit(ctx context.Context, s Store, optDir string, rt registry.Route, ptr *ChannelPointer, cur string) (proofVerdict, bool) { |
| + if g == nil { |
| + return proofVerdict{}, true |
| + } |
| + pol, perr := g.state() |
| + if perr != nil { |
| + if rt.Unit == "" { |
| + return proofVerdict{}, true |
| + } |
| + g.hold(rt, ptr.Version, "", "", "the proof policy file does not parse", cur) |
| + return proofVerdict{}, false |
| + } |
| + reqs := pol.Requires(rt.App) |
| + if len(reqs) == 0 { |
| + g.clear(rt.Host) |
| + return proofVerdict{}, true |
| + } |
| + acct := RouteAccount(rt) |
| + sha, err := g.manifestSHA(ctx, s, optDir, acct, rt.App, ptr) |
| + if err != nil { |
| + fmt.Printf(" %s: %s: reading its manifest for the proof check: %v\n", rt.Host, ptr.Version, err) |
| + return proofVerdict{}, false |
| + } |
| + v := proofVerdict{required: true, sha: sha} |
| + if sha == "" { |
| + return v, true // no build for this platform: ensureStaged says so |
| + } |
| + if IsHealthy(g.bakeDir, acct, rt.App, sha) { |
| + g.clear(rt.Host) |
| + return v, true |
| + } |
| + g.mu.Lock() |
| + until, cached := g.misses[missKey{rt.Host, sha}] |
| + g.mu.Unlock() |
| + if cached && g.now().Before(until) { |
| + return v, false |
| + } |
| + var reasons, names []string |
| + for _, e := range reqs { |
| + names = append(names, e.Prover) |
| + p, err := GetProof(ctx, s, acct, rt.App, sha, e.Prover) |
| + if errors.Is(err, os.ErrNotExist) { |
| + reasons = append(reasons, "no proof from "+e.Prover) |
| + continue |
| + } |
| + if err == nil { |
| + err = p.Check(e.PublicKey(), rt.App, adoptPlatform, sha, e.Prover) |
| + } |
| + if err != nil { |
| + reasons = append(reasons, fmt.Sprintf("proof from %s: %v", e.Prover, err)) |
| + continue |
| + } |
| + g.clear(rt.Host) |
| + v.proof = p |
| + return v, true |
| + } |
| + g.hold(rt, ptr.Version, sha, strings.Join(names, " or "), strings.Join(reasons, "; "), cur) |
| + return v, false |
| +} |
| + |
| +// manifestSHA is the manifest's sha256 for this platform, from the staged |
| +// sibling when there is one and the bucket otherwise, cached per promotion. |
| +func (g *ProofGate) manifestSHA(ctx context.Context, s Store, optDir, acct, app string, ptr *ChannelPointer) (string, error) { |
| + k := promotionKey{acct, app, ptr.Version, ptr.PromotedAt.Unix()} |
| + g.mu.Lock() |
| + sha, ok := g.shas[k] |
| + g.mu.Unlock() |
| + if ok { |
| + return sha, nil |
| + } |
| + m, err := ReadStagedManifest(optDir, acct, ptr.Version, app) |
| + if err != nil || m == nil { |
| + m, err = GetManifest(ctx, s, acct, app, ptr.Version) |
| + if err != nil { |
| + return "", err |
| + } |
| + } |
| + sha = m.Artifacts[adoptPlatform].SHA256 |
| + g.mu.Lock() |
| + g.shas[k] = sha |
| + g.mu.Unlock() |
| + return sha, nil |
| +} |
| + |
| +// bind is the staged-bytes check (spec §4.5 check 2). false means adoptHost |
| +// returns; a mismatch is a refusal record naming both hashes. |
| +func (g *ProofGate) bind(optDir string, rt registry.Route, ptr *ChannelPointer, v proofVerdict, now time.Time) bool { |
| + if g == nil || !v.required { |
| + return true |
| + } |
| + acct := RouteAccount(rt) |
| + got, err := FileSHA256(VersionPath(optDir, acct, ptr.Version, rt.App)) |
| + if err != nil { |
| + fmt.Printf(" %s: %s: hashing the staged build for the proof check: %v; not pointed live\n", rt.Host, ptr.Version, err) |
| + return false |
| + } |
| + if v.proof != nil && got == v.proof.SHA256 { |
| + return true |
| + } |
| + if IsHealthy(g.bakeDir, acct, rt.App, got) { |
| + return true |
| + } |
| + want := v.sha |
| + if v.proof != nil { |
| + want = v.proof.SHA256 |
| + } |
| + reason := fmt.Sprintf("the staged bytes are sha256 %s, not the proven %s", got, want) |
| + fmt.Printf(" %s: %s: %s; not pointed live\n", rt.Host, ptr.Version, reason) |
| + if g.bakeDir != "" { |
| + if err := WriteRefusal(g.bakeDir, Refusal{Host: rt.Host, App: rt.App, Version: ptr.Version, SHA256: got, |
| + Platform: adoptPlatform, PointerPromotedAt: ptr.PromotedAt, RefusedAt: now.UTC(), |
| + Reason: "not proven: " + reason}); err != nil { |
| + fmt.Printf(" %s: writing refusal record: %v\n", rt.Host, err) |
| + } |
| + } |
| + g.clear(rt.Host) |
| + return false |
| +} |
| +``` |
| + |
| +- [ ] **Step 4: Wire it into `adopt.go`** |
| + |
| +In `passConfig`, after `startUnit`: |
| + |
| +```go |
| + // proof is D3's gate (WithProofGate, proofgate.go). nil, the default for |
| + // every existing caller and test, means no proof is read or required. |
| + proof *ProofGate |
| +``` |
| + |
| +Below `WithStartUnit`: |
| + |
| +```go |
| +// WithProofGate arms D3 (proofgate.go): a build of an app the policy file |
| +// lists is adopted only with a verified proof for its bytes, or from this |
| +// box's healthy history. Pass the SAME gate to every pass of a process, so |
| +// its 30 s miss cache and its waiting list survive between ticks. |
| +func WithProofGate(g *ProofGate) PassOption { |
| + return func(c *passConfig) { c.proof = g } |
| +} |
| +``` |
| + |
| +In `PassFilter`, right after the `for _, opt := range opts { opt(&cfg) }` loop: |
| + |
| +```go |
| + // D3: re-read the proof policy at the start of every pass, so an edit |
| + // reaches the agent and root alike without a restart (spec §4.5). |
| + cfg.proof.BeginPass() |
| +``` |
| + |
| +In `adoptHost`, replace |
| + |
| +```go |
| + if cur == ver { |
| + return // up to date — stay quiet on the periodic tick |
| + } |
| +``` |
| + |
| +with |
| + |
| +```go |
| + if cur == ver { |
| + cfg.proof.clear(rt.Host) // live on its head: nothing left to wait for |
| + return // up to date — stay quiet on the periodic tick |
| + } |
| +``` |
| + |
| +Immediately after Plan 1's refusal block (the `if cfg.bakeDir != "" { if r, err := ReadRefusal(...` block that prints `refused: ...`), and before `m, err := ensureStaged(...)`, insert: |
| + |
| +```go |
| + // D3's cheap gate (proofgate.go, spec §4.5 check 1): before ensureStaged, |
| + // so an unproven build is neither downloaded nor relayed to root. |
| + verdict, admitted := cfg.proof.admit(ctx, s, optDir, rt, ptr, cur) |
| + if !admitted { |
| + return |
| + } |
| +``` |
| + |
| +Immediately after Plan 2's D2 block (the `if cfg.bakeDir != "" && newProg != "" { ... }` block that prints `not pointed live (ship with --program-change ...)`), and before the `// The capability converge for the NEW version` comment, insert: |
| + |
| +```go |
| + // D3's binding check (spec §4.5 check 2): the staged bytes themselves |
| + // must be the proven ones, or a build this box has run well. |
| + if !cfg.proof.bind(optDir, rt, ptr, verdict, now) { |
| + return |
| + } |
| +``` |
| + |
| +- [ ] **Step 5: Run the release suite** |
| + |
| +Run: `GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/release -count=1` |
| +Expected: PASS, every new test and every existing one: no existing caller passes `WithProofGate`, and every gate method is nil-safe. |
| + |
| +- [ ] **Step 6: Commit** |
| + |
| +```bash |
| +git add internal/release/proofgate.go internal/release/proofgate_test.go internal/release/adopt.go |
| +git commit -m "release: adoptHost requires a dev proof for listed apps, checked before staging and on the staged bytes" |
| +``` |
| + |
| +--- |
| + |
| +### Task 5: A running unit's build, from its bytes |
| + |
| +**Files:** |
| +- Modify: `internal/host/unitsettle.go` (Plan 1): `unitProps` gains `ActiveEnterTimestamp`; `systemdUnits.Show` asks for it; `parseShow` reads it |
| +- Modify: `internal/host/agent.go`: `checkedUnitRestart` (Plan 1 Task 6) gains a `settled` hook; its two agent-loop callers (adoptLoop's and fastConvergeLoop's `makeRestartRoute` calls) pass `nil` |
| +- Modify: `internal/host/unitsettle_test.go`: Plan 1's three `checkedUnitRestart(...)` calls (`TestNamedRestartBindingMismatchKeepsTheError`, `TestNamedRestartShowFailsFallsBack`, `TestNamedRestartStoppedUnitIsNotChecked`) gain a final `nil` argument |
| +- Create: `internal/host/proofobserve.go` |
| +- Test: `internal/host/proofobserve_test.go` |
| + |
| +**Interfaces:** |
| +- Consumes: Plan 1's `unitSystem`, `unitProps`, `parseShow`, `settleUnit`, `routeReady`, `fakeUnits`, `liveTo`; Tasks 1 and 3's `release.FileSHA256`, `release.RecordHealthy`, `release.ReadHealthy`. |
| +- Produces: |
| + - `unitProps.ActiveEnterTimestamp time.Time` |
| + - `func parseSystemdTime(v string) time.Time` |
| + - `func checkedUnitRestart(sys unitSystem, optDir string, getenv func(string) string, settled func(rt registry.Route)) func(rt registry.Route) error` (changes Plan 1's signature: `settled` runs only when the settle check passed) |
| + - `type liveBuild struct { SHA256, Version string; ActiveSince time.Time }` |
| + - `type proofObserver struct { sys unitSystem; procExe func(pid int) string; optDir string; now func() time.Time; ready func(rt registry.Route) func() error }` |
| + - `func (o proofObserver) observe(rt registry.Route, after time.Duration) (liveBuild, error)` |
| + - `func productionProofObserver(optDir string) proofObserver` |
| + - `func procExePath(pid int) string` |
| + - `func recordSettled(sys unitSystem, procExe func(pid int) string, optDir, bakeDir string, now func() time.Time) func(rt registry.Route)` |
| + |
| +- [ ] **Step 1: Write the failing tests** |
| + |
| +```go |
| +package host |
| + |
| +import ( |
| + "errors" |
| + "net" |
| + "os" |
| + "path/filepath" |
| + "strings" |
| + "testing" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/registry" |
| + "github.com/carlosframework/platform/internal/release" |
| +) |
| + |
| +func TestParseShowReadsActiveEnterTimestamp(t *testing.T) { |
| + p := parseShow("ActiveState=active\nActiveEnterTimestamp=Mon 2026-10-06 09:30:30 UTC\n") |
| + if want := time.Date(2026, 10, 6, 9, 30, 30, 0, time.UTC); !p.ActiveEnterTimestamp.Equal(want) { |
| + t.Fatalf("ActiveEnterTimestamp = %v, want %v", p.ActiveEnterTimestamp, want) |
| + } |
| + if !parseShow("ActiveEnterTimestamp=\n").ActiveEnterTimestamp.IsZero() { |
| + t.Fatal("an empty timestamp is not zero") |
| + } |
| +} |
| + |
| +const observedHost = "platform.example.com" |
| + |
| +// observeFixture: the route's live link names versions/v1/console, and by |
| +// default the unit runs that very file, bound, active for 20 minutes, ready. |
| +func observeFixture(t *testing.T) (proofObserver, registry.Route, string, time.Time) { |
| + t.Helper() |
| + opt := t.TempDir() |
| + liveTo(t, opt, observedHost, "v1") |
| + staged := filepath.Join(opt, "versions", "v1", "console") |
| + now := time.Date(2026, 10, 6, 10, 0, 0, 0, time.UTC) |
| + o := proofObserver{ |
| + sys: &fakeUnits{shows: []unitProps{{ActiveState: "active", InvocationID: "i1", MainPID: 7, |
| + ExecPath: release.LiveLink(opt, observedHost), ActiveEnterTimestamp: now.Add(-20 * time.Minute)}}}, |
| + procExe: func(int) string { return staged }, |
| + optDir: opt, |
| + now: func() time.Time { return now }, |
| + ready: func(registry.Route) func() error { return nil }, |
| + } |
| + rt := registry.Route{Host: observedHost, App: "console", Unit: "console.service"} |
| + return o, rt, staged, now |
| +} |
| + |
| +func TestProofObserveAcceptsABakedBuild(t *testing.T) { |
| + o, rt, staged, now := observeFixture(t) |
| + b, err := o.observe(rt, 15*time.Minute) |
| + if err != nil { |
| + t.Fatalf("observe: %v", err) |
| + } |
| + want, _ := release.FileSHA256(staged) |
| + if b.SHA256 != want || b.Version != "v1" || !b.ActiveSince.Equal(now.Add(-20*time.Minute)) { |
| + t.Fatalf("build = %+v", b) |
| + } |
| +} |
| + |
| +// Spec §7: the mint conditions, each false in turn. |
| +func TestProofObserveConditionsEachFalse(t *testing.T) { |
| + for name, c := range map[string]struct { |
| + edit func(t *testing.T, o *proofObserver, now time.Time) |
| + want string |
| + }{ |
| + "binding": {func(t *testing.T, o *proofObserver, now time.Time) { |
| + o.sys.(*fakeUnits).shows[0].ExecPath = release.LiveLink(o.optDir, "console.example.com") |
| + }, "binding"}, |
| + "not active": {func(t *testing.T, o *proofObserver, now time.Time) { |
| + o.sys.(*fakeUnits).shows[0].ActiveState = "failed" |
| + }, "failed"}, |
| + "too young": {func(t *testing.T, o *proofObserver, now time.Time) { |
| + o.sys.(*fakeUnits).shows[0].ActiveEnterTimestamp = now.Add(-5 * time.Minute) |
| + }, "needs 15m"}, |
| + "restarted during the check": {func(t *testing.T, o *proofObserver, now time.Time) { |
| + f := o.sys.(*fakeUnits) |
| + second := f.shows[0] |
| + second.InvocationID = "i2" |
| + f.shows = append(f.shows, second) |
| + }, "restarted"}, |
| + "exe and live link differ": {func(t *testing.T, o *proofObserver, now time.Time) { |
| + other := filepath.Join(t.TempDir(), "console") |
| + if err := os.WriteFile(other, []byte("a different build"), 0o755); err != nil { |
| + t.Fatal(err) |
| + } |
| + o.procExe = func(int) string { return other } |
| + }, "is not the live link's build"}, |
| + "not ready": {func(t *testing.T, o *proofObserver, now time.Time) { |
| + o.ready = func(registry.Route) func() error { return func() error { return errors.New("status 503") } } |
| + }, "not ready"}, |
| + } { |
| + t.Run(name, func(t *testing.T) { |
| + o, rt, _, now := observeFixture(t) |
| + c.edit(t, &o, now) |
| + if _, err := o.observe(rt, 15*time.Minute); err == nil || !strings.Contains(err.Error(), c.want) { |
| + t.Fatalf("observe = %v, want an error containing %q", err, c.want) |
| + } |
| + }) |
| + } |
| +} |
| + |
| +// D1's settle passing is the hook that records a healthy build, hashed from |
| +// the executable that passed. |
| +func TestSettledRestartRecordsTheRunningExe(t *testing.T) { |
| + opt, bake := t.TempDir(), t.TempDir() |
| + liveTo(t, opt, observedHost, "new") |
| + staged, err := filepath.EvalSymlinks(filepath.Join(opt, "versions", "new", "console")) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + ln, err := net.Listen("tcp", "127.0.0.1:0") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + t.Cleanup(func() { ln.Close() }) |
| + rt := registry.Route{Host: observedHost, App: "console", Unit: "console.service", Addr: ln.Addr().String()} |
| + f := &fakeUnits{shows: []unitProps{ |
| + {ActiveState: "active", InvocationID: "old", ExecPath: filepath.Join(opt, "live", observedHost)}, |
| + {ActiveState: "active", InvocationID: "new", MainPID: 7}, |
| + }, exe: map[int]string{7: staged}} |
| + settled := recordSettled(f, func(int) string { return staged }, opt, bake, time.Now) |
| + if err := checkedUnitRestart(f, opt, func(string) string { return "" }, settled)(rt); err != nil { |
| + t.Fatalf("restart: %v", err) |
| + } |
| + sha, _ := release.FileSHA256(staged) |
| + h := release.ReadHealthy(bake, "bab", "console") |
| + if len(h) != 1 || h[0].SHA256 != sha || h[0].How != "settle" || h[0].Version != "new" { |
| + t.Fatalf("history = %+v, want the settled build", h) |
| + } |
| +} |
| + |
| +func TestUnsettledRestartRecordsNothing(t *testing.T) { |
| + opt, bake := t.TempDir(), t.TempDir() |
| + liveTo(t, opt, observedHost, "new") |
| + rt := registry.Route{Host: observedHost, App: "console", Unit: "console.service"} |
| + f := &fakeUnits{shows: []unitProps{{ActiveState: "active", InvocationID: "old", |
| + ExecPath: filepath.Join(opt, "live", "console.example.com")}}} |
| + settled := recordSettled(f, func(int) string { return "/nonexistent" }, opt, bake, time.Now) |
| + if err := checkedUnitRestart(f, opt, func(string) string { return "" }, settled)(rt); err != nil { |
| + t.Fatalf("restart: %v", err) |
| + } |
| + if h := release.ReadHealthy(bake, "bab", "console"); len(h) != 0 { |
| + t.Fatalf("a restart that was not checked recorded %+v", h) |
| + } |
| +} |
| +``` |
| + |
| +- [ ] **Step 2: Run them to verify they fail** |
| + |
| +Run: `GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/host -run 'ActiveEnterTimestamp|ProofObserve|SettledRestart|UnsettledRestart' -count=1` |
| +Expected: FAIL, `unknown field ActiveEnterTimestamp in struct literal of type unitProps`. |
| + |
| +- [ ] **Step 3: Extend `unitsettle.go`** |
| + |
| +In `unitProps`, after `ExecPath`: |
| + |
| +```go |
| + ActiveEnterTimestamp time.Time // when the unit last entered active; zero when unknown |
| +``` |
| + |
| +In `systemdUnits.Show`, add `"-p", "ActiveEnterTimestamp"` after `"-p", "ExecStart"`. |
| + |
| +In `parseShow`'s switch, add: |
| + |
| +```go |
| + case "ActiveEnterTimestamp": |
| + p.ActiveEnterTimestamp = parseSystemdTime(v) |
| +``` |
| + |
| +and below `parseShow`: |
| + |
| +```go |
| +// parseSystemdTime reads systemctl show's default timestamp form, |
| +// "Mon 2026-10-06 09:30:30 UTC". Boxes run in UTC. "" and "n/a" are zero, |
| +// which every caller reads as "unknown", never as "long ago". |
| +func parseSystemdTime(v string) time.Time { |
| + t, err := time.Parse("Mon 2006-01-02 15:04:05 MST", strings.TrimSpace(v)) |
| + if err != nil { |
| + return time.Time{} |
| + } |
| + return t |
| +} |
| +``` |
| + |
| +- [ ] **Step 4: Add the hook to `checkedUnitRestart`** |
| + |
| +Change its signature to: |
| + |
| +```go |
| +func checkedUnitRestart(sys unitSystem, optDir string, getenv func(string) string, settled func(rt registry.Route)) func(rt registry.Route) error { |
| +``` |
| + |
| +and replace its success block |
| + |
| +```go |
| + if err == nil { |
| + log.Printf("adopt: %s: %s settled on %s", rt.Host, rt.Unit, filepath.Dir(want)) |
| + } |
| + return err |
| +``` |
| + |
| +with |
| + |
| +```go |
| + if err == nil { |
| + log.Printf("adopt: %s: %s settled on %s", rt.Host, rt.Unit, filepath.Dir(want)) |
| + // D3 (proofobserve.go): a build that passed the settle check is a |
| + // build this box has run well. |
| + if settled != nil { |
| + settled(rt) |
| + } |
| + } |
| + return err |
| +``` |
| + |
| +In adoptLoop and fastConvergeLoop, the two `checkedUnitRestart(systemdUnits{}, optDir, os.Getenv)` calls Plan 1 Task 6 wrote become `checkedUnitRestart(systemdUnits{}, optDir, os.Getenv, nil)`: the sandboxed agent cannot hash another unit's executable, and spec §4.5 has root record the history. Add the `nil` to Plan 1's three test calls listed under Files. |
| + |
| +- [ ] **Step 5: Implement `proofobserve.go`** |
| + |
| +```go |
| +// What a named unit is running, established from the bytes — D3 of the |
| +// console release defences (spec 2026-10-06-console-release-defences-design.md |
| +// §4.2, §4.3, §4.5). |
| +// |
| +// observe checks the first three of spec §4.3's mint conditions: the binding |
| +// is verified, the unit is active in one invocation for at least `after`, the |
| +// sha256 of /proc/<MainPID>/exe equals the live link's bytes, and the route is |
| +// ready now. The root pass mints on it (proofroot.go) and records the build |
| +// in the healthy history on it. recordSettled is the other way into that |
| +// history: the hook D1's checkedUnitRestart runs when the settle check passes. |
| +package host |
| + |
| +import ( |
| + "fmt" |
| + "log" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/registry" |
| + "github.com/carlosframework/platform/internal/release" |
| +) |
| + |
| +type liveBuild struct { |
| + SHA256 string |
| + Version string |
| + ActiveSince time.Time |
| +} |
| + |
| +type proofObserver struct { |
| + sys unitSystem |
| + procExe func(pid int) string // "/proc/<pid>/exe" in production |
| + optDir string |
| + now func() time.Time |
| + ready func(rt registry.Route) func() error // routeReady in production; nil func = no upstream |
| +} |
| + |
| +func productionProofObserver(optDir string) proofObserver { |
| + return proofObserver{sys: systemdUnits{}, procExe: procExePath, optDir: optDir, now: time.Now, |
| + ready: func(rt registry.Route) func() error { return routeReady(rt, 2*time.Second) }} |
| +} |
| + |
| +func procExePath(pid int) string { return fmt.Sprintf("/proc/%d/exe", pid) } |
| + |
| +func (o proofObserver) observe(rt registry.Route, after time.Duration) (liveBuild, error) { |
| + link := release.LiveLink(o.optDir, rt.Host) |
| + before, err := o.sys.Show(rt.Unit) |
| + if err != nil { |
| + return liveBuild{}, fmt.Errorf("cannot read %s: %v", rt.Unit, err) |
| + } |
| + if before.ExecPath != link { |
| + return liveBuild{}, fmt.Errorf("binding: %s runs %s, not %s", rt.Unit, before.ExecPath, link) |
| + } |
| + if before.ActiveState != "active" || before.MainPID <= 0 { |
| + return liveBuild{}, fmt.Errorf("%s is %s", rt.Unit, before.ActiveState) |
| + } |
| + if before.ActiveEnterTimestamp.IsZero() { |
| + return liveBuild{}, fmt.Errorf("%s reports no ActiveEnterTimestamp", rt.Unit) |
| + } |
| + if up := o.now().Sub(before.ActiveEnterTimestamp); up < after { |
| + return liveBuild{}, fmt.Errorf("%s has been active for %s, needs %s", rt.Unit, up.Round(time.Second), after) |
| + } |
| + exe, err := release.FileSHA256(o.procExe(before.MainPID)) |
| + if err != nil { |
| + return liveBuild{}, fmt.Errorf("hashing the running executable: %v", err) |
| + } |
| + live, err := release.FileSHA256(link) |
| + if err != nil { |
| + return liveBuild{}, fmt.Errorf("hashing the live link's build: %v", err) |
| + } |
| + if exe != live { |
| + return liveBuild{}, fmt.Errorf("the running executable (sha256 %s) is not the live link's build (sha256 %s)", exe, live) |
| + } |
| + if ready := o.ready(rt); ready != nil { |
| + if err := ready(); err != nil { |
| + return liveBuild{}, fmt.Errorf("not ready: %v", err) |
| + } |
| + } |
| + again, err := o.sys.Show(rt.Unit) |
| + if err != nil || again.ActiveState != "active" || again.InvocationID != before.InvocationID || again.MainPID != before.MainPID { |
| + return liveBuild{}, fmt.Errorf("%s restarted during the check", rt.Unit) |
| + } |
| + ver, _ := release.LiveVersion(o.optDir, rt.Host) |
| + return liveBuild{SHA256: exe, Version: ver, ActiveSince: before.ActiveEnterTimestamp}, nil |
| +} |
| + |
| +// recordSettled is checkedUnitRestart's settled hook on the root pass: hash |
| +// the executable that just passed the settle check and add it to the app's |
| +// healthy history. Every failure is a log line; a missed record costs only a |
| +// proof wait later. |
| +func recordSettled(sys unitSystem, procExe func(pid int) string, optDir, bakeDir string, now func() time.Time) func(rt registry.Route) { |
| + return func(rt registry.Route) { |
| + p, err := sys.Show(rt.Unit) |
| + if err != nil || p.ActiveState != "active" || p.MainPID <= 0 { |
| + log.Printf("adopt: %s: settled, but cannot read %s's process for the healthy history (%v)", rt.Host, rt.Unit, err) |
| + return |
| + } |
| + sha, err := release.FileSHA256(procExe(p.MainPID)) |
| + if err != nil { |
| + log.Printf("adopt: %s: settled, but hashing its executable failed: %v", rt.Host, err) |
| + return |
| + } |
| + ver, _ := release.LiveVersion(optDir, rt.Host) |
| + if err := release.RecordHealthy(bakeDir, release.RouteAccount(rt), rt.App, |
| + release.HealthyBuild{SHA256: sha, Version: ver, How: "settle", At: now().UTC()}); err != nil { |
| + log.Printf("adopt: %s: recording the healthy build: %v", rt.Host, err) |
| + } |
| + } |
| +} |
| +``` |
| + |
| +- [ ] **Step 6: Run the host suite** |
| + |
| +Run: `GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false nice -n 10 go test ./internal/host -count=1` |
| +Expected: PASS. `TestSettledRestartRecordsTheRunningExe` settles on its first poll (the TCP listener answers the connect), so it takes well under a second. |
| + |
| +- [ ] **Step 7: Commit** |
| + |
| +```bash |
| +git add internal/host/unitsettle.go internal/host/unitsettle_test.go internal/host/agent.go internal/host/proofobserve.go internal/host/proofobserve_test.go |
| +git commit -m "host: establish a unit's running build from its bytes, and record settled builds as healthy" |
| +``` |
| + |
| +--- |
| + |
| +### Task 6: The root pass mints, records live builds, and says when the requirement is off |
| + |
| +**Files:** |
| +- Create: `internal/host/proofroot.go` |
| +- Modify: `internal/host/agent.go`: `rootRestartRoute` (3242) gains a `settled` parameter; `RunAdopt` (3456): its `rootRestartRoute` call (3536), its `passOpts` (3660), and a `proofRoot` run right after `release.Pass` (3665) |
| +- Modify: `internal/host/agent_test.go`: the three `rootRestartRoute(...)` calls (lines 718, 836, 963) gain a final `nil` argument |
| +- Test: `internal/host/proofroot_test.go` |
| + |
| +**Interfaces:** |
| +- Consumes: Task 5's `proofObserver`, `liveBuild`, `recordSettled`, `procExePath`, `productionProofObserver`; Tasks 1-4 (`release.SignProof`, `GetProof`, `PutProof`, `AdoptPlatform`, `LoadProofPolicy`, `ProofPolicyPath`, `RecordHealthy`, `ReadHealthy`, `NewProofGate`, `WithProofGate`); Plan 1's `release.ListRefusals`. |
| +- Produces: |
| + - `type proofRoot struct { obs proofObserver; reg *registry.Registry; store release.Store; bakeDir, policyPath string; readKey func(path string) (ed25519.PrivateKey, error); hourly bool; logf func(format string, a ...any) }` |
| + - `func (r proofRoot) run(ctx context.Context)` |
| + - `func readProvingKey(path string) (ed25519.PrivateKey, error)` (refuses a key file any non-owner can read) |
| + - `func rootRestartRoute(reg *registry.Registry, optDir string, settled func(rt registry.Route)) func(host string) error` |
| + |
| +- [ ] **Step 1: Write the failing tests** |
| + |
| +```go |
| +package host |
| + |
| +import ( |
| + "context" |
| + "crypto/ed25519" |
| + "fmt" |
| + "os" |
| + "path/filepath" |
| + "strings" |
| + "testing" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/registry" |
| + "github.com/carlosframework/platform/internal/release" |
| +) |
| + |
| +type proofRootEnv struct { |
| + root proofRoot |
| + store release.Store |
| + bake string |
| + keyFile string |
| + pub ed25519.PublicKey |
| + sha string |
| + now time.Time |
| + logs []string |
| +} |
| + |
| +func provePolicy(keyFile string) string { |
| + return fmt.Sprintf(`{"schema":1,"prove":{"apps":["console"],"prover":"dev","after":"15m","key_file":%q}}`, keyFile) |
| +} |
| + |
| +// newProofRootEnv: one named-unit console route, bound, running its live |
| +// build for 20 minutes, ready. policy builds the policy file from the key |
| +// file's path; nil writes no policy file. |
| +func newProofRootEnv(t *testing.T, policy func(keyFile string) string) *proofRootEnv { |
| + t.Helper() |
| + opt, bake, etc := t.TempDir(), t.TempDir(), t.TempDir() |
| + liveTo(t, opt, observedHost, "v1") |
| + reg := restartTestRegistry(t) |
| + if err := reg.Upsert(registry.Route{Host: observedHost, Kind: "instance", App: "console", |
| + Channel: "canary/console", Unit: "console.service", Status: "ready"}); err != nil { |
| + t.Fatal(err) |
| + } |
| + seed, pubB64, err := release.GenerateKey() |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + pub, err := release.ParsePublicKey(pubB64, "test") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + keyFile := filepath.Join(etc, "proving.key") |
| + if err := os.WriteFile(keyFile, []byte(seed+"\n"), 0o600); err != nil { |
| + t.Fatal(err) |
| + } |
| + policyPath := filepath.Join(etc, "proof-policy.json") |
| + if policy != nil { |
| + if err := os.WriteFile(policyPath, []byte(policy(keyFile)), 0o644); err != nil { |
| + t.Fatal(err) |
| + } |
| + } |
| + staged := filepath.Join(opt, "versions", "v1", "console") |
| + sha, err := release.FileSHA256(staged) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + e := &proofRootEnv{store: release.NewDirStore(t.TempDir()), bake: bake, keyFile: keyFile, pub: pub, sha: sha, |
| + now: time.Date(2026, 10, 6, 10, 0, 0, 0, time.UTC)} |
| + units := &fakeUnits{shows: []unitProps{{ActiveState: "active", InvocationID: "i1", MainPID: 7, |
| + ExecPath: release.LiveLink(opt, observedHost), ActiveEnterTimestamp: e.now.Add(-20 * time.Minute)}}} |
| + e.root = proofRoot{ |
| + obs: proofObserver{sys: units, procExe: func(int) string { return staged }, optDir: opt, |
| + now: func() time.Time { return e.now }, ready: func(registry.Route) func() error { return nil }}, |
| + reg: reg, store: e.store, bakeDir: bake, policyPath: policyPath, readKey: readProvingKey, |
| + logf: func(f string, a ...any) { e.logs = append(e.logs, fmt.Sprintf(f, a...)) }, |
| + } |
| + return e |
| +} |
| + |
| +func (e *proofRootEnv) proof(t *testing.T) *release.Proof { |
| + t.Helper() |
| + p, err := release.GetProof(context.Background(), e.store, "bab", "console", e.sha, "dev") |
| + if err != nil { |
| + return nil |
| + } |
| + return p |
| +} |
| + |
| +func TestProofRootMintsWhenEveryConditionHolds(t *testing.T) { |
| + e := newProofRootEnv(t, provePolicy) |
| + e.root.run(context.Background()) |
| + p := e.proof(t) |
| + if p == nil { |
| + t.Fatalf("no proof minted; logs: %v", e.logs) |
| + } |
| + if err := p.Check(e.pub, "console", release.AdoptPlatform(), e.sha, "dev"); err != nil { |
| + t.Fatalf("minted proof: %v", err) |
| + } |
| + if p.Version != "v1" || !p.ActiveSince.Equal(e.now.Add(-20*time.Minute)) || !p.ProvenAt.Equal(e.now) { |
| + t.Fatalf("proof = %+v", p) |
| + } |
| +} |
| + |
| +func TestProofRootDoesNotMintARefusedBuild(t *testing.T) { |
| + e := newProofRootEnv(t, provePolicy) |
| + if err := release.WriteRefusal(e.bake, release.Refusal{Host: observedHost, App: "console", Version: "v1", SHA256: e.sha}); err != nil { |
| + t.Fatal(err) |
| + } |
| + e.root.run(context.Background()) |
| + if e.proof(t) != nil { |
| + t.Fatal("minted a proof for a build a refusal record names") |
| + } |
| +} |
| + |
| +func TestProofRootNeverOverwritesAProof(t *testing.T) { |
| + e := newProofRootEnv(t, provePolicy) |
| + seed, _ := os.ReadFile(e.keyFile) |
| + key, err := release.ParseSigningKey(string(seed), "test") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + first, err := release.SignProof(key, release.Proof{App: "console", Platform: release.AdoptPlatform(), SHA256: e.sha, |
| + Version: "v1", Prover: "dev", ActiveSince: e.now.Add(-time.Hour), ProvenAt: e.now.Add(-time.Minute)}) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := release.PutProof(context.Background(), e.store, "bab", first); err != nil { |
| + t.Fatal(err) |
| + } |
| + e.root.run(context.Background()) |
| + if p := e.proof(t); p == nil || !p.ProvenAt.Equal(first.ProvenAt) { |
| + t.Fatalf("proof = %+v, want the first one untouched", p) |
| + } |
| +} |
| + |
| +func TestProofRootRefusesAKeyOthersCanRead(t *testing.T) { |
| + e := newProofRootEnv(t, provePolicy) |
| + if err := os.Chmod(e.keyFile, 0o644); err != nil { |
| + t.Fatal(err) |
| + } |
| + e.root.run(context.Background()) |
| + if e.proof(t) != nil { |
| + t.Fatal("minted with a world-readable key") |
| + } |
| + if !strings.Contains(strings.Join(e.logs, "\n"), "readable by root only") { |
| + t.Fatalf("logs = %v, want the key-mode refusal", e.logs) |
| + } |
| +} |
| + |
| +type noReadStore struct { |
| + release.Store |
| + t *testing.T |
| +} |
| + |
| +func (s noReadStore) Get(ctx context.Context, key string) ([]byte, string, error) { |
| + s.t.Errorf("bucket GET %s with no policy file", key) |
| + return s.Store.Get(ctx, key) |
| +} |
| + |
| +// Review focus 3's source: with no policy file at all, the build already |
| +// live enters the healthy history, and no proof is read or minted. |
| +func TestProofRootRecordsTheLiveBuildAsHealthy(t *testing.T) { |
| + e := newProofRootEnv(t, nil) |
| + e.root.store = noReadStore{Store: e.store, t: t} |
| + e.root.run(context.Background()) |
| + h := release.ReadHealthy(e.bake, "bab", "console") |
| + if len(h) != 1 || h[0].SHA256 != e.sha || h[0].How != "live" || h[0].Version != "v1" { |
| + t.Fatalf("history = %+v, want the live build", h) |
| + } |
| + if e.proof(t) != nil { |
| + t.Fatal("a proof was minted with no policy file") |
| + } |
| +} |
| + |
| +// A box that never had the requirement (Tito, a self-hoster) has nothing |
| +// switched off and must stay quiet; one that had it and lost it logs hourly. |
| +func TestProofRootHourlyLineOnlyWhereTheRequirementWasOnce(t *testing.T) { |
| + e := newProofRootEnv(t, func(string) string { return `{"schema":1}` }) |
| + e.root.hourly = true |
| + e.root.run(context.Background()) |
| + if strings.Contains(strings.Join(e.logs, "\n"), "requirement is off") { |
| + t.Fatalf("a box that never required a proof logged that the requirement is off: %v", e.logs) |
| + } |
| +} |
| + |
| +func TestProofRootHourlyLineWhenRequirementIsOff(t *testing.T) { |
| + e := newProofRootEnv(t, func(string) string { return `{"schema":1}` }) |
| + marker := filepath.Join(e.root.bakeDir, "proof-required", "console") |
| + if err := os.MkdirAll(filepath.Dir(marker), 0o755); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := os.WriteFile(marker, nil, 0o644); err != nil { |
| + t.Fatal(err) |
| + } |
| + e.root.hourly = true |
| + e.root.run(context.Background()) |
| + if !strings.Contains(strings.Join(e.logs, "\n"), "requirement is off for console") { |
| + t.Fatalf("logs = %v, want the hourly requirement-off line", e.logs) |
| + } |
| + e.logs = nil |
| + e.root.hourly = false |
| + e.root.run(context.Background()) |
| + if strings.Contains(strings.Join(e.logs, "\n"), "requirement is off") { |
| + t.Fatalf("a stage-only pass printed the hourly line: %v", e.logs) |
| + } |
| +} |
| +``` |
| + |
| +- [ ] **Step 2: Run them to verify they fail** |
| + |
| +Run: `GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/host -run 'ProofRoot' -count=1` |
| +Expected: FAIL, `undefined: proofRoot`. |
| + |
| +- [ ] **Step 3: Implement `proofroot.go`** |
| + |
| +```go |
| +// The root pass's D3 work (spec 2026-10-06-console-release-defences-design.md |
| +// §4.3, §4.5): after release.Pass, look at every named-unit route's live build. |
| +// |
| +// - A build meeting observe's conditions (bound, active in one invocation |
| +// for the policy's after, hashed from its running executable, ready) |
| +// enters the app's healthy history, on every box, policy file or not. |
| +// That is what puts the build already live when a requirement is switched |
| +// on into the history without a restart. |
| +// - On a proving box (a prove block lists the app) the same build is |
| +// minted a proof, unless a refusal record names it or the bucket already |
| +// has one. Minting needs the root-only key, which is why it is here. |
| +// - On the hourly pass, an app with healthy history and no require entry |
| +// gets one line saying the requirement is off: the break-glass trail. |
| +package host |
| + |
| +import ( |
| + "context" |
| + "crypto/ed25519" |
| + "errors" |
| + "fmt" |
| + "os" |
| + "path/filepath" |
| + "sort" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/registry" |
| + "github.com/carlosframework/platform/internal/release" |
| +) |
| + |
| +type proofRoot struct { |
| + obs proofObserver |
| + reg *registry.Registry |
| + store release.Store |
| + bakeDir string |
| + policyPath string |
| + readKey func(path string) (ed25519.PrivateKey, error) |
| + hourly bool // carlos-adopt.service, not -stage-only |
| + logf func(format string, a ...any) |
| +} |
| + |
| +func (r proofRoot) run(ctx context.Context) { |
| + pol, perr := release.LoadProofPolicy(r.policyPath) |
| + if perr != nil { |
| + r.logf("proof: %v; nothing is minted until it parses", perr) |
| + } |
| + routes, err := r.reg.List() |
| + if err != nil { |
| + r.logf("proof: list routes: %v", err) |
| + return |
| + } |
| + after := pol.ProveAfter() |
| + type acctApp struct{ acct, app string } |
| + seen := map[acctApp]bool{} |
| + for _, rt := range routes { |
| + if !release.Adoptable(rt) || rt.Unit == "" { |
| + continue |
| + } |
| + acct := release.RouteAccount(rt) |
| + seen[acctApp{acct, rt.App}] = true |
| + b, err := r.obs.observe(rt, after) |
| + if err != nil { |
| + if pol.Proves(rt.App) { |
| + r.logf("proof: %s: not minting yet: %v", rt.Host, err) |
| + } |
| + continue |
| + } |
| + if err := release.RecordHealthy(r.bakeDir, acct, rt.App, release.HealthyBuild{ |
| + SHA256: b.SHA256, Version: b.Version, How: "live", At: r.obs.now().UTC()}); err != nil { |
| + r.logf("proof: %s: recording the healthy build: %v", rt.Host, err) |
| + } |
| + if perr == nil && pol.Proves(rt.App) { |
| + r.mint(ctx, pol, rt, acct, b) |
| + } |
| + } |
| + if !r.hourly || perr != nil { |
| + return |
| + } |
| + keys := make([]acctApp, 0, len(seen)) |
| + for k := range seen { |
| + keys = append(keys, k) |
| + } |
| + sort.Slice(keys, func(i, j int) bool { return keys[i].acct+"/"+keys[i].app < keys[j].acct+"/"+keys[j].app }) |
| + for _, k := range keys { |
| + // The marker says this box has required a proof for the app at some |
| + // point. Only then is "off" news: a box that never required one |
| + // (Tito, a self-hoster) has nothing switched off. |
| + marker := filepath.Join(r.bakeDir, "proof-required", k.app) |
| + if len(pol.Requires(k.app)) > 0 { |
| + if err := os.MkdirAll(filepath.Dir(marker), 0o755); err == nil { |
| + _ = os.WriteFile(marker, nil, 0o644) |
| + } |
| + continue |
| + } |
| + if _, err := os.Stat(marker); err != nil { |
| + continue |
| + } |
| + if n := len(release.ReadHealthy(r.bakeDir, k.acct, k.app)); n > 0 { |
| + r.logf("proof: requirement is off for %s on this box: %s has no require entry for it (%d healthy builds on record)", |
| + k.app, r.policyPath, n) |
| + } |
| + } |
| +} |
| + |
| +func (r proofRoot) mint(ctx context.Context, pol *release.ProofPolicy, rt registry.Route, acct string, b liveBuild) { |
| + refusals, err := release.ListRefusals(r.bakeDir) |
| + if err != nil { |
| + r.logf("proof: %s: reading refusal records: %v; not minting", rt.Host, err) |
| + return |
| + } |
| + for _, ref := range refusals { |
| + if ref.SHA256 == b.SHA256 { |
| + r.logf("proof: %s: not minting %s: this box refused it (%s)", rt.Host, b.Version, ref.Reason) |
| + return |
| + } |
| + } |
| + prover := pol.Prove.Prover |
| + _, err = release.GetProof(ctx, r.store, acct, rt.App, b.SHA256, prover) |
| + switch { |
| + case err == nil: |
| + return // already proven |
| + case !errors.Is(err, os.ErrNotExist): |
| + r.logf("proof: %s: reading the bucket for an existing proof: %v; not minting", rt.Host, err) |
| + return |
| + } |
| + key, err := r.readKey(pol.Prove.KeyFile) |
| + if err != nil { |
| + r.logf("proof: %s: %v; not minting", rt.Host, err) |
| + return |
| + } |
| + p, err := release.SignProof(key, release.Proof{App: rt.App, Platform: release.AdoptPlatform(), SHA256: b.SHA256, |
| + Version: b.Version, Prover: prover, ActiveSince: b.ActiveSince.UTC().Truncate(time.Second), |
| + ProvenAt: r.obs.now().UTC().Truncate(time.Second)}) |
| + if err != nil { |
| + r.logf("proof: %s: signing: %v", rt.Host, err) |
| + return |
| + } |
| + if err := release.PutProof(ctx, r.store, acct, p); err != nil { |
| + r.logf("proof: %s: writing the proof: %v (the agent asks again in 2 minutes)", rt.Host, err) |
| + return |
| + } |
| + r.logf("proof: minted %s %s (sha256 %s) as %s, live since %s", rt.App, b.Version, b.SHA256, prover, |
| + b.ActiveSince.UTC().Format(time.RFC3339)) |
| +} |
| + |
| +// readProvingKey reads the root-only proving key, refusing one any other user |
| +// can read (spec §4.2: the key is root 0600). |
| +func readProvingKey(path string) (ed25519.PrivateKey, error) { |
| + fi, err := os.Stat(path) |
| + if err != nil { |
| + return nil, fmt.Errorf("proving key: %w", err) |
| + } |
| + if perm := fi.Mode().Perm(); perm&0o077 != 0 { |
| + return nil, fmt.Errorf("proving key %s is mode %04o; it must be readable by root only (0600)", path, perm) |
| + } |
| + b, err := os.ReadFile(path) |
| + if err != nil { |
| + return nil, fmt.Errorf("proving key: %w", err) |
| + } |
| + return release.ParseSigningKey(string(b), path) |
| +} |
| +``` |
| + |
| +- [ ] **Step 4: Wire it into `RunAdopt`** |
| + |
| +Change `rootRestartRoute` (after Plan 1 it takes `optDir`) to: |
| + |
| +```go |
| +func rootRestartRoute(reg *registry.Registry, optDir string, settled func(rt registry.Route)) func(host string) error { |
| + return makeRestartRoute(reg, systemctlTryRestart, checkedUnitRestart(systemdUnits{}, optDir, os.Getenv, settled), nil) |
| +} |
| +``` |
| + |
| +In `RunAdopt`, the `restart := rootRestartRoute(...)` line becomes: |
| + |
| +```go |
| + // settled: D1's settle passing records the build in the healthy history |
| + // (D3, proofobserve.go). Root is the process that can hash the unit's |
| + // executable. |
| + restart := rootRestartRoute(reg, release.OptDir(), |
| + recordSettled(systemdUnits{}, procExePath, release.OptDir(), dataDir(), time.Now)) |
| +``` |
| + |
| +Append to `passOpts`, after `release.WithStartUnit(...)`: |
| + |
| +```go |
| + // D3's gate: a fresh gate per root pass, so the policy file is read |
| + // at the start of this pass (BeginPass) like every other. |
| + release.WithProofGate(release.NewProofGate(release.ProofPolicyPath, dataDir())), |
| +``` |
| + |
| +Immediately after the `if err := release.Pass(...); err != nil { log.Fatalf(...) }` block, before `// Step 4: the advisories.`: |
| + |
| +```go |
| + // D3 (proofroot.go): record live builds as healthy, mint on a proving |
| + // box, and on the hourly pass say when a requirement is off. After the |
| + // pass, so it looks at what this pass left live. |
| + proofRoot{ |
| + obs: productionProofObserver(release.OptDir()), |
| + reg: reg, |
| + store: store, |
| + bakeDir: dataDir(), |
| + policyPath: release.ProofPolicyPath, |
| + readKey: readProvingKey, |
| + hourly: !*stageOnly, |
| + logf: log.Printf, |
| + }.run(ctx) |
| +``` |
| + |
| +In `internal/host/agent_test.go`, add a final `nil` to the `rootRestartRoute(...)` calls at lines 718, 836 and 963. |
| + |
| +- [ ] **Step 5: Run the host suite and the build** |
| + |
| +Run: `GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false nice -n 10 go test ./internal/host -count=1 && GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go build ./...` |
| +Expected: PASS, and the build succeeds. |
| + |
| +- [ ] **Step 6: Commit** |
| + |
| +```bash |
| +git add internal/host/proofroot.go internal/host/proofroot_test.go internal/host/agent.go internal/host/agent_test.go |
| +git commit -m "host: the root pass mints dev proofs and records healthy builds (console defences D3)" |
| +``` |
| + |
| +--- |
| + |
| +### Task 7: The agent: one gate, a mint asker, and a waiting route in the fast loop |
| + |
| +**Files:** |
| +- Create: `internal/host/proofask.go` |
| +- Modify: `internal/host/agent.go`: `materializeHook` (776), `adoptLoop`'s `release.Pass` options (868), `fastDeps` (1420) gains `proof` and `mintLeg`, `fastMembers` (1522), `fastTick` (1805), `fastConvergeLoop` (1908) |
| +- Test: `internal/host/proofask_test.go` |
| + |
| +**Interfaces:** |
| +- Consumes: Tasks 1, 2, 4 (`release.GetProof`, `PutProofPending`, `LoadProofPolicy`, `ProofPolicyPath`, `NewProofGate`, `WithProofGate`, `(*ProofGate).WaitingDue`); Task 5's `unitProps.ActiveEnterTimestamp`; `stageRequester` (agent.go:1224). |
| +- Produces: |
| + - `func agentProofGate() *release.ProofGate`: one gate per agent process |
| + - `const mintAskEvery = 2 * time.Minute` |
| + - `type mintAsker struct { reg *registry.Registry; store release.Store; sys unitSystem; optDir, policyPath string; request func(host, ver string, err error); now func() time.Time; ... }` |
| + - `func (a *mintAsker) tick(ctx context.Context)` |
| + - `fastDeps.proof *release.ProofGate`, `fastDeps.mintLeg func(ctx context.Context)` |
| + |
| +- [ ] **Step 1: Write the failing tests** |
| + |
| +```go |
| +package host |
| + |
| +import ( |
| + "context" |
| + "crypto/sha256" |
| + "encoding/hex" |
| + "fmt" |
| + "os" |
| + "path/filepath" |
| + "testing" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/registry" |
| + "github.com/carlosframework/platform/internal/release" |
| +) |
| + |
| +func mintAskerFixture(t *testing.T, activeFor time.Duration) (*mintAsker, *int, *time.Time, release.Store, string) { |
| + t.Helper() |
| + opt, etc := t.TempDir(), t.TempDir() |
| + liveTo(t, opt, observedHost, "v1") |
| + reg := restartTestRegistry(t) |
| + if err := reg.Upsert(registry.Route{Host: observedHost, Kind: "instance", App: "console", |
| + Channel: "canary/console", Unit: "console.service", Status: "ready"}); err != nil { |
| + t.Fatal(err) |
| + } |
| + policy := filepath.Join(etc, "proof-policy.json") |
| + if err := os.WriteFile(policy, []byte(provePolicy("/etc/carlos/proving.key")), 0o644); err != nil { |
| + t.Fatal(err) |
| + } |
| + t0 := time.Date(2026, 10, 6, 10, 0, 0, 0, time.UTC) |
| + clock := t0 |
| + asked := 0 |
| + store := release.NewDirStore(t.TempDir()) |
| + a := &mintAsker{reg: reg, store: store, optDir: opt, policyPath: policy, |
| + sys: &fakeUnits{shows: []unitProps{{ActiveState: "active", InvocationID: "i1", MainPID: 7, |
| + ActiveEnterTimestamp: t0.Add(-activeFor)}}}, |
| + request: func(string, string, error) { asked++ }, |
| + now: func() time.Time { return clock }} |
| + sha, err := release.FileSHA256(filepath.Join(opt, "versions", "v1", "console")) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + return a, &asked, &clock, store, sha |
| +} |
| + |
| +// Review focus 5: the relay can drop a request and a mint write can fail, so |
| +// the agent asks every 2 minutes until it sees the proof, then stops. |
| +func TestMintAskerReasksUntilTheProofAppears(t *testing.T) { |
| + a, asked, clock, store, sha := mintAskerFixture(t, 20*time.Minute) |
| + ctx := context.Background() |
| + t0 := *clock |
| + at := func(d time.Duration) { *clock = t0.Add(d); a.tick(ctx) } |
| + at(0) |
| + if *asked != 1 { |
| + t.Fatalf("asked %d times at first sight, want 1", *asked) |
| + } |
| + at(time.Minute) |
| + if *asked != 1 { |
| + t.Fatalf("asked %d times inside 2 minutes, want 1", *asked) |
| + } |
| + at(2 * time.Minute) |
| + if *asked != 2 { |
| + t.Fatalf("asked %d times after 2 minutes, want 2", *asked) |
| + } |
| + seed, _, err := release.GenerateKey() |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + key, err := release.ParseSigningKey(seed, "test") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + p, err := release.SignProof(key, release.Proof{App: "console", Platform: release.AdoptPlatform(), SHA256: sha, |
| + Version: "v1", Prover: "dev", ActiveSince: t0.Add(-20 * time.Minute), ProvenAt: t0}) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := release.PutProof(ctx, store, "bab", p); err != nil { |
| + t.Fatal(err) |
| + } |
| + at(4 * time.Minute) |
| + at(6 * time.Minute) |
| + if *asked != 2 { |
| + t.Fatalf("asked %d times after the proof appeared, want 2", *asked) |
| + } |
| + pend, err := release.GetProofPending(ctx, store, "bab", "console", sha, "dev") |
| + if err != nil || !pend.LiveSince.Equal(t0.Add(-20*time.Minute)) { |
| + t.Fatalf("pending = %+v, %v", pend, err) |
| + } |
| +} |
| + |
| +func TestMintAskerWaitsOutTheBake(t *testing.T) { |
| + a, asked, _, store, sha := mintAskerFixture(t, 5*time.Minute) |
| + a.tick(context.Background()) |
| + if *asked != 0 { |
| + t.Fatalf("asked %d times five minutes into a fifteen-minute bake", *asked) |
| + } |
| + if _, err := release.GetProofPending(context.Background(), store, "bab", "console", sha, "dev"); err != nil { |
| + t.Fatalf("no pending note for a baking build: %v", err) |
| + } |
| +} |
| + |
| +// Spec §4.5, codex 14: a non-canary listed route with a live version stays in |
| +// the fast set while it waits (at the gate's 30 s cadence), and leaves once |
| +// it adopts. |
| +func TestFastMembersKeepsAWaitingRoute(t *testing.T) { |
| + f := newFastFixture(t) |
| + ctx := context.Background() |
| + host := "edge.example.com" |
| + f.addRoute(t, host, "edge") |
| + f.stageLive(t, host, "v1") |
| + plat := release.AdoptPlatform() |
| + for ver, body := range map[string]string{"v1": "staged build", "v2": "build two"} { |
| + m := release.Manifest{App: "fastapp", Version: ver, Kind: "binary", Artifacts: map[string]release.Artifact{plat: {}}} |
| + if err := release.Ship(ctx, f.store, "bab", m, map[string][]byte{plat: []byte(body)}); err != nil { |
| + t.Fatalf("Ship %s: %v", ver, err) |
| + } |
| + } |
| + if err := release.Promote(ctx, f.store, f.priv, "bab", "fastapp", "edge", "v2"); err != nil { |
| + t.Fatal(err) |
| + } |
| + seed, pubB64, err := release.GenerateKey() |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + prover, err := release.ParseSigningKey(seed, "test") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + policy := filepath.Join(t.TempDir(), "proof-policy.json") |
| + if err := os.WriteFile(policy, []byte(fmt.Sprintf(`{"schema":1,"require":[{"app":"fastapp","prover":"dev","key":%q}]}`, pubB64)), 0o644); err != nil { |
| + t.Fatal(err) |
| + } |
| + clock := time.Now() |
| + gate := release.NewProofGate(policy, f.dataDir).WithClock(func() time.Time { return clock }) |
| + d := f.deps(nil) |
| + d.passOpts = append(d.passOpts, release.WithProofGate(gate)) |
| + d.proof = gate |
| + |
| + if got := d.fastMembers(clock); len(got) != 0 { |
| + t.Fatalf("members = %v before anything waits", got) |
| + } |
| + // The serial-triggered full pass sees v2 and finds no proof. |
| + if err := release.Pass(ctx, f.reg, f.store, f.pub, f.optDir, d.restart, d.passOpts...); err != nil { |
| + t.Fatal(err) |
| + } |
| + if v, _ := release.LiveVersion(f.optDir, host); v != "v1" { |
| + t.Fatalf("live = %q, want v1 while unproven", v) |
| + } |
| + clock = clock.Add(31 * time.Second) |
| + if got := d.fastMembers(clock); len(got) != 1 || got[0] != host { |
| + t.Fatalf("members = %v, want the waiting non-canary route", got) |
| + } |
| + sum := sha256.Sum256([]byte("build two")) |
| + p, err := release.SignProof(prover, release.Proof{App: "fastapp", Platform: plat, SHA256: hex.EncodeToString(sum[:]), |
| + Version: "v2", Prover: "dev", ActiveSince: clock.Add(-time.Hour).UTC().Truncate(time.Second), |
| + ProvenAt: clock.UTC().Truncate(time.Second)}) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := release.PutProof(ctx, f.store, "bab", p); err != nil { |
| + t.Fatal(err) |
| + } |
| + fastTick(ctx, d, clock) |
| + if v, _ := release.LiveVersion(f.optDir, host); v != "v2" { |
| + t.Fatalf("live = %q, want v2 once proven", v) |
| + } |
| + clock = clock.Add(31 * time.Second) |
| + if got := d.fastMembers(clock); len(got) != 0 { |
| + t.Fatalf("members = %v after adopting, want none", got) |
| + } |
| +} |
| +``` |
| + |
| +- [ ] **Step 2: Run them to verify they fail** |
| + |
| +Run: `GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/host -run 'MintAsker|FastMembersKeepsAWaitingRoute' -count=1` |
| +Expected: FAIL, `undefined: mintAsker`. |
| + |
| +- [ ] **Step 3: Implement `proofask.go`** |
| + |
| +```go |
| +// The agent's half of D3 (spec 2026-10-06-console-release-defences-design.md |
| +// §4.3, §4.5). |
| +// |
| +// agentProofGate is the one release.ProofGate this process passes to every |
| +// pass it runs (adoptLoop, fastConvergeLoop, the cold-wake hook), so the miss |
| +// cache, the waiting list fastMembers reads and the status line all agree. |
| +// |
| +// mintAsker runs on the proving box's fast loop. Minting needs the root-only |
| +// key, and root passes run hourly or on request, so when a listed app's live |
| +// build has run for the policy's after with no proof in the bucket, the asker |
| +// requests a root pass through the stage-request relay. It asks again every |
| +// mintAskEvery until it sees the proof (codex 5): the relay can miss a request |
| +// written while carlos-stage.service is running, and a mint write can fail. |
| +// It also leaves a pending note the first time it sees a build live, which is |
| +// how `carlos proof get` can say how long the build has had. |
| +// |
| +// It cannot check everything a mint needs (it cannot read another unit's |
| +// /proc/<pid>/exe), and need not: the root pass checks every condition again. |
| +// When systemctl show is denied here, it asks anyway and root decides. |
| +package host |
| + |
| +import ( |
| + "context" |
| + "errors" |
| + "log" |
| + "os" |
| + "path/filepath" |
| + "sync" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/registry" |
| + "github.com/carlosframework/platform/internal/release" |
| +) |
| + |
| +var ( |
| + proofGateOnce sync.Once |
| + proofGateInst *release.ProofGate |
| +) |
| + |
| +func agentProofGate() *release.ProofGate { |
| + proofGateOnce.Do(func() { proofGateInst = release.NewProofGate(release.ProofPolicyPath, dataDir()) }) |
| + return proofGateInst |
| +} |
| + |
| +const mintAskEvery = 2 * time.Minute |
| + |
| +type mintAsker struct { |
| + reg *registry.Registry |
| + store release.Store |
| + sys unitSystem |
| + optDir string |
| + policyPath string |
| + request func(host, ver string, err error) // stageRequester in production |
| + now func() time.Time |
| + |
| + last map[string]time.Time // host -> when it was last looked at |
| + proven map[string]string // host -> sha256 seen proven |
| + pending map[string]string // host -> sha256 whose pending note is written |
| + shaByPath map[string]string // staged file -> sha256 (version dirs are immutable) |
| +} |
| + |
| +func (a *mintAsker) tick(ctx context.Context) { |
| + pol, err := release.LoadProofPolicy(a.policyPath) |
| + if err != nil || pol == nil || pol.Prove == nil { |
| + return |
| + } |
| + routes, err := a.reg.List() |
| + if err != nil { |
| + return |
| + } |
| + if a.last == nil { |
| + a.last, a.proven, a.pending, a.shaByPath = map[string]time.Time{}, map[string]string{}, map[string]string{}, map[string]string{} |
| + } |
| + now := a.now() |
| + after := pol.ProveAfter() |
| + prover := pol.Prove.Prover |
| + for _, rt := range routes { |
| + if !release.Adoptable(rt) || rt.Unit == "" || !pol.Proves(rt.App) { |
| + continue |
| + } |
| + if last, ok := a.last[rt.Host]; ok && now.Sub(last) < mintAskEvery { |
| + continue |
| + } |
| + a.last[rt.Host] = now |
| + target, err := filepath.EvalSymlinks(release.LiveLink(a.optDir, rt.Host)) |
| + if err != nil { |
| + continue |
| + } |
| + sha, ok := a.shaByPath[target] |
| + if !ok { |
| + if sha, err = release.FileSHA256(target); err != nil { |
| + continue |
| + } |
| + a.shaByPath[target] = sha |
| + } |
| + if a.proven[rt.Host] == sha { |
| + continue |
| + } |
| + ver, _ := release.LiveVersion(a.optDir, rt.Host) |
| + acct := release.RouteAccount(rt) |
| + if p, err := a.sys.Show(rt.Unit); err == nil { |
| + if p.ActiveState != "active" { |
| + continue |
| + } |
| + if !p.ActiveEnterTimestamp.IsZero() { |
| + if a.pending[rt.Host] != sha { |
| + if err := release.PutProofPending(ctx, a.store, acct, release.ProofPending{App: rt.App, |
| + Platform: release.AdoptPlatform(), SHA256: sha, Version: ver, Prover: prover, |
| + LiveSince: p.ActiveEnterTimestamp.UTC(), After: pol.Prove.After}); err != nil { |
| + log.Printf("agent: %s: writing the proof pending note: %v", rt.Host, err) |
| + } else { |
| + a.pending[rt.Host] = sha |
| + } |
| + } |
| + if now.Sub(p.ActiveEnterTimestamp) < after { |
| + continue |
| + } |
| + } |
| + } |
| + _, err = release.GetProof(ctx, a.store, acct, rt.App, sha, prover) |
| + switch { |
| + case err == nil: |
| + a.proven[rt.Host] = sha |
| + continue |
| + case !errors.Is(err, os.ErrNotExist): |
| + log.Printf("agent: %s: reading the bucket for %s's proof: %v", rt.Host, ver, err) |
| + continue |
| + } |
| + a.request(rt.Host, ver, nil) |
| + log.Printf("agent: %s has run %s for %s with no proof in the bucket; asked for a root pass to mint one", rt.Host, ver, after) |
| + } |
| +} |
| +``` |
| + |
| +- [ ] **Step 4: Wire it into `agent.go`** |
| + |
| +`fastDeps`, after `systemLeg`: |
| + |
| +```go |
| + proof *release.ProofGate // D3's gate (agentProofGate); nil = no proof waits to keep |
| + mintLeg func(ctx context.Context) // D3's mint asker (proofask.go); nil = not armed |
| +``` |
| + |
| +`fastMembers`, immediately after the `if isCanaryRoute(rt) { ... continue }` block: |
| + |
| +```go |
| + // D3 (spec §4.5, codex 14): a listed route waiting for a dev proof |
| + // stays in the sweep although it has a live version, whenever its |
| + // 30 s miss has run out, so it adopts soon after the proof arrives |
| + // rather than at the next serial bump or the hourly floor. |
| + if d.proof.WaitingDue(rt.Host) { |
| + seen[rt.Host] = true |
| + members = append(members, rt.Host) |
| + continue |
| + } |
| +``` |
| + |
| +`fastTick`, immediately before `members := d.fastMembers(now)`: |
| + |
| +```go |
| + // D3's mint asker: on a proving box, ask for a root pass every 2 minutes |
| + // while a listed build has baked with no proof. Elsewhere it reads one |
| + // absent file and returns. |
| + if d.mintLeg != nil { |
| + d.mintLeg(ctx) |
| + } |
| +``` |
| + |
| +`fastConvergeLoop`, immediately after `opts := fastPassOpts(dd, stageRequester(dd), restart, makeStartRoute(reg, systemctlStartUnit))`: |
| + |
| +```go |
| + // D3: the process's one proof gate rides every pass this loop runs, and |
| + // fastMembers reads its waiting list. The mint asker has its own |
| + // stage requester, so its 2-minute asks never spend the stage budget. |
| + opts = append(opts, release.WithProofGate(agentProofGate())) |
| + d.proof = agentProofGate() |
| + d.mintLeg = (&mintAsker{reg: reg, store: store, sys: systemdUnits{}, optDir: optDir, |
| + policyPath: release.ProofPolicyPath, request: stageRequester(dd), now: time.Now}).tick |
| +``` |
| + |
| +`adoptLoop`'s `release.Pass(...)` call: add `release.WithProofGate(agentProofGate()),` after `release.WithStartUnit(makeStartRoute(reg, systemctlStartUnit))`. |
| + |
| +`materializeHook`: its `release.MaterializeHost(ctx, reg, store, pub, optDir, dd, host, release.OnStageBlocked(stageReq))` call becomes: |
| + |
| +```go |
| + return release.MaterializeHost(ctx, reg, store, pub, optDir, dd, host, |
| + release.OnStageBlocked(stageReq), release.WithProofGate(agentProofGate())) |
| +``` |
| + |
| +so a cold wake cannot adopt an unproven build of a listed app. |
| + |
| +- [ ] **Step 5: Run the host suite** |
| + |
| +Run: `GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false nice -n 10 go test ./internal/host -count=1` |
| +Expected: PASS. Existing fast-loop tests leave `proof` and `mintLeg` nil, so their behaviour is unchanged. |
| + |
| +- [ ] **Step 6: Commit** |
| + |
| +```bash |
| +git add internal/host/proofask.go internal/host/proofask_test.go internal/host/agent.go |
| +git commit -m "host: the agent shares one proof gate, keeps a waiting route in the fast loop, and asks root to mint" |
| +``` |
| + |
| +--- |
| + |
| +### Task 8: "waiting for a dev proof" in `carlos status --live` |
| + |
| +**Files:** |
| +- Modify: `internal/status/snapshot.go`: new `ProofWait` type beside Plan 1's `ReleaseRefusal`; `Probes` (41) and `BoxMetrics` (213) gain `ProofWaits`; the emptiness check (700) and the copy (719) |
| +- Create: `internal/host/proofstatus.go` |
| +- Modify: `internal/host/agent.go`: the status probe block (2939), beside Plan 1's release-refusal probe |
| +- Modify: `cli.go` `printLiveBoxes` (3418) |
| +- Test: `internal/status/proofwait_test.go`, `statuslive_test.go` (append) |
| + |
| +**Interfaces:** |
| +- Consumes: Task 4's `(*ProofGate).Waits`, `release.ProofWait`; Task 7's `agentProofGate`. |
| +- Produces: |
| + - `type ProofWait struct { Host, Version, SHA256, Prover, Reason string; Since time.Time }` (status package; JSON `host`, `version`, `sha256`, `prover`, `reason`, `since`) |
| + - `Probes.ProofWaits []ProofWait`, `BoxMetrics.ProofWaits []ProofWait` (`proof_waits,omitempty`) |
| + - `func proofWaitProbe() []status.ProofWait` |
| + |
| +- [ ] **Step 1: Write the failing tests** |
| + |
| +`internal/status/proofwait_test.go`: |
| + |
| +```go |
| +package status |
| + |
| +import ( |
| + "testing" |
| + "time" |
| +) |
| + |
| +func TestCollectBoxCarriesProofWaits(t *testing.T) { |
| + now := time.Date(2026, 10, 6, 10, 0, 0, 0, time.UTC) |
| + box := CollectBox(nil, Probes{ProofWaits: []ProofWait{{Host: "console.example.com", Version: "v2", |
| + SHA256: "abc", Prover: "dev", Reason: "no proof from dev", Since: now}}}, now) |
| + if box.Metrics == nil || len(box.Metrics.ProofWaits) != 1 || box.Metrics.ProofWaits[0].Host != "console.example.com" { |
| + t.Fatalf("metrics = %+v, want the proof wait carried", box.Metrics) |
| + } |
| +} |
| +``` |
| + |
| +Append to `statuslive_test.go`: |
| + |
| +```go |
| +func TestPrintLiveBoxesShowsAProofWait(t *testing.T) { |
| + now := time.Date(2026, 10, 6, 10, 0, 0, 0, time.UTC) |
| + var buf bytes.Buffer |
| + printLiveBoxes(&buf, []status.Box{{GeneratedAt: now, Label: "ada-stargazing", Metrics: &status.BoxMetrics{ |
| + ProofWaits: []status.ProofWait{{Host: "console.example.com", Version: "v2", |
| + SHA256: "e47fcc1e0123456789abcdef", Prover: "dev", Since: now.Add(-time.Hour)}}, |
| + }}}, now) |
| + want := " waiting for a dev proof: console.example.com v2 (sha256 e47fcc1e0123) since 2026-10-06T09:00:00Z" |
| + if !strings.Contains(buf.String(), want) { |
| + t.Fatalf("missing %q in:\n%s", want, buf.String()) |
| + } |
| +} |
| +``` |
| + |
| +- [ ] **Step 2: Run them to verify they fail** |
| + |
| +Run: `GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/status . -run 'ProofWait' -count=1` |
| +Expected: FAIL, `unknown field ProofWaits in struct literal of type Probes`. |
| + |
| +- [ ] **Step 3: Implement** |
| + |
| +`internal/status/snapshot.go`, beside `ConfigRefusal`: |
| + |
| +```go |
| +// ProofWait is a host this box is holding on its current build because its |
| +// channel head has no dev proof yet (D3, release.ProofGate), or because the |
| +// box's proof policy file does not parse. PRIVATE like the rest of |
| +// Box.Metrics: Host is a real route name. |
| +type ProofWait struct { |
| + Host string `json:"host"` |
| + Version string `json:"version"` |
| + SHA256 string `json:"sha256,omitempty"` |
| + Prover string `json:"prover,omitempty"` |
| + Reason string `json:"reason"` |
| + Since time.Time `json:"since"` |
| +} |
| +``` |
| + |
| +In `Probes`, after `ConfigGuard`: |
| + |
| +```go |
| + // ProofWaits is the hosts held on their current build waiting for a dev |
| + // proof (D3). Private Metrics only: they name hosts. |
| + ProofWaits []ProofWait |
| +``` |
| + |
| +In `BoxMetrics`, after `ConfigGuard`: |
| + |
| +```go |
| + // ProofWaits is the hosts this box is holding on their current build |
| + // until a dev proof for the new head arrives (D3). Hostname-bearing, so |
| + // it lives in this private block only. Omitted when empty. |
| + ProofWaits []ProofWait `json:"proof_waits,omitempty"` |
| +``` |
| + |
| +In `CollectBox`, add `len(p.ProofWaits) > 0 ||` to the Metrics emptiness condition beside `len(p.ConfigRefusals) > 0 ||`, and in the `&BoxMetrics{...}` literal add: |
| + |
| +```go |
| + ProofWaits: append([]ProofWait(nil), p.ProofWaits...), |
| +``` |
| + |
| +`internal/host/proofstatus.go`: |
| + |
| +```go |
| +package host |
| + |
| +import "github.com/carlosframework/platform/internal/status" |
| + |
| +// proofWaitProbe feeds status.Probes.ProofWaits from the agent's proof gate. |
| +func proofWaitProbe() []status.ProofWait { |
| + var out []status.ProofWait |
| + for _, w := range agentProofGate().Waits() { |
| + out = append(out, status.ProofWait{Host: w.Host, Version: w.Version, SHA256: w.SHA256, |
| + Prover: w.Prover, Reason: w.Reason, Since: w.Since}) |
| + } |
| + return out |
| +} |
| +``` |
| + |
| +`internal/host/agent.go`, right after `p.ConfigGuard = configGuardProbe()`: |
| + |
| +```go |
| + // Hosts held for a dev proof (D3): private Metrics only. |
| + p.ProofWaits = proofWaitProbe() |
| +``` |
| + |
| +`cli.go` `printLiveBoxes`, after the box's main `Fprintf` (and after Plan 1's refusal lines): |
| + |
| +```go |
| + for _, pw := range m.ProofWaits { |
| + sha := pw.SHA256 |
| + if len(sha) > 12 { |
| + sha = sha[:12] |
| + } |
| + fmt.Fprintf(w, " waiting for a %s proof: %s %s (sha256 %s) since %s\n", |
| + orDash(pw.Prover), pw.Host, pw.Version, orDash(sha), pw.Since.UTC().Format(time.RFC3339)) |
| + } |
| +``` |
| + |
| +- [ ] **Step 4: Run the tests** |
| + |
| +Run: `GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test ./internal/status ./internal/host . -run 'ProofWait|LiveBoxes|Snapshot|CollectBox' -count=1` |
| +Expected: PASS. |
| + |
| +- [ ] **Step 5: Commit** |
| + |
| +```bash |
| +git add internal/status/snapshot.go internal/status/proofwait_test.go internal/host/proofstatus.go internal/host/agent.go cli.go statuslive_test.go |
| +git commit -m "status: show a host waiting for a dev proof in status --live" |
| +``` |
| + |
| +--- |
| + |
| +### Task 9: `carlos proof keygen | get | put` |
| + |
| +**Files:** |
| +- Create: `proof_cli.go` |
| +- Modify: `main.go`: `case "proof":` beside `case "release-keygen":` (133), and a usage line beside `release-keygen` (265) |
| +- Test: `proof_cli_test.go` |
| + |
| +**Interfaces:** |
| +- Consumes: Task 1 (`release.GenerateKey`, `ParseProof`, `PutProof`, `GetProof`, `GetProofPending`, `ProofsPrefix`, `ProofKey`); `openStore`, `accountFlags`, `appFlag`, `flagUsage`, `stdout` (cli.go, flags.go). |
| +- Produces: |
| + - `func runProof(args []string)` |
| + - `func proofKeygen(path string, w io.Writer) error` |
| + - `func proofGet(ctx context.Context, s release.Store, acct, app, sha, prover string, w io.Writer) error` |
| + - `func proofPut(ctx context.Context, s release.Store, acct, app string, raw []byte, w io.Writer) error` |
| + |
| +- [ ] **Step 1: Write the failing tests** |
| + |
| +```go |
| +package main |
| + |
| +import ( |
| + "bytes" |
| + "context" |
| + "crypto/ed25519" |
| + "encoding/base64" |
| + "encoding/json" |
| + "os" |
| + "path/filepath" |
| + "strings" |
| + "testing" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/release" |
| +) |
| + |
| +const cliSHA = "e47fcc1e0123456789abcdef0123456789abcdef0123456789abcdef01234567" |
| + |
| +func TestProofKeygenWritesARootOnlyKey(t *testing.T) { |
| + path := filepath.Join(t.TempDir(), "proving.key") |
| + var out bytes.Buffer |
| + if err := proofKeygen(path, &out); err != nil { |
| + t.Fatal(err) |
| + } |
| + fi, err := os.Stat(path) |
| + if err != nil || fi.Mode().Perm() != 0o600 { |
| + t.Fatalf("key file mode = %v, %v; want 0600", fi.Mode().Perm(), err) |
| + } |
| + seed, _ := os.ReadFile(path) |
| + key, err := release.ParseSigningKey(string(seed), path) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + pub := base64.StdEncoding.EncodeToString(key.Public().(ed25519.PublicKey)) |
| + if !strings.Contains(out.String(), "public key: "+pub+"\n") { |
| + t.Fatalf("printed %q, want the key file's public half %s", out.String(), pub) |
| + } |
| + if err := proofKeygen(path, &out); err == nil || !strings.Contains(err.Error(), "already exists") { |
| + t.Fatalf("second keygen = %v, want a refusal to overwrite", err) |
| + } |
| +} |
| + |
| +func signedCLIProof(t *testing.T) *release.Proof { |
| + t.Helper() |
| + seed, _, err := release.GenerateKey() |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + key, err := release.ParseSigningKey(seed, "test") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + p, err := release.SignProof(key, release.Proof{App: "console", Platform: "linux-arm64", SHA256: cliSHA, |
| + Version: "f4478b15", Prover: "dev", ActiveSince: time.Date(2026, 10, 6, 9, 0, 0, 0, time.UTC), |
| + ProvenAt: time.Date(2026, 10, 6, 9, 15, 0, 0, time.UTC)}) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + return p |
| +} |
| + |
| +// get against dev, put against flagship, byte for byte. |
| +func TestProofGetThenPutCarriesIt(t *testing.T) { |
| + ctx := context.Background() |
| + dev, flagship := release.NewDirStore(t.TempDir()), release.NewDirStore(t.TempDir()) |
| + p := signedCLIProof(t) |
| + if err := release.PutProof(ctx, dev, "bab", p); err != nil { |
| + t.Fatal(err) |
| + } |
| + var got bytes.Buffer |
| + if err := proofGet(ctx, dev, "bab", "console", cliSHA, "", &got); err != nil { |
| + t.Fatalf("proof get: %v", err) |
| + } |
| + var out bytes.Buffer |
| + if err := proofPut(ctx, flagship, "bab", "console", got.Bytes(), &out); err != nil { |
| + t.Fatalf("proof put: %v", err) |
| + } |
| + carried, err := release.GetProof(ctx, flagship, "bab", "console", cliSHA, "dev") |
| + if err != nil || carried.Sig != p.Sig { |
| + t.Fatalf("carried = %+v, %v", carried, err) |
| + } |
| +} |
| + |
| +func TestProofGetNotYetNamesTheBake(t *testing.T) { |
| + ctx := context.Background() |
| + dev := release.NewDirStore(t.TempDir()) |
| + if err := release.PutProofPending(ctx, dev, "bab", release.ProofPending{App: "console", Platform: "linux-arm64", |
| + SHA256: cliSHA, Version: "f4478b15", Prover: "dev", |
| + LiveSince: time.Date(2026, 10, 6, 9, 30, 30, 0, time.UTC), After: "15m"}); err != nil { |
| + t.Fatal(err) |
| + } |
| + err := proofGet(ctx, dev, "bab", "console", cliSHA, "", &bytes.Buffer{}) |
| + want := "not proven yet: " + cliSHA + " has been live on dev since 2026-10-06T09:30:30Z, proven after 15m" |
| + if err == nil || err.Error() != want { |
| + t.Fatalf("err = %v, want %q", err, want) |
| + } |
| + err = proofGet(ctx, release.NewDirStore(t.TempDir()), "bab", "console", cliSHA, "", &bytes.Buffer{}) |
| + if err == nil || !strings.Contains(err.Error(), "has not been seen live") { |
| + t.Fatalf("err = %v, want the never-seen text", err) |
| + } |
| +} |
| + |
| +func TestProofPutRefusesAnotherApp(t *testing.T) { |
| + var b bytes.Buffer |
| + p := signedCLIProof(t) |
| + raw, _ := json.MarshalIndent(p, "", " ") |
| + if err := proofPut(context.Background(), release.NewDirStore(t.TempDir()), "bab", "other", raw, &b); err == nil { |
| + t.Fatal("put a console proof under another app") |
| + } |
| + if err := proofPut(context.Background(), release.NewDirStore(t.TempDir()), "bab", "console", []byte(`{"schema":1}`), &b); err == nil { |
| + t.Fatal("put a malformed proof") |
| + } |
| +} |
| +``` |
| + |
| +- [ ] **Step 2: Run them to verify they fail** |
| + |
| +Run: `GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test . -run 'TestProof' -count=1` |
| +Expected: FAIL, `undefined: proofKeygen`. |
| + |
| +- [ ] **Step 3: Implement `proof_cli.go`** |
| + |
| +```go |
| +// carlos proof — D3 of the console release defences (spec |
| +// 2026-10-06-console-release-defences-design.md §4.2-§4.4). |
| +// |
| +// keygen runs as root on the proving console box. get runs against the |
| +// proving deployment's bucket and prints the proof; put runs against the |
| +// requiring deployment's bucket. Direct mode only (CARLOS_DEPLOYMENT_BUCKET or |
| +// _DIR): these are operator verbs, and a proof lives only in a bucket. put |
| +// checks shape, not the signature: only the requiring box holds the prover's |
| +// public key, and it checks. |
| +package main |
| + |
| +import ( |
| + "context" |
| + "encoding/json" |
| + "errors" |
| + "flag" |
| + "fmt" |
| + "io" |
| + "log" |
| + "os" |
| + "path" |
| + "strings" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/release" |
| +) |
| + |
| +const proofUsage = "usage: carlos proof keygen [--out <file>] | get --app <a> --sha256 <hex> [--prover <name>] | put --app <a> <file>" |
| + |
| +func runProof(args []string) { |
| + if len(args) == 0 { |
| + log.Fatal(proofUsage) |
| + } |
| + ctx := context.Background() |
| + switch args[0] { |
| + case "keygen": |
| + fs := flag.NewFlagSet("proof keygen", flag.ExitOnError) |
| + out := fs.String("out", "/etc/carlos/proving.key", "where to write the private key (root only, mode 0600)") |
| + fs.Usage = flagUsage(fs, "") |
| + fs.Parse(args[1:]) |
| + if fs.NArg() != 0 { |
| + log.Fatal(proofUsage) |
| + } |
| + if err := proofKeygen(*out, stdout); err != nil { |
| + log.Fatal(err) |
| + } |
| + case "get": |
| + fs := flag.NewFlagSet("proof get", flag.ExitOnError) |
| + app := appFlag(fs, "") |
| + sha := fs.String("sha256", "", "the artifact's sha256, hex") |
| + prover := fs.String("prover", "", "whose proof (default: the only one)") |
| + _, _, acctOf := accountFlags(fs) |
| + fs.Usage = flagUsage(fs, "") |
| + fs.Parse(args[1:]) |
| + if *app == "" || *sha == "" || fs.NArg() != 0 { |
| + log.Fatal(proofUsage) |
| + } |
| + if err := proofGet(ctx, openStore(), acctOf(), *app, *sha, *prover, stdout); err != nil { |
| + log.Fatal(err) |
| + } |
| + case "put": |
| + fs := flag.NewFlagSet("proof put", flag.ExitOnError) |
| + app := appFlag(fs, "") |
| + _, _, acctOf := accountFlags(fs) |
| + fs.Usage = flagUsage(fs, "") |
| + fs.Parse(args[1:]) |
| + if *app == "" || fs.NArg() != 1 { |
| + log.Fatal(proofUsage) |
| + } |
| + raw, err := os.ReadFile(fs.Arg(0)) |
| + if err != nil { |
| + log.Fatal(err) |
| + } |
| + if err := proofPut(ctx, openStore(), acctOf(), *app, raw, stdout); err != nil { |
| + log.Fatal(err) |
| + } |
| + default: |
| + log.Fatal(proofUsage) |
| + } |
| +} |
| + |
| +func proofKeygen(file string, w io.Writer) error { |
| + seed, pub, err := release.GenerateKey() |
| + if err != nil { |
| + return err |
| + } |
| + f, err := os.OpenFile(file, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o600) |
| + if errors.Is(err, os.ErrExist) { |
| + return fmt.Errorf("%s already exists. A proving key is never overwritten: delete it first if you mean to replace it, then update every require entry that names its public key", file) |
| + } |
| + if err != nil { |
| + return err |
| + } |
| + if _, err := f.WriteString(seed + "\n"); err != nil { |
| + f.Close() |
| + return err |
| + } |
| + if err := f.Close(); err != nil { |
| + return err |
| + } |
| + fmt.Fprintf(w, "wrote the proving key to %s (root only, mode 0600)\n", file) |
| + fmt.Fprintf(w, "public key: %s\n", pub) |
| + fmt.Fprintf(w, "a requiring box names it in its proof policy: {\"app\":\"<app>\",\"prover\":\"<prover>\",\"key\":%q}\n", pub) |
| + return nil |
| +} |
| + |
| +func proofGet(ctx context.Context, s release.Store, acct, app, sha, prover string, w io.Writer) error { |
| + if prover == "" { |
| + keys, err := s.List(ctx, release.ProofsPrefix(acct, app, sha)) |
| + if err != nil { |
| + return err |
| + } |
| + var names []string |
| + for _, k := range keys { |
| + base := path.Base(k) |
| + if strings.HasSuffix(base, ".pending.json") { |
| + continue |
| + } |
| + if n, ok := strings.CutSuffix(base, ".json"); ok { |
| + names = append(names, n) |
| + } |
| + } |
| + switch len(names) { |
| + case 0: |
| + return proofNotYet(ctx, s, acct, app, sha, keys) |
| + case 1: |
| + prover = names[0] |
| + default: |
| + return fmt.Errorf("%s has proofs from %s; choose one with --prover", sha, strings.Join(names, ", ")) |
| + } |
| + } |
| + p, err := release.GetProof(ctx, s, acct, app, sha, prover) |
| + if errors.Is(err, os.ErrNotExist) { |
| + return proofNotYet(ctx, s, acct, app, sha, []string{release.ProofPendingKey(acct, app, sha, prover)}) |
| + } |
| + if err != nil { |
| + return err |
| + } |
| + b, err := json.MarshalIndent(p, "", " ") |
| + if err != nil { |
| + return err |
| + } |
| + fmt.Fprintf(w, "%s\n", b) |
| + return nil |
| +} |
| + |
| +// proofNotYet explains a missing proof from the proving box's pending note |
| +// when there is one among keys. |
| +func proofNotYet(ctx context.Context, s release.Store, acct, app, sha string, keys []string) error { |
| + for _, k := range keys { |
| + prover, ok := strings.CutSuffix(path.Base(k), ".pending.json") |
| + if !ok { |
| + continue |
| + } |
| + pend, err := release.GetProofPending(ctx, s, acct, app, sha, prover) |
| + if err != nil { |
| + continue |
| + } |
| + return fmt.Errorf("not proven yet: %s has been live on %s since %s, proven after %s", |
| + sha, pend.Prover, pend.LiveSince.UTC().Format(time.RFC3339), pend.After) |
| + } |
| + return fmt.Errorf("not proven yet: %s has not been seen live on the proving deployment", sha) |
| +} |
| + |
| +func proofPut(ctx context.Context, s release.Store, acct, app string, raw []byte, w io.Writer) error { |
| + p, err := release.ParseProof(raw) |
| + if err != nil { |
| + return err |
| + } |
| + if p.App != app { |
| + return fmt.Errorf("this proof is for %s, not %s", p.App, app) |
| + } |
| + if err := release.PutProof(ctx, s, acct, p); err != nil { |
| + return err |
| + } |
| + fmt.Fprintf(w, "put %s's proof of %s %s (sha256 %s) at %s\n", p.Prover, p.App, p.Version, p.SHA256, |
| + release.ProofKey(acct, p.App, p.SHA256, p.Prover)) |
| + return nil |
| +} |
| +``` |
| + |
| +`main.go`: below `case "release-keygen":` / `runKeygen(args)`, add |
| + |
| +```go |
| + case "proof": |
| + runProof(args) |
| +``` |
| + |
| +and below the `release-keygen` usage line: |
| + |
| +``` |
| + proof dev proofs a box requires before adopting: keygen | get | put |
| +``` |
| + |
| +- [ ] **Step 4: Run the tests and the build** |
| + |
| +Run: `GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go test . -run 'TestProof' -count=1 && GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go build ./...` |
| +Expected: PASS, and the build succeeds. |
| + |
| +- [ ] **Step 5: Commit** |
| + |
| +```bash |
| +git add proof_cli.go proof_cli_test.go main.go |
| +git commit -m "carlos proof: keygen on the proving box, get and put to carry a proof" |
| +``` |
| + |
| +--- |
| + |
| +### Task 10: Operator note and the full gate |
| + |
| +**Files:** |
| +- Modify: `README.md`: the release-notes area under "Releasing the platform" that Plans 1 and 2 used |
| + |
| +- [ ] **Step 1: Add the note** |
| + |
| +```markdown |
| +**Dev proofs (console release defences, D3).** Off unless a box has |
| +`/etc/carlos/proof-policy.json` (root, mode 0644). On the proving console box |
| +(dev), run `carlos proof keygen` as root, then write the policy file with a |
| +`prove` block: the apps, the prover's name, how long a build must run |
| +(`after`, 15m) and `key_file`. Dev then mints a proof for each build that runs |
| +healthily for that long. On a requiring box (flagship), a `require` entry |
| +names the app, the prover and its public key. A new build of that app is then |
| +adopted only with a proof for those exact bytes, or if the box has already run |
| +that build healthily. Carry a proof with `carlos proof get` against dev and |
| +`carlos proof put` against flagship. `carlos status --live` shows a host that |
| +is waiting. A policy file that does not parse holds every named-unit route on |
| +its current build and says so in the journal. To switch the requirement off, |
| +delete the `require` entry (or the file) as root and run |
| +`carlos adopt --stage-only`. The box then logs once an hour that it is off. |
| +Write policy files over SSM, never through tofu `user_data`. |
| +``` |
| + |
| +This is operator documentation, not member-facing copy. Keep its sentences |
| +plain and avoid em dashes in new prose (`[[copy-voice-no-em-dashes]]`). The |
| +CLI strings in Tasks 4, 6, 7 and 9 follow the same rule. |
| + |
| +- [ ] **Step 2: Run the full gate** |
| + |
| +Run, niced and serial (`[[cpu-budget-docs-t1]]`): |
| + |
| +```bash |
| +GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false nice -n 10 go vet ./... && \ |
| +GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false nice -n 10 go test ./... -count=1 && \ |
| +GOOS=darwin GOARCH=arm64 GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go build ./... && \ |
| +GOOS=windows GOARCH=amd64 GOCACHE=$TMPDIR/gocache GOFLAGS=-buildvcs=false go build ./... |
| +``` |
| + |
| +Expected: all pass. The new host files use `/proc` paths and `os` calls only, |
| +no linux-only syscalls, so they need no build tag (`[[cross-compile-gate]]`). |
| + |
| +- [ ] **Step 3: Commit** |
| + |
| +```bash |
| +git add README.md |
| +git commit -m "README: operator note for dev proofs" |
| +``` |
amadan