amadan
#162: implementation plan, ten tasks from the record to the branch gate
Summary
each task carries its failing test, the exact edit sites read out of the code, and its own commit. Task 6 step 4 is the assertion the whole design rests on: the edge serves the account host from the local replica with no backhaul attempt.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 file changed,
+1556
−0
docs/superpowers/plans/2026-09-21-replica-placements.md+1556 −0
CI passed — run details
diff --git a/docs/superpowers/plans/2026-09-21-replica-placements.md b/docs/superpowers/plans/2026-09-21-replica-placements.md| new file mode 100644 |
| index 000000000..83c21951c |
| --- /dev/null |
| +++ b/docs/superpowers/plans/2026-09-21-replica-placements.md |
| @@ -0,0 +1,1556 @@ |
| +# Replica placements implementation plan |
| + |
| +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. |
| + |
| +**Goal:** An account can be served in a second placement by a replica instance: the edge on the replica's box resolves the account's public host to the replica's own route, instead of backhauling it home. |
| + |
| +**Architecture:** Three additive record fields (`Role`, `Replicas`, `Home`) carry the opt-in and the link. The replica's route row carries `replica_of`, written by `routeFor`; `domains.ReconcileAliases` reads that column and derives a `route_aliases` row mapping the account's public host to the replica's host, so `Edge.Reload`'s existing alias fold serves it locally and `backhaul` is never reached. The edges-file writer marks that alias `role: "replica"`, which every home-map reader already drops, so the account host stays single-valued fleet-wide. A siblings file written beside each instance's data carries the sibling's dial address and a per-pair HMAC key, so titogo's lease and drain calls need no new platform channel. |
| + |
| +**Tech Stack:** Go 1.x, SQLite (`internal/registry`), S3-shaped bucket store (`internal/release`), systemd units, `go test`. |
| + |
| +**Spec:** `docs/superpowers/specs/2026-09-21-replica-placements-design.md` (revision 2). Read it before Task 1; every task argues from a numbered section of it. |
| + |
| +## Global Constraints |
| + |
| +- **Additive only.** An account with no replicas must produce byte-identical output at every layer. Every new field is `omitempty`; every new column is `NOT NULL DEFAULT ''`. Spec §1. |
| +- **Go environment inside the sandbox:** `export GOCACHE=$TMPDIR/gocache GOFLAGS=-mod=mod GOPROXY=off` before every go command; env does not persist between Bash calls. Socket-binding tests (`internal/edge`, `internal/host`, `internal/activator`) fail inside the sandbox with "socket: operation not permitted" regardless of the change — run those packages outside the sandbox with `TMPDIR=/home/paulca/t GOTMPDIR=/home/paulca/t`. |
| +- **Worktree:** `/home/paulca/amadan.net/carlos/platform-162`, branch `replica-placements-162`. Never work in `/home/paulca/amadan.net/carlos/platform` — a different session has uncommitted work there. |
| +- **Commits:** `GIT_AUTHOR_NAME=claude GIT_AUTHOR_EMAIL='claude~paul@keymail.dev' GIT_COMMITTER_NAME=claude GIT_COMMITTER_EMAIL='claude~paul@keymail.dev' git commit`. End every message with `Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>`. Commit per task, never per file. |
| +- **Vocabulary, fixed across every task.** `Record.Role` (`""` | `"replica"`), `Record.Replicas []string`, `Record.Home string`, `provision.ValidRole`, `provision.ValidReplicas`, `provision.MaxReplicas = 4`, `registry.Route.ReplicaOf`, `registry.(*Registry).SetReplicaOf`, edges-file role string `"replica"`, env var `CARLOS_SIBLINGS_FILE`, siblings file name `siblings.json`. |
| +- **A new `carlos` verb is not done until `docs/site/cli.md` names it** (AGENTS.md; `internal/docsite` fails the gate otherwise). |
| + |
| +--- |
| + |
| +### Task 1: The record's three fields and their rails |
| + |
| +**Files:** |
| +- Modify: `internal/provision/provision.go` (struct close at `:142`; `ValidSteering` at `:354-367`; `ValidateShape` at `:392-451`; `ValidRegion` at `:551`) |
| +- Test: `internal/provision/provision_test.go` |
| + |
| +**Interfaces:** |
| +- Consumes: nothing. |
| +- Produces: `Record.Role string`, `Record.Replicas []string`, `Record.Home string`; `func ValidRole(s string) bool`; `const MaxReplicas = 4`; `func ValidReplicas(regions []string) ([]string, error)` returning the canonical (trimmed, deduplicated, sorted) set; `ValidateShape` refusals for every rail in spec §2. |
| + |
| +- [ ] **Step 1: Write the failing tests** |
| + |
| +Add to `internal/provision/provision_test.go`: |
| + |
| +```go |
| +func TestValidRole(t *testing.T) { |
| + for _, ok := range []string{"", "replica"} { |
| + if !ValidRole(ok) { |
| + t.Errorf("ValidRole(%q) = false, want true", ok) |
| + } |
| + } |
| + for _, bad := range []string{"Replica", "home", "read", "replica "} { |
| + if ValidRole(bad) { |
| + t.Errorf("ValidRole(%q) = true, want false", bad) |
| + } |
| + } |
| +} |
| + |
| +func TestValidReplicas(t *testing.T) { |
| + got, err := ValidReplicas([]string{" uk ", "ie", "uk", ""}) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if want := []string{"ie", "uk"}; !slices.Equal(got, want) { |
| + t.Errorf("ValidReplicas = %v, want %v", got, want) |
| + } |
| + if _, err := ValidReplicas([]string{"UK"}); err == nil { |
| + t.Error("ValidReplicas accepted an upper-case placement code") |
| + } |
| + if _, err := ValidReplicas([]string{"a", "b", "c", "d", "e"}); err == nil { |
| + t.Errorf("ValidReplicas accepted more than %d placements", MaxReplicas) |
| + } |
| +} |
| + |
| +func TestValidateShapeReplicaRails(t *testing.T) { |
| + base := Record{Schema: 1, Host: "acme.go.tito.io", Slug: "acme", App: "tito", Account: "abc", Channel: "stable"} |
| + replica := base |
| + replica.Role, replica.Home, replica.Region = "replica", "acme.go.tito.io", "uk" |
| + if err := ValidateShape(replica); err != nil { |
| + t.Fatalf("a well-formed replica record was refused: %v", err) |
| + } |
| + for name, mut := range map[string]func(*Record){ |
| + "unknown role": func(r *Record) { r.Role = "read" }, |
| + "replica of a static": func(r *Record) { r.Kind = "static" }, |
| + "replica with no home": func(r *Record) { r.Home = "" }, |
| + "replica with no region": func(r *Record) { r.Region = "" }, |
| + "replica carrying replicas": func(r *Record) { r.Replicas = []string{"ie"} }, |
| + } { |
| + rec := replica |
| + mut(&rec) |
| + if err := ValidateShape(rec); err == nil { |
| + t.Errorf("%s: ValidateShape accepted it", name) |
| + } |
| + } |
| + |
| + home := base |
| + home.Region, home.Steering, home.Replicas = "ie", "latency", []string{"uk"} |
| + if err := ValidateShape(home); err != nil { |
| + t.Fatalf("a well-formed home record was refused: %v", err) |
| + } |
| + for name, mut := range map[string]func(*Record){ |
| + "replicas without steering": func(r *Record) { r.Steering = "" }, |
| + "replica placement is the record's own region": func(r *Record) { r.Replicas = []string{"ie"} }, |
| + "empty placement": func(r *Record) { r.Replicas = []string{""} }, |
| + "bad placement": func(r *Record) { r.Replicas = []string{"UK"} }, |
| + "too many placements": func(r *Record) { r.Replicas = []string{"a", "b", "c", "d", "e"} }, |
| + } { |
| + rec := home |
| + mut(&rec) |
| + if err := ValidateShape(rec); err == nil { |
| + t.Errorf("%s: ValidateShape accepted it", name) |
| + } |
| + } |
| +} |
| + |
| +func TestRecordReplicaFieldsAreAdditive(t *testing.T) { |
| + raw, err := json.Marshal(Record{Schema: 1, Host: "acme.go.tito.io", Channel: "stable"}) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + for _, key := range []string{"role", "replicas", "home"} { |
| + if bytes.Contains(raw, []byte(`"`+key+`"`)) { |
| + t.Errorf("a record with no replica fields marshalled %q: the field is not omitempty", key) |
| + } |
| + } |
| +} |
| +``` |
| + |
| +- [ ] **Step 2: Run them and watch them fail** |
| + |
| +Run: `export GOCACHE=$TMPDIR/gocache GOFLAGS=-mod=mod GOPROXY=off; go test ./internal/provision/ -run 'ValidRole|ValidReplicas|ReplicaRails|ReplicaFieldsAreAdditive' -v` |
| +Expected: FAIL — `undefined: ValidRole`, `undefined: ValidReplicas`, `undefined: MaxReplicas`, and unknown fields `Role`/`Home`/`Replicas`. |
| + |
| +- [ ] **Step 3: Add the fields** |
| + |
| +In `internal/provision/provision.go`, immediately before the `Record` struct's closing brace (`:142`, after `Gate`): |
| + |
| +```go |
| + // --- replica placements (amadan carlos/platform #162) --- |
| + |
| + // Role is what this instance IS within its app: "" (the default every |
| + // record written before this field existed decodes to) is the serving |
| + // home, the instance whose data is authoritative. "replica" is a |
| + // SECOND instance of the same account holding a one-way mirror of it, |
| + // placed near the buyers (spec §2). Written only at create; no PATCH |
| + // flips a record into or out of the role, because which database is |
| + // authoritative is a data-plane fact, not a shape edit. ValidRole is |
| + // the door. |
| + Role string `json:"role,omitempty"` |
| + |
| + // Replicas names the placements an account's replicas live in — the |
| + // HOME record's own opt-in and the residence gate (spec §2). Never |
| + // defaulted, never inferred from Region. A record carrying any |
| + // placement here must also carry Steering "latency", or the account's |
| + // host resolves to one edge and the replica serves nothing (spec §5). |
| + Replicas []string `json:"replicas,omitempty"` |
| + |
| + // Home is the public host a REPLICA record mirrors: the home record's |
| + // own host. Written at create and never converged — repointing a |
| + // replica is delete-and-recreate, because what it serves is a mirror |
| + // OF something. Carried onto the route row as replica_of, which is |
| + // what the alias derivation and the edges-file writer read (spec §3). |
| + Home string `json:"home,omitempty"` |
| +``` |
| + |
| +- [ ] **Step 4: Add `ValidRole`, `MaxReplicas` and `ValidReplicas`** |
| + |
| +In `internal/provision/provision.go`, directly after `ValidSteering` (`:367`): |
| + |
| +```go |
| +// ValidRole pins the record's role vocabulary (spec §2): "" (the serving |
| +// home — every record written before the field existed decodes to it, |
| +// which is what makes the field additive) or "replica". Case-sensitive |
| +// and closed, ValidSteering's discipline: a value a future reader has |
| +// never heard of is refused where it is typed, rather than discovered as |
| +// a row nothing routes for. |
| +func ValidRole(s string) bool { |
| + switch s { |
| + case "", "replica": |
| + return true |
| + } |
| + return false |
| +} |
| + |
| +// MaxReplicas caps the placement set one record may name. Each entry is a |
| +// decision about where an account's order data lives until a drain |
| +// confirms (spec §2), not a cache hint, so the cap is small and deliberate. |
| +const MaxReplicas = 4 |
| + |
| +// ValidReplicas normalizes and validates a replica placement set: trim, |
| +// drop empties, deduplicate, sort into the canonical order, and check each |
| +// element against ValidRegion — the same vocabulary a record's own Region |
| +// speaks. Exported because the console's door canonicalizes the request |
| +// body through the SAME derivation ValidateShape re-checks, so neither |
| +// side trusts the other about what is in the bucket. |
| +func ValidReplicas(regions []string) ([]string, error) { |
| + if len(regions) > MaxReplicas { |
| + return nil, fmt.Errorf("replicas: %d placements, at most %d — a replica holds the account's order data in the placement it names until a drain confirms, so each one is a residence decision", len(regions), MaxReplicas) |
| + } |
| + seen := make(map[string]bool, len(regions)) |
| + out := make([]string, 0, len(regions)) |
| + for _, r := range regions { |
| + r = strings.TrimSpace(r) |
| + if r == "" { |
| + return nil, errors.New("replicas: empty placement code") |
| + } |
| + if !ValidRegion(r) { |
| + return nil, fmt.Errorf("replica placement %q: want a placement code (e.g. ie, uk)", r) |
| + } |
| + if seen[r] { |
| + continue |
| + } |
| + seen[r] = true |
| + out = append(out, r) |
| + } |
| + sort.Strings(out) |
| + return out, nil |
| +} |
| +``` |
| + |
| +- [ ] **Step 5: Add the `ValidateShape` rails** |
| + |
| +In `ValidateShape`, directly after the `ValidSteering` check (`:396-398`): |
| + |
| +```go |
| + if !ValidRole(rec.Role) { |
| + return fmt.Errorf("role %q: want \"\" or \"replica\"", rec.Role) |
| + } |
| + if rec.Role == "replica" { |
| + // A replica is a process holding a mirror of live data — never a |
| + // file tree, never a foreign upstream the platform did not spawn. |
| + if rec.KindOrDefault() != "instance" { |
| + return errors.New("role replica with kind " + rec.KindOrDefault() + ": a replica is an instance of the account — drop kind or drop role") |
| + } |
| + if rec.Home == "" { |
| + return errors.New("role replica with no home: a replica names the public host it mirrors") |
| + } |
| + // ValidRegion("") is true, so the placement rail is written out. |
| + if rec.Region == "" { |
| + return errors.New("role replica with no region: a replica is always in a named placement") |
| + } |
| + if len(rec.Replicas) > 0 { |
| + return errors.New("role replica with replicas: a replica of a replica is not a shape the platform carries") |
| + } |
| + } |
| + if len(rec.Replicas) > 0 { |
| + norm, err := ValidReplicas(rec.Replicas) |
| + if err != nil { |
| + return err |
| + } |
| + if slices.Contains(norm, rec.Region) && rec.Region != "" { |
| + return fmt.Errorf("replica placement %q is the record's own placement: a replica lives somewhere else", rec.Region) |
| + } |
| + // The home region normalizes to "" on a record (homeRegionIn, |
| + // console apiinstancemove.go), so this rail cannot see a home in |
| + // the deployment's own region naming it as a replica placement. |
| + // That comparison lives at the door, which knows the code (spec §2). |
| + if rec.Steering != "latency" { |
| + return errors.New("replicas without steering latency: the account's host would resolve to one edge and the replica would serve nothing — set steering first") |
| + } |
| + } |
| +``` |
| + |
| +- [ ] **Step 6: Run the tests and watch them pass** |
| + |
| +Run: `export GOCACHE=$TMPDIR/gocache GOFLAGS=-mod=mod GOPROXY=off; go test ./internal/provision/ -run 'ValidRole|ValidReplicas|ReplicaRails|ReplicaFieldsAreAdditive' -v` |
| +Expected: PASS, all four. |
| + |
| +- [ ] **Step 7: Run the whole package and commit** |
| + |
| +Run: `export GOCACHE=$TMPDIR/gocache GOFLAGS=-mod=mod GOPROXY=off; go test ./internal/provision/` |
| +Expected: PASS. If an existing test fails, it is a real regression — the fields are additive and nothing else changed. |
| + |
| +```bash |
| +git add internal/provision/provision.go internal/provision/provision_test.go |
| +GIT_AUTHOR_NAME=claude GIT_AUTHOR_EMAIL='claude~paul@keymail.dev' GIT_COMMITTER_NAME=claude GIT_COMMITTER_EMAIL='claude~paul@keymail.dev' \ |
| + git commit -m "#162: Role, Replicas and Home on the instance record |
| + |
| +Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>" |
| +``` |
| + |
| +--- |
| + |
| +### Task 2: `routes.replica_of` — the column both passes read |
| + |
| +**Files:** |
| +- Modify: `internal/registry/registry.go` (`Route` close `:260`; migrations `:442-473`; `routeCols` `:722-727`; `scanRoute` `:729-744`; `Upsert` `:916-971`; `SetSteering` `:1300-1312` is the setter precedent) |
| +- Test: `internal/registry/registry_test.go` |
| + |
| +**Interfaces:** |
| +- Consumes: nothing. |
| +- Produces: `registry.Route.ReplicaOf string`; `func (r *Registry) SetReplicaOf(host, replicaOf string) error`. |
| + |
| +`routeCols` is 40 names today and its doc comment says so. Adding one means FIVE edits — the comment's count, `routeCols`, `scanRoute`'s destinations, `Upsert`'s placeholder row (40 `?` become 41) and `Upsert`'s argument list. A miscount is a silent column shift; `TestUpsertRoundTripsAllColumns` is what catches it. |
| + |
| +- [ ] **Step 1: Write the failing test** |
| + |
| +```go |
| +func TestReplicaOfRoundTrips(t *testing.T) { |
| + reg, err := Open(filepath.Join(t.TempDir(), "registry.db")) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + defer reg.Close() |
| + if err := reg.Upsert(Route{Host: "r7x2.go.tito.io", Kind: "instance", Status: "ready", ReplicaOf: "acme.go.tito.io"}); err != nil { |
| + t.Fatal(err) |
| + } |
| + got, err := reg.Get("r7x2.go.tito.io") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if got.ReplicaOf != "acme.go.tito.io" { |
| + t.Fatalf("ReplicaOf = %q, want acme.go.tito.io", got.ReplicaOf) |
| + } |
| + if err := reg.SetReplicaOf("r7x2.go.tito.io", ""); err != nil { |
| + t.Fatal(err) |
| + } |
| + if got, err := reg.Get("r7x2.go.tito.io"); err != nil { |
| + t.Fatal(err) |
| + } else if got.ReplicaOf != "" { |
| + t.Fatalf("SetReplicaOf did not clear the column: %q", got.ReplicaOf) |
| + } |
| + if err := reg.SetReplicaOf("nosuch.go.tito.io", "x"); err == nil { |
| + t.Error("SetReplicaOf on an unknown host returned nil, want an error") |
| + } |
| +} |
| +``` |
| + |
| +If `Get` is not this package's row accessor, use whatever `registry_test.go` already uses. |
| + |
| +- [ ] **Step 2: Run it and watch it fail** |
| + |
| +Run: `export GOCACHE=$TMPDIR/gocache GOFLAGS=-mod=mod GOPROXY=off; go test ./internal/registry/ -run ReplicaOf -v` |
| +Expected: FAIL — `unknown field ReplicaOf`, `reg.SetReplicaOf undefined`. |
| + |
| +- [ ] **Step 3: Add the field** |
| + |
| +In `Route`, immediately before `CreatedAt int64` (`:259`): |
| + |
| +```go |
| + // ReplicaOf is the public host this route is a replica OF (#162): the |
| + // home record's host, carried from Record.Home by routeFor's mint and |
| + // held steady by convergeOwned. "" on every row that predates the |
| + // column and on every ordinary route = today's routing, byte for byte. |
| + // |
| + // It exists because two passes must know this row is an account's |
| + // replica and neither can read bucket records: |
| + // domains.ReconcileAliases derives the alias `ReplicaOf -> Host` from |
| + // it (internal/provision imports internal/domains, so the reverse is a |
| + // cycle), and the edges-file writer marks that alias role "replica". |
| + // Not in Fingerprint(), in Steering's shadow: it changes what the |
| + // ALIAS derivation produces, not what this box dials — and the alias |
| + // row it produces IS in the fingerprint. |
| + ReplicaOf string |
| +``` |
| + |
| +- [ ] **Step 4: Migration, `routeCols`, `scanRoute`, `Upsert`** |
| + |
| +Append to the migration list (after `max_run_seconds`, `:472`): |
| + |
| +```go |
| + `ALTER TABLE routes ADD COLUMN replica_of TEXT NOT NULL DEFAULT ''`, |
| +``` |
| + |
| +`routeCols`: change "40 names" to "41 names" in the doc comment and append `, replica_of` to its last line. `scanRoute`: append `&t.ReplicaOf` to the `scan(...)` arguments. `Upsert`: one more `?` in `VALUES` (41), `replica_of=excluded.replica_of,` beside `steering=excluded.steering,` in the `ON CONFLICT` list (a plain overwrite, like steering — the record is authoritative both ways), and `t.ReplicaOf` appended to the argument list. |
| + |
| +- [ ] **Step 5: Add the setter** |
| + |
| +After `SetSteering` (`:1312`): |
| + |
| +```go |
| +// SetReplicaOf repoints a route's replica link (#162) as a narrow UPDATE. |
| +// The record's value is authoritative in BOTH directions, including down |
| +// to "": a row that stopped being a replica must lose the link, or the |
| +// domains pass goes on deriving an alias that serves an account from a |
| +// box its record no longer places it in. Errors on an unmatched host, |
| +// SetChannel's contract: the reconciler must notice a row vanishing |
| +// underneath rather than silently do nothing. |
| +func (r *Registry) SetReplicaOf(host, replicaOf string) error { |
| + res, err := r.sql.Exec(`UPDATE routes SET replica_of = ? WHERE host = ?`, replicaOf, host) |
| + if err != nil { |
| + return err |
| + } |
| + n, err := res.RowsAffected() |
| + if err != nil { |
| + return err |
| + } |
| + if n == 0 { |
| + return fmt.Errorf("no route for host %q", host) |
| + } |
| + return nil |
| +} |
| +``` |
| + |
| +- [ ] **Step 6: Run the package** |
| + |
| +Run: `export GOCACHE=$TMPDIR/gocache GOFLAGS=-mod=mod GOPROXY=off; go test ./internal/registry/` |
| +Expected: PASS, including `TestUpsertRoundTripsAllColumns`. |
| + |
| +- [ ] **Step 7: Commit** |
| + |
| +```bash |
| +git add internal/registry/registry.go internal/registry/registry_test.go |
| +GIT_AUTHOR_NAME=claude GIT_AUTHOR_EMAIL='claude~paul@keymail.dev' GIT_COMMITTER_NAME=claude GIT_COMMITTER_EMAIL='claude~paul@keymail.dev' \ |
| + git commit -m "#162: routes.replica_of, the mark both passes read |
| + |
| +Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>" |
| +``` |
| + |
| +--- |
| + |
| +### Task 3: Reconcile mints a replica route — `replica_of`, the home's channel, the protect rule |
| + |
| +**Files:** |
| +- Modify: `internal/provision/reconcile.go` (`ReconcileConfig` `:34`; `routeFor` `:348-364`; `convergeOwned` `:572-628`; record loop `:649-736`; removal phase `:1048-1090`) |
| +- Test: `internal/provision/reconcile_test.go` |
| + |
| +**Interfaces:** |
| +- Consumes: Task 1's `Record.Role`/`Home`, Task 2's `Route.ReplicaOf` and `SetReplicaOf`. |
| +- Produces: `routeFor(rec Record, home Record, dataDir, seed string) registry.Route` (new second parameter — update call sites `:871`, `:947`, `:965` and the test call sites at `reconcile_test.go:925, 2508, 2527, 2540, 2556, 2704, 2708`); `convergeOwned(reg, existing, rec, home Record, dryRun bool)` (call sites `:888`, `:982`, `:997`); `ReconcileConfig.OnRecords func(all map[string]Record)`, called once per pass with every decoded record, in scope or not — Task 9 consumes it. |
| + |
| +Spec §2. Three code facts decide this: a record with `Channel == ""` is protected at `:718` before scope is asked; `recs` is in-scope only (`:654`, `:725`), so the home record is dropped on the replica's box; and `convergeOwned` converges the row back to the record's own channel every pass (`:573`). |
| + |
| +- [ ] **Step 1: Write the failing tests** |
| + |
| +Copy the shape of `TestReconcileConvergesSteering` (`reconcile_test.go:523`): |
| + |
| +```go |
| +func replicaSetup(t *testing.T) (*release.DirStore, string, Record, Record) { |
| + t.Helper() |
| + s, sqid := setup(t) |
| + home := acmeRecord(sqid) |
| + home.Region, home.Steering, home.Replicas = "ie", "latency", []string{"uk"} |
| + if err := CreateRecord(ctx, s, home); err != nil { |
| + t.Fatal(err) |
| + } |
| + replica := acmeRecord(sqid) |
| + replica.Host, replica.Slug = "r7x2k9m4qa.go.tito.io", "r7x2k9m4qa" |
| + replica.Role, replica.Home, replica.Region = "replica", home.Host, "uk" |
| + replica.Replicas, replica.Steering = nil, "" |
| + if err := CreateRecord(ctx, s, replica); err != nil { |
| + t.Fatal(err) |
| + } |
| + return s, sqid, home, replica |
| +} |
| + |
| +func TestReconcileMintsReplicaRouteOnItsOwnBoxOnly(t *testing.T) { |
| + s, _, _, replica := replicaSetup(t) |
| + uk := regWith(t) |
| + if err := Reconcile(ctx, s, uk, ReconcileConfig{Region: "uk", Pools: []string{"tito"}, DataDir: "/data"}); err != nil { |
| + t.Fatal(err) |
| + } |
| + rt := mustGet(t, uk, replica.Host) |
| + if rt.ReplicaOf != "acme.go.tito.io" { |
| + t.Errorf("ReplicaOf = %q, want acme.go.tito.io", rt.ReplicaOf) |
| + } |
| + if rt.Channel != "stable" { |
| + t.Errorf("channel = %q, want the home's stable", rt.Channel) |
| + } |
| + ie := regWith(t) |
| + if err := Reconcile(ctx, s, ie, ReconcileConfig{Region: "ie", Pools: []string{"tito"}, DataDir: "/data"}); err != nil { |
| + t.Fatal(err) |
| + } |
| + if _, err := ie.Get(replica.Host); err == nil { |
| + t.Error("the home's box minted a route for the replica") |
| + } |
| + if got := mustGet(t, ie, "acme.go.tito.io"); got.ReplicaOf != "" { |
| + t.Errorf("the home's own row carries ReplicaOf %q, want empty", got.ReplicaOf) |
| + } |
| +} |
| + |
| +func TestReconcileReplicaFollowsHomeChannel(t *testing.T) { |
| + s, sqid, home, replica := replicaSetup(t) |
| + uk := regWith(t) |
| + cfg := ReconcileConfig{Region: "uk", Pools: []string{"tito"}, DataDir: "/data"} |
| + if err := Reconcile(ctx, s, uk, cfg); err != nil { |
| + t.Fatal(err) |
| + } |
| + home.Channel = "canary" |
| + putReplace(t, s, RecordKey(sqid, "tito", home.Host), mustJSON(t, home)) |
| + if err := Reconcile(ctx, s, uk, cfg); err != nil { |
| + t.Fatal(err) |
| + } |
| + if got := mustGet(t, uk, replica.Host).Channel; got != "canary" { |
| + t.Errorf("channel = %q, want canary: the replica must follow its home", got) |
| + } |
| +} |
| + |
| +func TestReconcileProtectsReplicaWhoseHomeIsGone(t *testing.T) { |
| + s, sqid, home, replica := replicaSetup(t) |
| + uk := regWith(t) |
| + cfg := ReconcileConfig{Region: "uk", Pools: []string{"tito"}, DataDir: "/data"} |
| + if err := Reconcile(ctx, s, uk, cfg); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := DeleteRecord(ctx, s, sqid, "tito", home.Host); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := Reconcile(ctx, s, uk, cfg); err != nil { |
| + t.Fatal(err) |
| + } |
| + if _, err := uk.Get(replica.Host); err != nil { |
| + t.Fatal("the replica's route was removed when its home record went: a mirror of nothing is protected, not retired") |
| + } |
| +} |
| + |
| +func TestReconcileReportsEveryRecord(t *testing.T) { |
| + s, _, home, replica := replicaSetup(t) |
| + var seen map[string]Record |
| + cfg := ReconcileConfig{Region: "uk", Pools: []string{"tito"}, DataDir: "/data", |
| + OnRecords: func(all map[string]Record) { seen = all }} |
| + if err := Reconcile(ctx, s, regWith(t), cfg); err != nil { |
| + t.Fatal(err) |
| + } |
| + if _, ok := seen[home.Host]; !ok { |
| + t.Error("OnRecords did not carry the out-of-scope home record") |
| + } |
| + if _, ok := seen[replica.Host]; !ok { |
| + t.Error("OnRecords did not carry the in-scope replica record") |
| + } |
| +} |
| +``` |
| + |
| +- [ ] **Step 2: Run them and watch them fail** |
| + |
| +Run: `export GOCACHE=$TMPDIR/gocache GOFLAGS=-mod=mod GOPROXY=off; go test ./internal/provision/ -run 'Replica|ReportsEveryRecord' -v` |
| +Expected: FAIL — `OnRecords` undefined; no `ReplicaOf`; channel wrong. |
| + |
| +- [ ] **Step 3: Collect every record, not only the in-scope ones** |
| + |
| +Beside `recs` (`:654`): |
| + |
| +```go |
| + all := map[string]Record{} // host -> record, EVERY decoded record |
| +``` |
| + |
| +Immediately before the scope check at `:725` (so the shape and channel rails have already passed): |
| + |
| +```go |
| + // Every decoded record, in scope or not: a replica's box needs its |
| + // HOME record (whose Region is another box) for the route's |
| + // channel, and the siblings pass needs both sides. Scope is about |
| + // what this box MINTS, never about what it may read. |
| + all[rec.Host] = rec |
| +``` |
| + |
| +After the collection loop closes (`:736`): |
| + |
| +```go |
| + if cfg.OnRecords != nil { |
| + cfg.OnRecords(all) |
| + } |
| +``` |
| + |
| +And in `ReconcileConfig`: |
| + |
| +```go |
| + // OnRecords, when set, is called once per pass with every record this |
| + // pass decoded — in scope or not. The siblings-file pass (spec §7) |
| + // needs both sides of a home/replica pair and only one is ever in |
| + // scope on a given box; this hands it the map the pass already built |
| + // rather than making it list apps/ a second time. |
| + OnRecords func(all map[string]Record) |
| +``` |
| + |
| +- [ ] **Step 4: Teach the mint and the converge about the home record** |
| + |
| +`routeFor` becomes `func routeFor(rec Record, home Record, dataDir, seed string) registry.Route`, and inside, after `Steering: rec.Steering,` is set: |
| + |
| +```go |
| + if rec.Role == "replica" { |
| + // The link the domains pass and the edges-file writer read |
| + // (spec §3), and the HOME's channel — one channel across the pair, |
| + // and the replica record carries none of its own (spec §2). A |
| + // caller that could not find the home record must not reach here. |
| + rt.ReplicaOf = rec.Home |
| + rt.Channel = home.Channel |
| + } |
| +``` |
| + |
| +`convergeOwned` becomes `func convergeOwned(reg *registry.Registry, existing registry.Route, rec Record, home Record, dryRun bool)`, its channel arm reads: |
| + |
| +```go |
| + channel := rec.Channel |
| + if rec.Role == "replica" { |
| + channel = home.Channel |
| + } |
| + if channel != "" && existing.Channel != channel { |
| + // ... unchanged body, using `channel` in place of rec.Channel |
| + } |
| +``` |
| + |
| +and gains a link arm beside the steering arm: |
| + |
| +```go |
| + if rec.Role == "replica" && existing.ReplicaOf != rec.Home { |
| + if dryRun { |
| + log.Printf("provision: dry-run: would converge %s replica_of %q -> %q", rec.Host, existing.ReplicaOf, rec.Home) |
| + } else if err := reg.SetReplicaOf(rec.Host, rec.Home); err != nil { |
| + log.Printf("provision: converge replica_of on %s: %v", rec.Host, err) |
| + } else { |
| + log.Printf("provision: converged %s replica_of %q -> %q", rec.Host, existing.ReplicaOf, rec.Home) |
| + } |
| + } |
| +``` |
| + |
| +Every call site passes `all[rec.Home]`, which is a zero `Record{}` for a non-replica and is read by neither branch. |
| + |
| +- [ ] **Step 5: Protect a replica whose home is missing** |
| + |
| +In the mint/converge arm, before either call: |
| + |
| +```go |
| + if rec.Role == "replica" { |
| + if _, ok := all[rec.Home]; !ok { |
| + // A mirror of nothing. Protect rather than mint on a guess: |
| + // the row keeps serving what it serves and the removal |
| + // phase leaves it alone (spec §2). |
| + log.Printf("provision: not minting replica %s: its home record %s is absent or unreadable this pass", rec.Host, rec.Home) |
| + protect[rec.Host] = true |
| + continue |
| + } |
| + } |
| +``` |
| + |
| +`protect` is the pass's local map at `:663`; the removal phase reads it at `:1076`. |
| + |
| +- [ ] **Step 6: Run and watch them pass** |
| + |
| +Run: `export GOCACHE=$TMPDIR/gocache GOFLAGS=-mod=mod GOPROXY=off; go test ./internal/provision/` |
| +Expected: PASS. The seven existing `routeFor` test call sites need the new `Record{}` argument — mechanical. |
| + |
| +- [ ] **Step 7: Commit** |
| + |
| +```bash |
| +git add internal/provision/reconcile.go internal/provision/reconcile_test.go |
| +GIT_AUTHOR_NAME=claude GIT_AUTHOR_EMAIL='claude~paul@keymail.dev' GIT_COMMITTER_NAME=claude GIT_COMMITTER_EMAIL='claude~paul@keymail.dev' \ |
| + git commit -m "#162: mint a replica route with its home's channel and replica_of |
| + |
| +Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>" |
| +``` |
| + |
| +--- |
| + |
| +### Task 4: The domains pass derives the replica alias |
| + |
| +**Files:** |
| +- Modify: `internal/domains/aliases.go` (`ReconcileAliases` `:68-84`, sweep `:361-368`) |
| +- Test: `internal/domains/aliases_test.go` |
| + |
| +**Interfaces:** |
| +- Consumes: Task 2's `Route.ReplicaOf`. |
| +- Produces: a `route_aliases` row `{Alias: rt.ReplicaOf, Host: rt.Host, Account: rt.Account, App: rt.App, CreatedAt: now().Unix()}` for every local route with a non-empty `ReplicaOf`, present in `desired` so the sweep keeps it. |
| + |
| +Spec §3. `byHost` is built from `reg.List()` (`:78-81`) — exactly where `ReplicaOf` now lives. Helpers in this package: `aliasRegWith`, `putClaim`, `aliasNow` (fixed clock `time.Unix(1755000000, 0)`). |
| + |
| +- [ ] **Step 1: Write the failing tests** |
| + |
| +```go |
| +func TestReconcileAliasesDerivesReplicaAlias(t *testing.T) { |
| + s := release.NewDirStore(t.TempDir()) |
| + reg := aliasRegWith(t, registry.Route{ |
| + Host: "r7x2k9m4qa.go.tito.io", App: "tito", Account: "bab", Kind: "instance", |
| + Status: "ready", ReplicaOf: "acme.go.tito.io", |
| + }) |
| + for i := 0; i < 2; i++ { // twice: the second pass is the sweep's chance to eat it |
| + if err := ReconcileAliases(context.Background(), s, reg, aliasNow(), nil, nil); err != nil { |
| + t.Fatal(err) |
| + } |
| + } |
| + aliases, err := reg.ListAliases() |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if len(aliases) != 1 { |
| + t.Fatalf("got %d aliases, want 1: %+v", len(aliases), aliases) |
| + } |
| + if a := aliases[0]; a.Alias != "acme.go.tito.io" || a.Host != "r7x2k9m4qa.go.tito.io" || |
| + a.Account != "bab" || a.App != "tito" { |
| + t.Fatalf("replica alias row mismatch: %+v", a) |
| + } |
| +} |
| + |
| +func TestReconcileAliasesSweepsAReplicaAliasWhoseRouteIsGone(t *testing.T) { |
| + s := release.NewDirStore(t.TempDir()) |
| + reg := aliasRegWith(t, registry.Route{ |
| + Host: "r7x2k9m4qa.go.tito.io", App: "tito", Account: "bab", Kind: "instance", |
| + Status: "ready", ReplicaOf: "acme.go.tito.io", |
| + }) |
| + if err := ReconcileAliases(context.Background(), s, reg, aliasNow(), nil, nil); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := reg.Delete("r7x2k9m4qa.go.tito.io"); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := ReconcileAliases(context.Background(), s, reg, aliasNow(), nil, nil); err != nil { |
| + t.Fatal(err) |
| + } |
| + if aliases, err := reg.ListAliases(); err != nil { |
| + t.Fatal(err) |
| + } else if len(aliases) != 0 { |
| + t.Fatalf("the replica alias outlived its route: %+v", aliases) |
| + } |
| +} |
| + |
| +func TestReconcileAliasesReplicaAliasLosesToARealRoute(t *testing.T) { |
| + s := release.NewDirStore(t.TempDir()) |
| + reg := aliasRegWith(t, |
| + registry.Route{Host: "acme.go.tito.io", App: "tito", Account: "bab", Kind: "instance", Status: "ready"}, |
| + registry.Route{Host: "r7x2k9m4qa.go.tito.io", App: "tito", Account: "bab", Kind: "instance", |
| + Status: "ready", ReplicaOf: "acme.go.tito.io"}, |
| + ) |
| + if err := ReconcileAliases(context.Background(), s, reg, aliasNow(), nil, nil); err != nil { |
| + t.Fatal(err) |
| + } |
| + if aliases, err := reg.ListAliases(); err != nil { |
| + t.Fatal(err) |
| + } else if len(aliases) != 0 { |
| + t.Fatalf("an alias was derived over a real local route: %+v", aliases) |
| + } |
| +} |
| +``` |
| + |
| +- [ ] **Step 2: Run them and watch them fail** |
| + |
| +Run: `export GOCACHE=$TMPDIR/gocache GOFLAGS=-mod=mod GOPROXY=off; go test ./internal/domains/ -run ReplicaAlias -v` |
| +Expected: FAIL — 0 aliases, want 1. |
| + |
| +- [ ] **Step 3: Derive it** |
| + |
| +After `desired := map[string]bool{}` (`:84`), before the claims loop: |
| + |
| +```go |
| + // Replica placements (#162): a route carrying ReplicaOf is an account's |
| + // replica on this box, and the account's public host must resolve to it |
| + // HERE while every other box goes on backhauling that host home. |
| + // Derived from the route row because this pass cannot read records — |
| + // internal/provision imports this package, so the reverse is a cycle. |
| + for _, rt := range routes { |
| + if rt.ReplicaOf == "" { |
| + continue |
| + } |
| + if _, taken := byHost[rt.ReplicaOf]; taken { |
| + // A real local route owns that name: the home landed on this |
| + // box. Routes beat aliases in the edge's fold anyway |
| + // (edge.go:747-749), so this row would never win. |
| + log.Printf("domains: replica alias %s skipped: a real route owns that host", rt.ReplicaOf) |
| + continue |
| + } |
| + desired[rt.ReplicaOf] = true |
| + if err := reg.UpsertAlias(registry.RouteAlias{ |
| + Alias: rt.ReplicaOf, Host: rt.Host, Account: rt.Account, App: rt.App, |
| + CreatedAt: now().Unix(), |
| + }); err != nil { |
| + log.Printf("domains: upsert replica alias %s: %v", rt.ReplicaOf, err) |
| + } |
| + } |
| +``` |
| + |
| +- [ ] **Step 4: Run the package and commit** |
| + |
| +Run: `export GOCACHE=$TMPDIR/gocache GOFLAGS=-mod=mod GOPROXY=off; go test ./internal/domains/` |
| +Expected: PASS. |
| + |
| +```bash |
| +git add internal/domains/aliases.go internal/domains/aliases_test.go |
| +GIT_AUTHOR_NAME=claude GIT_AUTHOR_EMAIL='claude~paul@keymail.dev' GIT_COMMITTER_NAME=claude GIT_COMMITTER_EMAIL='claude~paul@keymail.dev' \ |
| + git commit -m "#162: the domains pass derives a replica's alias from replica_of |
| + |
| +Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>" |
| +``` |
| + |
| +--- |
| + |
| +### Task 5: The edges file marks the replica alias, and publishes it unconditionally |
| + |
| +**Files:** |
| +- Modify: `internal/host/edgesfile.go` (`edgeHost` `:36-59`; alias section `:246-269`) |
| +- Test: `internal/host/edgesfile_test.go` |
| + |
| +**Interfaces:** |
| +- Consumes: Task 2's `Route.ReplicaOf`, Task 4's alias row. |
| +- Produces: an edges-file entry `{"role":"replica","alias":true}` for the account's public host on the replica's box. No new struct field — `Role` carries it. |
| + |
| +Spec §4. The replica's own host keeps `role: "home"`; that is how home and every other box reach it by backhaul. The `dns_status == "pointing"` gate (`:255`) must not apply to a replica alias: it exists for tenant-chosen names the shared issuer would order for, and §5's whole point is that the name does not resolve everywhere yet. |
| + |
| +- [ ] **Step 1: Write the failing test** |
| + |
| +```go |
| +func TestEdgesFilePublishesReplicaAliasWithoutADNSStamp(t *testing.T) { |
| + s, reg := release.NewDirStore(t.TempDir()), edgesTestRegistry(t) |
| + if err := reg.Upsert(registry.Route{ |
| + Host: "r7x2k9m4qa.go.tito.io", Kind: "instance", Status: "ready", |
| + Socket: "/run/carlos/r7x2k9m4qa.go.tito.io.sock", Provisioned: true, |
| + Account: "bab", App: "tito", ReplicaOf: "acme.go.tito.io", |
| + }); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := reg.UpsertAlias(registry.RouteAlias{ |
| + Alias: "acme.go.tito.io", Host: "r7x2k9m4qa.go.tito.io", |
| + Account: "bab", App: "tito", // deliberately unstamped |
| + }); err != nil { |
| + t.Fatal(err) |
| + } |
| + var lastBody []byte |
| + if err := edgesFilePass(context.Background(), s, reg, |
| + edgesFile{Box: "uk-1", Placement: "uk"}, "203.0.113.9", &lastBody); err != nil { |
| + t.Fatal(err) |
| + } |
| + got, _, err := s.Get(context.Background(), "control/edges/uk-1.json") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + var f edgesFile |
| + if err := json.Unmarshal(got, &f); err != nil { |
| + t.Fatal(err) |
| + } |
| + if h, ok := f.Hosts["acme.go.tito.io"]; !ok || h.Role != "replica" || !h.Alias { |
| + t.Fatalf("public host entry = %+v, want role replica alias true: %+v", h, f.Hosts) |
| + } |
| + if h, ok := f.Hosts["r7x2k9m4qa.go.tito.io"]; !ok || h.Role != "home" { |
| + t.Fatalf("the replica's own host must stay role home: %+v", f.Hosts) |
| + } |
| +} |
| +``` |
| + |
| +- [ ] **Step 2: Run it and watch it fail** |
| + |
| +Run: `export GOCACHE=$TMPDIR/gocache GOFLAGS=-mod=mod GOPROXY=off; go test ./internal/host/ -run ReplicaAlias -v` |
| +Expected: FAIL — the public host is absent (the unstamped alias is withheld). |
| + |
| +- [ ] **Step 3: Mark it and exempt it from the stamp gate** |
| + |
| +In the alias loop, from the `a.DNSStatus != "pointing"` check: |
| + |
| +```go |
| + // A replica alias (#162) is a platform row: the domains pass derived |
| + // it from a route THIS box wrote (routes.replica_of), and the name |
| + // is the account's own public host, already issued and mirrored for |
| + // its home. Published without waiting for a dns_status stamp — that |
| + // stamp answers a question about a tenant-chosen name, and |
| + // withholding a replica alias would withhold exactly the rows this |
| + // feature exists to publish, silently. |
| + replica := byHost[a.Host].ReplicaOf == a.Alias |
| + if !replica && a.DNSStatus != "pointing" { |
| + // ... unchanged withheld-log branch |
| + continue |
| + } |
| + withheldAliasSaid.Delete(a.Alias) |
| + if _, taken := hosts[a.Alias]; taken { |
| + continue |
| + } |
| + if replica { |
| + // Role "replica", never "home": every home-map reader drops an |
| + // unknown role (foldHomes), which is what keeps the account's |
| + // host single-valued fleet-wide. A "home" entry here would give |
| + // a third box two dial targets settled by sort order. |
| + hosts[a.Alias] = edgeHost{Role: "replica", Alias: true} |
| + continue |
| + } |
| + hosts[a.Alias] = edgeHost{Role: "home", Alias: true} |
| +``` |
| + |
| +If the pass holds no `byHost` map, build one from the `routes` slice it already reads (`:174`). |
| + |
| +- [ ] **Step 4: Run and commit** |
| + |
| +Run: `export GOCACHE=$TMPDIR/gocache GOFLAGS=-mod=mod GOPROXY=off; go test ./internal/host/ -run Edges -v` |
| +Expected: PASS, including `TestEdgesFilePublishesPointingAliasesOfServedPrimaries` (`:669`) — an ordinary alias still waits for its stamp. |
| + |
| +```bash |
| +git add internal/host/edgesfile.go internal/host/edgesfile_test.go |
| +GIT_AUTHOR_NAME=claude GIT_AUTHOR_EMAIL='claude~paul@keymail.dev' GIT_COMMITTER_NAME=claude GIT_COMMITTER_EMAIL='claude~paul@keymail.dev' \ |
| + git commit -m "#162: publish the replica alias as role replica, stamp or no stamp |
| + |
| +Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>" |
| +``` |
| + |
| +--- |
| + |
| +### Task 6: The home map ignores role `replica` on purpose |
| + |
| +**Files:** |
| +- Modify: `internal/edge/homemap.go` (`foldHomes` `:529-584`) |
| +- Test: `internal/edge/homemap_test.go` |
| + |
| +**Interfaces:** |
| +- Consumes: Task 5's edges-file entry. |
| +- Produces: no API change. `const edgeRoleReplica = "replica"` and an explicit `case edgeRoleReplica: continue`. |
| + |
| +Spec §4. The `default: continue` arm (`:563-564`) already drops it; this task makes the ignore a decision and pins it with a test, so a later change to the switch cannot quietly hand a replica box the dial. |
| + |
| +- [ ] **Step 1: Write the test** |
| + |
| +```go |
| +func TestFoldHomesIgnoresReplicaEntries(t *testing.T) { |
| + files := map[string]edgesFileView{ |
| + "ie-1": {V: 1, Box: "ie-1", EIP: "198.51.100.4", Hosts: map[string]edgesHostView{ |
| + "acme.go.tito.io": {Role: "home", Platform: true}, |
| + }}, |
| + "uk-1": {V: 1, Box: "uk-1", EIP: "203.0.113.9", Hosts: map[string]edgesHostView{ |
| + "acme.go.tito.io": {Role: "replica", Alias: true}, |
| + "r7x2k9m4qa.go.tito.io": {Role: "home", Platform: true}, |
| + }}, |
| + } |
| + got := foldHomes(files, "us-1") |
| + if e, ok := got["acme.go.tito.io"]; !ok || e.box != "ie-1" || e.eip != "198.51.100.4" { |
| + t.Fatalf("account host resolved to %+v, want home box ie-1: a replica entry must never win the dial", e) |
| + } |
| + if e, ok := got["r7x2k9m4qa.go.tito.io"]; !ok || e.box != "uk-1" { |
| + t.Fatalf("the replica's own host must still be dialable: %+v", e) |
| + } |
| +} |
| +``` |
| + |
| +- [ ] **Step 2: Run it** |
| + |
| +Run: `export GOCACHE=$TMPDIR/gocache GOFLAGS=-mod=mod GOPROXY=off; go test ./internal/edge/ -run FoldHomesIgnoresReplica -v` |
| +Expected: PASS already. **If it FAILS, stop and report** — the whole design rests on this. |
| + |
| +- [ ] **Step 3: Make the ignore explicit** |
| + |
| +Beside `edgeRoleHome`/`edgeRoleWildcard`: |
| + |
| +```go |
| +const edgeRoleReplica = "replica" |
| +``` |
| + |
| +and in `foldHomes`'s role switch, before `default:`: |
| + |
| +```go |
| + case edgeRoleReplica: |
| + // #162: that box serves the account from a one-way mirror |
| + // and says so, so its own edge resolves the host locally |
| + // while the FLEET goes on treating the home as the only |
| + // dial target. Dropping it here keeps this map |
| + // single-valued; publishing it as "home" would hand a third |
| + // box two targets settled by sort order, and the account's |
| + // traffic would land on a mirror. |
| + continue |
| +``` |
| + |
| +- [ ] **Step 4: Pin the whole point of the feature — local serve, no backhaul** |
| + |
| +This is spec §11 test 8 and the one assertion the design exists for. In `internal/edge/alias_test.go`, beside the existing alias tests: |
| + |
| +```go |
| +func TestReplicaAliasServesLocallyWithoutBackhaul(t *testing.T) { |
| + // A replica's own route, and the account's public host as an alias of |
| + // it — exactly what Tasks 3 and 4 put in the registry on a replica box. |
| + // The home map is deliberately seeded with the account host homed on |
| + // ANOTHER box: if the alias fold did not win, this request would be |
| + // backhauled there, and the test's upstream would never be reached. |
| + e, reg := newTestEdge(t) // whatever this package's edge fixture is called |
| + sock := serveUnix(t, func(w http.ResponseWriter, r *http.Request) { |
| + io.WriteString(w, "replica") |
| + }) |
| + if err := reg.Upsert(registry.Route{ |
| + Host: "r7x2k9m4qa.go.tito.io", Kind: "instance", Status: "ready", |
| + Socket: sock, Provisioned: true, Account: "bab", App: "tito", |
| + ReplicaOf: "acme.go.tito.io", |
| + }); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := reg.UpsertAlias(registry.RouteAlias{ |
| + Alias: "acme.go.tito.io", Host: "r7x2k9m4qa.go.tito.io", Account: "bab", App: "tito", |
| + }); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := e.Reload(); err != nil { |
| + t.Fatal(err) |
| + } |
| + rec := httptest.NewRecorder() |
| + req := httptest.NewRequest("GET", "https://acme.go.tito.io/checkout", nil) |
| + req.Host = "acme.go.tito.io" |
| + e.ServeHTTP(rec, req) |
| + if got := rec.Body.String(); got != "replica" { |
| + t.Fatalf("body = %q, want the local replica's answer — the request was not served locally", got) |
| + } |
| +} |
| +``` |
| + |
| +Use the fixture helpers this package already has (`alias_test.go` and `edge_test.go` between them build an edge, a registry and a unix upstream); do not invent new ones. |
| + |
| +Run: `TMPDIR=/home/paulca/t GOTMPDIR=/home/paulca/t GOCACHE=/home/paulca/t/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/edge/ -run 'ReplicaAlias|FoldHomes' -v` |
| +Expected: PASS. A failure here means the alias fold does not do what spec §3 rests on — stop and report rather than adapting the test. |
| + |
| +- [ ] **Step 5: Run the package and commit** |
| + |
| +Run: `TMPDIR=/home/paulca/t GOTMPDIR=/home/paulca/t GOCACHE=/home/paulca/t/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/edge/` |
| +Expected: PASS. |
| + |
| +```bash |
| +git add internal/edge/homemap.go internal/edge/homemap_test.go internal/edge/alias_test.go |
| +GIT_AUTHOR_NAME=claude GIT_AUTHOR_EMAIL='claude~paul@keymail.dev' GIT_COMMITTER_NAME=claude GIT_COMMITTER_EMAIL='claude~paul@keymail.dev' \ |
| + git commit -m "#162: foldHomes ignores role replica as a decision, not an accident |
| + |
| +Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>" |
| +``` |
| + |
| +--- |
| + |
| +### Task 7: The console doors — create, PATCH, delete, move, and the old-writer guard |
| + |
| +**Files:** |
| +- Modify: `internal/console/apiinstances.go` (create `:578-830`; update `:1041-1281`; delete `:1288-1350`; token `:1366-1406` — comment only) |
| +- Modify: `internal/provision/maintenance.go` (`protectedFieldsEqual` `:79-94`) |
| +- Test: `internal/console/apiinstances_test.go`, `internal/provision/maintenance_test.go` |
| + |
| +**Interfaces:** |
| +- Consumes: Tasks 1-3. |
| +- Produces: `POST /api/cli/apps/{account}/{app}/instances` accepts `{"role":"replica","home":"<public host>","region":"<code>"}` and returns the minted host in the existing create response; `PATCH` accepts `{"replicas":["uk"]}`; both refuse per spec §6. No new endpoint. |
| + |
| +Spec §6. Existing helpers: `s.homeRegionIn`/`s.homeRegionOut` (`apiinstancemove.go:136-148`), `instanceRegionAllowed` (`:1573`), `placements(ctx, s.cfg.Releases)` (`instancesplacement.go`), `s.appPlacement(reg, acct, app)` (`:701`). Test helpers: `instanceFixture`, `enableInstances`, `instancesURL`, `rawReq`, `cliToken`, `titoManifest`. |
| + |
| +- [ ] **Step 1: Write the failing tests** |
| + |
| +```go |
| +func TestAPIInstanceCreateReplica(t *testing.T) { |
| + fx, tok := instanceFixture(t, "tito") |
| + enableInstances(t, fx) |
| + // A steered home in ie, opted into uk. |
| + home := createInstance(t, fx, tok, `{"host":"acme.go.tito.io","region":"ie","channel":"stable"}`) |
| + patchInstance(t, fx, tok, home, `{"steering":"latency"}`) |
| + patchInstance(t, fx, tok, home, `{"replicas":["uk"]}`) |
| + |
| + status, body := createInstanceRaw(t, fx, tok, `{"role":"replica","home":"acme.go.tito.io","region":"uk"}`) |
| + if status != http.StatusOK && status != http.StatusCreated { |
| + t.Fatalf("create replica = %d: %s", status, body) |
| + } |
| + var got struct { |
| + Host string `json:"host"` |
| + Role string `json:"role"` |
| + } |
| + if err := json.Unmarshal(body, &got); err != nil { |
| + t.Fatal(err) |
| + } |
| + if got.Role != "replica" || got.Host == "" || got.Host == "acme.go.tito.io" { |
| + t.Fatalf("create replica returned %+v, want a minted host of its own with role replica", got) |
| + } |
| + |
| + for name, payload := range map[string]string{ |
| + "replica of a replica": `{"role":"replica","home":"` + got.Host + `","region":"uk"}`, |
| + "placement not opted in": `{"role":"replica","home":"acme.go.tito.io","region":"de"}`, |
| + "home's own placement": `{"role":"replica","home":"acme.go.tito.io","region":"ie"}`, |
| + "caller-supplied host": `{"role":"replica","home":"acme.go.tito.io","region":"uk","host":"mine.go.tito.io"}`, |
| + "caller-supplied channel": `{"role":"replica","home":"acme.go.tito.io","region":"uk","channel":"canary"}`, |
| + "no such home": `{"role":"replica","home":"nope.go.tito.io","region":"uk"}`, |
| + } { |
| + if status, body := createInstanceRaw(t, fx, tok, payload); status < 400 { |
| + t.Errorf("%s: create returned %d, want a refusal: %s", name, status, body) |
| + } |
| + } |
| +} |
| + |
| +func TestAPIInstanceReplicasPatchRails(t *testing.T) { |
| + fx, tok := instanceFixture(t, "tito") |
| + enableInstances(t, fx) |
| + home := createInstance(t, fx, tok, `{"host":"acme.go.tito.io","region":"ie","channel":"stable"}`) |
| + |
| + if status, body := patchInstanceRaw(t, fx, tok, home, `{"replicas":["uk"]}`); status < 400 { |
| + t.Errorf("replicas accepted on an unsteered record: %d %s", status, body) |
| + } |
| + patchInstance(t, fx, tok, home, `{"steering":"latency"}`) |
| + patchInstance(t, fx, tok, home, `{"replicas":["uk"]}`) |
| + |
| + if status, _ := patchInstanceRaw(t, fx, tok, home, `{"steering":""}`); status < 400 { |
| + t.Error("steering was turned off under a live replica set") |
| + } |
| + createInstance(t, fx, tok, `{"role":"replica","home":"acme.go.tito.io","region":"uk"}`) |
| + if status, _ := patchInstanceRaw(t, fx, tok, home, `{"replicas":[]}`); status < 400 { |
| + t.Error("a placement was removed while a replica record still lived there") |
| + } |
| + if status, _ := deleteInstanceRaw(t, fx, tok, home); status != http.StatusConflict { |
| + t.Errorf("delete of a home with replicas = %d, want 409", status) |
| + } |
| +} |
| + |
| +func TestProtectedFieldsRefuseAnOldWriter(t *testing.T) { |
| + stored := Record{Schema: 1, Host: "acme.go.tito.io", Channel: "stable", |
| + Steering: "latency", Replicas: []string{"uk"}} |
| + dropped := stored |
| + dropped.Replicas = nil |
| + if protectedFieldsEqual(stored, dropped) { |
| + t.Error("a writer that dropped replicas was treated as equal: an old console would silently strip the opt-in") |
| + } |
| + replica := Record{Schema: 1, Host: "r7.go.tito.io", Channel: "stable", Role: "replica", Home: "acme.go.tito.io"} |
| + flat := replica |
| + flat.Role, flat.Home = "", "" |
| + if protectedFieldsEqual(replica, flat) { |
| + t.Error("a writer that dropped role/home was treated as equal") |
| + } |
| +} |
| +``` |
| + |
| +Write the small `createInstance`/`createInstanceRaw`/`patchInstance`/`patchInstanceRaw`/`deleteInstanceRaw` helpers in the test file if the package has no equivalents — each is a `rawReq` against `instancesURL` with the bearer token, failing the test on an unexpected status. |
| + |
| +- [ ] **Step 2: Run them and watch them fail** |
| + |
| +Run: `export GOCACHE=$TMPDIR/gocache GOFLAGS=-mod=mod GOPROXY=off; go test ./internal/console/ -run 'Replica|ProtectedFields' -v; go test ./internal/provision/ -run ProtectedFields -v` |
| +Expected: FAIL — the create door ignores `role`, the PATCH door ignores `replicas`, delete returns 200. |
| + |
| +- [ ] **Step 3: The create door** |
| + |
| +Add `Role`, `Home` and `Replicas` to the request struct (`:578`). After the manifest and placement lookups (`:701-712`) and before the host validation: |
| + |
| +```go |
| + role := strings.TrimSpace(req.Role) |
| + if !provision.ValidRole(role) { |
| + jsonError(w, http.StatusBadRequest, "role "+role+": want \"\" or \"replica\"") |
| + return |
| + } |
| + var homeRec *provision.Record |
| + if role == "replica" { |
| + // Every rail here is a CROSS-RECORD one, which no other door in |
| + // this file has: ValidateShape cannot see a sibling record and the |
| + // reconcile pass will not refuse anything (spec §6). |
| + if placed != "" { |
| + jsonError(w, http.StatusBadRequest, "this app is placed on a fleet, so a replica has no placement of its own to choose — unplace the app or drop role") |
| + return |
| + } |
| + if strings.TrimSpace(req.Host) != "" { |
| + jsonError(w, http.StatusBadRequest, "a replica's host is minted by the platform — drop host") |
| + return |
| + } |
| + if strings.TrimSpace(req.Channel) != "" { |
| + jsonError(w, http.StatusBadRequest, "a replica runs its home's channel — drop channel") |
| + return |
| + } |
| + rec, err := provision.GetRecord(ctx, s.cfg.Releases, acct.Sqid, app, strings.TrimSpace(req.Home)) |
| + if err != nil { |
| + jsonError(w, http.StatusBadRequest, "home "+req.Home+": no such instance in this app") |
| + return |
| + } |
| + if rec.Role != "" { |
| + jsonError(w, http.StatusBadRequest, "home "+req.Home+" is itself a replica — a replica of a replica is not a shape the platform carries") |
| + return |
| + } |
| + if rec.Steering != "latency" { |
| + jsonError(w, http.StatusBadRequest, "home "+req.Home+" is not latency-steered, so its host resolves to one edge and a replica would serve nothing — set steering first") |
| + return |
| + } |
| + // The home region is "" on a record and a code at the door, so the |
| + // comparison goes through homeRegionOut (spec §2). |
| + if s.homeRegionOut(region) == s.homeRegionOut(rec.Region) { |
| + jsonError(w, http.StatusBadRequest, "region "+s.homeRegionOut(region)+" is the home's own placement — a replica lives somewhere else") |
| + return |
| + } |
| + if !slices.Contains(rec.Replicas, region) { |
| + jsonError(w, http.StatusBadRequest, "region "+s.homeRegionOut(region)+" is not one this account opted into — set replicas on "+rec.Host+" first") |
| + return |
| + } |
| + homeRec = &rec |
| + } |
| +``` |
| + |
| +Mint the host for a replica where `host` is otherwise taken from the request (`:626`): |
| + |
| +```go |
| + host := strings.TrimSpace(req.Host) |
| + if role == "replica" { |
| + // A name of its own, not a secret: it is published in the box's |
| + // edges file by design (spec §6). Create-only writes mean a lost |
| + // race is a conflict, so retry a fresh name twice before failing. |
| + host = mintReplicaHost(m.Domain) |
| + } |
| +``` |
| + |
| +with, beside the other helpers in this file: |
| + |
| +```go |
| +// mintReplicaHost returns one label under domain for a replica instance: |
| +// "r" plus 10 characters of crypto/rand base32 (Crockford, lower case). |
| +// Not a secret — see spec §6 — just a name no tenant chose. |
| +func mintReplicaHost(domain string) string { |
| + const alphabet = "0123456789abcdefghjkmnpqrstvwxyz" |
| + b := make([]byte, 10) |
| + if _, err := rand.Read(b); err != nil { |
| + panic("replica host: " + err.Error()) // crypto/rand does not fail |
| + } |
| + for i, v := range b { |
| + b[i] = alphabet[int(v)%len(alphabet)] |
| + } |
| + return "r" + string(b) + "." + domain |
| +} |
| +``` |
| + |
| +In the record construction (`:774`), carry the three fields and the home's channel: |
| + |
| +```go |
| + Role: role, |
| + Home: strings.TrimSpace(req.Home), |
| +``` |
| + |
| +and where `channel` is resolved, `if homeRec != nil { channel = homeRec.Channel }`. |
| + |
| +Around the `CreateRecord` call, retry twice on `release.ErrCASConflict` for a replica, minting a fresh host each time. |
| + |
| +- [ ] **Step 4: The PATCH door** |
| + |
| +Add `Replicas *[]string` to the update request struct (`:1094`). After the record is read and `stored := *rec` is taken (`:1136`): |
| + |
| +```go |
| + if body.Replicas != nil { |
| + if rec.Role == "replica" { |
| + jsonError(w, http.StatusBadRequest, "a replica does not carry replicas of its own") |
| + return |
| + } |
| + norm, err := provision.ValidReplicas(*body.Replicas) |
| + if err != nil { |
| + jsonError(w, http.StatusBadRequest, err.Error()) |
| + return |
| + } |
| + for _, code := range norm { |
| + if s.homeRegionOut(code) == s.homeRegionOut(rec.Region) { |
| + jsonError(w, http.StatusBadRequest, "replica placement "+code+" is this instance's own placement") |
| + return |
| + } |
| + if !instanceRegionAllowed(m.Regions, code) { |
| + jsonError(w, http.StatusBadRequest, "replica placement "+code+" is not one this app runs in") |
| + return |
| + } |
| + if caps, published := placements(ctx, s.cfg.Releases); published && !caps[code].Hosting { |
| + jsonError(w, http.StatusBadRequest, "replica placement "+code+" cannot host an instance in this deployment") |
| + return |
| + } |
| + } |
| + gone, err := replicaPlacementsInUse(ctx, s.cfg.Releases, acct.Sqid, app, rec.Host, norm) |
| + if err != nil { |
| + jsonError(w, http.StatusBadGateway, "could not check this app's replicas") |
| + return |
| + } |
| + if len(gone) > 0 { |
| + jsonError(w, http.StatusConflict, "a replica still lives in "+strings.Join(gone, ", ")+": delete the replica instance first, once its data has drained") |
| + return |
| + } |
| + rec.Replicas = norm |
| + } |
| + if body.Steering != nil && *body.Steering != "latency" && len(rec.Replicas) > 0 { |
| + jsonError(w, http.StatusConflict, "this instance has replica placements, which depend on latency steering — remove the placements first") |
| + return |
| + } |
| +``` |
| + |
| +with a helper beside the other record readers: |
| + |
| +```go |
| +// replicaPlacementsInUse returns the placements of `home` that still hold a |
| +// replica record and are NOT in `keep`. A truncated or failed listing is an |
| +// error, never an empty answer: a partial view would let a residence opt-in |
| +// be withdrawn from underneath live order data (spec §6). |
| +func replicaPlacementsInUse(ctx context.Context, s release.Store, acct, app, home string, keep []string) ([]string, error) { |
| + recs, truncated, err := provision.ListRecords(ctx, s, acct, app, instanceListMax) |
| + if err != nil { |
| + return nil, err |
| + } |
| + if truncated { |
| + return nil, errors.New("instance list truncated") |
| + } |
| + var out []string |
| + for _, r := range recs { |
| + if r.Role == "replica" && r.Home == home && !slices.Contains(keep, r.Region) { |
| + out = append(out, r.Region) |
| + } |
| + } |
| + return out, nil |
| +} |
| +``` |
| + |
| +- [ ] **Step 5: The delete and move doors** |
| + |
| +In `handleAPIInstanceDelete`, after `refuseControlPlane` (`:1308`) and before `provision.DeleteRecord`: |
| + |
| +```go |
| + // A home with replicas is refused: its mirrors would go on serving the |
| + // account's host from a route whose record has gone (spec §6). Deleting |
| + // the REPLICA is the caller's to sequence after its data has drained — |
| + // the platform cannot see drain state. |
| + if rec, err := provision.GetRecord(ctx, s.cfg.Releases, acct.Sqid, app, host); err == nil && rec.Role == "" { |
| + live, err := replicaPlacementsInUse(ctx, s.cfg.Releases, acct.Sqid, app, host, nil) |
| + if err != nil { |
| + jsonError(w, http.StatusBadGateway, "could not check this instance's replicas") |
| + return |
| + } |
| + if len(live) > 0 || len(rec.Replicas) > 0 { |
| + jsonError(w, http.StatusConflict, "this instance has replicas in "+strings.Join(append(live, rec.Replicas...), ", ")+" — delete them first") |
| + return |
| + } |
| + } |
| +``` |
| + |
| +In the move door (`apiinstancemove.go`), refuse a move whose record is `role=replica` ("repointing a replica is delete-and-recreate"), and refuse a move of a home into any placement in its own `Replicas`. |
| + |
| +- [ ] **Step 6: The old-writer guard** |
| + |
| +In `internal/provision/maintenance.go`, extend `protectedFieldsEqual` to compare `Role`, `Home` and `Replicas` (use `slices.Equal` for the set) and mention in its doc comment that a console build predating these fields would otherwise strip them on a read-modify-write PATCH, taking the residence opt-in and the replica's link with them. |
| + |
| +- [ ] **Step 7: Run and commit** |
| + |
| +Run: `export GOCACHE=$TMPDIR/gocache GOFLAGS=-mod=mod GOPROXY=off; go test ./internal/console/ ./internal/provision/` |
| +Expected: PASS. |
| + |
| +```bash |
| +git add internal/console/ internal/provision/maintenance.go internal/provision/maintenance_test.go |
| +GIT_AUTHOR_NAME=claude GIT_AUTHOR_EMAIL='claude~paul@keymail.dev' GIT_COMMITTER_NAME=claude GIT_COMMITTER_EMAIL='claude~paul@keymail.dev' \ |
| + git commit -m "#162: console doors for replica creation, the opt-in and its refusals |
| + |
| +Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>" |
| +``` |
| + |
| +--- |
| + |
| +### Task 8: The list, get and CLI surface |
| + |
| +**Files:** |
| +- Modify: `internal/console/apiinstances.go` (`instanceListRecord` `:143-193`, `instanceGetRecord` `:521-529` — both embed `provision.Record`, so the three fields ride along; verify and add nothing if so) |
| +- Modify: `instances_cli.go` (list columns `:372-393`; create flags `:262-303`) |
| +- Modify: `docs/site/cli.md` |
| +- Test: `instances_cli_test.go` (or the root-package test file that covers the instances verb) |
| + |
| +**Interfaces:** |
| +- Consumes: Task 7's API. |
| +- Produces: `carlos instances` prints a ROLE column; `carlos instances create --role replica --home <host> --region <code>` prints the minted host. |
| + |
| +- [ ] **Step 1: Write the failing test** |
| + |
| +```go |
| +func TestInstancesListShowsRole(t *testing.T) { |
| + var buf bytes.Buffer |
| + printInstances(&buf, []instanceListRecord{ |
| + {Record: provision.Record{Host: "acme.go.tito.io", Region: "ie", Channel: "stable"}}, |
| + {Record: provision.Record{Host: "r7x2k9m4qa.go.tito.io", Region: "uk", Channel: "stable", |
| + Role: "replica", Home: "acme.go.tito.io"}}, |
| + }) |
| + out := buf.String() |
| + if !strings.Contains(out, "ROLE") { |
| + t.Fatalf("no ROLE column:\n%s", out) |
| + } |
| + for _, line := range strings.Split(out, "\n") { |
| + if strings.HasPrefix(line, "acme.go.tito.io") && strings.Contains(line, "replica") { |
| + t.Errorf("a home was rendered as a replica:\n%s", line) |
| + } |
| + if strings.HasPrefix(line, "r7x2k9m4qa.go.tito.io") && !strings.Contains(line, "replica") { |
| + t.Errorf("a replica was not marked:\n%s", line) |
| + } |
| + } |
| +} |
| + |
| +func TestInstancesCreateReplicaNeedsAHome(t *testing.T) { |
| + err := runInstancesCreate([]string{"--app", "tito", "--role", "replica", "--region", "uk"}) |
| + if err == nil || !strings.Contains(err.Error(), "home") { |
| + t.Fatalf("err = %v, want a local refusal naming --home before any request", err) |
| + } |
| +} |
| +``` |
| + |
| +Match the real names in `instances_cli.go` — if the printer is inline in the verb rather than a `printInstances` function, extract it in Step 3 so it can be tested, and say so in the commit message. |
| + |
| +- [ ] **Step 2: Run it and watch it fail** |
| + |
| +Run: `export GOCACHE=$TMPDIR/gocache GOFLAGS=-mod=mod GOPROXY=off; go test . -run Instances -v` |
| + |
| +- [ ] **Step 3: Add the column and the flags** |
| + |
| +Add `ROLE` to the header and `rec.Role` to each row (empty for a home — never the word "home", which would imply the field is set). Add `--role` and `--home` to the create verb, refuse `--role replica` without `--home` locally, and print the minted host from the response. |
| + |
| +- [ ] **Step 4: Document it** |
| + |
| +In `docs/site/cli.md`, under the instances section, document `--role` and `--home` with a worked example: create the home, steer it, set `--replicas`, then create the replica. `internal/docsite` fails the gate over an undocumented verb or flag, so run `go test ./internal/docsite/` too. |
| + |
| +- [ ] **Step 5: Run and commit** |
| + |
| +Run: `export GOCACHE=$TMPDIR/gocache GOFLAGS=-mod=mod GOPROXY=off; go test . ./internal/docsite/ ./internal/console/` |
| +Expected: PASS. |
| + |
| +```bash |
| +git add instances_cli.go docs/site/cli.md internal/console/apiinstances.go *_test.go |
| +GIT_AUTHOR_NAME=claude GIT_AUTHOR_EMAIL='claude~paul@keymail.dev' GIT_COMMITTER_NAME=claude GIT_COMMITTER_EMAIL='claude~paul@keymail.dev' \ |
| + git commit -m "#162: role and home on the instance surface, CLI and docs |
| + |
| +Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>" |
| +``` |
| + |
| +--- |
| + |
| +### Task 9: The siblings file |
| + |
| +**Files:** |
| +- Create: `internal/host/siblings.go`, `internal/host/siblings_test.go` |
| +- Modify: `internal/host/agent.go` (the instances loop around `:2336`; the edge-ready callback at `:510`) |
| +- Modify: `internal/edge/homemap.go` or `internal/edge/edge.go` (exported accessors) |
| +- Modify: `internal/activator/backend_exec.go` (`agentOwnedVars` `:977-995`) |
| +- Test: `internal/host/siblings_test.go`, `internal/activator/backend_exec_test.go` |
| + |
| +**Interfaces:** |
| +- Consumes: Task 3's `ReconcileConfig.OnRecords`. |
| +- Produces: |
| + - `func (e *edge.Edge) SiblingEIP(host string) (eip, box string, ok bool)` — wrapper over `homeMap.Home` (`homemap.go:611`); |
| + - `func (e *edge.Edge) MeshKeys() (current, next []byte)` — wrapper over `homeMap.meshKeys` (`homemap.go:689`); |
| + - `func buildSiblings(all map[string]provision.Record, self string, eipOf func(host string) (string, bool), keys [][]byte) map[string]siblingsFile` — the pure core, keyed by the host whose file it is; |
| + - `func siblingsPass(...) error` — writes each file under `pathsFor(rt).dataDir`; |
| + - `ExecBackend.SiblingsFile func(host string) string` and `CARLOS_SIBLINGS_FILE` in `agentOwnedVars`. |
| + |
| +Spec §7. Keys are per pair: `HMAC(meshKey, "replica/" + account + "/" + homeHost + "/" + replicaHost)`, current first, previous second while a rotation is in flight. The file is `siblings.json` in the instance's data directory — `/data/instances/<id>` for a nested row, `/data` for a flat one (`provision.InstanceDir`; `internal/host/schedule_wiring.go:193-203` computes both already). Write with `instancefs.WriteAtomic(dir, "siblings.json", data, 0o600)` — the directory is tenant-writable ground and a plain open would follow a planted symlink. |
| + |
| +- [ ] **Step 1: Write the failing tests for the pure core** |
| + |
| +```go |
| +func TestBuildSiblingsPairsHomeAndReplica(t *testing.T) { |
| + all := map[string]provision.Record{ |
| + "acme.go.tito.io": {Host: "acme.go.tito.io", Account: "bab", App: "tito", Region: "ie", |
| + Steering: "latency", Replicas: []string{"uk"}}, |
| + "r7x2.go.tito.io": {Host: "r7x2.go.tito.io", Account: "bab", App: "tito", Region: "uk", |
| + Role: "replica", Home: "acme.go.tito.io"}, |
| + } |
| + eip := func(host string) (string, bool) { |
| + return map[string]string{"acme.go.tito.io": "198.51.100.4", "r7x2.go.tito.io": "203.0.113.9"}[host], true |
| + } |
| + got := buildSiblings(all, "uk", eip, [][]byte{[]byte("mesh-current")}) |
| + |
| + home := got["acme.go.tito.io"] |
| + if home.Role != "home" || len(home.Replicas) != 1 { |
| + t.Fatalf("home file = %+v, want one replica", home) |
| + } |
| + if home.Replicas[0].Addr != "203.0.113.9:443" || home.Replicas[0].Placement != "uk" { |
| + t.Errorf("replica entry = %+v", home.Replicas[0]) |
| + } |
| + replica := got["r7x2.go.tito.io"] |
| + if replica.Role != "replica" || replica.Home == nil || replica.Home.Addr != "198.51.100.4:443" { |
| + t.Fatalf("replica file = %+v", replica) |
| + } |
| + if len(home.Replicas[0].Keys) != 1 || home.Replicas[0].Keys[0] != replica.Home.Keys[0] { |
| + t.Error("the two sides of a pair must carry the same key") |
| + } |
| +} |
| + |
| +func TestBuildSiblingsKeysArePerPairAndRotate(t *testing.T) { |
| + all := map[string]provision.Record{ |
| + "acme.go.tito.io": {Host: "acme.go.tito.io", Account: "bab", App: "tito", Region: "ie", Steering: "latency", Replicas: []string{"uk", "de"}}, |
| + "r7x2.go.tito.io": {Host: "r7x2.go.tito.io", Account: "bab", App: "tito", Region: "uk", Role: "replica", Home: "acme.go.tito.io"}, |
| + "r8y3.go.tito.io": {Host: "r8y3.go.tito.io", Account: "bab", App: "tito", Region: "de", Role: "replica", Home: "acme.go.tito.io"}, |
| + } |
| + eip := func(host string) (string, bool) { return "203.0.113.9", true } |
| + got := buildSiblings(all, "ie", eip, [][]byte{[]byte("cur"), []byte("prev")}) |
| + home := got["acme.go.tito.io"] |
| + if len(home.Replicas) != 2 { |
| + t.Fatalf("want two replicas, got %+v", home.Replicas) |
| + } |
| + if home.Replicas[0].Keys[0] == home.Replicas[1].Keys[0] { |
| + t.Error("two replicas of one home share a key: a compromised replica could speak as its sibling") |
| + } |
| + if len(home.Replicas[0].Keys) != 2 { |
| + t.Error("a rotation in flight must publish both keys, current first") |
| + } |
| +} |
| + |
| +func TestBuildSiblingsOmitsASiblingWithNoEIPAndStillWritesAnEmptyHome(t *testing.T) { |
| + all := map[string]provision.Record{ |
| + "acme.go.tito.io": {Host: "acme.go.tito.io", Account: "bab", App: "tito", Region: "ie", Steering: "latency", Replicas: []string{"uk"}}, |
| + "r7x2.go.tito.io": {Host: "r7x2.go.tito.io", Account: "bab", App: "tito", Region: "uk", Role: "replica", Home: "acme.go.tito.io"}, |
| + } |
| + got := buildSiblings(all, "ie", func(string) (string, bool) { return "", false }, [][]byte{[]byte("cur")}) |
| + home, ok := got["acme.go.tito.io"] |
| + if !ok { |
| + t.Fatal("a home with no reachable replica still gets a file, so the app can tell 'no replicas' from 'no file'") |
| + } |
| + if len(home.Replicas) != 0 { |
| + t.Errorf("a sibling with no EIP must be omitted, not written with an empty addr: %+v", home.Replicas) |
| + } |
| +} |
| +``` |
| + |
| +- [ ] **Step 2: Run them and watch them fail** |
| + |
| +Run: `export GOCACHE=$TMPDIR/gocache GOFLAGS=-mod=mod GOPROXY=off; go test ./internal/host/ -run BuildSiblings -v` |
| +Expected: FAIL — `undefined: buildSiblings`. |
| + |
| +- [ ] **Step 3: Write `internal/host/siblings.go`** |
| + |
| +The file types, exactly as spec §7 shows them: |
| + |
| +```go |
| +type siblingsFile struct { |
| + Role string `json:"role"` |
| + Host string `json:"host"` |
| + Placement string `json:"placement,omitempty"` |
| + Home *siblingEntry `json:"home,omitempty"` |
| + Replicas []siblingEntry `json:"replicas,omitempty"` |
| +} |
| + |
| +type siblingEntry struct { |
| + Host string `json:"host"` |
| + Placement string `json:"placement,omitempty"` |
| + Addr string `json:"addr"` |
| + Keys []string `json:"keys"` |
| +} |
| +``` |
| + |
| +`buildSiblings` walks `all`, pairs every `role=replica` record with `all[rec.Home]`, derives the pair key with `hmac.New(sha256.New, meshKey)` over `"replica/"+account+"/"+home+"/"+replica`, base64-encodes it, and emits a file for each side. It writes a file for a home with `Replicas` set even when no replica record exists yet. Nothing here touches the filesystem — that is `siblingsPass`, which resolves each host's data directory and calls `instancefs.WriteAtomic`, and which removes a stale `siblings.json` when a host is no longer part of any pair. |
| + |
| +- [ ] **Step 4: Wire it** |
| + |
| +In `internal/edge`, add the two exported accessors (they are one-line wrappers; `meshKeys` returns `(current, next)` — publish both, current first, dropping nils). In `agent.go`, publish them through the edge-ready callback with the `setUpstreamSource` pattern (`:2490-2510`), and call `siblingsPass` in the instances loop right after `provision.Reconcile`, using the map `OnRecords` captured. In `agentOwnedVars`, add: |
| + |
| +```go |
| + if b.SiblingsFile != nil { |
| + if p := b.SiblingsFile(rt.Host); p != "" { |
| + childVars = withVar(childVars, "CARLOS_SIBLINGS_FILE", p) |
| + } else { |
| + childVars = withoutVars(childVars, "CARLOS_SIBLINGS_FILE") |
| + } |
| + } else { |
| + childVars = withoutVars(childVars, "CARLOS_SIBLINGS_FILE") |
| + } |
| +``` |
| + |
| +and a backend test asserting the variable is set when a file exists and stripped when it does not — `CARLOS_REGION`'s own test is the shape. |
| + |
| +- [ ] **Step 5: Test the writer against a real directory** |
| + |
| +Run outside the sandbox (this package binds sockets elsewhere): `TMPDIR=/home/paulca/t GOTMPDIR=/home/paulca/t GOCACHE=/home/paulca/t/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./internal/host/ ./internal/activator/` |
| +Expected: PASS, with a test asserting mode 0600 and that a second pass with an unchanged pair rewrites nothing. |
| + |
| +- [ ] **Step 6: Commit** |
| + |
| +```bash |
| +git add internal/host/siblings.go internal/host/siblings_test.go internal/host/agent.go internal/edge/ internal/activator/ |
| +GIT_AUTHOR_NAME=claude GIT_AUTHOR_EMAIL='claude~paul@keymail.dev' GIT_COMMITTER_NAME=claude GIT_COMMITTER_EMAIL='claude~paul@keymail.dev' \ |
| + git commit -m "#162: the siblings file — addresses and per-pair keys, no new channel |
| + |
| +Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>" |
| +``` |
| + |
| +--- |
| + |
| +### Task 10: The whole tree, the cross-compile gate, and the branch |
| + |
| +**Files:** none new. |
| + |
| +- [ ] **Step 1: The additivity proof** |
| + |
| +Write one test (in `internal/host/edgesfile_test.go`) asserting that a registry with no replica rows produces an edges file byte-identical to the same registry before this branch — capture the expected body as a literal in the test, not as a computed value. |
| + |
| +- [ ] **Step 2: Run everything** |
| + |
| +Run: `TMPDIR=/home/paulca/t GOTMPDIR=/home/paulca/t GOCACHE=/home/paulca/t/gocache GOFLAGS=-mod=mod GOPROXY=off go test ./...` |
| +Expected: PASS. Any failure reproduced on a clean `origin/main` worktree (`git worktree add --detach`) before it is called a regression of this branch. |
| + |
| +- [ ] **Step 3: The cross-compile gate** |
| + |
| +Run: `GOOS=darwin go build ./... && GOOS=windows go build ./...` |
| +Expected: both succeed. CI builds darwin and windows; a linux-only syscall in `internal/host` needs a `//go:build linux` file and a `!linux` stub. |
| + |
| +- [ ] **Step 4: Push, watch CI, and hand off** |
| + |
| +```bash |
| +git push origin replica-placements-162:refs/heads/replica-placements-162 |
| +amadan ci status carlos/platform -branch replica-placements-162 |
| +amadan task list carlos/platform -branch replica-placements-162 # every task at its real status |
| +amadan branch handoff carlos/platform -branch replica-placements-162 -state - -next "..." -decided "..." |
| +``` |
| + |
| +Do **not** merge. The branch waits for Paul: the rollout depends on amadan #169 (steering is armed nowhere), and a replica that cannot be steered to serves nothing. |