amadan
plan: claim Plan 7b rev 1, the shared generation selector (internal/genselect)
Read-only, against moves spec §5.1 at 7.13 with the model's restoreCalc as reference; the model's pure-selection witnesses ported as table tests.
Co-Authored-By: Codex <noreply@openai.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 file changed,
+1111
−0
docs/superpowers/plans/2026-10-05-host-lifecycle-claim-7b-genselect.md+1111 −0
CI passed — run details
diff --git a/docs/superpowers/plans/2026-10-05-host-lifecycle-claim-7b-genselect.md b/docs/superpowers/plans/2026-10-05-host-lifecycle-claim-7b-genselect.md| new file mode 100644 |
| index 000000000..25ee253fe |
| --- /dev/null |
| +++ b/docs/superpowers/plans/2026-10-05-host-lifecycle-claim-7b-genselect.md |
| @@ -0,0 +1,1111 @@ |
| +# Host Lifecycle Claim, Plan 7b: the shared generation selector |
| + |
| +> **For agentic workers:** Implement this plan task by task. Steps use checkbox (`- [ ]`) syntax. Read the sources below before changing code; the commands are for the implementation session. |
| + |
| +**Goal:** provide one read-only `internal/genselect` library for the moves restore path and Plan 7's pull door. Given an authorized holder and its own region's versioned bucket, return the selected generation, the ordered exact object versions, the root used and the marker version read. |
| + |
| +**Architecture:** `Select` takes `hostclaim.Holder`, not a second claim schema. One complete `ListVersions` supplies the marker identity, generation occupancy, current files and historical pinned-version metadata. `GetVersion` reads the marker at that identity. `OpenVersion` reads only the chosen level-9 root's header. The consumer verifies complete bytes and the resulting database. The narrow `Reader` interface exposes no mutation. No production caller changes in this branch. |
| + |
| +**Tech Stack:** Go 1.26; standard library; `internal/hostclaim`; `lease.VersionedStore`; `leasetest.VersionedBucket`; Litestream **v0.5.14** and `superfly/ltx` **v0.5.1**. No new dependencies, IAM changes, deployment changes or live tests. |
| + |
| +**Spec and source baseline:** |
| + |
| +- Moves rev **7.13**, `/home/paulca/amadan.net/carlos/platform-moves-design/docs/superpowers/specs/2026-10-02-move-any-app-design.md`: §5.1 steps 1–9; §6.4's `generation.json` row and start check (INV-4). Model worktree HEAD at authoring: `8cdc8bbb`. |
| +- Claim rev **6.14**, `docs/superpowers/specs/2026-10-02-host-lifecycle-claim-design.md`: C3a, C6, §6.2a and §6.4a's operative “Admissions, coverage and retirement”, item 3. Historical readings do not override that subsection. |
| +- Model paths below are relative to `/home/paulca/amadan.net/carlos/platform-moves-design/internal/sim/hostlife/`: `protocol_test.go:173` (`restoreCalc`), `:187` (`restoreWith`), `:1154` (`restoreAsNext`), `:1188` (`unfinishedChain`); `ltx_test.go:667` (`plan`); `moves_test.go:311` (the seal's baseline selection). Witness ports cite their functions beside their table entries. |
| +- `origin/main` at `04cd2d78`: `internal/lease/versioned.go` and `internal/lease/leasetest/bucket.go`. `feat/hostclaim-library` at `47426cc1`: `internal/hostclaim/claim.go` (`Holder`, `AdmissionRecord`) and `generation.go` (`RootGeneration`). Read against those real interfaces, not this planning checkout's older source. |
| +- Module cache: `github.com/benbjohnson/litestream@v0.5.14/replica.go:1441` (`CalcRestorePlan`) and `:1626` (`restoreCandidateBetter`); `github.com/superfly/ltx@v0.5.1/ltx.go:623` (`IsContiguous`). Task 3 compares against the actual planner. |
| + |
| +**Depends on:** Plans 1 and 2. Cut `feat/hostclaim-genselect` from `feat/hostclaim-library`, then `git merge origin/main`. Plan 2 is already on main. Authoring this plan changes only this Markdown file, without staging or committing. |
| + |
| +## Global Constraints |
| + |
| +- **Authority and region:** the caller supplies the holder from its fresh authorized claim read and binds `Bucket` to `Region`. The selector does not grant admission or derive authority from region equality. It checks the location is exactly `instances/<host>` or `instances/<host>/<incarnation>` (accepting a trailing slash). It never constructs another incarnation or searches another bucket. A needed predecessor in a different region returns `ErrRegion`. |
| +- **Read-only:** only `ListVersions`, `GetVersion` and `OpenVersion`. No marker publication, CAS, upload, anchor creation, certificate, parking, deletion, filesystem mutation or replication open. Input claims and storage remain unchanged. |
| +- **Origin:** genuinely absent `<location>/generation.json` means `@root`, with files directly in `<location>/<level>/`. The holder's origin is `Generation == ""`; `hostclaim.RootGeneration == "@root"` is accepted in predecessor records and returned as output, never as a holder generation. A marker read error or a current marker tombstone is not absence. |
| +- **Unfinished run:** walk admissions newest to oldest, requiring the recorded generation and region at every step. Warm records preserve the same predecessor; cold links must be uncovered. Walk all the way to the marker before using any candidate, even if the holder generation has data. Cycles, broken links and other disagreements refuse. Choose the newest generation with any listed key, including metadata or history under a tombstone; otherwise choose the marker's generation. This follows the model's `hasFiles`/`bucket.keys`: history does not prove emptiness. Metadata-only or tombstoned-only selected generations refuse instead of falling back to an older baseline. |
| +- **Complete listing:** `VersionedStore.ListVersions` drains every page or returns an error. Discard any partial slice accompanying an error. Missing/duplicate `IsLatest` entries or a key outside the requested location also refuse. This interface cannot detect a provider that silently truncates a successful result: Plan 2's pagination contract and its fake/server tests are part of the dependency. `PageSize = 1` and `FailListAfterPages = 1` exercise this boundary here. |
| +- **Layout:** generation `g` uses `<location>/g-<g>/`; origin uses `<location>/`. Only exact direct `<%04x level>/<%016x min>-<%016x max>.ltx` coordinates with levels **0–9**, `min >= 1` and `max >= min` enter the plan. Deeper child locations and other generations never enter it. Malformed LTX names in a recognized level refuse. Metadata is occupancy evidence, never a restore file. |
| +- **Marker:** the wire fields are `op`, `incarnation`, `host`, `box`, `installed_at`, `manifest_sha256`, `snapshot_key`, `snapshot_version`, `snapshot_txid`, `snapshot_crc64`. TXID and CRC fields use JSON unsigned numbers; `installed_at` uses Go's RFC3339 `time.Time` encoding. Preserve the marker and its immutable version in the result. Require host and incarnation agreement. Permit additive unknown fields, refuse malformed/trailing JSON and a missing/null `snapshot_txid`. A fresh published generation has `snapshot_txid = 0` and empty snapshot identity; its first full `[1,1]` upload is ordinary level-0 data. No zero-TXID LTX file is valid. |
| +- **Root:** use the current level-9 snapshot with greatest `MaxTXID >= snapshot_txid`; its range starts at **1**. “Newest” means highest TXID, not modification time or version ordering. A late duplicate at the pinned key is never substituted for `snapshot_version`. If no different current successor qualifies, find the exact pinned key/version in the version listing and open that version. Missing pins refuse on this fallback path; a valid current successor does not require an expired old pin to remain readable (§5.1 steps 4–5, 8). The root header must match the selected coordinates. A corrupt successor refuses; it never triggers fallback to an older snapshot. |
| +- **Prepared roots:** moves can supply `Request.Prepared` from an authorized completed build journal while its marker is still lagging (`restorePinned`/`restoreAsNext`). The pin applies only to the selected generation and must agree with an existing marker pin. It is not inferred from `Coverage` or from `build-intent.json`. Ordinary fresh unfinished generations have no pin and start at TXID **1**. The door leaves `Prepared` nil; if it cannot reconstruct a prepared generation from current files it refuses. Integrating moves' journal and pre-admission sealed-transfer adapter is consumer work, not a second planner. |
| +- **Continuation:** after the root, consider current levels **0–8** only. Require `MinTXID <= current+1` and `MaxTXID > current`; rank by highest max, lowest min, then highest level. Repeat until none extends, then refuse if any current continuation remains beyond the final TXID. Guard the `uint64` maximum against overflow. Noncurrent continuations never repair gaps. Equal coordinate/level ties are the same canonical key and have only one current version. |
| +- **Exact versions:** empty, whitespace-only and literal `null` version IDs refuse for the marker, pin and current LTX candidates. Selected delete-marker versions refuse distinctly. Retention tombstones remove files from the current candidate set; they do not globally invalidate a healthy compacted replica, or prohibit reading a different, explicitly pinned historical version at that key. No bare-key GET and no “try current” fallback. |
| +- **Result and verification:** `Files` is ordered, with exact `{key, version_id, level, min_txid, max_txid, size}`. `Root` is its first file (a level-9 snapshot or the first contiguous file starting at 1). `MaxTXID` is the restored position; `MaxLevel0TXID` is the newest current level-0 position for the consumer's anchor decision. This is a selection result, not proof that bytes are valid. The root header read is **100 bytes**; it does not run CRC verification. Consumer requirements are in Task 4. |
| +- **Errors:** every failure returns a nil plan. `ErrListing`, `ErrDisagreement`, `ErrIncarnation`, `ErrMarker`, `ErrVersionID`, `ErrDeleteMarker`, `ErrPinnedMissing`, `ErrPinnedMismatch`, `ErrGap`, `ErrEmpty`, `ErrRegion`, `ErrInput`, and `ErrRoot` are distinguishable with `errors.Is`. Wrapped read failures retain their underlying error. Empty origin data also returns `ErrEmpty`; only a separately authorized bootstrap path may start an empty database. |
| +- **Portability and cache:** no OS-specific code. Use the default Go build cache, never a cache under `/tmp` or a new cache per task. In this sandbox the default is read-only; the existing shared on-disk `/home/paulca/.cache/tmp/gocache` is the fallback. CI includes Darwin, Windows and Linux/arm64 compilation. |
| + |
| +## Review Focus |
| + |
| +1. An unexplained marker disagreement must refuse even when the holder's generation contains files. A covered predecessor or foreign-region gap must never become an empty baseline (Tasks 1–2). |
| +2. A deleted or replaced pinned snapshot must never be read by current key. Conversely, normal retention must permit a verified successor and tombstoned old continuations (Tasks 1–3). |
| +3. A compacted `[10,13]` after snapshot 11 is valid; `[13,13]` alone is a gap. The anchor `[11,11]` is not a continuation. Test this against Litestream, not only against our own comparator (Tasks 1–3). |
| +4. The baseline's emptiness requires every page. Any object history or metadata in a newer generation prevents silent fallback to older data (Tasks 1–2). |
| +5. The selector cannot certify CRCs, SQLite integrity, source permissions, admission freshness or spawn safety. The consumer must satisfy the integration checklist before the library enables a door or restore (Task 4). |
| + |
| +## File Structure |
| + |
| +| File | Responsibility | |
| +|---|---| |
| +| `internal/genselect/select.go` | Read-only API, marker/pin schema, complete listing, unfinished-run resolution, exact-version planning and root header check | |
| +| `internal/genselect/select_test.go` | Model witness ports, versioned-bucket refusal cases, immutable selection and baseline tests | |
| +| `internal/genselect/compatibility_test.go` | Direct comparison with pinned Litestream `CalcRestorePlan` | |
| + |
| +### Task 1: Write the model witnesses and refusal tests first |
| + |
| +**Files:** create `internal/genselect/select_test.go`. |
| + |
| +- [ ] **Step 1: Create the implementation branch in a clean implementation checkout.** These commands are not part of writing this plan. |
| + |
| +```bash |
| +git switch -c feat/hostclaim-genselect feat/hostclaim-library |
| +git merge --no-edit origin/main |
| +git merge-base --is-ancestor 3ca43d3 HEAD |
| +``` |
| + |
| +Expected: Plans 1 and 2 are present. Resolve a dependency conflict before implementing the selector; do not change the storage or claim interfaces to fit this plan. |
| + |
| +- [ ] **Step 2: Write the complete test file below.** The fake's header-only LTX bodies are intentional: these tests exercise selection and the header check, not the consumer's mandatory CRC and SQLite checks. Each source citation names the model function whose pure selection behavior is ported; the multi-entry chain variants extend the witnesses with `restoreAsNext` and `unfinishedChain`, rather than claiming that the model witness itself tests every row. |
| + |
| +```go |
| +package genselect |
| + |
| +import ( |
| + "context" |
| + "encoding/json" |
| + "errors" |
| + "fmt" |
| + "io" |
| + "reflect" |
| + "strings" |
| + "testing" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/hostclaim" |
| + "github.com/carlosframework/platform/internal/lease" |
| + "github.com/carlosframework/platform/internal/lease/leasetest" |
| + "github.com/superfly/ltx" |
| +) |
| + |
| +type fixture struct { |
| + t *testing.T |
| + b *leasetest.VersionedBucket |
| + req Request |
| + marker Marker |
| + markerVersion string |
| + pin File |
| + store Reader |
| +} |
| + |
| +func newFixture(t *testing.T) *fixture { |
| + b := leasetest.NewVersionedBucket() |
| + b.AddBucket("data") |
| + b.PageSize = 1 |
| + f := &fixture{t: t, b: b, store: b, req: Request{Bucket: "data", Region: "ie", Host: "h", |
| + Holder: hostclaim.Holder{Incarnation: "inc", Location: "instances/h/inc", Generation: "g0"}}} |
| + f.pin = f.file("g0", 9, 1, 11) |
| + f.file("g0", 0, 11, 11) |
| + f.marker = Marker{Op: "g0", Incarnation: "inc", Host: "h", Box: "box", InstalledAt: time.Unix(1, 0).UTC(), |
| + ManifestSHA256: strings.Repeat("a", 64), Pin: Pin{SnapshotKey: f.pin.Key, SnapshotVersion: f.pin.VersionID, SnapshotTXID: 11}} |
| + f.publish() |
| + return f |
| +} |
| + |
| +func header(t *testing.T, min, max uint64) []byte { |
| + t.Helper() |
| + h := ltx.Header{Version: ltx.Version, Flags: ltx.HeaderFlagNoChecksum, PageSize: 4096, |
| + MinTXID: ltx.TXID(min), MaxTXID: ltx.TXID(max)} |
| + b, err := h.MarshalBinary() |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + return b |
| +} |
| + |
| +func (f *fixture) file(gen string, level int, min, max uint64) File { |
| + f.t.Helper() |
| + key := fmt.Sprintf("%s%04x/%016x-%016x.ltx", prefix(strings.TrimSuffix(f.req.Holder.Location, "/")+"/", origin(gen)), level, min, max) |
| + b := header(f.t, min, max) |
| + id := f.b.Put("data", key, b) |
| + return File{Key: key, VersionID: id, Level: level, MinTXID: min, MaxTXID: max, Size: int64(len(b))} |
| +} |
| + |
| +func (f *fixture) publish() { |
| + f.t.Helper() |
| + b, err := json.Marshal(f.marker) |
| + if err != nil { |
| + f.t.Fatal(err) |
| + } |
| + f.markerVersion = f.b.Put("data", f.req.Holder.Location+"/generation.json", b) |
| +} |
| + |
| +func (f *fixture) chain(base string) { |
| + f.req.Holder.Generation = "g2" |
| + f.req.Holder.Admissions = []hostclaim.AdmissionRecord{ |
| + {Generation: "g1", Predecessor: base, Region: "ie", PredecessorRegion: "ie", Mode: "cold"}, |
| + {Generation: "g2", Predecessor: "g1", Region: "ie", PredecessorRegion: "ie", Mode: "cold"}, |
| + } |
| +} |
| + |
| +func (f *fixture) origin() { |
| + f.req.Holder.Generation = "" |
| + if err := f.b.DeleteVersion(f.t.Context(), "data", f.req.Holder.Location+"/generation.json", f.markerVersion); err != nil { |
| + f.t.Fatal(err) |
| + } |
| + f.markerVersion = "" |
| +} |
| + |
| +// These are pure-selection ports, not simulations of wake, park or replication. |
| +// Paths refer to platform-moves-design/internal/sim/hostlife at rev 7.13. |
| +func TestModelWitnesses(t *testing.T) { |
| + tests := []struct { |
| + name, source string |
| + setup func(*fixture) []File |
| + gen string |
| + wantErr error |
| + }{ |
| + {"wRestoreOverlap", "witness_test.go:1204", func(f *fixture) []File { |
| + return []File{f.pin, f.file("g0", 1, 10, 13)} |
| + }, "g0", nil}, |
| + {"wRestoreGap", "witness_test.go:1233", func(f *fixture) []File { |
| + f.file("g0", 0, 13, 13) |
| + return nil |
| + }, "", ErrGap}, |
| + {"wRootAdvance", "witness_test.go:1295", func(f *fixture) []File { |
| + f.file("g0", 0, 13, 13) |
| + root := f.file("g0", 9, 1, 20) |
| + f.b.BareDelete("data", f.pin.Key) |
| + return []File{root, f.file("g0", 0, 21, 21)} |
| + }, "g0", nil}, |
| + {"wOriginSwept/origin", "witness_test.go:2373", func(f *fixture) []File { |
| + f.origin() |
| + return []File{f.file("", 0, 1, 1), f.file("", 1, 2, 5)} |
| + }, hostclaim.RootGeneration, nil}, |
| + {"wOriginSwept/unfinished_chain", "witness_test.go:2373; protocol_test.go:1188", func(f *fixture) []File { |
| + f.origin() |
| + f.chain(hostclaim.RootGeneration) |
| + return []File{f.file("", 0, 1, 3)} |
| + }, hostclaim.RootGeneration, nil}, |
| + {"wBaselineParked/all_empty", "witness_test.go:2168; protocol_test.go:1154", func(f *fixture) []File { |
| + f.chain("g0") |
| + return []File{f.pin} |
| + }, "g0", nil}, |
| + {"wBaselineParked/newest_nonempty", "witness_test.go:2168; protocol_test.go:1172", func(f *fixture) []File { |
| + f.chain("g0") |
| + return []File{f.file("g1", 0, 1, 1)} |
| + }, "g1", nil}, |
| + {"wBaselineParked/holder_nonempty", "protocol_test.go:1161", func(f *fixture) []File { |
| + f.chain("g0") |
| + f.file("g1", 0, 1, 9) |
| + return []File{f.file("g2", 0, 1, 1)} |
| + }, "g2", nil}, |
| + } |
| + for _, tt := range tests { |
| + t.Run(tt.name, func(t *testing.T) { |
| + t.Log("port of", tt.source) |
| + f := newFixture(t) |
| + want := tt.setup(f) |
| + before := f.b.Snapshot() |
| + f.b.FailOn = func(op, key string) error { |
| + if op != "list" && op != "get" { |
| + t.Fatalf("mutation %s %s", op, key) |
| + } |
| + return nil |
| + } |
| + got, err := Select(t.Context(), f.store, f.req) |
| + if !errors.Is(err, tt.wantErr) { |
| + t.Fatalf("got %v, want %v", err, tt.wantErr) |
| + } |
| + if f.b.Snapshot() != before { |
| + t.Fatal("selection mutated storage") |
| + } |
| + if tt.wantErr != nil { |
| + if got != nil { |
| + t.Fatal("partial plan") |
| + } |
| + return |
| + } |
| + if got.Generation != tt.gen || !reflect.DeepEqual(got.Files, want) { |
| + t.Fatalf("got %+v, want %s %+v", got, tt.gen, want) |
| + } |
| + if got.MarkerVersion != f.markerVersion || got.Root == nil || *got.Root != want[0] || got.MaxTXID != want[len(want)-1].MaxTXID { |
| + t.Fatalf("root, marker or final TXID: %+v", got) |
| + } |
| + }) |
| + } |
| +} |
| + |
| +type altered struct { |
| + Reader |
| + list func([]lease.ObjectVersion) ([]lease.ObjectVersion, error) |
| + open func(context.Context, string, string, string) (io.ReadCloser, error) |
| +} |
| + |
| +func (s altered) ListVersions(ctx context.Context, bucket, base string) ([]lease.ObjectVersion, error) { |
| + vs, err := s.Reader.ListVersions(ctx, bucket, base) |
| + if err != nil { |
| + return nil, err |
| + } |
| + if s.list != nil { |
| + return s.list(vs) |
| + } |
| + return vs, nil |
| +} |
| +func (s altered) OpenVersion(ctx context.Context, bucket, key, id string) (io.ReadCloser, error) { |
| + if s.open != nil { |
| + return s.open(ctx, bucket, key, id) |
| + } |
| + return s.Reader.OpenVersion(ctx, bucket, key, id) |
| +} |
| + |
| +func TestRefusals(t *testing.T) { |
| + tests := []struct { |
| + name string |
| + setup func(*fixture) |
| + want error |
| + }{ |
| + {"marker_holder_disagreement", func(f *fixture) { f.req.Holder.Generation = "unknown" }, ErrDisagreement}, |
| + {"nonempty_unexplained_disagreement", func(f *fixture) { f.req.Holder.Generation = "unknown"; f.file("unknown", 0, 1, 1) }, ErrDisagreement}, |
| + {"covered_chain", func(f *fixture) { f.chain("g0"); f.req.Holder.Admissions[0].Covered = true }, ErrDisagreement}, |
| + {"broken_chain", func(f *fixture) { f.chain("g0"); f.req.Holder.Admissions[1].Predecessor = "missing" }, ErrDisagreement}, |
| + {"cyclic_chain", func(f *fixture) { f.chain("g0"); f.req.Holder.Admissions[0].Predecessor = "g2" }, ErrDisagreement}, |
| + {"foreign_baseline", func(f *fixture) { f.chain("g0"); f.req.Holder.Admissions[0].PredecessorRegion = "us" }, ErrRegion}, |
| + {"incomplete_listing", func(f *fixture) { f.b.FailListAfterPages = 1 }, ErrListing}, |
| + {"partial_data_with_error", func(f *fixture) { |
| + f.store = altered{Reader: f.b, list: func(vs []lease.ObjectVersion) ([]lease.ObjectVersion, error) { return vs, io.ErrUnexpectedEOF }} |
| + }, ErrListing}, |
| + {"incomplete_unfinished_listing", func(f *fixture) { f.chain("g0"); f.b.FailListAfterPages = 1 }, ErrListing}, |
| + {"pinned_version_missing", func(f *fixture) { |
| + f.b.Put("data", f.pin.Key, header(f.t, 1, 11)) |
| + if err := f.b.DeleteVersion(f.t.Context(), "data", f.pin.Key, f.pin.VersionID); err != nil { |
| + f.t.Fatal(err) |
| + } |
| + }, ErrPinnedMissing}, |
| + {"pinned_version_vanished_after_listing", func(f *fixture) { |
| + f.store = altered{Reader: f.b, open: func(context.Context, string, string, string) (io.ReadCloser, error) { |
| + return nil, lease.ErrMirrorNotFound |
| + }} |
| + }, ErrPinnedMissing}, |
| + {"pinned_header_mismatch", func(f *fixture) { |
| + f.marker.SnapshotVersion = f.b.Put("data", f.pin.Key, header(f.t, 1, 12)) |
| + f.publish() |
| + }, ErrPinnedMismatch}, |
| + {"pinned_key_mismatch", func(f *fixture) { f.marker.SnapshotTXID = 12; f.publish() }, ErrPinnedMismatch}, |
| + {"pinned_delete_marker", func(f *fixture) { f.marker.SnapshotVersion = f.b.BareDelete("data", f.pin.Key); f.publish() }, ErrDeleteMarker}, |
| + {"selected_delete_marker_on_open", func(f *fixture) { |
| + f.store = altered{Reader: f.b, open: func(context.Context, string, string, string) (io.ReadCloser, error) { |
| + return nil, lease.ErrIsDeleteMarker |
| + }} |
| + }, ErrDeleteMarker}, |
| + {"marker_delete_marker", func(f *fixture) { f.b.BareDelete("data", f.req.Holder.Location+"/generation.json") }, ErrDeleteMarker}, |
| + {"marker_read_error", func(f *fixture) { |
| + f.b.FailOn = func(op, key string) error { |
| + if op == "get" { |
| + return io.ErrUnexpectedEOF |
| + } |
| + return nil |
| + } |
| + }, ErrMarker}, |
| + {"marker_disappears", func(f *fixture) { |
| + f.b.FailOn = func(op, key string) error { |
| + if op == "get" { |
| + return lease.ErrMirrorNotFound |
| + } |
| + return nil |
| + } |
| + }, ErrMarker}, |
| + {"marker_incarnation", func(f *fixture) { f.marker.Incarnation = "previous"; f.publish() }, ErrIncarnation}, |
| + {"marker_host", func(f *fixture) { f.marker.Host = "other"; f.publish() }, ErrMarker}, |
| + {"malformed_marker", func(f *fixture) { f.b.Put("data", f.req.Holder.Location+"/generation.json", []byte("{")) }, ErrMarker}, |
| + {"trailing_marker_json", func(f *fixture) { |
| + b, _ := json.Marshal(f.marker) |
| + f.b.Put("data", f.req.Holder.Location+"/generation.json", append(b, []byte(" {}")...)) |
| + }, ErrMarker}, |
| + {"empty_pin_version", func(f *fixture) { f.marker.SnapshotVersion = ""; f.publish() }, ErrVersionID}, |
| + {"null_pin_version", func(f *fixture) { f.marker.SnapshotVersion = "null"; f.publish() }, ErrVersionID}, |
| + {"empty_current_version", func(f *fixture) { alterVersion(f, "") }, ErrVersionID}, |
| + {"null_current_version", func(f *fixture) { alterVersion(f, "null") }, ErrVersionID}, |
| + {"tombstoned_unfinished_is_not_empty", func(f *fixture) { |
| + f.chain("g0") |
| + dead := f.file("g2", 0, 1, 1) |
| + f.b.BareDelete("data", dead.Key) |
| + }, ErrEmpty}, |
| + {"metadata_only_unfinished", func(f *fixture) { |
| + f.chain("g0") |
| + f.b.Put("data", f.req.Holder.Location+"/g-g2/build-intent.json", []byte("{}")) |
| + }, ErrEmpty}, |
| + {"empty_published_generation", func(f *fixture) { |
| + f.marker.Op = "empty" |
| + f.marker.Pin = Pin{} |
| + f.req.Holder.Generation = "empty" |
| + f.publish() |
| + }, ErrEmpty}, |
| + {"bad_successor_no_fallback", func(f *fixture) { r := f.file("g0", 9, 1, 20); f.b.Put("data", r.Key, header(f.t, 1, 19)) }, ErrRoot}, |
| + {"missing_snapshot_txid", func(f *fixture) { |
| + b, _ := json.Marshal(f.marker) |
| + var fields map[string]any |
| + json.Unmarshal(b, &fields) |
| + delete(fields, "snapshot_txid") |
| + b, _ = json.Marshal(fields) |
| + f.b.Put("data", f.req.Holder.Location+"/generation.json", b) |
| + }, ErrMarker}, |
| + {"malformed_current_ltx", func(f *fixture) { f.b.Put("data", f.req.Holder.Location+"/g-g0/0000/not-a-range.ltx", []byte("bad")) }, ErrListing}, |
| + {"invalid_snapshot_min", func(f *fixture) { f.file("g0", 9, 2, 20) }, ErrRoot}, |
| + {"gap_without_root", func(f *fixture) { f.origin(); f.file("", 0, 2, 2) }, ErrGap}, |
| + {"location_escape", func(f *fixture) { f.req.Holder.Location = "instances/h/../other" }, ErrInput}, |
| + {"holder_root_sentinel", func(f *fixture) { f.req.Holder.Generation = hostclaim.RootGeneration }, ErrInput}, |
| + } |
| + for _, tt := range tests { |
| + t.Run(tt.name, func(t *testing.T) { |
| + f := newFixture(t) |
| + tt.setup(f) |
| + before := f.b.Snapshot() |
| + got, err := Select(t.Context(), f.store, f.req) |
| + if got != nil || !errors.Is(err, tt.want) { |
| + t.Fatalf("got %+v / %v, want %v", got, err, tt.want) |
| + } |
| + if f.b.Snapshot() != before { |
| + t.Fatal("mutated storage") |
| + } |
| + }) |
| + } |
| +} |
| + |
| +func alterVersion(f *fixture, id string) { |
| + f.file("g0", 0, 12, 12) |
| + f.store = altered{Reader: f.b, list: func(vs []lease.ObjectVersion) ([]lease.ObjectVersion, error) { |
| + for i := range vs { |
| + if strings.Contains(vs[i].Key, "0000/000000000000000c-") { |
| + vs[i].VersionID = id |
| + } |
| + } |
| + return vs, nil |
| + }} |
| +} |
| + |
| +func TestSelectionDetails(t *testing.T) { |
| + tests := []struct { |
| + name string |
| + setup func(*fixture) []File |
| + }{ |
| + {"pinned_noncurrent_duplicate", func(f *fixture) []File { |
| + f.b.Put("data", f.pin.Key, header(f.t, 1, 11)) |
| + return []File{f.pin} |
| + }}, |
| + {"pinned_under_retention_tombstone", func(f *fixture) []File { |
| + f.b.BareDelete("data", f.pin.Key) |
| + return []File{f.pin} |
| + }}, |
| + {"retention_tombstones_ignored", func(f *fixture) []File { |
| + f.b.BareDelete("data", f.file("g0", 0, 13, 13).Key) |
| + return []File{f.pin} |
| + }}, |
| + {"max_then_min_then_level", func(f *fixture) []File { |
| + f.file("g0", 8, 12, 19) |
| + f.file("g0", 8, 12, 20) |
| + f.file("g0", 1, 10, 20) |
| + return []File{f.pin, f.file("g0", 2, 10, 20)} |
| + }}, |
| + {"origin_snapshot_and_isolation", func(f *fixture) []File { |
| + f.origin() |
| + f.file("other", 9, 1, 999) |
| + f.b.Put("data", f.req.Holder.Location+"/child/0009/0000000000000001-000000000000ffff.ltx", header(f.t, 1, 65535)) |
| + return []File{f.file("", 9, 1, 20), f.file("", 0, 21, 21)} |
| + }}, |
| + {"legacy_location", func(f *fixture) []File { |
| + f.req.Holder.Location = "instances/h" |
| + f.req.Holder.Generation = "" |
| + return []File{f.file("", 0, 1, 1)} |
| + }}, |
| + {"fresh_zero_pin_marker", func(f *fixture) []File { |
| + f.marker.Op = "fresh" |
| + f.marker.Pin = Pin{} |
| + f.req.Holder.Generation = "fresh" |
| + f.publish() |
| + return []File{f.file("fresh", 0, 1, 1)} |
| + }}, |
| + {"warm_record_in_unfinished_run", func(f *fixture) []File { |
| + f.chain("g0") |
| + f.req.Holder.Admissions = append(f.req.Holder.Admissions, hostclaim.AdmissionRecord{Generation: "g2", Predecessor: "g2", Region: "ie", PredecessorRegion: "ie", Mode: "warm"}) |
| + return []File{f.pin} |
| + }}, |
| + {"prepared_unpublished_root", func(f *fixture) []File { |
| + f.chain("g0") |
| + r := f.file("g2", 9, 1, 30) |
| + f.file("g2", 0, 30, 30) |
| + f.req.Prepared = &Prepared{Generation: "g2", Pin: Pin{SnapshotKey: r.Key, SnapshotVersion: r.VersionID, SnapshotTXID: 30}} |
| + f.b.Put("data", r.Key, header(f.t, 1, 30)) |
| + return []File{r} |
| + }}, |
| + {"root_advance_past_missing_old_pin", func(f *fixture) []File { |
| + if err := f.b.DeleteVersion(f.t.Context(), "data", f.pin.Key, f.pin.VersionID); err != nil { |
| + f.t.Fatal(err) |
| + } |
| + return []File{f.file("g0", 9, 1, 30)} |
| + }}, |
| + } |
| + for _, tt := range tests { |
| + t.Run(tt.name, func(t *testing.T) { |
| + f := newFixture(t) |
| + want := tt.setup(f) |
| + got, err := Select(t.Context(), f.store, f.req) |
| + if err != nil || !reflect.DeepEqual(got.Files, want) { |
| + t.Fatalf("got %+v / %v, want %+v", got, err, want) |
| + } |
| + }) |
| + } |
| +} |
| + |
| +``` |
| + |
| +- [ ] **Step 3: Run the tests and observe failure before adding the implementation.** |
| + |
| +```bash |
| +go test ./internal/genselect -count=1 |
| +``` |
| + |
| +Expected: compilation fails because `Reader`, `Request`, `Select` and the other new API symbols do not exist. Once implemented, each case must check exact version identities and refusal must return no partial plan. |
| + |
| +### Task 2: Implement the read-only selector |
| + |
| +**Files:** create `internal/genselect/select.go`. |
| + |
| +**Interfaces:** consumes the read-only subset of `lease.VersionedStore` and the actual `hostclaim.Holder`; produces `Plan`, `File`, `Marker`, `Pin`, `Prepared`, and typed errors. `Prepared` is a caller-supplied pin for an authorized completed build, not permission to construct or publish a generation. |
| + |
| +- [ ] **Step 1: Write the complete implementation.** |
| + |
| +```go |
| +package genselect |
| + |
| +import ( |
| + "bytes" |
| + "context" |
| + "encoding/json" |
| + "errors" |
| + "fmt" |
| + "io" |
| + "strconv" |
| + "strings" |
| + "time" |
| + |
| + "github.com/carlosframework/platform/internal/hostclaim" |
| + "github.com/carlosframework/platform/internal/lease" |
| + "github.com/superfly/ltx" |
| +) |
| + |
| +var ( |
| + ErrListing = errors.New("genselect: incomplete or invalid listing") |
| + ErrDisagreement = errors.New("genselect: marker and holder disagree") |
| + ErrIncarnation = errors.New("genselect: marker incarnation mismatch") |
| + ErrMarker = errors.New("genselect: malformed marker") |
| + ErrVersionID = errors.New("genselect: immutable version id required") |
| + ErrDeleteMarker = errors.New("genselect: selected version is a delete marker") |
| + ErrPinnedMissing = errors.New("genselect: pinned snapshot version missing") |
| + ErrPinnedMismatch = errors.New("genselect: pinned snapshot mismatch") |
| + ErrGap = errors.New("genselect: non-contiguous ltx files") |
| + ErrEmpty = errors.New("genselect: no restorable data") |
| + ErrRegion = errors.New("genselect: baseline belongs to another region") |
| + ErrInput = errors.New("genselect: invalid input") |
| + ErrRoot = errors.New("genselect: invalid root") |
| +) |
| + |
| +type Reader interface { |
| + ListVersions(context.Context, string, string) ([]lease.ObjectVersion, error) |
| + GetVersion(context.Context, string, string, string) ([]byte, error) |
| + OpenVersion(context.Context, string, string, string) (io.ReadCloser, error) |
| +} |
| + |
| +var _ Reader = (lease.VersionedStore)(nil) |
| + |
| +type Pin struct { |
| + SnapshotKey string `json:"snapshot_key"` |
| + SnapshotVersion string `json:"snapshot_version"` |
| + SnapshotTXID uint64 `json:"snapshot_txid"` |
| + SnapshotCRC64 uint64 `json:"snapshot_crc64"` |
| +} |
| + |
| +type Marker struct { |
| + Op string `json:"op"` |
| + Incarnation string `json:"incarnation"` |
| + Host string `json:"host"` |
| + Box string `json:"box"` |
| + InstalledAt time.Time `json:"installed_at"` |
| + ManifestSHA256 string `json:"manifest_sha256"` |
| + Pin |
| +} |
| + |
| +// Prepared is supplied only from an authorized, completed build journal. |
| +type Prepared struct { |
| + Generation string |
| + Pin |
| +} |
| + |
| +type Request struct { |
| + Bucket, Region, Host string |
| + Holder hostclaim.Holder |
| + Prepared *Prepared |
| +} |
| + |
| +type File struct { |
| + Key string `json:"key"` |
| + VersionID string `json:"version_id"` |
| + Level int `json:"level"` |
| + MinTXID uint64 `json:"min_txid"` |
| + MaxTXID uint64 `json:"max_txid"` |
| + Size int64 `json:"size"` |
| +} |
| + |
| +type Plan struct { |
| + Generation string |
| + Files []File |
| + Root *File |
| + Marker *Marker |
| + MarkerVersion string |
| + Pin *Pin |
| + MaxTXID uint64 |
| + MaxLevel0TXID uint64 |
| +} |
| + |
| +type candidate struct{ generation, region string } |
| + |
| +func origin(g string) string { |
| + if g == "" { |
| + return hostclaim.RootGeneration |
| + } |
| + return g |
| +} |
| + |
| +func component(s string) bool { |
| + return s != "" && s != "." && s != ".." && !strings.ContainsAny(s, "/\\\x00") |
| +} |
| + |
| +func generation(s string) bool { return component(s) && s != hostclaim.RootGeneration } |
| + |
| +func versionID(s string) bool { return strings.TrimSpace(s) != "" && s != "null" } |
| + |
| +func prefix(location, gen string) string { |
| + if gen == hostclaim.RootGeneration { |
| + return location |
| + } |
| + return location + "g-" + gen + "/" |
| +} |
| + |
| +func coordinates(key, base string) (File, bool) { |
| + if !strings.HasPrefix(key, base) { |
| + return File{}, false |
| + } |
| + parts := strings.Split(strings.TrimPrefix(key, base), "/") |
| + if len(parts) != 2 || len(parts[0]) != 4 { |
| + return File{}, false |
| + } |
| + level, err := strconv.ParseUint(parts[0], 16, 16) |
| + if err != nil || level > 9 || fmt.Sprintf("%04x", level) != parts[0] { |
| + return File{}, false |
| + } |
| + min, max, err := ltx.ParseFilename(parts[1]) |
| + if err != nil || min == 0 || min > max || ltx.FormatFilename(min, max) != parts[1] { |
| + return File{}, false |
| + } |
| + return File{Key: key, Level: int(level), MinTXID: uint64(min), MaxTXID: uint64(max)}, true |
| +} |
| + |
| +func exactFile(v lease.ObjectVersion, base string) (File, error) { |
| + f, ok := coordinates(v.Key, base) |
| + if !ok || v.Size < 0 { |
| + return File{}, ErrListing |
| + } |
| + if !versionID(v.VersionID) { |
| + return File{}, ErrVersionID |
| + } |
| + if v.DeleteMarker { |
| + return File{}, ErrDeleteMarker |
| + } |
| + f.VersionID, f.Size = v.VersionID, v.Size |
| + return f, nil |
| +} |
| + |
| +// Select returns identities only. The consumer must verify every complete file. |
| +func Select(ctx context.Context, s Reader, req Request) (*Plan, error) { |
| + h := req.Holder |
| + location := strings.TrimSuffix(h.Location, "/") |
| + if s == nil || req.Bucket == "" || req.Region == "" || !component(req.Host) || !component(h.Incarnation) || |
| + (location != "instances/"+req.Host && location != "instances/"+req.Host+"/"+h.Incarnation) || |
| + (h.Generation != "" && !generation(h.Generation)) { |
| + return nil, ErrInput |
| + } |
| + location += "/" |
| + // VersionedStore drains all pages before returning. Never use partial data. |
| + versions, err := s.ListVersions(ctx, req.Bucket, location) |
| + if err != nil { |
| + return nil, fmt.Errorf("%w: %w", ErrListing, err) |
| + } |
| + latest := make(map[string]lease.ObjectVersion) |
| + keys := make(map[string]bool) |
| + for _, v := range versions { |
| + if !strings.HasPrefix(v.Key, location) { |
| + return nil, ErrListing |
| + } |
| + keys[v.Key] = true |
| + if v.IsLatest { |
| + if _, exists := latest[v.Key]; exists { |
| + return nil, ErrListing |
| + } |
| + latest[v.Key] = v |
| + } |
| + } |
| + for k := range keys { |
| + if _, ok := latest[k]; !ok { |
| + return nil, ErrListing |
| + } |
| + } |
| + p := &Plan{Files: []File{}} |
| + markerGen := hostclaim.RootGeneration |
| + if mv, ok := latest[location+"generation.json"]; ok { |
| + if !versionID(mv.VersionID) { |
| + return nil, ErrVersionID |
| + } |
| + if mv.DeleteMarker { |
| + return nil, ErrDeleteMarker |
| + } |
| + data, err := s.GetVersion(ctx, req.Bucket, mv.Key, mv.VersionID) |
| + if err != nil { |
| + return nil, fmt.Errorf("%w: %w", ErrMarker, err) |
| + } |
| + var fields map[string]json.RawMessage |
| + if err := json.Unmarshal(data, &fields); err != nil { |
| + return nil, ErrMarker |
| + } |
| + if raw, ok := fields["snapshot_txid"]; !ok || string(raw) == "null" { |
| + return nil, ErrMarker |
| + } |
| + var m Marker |
| + dec := json.NewDecoder(bytes.NewReader(data)) |
| + // Unknown additive fields are allowed. Malformed/trailing JSON is not. |
| + if err := dec.Decode(&m); err != nil { |
| + return nil, fmt.Errorf("%w: %w", ErrMarker, err) |
| + } |
| + if err := dec.Decode(new(any)); err != io.EOF { |
| + return nil, ErrMarker |
| + } |
| + if m.Incarnation != h.Incarnation { |
| + return nil, ErrIncarnation |
| + } |
| + if m.Host != req.Host || !generation(m.Op) || m.Box == "" || m.InstalledAt.IsZero() { |
| + return nil, ErrMarker |
| + } |
| + markerGen, p.Marker, p.MarkerVersion = m.Op, &m, mv.VersionID |
| + } |
| + run, err := unfinished(h, markerGen, req.Region) |
| + if err != nil { |
| + return nil, err |
| + } |
| + selected := candidate{markerGen, req.Region} |
| + for _, c := range run { |
| + if c.region != "" && c.region != req.Region { |
| + return nil, ErrRegion |
| + } |
| + if c.generation == markerGen || hasObject(latest, location, c.generation) { |
| + selected = c |
| + break |
| + } |
| + } |
| + p.Generation = selected.generation |
| + base := prefix(location, p.Generation) |
| + var pin *Pin |
| + if p.Marker != nil && p.Generation == markerGen { |
| + cp := p.Marker.Pin |
| + pin = &cp |
| + } |
| + if req.Prepared != nil && req.Prepared.Generation == p.Generation { |
| + if pin != nil && *pin != req.Prepared.Pin { |
| + return nil, ErrPinnedMismatch |
| + } |
| + cp := req.Prepared.Pin |
| + pin = &cp |
| + } |
| + if pin != nil { |
| + if pin.SnapshotTXID == 0 { |
| + if pin.SnapshotKey != "" || pin.SnapshotVersion != "" || pin.SnapshotCRC64 != 0 { |
| + return nil, ErrPinnedMismatch |
| + } |
| + } else { |
| + f, ok := coordinates(pin.SnapshotKey, base) |
| + if !ok || f.Level != 9 || f.MinTXID != 1 || f.MaxTXID != pin.SnapshotTXID { |
| + return nil, ErrPinnedMismatch |
| + } |
| + if !versionID(pin.SnapshotVersion) { |
| + return nil, ErrVersionID |
| + } |
| + } |
| + p.Pin = pin |
| + } |
| + var current []File |
| + for _, v := range latest { |
| + f, ok := coordinates(v.Key, base) |
| + if !ok && strings.HasPrefix(v.Key, base) { |
| + rel := strings.TrimPrefix(v.Key, base) |
| + parts := strings.Split(rel, "/") |
| + if len(parts) == 2 && len(parts[0]) == 4 && strings.HasSuffix(parts[1], ".ltx") { |
| + if level, e := strconv.ParseUint(parts[0], 16, 16); e == nil && level <= 9 { |
| + return nil, ErrListing |
| + } |
| + } |
| + } |
| + if !ok || v.DeleteMarker { |
| + continue |
| + } // Retention tombstones are not current files. |
| + f, err = exactFile(v, base) |
| + if err != nil { |
| + return nil, err |
| + } |
| + if f.Level == 9 && f.MinTXID != 1 { |
| + return nil, ErrRoot |
| + } |
| + current = append(current, f) |
| + if f.Level == 0 && f.MaxTXID > p.MaxLevel0TXID { |
| + p.MaxLevel0TXID = f.MaxTXID |
| + } |
| + } |
| + floor := uint64(0) |
| + if pin != nil { |
| + floor = pin.SnapshotTXID |
| + } |
| + for _, f := range current { |
| + // A late duplicate at the pinned key never replaces its acknowledged version. |
| + if pin != nil && floor > 0 && f.Key == pin.SnapshotKey { |
| + continue |
| + } |
| + if f.Level == 9 && f.MinTXID == 1 && f.MaxTXID >= floor && (p.Root == nil || better(f, *p.Root)) { |
| + cp := f |
| + p.Root = &cp |
| + } |
| + } |
| + pinned := false |
| + if p.Root == nil && floor > 0 { |
| + var pv *lease.ObjectVersion |
| + for _, v := range versions { |
| + if v.Key == pin.SnapshotKey && v.VersionID == pin.SnapshotVersion { |
| + cp := v |
| + pv = &cp |
| + break |
| + } |
| + } |
| + if pv == nil { |
| + return nil, ErrPinnedMissing |
| + } |
| + f, err := exactFile(*pv, base) |
| + if err != nil { |
| + return nil, err |
| + } |
| + p.Root, pinned = &f, true |
| + } |
| + if p.Root != nil { |
| + if err := checkRoot(ctx, s, req.Bucket, *p.Root, pinned); err != nil { |
| + return nil, err |
| + } |
| + p.Files = append(p.Files, *p.Root) |
| + p.MaxTXID = p.Root.MaxTXID |
| + } |
| + for { |
| + var best *File |
| + for _, f := range current { |
| + if f.Level == 9 || p.MaxTXID == ^uint64(0) || !ltx.IsContiguous(ltx.TXID(p.MaxTXID), ltx.TXID(f.MinTXID), ltx.TXID(f.MaxTXID)) { |
| + continue |
| + } |
| + if best == nil || better(f, *best) { |
| + cp := f |
| + best = &cp |
| + } |
| + } |
| + if best == nil { |
| + break |
| + } |
| + if p.Root == nil { |
| + cp := *best |
| + p.Root = &cp |
| + } |
| + p.Files = append(p.Files, *best) |
| + p.MaxTXID = best.MaxTXID |
| + } |
| + for _, f := range current { |
| + if f.Level < 9 && f.MaxTXID > p.MaxTXID { |
| + return nil, ErrGap |
| + } |
| + } |
| + if len(p.Files) == 0 { |
| + return nil, ErrEmpty |
| + } |
| + return p, nil |
| +} |
| + |
| +func unfinished(h hostclaim.Holder, marker, region string) ([]candidate, error) { |
| + gen := origin(h.Generation) |
| + if gen == marker { |
| + return []candidate{{marker, region}}, nil |
| + } |
| + var out []candidate |
| + seen := map[string]bool{} |
| + for i := len(h.Admissions) - 1; i >= 0; i-- { |
| + a := h.Admissions[i] |
| + if origin(a.Generation) != gen || a.Region != region { |
| + return nil, ErrDisagreement |
| + } |
| + if a.Mode == "warm" { |
| + if origin(a.Predecessor) != gen || a.PredecessorRegion != region { |
| + return nil, ErrDisagreement |
| + } |
| + continue |
| + } |
| + if a.Mode != "cold" || a.Covered || !generation(a.Generation) || seen[gen] { |
| + return nil, ErrDisagreement |
| + } |
| + seen[gen] = true |
| + out = append(out, candidate{gen, region}) |
| + gen, region = origin(a.Predecessor), a.PredecessorRegion |
| + if gen == marker { |
| + return append(out, candidate{gen, region}), nil |
| + } |
| + if !generation(gen) || region == "" { |
| + return nil, ErrDisagreement |
| + } |
| + } |
| + return nil, ErrDisagreement |
| +} |
| + |
| +func hasObject(latest map[string]lease.ObjectVersion, location, gen string) bool { |
| + base := prefix(location, gen) |
| + for k := range latest { |
| + if !strings.HasPrefix(k, base) { |
| + continue |
| + } |
| + if gen == hostclaim.RootGeneration { |
| + if _, ok := coordinates(k, base); !ok { |
| + continue |
| + } |
| + } |
| + return true |
| + } |
| + return false |
| +} |
| + |
| +func better(a, b File) bool { |
| + if a.MaxTXID != b.MaxTXID { |
| + return a.MaxTXID > b.MaxTXID |
| + } |
| + if a.MinTXID != b.MinTXID { |
| + return a.MinTXID < b.MinTXID |
| + } |
| + return a.Level > b.Level |
| +} |
| + |
| +func checkRoot(ctx context.Context, s Reader, bucket string, f File, pinned bool) error { |
| + category := ErrRoot |
| + if pinned { |
| + category = ErrPinnedMismatch |
| + } |
| + rc, err := s.OpenVersion(ctx, bucket, f.Key, f.VersionID) |
| + if err != nil { |
| + if errors.Is(err, lease.ErrIsDeleteMarker) { |
| + return fmt.Errorf("%w: %w", ErrDeleteMarker, err) |
| + } |
| + if pinned && errors.Is(err, lease.ErrMirrorNotFound) { |
| + return fmt.Errorf("%w: %w", ErrPinnedMissing, err) |
| + } |
| + return fmt.Errorf("%w: %w", category, err) |
| + } |
| + dec := ltx.NewDecoder(rc) |
| + err = dec.DecodeHeader() |
| + closeErr := rc.Close() |
| + if err != nil { |
| + return fmt.Errorf("%w: %w", category, err) |
| + } |
| + if closeErr != nil { |
| + return fmt.Errorf("%w: %w", category, closeErr) |
| + } |
| + h := dec.Header() |
| + if uint64(h.MinTXID) != f.MinTXID || uint64(h.MaxTXID) != f.MaxTXID { |
| + return category |
| + } |
| + return nil |
| +} |
| + |
| +``` |
| + |
| +- [ ] **Step 2: Format and pass the witness and refusal tests.** |
| + |
| +```bash |
| +gofmt -w internal/genselect/select.go internal/genselect/select_test.go |
| +go test ./internal/genselect -count=1 -v |
| +``` |
| + |
| +Expected: all witness, refusal and selection-detail rows pass. The selected generation is determined by admissions, never by the largest TXID across generations. A successful selection does not write a marker or anchor. |
| + |
| +### Task 3: Check the pinned library's planner |
| + |
| +**Files:** create `internal/genselect/compatibility_test.go`. |
| + |
| +- [ ] **Step 1: Write this complete comparison test.** The file replica is only the reference planner's fixture; its on-disk decimal-level layout is not used to parse S3 keys. The selector still runs against `leasetest.VersionedBucket`. |
| + |
| +```go |
| +package genselect |
| + |
| +import ( |
| + "io" |
| + "log/slog" |
| + "os" |
| + "path/filepath" |
| + "testing" |
| + "time" |
| + |
| + "github.com/benbjohnson/litestream" |
| + filereplica "github.com/benbjohnson/litestream/file" |
| + "github.com/superfly/ltx" |
| +) |
| + |
| +func TestCalcRestorePlanCompatibility(t *testing.T) { |
| + type coords struct { |
| + level int |
| + min, max uint64 |
| + } |
| + tests := []struct { |
| + name string |
| + files []coords |
| + }{ |
| + {"compacted_overlap", []coords{{9, 1, 11}, {1, 10, 13}}}, |
| + {"genuine_gap", []coords{{9, 1, 11}, {0, 13, 13}}}, |
| + {"successor_snapshot", []coords{{9, 1, 11}, {9, 1, 20}, {0, 13, 13}, {0, 21, 21}}}, |
| + {"origin", []coords{{0, 1, 1}, {1, 2, 5}}}, |
| + {"tie_breaks", []coords{{9, 1, 11}, {8, 12, 19}, {8, 12, 20}, {1, 10, 20}, {2, 10, 20}}}, |
| + {"anchor_is_not_continuation", []coords{{9, 1, 11}, {0, 11, 11}, {0, 12, 12}}}, |
| + } |
| + for _, tt := range tests { |
| + t.Run(tt.name, func(t *testing.T) { |
| + f := newFixture(t) |
| + f.origin() |
| + client := filereplica.NewReplicaClient(t.TempDir()) |
| + for _, c := range tt.files { |
| + f.file("", c.level, c.min, c.max) |
| + path := client.LTXFilePath(c.level, ltx.TXID(c.min), ltx.TXID(c.max)) |
| + if err := os.MkdirAll(filepath.Dir(path), 0700); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := os.WriteFile(path, header(t, c.min, c.max), 0600); err != nil { |
| + t.Fatal(err) |
| + } |
| + } |
| + want, wantErr := litestream.CalcRestorePlan(t.Context(), client, 0, time.Time{}, slog.New(slog.NewTextHandler(io.Discard, nil))) |
| + got, err := Select(t.Context(), f.b, f.req) |
| + if (err != nil) != (wantErr != nil) { |
| + t.Fatalf("selector %v; litestream %v", err, wantErr) |
| + } |
| + if err != nil { |
| + return |
| + } |
| + if len(got.Files) != len(want) { |
| + t.Fatalf("length %d != %d", len(got.Files), len(want)) |
| + } |
| + for i, v := range want { |
| + x := got.Files[i] |
| + if x.Level != v.Level || x.MinTXID != uint64(v.MinTXID) || x.MaxTXID != uint64(v.MaxTXID) { |
| + t.Fatalf("file %d: %+v != %+v", i, x, v) |
| + } |
| + } |
| + }) |
| + } |
| +} |
| + |
| +``` |
| + |
| +- [ ] **Step 2: Run the compatibility cases and race detector.** |
| + |
| +```bash |
| +gofmt -w internal/genselect/compatibility_test.go |
| +go test ./internal/genselect -count=1 |
| +go test -race ./internal/genselect -count=1 |
| +``` |
| + |
| +Expected: equal ordered level/range selections for origin, overlap, root advancement, ties and anchors; both planners refuse the genuine gap. Version-only requirements, absent-marker handling and unfinished-run authorization remain the responsibility of Task 1's tests, because Litestream has no claim or object-version interface. |
| + |
| +- [ ] **Step 3: Prove the overlap witness detects the old rule.** Temporarily replace the `ltx.IsContiguous` predicate in `Select` with exact adjacency (`MinTXID == current+1`, retaining `MaxTXID > current`), then run: |
| + |
| +```bash |
| +go test ./internal/genselect -run 'TestModelWitnesses/wRestoreOverlap|TestCalcRestorePlanCompatibility/compacted_overlap' -count=1 |
| +``` |
| + |
| +Expected: failure. Restore the implementation exactly, then rerun the package. Never commit the mutation. |
| + |
| +### Task 4: Verify the library and preserve the rollout boundary |
| + |
| +**Files:** no production files. |
| + |
| +- [ ] **Step 1: Run the repository gate in the implementation checkout.** |
| + |
| +```bash |
| +go build ./... |
| +go vet ./... |
| +go test ./... -count=1 |
| +gofmt -l . |
| +GOOS=darwin GOARCH=arm64 go build ./... |
| +GOOS=windows GOARCH=amd64 go build ./... |
| +GOOS=linux GOARCH=arm64 go build ./... |
| +git diff --check |
| +``` |
| + |
| +Expected: all commands pass; `gofmt -l .` and `git diff --check` are empty. If the default cache is read-only here, prefix the Go commands with `GOCACHE=/home/paulca/.cache/tmp/gocache`; that directory already exists on disk and is shared. Keep the module cache unchanged. No AWS calls or production sandbox experiments are needed. |
| + |
| +- [ ] **Step 2: Review the consumer obligations before declaring a caller integrated.** These are integration acceptance criteria, not changes authorized by this library branch: |
| + |
| +1. Obtain and authorize a fresh claim and record. For the door, keep Plan 7's O1/O2 comparison of the complete holder and operation and all existing membership, class, residency and region gates. Pass only the authorized local bucket. A `Plan` grants no authority to spawn or read another region. |
| +2. Open every file by its selected key **and version ID**, check header range against `File`, consume the whole stream, enforce the listed size and run `ltx.Decoder.Verify` through the trailer. Verify the pinned root's CRC-64 against `Pin.SnapshotCRC64` when that exact pin is used; a successor must pass its own `Verify`, not compare its checksum to the old pin's. If a selected version has disappeared or is a delete marker, fail the restore. Do not substitute a current version or older root. |
| +3. Apply in the returned order into a disposable database. Run SQLite `PRAGMA integrity_check` and require `ok`. At an install, require the exact sealed final TXID **N**; for warm continuation, require the certificate's `max_txid`. A failed check produces no published database and no app start. Map errors to member-safe refusals and the operation's alert/page policy. |
| +4. Do not compare pre/post database checksums: the pinned level-0 writer, snapshot writer and compactor use `HeaderFlagNoChecksum`, and their pre/post checksums are zero (§5.1 step 5). Per-file CRC and SQLite integrity are both still required. |
| +5. For replication initialization, compare the verified restored position with the newest current level-0 position, rechecking that position at activation under the existing guard. An install's `[N,N]` anchor must be fully uploaded and acknowledged before replication opens. A restore above current level 0 needs an anchor; a normal replica whose newest level-0 file already reaches the restored position needs none. `Select` only returns `MaxLevel0TXID`; it never creates an anchor, writes a TXID sidecar, initializes replication or claims a PUT was acknowledged. |
| +6. Before spawn, perform INV-4's fresh marker/holder, `.files-gen`, and database-generation/location checks. A lagging marker selects a baseline; it does not authorize starting the newer generation before its publication protocol completes. All admission CAS, purge, app-start/readiness ordering, prepared-generation one-use rules, certificate/request-ledger rules, writer prefix binding and retirement remain with the lifecycle consumers. |
| + |
| +- [ ] **Step 3: Obtain the required production-path review and land through amadan only after the implementation gate passes.** Follow `AGENTS.md`: describe and review the branch, squash its implementation commits to one, push to amadan, check CI on that exact tip and use `amadan branch merge -expect`. Read main's CI after landing. No branch creation, staging, committing, pushing or landing happens during this plan-writing session. |
| + |
| +## Rollout note |
| + |
| +This is a library-only dependency. Plan 7's nonempty-generation refusal stays in place until a separate follow-up integrates `genselect.Select` into its listing path, signs only the exact versions returned, and tests generation changes, unfinished runs, pinned historical roots, missing versions, pagination failures and the complete O1/O2 authorization boundary over HTTP. That follow-up must adapt the selected plan to the door's level-based protocol without relisting a different snapshot on each request, and preserve the same version identities through all GETs. Plan 8 must not enable the generation-aware door before those tests pass. |
| + |
| +The moves plans import this package and supply completed-journal pins for prepared generations. They must bind sealed transfer versions as the source inventory when building from a seal, without substituting today's staged listing; their adapter and byte-verifying restore remain separate tasks. They do not copy this selector. The library does not remove the need for their admission, initialization, final-TXID and INV-4 integration tests. |
| + |
| +## Self-review: requirements mapped to tasks |
| + |
| +| Moves §5.1 clause | Task and boundary | |
| +|---|---| |
| +| Location/generation definition and exact S3 layout | Task 2 `prefix`, `coordinates`, holder-location check; Task 1 origin/legacy/child-isolation tests | |
| +| 1: sealed transfer, exact versions, required final N, 16 MiB cut bar | Task 2 exact version output; Task 4 consumer checks 1–3 and rollout's sealed-inventory adapter. Sealing, transfer and cut budget are moves responsibilities | |
| +| 2: staged restore, snapshot `[1,N]`, anchor `[N,N]`, durable build intent then acknowledged artifacts then completed manifest | Task 2 `Prepared` pin and root range; Task 3 anchor comparison; Task 4 checks 2–5. No builder or manifest publication here | |
| +| 3: snapshot key/version/TXID/CRC identity; late duplicate never read | Task 2 `Marker`, `Pin`, exact historical fallback; Task 1 duplicate, missing-pin, mismatch and delete-marker cases; Task 4 CRC check | |
| +| 4: newest qualifying current level-9 root, current levels 0–8, overlap and all three tie breaks | Task 2 root/continuation loops; Tasks 1 and 3 overlap, gap, advancement and tie cases | |
| +| 5: missing/wrong pin, Verify failure, gap, integrity_check, required install TXID; no pre/post checksum test | Task 1 distinct refusal cases; Task 2 header/pin/gap errors; Task 4 checks 2–4 own full-byte verification and SQLite | |
| +| 6: a gap refuses rather than returning a shortened database | Task 1 `wRestoreGap`, gap without root; Task 3 comparison; every error returns nil | |
| +| 7: acknowledged anchor, remote level-0 position, no oracle writes, no TXID-sidecar shortcut | Task 2 `MaxLevel0TXID` and read-only interface; Task 3 anchor skipped as continuation; Task 4 check 5 owns initialization | |
| +| 8: rotation/retention/root advancement; current versus noncurrent selection; newest level 0 | Tasks 1–3 advancement and retention cases; Task 2 current-only candidates and max-level-0 output. Protecting versions from sweep/retention is not this read-only API | |
| +| 9: writer-session isolation and fresh claim admission before app start | Task 2 confines selection to one authorized location/generation; Task 4 checks 1 and 6 preserve admission/CAS and writer binding responsibilities | |
| +| 9: purge/remote baseline, warm certificate target, cold prepared/fresh generation, empty bootstrap and writes_lost | Tasks 1–2 baseline resolution and prepared pin; Task 4 checks 3 and 6 preserve purge, certificate, bootstrap and loss-declaration decisions | |
| +| 9: unfinished run and absent-marker origin | Task 2 `unfinished`, `hasObject`; Task 1 `wBaselineParked`, `wOriginSwept`, complete-listing and covered-chain refusals | |
| +| 9: publication, per-region coverage, predecessor parking, retry policy, one-use close certificates, generation-bound request ledger | Task 4 check 6 and rollout: caller-owned side effects, never performed or certified by `Select` | |
| +| §6.4 marker table and INV-4 start check | Task 2 full marker schema plus host/incarnation check; Task 4 check 6 retains files/database marker checks | |
| + |
| +| Brief item | Task and evidence | |
| +|---|---| |
| +| 1: read-only | Task 2 `Reader`; Task 1 storage snapshots and mutation-rejecting witness hook | |
| +| 2: absent marker means origin | Task 1 `wOriginSwept`; Task 2 marker default and root layout | |
| +| 3: unfinished chain, newest object-bearing baseline, complete pages, other disagreement refuses | Tasks 1–2 chain, history/metadata occupancy, foreign baseline, partial-page/error tests | |
| +| 4: exact pin, successor snapshot, anchor handling | Tasks 1–3 pin/root/anchor cases; Task 4 byte verification and acknowledged initialization | |
| +| 5: CalcRestorePlan continuation and tie order | Tasks 2–3 `ltx.IsContiguous`, comparator and direct library comparison | |
| +| 6: distinct fail-closed errors | Tasks 1–2 error table, late disappearance, bad versions, header mismatch, deleted pins and no fallback | |
| +| 7: generation, ordered exact versions, root, marker version; consumer checks | Task 2 `Plan`/`File`; Task 1 exact struct assertions; Task 4 checks 2–5 | |
| +| 8: named model witness ports on VersionedBucket | Task 1 table with model function names/file:line; Task 3 real planner comparison | |
| +| Claim C3a, C6, §6.2a, §6.4a item 3 | Task 2 exact namespace, complete reads and uncovered predecessor walk; Task 1 corresponding refusals | |
| +| Branch, library scope, rollout and cross-platform gate | Tasks 1 and 4, rollout note | |
| + |
| +## Authoring verification |
| + |
| +The embedded code was compiled using a Go overlay, with Plan 1's actual `hostclaim` sources and `origin/main`'s actual `lease`/`leasetest` sources. Scratch files were outside the repository under the on-disk `$TMPDIR`. The default cache was read-only, so verification used the existing shared `$TMPDIR/gocache`. Go's test runner and vet cannot `chdir` into an overlay-only package directory; the check therefore built the test binary with `go test -c -vet=off` and ran that binary from scratch. This is a compile and package-test check, not a claim that the implementation branch's repository-wide gate or CI has run. |
| + |
| +The Linux test binary passed **60 table cases**, including the six direct comparisons with `CalcRestorePlan`. Replacing overlap with exact adjacency made both the model overlap port and the library-comparison overlap case fail, as intended. The unmodified code remains in this plan. The final source also passed package builds for **Darwin/arm64**, **Windows/amd64** and **Linux/arm64**. All three embedded Go blocks were checked byte-for-byte against the formatted source used for those checks. |
| + |
| +Paseo could not connect to its daemon, so the requested Opus review was unavailable. The tables above record the source-based self-review; an Opus review remains part of Task 4 before an implementation is landed. |