amadan
plan: claim Plan 7b rev 2, SelectPrepared for prepared restores
Answers the review's P2: a prepared generation is restored through its own explicitly authorized entry point and never falls back to a baseline.
Co-Authored-By: Codex <noreply@openai.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 file changed,
+275
−55
docs/superpowers/plans/2026-10-05-host-lifecycle-claim-7b-genselect.md+275 −55
CI passed — run details
diff --git a/docs/superpowers/plans/2026-10-05-host-lifecycle-claim-7b-genselect.md b/docs/superpowers/plans/2026-10-05-host-lifecycle-claim-7b-genselect.md| index 25ee253fe..c83bc0299 100644 |
| --- a/docs/superpowers/plans/2026-10-05-host-lifecycle-claim-7b-genselect.md |
| +++ b/docs/superpowers/plans/2026-10-05-host-lifecycle-claim-7b-genselect.md |
| @@ -1,10 +1,12 @@ |
| # Host Lifecycle Claim, Plan 7b: the shared generation selector |
| +Rev 2 answers the independent review's P2 (prepared restores). |
| + |
| > **For agentic workers:** Implement this plan task by task. Steps use checkbox (`- [ ]`) syntax. Read the sources below before changing code; the commands are for the implementation session. |
| **Goal:** provide one read-only `internal/genselect` library for the moves restore path and Plan 7's pull door. Given an authorized holder and its own region's versioned bucket, return the selected generation, the ordered exact object versions, the root used and the marker version read. |
| -**Architecture:** `Select` takes `hostclaim.Holder`, not a second claim schema. One complete `ListVersions` supplies the marker identity, generation occupancy, current files and historical pinned-version metadata. `GetVersion` reads the marker at that identity. `OpenVersion` reads only the chosen level-9 root's header. The consumer verifies complete bytes and the resulting database. The narrow `Reader` interface exposes no mutation. No production caller changes in this branch. |
| +**Architecture:** `Select` resolves the ordinary baseline from `hostclaim.Holder`, the marker and the unfinished admissions run. `SelectPrepared` restores exactly the generation named by an explicitly authorized `PreparedPin`, without reading the marker or admissions. Both use one complete `ListVersions` call and share exact-version root selection and continuation. `GetVersion` reads the marker only in `Select`; `OpenVersion` reads the chosen level-9 root's header. The consumer verifies complete bytes and the resulting database. The narrow `Reader` interface exposes no mutation. No production caller changes in this branch. |
| **Tech Stack:** Go 1.26; standard library; `internal/hostclaim`; `lease.VersionedStore`; `leasetest.VersionedBucket`; Litestream **v0.5.14** and `superfly/ltx` **v0.5.1**. No new dependencies, IAM changes, deployment changes or live tests. |
| @@ -12,7 +14,7 @@ |
| - Moves rev **7.13**, `/home/paulca/amadan.net/carlos/platform-moves-design/docs/superpowers/specs/2026-10-02-move-any-app-design.md`: §5.1 steps 1–9; §6.4's `generation.json` row and start check (INV-4). Model worktree HEAD at authoring: `8cdc8bbb`. |
| - Claim rev **6.14**, `docs/superpowers/specs/2026-10-02-host-lifecycle-claim-design.md`: C3a, C6, §6.2a and §6.4a's operative “Admissions, coverage and retirement”, item 3. Historical readings do not override that subsection. |
| -- Model paths below are relative to `/home/paulca/amadan.net/carlos/platform-moves-design/internal/sim/hostlife/`: `protocol_test.go:173` (`restoreCalc`), `:187` (`restoreWith`), `:1154` (`restoreAsNext`), `:1188` (`unfinishedChain`); `ltx_test.go:667` (`plan`); `moves_test.go:311` (the seal's baseline selection). Witness ports cite their functions beside their table entries. |
| +- Model paths below are relative to `/home/paulca/amadan.net/carlos/platform-moves-design/internal/sim/hostlife/`: `protocol_test.go:173` (`restoreCalc`), `:183` (`restorePinned`), `:187` (`restoreWith`), `:658` (`resumeOpen` prepared branch), `:1154` (`restoreAsNext`), `:1188` (`unfinishedChain`); `ltx_test.go:667` (`plan`); `moves_test.go:311` (the seal's baseline selection). Witness ports cite their functions beside their table entries. |
| - `origin/main` at `04cd2d78`: `internal/lease/versioned.go` and `internal/lease/leasetest/bucket.go`. `feat/hostclaim-library` at `47426cc1`: `internal/hostclaim/claim.go` (`Holder`, `AdmissionRecord`) and `generation.go` (`RootGeneration`). Read against those real interfaces, not this planning checkout's older source. |
| - Module cache: `github.com/benbjohnson/litestream@v0.5.14/replica.go:1441` (`CalcRestorePlan`) and `:1626` (`restoreCandidateBetter`); `github.com/superfly/ltx@v0.5.1/ltx.go:623` (`IsContiguous`). Task 3 compares against the actual planner. |
| @@ -20,18 +22,18 @@ |
| ## Global Constraints |
| -- **Authority and region:** the caller supplies the holder from its fresh authorized claim read and binds `Bucket` to `Region`. The selector does not grant admission or derive authority from region equality. It checks the location is exactly `instances/<host>` or `instances/<host>/<incarnation>` (accepting a trailing slash). It never constructs another incarnation or searches another bucket. A needed predecessor in a different region returns `ErrRegion`. |
| +- **Authority and region:** the caller supplies the holder from its fresh authorized claim read and binds `Bucket` to `Region`. The selector does not grant admission or derive authority from region equality. It checks the location is exactly `instances/<host>` or `instances/<host>/<incarnation>` (accepting a trailing slash). It never constructs another incarnation or searches another bucket. In `Select`, a needed predecessor in a different region returns `ErrRegion`. For `SelectPrepared`, the caller explicitly authorizes the completed-journal generation and pin for this holder, operation and destination region; the library does not infer that authority from admissions or a marker. |
| - **Read-only:** only `ListVersions`, `GetVersion` and `OpenVersion`. No marker publication, CAS, upload, anchor creation, certificate, parking, deletion, filesystem mutation or replication open. Input claims and storage remain unchanged. |
| -- **Origin:** genuinely absent `<location>/generation.json` means `@root`, with files directly in `<location>/<level>/`. The holder's origin is `Generation == ""`; `hostclaim.RootGeneration == "@root"` is accepted in predecessor records and returned as output, never as a holder generation. A marker read error or a current marker tombstone is not absence. |
| -- **Unfinished run:** walk admissions newest to oldest, requiring the recorded generation and region at every step. Warm records preserve the same predecessor; cold links must be uncovered. Walk all the way to the marker before using any candidate, even if the holder generation has data. Cycles, broken links and other disagreements refuse. Choose the newest generation with any listed key, including metadata or history under a tombstone; otherwise choose the marker's generation. This follows the model's `hasFiles`/`bucket.keys`: history does not prove emptiness. Metadata-only or tombstoned-only selected generations refuse instead of falling back to an older baseline. |
| +- **Origin (`Select`):** genuinely absent `<location>/generation.json` means `@root`, with files directly in `<location>/<level>/`. The holder's origin is `Generation == ""`; `hostclaim.RootGeneration == "@root"` is accepted in predecessor records and returned as output, never as a holder generation. A marker read error or a current marker tombstone is not absence. |
| +- **Unfinished run (`Select`):** walk admissions newest to oldest, requiring the recorded generation and region at every step. Warm records preserve the same predecessor; cold links must be uncovered. Walk all the way to the marker before using any candidate, even if the holder generation has data. Cycles, broken links and other disagreements refuse. Choose the newest generation with any listed key, including metadata or history under a tombstone; otherwise choose the marker's generation. This follows the model's `hasFiles`/`bucket.keys`: history does not prove emptiness. Metadata-only or tombstoned-only selected generations refuse instead of falling back to an older baseline. |
| - **Complete listing:** `VersionedStore.ListVersions` drains every page or returns an error. Discard any partial slice accompanying an error. Missing/duplicate `IsLatest` entries or a key outside the requested location also refuse. This interface cannot detect a provider that silently truncates a successful result: Plan 2's pagination contract and its fake/server tests are part of the dependency. `PageSize = 1` and `FailListAfterPages = 1` exercise this boundary here. |
| - **Layout:** generation `g` uses `<location>/g-<g>/`; origin uses `<location>/`. Only exact direct `<%04x level>/<%016x min>-<%016x max>.ltx` coordinates with levels **0–9**, `min >= 1` and `max >= min` enter the plan. Deeper child locations and other generations never enter it. Malformed LTX names in a recognized level refuse. Metadata is occupancy evidence, never a restore file. |
| -- **Marker:** the wire fields are `op`, `incarnation`, `host`, `box`, `installed_at`, `manifest_sha256`, `snapshot_key`, `snapshot_version`, `snapshot_txid`, `snapshot_crc64`. TXID and CRC fields use JSON unsigned numbers; `installed_at` uses Go's RFC3339 `time.Time` encoding. Preserve the marker and its immutable version in the result. Require host and incarnation agreement. Permit additive unknown fields, refuse malformed/trailing JSON and a missing/null `snapshot_txid`. A fresh published generation has `snapshot_txid = 0` and empty snapshot identity; its first full `[1,1]` upload is ordinary level-0 data. No zero-TXID LTX file is valid. |
| +- **Marker (`Select` only):** the wire fields are `op`, `incarnation`, `host`, `box`, `installed_at`, `manifest_sha256`, `snapshot_key`, `snapshot_version`, `snapshot_txid`, `snapshot_crc64`. TXID and CRC fields use JSON unsigned numbers; `installed_at` uses Go's RFC3339 `time.Time` encoding. Preserve the marker and its immutable version in the result. Require host and incarnation agreement. Permit additive unknown fields, refuse malformed/trailing JSON and a missing/null `snapshot_txid`. A fresh published generation has `snapshot_txid = 0` and empty snapshot identity; its first full `[1,1]` upload is ordinary level-0 data. No zero-TXID LTX file is valid. |
| - **Root:** use the current level-9 snapshot with greatest `MaxTXID >= snapshot_txid`; its range starts at **1**. “Newest” means highest TXID, not modification time or version ordering. A late duplicate at the pinned key is never substituted for `snapshot_version`. If no different current successor qualifies, find the exact pinned key/version in the version listing and open that version. Missing pins refuse on this fallback path; a valid current successor does not require an expired old pin to remain readable (§5.1 steps 4–5, 8). The root header must match the selected coordinates. A corrupt successor refuses; it never triggers fallback to an older snapshot. |
| -- **Prepared roots:** moves can supply `Request.Prepared` from an authorized completed build journal while its marker is still lagging (`restorePinned`/`restoreAsNext`). The pin applies only to the selected generation and must agree with an existing marker pin. It is not inferred from `Coverage` or from `build-intent.json`. Ordinary fresh unfinished generations have no pin and start at TXID **1**. The door leaves `Prepared` nil; if it cannot reconstruct a prepared generation from current files it refuses. Integrating moves' journal and pre-admission sealed-transfer adapter is consumer work, not a second planner. |
| +- **Prepared roots (R9):** `SelectPrepared(ctx, reader, req, PreparedPin{Generation, SnapshotKey, SnapshotVersion, SnapshotTXID, SnapshotCRC64})` is a separate, explicitly authorized restore entry point (moves §5.1 / §7.14; model `protocol_test.go:658`). It lists only that generation's prefix, selects that generation even when the target marker is absent and source admissions are covered or trimmed, and never reads the marker or walks admissions. The root is the exact pin or a verified current successor in that same generation under §5.1 step 4. Missing roots return `ErrPinnedMissing`; mismatched coordinates or headers return `ErrPinnedMismatch`; an invalid successor returns `ErrRoot`. A zero-TXID prepared pin refuses with `ErrPinnedMismatch`; an empty prepared generation with a valid positive pin returns `ErrPinnedMissing`. None may select an older baseline. `Request` has no `Prepared` field. The door uses `Select`, which retains all ordinary disagreement refusals and permits fresh unfinished generations to start at TXID **1**. Journal authorization and pre-admission sealed-transfer integration remain consumer work. |
| - **Continuation:** after the root, consider current levels **0–8** only. Require `MinTXID <= current+1` and `MaxTXID > current`; rank by highest max, lowest min, then highest level. Repeat until none extends, then refuse if any current continuation remains beyond the final TXID. Guard the `uint64` maximum against overflow. Noncurrent continuations never repair gaps. Equal coordinate/level ties are the same canonical key and have only one current version. |
| - **Exact versions:** empty, whitespace-only and literal `null` version IDs refuse for the marker, pin and current LTX candidates. Selected delete-marker versions refuse distinctly. Retention tombstones remove files from the current candidate set; they do not globally invalidate a healthy compacted replica, or prohibit reading a different, explicitly pinned historical version at that key. No bare-key GET and no “try current” fallback. |
| -- **Result and verification:** `Files` is ordered, with exact `{key, version_id, level, min_txid, max_txid, size}`. `Root` is its first file (a level-9 snapshot or the first contiguous file starting at 1). `MaxTXID` is the restored position; `MaxLevel0TXID` is the newest current level-0 position for the consumer's anchor decision. This is a selection result, not proof that bytes are valid. The root header read is **100 bytes**; it does not run CRC verification. Consumer requirements are in Task 4. |
| +- **Result and verification:** `Files` is ordered, with exact `{key, version_id, level, min_txid, max_txid, size}`. `Root` is its first file (a level-9 snapshot or the first contiguous file starting at 1). `MaxTXID` is the restored position; `MaxLevel0TXID` is the newest current level-0 position for the consumer's anchor decision. `SelectPrepared` returns the supplied pin in `Pin`, with `Marker == nil` and `MarkerVersion == ""`. This is a selection result, not proof that bytes are valid. The root header read is **100 bytes**; it does not run CRC verification. Consumer requirements are in Task 4. |
| - **Errors:** every failure returns a nil plan. `ErrListing`, `ErrDisagreement`, `ErrIncarnation`, `ErrMarker`, `ErrVersionID`, `ErrDeleteMarker`, `ErrPinnedMissing`, `ErrPinnedMismatch`, `ErrGap`, `ErrEmpty`, `ErrRegion`, `ErrInput`, and `ErrRoot` are distinguishable with `errors.Is`. Wrapped read failures retain their underlying error. Empty origin data also returns `ErrEmpty`; only a separately authorized bootstrap path may start an empty database. |
| - **Portability and cache:** no OS-specific code. Use the default Go build cache, never a cache under `/tmp` or a new cache per task. In this sandbox the default is read-only; the existing shared on-disk `/home/paulca/.cache/tmp/gocache` is the fallback. CI includes Darwin, Windows and Linux/arm64 compilation. |
| @@ -41,7 +43,8 @@ |
| 2. A deleted or replaced pinned snapshot must never be read by current key. Conversely, normal retention must permit a verified successor and tombstoned old continuations (Tasks 1–3). |
| 3. A compacted `[10,13]` after snapshot 11 is valid; `[13,13]` alone is a gap. The anchor `[11,11]` is not a continuation. Test this against Litestream, not only against our own comparator (Tasks 1–3). |
| 4. The baseline's emptiness requires every page. Any object history or metadata in a newer generation prevents silent fallback to older data (Tasks 1–2). |
| -5. The selector cannot certify CRCs, SQLite integrity, source permissions, admission freshness or spawn safety. The consumer must satisfy the integration checklist before the library enables a door or restore (Task 4). |
| +5. R9 and the independent review's P2: prepared restores must work without a target marker or usable source admissions, preserve exact pin identity, and refuse missing/mismatched roots and empty prepared generations. `Select` must still refuse those unexplained disagreements. Tests assert that `SelectPrepared` never reads the marker or lists another generation (Tasks 1–2). |
| +6. The selector cannot certify CRCs, SQLite integrity, source permissions, admission freshness or spawn safety. The consumer must satisfy the integration checklist before the library enables a door or restore (Task 4). |
| ## File Structure |
| @@ -429,14 +432,6 @@ func TestSelectionDetails(t *testing.T) { |
| f.req.Holder.Admissions = append(f.req.Holder.Admissions, hostclaim.AdmissionRecord{Generation: "g2", Predecessor: "g2", Region: "ie", PredecessorRegion: "ie", Mode: "warm"}) |
| return []File{f.pin} |
| }}, |
| - {"prepared_unpublished_root", func(f *fixture) []File { |
| - f.chain("g0") |
| - r := f.file("g2", 9, 1, 30) |
| - f.file("g2", 0, 30, 30) |
| - f.req.Prepared = &Prepared{Generation: "g2", Pin: Pin{SnapshotKey: r.Key, SnapshotVersion: r.VersionID, SnapshotTXID: 30}} |
| - f.b.Put("data", r.Key, header(f.t, 1, 30)) |
| - return []File{r} |
| - }}, |
| {"root_advance_past_missing_old_pin", func(f *fixture) []File { |
| if err := f.b.DeleteVersion(f.t.Context(), "data", f.pin.Key, f.pin.VersionID); err != nil { |
| f.t.Fatal(err) |
| @@ -456,6 +451,187 @@ func TestSelectionDetails(t *testing.T) { |
| } |
| } |
| +// Port of resumeOpen's prepared branch (protocol_test.go:658) and |
| +// restorePinned/restoreWith (:183/:187), moves §5.1 / §7.14. |
| +func TestPreparedRestore(t *testing.T) { |
| + tests := []struct { |
| + name string |
| + setup func(*fixture, *PreparedPin, File) []File |
| + want error |
| + ordinaryRefuses bool |
| + }{ |
| + {"absent_target_marker_covered_source", func(f *fixture, p *PreparedPin, root File) []File { |
| + f.origin() |
| + f.req.Holder.Generation = "g2" |
| + return []File{root, f.file("g2", 1, 29, 33)} |
| + }, nil, true}, |
| + {"absent_target_marker_trimmed_source", func(f *fixture, p *PreparedPin, root File) []File { |
| + f.origin() |
| + f.req.Holder.Generation = "g2" |
| + f.req.Holder.Admissions = f.req.Holder.Admissions[1:] |
| + return []File{root} |
| + }, nil, true}, |
| + {"unpublished_historical_pin", func(f *fixture, p *PreparedPin, root File) []File { |
| + f.b.Put("data", root.Key, header(f.t, 1, 30)) |
| + return []File{root} |
| + }, nil, false}, |
| + {"marker_pin_disagreement_ignored", func(f *fixture, p *PreparedPin, root File) []File { |
| + f.marker.Op = "g2" // Its old g0 pin cannot override the authorized pin. |
| + f.publish() |
| + return []File{root} |
| + }, nil, false}, |
| + {"malformed_marker_ignored", func(f *fixture, p *PreparedPin, root File) []File { |
| + f.b.Put("data", f.req.Holder.Location+"/generation.json", []byte("{")) |
| + return []File{root} |
| + }, nil, false}, |
| + {"marker_tombstone_ignored", func(f *fixture, p *PreparedPin, root File) []File { |
| + f.b.BareDelete("data", f.req.Holder.Location+"/generation.json") |
| + return []File{root} |
| + }, nil, false}, |
| + {"missing_prepared_root_no_current_key_fallback", func(f *fixture, p *PreparedPin, root File) []File { |
| + f.b.Put("data", root.Key, header(f.t, 1, 30)) |
| + deletePreparedRoot(f, root) |
| + return nil |
| + }, ErrPinnedMissing, false}, |
| + {"empty_prepared_generation_never_older_baseline", func(f *fixture, p *PreparedPin, root File) []File { |
| + f.chain("g0") |
| + deletePreparedRoot(f, root) |
| + versions, err := f.b.ListVersions(f.t.Context(), "data", prefix(f.req.Holder.Location+"/", "g2")) |
| + if err != nil { |
| + f.t.Fatal(err) |
| + } |
| + for _, v := range versions { |
| + if err := f.b.DeleteVersion(f.t.Context(), "data", v.Key, v.VersionID); err != nil { |
| + f.t.Fatal(err) |
| + } |
| + } |
| + baseline, err := Select(f.t.Context(), f.b, f.req) |
| + if err != nil || baseline.Generation != "g0" { |
| + f.t.Fatalf("ordinary baseline: %+v / %v", baseline, err) |
| + } |
| + return nil |
| + }, ErrPinnedMissing, false}, |
| + {"zero_txid_prepared_pin_refuses", func(f *fixture, p *PreparedPin, root File) []File { |
| + *p = PreparedPin{Generation: "empty"} |
| + return nil |
| + }, ErrPinnedMismatch, false}, |
| + {"prepared_pin_key_mismatch", func(f *fixture, p *PreparedPin, root File) []File { |
| + p.SnapshotTXID++ |
| + return nil |
| + }, ErrPinnedMismatch, false}, |
| + {"prepared_pin_other_generation", func(f *fixture, p *PreparedPin, root File) []File { |
| + p.SnapshotKey = f.pin.Key |
| + p.SnapshotVersion = f.pin.VersionID |
| + p.SnapshotTXID = f.pin.MaxTXID |
| + return nil |
| + }, ErrPinnedMismatch, false}, |
| + {"prepared_pin_header_mismatch", func(f *fixture, p *PreparedPin, root File) []File { |
| + p.SnapshotVersion = f.b.Put("data", root.Key, header(f.t, 1, 29)) |
| + return nil |
| + }, ErrPinnedMismatch, false}, |
| + {"prepared_pin_empty_version", func(f *fixture, p *PreparedPin, root File) []File { |
| + p.SnapshotVersion = "" |
| + return nil |
| + }, ErrVersionID, false}, |
| + {"prepared_pin_deleted_version", func(f *fixture, p *PreparedPin, root File) []File { |
| + p.SnapshotVersion = f.b.BareDelete("data", root.Key) |
| + return nil |
| + }, ErrDeleteMarker, false}, |
| + {"prepared_root_advance_past_missing_pin", func(f *fixture, p *PreparedPin, root File) []File { |
| + deletePreparedRoot(f, root) |
| + f.file("other", 9, 1, 999) |
| + return []File{f.file("g2", 9, 1, 40), f.file("g2", 0, 41, 41)} |
| + }, nil, false}, |
| + {"prepared_bad_successor_no_fallback", func(f *fixture, p *PreparedPin, root File) []File { |
| + r := f.file("g2", 9, 1, 40) |
| + f.b.Put("data", r.Key, header(f.t, 1, 39)) |
| + return nil |
| + }, ErrRoot, false}, |
| + {"prepared_incomplete_listing", func(f *fixture, p *PreparedPin, root File) []File { |
| + f.b.FailListAfterPages = 1 |
| + return nil |
| + }, ErrListing, false}, |
| + } |
| + for _, tt := range tests { |
| + t.Run(tt.name, func(t *testing.T) { |
| + f := newFixture(t) |
| + f.req.Holder.Generation = "g2" |
| + f.req.Holder.Admissions = []hostclaim.AdmissionRecord{ |
| + {Generation: "source", Predecessor: "older", Region: "us", PredecessorRegion: "us", Mode: "cold", Covered: true}, |
| + {Generation: "g2", Predecessor: "source", Region: "ie", PredecessorRegion: "us", Mode: "cold"}, |
| + } |
| + root := f.file("g2", 9, 1, 30) |
| + f.file("g2", 0, 30, 30) |
| + prepared := PreparedPin{Generation: "g2", SnapshotKey: root.Key, SnapshotVersion: root.VersionID, SnapshotTXID: 30, SnapshotCRC64: 123} |
| + want := tt.setup(f, &prepared, root) |
| + if tt.ordinaryRefuses { |
| + got, err := Select(t.Context(), f.store, f.req) |
| + if got != nil || !errors.Is(err, ErrDisagreement) { |
| + t.Fatalf("ordinary selection: %+v / %v", got, err) |
| + } |
| + } |
| + before := f.b.Snapshot() |
| + holderBefore, _ := json.Marshal(f.req.Holder) |
| + f.b.FailOn = func(op, key string) error { |
| + if op != "list" && op != "get" { |
| + t.Fatalf("mutation %s %s", op, key) |
| + } |
| + return nil |
| + } |
| + reader := preparedReader{Reader: f.store, t: t, base: prefix(f.req.Holder.Location+"/", prepared.Generation)} |
| + got, err := SelectPrepared(t.Context(), reader, f.req, prepared) |
| + holderAfter, _ := json.Marshal(f.req.Holder) |
| + if f.b.Snapshot() != before || string(holderBefore) != string(holderAfter) { |
| + t.Fatal("selection mutated inputs or storage") |
| + } |
| + if !errors.Is(err, tt.want) { |
| + t.Fatalf("got %+v / %v, want %v", got, err, tt.want) |
| + } |
| + if tt.want != nil { |
| + if got != nil { |
| + t.Fatal("partial plan") |
| + } |
| + return |
| + } |
| + wantPin := Pin{SnapshotKey: prepared.SnapshotKey, SnapshotVersion: prepared.SnapshotVersion, SnapshotTXID: prepared.SnapshotTXID, SnapshotCRC64: prepared.SnapshotCRC64} |
| + if got == nil || got.Generation != prepared.Generation || !reflect.DeepEqual(got.Files, want) || |
| + got.Root == nil || *got.Root != want[0] || got.MaxTXID != want[len(want)-1].MaxTXID || |
| + got.Marker != nil || got.MarkerVersion != "" || got.Pin == nil || *got.Pin != wantPin { |
| + t.Fatalf("got %+v, want prepared %s %+v", got, prepared.Generation, want) |
| + } |
| + }) |
| + } |
| +} |
| + |
| +func deletePreparedRoot(f *fixture, root File) { |
| + f.t.Helper() |
| + if err := f.b.DeleteVersion(f.t.Context(), "data", root.Key, root.VersionID); err != nil { |
| + f.t.Fatal(err) |
| + } |
| +} |
| + |
| +type preparedReader struct { |
| + Reader |
| + t *testing.T |
| + base string |
| +} |
| + |
| +func (s preparedReader) ListVersions(ctx context.Context, bucket, base string) ([]lease.ObjectVersion, error) { |
| + if base != s.base { |
| + s.t.Fatalf("prepared listing escaped generation: %s", base) |
| + } |
| + return s.Reader.ListVersions(ctx, bucket, base) |
| +} |
| +func (s preparedReader) GetVersion(context.Context, string, string, string) ([]byte, error) { |
| + s.t.Fatal("prepared restore must not read a marker") |
| + return nil, ErrMarker |
| +} |
| +func (s preparedReader) OpenVersion(ctx context.Context, bucket, key, id string) (io.ReadCloser, error) { |
| + if !strings.HasPrefix(key, s.base) { |
| + s.t.Fatalf("prepared open escaped generation: %s", key) |
| + } |
| + return s.Reader.OpenVersion(ctx, bucket, key, id) |
| +} |
| ``` |
| - [ ] **Step 3: Run the tests and observe failure before adding the implementation.** |
| @@ -464,13 +640,13 @@ func TestSelectionDetails(t *testing.T) { |
| go test ./internal/genselect -count=1 |
| ``` |
| -Expected: compilation fails because `Reader`, `Request`, `Select` and the other new API symbols do not exist. Once implemented, each case must check exact version identities and refusal must return no partial plan. |
| +Expected: compilation fails because `Reader`, `Request`, `Select`, `SelectPrepared`, `PreparedPin` and the other new API symbols do not exist. Once implemented, each case must check exact version identities and refusal must return no partial plan. |
| ### Task 2: Implement the read-only selector |
| **Files:** create `internal/genselect/select.go`. |
| -**Interfaces:** consumes the read-only subset of `lease.VersionedStore` and the actual `hostclaim.Holder`; produces `Plan`, `File`, `Marker`, `Pin`, `Prepared`, and typed errors. `Prepared` is a caller-supplied pin for an authorized completed build, not permission to construct or publish a generation. |
| +**Interfaces:** consumes the read-only subset of `lease.VersionedStore` and the actual `hostclaim.Holder`; exposes `Select` and `SelectPrepared`, with `Plan`, `File`, `Marker`, `Pin`, `PreparedPin`, and typed errors. `PreparedPin` identifies an explicitly authorized completed build. It grants no permission to construct, publish or start a generation. `Request` has no prepared-restore option. |
| - [ ] **Step 1: Write the complete implementation.** |
| @@ -534,16 +710,20 @@ type Marker struct { |
| Pin |
| } |
| -// Prepared is supplied only from an authorized, completed build journal. |
| -type Prepared struct { |
| - Generation string |
| - Pin |
| +// PreparedPin is supplied only from an authorized, completed build journal. |
| +// The caller binds this exact generation and pin to the authorized holder, |
| +// operation and destination region before calling SelectPrepared. |
| +type PreparedPin struct { |
| + Generation string |
| + SnapshotKey string |
| + SnapshotVersion string |
| + SnapshotTXID uint64 |
| + SnapshotCRC64 uint64 |
| } |
| type Request struct { |
| Bucket, Region, Host string |
| Holder hostclaim.Holder |
| - Prepared *Prepared |
| } |
| type File struct { |
| @@ -624,40 +804,57 @@ func exactFile(v lease.ObjectVersion, base string) (File, error) { |
| return f, nil |
| } |
| -// Select returns identities only. The consumer must verify every complete file. |
| -func Select(ctx context.Context, s Reader, req Request) (*Plan, error) { |
| +func requestLocation(s Reader, req Request) (string, error) { |
| h := req.Holder |
| location := strings.TrimSuffix(h.Location, "/") |
| if s == nil || req.Bucket == "" || req.Region == "" || !component(req.Host) || !component(h.Incarnation) || |
| (location != "instances/"+req.Host && location != "instances/"+req.Host+"/"+h.Incarnation) || |
| (h.Generation != "" && !generation(h.Generation)) { |
| - return nil, ErrInput |
| + return "", ErrInput |
| } |
| - location += "/" |
| + return location + "/", nil |
| +} |
| + |
| +func listVersions(ctx context.Context, s Reader, bucket, base string) ([]lease.ObjectVersion, map[string]lease.ObjectVersion, error) { |
| // VersionedStore drains all pages before returning. Never use partial data. |
| - versions, err := s.ListVersions(ctx, req.Bucket, location) |
| + versions, err := s.ListVersions(ctx, bucket, base) |
| if err != nil { |
| - return nil, fmt.Errorf("%w: %w", ErrListing, err) |
| + return nil, nil, fmt.Errorf("%w: %w", ErrListing, err) |
| } |
| latest := make(map[string]lease.ObjectVersion) |
| keys := make(map[string]bool) |
| for _, v := range versions { |
| - if !strings.HasPrefix(v.Key, location) { |
| - return nil, ErrListing |
| + if !strings.HasPrefix(v.Key, base) { |
| + return nil, nil, ErrListing |
| } |
| keys[v.Key] = true |
| if v.IsLatest { |
| if _, exists := latest[v.Key]; exists { |
| - return nil, ErrListing |
| + return nil, nil, ErrListing |
| } |
| latest[v.Key] = v |
| } |
| } |
| for k := range keys { |
| if _, ok := latest[k]; !ok { |
| - return nil, ErrListing |
| + return nil, nil, ErrListing |
| } |
| } |
| + return versions, latest, nil |
| +} |
| + |
| +// Select resolves the ordinary baseline using the marker and unfinished run. |
| +// It returns identities only. The consumer must verify every complete file. |
| +func Select(ctx context.Context, s Reader, req Request) (*Plan, error) { |
| + location, err := requestLocation(s, req) |
| + if err != nil { |
| + return nil, err |
| + } |
| + versions, latest, err := listVersions(ctx, s, req.Bucket, location) |
| + if err != nil { |
| + return nil, err |
| + } |
| + h := req.Holder |
| p := &Plan{Files: []File{}} |
| markerGen := hostclaim.RootGeneration |
| if mv, ok := latest[location+"generation.json"]; ok { |
| @@ -716,13 +913,37 @@ func Select(ctx context.Context, s Reader, req Request) (*Plan, error) { |
| cp := p.Marker.Pin |
| pin = &cp |
| } |
| - if req.Prepared != nil && req.Prepared.Generation == p.Generation { |
| - if pin != nil && *pin != req.Prepared.Pin { |
| - return nil, ErrPinnedMismatch |
| - } |
| - cp := req.Prepared.Pin |
| - pin = &cp |
| + return selectFiles(ctx, s, req.Bucket, base, versions, latest, p, pin) |
| +} |
| + |
| +// SelectPrepared restores only the explicitly authorized prepared generation. |
| +// It never reads the marker or walks admissions, and never chooses a baseline. |
| +func SelectPrepared(ctx context.Context, s Reader, req Request, prepared PreparedPin) (*Plan, error) { |
| + location, err := requestLocation(s, req) |
| + if err != nil { |
| + return nil, err |
| + } |
| + if !generation(prepared.Generation) { |
| + return nil, ErrInput |
| + } |
| + // Empty prepared roots require a separate bootstrap protocol, not fallback. |
| + if prepared.SnapshotTXID == 0 { |
| + return nil, ErrPinnedMismatch |
| + } |
| + base := prefix(location, prepared.Generation) |
| + versions, latest, err := listVersions(ctx, s, req.Bucket, base) |
| + if err != nil { |
| + return nil, err |
| } |
| + pin := &Pin{SnapshotKey: prepared.SnapshotKey, SnapshotVersion: prepared.SnapshotVersion, |
| + SnapshotTXID: prepared.SnapshotTXID, SnapshotCRC64: prepared.SnapshotCRC64} |
| + p := &Plan{Generation: prepared.Generation, Files: []File{}} |
| + return selectFiles(ctx, s, req.Bucket, base, versions, latest, p, pin) |
| +} |
| + |
| +// Both entry points share exact-version root selection and continuation. |
| +func selectFiles(ctx context.Context, s Reader, bucket, base string, versions []lease.ObjectVersion, |
| + latest map[string]lease.ObjectVersion, p *Plan, pin *Pin) (*Plan, error) { |
| if pin != nil { |
| if pin.SnapshotTXID == 0 { |
| if pin.SnapshotKey != "" || pin.SnapshotVersion != "" || pin.SnapshotCRC64 != 0 { |
| @@ -754,7 +975,7 @@ func Select(ctx context.Context, s Reader, req Request) (*Plan, error) { |
| if !ok || v.DeleteMarker { |
| continue |
| } // Retention tombstones are not current files. |
| - f, err = exactFile(v, base) |
| + f, err := exactFile(v, base) |
| if err != nil { |
| return nil, err |
| } |
| @@ -800,7 +1021,7 @@ func Select(ctx context.Context, s Reader, req Request) (*Plan, error) { |
| p.Root, pinned = &f, true |
| } |
| if p.Root != nil { |
| - if err := checkRoot(ctx, s, req.Bucket, *p.Root, pinned); err != nil { |
| + if err := checkRoot(ctx, s, bucket, *p.Root, pinned); err != nil { |
| return nil, err |
| } |
| p.Files = append(p.Files, *p.Root) |
| @@ -928,7 +1149,6 @@ func checkRoot(ctx context.Context, s Reader, bucket string, f File, pinned bool |
| } |
| return nil |
| } |
| - |
| ``` |
| - [ ] **Step 2: Format and pass the witness and refusal tests.** |
| @@ -938,7 +1158,7 @@ gofmt -w internal/genselect/select.go internal/genselect/select_test.go |
| go test ./internal/genselect -count=1 -v |
| ``` |
| -Expected: all witness, refusal and selection-detail rows pass. The selected generation is determined by admissions, never by the largest TXID across generations. A successful selection does not write a marker or anchor. |
| +Expected: all witness, refusal, selection-detail and prepared-restore rows pass. `Select` resolves the ordinary baseline from the marker and admissions. `SelectPrepared` selects only its authorized generation and pin; it never resolves a baseline. Neither entry point compares TXIDs across generations. A successful selection does not write a marker or anchor. |
| ### Task 3: Check the pinned library's planner |
| @@ -1013,7 +1233,6 @@ func TestCalcRestorePlanCompatibility(t *testing.T) { |
| }) |
| } |
| } |
| - |
| ``` |
| - [ ] **Step 2: Run the compatibility cases and race detector.** |
| @@ -1026,7 +1245,7 @@ go test -race ./internal/genselect -count=1 |
| Expected: equal ordered level/range selections for origin, overlap, root advancement, ties and anchors; both planners refuse the genuine gap. Version-only requirements, absent-marker handling and unfinished-run authorization remain the responsibility of Task 1's tests, because Litestream has no claim or object-version interface. |
| -- [ ] **Step 3: Prove the overlap witness detects the old rule.** Temporarily replace the `ltx.IsContiguous` predicate in `Select` with exact adjacency (`MinTXID == current+1`, retaining `MaxTXID > current`), then run: |
| +- [ ] **Step 3: Prove the overlap witness detects the old rule.** Temporarily replace the `ltx.IsContiguous` predicate in shared `selectFiles` with exact adjacency (`MinTXID == current+1`, retaining `MaxTXID > current`), then run: |
| ```bash |
| go test ./internal/genselect -run 'TestModelWitnesses/wRestoreOverlap|TestCalcRestorePlanCompatibility/compacted_overlap' -count=1 |
| @@ -1055,11 +1274,11 @@ Expected: all commands pass; `gofmt -l .` and `git diff --check` are empty. If t |
| - [ ] **Step 2: Review the consumer obligations before declaring a caller integrated.** These are integration acceptance criteria, not changes authorized by this library branch: |
| -1. Obtain and authorize a fresh claim and record. For the door, keep Plan 7's O1/O2 comparison of the complete holder and operation and all existing membership, class, residency and region gates. Pass only the authorized local bucket. A `Plan` grants no authority to spawn or read another region. |
| +1. Obtain and authorize a fresh claim and record. For the door, keep Plan 7's O1/O2 comparison of the complete holder and operation and all existing membership, class, residency and region gates. Pass only the authorized local bucket. Use `Select` for the door and ordinary baseline reads. Use `SelectPrepared` only after authorizing the exact completed-journal generation and pin against the fresh holder, operation and destination region, including the prepared-generation one-use rules. Never retry a prepared refusal through `Select`; keep the same prepared generation or refuse the restore. A `Plan` grants no authority to spawn or read another region. |
| 2. Open every file by its selected key **and version ID**, check header range against `File`, consume the whole stream, enforce the listed size and run `ltx.Decoder.Verify` through the trailer. Verify the pinned root's CRC-64 against `Pin.SnapshotCRC64` when that exact pin is used; a successor must pass its own `Verify`, not compare its checksum to the old pin's. If a selected version has disappeared or is a delete marker, fail the restore. Do not substitute a current version or older root. |
| 3. Apply in the returned order into a disposable database. Run SQLite `PRAGMA integrity_check` and require `ok`. At an install, require the exact sealed final TXID **N**; for warm continuation, require the certificate's `max_txid`. A failed check produces no published database and no app start. Map errors to member-safe refusals and the operation's alert/page policy. |
| 4. Do not compare pre/post database checksums: the pinned level-0 writer, snapshot writer and compactor use `HeaderFlagNoChecksum`, and their pre/post checksums are zero (§5.1 step 5). Per-file CRC and SQLite integrity are both still required. |
| -5. For replication initialization, compare the verified restored position with the newest current level-0 position, rechecking that position at activation under the existing guard. An install's `[N,N]` anchor must be fully uploaded and acknowledged before replication opens. A restore above current level 0 needs an anchor; a normal replica whose newest level-0 file already reaches the restored position needs none. `Select` only returns `MaxLevel0TXID`; it never creates an anchor, writes a TXID sidecar, initializes replication or claims a PUT was acknowledged. |
| +5. For replication initialization, compare the verified restored position with the newest current level-0 position, rechecking that position at activation under the existing guard. An install's `[N,N]` anchor must be fully uploaded and acknowledged before replication opens. A restore above current level 0 needs an anchor; a normal replica whose newest level-0 file already reaches the restored position needs none. Both entry points only return `MaxLevel0TXID`; they never create an anchor, write a TXID sidecar, initialize replication or claim a PUT was acknowledged. |
| 6. Before spawn, perform INV-4's fresh marker/holder, `.files-gen`, and database-generation/location checks. A lagging marker selects a baseline; it does not authorize starting the newer generation before its publication protocol completes. All admission CAS, purge, app-start/readiness ordering, prepared-generation one-use rules, certificate/request-ledger rules, writer prefix binding and retirement remain with the lifecycle consumers. |
| - [ ] **Step 3: Obtain the required production-path review and land through amadan only after the implementation gate passes.** Follow `AGENTS.md`: describe and review the branch, squash its implementation commits to one, push to amadan, check CI on that exact tip and use `amadan branch merge -expect`. Read main's CI after landing. No branch creation, staging, committing, pushing or landing happens during this plan-writing session. |
| @@ -1068,7 +1287,7 @@ Expected: all commands pass; `gofmt -l .` and `git diff --check` are empty. If t |
| This is a library-only dependency. Plan 7's nonempty-generation refusal stays in place until a separate follow-up integrates `genselect.Select` into its listing path, signs only the exact versions returned, and tests generation changes, unfinished runs, pinned historical roots, missing versions, pagination failures and the complete O1/O2 authorization boundary over HTTP. That follow-up must adapt the selected plan to the door's level-based protocol without relisting a different snapshot on each request, and preserve the same version identities through all GETs. Plan 8 must not enable the generation-aware door before those tests pass. |
| -The moves plans import this package and supply completed-journal pins for prepared generations. They must bind sealed transfer versions as the source inventory when building from a seal, without substituting today's staged listing; their adapter and byte-verifying restore remain separate tasks. They do not copy this selector. The library does not remove the need for their admission, initialization, final-TXID and INV-4 integration tests. |
| +The moves plans import this package and call `SelectPrepared` with explicitly authorized completed-journal pins for prepared generations; ordinary baseline reads use `Select`. A prepared refusal must not trigger a retry through `Select` or any older baseline. They must bind sealed transfer versions as the source inventory when building from a seal, without substituting today's staged listing; their adapter and byte-verifying restore remain separate tasks. They do not copy this selector. The library does not remove the need for their admission, initialization, final-TXID and INV-4 integration tests. |
| ## Self-review: requirements mapped to tasks |
| @@ -1076,7 +1295,7 @@ The moves plans import this package and supply completed-journal pins for prepar |
| |---|---| |
| | Location/generation definition and exact S3 layout | Task 2 `prefix`, `coordinates`, holder-location check; Task 1 origin/legacy/child-isolation tests | |
| | 1: sealed transfer, exact versions, required final N, 16 MiB cut bar | Task 2 exact version output; Task 4 consumer checks 1–3 and rollout's sealed-inventory adapter. Sealing, transfer and cut budget are moves responsibilities | |
| -| 2: staged restore, snapshot `[1,N]`, anchor `[N,N]`, durable build intent then acknowledged artifacts then completed manifest | Task 2 `Prepared` pin and root range; Task 3 anchor comparison; Task 4 checks 2–5. No builder or manifest publication here | |
| +| 2: staged restore, snapshot `[1,N]`, anchor `[N,N]`, durable build intent then acknowledged artifacts then completed manifest | Task 2 `SelectPrepared` / `PreparedPin` and root range; Task 3 anchor comparison; Task 4 checks 2–5. No builder or manifest publication here | |
| | 3: snapshot key/version/TXID/CRC identity; late duplicate never read | Task 2 `Marker`, `Pin`, exact historical fallback; Task 1 duplicate, missing-pin, mismatch and delete-marker cases; Task 4 CRC check | |
| | 4: newest qualifying current level-9 root, current levels 0–8, overlap and all three tie breaks | Task 2 root/continuation loops; Tasks 1 and 3 overlap, gap, advancement and tie cases | |
| | 5: missing/wrong pin, Verify failure, gap, integrity_check, required install TXID; no pre/post checksum test | Task 1 distinct refusal cases; Task 2 header/pin/gap errors; Task 4 checks 2–4 own full-byte verification and SQLite | |
| @@ -1084,9 +1303,9 @@ The moves plans import this package and supply completed-journal pins for prepar |
| | 7: acknowledged anchor, remote level-0 position, no oracle writes, no TXID-sidecar shortcut | Task 2 `MaxLevel0TXID` and read-only interface; Task 3 anchor skipped as continuation; Task 4 check 5 owns initialization | |
| | 8: rotation/retention/root advancement; current versus noncurrent selection; newest level 0 | Tasks 1–3 advancement and retention cases; Task 2 current-only candidates and max-level-0 output. Protecting versions from sweep/retention is not this read-only API | |
| | 9: writer-session isolation and fresh claim admission before app start | Task 2 confines selection to one authorized location/generation; Task 4 checks 1 and 6 preserve admission/CAS and writer binding responsibilities | |
| -| 9: purge/remote baseline, warm certificate target, cold prepared/fresh generation, empty bootstrap and writes_lost | Tasks 1–2 baseline resolution and prepared pin; Task 4 checks 3 and 6 preserve purge, certificate, bootstrap and loss-declaration decisions | |
| +| 9: purge/remote baseline, warm certificate target, cold prepared/fresh generation, empty bootstrap and writes_lost | Tasks 1–2 separate `Select` baseline resolution and `SelectPrepared` pin; Task 4 checks 3 and 6 preserve purge, certificate, bootstrap and loss-declaration decisions | |
| | 9: unfinished run and absent-marker origin | Task 2 `unfinished`, `hasObject`; Task 1 `wBaselineParked`, `wOriginSwept`, complete-listing and covered-chain refusals | |
| -| 9: publication, per-region coverage, predecessor parking, retry policy, one-use close certificates, generation-bound request ledger | Task 4 check 6 and rollout: caller-owned side effects, never performed or certified by `Select` | |
| +| 9: publication, per-region coverage, predecessor parking, retry policy, one-use close certificates, generation-bound request ledger | Task 4 check 6 and rollout: caller-owned side effects, never performed or certified by either entry point | |
| | §6.4 marker table and INV-4 start check | Task 2 full marker schema plus host/incarnation check; Task 4 check 6 retains files/database marker checks | |
| | Brief item | Task and evidence | |
| @@ -1099,13 +1318,14 @@ The moves plans import this package and supply completed-journal pins for prepar |
| | 6: distinct fail-closed errors | Tasks 1–2 error table, late disappearance, bad versions, header mismatch, deleted pins and no fallback | |
| | 7: generation, ordered exact versions, root, marker version; consumer checks | Task 2 `Plan`/`File`; Task 1 exact struct assertions; Task 4 checks 2–5 | |
| | 8: named model witness ports on VersionedBucket | Task 1 table with model function names/file:line; Task 3 real planner comparison | |
| -| Claim C3a, C6, §6.2a, §6.4a item 3 | Task 2 exact namespace, complete reads and uncovered predecessor walk; Task 1 corresponding refusals | |
| +| Claim C3a, C6, §6.2a, §6.4a item 3 | Task 2 exact namespace, complete reads and uncovered predecessor walk in `Select`; separate prepared-generation restore in `SelectPrepared`. Task 1 corresponding refusals and prepared ports | |
| +| R9; independent review P2 (prepared restores) | Tasks 1–2 split `Select` / `SelectPrepared`, remove `Request.Prepared`, and share `selectFiles`. `TestPreparedRestore` ports `protocol_test.go:658`: absent target marker with covered/trimmed source admissions succeeds only through the prepared path; missing/mismatched roots and an empty prepared generation refuse; exact pin or same-generation successor only. `TestRefusals` and prepared cases retain ordinary disagreement refusals. Task 4 and rollout forbid fallback through `Select` | |
| | Branch, library scope, rollout and cross-platform gate | Tasks 1 and 4, rollout note | |
| ## Authoring verification |
| The embedded code was compiled using a Go overlay, with Plan 1's actual `hostclaim` sources and `origin/main`'s actual `lease`/`leasetest` sources. Scratch files were outside the repository under the on-disk `$TMPDIR`. The default cache was read-only, so verification used the existing shared `$TMPDIR/gocache`. Go's test runner and vet cannot `chdir` into an overlay-only package directory; the check therefore built the test binary with `go test -c -vet=off` and ran that binary from scratch. This is a compile and package-test check, not a claim that the implementation branch's repository-wide gate or CI has run. |
| -The Linux test binary passed **60 table cases**, including the six direct comparisons with `CalcRestorePlan`. Replacing overlap with exact adjacency made both the model overlap port and the library-comparison overlap case fail, as intended. The unmodified code remains in this plan. The final source also passed package builds for **Darwin/arm64**, **Windows/amd64** and **Linux/arm64**. All three embedded Go blocks were checked byte-for-byte against the formatted source used for those checks. |
| +Rev 2 was recompiled and the Linux test binary passed **76 table cases**, including **17 prepared-restore cases** and the six direct comparisons with `CalcRestorePlan`. The prepared cases cover absent target markers with covered or trimmed source admissions, ordinary disagreement refusals, empty generations, missing/mismatched pins, marker isolation and same-generation successor advancement. Replacing overlap with exact adjacency made both the model overlap port and the library-comparison overlap case fail, as intended. The unmodified code remains in this plan. The final source also passed package builds for **Darwin/arm64**, **Windows/amd64** and **Linux/arm64**. All three embedded Go blocks were checked byte-for-byte against the formatted source used for those checks. |
| -Paseo could not connect to its daemon, so the requested Opus review was unavailable. The tables above record the source-based self-review; an Opus review remains part of Task 4 before an implementation is landed. |
| +Paseo could not connect to its daemon during rev 2 verification, so an Opus review was unavailable. The tables above record the source-based self-review; an Opus review remains part of Task 4 before an implementation is landed. |