| index b9972fd0d..c3af46094 100644 |
| --- a/docs/superpowers/plans/2026-10-17-region-ha-2b-joining.md |
| +++ b/docs/superpowers/plans/2026-10-17-region-ha-2b-joining.md |
| @@ -58,10 +58,16 @@ |
| | Finding or ruling | What changed | |
| |---|---| |
| | New (P2): the timeout does not establish a 75-second commit ceiling | V1: the claim is withdrawn everywhere. The guarantee, stated as it is: every store call of the invite and the enrolling write is issued before the deadline (60 s from the start of the membership describe); the console waits at most 15 s for any call; a write whose result the console did not see is unknown and is settled on a later pass by reading the roster. A cancelled write can still commit later; such an enrolment rests on a membership read that was true when the write was issued, and is the same exposure as an instance detached after enrolment, which spec 3's worker reconciles. No mechanism is added. Task 5's `TestAWriteTheConsoleGaveUpOnCanCommitLater` documents the exposure: a delayed `Put` commits after the next pass found the box absent and refused it on a fresh describe (Tasks 4, 5) | |
| -| platform-1e: the per-call context reaches the SDK | V2: Task 4's `TestTheBoundReachesTheSDKRequest` builds the console's stores as `claim_runtime.go` does, over `release.NewS3Store` with only the network replaced, and records each HTTP request's context deadline: 15 seconds from the call, or the earlier write deadline. A wrapper that drops the context fails it (Task 4) | |
| +| platform-1e: the per-call context reaches the SDK | V2: Task 4's `TestTheBoundReachesTheSDKRequest` builds the console's stores as `claim_runtime.go` does, over `release.NewS3Store` with only the network replaced, and records each HTTP request's context deadline: 15 seconds from the call, or the earlier write deadline. A wrapper that drops the context fails it (Task 4; round 6 drives a successful `PUT` through every wrapper) | |
| | platform-1e: the two host.env rows | V3: `writeHostEnv` and `checkedDir` re-marked `census-ruling-region-ha-2b` (Task 2) | |
| | platform-1e: late commits land only where they were aimed | V4: every write in the invitation and enrolment paths is conditional, shown from main's code in "Global Constraints: Freshness bound" | |
| |
| +**Review round 6.** Astra's sixth review (`CODEX-REVIEW-REGION-HA-2B-6.md`) closed round 5's finding and judged the plan ready to build, with one P3 that platform-1e's acceptance of revision 5 also asks for (W1): |
| + |
| +| Finding or ruling | What changed | |
| +|---|---| |
| +| P3: the SDK test misses successful writes through the wrapper stack; W1 (platform-1e) | `TestTheBoundReachesTheSDKRequest`'s fake bucket now honours S3's conditional headers and is seeded with a root-signed roster, so the conditional roster `PUT` succeeds through the deadline store, the mutation boundary, the conditional record store and the S3 store, and the test asserts that `PUT`'s deadline and its `If-Match`. Dropping the context where any wrapper forwards `Put` fails it (Task 4) | |
| + |
| ## Scope |
| |
| Spec 2 is four plans. This is **2b**. What 2a left it, and what it leaves the others: |
| @@ -213,14 +219,14 @@ nice -n 19 go test ./internal/console/ -run 'TestClaimAuthoritySourceCensus' -co |
| 14. **Ownership is reconciled on a retry (S2).** Task 3's `TestInstallChannelReconcilesOwnershipOnARetry` and `TestInstallChannelRefusesASymlinkedParent`. |
| 15. **A group box never publishes status under a key it doesn't own (T1).** Task 6's `TestAGroupBoxPublishesOnlyUnderItsReference`, through `statusLoop`'s own tick and `WriteBox`: while IMDS fails, another box's report under the colliding label is untouched and nothing is written, with the switches on or off; the reference is asked again each tick, and on recovery the box publishes under it. `TestABoxThatIsNotAGroupBoxKeepsItsLabelAndKey` compares every other box's keys with `statusTick`'s. |
| 16. **Replaced boxes' history never hides the running ones (T2).** Task 6's `TestStaleReportsDoNotHideLiveBoxes`: 250 day-old reports and two live boxes: both live boxes found, and the reading complete. |
| -17. **No store call is issued past its bound, the console waits no longer than its timeout, and an unknown outcome is settled by the roster (U1, U2, V1, V2).** Task 4's `TestTheBoundReachesTheSDKRequest`: through the console's production stores, each SDK request carries 15 seconds from its call, or the earlier write deadline. Task 5's `TestAWriteTheConsoleGaveUpOnCanCommitLater`, the accepted exposure: a `Put` the console gave up on commits after the next pass found the box absent and refused it on a fresh describe without writing, and the pass after that has nothing to do. Task 5's `TestAStoreThatIgnoresItsContextIsCutOffAtTheHardTimeout`: a store that ignores its context and never answers is cut off at the hard timeout, the outcome unknown, the attempt and its wait unchanged. `TestAnUnknownEnrolmentIsSettledByTheRoster`: an enrolling write that did not land goes on with the same attempt and enrols next pass; one that landed, whether cut off in flight or answering success only after the deadline, is unknown, and the next pass, reading the roster, has nothing left to do: no step and no describe. |
| +17. **No store call is issued past its bound, the console waits no longer than its timeout, and an unknown outcome is settled by the roster (U1, U2, V1, V2).** Task 4's `TestTheBoundReachesTheSDKRequest`: through the console's production stores, a successful conditional roster `PUT` and every other SDK request carry 15 seconds from their call, or the earlier write deadline. Task 5's `TestAWriteTheConsoleGaveUpOnCanCommitLater`, the accepted exposure: a `Put` the console gave up on commits after the next pass found the box absent and refused it on a fresh describe without writing, and the pass after that has nothing to do. Task 5's `TestAStoreThatIgnoresItsContextIsCutOffAtTheHardTimeout`: a store that ignores its context and never answers is cut off at the hard timeout, the outcome unknown, the attempt and its wait unchanged. `TestAnUnknownEnrolmentIsSettledByTheRoster`: an enrolling write that did not land goes on with the same attempt and enrols next pass; one that landed, whether cut off in flight or answering success only after the deadline, is unknown, and the next pass, reading the roster, has nothing left to do: no step and no describe. |
| 18. **host.env takes nothing that could change a box's authority unchecked (runHostEnv conditions, U3).** Task 2's `TestWriteTakesOnlyAllowedNamesAndSafeValues`, `TestHostEnvCommand` (no flag moves the path) and `TestWriteGoesOnlyThroughDirectoriesRootAloneCanWrite` (a group-writable `etc/carlos`, a world-writable `etc`, an `etc/carlos` another user owns, a symlinked `etc/carlos` and a symlinked `etc` each refused with nothing written; a normal write lands 0600, replaces the old file by rename, never in place, and leaves no temporary file); Task 6's `TestSelectModeHoldsAStoreValueTheRosterDisagreesWith`; Task 7's `TestRootJoinTakesNothingFromItsTrigger`. |
| |
| ## Dry run |
| |
| Every code block below was written into this branch on main `6834659c` (plan 2a merged), built and vetted for linux, darwin and windows, and its tests run as each task's steps say, with `nice -n 19` and a `-run` filter; the whole `internal/host` suite was never run. The plan was then replayed from a clean base, task by task in order, by applying its own code blocks: every intermediate state built on all three platforms, vetted, passed its task's tests and all three censuses, and the census values here are those runs'. The copy gate was checked without approving anything: with every row pending, `TestRegionHA2bCopyRecordIsComplete`, `TestRegionHA2bCopyHasNoEmDash` and `TestRegionHA2bBuiltCopyIsApplied` pass, and `TestRegionHA2bBuiltCopyIsApproved` fails naming exactly the built ids (21 after Task 2, 66 after Task 9). With a scratch record that marks every row approved, the applied check passes for all 66 rows, and fails for a row whose text was edited. |
| |
| -Rounds 1 to 5 were dry-run the same way, each from a clean base for every task. Round 4 ran 51 mutations through `go test -overlay` from `$TMPDIR`, never editing source in place, and every one was caught by the test named: 37 kept from rounds 1 to 3 and run again on round 4's code (the group check and `EnrollAutomatic`'s own read; a failed read counted against the attempt or reported as a non-member; no backoff; enrolment without the guard-ready report; seed overwriting shared values or not reading one back; install-channel not handing its directories over, handing over only the ones it created, or following a symlinked parent; the relaunch hold, or `relaunched` ignoring `pendingTime`; joining marking the box hosting; the bound store never refusing; `Invite` or `enroll` run unbound; a missed deadline counted against the attempt; a group box keeping its label, or every box taken for one; `PrivateBoxReport`, the pass or `checkConfirmed` trusting a chosen key; the reference without the account; `Fetch` without the marker; a log line without the attempt id; the report read under another key; a group box's status tick falling back to its label, or resolving its reference once; the status listing ignoring write times, or keeping history; the bound with no deadline context; a read of exactly 60 seconds taken as fresh, and a store call exactly at the deadline issued; no allowlist; no cross-check against the roster; and root's join reading its trigger), and 14 for U1 to U3 (no hard per-call timeout; a success after the deadline accepted; an unanswered call reported as success; a cut-off call taken at the store's word; an unknown counted against the attempt, or the attempt forgotten; and for host.env, symlinks followed, `etc` opened unchecked, a group- or world-writable directory accepted, a directory another user owns accepted, the file rewritten in place, the temporary file left behind, `--out` accepted again, and a write that bypasses the held directory). Round 5 changed only a comment in production code, so those stand; it added 9 mutations, all caught. For V1, `TestAWriteTheConsoleGaveUpOnCanCommitLater` catches a cut-off write reported as success and an absent box enrolled without a fresh describe. For V2, `TestTheBoundReachesTheSDKRequest` catches the bound passing its own context instead of the per-call one, the bound dropping the context, `S3Store.Get` or `S3Store.Put` dropping it, the mutation boundary's `Get` or its `Put` pre-read dropping it, and the conditional record store's `Get` dropping it. The fsyncs are not observable in a test and were not mutated. The claim-authority census reads source with `go/parser`, which an overlay does not reach, so in round 4 its four mutations overlaid the census test instead, to parse a mutated `enrollpass.go`: with the pass's start removed, its construction duplicated, its invocation removed, or its start duplicated, the census went red each time, and unmutated it stayed green; round 5 does not touch that file. With a scratch record marking every row approved, the copy gate passed, and went red with RHB-C67's text edited. `internal/hostenv` and `internal/hostactors/...` passed in full on the final tree, the new tests also under `-race`, and the console's enrolment and authority-census tests on their own. The code was then removed, leaving the plan, its copy record and the spec. |
| +Rounds 1 to 5 were dry-run the same way, each from a clean base for every task. Round 6 changed only Task 4's `TestTheBoundReachesTheSDKRequest`, and was dry-run from a clean base through Task 4. Round 4 ran 51 mutations through `go test -overlay` from `$TMPDIR`, never editing source in place, and every one was caught by the test named: 37 kept from rounds 1 to 3 and run again on round 4's code (the group check and `EnrollAutomatic`'s own read; a failed read counted against the attempt or reported as a non-member; no backoff; enrolment without the guard-ready report; seed overwriting shared values or not reading one back; install-channel not handing its directories over, handing over only the ones it created, or following a symlinked parent; the relaunch hold, or `relaunched` ignoring `pendingTime`; joining marking the box hosting; the bound store never refusing; `Invite` or `enroll` run unbound; a missed deadline counted against the attempt; a group box keeping its label, or every box taken for one; `PrivateBoxReport`, the pass or `checkConfirmed` trusting a chosen key; the reference without the account; `Fetch` without the marker; a log line without the attempt id; the report read under another key; a group box's status tick falling back to its label, or resolving its reference once; the status listing ignoring write times, or keeping history; the bound with no deadline context; a read of exactly 60 seconds taken as fresh, and a store call exactly at the deadline issued; no allowlist; no cross-check against the roster; and root's join reading its trigger), and 14 for U1 to U3 (no hard per-call timeout; a success after the deadline accepted; an unanswered call reported as success; a cut-off call taken at the store's word; an unknown counted against the attempt, or the attempt forgotten; and for host.env, symlinks followed, `etc` opened unchecked, a group- or world-writable directory accepted, a directory another user owns accepted, the file rewritten in place, the temporary file left behind, `--out` accepted again, and a write that bypasses the held directory). Round 5 changed only a comment in production code, so those stand; it added 9 mutations, all caught. For V1, `TestAWriteTheConsoleGaveUpOnCanCommitLater` catches a cut-off write reported as success and an absent box enrolled without a fresh describe. For V2, `TestTheBoundReachesTheSDKRequest` catches the bound passing its own context instead of the per-call one, the bound dropping the context, `S3Store.Get` or `S3Store.Put` dropping it, the mutation boundary's `Get` or its `Put` pre-read dropping it, and the conditional record store's `Get` dropping it. Round 6 (W1) re-ran it with the successful conditional `PUT` through every wrapper, and 8 more mutations were caught: the deadline store's `Put`, the mutation boundary's `Put` forwarding and the conditional record store's `Put` forwarding each dropping the context, the bound passing its own context, the boundary's pre-read, `S3Store.Put` and `S3Store.Get` dropping it, and the roster `PUT` sent without `If-Match`. The fsyncs are not observable in a test and were not mutated. The claim-authority census reads source with `go/parser`, which an overlay does not reach, so in round 4 its four mutations overlaid the census test instead, to parse a mutated `enrollpass.go`: with the pass's start removed, its construction duplicated, its invocation removed, or its start duplicated, the census went red each time, and unmutated it stayed green; round 5 does not touch that file. With a scratch record marking every row approved, the copy gate passed, and went red with RHB-C67's text edited. `internal/hostenv` and `internal/hostactors/...` passed in full on the final tree, the new tests also under `-race`, and the console's enrolment and authority-census tests on their own. The code was then removed, leaving the plan, its copy record and the spec. |
| |
| --- |
| |
| @@ -2724,7 +2730,7 @@ The read (C2) is a member only when the instance is in the region's group, `Pend |
| - an unknown write keeps the attempt and its wait, and the next pass reads the roster as every pass does (Task 5, U1); |
| - it bounds the console's waiting, not the bucket's commit: a call cut off may still be applied later. Every write it carries is conditional ("Global Constraints: Freshness bound", V4), so such a write lands only on the roster revision it was built against. |
| |
| -The store is wrapped because it can't be trusted to do this: `release.DirStore` ignores its context (`release.S3Store` passes it to the AWS SDK, which does honour it). The pass's S3 store is the console's claim data boundary (`internal/console/claim_runtime.go:42`, `NewConditionalRecordStore`, then `NewObservedMutationBoundary`), whose S3 client the console shares; U2 preferred a dedicated client for the pass, but a second client means a second boundary, with its own census-selected construction and its own mutation observer. The wrapper gives the same hard limit without one: each call's own 15-second context reaches the SDK's HTTP request, and `call` stops waiting at that point whatever the store does. `TestTheBoundReachesTheSDKRequest` (V2) holds that: it builds the console's stores as `claim_runtime.go:33` to `:50` does, the production mutation check included, over `release.NewS3Store` with only the network replaced by a recorder, and asserts every request the SDK sends (a `Get` through the boundary, a `Put`'s pre-read through it, and the S3 store's own `Put`) carries 15 seconds from its call, or the write deadline when that is earlier. `export_test.go` gives the external test `writeBound`; the test is external because `hostops` imports `hostactors`. `EnrollAutomatic` runs main's `enroll` on the bound directory. The tests drive a describe of exactly 60 seconds, a clock that reaches the deadline between the check and the roster write, and (Task 5) a store read that answers at the deadline inside the sealed functions, a `Put` still in flight there, a store that ignores its context and never answers, and a `Put` that answers success after the deadline. |
| +The store is wrapped because it can't be trusted to do this: `release.DirStore` ignores its context (`release.S3Store` passes it to the AWS SDK, which does honour it). The pass's S3 store is the console's claim data boundary (`internal/console/claim_runtime.go:42`, `NewConditionalRecordStore`, then `NewObservedMutationBoundary`), whose S3 client the console shares; U2 preferred a dedicated client for the pass, but a second client means a second boundary, with its own census-selected construction and its own mutation observer. The wrapper gives the same hard limit without one: each call's own 15-second context reaches the SDK's HTTP request, and `call` stops waiting at that point whatever the store does. `TestTheBoundReachesTheSDKRequest` (V2, W1) holds that: it builds the console's stores as `claim_runtime.go:33` to `:50` does, the production mutation check included, over `release.NewS3Store` with only the network replaced by a recorder that honours S3's conditional headers. The bucket is seeded with a root-signed roster, so the enrolling roster write is a successful conditional `PUT` through the whole stack: the deadline store, the observed mutation boundary (its pre-read, the production check, which admits a root-signed roster, and its forwarding), the conditional record store's forwarding and the S3 store. The test asserts the SDK sends exactly the roster read's `GET`, then the write's pre-read `GET` and its `PUT`, that the `PUT` carries `If-Match` with the ETag read, that a stale ETag is then refused, and that every request carries 15 seconds from its call, or the write deadline when that is earlier. `export_test.go` gives the external test `writeBound`; the test is external because `hostops` imports `hostactors`. `EnrollAutomatic` runs main's `enroll` on the bound directory. The tests drive a describe of exactly 60 seconds, a clock that reaches the deadline between the check and the roster write, and (Task 5) a store read that answers at the deadline inside the sealed functions, a `Put` still in flight there, a store that ignores its context and never answers, and a `Put` that answers success after the deadline. |
| |
| **Group references (S3).** `GroupBoxReference` is platform-1e's function: `g-` and the first 20 hex digits of `sha256("carlos-box-ref-v1|" + account + "|" + instance)`. `checkConfirmed` refuses a request whose reference is not the attempt instance's group reference, before the cloud is asked, and `PrivateBoxReport` reads a group box's report under the reference derived from the instance its entry pins, so `checkMembership`'s `VerifyReport`, the census and activation never read a group box's report under a key the box chose. A report another box writes there names its own instance and enrollment, and `VerifyReport` refuses it. Non-group boxes are read exactly as before. `deadlineStore.Put` is a new site in the record-write census (nonrecord). |
| |
| @@ -3057,8 +3063,11 @@ func BindForTest(ctx context.Context, d Directory, observedAt time.Time, now fun |
| package hostactors_test |
| |
| import ( |
| + "bytes" |
| "context" |
| "crypto/ed25519" |
| + "errors" |
| + "fmt" |
| "io" |
| "net/http" |
| "strings" |
| @@ -3074,18 +3083,28 @@ import ( |
| ) |
| |
| // sdkRequest is one HTTP request the AWS SDK sent: its method, its path, |
| -// and how long its context had left when it was sent (zero: no deadline). |
| +// its If-Match condition, and how long its context had left when it was |
| +// sent (zero: no deadline). |
| type sdkRequest struct { |
| - method, path string |
| - left time.Duration |
| + method, path, ifMatch string |
| + left time.Duration |
| } |
| |
| -// recordingHTTP stands in for the network under the SDK. It records each |
| -// request's context deadline and answers as an empty bucket: a GET is |
| -// NoSuchKey, a PUT succeeds. |
| +// recordingHTTP stands in for the network under the SDK: a bucket that |
| +// honours S3's conditional headers. It records each request's context |
| +// deadline. A GET answers the object and its ETag, or NoSuchKey; a PUT |
| +// with If-Match succeeds only on the current ETag, one with |
| +// If-None-Match: * only when the key is absent, and gets a new ETag. |
| type recordingHTTP struct { |
| - mu sync.Mutex |
| - seen []sdkRequest |
| + mu sync.Mutex |
| + seen []sdkRequest |
| + objects map[string]bucketObject |
| + next int |
| +} |
| + |
| +type bucketObject struct { |
| + body []byte |
| + etag string |
| } |
| |
| func (h *recordingHTTP) Do(r *http.Request) (*http.Response, error) { |
| @@ -3093,24 +3112,48 @@ func (h *recordingHTTP) Do(r *http.Request) (*http.Response, error) { |
| if dl, ok := r.Context().Deadline(); ok { |
| left = time.Until(dl) |
| } |
| - h.mu.Lock() |
| - h.seen = append(h.seen, sdkRequest{r.Method, r.URL.Path, left}) |
| - h.mu.Unlock() |
| + var body []byte |
| if r.Body != nil { |
| - _, _ = io.Copy(io.Discard, r.Body) |
| + body, _ = io.ReadAll(r.Body) |
| + } |
| + h.mu.Lock() |
| + defer h.mu.Unlock() |
| + h.seen = append(h.seen, sdkRequest{r.Method, r.URL.Path, r.Header.Get("If-Match"), left}) |
| + resp := &http.Response{Header: http.Header{}, Request: r, Proto: "HTTP/1.1", ProtoMajor: 1, ProtoMinor: 1, Body: io.NopCloser(strings.NewReader(""))} |
| + reply := func(code int, xml string) (*http.Response, error) { |
| + resp.StatusCode = code |
| + resp.Header.Set("Content-Type", "application/xml") |
| + resp.Body = io.NopCloser(strings.NewReader(`<?xml version="1.0" encoding="UTF-8"?><Error><Code>` + xml + `</Code><Message>fake</Message></Error>`)) |
| + return resp, nil |
| } |
| - resp := &http.Response{Header: http.Header{}, Request: r, Proto: "HTTP/1.1", ProtoMajor: 1, ProtoMinor: 1} |
| + key := r.URL.Path |
| + cur, exists := h.objects[key] |
| switch r.Method { |
| case http.MethodPut: |
| + if m := r.Header.Get("If-Match"); m != "" && (!exists || strings.Trim(m, `"`) != strings.Trim(cur.etag, `"`)) { |
| + return reply(http.StatusPreconditionFailed, "PreconditionFailed") |
| + } |
| + if r.Header.Get("If-None-Match") == "*" && exists { |
| + return reply(http.StatusPreconditionFailed, "PreconditionFailed") |
| + } |
| + h.next++ |
| + obj := bucketObject{body, fmt.Sprintf(`"e%d"`, h.next)} |
| + h.objects[key] = obj |
| resp.StatusCode = http.StatusOK |
| - resp.Header.Set("ETag", `"e1"`) |
| - resp.Body = io.NopCloser(strings.NewReader("")) |
| - default: |
| - resp.StatusCode = http.StatusNotFound |
| - resp.Header.Set("Content-Type", "application/xml") |
| - resp.Body = io.NopCloser(strings.NewReader(`<?xml version="1.0" encoding="UTF-8"?><Error><Code>NoSuchKey</Code><Message>none</Message></Error>`)) |
| + resp.Header.Set("ETag", obj.etag) |
| + return resp, nil |
| + case http.MethodGet: |
| + if !exists { |
| + return reply(http.StatusNotFound, "NoSuchKey") |
| + } |
| + resp.StatusCode = http.StatusOK |
| + resp.Header.Set("ETag", cur.etag) |
| + resp.Header.Set("Content-Length", fmt.Sprint(len(cur.body))) |
| + resp.Body = io.NopCloser(bytes.NewReader(cur.body)) |
| + resp.ContentLength = int64(len(cur.body)) |
| + return resp, nil |
| } |
| - return resp, nil |
| + return reply(http.StatusNotFound, "NoSuchKey") |
| } |
| |
| func (h *recordingHTTP) take() []sdkRequest { |
| @@ -3126,7 +3169,7 @@ func (h *recordingHTTP) take() []sdkRequest { |
| // the network replaced), the conditional record store over it, and the |
| // observed mutation boundary with the production mutation check over |
| // that: the store the enrolment pass's Directory carries. |
| -func productionStores(t *testing.T) (raw *release.S3Store, data release.Store, h *recordingHTTP) { |
| +func productionStores(t *testing.T) (raw *release.S3Store, data release.Store, h *recordingHTTP, root release.Signer) { |
| t.Helper() |
| t.Setenv("AWS_ACCESS_KEY_ID", "k") |
| t.Setenv("AWS_SECRET_ACCESS_KEY", "s") |
| @@ -3140,9 +3183,9 @@ func productionStores(t *testing.T) (raw *release.S3Store, data release.Store, h |
| if err != nil { |
| t.Fatal(err) |
| } |
| - h = &recordingHTTP{} |
| + h = &recordingHTTP{objects: map[string]bucketObject{}} |
| raw.Client = awss3.New(raw.Client.Options(), func(o *awss3.Options) { o.HTTPClient = h; o.RetryMaxAttempts = 1 }) |
| - pub, _, err := ed25519.GenerateKey(nil) |
| + pub, priv, err := ed25519.GenerateKey(nil) |
| if err != nil { |
| t.Fatal(err) |
| } |
| @@ -3162,17 +3205,22 @@ func productionStores(t *testing.T) (raw *release.S3Store, data release.Store, h |
| if err != nil { |
| t.Fatal(err) |
| } |
| - return raw, data, h |
| + return raw, data, h, release.RootSigner(priv) |
| } |
| |
| -// V2 (platform-1e): the per-call context of a bound write reaches the AWS |
| -// SDK's HTTP request through the console's production stores, so the |
| -// 15-second limit is enforced where the request is sent. Each request |
| -// carries a deadline of 15 seconds from its call, or the write deadline |
| -// when that comes first. A wrapper anywhere on the path that drops the |
| -// context, or a bound that does not give each call its own, fails here. |
| +// V2 and W1 (platform-1e): the per-call context of a bound write reaches |
| +// the AWS SDK's HTTP request through the console's production stores, so |
| +// the 15-second limit is enforced where the request is sent. The roster |
| +// write is a successful conditional PUT through the whole stack: the |
| +// bound's deadline store, the observed mutation boundary (its pre-read, |
| +// the production mutation check, which admits a root-signed roster, and |
| +// its forwarding), the conditional record store's forwarding and the S3 |
| +// store. Every request it sends, the PUT included, carries a deadline of |
| +// 15 seconds from its call, or the write deadline when that comes first. |
| +// A wrapper anywhere on the path that drops the context, or a bound that |
| +// does not give each call its own, fails here. |
| func TestTheBoundReachesTheSDKRequest(t *testing.T) { |
| - raw, data, h := productionStores(t) |
| + _, data, h, root := productionStores(t) |
| for name, c := range map[string]struct { |
| readAge time.Duration |
| low, high time.Duration |
| @@ -3181,36 +3229,58 @@ func TestTheBoundReachesTheSDKRequest(t *testing.T) { |
| "the earlier write deadline": {50 * time.Second, 9 * time.Second, 10 * time.Second}, |
| } { |
| t.Run(name, func(t *testing.T) { |
| - check := func(what string, got []sdkRequest, method string) { |
| + check := func(what string, got []sdkRequest, methods ...string) []sdkRequest { |
| t.Helper() |
| - saw := false |
| + var sent []string |
| for _, r := range got { |
| - saw = saw || r.method == method |
| + sent = append(sent, r.method) |
| if r.left <= c.low || r.left > c.high { |
| t.Errorf("%s: %s %s had %s left, want (%s, %s]", what, r.method, r.path, r.left, c.low, c.high) |
| } |
| } |
| - if !saw { |
| - t.Fatalf("%s: no %s reached the SDK: %+v", what, method, got) |
| + if strings.Join(sent, " ") != strings.Join(methods, " ") { |
| + t.Fatalf("%s: the SDK sent %v, want %v", what, sent, methods) |
| } |
| + return got |
| + } |
| + // The bucket holds a roster the pass has read. |
| + h.mu.Lock() |
| + h.objects = map[string]bucketObject{} |
| + h.mu.Unlock() |
| + before, err := release.SignRootEnvelope(root, map[string]int{"revision": 1}) |
| + if err != nil { |
| + t.Fatal(err) |
| } |
| - // Through the pass's store: the Directory over the boundary. |
| d := hostactors.Directory{Store: data, Bucket: "deployment"} |
| + etag, err := data.Put(context.Background(), hostactors.Key, before, "") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + h.take() |
| bd, bctx, cancel := hostactors.BindForTest(context.Background(), d, time.Now().Add(-c.readAge), time.Now) |
| defer cancel() |
| - _, _, _ = bd.Store.Get(bctx, hostactors.Key) |
| - check("Get through the boundary", h.take(), http.MethodGet) |
| - _, _ = bd.Store.Put(bctx, hostactors.Key, []byte("{}"), `"e0"`) |
| - check("Put through the boundary", h.take(), http.MethodGet) |
| - // The S3 store's own Put, the request a roster or invitation |
| - // write ends in. |
| - rd := hostactors.Directory{Store: raw, Bucket: "deployment"} |
| - rbd, rctx, rcancel := hostactors.BindForTest(context.Background(), rd, time.Now().Add(-c.readAge), time.Now) |
| - defer rcancel() |
| - if _, err := rbd.Store.Put(rctx, "control/host-actor-invitations/g-0123456789abcdef0123.json", []byte("{}"), ""); err != nil { |
| + if _, got, err := bd.Store.Get(bctx, hostactors.Key); err != nil || got != etag { |
| + t.Fatalf("roster read: %q %v", got, err) |
| + } |
| + check("the roster read", h.take(), http.MethodGet) |
| + // The enrolling write: conditional on that ETag, and admitted. |
| + next, err := release.SignRootEnvelope(root, map[string]int{"revision": 2}) |
| + if err != nil { |
| t.Fatal(err) |
| } |
| - check("the S3 store's Put", h.take(), http.MethodPut) |
| + after, err := bd.Store.Put(bctx, hostactors.Key, next, etag) |
| + if err != nil || after == etag { |
| + t.Fatalf("the conditional roster write: %q %v", after, err) |
| + } |
| + sent := check("the roster write", h.take(), http.MethodGet, http.MethodPut) |
| + if strings.Trim(sent[1].ifMatch, `"`) != strings.Trim(etag, `"`) { |
| + t.Fatalf("the roster PUT was sent with If-Match %q, want the ETag read, %q", sent[1].ifMatch, etag) |
| + } |
| + // And it stays conditional: the old ETag now loses. |
| + if _, err := bd.Store.Put(bctx, hostactors.Key, next, etag); !errors.Is(err, release.ErrCASConflict) { |
| + t.Fatalf("a stale ETag was not refused: %v", err) |
| + } |
| + h.take() |
| }) |
| } |
| } |
amadan