amadan

oficina / memoria Public

🤖 feat: S1 and S3 clients, where a failed poll must not read as empty

Slice 1's two remaining sibling calls. The whole weight is on one
distinction: **a failed enumeration returns no information, not no
documents.** §5.1 and gate 7 — the second reading purges the corpus, and
it is this codebase's own worst failure shape (the plausible empty list)
sitting on the one call whose result authorises deletion.

The module refuses to decode a non-2xx for the same reason. What this
adds above it is the case with no status at all: a call that never
reached a server, which DecodeS1 never sees because it is never invoked.

S3 carries an item, a revision and nobody (§6.1). The test asserts that
as a closed set over the query, not as a list of forbidden parameter
names — `query.Get("asker") == ""` passes for `user`, `on_behalf_of`,
and every other spelling of the same mistake.

One correction on the way in: the first S3 test asserted memoria must
not be able to tell 404 from 403. Wrong. "404, never 403" binds the
*sibling*; the module deliberately keeps a 403 distinguishable, because
folding it into ErrNotAvailable hides a leaking sibling from the only
person who could fix it. The test now asserts the breach surfaces.

Nine mutations, all verified red and reverted: the transport error
swallowed; DecodeS1's error ignored; the cursor dropped; Feed always
erroring; the status unconsulted; the breach folded away; an empty body
returned beside an error; an asker added to the URL; the revision
dropped. Survivors: none. An independent pick still owes — §1.7's rule
is that the author's own mutations prove the author consistent, not the
gate sound.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H3VKtXnq6fSmM2F11Bjcsy
Paul Campbell pushed by paul@keymail.dev 829acc6555a69699a5f188dfdf3597f99673718a parent aecdfb0
2 files changed, +355 −0
  • internal/memoria/sibling/feed.go +108 −0
  • internal/memoria/sibling/feed_test.go +247 −0
diff --git a/internal/memoria/sibling/feed.go b/internal/memoria/sibling/feed.go
new file mode 100644
index 0000000..44dd3be
--- /dev/null
+++ b/internal/memoria/sibling/feed.go
@@ -0,0 +1,108 @@
+package sibling
+
+import (
+ "context"
+ "fmt"
+ "io"
+ "net/http"
+ "net/url"
+
+ "amadan.net/oficina/suite"
+)
+
+// maxFeedResponse and maxContentResponse cap what a sibling can make
+// memoria hold in memory on one call.
+//
+// A feed page is four small fields a row, so 4 MiB is thousands of rows and
+// far past any sane page. A body is the same 2 MiB an upload is capped at,
+// plus slack for transport framing — a sibling serving more than that has
+// gone wrong, and reading it all to discover that is a second failure.
+// Ceilings on damage rather than measurements, said out loud because
+// AGENTS.md requires a cap to carry its reason.
+const (
+ maxFeedResponse = 4 << 20
+ maxContentResponse = (2 << 20) + (64 << 10)
+)
+
+// Feed polls one sibling's change feed from a stored cursor.
+//
+// **A failure is an error, never an empty feed**, and that distinction is
+// the whole reason this returns one. §5.1 and gate 7: an enumeration diff
+// must tell an authoritative empty result from a failure *before* deleting
+// anything, because a timed-out, rate-limited or half-paginated poll
+// returns no information rather than no documents — and treating the second
+// as the first purges the corpus silently. The module refuses to decode a
+// non-2xx for that reason; this is the half above it, where a call that
+// never got a status at all lives.
+//
+// The cursor is opaque and the sibling's. A caller stores the one that
+// comes back **only on success**: advancing past a page nobody read is how
+// a consumer skips changes it will never be offered again.
+func (c *Client) Feed(ctx context.Context, s Sibling, cursor string) (suite.Feed, error) {
+ ctx, cancel := context.WithTimeout(ctx, Deadline)
+ defer cancel()
+
+ u := s.BaseURL + suite.RouteS1
+ if cursor != "" {
+ u += "?cursor=" + url.QueryEscape(cursor)
+ }
+ req, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
+ if err != nil {
+ return suite.Feed{}, fmt.Errorf("sibling %s: build S1 request: %w", s.Name, err)
+ }
+ req.Header.Set("Authorization", "Bearer "+s.Token)
+
+ res, err := c.http.Do(req)
+ if err != nil {
+ return suite.Feed{}, fmt.Errorf("sibling %s: S1 unreachable: %w", s.Name, err)
+ }
+ defer res.Body.Close()
+
+ raw, err := io.ReadAll(io.LimitReader(res.Body, maxFeedResponse))
+ if err != nil {
+ return suite.Feed{}, fmt.Errorf("sibling %s: reading S1: %w", s.Name, err)
+ }
+ feed, err := suite.DecodeS1(res.StatusCode, raw)
+ if err != nil {
+ return suite.Feed{}, fmt.Errorf("sibling %s: %w", s.Name, err)
+ }
+ return feed, nil
+}
+
+// Content fetches one item's body. **`(id, revision)`, and no asker.**
+//
+// §6.1: the ingest call is custody-scoped and person-less, and the sibling
+// answers only for items that are team-visible. A per-person S3 was draft
+// 2's error and could not have been made — there is nobody there on an
+// ingest — and §1.3 records that the fix was deleting the false claim
+// rather than adding a person to the call.
+//
+// A missing item is **404, never 403**, so the two are indistinguishable
+// and nobody learns which ids exist. The module's DecodeS3 turns both into
+// ErrNotAvailable and this returns that unchanged: a caller that branched
+// on which one it was would rebuild the oracle §4.2 closes — the same
+// oracle the no-row S2 rule opened and had to be withdrawn for (§4.1).
+func (c *Client) Content(ctx context.Context, s Sibling, itemID string, revision int64) ([]byte, error) {
+ ctx, cancel := context.WithTimeout(ctx, Deadline)
+ defer cancel()
+
+ u := fmt.Sprintf("%s%s?item_id=%s&revision=%d",
+ s.BaseURL, suite.RouteS3, url.QueryEscape(itemID), revision)
+ req, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
+ if err != nil {
+ return nil, fmt.Errorf("sibling %s: build S3 request: %w", s.Name, err)
+ }
+ req.Header.Set("Authorization", "Bearer "+s.Token)
+
+ res, err := c.http.Do(req)
+ if err != nil {
+ return nil, fmt.Errorf("sibling %s: S3 unreachable: %w", s.Name, err)
+ }
+ defer res.Body.Close()
+
+ raw, err := io.ReadAll(io.LimitReader(res.Body, maxContentResponse))
+ if err != nil {
+ return nil, fmt.Errorf("sibling %s: reading S3: %w", s.Name, err)
+ }
+ return suite.DecodeS3(res.StatusCode, raw)
+}
diff --git a/internal/memoria/sibling/feed_test.go b/internal/memoria/sibling/feed_test.go
new file mode 100644
index 0000000..e11f83c
--- /dev/null
+++ b/internal/memoria/sibling/feed_test.go
@@ -0,0 +1,247 @@
+package sibling
+
+import (
+ "context"
+ "encoding/json"
+ "errors"
+ "net/http"
+ "net/url"
+ "reflect"
+ "testing"
+ "time"
+
+ "amadan.net/oficina/suite"
+)
+
+// Gate 7's rule at the call boundary: **an empty feed and a failed poll
+// must never be the same value.**
+//
+// §5.1: "A timed-out, rate-limited, partially-paginated or credential-expired
+// enumeration returns *no information*, not *no documents*; treating the
+// second as the first purges the corpus silently." That is the worst
+// failure this codebase has a name for — the plausible empty list — and it
+// arrives here, on the one call whose result authorises deletion.
+//
+// The module refuses to decode a non-2xx for the same reason. What this
+// asserts is the half above it: a call that never reached a server has no
+// status for the module to refuse, and must still not read as "nothing
+// changed".
+//
+// TWO-SIDED. Every "this is an error" assertion is satisfied by a Feed that
+// always errors, so the last subtest requires a real page to decode and its
+// cursor to come back. Without it, `return suite.Feed{}, errors.New("no")`
+// passes this file.
+//
+// MUTATION-TESTED 2026-09-06, each verified red then reverted:
+//
+// Feed swallows the transport error and returns suite.Feed{}, nil
+// -> unreachable case red
+// Feed ignores DecodeS1's error and returns the zero Feed
+// -> 500 case red, and the ambiguous-body case red
+// Feed sends no cursor at all
+// -> happy-path case red
+// Feed always errors
+// -> happy-path case red
+func TestAFailedPollIsNeverAnEmptyFeed(t *testing.T) {
+ ctx := context.Background()
+
+ t.Run("a sibling that cannot be reached is an error, not zero rows", func(t *testing.T) {
+ s := serve(t, func(w http.ResponseWriter, r *http.Request) {
+ hj, ok := w.(http.Hijacker)
+ if !ok {
+ t.Skip("no hijacker")
+ }
+ conn, _, err := hj.Hijack()
+ if err != nil {
+ t.Fatalf("hijack: %v", err)
+ }
+ conn.Close()
+ })
+ feed, err := newTestClient().Feed(ctx, s, "")
+ if err == nil {
+ t.Fatalf("got feed %+v and no error: an unreachable sibling read as "+
+ "'nothing changed', which is what authorises a purge", feed)
+ }
+ if len(feed.Rows) != 0 || feed.Cursor != "" {
+ t.Errorf("a failed poll returned data: %+v", feed)
+ }
+ })
+
+ t.Run("a non-2xx is an error, not zero rows", func(t *testing.T) {
+ s := serve(t, func(w http.ResponseWriter, r *http.Request) {
+ w.WriteHeader(http.StatusInternalServerError)
+ _, _ = w.Write([]byte(`{"cursor":"c2","rows":[]}`))
+ })
+ // The body is a perfectly good empty feed. The status is not, and
+ // the status wins: a sibling erroring while returning a plausible
+ // body is exactly how a purge gets authorised by a failure.
+ if _, err := newTestClient().Feed(ctx, s, ""); err == nil {
+ t.Error("a 500 carrying a well-formed empty feed decoded as success")
+ }
+ })
+
+ t.Run("an ambiguous body is an error", func(t *testing.T) {
+ s := serve(t, func(w http.ResponseWriter, r *http.Request) {
+ // A duplicated key: encoding/json resolves it silently, and a
+ // duplicated `rows` would lose every tombstone in the first copy.
+ _, _ = w.Write([]byte(`{"cursor":"c","rows":[],"rows":[{"item_id":"x","revision":1,"gone":true,"touched_at":"2026-01-01T00:00:00Z"}]}`))
+ })
+ if _, err := newTestClient().Feed(ctx, s, ""); err == nil {
+ t.Error("an ambiguous body decoded as success")
+ }
+ })
+
+ t.Run("a real page decodes, or none of the above proves anything", func(t *testing.T) {
+ var gotCursor string
+ s := serve(t, func(w http.ResponseWriter, r *http.Request) {
+ gotCursor = r.URL.Query().Get("cursor")
+ _ = json.NewEncoder(w).Encode(suite.Feed{
+ Cursor: "next-cursor",
+ Rows: []suite.FeedRow{
+ {ItemID: "doc-a", Revision: 3, Gone: false, TouchedAt: time.Unix(0, 0).UTC()},
+ {ItemID: "doc-b", Revision: 9, Gone: true, TouchedAt: time.Unix(0, 0).UTC()},
+ },
+ })
+ })
+ feed, err := newTestClient().Feed(ctx, s, "stored-cursor")
+ if err != nil {
+ t.Fatalf("a good page failed to decode: %v — if Feed always errors, "+
+ "every assertion above passes for the wrong reason", err)
+ }
+ if gotCursor != "stored-cursor" {
+ t.Errorf("cursor sent = %q, want the stored one: a poll that forgets "+
+ "its position re-reads the whole feed", gotCursor)
+ }
+ if feed.Cursor != "next-cursor" || len(feed.Rows) != 2 {
+ t.Errorf("decoded %+v, want 2 rows and the sibling's next cursor", feed)
+ }
+ if !feed.Rows[1].Gone {
+ t.Error("the tombstone did not survive decoding, which is the row " +
+ "that carries a deletion")
+ }
+ })
+}
+
+// The S3 fetch path, and what memoria owns on it.
+//
+// Not the indistinguishability itself: "404, never 403" is an obligation on
+// the *sibling*, and the module deliberately refuses to fold a 403 into
+// ErrNotAvailable — doing so would hide a leaking sibling from everyone
+// downstream, including the operator who could fix it. A first draft of
+// this test asserted the opposite, that memoria must not be able to tell
+// the two apart. That was wrong: the consumer noticing a breach is not the
+// leak, the breach is.
+//
+// What memoria owns is narrower, and all of it is here:
+//
+// a failure is never an empty body — `nil, nil` writes emptiness over a
+// real document, gate 7's shape one row down;
+// a breach stays a breach, not a shrug;
+// the call names an item and a revision and **nobody**. §6.1: the ingest
+// call is custody-scoped and person-less.
+//
+// TWO-SIDED: the last subtest needs a real body back, or a Content that
+// only ever errors satisfies every other assertion in this function.
+//
+// MUTATION-TESTED 2026-09-06, each verified red then reverted:
+//
+// return raw, nil without consulting the status
+// -> 404 case red, breach case red
+// fold ErrContractBreach into ErrNotAvailable
+// -> breach case red
+// on error return []byte{} instead of nil
+// -> 404 case red
+// add &asker=... to the URL
+// -> closed-set case red
+// drop the revision from the URL
+// -> closed-set case red
+func TestContentFetch(t *testing.T) {
+ ctx := context.Background()
+
+ t.Run("a missing item is ErrNotAvailable and not an empty body", func(t *testing.T) {
+ s := serve(t, func(w http.ResponseWriter, r *http.Request) {
+ w.WriteHeader(http.StatusNotFound)
+ })
+ body, err := newTestClient().Content(ctx, s, "doc-a", 1)
+ if !errors.Is(err, suite.ErrNotAvailable) {
+ t.Fatalf("404 = %v, want ErrNotAvailable", err)
+ }
+ if body != nil {
+ t.Errorf("got body %q alongside an error: a caller that ingests "+
+ "this writes emptiness over a document it still has", body)
+ }
+ })
+
+ t.Run("a sibling answering 403 is a breach, not a shrug", func(t *testing.T) {
+ s := serve(t, func(w http.ResponseWriter, r *http.Request) {
+ w.WriteHeader(http.StatusForbidden)
+ })
+ _, err := newTestClient().Content(ctx, s, "doc-a", 1)
+ if !errors.Is(err, suite.ErrContractBreach) {
+ t.Errorf("403 = %v, want ErrContractBreach — 403 means \"this exists "+
+ "and you may not have it\", the one fact custody hides, and "+
+ "quietly reading it as ErrNotAvailable keeps that leak invisible "+
+ "to the only person who could fix it", err)
+ }
+ })
+
+ t.Run("an unreachable sibling is an error, not an empty body", func(t *testing.T) {
+ s := serve(t, func(w http.ResponseWriter, r *http.Request) {
+ hj, ok := w.(http.Hijacker)
+ if !ok {
+ t.Skip("no hijacker")
+ }
+ conn, _, err := hj.Hijack()
+ if err != nil {
+ t.Fatalf("hijack: %v", err)
+ }
+ conn.Close()
+ })
+ body, err := newTestClient().Content(ctx, s, "doc-a", 1)
+ if err == nil {
+ t.Fatalf("got %q and no error from a sibling that never answered", body)
+ }
+ if body != nil {
+ t.Errorf("a failed fetch returned a body: %q", body)
+ }
+ })
+
+ t.Run("the call names an item and a revision and nobody", func(t *testing.T) {
+ var query url.Values
+ var headers http.Header
+ s := serve(t, func(w http.ResponseWriter, r *http.Request) {
+ query, headers = r.URL.Query(), r.Header.Clone()
+ _, _ = w.Write([]byte("the body"))
+ })
+ body, err := newTestClient().Content(ctx, s, "doc-a", 7)
+ if err != nil {
+ t.Fatalf("a good fetch failed: %v — with this red, every assertion "+
+ "above passes for a Content that only ever errors", err)
+ }
+ if string(body) != "the body" {
+ t.Errorf("body = %q, want %q", body, "the body")
+ }
+
+ // A closed set, not a list of forbidden names. §6.1 says the call
+ // carries no person, and the way to assert that is to enumerate what
+ // it may carry and fail on anything else — a check spelled
+ // `query.Get("asker") == ""` passes for `user`, `on_behalf_of`, and
+ // every other spelling of the same mistake.
+ want := url.Values{"item_id": {"doc-a"}, "revision": {"7"}}
+ if !reflect.DeepEqual(query, want) {
+ t.Errorf("S3 query = %v, want exactly %v", query, want)
+ }
+ if got := headers.Get("Authorization"); got != "Bearer test-consumer-token" {
+ t.Errorf("Authorization = %q, want the consumer token", got)
+ }
+ // The consumer token identifies the *app* (§4.3). A person would
+ // have to arrive as some second header, so the headers get the same
+ // treatment as the query: named ones checked, and the rest is what
+ // the Go client adds on its own.
+ for _, h := range []string{"From", "X-Asker", "X-User", "Cookie"} {
+ if v := headers.Get(h); v != "" {
+ t.Errorf("S3 carried %s: %q — the ingest call has nobody on it", h, v)
+ }
+ }
+ })
+}