amadan
docs(spec): draft 3 — answer the draft-2 review, and Paul's numeric ask
GPT Sol (via Codex on Paul's subscription) reviewed draft 2: five fixes confirmed answered, one blocker, six live findings. Draft 3 answers them. The blocker is not mine to close, and §1.3 says so plainly. memoria pulls plaintext bodies from siblings, and home §2.2, Sheets PRODUCT.md principle 3 and Docs §8.2 each forbid content crossing an instance boundary. Paul's one-trust-domain ruling stands, but memoria cannot enact another app's settled decision in its own table. Slice 1 is blocked on agreement, not on code. "Nothing left to reconcile" was still false: S2 fixed sibling content while admission and memoria-owned content both used memoria's own members table, so a person removed everywhere else could still read the team's Notion, Drive, mail and uploads. Admission is now relayed from the siblings (decision 11) — memoria holds no membership answer of its own. A stale revision is not a denial. Draft 2 mapped it to NOT_VISIBLE, which is silently omitted, recreating the exact failure it claimed to remove. S2 is now four-valued with STALE, and its decoder is total: every non-answer becomes UNKNOWN rather than falling through to omission. S3 promised a per-person grants join while taking no person. It carries the person ref now. Derived artefacts have an algorithm for external and mixed provenance; authorities are ANDed. Restore has a durable tombstone log independent of the database, and snapshots are not retired while a restorable backup could reference them. §6.5 gives live authorization a deadline, a concurrency policy and a partial-result contract — and states plainly that a reliably slow sibling degrades the primary search path, with no design that fixes it while failing closed. §7's numbers are measured now, not asserted, and measuring found draft 2 wrong: the FTS figure was out by half, so the ceiling is ~90-110k chunks rather than 150k. Draft 2 moved that number the wrong way while claiming to have fixed the arithmetic. §8.7 answers Paul's "average revenue in June the last 3 years". Fable and GPT Sol converged: Sheets computes via an enumerated-function S6 call, never expression text, so memoria is never a formula author; retrieval resolves named slots and refuses to show a number unless every slot has exactly one authorized operand. Both independently forbade the same anti-design — a RAG calculator letting a model average numeric strings it retrieved, which cites correctly while being wrong. Gates rewritten: gate 5 now requires two people to get different answers from the sibling's real grants join, because "an authorization call happened" stays green with a constant VISIBLE.
Reviewed-by: gpt-5 (self-reported; Codex labelled it gpt-5.6-sol) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SiWVaR69Vb2Lzc9wgfit1v
1 file changed,
+410
−67
docs/superpowers/specs/2026-09-03-memoria-design.md+410 −67
diff --git a/docs/superpowers/specs/2026-09-03-memoria-design.md b/docs/superpowers/specs/2026-09-03-memoria-design.md| index fa611d6..8b5c6f4 100644 |
| --- a/docs/superpowers/specs/2026-09-03-memoria-design.md |
| +++ b/docs/superpowers/specs/2026-09-03-memoria-design.md |
| @@ -3,18 +3,23 @@ |
| The Oficina suite's memory. One instance per team, at |
| `<team>.memoria.oficina.works`. |
| -**Status:** draft 2, 2026-09-03. |
| +**Status:** draft 3, 2026-09-03. |
| -Draft 1 was reviewed independently by **GPT Sol** (via commandcode) and |
| -by **Fable**, both against the same adversarial brief, and both returned |
| -**REQUEST CHANGES** — 12 and 21 findings, three structural gaps found by |
| -both. This draft answers all of them. §1.2 records what changed and why, |
| -because two corrections reverse a settled decision and a future reader is |
| -owed the reason. |
| +Review history, per AGENTS.md §Review. None of these models wrote the |
| +draft they reviewed; all drafts were written by Opus. |
| -Per AGENTS.md §Review, the reviewing models are recorded above; neither |
| -wrote draft 1, which was written by Opus. **This draft has not itself |
| -been reviewed** — that pass is owed before approval. |
| +| Draft | Reviewers | Verdict | |
| +|---|---|---| |
| +| 1 | GPT Sol (commandcode), Fable | REQUEST CHANGES ×2 — 12 and 21 findings, three structural gaps found by both | |
| +| 2 | GPT Sol via Codex on a ChatGPT subscription — **it reported its own model id as `gpt-5`, which disagrees with the `gpt-5.6-sol` Codex labelled it; recorded as the discrepancy it is rather than resolved** | REQUEST CHANGES — 4 ANSWERED, 1 REWORDED, 6 PARTIAL, 1 blocker and 6 new findings | |
| +| 3 | none yet | — | |
| + |
| +§1.2 records what the draft-1 reviews changed; §1.3 what the draft-2 |
| +review changed. **Draft 3 has not been reviewed.** |
| + |
| +Numbers in §7 are now measured rather than asserted, and recorded in |
| +`docs/BENCHMARKS.md` with the machine and command. Doing that found draft |
| +2's FTS figure wrong by half. |
| --- |
| @@ -59,7 +64,9 @@ embedding is a plaintext cell in a costume — §3.1. |
| | 6 | **Embeddings and FTS5 postings are content.** They never reach replicated SQLite or the bucket unsealed. | Treating a vector as a derived artefact outside §0's scope, on the grounds that no plaintext bytes appear | Fable trust pass, adopted 2026-09-03 | |
| | 7 | **Every SIBLING call is a pull. memoria exposes no sibling-facing surface.** Its ingest endpoint serves external connectors only, which are memoria-owned content, not a sibling push. | An outbox per sibling; and draft 1's §9, which specified the push surface this decision forbids | Fable contract pass; corrected after review | |
| | 8 | **Provenance or deletion, at `(id, revision)` granularity.** A derived artefact whose provenance stops resolving *or whose source changed* is deleted or regenerated. memoria never fine-tunes. | Draft 1's id-level rule, which left "the Q3 target is X" standing after the source said Y | Fable ingestion pass; corrected after review | |
| -| 9 | **Name: `memoria`.** House convention is plain Spanish office nouns. A *memoria* is both a memory and, in Spanish office usage, a written record. | `archivo`; `cerebro`; `base`; anything English | Paul, 2026-09-03 | |
| +| 10 | **memoria never does arithmetic. A number comes from a cell, or from Sheets' engine via S6.** S6 takes an enumerated function over cell refs — never expression text — so memoria is never a formula author. | memoria computing and showing its working (a second calculator, arrived at one `mean` at a time); writing a formula into a scratch sheet (a write, forbidden by §6.4) | Paul asked for numeric questions, 2026-09-03; shape ruled by Fable and GPT Sol independently, converging | |
| +| 11 | **Admission is derived from the siblings, not from a members table memoria maintains.** A person may enter memoria if at least one sibling says they are a member of this team. | Draft 2's own members table, which left a person removed everywhere else still able to read memoria-owned content | after the draft-2 review, 2026-09-03 | |
| +| 12 | **Name: `memoria`.** House convention is plain Spanish office nouns. A *memoria* is both a memory and, in Spanish office usage, a written record. | `archivo`; `cerebro`; `base`; anything English | Paul, 2026-09-03 | |
| ### 1.1 The proposal review, and what it killed |
| @@ -136,6 +143,62 @@ same contradiction between §9 and decision 7, and the same over-claim in |
| that the README was false about egress volume (§3.2) — the single most |
| product-damaging finding of the two. |
| +### 1.3 What the draft-2 review changed |
| + |
| +Draft 2 was reviewed by GPT Sol via Codex. It confirmed five fixes as |
| +answered (the search bypass, the pull/push contradiction, the barrier |
| +count, the rotation protocol, the corpus-count inconsistency) and found |
| +one blocker plus six live findings. Draft 3 answers them as follows. |
| + |
| +**The blocker is not mine to close, and that is the honest position.** |
| +memoria declares the suite one trust domain and pulls plaintext bodies |
| +from siblings. Three source specs currently forbid exactly that — home |
| +§2.2 *"Never federates. Data does not cross an instance boundary."*, |
| +Sheets PRODUCT.md principle 3 *"Never design a flow that implies data can |
| +move to another instance"*, Docs §8.2 *"'Sharing is membership' forbids |
| +data crossing the instance boundary."* **memoria cannot overturn another |
| +app's settled decision in its own table.** Decision 1 is Paul's ruling and |
| +stands, but it binds Oficina only once the source specs adopt it. Threads |
| +are open (§13); until they are answered, slice 1 is blocked on agreement, |
| +not on code. |
| + |
| +**"Nothing left to reconcile" was still false.** S2 fixed sibling |
| +content, but admission used memoria's own members table and so did |
| +memoria-owned content — so a person removed from Sheets and Docs, and left |
| +in memoria, could still read the team's Notion, Drive, mail and uploads. |
| +**Fixed by decision 11:** admission is derived from the siblings. |
| + |
| +**A stale revision is not a denial.** Draft 2 mapped it to |
| +`NOT_VISIBLE`, which §4.1 silently omits — recreating the |
| +plausible-empty-list failure the same draft claimed to have removed. |
| +**Fixed by adding `STALE`** (§4.1), which withholds loudly and refetches. |
| + |
| +**S3 could not do what it promised.** It took `(id, revision)` with no |
| +asker, yet claimed the sibling "re-runs its grants join at fetch time". |
| +There was no person in the call. **Fixed:** S3 carries the person ref too |
| +(§6.1). |
| + |
| +**Derived artefacts had no algorithm for external or mixed provenance** — |
| +S2 exists only at siblings. **Fixed** in §5.2. |
| + |
| +**Restore had no durable deletion state.** Replaying tombstones from a |
| +restored backup replays a log that predates the deletion, and retiring the |
| +superseded snapshot leaves a restored database pointing at a blob that is |
| +gone. **Fixed** in §7.4. |
| + |
| +**Live authorization had no availability design** — no deadline, no |
| +concurrency policy, no partial-result contract, on the product's primary |
| +path. **Fixed** in §6.5, and it is the finding most likely to change the |
| +product rather than the spec. |
| + |
| +**The gates were weaker than the claims.** Gate 5 proved an |
| +authorization call happened, which stays green with a constant `VISIBLE` |
| +or the wrong person ref. **§10 rewritten** accordingly. |
| + |
| +One finding was withdrawn by the reviewer on its own second look (S2 |
| +transport failures needing a separate finding), and its substance is |
| +folded into gate 6 anyway. |
| + |
| --- |
| ## 2. What memoria is |
| @@ -332,6 +395,13 @@ draft-1 version was false about egress volume, which PRODUCT.md principle |
| > With the assistant switched off, nothing leaves this instance — |
| > memoria searches your content where it sits. |
| > |
| +> **What you connect, memoria keeps.** Disconnecting a Notion space or a |
| +> Drive folder, or revoking our access to it, stops memoria seeing new |
| +> changes — it does not remove what memoria already holds. That copy |
| +> stays readable to your team until someone deletes it *in memoria*. |
| +> Documents in Sheets and Docs work the other way round: those apps stay |
| +> in charge, and memoria asks them on every search. |
| +> |
| > **Switching the assistant on sends your content to the provider named |
| > in your settings — not just the passages behind an answer, but every |
| > document in the corpus, once, when it is indexed.** That is how the |
| @@ -352,18 +422,38 @@ Every path that would speak a passage originating in a sibling app calls |
| ``` |
| S2(person_ref, [(item_id, revision), …]) |
| - → per item: VISIBLE | NOT_VISIBLE | UNKNOWN |
| + → per item: VISIBLE | NOT_VISIBLE | STALE | UNKNOWN |
| ``` |
| The sibling answers from the grants join it already maintains — Sheets |
| §4.5, "rows this team can see, plus rows granted to me individually, |
| minus rows I have been removed from". memoria evaluates none of it. |
| -**Three-valued, deliberately.** `UNKNOWN` is what a hibernating or broken |
| -sibling returns, and it is *not* `NOT_VISIBLE`. Collapsing them is how a |
| -suite produces "a plausible empty list, not an error" — the failure shape |
| -sheets-core AGENTS.md names as the worst available. `UNKNOWN` triggers |
| -loud withholding (§8.1); `NOT_VISIBLE` silently omits, correctly. |
| +**Four-valued, and each value exists because collapsing it into another |
| +produces a plausible empty list — the failure shape sheets-core AGENTS.md |
| +names as the worst available.** |
| + |
| +| Result | Means | memoria does | |
| +|---|---|---| |
| +| `VISIBLE` | this person may read this revision | render it | |
| +| `NOT_VISIBLE` | this person may not read it | omit it silently — correct, and the only value that may be silent | |
| +| `STALE` | the item exists and is readable, but the sibling has moved past this revision | **withhold loudly and refetch.** Not a denial | |
| +| `UNKNOWN` | the sibling did not answer usefully | withhold loudly | |
| + |
| +**`STALE` is here because draft 2 did not have it**, and mapped a stale |
| +revision to `NOT_VISIBLE` — which is silently omitted. An ingestion delay |
| +therefore looked exactly like "you have no matching results", recreating |
| +the failure draft 2 claimed to have removed. A synchronisation condition |
| +is not an authorization decision and must not share its value. |
| + |
| +**`UNKNOWN` is synthesised by the caller, not only returned by the |
| +sibling.** A sibling that never answers cannot return anything, so the |
| +client maps every non-answer to `UNKNOWN`: timeout, non-2xx, malformed |
| +body, an item missing from the response, a duplicate item, a short |
| +response to a 256-item request. **The decoder is total** — every requested |
| +pair gets exactly one result or the whole call is `UNKNOWN`. Without that |
| +rule a missing entry falls through to omission, which is the silent path |
| +again. Gate 6. |
| **This subsumes membership.** There is no separate "is P a member?" |
| question and no membership synchronisation, because a non-member's items |
| @@ -372,8 +462,8 @@ unreconciled tables of §1.2 are simply not consulted. |
| **Revision matters.** A document that stays team-visible but whose newer |
| revision removed a sentence must not have its stale chunk spoken: a |
| -revision behind the sibling's current one answers `NOT_VISIBLE`, and |
| -memoria re-fetches on the next pull. |
| +revision behind the sibling's current one answers **`STALE`**, and memoria |
| +refetches rather than omitting. |
| **Batch cap: 256 pairs per call**, multiple calls permitted. Draft 1's 64 |
| was tied to answer top-k and did not survive paginated search or |
| @@ -395,12 +485,39 @@ memoria*. §5.1's enumeration diff is a convenience that eventually |
| notices upstream deletions, not an authorization mechanism, and §3.6's |
| claim does not pretend otherwise. |
| -### 4.3 Using memoria at all |
| +### 4.3 Admission is derived from the siblings |
| A person reaches memoria with a home assertion whose `aud` is memoria's |
| -host, and memoria decides admission from its own grants (home §5.5). |
| -Admission is not authorization: an admitted person still sees only what |
| -§4.1 and §4.2 allow. No guests, no anonymous link viewers. |
| +host. **Whether they are admitted is asked of the siblings, not answered |
| +from a table memoria maintains** (decision 11). |
| + |
| +Draft 2 had memoria keep its own members table, and the draft-2 review |
| +showed why that fails. S2 protected sibling content, but admission and |
| +§4.2's memoria-owned content both used memoria's own list — so a person |
| +removed from the team in Sheets and Docs, and not removed in memoria, |
| +could still sign in and read the team's Notion pages, Drive files, |
| +forwarded mail and uploads. The membership drift decision 3 was written to |
| +eliminate survived in exactly the half nobody was looking at. |
| + |
| +**The rule: a person is admitted if at least one sibling answers that they |
| +are a member of this team.** That is one more call, `S7(person_ref) → |
| +MEMBER | NOT_MEMBER | UNKNOWN`, and it is cached for the life of a session |
| +rather than per request, because it changes on the timescale of employment |
| +rather than of editing. |
| + |
| +Three consequences worth stating: |
| + |
| +- **`UNKNOWN` from every sibling refuses the session.** Fail closed: an |
| + unreachable suite means nobody new signs in, which is an outage, where |
| + the alternative is admitting someone nobody can vouch for. |
| +- **A session already minted survives a sibling outage** for its TTL, |
| + exactly as home §2.1 accepts for the suite as a whole. |
| +- **memoria still holds no authorization model.** It holds no answer of |
| + its own — it relays one, and §4.1 still governs every passage after the |
| + door. |
| + |
| +No guests, no anonymous link viewers. Admission is not authorization: an |
| +admitted person still sees only what §4.1 and §4.2 allow. |
| --- |
| @@ -455,13 +572,33 @@ them stops resolving *or moves to a new revision*.** Draft 1's rule was |
| id-level, which left "the Q3 target is X" standing after the source |
| changed to Y — provenance that resolves and is wrong. |
| -**A derived artefact expands to its full provenance before §4.1 runs.** |
| -Speaking a summary means asking S2 about *every* source behind it, and |
| -**every one must return `VISIBLE`**. One `NOT_VISIBLE` withholds the |
| -artefact; one `UNKNOWN` withholds it loudly. Provenance larger than the |
| -batch cap spans several calls; all must pass. Without this a summary is a |
| +**A derived artefact expands to its full provenance before §4.1 runs**, |
| +and **every source must pass, under whichever authority owns it.** One |
| +failure withholds the whole artefact. Without this a summary is a |
| laundering channel for a single downgraded input. |
| +Draft 2 said this and specified it only for siblings, which the draft-2 |
| +review caught: S2 exists nowhere else, so an external source had no check |
| +at all and a *mixed* artefact had no rule for combining two authorities. |
| +The full algorithm: |
| + |
| +1. **Partition the provenance** by owner — sibling pairs on one side, |
| + memoria-owned ids on the other. |
| +2. **Sibling pairs go to S2**, batched, cap 256 per call, as many calls as |
| + the provenance needs. `VISIBLE` for every pair or the artefact is |
| + withheld. `STALE` on any pair means the artefact is out of date, not |
| + forbidden: regenerate it, do not speak it. |
| +3. **memoria-owned ids are checked against memoria's own grants** — which |
| + for that content is the authority, not a copy (§4.2). Existence plus |
| + team-visibility in memoria is the whole test. |
| +4. **A mixed artefact needs both to pass.** There is no precedence rule |
| + and no fallback: authorities are ANDed, because an artefact is a claim |
| + about all its sources at once. |
| +5. **Provenance that cannot be partitioned** — a source whose owner is |
| + unrecorded — withholds the artefact and is logged as a defect. A |
| + derived artefact that cannot say where it came from is decision 8's |
| + unresolvable-provenance case and gets deleted, not served. |
| + |
| **Where derived artefacts live:** memoria's own store, sealed like any |
| chunk, indexed and searchable — and therefore subject to §4.1 through |
| their provenance, never on their own account. They exist only with AI on, |
| @@ -505,10 +642,12 @@ memoria pulls on its own wake and on each question. It runs no timers. |
| | # | Call | Notes | |
| |---|---|---| |
| | S1 | Change feed | Cursor is an opaque position in the sibling's event log, issued by the sibling, stored per consumer. A row is `(item_id, revision, gone, touched_at)` — **no title, no content, no visibility flag** — so it passes the plaintext-grep gate unaided. Retention is a **per-consumer TTL** (home §9.1: "any 'prune when all consumers have caught up' is unbounded in a hibernating fleet"). Past TTL, or a restore-rewound cursor, takes a full re-announce. | |
| -| S2 | Authorization re-check | `(person_ref, [(item_id, revision)]) → VISIBLE / NOT_VISIBLE / UNKNOWN`. §4.1 specifies it. Cap 256 pairs. | |
| -| S3 | Content fetch | `(id, revision)` → plaintext body over TLS into memoria's RAM only. The sibling re-runs its grants join at fetch time and **404s, never 403s**. | |
| +| S2 | Authorization re-check | `(person_ref, [(item_id, revision)]) → VISIBLE / NOT_VISIBLE / STALE / UNKNOWN` per item. §4.1 specifies it, including the total decoder. Cap 256 pairs. | |
| +| S3 | Content fetch | **`(person_ref, id, revision)`** → plaintext body over TLS into memoria's RAM only. The sibling re-runs its grants join and **404s, never 403s**. | |
| | S4 | Tombstone | *a feed row.* `gone` covers deletion and departure from team-visibility, indistinguishably. | |
| | S5 | Consumer expiry | *no call.* TTL expiry is the decommission verb. | |
| +| S6 | Aggregate | `(person_ref, fn, [(sheet, tab, row, col, revision)…]) → {value, display, format_code}` or `STALE(cell)` / `NOT_VISIBLE` / `UNKNOWN` / `NON_NUMERIC(cell)` / `MIXED_UNITS`. §8.7. Cap 64 cells. Never cached. | |
| +| S7 | Membership | `(person_ref) → MEMBER / NOT_MEMBER / UNKNOWN`. §4.3's admission relay. Cached per session, not per request. | |
| **Draft 1's S1 row carried a `visible-to-team` field, and that was a |
| leak.** With it, a `visible=false, gone=false` row tells memoria that a |
| @@ -516,10 +655,22 @@ guest-only item exists, at what revision, and when it was touched — |
| exactly what S3's "404s, never 403s" was designed to hide, and unnamed in |
| the leak list. Removed: departure from team-visibility is simply `gone`. |
| -**"A stale index costs a miss, never a leak" — narrowed.** Draft 1 stated |
| -it unconditionally and it was false for retraction-by-edit. It holds now |
| -only because S2 carries the revision (§4.1): a stale chunk fails its |
| -re-check. The two clauses must travel together. |
| +**"A stale index costs a miss, never a leak" — narrowed twice.** Draft 1 |
| +stated it unconditionally and it was false for retraction-by-edit. Draft 2 |
| +fixed that with a revision on S2 but mapped the failure to `NOT_VISIBLE`, |
| +which is silently omitted — so it cost a *silent* miss, which is the |
| +failure it was supposed to prevent. It holds now only because S2 carries |
| +the revision **and** answers `STALE` distinctly. All three clauses travel |
| +together or the sentence is false. |
| + |
| +**S3 carries the person, and draft 2's did not.** Draft 2's S3 took |
| +`(id, revision)` and promised the sibling "re-runs its grants join at |
| +fetch time" — with no asker in the call, which the draft-2 review named as |
| +objectively impossible. The alternative considered was an S2-minted |
| +capability that S3 consumes; rejected as a second credential type to |
| +issue, bind and expire, where passing the person ref costs nothing and |
| +keeps one authority. **Every sibling call that touches content carries the |
| +person it is for.** |
| ### 6.2 Authentication, and rotation |
| @@ -589,6 +740,36 @@ Any write. Any sibling item outside the team-visible custody rule. The |
| home in the loop. Content leaving the four hosts, except §8.3's named |
| provider, which is a disclosed exception and not a sibling. |
| +### 6.5 Availability: the deadline, and what a slow sibling costs |
| + |
| +Draft 2 had none of this, and the draft-2 review was right that it is the |
| +finding most likely to change the product rather than the spec. Every |
| +search now calls every contributing sibling, any of which may be cold, and |
| +"withhold loudly" said what to do on failure without saying when failure |
| +is declared. |
| + |
| +- **Deadline: 800 ms per sibling, all siblings called concurrently.** One |
| + page of results is one round of calls, so the page's floor is the |
| + slowest sibling, not their sum. The number is a starting assumption, not |
| + a measurement — §13 carries the obligation to measure it before slice 1 |
| + ships. |
| +- **A sibling past its deadline is `UNKNOWN`**, which withholds its |
| + passages loudly and renders the rest. **Partial pages are the normal |
| + case, not an error path**, and the page names what is missing and from |
| + which app. |
| +- **A cold sibling is a wake, and a wake is slower than a deadline.** The |
| + honest consequence: the first search after a quiet period reliably |
| + withholds. So memoria **fires S7 and a warm-up S2 on its own wake**, |
| + before anyone has searched, to spend that latency where nobody is |
| + waiting. |
| +- **A reliably slow sibling produces permanent withholding**, which is |
| + worse than it sounds: the product's primary path degrades to "Sheets did |
| + not answer" forever, and a reader learns to ignore the notice. There is |
| + no design that fixes this while failing closed. **The stated position |
| + is that this is a suite health problem surfaced by memoria, not a |
| + memoria problem to route around** — and if it happens, the answer is to |
| + fix or drop the source, never to cache its authorization. |
| + |
| --- |
| ## 7. The index |
| @@ -644,32 +825,61 @@ one core of this machine, measured at **3.2 GFLOP/s** (pure Go, no SIMD): |
| | 250k × 512 | 256 M | 80 ms | |
| | 1M × 512 | 1.02 G | 320 ms | |
| -**FTS index size.** Measured at **104 MB** for 90k chunks — but on a |
| -synthetic corpus of random 4–9 letter words, whose vocabulary inflates a |
| -postings list relative to English. Real English will be smaller. **Taken |
| -as 104 MB because it is the only measurement in hand, and flagged as |
| -conservative rather than accurate.** Draft 1 asserted 60 MB with no |
| -derivation. |
| - |
| -**Rebuild vs snapshot.** Building that index took **19 s**. Loading a |
| -sealed snapshot does not. **Snapshot, never rebuild.** |
| +**FTS index size — remeasured, and draft 2 was wrong by half.** Draft 2 |
| +reported 104 MB from a synthetic corpus of random letters. The draft-2 |
| +review showed that figure could not describe the stated schema, because a |
| +default FTS5 table keeps its own copy of the inserted text. Measured |
| +properly, on the real corpus, chunked by memoria's own chunker |
| +(`docs/BENCHMARKS.md` carries the machine and command): |
| + |
| +| schema | index | bytes/chunk | build | query | |
| +|---|---|---|---|---| |
| +| `fts5(body, …)` — default | **200 MB** | 2329 | 6.8 s | 1.09 ms | |
| +| `fts5(body, content='', …)` | **49 MB** | 574 | 6.2 s | 1.09 ms | |
| + |
| +**A contentless table is 4× smaller and queries just as fast, but cannot |
| +return column values — so `snippet()` does not work and memoria must build |
| +its own highlight from the sealed chunk.** That is the trade, recorded as a |
| +decision: slice 0 uses the default table because a snippet is free there |
| +and the corpus is tiny; the snapshot design uses contentless, because 49 MB |
| +against 200 MB decides the wake budget. |
| + |
| +**A chunk is smaller than draft 2 assumed.** Measured mean: **202 words / |
| +1325 bytes**, not 400 words / 1.6 KB. The chunker targets 400 words but |
| +folds runts and never straddles a heading, and real documents have |
| +headings every few paragraphs. Every figure here follows from the measured |
| +size. |
| + |
| +**Rebuild vs snapshot.** Building the index took **6.8 s** at 90k chunks. |
| +Loading a sealed snapshot does not. **Snapshot, never rebuild** — above |
| +slice 0, where a rebuild is milliseconds. |
| ### 7.3 The wake budget, and the real ceiling |
| -Snapshot at 90k chunks: 46 MB vectors + 104 MB FTS = **150 MB**, ≈1.67 KB |
| -per chunk. |
| +Snapshot at 90k chunks, measured: |
| + |
| +| design | FTS | vectors | separate text | total | |
| +|---|---|---|---|---| |
| +| default FTS (text is in the index) | 200 MB | 44 MB | — | **244 MB** | |
| +| contentless FTS + the text snippets need | 49 MB | 44 MB | 114 MB | **207 MB** | |
| **Stated assumption, which draft 1 omitted:** bucket read at 50–100 MB/s, |
| AES-GCM at ~1 GB/s. So 150 MB loads in **1.5–3.0 s plus 0.15 s decrypt**. |
| -A **5 s budget** therefore admits 250–500 MB, i.e. **150k–300k chunks**. |
| -Taking the conservative bandwidth: **~150k chunks**. |
| +A **5 s budget** at the conservative bandwidth admits ~250 MB. Against the |
| +measured footprints that is **about 90,000–110,000 chunks**. |
| + |
| +**Both earlier drafts got this wrong, in opposite directions, and the |
| +correction is worth keeping.** Draft 1 said ~100k and called it "the same |
| +ceiling the brute-force arithmetic gives, which is tidy" — a coincidence it |
| +had manufactured. Draft 2 called that out and derived ~150k instead, from |
| +an FTS figure that was wrong by half. So **draft 2 moved the number the |
| +wrong way while claiming to have fixed the arithmetic, and draft 1's |
| +~100k was closer by accident.** Measurement, not rederivation, is what |
| +settled it. |
| -**Draft 1 claimed ~100k and called it "the same ceiling the brute-force |
| -arithmetic gives, which is tidy". Both reviewers showed that was |
| -manufactured**, and they were right twice: 250 MB ÷ 1.67 KB is ~150k, not |
| -100k; and brute-force cosine is nowhere near binding at that size — §7.2 |
| -shows 250k chunks answering in 80 ms. |
| +Brute-force cosine remains nowhere near binding: §7.2 measures 250k chunks |
| +at 80 ms. |
| **The honest conclusion draft 1 missed: the wake budget binds first, and |
| ANN does not help it.** An HNSW graph makes the snapshot *larger*. So the |
| @@ -693,6 +903,21 @@ reviewer nor draft 1 caught. |
| from the tombstone log at wake, before first query. |
| - **A restore from backup takes the re-announce path** and replays |
| memoria-owned tombstones before serving. |
| +- **The tombstone log is durable independently of the database, and this |
| + is what draft 2 was missing.** Restoring a backup from before a deletion |
| + also restores a tombstone log from before it, so replaying that log |
| + replays nothing: the deletion is simply gone. So tombstones are written |
| + to their own append-only object in the bucket, keyed by a monotonic |
| + deletion sequence, retained beyond any restorable backup's age, and |
| + **replayed from the sequence the restored database stops at** rather |
| + than from the log it happens to carry. |
| +- **A snapshot is not retired while any restorable backup could point at |
| + it.** Draft 2's retirement rule created the mirror-image bug: a restored |
| + database referencing a superseded generation finds the blob deleted and |
| + has no index at all. Retirement waits out the backup retention window, |
| + and a snapshot referenced by a missing generation triggers a rebuild |
| + rather than an empty index — an empty index would serve "nothing |
| + matched" over a full corpus. |
| --- |
| @@ -730,7 +955,7 @@ calling no model: |
| | Class | Signal | Route | |
| |---|---|---| |
| | Navigational | "where is the Q3 budget" | **No model.** Result list. | |
| -| Numeric / aggregate | asks for a computed value | **No model.** Cite the cell or range; Sheets is the app of record. | |
| +| Numeric / aggregate | asks for a computed value | **No model does the arithmetic.** Resolve slots, then S6 — §8.7. | |
| | Who knows about X | asks about people | **No model.** Authorship markers on passages that already passed §4.1, aggregated. | |
| | Single-source factual | large top-1 BM25 margin, one document | Cheap model, extractive. | |
| | Synthesis | top-K spread across ≥3 documents | Frontier model. | |
| @@ -813,6 +1038,82 @@ invalidated artefact and is unbudgeted until the feature exists. |
| The money is recurring query tokens by an order of magnitude — the |
| economic argument for §8.2's router and its three model-free classes. |
| +### 8.7 Numeric and aggregate questions |
| + |
| +Paul's motivating example, 2026-09-03: *"what's average revenue in June the |
| +last 3 years"*. Ruled independently by Fable and GPT Sol, which converged; |
| +where they differ, the stricter reading is taken. |
| + |
| +**The tension this resolves.** §2 says memoria never recomputes a number, |
| +because a second calculator eventually disagrees with the first. But no |
| +cell contains "average June revenue over three years", so the question |
| +either gets forbidden or the rule gets read properly. What the rule |
| +protects is not arithmetic — averaging three doubles is deterministic — |
| +it is **selection, units and formatting**: actual versus budget, one year |
| +in thousands, `1234567.833` versus `€1,234,568`. Sheets §5 already names a |
| +second formatter as the same bug class one layer down. |
| + |
| +**So: Sheets computes, memoria never does** (decision 10). S6 takes an |
| +**enumerated function over cell references** — `SUM`, `AVERAGE`, `MIN`, |
| +`MAX`, `COUNT` — and never expression text, so memoria is not a formula |
| +author and cannot grow into one. Rejected alternatives: memoria computing |
| +and showing its working (a second calculator, arrived at one `mean` at a |
| +time); writing a formula into a scratch sheet (a write, forbidden by §6.4, |
| +and a citation that cites itself). |
| + |
| +**Chunking a spreadsheet**, which §13 carried as unsolved. Two chunk kinds |
| +per tab: |
| + |
| +- **One structure chunk per tab** — sheet title, tab name, dimensions, the |
| + header band, the leftmost text column, number formats named in words, |
| + and every date-like header emitted twice, as displayed and canonicalised |
| + (`Jun 2024` *and* `2024-06`). This is the chunk that carries the |
| + vocabulary. |
| +- **Region row chunks**, one per row of a header-plus-contiguous-block |
| + region, rendered as **facts rather than positions**: *"FY2024 › P&L › |
| + Revenue: Jan 2024 = €1,200,000; … Jun 2024 = €1,350,000"*. Row label is |
| + the nearest text run to the left, column label the header band above, |
| + concatenated top-down. Sheet and tab prepend to every chunk. Values are |
| + the **display string from S3**, never a double memoria formatted. |
| + |
| +That is what makes `revenue`, `June` and `2024` literal BM25 terms for a |
| +cell whose content is the bare number `1350000`. **Header detection is |
| +deterministic and conservative: where competing header bands make the |
| +meaning ambiguous, index the alternatives as structure and do not |
| +manufacture a fact.** |
| + |
| +**Retrieval, because top-k cannot prove completeness.** Decompose into |
| +named slots — `(Revenue, June, 2024)`, `(…, 2025)`, `(…, 2026)` — and |
| +resolve each by exact metadata lookup against the structure catalogue, |
| +using top-k only to find candidate regions. Recall becomes a per-slot |
| +count: |
| + |
| +- exactly one current, authorized operand → filled; |
| +- zero → a named gap; |
| +- **two or more → ambiguity, never extra evidence**, and never collapsed. |
| + |
| +**S6 is called only when every slot is filled with compatible units.** Any |
| +gap, ambiguity, `STALE` or `UNKNOWN` yields the slot table with citations |
| +and **no number**: *"Found June 2024 and June 2023; nothing matched June |
| +2025. No average shown."* A two-of-three average is the plausible-wrong |
| +answer that §8.1's loud withholding exists to prevent, and it is worse |
| +than an empty result because it is actionable. |
| + |
| +**The answer names its basis**, including which years "last three" meant, |
| +because that is ambiguous on any given date. |
| + |
| +**What must not be built.** A retrieval-augmented calculator: retrieve |
| +top-k, let a model pick numeric strings, average them, present it as |
| +grounded. Both reviewers named this unprompted. It stacks three silent |
| +errors — incomplete recall, wrong operand selection, arithmetic and |
| +formatting outside Sheets — and every one of them cites correctly while |
| +being wrong. Also forbidden: memoria parsing a display string back into a |
| +number (`"€1.2M"`), which is a second parser and therefore a second |
| +calculator; and caching an S6 result, which outlives its cells. |
| + |
| +Gate 14 extends: **every digit in an answer sentence traces to a cited |
| +passage or an S6 result.** |
| + |
| --- |
| ## 9. Surfaces |
| @@ -869,11 +1170,18 @@ renders on its own origin. The home never sees `q`. |
| over S1 rows, asserting the absence of the field draft 1 had. |
| 4. **No content in logs**, covering query text, answer text, cited |
| snippets and any model-call ledger. |
| -5. **Every render path calls §4.1.** Enumerate the passage-rendering |
| - entry points and assert each one's authorization call — the gate that |
| - would have caught gap 2. A new render path with no S2 call fails. |
| -6. **`UNKNOWN` is never treated as `NOT_VISIBLE`.** A sibling returning |
| - `UNKNOWN` produces a stated withholding, never a shorter list. |
| +5. **Two people get different answers.** End-to-end, two assertions, one |
| + sibling holding a document granted to one of them: the passage renders |
| + for one person and not the other, and the sibling's own grants join is |
| + what decides. Draft 2's version asserted only that an authorization |
| + call happened, which the draft-2 review correctly called weaker than |
| + the claim — it stays green with a constant `VISIBLE`, with the wrong |
| + person ref, or with authorization performed *after* disclosure. |
| +6. **Every S2 result is handled, and the decoder is total.** `UNKNOWN`, |
| + `STALE` and `NOT_VISIBLE` each produce their specified behaviour; |
| + `STALE` withholds loudly rather than omitting; and a timeout, non-2xx, |
| + malformed body, missing item, duplicate item or short response all map |
| + to `UNKNOWN` rather than falling through to omission. |
| 7. **An enumeration diff distinguishes empty from failed**, and deletes |
| nothing on a timeout, credential error or partial page. |
| 8. **Circuit breaker** — a wholesale `NOT_VISIBLE` or 404 response states |
| @@ -892,6 +1200,26 @@ renders on its own origin. The home never sees `q`. |
| 15. **AI-off floor** — every affordance in §8.1 works scriptless with the |
| provider unreachable, **and no egress occurs**, asserted by a network |
| guard rather than by inspection. |
| +16. **A sibling credential is refused at the connector routes**, and a |
| + connector credential at nothing else. §9.1 asserts the two sets are |
| + disjoint; nothing tested it. |
| +17. **No sibling-facing endpoint exists.** Enumerate memoria's routes and |
| + fail any inbound route not on the person-facing or connector list — |
| + the regression gate for the push surface draft 1 shipped by accident. |
| +18. **External-source authorization and disclosure.** A memoria-owned |
| + document is governed by memoria's grants, an upstream revocation does |
| + *not* remove it, and the copy the README promises is deletable is |
| + deletable. |
| +19. **Mixed-provenance authorization** — a derived artefact with one |
| + sibling source and one memoria-owned source requires both, spans |
| + multiple S2 batches, and withholds when either fails. |
| +20. **The wake budget and the corpus ceiling**, run against a real corpus |
| + rather than asserted, failing when the measured footprint at the |
| + stated chunk count exceeds the budget. §7's numbers were wrong twice |
| + with no gate to notice. |
| +21. **Admission is relayed, not held.** A person no sibling calls a member |
| + is refused, and `UNKNOWN` from every sibling refuses rather than |
| + admits. |
| --- |
| @@ -944,13 +1272,28 @@ rows. |
| ## 13. Open |
| -- **§Review pass on this draft.** Draft 1 was reviewed by GPT Sol and |
| - Fable; draft 2 answers them and has been reviewed by nobody. |
| +- **§Review pass on draft 3.** Drafts 1 and 2 were reviewed; this one has |
| + not been. |
| +- **BLOCKER, and not mine to close: the source apps must adopt decision |
| + 1.** home §2.2, Sheets PRODUCT.md principle 3 and Docs §8.2 each |
| + currently forbid content crossing an instance boundary. Paul ruled the |
| + suite is one trust domain, and memoria cannot enact that in its own |
| + table. `oficina/docs` discussion 33 is open; Sheets and home need the |
| + same. **Slice 1 is blocked on agreement, not on code.** |
| +- **Measure S6 and S2 latency against a cold sibling** before slice 1 |
| + ships. §6.5's 800 ms deadline is an assumption, and it decides whether |
| + the primary search path degrades to permanent withholding. |
| - **Does CARLOS provide non-replicated scratch storage?** Unverified, and |
| §7.1 needs it or pays a download on every wake. |
| -- **The FTS index size is measured on a synthetic corpus** whose random |
| - vocabulary inflates it. Re-measure on real English before §7.3's |
| - ceiling is trusted as anything but conservative. |
| +- **The FTS index is now measured on the real corpus** (§7.2, |
| + `docs/BENCHMARKS.md`), but on a *cycled* one — repeated vocabulary makes |
| + the postings list smaller than genuinely varied English. The ceiling is |
| + therefore optimistic, not conservative. |
| +- **Bucket bandwidth and AES-GCM throughput are still assumptions** |
| + (50–100 MB/s, ~1 GB/s), and the ceiling is most sensitive to them. So is |
| + the rest of a cold start: opening SQLite, allocating the vector matrix, |
| + parsing the snapshot and building id maps are all outside the 5-second |
| + budget as computed. |
| - **OpenRouter's embeddings endpoint** is reported OpenAI-compatible at |
| `/api/v1/embeddings`, but the documentation page 404'd when checked. If |
| embeddings and chat both route through OpenRouter, **two parties see |
| @@ -961,10 +1304,10 @@ rows. |
| - **A `carlos/platform` discussion** noting the app boundary is the |
| implementer's to declare, and Oficina declares it at the suite — |
| informational, not a doctrine proposal (decision 2). |
| -- **Chunking is unspecified**, and §8.2's numeric route depends on it: a |
| - spreadsheet is not prose, and a 400-token chunk of a grid may be |
| - meaningless. Sheets likely needs a per-tab or per-region strategy Docs |
| - does not. |
| +- **Spreadsheet chunking is now specified (§8.7) but not built or tested |
| + against a real sheet.** The structure-chunk and region-row design is two |
| + models' reasoning, not a measurement, and header detection on real |
| + spreadsheets is where it will meet trouble. |
| - **S2's latency budget.** Every query now makes a live call to each |
| contributing sibling, any of which may be cold. Measure before slice 1 |
| ships; if a cold wake is seconds, the floor needs a stated behaviour |