amadan
Spec §8.8: a deterministic lookup box above the results
Paul, 2026-09-14: searching "phone number" should put the number at the top of the page, and the constraint given with it is that as much ordering of the data as possible happens deterministically before any inference runs on it. So §8.8 is §8.2's fourth model-free class, recorded as decision 15: a validated fact — eircode, phone, IBAN, VAT, email, URL — extracted by recogniser, displayed as the document's own span with the document's own label, above the results and never instead of them. Three rulings do the work. Extraction runs twice, at ingest for reachability and at query time for the box, and the query-time half runs over Page.Results so §8.4's invariant holds structurally rather than by care. The ordering rule-then-extract-then-count is load-bearing: an ambiguity counted before §4.1 reports the existence of a passage the reader was just refused. And ambiguity enumerates here where §8.7 refuses, because two phone numbers are two right answers to an underspecified question while a two-of-three average is a wrong number. A sealed facts table is rejected: its kind column would be a plaintext inventory of which kinds of sensitive data the team holds, which is the bare-SHA external_id failure with a different name. Gates 34-37, with the author's mutations listed as a starting point and labelled as not counting (§1.9). Dependants updated rather than left: §8.2's class count, §8.6's two references to it, §12's deferred rows. §13's record of what Paul said on 2026-09-04 is annotated, not rewritten.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 file changed,
+278
−5
docs/superpowers/specs/2026-09-03-memoria-design.md+278 −5
diff --git a/docs/superpowers/specs/2026-09-03-memoria-design.md b/docs/superpowers/specs/2026-09-03-memoria-design.md| index edc74a6..6cab445 100644 |
| --- a/docs/superpowers/specs/2026-09-03-memoria-design.md |
| +++ b/docs/superpowers/specs/2026-09-03-memoria-design.md |
| @@ -108,6 +108,7 @@ embedding is a plaintext cell in a costume — §3.1. |
| | 12 | **Name: `memoria`.** House convention is plain Spanish office nouns. A *memoria* is both a memory and, in Spanish office usage, a written record. | `archivo`; `cerebro`; `base`; anything English | Paul, 2026-09-03 | |
| | 13 | **The provider is TensorX, and memoria supports exactly one.** A team owner's choice is on or off, not which — a configurable base URL would let an owner reinstate a proxy while the answer still printed "one named provider". Chosen for EU jurisdiction and open weights, criteria decision 4 did not state, at a known cost in id-pinning and frontier quality (§8.3). | OpenRouter (a proxy: two parties see content); OpenAI direct (better id-pinning, worse jurisdiction and cost) | Paul, 2026-09-04, both halves — *"we go with exactly one for now: TensorX. If we want to swap that later we can."* Swapping the *generation* provider is a code change and a review, not a setting; swapping the *embedding* provider is that plus a full corpus re-embed unless the new host serves the same open weights (§8.3, §13) | |
| | 14 | **memoria is a search page, not a chat.** One GET search form, `?q=`, one query in and one page out, with `/documents/{id}` as the only other passage surface. This follows from decision 5, not from decision 3 — §8.4's invariant is what decision 3 requires, and a conversational surface could satisfy it. | A chat UI with citations; a threaded follow-up. Deferred to §12, not forbidden — the invariant forbids the leak, this row chooses the shape | Spec author, 2026-09-04, answering a question Paul asked; **confirmed by Paul 2026-09-05**, parking the conversational surface in §12 rather than forbidding it | |
| +| 15 | **The lookup box calls no model, ever.** A validated fact — eircode, phone, IBAN, VAT, email, URL — is extracted deterministically and shown above the results with its citation. A kind with no validator is not a kind. §8.8 | A cheap extractive model picking the span (§8.7's retrieval-augmented calculator with a different noun); a model composing the label; postal address as a kind, which has nothing to validate | Paul, 2026-09-14 — *"do as much deterministic ordering of the data before we go doing inference on it"* | |
| **Three of these are now suite rulings, and this spec cites them rather |
| than restating them.** Decision 1 is **suite ruling 1**, decision 3 is |
| @@ -2365,14 +2366,16 @@ a provider — and per §3.2, nothing leaves the instance in this mode. |
| ### 8.2 Routing, decided after retrieval |
| -Retrieval is free, so the router runs on results. Five classes, three |
| -calling no model: |
| +Retrieval is free, so the router runs on results. Six classes, four |
| +calling no model — and §8.8's arrived last precisely because it is the |
| +cheapest: the deterministic classes are the ones worth growing. |
| | Class | Signal | Route | |
| |---|---|---| |
| | Navigational | "where is the Q3 budget" | **No model.** Result list. | |
| | Numeric / aggregate | asks for a computed value | **No model does the arithmetic.** Resolve slots, then S6 — §8.7. | |
| | Who knows about X | asks about people | **No model.** Authorship markers on passages that already passed §4.1, aggregated. | |
| +| Lookup | names a validated fact kind — "phone number", "eircode", "VAT number" | **No model.** Recognisers over the ruled results; the box, or nothing — §8.8. | |
| | Single-source factual | large top-1 BM25 margin, one document | Cheap model, extractive. | |
| | Synthesis | top-K spread across ≥3 documents | Frontier model. | |
| @@ -2680,14 +2683,17 @@ below its $0.02–0.10 range rather than sitting in it. |
| Most of that came from §8.2 pinning dated ids from one family, not from the |
| provider switch, and none of it is a reason to route more queries to the |
| -frontier tier — §8.2's three model-free classes remain the design, and this |
| +frontier tier — §8.2's model-free classes remain the design, and there are |
| +four of them since §8.8, not three. This |
| section's closing sentence still holds. |
| Derived-artefact regeneration (§5.2) adds a synthesis-class call per |
| invalidated artefact and is unbudgeted until the feature exists. |
| The money is recurring query tokens by an order of magnitude — the |
| -economic argument for §8.2's router and its three model-free classes. |
| +economic argument for §8.2's router and its four model-free classes — and |
| +§8.8, which added the fourth, is the only one of them that reduces the |
| +query count rather than routing it cheaper. |
| ### 8.7 Numeric and aggregate questions |
| @@ -2886,6 +2892,268 @@ calculator; and caching an S6 result, which outlives its cells. |
| Gate 14 extends: **every digit in an answer sentence traces to a cited |
| passage or an S6 result.** |
| +### 8.8 The lookup box: deterministic extraction above the list |
| + |
| +Paul's motivating example, 2026-09-14: a search for `phone number` should |
| +put the answer at the top of the page — *"Tito office number: +353 (0)1 |
| +547 7854"* — and the same for an eircode, a postcode, a VAT number. And |
| +his constraint, given in the same breath, which is the whole of this |
| +section's shape: **do as much deterministic ordering of the data as |
| +possible before any inference runs on it.** |
| + |
| +**This is §8.2's fourth model-free class.** That table already has three — |
| +navigational, numeric/aggregate, who-knows-about-X — and the third is the |
| +closest precedent: it answers a real question from structured markers on |
| +passages that already passed §4.1, with no model anywhere. A lookup box is |
| +that shape applied to facts a regular expression can *validate* rather |
| +than merely spot. |
| + |
| +**Decision 15: the lookup box calls no model, ever.** Not a cheap one, not |
| +an extractive one. The kinds it serves are the kinds that have a |
| +validator; a kind with no validator is not in the box and is not the |
| +box's problem (§12). The moment a model picks which span to show, every |
| +argument below collapses into §8.7's forbidden retrieval-augmented |
| +calculator with a different noun. |
| + |
| +It does not touch §3.2. No egress, no provider, no embedding — the box |
| +works on an instance with AI switched off, which is decision 5's floor and |
| +therefore the whole product today. |
| + |
| +#### Two extractions, in two places, for two reasons |
| + |
| +Conflating these is where this goes wrong, so they are named separately |
| +and neither is optional. |
| + |
| +**At ingest, to make a chunk reachable.** A document that says `D02 XY45` |
| +and never says the word "eircode" cannot be retrieved by someone searching |
| +for one. So the recognisers run over chunk text at ingest and their output |
| +is folded into what the in-memory index holds for that chunk: **the kind |
| +words** (`eircode`), and **the canonicalised value** (`015477854` beside |
| +`+353 (0)1 547 7854`). That second half is §8.7's spreadsheet rule |
| +verbatim — *"every date-like header emitted twice, as displayed and |
| +canonicalised"* — and it is what makes a phone number findable by someone |
| +who types it without the spaces. |
| + |
| +**At query time, over the ruled results, to fill the box.** The box is |
| +memoria speaking a passage, so §8.4's invariant binds it: every passage in |
| +a response passed §4.1 **in the request that produced it**. The box is |
| +therefore derived from `Page.Results` — after `Store.rule` — and from |
| +nothing else. There is no second retrieval path, so there is no second |
| +authorization path to get wrong. |
| + |
| +**The ordering is load-bearing and it is the ruling of this subsection: |
| +rule, then extract, then count candidates.** Extracting before ruling |
| +would let a withheld passage contribute a candidate, and the ambiguity |
| +count is the leak: two candidates where the reader may see one reports the |
| +existence of a passage §4.1 just refused them. Counting after ruling gives |
| +the right answer for the right reason — for that reader, at that moment, |
| +there is one. |
| + |
| +#### Why there is no facts table |
| + |
| +The obvious design is a sealed `facts` table written at ingest, keyed by |
| +chunk, read back at query time. It is rejected on three grounds, and the |
| +third is the one that decides it: |
| + |
| +- It is **derived state**, so it inherits AGENTS.md's ordering rule in |
| + both directions and §5.2's invalidation — a new way for the index and |
| + the database to disagree, added to buy nothing. |
| +- It is **a migration and a schema** for a computation that costs |
| + microseconds. Recognisers over twenty-five chunks of ~200 words is less |
| + work than the twenty-five AEAD opens that hand them the text. |
| +- **Its `kind` column is a plaintext inventory of which kinds of sensitive |
| + data this team holds.** A row reading `kind='iban'` tells anyone with |
| + the bucket that the corpus contains bank details, without opening one. |
| + That is the same object as the bare-SHA `external_id` §5 already |
| + retired: an identifier derived from plaintext that answers a question |
| + about the plaintext. Sealing the value and leaving the kind readable is |
| + precisely the half-measure that failed there. |
| + |
| +Computing at query time means **nothing new exists at rest at all**. Gate |
| +1's plaintext grep and `TestIndexIsNotOnDisk` already cover the only two |
| +places anything could land, and they cover the new index column for free, |
| +which is why this section adds no gate for it. |
| + |
| +#### The recognisers, and what is not one |
| + |
| +A kind is in the box when a query word names it **and** a candidate |
| +validates. Validation is not pattern-spotting: an eircode has a fixed |
| +routing-key alphabet, an IBAN has a mod-97 check, an Irish VAT number has |
| +a checksum. What validates is what may sit above the results. |
| + |
| +| Kind | Primed by | Validated by | |
| +|---|---|---| |
| +| `eircode` | `eircode`, `postcode`, `post code` | routing key from the published letter set, plus four characters | |
| +| `phone` | `phone`, `telephone`, `tel`, `mobile`, `phone number` | E.164, or an Irish national format; length and prefix bounds | |
| +| `email` | `email`, `email address`, `e-mail` | `local@domain`, domain carrying a dot, no whitespace | |
| +| `url` | `url`, `link`, `website` | scheme in {`http`, `https`} plus a host | |
| +| `iban` | `iban`, `bank details`, `account number` | country prefix, length by country, mod-97 | |
| +| `vat` | `vat`, `vat number` | Irish VAT format and its checksum | |
| + |
| +**`address` on its own primes nothing**, and the reason is the interesting |
| +one: it is ambiguous between a postal address and an email address, and |
| +only one of those has a validator. `email address` primes `email`. A |
| +postal address has no checksum, no fixed grammar and no length — there is |
| +nothing to validate, so under decision 15 it is not a kind. §12 carries |
| +it. |
| + |
| +**`number` on its own primes nothing either.** The lexicon maps |
| +distinctive words and phrases; generic words count only in combination. |
| +This is `query.openers`' trade made again and in the same direction: a |
| +missed kind costs a box that would have helped, a wrong kind costs a box |
| +that asserts something nobody asked about. A generous lexicon is |
| +affordable *only* because the box still requires a validated candidate — |
| +"mobile app" primes `phone` and then finds no phone number and renders |
| +nothing. |
| + |
| +**It complements Enter-priming rather than competing with it.** `query. |
| +IsQuestion` declines to prime a result for anything opening with an |
| +interrogative, so "what's the office phone number?" navigates nowhere |
| +today. Those are disproportionately the queries a lookup box answers. The |
| +two heuristics divide the same space and neither needs to know about the |
| +other. |
| + |
| +#### What the box displays, and where every word of it comes from |
| + |
| +**The value is the document's own span, byte for byte.** `+353 (0)1 547 |
| +7854` is displayed as `+353 (0)1 547 7854`. Canonicalisation happens for |
| +matching and for the index, never for display. This is §8.7's rule one |
| +noun over — *a second formatter is the same bug class as a second |
| +calculator* — and it has teeth here: normalising to E.164 drops the |
| +extension a document wrote after the number, and reformatting an IBAN |
| +regroups digits a bank grouped deliberately. |
| + |
| +**The label is taken from the document and is never composed.** Three |
| +sources, in order, and the first that applies wins: |
| + |
| +1. **The run to the left of the value on its own line**, ending in a |
| + separator (`:`, `—`, `-`, `=`, tab) — `Office: +353 …` yields `Office`. |
| + **Capped at 40 characters, and the value must not be mid-sentence**: |
| + without that, a paragraph mentioning a number yields the preceding |
| + clause as a label, which is a sentence memoria composed out of |
| + fragments and presented as a fact's name. |
| +2. **The chunk's heading trail** — `Tito › Contact`. |
| +3. **The document title.** |
| + |
| +Conservative in §8.7's sense: where the line offers no short label, fall |
| +back rather than manufacture one. Nothing here title-cases, expands or |
| +rewords; a label the document did not write is a label a model wrote. |
| + |
| +So Paul's example renders as `Tito › Contact — Office: +353 (0)1 547 |
| +7854`, carrying the same `/documents/{id}#p{n}` link every result carries. |
| +Less shapely than the mock-up, and every character of it is checkable. |
| + |
| +**The box never appears without its citation, and never instead of the |
| +list.** §8.4: citations with an optional answer, never the reverse. It is |
| +one element above the results, exactly where the 🤖 paragraph will go, and |
| +subject to the same sentence — it is a garnish on the search page, which |
| +is the product (decision 5). |
| + |
| +**Zero candidates renders nothing at all** — not "no phone number found". |
| +The corpus is not the world, and a confident absence is this codebase's |
| +own worst failure shape. The results below already say what was searched |
| +and how much of it there is. |
| + |
| +#### Ambiguity enumerates here, where §8.7 refuses |
| + |
| +§8.7's slot rule is *"two or more → ambiguity, never extra evidence, and |
| +never collapsed"*, and it resolves that by showing no number. **The |
| +lookup box shows all of them instead, and the difference is not |
| +inconsistency.** |
| + |
| +An average built from two of three operands is a *wrong number*: the |
| +question had one answer and memoria produced a different one. Two phone |
| +numbers in a corpus are two *right answers* to an underspecified question. |
| +§8.7's ambiguity is incompleteness, which can only be refused; this |
| +section's is underspecification, which is honestly served by showing the |
| +reader both, labelled and cited, and letting them see that the corpus |
| +holds two. |
| + |
| +**Never by picking.** Not highest BM25, not first, not longest. The rank |
| +ordering exists to sort a result list and carries no claim about which |
| +phone number is the office one. |
| + |
| +**Capped at three, and above the cap there is no box.** A box enumerating |
| +twelve numbers is the result list with extra steps and a false air of |
| +authority; at that point the honest surface is the one already below it. |
| + |
| +#### Injection, and the one place ambiguity pays for itself |
| + |
| +§8.5's mitigation for a poisoned document is that every claim carries an |
| +openable citation, and that it is a partial one. The box makes that |
| +sharper, because it promotes a single document's assertion to the top of |
| +the page with a validated-looking format around it. `Office phone: +353 1 |
| +000 0000` in an uploaded file is a lookup box, and no model was involved |
| +in believing it. |
| + |
| +Two things bound it, and neither is a fix: |
| + |
| +- The value is a cited span of a passage that passed §4.1, so the reader |
| + can open it, and a poisoned document is a document somebody put in this |
| + corpus. |
| +- **The ambiguity rule fails safe against poisoning.** A planted second |
| + number does not replace the real one and does not outrank it — it makes |
| + the box show both, or vanish past the cap. The attack that works against |
| + a single-answer box degrades into a visibly contradictory page here, and |
| + that is a property worth keeping rather than an accident to optimise |
| + away. |
| + |
| +#### One stated limit |
| + |
| +**The box only sees what retrieval returned.** A fact in a document that |
| +does not reach the ruled top-25 is a fact the box does not have, and the |
| +page will show results without it. The ingest-time facts are what make |
| +this rare — an unlabelled eircode is reachable by the word "eircode" that |
| +is nowhere near it — but the facts column is weighted to buy |
| +**reachability, not rank** (see `FactWeight`), so a large corpus can still |
| +push the fact-bearing chunk off the page. |
| + |
| +Named rather than hidden, with its trigger in §12: a second, |
| +kind-restricted retrieval pass when the lookup class fires and the ruled |
| +results carry no candidate. It is not built now because it is a second |
| +query on the hot path bought for a case nobody has yet observed. |
| + |
| +#### Gates |
| + |
| +**34. The box never speaks a withheld passage, and the count happens |
| +after the ruling.** Two documents from a sibling, each carrying a |
| +different validated eircode. Presence first: with both `VISIBLE`, the box |
| +renders and shows two candidates. Then rule one `NOT_VISIBLE` — the box |
| +shows exactly the surviving one, as a single answer, and §6.5's notice is |
| +on the page. Then rule both `NOT_VISIBLE` — **no box at all**, and the |
| +notice stands. Mutation: run extraction over `all` rather than over |
| +`page.Results` in `Store.rule`; the second case goes red on the count and |
| +the third on the box's existence. Assert the box's fields, not the |
| +rendered copy — §4.1's `Outcome.Record` lesson, where two dispositions |
| +rendered identical pages. |
| + |
| +**35. Two candidates are never collapsed to one.** Presence: one |
| +validated phone number, one box, one value. Add a second document with a |
| +different one: both are named, neither is dropped, and the higher-scoring |
| +one is not presented as the answer. Add two more: no box. Mutation: |
| +return `candidates[0]` when `len(candidates) > 1` and the second case goes |
| +red; raise the cap and the fourth does. |
| + |
| +**36. The displayed value is byte-identical to the document's.** Seed |
| +`+353 (0)1 547 7854` and assert that exact string reaches the box. |
| +Mutation: canonicalise on the display path — the direction the code |
| +already canonicalises for the index, so this is one line's difference |
| +and the gate exists because it is — and it goes red. |
| + |
| +**37. The label is the document's or the fallback's, never a fragment.** |
| +Three fixtures: a labelled line (`Office: …` → `Office`), a number |
| +mid-paragraph (→ the heading trail, not the preceding clause), and a |
| +label longer than the cap (→ the heading trail). Mutation: drop the |
| +mid-sentence condition and the second fixture yields a composed label — |
| +red. |
| + |
| +**Gate 14 extends to this box**: every character of it traces to a cited |
| +passage or to the fixed lexicon. Nothing in it is authored. |
| + |
| +**And the mutations above are the author's, which §1.9 says is exactly |
| +what does not count.** They are listed so a reviewer has somewhere to |
| +start, not as the pass. The gate review picks its own. |
| + |
| --- |
| ## 9. Surfaces |
| @@ -3559,6 +3827,8 @@ commit. Fixed 2026-09-05; §1.9's lesson arriving inside the fix for §1.8.* |
| | **A source-of-record surface, so a hand-off can include memoria's own material** | Someone wants "save this answer as a Doc" to work for answers built on the team's own uploads. Suite §7 makes the *receiver* check every source with its app of record, and memoria serves nobody to ask (decision 7), so such a hand-off is refused today — §9.1. Opening it needs its own decision, its own §3.2 leak entry and its own gate, and it is Paul's: decision 7 is settled | |
| | Calendar as a source | Calendar has a spec and content; bodies off by default even then | |
| | Mail beyond forward-or-label | Never by sweep. A mail repo exists **and** a per-thread audience maps onto a team | |
| +| **A kind-restricted retrieval pass for §8.8** | The lookup class fires, the ruled results carry no candidate, and somebody reports a fact they can see in a document the box did not surface. A second query on the hot path, bought for a case nobody has observed | |
| +| **Postal address as a lookup kind** | Decision 15 keeps it out because there is nothing to validate. The two openings are an anchored one — an eircode certifies the lines above it, deterministically — and §8.2's extractive single-source class, which is a model and therefore a decision-15 amendment | |
| | Cached answers | Query cost becomes visible. Requires §5.2's provenance invalidation first. **Triggering this expires §8.4's "stored nowhere"** | |
| | A conversational surface | Someone wants one, and it keeps §8.4's invariant: no passage crosses a request, in a prompt or a response. Decision 14 chose the shape on decision 5's grounds, not decision 3's, so this is a product argument to have — not a rule to break. §6.5 affords it: a four-turn transcript re-authorizes in one batch, one round | |
| | A runtime model-call ledger | Someone needs to audit routing. Sealed and provenance-stamped when it lands (§8.3) | |
| @@ -3954,7 +4224,10 @@ on 2026-09-06 when the branches merged. |
| complex reasoning."* — not the obligation to measure it, which came from |
| the code review and stands. It reads §8.2 right: three of five classes |
| call no model, and §8.7 sends arithmetic to Sheets, so a model sees only |
| - what survives retrieval, filtering and authorization. |
| + what survives retrieval, filtering and authorization. *(That count was |
| + right when he said it. §8.8 made it four of six on 2026-09-14; the |
| + sentence is left as spoken, per this section's rule about struck items, |
| + and the ratio moved in the direction the argument wanted.)* |
| - **Two tiers stay, and so does the measurement.** The spec asked whether |
| the frontier tier was a habit. Paul ruled on 2026-09-05 that two tiers |
| make sense *and* that measuring makes sense — two answers, not one, and |