amadan
Spec §8.9, draft 2: what the verbatim guard actually proves
Reviewed by Codex GPT-6-Astra (dispatched: codex/gpt-6-astra via Paseo; self-reported "Codex / GPT-6, exact variant not exposed"). REQUEST CHANGES, eight must-fix. Every factual claim was checked against the text before acting on it, and every one held. **The central claim overstated what it guaranteed.** Draft 1 said "the model proposes, a string comparison disposes" and that a hallucinated address "cannot reach the page". The check proves where a value's BYTES came from. It does not prove the value is an address, the whole address, the relevant one — or that the model returned every address it saw. The empty string passes strings.Contains. "Dublin" from an unrelated sentence passes. And if four addresses are in the passages and the model returns one, every check passes and the box shows one as though it were the only one: the model can pick, and §8.8's "never picks" does not survive. That omission is undetectable from inside memoria. So §8.9 now states the narrow guarantee — byte-exact span of a sent, ruled passage, the document's label, a working citation — says in so many words that classification, completeness and relevance are model judgements, and lists the three §8.8 guarantees that do not carry over. A model row never inherits the deterministic layer's claim: it is marked 🤖 and names its model, which is also decision 4's naming rule reaching a box that has no answer paragraph. **Selection is now fully specified.** Proposals name their passage; the text is located by exact byte search in THAT passage, no normalisation; the first occurrence by offset; word-boundary ends; not inside a span the recognisers found; ≥12 bytes with a digit, comma or line break. The result is an immutable (passage, start, end) coordinate used for slicing, labelling and citing, and the model's own string is discarded. Truncated responses are rejected whole. **Eligibility is a property of the kind, never of the outcome.** Draft 1 said the model runs "where the deterministic layer produced nothing", which invites treating an empty box as nothing; §8.8's box is empty for five reasons and none licenses asking a model for a phone number. A fixed allowlist of model kinds, today only address; kinds independent in a mixed query. **Dependants I had missed, now fixed.** §8.8's own body still said "the fact box calls no model, ever" — I had amended the decisions table and not the section it governs. "address on its own primes nothing" was still live. §8.1 still cited decision 15 as "calls no model". And draft 1 said gate 31 enforces that nothing calls a provider before slice 2; gate 31 forbids logging a response body and says nothing of the kind — a rule attributed to a gate that does not state it, which is this repo's worst recorded failure shape. **Attribution.** Decision 15's row now separates Paul's ruling, quoted, from the author's implementation constraints, listed as proposed and overridable. §13 had paraphrased him as "then a cheap model"; he said "best value". The claim that a string-checked task "does not improve with a better model" is withdrawn — a better model extracts better; the tier is a chosen cost-quality trade. Also: the payload is bounded at the top eight ruled results, matching §8.6's answer budget; §3.2 names the second trigger, including searches that produce no answer; decision 16's compatibility is stated; gate 39 is rewritten to assert on recorded requests as well as rendered pages, because draft 1's could not see an egress that had already happened by the time the box rendered nothing. The eircode-anchored address block — the deterministic half of Address, and exactly what Paul's rule asks for first — is recorded in §12 as the next step.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 file changed,
+291
−169
docs/superpowers/specs/2026-09-03-memoria-design.md+291 −169
diff --git a/docs/superpowers/specs/2026-09-03-memoria-design.md b/docs/superpowers/specs/2026-09-03-memoria-design.md| index ad1a92d..fb90f18 100644 |
| --- a/docs/superpowers/specs/2026-09-03-memoria-design.md |
| +++ b/docs/superpowers/specs/2026-09-03-memoria-design.md |
| @@ -108,7 +108,7 @@ embedding is a plaintext cell in a costume — §3.1. |
| | 12 | **Name: `memoria`.** House convention is plain Spanish office nouns. A *memoria* is both a memory and, in Spanish office usage, a written record. | `archivo`; `cerebro`; `base`; anything English | Paul, 2026-09-03 | |
| | 13 | **The provider is TensorX, and memoria supports exactly one.** A team owner's choice is on or off, not which — a configurable base URL would let an owner reinstate a proxy while the answer still printed "one named provider". Chosen for EU jurisdiction and open weights, criteria decision 4 did not state, at a known cost in id-pinning and frontier quality (§8.3). | OpenRouter (a proxy: two parties see content); OpenAI direct (better id-pinning, worse jurisdiction and cost) | Paul, 2026-09-04, both halves — *"we go with exactly one for now: TensorX. If we want to swap that later we can."* Swapping the *generation* provider is a code change and a review, not a setting; swapping the *embedding* provider is that plus a full corpus re-embed unless the new host serves the same open weights (§8.3, §13) | |
| | 14 | **memoria is a search page, not a chat.** One GET search form, `?q=`, one query in and one page out, with `/documents/{id}` as the only other passage surface. This follows from decision 5, not from decision 3 — §8.4's invariant is what decision 3 requires, and a conversational surface could satisfy it. | A chat UI with citations; a threaded follow-up. Deferred to §12, not forbidden — the invariant forbids the leak, this row chooses the shape | Spec author, 2026-09-04, answering a question Paul asked; **confirmed by Paul 2026-09-05**, parking the conversational surface in §12 rather than forbidding it | |
| -| 15 | ~~**The fact box calls no model, ever.**~~ **AMENDED BY PAUL, 2026-09-19.** The rule is **deterministic first, then a model for what deterministic analysis cannot reach**: *"do as much as possible with deterministic analysis, and then attempt to build metadata with the best value models via TensorX that make sense."* A validated fact is still extracted by recogniser and shown with its citation; a kind with **no** validator — a postal address — is no longer excluded, it is the model's half. §8.8, §8.9 | The strict reading, which was the spec author's and not Paul's, and which this row carried for five days: it foreclosed the obvious next request and made "has a checksum" the boundary of the feature rather than the boundary of the deterministic layer. **What is NOT amended:** a model never picks among validated candidates, never composes a label, and never produces a value that is not a verbatim span of a cited passage | **Paul, 2026-09-19**, ruling on the question §13 had recorded as open since 2026-09-14. The strict version was mine; he reversed it, and the reversal is the general rule and not a carve-out for addresses | |
| +| 15 | ~~**The fact box calls no model, ever.**~~ **AMENDED BY PAUL, 2026-09-19.** **Paul's ruling:** *"do as much as possible with deterministic analysis, and then attempt to build metadata with the best value models via TensorX that make sense"* — asked specifically whether a postal address, which has no validator, should be extractable by a model, and answering yes in general terms. **The implementation constraints are the spec author's, proposed and not ruled:** extractive only; every value a byte-exact span of a sent passage; a model never touches a validator-backed kind; labels never model-written; model rows marked 🤖 and named; `model.cheap` as the initial tier. §8.8 (the deterministic layer), §8.9 (the model half) | The strict reading, which was the spec author's and not Paul's and which this row carried from 2026-09-14 to 2026-09-19: it made "has a checksum" the boundary of the feature rather than the boundary of the deterministic layer | **The ruling is Paul's, 2026-09-19.** The constraints listed in the first column are the author's and §13 records them as open to his override | |
| **Three of these are now suite rulings, and this spec cites them rather |
| than restating them.** Decision 1 is **suite ruling 1**, decision 3 is |
| @@ -847,6 +847,17 @@ said.** Draft 1 stated only query-time egress: ~10 chunks × 500 tokens |
| per question, so that over months "the frequently-consulted part of the |
| corpus has left in full". That understated it by the corpus. |
| +**§8.9 adds a second query-time trigger, and it is not the same as the |
| +first.** Same recipient, same category of transmission — passages that |
| +passed §4.1 in this request — but a different trigger and a bounded, |
| +possibly different payload: the top eight ruled results are sent when a |
| +query names a model kind (a postal address) and deterministic extraction |
| +cannot serve it, **whether or not the query would have produced a |
| +generated answer at all.** A search for "office address" that §8.2 routes |
| +as navigational still sends passages. The disclosure has to say that in |
| +the same breath as the answer path; "passages behind an answer" is no |
| +longer the whole of it. |
| + |
| **Embeddings come from a hosted API (§7.2). So with AI on, every chunk of |
| every document leaves at ingest, on day one.** §8.6 prices it: 24M tokens |
| for a 90k-chunk corpus. The frequently-consulted part leaving is the |
| @@ -2218,7 +2229,8 @@ paginated, rastrillo `server` shape. Lexical only, since embeddings need |
| a provider — and per §3.2, nothing leaves the instance in this mode. |
| **§8.8's fact box is part of this floor and not an addition above it.** It |
| -calls no model (decision 15), so it is present in exactly the mode this |
| +calls no model in its deterministic layer (§8.8, decision 15 as |
| +amended), so that layer is present in exactly the mode this |
| paragraph describes; gate 15 covers it with everything else here, and it |
| renders inside the results region rather than beside it. |
| @@ -2701,9 +2713,12 @@ section's closing sentence still holds. |
| Derived-artefact regeneration (§5.2) adds a synthesis-class call per |
| invalidated artefact and is unbudgeted until the feature exists. **So does |
| §8.9**, at one cheap-tier call per query that names a valueless kind and |
| -whose recognisers found nothing — a small fraction of a small fraction, |
| -and cheap-tier by rule rather than by choice, since a task checked by |
| -string comparison does not improve with a better model. |
| +whose recognisers found nothing — a small fraction of a small fraction. |
| +**Cheap tier by choice, not by necessity**: Paul's ruling says "best value |
| +models", and extraction from eight passages is the cheap tier's work under |
| +§8.2. A better model would extract better — more complete addresses, |
| +cleaner boundaries, fewer omissions — so this is a cost-quality trade to be |
| +re-measured, not a consequence of the verbatim check. |
| The money is recurring query tokens by an order of magnitude — the |
| economic argument for §8.2's router and its three model-free classes. |
| @@ -2920,15 +2935,18 @@ codebase — `TestALookupPrimesTheFirstResult` is about Enter, not about |
| this — and a second meaning for the word in the same package is the name |
| collision this repo's reviews keep finding. |
| -**Decision 15: the fact box calls no model, ever.** Not a cheap one, not |
| -an extractive one. The kinds it serves are the kinds that have a |
| -validator; a kind with no validator is not in the box and is not the |
| -box's problem (§12). The moment a model picks which span to show, every |
| -argument below collapses into §8.7's forbidden retrieval-augmented |
| -calculator with a different noun. |
| - |
| -**It is a fourth way memoria answers without a model, and it is not a |
| -class.** Drafts 1 and 2 both called it *"§8.2's fourth model-free class"* |
| +**This section is the deterministic layer, and it calls no model.** The |
| +kinds it serves are the kinds that have a validator. Decision 15 originally |
| +said the whole fact box calls no model, *ever*, and excluded any kind |
| +without a validator; **Paul amended it on 2026-09-19** and §8.9 is the |
| +model half he ruled in. Nothing in this section changes because of that: |
| +a validator-backed kind never reaches a model, and every guarantee below |
| +is a guarantee about *this* layer. Where a sentence here says "no model", |
| +it means this layer and not the request — §8.9 may call one afterwards, |
| +for a different kind, and §8.2's router may call one after that. |
| + |
| +**Its deterministic layer is a fourth way memoria answers without a model, |
| +and it is not a class.** Drafts 1 and 2 both called it *"§8.2's fourth model-free class"* |
| while also calling it a layer, which cannot both be true: a row in §8.2's |
| table is a **route**, and a route is terminal. A query naming a kind would |
| have stopped at the box, so *"what's the office phone number?"* with |
| @@ -2937,6 +2955,13 @@ class would have given one. The box is attempted, and then §8.2 routes the |
| query exactly as it would have. Paul's constraint is deterministic |
| **first**, not deterministic **instead**. |
| +**"Model-free" in §8.2 and §8.6 is a claim about routing, not about the |
| +request.** §8.9 now sits between this box and the router and may call the |
| +cheap tier for a postal address. A navigational query that names |
| +`address` can therefore cause a model call that no router class did — |
| +the router's three model-free classes are still model-free; the request |
| +carrying them may not be. |
| + |
| **This section's own path touches nothing in §3.2.** No egress, no |
| provider, no embedding — the deterministic box works on an instance with |
| AI switched off, which is decision 5's floor and therefore the whole |
| @@ -3028,7 +3053,8 @@ box is a function of the page, not of the corpus, and nobody should read |
| it as the latter. |
| **Whether the cap makes the box inert on the corpus that motivated it is |
| -unmeasured, and that measurement is owed before decision 15 is settled.** |
| +unmeasured.** Decision 15's principle is settled; this measurement is |
| +about whether the cap of three is the right number, and it is still owed. |
| A real top-25 for "phone number" on Paul's live corpus may well carry more |
| than three distinct valid numbers. §13 records it. The remedy if it does |
| is a narrower kind lexicon or a per-kind cap, **not** a rule that picks. |
| @@ -3092,6 +3118,7 @@ validates. |
| | `vat` | `vat`, `vat number` | modern Irish format and its check letter | |
| | `phone` | `phone`, `telephone`, `tel`, `mobile`, `phone number` | **structural, and see below** | |
| | `email` | `email`, `email address`, `e-mail` | `local@domain`, domain carrying a dot, no whitespace — **weak, and see below** | |
| +| `address` | `address`, `postal address`, `office address` | **none — this layer never produces one.** A §8.9 model kind; see there | |
| **Three of these are checksums and two are not, and the section says so |
| rather than implying five.** An eircode's routing key is a closed set, an |
| @@ -3174,11 +3201,13 @@ qualifies and the box is inert immediately. It also collides with §8.5's |
| the box carries is its citation, to `/documents/{id}#p{n}`. No `mailto:`, |
| no `href` built from a document's bytes. |
| -**`address` on its own primes nothing.** It is ambiguous between a postal |
| -address and an email address, and only one of those has anything |
| -resembling a validator. `email address` primes `email`. A postal address |
| -has no checksum, no fixed grammar and no length, so under decision 15 it |
| -is not a kind. §12 carries it. |
| +**`address` primes the postal-address kind, and `email address` primes |
| +`email`** — longest phrase first, so the two cannot collide. Postal |
| +address has no checksum, no fixed grammar and no length, so this layer has |
| +nothing to validate it with and produces nothing for it. **It is §8.9's |
| +first model kind**, which is what decision 15's amendment changed: before |
| +it, "no validator" meant "not a kind"; now it means "not this layer's |
| +kind". |
| **`number` on its own primes nothing either.** The lexicon maps |
| distinctive words and phrases; generic words count only in combination. |
| @@ -3417,145 +3446,231 @@ with deterministic analysis, and then attempt to build metadata with the |
| best value models via TensorX that make sense."* |
| The question put to him was narrow — should a postal address, which has |
| -no checksum and therefore no validator, be extractable by a cheap model? |
| -**The answer he gave is general**, and this section records the general |
| -form rather than the carve-out: the deterministic layer is the floor and |
| -not the ceiling, and where it cannot reach, a cheap model may — under |
| -conditions that keep every guarantee §8.8 establishes. |
| - |
| -**Not built. This is slice 2**, because nothing in this binary calls a |
| -provider before slice 2 and gate 31 enforces it. What lands now is the |
| -ruling and its conditions, so that the section is written before the code |
| -rather than after — which is the order §1.9 exists to insist on. |
| - |
| -#### The ordering, which is the whole of the ruling |
| - |
| -1. **Recognisers run first, always.** If a validated candidate exists for |
| - a wanted kind, §8.8 answers and no model is called. The deterministic |
| - answer is never second-guessed, never re-ranked, and never handed to a |
| - model for an opinion. |
| -2. **A model runs only where the deterministic layer produced nothing**, |
| - for a kind that has no validator to produce it with. |
| -3. **With AI off, step 2 does not happen and step 1 is unchanged.** |
| - Decision 5's floor is intact: the search page and §8.8's box are the |
| - product, and this section is a garnish on a garnish. |
| - |
| -**The failure this ordering prevents is a model improving on a checksum.** |
| -A mod-97 check is right or wrong; a model asked to "find the IBAN" in the |
| -same passage can be neither. Letting it near a kind that validates would |
| -trade a provable answer for a plausible one, which is the direction §8.7 |
| -spends a whole section refusing. |
| - |
| -#### The condition that makes it safe: the model proposes, a string |
| -#### comparison disposes |
| - |
| -**Every value the model returns must appear verbatim in one of the |
| -passages that were sent to it, and memoria checks that itself.** Not |
| -"should"; the check is a `strings.Contains` against passages memoria |
| -already holds, and a value that fails it is dropped without being |
| -rendered. |
| - |
| -This is the load-bearing sentence of the section, and it is what makes |
| -the rest of §8.8 survive intact: |
| - |
| -- **A hallucinated address cannot reach the page.** Not because the model |
| - is good, but because a string that is not in the corpus fails a string |
| - comparison. The model's job is reduced to *pointing*, and pointing is |
| - checkable. |
| -- **§8.8's display rule is unchanged**: the span shown is the document's |
| - bytes, because it is located in the document by the check above and |
| - sliced from there. The model's own output text is never rendered — it is |
| - a lookup key and then it is discarded. |
| -- **The label is still the document's**, derived by §8.8's five rules from |
| - the span's position once located. **A model never writes a label**, and |
| - this is not a stylistic preference: a composed label is the one part of |
| - the box that no string comparison can verify. |
| -- **Gate 14 and §8.4 hold without amendment.** Every character of the box |
| - still traces to a cited passage, and the citation is the passage the |
| - span was found in. |
| - |
| -**Ambiguity, the cap and collision are §8.8's, unchanged.** If the model |
| -points at three addresses, three render; at four, none. It never picks, |
| -for the same reason the deterministic layer never picks — rank order |
| -carries no claim about which address is the office one, and a model's |
| -preference carries less. |
| - |
| -#### Egress, and what is not new about it |
| - |
| -**Passages go to TensorX at query time.** That is the same egress §8.3 |
| -already describes for the 🤖 paragraph: the same provider, the same |
| -passages, the same moment in the same request, after §4.1 has ruled. It |
| -is **not a new entry on §3.2's leak list** and should not be written as |
| -one — the leak is "query-time passages reach the named provider when AI |
| -is on", and it exists the moment slice 2 does. |
| - |
| -What *is* new is the trigger. §8.2's router sends passages when a query |
| -needs an answer; this sends them when a query named a kind and |
| -deterministic extraction found nothing. A team with AI on now has a |
| -second reason for a request to leave the instance, and §8.3's disclosure |
| -copy should say so in the same breath as the first. |
| - |
| -**Model and tier:** `model.cheap` (§8.2, §8.3 — `deepseek/deepseek-v4-flash-0731` |
| -at the time of writing), because this is extraction from a handful of |
| -passages and §8.2 already classes that as the cheap tier's work. **Never |
| -the frontier tier**, whatever the kind: a task whose output is checked by |
| -string comparison does not improve with a better model, it only costs |
| -more. |
| - |
| -**Cost** is §8.6's to absorb and is unbudgeted until this is built. The |
| -shape: one cheap call per query that names a valueless kind and finds |
| -nothing, which is a small fraction of a small fraction. |
| +no checksum and therefore no validator, be extractable by a model? **The |
| +answer he gave is general**: the deterministic layer is the floor and not |
| +the ceiling, and where it cannot reach, a best-value TensorX model may. |
| +**Everything below that sentence is the spec author's proposal for how**, |
| +and decision 15's row says which parts are his and which are not. |
| + |
| +**Draft 1 of this section overclaimed, and a review by Codex GPT-6-Astra |
| +(2026-09-27) is why it no longer does.** It said "the model proposes, a |
| +string comparison disposes" and that a hallucinated address "cannot reach |
| +the page". The check it described proves where a value's **bytes** came |
| +from. It does not prove the value is an address, that it is the whole |
| +address, that it is the relevant one, or that the model returned every |
| +address it was shown. Those are four model judgements, and this section |
| +now says so instead of implying the check makes them. |
| + |
| +#### Which kinds, decided by the kind and never by the outcome |
| + |
| +**Eligibility is a property of the kind.** A fixed allowlist of *model |
| +kinds* — today exactly one, `address` — may be sent to a model. **A |
| +validator-backed kind never is**, whatever the deterministic layer did |
| +with it: produced candidates, produced none, or produced some that the cap, |
| +a collision or an over-long label then suppressed (§8.8). Draft 1 said the |
| +model runs "where the deterministic layer produced nothing", which invites |
| +treating an empty box as nothing; the box is empty for five different |
| +reasons in §8.8, and none of them is licence to ask a model for a phone |
| +number. |
| + |
| +**Kinds are independent.** "Phone number and postal address" is two |
| +requests: the phone goes to §8.8, the address to this section, and a |
| +validated phone does not suppress the address request or the reverse. |
| + |
| +**"No model is called" is a claim about this metadata path, per kind.** |
| +§8.2's router runs after the box and may call a model for an answer |
| +regardless; §8.8 says so and this does not change it. |
| + |
| +**With AI off, nothing in this section happens** and §8.8 is unchanged. |
| +Decision 5's floor is the search page and the deterministic box. |
| + |
| +**Not built. This is slice 2**, because §11 sequences every provider call |
| +there. Draft 1 said gate 31 enforced that; it does not — gate 31 forbids |
| +logging a provider's response body and says nothing about when a provider |
| +may be called. The sequencing is §11's, and until this section is built |
| +the claim is enforced only by there being no provider client in the |
| +binary. |
| + |
| +#### What is sent |
| + |
| +**The top eight ruled results, by rank** — the passages §4.1 allowed in |
| +this request, bounded at eight to match §8.6's answer-time budget. Not |
| +the twenty-five §8.8 scans: an address in result eleven is a recall limit |
| +this section accepts in exchange for a bounded payload and a response |
| +inside the deadline. Each passage is sent with an index, and nothing else |
| +about it — no title, no path, no id. |
| + |
| +**Egress is the existing category with a new trigger**, and §3.2 now says |
| +both halves: the same recipient and the same kind of transmission as the |
| +answer path, but sent when a query names a model kind whether or not a |
| +generated answer would follow. The disclosure copy names this trigger |
| +alongside the answer path. |
| + |
| +**If decision 16 lands** — TensorX as zero-data-retention, treated as |
| +equivalent to local inference, proposed on `design/zdr-treat-as-local` — |
| +the retention wording here and in §3.2 aligns with it. **The transmission |
| +disclosure does not go**: decision 16 changes what is retained, not what |
| +is sent, and it says so itself. |
| + |
| +#### What comes back, and what memoria checks |
| + |
| +**The response contract** asks for every postal address in the sent |
| +passages, as a list of `{passage: <index>, text: <exact bytes>}`, and |
| +nothing else. **A response the provider marks truncated is rejected |
| +whole**, not partially used; a malformed one likewise. |
| + |
| +**Each proposal becomes a coordinate or is dropped.** For every returned |
| +`{passage, text}`: |
| + |
| +1. `passage` must name a passage that was sent. A reference to a passage |
| + that was not sent — including an authorised result ranked ninth, which |
| + was never in the request — is dropped. |
| +2. `text` must be non-empty, at least 12 bytes, and contain a digit, a |
| + comma or a line break — **structural, not a validator**, and there to |
| + stop `Dublin` or `1` passing because they occur somewhere. |
| +3. **`text` is located in that passage by exact byte search — no |
| + normalisation of case, whitespace, quotes or Unicode form.** NBSP versus |
| + space, curly versus straight quotes, a line break flattened to a space: |
| + each is a refusal, deliberately. A span crossing two passages is a |
| + refusal. These are real recall limits and they are chosen over any |
| + mapping back to source bytes, which would be a second parser (§8.7). |
| +4. **The first occurrence in that passage, by byte offset, is the one** — |
| + deterministic, and never the provider's return order or a map's. |
| +5. The occurrence must **begin and end on a word boundary** and must **not |
| + lie inside a span §8.8's recognisers found** — a street name inside an |
| + email address is not an address. |
| +6. The candidate is now the immutable coordinate **(passage, start byte, |
| + end byte)**. The span displayed is sliced from the document at it; the |
| + label is derived by §8.8's five rules at `start`; the citation is that |
| + passage. **The model's own string is discarded** — its only role was to |
| + be found. |
| + |
| +**Identity is the byte-exact span.** An address has no canonical form, so |
| +two spellings are two candidates; the same bytes in two passages are one, |
| +cited to the better-ranked passage, as §8.8 does. |
| + |
| +**Cap, collision and over-long labels then apply to the accepted |
| +proposals**, per §8.8, with the same results: four accepted addresses |
| +render nothing, a colliding citation line renders nothing, an over-long |
| +label silences the kind. |
| + |
| +#### What does not survive, stated rather than implied |
| + |
| +Draft 1 said every §8.8 guarantee survives. Three do not, and a reader |
| +of a model row has to be told: |
| + |
| +- **"Never picks" does not hold.** §8.8 scans every recognised candidate |
| + in its result set; this section sees only what the model chose to |
| + return. If the passages hold four addresses and the model returns one, |
| + every check passes and the box shows one address as though it were the |
| + only one. **That omission is undetectable from inside memoria.** The |
| + exhaustive contract and the truncation refusal reduce it; nothing |
| + removes it. |
| +- **The cap and collision rules cannot fire on what was never returned.** |
| + An omitted fourth address means a box of three where §8.8 would have |
| + shown none. |
| +- **§8.8's poisoning argument weakens.** A planted second phone number |
| + makes a deterministic box show both or vanish. A planted instruction in |
| + a passage can make the model omit the real address instead — the attack |
| + §8.8 says degrades into a visible contradiction degrades here into a |
| + silent omission. |
| + |
| +**So a model row never inherits the deterministic layer's claim.** It is |
| +marked 🤖, names the model that produced it (decision 4: every answer |
| +names its provider), and is visually distinct from a row a checksum |
| +vouched for. A reader must be able to tell "validated" from "a model |
| +pointed here, and the text really is in the document" at a glance. |
| + |
| +**What the checks do guarantee, narrowly:** every accepted value is a |
| +byte-exact span of a passage that passed §4.1 in this request and was |
| +sent; its label is the document's; its citation opens on it. **Its |
| +classification, completeness and relevance are the model's judgement.** |
| + |
| +#### Model and tier |
| + |
| +`model.cheap` (§8.2, §8.3 — `deepseek/deepseek-v4-flash-0731` at the time |
| +of writing) **is the chosen initial tier, not a consequence**. Paul asked |
| +for "best value models"; extraction from eight passages is the cheap |
| +tier's work under §8.2. A better model would extract better, and draft 1's |
| +claim that a string-checked task "does not improve with a better model" |
| +was wrong. Re-measure against labelled cases before treating the tier as |
| +settled. |
| + |
| +**Deadline: 800 ms**, §6.5's sibling deadline reused. The call rides the |
| +request's context, so an abandoned keystroke on the fragment route cancels |
| +it. Past the deadline, or on any provider error, the box renders without |
| +model rows and the page is otherwise unchanged. **Nothing from the request |
| +or the response is logged** — status and a fixed string only (gates 4, 31). |
| #### Injection |
| -§8.5 is unchanged and its mitigation is the same partial one. A poisoned |
| -document can make the model point at an attacker's string, and the verbatim |
| -check will pass — because the string really is in the corpus. What bounds |
| -it is what bounds §8.8: the passage passed §4.1, the citation is openable, |
| -and somebody in this team put the document there. **The check defends |
| -against the model inventing; it does not defend against the corpus lying**, |
| -and those are different problems with different answers. |
| +§8.5 is unchanged and its mitigation is the same partial one, with the |
| +omission case above added to it. **The checks defend against invented |
| +bytes; they defend against neither the corpus lying nor the model being |
| +steered into silence.** |
| #### What "that make sense" leaves open, and it is Paul's |
| -The ruling names a principle and one instance. **Which metadata is worth |
| -building this way is a product judgement and §13 records it as his**, not |
| -because the spec is being timid but because the answer is a list that will |
| -grow and each entry costs a query-time model call. |
| +Which metadata is worth a query-time model call. Postal address is ruled |
| +in. Opening hours, a person's role, an effective date, a registration |
| +number in a format memoria does not know are plausible and unruled. **The |
| +proposed test** is "is the answer a verbatim span somebody would want |
| +cited" — anything needing summary is an answer, not metadata, and §8.2 |
| +owns answers. §13 records it. |
| -The first entry is **postal address**, which is what he was asked about |
| -and which §12 no longer defers. Plausible next ones, none of them ruled: |
| -opening hours, a person's role or title, a document's effective date, a |
| -company registration number in a jurisdiction whose format memoria does |
| -not know. **The test for admitting one is not "could a model do it" — a |
| -model could do all of them — but "is the answer a verbatim span somebody |
| -would want cited".** Anything that needs summarising is not metadata, it |
| -is an answer, and §8.2's router already owns answers. |
| +**And the deterministic half of Address is not built yet, which is the |
| +next step Paul's rule asks for.** A validated eircode certifies the |
| +address block it ends; extracting that block deterministically would serve |
| +most Irish addresses with no model at all. §12 carries it. |
| #### Gate |
| -**39. A value the model returns that is not in a sent passage never |
| -renders** (§8.9). Three fixtures against a stubbed provider, and the |
| -first is the presence half: |
| - |
| -- the stub returns a span that **is** in a passage → the box renders it, |
| - with the document's bytes and the document's label; |
| -- the stub returns a plausible address that is **in no passage** → **no |
| - box**, and nothing is logged that contains it (gate 4, and a provider's |
| - response body is content — gate 31); |
| -- the stub returns a span that is in a passage belonging to a document |
| - **§4.1 withheld from this reader** → no box. The passages sent are the |
| - ruled ones, so this should be unreachable; the fixture exists because |
| - "unreachable" is what §1.9 says to test rather than assume. |
| - |
| -Mutation: render the model's own output text rather than locating it and |
| -slicing the document, and fixture 2 goes red. Send the unruled passages |
| -and fixture 3 goes red. **And the deterministic-first ordering needs its |
| -own:** stub the provider to return a *different* phone number from the |
| -one the recogniser validated, and assert the recogniser's answer is what |
| -renders and **that the provider was never called at all** — a gate that |
| -checks only the rendered value cannot tell "called and ignored" from |
| -"never called", and the first of those is an egress nobody authorised. |
| +**39. The model half sends only what it may, accepts only what it can |
| +locate, and never touches a validator-backed kind** (§8.9). Against a stub |
| +provider that records every request body and returns scripted responses. |
| +**Every case asserts on the recorded requests as well as on the page**, |
| +because draft 1's version could not see an egress that had already |
| +happened by the time the box rendered nothing. |
| + |
| +- **Presence.** A scripted span that is in a sent passage renders, as the |
| + full tuple `(span, title, trail, label, DocumentID, Ordinal)`, marked 🤖 |
| + with the model named — and the recorded request contains exactly the |
| + top eight ruled passages. |
| +- **A span in no passage** → no row; the stub confirms the response was |
| + delivered, so "no row" is a refusal and not a failed call. |
| +- **A span in an authorised result ranked ninth**, which was never sent → |
| + no row. A check run against all authorised results rather than the sent |
| + set passes every other fixture. |
| +- **A withheld document**, with a visible control in the same search: the |
| + recorded request contains the visible passage and **not** the withheld |
| + one, and a span present only in the withheld one renders nothing. |
| +- **The same text in a visible and a withheld passage** → one row, cited |
| + to the visible passage, every tuple field checked. |
| +- **Two occurrences in one passage with different labels** → the first by |
| + byte offset, both label and citation. |
| +- **Normalisation refusals:** NBSP, curly quotes, a flattened line break — |
| + each returned in the "wrong" form → no row. |
| +- **Short and embedded spans:** `Dublin`, the empty string, and a |
| + street-name fragment inside a recognised email → no row. |
| +- **A truncated response** → no model rows at all. |
| +- **Validator-backed kinds never reach the provider**, counted at the stub |
| + by purpose: zero valid candidates, candidates suppressed by the cap, by |
| + a collision, by an over-long label, and a mixed `phone number and postal |
| + address` query — which must send exactly one request, for the address. |
| +- **AI off** → zero requests for every fixture above; **with a presence |
| + half**, the same query with AI on does send one. |
| +- **A provider error and a timeout** → the page renders without model rows, |
| + and nothing from either is in the log. |
| +- **Four addresses in the passages, one returned** → one row renders. This |
| + is the undetectable omission, asserted as the documented behaviour so |
| + that it is a known limit and not a surprise; the day something can detect |
| + it, this case goes red on purpose. |
| + |
| +Draft 1 listed "render the model's string instead of the source slice" as |
| +a mutation this gate catches. **It cannot, and it does not matter:** for an |
| +exact match the bytes are identical. What matters is the coordinate — which |
| +occurrence, which passage, which citation — and the tuple assertions above |
| +are what see a wrong one. |
| --- |
| @@ -4378,6 +4493,7 @@ commit. Fixed 2026-09-05; §1.9's lesson arriving inside the fix for §1.8.* |
| | Mail beyond forward-or-label | Never by sweep. A mail repo exists **and** a per-thread audience maps onto a team | |
| | **Reaching a fact the ranked query does not** | Somebody reports a fact they can see in a document the box did not surface. §8.8's candidates are the ruled top-25, so an eircode in a document that never says "eircode" is found only if something else about it ranks. **Whatever this turns out to be, it is not "index the kind words"** — §8.8 measures that collapsing `bm25()` for the injected term from 2.7169 to 0.0000 | |
| | ~~**Postal address as a fact kind**~~ | **PROMOTED 2026-09-19 to §8.9, slice 2.** Decision 15's amendment makes "nothing to validate" the reason a model handles it rather than the reason it is excluded. Kept struck rather than deleted, because the row's reasoning is what the amendment overturned | |
| +| **An eircode-anchored address block** | **The next step, and the one Paul's rule asks for.** A validated eircode certifies the address block it ends; extracting the contiguous lines above it is deterministic and would serve most Irish addresses with no model call at all. Deferred from §8.9's first branch only because that branch is already memoria's first provider client. Trigger: that branch landing | |
| | **A durable, sealed facts table** | The per-query extraction shows up in a profile. §8.8 rejects it today on derived-state grounds and because it would freeze the recognisers at ingest — **not** on leak grounds, which is what draft 1 argued and which attacked an unsealed design nobody proposed. Note the direction: storing facts makes recogniser improvements *harder* to ship, so this row gets triggered by cost and never by correctness | |
| | Cached answers | Query cost becomes visible. Requires §5.2's provenance invalidation first. **Triggering this expires §8.4's "stored nowhere"** | |
| | A conversational surface | Someone wants one, and it keeps §8.4's invariant: no passage crosses a request, in a prompt or a response. Decision 14 chose the shape on decision 5's grounds, not decision 3's, so this is a product argument to have — not a rule to break. §6.5 affords it: a four-turn transcript re-authorizes in one batch, one round | |
| @@ -4393,11 +4509,13 @@ commit. Fixed 2026-09-05; §1.9's lesson arriving inside the fix for §1.8.* |
| **Five items need Paul and are marked `NEEDS PAUL`:** the 12-hour session |
| TTL, §8.7's unit-override switch, the CI runner's repo list, the |
| disclosure route for the rastrillo SSRF (`SECURITY-DRAFT-rastrillo-signin.md`, |
| -unfiled), and **decision 15's scope**. **Nothing else here is a ruling; |
| -the rest is work, measurement or somebody else's repo.** |
| +unfiled), and **§8.9's constraints and kind list** (what replaced |
| +decision 15's scope when Paul ruled on it). **Nothing else here is a |
| +ruling; the rest is work, measurement or somebody else's repo.** |
| - - **§8.8's cap is unmeasured against the corpus that motivated it, and |
| - the measurement is owed before decision 15 is settled.** The box shows |
| + - **§8.8's cap is unmeasured against the corpus that motivated it.** |
| + Decision 15's principle is settled; whether three is the right cap is |
| + not. The box shows |
| nothing when a search's ruled top-25 carries four or more distinct |
| validated values of one kind. Nobody has counted how many distinct |
| phone numbers a real top-25 for "phone number" carries on Paul's live |
| @@ -4413,21 +4531,24 @@ the rest is work, measurement or somebody else's repo.** |
| nothing. Defensible, and recorded here for the same reason decision |
| 15's widening is: §1.9's lesson is that the load-bearing half of a |
| decision must not live only in a commit message. |
| - - **~~Decision 15 records more than Paul ruled.~~ CLOSED 2026-09-19, |
| - and he reversed it.** The strict reading — no model, ever — was the |
| - spec author's. Paul ruled the opposite and gave the general rule: |
| - deterministic as far as it goes, then a cheap model for what it |
| - cannot reach. §8.9. **What replaces it as open is narrower and is |
| - his:** *which* metadata is worth a query-time model call. Postal |
| + - **Decision 15's scope: CLOSED 2026-09-19, and Paul reversed it.** The |
| + strict reading — no model, ever — was the spec author's. His ruling, |
| + verbatim: *"do as much as possible with deterministic analysis, and |
| + then attempt to build metadata with the best value models via TensorX |
| + that make sense."* (An earlier revision of this item paraphrased that |
| + as "then a cheap model", which put the author's tier choice into his |
| + mouth. He said "best value".) §8.9. |
| + - **NEEDS PAUL — what replaced it, and it is narrower.** Two things, both |
| + his: (a) *which* metadata is worth a query-time model call — postal |
| address is ruled in; opening hours, a role, an effective date, a |
| - foreign company number are all plausible and none is ruled. The spec's |
| - proposed test is "is the answer a verbatim span somebody would want |
| - cited" — anything needing summary is an answer, not metadata, and |
| - §8.2 already owns answers. |
| - - **The struck text is kept below**, because what the amendment |
| - overturned is only legible beside what it replaced. |
| - |
| - - **Decision 15 records more than Paul ruled, and says so rather than |
| + foreign registration number are plausible and unruled; the proposed |
| + test is "is the answer a verbatim span somebody would want cited". |
| + (b) **The implementation constraints in decision 15's row are the |
| + author's**, not his: extractive only, byte-exact spans, validator-backed |
| + kinds never sent, no model-written labels, 🤖 marking, `model.cheap` as |
| + the initial tier. They are proposed as the conditions under which his |
| + ruling is safe to build; any of them is his to override. |
| + - ~~**Decision 15 records more than Paul ruled, and says so rather than |
| quietly widening.** What he asked for on 2026-09-14 is the box, and |
| the constraint — *"do as much deterministic ordering of the data |
| before we go doing inference on it"*. The **"ever"** is the spec |
| @@ -4439,8 +4560,9 @@ the rest is work, measurement or somebody else's repo.** |
| first, a model second where nothing validates — that is one row in |
| §12 promoted and decision 15 amended, not a redesign. The gate |
| review raised this as §1.9's failure repeating: the load-bearing |
| - half of a decision living in a commit message. **Nothing else here is a ruling; the rest is work, measurement |
| -or somebody else's repo.** |
| + half of a decision living in a commit message.~~ *Superseded |
| + 2026-09-19 by Paul's ruling above; kept because what the amendment |
| + overturned is only legible beside what it replaced.* |
| *Struck items are kept with their reasoning rather than deleted. Several |
| closed on 2026-09-05 and 2026-09-06 when the suite doctrine landed, and in |