| index e751aa2..5a08ef6 100644 |
| --- a/docs/superpowers/specs/2026-09-03-memoria-design.md |
| +++ b/docs/superpowers/specs/2026-09-03-memoria-design.md |
| @@ -2905,10 +2905,10 @@ passage or an S6 result.** |
| |
| Paul's motivating example, 2026-09-14: a search for `phone number` should |
| put the answer at the top of the page — *"Tito office number: +353 (0)1 |
| -547 7854"* — and the same for an eircode, a postcode, a VAT number. And |
| -his constraint, given in the same breath, which is the whole of this |
| -section's shape: **do as much deterministic ordering of the data as |
| -possible before any inference runs on it.** |
| +547 7854"* — and the same for an eircode, a VAT number. And his |
| +constraint, given in the same breath, which is the whole of this section's |
| +shape: **do as much deterministic ordering of the data as possible before |
| +any inference runs on it.** |
| |
| **Called the fact box, and not the lookup box, which is what draft 1 |
| called it.** `query.IsQuestion`'s complement is already a lookup in this |
| @@ -2916,179 +2916,183 @@ codebase — `TestALookupPrimesTheFirstResult` is about Enter, not about |
| this — and a second meaning for the word in the same package is the name |
| collision this repo's reviews keep finding. |
| |
| -**It is §8.2's fourth model-free class.** That table already has three — |
| -navigational, numeric/aggregate, who-knows-about-X — and the third is the |
| -closest precedent: it answers a real question from structured markers on |
| -passages that already passed §4.1, with no model anywhere. The fact box is |
| -that shape applied to facts a regular expression can *validate* rather |
| -than merely spot. |
| - |
| **Decision 15: the fact box calls no model, ever.** Not a cheap one, not |
| an extractive one. The kinds it serves are the kinds that have a |
| -validator; a kind with no validator is not in the box and is not the box's |
| -problem (§12). The moment a model picks which span to show, every argument |
| -below collapses into §8.7's forbidden retrieval-augmented calculator with |
| -a different noun. |
| - |
| -**It is a layer before the router, not a row in it.** Draft 1 put it in |
| -§8.2's table as a sixth class, which read as terminal: a query naming a |
| -kind would stop there, and *"what's the office phone number?"* with no |
| -validated number in reach would get no answer where §8.2's single-source |
| -class would have given one. Paul's constraint says deterministic |
| -**first**, not deterministic **instead**. So: the box is attempted, and |
| -then §8.2 routes the query exactly as it would have. Box and answer can |
| -both appear; neither suppresses the other. |
| +validator; a kind with no validator is not in the box and is not the |
| +box's problem (§12). The moment a model picks which span to show, every |
| +argument below collapses into §8.7's forbidden retrieval-augmented |
| +calculator with a different noun. |
| + |
| +**It is a fourth way memoria answers without a model, and it is not a |
| +class.** Drafts 1 and 2 both called it *"§8.2's fourth model-free class"* |
| +while also calling it a layer, which cannot both be true: a row in §8.2's |
| +table is a **route**, and a route is terminal. A query naming a kind would |
| +have stopped at the box, so *"what's the office phone number?"* with |
| +nothing validated in reach would get no answer where the single-source |
| +class would have given one. The box is attempted, and then §8.2 routes the |
| +query exactly as it would have. Paul's constraint is deterministic |
| +**first**, not deterministic **instead**. |
| |
| It does not touch §3.2. No egress, no provider, no embedding — the box |
| works on an instance with AI switched off, which is decision 5's floor and |
| therefore the whole product today. It is part of §8.1's floor and gate 15 |
| -covers it: scriptless, the box arrives with the page. |
| - |
| -#### Retrieval: a second index, not a wider one |
| - |
| -Draft 1 folded the recognisers' output into the chunk's existing indexed |
| -text, so that a document saying `D02 XY45` and never saying "eircode" |
| -would still be reachable by someone searching for one. **The gate review |
| -killed that, and the reason is in `index.go`'s own comments.** |
| - |
| -FTS5 computes a term's IDF across the table, not per column. Injecting the |
| -word `phone` into every chunk that happens to carry a phone number lowers |
| -`phone`'s IDF corpus-wide — degrading the ordinary prose ranking that |
| -`MatchQuery`'s `NEAR` operand was added to fix, and degrading it for |
| -exactly the query the NEAR comment works through. A new column with a low |
| -weight does not avoid this; weights scale the per-column contribution and |
| -leave IDF alone. Draft 1's `FactWeight` was reasoned from `PathWeight`, |
| -and the analogy is unsound: path is weighted low because ancestry |
| -**repeats into every chunk of a document**, which is not why facts would |
| -need holding down. |
| - |
| -**So facts get their own in-memory FTS5 table, with its own IDF space, and |
| -the main index is untouched.** Columns: kind, canonical value, display |
| -span, and the chunk coordinates. Same lifetime as the main index, same |
| -`RemoveDocument`, rebuilt from the sealed chunk rows on wake. Nothing new |
| -exists at rest — it is the postings argument applied to the same bytes for |
| -the same reason (decision 6), and `TestIndexIsBackedByNoFile` covers the |
| -new table for free, which is why this section adds no gate for it. If §12's |
| -index snapshot ever lands, the facts table is part of the index and is |
| -sealed with it. |
| - |
| -That buys three things draft 1 paid for or went without: |
| - |
| -- **No ranking interaction at all**, so the `NEAR` measurement stands. |
| -- **The kind word retrieves the fact directly** — an unlabelled eircode is |
| - found by `eircode`, and a phone number written `+353 (0)1 547 7854` is |
| - found by someone typing `015477854`. This is §8.7's spreadsheet rule |
| - verbatim: *"every date-like header emitted twice, as displayed and |
| - canonicalised"*. |
| -- **No top-k reachability limit.** Draft 1 had to state one — a fact in a |
| - document that did not rank could not fill the box — and defer the fix to |
| - §12. The facts table is the fix, and it is cheaper than the deferral |
| - was: one extra MATCH against a table holding a handful of rows per |
| - document, run only when a kind is named. |
| - |
| -#### Authorization: one ruling, over the union |
| - |
| -The box is memoria speaking a passage, so §8.4's invariant binds it: |
| -every passage in a response passed §4.1 **in the request that produced |
| -it**. The facts table is a second source of candidate passages, and a |
| -second source of passages is exactly the thing that grows a second |
| -authorization path. |
| - |
| -**It does not get one.** The fact hits and the ranked hits are unioned |
| -into a single candidate set and handed to `store.rule` once. One S2 round, |
| -one batch, §6.5's deadline unchanged. Then: |
| - |
| -- the results list renders from the ruled ranked hits, as today; |
| -- the box is built from the ruled fact hits, and from nothing else. |
| - |
| -**The ordering is load-bearing and it is the ruling of this subsection: |
| -rule, then extract, then count candidates.** Extracting before ruling |
| -would let a withheld passage contribute a candidate, and the *count* is |
| -the leak — two candidates where the reader may see one reports the |
| -existence of a passage §4.1 just refused them, and so does a box that |
| -vanishes for passing the cap because of documents the reader cannot see. |
| -Counting after ruling gives the right answer for the right reason: for |
| -that reader, at that moment, there is one. |
| - |
| -**A `NOT_VISIBLE` produces no notice here, and draft 1's gate demanded |
| -one.** §4.1's table makes `OmitSilently` the only disposition that may be |
| -silent, and `sibling.Outcome.Record` implements exactly that. Draft 1's |
| -gate 34 asserted *"the box shows the survivor and §6.5's notice is on the |
| -page"* — which is either unreachable against correct code or, worse, the |
| -thing somebody makes true to turn the gate green. That is §1.10's failure |
| -shape: a rule written while citing the section that forbids it. The two |
| -cases are now separate fixtures and the silent one is the one that |
| -matters. |
| - |
| -**If the box's passage is not among the rendered results, it is appended |
| -to them.** §8.4 says an answer is never served without its list, and a |
| -fact drawn from a chunk the ranked query did not return would otherwise be |
| -an answer whose passage is on no list. Appended last rather than merged by |
| -score: the two queries score against different IDF spaces and their |
| -numbers are not comparable, so a merged position would be a ranking |
| -memoria invented. |
| - |
| -**A chunk deleted between the ruling and the box's read is dropped |
| +covers it. **Nothing on this path is logged**, gate 4's rule applying |
| +unchanged: a fact, its label and its kind are all content, and the query |
| +that named the kind is content too. |
| + |
| +#### One extraction, over the ruled results, and why drafts 1 and 2 had two |
| + |
| +**Candidates are the validated facts carried by the passages this search |
| +already returned and §4.1 already allowed.** There is no second index, no |
| +second query, no ingest-time half and nothing stored. `Store.Search` |
| +produces `Page.Results`; the box is extracted from those and from nothing |
| +else. |
| + |
| +That makes §8.4's invariant structural rather than careful: every passage |
| +in a response passed §4.1 **in the request that produced it**, and a |
| +candidate that is not in `Page.Results` does not exist. There is no |
| +second retrieval path, so there is no second authorization path to get |
| +wrong. |
| + |
| +**Both earlier drafts built a retrieval mechanism for reachability, and |
| +the cap is what killed it.** Draft 1 folded kind words into the main |
| +index; draft 2, after the IDF objection below, gave facts their own |
| +in-memory table queried by kind. Writing the retrieval step out |
| +concretely is what found the contradiction: |
| + |
| +- this section caps candidates at three; |
| +- a facts table is queried `WHERE kind = 'phone'`, which is corpus-wide; |
| +- a real corpus holds dozens of phone numbers — `index/rank_test.go`'s |
| + twelve-chunk fixture already holds two, plus four account numbers and a |
| + VAT number. |
| + |
| +So the cap fires on the first query anyone types, on every kind, |
| +permanently, and the box never appears. **The only way to bound a |
| +corpus-wide fact query is to rank the candidates and keep the best three, |
| +which is exactly what "never by picking" forbids** — and a `LIMIT` before |
| +the ruling is worse still, because a withheld document then eats one of |
| +the slots and "count after the ruling" becomes unimplementable. The gate |
| +review reached this independently and from the other side. |
| + |
| +The ruled top-25 is already a query-scoped candidate set. Once candidates |
| +come from it, the facts table has no query-time job left, and its only |
| +other job was the ingest-time ranking lift. |
| + |
| +**Why the deferred fix is not "index the kind words", measured rather |
| +than asserted.** FTS5 computes a term's IDF across the table, so a term |
| +present in most rows scores near zero. On `modernc.org/sqlite`, 20 rows, |
| +best `bm25()` for `"phone"`: **-2.7169** with the term in one row, |
| +**-0.0000** with it injected into all twenty. A collapse, not a |
| +degradation. |
| + |
| +**Be exact about what that measures, because draft 2 overclaimed it.** |
| +Draft 2 said the injection would degrade *"exactly the query the NEAR |
| +comment works through"*, and on that comment's own fixture it would not: |
| +the chunk holding the office number already contains the word `phone`, so |
| +injecting it there moves that chunk's term frequency and length, not the |
| +corpus IDF. The real exposure is a corpus where **many chunks carry |
| +numbers and few say `phone`** — which is the shape a large real corpus has |
| +and the fixture does not. The number stands; the sentence it was attached |
| +to did not. |
| + |
| +#### What this costs, and it is the thing to measure next |
| + |
| +**The reachability limit is back, and draft 2 claimed to have removed |
| +it.** A fact in a document that does not reach the ruled top-25 is a fact |
| +the box does not have: an eircode written `D02 XY45` in a document that |
| +never says "eircode" is found only if something else about it ranks. |
| +Draft 1 named this and deferred the fix; draft 2's table did not actually |
| +remove it, since that table ran only when a kind was named and was inert |
| +corpus-wide under the cap. §12 carries it. |
| + |
| +**The box's presence depends on the retrieval limit, which is a real |
| +edge and is stated rather than discovered.** `Search(ctx, q, 25)` is a |
| +hard cap, so a fourth distinct value sitting at rank 26 leaves a box of |
| +three, and the same value at rank 25 removes the box entirely. That is |
| +not wrong — the box describes what this search found — but it means the |
| +box is a function of the page, not of the corpus, and nobody should read |
| +it as the latter. |
| + |
| +**Whether the cap makes the box inert on the corpus that motivated it is |
| +unmeasured, and that measurement is owed before decision 15 is settled.** |
| +A real top-25 for "phone number" on Paul's live corpus may well carry more |
| +than three distinct valid numbers. §13 records it. The remedy if it does |
| +is a narrower kind lexicon or a per-kind cap, **not** a rule that picks. |
| + |
| +**§4.1 is unaffected, in the direction that matters.** A reader for whom |
| +passages are withheld draws candidates from a smaller set, never a larger |
| +one — `rule` drops withheld results and §12's over-fetch-and-refill is |
| +deliberately not built, so the slot stays lost. Fewer candidates cannot |
| +be a leak. |
| + |
| +**A chunk row that is gone by the time the box reads it is skipped |
| silently**, exactly as `Store.Search`'s `livePassage` check drops a stale |
| -hit, and for the same reason: the passage is genuinely gone, so its |
| -absence is the true answer rather than a truncation of one. It does not |
| -fail the page. |
| +hit, and for the reason recorded there. It does not fail the search: the |
| +box is a garnish and trading the page for it is the wrong direction. |
| |
| -**And the box needs plaintext the search path does not currently open.** |
| +**And the box needs plaintext the search path does not open.** |
| `Store.Search` opens titles, paths and item ids under their own purposes; |
| -chunk bodies are opened only by `indexDocument` and `Passages`. Snippets |
| -come from FTS5 and are 24 tokens with `<mark>` in them — too short and too |
| -marked-up to take a label from. So the box reads the sealed chunk rows for |
| -its own candidates, after the ruling, under `seal.PurposeChunk`: at most |
| -three rows, since the cap is three. Draft 1 claimed this was cheaper than |
| -the opens the search path already did, which was a statement about a |
| -codebase this is not. |
| +chunk bodies are opened only by `indexDocument` and `Passages`. FTS5 |
| +snippets are 24 tokens with `<mark>` in them — too short and too marked-up |
| +to take a span or a label from. So the box re-reads the sealed chunk rows |
| +for the ruled results, after the ruling, under `seal.PurposeChunk`: at |
| +most 25 rows, one `IN` query, 25 AEAD opens. |
| |
| -#### Why there is no facts table in SQLite |
| +#### Why nothing is stored |
| |
| -The in-memory table above is the index. The design this rejects is a |
| -**sealed, durable** `facts` table, written at ingest, read at query time. |
| -Two grounds carry it, and draft 1 leaned on a third that attacked a design |
| -nobody proposed: |
| +The rejected design is a **sealed, durable** `facts` table, written at |
| +ingest, read at query time. Two grounds, and draft 1's third is withdrawn: |
| |
| - It is **derived state**, so it inherits AGENTS.md's ordering rule in |
| both directions and §5.2's invalidation — a new way for the index and |
| - the database to disagree, added to buy nothing the in-memory table does |
| - not already give. |
| -- It is **a migration and a schema** for a computation that costs |
| - microseconds, and it **freezes the recognisers at the version that ran |
| - at ingest**. That is the chunker problem `Passages`' comment records: |
| - improve a recogniser and every existing document keeps the old answer |
| - until somebody re-ingests it. |
| + the database to disagree, bought for nothing that query-time extraction |
| + does not already give. |
| +- It **freezes the recognisers at the version that ran at ingest**, so |
| + improving one leaves every existing document answering with the old |
| + one until somebody re-ingests it. **This is not the `Passages` problem |
| + and draft 2 cited it as though it were.** That comment records the |
| + opposite lesson — re-running the *chunker* at view time misaligned |
| + citations, and freezing at ingest is the fix — and it does not transfer, |
| + because a recogniser change moves no ordinal and breaks no citation. |
| + The staleness argument stands on its own or not at all. |
| |
| Draft 1's third ground was that a plaintext `kind` column would be an |
| inventory of which kinds of sensitive data the team holds — true, and |
| -irrelevant, because the proposal was for a sealed table. **The honest |
| -residual for a fully sealed one is the per-chunk fact-row count**, which |
| -is real, much smaller, and not on its own a reason. The commit message |
| -called the weak ground decisive; it was not. |
| +irrelevant, because the proposal was for a sealed table. The honest |
| +residual for a sealed one is the per-chunk fact-row count, which is real, |
| +much smaller, and not on its own a reason. |
| + |
| +**Nothing new exists at rest, and nothing new exists in the index |
| +either.** That is the strongest property this section has and it came |
| +from deleting things. |
| |
| #### The recognisers, and what is not one |
| |
| A kind is in the box when a query word names it **and** a candidate |
| -validates. Validation is not pattern-spotting: an eircode has a fixed |
| -routing-key alphabet, an IBAN has a mod-97 check, an Irish VAT number has |
| -a checksum. |
| +validates. |
| |
| | Kind | Primed by | Validated by | |
| |---|---|---| |
| -| `eircode` | `eircode`, `postcode`, `post code` | routing key from the published letter set, plus four characters from it | |
| -| `phone` | `phone`, `telephone`, `tel`, `mobile`, `phone number` | canonicalises to E.164 or to an Irish national number; length and prefix bounds | |
| +| `eircode` | `eircode`, `postcode`, `post code` | routing key — a letter from `ACDEFHKNPRTVWXY`, a digit, then a digit or `W` — plus four characters from that set and the digits | |
| | `iban` | `iban`, `bank details` | country prefix, length by country, mod-97 | |
| -| `vat` | `vat`, `vat number` | Irish VAT format and its checksum | |
| -| `email` | `email`, `email address`, `e-mail` | `local@domain`, domain carrying a dot, no whitespace — **and see below** | |
| - |
| -**`url` was in draft 1 and is dropped.** `scheme://host` is |
| -pattern-spotting with no validator behind it, so every link in every |
| -document qualifies; on any real corpus a search for "link" passes the cap |
| -immediately and the box is inert. It also collides with §8.5's *"refuse |
| -links to non-suite hosts"* for no gain, since a document full of links is |
| -what the results list is for. |
| +| `vat` | `vat`, `vat number` | modern Irish format and its check letter | |
| +| `phone` | `phone`, `telephone`, `tel`, `mobile`, `phone number` | **structural, and see below** | |
| +| `email` | `email`, `email address`, `e-mail` | `local@domain`, domain carrying a dot, no whitespace — **weak, and see below** | |
| + |
| +**Three of these are checksums and two are not, and the section says so |
| +rather than implying five.** An eircode's routing key is a closed set, an |
| +IBAN has a mod-97 check, an Irish VAT number has a check letter derived |
| +from its digits. These can be wrong, and a wrong one is refused. |
| + |
| +**`phone` has no checksum, so it gets a structural rule that is stated |
| +here rather than left to the code: a bare run of digits is never a phone |
| +number.** The span must carry a `+`, or separators grouping it. This is |
| +not a hack bolted on to a weak validator — it is how documents write phone |
| +numbers, and without it `rank_test.go`'s own corpus yields false |
| +positives: `DUNS/D-U-N-S number is 985632364` is nine digits and a |
| +plausible Irish mobile. **The cost is stated too:** a document writing |
| +`0871234567` with no separators is not boxed. That is the safe direction, |
| +and it is the direction a refusal should fail in. |
| |
| **`email` is kept and its weakness is stated rather than hidden.** |
| `local@domain` is closer to pattern-spotting than to validation. On a |
| @@ -3098,18 +3102,20 @@ signature qualifies, the cap is passed, and the box is inert for that |
| kind. That is the cap working, and the failure direction is silence rather |
| than a wrong address. |
| |
| +**`url` was in draft 1 and is dropped.** `scheme://host` is |
| +pattern-spotting with nothing behind it, so every link in every document |
| +qualifies and the box is inert immediately. It also collides with §8.5's |
| +*"refuse links to non-suite hosts"* for no gain. |
| + |
| **Nothing in the box is ever an anchor.** The value is text; the only link |
| the box carries is its citation, to `/documents/{id}#p{n}`. No `mailto:`, |
| -no `href` built from a document's bytes — §8.5's *"no remote loads in |
| -rendered answers"* is about exactly this and it does not need a second |
| -statement, only an obeying. |
| +no `href` built from a document's bytes. |
| |
| -**`address` on its own primes nothing**, and the reason is the interesting |
| -one: it is ambiguous between a postal address and an email address, and |
| -only one of those has anything resembling a validator. `email address` |
| -primes `email`. A postal address has no checksum, no fixed grammar and no |
| -length — there is nothing to validate, so under decision 15 it is not a |
| -kind. §12 carries it. |
| +**`address` on its own primes nothing.** It is ambiguous between a postal |
| +address and an email address, and only one of those has anything |
| +resembling a validator. `email address` primes `email`. A postal address |
| +has no checksum, no fixed grammar and no length, so under decision 15 it |
| +is not a kind. §12 carries it. |
| |
| **`number` on its own primes nothing either.** The lexicon maps |
| distinctive words and phrases; generic words count only in combination. |
| @@ -3119,113 +3125,127 @@ distinctive words and phrases; generic words count only in combination. |
| safe direction is not priming"* — because a wrong prime navigates away |
| from a page somebody was reading. The fact lexicon can afford to be |
| generous, and the reason is the validator, not a shared instinct: "mobile |
| -app" primes `phone`, finds no phone number, and renders nothing. A |
| -generous lexicon costs a box that does not appear. |
| +app" primes `phone`, finds no phone number, and renders nothing. |
| |
| **It complements Enter-priming rather than competing with it.** |
| -`IsQuestion` declines to prime a result for anything ending in `?` or |
| -opening with an interrogative, so "what's the office phone number?" |
| -navigates nowhere today. Those are disproportionately the queries a fact |
| -box answers. Where both fire — "office phone number", no question mark — |
| -**Enter still goes to the first result, not to the box's citation.** The |
| -box is a thing to read, not a destination; changing what Enter means |
| -depending on whether a regular expression matched is the unreliable-Enter |
| -failure `PrimesFirstResult` exists to avoid. |
| - |
| -#### The span, the canonical form, and §8.7's parser rule |
| - |
| -Two strings come out of a recogniser and they have different jobs. |
| - |
| -**The span is what the document wrote**, byte for byte, including the |
| -punctuation, the `(0)`, and any extension after it. It is what the box |
| -displays and the only thing the box displays. |
| - |
| -**The canonical form is match-only.** It is what the facts table indexes |
| -so that `015477854` finds `+353 (0)1 547 7854`, and what the validator |
| -runs on. **It is never rendered and never arithmetic is reported from |
| -it.** |
| - |
| -That distinction is load-bearing against two of this spec's own rules, and |
| -draft 1 tripped over both: |
| +`IsQuestion` declines to prime for anything ending in `?` or opening with |
| +an interrogative, so "what's the office phone number?" navigates nowhere |
| +today, and those are disproportionately the queries a fact box answers. |
| +Where both fire — "office phone number", no question mark — **Enter still |
| +goes to the first result, not to the box's citation.** The box is a thing |
| +to read, not a destination; changing what Enter means depending on whether |
| +a regular expression matched is the unreliable-Enter failure |
| +`PrimesFirstResult` exists to avoid. |
| + |
| +#### The span, the canonical form, and this spec's two arithmetic rules |
| + |
| +**The span is what the document wrote**, byte for byte, including |
| +punctuation, the `(0)`, and any extension written after it. It is what the |
| +box displays and the only thing the box displays. Separators inside a span |
| +include **U+00A0**, because Word and Notion paste non-breaking spaces and |
| +the chunker does not normalise interior ones — a recogniser written with |
| +Go's `\s`, which does not match U+00A0, silently finds nothing in pasted |
| +text. |
| + |
| +**The canonical form is match-only.** It is what candidate identity is |
| +keyed on, and what the validator runs on. It is **never rendered and never |
| +an operand.** An Irish national number canonicalises to E.164 so that |
| +`01 547 7854` and `+353 (0)1 547 7854` are recognised as one number rather |
| +than two candidates. |
| + |
| +Two of this spec's own rules have to be read against that, and both are |
| +ruled here rather than left to be found: |
| |
| - **§8.7 forbids *"memoria parsing a display string back into a number … |
| - a second parser"*.** What it forbids is parsing a display string into a |
| - value memoria then *shows or computes with* — the `"€1.2M"` case, where |
| - the parse feeds an answer. A canonical form that exists only as an index |
| - term, is never displayed, and is never an operand is not that. Said |
| - here explicitly so the next review finds the ruling rather than the |
| - apparent contradiction. |
| + a second parser"*.** What it forbids is a parse that feeds something |
| + memoria then shows or computes with — the `"€1.2M"` case. A canonical |
| + form that is never displayed and is never an operand is not that. |
| - **Decision 10 says memoria never does arithmetic.** A mod-97 check |
| - digit is arithmetic, and it is not a number memoria reports — it decides |
| + digit is arithmetic. It is not a number memoria reports: it decides |
| whether a string is an IBAN at all. Decision 10 governs numbers in |
| - answers. A checksum is a predicate. Also stated rather than left to be |
| - found. |
| - |
| -**Draft 1's own example failed draft 1's own validator**, which is the |
| -review finding worth keeping visible: `+353 (0)1 547 7854` is not E.164 |
| -— E.164 has no spaces and no `(0)` — so "validated by E.164" was a rule |
| -the motivating case did not pass. The table above now says *canonicalises |
| -to* E.164, which is what was always meant and is what the code will do. |
| - |
| -#### The label, and the sentence it comes from |
| - |
| -**The label is taken from the document and is never composed.** Three |
| -sources, in order, first match wins: |
| - |
| -1. **The run back to the start of the enclosing sentence or line, if it |
| - ends in a separator** (`:`, `—`, `-`, `=`, tab) and is **60 characters |
| - or fewer**. `Office: +353 …` yields `Office`. So does *"The office |
| - phone number is: +353 …"*, yielding `The office phone number is`. |
| -2. **The chunk's heading trail** — `Tito › Contact`. |
| -3. **The document title**, for an upload with no headings at all, which is |
| - the common shape for a pasted file and must not fall through to an |
| - empty label. |
| - |
| -**Draft 1 required the value to be at the start of its own line and capped |
| -the label at 40 characters, and the repo's own fixture is the case that |
| -breaks.** `index/rank_test.go:56` — built from the live instance where |
| -Paul hit this — holds one chunk reading: |
| + answers; a checksum is a predicate. **That is a narrowing of Paul's |
| + decision made by the spec author, and it is filed in §13 beside decision |
| + 15's, rather than one being marked and the other slipped through.** |
| + |
| +**Draft 1's own example failed draft 1's own validator**, which is worth |
| +keeping visible: `+353 (0)1 547 7854` is not E.164 — E.164 has no spaces |
| +and no `(0)` — so "validated by E.164" was a rule the motivating case did |
| +not pass. The table says *canonicalises to*, which is what was meant. |
| + |
| +#### The label, and the four ways it goes wrong |
| + |
| +**The label is taken from the document and is never composed.** Given the |
| +span's position in the chunk text: |
| + |
| +1. **Take the run back to the start of the enclosing sentence or line.** |
| + **If that run contains another recognised span, start after it** — in |
| + `Phone: 01 547 7854 - 01 598 1234` the second number's run would |
| + otherwise be `Phone: 01 547 7854 -`, labelling the second number with |
| + the first one's value. |
| +2. **If what remains ends in a separator** — `:`, `—`, `–`, `-`, `=`, tab |
| + — **and is non-empty after removing it and trimming, and is 60 |
| + characters or fewer, that is the label.** |
| +3. **Empty after the separator → fall through.** A markdown bullet is the |
| + case: the chunker keeps lines verbatim, so `- +353 1 547 7854` gives a |
| + run of `-`, which "ends in a separator" and yields nothing. Two such |
| + bullets give two empty labels, collide, and would have suppressed the |
| + box for a plain contact list. |
| +4. **Longer than 60 → no box for that kind.** Not the trail instead. The |
| + document labelled this value and the label will not fit, so |
| + substituting a vaguer one is memoria replacing the document's words |
| + with its own: *"The number you must never give to customers because it |
| + rings the CEO directly is: 01 598 1234"* becomes `Contacts`, sitting |
| + beside `Reception — 01 547 7854`. Dropping the candidate instead is |
| + worse, because that turns a two-candidate ambiguity into a confident |
| + single answer. |
| +5. **Otherwise the chunk's heading trail**, and if there is none — an |
| + upload with no headings, which is the common pasted-file shape — **the |
| + document title**, never the empty string. |
| + |
| +**Draft 1 required the value to start its own line and capped the label at |
| +40, and the repo's own fixture is what breaks that.** |
| +`index/rank_test.go:56` — built from the live instance where Paul hit this |
| +— holds one chunk reading: |
| |
| > *"Turn the knob. The office phone number is: +353 (0)1 547 7854. We |
| > also have the following unused phone numbers: +353 (0)1 598 1234."* |
| |
| -Under draft 1 both numbers are mid-paragraph, both fall through to the |
| -heading trail, and both come back labelled `Office Address`: two |
| -identical labels, one of which the document explicitly calls **unused**. |
| -The box would have shown both as equals while the disambiguating word sat |
| -one clause away, unread. That is §8.7's *"cites correctly while being |
| -wrong"* in a lookup coat, and draft 1's "never by picking" made it |
| -unfixable from inside. |
| - |
| -Sentence-anchored labelling fixes it by doing less, not more: the two |
| -candidates now carry *"The office phone number is"* and *"We also have |
| -the following unused phone numbers"*, and the reader can see which is |
| -which because the document said so. Nothing was summarised, ranked or |
| -chosen. |
| - |
| -**The sentence split is naive and conservative on purpose** — a full stop, |
| -question mark or exclamation mark followed by a space, or a newline. It is |
| -not a sentence tokeniser and must not grow into one; where it gets the |
| -boundary wrong the run fails the separator test and the label falls to the |
| -trail, which is the safe direction. |
| +Under draft 1 both numbers are mid-paragraph, both fall to the heading |
| +trail, and both come back labelled `Office Address`: two identical labels, |
| +one of which the document explicitly calls **unused**. The box would have |
| +shown them as equals with the disambiguating word one clause away. That is |
| +§8.7's *"cites correctly while being wrong"* in a lookup coat. Sentence |
| +anchoring fixes it by doing less: the labels become *"The office phone |
| +number is"* and *"We also have the following unused phone numbers"*, and |
| +the reader can see which is which because the document said so. |
| + |
| +**The sentence split is naive, and draft 2's claim that it fails safely is |
| +false.** A boundary placed too *late* yields a shorter run that still |
| +passes every test above: `Dr. Smith's direct line: 01 …` yields |
| +`Smith's direct line`, and `St. Stephen's Green office: …` yields |
| +`Stephen's Green office`. **That is a label the document did not write, |
| +produced by the rule whose entire justification is that it never |
| +composes.** A short fixed exception list — `Dr. Mr. Mrs. Ms. St. No. ext. |
| +Ltd. Co.` — covers the common cases and is **stated as incomplete**. It |
| +must not grow into a sentence tokeniser; where it is wrong it is wrong in |
| +the composing direction, and saying so is the difference between a known |
| +limit and a discovered one. |
| |
| **60 characters is a guess.** What is not a guess is that it must clear |
| -the fixture above, whose longer label is 47. |
| - |
| -Nothing here title-cases, expands, truncates or rewords. A label the |
| -document did not write is a label memoria wrote. |
| +the motivating fixture, whose longer label is 47. |
| |
| -So Paul's example renders as `The office phone number is — +353 (0)1 547 |
| -7854`, under the heading trail, carrying the same |
| -`/documents/{id}#p{n}` link every result carries. |
| - |
| -#### Candidates, ambiguity, and the cap |
| +#### Candidates, ambiguity, the cap, and one box per kind |
| |
| **A candidate is one canonical value.** The same number in five forwarded |
| signatures is **one** candidate, cited to the best-ranked passage carrying |
| -it — not five, and not a reason to pass the cap. Two different numbers in |
| -one chunk are **two** candidates; the fixture above is exactly that shape, |
| -and treating a chunk as the unit would have hidden the second one. |
| +it. Two different numbers in one chunk are **two** — the fixture above is |
| +exactly that shape, and treating a chunk as the unit would hide the |
| +second. |
| + |
| +**Every rule below is applied per kind, and a multi-kind query renders one |
| +box per kind.** A cap counted across kinds would let two phone numbers and |
| +two email addresses suppress each other, and a collision check across |
| +kinds would suppress a phone and an email that happened to share a label. |
| |
| §8.7's slot rule is *"two or more → ambiguity, never extra evidence, and |
| never collapsed"*, and it resolves that by showing no number. **The fact |
| @@ -3234,23 +3254,28 @@ An average built from two of three operands is a *wrong number*: the |
| question had one answer and memoria produced a different one. Two phone |
| numbers are two *right answers* to an underspecified question. §8.7's |
| ambiguity is incompleteness, which can only be refused; this one is |
| -underspecification, which is honestly served by showing both, labelled and |
| -cited. |
| +underspecification, honestly served by showing both, labelled and cited. |
| |
| -**Never by picking.** Not highest BM25, not first, not longest. The rank |
| -ordering sorts a result list and carries no claim about which number is |
| -the office one. |
| +**Never by picking.** Not highest BM25, not first, not longest. Rank order |
| +sorts a result list and carries no claim about which number is the office |
| +one. |
| |
| -**Two candidates that would render the same label render no box.** This is |
| -the guard behind the sentence-anchored rule rather than a replacement for |
| -it: where both fall through to the same heading trail, the page would show |
| -one label twice against two different values, which is not an enumeration |
| -but a contradiction with no way for the reader to resolve it. The results |
| -list, with its snippets, is the honest surface for that. |
| +**A box row carries the document title and the heading trail as well as |
| +the label**, exactly as a result row carries both and for the reason |
| +recorded there: two pages called "Contact" under different documents are |
| +the same row to a reader who cannot see which is which. |
| |
| -**Capped at three inclusive: three renders, four renders nothing.** A box |
| -enumerating twelve numbers is the result list with extra steps and a false |
| -air of authority. |
| +**Two candidates whose whole citation line collides — same title, same |
| +trail, same label — render no box.** Keyed on the line rather than the |
| +label alone, because a supplier page and the office page both falling to |
| +the trail `Contact` are perfectly disambiguated by their titles, and |
| +suppressing that box would be losing a real answer to a rule aimed at a |
| +different problem. Where the whole line does collide, the page would show |
| +one identical row twice against two different values — not an enumeration |
| +but a contradiction with no way for the reader to resolve it, and the |
| +results list is the honest surface for that. |
| + |
| +**Capped at three inclusive: three renders, four renders nothing.** |
| |
| #### Where it sits on the page |
| |
| @@ -3258,18 +3283,17 @@ Inside the `results` template definition, not above it. `results.html` is |
| rendered two ways from one definition — inside the page on a scriptless |
| submit, alone on `GET /search` for `search.js` — and a box outside it |
| would sit above a fragment that swapped underneath it, leaving query N−1's |
| -answer over query N's results on every keystroke. That is §8.4 inverted, |
| -several times a second. |
| +answer over query N's results on every keystroke. |
| |
| **Below §6.5's notices and above the list.** `results.html` already puts |
| the withholding callout above the results because *"a reader who scrolls |
| past the last result has already decided what the page means"*. A |
| -confident single answer rendered above the line saying the page is partial |
| -is the same mistake with the order reversed. |
| +confident single answer above the line saying the page is partial is the |
| +same mistake reversed. |
| |
| -**Search page only.** `/documents/{id}` renders a whole document and |
| -answers no query; decision 14 makes those the only two passage surfaces |
| -and this is a feature of one of them. |
| +**Search page only.** Decision 14 makes the search page and |
| +`/documents/{id}` the only two passage surfaces, and this is a feature of |
| +one of them. |
| |
| **Zero candidates renders nothing at all** — not "no phone number found". |
| The corpus is not the world, and a confident absence is this codebase's |
| @@ -3279,8 +3303,7 @@ own worst failure shape. |
| traces to a cited passage. **Its second does not.** "An all-dropped answer |
| says so rather than rendering empty" exists because an empty 🤖 paragraph |
| is indistinguishable from AI being off; a box that does not appear is |
| -indistinguishable from nothing, which is what it means. Draft 1 extended |
| -gate 14 whole and contradicted itself two paragraphs earlier. |
| +indistinguishable from nothing, which is what it means. |
| |
| #### Injection, and the one place ambiguity pays for itself |
| |
| @@ -3294,22 +3317,27 @@ it. |
| Two things bound it and neither is a fix: |
| |
| - The value is a cited span of a passage that passed §4.1, so the reader |
| - can open it, and a poisoned document is a document somebody in this team |
| - put in this corpus. |
| + can open it, and a poisoned document is one somebody in this team put in |
| + this corpus. |
| - **The ambiguity rule fails safe against poisoning.** A planted second |
| number does not replace the real one and does not outrank it — it makes |
| - the box show both, or show nothing if the labels collide, or vanish past |
| - the cap. The attack that works against a single-answer box degrades into |
| - a visibly contradictory page, which is a property worth keeping rather |
| - than an accident to optimise away. |
| + the box show both, or show nothing if the whole citation line collides, |
| + or vanish past the cap. The attack that works against a single-answer |
| + box degrades into a visibly contradictory page, which is a property |
| + worth keeping rather than an accident to optimise away. |
| |
| #### Sequencing |
| |
| -**Slice 0**, with one exception. Recognisers, the facts table, the |
| -lexicon, the box and gates 35–38 need no sibling and no model, and the |
| -box is part of decision 5's floor. **Gate 34 needs slice 1**, because its |
| -fixtures are S2 dispositions; it is written with the rest and runs when |
| -`memoriatest`'s sibling stub can answer, which is now. |
| +**Slice 0, entirely.** Recognisers, the lexicon, the label rule, the box |
| +and gates 35–37 need no sibling and no model. **Gate 34 needs a custody |
| +fake that rules per item**, which `fixedCustody` is not — it returns one |
| +disposition for every candidate from a single field — so the gate lands |
| +with a fake extended to answer per item id, in `internal/memoria` at store |
| +level, which is where its "assert the struct, not the page" demand is |
| +natural anyway. Drafts 1 and 2 both said this gate could run against |
| +"`memoriatest`'s sibling stub, which exists"; there is no such stub, and |
| +that is the renamed-test failure §10 opens with, committed twice in the |
| +section that cites it. |
| |
| --- |
| |
| @@ -3914,110 +3942,132 @@ The gates: |
| are distinguishable after five submissions, and a refused address |
| consumes no per-IP budget either. The budget is inside signin and not |
| reachable from here; it closes with the upstream fix. |
| -34. **The fact box never speaks a withheld passage, and the count happens |
| - after the ruling** (§8.8). Needs slice 1's S2, which is the only gate |
| - here that does. Four fixtures, and the shape of each is chosen so the |
| - author's own mutations are not the ones it rests on: |
| +34. **The box never speaks a withheld passage, and every count happens |
| + after the ruling** (§8.8). Lives in `internal/memoria`, at store |
| + level, against **a custody fake that rules per item id** — |
| + `fixedCustody` returns one disposition for every candidate from a |
| + single `give` field and cannot express these fixtures at all. Six, |
| + and the last three are the ones an author does not write: |
| |
| - **Presence.** Two sibling documents, two different validated |
| - eircodes, both `VISIBLE`. The box renders and carries **two** |
| - candidates. Without this the three absences below are assertions |
| - about an empty world — §10's second rule. |
| - - **`NOT_VISIBLE` is silent.** Rule one of them `NOT_VISIBLE`. The box |
| - shows **exactly the survivor, as a single answer, and `Notices` is |
| - empty.** Draft 1 of §8.8 demanded a notice here, which §4.1's table |
| - forbids and `Outcome.Record` will not produce — a gate that could |
| - only be made green by building the existence oracle §4.1 exists to |
| - deny. The empty-notice assertion is now the load-bearing half. |
| - - **`UNKNOWN` is loud.** Same fixture, `UNKNOWN` instead: the box |
| - shows the survivor **and** §6.5's notice is on the page. This is |
| - what draft 1 meant and attached to the wrong disposition. |
| - - **The cap is counted after the ruling too.** Four documents, four |
| - distinct values, one `NOT_VISIBLE` — the box shows **three**. Under |
| + eircodes, both `VISIBLE` → the box carries **two** candidates. |
| + Without this the absences below are assertions about an empty |
| + world (§10's second rule). |
| + - **`NOT_VISIBLE` is silent.** One of them withheld → the box shows |
| + **exactly the survivor, and `Notices` is empty.** Draft 1 demanded |
| + a notice here, which §4.1's table forbids and `Outcome.Record` will |
| + not produce: a gate that could only be made green by building the |
| + existence oracle §4.1 exists to deny. |
| + - **`UNKNOWN` is loud.** Same fixture, `UNKNOWN` instead → the |
| + survivor **and** §6.5's notice. |
| + - **The cap counts after the ruling.** Four documents, four distinct |
| + values, one `NOT_VISIBLE` → **three** candidates and a box. Under |
| "extract over the ruled set, count over the unruled one" the reader |
| loses the box entirely because of a document they may not see, and |
| - a two-document fixture cannot tell the two implementations apart. |
| - |
| - **Assert the struct, not the page.** Each case asserts the candidate |
| - tuple `(display span, DocumentID, Ordinal)` and the notice slice — |
| - never the rendered copy. Two dispositions rendering an identical page |
| - is how `Outcome.Record` filed silent rulings undetected (§1.9), and a |
| - citation cross-wired to the withheld document survives any assertion |
| - that reads the value alone. |
| -35. **Candidates are canonical values; ambiguity is enumerated, never |
| - collapsed; the cap is three** (§8.8). One gate because the three |
| - rules share a definition and a gate that splits them cannot see the |
| - interactions: |
| + no two-document fixture can tell those implementations apart. |
| + - **The citation is the survivor's.** The **same** eircode in a |
| + `VISIBLE` and a `NOT_VISIBLE` document → one candidate, cited to |
| + the visible passage. Every fixture above uses distinct values per |
| + document, so "cite from `all` instead of `page.Results`" passes all |
| + of them; this is the only one that can see it. |
| + - **Collision counts after the ruling too.** Two documents sharing a |
| + title and trail, both falling to the trail for a label, one |
| + `NOT_VISIBLE` → the survivor **is** boxed. Computed over the |
| + unruled set it collides and the box vanishes, which is the same |
| + leak as the cap case wearing the collision rule's coat. |
| + |
| + **Assert the struct, not the page** — the candidate tuple |
| + `(span, title, trail, label, DocumentID, Ordinal)` and the notice |
| + slice, never rendered copy. Two dispositions rendering an identical |
| + page is how `Outcome.Record` filed silent rulings undetected (§1.9). |
| +35. **Candidates are canonical values, ambiguity is enumerated, and the |
| + cap is three** (§8.8). One gate, because the rules share a definition |
| + and a gate that splits them cannot see the interactions: |
| |
| - one validated number → one box, one value **(presence)**; |
| - - a second, different number → **both named**, neither dropped, and |
| - the higher-scoring one not presented as the answer; |
| + - a second, different number → **both named**, neither dropped, the |
| + higher-scoring one not presented as the answer, **and in first-seen |
| + order** — order asserted, since a map's range is not an order; |
| - **three → still a box. Four → none.** Both, because a cap written |
| - `>= 3` passes a suite that tests only one, two and four; |
| - - **the same number in five documents → one candidate and a box**, |
| - cited to the best-ranked passage. This is the commonest real shape |
| - — a signature repeated through forwarded mail — and an |
| - implementation counting passages rather than values kills the box |
| - on it; |
| - - **two numbers in one chunk → two candidates.** `index/rank_test.go`'s |
| - office fixture is exactly this, and an implementation counting |
| - chunks silently hides the second; |
| - - **two candidates whose labels collide → no box.** |
| - |
| - Mutation: return `candidates[0]` when there is more than one, and case |
| - 2 goes red. Count passages instead of canonical values and cases 4 and |
| - 5 go red in opposite directions, which is the pair worth having. |
| + `>= 3` passes a suite testing only one, two and four; |
| + - **two spellings of one number** — `+353 1 547 7854` and |
| + `01 547 7854` → **one** candidate. A dedupe keyed on the span |
| + rather than the canonical form passes a same-spelling fixture; |
| + - the same number in five documents → one candidate, cited to the |
| + best-ranked; |
| + - **two numbers in one chunk → two candidates.** |
| + `index/rank_test.go`'s office fixture is exactly this, and an |
| + implementation counting chunks hides the second; |
| + - **a two-kind query with two of each** → **two boxes**, not none. |
| + A cap counted across kinds suppresses both, and this is the fixture |
| + that sees it. |
| + |
| + Mutation: return `candidates[0]` when there is more than one and case |
| + 2 goes red. Count passages instead of canonical values and cases 4 |
| + and 5 go red in opposite directions, which is the pair worth having. |
| 36. **The displayed value is byte-identical to the document's** (§8.8). |
| The canonical form exists for matching and must never reach the page. |
| - Three fixtures, and the second and third are the ones an author does |
| - not think of: `+353 (0)1 547 7854`; the same number followed by |
| - `ext. 12`; and one written with a non-breaking space. Assert the exact |
| - bytes of the span in every case. Mutation: normalise ` ` to a |
| - space before matching **and display the normalised span** — the shape |
| - a recogniser naturally grows, and invisible to a fixture using only |
| - ASCII spaces. |
| -37. **The label is the document's, or the fallback's, never a fragment** |
| - (§8.8). Five fixtures: a labelled line (`Office: …` → `Office`); a |
| - value mid-paragraph after a separator (→ the sentence-anchored run, |
| - which is the fixture `rank_test.go` already holds and the case draft |
| - 1 got wrong); a value mid-sentence with **no** separator before it |
| - (→ the heading trail, never the preceding clause); a run one |
| - character over the 60-cap (→ the heading trail); and an upload with |
| - **no headings at all** (→ the document title, never the empty |
| - string). Mutation: drop the separator condition and fixture 3 yields a |
| - composed label; drop source 3 and fixture 5 yields `""`, which every |
| - fixture carrying a heading would have hidden. |
| -38. **The facts table does not disturb the main ranking, and it reaches |
| - what the main index cannot** (§8.8). The two halves of the retrieval |
| - design, and the first is the one draft 1 would have failed: |
| - |
| - - **Reachability, the presence half.** A chunk whose only connection |
| - to the word `eircode` is that it contains one is returned for the |
| - query `eircode`; a phone number written `+353 (0)1 547 7854` is |
| - returned for `015477854`. Without this the whole ingest-time half |
| - is unasserted. |
| - - **Non-interference, measured.** Re-run `TestOrRecallStillRanksBestMatchFirst` |
| - and the `NEAR` measurement in `index/rank_test.go` with the facts |
| - table populated, and assert the scores are **unchanged**. Draft 1 |
| - folded kind words into the main index, where FTS5's table-wide IDF |
| - would have diluted `phone` across every chunk carrying a number — |
| - degrading the exact query that comment works through. A separate |
| - table is what makes this assertable at all. |
| - |
| - Mutation: index the kind words into the main table's body column and |
| - the second half goes red on the recorded scores; drop the canonical |
| - form and the first half's second case goes red. |
| + Fixtures, with the expected bytes written out in each: `+353 (0)1 547 |
| + 7854`; the same followed by `ext. 12`, where the expected span is |
| + `+353 (0)1 547 7854 ext. 12` and the recogniser must be specified to |
| + include it; and one written with **U+00A0** between the groups. |
| + Mutation: normalise ` ` to a space before matching **and display |
| + the normalised span** — the shape a recogniser naturally grows, and |
| + invisible to any fixture using only ASCII spaces. **Note which way |
| + the NBSP fixture fails:** Go's `\s` does not match U+00A0, so a |
| + recogniser written with it finds nothing and the fixture goes red on |
| + presence rather than on the bytes. That is the design demand, not an |
| + accident. |
| +37. **The label is the document's, or the fallback's, and never composed** |
| + (§8.8's five rules, one fixture each plus the four failures the gate |
| + review found): |
| + |
| + - `Office: …` → `Office`; |
| + - the motivating fixture's mid-paragraph value after a separator → |
| + the sentence-anchored run, which draft 1 got wrong; |
| + - a value mid-sentence with **no** separator before it → the heading |
| + trail, never the preceding clause; |
| + - **a markdown bullet**, `- +353 1 547 7854` → the run is `-`, empty |
| + once the separator is removed, so it **falls through to the trail** |
| + rather than yielding an empty label. Two such bullets must still |
| + box, which an implementation returning `""` turns into a collision; |
| + - **two values adjacent**, `Phone: 01 547 7854 - 01 598 1234` → the |
| + second label must not be `Phone: 01 547 7854`. Drop the |
| + earlier-span rule and this fixture labels one number with another; |
| + - **an abbreviation**, `Dr. Smith's direct line: 01 …` → the whole |
| + run, not `Smith's direct line`. This is the composing direction and |
| + the exception list is the only thing holding it; |
| + - **60 and 61 characters**, both. At 60 the label renders; at 61 |
| + there is **no box for that kind** — not the trail instead. A cap |
| + written `< 60` passes a suite that only tests "one over"; |
| + - an upload with **no headings at all** → the document title, never |
| + `""`. Every fixture carrying a heading hides this. |
| +38. ~~**The facts table does not disturb the main ranking.**~~ **Struck |
| + before it was built, 2026-09-14.** §8.8 draft 3 removed the facts |
| + table, so there is nothing to prove non-interference for — and the |
| + gate as drafted was unreachable anyway: a test that injects a term |
| + into every row and asserts `bm25()` collapses is a test of SQLite, |
| + and no mutation of this repo can turn it red. AGENTS.md's rule is |
| + that a defence nothing can reach is the same object as a guard |
| + written to pass, and that the answer is to make it reachable or |
| + delete it, never to leave it with a comment. The **measurement** |
| + survives as prose in §8.8, where it explains why §12's eventual |
| + reachability fix is not "index the kind words". The number kept its |
| + place in this list rather than being reused, per §13's convention for |
| + struck items. |
| |
| ## 11. Sequencing |
| |
| **Slice 0 — buildable today, no contract, no blockers.** Scaffold, sealed |
| store, §7 proven end to end: chunk, seal, wake, BM25 search, scriptless |
| UI with citations, gates 1–4 and 15. **§8.8's fact box lands here too**, |
| -added 2026-09-14: recognisers, the in-memory facts table, the kind |
| -lexicon, the box, and gates 35–38, none of which needs a sibling or a |
| -model. Gate 34 is the exception — its fixtures are S2 dispositions — and |
| -it is written with the rest and runs against slice 1's stub, which |
| -exists. Content by upload and paste only, so |
| +added 2026-09-14: recognisers, the kind lexicon, the label rule, the box, |
| +and gates 35–37 — nothing stored, nothing indexed, no sibling and no |
| +model. **Gate 34 comes with it and needs a custody fake that rules per |
| +item id**, which `fixedCustody` is not. Drafts 1 and 2 of §8.8 both said |
| +it could run against "`memoriatest`'s sibling stub, which exists"; no such |
| +stub exists, and writing that twice in the section that cites §10's |
| +renamed-test rule is the failure that rule is about. Content by upload and paste only, so |
| §4.2 governs and no person ref is needed. **No embeddings, therefore no |
| egress at all.** This is the design Docs §15 item 2 deferred for want of |
| one, so proving it unblocks Docs regardless of memoria's fate. |
| @@ -4082,8 +4132,9 @@ commit. Fixed 2026-09-05; §1.9's lesson arriving inside the fix for §1.8.* |
| | **A source-of-record surface, so a hand-off can include memoria's own material** | Someone wants "save this answer as a Doc" to work for answers built on the team's own uploads. Suite §7 makes the *receiver* check every source with its app of record, and memoria serves nobody to ask (decision 7), so such a hand-off is refused today — §9.1. Opening it needs its own decision, its own §3.2 leak entry and its own gate, and it is Paul's: decision 7 is settled | |
| | Calendar as a source | Calendar has a spec and content; bodies off by default even then | |
| | Mail beyond forward-or-label | Never by sweep. A mail repo exists **and** a per-thread audience maps onto a team | |
| +| **Reaching a fact the ranked query does not** | Somebody reports a fact they can see in a document the box did not surface. §8.8's candidates are the ruled top-25, so an eircode in a document that never says "eircode" is found only if something else about it ranks. **Whatever this turns out to be, it is not "index the kind words"** — §8.8 measures that collapsing `bm25()` for the injected term from 2.7169 to 0.0000 | |
| | **Postal address as a fact kind** | Decision 15 keeps it out because there is nothing to validate. The two openings are an anchored one — an eircode certifies the lines above it, deterministically — and §8.2's extractive single-source class, which is a model and therefore a decision-15 amendment | |
| -| **A durable, sealed facts table** | Recogniser improvements need to reach old documents without a re-ingest, or the per-query extraction shows up in a profile. §8.8 rejects it today on derived-state and staleness grounds, not on leak grounds — the leak argument draft 1 gave attacked an unsealed design nobody proposed | |
| +| **A durable, sealed facts table** | The per-query extraction shows up in a profile. §8.8 rejects it today on derived-state grounds and because it would freeze the recognisers at ingest — **not** on leak grounds, which is what draft 1 argued and which attacked an unsealed design nobody proposed. Note the direction: storing facts makes recogniser improvements *harder* to ship, so this row gets triggered by cost and never by correctness | |
| | Cached answers | Query cost becomes visible. Requires §5.2's provenance invalidation first. **Triggering this expires §8.4's "stored nowhere"** | |
| | A conversational surface | Someone wants one, and it keeps §8.4's invariant: no passage crosses a request, in a prompt or a response. Decision 14 chose the shape on decision 5's grounds, not decision 3's, so this is a product argument to have — not a rule to break. §6.5 affords it: a four-turn transcript re-authorizes in one batch, one round | |
| | A runtime model-call ledger | Someone needs to audit routing. Sealed and provenance-stamped when it lands (§8.3) | |
| @@ -4098,8 +4149,26 @@ commit. Fixed 2026-09-05; §1.9's lesson arriving inside the fix for §1.8.* |
| **Five items need Paul and are marked `NEEDS PAUL`:** the 12-hour session |
| TTL, §8.7's unit-override switch, the CI runner's repo list, the |
| disclosure route for the rastrillo SSRF (`SECURITY-DRAFT-rastrillo-signin.md`, |
| -unfiled), and **decision 15's scope**. |
| - |
| +unfiled), and **decision 15's scope**. **Nothing else here is a ruling; |
| +the rest is work, measurement or somebody else's repo.** |
| + |
| + - **§8.8's cap is unmeasured against the corpus that motivated it, and |
| + the measurement is owed before decision 15 is settled.** The box shows |
| + nothing when a search's ruled top-25 carries four or more distinct |
| + validated values of one kind. Nobody has counted how many distinct |
| + phone numbers a real top-25 for "phone number" carries on Paul's live |
| + corpus, and if the answer is "more than three" the feature is inert on |
| + the very corpus it was asked for. **The remedy if so is a narrower |
| + lexicon or a per-kind cap — never a rule that picks**, which is the |
| + one thing §8.8 cannot trade away. Run it against the live data or |
| + through `tools/bench`. |
| + - **Decision 10 is narrowed by §8.8 and the narrowing is the spec |
| + author's, not Paul's.** "memoria never does arithmetic" now reads as |
| + "never reports a number it computed"; a mod-97 check digit is |
| + arithmetic on document content that decides a predicate and reports |
| + nothing. Defensible, and recorded here for the same reason decision |
| + 15's widening is: §1.9's lesson is that the load-bearing half of a |
| + decision must not live only in a commit message. |
| - **Decision 15 records more than Paul ruled, and says so rather than |
| quietly widening.** What he asked for on 2026-09-14 is the box, and |
| the constraint — *"do as much deterministic ordering of the data |