Address Fable's review: the plaintext gate, member ids, and the event log
REQUEST CHANGES, three P1s, all verified against the siblings before acting. The gate. Sheets has TestNoPlaintextValueColumns and memoria has TestPlaintextNeverReachesTheDatabase; fichas had neither, so `Name string` could be added to Card and CI would pass — in the app whose defining property is that it does not do that. There are now two: a reflection walk over Models against an explicit allowlist that IS the leak list, and a file gate that boots the real app and greps every byte it wrote, database and WAL, for the catalogue's sealed labels. The second has a companion test proving the grep can fail, because a canary nobody has seen fail is a canary that passes vacuously. Addresses. Card.Owner and Entry.Actor were address strings, indexed, beside an indexed updated_at — publishing per-person card counts and activity cadence, which is the shape memoria seals its breaker tally for. The Sheets precedent cited for it says the opposite: Event.ActorID and Cell.UpdatedBy are uint64 FKs to a Member row, and the address is plaintext exactly once, in one table, as a published leak. There is now a Member table storing the address memoria's way — AddressPrint for lookup, SealedAddress for display — and OwnerID/ActorID are int64. The event log could not keep what §5 promises. "Which fields changed" is kept permanently and had nowhere to live but inside the payload, which is the thing erasure empties; and an empty payload could not be told apart from one that never carried values, which is §5's named failure — a silently broken chain that still offers restore. Event gains ChangedFields, PayloadErasedAt and Revision. Also: prints now go through printFor, which refuses a field that is not TierPrint and canonicalises first, because an HMAC over raw bytes makes Ana@Rothar.ie and ana@rothar.ie different values and the dedupe quietly misses. Seal vectors pin the HKDF info string, which every other test in that package passes happily without. Seed is transactional and reconciling — a field added to the catalogue now reaches a team that already has the type, a changed tier refuses to boot, and one key claimed by two types is refused rather than silently ignored, which is a gap the atomicity test found. Nullable bools are NOT NULL (suite/s1.go refuses a feed row whose gone is absent), six foreign keys exist, enum options are sealed, and the "Sheets field for field" overclaim is corrected in both code and spec. The spec gains §4.1 case 4 for shipped structural vocabulary, rather than leaving four columns justified by a case the rule does not have.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
11 files changed,
+808
−108
docs/superpowers/specs/2026-09-19-fichas-design.md+13 −3internal/fichas/catalogue.go+122 −53internal/fichas/catalogue_test.go+80 −0internal/fichas/migrations/0001_init.sql+21 −13internal/fichas/migrations/schema.sql+21 −13internal/fichas/models.go+124 −26internal/fichas/models_test.go+95 −0internal/fichas/print.go+84 −0internal/fichas/print_test.go+76 −0internal/fichas/seal/vectors_test.go+72 −0internal/fichastest/gates_test.go+100 −0
CI passed — run details
CI log
=== vet === go vet ./... === fmt === === test === go test ./... ? amadan.net/oficina/fichas/cmd/fichas [no test files] ok amadan.net/oficina/fichas/internal/fichas 0.040s ? amadan.net/oficina/fichas/internal/fichas/icons [no test files] ok amadan.net/oficina/fichas/internal/fichas/seal 0.002s ok amadan.net/oficina/fichas/internal/fichastest 0.034s === migration-check === go run github.com/carlosframework/rastrillo/cmd/rastrillo migration check rastrillo migration check: models and migrations agree
diff --git a/docs/superpowers/specs/2026-09-19-fichas-design.md b/docs/superpowers/specs/2026-09-19-fichas-design.md| index ea1db4c..3afea8e 100644 |
| --- a/docs/superpowers/specs/2026-09-19-fichas-design.md |
| +++ b/docs/superpowers/specs/2026-09-19-fichas-design.md |
| @@ -1,8 +1,10 @@ |
| # 🤖 fichas — design |
| -**Status:** draft, revision 1, 2026-09-19. **Nothing here is built.** No |
| -repo, no module, no binary — this file and its directory are the whole of |
| -fichas today. |
| +**Status:** draft, revision 2, 2026-09-20. **Partly built.** The repo, the |
| +schema, the seal, the seeded catalogue, the config and the two plaintext |
| +gates exist; no write path, no surface, nothing deployed. Revision 1 said |
| +"nothing here is built", which went stale within a day — this line is the |
| +one to keep current. |
| A *ficha* is an index card; a *fichero* is the card cabinet. |
| @@ -132,6 +134,14 @@ without the key — three cases, and no others: |
| 2. **Structural integrity** — ids, foreign keys, kind routing. |
| 3. **The reach mark.** Under 2026-09-19's ruling the sibling evaluates its |
| own "offered" mark, with no asker and no unsealing. |
| +4. **Shipped structural vocabulary** — a field's `key`, `kind`, `tier` and |
| + `ordinal`, and an entry's `source`. **Added 2026-09-20**, because the |
| + code needed a fourth case and inventing one per column at review time is |
| + how a rule stops being stateable. These are the app's own words, never a |
| + team's. `tier` in particular has a reason the first three do not cover: |
| + the plaintext gate, and any reviewer, must be able to read the leak list |
| + off the file **without the key**. A sealed tier would mean auditing what |
| + is exposed requires holding the DEK, which inverts the point. |
| **Prints are for high-cardinality values only.** A keyed print over a |
| five-value `stage` enum yields five visible equality classes, and frequency |
diff --git a/internal/fichas/catalogue.go b/internal/fichas/catalogue.go| index c52c2c7..143e8c5 100644 |
| --- a/internal/fichas/catalogue.go |
| +++ b/internal/fichas/catalogue.go |
| @@ -1,6 +1,7 @@ |
| package fichas |
| import ( |
| + "encoding/json" |
| "fmt" |
| "amadan.net/oficina/fichas/internal/fichas/seal" |
| @@ -41,9 +42,14 @@ const ( |
| FieldOccurredAt = "f_occurred_at" |
| FieldBody = "f_body" |
| FieldDuration = "f_duration" |
| - FieldCallsRef = "f_calls_ref" |
| ) |
| +// There is deliberately no f_calls_ref field. The Calls meeting an entry |
| +// came from is a Ref row with EntryID set, not a field value: memoria's |
| +// seal keeps purposes separate by WHO RECEIVES the value, and an id that |
| +// goes back on the wire to Calls is its own kind. Sealed as an ordinary |
| +// field value it would share a purpose with a phone number. |
| + |
| // Field kinds — this app's vocabulary for a value's shape, plaintext |
| // because it is ours and not the team's content. |
| const ( |
| @@ -59,6 +65,10 @@ type seedField struct { |
| Label string |
| Kind string |
| Tier Tier |
| + // Options is a KindEnum's permitted values, sealed at seed time. |
| + // "Lead", "Won" are user-facing vocabulary and never reach the |
| + // durable file in the clear. |
| + Options []string |
| } |
| type seedType struct { |
| @@ -77,22 +87,23 @@ var catalogue = []seedType{ |
| // Printed: high-cardinality, and the equality question is |
| // worth answering without the key — "is this the same |
| // company", "do we already have this contact". |
| - {FieldName, "Name", KindText, TierPrint}, |
| - {FieldDomain, "Domain", KindText, TierPrint}, |
| - {FieldEmail, "Email", KindText, TierPrint}, |
| - {FieldPhone, "Phone", KindText, TierPrint}, |
| + {Key: FieldName, Label: "Name", Kind: KindText, Tier: TierPrint}, |
| + {Key: FieldDomain, Label: "Domain", Kind: KindText, Tier: TierPrint}, |
| + {Key: FieldEmail, Label: "Email", Kind: KindText, Tier: TierPrint}, |
| + {Key: FieldPhone, Label: "Phone", Kind: KindText, Tier: TierPrint}, |
| // Sealed. Stage is low-cardinality, so a print would publish |
| // its equality classes and frequency analysis would order |
| // them — and a pipeline's composition is commercially |
| // sensitive. Grouped in the in-memory index instead. |
| - {FieldStage, "Stage", KindEnum, TierSealed}, |
| + {Key: FieldStage, Label: "Stage", Kind: KindEnum, Tier: TierSealed, |
| + Options: []string{"Lead", "Contacted", "Proposal", "Won", "Lost"}}, |
| // Sealed although it is the field most wanted for ranges |
| // ("no call in six weeks"). §4.1: the index is in memory, so |
| // sealed is not unqueryable, and plaintext here would buy |
| // nothing the index does not already give. |
| - {FieldNextAction, "Next action", KindDate, TierSealed}, |
| - {FieldNotes, "Notes", KindText, TierSealed}, |
| + {Key: FieldNextAction, Label: "Next action", Kind: KindDate, Tier: TierSealed}, |
| + {Key: FieldNotes, Label: "Notes", Kind: KindText, Tier: TierSealed}, |
| }, |
| }, |
| { |
| @@ -102,65 +113,123 @@ var catalogue = []seedType{ |
| // Deliberately distinct from Entry.CreatedAt, which is |
| // plaintext because the feed needs to know a row was written. |
| // WHEN YOU CALLED THEM is content, and is sealed. |
| - {FieldOccurredAt, "When", KindDate, TierSealed}, |
| - {FieldBody, "What was said", KindText, TierSealed}, |
| - {FieldDuration, "Duration", KindNumber, TierSealed}, |
| - // Printed for dedupe on re-import: capture from Calls must be |
| - // able to ask "have I already logged this meeting" without |
| - // unsealing every entry on the card. |
| - {FieldCallsRef, "Calls meeting", KindRef, TierPrint}, |
| + {Key: FieldOccurredAt, Label: "When", Kind: KindDate, Tier: TierSealed}, |
| + {Key: FieldBody, Label: "What was said", Kind: KindText, Tier: TierSealed}, |
| + {Key: FieldDuration, Label: "Duration", Kind: KindNumber, Tier: TierSealed}, |
| }, |
| }, |
| } |
| -// Seed writes the catalogue if it is not already there, and is safe to run |
| -// on every boot. |
| +// Seed writes the catalogue, and reconciles it, on every boot. |
| // |
| // It matches on Key and never on Label, because a Label is sealed and |
| -// sealing is not deterministic — two seals of "Customer" differ, so a |
| -// label comparison would insert a duplicate type on every start. |
| +// sealing is not deterministic — two seals of "Customer" differ, so a label |
| +// comparison would insert a duplicate type on every start. |
| +// |
| +// **A missing field is added; a changed Kind or Tier refuses to boot.** The |
| +// first draft skipped an existing type entirely, which meant a later |
| +// release adding a field would never reach a team that already had the |
| +// type — while the design says adding a field is a row and not a migration. |
| +// The reasoning it borrowed was about changing a TIER, which really is a |
| +// re-encryption of every value under it, and applying that to an ADDITION |
| +// was a bug the comment had talked itself into. Refusing loudly on a tier |
| +// change is the other half: seeding past it would leave the definition |
| +// lying about the values already sealed under it. |
| // |
| -// It does not update an existing row. Changing a shipped field's Tier is a |
| -// re-encryption of every value under it, not a seed-time edit, and doing it |
| -// silently here would leave old values sealed under a tier the definition |
| -// no longer claims. |
| +// All of it in one transaction, because a crash between the type row and |
| +// its last field used to leave a partial type that the skip-if-present |
| +// path would then ignore forever. |
| func Seed(db *gorm.DB, s *seal.Sealer) error { |
| - for _, t := range catalogue { |
| - var ct CardType |
| - err := db.Where("key = ?", t.Key).First(&ct).Error |
| - switch { |
| - case err == nil: |
| - continue // already seeded; see the doc comment |
| - case err != gorm.ErrRecordNotFound: |
| - return fmt.Errorf("seed: looking up %s: %w", t.Key, err) |
| + return db.Transaction(func(tx *gorm.DB) error { |
| + for _, t := range catalogue { |
| + ct, err := seedType_(tx, s, t) |
| + if err != nil { |
| + return err |
| + } |
| + for i, f := range t.Fields { |
| + if err := seedField_(tx, s, ct.ID, i, f); err != nil { |
| + return err |
| + } |
| + } |
| } |
| + return nil |
| + }) |
| +} |
| - label, err := s.Seal(seal.PurposeLabel, []byte(t.Label)) |
| - if err != nil { |
| - return fmt.Errorf("seed: sealing the label for %s: %w", t.Key, err) |
| +func seedType_(tx *gorm.DB, s *seal.Sealer, t seedType) (CardType, error) { |
| + var ct CardType |
| + err := tx.Where("key = ?", t.Key).First(&ct).Error |
| + if err == nil { |
| + return ct, nil |
| + } |
| + if err != gorm.ErrRecordNotFound { |
| + return ct, fmt.Errorf("seed: looking up %s: %w", t.Key, err) |
| + } |
| + label, err := s.Seal(seal.PurposeLabel, []byte(t.Label)) |
| + if err != nil { |
| + return ct, fmt.Errorf("seed: sealing the label for %s: %w", t.Key, err) |
| + } |
| + ct = CardType{Key: t.Key, SealedLabel: label} |
| + if err := tx.Create(&ct).Error; err != nil { |
| + return ct, fmt.Errorf("seed: creating %s: %w", t.Key, err) |
| + } |
| + return ct, nil |
| +} |
| + |
| +func seedField_(tx *gorm.DB, s *seal.Sealer, cardTypeID int64, ordinal int, f seedField) error { |
| + var fd FieldDef |
| + err := tx.Where("key = ?", f.Key).First(&fd).Error |
| + switch { |
| + case err == nil: |
| + // A key is unique across the whole table, so finding one under a |
| + // DIFFERENT type means two definitions are claiming one key — |
| + // and a print is derived from the key, so they would share an |
| + // equality space. Refuse rather than silently ignoring the |
| + // second, which is what returning early here used to do. |
| + if fd.CardTypeID != cardTypeID { |
| + return fmt.Errorf( |
| + "seed: %s is already defined under another card type; a field key is unique across the "+ |
| + "catalogue because the print purpose derives from it", f.Key) |
| } |
| - ct = CardType{Key: t.Key, SealedLabel: label} |
| - if err := db.Create(&ct).Error; err != nil { |
| - return fmt.Errorf("seed: creating %s: %w", t.Key, err) |
| + // Label and ordinal may drift freely — they are presentation. |
| + // Kind and Tier may not: both describe how the values already in |
| + // the database were written. |
| + if fd.Kind != f.Kind || fd.Tier != f.Tier { |
| + return fmt.Errorf( |
| + "seed: %s is stored as kind=%s tier=%s but the catalogue now says kind=%s tier=%s; "+ |
| + "changing either is a re-encryption of every value under it, not a seed-time edit", |
| + f.Key, fd.Kind, fd.Tier, f.Kind, f.Tier) |
| } |
| + return nil |
| + case err != gorm.ErrRecordNotFound: |
| + return fmt.Errorf("seed: looking up %s: %w", f.Key, err) |
| + } |
| - for i, f := range t.Fields { |
| - fl, err := s.Seal(seal.PurposeLabel, []byte(f.Label)) |
| - if err != nil { |
| - return fmt.Errorf("seed: sealing the label for %s: %w", f.Key, err) |
| - } |
| - fd := FieldDef{ |
| - CardTypeID: ct.ID, |
| - Key: f.Key, |
| - SealedLabel: fl, |
| - Kind: f.Kind, |
| - Tier: f.Tier, |
| - Ordinal: i, |
| - } |
| - if err := db.Create(&fd).Error; err != nil { |
| - return fmt.Errorf("seed: creating %s: %w", f.Key, err) |
| - } |
| + label, err := s.Seal(seal.PurposeLabel, []byte(f.Label)) |
| + if err != nil { |
| + return fmt.Errorf("seed: sealing the label for %s: %w", f.Key, err) |
| + } |
| + var options []byte |
| + if len(f.Options) > 0 { |
| + raw, err := json.Marshal(f.Options) |
| + if err != nil { |
| + return fmt.Errorf("seed: encoding the options for %s: %w", f.Key, err) |
| + } |
| + if options, err = s.Seal(seal.PurposeLabel, raw); err != nil { |
| + return fmt.Errorf("seed: sealing the options for %s: %w", f.Key, err) |
| } |
| } |
| + fd = FieldDef{ |
| + CardTypeID: cardTypeID, |
| + Key: f.Key, |
| + SealedLabel: label, |
| + SealedOptions: options, |
| + Kind: f.Kind, |
| + Tier: f.Tier, |
| + Ordinal: ordinal, |
| + } |
| + if err := tx.Create(&fd).Error; err != nil { |
| + return fmt.Errorf("seed: creating %s: %w", f.Key, err) |
| + } |
| return nil |
| } |
diff --git a/internal/fichas/catalogue_test.go b/internal/fichas/catalogue_test.go| index 5d3d297..a918779 100644 |
| --- a/internal/fichas/catalogue_test.go |
| +++ b/internal/fichas/catalogue_test.go |
| @@ -136,3 +136,83 @@ func TestFieldKeysAreUnique(t *testing.T) { |
| } |
| } |
| } |
| + |
| +// Seed reconciles rather than skipping: a later release that adds a field |
| +// must reach a team that already has the type. The first draft returned |
| +// early on an existing type and never looked at its fields. |
| +func TestSeedAddsAFieldToAnExistingType(t *testing.T) { |
| + g, s := testDB(t), testSealer(t) |
| + if err := Seed(g, s); err != nil { |
| + t.Fatalf("first Seed: %v", err) |
| + } |
| + |
| + added := seedField{Key: "f_website", Label: "Website", Kind: KindText, Tier: TierPrint} |
| + restore := catalogue |
| + catalogue = append([]seedType(nil), catalogue...) |
| + catalogue[0].Fields = append(append([]seedField(nil), catalogue[0].Fields...), added) |
| + t.Cleanup(func() { catalogue = restore }) |
| + |
| + if err := Seed(g, s); err != nil { |
| + t.Fatalf("second Seed: %v", err) |
| + } |
| + var fd FieldDef |
| + if err := g.Where("key = ?", added.Key).First(&fd).Error; err != nil { |
| + t.Fatalf("a field added to the catalogue never reached an existing database: %v", err) |
| + } |
| + if fd.Tier != added.Tier || fd.Kind != added.Kind { |
| + t.Fatalf("added field stored as kind=%s tier=%s", fd.Kind, fd.Tier) |
| + } |
| +} |
| + |
| +// Changing a shipped field's tier is a re-encryption of every value under |
| +// it. Seeding past it would leave the definition lying about what is in |
| +// the database, so it must refuse to boot. |
| +func TestSeedRefusesAChangedTier(t *testing.T) { |
| + g, s := testDB(t), testSealer(t) |
| + if err := Seed(g, s); err != nil { |
| + t.Fatalf("first Seed: %v", err) |
| + } |
| + |
| + restore := catalogue |
| + catalogue = append([]seedType(nil), catalogue...) |
| + fields := append([]seedField(nil), catalogue[0].Fields...) |
| + for i := range fields { |
| + if fields[i].Key == FieldNotes { |
| + fields[i].Tier = TierPrint // was TierSealed |
| + } |
| + } |
| + catalogue[0].Fields = fields |
| + t.Cleanup(func() { catalogue = restore }) |
| + |
| + if err := Seed(g, s); err == nil { |
| + t.Fatal("Seed accepted a changed tier on a shipped field") |
| + } |
| +} |
| + |
| +// A partial type used to be possible: a crash between the type row and its |
| +// last field left a type the skip-if-present path then ignored forever. |
| +// One transaction is what prevents it. |
| +func TestSeedIsAtomic(t *testing.T) { |
| + g, s := testDB(t), testSealer(t) |
| + restore := catalogue |
| + broken := append([]seedType(nil), catalogue...) |
| + // The SECOND type claims a field key the first already defined. That |
| + // fails after the first type's rows are written, so a rollback is the |
| + // only thing that can leave the database clean. |
| + broken[1] = seedType{ |
| + Key: broken[1].Key, |
| + Label: broken[1].Label, |
| + Fields: append(append([]seedField(nil), broken[1].Fields...), broken[0].Fields[0]), |
| + } |
| + catalogue = broken |
| + t.Cleanup(func() { catalogue = restore }) |
| + |
| + if err := Seed(g, s); err == nil { |
| + t.Fatal("Seed accepted one field key claimed by two card types") |
| + } |
| + var types int64 |
| + g.Model(&CardType{}).Count(&types) |
| + if types != 0 { |
| + t.Fatalf("a failed seed left %d type(s) behind; the transaction did not roll back", types) |
| + } |
| +} |
diff --git a/internal/fichas/migrations/0001_init.sql b/internal/fichas/migrations/0001_init.sql| index afa85d6..1b717f7 100644 |
| --- a/internal/fichas/migrations/0001_init.sql |
| +++ b/internal/fichas/migrations/0001_init.sql |
| @@ -1,18 +1,22 @@ |
| +CREATE TABLE `members` (`id` integer PRIMARY KEY AUTOINCREMENT,`address_print` text,`sealed_address` blob,`created_at` datetime,`updated_at` datetime); |
| + |
| +CREATE UNIQUE INDEX `idx_members_address_print` ON `members`(`address_print`); |
| + |
| CREATE TABLE `card_types` (`id` integer PRIMARY KEY AUTOINCREMENT,`key` text,`sealed_label` blob,`created_at` datetime,`updated_at` datetime); |
| CREATE UNIQUE INDEX `idx_card_types_key` ON `card_types`(`key`); |
| -CREATE TABLE `field_defs` (`id` integer PRIMARY KEY AUTOINCREMENT,`card_type_id` integer,`key` text,`sealed_label` blob,`kind` text,`tier` text,`ordinal` integer,`created_at` datetime,`updated_at` datetime); |
| +CREATE TABLE `field_defs` (`id` integer PRIMARY KEY AUTOINCREMENT,`card_type_id` integer NOT NULL,`key` text,`sealed_label` blob,`sealed_options` blob,`kind` text,`tier` text,`ordinal` integer,`created_at` datetime,`updated_at` datetime,CONSTRAINT `fk_card_types_fields` FOREIGN KEY (`card_type_id`) REFERENCES `card_types`(`id`) ON DELETE RESTRICT); |
| -CREATE INDEX `idx_field_defs_key` ON `field_defs`(`key`); |
| +CREATE UNIQUE INDEX `idx_field_defs_key` ON `field_defs`(`key`); |
| CREATE INDEX `idx_field_defs_card_type_id` ON `field_defs`(`card_type_id`); |
| -CREATE TABLE `cards` (`id` integer PRIMARY KEY AUTOINCREMENT,`card_type_id` integer,`revision` integer NOT NULL DEFAULT 1,`gone` numeric,`offered` numeric,`owner` text,`created_at` datetime,`updated_at` datetime); |
| +CREATE TABLE `cards` (`id` integer PRIMARY KEY AUTOINCREMENT,`card_type_id` integer NOT NULL,`revision` integer NOT NULL DEFAULT 1,`gone` numeric NOT NULL DEFAULT false,`offered` numeric NOT NULL DEFAULT false,`owner_id` integer,`created_at` datetime,`updated_at` datetime); |
| CREATE INDEX `idx_cards_updated_at` ON `cards`(`updated_at`); |
| -CREATE INDEX `idx_cards_owner` ON `cards`(`owner`); |
| +CREATE INDEX `idx_cards_owner_id` ON `cards`(`owner_id`); |
| CREATE INDEX `idx_cards_offered` ON `cards`(`offered`); |
| @@ -20,37 +24,41 @@ CREATE INDEX `idx_cards_gone` ON `cards`(`gone`); |
| CREATE INDEX `idx_cards_card_type_id` ON `cards`(`card_type_id`); |
| -CREATE TABLE `card_values` (`card_id` integer,`field_def_id` integer,`sealed_value` blob,`print` text,PRIMARY KEY (`card_id`,`field_def_id`)); |
| +CREATE TABLE `card_values` (`card_id` integer,`field_def_id` integer,`sealed_value` blob,`print` text,PRIMARY KEY (`card_id`,`field_def_id`),CONSTRAINT `fk_cards_values` FOREIGN KEY (`card_id`) REFERENCES `cards`(`id`) ON DELETE CASCADE); |
| CREATE INDEX `idx_card_values_print` ON `card_values`(`print`); |
| -CREATE TABLE `entries` (`id` integer PRIMARY KEY AUTOINCREMENT,`card_id` integer,`kind` text,`source` text,`actor` text,`created_at` datetime); |
| +CREATE TABLE `entries` (`id` integer PRIMARY KEY AUTOINCREMENT,`card_id` integer NOT NULL,`kind` text NOT NULL,`source` text,`actor_id` integer,`created_at` datetime,CONSTRAINT `fk_cards_entries` FOREIGN KEY (`card_id`) REFERENCES `cards`(`id`) ON DELETE CASCADE); |
| CREATE INDEX `idx_entries_created_at` ON `entries`(`created_at`); |
| -CREATE INDEX `idx_entries_actor` ON `entries`(`actor`); |
| +CREATE INDEX `idx_entries_actor_id` ON `entries`(`actor_id`); |
| CREATE INDEX `idx_entries_kind` ON `entries`(`kind`); |
| CREATE INDEX `idx_entries_card_id` ON `entries`(`card_id`); |
| -CREATE TABLE `entry_values` (`entry_id` integer,`field_def_id` integer,`sealed_value` blob,`print` text,PRIMARY KEY (`entry_id`,`field_def_id`)); |
| +CREATE TABLE `entry_values` (`entry_id` integer,`field_def_id` integer,`sealed_value` blob,`print` text,PRIMARY KEY (`entry_id`,`field_def_id`),CONSTRAINT `fk_entries_values` FOREIGN KEY (`entry_id`) REFERENCES `entries`(`id`) ON DELETE CASCADE); |
| CREATE INDEX `idx_entry_values_print` ON `entry_values`(`print`); |
| -CREATE TABLE `refs` (`id` integer PRIMARY KEY AUTOINCREMENT,`card_id` integer,`sealed_target` blob,`print` text,`created_at` datetime); |
| +CREATE TABLE `refs` (`id` integer PRIMARY KEY AUTOINCREMENT,`card_id` integer NOT NULL,`entry_id` integer,`sealed_target` blob,`print` text,`created_at` datetime,CONSTRAINT `fk_entries_refs` FOREIGN KEY (`entry_id`) REFERENCES `entries`(`id`) ON DELETE CASCADE,CONSTRAINT `fk_cards_refs` FOREIGN KEY (`card_id`) REFERENCES `cards`(`id`) ON DELETE CASCADE); |
| -CREATE INDEX `idx_refs_print` ON `refs`(`print`); |
| +CREATE UNIQUE INDEX `idx_refs_card_print` ON `refs`(`print`); |
| + |
| +CREATE INDEX `idx_refs_entry_id` ON `refs`(`entry_id`); |
| CREATE INDEX `idx_refs_card_id` ON `refs`(`card_id`); |
| -CREATE TABLE `events` (`id` integer PRIMARY KEY AUTOINCREMENT,`kind` text,`object_kind` text,`object_id` integer,`actor_id` text,`payload_sealed` blob,`trigger_id` integer,`created_at` datetime); |
| +CREATE TABLE `events` (`id` integer PRIMARY KEY AUTOINCREMENT,`kind` text NOT NULL,`object_kind` text NOT NULL,`object_id` integer,`actor_id` integer,`revision` integer,`changed_fields` text,`payload_sealed` blob,`payload_erased_at` datetime,`trigger_id` integer,`created_at` datetime); |
| CREATE INDEX `idx_events_created_at` ON `events`(`created_at`); |
| -CREATE INDEX `idx_events_object_id` ON `events`(`object_id`); |
| +CREATE INDEX `idx_events_trigger_id` ON `events`(`trigger_id`); |
| + |
| +CREATE INDEX `idx_events_actor_id` ON `events`(`actor_id`); |
| -CREATE INDEX `idx_events_object_kind` ON `events`(`object_kind`); |
| +CREATE INDEX `idx_events_object` ON `events`(`object_kind`,`object_id`); |
| CREATE INDEX `idx_events_kind` ON `events`(`kind`); |
diff --git a/internal/fichas/migrations/schema.sql b/internal/fichas/migrations/schema.sql| index 6e7efe5..a6efd7b 100644 |
| --- a/internal/fichas/migrations/schema.sql |
| +++ b/internal/fichas/migrations/schema.sql |
| @@ -3,19 +3,21 @@ |
| CREATE TABLE `card_types` (`id` integer PRIMARY KEY AUTOINCREMENT,`key` text,`sealed_label` blob,`created_at` datetime,`updated_at` datetime); |
| -CREATE TABLE `card_values` (`card_id` integer,`field_def_id` integer,`sealed_value` blob,`print` text,PRIMARY KEY (`card_id`,`field_def_id`)); |
| +CREATE TABLE `card_values` (`card_id` integer,`field_def_id` integer,`sealed_value` blob,`print` text,PRIMARY KEY (`card_id`,`field_def_id`),CONSTRAINT `fk_cards_values` FOREIGN KEY (`card_id`) REFERENCES `cards`(`id`) ON DELETE CASCADE); |
| -CREATE TABLE `cards` (`id` integer PRIMARY KEY AUTOINCREMENT,`card_type_id` integer,`revision` integer NOT NULL DEFAULT 1,`gone` numeric,`offered` numeric,`owner` text,`created_at` datetime,`updated_at` datetime); |
| +CREATE TABLE `cards` (`id` integer PRIMARY KEY AUTOINCREMENT,`card_type_id` integer NOT NULL,`revision` integer NOT NULL DEFAULT 1,`gone` numeric NOT NULL DEFAULT false,`offered` numeric NOT NULL DEFAULT false,`owner_id` integer,`created_at` datetime,`updated_at` datetime); |
| -CREATE TABLE `entries` (`id` integer PRIMARY KEY AUTOINCREMENT,`card_id` integer,`kind` text,`source` text,`actor` text,`created_at` datetime); |
| +CREATE TABLE `entries` (`id` integer PRIMARY KEY AUTOINCREMENT,`card_id` integer NOT NULL,`kind` text NOT NULL,`source` text,`actor_id` integer,`created_at` datetime,CONSTRAINT `fk_cards_entries` FOREIGN KEY (`card_id`) REFERENCES `cards`(`id`) ON DELETE CASCADE); |
| -CREATE TABLE `entry_values` (`entry_id` integer,`field_def_id` integer,`sealed_value` blob,`print` text,PRIMARY KEY (`entry_id`,`field_def_id`)); |
| +CREATE TABLE `entry_values` (`entry_id` integer,`field_def_id` integer,`sealed_value` blob,`print` text,PRIMARY KEY (`entry_id`,`field_def_id`),CONSTRAINT `fk_entries_values` FOREIGN KEY (`entry_id`) REFERENCES `entries`(`id`) ON DELETE CASCADE); |
| -CREATE TABLE `events` (`id` integer PRIMARY KEY AUTOINCREMENT,`kind` text,`object_kind` text,`object_id` integer,`actor_id` text,`payload_sealed` blob,`trigger_id` integer,`created_at` datetime); |
| +CREATE TABLE `events` (`id` integer PRIMARY KEY AUTOINCREMENT,`kind` text NOT NULL,`object_kind` text NOT NULL,`object_id` integer,`actor_id` integer,`revision` integer,`changed_fields` text,`payload_sealed` blob,`payload_erased_at` datetime,`trigger_id` integer,`created_at` datetime); |
| -CREATE TABLE `field_defs` (`id` integer PRIMARY KEY AUTOINCREMENT,`card_type_id` integer,`key` text,`sealed_label` blob,`kind` text,`tier` text,`ordinal` integer,`created_at` datetime,`updated_at` datetime); |
| +CREATE TABLE `field_defs` (`id` integer PRIMARY KEY AUTOINCREMENT,`card_type_id` integer NOT NULL,`key` text,`sealed_label` blob,`sealed_options` blob,`kind` text,`tier` text,`ordinal` integer,`created_at` datetime,`updated_at` datetime,CONSTRAINT `fk_card_types_fields` FOREIGN KEY (`card_type_id`) REFERENCES `card_types`(`id`) ON DELETE RESTRICT); |
| -CREATE TABLE `refs` (`id` integer PRIMARY KEY AUTOINCREMENT,`card_id` integer,`sealed_target` blob,`print` text,`created_at` datetime); |
| +CREATE TABLE `members` (`id` integer PRIMARY KEY AUTOINCREMENT,`address_print` text,`sealed_address` blob,`created_at` datetime,`updated_at` datetime); |
| + |
| +CREATE TABLE `refs` (`id` integer PRIMARY KEY AUTOINCREMENT,`card_id` integer NOT NULL,`entry_id` integer,`sealed_target` blob,`print` text,`created_at` datetime,CONSTRAINT `fk_entries_refs` FOREIGN KEY (`entry_id`) REFERENCES `entries`(`id`) ON DELETE CASCADE,CONSTRAINT `fk_cards_refs` FOREIGN KEY (`card_id`) REFERENCES `cards`(`id`) ON DELETE CASCADE); |
| CREATE UNIQUE INDEX `idx_card_types_key` ON `card_types`(`key`); |
| @@ -27,11 +29,11 @@ CREATE INDEX `idx_cards_gone` ON `cards`(`gone`); |
| CREATE INDEX `idx_cards_offered` ON `cards`(`offered`); |
| -CREATE INDEX `idx_cards_owner` ON `cards`(`owner`); |
| +CREATE INDEX `idx_cards_owner_id` ON `cards`(`owner_id`); |
| CREATE INDEX `idx_cards_updated_at` ON `cards`(`updated_at`); |
| -CREATE INDEX `idx_entries_actor` ON `entries`(`actor`); |
| +CREATE INDEX `idx_entries_actor_id` ON `entries`(`actor_id`); |
| CREATE INDEX `idx_entries_card_id` ON `entries`(`card_id`); |
| @@ -41,19 +43,25 @@ CREATE INDEX `idx_entries_kind` ON `entries`(`kind`); |
| CREATE INDEX `idx_entry_values_print` ON `entry_values`(`print`); |
| +CREATE INDEX `idx_events_actor_id` ON `events`(`actor_id`); |
| + |
| CREATE INDEX `idx_events_created_at` ON `events`(`created_at`); |
| CREATE INDEX `idx_events_kind` ON `events`(`kind`); |
| -CREATE INDEX `idx_events_object_id` ON `events`(`object_id`); |
| +CREATE INDEX `idx_events_object` ON `events`(`object_kind`,`object_id`); |
| -CREATE INDEX `idx_events_object_kind` ON `events`(`object_kind`); |
| +CREATE INDEX `idx_events_trigger_id` ON `events`(`trigger_id`); |
| CREATE INDEX `idx_field_defs_card_type_id` ON `field_defs`(`card_type_id`); |
| -CREATE INDEX `idx_field_defs_key` ON `field_defs`(`key`); |
| +CREATE UNIQUE INDEX `idx_field_defs_key` ON `field_defs`(`key`); |
| + |
| +CREATE UNIQUE INDEX `idx_members_address_print` ON `members`(`address_print`); |
| CREATE INDEX `idx_refs_card_id` ON `refs`(`card_id`); |
| -CREATE INDEX `idx_refs_print` ON `refs`(`print`); |
| +CREATE UNIQUE INDEX `idx_refs_card_print` ON `refs`(`print`); |
| + |
| +CREATE INDEX `idx_refs_entry_id` ON `refs`(`entry_id`); |
diff --git a/internal/fichas/models.go b/internal/fichas/models.go| index 05de872..c7be49a 100644 |
| --- a/internal/fichas/models.go |
| +++ b/internal/fichas/models.go |
| @@ -7,6 +7,7 @@ import "time" |
| // out what the database should look like, and `rastrillo migration check` |
| // fails CI when it and the migrations disagree. |
| var Models = []any{ |
| + &Member{}, |
| &CardType{}, |
| &FieldDef{}, |
| &Card{}, |
| @@ -35,6 +36,36 @@ var Models = []any{ |
| // the suite contract's feed row, referential integrity, and the reach mark |
| // a sibling evaluates for itself. Everything else is sealed. |
| +// Member is a person this instance has seen. It exists so that "who owns |
| +// this card" and "who wrote this entry" are FOREIGN KEYS rather than |
| +// addresses repeated in the clear on every row. |
| +// |
| +// The first draft put the address string directly on Card.Owner and |
| +// Entry.Actor and cited Sheets as precedent. **That citation was wrong.** |
| +// Sheets' Event.ActorID and Cell.UpdatedBy are uint64 FKs to a Member row; |
| +// the address is plaintext exactly ONCE, in one table, and Sheets §2.2 |
| +// publishes that as a stated leak. Repeating it per card, indexed, beside |
| +// an indexed updated_at, would have published per-person card counts and |
| +// activity cadence — the shape memoria seals its breaker tally for. |
| +// |
| +// So the address is stored memoria's way rather than Sheets': an |
| +// AddressPrint for lookup and a SealedAddress for display. A print is |
| +// admissible here although a roster is low-cardinality, because the print |
| +// is never compared across a field boundary — it answers exactly one |
| +// question, "which member is this", for a caller who already holds the |
| +// address. |
| +type Member struct { |
| + ID int64 |
| + // AddressPrint is the keyed fingerprint used to find a member by |
| + // address without holding one. Unique: two rows for one person would |
| + // split their cards in half. |
| + AddressPrint string `gorm:"uniqueIndex;size:64"` |
| + // SealedAddress is the address itself, for display. |
| + SealedAddress []byte |
| + CreatedAt time.Time |
| + UpdatedAt time.Time |
| +} |
| + |
| // Tier is where a field's values live: the vocabulary of §4.2. |
| // |
| // TierPrint is a keyed one-way fingerprint and answers equality only — never |
| @@ -68,15 +99,27 @@ type CardType struct { |
| SealedLabel []byte |
| CreatedAt time.Time |
| UpdatedAt time.Time |
| + // RESTRICT, not CASCADE: deleting a type out from under live cards |
| + // must fail rather than take the cards with it. |
| + Fields []FieldDef `gorm:"constraint:OnDelete:RESTRICT"` |
| } |
| // FieldDef is one typed field on a card type. Adding a field is a row, not |
| // a migration. |
| type FieldDef struct { |
| - ID int64 |
| - CardTypeID int64 `gorm:"index"` |
| - Key string `gorm:"index;size:64"` // opaque, as CardType.Key |
| + ID int64 |
| + CardTypeID int64 `gorm:"index;not null"` |
| + // Key is unique across the whole table, not merely within a type. A |
| + // print is derived from the key (seal.Print), so two definitions |
| + // sharing one would share an equality space — and the schema, not a |
| + // test over a Go slice, is where that has to be refused. |
| + Key string `gorm:"uniqueIndex;size:64"` // opaque, as CardType.Key |
| SealedLabel []byte |
| + // SealedOptions is a KindEnum's permitted values, as a sealed JSON |
| + // array. Sealed because "Lead", "Contract sent", "Won" are precisely |
| + // the user vocabulary §4.3 exists to keep out of the durable file. |
| + // Empty for every other kind. |
| + SealedOptions []byte |
| // Kind is the value's shape — text, date, enum, number — and drives |
| // validation and how the in-memory index treats it. Plaintext: it is |
| // this app's vocabulary, not the team's content. |
| @@ -92,23 +135,33 @@ type FieldDef struct { |
| // CardValue rows; nothing about what it *says* is in this struct. |
| type Card struct { |
| ID int64 |
| - CardTypeID int64 `gorm:"index"` |
| + CardTypeID int64 `gorm:"index;not null"` |
| // Revision, Gone and UpdatedAt are the suite contract's feed row |
| // (S1: item_id, revision, gone, touched_at) and are plaintext so the |
| // feed can be served without unsealing and passes the plaintext-grep |
| // gate unaided. memoria §3.2 already concedes update timestamps as a |
| // known leak; this is the same trade, made deliberately. |
| Revision int64 `gorm:"not null;default:1"` |
| - Gone bool `gorm:"index"` |
| + // not null with a default, because a nullable bool does not behave: |
| + // with a NULL row, `gone = 0`, `gone != 1` and `NOT gone` all miss |
| + // it — and suite/s1.go refuses a feed row whose gone is absent. |
| + Gone bool `gorm:"index;not null;default:false"` |
| // Offered is the reach mark. Under the 2026-09-19 credential ruling |
| // the sibling evaluates its own mark, with no asker and no unsealing, |
| // so this must be answerable by SQL against the durable file. |
| - Offered bool `gorm:"index"` |
| - // Owner is a roster reference, not content. Sheets keeps actor_id |
| - // plaintext and this follows it. |
| - Owner string `gorm:"index;size:128"` |
| + Offered bool `gorm:"index;not null;default:false"` |
| + // OwnerID is a Member row, never an address. See Member. |
| + OwnerID int64 `gorm:"index"` |
| CreatedAt time.Time |
| UpdatedAt time.Time `gorm:"index"` |
| + |
| + // Deleting a card takes its values, entries and references with it. |
| + // Its EVENTS are deliberately absent from this list: the log must |
| + // outlive the object, or "who deleted this" has no row to live on |
| + // (§5). |
| + Values []CardValue `gorm:"constraint:OnDelete:CASCADE"` |
| + Entries []Entry `gorm:"constraint:OnDelete:CASCADE"` |
| + Refs []Ref `gorm:"constraint:OnDelete:CASCADE"` |
| } |
| // CardValue is one field's value on one card. Sealed, with an optional |
| @@ -128,19 +181,23 @@ type CardValue struct { |
| // never an edit, because an entry is a fact about a past moment. |
| type Entry struct { |
| ID int64 |
| - CardID int64 `gorm:"index"` |
| - Kind string `gorm:"index;size:32"` |
| + CardID int64 `gorm:"index;not null"` |
| + Kind string `gorm:"index;size:32;not null"` |
| // Source is where it came from — "manual", "calls". Routing, not |
| // content. |
| Source string `gorm:"size:32"` |
| - // Actor names the real writer, per suite ruling 9. A generated entry |
| - // is written by the app as itself and says so; it is never attributed |
| - // to whoever triggered it. |
| - Actor string `gorm:"index;size:128"` |
| + // ActorID names the real writer, per suite ruling 9 — a Member row, |
| + // never an address. Zero means the app wrote it as itself, which a |
| + // generated entry does; it is never attributed to whoever triggered |
| + // it. |
| + ActorID int64 `gorm:"index"` |
| // CreatedAt is when the row was written — the feed needs that. |
| // When the call actually happened is an EntryValue, sealed, because |
| // "when you called them" is content. |
| CreatedAt time.Time `gorm:"index"` |
| + |
| + Values []EntryValue `gorm:"constraint:OnDelete:CASCADE"` |
| + Refs []Ref `gorm:"constraint:OnDelete:CASCADE"` |
| } |
| // EntryValue is CardValue's twin for entries. |
| @@ -159,16 +216,35 @@ type EntryValue struct { |
| // app, that a private item exists. That is the leak S1's visibility flag |
| // was removed to prevent. |
| type Ref struct { |
| - ID int64 |
| - CardID int64 `gorm:"index"` |
| + ID int64 |
| + CardID int64 `gorm:"index;not null"` |
| + // EntryID scopes a reference to one entry rather than to the card — |
| + // the Calls meeting an entry came from. Nil for a card-level |
| + // reference. |
| + // |
| + // This exists because the alternative was storing a sibling's item id |
| + // as an ordinary field value, which would seal it under PurposeValue. |
| + // memoria's seal keeps purposes separate "by who receives them": an |
| + // id that goes back on the wire to another app is its own kind, and |
| + // sealing it as a generic value loses that rule. |
| + EntryID *int64 `gorm:"index"` |
| SealedTarget []byte |
| - Print string `gorm:"index;size:64"` // dedupe only |
| - CreatedAt time.Time |
| + // Print dedupes on re-import. Unique per card so that |
| + // check-then-insert cannot race a duplicate in. |
| + Print string `gorm:"uniqueIndex:idx_refs_card_print;size:64"` |
| + CreatedAt time.Time |
| } |
| // Event is the change log: what changed, who did it (§5, Paul 2026-09-19). |
| -// Deliberately Sheets' events table, field for field, rather than a second |
| -// design that drifts from it. |
| +// |
| +// Sheets' events table is the model, and an earlier comment here claimed |
| +// it was copied "field for field", which was false. What is copied is the |
| +// shape that matters: an append-only id that doubles as a cursor, a kind, |
| +// a payload sealed under "event.payload", and trigger_id. What differs: |
| +// tab_id becomes (object_kind, object_id) because fichas logs more than |
| +// one kind of object; derived_count is dropped as a spreadsheet's concern; |
| +// and Revision, ChangedFields and PayloadErasedAt are added for §5's |
| +// retention rule, which Sheets has no equivalent of. |
| // |
| // Retention is the load-bearing part and it is not tidiness. Kind, |
| // ObjectKind, ObjectID, ActorID and CreatedAt are kept permanently and |
| @@ -178,17 +254,39 @@ type Ref struct { |
| // unbroken chain and erasure has to be able to break it. |
| type Event struct { |
| ID int64 |
| - Kind string `gorm:"index;size:32"` |
| - ObjectKind string `gorm:"index;size:16"` |
| - ObjectID int64 `gorm:"index"` |
| - ActorID string `gorm:"size:128"` |
| + Kind string `gorm:"index;size:32;not null"` |
| + ObjectKind string `gorm:"size:16;not null;index:idx_events_object"` |
| + ObjectID int64 `gorm:"index:idx_events_object"` |
| + // ActorID is a Member row, or zero for the app acting as itself. |
| + // Indexed because "who deleted this" is the one audit question the |
| + // durable table has to answer without the key. |
| + ActorID int64 `gorm:"index"` |
| + // Revision is the object's revision AFTER this event. Restore needs |
| + // it to check the diff chain is contiguous, and it is what lets the |
| + // log answer "at what revision" for S1 rather than reading the |
| + // current card row. |
| + Revision int64 |
| + // ChangedFields lists the FieldDef ids this event touched, as a JSON |
| + // array of integers. **Plaintext, and it must be**: §5 keeps "which |
| + // fields changed" permanently, and the only other place to put it is |
| + // inside PayloadSealed — which is the thing erasure empties. Ids are |
| + // opaque and structural, so this is §4.1 case 2. |
| + ChangedFields string `gorm:"size:512"` |
| // PayloadSealed is sealed under its own purpose, as Sheets seals |
| // "event.payload". Emptied when the window passes; the row stays. |
| PayloadSealed []byte |
| + // PayloadErasedAt records that the payload was deliberately removed. |
| + // |
| + // Without it an empty payload is ambiguous between "this event never |
| + // carried values" and "these values were erased" — and a restore path |
| + // that cannot tell them apart is §5's named failure: a silently broken |
| + // chain that still offers a restore button and reconstructs a wrong |
| + // object. |
| + PayloadErasedAt *time.Time |
| // TriggerID is the event that caused this one. Sheets uses it for a |
| // recalc's input; fichas uses it for the derived summary, so that a |
| // machine-written change names the entry that provoked it and the |
| // actor stays honest (§5, §7). |
| - TriggerID int64 |
| + TriggerID int64 `gorm:"index"` |
| CreatedAt time.Time `gorm:"index"` |
| } |
diff --git a/internal/fichas/models_test.go b/internal/fichas/models_test.go| new file mode 100644 |
| index 0000000..feba572 |
| --- /dev/null |
| +++ b/internal/fichas/models_test.go |
| @@ -0,0 +1,95 @@ |
| +package fichas |
| + |
| +import ( |
| + "reflect" |
| + "testing" |
| +) |
| + |
| +// plaintextColumns is the allowlist of string columns permitted to reach |
| +// the durable file, and **it is this app's leak list**. Adding a column |
| +// here is a deliberate act with a reason; adding one to a model without |
| +// editing this map fails the build of the gate below. |
| +// |
| +// §4.1 permits plaintext in three cases and no others: |
| +// |
| +// 1. the suite contract's feed row, which must be answerable without the |
| +// key and must pass a plaintext grep unaided; |
| +// 2. referential integrity and routing; |
| +// 3. the reach mark, which a sibling evaluates with no asker and no |
| +// unsealing. |
| +// |
| +// Everything below is case 2 or case 3, plus keyed one-way prints, which |
| +// are not plaintext content: a print is an HMAC under a purpose key and |
| +// yields nothing to someone holding the file. |
| +// |
| +// Sheets and memoria each have a gate of this shape |
| +// (TestNoPlaintextValueColumns, TestPlaintextNeverReachesTheDatabase). |
| +// fichas had neither until this test, which meant `Name string` could be |
| +// added to Card and CI would pass — in the app whose defining property is |
| +// that it does not do that. |
| +var plaintextColumns = map[string]string{ |
| + // Keyed one-way prints. Not content. |
| + "Member.AddressPrint": "print: finds a member without holding an address", |
| + "CardValue.Print": "print: equality on a TierPrint field", |
| + "EntryValue.Print": "print: equality on a TierPrint field", |
| + "Ref.Print": "print: dedupe on re-import", |
| + |
| + // Case 2 — routing and referential integrity. Every one of these is |
| + // this app's own vocabulary, chosen by us, never the team's words. |
| + "CardType.Key": "opaque constant, ours; code selects on it", |
| + "FieldDef.Key": "opaque constant, ours; the print purpose derives from it", |
| + "FieldDef.Kind": "the value's shape — text, date, enum — our vocabulary", |
| + "FieldDef.Tier": "which tier a field's values live in; see the note below", |
| + "Entry.Kind": "routing: call, note", |
| + "Entry.Source": "routing: manual, calls", |
| + "Event.Kind": "routing: created, updated, deleted", |
| + "Event.ObjectKind": "routing: card, entry", |
| + "Event.ChangedFields": "JSON array of FieldDef ids — opaque, and §5 keeps it permanently", |
| +} |
| + |
| +// FieldDef.Tier is plaintext for a reason worth stating because the design |
| +// does not give it: this gate, and any reviewer, must be able to read the |
| +// leak list off the file WITHOUT the key. A sealed tier would mean the only |
| +// way to audit what is exposed is to hold the DEK, which inverts the point |
| +// of writing a leak list down. |
| + |
| +// TestNoPlaintextColumnEscapesTheAllowlist walks every model and fails on a |
| +// string column that is not on the list above. |
| +// |
| +// []byte columns are not checked: by convention in this app every one of |
| +// them is sealed, and the file gate in fichastest proves that for the |
| +// values this app actually writes. |
| +func TestNoPlaintextColumnEscapesTheAllowlist(t *testing.T) { |
| + for _, m := range Models { |
| + st := reflect.TypeOf(m).Elem() |
| + for i := 0; i < st.NumField(); i++ { |
| + f := st.Field(i) |
| + if f.Type.Kind() != reflect.String { |
| + continue |
| + } |
| + name := st.Name() + "." + f.Name |
| + if _, ok := plaintextColumns[name]; !ok { |
| + t.Errorf("%s is a plaintext string column and is not on the allowlist.\n"+ |
| + "If it must be plaintext, add it to plaintextColumns with the §4.1 case it falls under.\n"+ |
| + "If it holds anything a person typed, seal it.", name) |
| + } |
| + } |
| + } |
| +} |
| + |
| +// The allowlist must not rot: an entry naming a field that no longer exists |
| +// makes the list stop being a description of the schema. |
| +func TestAllowlistHasNoStaleEntries(t *testing.T) { |
| + live := map[string]bool{} |
| + for _, m := range Models { |
| + st := reflect.TypeOf(m).Elem() |
| + for i := 0; i < st.NumField(); i++ { |
| + live[st.Name()+"."+st.Field(i).Name] = true |
| + } |
| + } |
| + for name := range plaintextColumns { |
| + if !live[name] { |
| + t.Errorf("plaintextColumns names %s, which no longer exists", name) |
| + } |
| + } |
| +} |
diff --git a/internal/fichas/print.go b/internal/fichas/print.go| new file mode 100644 |
| index 0000000..300bf3d |
| --- /dev/null |
| +++ b/internal/fichas/print.go |
| @@ -0,0 +1,84 @@ |
| +package fichas |
| + |
| +import ( |
| + "fmt" |
| + "strings" |
| + "unicode" |
| + |
| + "amadan.net/oficina/fichas/internal/fichas/seal" |
| +) |
| + |
| +// printFor returns the equality print for a value of fd, or "" when fd is |
| +// not a printed field. |
| +// |
| +// Two jobs, both of which seal.Print deliberately does not do. |
| +// |
| +// **It refuses a field that is not TierPrint.** seal.Print takes a bare |
| +// string and its doc comment was the only guard; a comment is not a guard. |
| +// Printing a TierSealed field would publish equality classes for exactly |
| +// the values §4.2 sealed to avoid publishing — `stage` has five, and |
| +// frequency analysis puts five classes in order. |
| +// |
| +// **It canonicalises first.** An HMAC over raw bytes makes `Ana@Rothar.ie` |
| +// and `ana@rothar.ie` different values, and `+353 1 555 0100` different |
| +// from `+35315550100`. The print exists to answer "do we already have this |
| +// contact"; without canonicalisation it misses on the commonest real |
| +// variance and the feature quietly does not work. memoria normalises an |
| +// address before fingerprinting for the same reason. |
| +func printFor(s *seal.Sealer, fd FieldDef, value string) (string, error) { |
| + if fd.Tier != TierPrint { |
| + return "", fmt.Errorf( |
| + "fichas: %s is tier %s and must not be printed; printing it would publish its equality classes", |
| + fd.Key, fd.Tier) |
| + } |
| + return s.Print(fd.Key, []byte(canonical(fd.Kind, value))) |
| +} |
| + |
| +// canonical folds away the variance that should not change identity. |
| +// |
| +// Conservative on purpose: it removes case, surrounding space and, for a |
| +// phone number, formatting. It does NOT try to parse or validate — a |
| +// canonicaliser that rejects is a validator, and a validator here would |
| +// silently drop values out of the print space rather than refusing them |
| +// where a person could see it. |
| +func canonical(kind, v string) string { |
| + v = strings.TrimSpace(v) |
| + switch kind { |
| + case KindNumber: |
| + return v |
| + case KindText, KindEnum, KindDate, KindRef: |
| + // Phone numbers arrive as text. Keeping digits and a leading + |
| + // makes "+353 1 555 0100" and "+35315550100" one value; anything |
| + // with a letter in it is left alone, so this cannot mangle a name. |
| + if looksLikeAPhoneNumber(v) { |
| + return stripPhoneFormatting(v) |
| + } |
| + return strings.ToLower(strings.Join(strings.Fields(v), " ")) |
| + default: |
| + return strings.ToLower(v) |
| + } |
| +} |
| + |
| +func looksLikeAPhoneNumber(v string) bool { |
| + digits := 0 |
| + for _, r := range v { |
| + switch { |
| + case unicode.IsDigit(r): |
| + digits++ |
| + case r == '+' || r == '-' || r == ' ' || r == '(' || r == ')' || r == '.': |
| + default: |
| + return false |
| + } |
| + } |
| + return digits >= 6 |
| +} |
| + |
| +func stripPhoneFormatting(v string) string { |
| + var b strings.Builder |
| + for i, r := range v { |
| + if unicode.IsDigit(r) || (r == '+' && i == 0) { |
| + b.WriteRune(r) |
| + } |
| + } |
| + return b.String() |
| +} |
diff --git a/internal/fichas/print_test.go b/internal/fichas/print_test.go| new file mode 100644 |
| index 0000000..daf46fc |
| --- /dev/null |
| +++ b/internal/fichas/print_test.go |
| @@ -0,0 +1,76 @@ |
| +package fichas |
| + |
| +import "testing" |
| + |
| +func fd(key, kind string, tier Tier) FieldDef { |
| + return FieldDef{Key: key, Kind: kind, Tier: tier} |
| +} |
| + |
| +// The guard seal.Print could not enforce. |
| +func TestPrintForRefusesAFieldThatIsNotPrinted(t *testing.T) { |
| + s := testSealer(t) |
| + for _, tier := range []Tier{TierSealed, TierPlain, Tier("")} { |
| + if _, err := printFor(s, fd(FieldStage, KindEnum, tier), "Lead"); err == nil { |
| + t.Errorf("a %q field was printed; its equality classes would be published", tier) |
| + } |
| + } |
| +} |
| + |
| +func TestPrintForAcceptsAPrintedField(t *testing.T) { |
| + s := testSealer(t) |
| + got, err := printFor(s, fd(FieldEmail, KindText, TierPrint), "ana@rothar.ie") |
| + if err != nil { |
| + t.Fatalf("printFor: %v", err) |
| + } |
| + if got == "" { |
| + t.Fatal("printFor returned an empty print with no error") |
| + } |
| +} |
| + |
| +// Without this the dedupe the print exists for misses on the commonest |
| +// real variance, and "do we already have this contact" quietly stops |
| +// working. |
| +func TestPrintForFoldsTheVarianceThatIsNotIdentity(t *testing.T) { |
| + s := testSealer(t) |
| + email := fd(FieldEmail, KindText, TierPrint) |
| + phone := fd(FieldPhone, KindText, TierPrint) |
| + |
| + same := func(f FieldDef, a, b string) { |
| + t.Helper() |
| + pa, err := printFor(s, f, a) |
| + if err != nil { |
| + t.Fatalf("printFor(%q): %v", a, err) |
| + } |
| + pb, err := printFor(s, f, b) |
| + if err != nil { |
| + t.Fatalf("printFor(%q): %v", b, err) |
| + } |
| + if pa != pb { |
| + t.Errorf("%q and %q print differently under %s", a, b, f.Key) |
| + } |
| + } |
| + |
| + same(email, "ana@rothar.ie", "Ana@Rothar.ie") |
| + same(email, "ana@rothar.ie", " ana@rothar.ie ") |
| + same(phone, "+353 1 555 0100", "+35315550100") |
| + same(phone, "(01) 555-0100", "015550100") |
| +} |
| + |
| +// Canonicalisation must not merge things that are genuinely different. |
| +func TestPrintForKeepsDifferentValuesDifferent(t *testing.T) { |
| + s := testSealer(t) |
| + name := fd(FieldName, KindText, TierPrint) |
| + a, _ := printFor(s, name, "Rothar Ltd") |
| + b, _ := printFor(s, name, "Rothar Limited") |
| + if a == b { |
| + t.Fatal("two different names print the same") |
| + } |
| +} |
| + |
| +// A name with digits in it must not be treated as a phone number and have |
| +// its letters stripped. |
| +func TestCanonicalDoesNotMangleANameWithDigits(t *testing.T) { |
| + if got := canonical(KindText, "Studio 54 Ltd"); got != "studio 54 ltd" { |
| + t.Fatalf("canonical mangled a name: %q", got) |
| + } |
| +} |
diff --git a/internal/fichas/seal/vectors_test.go b/internal/fichas/seal/vectors_test.go| new file mode 100644 |
| index 0000000..608dcf9 |
| --- /dev/null |
| +++ b/internal/fichas/seal/vectors_test.go |
| @@ -0,0 +1,72 @@ |
| +package seal |
| + |
| +import ( |
| + "encoding/hex" |
| + "testing" |
| +) |
| + |
| +// vectorDEK is a fixed, published key. It protects nothing; it exists so |
| +// the derivation below is reproducible by anyone reading this file. |
| +func vectorDEK() []byte { |
| + k := make([]byte, 32) |
| + for i := range k { |
| + k[i] = byte(i) |
| + } |
| + return k |
| +} |
| + |
| +// TestPurposeKeyVectors pins the key derivation. |
| +// |
| +// The package comment says the HKDF info string "must never be changed: it |
| +// is baked into every byte already sealed" — and until this test nothing |
| +// enforced that. Every other test in this package passes just as happily |
| +// if "oficina/fichas:" becomes "oficina/fichas/", or if the salt changes, |
| +// or if the hash does: round-trips still round-trip, prints are still |
| +// deterministic, purposes are still separated. What breaks is only visible |
| +// against an EXISTING database, at which point the corpus is unreadable |
| +// and the change has shipped. |
| +// |
| +// Sheets has a test of this shape. These vectors are the whole contract: |
| +// if one of them changes, every value already sealed by a deployed fichas |
| +// is lost, and no other test in this repo will tell you. |
| +func TestPurposeKeyVectors(t *testing.T) { |
| + s, err := New(vectorDEK()) |
| + if err != nil { |
| + t.Fatalf("New: %v", err) |
| + } |
| + for purpose, want := range map[string]string{ |
| + PurposeLabel: "957bfc889b61cbbb3acfe4a86500622c338a30e43d158068efd64fa69afbcb94", |
| + PurposeValue: "43182def144d4b665177715e00b1cbcefae8f38f924393beb9332c131c9c4c0d", |
| + PurposeEventPayload: "2252fd1b0ee0a87fc53d2090c39949bfcc402df882e2d6c63a96c30023d36358", |
| + PurposeRefTarget: "d1390ea00de365f9cc6c87b049c7ee3ded5f1190ff7f01f74032d24986d40543", |
| + printPurpose("f_email"): "2343a23362e9fdd244537a3375968e5b259e14b78436d230749fba3055766b8e", |
| + refPrintPurpose: "f9723057c036c5b6973765134a02255dde2e0f34777c2abf729c5936a43da8b0", |
| + } { |
| + key, err := s.purposeKey(purpose) |
| + if err != nil { |
| + t.Fatalf("purposeKey(%q): %v", purpose, err) |
| + } |
| + if got := hex.EncodeToString(key); got != want { |
| + t.Errorf("purposeKey(%q) = %s, want %s\n"+ |
| + "This is not a test to update. Every value already sealed under this purpose "+ |
| + "by a deployed instance is now unreadable.", purpose, got, want) |
| + } |
| + } |
| +} |
| + |
| +// The purpose strings themselves are part of the contract: renaming a |
| +// constant's VALUE is the same break as changing the info string. |
| +func TestPurposeNamesAreStable(t *testing.T) { |
| + for got, want := range map[string]string{ |
| + PurposeLabel: "type.label", |
| + PurposeValue: "field.value", |
| + PurposeEventPayload: "event.payload", |
| + PurposeRefTarget: "ref.target", |
| + printPurpose("f_email"): "field.print:f_email", |
| + refPrintPurpose: "ref.print", |
| + } { |
| + if got != want { |
| + t.Errorf("a purpose string changed: got %q, want %q", got, want) |
| + } |
| + } |
| +} |
diff --git a/internal/fichastest/gates_test.go b/internal/fichastest/gates_test.go| new file mode 100644 |
| index 0000000..39ec391 |
| --- /dev/null |
| +++ b/internal/fichastest/gates_test.go |
| @@ -0,0 +1,100 @@ |
| +package fichastest |
| + |
| +import ( |
| + "io" |
| + "log/slog" |
| + "os" |
| + "path/filepath" |
| + "strings" |
| + "testing" |
| + |
| + fichas "amadan.net/oficina/fichas/internal/fichas" |
| + "github.com/carlosframework/rastrillo/db" |
| +) |
| + |
| +// TestPlaintextNeverReachesTheDatabase is the gate this app exists to pass. |
| +// memoria has one of the same shape (TestPlaintextNeverReachesTheDatabase), |
| +// Sheets has the schema-side twin, and fichas had neither until now. |
| +// |
| +// It boots the real app over a real file, which seeds the catalogue, then |
| +// reads every byte the app wrote — the database AND its write-ahead log, |
| +// because a value can sit in the WAL for a long time before it is |
| +// checkpointed, and a gate that greps only the .db is a gate that passes |
| +// while the plaintext is on disk next to it. |
| +func TestPlaintextNeverReachesTheDatabase(t *testing.T) { |
| + dir := t.TempDir() |
| + logger := slog.New(slog.NewTextHandler(io.Discard, nil)) |
| + |
| + d, err := db.Open(filepath.Join(dir, "fichas.db"), logger) |
| + if err != nil { |
| + t.Fatalf("db.Open: %v", err) |
| + } |
| + if _, err := fichas.App(d, testOrigin, testSealer(t), logger); err != nil { |
| + t.Fatalf("App: %v", err) |
| + } |
| + // Close so the WAL is flushed and every byte is on disk to be read. |
| + if err := d.Close(); err != nil { |
| + t.Fatalf("Close: %v", err) |
| + } |
| + |
| + // Labels the seeded catalogue writes. Each is sealed, so none may |
| + // appear. "Customer" is deliberately NOT in this list: the card type's |
| + // opaque key is `t_customer`, which legitimately contains it, and a |
| + // canary that the schema is allowed to match is a canary that teaches |
| + // the next person to weaken the test. |
| + canaries := []string{ |
| + "Call log", |
| + "Next action", |
| + "What was said", |
| + "Duration", |
| + "Lead", |
| + "Contacted", |
| + "Proposal", |
| + } |
| + |
| + entries, err := os.ReadDir(dir) |
| + if err != nil { |
| + t.Fatalf("reading %s: %v", dir, err) |
| + } |
| + if len(entries) == 0 { |
| + t.Fatal("the app wrote no files: this gate would pass vacuously") |
| + } |
| + |
| + checked := 0 |
| + for _, e := range entries { |
| + if e.IsDir() { |
| + continue |
| + } |
| + raw, err := os.ReadFile(filepath.Join(dir, e.Name())) |
| + if err != nil { |
| + t.Fatalf("reading %s: %v", e.Name(), err) |
| + } |
| + checked++ |
| + for _, c := range canaries { |
| + if strings.Contains(string(raw), c) { |
| + t.Errorf("%s contains the plaintext %q", e.Name(), c) |
| + } |
| + } |
| + } |
| + if checked == 0 { |
| + t.Fatal("no files were read: this gate would pass vacuously") |
| + } |
| + t.Logf("checked %d file(s) for %d canaries", checked, len(canaries)) |
| +} |
| + |
| +// The gate above is only worth anything if it can fail. This proves the |
| +// grep actually looks: the same canary, written in the clear to the same |
| +// directory, must be found. |
| +func TestTheGateCanFail(t *testing.T) { |
| + dir := t.TempDir() |
| + if err := os.WriteFile(filepath.Join(dir, "leak.db"), []byte("x Next action x"), 0o600); err != nil { |
| + t.Fatalf("WriteFile: %v", err) |
| + } |
| + raw, err := os.ReadFile(filepath.Join(dir, "leak.db")) |
| + if err != nil { |
| + t.Fatalf("ReadFile: %v", err) |
| + } |
| + if !strings.Contains(string(raw), "Next action") { |
| + t.Fatal("the grep this gate relies on does not find a plaintext canary") |
| + } |
| +} |