oficina / docs Public

Verify principal foreign keys and legacy rollback probes

Paul Campbell pushed by paul@keymail.dev 1 month ago 10a8364d457c3e3835fb523440ce54229848c623 parent 07099a9

7 files changed, +187 −49

implementation/home-principal-store.md +17 −0
internal/docs/principalstore/store.go +7 −0
internal/docs/principalstore/store_test.go +29 −0
internal/docs/principalstore/testdata/compat_main.go.txt +99 −41
internal/docs/principalstore/testdata/compat_new.go.txt +20 −6
internal/docs/principalstore/testdata/compat_old.go.txt +7 −2
scripts/check-principal-compat.py +8 −0

CI passed — run details

diff --git a/implementation/home-principal-store.md b/implementation/home-principal-store.md

new file mode 100644

index 0000000..aef3f0e

--- /dev/null

+++ b/implementation/home-principal-store.md

@@ -0,0 +1,17 @@

+# Unmounted Home principal ledger

+This data-only slice implements designa5a791819e65f573a8c329b8cb523771d659c975 (Docs53), based on integration1e862f8. Docs55/task68 owns review. `principalstore.Schema` remains separate and unmounted. No Actor, session, route, roster admission, grant query or production configuration changes are included.

+API: `New(db, Config{Issuer, Team, RequireMembership})`, `Person`, `Resolve`, and `EnsureHome`. Principal exposes ID, kind and an optional verified local address pointer. A Home-only address is nil; it is never an empty or invented people.email. Person/Resolve are identity-only reads. EnsureHome receives an already authenticated, consumed Home identity and refuses creation without the explicit membership-check callback. Constructor verifies foreign_keys is enabled.

+Roster seam agreed with consumer_implementation: request names exact pinned issuer, team and subject. Missing/untrusted/expired/incomplete cache or failed cold refresh maps to ErrUnavailable; a current complete snapshot lacking the member maps to ErrRefused. The callback may refresh before the SQL transaction, never while holding the writer. Success is a bounded observation, not durable authority. No role, epoch, timestamp or standing is stored in principal records; every actual access needs a fresh authority decision in later integration. No session is minted here.

+The SQL migration performs its census and ledger copy in one migration transaction, keeping the original people/link FK tables. A failing census leaves no partial principal schema. Local IDs remain below2^52, Home-only IDs span2^52 through2^53-1. The old people's AFTER INSERT mirror preserves the assigned ID; immutable identity and cross-table mapping guards refuse reuse/reassignment/conflict. Home allocation takes the writer lock before checking either mapping, then inserts the allocator advance/principal/mapping atomically. Neither EnsureHome nor New creates samples or documents. Exhaustion refuses instead of wrapping. New tables have actual FKs; old owner/author fields gain reference-validation triggers while preserving their0 sentinel.

+Focused tests passed with the race detector. They cover typed absence, old/new IDs and LastInsertId, callback errors and missing callback, identity-only resolution, concurrent first arrival, link conflicts, immutable IDs, constraints,0-sentinel updates, foreign_keys, atomic preflight failure, and migration idempotency/older known migration sets. An opaque encoding1 fixture separately proves the migration preserves update/snapshot bytes, encoding flags and key-check/fingerprint bytes without decoding or rewriting them; it does not claim a valid encrypted-document replay test.

+`scripts/check-principal-compat.py --artifacts <new temporary directory>` builds helper executables against actual frozen87662f5,8a126727,3ed7665 and candidate source. The helper calls those versions' real App.New/migrate.Apply, EnsurePerson, Create, ApplyUpdate, Grant and Text methods. Three disposable old→new→old→new cycles preserve local allocations1,2,3,4, Home principal2^52, a Home-owned document, original document sqid/owner/grant author, real Yjs text, update+snapshot hashes and encoding0. The helpers do not authenticate a Home session or certify those older binaries as full suite rollback releases. 3ed-lineage runtime/session rollback still needs the combined admission review; these probes establish data/allocator compatibility only.

+Initial exact-candidate proof at07099a9 is recorded in /tmp/oficina-principal-compat-run1/verified.json. Final candidate is rerun after changes and recorded in the branch discussion. Full repo gate passed at the initial implementation; final checks are recorded separately rather than inferred. Two overlays fail the intended tests: substituting a nonnil empty verified address for absence, and removing the people ID-immutability trigger. This slice does not yet change Actor/grant queries, so it does not claim a malformed-empty-grant redemption integration test.

+Before mounting: combine with authoritative roster admission and the typed Actor/grant/addressOf changes from the design; preserve existing-account linking, production identity keys/content encoding and original person IDs. Run the named runtime rollback and platform-fenced restore drill. No live database census, content activation, secrets rotation, canary or deployment was performed by this branch.

diff --git a/internal/docs/principalstore/store.go b/internal/docs/principalstore/store.go

index b9d727d..e3409cb 100644

--- a/internal/docs/principalstore/store.go

+++ b/internal/docs/principalstore/store.go

@@ -66,6 +66,13 @@ func New(db *sql.DB, cfg Config) (*Store, error) {

if db == nil || !origin(cfg.Issuer) || !opaque(cfg.Team, 64) {

return nil, ErrRefused

}

+ var foreignKeys int

+ if err := db.QueryRow("PRAGMA foreign_keys").Scan(&foreignKeys); err != nil {

+ return nil, ErrUnavailable

+ }

+ if foreignKeys != 1 {

+ return nil, ErrRefused

+ }

return &Store{db: db, cfg: cfg}, nil

}

diff --git a/internal/docs/principalstore/store_test.go b/internal/docs/principalstore/store_test.go

index 715f234..c443785 100644

--- a/internal/docs/principalstore/store_test.go

+++ b/internal/docs/principalstore/store_test.go

@@ -243,3 +243,32 @@ func TestPreflightIsAtomicAndPreservesExistingSchema(t *testing.T) {

})

}

+func TestRequiresForeignKeysAndPreservesEncodedBytes(t *testing.T) {

+ d, _ := fixture(t, false)

+ id := local(t, d, "owner@example.invalid")

+ exec(t, d, `INSERT INTO documents(sqid,title,guid,client_id,created_at,updated_at,owner_id) VALUES('doc','kept','guid',1,CURRENT_TIMESTAMP,CURRENT_TIMESTAMP,?)`, id)

+ exec(t, d, `INSERT INTO doc_updates(doc_id,seq,update_bin,created_at,content_encoding)VALUES(1,1,X'00FFAABB',CURRENT_TIMESTAMP,1)`)

+ exec(t, d, `INSERT INTO doc_snapshots(doc_id,seq,state,abi_version,created_at,content_encoding)VALUES(1,1,X'010203FF',2,CURRENT_TIMESTAMP,1)`)

+ exec(t, d, `INSERT INTO content_sealing VALUES(1,1,'fixed-key-fingerprint',X'090807',0)`)

+ if _, e := migrate.Apply(context.Background(), d, p.Schema); e != nil {

+ t.Fatal(e)

+ }

+ var update, snapshot, key string

+ if e := d.Writer().QueryRow(`SELECT hex(update_bin)||':'||content_encoding FROM doc_updates`).Scan(&update); e != nil {

+ t.Fatal(e)

+ }

+ if e := d.Writer().QueryRow(`SELECT hex(state)||':'||content_encoding FROM doc_snapshots`).Scan(&snapshot); e != nil {

+ t.Fatal(e)

+ }

+ if e := d.Writer().QueryRow(`SELECT key_fingerprint||':'||hex(key_check)||':'||encoding FROM content_sealing`).Scan(&key); e != nil {

+ t.Fatal(e)

+ }

+ if update != "00FFAABB:1" || snapshot != "010203FF:1" || key != "fixed-key-fingerprint:090807:1" {

+ t.Fatal(update, snapshot, key)

+ }

+ exec(t, d, `PRAGMA foreign_keys=OFF`)

+ if _, e := p.New(d.Writer(), p.Config{Issuer: issuer, Team: "A1"}); e != p.ErrRefused {

+ t.Fatal("foreign keys disabled", e)

+ }

diff --git a/internal/docs/principalstore/testdata/compat_main.go.txt b/internal/docs/principalstore/testdata/compat_main.go.txt

index f973ce8..a4d3ddd 100644

--- a/internal/docs/principalstore/testdata/compat_main.go.txt

+++ b/internal/docs/principalstore/testdata/compat_main.go.txt

@@ -1,44 +1,102 @@

package main

-import(

- "context"

- "crypto/sha256"

- "encoding/hex"

- "encoding/json"

- "flag"

- "fmt"

- "log/slog"

- "os"

- "io"

- "github.com/carlosframework/docs/internal/docs"

- "github.com/carlosframework/docs/internal/core"

- "github.com/carlosframework/rastrillo/db"

+import (

+ "context"

+ "crypto/sha256"

+ "encoding/hex"

+ "encoding/json"

+ "flag"

+ "fmt"

+ "github.com/carlosframework/docs/internal/core"

+ "github.com/carlosframework/docs/internal/docs"

+ "github.com/carlosframework/rastrillo/db"

+ "io"

+ "log/slog"

+ "os"

)

-func must(e error){if e!=nil{panic(e)}}

-func main(){

- path:=flag.String("db","","disposable database");address:=flag.String("address","","synthetic local account");phase:=flag.String("phase","","phase");flag.Parse()

- ctx:=context.Background();app,e:=docs.New(docs.Config{DBPath:*path,Env:"development",Origin:"https://compat.example",SampleDocuments:"false",Logger:slog.New(slog.NewTextHandler(io.Discard,nil))});must(e);defer app.Close()

- d,e:=db.Open(*path,nil);must(e);defer d.Close()

- id,e:=app.Store().EnsurePerson(ctx,*address);must(e)

- principal:=principalStep(ctx,d,*phase)

- if principal>0 {var n int;must(d.Writer().QueryRow("SELECT COUNT(*) FROM documents WHERE owner_id=?",principal).Scan(&n));if n==0{_,e:=app.Store().Create(ctx,principal,"home-owned");must(e)}}

- var sqid string;e=d.Writer().QueryRow(`SELECT sqid FROM documents WHERE title='kept-content'`).Scan(&sqid)

- if e!=nil{

- doc,e:=app.Store().Create(ctx,id,"kept-content");must(e);sqid=doc.Sqid

- update,e:=os.ReadFile("fixtures/vectors/fragment/hello.update.bin");must(e)

- _,e=app.Store().ApplyUpdate(ctx,doc,docs.Actor{PersonID:id,Subject:*address},update);must(e)

- must(app.Store().Grant(ctx,id,sqid,"reader@example.invalid",docs.RoleWriter))

- _,e=d.Writer().Exec(`INSERT INTO doc_snapshots(doc_id,seq,state,abi_version,created_at)VALUES(?,1,?,?,CURRENT_TIMESTAMP)`,doc.ID,update,core.ABIVersion);must(e)

- _,e=d.Writer().Exec(`INSERT INTO documents(sqid,title,guid,client_id,created_at,updated_at,owner_id)VALUES('orphan','orphan','orphan',1,CURRENT_TIMESTAMP,CURRENT_TIMESTAMP,0)`);must(e)

- _,e=d.Writer().Exec(`INSERT INTO grants(doc_id,subject,role,created_by)VALUES(0,'sentinel@example.invalid','reader',0)`);must(e)

- }

- var owner int64;must(d.Writer().QueryRow(`SELECT owner_id FROM documents WHERE sqid=?`,sqid).Scan(&owner))

- doc,e:=app.Store().Get(ctx,owner,sqid);must(e);text,e:=app.Store().Text(ctx,doc);must(e);if text!="hello world"{panic(text)}

- if _,e=d.Writer().Exec(`UPDATE documents SET owner_id=owner_id`);e!=nil{panic(e)}

- if _,e=d.Writer().Exec(`UPDATE grants SET created_by=created_by`);e!=nil{panic(e)}

- rows,e:=d.Writer().Query(`SELECT update_bin,content_encoding FROM doc_updates WHERE doc_id=(SELECT id FROM documents WHERE title='kept-content') UNION ALL SELECT state,content_encoding FROM doc_snapshots WHERE doc_id=(SELECT id FROM documents WHERE title='kept-content')`);must(e);h:=sha256.New();encodings:=[]int{};for rows.Next(){var b []byte;var encoding int;must(rows.Scan(&b,&encoding));h.Write(b);encodings=append(encodings,encoding)};must(rows.Err());rows.Close()

- var author int64;must(d.Writer().QueryRow(`SELECT created_by FROM grants WHERE subject='reader@example.invalid'`).Scan(&author))

- var homeCount int;must(d.Writer().QueryRow(`SELECT COUNT(*) FROM documents WHERE owner_id>=4503599627370496`).Scan(&homeCount))

- var ledger int;d.Writer().QueryRow(`SELECT COUNT(*) FROM docs_principals`).Scan(&ledger)

- out:=map[string]any{"phase":*phase,"local_id":id,"home_id":principal,"owner":owner,"grant_author":author,"sqid":sqid,"content_hash":hex.EncodeToString(h.Sum(nil)),"encodings":encodings,"text":text,"principal_count":ledger,"home_owned":homeCount}

- b,e:=json.Marshal(out);must(e);fmt.Println(string(b))

+func must(e error) {

+ if e != nil {

+ panic(e)

+ }

+func main() {

+ path := flag.String("db", "", "disposable database")

+ address := flag.String("address", "", "synthetic local account")

+ phase := flag.String("phase", "", "phase")

+ flag.Parse()

+ ctx := context.Background()

+ app, e := docs.New(docs.Config{DBPath: *path, Env: "development", Origin: "https://compat.example", SampleDocuments: "false", Logger: slog.New(slog.NewTextHandler(io.Discard, nil))})

+ must(e)

+ defer app.Close()

+ d, e := db.Open(*path, nil)

+ must(e)

+ defer d.Close()

+ id, e := app.Store().EnsurePerson(ctx, *address)

+ must(e)

+ principal := principalStep(ctx, d, *phase)

+ if principal > 0 {

+ var n int

+ must(d.Writer().QueryRow("SELECT COUNT(*) FROM documents WHERE owner_id=?", principal).Scan(&n))

+ if n == 0 {

+ _, e := app.Store().Create(ctx, principal, "home-owned")

+ must(e)

+ }

+ var sqid string

+ e = d.Writer().QueryRow(`SELECT sqid FROM documents WHERE title='kept-content'`).Scan(&sqid)

+ if e != nil {

+ doc, e := app.Store().Create(ctx, id, "kept-content")

+ must(e)

+ sqid = doc.Sqid

+ update, e := os.ReadFile("fixtures/vectors/fragment/hello.update.bin")

+ must(e)

+ _, e = app.Store().ApplyUpdate(ctx, doc, docs.Actor{PersonID: id, Subject: *address}, update)

+ must(e)

+ must(app.Store().Grant(ctx, id, sqid, "reader@example.invalid", docs.RoleWriter))

+ _, e = d.Writer().Exec(`INSERT INTO doc_snapshots(doc_id,seq,state,abi_version,created_at)VALUES(?,1,?,?,CURRENT_TIMESTAMP)`, doc.ID, update, core.ABIVersion)

+ must(e)

+ _, e = d.Writer().Exec(`INSERT INTO documents(sqid,title,guid,client_id,created_at,updated_at,owner_id)VALUES('orphan','orphan','orphan',1,CURRENT_TIMESTAMP,CURRENT_TIMESTAMP,0)`)

+ must(e)

+ _, e = d.Writer().Exec(`INSERT INTO grants(doc_id,subject,role,created_by)VALUES(0,'sentinel@example.invalid','reader',0)`)

+ must(e)

+ }

+ var owner int64

+ must(d.Writer().QueryRow(`SELECT owner_id FROM documents WHERE sqid=?`, sqid).Scan(&owner))

+ doc, e := app.Store().Get(ctx, owner, sqid)

+ must(e)

+ text, e := app.Store().Text(ctx, doc)

+ must(e)

+ if text != "hello world" {

+ panic(text)

+ }

+ if _, e = d.Writer().Exec(`UPDATE documents SET owner_id=owner_id`); e != nil {

+ panic(e)

+ }

+ if _, e = d.Writer().Exec(`UPDATE grants SET created_by=created_by`); e != nil {

+ panic(e)

+ }

+ rows, e := d.Writer().Query(`SELECT update_bin,content_encoding FROM doc_updates WHERE doc_id=(SELECT id FROM documents WHERE title='kept-content') UNION ALL SELECT state,content_encoding FROM doc_snapshots WHERE doc_id=(SELECT id FROM documents WHERE title='kept-content')`)

+ must(e)

+ h := sha256.New()

+ encodings := []int{}

+ for rows.Next() {

+ var b []byte

+ var encoding int

+ must(rows.Scan(&b, &encoding))

+ h.Write(b)

+ encodings = append(encodings, encoding)

+ }

+ must(rows.Err())

+ rows.Close()

+ var author int64

+ must(d.Writer().QueryRow(`SELECT created_by FROM grants WHERE subject='reader@example.invalid'`).Scan(&author))

+ var homeCount int

+ must(d.Writer().QueryRow(`SELECT COUNT(*) FROM documents WHERE owner_id>=4503599627370496`).Scan(&homeCount))

+ var ledger int

+ d.Writer().QueryRow(`SELECT COUNT(*) FROM docs_principals`).Scan(&ledger)

+ out := map[string]any{"phase": *phase, "local_id": id, "home_id": principal, "owner": owner, "grant_author": author, "sqid": sqid, "content_hash": hex.EncodeToString(h.Sum(nil)), "encodings": encodings, "text": text, "principal_count": ledger, "home_owned": homeCount}

+ b, e := json.Marshal(out)

+ must(e)

+ fmt.Println(string(b))

}

diff --git a/internal/docs/principalstore/testdata/compat_new.go.txt b/internal/docs/principalstore/testdata/compat_new.go.txt

index 23a23be..835cbb1 100644

--- a/internal/docs/principalstore/testdata/compat_new.go.txt

+++ b/internal/docs/principalstore/testdata/compat_new.go.txt

@@ -1,8 +1,22 @@

package main

-import("context";"github.com/carlosframework/rastrillo/db";"github.com/carlosframework/rastrillo/migrate";"github.com/carlosframework/docs/internal/docs/linkstore";p "github.com/carlosframework/docs/internal/docs/principalstore")

-func principalStep(ctx context.Context,d *db.DB,phase string)int64{

- _,e:=migrate.Apply(ctx,d,migrate.Merge(linkstore.Schema,p.Schema));must(e)

- s,e:=p.New(d.Writer(),p.Config{Issuer:"https://id.oficina.works",Team:"A1",RequireMembership:func(context.Context,p.MembershipRequest)error{return nil}});must(e)

- v,e:=s.EnsureHome(ctx,p.HomeIdentity{Issuer:"https://id.oficina.works",Subject:"synthetic-member"});must(e)

- if v.VerifiedAddress!=nil{panic("invented address")};return v.ID

+import (

+ "context"

+ "github.com/carlosframework/docs/internal/docs/linkstore"

+ p "github.com/carlosframework/docs/internal/docs/principalstore"

+ "github.com/carlosframework/rastrillo/db"

+ "github.com/carlosframework/rastrillo/migrate"

+func principalStep(ctx context.Context, d *db.DB, phase string) int64 {

+ _, e := migrate.Apply(ctx, d, migrate.Merge(linkstore.Schema, p.Schema))

+ must(e)

+ s, e := p.New(d.Writer(), p.Config{Issuer: "https://id.oficina.works", Team: "A1", RequireMembership: func(context.Context, p.MembershipRequest) error { return nil }})

+ must(e)

+ v, e := s.EnsureHome(ctx, p.HomeIdentity{Issuer: "https://id.oficina.works", Subject: "synthetic-member"})

+ must(e)

+ if v.VerifiedAddress != nil {

+ panic("invented address")

+ }

+ return v.ID

}

diff --git a/internal/docs/principalstore/testdata/compat_old.go.txt b/internal/docs/principalstore/testdata/compat_old.go.txt

index c16bb3c..cc7b454 100644

--- a/internal/docs/principalstore/testdata/compat_old.go.txt

+++ b/internal/docs/principalstore/testdata/compat_old.go.txt

@@ -1,3 +1,8 @@

package main

-import("context";"github.com/carlosframework/rastrillo/db")

-func principalStep(context.Context,*db.DB,string)int64{return 0}

+import (

+ "context"

+ "github.com/carlosframework/rastrillo/db"

+func principalStep(context.Context, *db.DB, string) int64 { return 0 }

diff --git a/scripts/check-principal-compat.py b/scripts/check-principal-compat.py

index 1e78f44..081f730 100755

--- a/scripts/check-principal-compat.py

+++ b/scripts/check-principal-compat.py

@@ -5,6 +5,7 @@ import io

import json

import pathlib

import subprocess

+import sqlite3

import tarfile

parser = argparse.ArgumentParser()

@@ -51,7 +52,14 @@ for old in ["old876", "old8a", "old3ed"]:

raise RuntimeError("Home identity/ownership changed")

if index == 2 and result["home_owned"] != 1:

raise RuntimeError("old binary lost Home ownership")

+ if index and result["principal_count"] != index + 2:

+ raise RuntimeError("legacy insert did not mirror the canonical ledger")

phases.append(result)

+ with sqlite3.connect(db) as check:

+ if check.execute("PRAGMA foreign_key_check").fetchall():

+ raise RuntimeError("foreign key damage across rollback")

+ if check.execute("SELECT COUNT(*) FROM people WHERE email='' OR id>=4503599627370496").fetchone()[0]:

+ raise RuntimeError("Home-only identity leaked into legacy people")

results[old] = phases

(out / "verified.json").write_text(json.dumps({"candidate": current, "versions": records, "cycles": results}, indent=2))

print(out / "verified.json")