amadan
Search, sort and filter that stay instant at thousands of rows
Paul: "any search we ever do across records needs to feel instant, even when searching 1000s of rows." Measured against that, not asserted. What made it slow: every list request decrypted every card, and every decryption re-derived its key — HKDF, key schedule, GCM table — at 1.6µs a value. At 5,000 cards that was ~63ms before any searching began. The sealer now derives each purpose's AEAD once: 1,578ns → 100ns an Open. The key vectors still pass, so nothing about what is derived changed, and a 64-goroutine race test is clean. index.go is an in-memory index of decrypted cards — §4.1's "sealed is not unqueryable" made real, the same shape as memoria's in-memory FTS. Its change feed is the event log: every write already lands an Event in the same transaction as the change, so MAX(events.id) answers "has anything changed" and the events past it say what. No write path tells the index anything, which is what stops it going quietly stale. A type or field event rebuilds; a card event reloads one card. Search needs every term, case-insensitive, across every value and the latest call. Filters are equality on choice fields only — a free-text equality filter in a URL is a search that silently misses on case and spacing, so it is ignored. Sorting puts empty values last in both directions and ties on id so paging is stable; the comparator touches only prepared data. 100 rows a page. Measured at 5,000 customers: a search with a filter and a sort is 0.25ms in the index; a full page request, render included, is 1–7ms; a cold load on wake is 88ms, once. Everything lives in the URL (list.go), so every state is a link and the page works with scripts off. live.js makes the search box update the results as you type: debounced, a newer keystroke aborts an older request so a slow answer never overwrites a fresher one, and the URL follows via replaceState. ⌘↩ (Ctrl+Enter elsewhere) submits from any textarea, via requestSubmit so validation and busy.js see an ordinary submit. The browser test file had not compiled for most of fichas' life — it called App with the scaffold's signature, and its build tag kept it out of make ci. Fixed, and extended: TestLiveSearch types into the box and checks the results narrow with no page load (a window marker that only survives if nothing reloaded), and TestCmdEnterSubmits presses ⌘↩ in the call-log textarea. Both were run with the scripts removed from the page, and both failed. `make browser` runs them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
24 files changed,
+1508
−88
Makefile+8 −0internal/fichas/app.go+1 −1internal/fichas/catalogue.go+11 −0internal/fichas/catalogue_test.go+2 −2internal/fichas/handlers.go+31 −11internal/fichas/index.go+481 −0internal/fichas/index_test.go+182 −0internal/fichas/list.go+259 −0internal/fichas/migrations/0003_alter_card_types.sql+4 −0internal/fichas/migrations/schema.sql+26 −1internal/fichas/models.go+11 −2internal/fichas/models_test.go+1 −0internal/fichas/render.go+3 −0internal/fichas/seal/bench_test.go+47 −0internal/fichas/seal/seal.go+32 −2internal/fichas/static/fichas.css+5 −0internal/fichas/static/live.js+87 −0internal/fichas/static/shortcuts.js+40 −0internal/fichas/store_test.go+1 −1internal/fichas/templates/index.html+57 −41internal/fichas/templates/shell.html+2 −0internal/fichastest/browser_test.go+120 −19internal/fichastest/surface_test.go+80 −7scripts/preview/main.go+17 −1
CI passed — run details
CI log
=== vet === go vet ./... === fmt === === test === go test ./... ? amadan.net/oficina/fichas/cmd/fichas [no test files] ok amadan.net/oficina/fichas/internal/fichas 0.512s ? amadan.net/oficina/fichas/internal/fichas/icons [no test files] ok amadan.net/oficina/fichas/internal/fichas/seal 0.012s ok amadan.net/oficina/fichas/internal/fichastest 0.292s ? amadan.net/oficina/fichas/scripts/mockups [no test files] ? amadan.net/oficina/fichas/scripts/preview [no test files] ? amadan.net/oficina/fichas/scripts/writeestilo [no test files] === migration-check === go run github.com/carlosframework/rastrillo/cmd/rastrillo migration check rastrillo migration check: models and migrations agree
diff --git a/Makefile b/Makefile| index 178f69d..1a8e365 100644 |
| --- a/Makefile |
| +++ b/Makefile |
| @@ -111,3 +111,11 @@ migration-check: |
| # instead of at boot on whatever machine notices next. If the app |
| # declares manifest resources, also add: rastrillo generate --check |
| ci: vet fmt-check test migration-check |
| + |
| +# browser runs the real-Chromium tests (internal/fichastest/browser_test.go). |
| +# Behind a build tag because CI runners may have no Chromium — which is |
| +# exactly how that file sat uncompiled for most of fichas' life. Run it |
| +# before any change to live.js, shortcuts.js or busy.js. |
| +.PHONY: browser |
| +browser: |
| + go test -tags browser ./internal/fichastest/ |
diff --git a/internal/fichas/app.go b/internal/fichas/app.go| index 6df16dc..40f24eb 100644 |
| --- a/internal/fichas/app.go |
| +++ b/internal/fichas/app.go |
| @@ -52,7 +52,7 @@ func App(d *db.DB, cfg Config, logger *slog.Logger) (*http.ServeMux, *app, error |
| return nil, nil, err |
| } |
| - a := &app{db: d, sealer: cfg.Sealer, cfg: cfg, logger: logger} |
| + a := &app{db: d, sealer: cfg.Sealer, cfg: cfg, ix: NewIndex(d.G, cfg.Sealer), logger: logger} |
| a.owners = cfg.Owners |
diff --git a/internal/fichas/catalogue.go b/internal/fichas/catalogue.go| index d1f6a54..fda15d1 100644 |
| --- a/internal/fichas/catalogue.go |
| +++ b/internal/fichas/catalogue.go |
| @@ -58,8 +58,19 @@ const ( |
| KindEnum = "enum" |
| KindNumber = "number" |
| KindRef = "ref" |
| + // KindLongText is paragraphs: a textarea, and cut short on a card |
| + // face. The seeded Notes field predates it and is KindText — changing |
| + // a shipped field's kind refuses to boot (Seed) — so isLong also |
| + // treats FieldNotes as long. |
| + KindLongText = "longtext" |
| ) |
| +// FaceLatestCall is the FaceKeys token for "the most recent entry". |
| +const FaceLatestCall = "_latest_call" |
| + |
| +// isLong reports whether a field is paragraphs rather than a line. |
| +func isLong(fd FieldDef) bool { return fd.Kind == KindLongText || fd.Key == FieldNotes } |
| + |
| type seedField struct { |
| Key string |
| Label string |
diff --git a/internal/fichas/catalogue_test.go b/internal/fichas/catalogue_test.go| index a918779..67bbfb5 100644 |
| --- a/internal/fichas/catalogue_test.go |
| +++ b/internal/fichas/catalogue_test.go |
| @@ -13,7 +13,7 @@ import ( |
| "gorm.io/gorm" |
| ) |
| -func testDB(t *testing.T) *gorm.DB { |
| +func testDB(t testing.TB) *gorm.DB { |
| t.Helper() |
| logger := slog.New(slog.NewTextHandler(io.Discard, nil)) |
| d, err := db.Open(filepath.Join(t.TempDir(), "seed.db"), logger) |
| @@ -27,7 +27,7 @@ func testDB(t *testing.T) *gorm.DB { |
| return d.G |
| } |
| -func testSealer(t *testing.T) *seal.Sealer { |
| +func testSealer(t testing.TB) *seal.Sealer { |
| t.Helper() |
| k := make([]byte, 32) |
| for i := range k { |
diff --git a/internal/fichas/handlers.go b/internal/fichas/handlers.go| index 979e212..ae6257b 100644 |
| --- a/internal/fichas/handlers.go |
| +++ b/internal/fichas/handlers.go |
| @@ -23,6 +23,9 @@ type app struct { |
| // the compiler will catch. |
| sealer *seal.Sealer |
| cfg Config |
| + // ix is the decrypted in-memory index that search, sort and filter |
| + // run over. It follows the event log, so no write path updates it. |
| + ix *Index |
| auth *auth.Auth |
| owners []string |
| // testIdentity and allowUnauthenticated let a test act as a person |
| @@ -160,27 +163,44 @@ func pluralLabel(s string) string { |
| } |
| // index is the record list, as a table or as cards — the view switch Paul |
| -// asked for. The view is a query parameter so it is a link, works with |
| -// scripts off, and can be bookmarked. |
| +// asked for — searched, filtered, sorted and paged from the in-memory index. |
| +// All of it lives in the URL (list.go), so every state is a link and the |
| +// page works with scripts off; live.js only makes the search box update the |
| +// results as you type. |
| func (a *app) index(w http.ResponseWriter, r *http.Request) { |
| - cards, err := ListCards(r.Context(), a.db.G, a.sealer, TypeCustomer) |
| - if err != nil { |
| + if err := a.ix.Sync(r.Context()); err != nil { |
| a.fail(w, r, err) |
| return |
| } |
| - fields, err := CardFields(r.Context(), a.db.G, a.sealer, TypeCustomer) |
| + fields, err := a.ix.Fields(TypeCustomer) |
| if err != nil { |
| a.fail(w, r, err) |
| return |
| } |
| - view := r.URL.Query().Get("view") |
| - if view != "cards" { |
| - view = "table" |
| + l := parseList(r, "/", fields) |
| + res, err := a.ix.Query(TypeCustomer, l.query()) |
| + if err != nil { |
| + a.fail(w, r, err) |
| + return |
| } |
| + filtered := l.Q != "" || len(l.Filters) > 0 |
| + clear := l.with(func(s *listState) { s.Q = "" }) |
| a.page(w, r, "index", map[string]any{ |
| - "Cards": cards, |
| - "Fields": fields, |
| - "View": view, |
| + "Result": res, |
| + "Cards": res.Cards, |
| + "Fields": fields, |
| + "View": l.View, |
| + "Q": l.Q, |
| + "Hidden": l.hidden(), |
| + "ClearHref": clear.href(), |
| + "Filters": l.filterMenus(fields), |
| + "Columns": l.columns(fields), |
| + "Pages": l.pages(res), |
| + "Count": countLine(res, filtered), |
| + "Filtered": filtered, |
| + "TableHref": l.with(func(s *listState) { s.View = "table" }).href(), |
| + "CardsHref": l.with(func(s *listState) { s.View = "cards" }).href(), |
| + "ResetHref": "/" + map[bool]string{true: "?view=cards", false: ""}[l.View == "cards"], |
| }) |
| } |
diff --git a/internal/fichas/index.go b/internal/fichas/index.go| new file mode 100644 |
| index 0000000..fbe9590 |
| --- /dev/null |
| +++ b/internal/fichas/index.go |
| @@ -0,0 +1,481 @@ |
| +package fichas |
| + |
| +import ( |
| + "context" |
| + "fmt" |
| + "sort" |
| + "strconv" |
| + "strings" |
| + "sync" |
| + |
| + "amadan.net/oficina/fichas/internal/fichas/seal" |
| + "gorm.io/gorm" |
| +) |
| + |
| +// Index is the in-memory, decrypted view of every live card, which search, |
| +// sort and filter run over. |
| +// |
| +// §4.1: sealed is not unqueryable. The durable file cannot answer a search — |
| +// every value in it is ciphertext — so the values come back into memory to |
| +// be searched, exactly as memoria's index lives in memory and is rebuilt from |
| +// sealed rows on wake. Paul, 2026-09-29: "any search we ever do across |
| +// records needs to feel instant, even when searching 1000s of rows." Over |
| +// plain Go values it is well under a millisecond at that size; decrypting |
| +// every card per request, which is what the list page did before this, was |
| +// tens of milliseconds and grew with every card. |
| +// |
| +// **The event log is its change feed.** Every write in store.go lands an |
| +// Event in the same transaction as the change (the ruling that a change that |
| +// is not logged did not happen), so the highest event id is a complete, |
| +// cheap answer to "has anything changed since I last looked", and the events |
| +// past it say exactly what. No write path has to remember to tell the index |
| +// anything, which is the property that keeps it from going quietly stale. |
| +// |
| +// It holds decrypted content in RAM for the life of the process. That is |
| +// memoria §3.2's "from a live host" entry, made here for the same reason: |
| +// the alternative is not a safer index, it is no search at all. |
| +type Index struct { |
| + db *gorm.DB |
| + s *seal.Sealer |
| + |
| + mu sync.RWMutex |
| + loaded bool |
| + seen int64 // highest event id reflected in the index |
| + types map[int64]*typeIndex |
| +} |
| + |
| +// typeIndex is one card type's cards and definitions. |
| +type typeIndex struct { |
| + ct CardType |
| + label string |
| + plural string |
| + defs []FieldDef // by ordinal |
| + blank []FieldView |
| + cards map[int64]*indexedCard |
| +} |
| + |
| +type indexedCard struct { |
| + view CardView |
| + search string // every value and the latest entry, lowercased, for matching |
| + // lowered holds each field's value lowercased, computed once at load so |
| + // a sort compares prepared strings instead of lowercasing both sides on |
| + // every comparison — which was most of a 15ms sort at 5,000 cards. |
| + lowered map[string]string |
| +} |
| + |
| +// NewIndex returns an empty index; it loads on first use. |
| +func NewIndex(db *gorm.DB, s *seal.Sealer) *Index { |
| + return &Index{db: db, s: s, types: map[int64]*typeIndex{}} |
| +} |
| + |
| +// Sync brings the index up to date with the database. |
| +// |
| +// Cheap when nothing has changed: one MAX over the events primary key. |
| +func (ix *Index) Sync(ctx context.Context) error { |
| + g := ix.db.WithContext(ctx) |
| + var latest int64 |
| + if err := g.Model(&Event{}).Select("COALESCE(MAX(id), 0)").Scan(&latest).Error; err != nil { |
| + return fmt.Errorf("fichas: reading the event log head: %w", err) |
| + } |
| + |
| + ix.mu.RLock() |
| + current := ix.loaded && latest <= ix.seen |
| + ix.mu.RUnlock() |
| + if current { |
| + return nil |
| + } |
| + |
| + ix.mu.Lock() |
| + defer ix.mu.Unlock() |
| + if ix.loaded && latest <= ix.seen { |
| + return nil // another request caught it up while this one waited |
| + } |
| + if !ix.loaded { |
| + // seen is read BEFORE loading, so a write that lands during the |
| + // load is replayed on the next Sync rather than missed. Replaying |
| + // a card that is already current is harmless. |
| + if err := ix.loadAll(g); err != nil { |
| + return err |
| + } |
| + ix.seen, ix.loaded = latest, true |
| + return nil |
| + } |
| + |
| + var events []Event |
| + if err := g.Where("id > ?", ix.seen).Order("id").Find(&events).Error; err != nil { |
| + return fmt.Errorf("fichas: reading new events: %w", err) |
| + } |
| + cards := map[int64]bool{} |
| + for _, e := range events { |
| + switch e.ObjectKind { |
| + case ObjectCard: |
| + cards[e.ObjectID] = true |
| + case ObjectEntry: |
| + // The card-level entry_added event carries the card; the |
| + // entry's own event adds nothing the index needs. |
| + default: |
| + // A type or field changed: labels, options or the field |
| + // set itself. Rare, so rebuild rather than patch. |
| + if err := ix.loadAll(g); err != nil { |
| + return err |
| + } |
| + ix.seen = latest |
| + return nil |
| + } |
| + } |
| + for id := range cards { |
| + if err := ix.reloadCard(g, id); err != nil { |
| + return err |
| + } |
| + } |
| + ix.seen = latest |
| + return nil |
| +} |
| + |
| +func (ix *Index) loadAll(g *gorm.DB) error { |
| + var types []CardType |
| + if err := g.Where("scope = ?", ScopeCard).Order("id").Find(&types).Error; err != nil { |
| + return fmt.Errorf("fichas: loading card types: %w", err) |
| + } |
| + fresh := map[int64]*typeIndex{} |
| + for _, ct := range types { |
| + ti, err := ix.loadType(g, ct) |
| + if err != nil { |
| + return err |
| + } |
| + fresh[ct.ID] = ti |
| + } |
| + ix.types = fresh |
| + return nil |
| +} |
| + |
| +func (ix *Index) loadType(g *gorm.DB, ct CardType) (*typeIndex, error) { |
| + label, err := openLabel(ix.s, ct.SealedLabel) |
| + if err != nil { |
| + return nil, err |
| + } |
| + plural, err := openLabel(ix.s, ct.SealedPlural) |
| + if err != nil { |
| + return nil, err |
| + } |
| + if plural == "" { |
| + plural = pluralLabel(label) |
| + } |
| + var defs []FieldDef |
| + if err := g.Where("card_type_id = ?", ct.ID).Find(&defs).Error; err != nil { |
| + return nil, fmt.Errorf("fichas: loading field definitions: %w", err) |
| + } |
| + sort.Slice(defs, func(i, j int) bool { return defs[i].Ordinal < defs[j].Ordinal }) |
| + blank, err := fieldViews(ix.s, append([]FieldDef(nil), defs...), nil) |
| + if err != nil { |
| + return nil, err |
| + } |
| + ti := &typeIndex{ct: ct, label: label, plural: plural, defs: defs, blank: blank, cards: map[int64]*indexedCard{}} |
| + |
| + var cards []Card |
| + if err := g.Where("card_type_id = ? AND gone = ?", ct.ID, false).Find(&cards).Error; err != nil { |
| + return nil, fmt.Errorf("fichas: loading cards: %w", err) |
| + } |
| + if len(cards) == 0 { |
| + return ti, nil |
| + } |
| + ids := make([]int64, len(cards)) |
| + for i, c := range cards { |
| + ids[i] = c.ID |
| + } |
| + values, err := valuesByCard(g, ids) |
| + if err != nil { |
| + return nil, err |
| + } |
| + latest, err := latestEntries(g, ix.s, ids) |
| + if err != nil { |
| + return nil, err |
| + } |
| + for _, c := range cards { |
| + ic, err := ix.build(ti, c, values[c.ID], latest[c.ID]) |
| + if err != nil { |
| + return nil, err |
| + } |
| + ti.cards[c.ID] = ic |
| + } |
| + return ti, nil |
| +} |
| + |
| +// reloadCard refreshes one card, or drops it if it is gone. |
| +func (ix *Index) reloadCard(g *gorm.DB, id int64) error { |
| + var c Card |
| + err := g.First(&c, id).Error |
| + if err == gorm.ErrRecordNotFound { |
| + for _, ti := range ix.types { |
| + delete(ti.cards, id) |
| + } |
| + return nil |
| + } |
| + if err != nil { |
| + return fmt.Errorf("fichas: reloading card %d: %w", id, err) |
| + } |
| + ti, ok := ix.types[c.CardTypeID] |
| + if !ok { |
| + return nil // a type the index has not loaded: the next full load has it |
| + } |
| + if c.Gone { |
| + delete(ti.cards, id) |
| + return nil |
| + } |
| + values, err := valuesByCard(g, []int64{id}) |
| + if err != nil { |
| + return err |
| + } |
| + latest, err := latestEntries(g, ix.s, []int64{id}) |
| + if err != nil { |
| + return err |
| + } |
| + ic, err := ix.build(ti, c, values[id], latest[id]) |
| + if err != nil { |
| + return err |
| + } |
| + ti.cards[id] = ic |
| + return nil |
| +} |
| + |
| +func (ix *Index) build(ti *typeIndex, c Card, values map[int64][]byte, latest *EntryView) (*indexedCard, error) { |
| + fields, err := fieldViews(ix.s, append([]FieldDef(nil), ti.defs...), values) |
| + if err != nil { |
| + return nil, err |
| + } |
| + view := CardView{Card: c, TypeKey: ti.ct.Key, TypeLabel: ti.label, Fields: fields, Latest: latest} |
| + var b strings.Builder |
| + lowered := make(map[string]string, len(fields)) |
| + for _, f := range fields { |
| + if f.Value != "" { |
| + l := strings.ToLower(f.Value) |
| + lowered[f.Def.Key] = l |
| + b.WriteString(l) |
| + b.WriteByte('\n') |
| + } |
| + } |
| + if latest != nil { |
| + b.WriteString(strings.ToLower(latest.Body())) |
| + } |
| + return &indexedCard{view: view, search: b.String(), lowered: lowered}, nil |
| +} |
| + |
| +func valuesByCard(g *gorm.DB, ids []int64) (map[int64]map[int64][]byte, error) { |
| + var values []CardValue |
| + if err := g.Where("card_id IN ?", ids).Find(&values).Error; err != nil { |
| + return nil, fmt.Errorf("fichas: loading values: %w", err) |
| + } |
| + out := map[int64]map[int64][]byte{} |
| + for _, v := range values { |
| + if out[v.CardID] == nil { |
| + out[v.CardID] = map[int64][]byte{} |
| + } |
| + out[v.CardID][v.FieldDefID] = v.SealedValue |
| + } |
| + return out, nil |
| +} |
| + |
| +// Sort keys that are not field keys. |
| +const ( |
| + SortUpdated = "_updated" |
| + SortLastCall = "_last_call" |
| +) |
| + |
| +// Query is a search over one card type. |
| +type Query struct { |
| + Text string |
| + Filters map[string]string // choice field key → exact value |
| + Sort string // a field key, SortUpdated or SortLastCall |
| + Desc bool |
| + Page int // 1-based |
| + PerPage int |
| +} |
| + |
| +// Result is one page of a Query. |
| +type Result struct { |
| + Type CardType |
| + Label string |
| + Plural string |
| + Fields []FieldView // the type's fields, blank, for headers and filters |
| + Cards []CardView |
| + Total int // matches, across all pages |
| + All int // every live card of the type, unfiltered |
| + Page, Pages int |
| + First, Last int // 1-based positions of this page's first and last row |
| +} |
| + |
| +// Query answers a search from memory. Call Sync first. |
| +func (ix *Index) Query(typeKey string, q Query) (Result, error) { |
| + ix.mu.RLock() |
| + defer ix.mu.RUnlock() |
| + |
| + var ti *typeIndex |
| + for _, t := range ix.types { |
| + if t.ct.Key == typeKey { |
| + ti = t |
| + } |
| + } |
| + if ti == nil { |
| + return Result{}, gorm.ErrRecordNotFound |
| + } |
| + |
| + terms := strings.Fields(strings.ToLower(q.Text)) |
| + choice := map[string]bool{} |
| + for _, d := range ti.defs { |
| + if d.Kind == KindEnum { |
| + choice[d.Key] = true |
| + } |
| + } |
| + |
| + matched := make([]*indexedCard, 0, len(ti.cards)) |
| + for _, ic := range ti.cards { |
| + if !matchesTerms(ic.search, terms) || !matchesFilters(ic.view, q.Filters, choice) { |
| + continue |
| + } |
| + matched = append(matched, ic) |
| + } |
| + sortCards(matched, ti, q) |
| + |
| + res := Result{ |
| + Type: ti.ct, Label: ti.label, Plural: ti.plural, Fields: ti.blank, |
| + Total: len(matched), All: len(ti.cards), |
| + } |
| + per := q.PerPage |
| + if per <= 0 { |
| + per = 100 |
| + } |
| + res.Pages = (len(matched) + per - 1) / per |
| + if res.Pages == 0 { |
| + res.Pages = 1 |
| + } |
| + res.Page = q.Page |
| + if res.Page < 1 { |
| + res.Page = 1 |
| + } |
| + if res.Page > res.Pages { |
| + res.Page = res.Pages |
| + } |
| + start := (res.Page - 1) * per |
| + end := start + per |
| + if end > len(matched) { |
| + end = len(matched) |
| + } |
| + for _, ic := range matched[start:end] { |
| + res.Cards = append(res.Cards, ic.view) |
| + } |
| + if end > start { |
| + res.First, res.Last = start+1, end |
| + } |
| + return res, nil |
| +} |
| + |
| +// matchesTerms: every term must appear somewhere. "bright ana" finds the card |
| +// whose name has one and whose email has the other. |
| +func matchesTerms(hay string, terms []string) bool { |
| + for _, t := range terms { |
| + if !strings.Contains(hay, t) { |
| + return false |
| + } |
| + } |
| + return true |
| +} |
| + |
| +// matchesFilters applies equality filters on CHOICE fields only. A filter on |
| +// any other key is ignored rather than honoured: a free-text equality filter |
| +// in a URL is a search that silently misses on case and spacing. |
| +func matchesFilters(v CardView, filters map[string]string, choice map[string]bool) bool { |
| + for key, want := range filters { |
| + if !choice[key] || want == "" { |
| + continue |
| + } |
| + if v.Field(key).Value != want { |
| + return false |
| + } |
| + } |
| + return true |
| +} |
| + |
| +func sortCards(cards []*indexedCard, ti *typeIndex, q Query) { |
| + key := q.Sort |
| + kind := "" |
| + for _, d := range ti.defs { |
| + if d.Key == key { |
| + kind = d.Kind |
| + } |
| + } |
| + if key != SortLastCall && kind == "" { |
| + key = SortUpdated |
| + } |
| + desc := q.Desc |
| + if q.Sort == "" { |
| + desc = true // the default: most recently touched first |
| + } |
| + |
| + // The comparator touches only prepared data — the lowered map and the |
| + // card's timestamps — never Field(), which scans a card's fields, and |
| + // never a copy of the whole view. At 5,000 cards it runs ~60,000 times. |
| + sort.SliceStable(cards, func(i, j int) bool { |
| + ci, cj := cards[i], cards[j] |
| + idTie := ci.view.Card.ID < cj.view.Card.ID |
| + switch key { |
| + case SortUpdated: |
| + a, b := ci.view.Card.UpdatedAt.UnixNano(), cj.view.Card.UpdatedAt.UnixNano() |
| + return less(a < b, b < a, desc, idTie) |
| + case SortLastCall: |
| + at, bt := int64(0), int64(0) |
| + if ci.view.Latest != nil { |
| + at = ci.view.Latest.Entry.CreatedAt.UnixNano() |
| + } |
| + if cj.view.Latest != nil { |
| + bt = cj.view.Latest.Entry.CreatedAt.UnixNano() |
| + } |
| + // Never-called sorts last in either direction. |
| + if (at == 0) != (bt == 0) { |
| + return at != 0 |
| + } |
| + return less(at < bt, bt < at, desc, idTie) |
| + } |
| + av, aok := ci.lowered[key] |
| + bv, bok := cj.lowered[key] |
| + // An empty value sorts last in either direction: a column sorted |
| + // by next action should lead with the next action, not with the |
| + // cards that have none. |
| + if aok != bok { |
| + return aok |
| + } |
| + if kind == KindNumber { |
| + af, aerr := strconv.ParseFloat(av, 64) |
| + bf, berr := strconv.ParseFloat(bv, 64) |
| + if aerr == nil && berr == nil { |
| + return less(af < bf, bf < af, desc, idTie) |
| + } |
| + } |
| + return less(av < bv, bv < av, desc, idTie) |
| + }) |
| +} |
| + |
| +// less orders by lt/gt in the requested direction, falling back to id so the |
| +// order is total and paging is stable. |
| +func less(lt, gt, desc, idTie bool) bool { |
| + switch { |
| + case lt: |
| + return !desc |
| + case gt: |
| + return desc |
| + default: |
| + return idTie |
| + } |
| +} |
| + |
| +// Fields returns a type's field definitions, blank, for parsing a list URL. |
| +// Call Sync first. |
| +func (ix *Index) Fields(typeKey string) ([]FieldView, error) { |
| + ix.mu.RLock() |
| + defer ix.mu.RUnlock() |
| + for _, t := range ix.types { |
| + if t.ct.Key == typeKey { |
| + return t.blank, nil |
| + } |
| + } |
| + return nil, gorm.ErrRecordNotFound |
| +} |
diff --git a/internal/fichas/index_test.go b/internal/fichas/index_test.go| new file mode 100644 |
| index 0000000..4a8d816 |
| --- /dev/null |
| +++ b/internal/fichas/index_test.go |
| @@ -0,0 +1,182 @@ |
| +package fichas |
| + |
| +import ( |
| + "fmt" |
| + "testing" |
| + "time" |
| +) |
| + |
| +func names(r Result) []string { |
| + var out []string |
| + for _, c := range r.Cards { |
| + out = append(out, c.Title()) |
| + } |
| + return out |
| +} |
| + |
| +func mustQuery(t *testing.T, ix *Index, q Query) Result { |
| + t.Helper() |
| + if err := ix.Sync(ctx()); err != nil { |
| + t.Fatalf("Sync: %v", err) |
| + } |
| + r, err := ix.Query(TypeCustomer, q) |
| + if err != nil { |
| + t.Fatalf("Query: %v", err) |
| + } |
| + return r |
| +} |
| + |
| +// The index learns about every kind of write from the event log alone — |
| +// nothing in store.go tells it anything. |
| +func TestTheIndexFollowsTheEventLog(t *testing.T) { |
| + g, s := seeded(t) |
| + ix := NewIndex(g, s) |
| + if r := mustQuery(t, ix, Query{}); r.Total != 0 { |
| + t.Fatalf("a fresh instance has %d cards", r.Total) |
| + } |
| + |
| + card, _ := CreateCard(ctx(), g, s, TypeCustomer, Person(1), map[string]string{FieldName: "Cnoc Teoranta"}) |
| + if r := mustQuery(t, ix, Query{}); r.Total != 1 { |
| + t.Fatalf("a created card is not in the index (total %d)", r.Total) |
| + } |
| + |
| + UpdateCard(ctx(), g, s, card.ID, Person(1), map[string]string{FieldName: "Cnoc Teoranta Group"}) |
| + if got := names(mustQuery(t, ix, Query{})); len(got) != 1 || got[0] != "Cnoc Teoranta Group" { |
| + t.Fatalf("an update is not in the index: %v", got) |
| + } |
| + |
| + AddEntry(ctx(), g, s, card.ID, Person(1), TypeCallLog, "manual", map[string]string{FieldBody: "talked about the renewal"}) |
| + r := mustQuery(t, ix, Query{Text: "renewal"}) |
| + if r.Total != 1 || r.Cards[0].Latest == nil { |
| + t.Fatalf("a logged call is not searchable (total %d)", r.Total) |
| + } |
| + |
| + DeleteCard(ctx(), g, s, card.ID, Person(1)) |
| + if r := mustQuery(t, ix, Query{}); r.Total != 0 { |
| + t.Fatalf("a deleted card is still in the index") |
| + } |
| +} |
| + |
| +func seedMany(t *testing.T, n int) (*Index, func()) { |
| + t.Helper() |
| + g, s := seeded(t) |
| + stages := []string{"Lead", "Contacted", "Proposal", "Won", "Lost"} |
| + for i := 0; i < n; i++ { |
| + v := map[string]string{ |
| + FieldName: fmt.Sprintf("Company %04d", i), |
| + FieldEmail: fmt.Sprintf("hello%04d@example.com", i), |
| + FieldStage: stages[i%len(stages)], |
| + } |
| + if i%3 == 0 { |
| + v[FieldNextAction] = time.Date(2026, 10, 1+i%28, 0, 0, 0, 0, time.UTC).Format("2006-01-02") |
| + } |
| + if _, err := CreateCard(ctx(), g, s, TypeCustomer, Person(1), v); err != nil { |
| + t.Fatalf("seeding card %d: %v", i, err) |
| + } |
| + } |
| + return NewIndex(g, s), func() {} |
| +} |
| + |
| +func TestSearchNeedsEveryTerm(t *testing.T) { |
| + ix, _ := seedMany(t, 30) |
| + r := mustQuery(t, ix, Query{Text: "company 0012"}) |
| + if r.Total != 1 || r.Cards[0].Title() != "Company 0012" { |
| + t.Fatalf("two-term search found %v", names(r)) |
| + } |
| + if r := mustQuery(t, ix, Query{Text: "COMPANY"}); r.Total != 30 { |
| + t.Fatalf("search is case-sensitive: %d of 30", r.Total) |
| + } |
| + if r := mustQuery(t, ix, Query{Text: "company nosuchthing"}); r.Total != 0 { |
| + t.Fatalf("a term that matches nothing still matched %d", r.Total) |
| + } |
| +} |
| + |
| +func TestFiltersApplyToChoiceFieldsOnly(t *testing.T) { |
| + ix, _ := seedMany(t, 30) |
| + r := mustQuery(t, ix, Query{Filters: map[string]string{FieldStage: "Won"}}) |
| + if r.Total != 6 { |
| + t.Fatalf("Stage = Won matched %d, want 6", r.Total) |
| + } |
| + for _, c := range r.Cards { |
| + if c.Field(FieldStage).Value != "Won" { |
| + t.Fatalf("a filtered result has stage %q", c.Field(FieldStage).Value) |
| + } |
| + } |
| + // Equality on a free-text field would silently miss on case and |
| + // spacing, so it is ignored rather than honoured. |
| + if r := mustQuery(t, ix, Query{Filters: map[string]string{FieldName: "company 0001"}}); r.Total != 30 { |
| + t.Fatalf("a filter on a text field was applied (%d results)", r.Total) |
| + } |
| +} |
| + |
| +func TestSortingPutsEmptyValuesLast(t *testing.T) { |
| + ix, _ := seedMany(t, 30) |
| + for _, desc := range []bool{false, true} { |
| + r := mustQuery(t, ix, Query{Sort: FieldNextAction, Desc: desc}) |
| + seenEmpty := false |
| + prev := "" |
| + for _, c := range r.Cards { |
| + v := c.Field(FieldNextAction).Value |
| + if v == "" { |
| + seenEmpty = true |
| + continue |
| + } |
| + if seenEmpty { |
| + t.Fatalf("desc=%v: a card with a next action sorted after one without", desc) |
| + } |
| + if prev != "" && ((!desc && v < prev) || (desc && v > prev)) { |
| + t.Fatalf("desc=%v: %s came after %s", desc, v, prev) |
| + } |
| + prev = v |
| + } |
| + } |
| + r := mustQuery(t, ix, Query{Sort: FieldName}) |
| + if r.Cards[0].Title() != "Company 0000" || r.Cards[len(r.Cards)-1].Title() != "Company 0029" { |
| + t.Fatalf("name sort is %v…", names(r)[:3]) |
| + } |
| +} |
| + |
| +func TestPaging(t *testing.T) { |
| + ix, _ := seedMany(t, 250) |
| + r := mustQuery(t, ix, Query{Sort: FieldName, Page: 3, PerPage: 100}) |
| + if r.Pages != 3 || r.Page != 3 || len(r.Cards) != 50 || r.First != 201 || r.Last != 250 { |
| + t.Fatalf("page 3 of 250 at 100: pages=%d page=%d rows=%d first=%d last=%d", |
| + r.Pages, r.Page, len(r.Cards), r.First, r.Last) |
| + } |
| + if r := mustQuery(t, ix, Query{Page: 99}); r.Page != 3 { |
| + t.Fatalf("a page past the end is %d, want clamped to 3", r.Page) |
| + } |
| +} |
| + |
| +// BenchmarkSearch is the "instant at thousands of rows" claim, measured. |
| +// |
| +// go test ./internal/fichas -run xxx -bench Search -benchtime 200x |
| +func BenchmarkSearch(b *testing.B) { |
| + b.StopTimer() |
| + g, s := seeded(b) |
| + stages := []string{"Lead", "Contacted", "Proposal", "Won", "Lost"} |
| + for i := 0; i < 5000; i++ { |
| + CreateCard(ctx(), g, s, TypeCustomer, Person(1), map[string]string{ |
| + FieldName: fmt.Sprintf("Company %04d", i), |
| + FieldEmail: fmt.Sprintf("hello%04d@example.com", i), |
| + FieldStage: stages[i%5], |
| + FieldNotes: "Renewal is tied to their grant cycle; the budget holder signs off in March.", |
| + }) |
| + } |
| + ix := NewIndex(g, s) |
| + start := time.Now() |
| + if err := ix.Sync(ctx()); err != nil { |
| + b.Fatal(err) |
| + } |
| + b.ReportMetric(float64(time.Since(start).Milliseconds()), "ms-cold-load") |
| + b.StartTimer() |
| + for i := 0; i < b.N; i++ { |
| + if err := ix.Sync(ctx()); err != nil { |
| + b.Fatal(err) |
| + } |
| + r, err := ix.Query(TypeCustomer, Query{Text: "renewal 042", Filters: map[string]string{FieldStage: "Proposal"}, Sort: FieldName, PerPage: 100}) |
| + if err != nil || r.All != 5000 { |
| + b.Fatalf("query: %v (all=%d)", err, r.All) |
| + } |
| + } |
| +} |
diff --git a/internal/fichas/list.go b/internal/fichas/list.go| new file mode 100644 |
| index 0000000..82c0ddd |
| --- /dev/null |
| +++ b/internal/fichas/list.go |
| @@ -0,0 +1,259 @@ |
| +package fichas |
| + |
| +import ( |
| + "fmt" |
| + "net/http" |
| + "net/url" |
| + "sort" |
| + "strconv" |
| +) |
| + |
| +// listState is everything about a list page that lives in its URL: the |
| +// view, the search, the filters, the sort and the page. It lives in the URL |
| +// so that every state is a link — bookmarkable, shareable, and working with |
| +// scripts off, where the search box is an ordinary GET form. |
| +type listState struct { |
| + base string |
| + View string |
| + Q string |
| + Sort string |
| + Desc bool |
| + Filters map[string]string |
| + Page int |
| +} |
| + |
| +// parseList reads a list URL. Filters are read only for the type's choice |
| +// fields, so an arbitrary query parameter cannot become a filter. |
| +func parseList(r *http.Request, base string, fields []FieldView) listState { |
| + q := r.URL.Query() |
| + l := listState{base: base, View: q.Get("view"), Q: q.Get("q"), Sort: q.Get("sort"), Desc: q.Get("dir") == "desc", Filters: map[string]string{}} |
| + if l.View != "cards" { |
| + l.View = "table" |
| + } |
| + l.Page, _ = strconv.Atoi(q.Get("page")) |
| + for _, f := range fields { |
| + if f.Def.Kind == KindEnum { |
| + if v := q.Get(f.Def.Key); v != "" { |
| + l.Filters[f.Def.Key] = v |
| + } |
| + } |
| + } |
| + return l |
| +} |
| + |
| +func (l listState) query() Query { |
| + return Query{Text: l.Q, Filters: l.Filters, Sort: l.Sort, Desc: l.Desc, Page: l.Page, PerPage: 100} |
| +} |
| + |
| +// href renders a state as a URL. Parameters are written in a fixed order so |
| +// the same state is always the same link. |
| +func (l listState) href() string { |
| + v := url.Values{} |
| + if l.View == "cards" { |
| + v.Set("view", "cards") |
| + } |
| + if l.Q != "" { |
| + v.Set("q", l.Q) |
| + } |
| + keys := make([]string, 0, len(l.Filters)) |
| + for k := range l.Filters { |
| + keys = append(keys, k) |
| + } |
| + sort.Strings(keys) |
| + for _, k := range keys { |
| + v.Set(k, l.Filters[k]) |
| + } |
| + if l.Sort != "" { |
| + v.Set("sort", l.Sort) |
| + if l.Desc { |
| + v.Set("dir", "desc") |
| + } |
| + } |
| + if l.Page > 1 { |
| + v.Set("page", strconv.Itoa(l.Page)) |
| + } |
| + if len(v) == 0 { |
| + return l.base |
| + } |
| + return l.base + "?" + v.Encode() |
| +} |
| + |
| +// with returns a copy after a change. Any change except paging itself goes |
| +// back to page 1 — a new search on page 7 of the old one is a page of |
| +// nothing. |
| +func (l listState) with(change func(*listState)) listState { |
| + c := l |
| + c.Filters = map[string]string{} |
| + for k, v := range l.Filters { |
| + c.Filters[k] = v |
| + } |
| + c.Page = 0 |
| + change(&c) |
| + return c |
| +} |
| + |
| +// hidden is the state a search form must carry through, so typing a search |
| +// does not throw away the filters and sort. |
| +func (l listState) hidden() [][2]string { |
| + var out [][2]string |
| + if l.View == "cards" { |
| + out = append(out, [2]string{"view", "cards"}) |
| + } |
| + keys := make([]string, 0, len(l.Filters)) |
| + for k := range l.Filters { |
| + keys = append(keys, k) |
| + } |
| + sort.Strings(keys) |
| + for _, k := range keys { |
| + out = append(out, [2]string{k, l.Filters[k]}) |
| + } |
| + if l.Sort != "" { |
| + out = append(out, [2]string{"sort", l.Sort}) |
| + if l.Desc { |
| + out = append(out, [2]string{"dir", "desc"}) |
| + } |
| + } |
| + return out |
| +} |
| + |
| +// column is one sortable table header. |
| +type column struct { |
| + Label string |
| + Href string |
| + Sort string // "ascending", "descending" or "" |
| +} |
| + |
| +// columns are the table's headers: every field that is not long text, then |
| +// the latest call. Clicking a header sorts by it; clicking it again flips. |
| +func (l listState) columns(fields []FieldView) []column { |
| + var out []column |
| + add := func(label, key string) { |
| + c := column{Label: label} |
| + first := key == SortLastCall // most recent call first is the useful default |
| + if l.Sort == key { |
| + if l.Desc { |
| + c.Sort = "descending" |
| + } else { |
| + c.Sort = "ascending" |
| + } |
| + } |
| + c.Href = l.with(func(s *listState) { |
| + s.Sort = key |
| + if l.Sort == key { |
| + s.Desc = !l.Desc |
| + } else { |
| + s.Desc = first |
| + } |
| + }).href() |
| + out = append(out, c) |
| + } |
| + for _, f := range fields { |
| + if !isLong(f.Def) { |
| + add(f.Label, f.Def.Key) |
| + } |
| + } |
| + add("Last call", SortLastCall) |
| + return out |
| +} |
| + |
| +// filterMenus is one dropdown per choice field, for the ui dropdown partial. |
| +func (l listState) filterMenus(fields []FieldView) []map[string]any { |
| + var out []map[string]any |
| + for _, f := range fields { |
| + if f.Def.Kind != KindEnum { |
| + continue |
| + } |
| + key, current := f.Def.Key, l.Filters[f.Def.Key] |
| + items := []map[string]any{{ |
| + "Label": "All", "Current": current == "", |
| + "Href": l.with(func(s *listState) { delete(s.Filters, key) }).href(), |
| + }} |
| + for _, opt := range f.Options { |
| + opt := opt |
| + items = append(items, map[string]any{ |
| + "Label": opt, "Current": current == opt, |
| + "Href": l.with(func(s *listState) { s.Filters[key] = opt }).href(), |
| + }) |
| + } |
| + label := f.Label + ": All" |
| + if current != "" { |
| + label = f.Label + ": " + current |
| + } |
| + out = append(out, map[string]any{"Label": label, "Aria": "Filter by " + label, "Items": items}) |
| + } |
| + return out |
| +} |
| + |
| +// pages is the ui pagination partial's items: first, last, the current page |
| +// and its neighbours, with gaps between. |
| +func (l listState) pages(res Result) []map[string]any { |
| + if res.Pages <= 1 { |
| + return nil |
| + } |
| + at := func(p int) string { c := l; c.Page = p; return c.href() } |
| + var out []map[string]any |
| + if res.Page > 1 { |
| + out = append(out, map[string]any{"Label": "Previous", "Href": at(res.Page - 1)}) |
| + } else { |
| + out = append(out, map[string]any{"Label": "Previous", "Disabled": true}) |
| + } |
| + last := 0 |
| + for p := 1; p <= res.Pages; p++ { |
| + if p != 1 && p != res.Pages && (p < res.Page-1 || p > res.Page+1) { |
| + continue |
| + } |
| + if last != 0 && p > last+1 { |
| + out = append(out, map[string]any{"Gap": true}) |
| + } |
| + if p == res.Page { |
| + out = append(out, map[string]any{"Label": strconv.Itoa(p), "Current": true}) |
| + } else { |
| + out = append(out, map[string]any{"Label": strconv.Itoa(p), "Href": at(p)}) |
| + } |
| + last = p |
| + } |
| + if res.Page < res.Pages { |
| + out = append(out, map[string]any{"Label": "Next", "Href": at(res.Page + 1)}) |
| + } else { |
| + out = append(out, map[string]any{"Label": "Next", "Disabled": true}) |
| + } |
| + return out |
| +} |
| + |
| +// countLine says what the list is showing, in words. |
| +func countLine(res Result, filtered bool) string { |
| + noun := res.Plural |
| + switch { |
| + case res.All == 0: |
| + return "" |
| + case filtered && res.Total == 0: |
| + return fmt.Sprintf("No %s match. %s in all.", lower(noun), commas(res.All)) |
| + case filtered && res.Pages > 1: |
| + return fmt.Sprintf("%s–%s of %s matching %s (%s in all)", commas(res.First), commas(res.Last), commas(res.Total), lower(noun), commas(res.All)) |
| + case filtered: |
| + return fmt.Sprintf("%s of %s %s", commas(res.Total), commas(res.All), lower(noun)) |
| + case res.Pages > 1: |
| + return fmt.Sprintf("%s–%s of %s %s", commas(res.First), commas(res.Last), commas(res.All), lower(noun)) |
| + default: |
| + return fmt.Sprintf("%s %s", commas(res.All), lower(noun)) |
| + } |
| +} |
| + |
| +func commas(n int) string { |
| + s := strconv.Itoa(n) |
| + for i := len(s) - 3; i > 0; i -= 3 { |
| + s = s[:i] + "," + s[i:] |
| + } |
| + return s |
| +} |
| + |
| +func lower(s string) string { |
| + if s == "" { |
| + return s |
| + } |
| + b := []rune(s) |
| + if b[0] >= 'A' && b[0] <= 'Z' { |
| + b[0] += 'a' - 'A' |
| + } |
| + return string(b) |
| +} |
diff --git a/internal/fichas/migrations/0003_alter_card_types.sql b/internal/fichas/migrations/0003_alter_card_types.sql| new file mode 100644 |
| index 0000000..dc8dc67 |
| --- /dev/null |
| +++ b/internal/fichas/migrations/0003_alter_card_types.sql |
| @@ -0,0 +1,4 @@ |
| +ALTER TABLE `card_types` ADD `sealed_plural` blob; |
| + |
| +ALTER TABLE `card_types` ADD `face_keys` text; |
| + |
diff --git a/internal/fichas/migrations/schema.sql b/internal/fichas/migrations/schema.sql| index 764b493..6578887 100644 |
| --- a/internal/fichas/migrations/schema.sql |
| +++ b/internal/fichas/migrations/schema.sql |
| @@ -1,7 +1,32 @@ |
| -- Generated by rastrillo migration generate; DO NOT EDIT. |
| -- The schema every migration in this directory adds up to. |
| -CREATE TABLE `card_types` (`id` integer PRIMARY KEY AUTOINCREMENT,`key` text,`scope` text NOT NULL,`sealed_label` blob,`created_at` datetime,`updated_at` datetime); |
| +CREATE TRIGGER cards_are_not_hard_deleted |
| +BEFORE DELETE ON cards |
| +BEGIN |
| + SELECT RAISE(ABORT, 'cards are not deleted: set gone, which is what DeleteCard does'); |
| +END; |
| + |
| +CREATE TRIGGER entries_are_immutable |
| +BEFORE UPDATE ON entries |
| +BEGIN |
| + SELECT RAISE(ABORT, 'entries are immutable: write a new entry that supersedes this one'); |
| +END; |
| + |
| +CREATE TRIGGER entries_are_not_deleted |
| +BEFORE DELETE ON entries |
| +WHEN (SELECT COUNT(*) FROM cards WHERE cards.id = OLD.card_id) > 0 |
| +BEGIN |
| + SELECT RAISE(ABORT, 'entries are not deleted: mark the card gone instead'); |
| +END; |
| + |
| +CREATE TRIGGER entry_values_are_immutable |
| +BEFORE UPDATE ON entry_values |
| +BEGIN |
| + SELECT RAISE(ABORT, 'entry values are immutable: write a new entry that supersedes this one'); |
| +END; |
| + |
| +CREATE TABLE `card_types` (`id` integer PRIMARY KEY AUTOINCREMENT,`key` text,`scope` text NOT NULL,`sealed_label` blob,`created_at` datetime,`updated_at` datetime, `sealed_plural` blob, `face_keys` text); |
| CREATE TABLE `card_values` (`card_id` integer,`field_def_id` integer,`sealed_value` blob,`print` text,PRIMARY KEY (`card_id`,`field_def_id`),CONSTRAINT `fk_cards_values` FOREIGN KEY (`card_id`) REFERENCES `cards`(`id`) ON DELETE CASCADE); |
diff --git a/internal/fichas/models.go b/internal/fichas/models.go| index 75dcf32..b7b4925 100644 |
| --- a/internal/fichas/models.go |
| +++ b/internal/fichas/models.go |
| @@ -115,8 +115,17 @@ type CardType struct { |
| // that one rule covers the catalogue and whatever is added to it — |
| // a label that is sometimes plaintext is a rule nobody can apply. |
| SealedLabel []byte |
| - CreatedAt time.Time |
| - UpdatedAt time.Time |
| + // SealedPlural is what a list of these is called — "Customers". Sealed |
| + // for the same reason the label is: a user-added type's name is the |
| + // user's own words. Empty means "derive it from the label". |
| + SealedPlural []byte |
| + // FaceKeys is the card face — which fields the Cards view shows, in |
| + // order — as a JSON array of FieldDef keys, plus FaceLatestCall for |
| + // the latest entry. Plaintext under §4.1 case 4: it is a list of this |
| + // app's opaque keys, and never a value. Empty means the type's default. |
| + FaceKeys string `gorm:"size:2048"` |
| + CreatedAt time.Time |
| + UpdatedAt time.Time |
| // RESTRICT, not CASCADE: deleting a type out from under live cards |
| // must fail rather than take the cards with it. |
| Fields []FieldDef `gorm:"constraint:OnDelete:RESTRICT"` |
diff --git a/internal/fichas/models_test.go b/internal/fichas/models_test.go| index 286c1ee..ed66d5b 100644 |
| --- a/internal/fichas/models_test.go |
| +++ b/internal/fichas/models_test.go |
| @@ -45,6 +45,7 @@ var plaintextColumns = map[string]string{ |
| "Event.Kind": "routing: created, updated, deleted", |
| "Event.ObjectKind": "routing: card, entry", |
| "Event.ChangedFields": "JSON array of FieldDef ids — opaque, and §5 keeps it permanently", |
| + "CardType.FaceKeys": "the card face: a JSON list of this app's opaque field keys, never a value", |
| "CardType.Scope": "card or entry — this app's vocabulary, and what stops an entry taking a card type", |
| "Event.ActorReason": "why the app acted as itself — our words, never a team's; required for an app actor", |
| } |
diff --git a/internal/fichas/render.go b/internal/fichas/render.go| index bebf9a9..6b492b2 100644 |
| --- a/internal/fichas/render.go |
| +++ b/internal/fichas/render.go |
| @@ -47,6 +47,9 @@ func init() { |
| "ago": ago, |
| "dateAgo": dateAgo, |
| "stageTone": stageTone, |
| + "isLong": isLong, |
| + "lower": lower, |
| + "commas": commas, |
| }). |
| ParseFS(ui.Templates(), "*.html")). |
| ParseFS(appFS, "templates/"+shell+".html", "templates/"+name+".html")) |
diff --git a/internal/fichas/seal/bench_test.go b/internal/fichas/seal/bench_test.go| new file mode 100644 |
| index 0000000..c2f5b66 |
| --- /dev/null |
| +++ b/internal/fichas/seal/bench_test.go |
| @@ -0,0 +1,47 @@ |
| +package seal |
| + |
| +import "testing" |
| + |
| +func BenchmarkOpen(b *testing.B) { |
| + s, _ := New(vectorDEK()) |
| + ct, _ := s.Seal(PurposeValue, []byte("ana@brightwater.example")) |
| + b.ResetTimer() |
| + for i := 0; i < b.N; i++ { |
| + if _, err := s.Open(PurposeValue, ct); err != nil { |
| + b.Fatal(err) |
| + } |
| + } |
| +} |
| + |
| +// The cache must not change a single byte of what is derived or sealed — |
| +// TestPurposeKeyVectors pins the keys; this pins that a value sealed before |
| +// the cache existed still opens, from many goroutines at once. |
| +func TestTheCacheIsSafeUnderConcurrency(t *testing.T) { |
| + s, _ := New(vectorDEK()) |
| + ct, err := s.Seal(PurposeValue, []byte("same")) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + done := make(chan error, 64) |
| + for g := 0; g < 64; g++ { |
| + go func() { |
| + for i := 0; i < 200; i++ { |
| + got, err := s.Open(PurposeValue, ct) |
| + if err != nil || string(got) != "same" { |
| + done <- err |
| + return |
| + } |
| + if _, err := s.Seal(PurposeLabel, []byte("x")); err != nil { |
| + done <- err |
| + return |
| + } |
| + } |
| + done <- nil |
| + }() |
| + } |
| + for g := 0; g < 64; g++ { |
| + if err := <-done; err != nil { |
| + t.Fatal(err) |
| + } |
| + } |
| +} |
diff --git a/internal/fichas/seal/seal.go b/internal/fichas/seal/seal.go| index cf44388..2ef1745 100644 |
| --- a/internal/fichas/seal/seal.go |
| +++ b/internal/fichas/seal/seal.go |
| @@ -32,6 +32,7 @@ import ( |
| "errors" |
| "hash" |
| "io" |
| + "sync" |
| "golang.org/x/crypto/hkdf" |
| ) |
| @@ -82,7 +83,19 @@ const refPrintPurpose = "ref.print" |
| func sha256New() hash.Hash { return sha256.New() } |
| // Sealer holds the data encryption key. One per process. |
| -type Sealer struct{ dek []byte } |
| +// |
| +// It caches each purpose's derived AEAD. Deriving one is an HKDF expand, a |
| +// key schedule and a GCM table — about 1.5µs, which was nearly all of the |
| +// cost of an Open, paid again for every value on every list page. The key |
| +// for a purpose never changes, so it is derived once. A GCM AEAD holds only |
| +// read-only state after construction and is safe for concurrent Seal and |
| +// Open, which is how crypto/tls uses one per direction of a connection. |
| +type Sealer struct { |
| + dek []byte |
| + |
| + mu sync.RWMutex |
| + aeads map[string]cipher.AEAD |
| +} |
| // New returns a Sealer. The DEK must be exactly 32 bytes. |
| func New(dek []byte) (*Sealer, error) { |
| @@ -91,7 +104,7 @@ func New(dek []byte) (*Sealer, error) { |
| } |
| c := make([]byte, 32) |
| copy(c, dek) |
| - return &Sealer{dek: c}, nil |
| + return &Sealer{dek: c, aeads: map[string]cipher.AEAD{}}, nil |
| } |
| // purposeKey derives the AEAD key for purpose via HKDF over the DEK. A |
| @@ -111,6 +124,23 @@ func (s *Sealer) purposeKey(purpose string) ([]byte, error) { |
| } |
| func (s *Sealer) aeadFor(purpose string) (cipher.AEAD, error) { |
| + s.mu.RLock() |
| + a, ok := s.aeads[purpose] |
| + s.mu.RUnlock() |
| + if ok { |
| + return a, nil |
| + } |
| + a, err := s.deriveAEAD(purpose) |
| + if err != nil { |
| + return nil, err |
| + } |
| + s.mu.Lock() |
| + s.aeads[purpose] = a |
| + s.mu.Unlock() |
| + return a, nil |
| +} |
| + |
| +func (s *Sealer) deriveAEAD(purpose string) (cipher.AEAD, error) { |
| key, err := s.purposeKey(purpose) |
| if err != nil { |
| return nil, err |
diff --git a/internal/fichas/static/fichas.css b/internal/fichas/static/fichas.css| index 2fc5bb8..255cf5e 100644 |
| --- a/internal/fichas/static/fichas.css |
| +++ b/internal/fichas/static/fichas.css |
| @@ -224,3 +224,8 @@ |
| /* ---- A form's measure: fields read as a column, not a banner. */ |
| [fichas-form] { max-inline-size: 40rem; } |
| + |
| +/* ---- Sortable headers: the header IS the control, in text colour. */ |
| +[fichas-grid] th a { color: inherit; text-decoration: none; } |
| +[fichas-grid] th a:hover { text-decoration: underline; } |
| +[fichas-grid] th[aria-sort] a { color: var(--rst-text); } |
diff --git a/internal/fichas/static/live.js b/internal/fichas/static/live.js| new file mode 100644 |
| index 0000000..6cf1c29 |
| --- /dev/null |
| +++ b/internal/fichas/static/live.js |
| @@ -0,0 +1,87 @@ |
| +/* live.js — the search box updates the results as you type. |
| + * |
| + * Progressive enhancement over an ordinary GET form: with scripts off the |
| + * form submits and the server renders the same page. With them on, typing |
| + * fetches that same URL and swaps only the results region, so the search |
| + * box keeps focus and the cursor stays where it was. |
| + * |
| + * Why this is fast enough to feel instant: the server answers from an |
| + * in-memory index (index.go) — a search over 5,000 cards is a quarter of a |
| + * millisecond — so the cost is one round trip and a render. Keystrokes are |
| + * debounced so a word typed quickly is one request, and a request that a |
| + * newer keystroke has overtaken is aborted, so a slow answer can never |
| + * overwrite a fresher one. |
| + * |
| + * The URL follows along (history.replaceState), so a live search is still a |
| + * link you can bookmark or share, and Back is not littered with one entry |
| + * per keystroke. |
| + */ |
| +(function () { |
| + "use strict"; |
| + |
| + var DEBOUNCE_MS = 120; |
| + |
| + function init() { |
| + var bar = document.querySelector("[data-live-search]"); |
| + var results = document.querySelector("[data-live-results]"); |
| + if (!bar || !results || !window.fetch || !window.DOMParser) return; |
| + var form = bar.querySelector("form"); |
| + var input = form && form.querySelector('input[name="q"]'); |
| + if (!input) return; |
| + |
| + var timer = 0; |
| + var inflight = null; |
| + |
| + function urlFor() { |
| + var params = new URLSearchParams(new FormData(form)); |
| + if (!params.get("q")) params.delete("q"); |
| + var qs = params.toString(); |
| + return (form.getAttribute("action") || location.pathname) + (qs ? "?" + qs : ""); |
| + } |
| + |
| + function refresh() { |
| + var url = urlFor(); |
| + if (inflight) inflight.abort(); |
| + inflight = new AbortController(); |
| + results.setAttribute("aria-busy", "true"); |
| + fetch(url, { signal: inflight.signal, headers: { Accept: "text/html" }, credentials: "same-origin" }) |
| + .then(function (r) { |
| + if (!r.ok) throw new Error("status " + r.status); |
| + return r.text(); |
| + }) |
| + .then(function (text) { |
| + var doc = new DOMParser().parseFromString(text, "text/html"); |
| + var fresh = doc.querySelector("[data-live-results]"); |
| + if (!fresh) return; |
| + results.innerHTML = fresh.innerHTML; |
| + history.replaceState(null, "", url); |
| + }) |
| + .catch(function (err) { |
| + // An abort is a newer keystroke winning; anything else leaves |
| + // the page as it was and the form still submits normally. |
| + if (err && err.name === "AbortError") return; |
| + }) |
| + .finally(function () { |
| + results.removeAttribute("aria-busy"); |
| + }); |
| + } |
| + |
| + input.addEventListener("input", function () { |
| + clearTimeout(timer); |
| + timer = setTimeout(refresh, DEBOUNCE_MS); |
| + }); |
| + |
| + // Enter does the same thing immediately rather than reloading the page. |
| + form.addEventListener("submit", function (e) { |
| + e.preventDefault(); |
| + clearTimeout(timer); |
| + refresh(); |
| + }); |
| + } |
| + |
| + if (document.readyState === "loading") { |
| + document.addEventListener("DOMContentLoaded", init); |
| + } else { |
| + init(); |
| + } |
| +})(); |
diff --git a/internal/fichas/static/shortcuts.js b/internal/fichas/static/shortcuts.js| new file mode 100644 |
| index 0000000..d9b7064 |
| --- /dev/null |
| +++ b/internal/fichas/static/shortcuts.js |
| @@ -0,0 +1,40 @@ |
| +/* shortcuts.js — ⌘↩ (Ctrl+Enter elsewhere) submits the form a textarea is in. |
| + * |
| + * Progressive enhancement only: every form still submits with its button, |
| + * scripts on or off. requestSubmit rather than submit(), so the browser's own |
| + * validation runs and busy.js sees an ordinary submit event — the same path |
| + * as a click, never a second one. |
| + * |
| + * aria-keyshortcuts announces the shortcut to assistive technology; it is set |
| + * here rather than in the templates because the shortcut only exists when |
| + * this script does. |
| + */ |
| +(function () { |
| + "use strict"; |
| + |
| + function mark(root) { |
| + root.querySelectorAll("form textarea").forEach(function (t) { |
| + if (!t.hasAttribute("aria-keyshortcuts")) { |
| + t.setAttribute("aria-keyshortcuts", "Meta+Enter Control+Enter"); |
| + } |
| + }); |
| + } |
| + |
| + document.addEventListener("keydown", function (e) { |
| + if (e.key !== "Enter" || !(e.metaKey || e.ctrlKey) || e.isComposing) return; |
| + var t = e.target; |
| + if (!t || t.tagName !== "TEXTAREA" || !t.form) return; |
| + e.preventDefault(); |
| + if (typeof t.form.requestSubmit === "function") { |
| + t.form.requestSubmit(); |
| + } else { |
| + t.form.submit(); |
| + } |
| + }); |
| + |
| + if (document.readyState === "loading") { |
| + document.addEventListener("DOMContentLoaded", function () { mark(document); }); |
| + } else { |
| + mark(document); |
| + } |
| +})(); |
diff --git a/internal/fichas/store_test.go b/internal/fichas/store_test.go| index 992f7d7..db6591d 100644 |
| --- a/internal/fichas/store_test.go |
| +++ b/internal/fichas/store_test.go |
| @@ -12,7 +12,7 @@ import ( |
| func ctx() context.Context { return context.Background() } |
| -func seeded(t *testing.T) (*gorm.DB, *seal.Sealer) { |
| +func seeded(t testing.TB) (*gorm.DB, *seal.Sealer) { |
| t.Helper() |
| g, s := testDB(t), testSealer(t) |
| if err := Seed(g, s); err != nil { |
diff --git a/internal/fichas/templates/index.html b/internal/fichas/templates/index.html| index 8a74420..3a13b75 100644 |
| --- a/internal/fichas/templates/index.html |
| +++ b/internal/fichas/templates/index.html |
| @@ -1,57 +1,73 @@ |
| -{{define "title"}}Customers · fichas{{end}} |
| +{{define "title"}}{{.Result.Plural}} · fichas{{end}} |
| {{define "content"}} |
| <header fichas-appbar> |
| - <h1 fichas-appbar-title>Customers <span>{{len .Cards}} {{if eq (len .Cards) 1}}record{{else}}records{{end}}</span></h1> |
| - <div fichas-appbar-actions><a rst-btn="primary" href="/cards/new">{{icon "plus"}} Add a customer</a></div> |
| + <h1 fichas-appbar-title>{{.Result.Plural}}{{if .Result.All}} <span>{{commas .Result.All}} {{if eq .Result.All 1}}record{{else}}records{{end}}</span>{{end}}</h1> |
| + <div fichas-appbar-actions><a rst-btn="primary" href="/cards/new">{{icon "plus"}} Add a {{lower .Result.Label}}</a></div> |
| </header> |
| -{{if .Cards}} |
| +{{if .Result.All}} |
| <div fichas-toolbar> |
| - {{template "seg-tabs" (dict "Label" "View customers as" "Items" (list |
| - (dict "Label" "Table" "Href" "/?view=table" "Current" (eq .View "table")) |
| - (dict "Label" "Cards" "Href" "/?view=cards" "Current" (eq .View "cards"))))}} |
| + {{template "seg-tabs" (dict "Label" (printf "View %s as" .Result.Plural) "Items" (list |
| + (dict "Label" "Table" "Href" .TableHref "Current" (eq .View "table")) |
| + (dict "Label" "Cards" "Href" .CardsHref "Current" (eq .View "cards"))))}} |
| </div> |
| -{{if eq .View "table"}} |
| -<div fichas-grid-wrap> |
| -<table fichas-grid> |
| - <thead><tr> |
| - {{range .Fields}}{{if ne .Def.Key "f_notes"}}<th scope="col">{{.Label}}</th>{{end}}{{end}} |
| - <th scope="col">Last call</th> |
| - </tr></thead> |
| - <tbody> |
| - {{range $c := .Cards}} |
| - <tr> |
| - {{range $f := $.Fields}}{{if ne $f.Def.Key "f_notes"}}{{$v := ($c.Field $f.Def.Key).Value}} |
| - {{if eq $f.Def.Key "f_name"}}<td><a href="/cards/{{$c.Card.ID}}">{{$c.Title}}</a></td> |
| - {{else if not $v}}<td><span fichas-empty-cell>—</span></td> |
| - {{else if eq $f.Def.Key "f_stage"}}<td>{{template "status-pill" (dict "Label" $v "Tone" (stageTone $v))}}</td> |
| - {{else if eq $f.Def.Kind "date"}}<td><time datetime="{{$v}}" title="{{$v}}">{{dateAgo $v}}</time></td> |
| - {{else}}<td>{{$v}}</td>{{end}} |
| - {{end}}{{end}} |
| - <td>{{with $c.Latest}}<time datetime="{{.Entry.CreatedAt.Format "2006-01-02"}}" title="{{.Entry.CreatedAt.Format "2 January 2006, 15:04"}}">{{ago .Entry.CreatedAt}}</time>{{else}}<span fichas-empty-cell>never</span>{{end}}</td> |
| - </tr> |
| - {{end}} |
| - </tbody> |
| -</table> |
| +<div rst-lbar data-live-search> |
| + {{template "list-bar-search" (dict "Action" "/" "Query" .Q "Placeholder" (printf "Search %s" (lower .Result.Plural)) "Hidden" .Hidden "ClearHref" .ClearHref)}} |
| + {{range .Filters}}{{template "dropdown" .}}{{end}} |
| </div> |
| + |
| +<div id="results" data-live-results aria-live="polite"> |
| +{{if .Cards}} |
| + {{if eq .View "table"}} |
| + <div fichas-grid-wrap> |
| + <table fichas-grid> |
| + <thead><tr> |
| + {{range .Columns}}<th scope="col"{{with .Sort}} aria-sort="{{.}}"{{end}}><a href="{{.Href}}">{{.Label}}{{if eq .Sort "ascending"}} <span aria-hidden="true">↑</span>{{else if eq .Sort "descending"}} <span aria-hidden="true">↓</span>{{end}}</a></th>{{end}} |
| + </tr></thead> |
| + <tbody> |
| + {{range $c := .Cards}} |
| + <tr> |
| + {{range $f := $.Fields}}{{if not (isLong $f.Def)}}{{$v := ($c.Field $f.Def.Key).Value}} |
| + {{if eq $f.Def.Key "f_name"}}<td><a href="/cards/{{$c.Card.ID}}">{{$c.Title}}</a></td> |
| + {{else if not $v}}<td><span fichas-empty-cell>—</span></td> |
| + {{else if eq $f.Def.Kind "enum"}}<td>{{template "status-pill" (dict "Label" $v "Tone" (stageTone $v))}}</td> |
| + {{else if eq $f.Def.Kind "date"}}<td><time datetime="{{$v}}" title="{{$v}}">{{dateAgo $v}}</time></td> |
| + {{else}}<td>{{$v}}</td>{{end}} |
| + {{end}}{{end}} |
| + <td>{{with $c.Latest}}<time datetime="{{.Entry.CreatedAt.Format "2006-01-02"}}" title="{{.Entry.CreatedAt.Format "2 January 2006, 15:04"}}">{{ago .Entry.CreatedAt}}</time>{{else}}<span fichas-empty-cell>never</span>{{end}}</td> |
| + </tr> |
| + {{end}} |
| + </tbody> |
| + </table> |
| + </div> |
| + {{else}} |
| + <ul fichas-card-grid> |
| + {{range .Cards}} |
| + <li><article rst-card> |
| + <h2 fichas-card-title><a href="/cards/{{.Card.ID}}">{{.Title}}</a>{{with (.Field "f_stage").Value}}{{template "status-pill" (dict "Label" . "Tone" (stageTone .))}}{{end}}</h2> |
| + {{with (.Field "f_email").Value}}<p fichas-meta>{{.}}</p>{{end}} |
| + {{with .Latest}}{{with .Body}}<p fichas-card-quote>{{.}}</p>{{end}}{{end}} |
| + <p fichas-meta>{{with (.Field "f_next_action").Value}}Next action <time datetime="{{.}}" title="{{.}}">{{dateAgo .}}</time>{{else}}{{with .Latest}}Last call <time title="{{.Entry.CreatedAt.Format "2 January 2006"}}">{{ago .Entry.CreatedAt}}</time>{{else}}No calls yet{{end}}{{end}}</p> |
| + </article></li> |
| + {{end}} |
| + </ul> |
| + {{end}} |
| + <p rst-count-line>{{.Count}}</p> |
| + {{with .Pages}}{{template "pagination" (dict "Items" . "Label" "Pages of results")}}{{end}} |
| {{else}} |
| -<ul fichas-card-grid> |
| -{{range .Cards}} |
| - <li><article rst-card> |
| - <h2 fichas-card-title><a href="/cards/{{.Card.ID}}">{{.Title}}</a>{{with (.Field "f_stage").Value}}{{template "status-pill" (dict "Label" . "Tone" (stageTone .))}}{{end}}</h2> |
| - {{with (.Field "f_email").Value}}<p fichas-meta>{{.}}</p>{{end}} |
| - {{with .Latest}}{{with .Body}}<p fichas-card-quote>{{.}}</p>{{end}}{{end}} |
| - <p fichas-meta>{{with (.Field "f_next_action").Value}}Next action <time datetime="{{.}}" title="{{.}}">{{dateAgo .}}</time>{{else}}{{with .Latest}}Last call <time title="{{.Entry.CreatedAt.Format "2 January 2006"}}">{{ago .Entry.CreatedAt}}</time>{{else}}No calls yet{{end}}{{end}}</p> |
| - </article></li> |
| -{{end}} |
| -</ul> |
| + {{template "empty-state" (dict |
| + "Title" "Nothing matches" |
| + "Body" .Count |
| + "ActionHref" .ResetHref |
| + "ActionLabel" "Clear the search and filters")}} |
| {{end}} |
| +</div> |
| {{else}} |
| {{template "empty-state" (dict |
| - "Title" "No customers yet" |
| + "Title" (printf "No %s yet" (lower .Result.Plural)) |
| "Body" "A customer card holds what you know about one prospect, and every call you log against them. Start with the one you spoke to most recently." |
| "ActionHref" "/cards/new" |
| - "ActionLabel" "Add a customer")}} |
| + "ActionLabel" (printf "Add a %s" (lower .Result.Label)))}} |
| {{end}} |
| {{end}} |
diff --git a/internal/fichas/templates/shell.html b/internal/fichas/templates/shell.html| index feac1fc..f27585c 100644 |
| --- a/internal/fichas/templates/shell.html |
| +++ b/internal/fichas/templates/shell.html |
| @@ -15,6 +15,8 @@ |
| <link rel="stylesheet" href="{{asset "static/fichas.css"}}"> |
| <script defer src="{{asset "static/rastrillo.js"}}"></script> |
| <script defer src="{{asset "static/busy.js"}}"></script> |
| +<script defer src="{{asset "static/shortcuts.js"}}"></script> |
| +<script defer src="{{asset "static/live.js"}}"></script> |
| {{iconAssets}} |
| <script defer src="{{asset "static/select.js"}}"></script> |
| <script defer src="{{asset "static/datetime.js"}}"></script> |
diff --git a/internal/fichastest/browser_test.go b/internal/fichastest/browser_test.go| index 29f8453..a18490a 100644 |
| --- a/internal/fichastest/browser_test.go |
| +++ b/internal/fichastest/browser_test.go |
| @@ -1,24 +1,31 @@ |
| //go:build browser |
| -// The browser drive: the whole app in a real Chromium, wired exactly |
| -// as cmd/fichas/main.go wires it — loud on any console error, failed |
| -// request, 4xx/5xx, or JS value that leaked to the screen unrendered. |
| -// Not part of the plain suite: |
| +// The browser drive: the whole app in a real Chromium — loud on any console |
| +// error, failed request, 4xx/5xx, or JS value that leaked to the screen |
| +// unrendered. Not part of the plain suite: |
| // |
| // go test -tags browser ./... |
| // |
| -// It needs a Chromium: on PATH, via RASTRILLO_CHROME, or in a |
| -// Playwright cache. A skip is not a pass — with no browser this fails, |
| -// unless RASTRILLO_BROWSER_OPTIONAL is set, which makes the skip a |
| -// deliberate visible choice rather than an accident. |
| +// It needs a Chromium: on PATH, via RASTRILLO_CHROME, or in a Playwright |
| +// cache. A skip is not a pass — with no browser this fails, unless |
| +// RASTRILLO_BROWSER_OPTIONAL is set. |
| +// |
| +// **This file did not compile for most of fichas' life.** It called App with |
| +// the scaffold's original signature, and because the build tag keeps it out |
| +// of `make ci`, nothing noticed. A test behind a tag has to be run on |
| +// purpose, or it rots silently. |
| package fichastest |
| import ( |
| + "context" |
| + "encoding/base64" |
| "io" |
| "log/slog" |
| "net/http" |
| "path/filepath" |
| + "strings" |
| "testing" |
| + "time" |
| "github.com/carlosframework/rastrillo/db" |
| "github.com/carlosframework/rastrillo/harness" |
| @@ -27,27 +34,121 @@ import ( |
| fichas "amadan.net/oficina/fichas/internal/fichas" |
| ) |
| -// TestBrowserWalk is the minimal loud walk: the home screen arrives |
| -// with nothing wrong anywhere the rig watches. Grow it with the app — |
| -// each screen a flow reaches earns a Screen() line, and an expected |
| -// probe (a signed-out 401, say) earns a rig.Allow. |
| -func TestBrowserWalk(t *testing.T) { |
| - rig := harness.New(t, func(origin string) http.Handler { |
| - // The scaffold's own wiring, origin included: the rig hands |
| - // the app its localhost origin before serving — the same seam |
| - // main.go fills from FICHAS_ORIGIN. |
| +// browserApp is the app signed in as TestOwner, with the rig's own origin, |
| +// optionally seeded through the real store. |
| +func browserApp(t *testing.T, seed func(context.Context, *db.DB, fichas.Config)) *harness.Rig { |
| + return harness.New(t, func(origin string) http.Handler { |
| logger := slog.New(slog.NewTextHandler(io.Discard, nil)) |
| d, err := db.Open(filepath.Join(t.TempDir(), "app.db"), logger) |
| if err != nil { |
| t.Fatalf("db.Open: %v", err) |
| } |
| t.Cleanup(func() { d.Close() }) |
| - mux, err := fichas.App(d, origin, logger) |
| + cfg, err := fichas.LoadConfig(func(k string) string { |
| + return map[string]string{ |
| + "FICHAS_DEK": base64.StdEncoding.EncodeToString(testKey()), |
| + "FICHAS_INSTANCE_KEY": "browser", |
| + "FICHAS_OWNERS": TestOwner, |
| + "FICHAS_ORIGIN": origin, |
| + "FICHAS_ALLOW_LOGGED_MAIL": "1", |
| + }[k] |
| + }) |
| + if err != nil { |
| + t.Fatalf("LoadConfig: %v", err) |
| + } |
| + mux, err := fichas.AppForTest(d, cfg, logger, TestOwner) |
| if err != nil { |
| - t.Fatalf("App: %v", err) |
| + t.Fatalf("AppForTest: %v", err) |
| + } |
| + if seed != nil { |
| + seed(context.Background(), d, cfg) |
| } |
| return mux |
| }) |
| +} |
| + |
| +func TestBrowserWalk(t *testing.T) { |
| + rig := browserApp(t, nil) |
| rig.Run(chromedp.Navigate(rig.Origin + "/")) |
| rig.Screen("body", "home") |
| } |
| + |
| +// TestLiveSearch types into the search box and checks the results change |
| +// WITHOUT a page load — the claim live.js makes. A marker on window survives |
| +// only if the page was never reloaded. |
| +func TestLiveSearch(t *testing.T) { |
| + rig := browserApp(t, func(ctx context.Context, d *db.DB, cfg fichas.Config) { |
| + m, err := fichas.EnsureMember(ctx, d.G, cfg.Sealer, TestOwner) |
| + if err != nil { |
| + t.Fatalf("member: %v", err) |
| + } |
| + for _, name := range []string{"Brightwater Labs", "Halden Robotics", "Oriel Housing Trust"} { |
| + if _, err := fichas.CreateCard(ctx, d.G, cfg.Sealer, fichas.TypeCustomer, fichas.Person(m.ID), |
| + map[string]string{fichas.FieldName: name}); err != nil { |
| + t.Fatalf("seed: %v", err) |
| + } |
| + } |
| + }) |
| + |
| + var marker bool |
| + var results, location string |
| + rig.Run( |
| + chromedp.Navigate(rig.Origin+"/"), |
| + chromedp.WaitVisible(`input[name="q"]`), |
| + chromedp.Evaluate(`window.__noReload = true`, nil), |
| + chromedp.SendKeys(`input[name="q"]`, "halden"), |
| + chromedp.Sleep(600*time.Millisecond), |
| + chromedp.Text(`[data-live-results]`, &results), |
| + chromedp.Evaluate(`window.__noReload === true`, &marker), |
| + chromedp.Evaluate(`location.search`, &location), |
| + ) |
| + if !marker { |
| + t.Fatal("typing reloaded the page; live.js did not take over the search") |
| + } |
| + if !strings.Contains(results, "Halden Robotics") || strings.Contains(results, "Brightwater Labs") { |
| + t.Fatalf("the results did not narrow as you typed:\n%s", results) |
| + } |
| + if !strings.Contains(location, "q=halden") { |
| + t.Fatalf("the URL did not follow the search (location.search = %q)", location) |
| + } |
| + rig.Screen("body", "live-search") |
| +} |
| + |
| +// TestCmdEnterSubmits: ⌘↩ in a textarea submits its form. |
| +func TestCmdEnterSubmits(t *testing.T) { |
| + var id int64 |
| + rig := browserApp(t, func(ctx context.Context, d *db.DB, cfg fichas.Config) { |
| + m, _ := fichas.EnsureMember(ctx, d.G, cfg.Sealer, TestOwner) |
| + c, err := fichas.CreateCard(ctx, d.G, cfg.Sealer, fichas.TypeCustomer, fichas.Person(m.ID), |
| + map[string]string{fichas.FieldName: "Brightwater Labs"}) |
| + if err != nil { |
| + t.Fatalf("seed: %v", err) |
| + } |
| + id = c.ID |
| + }) |
| + var timeline string |
| + path := rig.Origin + "/cards/" + itoa(id) |
| + rig.Run( |
| + chromedp.Navigate(path), |
| + chromedp.WaitVisible(`textarea[name="f_body"]`), |
| + chromedp.SendKeys(`textarea[name="f_body"]`, "Called about the pilot"), |
| + chromedp.KeyEvent("\r", chromedp.KeyModifiers(4)), // 4 = Meta (⌘) |
| + chromedp.Sleep(800*time.Millisecond), |
| + chromedp.Text(`body`, &timeline), |
| + ) |
| + if !strings.Contains(timeline, "Called about the pilot") { |
| + t.Fatal("⌘↩ in the call-log textarea did not submit it") |
| + } |
| +} |
| + |
| +func itoa(n int64) string { |
| + b := []byte{} |
| + if n == 0 { |
| + return "0" |
| + } |
| + for n > 0 { |
| + b = append([]byte{byte('0' + n%10)}, b...) |
| + n /= 10 |
| + } |
| + return string(b) |
| +} |
diff --git a/internal/fichastest/surface_test.go b/internal/fichastest/surface_test.go| index b31cb85..17de05a 100644 |
| --- a/internal/fichastest/surface_test.go |
| +++ b/internal/fichastest/surface_test.go |
| @@ -1,6 +1,7 @@ |
| package fichastest |
| import ( |
| + "html" |
| "net/http" |
| "net/url" |
| "regexp" |
| @@ -175,21 +176,93 @@ func TestEditingHasItsOwnPage(t *testing.T) { |
| } |
| // Table and Cards are two views of one list, switched by a link so it works |
| -// with scripts off and can be bookmarked. |
| +// with scripts off and can be bookmarked. The test follows the tabs' own |
| +// links rather than matching a URL spelling. |
| func TestTheListHasTableAndCardsViews(t *testing.T) { |
| h := newApp(t) |
| createACustomer(t, h, "Cnoc Teoranta") |
| + tab := func(body, label string) string { |
| + m := regexp.MustCompile(`<a href="([^"]*)"[^>]*>` + label + `</a>`).FindStringSubmatch(body) |
| + if m == nil { |
| + t.Fatalf("no %s tab", label) |
| + } |
| + return html.UnescapeString(m[1]) |
| + } |
| table := get(t, h, "/").Body.String() |
| if !strings.Contains(table, "fichas-grid") || strings.Contains(table, "fichas-card-grid") { |
| - t.Error("the default view is not the table") |
| + t.Fatal("the default view is not the table") |
| } |
| - cards := get(t, h, "/?view=cards").Body.String() |
| + cards := get(t, h, tab(table, "Cards")).Body.String() |
| if !strings.Contains(cards, "fichas-card-grid") { |
| - t.Error("?view=cards does not show cards") |
| + t.Fatal("the Cards tab does not show cards") |
| } |
| - for _, body := range []string{table, cards} { |
| - if !strings.Contains(body, `href="/?view=table"`) || !strings.Contains(body, `href="/?view=cards"`) { |
| - t.Error("a list view is missing the Table · Cards switch") |
| + back := get(t, h, tab(cards, "Table")).Body.String() |
| + if !strings.Contains(back, "fichas-grid") || strings.Contains(back, "fichas-card-grid") { |
| + t.Fatal("the Table tab does not show the table") |
| + } |
| +} |
| + |
| +// ⌘↩ submits from a textarea. The behaviour is the browser's; what can be |
| +// pinned here is that every app page carries the script and it is served. |
| +func TestTheSubmitShortcutIsWiredIn(t *testing.T) { |
| + h := newApp(t) |
| + path := createACustomer(t, h, "Cnoc Teoranta") |
| + for _, p := range []string{"/", path, path + "/edit"} { |
| + body := get(t, h, p).Body.String() |
| + m := regexp.MustCompile(`/static/shortcuts\.[0-9a-f]{16}\.js`).FindString(body) |
| + if m == "" { |
| + t.Errorf("%s does not load shortcuts.js", p) |
| + continue |
| } |
| + if rec := get(t, h, m); rec.Code != http.StatusOK || !strings.Contains(rec.Body.String(), "requestSubmit") { |
| + t.Errorf("%s: shortcuts.js is not served", m) |
| + } |
| + } |
| +} |
| + |
| +// Search, filter and sort through the real page. The index's own tests pin |
| +// the semantics; these pin that the page wires them up and keeps each state |
| +// a link. |
| +func TestSearchFilterAndSortOnTheList(t *testing.T) { |
| + h := newApp(t) |
| + for _, c := range []struct{ name, stage string }{ |
| + {"Brightwater Labs", "Proposal"}, {"Cedar & Pike", "Lead"}, {"Halden Robotics", "Won"}, |
| + } { |
| + rec := post(t, h, "/cards", url.Values{"f_name": {c.name}, "f_stage": {c.stage}}) |
| + if rec.Code != http.StatusSeeOther { |
| + t.Fatalf("creating %s: %d", c.name, rec.Code) |
| + } |
| + } |
| + |
| + body := get(t, h, "/?q=bright").Body.String() |
| + if !strings.Contains(body, "Brightwater Labs") || strings.Contains(body, "Halden Robotics") { |
| + t.Error("search did not narrow the list") |
| + } |
| + if !strings.Contains(body, `value="bright"`) { |
| + t.Error("the search box does not show the current search") |
| + } |
| + |
| + body = get(t, h, "/?f_stage=Won").Body.String() |
| + if !strings.Contains(body, "Halden Robotics") || strings.Contains(body, "Brightwater Labs") { |
| + t.Error("the Stage filter did not apply") |
| + } |
| + if !strings.Contains(body, "Stage: Won") { |
| + t.Error("the filter menu does not say which filter is on") |
| + } |
| + |
| + body = get(t, h, "/?sort=f_name").Body.String() |
| + if a, c := strings.Index(body, "Brightwater Labs"), strings.Index(body, "Halden Robotics"); a < 0 || c < 0 || a > c { |
| + t.Error("sorting by name did not order the rows") |
| + } |
| + if !strings.Contains(body, `aria-sort="ascending"`) { |
| + t.Error("the sorted column does not say so to assistive technology") |
| + } |
| + |
| + body = get(t, h, "/?q=nothingmatchesthis").Body.String() |
| + if !strings.Contains(body, "Nothing matches") || !strings.Contains(body, "Clear the search and filters") { |
| + t.Error("a search with no results has no way back") |
| + } |
| + if !strings.Contains(body, "data-live-results") { |
| + t.Error("the results region live.js swaps is missing") |
| } |
| } |
diff --git a/scripts/preview/main.go b/scripts/preview/main.go| index 85da3f3..c482dc5 100644 |
| --- a/scripts/preview/main.go |
| +++ b/scripts/preview/main.go |
| @@ -20,6 +20,7 @@ import ( |
| "net/http" |
| "os" |
| "path/filepath" |
| + "strconv" |
| fichas "amadan.net/oficina/fichas/internal/fichas" |
| "github.com/carlosframework/rastrillo/db" |
| @@ -69,7 +70,6 @@ func seed(d *db.DB, cfg fichas.Config) error { |
| return err |
| } |
| me := fichas.Person(m.ID) |
| - type call struct{ body string } |
| customers := []struct { |
| values map[string]string |
| calls []string |
| @@ -91,6 +91,22 @@ func seed(d *db.DB, cfg fichas.Config) error { |
| fichas.FieldEmail: "it@oriel.example", fichas.FieldStage: "Won"}, |
| []string{"Signed. Rollout starts next month."}}, |
| } |
| + // PREVIEW_EXTRA=5000 adds that many more synthetic customers, to look at |
| + // and time the list at a realistic size. |
| + if n, _ := strconv.Atoi(os.Getenv("PREVIEW_EXTRA")); n > 0 { |
| + stages := []string{"Lead", "Contacted", "Proposal", "Won", "Lost"} |
| + for i := 0; i < n; i++ { |
| + customers = append(customers, struct { |
| + values map[string]string |
| + calls []string |
| + }{map[string]string{ |
| + fichas.FieldName: fmt.Sprintf("Company %04d", i), |
| + fichas.FieldEmail: fmt.Sprintf("hello%04d@company%04d.example", i, i), |
| + fichas.FieldStage: stages[i%5], |
| + fichas.FieldNotes: "Renewal is tied to their grant cycle; the budget holder signs off in March.", |
| + }, nil}) |
| + } |
| + } |
| for _, c := range customers { |
| card, err := fichas.CreateCard(ctx, d.G, cfg.Sealer, fichas.TypeCustomer, me, c.values) |
| if err != nil { |