amadan

oficina / sheets Public

Integrate workbook chrome and shared theme with durable batch saves

Paul Campbell pushed by paul@keymail.dev 20c269129733dcd57f5d1c4f03bcccd5f5f4ff2f parent 003b183parent d35cf94
29 files changed, +1446 −76
  • AGENTS.md +8 −16
  • docs/superpowers/specs/2026-09-07-atomic-cell-batch.md +22 −0
  • docs/superpowers/specs/2026-09-07-canary-fixture-bootstrap.md +62 −0
  • docs/superpowers/specs/2026-09-07-durable-tab-versions.md +56 −0
  • internal/guardstest/csstokens_test.go +2 −2
  • internal/sheets/app.go +1 −0
  • internal/sheets/handlers_cell_batch.go +65 −0
  • internal/sheets/handlers_cell_batch_test.go +130 −0
  • internal/sheets/migrations/0010_alter_tabs.sql +2 −0
  • internal/sheets/migrations/schema.sql +1 −1
  • internal/sheets/static/cellqueue.js +20 −37
  • internal/sheets/static/clipboard.js +5 −6
  • internal/sheets/static/grid.js +3 −3
  • internal/sheets/store/handle_blank_cap_test.go +12 −1
  • internal/sheets/store/handle_open.go +8 −1
  • internal/sheets/store/handle_report.go +3 −0
  • internal/sheets/store/handle_set.go +10 −1
  • internal/sheets/store/handle_version.go +27 −0
  • internal/sheets/store/handle_version_export_test.go +14 −0
  • internal/sheets/store/handle_version_limits_test.go +184 −0
  • internal/sheets/store/handle_version_migration_test.go +122 −0
  • internal/sheets/store/handle_version_watermark_test.go +203 −0
  • internal/sheets/store/handle_write_batch.go +96 −0
  • internal/sheets/store/handle_write_batch_test.go +251 −0
  • internal/sheets/store/models.go +7 −6
  • internal/sheets/theme.go +1 −1
  • tools/drive/batch-checks.mjs +52 −0
  • tools/drive/canary-batch.mjs +75 −0
  • tools/drive/clipboard.mjs +4 −1

CI passed — run details

diff --git a/AGENTS.md b/AGENTS.md
index 9d25ff7..59d3724 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -201,22 +201,14 @@ relying on remembering one, make it a test instead.
`doctor` binary alongside the module** — a `doctor` one release behind
compares against its own library copy and calls the upgrade you just did
the drift.
-- **`doctor --fix` does not re-copy `static/theme.css`. Copy it by hand.**
- Measured on the v0.22.0 → v0.23.0 upgrade: `doctor --fix -theme day`
- printed `drift theme.css` with the diff, rewrote `tokens.css`, and left
- `theme.css` untouched without saying it had. Unnamed, the theme is not
- compared at all (`custom or drifted — not compared`), which reads like
- a clean bill.
- This is the worst shape the frozen-theme problem takes, because the two
- files are coupled: v0.23.0's `tokens.css` writes
- `border-bottom: 1px solid var(--rst-header-rule)` and v0.22.0's `day`
- declares no such token. An undefined custom property makes the whole
- shorthand invalid at computed-value time, so it resolves to `initial` —
- `border-style: none`. The page header's rule does not go grey or go
- wrong; **it disappears**, silently, on every screen. So after every
- upgrade: `cp "$(go env GOMODCACHE)/github.com/carlosframework/rastrillo@<ver>/ui/themes/day.css"
- internal/sheets/static/theme.css`, then re-run `doctor -theme day` and
- require `all matching the library`.
+- **Theme bytes come from the Estilo module pin.** There is no app-local
+ `static/theme.css` to copy. On framework upgrades, update the Estilo pin
+ to the same released `day` body and run the theme asset tests. Estilo's
+ byte-for-byte test and this app's `theme_assets_test.go` detect drift;
+ `rastrillo doctor` reports the absent theme as not compared. Its token
+ checks still apply. Do not recreate an embedded theme file that the
+ served filesystem would shadow. This matters at v0.23.0 because tokens
+ require `--rst-header-rule`; an older theme silently loses the rule.
- Don't build on `.rst-page-header::after` — the rake line is retired in
favour of `--rst-header-rule` (rastrillo v0.23.0, design doc §6-v2.2).
It was retired because, built from the same parts in the same order, it
diff --git a/docs/superpowers/specs/2026-09-07-atomic-cell-batch.md b/docs/superpowers/specs/2026-09-07-atomic-cell-batch.md
new file mode 100644
index 0000000..c70b2c0
--- /dev/null
+++ b/docs/superpowers/specs/2026-09-07-atomic-cell-batch.md
@@ -0,0 +1,22 @@
+# Atomic cell batch implementation contract
+
+Status: revision 3; Claude Fable 5.1 (claude-fable-5-1) approved revision 2 design at 1aca44c with these wording amendments, and requested code changes for committed browser queue proof and obsolete comments. Focused code re-review remains pending. Implements the overnight suite readiness request, under the existing Sheets design and next-three batch-route item.
+
+- POST /api/sheets/{sheetID}/tabs/{tabID}/cells/batch accepts {cells: [{r,c,raw,fromR?,fromC?}], client?}. Require an editor using openTab, preserving sheet/tab pairing and 404 isolation.
+- Limit to 500 distinct destinations and 1 MiB body. Validate every coordinate, copy origin, raw length, translated raw length, and formula before any committed writes. Duplicate destinations, empty batches, and trailing JSON are invalid. Reuse existing fixed response strings; add no UI copy.
+- Hold the live handle mutex while a transaction-scoped independent handle is built from the DB. It shares no mutable graph or maps with the live handle. Set its next version at least to the live handle version. Ensure rows/columns and perform existing SetReport operations through that transaction handle. Existing nested transactions become savepoints; only the outer transaction commits. Reject capped recalculation and over-limit resulting cell count atomically.
+- Preserve existing per-cell input history and recalculation semantics. Merge reports by coordinate, keeping final values and marking directly edited cells as direct. Return only after outer commit; swap graph/maps/version into the live handle after commit. On failure discard the transaction handle; no partial history, axes, data, or live publication.
+- One response and SSE publication contain all changed cells and the final version. Keep the current 500-cell paste cap. The existing cell queue submits one batch instead of one request per cell. Larger clear selections retain serial batches of 500; a failed batch must stop the queue. Existing post currently absorbs failures; make it return explicit success for queue control while preserving existing callers.
+- No format, row/column semantics, tab lifecycle, formula grammar, or permission changes. Clipboard provenance is a separate next slice; this changes only transmission, no behaviour change to clipboard.js; its batching comments are updated.
+- Cost: a second tab graph is built per batch. Measure a 500-cell batch and document this bounded first implementation. Avoid adding a graph clone API or redesigning calculation before evidence demands it.
+- Tests: dependent formulas and clears, overlapping formulas, encryption/history, invalid late formula, injected persistence failure after earlier writes, axis rollback, shared live handle unchanged on failure, viewer/stranger/cross-sheet isolation, request bounds, one publication, client queue one-request and stop-on-failure behavior in tools/drive/batch-checks.mjs, called by clipboard.mjs and canary-batch.mjs. These use real editor clipboard actions, intercept a forced first-chunk failure, and await the queue completion barrier rather than assuming a delay means success. Browser checks are a deployed-canary gate, not part of make test. Run make test and exact-head CI. Build with make release, deploy isolated canary, run browser checks, obtain Fable branch review, then coordinate squash merge and production deployment with root.
+
+## Revision 2 review answers
+
+1. The independent handle is built with `buildHandle(tx, ...)`; its `db` is exactly `tx`. Recovery, count, axis allocation, input/event writes, and nested transactions all use that handle. No Open/registry path runs inside the transaction. TestCellBatchRecoversStaleCacheInsideTransaction deliberately seeds a stale cell after opening the live handle, then requires recovery and batch completion within three seconds. Building from the outer DB deadlocks that test.
+2. Measured with CELL_BATCH_BENCH_N and the committed TestCellBatchLargeTabTiming fixture: 500 changed cells on a 50,000-cell tab took 660ms; on a 200,000-cell tab took 2.326s on this development machine. These are transaction duration including rebuild, and therefore instance-wide writer blocking durations. The fixture has one feeder and many dependent formulas; paste changes leaf cells. Production measurements remain required. This initial bounded implementation accepts this cost to preserve atomic rollback without inventing graph cloning. The trigger for moving the rebuild out of the writer transaction is measured unacceptable production/pilot latency; it must preserve recovery and handle consistency. Each SetReport uses nested savepoints, which GORM retains until outer commit (roughly 1,000 per 500-cell batch).
+3. The first non-error operation after outer Transaction returns is swapping graph and mutable maps into the live handle under its mutex. Report assembly occurs inside the transaction before commit; response formatting happens after the live state is updated. There is no fallible operation between commit and the swap. Failure to encode or deliver the HTTP response does not invalidate the now-current live handle; reconnect/snapshot recovers delivery.
+4. A merged report may contain at most calc.MaxDirtyPerPass (5,000) distinct cells. Exceeding either an individual recalc cap or the merged report cap rejects and rolls back the whole batch. This deliberately differs from the single-cell path: the batch success contract guarantees complete final values without depending on the pre-existing resync behavior. No new user copy is introduced; the existing write failure string is reused. Supporting larger dependency fanouts is a later slice with dependable resync and explicit recovery semantics. TestCellBatchRejectsMergedReportPastCalculationCap proves the aggregate limit is applied even when each individual edit is within its cap.
+5. Both the implementation and failure-injection tests exist. The live handle is preserved on rollback and updated immediately on success. Local make test and remote CI were green at initial implementation 50914fb; revision 2 requires fresh checks and review.
+
+Revision 3 records the aggregate request bound explicitly: the 1 MiB body limit may reject a set of individually valid 8 KiB cells; callers must stay within both limits. Batch copy origins must include both coordinates, stricter than the legacy single-cell route. For pilot measurement, a p95 batch latency above 1 second at 50,000 cells or 3 seconds at 200,000 cells triggers the next transaction-duration design review; these are investigation thresholds, not a readiness claim.
diff --git a/docs/superpowers/specs/2026-09-07-canary-fixture-bootstrap.md b/docs/superpowers/specs/2026-09-07-canary-fixture-bootstrap.md
new file mode 100644
index 0000000..335b5bc
--- /dev/null
+++ b/docs/superpowers/specs/2026-09-07-canary-fixture-bootstrap.md
@@ -0,0 +1,62 @@
+# Isolated Sheets canary session fixture
+
+Status: proposed fallback for Fable review; do not deploy yet. Normal SMTP-based sign-in remains preferred and root has asked the user about a controlled mailbox. This design is only for testing unchanged session validation and editor behavior if that mailbox is unavailable.
+
+## Target and boundary
+
+The only permitted target is app sheets, account bds, instance overnight-sheets.bds.oncarlos.com, channel canary/overnight-batch, with named environment overnight-batch, separately bound by root. Root has verified current platform CLI supports instance environment binding. Never change the app's default environment, production origin or production channel. The fixture receives fresh canary SHEETS_DEK and SHEETS_INSTANCE_KEY and an owner address oficina-fixture@example.test. It does not read or copy production secrets, sessions, databases or content.
+
+Use a temporary reviewed bootstrap binary on this isolated instance, then replace it with the exact normal candidate binary. The bootstrap binary contains no login endpoint, alternate auth middleware, universal cookie, content endpoint or API backdoor. It initializes existing schema/session rows during startup only, then serves only platform health/version responses while awaiting replacement. It never serves the editor, so no browser behavior can accidentally be certified against bootstrap code.
+
+## Host and environment guards
+
+- Compile the single expected HTTPS origin and canary hostname into the bootstrap. Require SHEETS_ORIGIN to equal that origin exactly (scheme/host, no wildcard or suffix match).
+- Require a one-use fixture run ID in a dedicated canary environment variable, and require it to equal the run ID compiled into this bootstrap build. It is a scope guard, not an authentication mechanism.
+- Require SHEETS_OWNERS to equal exactly the synthetic owner address, with no additional addresses. Validate the key formats normally and require root's fresh-key provisioning record; do not compare against or obtain production key values.
+- The supported target is the platform exec-child contract: ExecBackend.Start invokes the binary with --socket <route.Socket> --db <route.DBPath or DataDir/host.db>. Read both through rastrillo.Resolve, not by parsing HTTP Host. The observed canary socket is exactly /run/carlos/overnight-sheets.bds.oncarlos.com.sock. The expected default database is /data/overnight-sheets.bds.oncarlos.com.db (platform DataDir defaults to /data). Root must confirm that resolved DB path from the route/activation configuration before building. Compile the confirmed absolute DB and socket paths, and reject any different path even if its basename matches.
+- Validate both exact cleaned absolute paths and exact host-derived basenames. Reject parent traversal, a symlink database file, or a resolved parent directory outside the compiled expected root. For a fresh nonexistent DB, resolve/check the existing parent directory before opening it; after opening, confirm the path still identifies the expected file. Do not derive the expected paths from fixture-supplied environment values. If the deployment uses a non-default DataDir, review and compile that one actual absolute path rather than accepting arbitrary directories.
+- STATE_DIRECTORY is not required for this exec-child shape. If present, it must not contradict the confirmed runtime paths, but its absence alone is not a failure. Supporting a systemd unit-backed shape later requires a separate explicit guard variant, not a basename-only fallback.
+- Before deployment, root verifies the control-plane route has this exact host, named environment and canary-only channel. Runtime path/origin guards complement that check; neither a release name nor an HTTP Host header alone establishes isolation.
+- Refuse bootstrap if any app content or member/grant/session row already exists. This is a fresh-fixture tool, not a repair or reset tool. Existing migration ledger rows alone are harmless. Never clear tables to make the precondition pass.
+
+## Session material and initialization
+
+Generate a random 32-byte session token locally using the framework's token format (base64 raw URL encoding), and compute SHA-256 hex with sessions.HashToken. Keep the raw token only in a chmod-0600 local Playwright storage-state file. Send only the hash, synthetic subject and bounded expiry through the dedicated canary environment. No token, key, cookie or sign-in link goes into stdout, logs, discussions, screenshots, commits or release metadata.
+
+Run sheets.BootSchema through the normal migrate.Apply path. Seed only one row in the existing sessions table in one transaction after rechecking the empty-state precondition. The pinned sessions schema contains token_hash, subject, method, auth_time, created_at and expires_at. Use:
+
+- token_hash: supplied 64-character lowercase hex hash, validated strictly;
+- subject: exactly oficina-fixture@example.test;
+- method: fixture (do not claim the mailbox was verified);
+- auth_time: empty;
+- created_at: current UTC RFC3339;
+- expires_at: a supplied absolute UTC time no later than two hours after creation, validated before insertion.
+
+No custom table or migration is added. A repeated bootstrap start may accept exactly the same single hash/subject/expiry row as an idempotent success but must not extend its expiry or mint another session. Any other session or app data causes refusal. Once the candidate browser creates app content, bootstrap is intentionally unable to run again on that database.
+
+The normal candidate's actor() resolves the session subject and memberFor admits/promotes only the synthetic owner listed in SHEETS_OWNERS. Do not seed a broader role or grant independently. Local tests must prove the seeded row is accepted by the real sessions.New lookup/adoption path, while a random token and an expired fixture row are rejected.
+
+Playwright state sets the cookie __Host-rastrillo_session for the exact HTTPS canary origin: Secure=true, HttpOnly=true, SameSite=Lax, path=/, no Domain attribute. The raw value never reaches bootstrap. The cookie is a fixture for the existing session store, not a signature forged using an application key.
+
+## Deployment sequence and assertions
+
+1. Record production process version and route/channel/environment bindings without reading secrets. Verify isolated canary has fresh keys and the expected empty database.
+2. After Fable approves bootstrap design/code and local fixture tests, deploy its uniquely stamped binary only to the explicit canary host/channel with carlos deploy. Verify bootstrap /api/version and that every non-platform route refuses; no app assertions yet.
+3. Remove the fixture-specific hash/run/expiry settings from the named environment before replacing bootstrap. They are not used by the normal candidate.
+4. Deploy the exact clean candidate built with make release. Poll process-served /api/version until it equals the intended candidate SHA; verify the canary channel no longer names the bootstrap release. Verify production still names its previously recorded release. Never treat an edge version header or a successful promote as process proof.
+5. Only now open Playwright with the private cookie state. Verify /sheets accepts this fixture session, unsigned and random-cookie requests cannot write, and then run deployed clipboard/batch, clear/reopen version regression, reload, second-observer and scriptless-table checks. Record candidate SHA alongside outcomes.
+6. Do not claim these checks exercised email delivery, pending-cookie magic-link binding, SMTP credentials or mailbox access. They certify real persisted-session validation and candidate editor behavior only. Normal SMTP sign-in remains a separate readiness check.
+
+## Teardown
+
+Use the candidate's normal POST /signout while the fixture cookie is present; confirm it deletes/revokes the row and the old cookie no longer accesses /sheets. Close browser contexts and delete the private storage-state/token files. Fixture expiry is a second bound if signout fails. Remove remaining fixture-specific named-environment variables. Keep the canary channel on the normal candidate or remove only this disposable instance through supported CLI, according to root's overnight verification needs. Never leave bootstrap selected by any channel. Remove its channel reference first; an immutable release artifact may remain in storage if the platform has no safe deletion verb, but its origin/path/run guards and missing fixture env must make it inert if mistakenly selected.
+
+Do not delete production routes, releases, keys, backups or configuration during teardown. Any authentication or cleanup failure is recorded as a failed/limited canary check, not hidden by reseeding.
+
+## Evidence required before use
+
+Fable review, wrong-host/wrong-origin/nonempty-database/expired-session/local-session-acceptance tests, supported CLI route/environment inspection, bootstrap replacement proof, and exact candidate process SHA. No bootstrap deployment has occurred as part of writing this proposal.
+
+## Native seed path
+
+The existing platform supports a first-boot seed argv on an instance record; ExecBackend.Seed invokes that release binary with the configured argv followed by --db <path>, under the same merged environment, only when no local database or replica exists. This is an existing supported lifecycle, not an HTTP auth mechanism. It can be used only if root verifies the actual instance seed configuration through the supported CLI/API and Fable reviews the separate bootstrap command. The command must retain the same origin/run/owner/empty-database/exact-DB-path guards; a one-shot native seed has no socket argument and must verify the corresponding expected route socket in root's control-plane preflight. It writes ordinary session hashes into the existing schema, exits, and introduces no fixture command into the normal candidate. The candidate's deployed manifest/instance record must not keep a bootstrap seed argv referring to unsupported candidate commands. The initial fallback remains the simpler reviewed startup-only bootstrap; no native seed configuration has been changed.
diff --git a/docs/superpowers/specs/2026-09-07-durable-tab-versions.md b/docs/superpowers/specs/2026-09-07-durable-tab-versions.md
new file mode 100644
index 0000000..0c35dde
--- /dev/null
+++ b/docs/superpowers/specs/2026-09-07-durable-tab-versions.md
@@ -0,0 +1,56 @@
+# Durable tab version watermark
+
+Status: design approved for implementation at 9f6abe2 by Fable 5.1 (claude-fable-5-1), after the four amendments requested at 91e9991. Implementation starts at 6ccb609. Amadan discussion 19/task 3. Branch review, exact-head CI, reconciliation with deployed chrome and canary verification remain required before production deployment.
+
+## Reproduction
+
+A 500-cell paste returns version 503. Release/reopen the tab handle, clear those 500 cells, and the clear returns 1004. Release/reopen again and write one new cell: version 4. The browser correctly refuses version 4 as older than 1004, so the newly committed edit is absent from the display. The same defect affects a single-cell edit after clearing the last surviving cell.
+
+buildHandle derives maxVersion only from current cells' InputVersion and CacheVersion. Deleting the newest cell rows destroys that ordering evidence. The in-memory handle counter survives only while some caller retains the handle; SSE subscribers deliberately do not retain it. An event count or timestamp is not a reliable substitute: neither is the previously issued version.
+
+## Invariant and schema
+
+For each tab, every committed cell input or persisted calculation stamp is no greater than a durable tab watermark. Opening a handle starts above both that watermark and all surviving input/cache versions. Clearing cells, recovering stale caches, and releasing every handle must never make a future edit's version go backwards.
+
+Add Tab.ContentVersion as a non-null integer with default zero through an additive generated migration. It is metadata of the same class as the existing cell InputVersion/CacheVersion, not authored content. Existing tabs start at zero and retain all content, names, history, formats, rows and columns. At open, the maximum of watermark and existing cell versions preserves older populated databases without a destructive migration/backfill. Already-empty pre-upgrade databases have no reconstructable historic version; deployment must reconnect/refetch browsers, and all future edits gain the durable invariant.
+
+SQLite integers are signed 64-bit. Refuse an exhausted/corrupt watermark before allocating a wrapped stamp; do not silently return to zero. No user-facing strings are added; existing fixed storage failure responses remain.
+
+## Mutation paths
+
+| Path | Required handling |
+| --- | --- |
+| SetReport -> setContent | Advance watermark to at least v in the same first transaction as sealed input and AppendInput |
+| SetReport -> setBlank | Advance watermark to at least v in the same transaction as deletion, dirty dependent markers and AppendInput |
+| persistTouched | Advance watermark to at least v in the same transaction as sealed cache updates and AppendRecalc |
+| recoverStale | Uses persistTouched on the transaction-scoped handle; recovery version is also preserved even if its cells are later cleared |
+| SetBatch | Every nested update uses staged.db=tx; outer rollback undoes watermark changes too; successful live swap retains staged.nextVersion |
+| Table edit, XLSX import, restore, undo | Already call Set or SetReport; inherit the same durable handling without parallel logic |
+| SetFormat, widths/heights, saved views | Existing behavior does not allocate content versions; retain it. This fix does not pretend to solve concurrent ordering of distinct equal-version formatting responses |
+| EnsureRow/EnsureCol | Allocate stable axes only, not a content change; no new version |
+| Row/column insert/delete/move | Not implemented in this repository; future structural operations must allocate and persist a tab stamp in the same mutation transaction. Do not delete a watermark with cell/axis rows |
+| Tab add/rename/reorder | Future metadata events have separate ordering semantics; do not masquerade as cell history |
+
+Implement one small transaction-scoped helper for monotonic advancement using an allowlisted update of content_version on Tab by ID. It never lowers a stored value. Both input write paths and persistTouched call it. No outer db calls occur while a transaction holds the one writer connection. Use SET content_version = MAX(content_version, ?) WHERE id = ? so RowsAffected=0 means the tab is missing, not that a lower stamp was safely ignored. A missing tab row must fail rather than silently discarding the watermark.
+
+Read the tab watermark while building a handle, before allocating nextVersion; a missing tab row fails Open. Preserve the existing deterministic recovery-stamp reservation, even when there are no stale cells, so a snapshot version handed out before release is no greater than the first edit stamp after reopen. Check exhaustion at Open and SetReport allocation (including staged SetReport calls inside SetBatch). Move the in-memory nextVersion increment above the blank crash seam after transaction one commits. A failed input/recalculation transaction must not commit its watermark; the existing dirty-marker behavior after a committed input transaction remains valid. A failed atomic batch must leave both the live handle and persisted watermark unchanged.
+
+## Regression coverage
+
+1. Real release/reopen sequence for paste500, clear500, new edit; require strictly increasing report versions and the correct final value. Existing candidate fails 503 ->1004 ->4.
+2. Single-cell last-cell clear followed by new edit after reopen; require the new stamp to exceed the clear stamp.
+3. Blank crash seam after the first transaction: release/reopen and require the next edit to exceed the committed blank's allocated stamp.
+4. Overlapping formula/dependent recomputation, removal of the newest cached cells, and reopen preserve ordering and correct values.
+5. Existing failure injection into a late SetBatch event must leave Tab.ContentVersion unchanged alongside inputs, history and axes. A subsequent successful edit must remain usable.
+6. Migrate a database built with BootSchema through the pinned pre-watermark migration IDs (Sheets 0001 through 0009 plus framework schema); seed encrypted names/content and history, then apply the new migration. Verify byte-for-byte ciphertext retention, default watermark, schema agreement and monotonic first post-upgrade edits.
+7. A high watermark in an otherwise empty tab survives Open; the next write uses a higher stamp. A lower attempted advance cannot reduce it. Exhaustion refuses without a write.
+8. Snapshot Version() before release/restart must be no greater than the first edit stamp after reopening. Test both populated and cleared tabs.
+9. A capped blank with more than MaxDirtyPerPass dependents must advance the watermark in transaction one alone. Mutation-test by removing only that advancement; the regression must fail.
+10. A retained handle starts a batch with nextVersion above the persisted watermark, commits a batch, then rolls back a late-failing batch. After release/reopen the next edit must exceed the successful report, and the failed batch must leave its watermark unchanged.
+11. Run existing crash/recovery, plain SQLite/WAL leakage and race tests. The watermark contains no plaintext values and adds no logs.
+
+## Review and deploy
+
+Keep this prerequisite independently attributable in discussion 19. Fable reviews the design before implementation and the final code before merge. Integrate with atomic batch only after passing full make test, rastrillo migration check, exact-head CI and deployed canary browser regression (paste, clear, new paste, reload, second observer). A migration-only deploy must not erase or replace any user data. Root coordinates production promotion and /api/version proof.
+
+The current migration head is 0009_alter_formats_ink_intent. Confirm current main has not acquired 0010 before final integration; assign a fresh additive migration ID if it has. The old-database fixture must select known IDs, not filter SQL text, so future migrations mentioning the column cannot be silently omitted.
diff --git a/internal/guardstest/csstokens_test.go b/internal/guardstest/csstokens_test.go
index c313fb5..dcbb4bf 100644
--- a/internal/guardstest/csstokens_test.go
+++ b/internal/guardstest/csstokens_test.go
@@ -22,7 +22,7 @@ var ourStylesheets = []string{
// structural ones copied by `rastrillo doctor` and the shared palette.
var tokenSources = []string{
"internal/sheets/static/tokens.css",
- "internal/sheets/static/theme.css",
+ "Estilo theme module",
}
var (
@@ -51,7 +51,7 @@ func TestEveryTokenWeUseExists(t *testing.T) {
for _, rel := range tokenSources {
var body []byte
var err error
- if rel == "internal/sheets/static/theme.css" {
+ if rel == "Estilo theme module" {
body = theme.CSS()
} else {
body, err = os.ReadFile(filepath.Join(root, rel))
diff --git a/internal/sheets/app.go b/internal/sheets/app.go
index 614c2d6..cbbe81f 100644
--- a/internal/sheets/app.go
+++ b/internal/sheets/app.go
@@ -241,6 +241,7 @@ func (a *App) Router() http.Handler {
// ask for it in.
r.Get("/api/sheets/{sheetID}/tabs/{tabID}/cells", a.handleCellsGet)
r.Post("/api/sheets/{sheetID}/tabs/{tabID}/cells", a.handleCellPost)
+ r.Post("/api/sheets/{sheetID}/tabs/{tabID}/cells/batch", a.handleCellBatchPost)
r.Post("/api/sheets/{sheetID}/tabs/{tabID}/format", a.handleFormatPost)
r.Post("/api/sheets/{sheetID}/tabs/{tabID}/size", a.handleSizePost)
r.Get("/api/sheets/{sheetID}/tabs/{tabID}/stream", a.handleStream)
diff --git a/internal/sheets/handlers_cell_batch.go b/internal/sheets/handlers_cell_batch.go
new file mode 100644
index 0000000..1cb390a
--- /dev/null
+++ b/internal/sheets/handlers_cell_batch.go
@@ -0,0 +1,65 @@
+package sheets
+
+import (
+ "encoding/json"
+ "io"
+ "net/http"
+
+ "amadan.net/oficina/sheets/internal/sheets/store"
+)
+
+func (a *App) handleCellBatchPost(w http.ResponseWriter, r *http.Request) {
+ actor, ok := a.actor(r)
+ if !ok {
+ http.Error(w, "Sign in first.", http.StatusForbidden)
+ return
+ }
+ h, ok := a.openTab(w, r, SheetEditor)
+ if !ok {
+ return
+ }
+ defer h.Release()
+
+ var in struct {
+ Cells []cellWrite `json:"cells"`
+ Client string `json:"client"`
+ }
+ dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, 1<<20))
+ if err := dec.Decode(&in); err != nil {
+ http.Error(w, "Malformed request.", http.StatusBadRequest)
+ return
+ }
+ var extra any
+ if err := dec.Decode(&extra); err != io.EOF || len(in.Cells) == 0 || len(in.Cells) > store.MaxCellBatch {
+ http.Error(w, "Malformed request.", http.StatusBadRequest)
+ return
+ }
+ writes := make([]store.CellWrite, len(in.Cells))
+ for i, cell := range in.Cells {
+ dr, dc, valid := cell.origin()
+ if !inCellRange(cell.R, cell.C) || !valid || (cell.FromR == nil) != (cell.FromC == nil) {
+ http.Error(w, "Cell out of range.", http.StatusBadRequest)
+ return
+ }
+ if len(cell.Raw) > maxRawBytes {
+ http.Error(w, "Cell input too long.", http.StatusRequestEntityTooLarge)
+ return
+ }
+ raw := translateCopied(cell.Raw, dr, dc)
+ if len(raw) > maxRawBytes {
+ http.Error(w, "Cell input too long.", http.StatusRequestEntityTooLarge)
+ return
+ }
+ writes[i] = store.CellWrite{Row: cell.R, Col: cell.C, Raw: raw}
+ }
+ rep, err := h.SetBatch(writes, actor.ID)
+ if err != nil {
+ http.Error(w, "That cell could not be written.", http.StatusBadRequest)
+ return
+ }
+ cells, styles := fromReport(rep, newPosIndex(h), h)
+ resp := gridResponse{Version: rep.Version, Cells: cells, Formats: styles, Client: in.Client}
+ a.touchSheet(sheetIDFrom(r))
+ a.publish(tabIDFrom(r), resp)
+ writeJSON(w, http.StatusOK, resp)
+}
diff --git a/internal/sheets/handlers_cell_batch_test.go b/internal/sheets/handlers_cell_batch_test.go
new file mode 100644
index 0000000..6bf7103
--- /dev/null
+++ b/internal/sheets/handlers_cell_batch_test.go
@@ -0,0 +1,130 @@
+package sheets
+
+import (
+ "encoding/json"
+ "fmt"
+ "net/http"
+ "net/http/httptest"
+ "strings"
+ "testing"
+
+ "amadan.net/oficina/sheets/internal/sheets/store"
+)
+
+func TestCellBatchPublishesOnlyFinalCommittedValues(t *testing.T) {
+ app, cookies := signIn(t)
+ s, path := newSheet(t, app, cookies, "Budget")
+ sub := app.hub.Subscribe(s.TabID)
+ defer sub.Close()
+ rec := postCell(t, app, cookies, path+"/batch", `{"cells":[{"r":1,"c":0,"raw":"=A1*2"},{"r":0,"c":0,"raw":"21"}],"client":"batch-client"}`)
+ if rec.Code != http.StatusOK {
+ t.Fatalf("batch %d: %s", rec.Code, rec.Body.String())
+ }
+ got := decodeGrid(t, rec)
+ if len(got.Cells) != 2 || got.Client != "batch-client" {
+ t.Fatalf("response %+v", got)
+ }
+ for _, c := range got.Cells {
+ if c.R == 1 && c.Display != "42" {
+ t.Fatalf("stale formula %+v", c)
+ }
+ }
+ select {
+ case m := <-sub.C():
+ if string(m.Data) != strings.TrimSpace(rec.Body.String()) {
+ t.Fatalf("publication differs: %s", m.Data)
+ }
+ default:
+ t.Fatal("no publication")
+ }
+ select {
+ case <-sub.C():
+ t.Fatal("intermediate publication")
+ default:
+ }
+ var events int64
+ app.db.Model(&store.Event{}).Where("tab_id = ? AND kind = ?", s.TabID, store.KindInput).Count(&events)
+ if events != 2 {
+ t.Fatalf("input events = %d", events)
+ }
+}
+
+func TestCellBatchRejectsInvalidRequestsWithoutWrites(t *testing.T) {
+ for _, body := range []string{
+ `{}`, `{"cells":[]}`, `{"cells":[{"r":0,"c":0,"raw":"7"}]} {}`,
+ `{"cells":[{"r":0,"c":0,"raw":"7"},{"r":0,"c":0,"raw":"8"}]}`,
+ `{"cells":[{"r":0,"c":0,"raw":"7"},{"r":-1,"c":0,"raw":"8"}]}`,
+ `{"cells":[{"r":0,"c":0,"raw":"7"},{"r":1,"c":0,"raw":"=SUM(("}]}`,
+ `{"cells":[{"r":0,"c":0,"raw":"7","fromR":0}]}`,
+ `{"cells":[{"r":0,"c":0,"raw":"7","fromR":-1,"fromC":0}]}`,
+ `{"cells":[{"r":0,"c":0,"raw":"` + strings.Repeat("x", maxRawBytes+1) + `"}]}`,
+ `{"cells":[{"r":0,"c":0,"raw":"` + strings.Repeat("x", 1<<20) + `"}]}`,
+ } {
+ app, cookies := signIn(t)
+ s, path := newSheet(t, app, cookies, "Budget")
+ sub := app.hub.Subscribe(s.TabID)
+ rec := postCell(t, app, cookies, path+"/batch", body)
+ if rec.Code != http.StatusBadRequest && rec.Code != http.StatusRequestEntityTooLarge {
+ t.Fatalf("invalid status %d", rec.Code)
+ }
+ if strings.Contains(rec.Body.String(), "SUM") || strings.Contains(rec.Body.String(), "xxx") {
+ t.Fatal("content in error")
+ }
+ snap := decodeGrid(t, do(t, app, as(httptest.NewRequest(http.MethodGet, path, nil), cookies)))
+ if len(snap.Cells) != 0 {
+ t.Fatalf("invalid batch wrote %+v", snap.Cells)
+ }
+ select {
+ case <-sub.C():
+ t.Fatal("failed batch published")
+ default:
+ }
+ sub.Close()
+ }
+ app, cookies := signIn(t)
+ _, path := newSheet(t, app, cookies, "Budget")
+ cells := make([]cellWrite, store.MaxCellBatch+1)
+ for i := range cells {
+ cells[i] = cellWrite{R: i, Raw: "1"}
+ }
+ body, _ := json.Marshal(map[string]any{"cells": cells})
+ if rec := postCell(t, app, cookies, path+"/batch", string(body)); rec.Code != http.StatusBadRequest {
+ t.Fatalf("oversized batch = %d", rec.Code)
+ }
+}
+
+func TestCellBatchUsesCopyTranslationAndRejectsWrongTab(t *testing.T) {
+ app, cookies := signIn(t)
+ s, path := newSheet(t, app, cookies, "Budget")
+ rec := postCell(t, app, cookies, path+"/batch", `{"cells":[{"r":1,"c":0,"raw":"=A1+$B$1","fromR":0,"fromC":0}]}`)
+ if rec.Code != 200 {
+ t.Fatalf("copy %d", rec.Code)
+ }
+ if got := decodeGrid(t, rec).Cells[0].Raw; got != "=A2+$B$1" {
+ t.Fatalf("translated = %q", got)
+ }
+ other, _ := newSheet(t, app, cookies, "Other")
+ wrong := fmt.Sprintf("/api/sheets/%d/tabs/%d/cells/batch", s.ID, other.TabID)
+ if rec := postCell(t, app, cookies, wrong, `{"cells":[{"r":0,"c":0,"raw":"1"}]}`); rec.Code != 404 {
+ t.Fatalf("cross sheet = %d", rec.Code)
+ }
+ if rec := postCell(t, app, nil, path+"/batch", `{"cells":[{"r":0,"c":0,"raw":"1"}]}`); rec.Code == 200 {
+ t.Fatal("unsigned write accepted")
+ }
+}
+
+func TestCellBatchEnforcesViewerCommenterAndStrangerIsolation(t *testing.T) {
+ for _, role := range []SheetRole{SheetViewer, SheetCommenter} {
+ app, cookies := signIn(t)
+ s, path := newSheet(t, app, cookies, "Budget")
+ guestOn(t, app, s.ID, "ada@elsewhere.example", role)
+ other, _ := newSheet(t, app, cookies, "Other")
+ guestOn(t, app, other.ID, "stranger@elsewhere.example", SheetEditor)
+ for _, email := range []string{"ada@elsewhere.example", "stranger@elsewhere.example"} {
+ guest := signInAs(t, app, email)
+ if rec := postCell(t, app, guest, path+"/batch", `{"cells":[{"r":0,"c":0,"raw":"1"}]}`); rec.Code != 404 {
+ t.Fatalf("%s %s batch = %d", role, email, rec.Code)
+ }
+ }
+ }
+}
diff --git a/internal/sheets/migrations/0010_alter_tabs.sql b/internal/sheets/migrations/0010_alter_tabs.sql
new file mode 100644
index 0000000..661e081
--- /dev/null
+++ b/internal/sheets/migrations/0010_alter_tabs.sql
@@ -0,0 +1,2 @@
+ALTER TABLE `tabs` ADD `content_version` integer NOT NULL DEFAULT 0;
+
diff --git a/internal/sheets/migrations/schema.sql b/internal/sheets/migrations/schema.sql
index a15b30a..6d8675c 100644
--- a/internal/sheets/migrations/schema.sql
+++ b/internal/sheets/migrations/schema.sql
@@ -23,7 +23,7 @@ CREATE TABLE `sheets` (`id` integer PRIMARY KEY AUTOINCREMENT,`title_sealed` blo
CREATE TABLE `tab_views` (`id` integer PRIMARY KEY AUTOINCREMENT,`tab_id` integer,`spec_sealed` blob,`updated_by` integer,`updated_at` datetime);
-CREATE TABLE `tabs` (`id` integer PRIMARY KEY AUTOINCREMENT,`sheet_id` integer,`name_sealed` blob,`position` integer,`frozen_rows` integer,`frozen_cols` integer);
+CREATE TABLE `tabs` (`id` integer PRIMARY KEY AUTOINCREMENT,`sheet_id` integer,`name_sealed` blob,`position` integer,`frozen_rows` integer,`frozen_cols` integer, `content_version` integer NOT NULL DEFAULT 0);
CREATE UNIQUE INDEX `idx_col_tab_pos` ON `cols`(`tab_id`,`position`);
diff --git a/internal/sheets/static/cellqueue.js b/internal/sheets/static/cellqueue.js
index 3e4124c..04e3bdb 100644
--- a/internal/sheets/static/cellqueue.js
+++ b/internal/sheets/static/cellqueue.js
@@ -1,43 +1,26 @@
-/* Send a batch of cell writes, safely.
- *
- * Its own module because "how do we send N cell writes without losing
- * one" is a different question from "what did the person copy", and
- * because the answer is subtle enough to be worth stating once in a
- * place both callers can read.
- *
- * **The writes go out ONE AT A TIME, and that is the whole point.**
- *
- * The first version sent six at once, on the reasonable ground that a
- * 200-cell paste in series is 200 sequential round trips. Responses to
- * concurrent writes come back in whatever order the server finishes
- * them, and grid.js correctly discards a paint whose version is older
- * than one already applied — so a six-cell paste landed five on screen.
- * Every cell was stored; one simply never painted, which looks exactly
- * like a cell that was never sent.
- *
- * The second version kept the six lanes and added a refetch afterwards
- * to repair them. That was worse: refetch() zeroes grid.js's version
- * before applying, so a snapshot still in flight when a live edit
- * arrives from another tab paints over that newer edit. A repair that
- * introduces a race is not a repair.
- *
- * Serial writes have neither problem. Versions arrive monotonically,
- * nothing is discarded, and no snapshot is needed to fix anything. It is
- * slower per cell, which is what the caller's own size cap is sized
- * against — the honest trade until a batch cell route exists, which is
- * the real answer and is named in the slice spec's deferred table.
- */
+/* Send one atomic batch per 500 cells, with chunks and invocations in order.
+ * Concurrent writes lost paints when older replies were discarded.
+ * A final refetch still raced with live edits. Serial chunks avoid both;
+ * stop on the first failed chunk. */
(function () {
'use strict';
- /* jobs is an array of cell-write bodies; post is grid.js's own post,
- * passed in rather than reached for, so this module never has to know
- * whether the grid booted. */
+ var tail = Promise.resolve();
window.SheetsCellQueue = function (jobs, post, done) {
- var i = 0;
- (function step() {
- if (i >= jobs.length) { if (done) done(); return; }
- post('/cells', jobs[i++]).then(step, step);
- }());
+ var run = function () {
+ var i = 0;
+ function step() {
+ if (i >= jobs.length) { if (done) done(); return true; }
+ var cells = jobs.slice(i, i + 500);
+ i += cells.length;
+ return post('/cells/batch', { cells: cells }).then(function (ok) {
+ if (ok === true) return step();
+ return false;
+ }, function () { return false; });
+ }
+ return step();
+ };
+ tail = tail.then(run, run);
+ return tail;
};
}());
diff --git a/internal/sheets/static/clipboard.js b/internal/sheets/static/clipboard.js
index 1efb886..0623c09 100644
--- a/internal/sheets/static/clipboard.js
+++ b/internal/sheets/static/clipboard.js
@@ -13,8 +13,8 @@
* anyway, which is silent data loss that looks fine until a reload.
*
* So this module takes the keydown AND the three clipboard events at
- * the capture phase, and every cell it writes goes through the same
- * POST a keystroke uses. tsv.js is the wire format and cellqueue.js
+ * the capture phase, and every cell it writes goes through the atomic batch
+ * route. tsv.js is the wire format and cellqueue.js
* sends the writes.
*
* It also carries spec §4.2's translation pass. A pasted formula names
@@ -28,8 +28,8 @@
var surface = document.querySelector('[data-sheet-surface]');
if (!surface) return;
- /* Sized against SERIAL writes (cellqueue.js), not against the store.
- * Raise it when a batch cell route lands, and not before. */
+ /* Keep a paste within one atomic batch. Raising this cap needs a
+ * separate decision about failures across multiple chunks. */
var MAX_CELLS = 500;
function ready() {
@@ -84,8 +84,7 @@
var TSV = window.SheetsTSV;
if (!TSV) return;
- /* cellqueue.js sends the writes. See its own comment for why they
- * go one at a time. */
+ /* cellqueue.js sends atomic batches in order and stops on failure. */
var writeAll = window.SheetsCellQueue;
if (!writeAll) return;
diff --git a/internal/sheets/static/grid.js b/internal/sheets/static/grid.js
index 6ab046d..b6c2f0b 100644
--- a/internal/sheets/static/grid.js
+++ b/internal/sheets/static/grid.js
@@ -150,9 +150,9 @@
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body)
}).then(function (resp) {
- if (!resp.ok) { refetch(); return; }
- return resp.json().then(applyBatch);
- }).catch(refetch);
+ if (!resp.ok) { refetch(); return false; }
+ return resp.json().then(function (payload) { applyBatch(payload); return true; });
+ }).catch(function () { refetch(); return false; });
}
function write(r, c, text) {
diff --git a/internal/sheets/store/handle_blank_cap_test.go b/internal/sheets/store/handle_blank_cap_test.go
index 3074408..f06ce08 100644
--- a/internal/sheets/store/handle_blank_cap_test.go
+++ b/internal/sheets/store/handle_blank_cap_test.go
@@ -137,9 +137,20 @@ func TestBlankFeederOverCapDependentsAreRecoverable(t *testing.T) {
t.Fatalf("test setup: only %d dependents wired, want more than MaxDirtyPerPass (%d)", len(touched), calc.MaxDirtyPerPass)
}
- if err := h.Set(feederRow, colA, "", 7); err != nil {
+ rep, err := h.SetReport(feederRow, colA, "", 7)
+ if err != nil {
t.Fatalf("blank feeder: %v", err)
}
+ if !rep.Capped {
+ t.Fatal("blank unexpectedly recalculated")
+ }
+ var watermark uint64
+ if err := db.Raw("SELECT content_version FROM tabs WHERE id = ?", localTab).Scan(&watermark).Error; err != nil {
+ t.Fatal(err)
+ }
+ if watermark < rep.Version {
+ t.Fatalf("capped blank stamp %d not retained by watermark %d", rep.Version, watermark)
+ }
h.Release()
// The real proof: reopen and read the dependents' actual VALUE, not
diff --git a/internal/sheets/store/handle_open.go b/internal/sheets/store/handle_open.go
index 9e76354..8f3498b 100644
--- a/internal/sheets/store/handle_open.go
+++ b/internal/sheets/store/handle_open.go
@@ -43,6 +43,10 @@ import (
// calc's evaluator to real ids, which is the more invasive but more
// durable fix. This handle does neither because it needs neither yet.
func buildHandle(db *gorm.DB, sealer *seal.Sealer, tabID uint64) (*Handle, error) {
+ var tab Tab
+ if err := db.Select("id", "content_version").First(&tab, tabID).Error; err != nil {
+ return nil, fmt.Errorf("store: load tab version: %w", err)
+ }
var rows []Row
if err := db.Where("tab_id = ?", tabID).Find(&rows).Error; err != nil {
return nil, fmt.Errorf("store: load rows: %w", err)
@@ -78,7 +82,7 @@ func buildHandle(db *gorm.DB, sealer *seal.Sealer, tabID uint64) (*Handle, error
}
var stale []calc.CellKey
- var maxVersion uint64
+ maxVersion := tab.ContentVersion
for _, c := range cells {
key, ok := h.cellKey(c.RowID, c.ColID)
@@ -125,6 +129,9 @@ func buildHandle(db *gorm.DB, sealer *seal.Sealer, tabID uint64) (*Handle, error
stale = append(stale, key)
}
}
+ if maxVersion >= maxContentVersion-1 {
+ return nil, errVersionExhausted
+ }
h.nextVersion = maxVersion + 1
// Crash recovery: any cell whose CacheVersion trails its InputVersion
diff --git a/internal/sheets/store/handle_report.go b/internal/sheets/store/handle_report.go
index 15111d8..835298f 100644
--- a/internal/sheets/store/handle_report.go
+++ b/internal/sheets/store/handle_report.go
@@ -105,6 +105,9 @@ func (h *Handle) SetReport(rowID, colID uint64, raw string, actor uint64) (Repor
return Report{}, err
}
+ if h.nextVersion >= maxContentVersion {
+ return Report{}, errVersionExhausted
+ }
v := h.nextVersion
prevRaw := h.rawText[key]
diff --git a/internal/sheets/store/handle_set.go b/internal/sheets/store/handle_set.go
index f4a8039..c64abbc 100644
--- a/internal/sheets/store/handle_set.go
+++ b/internal/sheets/store/handle_set.go
@@ -52,6 +52,9 @@ func (h *Handle) setBlank(rowID, colID uint64, key calc.CellKey, v uint64, actor
withinCap := len(touched) > 0 && len(touched) <= calc.MaxDirtyPerPass
if err := h.db.Transaction(func(tx *gorm.DB) error {
+ if err := h.advanceContentVersion(tx, v); err != nil {
+ return err
+ }
if err := tx.Where("tab_id = ? AND row_id = ? AND col_id = ?", h.tabID, rowID, colID).
Delete(&Cell{}).Error; err != nil {
return err
@@ -78,6 +81,7 @@ func (h *Handle) setBlank(rowID, colID uint64, key calc.CellKey, v uint64, actor
}); err != nil {
return nil, false, err
}
+ h.nextVersion++
// Test seam: a crash landing exactly here — transaction 1 durably
// committed, nothing after it has run yet — is what
// TestBlankCrashLeavesDependentsRecoverable simulates. Always nil in
@@ -85,7 +89,6 @@ func (h *Handle) setBlank(rowID, colID uint64, key calc.CellKey, v uint64, actor
if err := afterBlankTransactionOneCommits(); err != nil {
return nil, false, err
}
- h.nextVersion++
delete(h.rawText, key)
delete(h.keyToIDs, key)
h.graph.Clear(key)
@@ -124,6 +127,9 @@ func (h *Handle) setContent(rowID, colID uint64, key calc.CellKey, node calc.Nod
return nil, false, err
}
if err := h.db.Transaction(func(tx *gorm.DB) error {
+ if err := h.advanceContentVersion(tx, v); err != nil {
+ return err
+ }
res := tx.Model(&Cell{}).
Where("tab_id = ? AND row_id = ? AND col_id = ?", h.tabID, rowID, colID).
Updates(map[string]any{
@@ -228,6 +234,9 @@ func (h *Handle) recalcTouched(origin calc.CellKey, includeOrigin bool) (map[cal
// caused it — there was never a single-trigger claim to violate.
func (h *Handle) persistTouched(touched map[calc.CellKey]struct{}, v uint64) error {
return h.db.Transaction(func(tx *gorm.DB) error {
+ if err := h.advanceContentVersion(tx, v); err != nil {
+ return err
+ }
for key := range touched {
ids, ok := h.keyToIDs[key]
if !ok {
diff --git a/internal/sheets/store/handle_version.go b/internal/sheets/store/handle_version.go
new file mode 100644
index 0000000..765d787
--- /dev/null
+++ b/internal/sheets/store/handle_version.go
@@ -0,0 +1,27 @@
+package store
+
+import (
+ "errors"
+ "math"
+
+ "gorm.io/gorm"
+)
+
+const maxContentVersion uint64 = math.MaxInt64
+
+var errVersionExhausted = errors.New("store: content version exhausted")
+
+func (h *Handle) advanceContentVersion(tx *gorm.DB, version uint64) error {
+ if version >= maxContentVersion {
+ return errVersionExhausted
+ }
+ result := tx.Model(&Tab{}).Where("id = ?", h.tabID).
+ Select("content_version").Update("content_version", gorm.Expr("MAX(content_version, ?)", version))
+ if result.Error != nil {
+ return result.Error
+ }
+ if result.RowsAffected != 1 {
+ return gorm.ErrRecordNotFound
+ }
+ return nil
+}
diff --git a/internal/sheets/store/handle_version_export_test.go b/internal/sheets/store/handle_version_export_test.go
new file mode 100644
index 0000000..4ea4ba1
--- /dev/null
+++ b/internal/sheets/store/handle_version_export_test.go
@@ -0,0 +1,14 @@
+package store
+
+import "gorm.io/gorm"
+
+func AdvanceContentVersionForTest(db *gorm.DB, tabID, version uint64) error {
+ h := &Handle{tabID: tabID}
+ return db.Transaction(func(tx *gorm.DB) error { return h.advanceContentVersion(tx, version) })
+}
+
+func SetNextVersionForTest(h *Handle, version uint64) {
+ h.mu.Lock()
+ defer h.mu.Unlock()
+ h.nextVersion = version
+}
diff --git a/internal/sheets/store/handle_version_limits_test.go b/internal/sheets/store/handle_version_limits_test.go
new file mode 100644
index 0000000..b5f006f
--- /dev/null
+++ b/internal/sheets/store/handle_version_limits_test.go
@@ -0,0 +1,184 @@
+package store_test
+
+import (
+ "math"
+ "strconv"
+ "testing"
+
+ "amadan.net/oficina/sheets/internal/sheets/store"
+)
+
+func TestVersionWatermarkNeverLowersAndRequiresTab(t *testing.T) {
+ db, _, id := seeded(t)
+ if err := store.AdvanceContentVersionForTest(db, id, 100); err != nil {
+ t.Fatal(err)
+ }
+ if err := store.AdvanceContentVersionForTest(db, id, 4); err != nil {
+ t.Fatal(err)
+ }
+ var mark uint64
+ if err := db.Raw("SELECT content_version FROM tabs WHERE id = ?", id).Scan(&mark).Error; err != nil {
+ t.Fatal(err)
+ }
+ if mark != 100 {
+ t.Fatalf("watermark lowered to %d", mark)
+ }
+ if err := store.AdvanceContentVersionForTest(db, id+1000, 4); err == nil {
+ t.Fatal("missing tab accepted")
+ }
+}
+
+func TestOpenRequiresTabVersionRow(t *testing.T) {
+ db, s, id := seeded(t)
+ h, err := store.Open(db, s, id+1000)
+ if err == nil {
+ h.Release()
+ t.Fatal("missing tab opened")
+ }
+}
+
+func TestMissingTabRefusesRetainedHandleWrite(t *testing.T) {
+ db, s, id := seeded(t)
+ h, err := store.Open(db, s, id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer h.Release()
+ if err = db.Delete(&store.Tab{}, id).Error; err != nil {
+ t.Fatal(err)
+ }
+ if _, err = h.SetReport(rowA, colA, "9", actor); err == nil {
+ t.Fatal("write accepted without version row")
+ }
+ if h.Value(rowA, colA).Num != 5 {
+ t.Fatal("failed write changed graph")
+ }
+}
+
+func TestHighWatermarkSurvivesEmptyTabOpen(t *testing.T) {
+ db, s, id := seeded(t)
+ if err := db.Where("tab_id = ?", id).Delete(&store.Cell{}).Error; err != nil {
+ t.Fatal(err)
+ }
+ const high = math.MaxInt64 - 10
+ if err := store.AdvanceContentVersionForTest(db, id, high); err != nil {
+ t.Fatal(err)
+ }
+ h, err := store.Open(db, s, id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer h.Release()
+ rep, err := h.SetReport(rowA, colA, "9", actor)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if rep.Version <= high {
+ t.Fatalf("high watermark lost: %d", rep.Version)
+ }
+}
+
+func TestVersionExhaustionRefusesWithoutMutation(t *testing.T) {
+ for _, mark := range []int64{math.MaxInt64, math.MaxInt64 - 1, -1} {
+ t.Run(strconv.FormatInt(mark, 10), func(t *testing.T) {
+ db, s, id := seeded(t)
+ if err := db.Model(&store.Tab{}).Where("id = ?", id).Update("content_version", mark).Error; err != nil {
+ t.Fatal(err)
+ }
+ h, err := store.Open(db, s, id)
+ if err == nil {
+ h.Release()
+ t.Fatal("invalid watermark opened")
+ }
+ })
+ }
+ for _, batch := range []bool{false, true} {
+ t.Run(map[bool]string{false: "single", true: "batch"}[batch], func(t *testing.T) {
+ db, s, id := seeded(t)
+ h, err := store.Open(db, s, id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer h.Release()
+ var before store.Tab
+ if err = db.First(&before, id).Error; err != nil {
+ t.Fatal(err)
+ }
+ store.SetNextVersionForTest(h, math.MaxInt64)
+ if batch {
+ _, err = h.SetBatch([]store.CellWrite{{Row: 0, Col: 0, Raw: "9"}}, actor)
+ } else {
+ _, err = h.SetReport(rowA, colA, "9", actor)
+ }
+ if err == nil {
+ t.Fatal("exhausted allocation accepted")
+ }
+ var after store.Tab
+ if err = db.First(&after, id).Error; err != nil {
+ t.Fatal(err)
+ }
+ if after.ContentVersion != before.ContentVersion || h.Value(rowA, colA).Num != 5 {
+ t.Fatal("exhausted write mutated state")
+ }
+ })
+ }
+}
+
+func TestFormulaRecoveryVersionSurvivesClearingNewestCaches(t *testing.T) {
+ db, s, id := seeded(t)
+ h, err := store.Open(db, s, id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err = h.Set(rowB, colA, "=A1*2", actor); err != nil {
+ t.Fatal(err)
+ }
+ if err = h.Set(rowA, colA, "8", actor); err != nil {
+ t.Fatal(err)
+ }
+ if h.Value(rowB, colA).Num != 16 {
+ t.Fatal("formula did not recalculate")
+ }
+ h.Release()
+ if err = db.Model(&store.Cell{}).Where("tab_id = ? AND row_id = ?", id, rowB).Update("cache_version", 0).Error; err != nil {
+ t.Fatal(err)
+ }
+ h, err = store.Open(db, s, id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ var tab store.Tab
+ if err = db.First(&tab, id).Error; err != nil {
+ t.Fatal(err)
+ }
+ var cell store.Cell
+ if err = db.Where("tab_id = ? AND row_id = ?", id, rowB).First(&cell).Error; err != nil {
+ t.Fatal(err)
+ }
+ if tab.ContentVersion < cell.CacheVersion {
+ t.Fatal("recovery stamp not retained")
+ }
+ if err = h.Set(rowA, colA, "", actor); err != nil {
+ t.Fatal(err)
+ }
+ if h.Value(rowB, colA).Num != 0 {
+ t.Fatal("blank feeder did not recalculate")
+ }
+ last, err := h.SetReport(rowB, colA, "", actor)
+ if err != nil {
+ t.Fatal(err)
+ }
+ h.Release()
+ h, err = store.Open(db, s, id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer h.Release()
+ next, err := h.SetReport(rowA, colA, "9", actor)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if next.Version <= last.Version {
+ t.Fatal("deleting recovered cells lost watermark")
+ }
+}
diff --git a/internal/sheets/store/handle_version_migration_test.go b/internal/sheets/store/handle_version_migration_test.go
new file mode 100644
index 0000000..9887a6e
--- /dev/null
+++ b/internal/sheets/store/handle_version_migration_test.go
@@ -0,0 +1,122 @@
+package store_test
+
+import (
+ "bytes"
+ "context"
+ "path/filepath"
+ "strings"
+ "testing"
+
+ "amadan.net/oficina/sheets/internal/sheets"
+ "amadan.net/oficina/sheets/internal/sheets/calc"
+ "amadan.net/oficina/sheets/internal/sheets/seal"
+ "amadan.net/oficina/sheets/internal/sheets/store"
+ rdb "github.com/carlosframework/rastrillo/db"
+ "github.com/carlosframework/rastrillo/migrate"
+)
+
+func TestVersionWatermarkMigrationPreservesExistingContent(t *testing.T) {
+ d, err := rdb.Open(filepath.Join(t.TempDir(), "old.db"), nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer d.Close()
+ old := &migrate.Set{}
+ for _, m := range sheets.BootSchema.All() {
+ if !strings.HasPrefix(m.ID, "sheets/") || m.ID <= "sheets/0009_alter_formats_ink_intent" {
+ old.Add(m)
+ }
+ }
+ if _, err := migrate.Apply(context.Background(), d, old); err != nil {
+ t.Fatal(err)
+ }
+ s, err := seal.New(make32())
+ if err != nil {
+ t.Fatal(err)
+ }
+ name, err := s.Seal("tab.name", []byte("Original tab"))
+ if err != nil {
+ t.Fatal(err)
+ }
+ raw, err := s.Seal("cell.input", []byte("5"))
+ if err != nil {
+ t.Fatal(err)
+ }
+ cache, err := s.Seal("cell.cached", store.EncodeValueForTest(calc.Value{Kind: calc.KindNum, Num: 5}))
+ if err != nil {
+ t.Fatal(err)
+ }
+ payload, err := s.Seal("event.payload", []byte(`{"r":100,"c":200,"o":"","n":"5"}`))
+ if err != nil {
+ t.Fatal(err)
+ }
+ for _, q := range []struct {
+ sql string
+ args []any
+ }{
+ {"INSERT INTO events (id,tab_id,kind,actor_id,payload_sealed) VALUES (?,?,?,?,?)", []any{1, tabID, "input", actor, payload}},
+ {"INSERT INTO tabs (id,sheet_id,name_sealed,position) VALUES (?,?,?,?)", []any{tabID, 1, name, 0}},
+ {"INSERT INTO rows (id,tab_id,position) VALUES (?,?,?)", []any{rowA, tabID, 0}},
+ {"INSERT INTO cols (id,tab_id,position) VALUES (?,?,?)", []any{colA, tabID, 0}},
+ {"INSERT INTO cells (tab_id,row_id,col_id,input_sealed,cached_sealed,input_version,cache_version) VALUES (?,?,?,?,?,?,?)", []any{tabID, rowA, colA, raw, cache, 700, 700}},
+ } {
+ if err := d.G.Exec(q.sql, q.args...).Error; err != nil {
+ t.Fatal(err)
+ }
+ }
+ if _, err := migrate.Apply(context.Background(), d, sheets.BootSchema); err != nil {
+ t.Fatal(err)
+ }
+ var event store.Event
+ if err := d.G.First(&event, 1).Error; err != nil {
+ t.Fatal(err)
+ }
+ if !bytes.Equal(event.PayloadSealed, payload) || event.ActorID != actor {
+ t.Fatal("migration altered history")
+ }
+ var mark uint64
+ if err := d.G.Raw("SELECT content_version FROM tabs WHERE id = ?", tabID).Scan(&mark).Error; err != nil {
+ t.Fatal(err)
+ }
+ if mark != 0 {
+ t.Fatalf("new watermark default=%d", mark)
+ }
+ var cell store.Cell
+ if err := d.G.First(&cell).Error; err != nil {
+ t.Fatal(err)
+ }
+ if !bytes.Equal(cell.InputSealed, raw) || !bytes.Equal(cell.CachedSealed, cache) {
+ t.Fatal("migration altered encrypted cell")
+ }
+ var tab store.Tab
+ if err := d.G.First(&tab, tabID).Error; err != nil {
+ t.Fatal(err)
+ }
+ if !bytes.Equal(tab.NameSealed, name) {
+ t.Fatal("migration altered encrypted name")
+ }
+ h, err := store.Open(d.G, s, tabID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ clear, err := h.SetReport(rowA, colA, "", actor)
+ if err != nil {
+ t.Fatal(err)
+ }
+ h.Release()
+ if clear.Version <= 700 {
+ t.Fatalf("ignored old cell stamp: %d", clear.Version)
+ }
+ h, err = store.Open(d.G, s, tabID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer h.Release()
+ next, err := h.SetReport(rowA, colA, "6", actor)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if next.Version <= clear.Version {
+ t.Fatalf("post-upgrade versions %d -> %d", clear.Version, next.Version)
+ }
+}
diff --git a/internal/sheets/store/handle_version_watermark_test.go b/internal/sheets/store/handle_version_watermark_test.go
new file mode 100644
index 0000000..012b082
--- /dev/null
+++ b/internal/sheets/store/handle_version_watermark_test.go
@@ -0,0 +1,203 @@
+package store_test
+
+import (
+ "amadan.net/oficina/sheets/internal/sheets/store"
+ "errors"
+ "gorm.io/gorm"
+ "testing"
+)
+
+func TestBatchVersionsSurviveClearingAndReopen(t *testing.T) {
+ db, s, id := seeded(t)
+ h, e := store.Open(db, s, id)
+ if e != nil {
+ t.Fatal(e)
+ }
+ jobs := make([]store.CellWrite, 500)
+ for i := range jobs {
+ jobs[i] = store.CellWrite{Row: i, Col: 1, Raw: "1"}
+ }
+ a, e := h.SetBatch(jobs, actor)
+ if e != nil {
+ t.Fatal(e)
+ }
+ h.Release()
+ h, e = store.Open(db, s, id)
+ if e != nil {
+ t.Fatal(e)
+ }
+ for i := range jobs {
+ jobs[i].Raw = ""
+ }
+ b, e := h.SetBatch(jobs, actor)
+ if e != nil {
+ t.Fatal(e)
+ }
+ h.Release()
+ h, e = store.Open(db, s, id)
+ if e != nil {
+ t.Fatal(e)
+ }
+ defer h.Release()
+ c, e := h.SetBatch([]store.CellWrite{{Row: 0, Col: 1, Raw: "2"}}, actor)
+ if e != nil {
+ t.Fatal(e)
+ }
+ if c.Version <= b.Version {
+ t.Fatalf("version regression: paste=%d clear=%d next edit=%d; browser discards latest edit", a.Version, b.Version, c.Version)
+ }
+}
+
+func TestSingleCellVersionSurvivesLastCellClear(t *testing.T) {
+ db, s, id := seeded(t)
+ h, err := store.Open(db, s, id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ first, err := h.SetReport(rowA, colA, "8", actor)
+ if err != nil {
+ t.Fatal(err)
+ }
+ h.Release()
+ h, err = store.Open(db, s, id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ cleared, err := h.SetReport(rowA, colA, "", actor)
+ if err != nil {
+ t.Fatal(err)
+ }
+ h.Release()
+ h, err = store.Open(db, s, id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer h.Release()
+ next, err := h.SetReport(rowA, colA, "9", actor)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if cleared.Version <= first.Version || next.Version <= cleared.Version {
+ t.Fatalf("versions regressed: first=%d cleared=%d next=%d", first.Version, cleared.Version, next.Version)
+ }
+}
+
+func TestBlankCrashRetainsCommittedVersion(t *testing.T) {
+ db, s, id := seeded(t)
+ h, err := store.Open(db, s, id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ allocated := h.Version()
+ reset := store.InterruptBlankAfterFirstTransaction(func() error { return errors.New("simulated crash") })
+ _, err = h.SetReport(rowA, colA, "", actor)
+ reset()
+ h.Release()
+ if err == nil {
+ t.Fatal("crash injection failed")
+ }
+ h, err = store.Open(db, s, id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer h.Release()
+ next, err := h.SetReport(rowA, colA, "9", actor)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if next.Version <= allocated {
+ t.Fatalf("committed blank stamp=%d next=%d", allocated, next.Version)
+ }
+}
+
+func TestSnapshotVersionSurvivesReopen(t *testing.T) {
+ for _, clear := range []bool{false, true} {
+ t.Run(map[bool]string{false: "populated", true: "cleared"}[clear], func(t *testing.T) {
+ db, s, id := seeded(t)
+ h, err := store.Open(db, s, id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if clear {
+ if err = h.Set(rowA, colA, "", actor); err != nil {
+ t.Fatal(err)
+ }
+ }
+ snapshot := h.Version()
+ h.Release()
+ h, err = store.Open(db, s, id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer h.Release()
+ rep, err := h.SetReport(rowA, colA, "9", actor)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if rep.Version < snapshot {
+ t.Fatalf("snapshot %d exceeds new edit %d", snapshot, rep.Version)
+ }
+ })
+ }
+}
+
+func TestRetainedHandleBatchRollbackKeepsDurableVersion(t *testing.T) {
+ db, s, id := seeded(t)
+ h, err := store.Open(db, s, id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err = h.Set(rowA, colA, "", actor); err != nil {
+ t.Fatal(err)
+ }
+ var before uint64
+ if err = db.Raw("SELECT content_version FROM tabs WHERE id = ?", id).Scan(&before).Error; err != nil {
+ t.Fatal(err)
+ }
+ if h.Version() <= before {
+ t.Fatal("fixture did not retain an unused stamp")
+ }
+ rep, err := h.SetBatch([]store.CellWrite{{Row: 0, Col: 0, Raw: "8"}}, actor)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err = db.Raw("SELECT content_version FROM tabs WHERE id = ?", id).Scan(&before).Error; err != nil {
+ t.Fatal(err)
+ }
+ calls := 0
+ if err = db.Callback().Create().Before("gorm:create").Register("watermark-failure", func(tx *gorm.DB) {
+ if tx.Statement.Table == "events" {
+ calls++
+ if calls == 3 {
+ tx.AddError(errors.New("injected failure"))
+ }
+ }
+ }); err != nil {
+ t.Fatal(err)
+ }
+ _, err = h.SetBatch([]store.CellWrite{{Row: 0, Col: 0, Raw: ""}, {Row: 0, Col: 1, Raw: "9"}}, actor)
+ db.Callback().Create().Remove("watermark-failure")
+ if err == nil {
+ t.Fatal("batch failure not injected")
+ }
+ var after uint64
+ if err = db.Raw("SELECT content_version FROM tabs WHERE id = ?", id).Scan(&after).Error; err != nil {
+ t.Fatal(err)
+ }
+ if after != before {
+ t.Fatalf("watermark survived rollback: %d -> %d", before, after)
+ }
+ h.Release()
+ h, err = store.Open(db, s, id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer h.Release()
+ next, err := h.SetReport(rowA, colA, "10", actor)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if next.Version <= rep.Version {
+ t.Fatalf("last successful %d >= reopened edit %d", rep.Version, next.Version)
+ }
+}
diff --git a/internal/sheets/store/handle_write_batch.go b/internal/sheets/store/handle_write_batch.go
new file mode 100644
index 0000000..f3a2378
--- /dev/null
+++ b/internal/sheets/store/handle_write_batch.go
@@ -0,0 +1,96 @@
+package store
+
+import (
+ "amadan.net/oficina/sheets/internal/sheets/calc"
+ "errors"
+
+ "gorm.io/gorm"
+)
+
+const MaxCellBatch = 500
+
+type CellWrite struct {
+ Row, Col int
+ Raw string
+}
+
+var errCellBatch = errors.New("store: invalid cell batch")
+
+func (h *Handle) SetBatch(writes []CellWrite, actor uint64) (Report, error) {
+ if len(writes) == 0 || len(writes) > MaxCellBatch {
+ return Report{}, errCellBatch
+ }
+ seen := make(map[[2]int]bool, len(writes))
+ for _, w := range writes {
+ pos := [2]int{w.Row, w.Col}
+ if w.Row < 0 || w.Col < 0 || seen[pos] {
+ return Report{}, errCellBatch
+ }
+ seen[pos] = true
+ if _, _, err := contentNode(w.Raw); err != nil {
+ return Report{}, errCellBatch
+ }
+ }
+ h.mu.Lock()
+ defer h.mu.Unlock()
+
+ var staged *Handle
+ var result Report
+ err := h.db.Transaction(func(tx *gorm.DB) error {
+ var err error
+ staged, err = buildHandle(tx, h.sealer, h.tabID)
+ if err != nil {
+ return err
+ }
+ if staged.nextVersion < h.nextVersion {
+ staged.nextVersion = h.nextVersion
+ }
+ changed := make(map[CellCoord]ChangedCell)
+ direct := make(map[CellCoord]bool)
+ for _, w := range writes {
+ r, err := staged.EnsureRow(w.Row)
+ if err != nil {
+ return err
+ }
+ c, err := staged.EnsureCol(w.Col)
+ if err != nil {
+ return err
+ }
+ rep, err := staged.SetReport(r, c, w.Raw, actor)
+ if err != nil {
+ return err
+ }
+ if rep.Capped {
+ return errCellBatch
+ }
+ direct[CellCoord{r, c}] = true
+ result.Version = rep.Version
+ for _, cell := range rep.Cells {
+ changed[CellCoord{cell.RowID, cell.ColID}] = cell
+ }
+ if len(changed) > calc.MaxDirtyPerPass {
+ return errCellBatch
+ }
+ }
+ var count int64
+ if err := tx.Model(&Cell{}).Where("tab_id = ?", h.tabID).Count(&count).Error; err != nil {
+ return err
+ }
+ if count > MaxCellsPerTab {
+ return ErrTabTooLarge
+ }
+ for coord, cell := range changed {
+ cell.Derived = !direct[coord]
+ result.Cells = append(result.Cells, cell)
+ }
+ return nil
+ })
+ if err != nil {
+ return Report{}, err
+ }
+ h.graph, h.rawText, h.keyToIDs = staged.graph, staged.rawText, staged.keyToIDs
+ h.rowPos, h.colPos = staged.rowPos, staged.colPos
+ h.rowByPos, h.colByPos = staged.rowByPos, staged.colByPos
+ h.nextVersion = staged.nextVersion
+ return result, nil
+}
diff --git a/internal/sheets/store/handle_write_batch_test.go b/internal/sheets/store/handle_write_batch_test.go
new file mode 100644
index 0000000..59fec8f
--- /dev/null
+++ b/internal/sheets/store/handle_write_batch_test.go
@@ -0,0 +1,251 @@
+package store_test
+
+import (
+ "amadan.net/oficina/sheets/internal/sheets/seal"
+ "amadan.net/oficina/sheets/internal/sheets/sealtest"
+ "errors"
+ "os"
+ "path/filepath"
+ "strconv"
+ "testing"
+ "time"
+
+ "amadan.net/oficina/sheets/internal/sheets/store"
+ "gorm.io/gorm"
+)
+
+func TestCellBatchFinalValuesAndDirectOrigins(t *testing.T) {
+ h := openSeeded(t)
+ before := h.Version()
+ rep, err := h.SetBatch([]store.CellWrite{{Row: 1, Col: 0, Raw: "=A1*2"}, {Row: 0, Col: 0, Raw: "21"}}, actor)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if rep.Version <= before || h.Version() <= rep.Version {
+ t.Fatalf("versions %d %d %d", before, rep.Version, h.Version())
+ }
+ for _, c := range rep.Cells {
+ if c.Derived {
+ t.Fatalf("edited origin marked derived: %+v", c)
+ }
+ }
+ if got := findCell(t, rep, rowB, colA); got.Value.Num != 42 {
+ t.Fatalf("final formula = %+v", got)
+ }
+ rep, err = h.SetBatch([]store.CellWrite{{Row: 0, Col: 0, Raw: ""}}, actor)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if c := findCell(t, rep, rowB, colA); c.Value.Num != 0 || !c.Derived {
+ t.Fatalf("clear dependent = %+v", c)
+ }
+}
+
+func TestCellBatchRollsBackStorageHistoryAxesAndLiveHandle(t *testing.T) {
+ db, sealer, id := seeded(t)
+ h, err := store.Open(db, sealer, id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer h.Release()
+ other, err := store.Open(db, sealer, id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer other.Release()
+ before := h.Version()
+ var history int64
+ db.Model(&store.Event{}).Count(&history)
+ calls := 0
+ if err := db.Callback().Create().Before("gorm:create").Register("batch-failure", func(tx *gorm.DB) {
+ if tx.Statement.Table == "events" {
+ calls++
+ if calls == 3 {
+ tx.AddError(errors.New("injected failure"))
+ }
+ }
+ }); err != nil {
+ t.Fatal(err)
+ }
+ defer db.Callback().Create().Remove("batch-failure")
+ _, err = h.SetBatch([]store.CellWrite{{Row: 0, Col: 0, Raw: "21"}, {Row: 30, Col: 40, Raw: "99"}}, actor)
+ if err == nil {
+ t.Fatal("injected persistence failure accepted")
+ }
+ if got := other.Value(rowA, colA).Num; got != 5 {
+ t.Fatalf("live graph changed: %v", got)
+ }
+ if h.Version() != before {
+ t.Fatal("failed batch advanced version")
+ }
+ var now int64
+ db.Model(&store.Event{}).Count(&now)
+ if now != history {
+ t.Fatalf("history %d -> %d", history, now)
+ }
+ db.Model(&store.Row{}).Where("tab_id = ? AND position = ?", id, 30).Count(&now)
+ if now != 0 {
+ t.Fatal("row survived rollback")
+ }
+ db.Model(&store.Col{}).Where("tab_id = ? AND position = ?", id, 40).Count(&now)
+ if now != 0 {
+ t.Fatal("column survived rollback")
+ }
+ db.Callback().Create().Remove("batch-failure")
+ rep, err := h.SetBatch([]store.CellWrite{{Row: 0, Col: 0, Raw: "8"}}, actor)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if findCell(t, rep, rowA, colA).Value.Num != 8 || other.Value(rowA, colA).Num != 8 {
+ t.Fatal("live handle failed to recover")
+ }
+}
+
+func TestCellBatchValidatesAllInputsBeforeWriting(t *testing.T) {
+ for _, writes := range [][]store.CellWrite{
+ nil,
+ {{Row: 0, Col: 0, Raw: "7"}, {Row: 30, Col: 20, Raw: "=SUM(("}},
+ {{Row: 0, Col: 0, Raw: "7"}, {Row: 0, Col: 0, Raw: "8"}},
+ {{Row: -1, Col: 0, Raw: "7"}},
+ make([]store.CellWrite, store.MaxCellBatch+1),
+ } {
+ h := openSeeded(t)
+ v := h.Version()
+ if _, err := h.SetBatch(writes, actor); err == nil {
+ t.Fatal("invalid batch accepted")
+ }
+ if h.Value(rowA, colA).Num != 5 || h.Version() != v {
+ t.Fatal("invalid batch changed cell")
+ }
+ }
+}
+
+func TestCellBatchFiveHundredCells(t *testing.T) {
+ h := openSeeded(t)
+ writes := make([]store.CellWrite, store.MaxCellBatch)
+ for i := range writes {
+ writes[i] = store.CellWrite{Row: i, Col: 1, Raw: "=A1*2"}
+ }
+ start := time.Now()
+ rep, err := h.SetBatch(writes, actor)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if len(rep.Cells) != store.MaxCellBatch {
+ t.Fatalf("got %d cells", len(rep.Cells))
+ }
+ for _, c := range rep.Cells {
+ if c.Value.Num != 10 {
+ t.Fatalf("value = %+v", c)
+ }
+ }
+ t.Logf("500-cell atomic batch: %s", time.Since(start))
+}
+
+func TestCellBatchKeepsCellsAndHistorySealed(t *testing.T) {
+ path := filepath.Join(t.TempDir(), "test.db")
+ db := openTestDB(t, path)
+ sealer, err := seal.New(make32())
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := db.Create(&store.Tab{ID: tabID, SheetID: 1}).Error; err != nil {
+ t.Fatal(err)
+ }
+ h, err := store.Open(db, sealer, tabID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer h.Release()
+ if _, err := h.SetBatch([]store.CellWrite{{Row: 0, Col: 0, Raw: string(sealtest.Canaries[0])}, {Row: 1, Col: 0, Raw: "=A1"}}, actor); err != nil {
+ t.Fatal(err)
+ }
+ sealtest.AssertNoPlaintext(t, path)
+}
+
+func TestCellBatchRecoversStaleCacheInsideTransaction(t *testing.T) {
+ db, s, id := seeded(t)
+ h, err := store.Open(db, s, id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer h.Release()
+ if err := h.Set(rowB, colA, "=A1*2", actor); err != nil {
+ t.Fatal(err)
+ }
+ sealed, err := s.Seal("cell.input", []byte("9"))
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := db.Model(&store.Cell{}).Where("tab_id = ? AND row_id = ? AND col_id = ?", id, rowA, colA).Updates(map[string]any{"input_sealed": sealed, "input_version": h.Version() + 1}).Error; err != nil {
+ t.Fatal(err)
+ }
+ done := make(chan error, 1)
+ go func() { _, err := h.SetBatch([]store.CellWrite{{Row: 0, Col: 1, Raw: "=A2"}}, actor); done <- err }()
+ select {
+ case err := <-done:
+ if err != nil {
+ t.Fatal(err)
+ }
+ case <-time.After(3 * time.Second):
+ panic("batch recovery deadlocked")
+ }
+ if h.Value(rowB, colA).Num != 18 {
+ t.Fatal("stale dependent not recovered")
+ }
+}
+
+func TestCellBatchLargeTabTiming(t *testing.T) {
+ raw := os.Getenv("CELL_BATCH_BENCH_N")
+ if raw == "" {
+ t.Skip("set CELL_BATCH_BENCH_N to measure full-tab rebuild")
+ }
+ n, err := strconv.Atoi(raw)
+ if err != nil || n < 501 || n > store.MaxCellsPerTab {
+ t.Fatal("invalid CELL_BATCH_BENCH_N")
+ }
+ db := openTestDB(t, filepath.Join(t.TempDir(), "test.db"))
+ s, err := seal.New(make32())
+ if err != nil {
+ t.Fatal(err)
+ }
+ id := tabID
+ seedBlankRecoveryFixture(t, db, s, id, rowA, colA, n-1)
+ h, err := store.Open(db, s, id)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer h.Release()
+ writes := make([]store.CellWrite, 500)
+ for i := range writes {
+ writes[i] = store.CellWrite{Row: i + 1, Col: 0, Raw: "8"}
+ }
+ start := time.Now()
+ _, err = h.SetBatch(writes, actor)
+ elapsed := time.Since(start)
+ if err != nil {
+ t.Fatal(err)
+ }
+ t.Logf("%d-cell tab, 500 writes with transaction rebuild: %s", n, elapsed)
+}
+
+func TestCellBatchRejectsMergedReportPastCalculationCap(t *testing.T) {
+ db := openTestDB(t, filepath.Join(t.TempDir(), "test.db"))
+ s, err := seal.New(make32())
+ if err != nil {
+ t.Fatal(err)
+ }
+ seedBlankRecoveryFixture(t, db, s, tabID, rowA, colA, 4999)
+ h, err := store.Open(db, s, tabID)
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer h.Release()
+ _, err = h.SetBatch([]store.CellWrite{{Row: 0, Col: 0, Raw: "7"}, {Row: 0, Col: 1, Raw: "8"}}, actor)
+ if err == nil {
+ t.Fatal("oversized merged report accepted")
+ }
+ if h.Value(rowA, colA).Num != 5 {
+ t.Fatal("oversized report committed partial write")
+ }
+}
diff --git a/internal/sheets/store/models.go b/internal/sheets/store/models.go
index 0faed88..9bd5195 100644
--- a/internal/sheets/store/models.go
+++ b/internal/sheets/store/models.go
@@ -52,12 +52,13 @@ type Sheet struct {
// Tab is one sheet within a document — the tabs along the bottom.
type Tab struct {
- ID uint64 `gorm:"primaryKey"`
- SheetID uint64 `gorm:"index"`
- NameSealed []byte // seal purpose "tab.name"
- Position int
- FrozenRows int
- FrozenCols int
+ ID uint64 `gorm:"primaryKey"`
+ SheetID uint64 `gorm:"index"`
+ NameSealed []byte // seal purpose "tab.name"
+ Position int
+ FrozenRows int
+ FrozenCols int
+ ContentVersion uint64 `gorm:"not null;default:0"`
}
// Row and Col carry stable identity. Position is a property of the record,
diff --git a/internal/sheets/theme.go b/internal/sheets/theme.go
index 3d855d4..b7b6030 100644
--- a/internal/sheets/theme.go
+++ b/internal/sheets/theme.go
@@ -60,7 +60,7 @@ func modeFrom(r *http.Request) string {
// This is also the bug this function exists to prevent recurring. The
// attribute used to be handed rastrillo's THEME NAME — day, plain,
// signal — which are the palette files an app picks once at scaffold
-// (`rastrillo doctor` copies one to static/theme.css), not a per-person
+// (this app serves day through Estilo), not a per-person
// setting. Neither toggle rule matched, so an explicit choice never
// beat the OS and the switcher did nothing at all. Theme and scheme are
// orthogonal: this app ships `day` (ruling 7) and every person picks
diff --git a/tools/drive/batch-checks.mjs b/tools/drive/batch-checks.mjs
new file mode 100644
index 0000000..90bfe58
--- /dev/null
+++ b/tools/drive/batch-checks.mjs
@@ -0,0 +1,52 @@
+import assert from 'node:assert/strict';
+
+export async function checkCellQueue(page) {
+ const isBatch = request => request.method() === 'POST' && new URL(request.url()).pathname.endsWith('/cells/batch');
+ const requests = [];
+ const record = request => { if (isBatch(request)) requests.push(request); };
+ const text = Array.from({length: 25}, (_, r) => Array.from({length: 20}, (_, c) => String(r * 20 + c + 1)).join('\t')).join('\n');
+ await page.bringToFront();
+ await page.evaluate(async text => navigator.clipboard.writeText(text), text);
+ await page.locator('[data-sheet-grid] tbody tr:first-child td:nth-child(2)').click();
+ await page.evaluate(() => { window.SheetsGrid.worksheet().updateSelectionFromCoords(0, 0, 0, 0); window.SheetsGrid.focus(); });
+ page.on('request', record);
+ try {
+ const pasted = page.waitForRequest(isBatch);
+ await page.keyboard.press('Control+v');
+ await pasted;
+ await page.evaluate(() => window.SheetsCellQueue([], window.SheetsGrid.post));
+ assert.equal(requests.length, 1, 'paste must send one atomic request');
+ assert.equal(requests[0].postDataJSON().cells.length, 500);
+ await page.waitForFunction(() => window.SheetsGrid.rawAt(24, 19) === '500');
+ } finally {
+ page.off('request', record);
+ }
+
+ assert.equal(await page.evaluate(() => window.SheetsGrid.write(24, 20, '501')), true);
+ await page.evaluate(() => { window.SheetsGrid.worksheet().updateSelectionFromCoords(0, 0, 20, 24); window.SheetsGrid.focus(); });
+ const count = await page.evaluate(() => {
+ let n = 0;
+ for (let r = 0; r < 25; r++) for (let c = 0; c < 21; c++) if (window.SheetsGrid.rawAt(r, c) !== '') n++;
+ return n;
+ });
+ assert.ok(count > 500, 'cut must need at least two chunks');
+ const cuts = [];
+ const refuse = async route => {
+ cuts.push(route.request().postDataJSON());
+ await route.fulfill({status: 500, contentType: 'text/plain', body: ''});
+ };
+ await page.route('**/cells/batch', refuse);
+ try {
+ const attempted = page.waitForRequest(isBatch);
+ await page.keyboard.press('Control+x');
+ await attempted;
+ await page.evaluate(() => window.SheetsCellQueue([], window.SheetsGrid.post));
+ assert.equal(cuts.length, 1, 'failed first chunk must stop the cut queue');
+ assert.equal(cuts[0].cells.length, 500);
+ assert.ok(cuts[0].cells.every(cell => cell.raw === ''));
+ await page.waitForFunction(() => window.SheetsGrid.rawAt(0, 0) === '1' && window.SheetsGrid.rawAt(24, 20) === '501');
+ } finally {
+ await page.unroute('**/cells/batch', refuse);
+ }
+ return {pasteRequests: 1, pasteCells: 500, failedCutRequests: 1, failedCutStopped: true};
+}
diff --git a/tools/drive/canary-batch.mjs b/tools/drive/canary-batch.mjs
new file mode 100644
index 0000000..e37b64f
--- /dev/null
+++ b/tools/drive/canary-batch.mjs
@@ -0,0 +1,75 @@
+import assert from 'node:assert/strict';
+import fs from 'node:fs';
+import { checkCellQueue } from './batch-checks.mjs';
+const pw = await import(process.env.PLAYWRIGHT || 'playwright');
+const base = process.env.SHEETS_CANARY_ORIGIN;
+const state = process.env.SHEETS_CANARY_AUTH_STATE;
+if (base !== 'https://overnight-sheets.bds.oncarlos.com' || !state) throw new Error('Missing HTTPS canary origin or authentication state');
+if ((fs.statSync(state).mode & 0o077) !== 0) throw new Error('Authentication state must be private');
+const expected = process.env.SHEETS_CANARY_VERSION;
+if (!/^[a-f0-9]{7,40}$/.test(expected || '')) throw new Error('Expected candidate version is required');
+const browser = await pw.chromium.launch({headless:true,executablePath:process.env.CHROMIUM_EXECUTABLE_PATH});
+try {
+ const context = await browser.newContext({storageState:state, permissions:['clipboard-read','clipboard-write']});
+ let signinPosted = false;
+ await context.route('**/signin', route => {
+ if (route.request().method() === 'POST') { signinPosted = true; return route.abort(); }
+ return route.continue();
+ });
+ const version = await context.request.get(base+'/api/version');
+ assert.equal((await version.text()).trim(), expected);
+ const page = await context.newPage();
+ const errors=[];page.on('pageerror',e=>errors.push(e.message));
+ await page.goto(base+'/sheets');
+ assert.equal(new URL(page.url()).pathname,'/sheets');
+ const location = await page.evaluate(async()=>{
+ const r=await fetch('/sheets',{method:'POST',body:new URLSearchParams({title:'Atomic batch canary '+Date.now()})});
+ if(!r.ok)throw new Error('Fixture creation failed');return r.url;
+ });
+ await page.goto(location);
+ await page.waitForFunction(()=>window.SheetsGrid?.worksheet());
+ const ids=await page.locator('[data-sheet-surface]').evaluate(el=>({sheet:el.dataset.sheetId,tab:el.dataset.tabId}));
+ const api=`/api/sheets/${ids.sheet}/tabs/${ids.tab}/cells`;
+ const second=await context.newPage();second.on('pageerror',e=>errors.push(e.message));
+ await second.goto(location);await second.waitForFunction(()=>window.SheetsGrid?.worksheet());
+ const requests=[];page.on('request',r=>{if(r.method()==='POST'&&r.url().includes('/cells'))requests.push(r.url());});
+ const text=Array.from({length:25},(_,r)=>Array.from({length:20},(_,c)=>String(r*20+c+1)).join('\t')).join('\n');
+ await page.bringToFront();
+ await page.evaluate(async text=>{await navigator.clipboard.writeText(text);},text);
+ await page.locator('[data-sheet-grid] tbody tr:first-child td:nth-child(2)').click();
+ const start=Date.now();
+ await page.keyboard.press('Control+v');
+ await page.waitForFunction(()=>window.SheetsGrid.rawAt(24,19)==='500');
+ assert.equal(requests.length,1);assert.ok(requests[0].endsWith('/cells/batch'));
+ const pasteMs=Date.now()-start;
+ await second.bringToFront();
+ await second.waitForFunction(()=>window.SheetsGrid.rawAt(24,19)==='500');
+ await page.bringToFront();
+ await page.evaluate(()=>{window.SheetsGrid.worksheet().updateSelectionFromCoords(0,0,19,24);window.SheetsGrid.focus();});
+ await page.keyboard.press('Control+x');
+ await page.waitForFunction(()=>window.SheetsGrid.rawAt(0,0)===''&&window.SheetsGrid.rawAt(24,19)==='');
+ await page.evaluate(async text=>{await navigator.clipboard.writeText(text);},text);
+ await page.locator('[data-sheet-grid] tbody tr:first-child td:nth-child(2)').click();
+ await page.keyboard.press('Control+v');
+ await page.waitForFunction(()=>window.SheetsGrid.rawAt(24,19)==='500');
+ const snap=await page.evaluate(async api=>{const r=await fetch(api);return r.json();},api);
+ assert.equal(snap.cells.length,500);
+ const before=snap.cells.map(c=>[c.r,c.c,c.raw]).sort();
+ const refused=await page.evaluate(async api=>{
+ const r=await fetch(api+'/batch',{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify({cells:[{r:0,c:0,raw:'99999'},{r:24,c:19,raw:'=SUM(('}]})});return r.status;
+ },api);
+ assert.equal(refused,400);
+ const after=await page.evaluate(async api=>{const r=await fetch(api);return r.json();},api);
+ assert.deepEqual(after.cells.map(c=>[c.r,c.c,c.raw]).sort(),before);
+ await page.reload();await page.waitForFunction(()=>window.SheetsGrid?.worksheet()&&window.SheetsGrid.rawAt(24,19)==='500');
+ const nojs=await browser.newContext({storageState:state,javaScriptEnabled:false});
+ const table=await nojs.newPage();await table.goto(location+'/table');
+ assert.equal(await table.locator('a[aria-label="Edit T25"]').innerText(),'500');
+ assert.equal(await table.locator('a[aria-label="Edit A1"]').innerText(),'1');
+ const queue = await checkCellQueue(page);
+ assert.equal(signinPosted, false);
+ assert.deepEqual(errors,[]);
+ const result={origin:base,version:expected,queue,sheet:ids.sheet,tab:ids.tab,pasteCells:500,pasteRequests:1,pasteMs,secondPageObserved:true,clearThenPasteVisible:true,invalidBatchRolledBack:true,reloadPreserved:true,scriptlessTableLoaded:true};
+ if(process.env.SHEETS_CANARY_RESULT)fs.writeFileSync(process.env.SHEETS_CANARY_RESULT,JSON.stringify(result,null,2));
+ console.log(JSON.stringify(result));
+} finally {await browser.close();}
diff --git a/tools/drive/clipboard.mjs b/tools/drive/clipboard.mjs
index 97d58ec..b3e977e 100644
--- a/tools/drive/clipboard.mjs
+++ b/tools/drive/clipboard.mjs
@@ -17,13 +17,14 @@
* SHEETS_SOCK=$D/s.sock SHEETS_LOG=$D/s.log \
* PLAYWRIGHT=/path/to/playwright/index.js node tools/drive/clipboard.mjs
*/
+import { checkCellQueue } from './batch-checks.mjs';
import net from 'node:net';
import fs from 'node:fs';
const pw=await import(process.env.PLAYWRIGHT); const chromium=pw.chromium||(pw.default&&pw.default.chromium);
const proxy=net.createServer(c=>{const u=net.connect(process.env.SHEETS_SOCK);c.pipe(u);u.pipe(c);c.on('error',()=>u.destroy());u.on('error',()=>c.destroy());});
await new Promise(r=>proxy.listen(8799,'127.0.0.1',r));
const base='http://127.0.0.1:8799';
-const browser=await chromium.launch();
+const browser=await chromium.launch({executablePath:process.env.CHROMIUM_EXECUTABLE_PATH});
const ctx=await browser.newContext({viewport:{width:1440,height:900},permissions:['clipboard-read','clipboard-write']});
const page=await ctx.newPage();
const errors=[]; page.on('pageerror',e=>errors.push(String(e))); page.on('console',m=>{if(m.type()==='error')errors.push(m.text());});
@@ -127,6 +128,8 @@ await page.click('[data-cmd="paste"]'); await page.waitForTimeout(900);
// H8 is 7 rows and 6 columns from B1, so A1 becomes G8.
check('menu Paste writes a translated cell', await raw(7,7)==='=G8*2', 'got '+JSON.stringify(await raw(7,7)));
console.log('\n note text:',JSON.stringify(await page.textContent('[data-sheet-note]')));
+try { await checkCellQueue(page); ok('atomic paste and failed cut queue'); }
+catch (err) { check('atomic paste and failed cut queue', false, err.message); }
console.log('ERRORS:',JSON.stringify([...new Set(errors)]));
console.log(fails.length?('\nFAILURES: '+fails.join('; ')):'\nall checks passed');
await browser.close(); proxy.close(); process.exit(fails.length?1:0);