amadan
test: verify real clipboard batching and failure-stop behavior
6 files changed,
+145
−10
docs/superpowers/specs/2026-09-07-atomic-cell-batch.md+5 −3internal/sheets/static/cellqueue.js+4 −0internal/sheets/static/clipboard.js+5 −6tools/drive/batch-checks.mjs+52 −0tools/drive/canary-batch.mjs+75 −0tools/drive/clipboard.mjs+4 −1
CI passed — run details
Intent
Human
READ-ONLY Fable DESIGN review; no edits/deploys/secret reads. Repo /home/paulca/amadan.net/oficina/sheets-core/.claude/worktrees/overnight-versions exact HEAD1aca44c. Read AGENTS.md and docs/superpowers/specs/2026-09-07-canary-fixture-bootstrap.md. Root authorized isolated canary environment overnight-batch at overnight-sheets.bds.oncarlos.com, owners oficina-fixture@example.test, fresh keys provisioned. No SMTP sandbox response; fixture must certify normal persisted-session validation/editor only, never email delivery. Assess startup-only temporary bootstrap with existing BootSchema + single ordinary session hash, exact origin/compiled runID/owner, empty content/member/grant/session DB precondition, exact verified exec-child --socket and --db paths (missing STATE_DIRECTORY not fatal), replacement by unchanged candidate before any browser assertion, exact process SHA and channel proof, normal signout/teardown. Platform current source /home/paulca/amadan.net/carlos/platform if needed; discover ExecBackend.Start/Seed contract read-only, never read secrets.json. Session schema/validation in pinned rastrillo dependency. Candidate contains no auth or fixture backdoor. Bootstrap implementation does not exist yet. Explicit APPROVE or REQUEST CHANGES for design; name minimal missing guards/tests, avoid hypothetical unrelated mechanisms. Root must confirm actual /data/... DB path before building exact guards; native seed alternative is optional and not deployed. This review must not waive later code/test/deployment gates.
Human
READ-ONLY Fable revised SPEC and full ATOMIC code review; no edits/deploys. Exact repo /home/paulca/amadan.net/oficina/sheets-core/.claude/worktrees/overnight-versions HEAD1aca44c. Atomic implementation37c8055 inherits main4aea8f4; durable watermark on top has separate Fable design/code approvals /tmp/sheets-durable-versions-fable-review2.json and /tmp/sheets-watermark-code-review.json. Read AGENTS.md, docs/superpowers/specs/2026-09-07-atomic-cell-batch.md revision2, original REQUEST CHANGES /tmp/sheets-batch-spec-review.json, and all atomic diff4aea8f4..37c8055 (space after diff). Review handlers_cell_batch.go/app route, store/handle_write_batch.go, cellqueue.js/grid.js and their tests; assess against current watermark implementation1aca44c. Original five requests addressed: transaction-scoped recovery, measured50k/200k writer cost, immediate postcommit live swap, merged5000-cell report cap, stale-recovery regression with mutation proof /tmp/sheets-batch-mutation.log. Full current1aca44c make test and exact-head CI passed. Verify batching final formula values/clear/rollback/events, bounds/auth/copy translation, live publication/order and browser queue failure semantics. Explicit verdicts for revised SPEC and ATOMIC CODE. This code inherited a branch is not presumed reviewed. Human-copy gate unaffected (existing fixed strings only); production requires separately-approved reconciled chrome a75c7a9 plus its pendingcopy, and deployed canary browser gate remains pending. Isolated atomic-only canary may certify batch+durable code without changing production after root approves fixture. No unrelated tabs/bootstrap review in this pass. Keep blockers actionable with exact lines.
Attributed to this commit: 8.03 g CO₂e (356564 tokens). This is an attribution, not a measurement — usage counted from this commit's own marker forward to the next one on fix/durable-tab-versions, never backward from it. A turn that produced more than one commit books all of its tokens to the last one, and a concurrent session on this branch can add to the same figure. Some of it was priced using the shared default rate, because no model-specific factor row exists yet for at least one model involved. The Carbon view has the repo total, which is never derived from this attribution.
diff --git a/docs/superpowers/specs/2026-09-07-atomic-cell-batch.md b/docs/superpowers/specs/2026-09-07-atomic-cell-batch.md| index bc4614d..c70b2c0 100644 |
| --- a/docs/superpowers/specs/2026-09-07-atomic-cell-batch.md |
| +++ b/docs/superpowers/specs/2026-09-07-atomic-cell-batch.md |
| @@ -1,15 +1,15 @@ |
| # Atomic cell batch implementation contract |
| -Status: revision 2; Fable re-review pending. Revision 1 reviewed by Claude Fable (claude-fable-5-1): REQUEST CHANGES; answers below. Implements the overnight suite readiness request, under the existing Sheets design and next-three batch-route item. |
| +Status: revision 3; Claude Fable 5.1 (claude-fable-5-1) approved revision 2 design at 1aca44c with these wording amendments, and requested code changes for committed browser queue proof and obsolete comments. Focused code re-review remains pending. Implements the overnight suite readiness request, under the existing Sheets design and next-three batch-route item. |
| - POST /api/sheets/{sheetID}/tabs/{tabID}/cells/batch accepts {cells: [{r,c,raw,fromR?,fromC?}], client?}. Require an editor using openTab, preserving sheet/tab pairing and 404 isolation. |
| - Limit to 500 distinct destinations and 1 MiB body. Validate every coordinate, copy origin, raw length, translated raw length, and formula before any committed writes. Duplicate destinations, empty batches, and trailing JSON are invalid. Reuse existing fixed response strings; add no UI copy. |
| - Hold the live handle mutex while a transaction-scoped independent handle is built from the DB. It shares no mutable graph or maps with the live handle. Set its next version at least to the live handle version. Ensure rows/columns and perform existing SetReport operations through that transaction handle. Existing nested transactions become savepoints; only the outer transaction commits. Reject capped recalculation and over-limit resulting cell count atomically. |
| - Preserve existing per-cell input history and recalculation semantics. Merge reports by coordinate, keeping final values and marking directly edited cells as direct. Return only after outer commit; swap graph/maps/version into the live handle after commit. On failure discard the transaction handle; no partial history, axes, data, or live publication. |
| - One response and SSE publication contain all changed cells and the final version. Keep the current 500-cell paste cap. The existing cell queue submits one batch instead of one request per cell. Larger clear selections retain serial batches of 500; a failed batch must stop the queue. Existing post currently absorbs failures; make it return explicit success for queue control while preserving existing callers. |
| -- No format, row/column semantics, tab lifecycle, formula grammar, or permission changes. Clipboard provenance is a separate next slice; this changes only transmission, leaving clipboard.js untouched. |
| +- No format, row/column semantics, tab lifecycle, formula grammar, or permission changes. Clipboard provenance is a separate next slice; this changes only transmission, no behaviour change to clipboard.js; its batching comments are updated. |
| - Cost: a second tab graph is built per batch. Measure a 500-cell batch and document this bounded first implementation. Avoid adding a graph clone API or redesigning calculation before evidence demands it. |
| -- Tests: dependent formulas and clears, overlapping formulas, encryption/history, invalid late formula, injected persistence failure after earlier writes, axis rollback, shared live handle unchanged on failure, viewer/stranger/cross-sheet isolation, request bounds, one publication, client queue one-request and stop-on-failure behavior. Run make test and exact-head CI. Build with make release, deploy isolated canary, run browser checks, obtain Fable branch review, then coordinate squash merge and production deployment with root. |
| +- Tests: dependent formulas and clears, overlapping formulas, encryption/history, invalid late formula, injected persistence failure after earlier writes, axis rollback, shared live handle unchanged on failure, viewer/stranger/cross-sheet isolation, request bounds, one publication, client queue one-request and stop-on-failure behavior in tools/drive/batch-checks.mjs, called by clipboard.mjs and canary-batch.mjs. These use real editor clipboard actions, intercept a forced first-chunk failure, and await the queue completion barrier rather than assuming a delay means success. Browser checks are a deployed-canary gate, not part of make test. Run make test and exact-head CI. Build with make release, deploy isolated canary, run browser checks, obtain Fable branch review, then coordinate squash merge and production deployment with root. |
| ## Revision 2 review answers |
| @@ -18,3 +18,5 @@ Status: revision 2; Fable re-review pending. Revision 1 reviewed by Claude Fable |
| 3. The first non-error operation after outer Transaction returns is swapping graph and mutable maps into the live handle under its mutex. Report assembly occurs inside the transaction before commit; response formatting happens after the live state is updated. There is no fallible operation between commit and the swap. Failure to encode or deliver the HTTP response does not invalidate the now-current live handle; reconnect/snapshot recovers delivery. |
| 4. A merged report may contain at most calc.MaxDirtyPerPass (5,000) distinct cells. Exceeding either an individual recalc cap or the merged report cap rejects and rolls back the whole batch. This deliberately differs from the single-cell path: the batch success contract guarantees complete final values without depending on the pre-existing resync behavior. No new user copy is introduced; the existing write failure string is reused. Supporting larger dependency fanouts is a later slice with dependable resync and explicit recovery semantics. TestCellBatchRejectsMergedReportPastCalculationCap proves the aggregate limit is applied even when each individual edit is within its cap. |
| 5. Both the implementation and failure-injection tests exist. The live handle is preserved on rollback and updated immediately on success. Local make test and remote CI were green at initial implementation 50914fb; revision 2 requires fresh checks and review. |
| + |
| +Revision 3 records the aggregate request bound explicitly: the 1 MiB body limit may reject a set of individually valid 8 KiB cells; callers must stay within both limits. Batch copy origins must include both coordinates, stricter than the legacy single-cell route. For pilot measurement, a p95 batch latency above 1 second at 50,000 cells or 3 seconds at 200,000 cells triggers the next transaction-duration design review; these are investigation thresholds, not a readiness claim. |
diff --git a/internal/sheets/static/cellqueue.js b/internal/sheets/static/cellqueue.js| index 9c35585..04e3bdb 100644 |
| --- a/internal/sheets/static/cellqueue.js |
| +++ b/internal/sheets/static/cellqueue.js |
| @@ -1,3 +1,7 @@ |
| +/* Send one atomic batch per 500 cells, with chunks and invocations in order. |
| + * Concurrent writes lost paints when older replies were discarded. |
| + * A final refetch still raced with live edits. Serial chunks avoid both; |
| + * stop on the first failed chunk. */ |
| (function () { |
| 'use strict'; |
diff --git a/internal/sheets/static/clipboard.js b/internal/sheets/static/clipboard.js| index 1efb886..0623c09 100644 |
| --- a/internal/sheets/static/clipboard.js |
| +++ b/internal/sheets/static/clipboard.js |
| @@ -13,8 +13,8 @@ |
| * anyway, which is silent data loss that looks fine until a reload. |
| * |
| * So this module takes the keydown AND the three clipboard events at |
| - * the capture phase, and every cell it writes goes through the same |
| - * POST a keystroke uses. tsv.js is the wire format and cellqueue.js |
| + * the capture phase, and every cell it writes goes through the atomic batch |
| + * route. tsv.js is the wire format and cellqueue.js |
| * sends the writes. |
| * |
| * It also carries spec §4.2's translation pass. A pasted formula names |
| @@ -28,8 +28,8 @@ |
| var surface = document.querySelector('[data-sheet-surface]'); |
| if (!surface) return; |
| - /* Sized against SERIAL writes (cellqueue.js), not against the store. |
| - * Raise it when a batch cell route lands, and not before. */ |
| + /* Keep a paste within one atomic batch. Raising this cap needs a |
| + * separate decision about failures across multiple chunks. */ |
| var MAX_CELLS = 500; |
| function ready() { |
| @@ -84,8 +84,7 @@ |
| var TSV = window.SheetsTSV; |
| if (!TSV) return; |
| - /* cellqueue.js sends the writes. See its own comment for why they |
| - * go one at a time. */ |
| + /* cellqueue.js sends atomic batches in order and stops on failure. */ |
| var writeAll = window.SheetsCellQueue; |
| if (!writeAll) return; |
diff --git a/tools/drive/batch-checks.mjs b/tools/drive/batch-checks.mjs| new file mode 100644 |
| index 0000000..90bfe58 |
| --- /dev/null |
| +++ b/tools/drive/batch-checks.mjs |
| @@ -0,0 +1,52 @@ |
| +import assert from 'node:assert/strict'; |
| + |
| +export async function checkCellQueue(page) { |
| + const isBatch = request => request.method() === 'POST' && new URL(request.url()).pathname.endsWith('/cells/batch'); |
| + const requests = []; |
| + const record = request => { if (isBatch(request)) requests.push(request); }; |
| + const text = Array.from({length: 25}, (_, r) => Array.from({length: 20}, (_, c) => String(r * 20 + c + 1)).join('\t')).join('\n'); |
| + await page.bringToFront(); |
| + await page.evaluate(async text => navigator.clipboard.writeText(text), text); |
| + await page.locator('[data-sheet-grid] tbody tr:first-child td:nth-child(2)').click(); |
| + await page.evaluate(() => { window.SheetsGrid.worksheet().updateSelectionFromCoords(0, 0, 0, 0); window.SheetsGrid.focus(); }); |
| + page.on('request', record); |
| + try { |
| + const pasted = page.waitForRequest(isBatch); |
| + await page.keyboard.press('Control+v'); |
| + await pasted; |
| + await page.evaluate(() => window.SheetsCellQueue([], window.SheetsGrid.post)); |
| + assert.equal(requests.length, 1, 'paste must send one atomic request'); |
| + assert.equal(requests[0].postDataJSON().cells.length, 500); |
| + await page.waitForFunction(() => window.SheetsGrid.rawAt(24, 19) === '500'); |
| + } finally { |
| + page.off('request', record); |
| + } |
| + |
| + assert.equal(await page.evaluate(() => window.SheetsGrid.write(24, 20, '501')), true); |
| + await page.evaluate(() => { window.SheetsGrid.worksheet().updateSelectionFromCoords(0, 0, 20, 24); window.SheetsGrid.focus(); }); |
| + const count = await page.evaluate(() => { |
| + let n = 0; |
| + for (let r = 0; r < 25; r++) for (let c = 0; c < 21; c++) if (window.SheetsGrid.rawAt(r, c) !== '') n++; |
| + return n; |
| + }); |
| + assert.ok(count > 500, 'cut must need at least two chunks'); |
| + const cuts = []; |
| + const refuse = async route => { |
| + cuts.push(route.request().postDataJSON()); |
| + await route.fulfill({status: 500, contentType: 'text/plain', body: ''}); |
| + }; |
| + await page.route('**/cells/batch', refuse); |
| + try { |
| + const attempted = page.waitForRequest(isBatch); |
| + await page.keyboard.press('Control+x'); |
| + await attempted; |
| + await page.evaluate(() => window.SheetsCellQueue([], window.SheetsGrid.post)); |
| + assert.equal(cuts.length, 1, 'failed first chunk must stop the cut queue'); |
| + assert.equal(cuts[0].cells.length, 500); |
| + assert.ok(cuts[0].cells.every(cell => cell.raw === '')); |
| + await page.waitForFunction(() => window.SheetsGrid.rawAt(0, 0) === '1' && window.SheetsGrid.rawAt(24, 20) === '501'); |
| + } finally { |
| + await page.unroute('**/cells/batch', refuse); |
| + } |
| + return {pasteRequests: 1, pasteCells: 500, failedCutRequests: 1, failedCutStopped: true}; |
| +} |
diff --git a/tools/drive/canary-batch.mjs b/tools/drive/canary-batch.mjs| new file mode 100644 |
| index 0000000..e37b64f |
| --- /dev/null |
| +++ b/tools/drive/canary-batch.mjs |
| @@ -0,0 +1,75 @@ |
| +import assert from 'node:assert/strict'; |
| +import fs from 'node:fs'; |
| +import { checkCellQueue } from './batch-checks.mjs'; |
| +const pw = await import(process.env.PLAYWRIGHT || 'playwright'); |
| +const base = process.env.SHEETS_CANARY_ORIGIN; |
| +const state = process.env.SHEETS_CANARY_AUTH_STATE; |
| +if (base !== 'https://overnight-sheets.bds.oncarlos.com' || !state) throw new Error('Missing HTTPS canary origin or authentication state'); |
| +if ((fs.statSync(state).mode & 0o077) !== 0) throw new Error('Authentication state must be private'); |
| +const expected = process.env.SHEETS_CANARY_VERSION; |
| +if (!/^[a-f0-9]{7,40}$/.test(expected || '')) throw new Error('Expected candidate version is required'); |
| +const browser = await pw.chromium.launch({headless:true,executablePath:process.env.CHROMIUM_EXECUTABLE_PATH}); |
| +try { |
| + const context = await browser.newContext({storageState:state, permissions:['clipboard-read','clipboard-write']}); |
| + let signinPosted = false; |
| + await context.route('**/signin', route => { |
| + if (route.request().method() === 'POST') { signinPosted = true; return route.abort(); } |
| + return route.continue(); |
| + }); |
| + const version = await context.request.get(base+'/api/version'); |
| + assert.equal((await version.text()).trim(), expected); |
| + const page = await context.newPage(); |
| + const errors=[];page.on('pageerror',e=>errors.push(e.message)); |
| + await page.goto(base+'/sheets'); |
| + assert.equal(new URL(page.url()).pathname,'/sheets'); |
| + const location = await page.evaluate(async()=>{ |
| + const r=await fetch('/sheets',{method:'POST',body:new URLSearchParams({title:'Atomic batch canary '+Date.now()})}); |
| + if(!r.ok)throw new Error('Fixture creation failed');return r.url; |
| + }); |
| + await page.goto(location); |
| + await page.waitForFunction(()=>window.SheetsGrid?.worksheet()); |
| + const ids=await page.locator('[data-sheet-surface]').evaluate(el=>({sheet:el.dataset.sheetId,tab:el.dataset.tabId})); |
| + const api=`/api/sheets/${ids.sheet}/tabs/${ids.tab}/cells`; |
| + const second=await context.newPage();second.on('pageerror',e=>errors.push(e.message)); |
| + await second.goto(location);await second.waitForFunction(()=>window.SheetsGrid?.worksheet()); |
| + const requests=[];page.on('request',r=>{if(r.method()==='POST'&&r.url().includes('/cells'))requests.push(r.url());}); |
| + const text=Array.from({length:25},(_,r)=>Array.from({length:20},(_,c)=>String(r*20+c+1)).join('\t')).join('\n'); |
| + await page.bringToFront(); |
| + await page.evaluate(async text=>{await navigator.clipboard.writeText(text);},text); |
| + await page.locator('[data-sheet-grid] tbody tr:first-child td:nth-child(2)').click(); |
| + const start=Date.now(); |
| + await page.keyboard.press('Control+v'); |
| + await page.waitForFunction(()=>window.SheetsGrid.rawAt(24,19)==='500'); |
| + assert.equal(requests.length,1);assert.ok(requests[0].endsWith('/cells/batch')); |
| + const pasteMs=Date.now()-start; |
| + await second.bringToFront(); |
| + await second.waitForFunction(()=>window.SheetsGrid.rawAt(24,19)==='500'); |
| + await page.bringToFront(); |
| + await page.evaluate(()=>{window.SheetsGrid.worksheet().updateSelectionFromCoords(0,0,19,24);window.SheetsGrid.focus();}); |
| + await page.keyboard.press('Control+x'); |
| + await page.waitForFunction(()=>window.SheetsGrid.rawAt(0,0)===''&&window.SheetsGrid.rawAt(24,19)===''); |
| + await page.evaluate(async text=>{await navigator.clipboard.writeText(text);},text); |
| + await page.locator('[data-sheet-grid] tbody tr:first-child td:nth-child(2)').click(); |
| + await page.keyboard.press('Control+v'); |
| + await page.waitForFunction(()=>window.SheetsGrid.rawAt(24,19)==='500'); |
| + const snap=await page.evaluate(async api=>{const r=await fetch(api);return r.json();},api); |
| + assert.equal(snap.cells.length,500); |
| + const before=snap.cells.map(c=>[c.r,c.c,c.raw]).sort(); |
| + const refused=await page.evaluate(async api=>{ |
| + const r=await fetch(api+'/batch',{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify({cells:[{r:0,c:0,raw:'99999'},{r:24,c:19,raw:'=SUM(('}]})});return r.status; |
| + },api); |
| + assert.equal(refused,400); |
| + const after=await page.evaluate(async api=>{const r=await fetch(api);return r.json();},api); |
| + assert.deepEqual(after.cells.map(c=>[c.r,c.c,c.raw]).sort(),before); |
| + await page.reload();await page.waitForFunction(()=>window.SheetsGrid?.worksheet()&&window.SheetsGrid.rawAt(24,19)==='500'); |
| + const nojs=await browser.newContext({storageState:state,javaScriptEnabled:false}); |
| + const table=await nojs.newPage();await table.goto(location+'/table'); |
| + assert.equal(await table.locator('a[aria-label="Edit T25"]').innerText(),'500'); |
| + assert.equal(await table.locator('a[aria-label="Edit A1"]').innerText(),'1'); |
| + const queue = await checkCellQueue(page); |
| + assert.equal(signinPosted, false); |
| + assert.deepEqual(errors,[]); |
| + const result={origin:base,version:expected,queue,sheet:ids.sheet,tab:ids.tab,pasteCells:500,pasteRequests:1,pasteMs,secondPageObserved:true,clearThenPasteVisible:true,invalidBatchRolledBack:true,reloadPreserved:true,scriptlessTableLoaded:true}; |
| + if(process.env.SHEETS_CANARY_RESULT)fs.writeFileSync(process.env.SHEETS_CANARY_RESULT,JSON.stringify(result,null,2)); |
| + console.log(JSON.stringify(result)); |
| +} finally {await browser.close();} |
diff --git a/tools/drive/clipboard.mjs b/tools/drive/clipboard.mjs| index 61d8f16..c7cc6fd 100644 |
| --- a/tools/drive/clipboard.mjs |
| +++ b/tools/drive/clipboard.mjs |
| @@ -17,13 +17,14 @@ |
| * SHEETS_SOCK=$D/s.sock SHEETS_LOG=$D/s.log \ |
| * PLAYWRIGHT=/path/to/playwright/index.js node tools/drive/clipboard.mjs |
| */ |
| +import { checkCellQueue } from './batch-checks.mjs'; |
| import net from 'node:net'; |
| import fs from 'node:fs'; |
| const pw=await import(process.env.PLAYWRIGHT); const chromium=pw.chromium||(pw.default&&pw.default.chromium); |
| const proxy=net.createServer(c=>{const u=net.connect(process.env.SHEETS_SOCK);c.pipe(u);u.pipe(c);c.on('error',()=>u.destroy());u.on('error',()=>c.destroy());}); |
| await new Promise(r=>proxy.listen(8799,'127.0.0.1',r)); |
| const base='http://127.0.0.1:8799'; |
| -const browser=await chromium.launch(); |
| +const browser=await chromium.launch({executablePath:process.env.CHROMIUM_EXECUTABLE_PATH}); |
| const ctx=await browser.newContext({viewport:{width:1440,height:900},permissions:['clipboard-read','clipboard-write']}); |
| const page=await ctx.newPage(); |
| const errors=[]; page.on('pageerror',e=>errors.push(String(e))); page.on('console',m=>{if(m.type()==='error')errors.push(m.text());}); |
| @@ -123,6 +124,8 @@ await page.click('[data-cmd="paste"]'); await page.waitForTimeout(900); |
| // H8 is 7 rows and 6 columns from B1, so A1 becomes G8. |
| check('menu Paste writes a translated cell', await raw(7,7)==='=G8*2', 'got '+JSON.stringify(await raw(7,7))); |
| console.log('\n note text:',JSON.stringify(await page.textContent('[data-sheet-note]'))); |
| +try { await checkCellQueue(page); ok('atomic paste and failed cut queue'); } |
| +catch (err) { check('atomic paste and failed cut queue', false, err.message); } |
| console.log('ERRORS:',JSON.stringify([...new Set(errors)])); |
| console.log(fails.length?('\nFAILURES: '+fails.join('; ')):'\nall checks passed'); |
| await browser.close(); proxy.close(); process.exit(fails.length?1:0); |