amadan
The toolbar checks get their own group, and the header stops lying
Two follow-ups the last fix earned, both named by the session that found the regressions. THE TOOLBAR SECTION IS A GROUP. It was inline in run() and throws deliberately in two places — there is no point continuing without a selection — so when the ambiguous selector threw there, the run reported 5/6: one failure, and 38 checks simply absent rather than failed or skipped. The summary read like a smaller suite instead of like missing coverage. That is the same "cannot tell no from I can't see it" shape the group() helper was added for, and the comment at the call site had already named extracting it as the follow-up, on the reasoning that moving a hundred working checks while fixing them is how a harness stops being trustworthy. It was right, and then it demonstrated itself. Moved on its own, with no change to any check; 44/44 before and after. THE SCRIPT HEADER SAID CHROMIUM "CANNOT START AT ALL". It cannot start inside the sandbox, which is true and is not the same sentence. That paragraph is the root cause of everything the previous commit fixed: I read it, believed it, never tried, and merged a branch that described itself as impossible to browser-verify — with a real SC 1.4.10 failure and four blinded checks in it. Three sessions repeated the claim to each other; the paste session has since said plainly that it relayed the header without checking it either. It now says the limitation is the sandbox and that turning the sandbox off gets you a browser, in a sentence that cannot be skimmed. It no longer quotes one error as the diagnosis — the symptom has already moved once, from a refused socket to a user-data-directory failure, and quoting either as the reason is part of what made the old text read as settled. The open question for CI is correspondingly smaller: not "can this run here" but "may a gate step run unsandboxed", which belongs to whoever owns the runner.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2 files changed,
+52
−20
fixtures/browser-drive.mjs+26 −8scripts/browser-drive.sh+26 −12
CI passed — run details
diff --git a/fixtures/browser-drive.mjs b/fixtures/browser-drive.mjs| index 65e8815..5d40205 100644 |
| --- a/fixtures/browser-drive.mjs |
| +++ b/fixtures/browser-drive.mjs |
| @@ -310,6 +310,20 @@ async function run() { |
| await send('Page.navigate', { url: base + href }) |
| await sleep(2500) |
| +// editorChecks is the toolbar and editing section: it mounted, you |
| +// can type, the buttons and selects act on a real selection, and undo |
| +// undoes. |
| +// |
| +// EXTRACTED INTO A GROUP, which is the follow-up the comment at the |
| +// call site used to describe and defer. It throws deliberately in two |
| +// places — there is no point continuing without a selection — and |
| +// while it was inline in run() those aborts took every group after it |
| +// down with them. That cost was paid for real: a selector that had |
| +// stopped being unique threw here, and the run reported 5/6 rather |
| +// than one failure plus 38 other checks. The output read like a |
| +// smaller suite instead of like missing coverage, which is the exact |
| +// shape this file exists to refuse. |
| +async function editorChecks() { |
| check('the editor mounted', await evaluate(`!!document.querySelector('[docs-editor] p')`)) |
| check('the formatting bar rendered', await evaluate(`!!document.querySelector('[docs-toolbar]')`)) |
| check( |
| @@ -432,6 +446,10 @@ async function run() { |
| } catch (err) { |
| check('undo reverses the last change', false, err.message) |
| } |
| +} |
| + |
| + |
| + await group('the editor and toolbar checks', editorChecks) |
| // Each of these is wrapped so that a failure inside one cannot take |
| // the checks after it down with it. Proven necessary rather than |
| @@ -443,14 +461,14 @@ async function run() { |
| // tell no from I can't see it" shape this file exists to avoid, so |
| // an aborted group is now itself a failed check that names the group. |
| // |
| - // NOT yet true of the toolbar section above, which is inline in this |
| - // function and throws deliberately in two places ("could not select |
| - // a word to format"). Those aborts are correct in themselves — there |
| - // is no point continuing without a selection — but they still cost |
| - // every group below, and the fix is to extract that section the way |
| - // these three are extracted. Left as a follow-up rather than done |
| - // here, because moving a hundred lines of working checks at the same |
| - // time as fixing them is how a harness stops being trustworthy. |
| + // The toolbar section is a group too now. It used to be inline here |
| + // and this comment said extracting it was a follow-up, on the |
| + // reasoning that moving a hundred working checks while fixing them is |
| + // how a harness stops being trustworthy. That was right, and the |
| + // follow-up then demonstrated itself: an ambiguous selector threw |
| + // inside that section and the run reported 5/6 — one failure, and 38 |
| + // checks simply absent rather than failed or skipped. Moved on its |
| + // own, with no change to the checks themselves. |
| await group('the caret-placement checks', caretChecks) |
| await group('the jump-to palette checks', paletteChecks) |
| await group('the 320px reflow checks', reflowChecks) |
diff --git a/scripts/browser-drive.sh b/scripts/browser-drive.sh| index 5e54006..8c5154f 100755 |
| --- a/scripts/browser-drive.sh |
| +++ b/scripts/browser-drive.sh |
| @@ -6,19 +6,33 @@ |
| # Not in .amadan/ci.d/10-gate yet, deliberately, and the reason is |
| # bigger than "the runner needs a Chrome installed". |
| # |
| -# Chromium cannot start at all inside the agent sandbox this repo's |
| -# sessions run under: its process-singleton socket is refused |
| -# ("socket() failed: Operation not permitted") and ~/.config/chromium is |
| -# read-only. So putting this in the gate is not a question of installing |
| -# a package, it is a question of the gate step running with the sandbox |
| -# off — which is a real decision about what CI is allowed to do, and |
| -# belongs to whoever owns the runner rather than to the branch that |
| -# wrote the script. (Confirmed independently by the live-sync session, |
| -# which hit the same wall building its own two-tab check.) |
| +# Chromium does not start inside the agent sandbox as it is configured |
| +# by default — it fails early on something it cannot write or bind. |
| +# The exact symptom moves: it has been "socket() failed: Operation not |
| +# permitted" and it has been "Failed to create a unique user data |
| +# directory for headless". Neither is quoted here as the diagnosis, |
| +# because quoting one symptom as though it were the reason is part of |
| +# what made the old version of this paragraph read as final. |
| # |
| -# Until then it is run by hand, which is still worth far more than not |
| -# having it, as its first run showed (see fixtures/browser-drive.mjs's |
| -# own header). |
| +# BUT IT RUNS FINE WITH THE SANDBOX OFF, and this header said otherwise |
| +# for long enough to do damage. It read "cannot start at all", full |
| +# stop — and a session that believed it merged a branch describing |
| +# itself as impossible to browser-verify, having never tried. A real |
| +# WCAG 2.2 SC 1.4.10 failure and four blinded checks went to main behind |
| +# that sentence. The sandbox is a setting, not a property of the |
| +# machine; a session that needs a browser turns it off for this one |
| +# command and gets one. |
| +# |
| +# So the open question for the gate is smaller than this file used to |
| +# claim. Not "can this run here" — it can — but "may a CI step run |
| +# unsandboxed", which is a real decision about what CI is allowed to do |
| +# and belongs to whoever owns the runner rather than to the branch that |
| +# wrote the script. |
| +# |
| +# Until that is answered it is run by hand, which is worth far more than |
| +# not having it: its first run caught an editor that silently discarded |
| +# every edit, and the run after this file's own header was corrected |
| +# caught five regressions a green gate had passed. |
| # |
| # $DOCS_CHROME overrides which browser to use. |
| set -euo pipefail |