Every suite shard under a minute: the long drives in parts, 52 shards
The suite ran as 15 shards, the slowest five minutes, because single tests ran for minutes: the narrow height drive 290s, the HTML tab's renders 107s, the frame axe sweep 87s, the wide height drive 84s, the floors drive 62s. Each is now dealt into top-level tests a slice can hold, every assertion kept: the height drives by (locale, page) into 3 and 12 parts, the axe sweeps by leg and by theme, the HTML tab by language and width, the floors drive into the phone's leg and the two desktop legs. A check the whole drive made once is made where what it checks is read, or in every part, which is only stricter; the floors drive now looks for each idiom where the inventory says it renders. Under FIESTA_SWEEP=full a height drive is one part again, so the table is whole, and make heights-sweep prints it from both drives at once. The untagged gallery, cmd/fiesta and www packages leave root as test roles, the browser packages are sliced by weights measured today (the check, gallery and www browser tests and the three untagged packages), and the suite is 52 roles. Each shard now prints its wall and CPU time. A runner clones one commit deep with no tags, so gitleaks scanned one commit and the release tests compared nothing. make history, a step before the suite, fetches the whole history and the release tags into a shallow clone (a third of a second for Fiesta), gitleaks refuses a shallow clone, and FIESTA_TEST_REQUIRE_TAGS fails the release tests when git sees no tag for a release.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
12 files changed,
+681
−212
.amadan/ci.d/40-history+2 −0.amadan/ci.d/{50-suite@15 => 50-suite@52}+0 −0Makefile+77 −15check/coverage_browser_test.go+41 −13hack/gotest.sh+10 −4hack/suite.sh+11 −1hack/test-weights.txt+167 −51internal/gallery/a11y_browser_test.go+28 −13internal/gallery/code_browser_test.go+38 −4internal/gallery/heights_browser_test.go+249 −107internal/gallery/pages_a11y_browser_test.go+21 −4releases_test.go+37 −0
CI passed — run details
CI log
=== gofmt ===
=== no-rastrillo ===
mkdir -p "/var/lib/amadan-runner/w4/repos/rastrillo__fiesta/wt/gallery/.build/tmp"
go test -run '^(TestFiestaNeverImportsRastrillo|TestThePublicPackagesLinkNoBrowser)$' -count=1 .
go: downloading go1.26.10 (linux/arm64)
ok amadan.net/rastrillo/fiesta 0.721s
=== size ===
go test ./internal/build -run '^TestTheOneLinkIsUnder25KiBGzipped$' -count=1 -v
=== RUN TestTheOneLinkIsUnder25KiBGzipped
build_test.go:138: fiesta-plain.css: 152856 bytes, 22303 bytes gzipped (21.8 KiB of the 25.0 KiB limit)
build_test.go:138: fiesta-signal.css: 152951 bytes, 22414 bytes gzipped (21.9 KiB of the 25.0 KiB limit)
build_test.go:138: fiesta.css: 155770 bytes, 23018 bytes gzipped (22.5 KiB of the 25.0 KiB limit)
--- PASS: TestTheOneLinkIsUnder25KiBGzipped (0.03s)
PASS
ok amadan.net/rastrillo/fiesta/internal/build 0.031s
=== history ===
history: a shallow clone; fetching the whole history and the release tags
history: 25 commits; release tags: v1.0.0
=== suite ===
--- shard 0 of 52 ---
==> shard 0 of 52: make root
go build ./...
go vet ./...
FIESTA_TEST_REQUIRE_NODE=1 FIESTA_TEST_REQUIRE_GIT=1 FIESTA_TEST_REQUIRE_TAGS=1 ./hack/gotest.sh root 'internal/gallery@3 cmd/fiesta@2 internal/www@2'
ok amadan.net/rastrillo/fiesta 2.311s
ok amadan.net/rastrillo/fiesta/check 15.956s
ok amadan.net/rastrillo/fiesta/example 0.157s
? amadan.net/rastrillo/fiesta/hack/seed/english [no test files]
ok amadan.net/rastrillo/fiesta/internal/axe 0.100s
ok amadan.net/rastrillo/fiesta/internal/build 2.076s
ok amadan.net/rastrillo/fiesta/internal/codeview 0.004s
? amadan.net/rastrillo/fiesta/internal/harness [no test files]
ok amadan.net/rastrillo/fiesta/internal/markup 0.005s
ok amadan.net/rastrillo/fiesta/internal/nodetest 0.476s
ok amadan.net/rastrillo/fiesta/internal/site 7.036s
ok amadan.net/rastrillo/fiesta/internal/tokens 0.006s
suite: root took 88s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m20.900000s 0m4.340000s
--- shard 1 of 52 ---
==> shard 1 of 52: make test.internal.gallery~1of3
FIESTA_TEST_REQUIRE_NODE=1 FIESTA_TEST_REQUIRE_GIT=1 FIESTA_TEST_REQUIRE_TAGS=1 ./hack/gotest.sh test 'internal.gallery~1of3'
gotest: slice 1 of 3 of ./internal/gallery/ runs 33 of 102 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 127.153s
suite: test.internal.gallery~1of3 took 166s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m55.750000s 0m2.400000s
--- shard 2 of 52 ---
==> shard 2 of 52: make test.internal.gallery~2of3
FIESTA_TEST_REQUIRE_NODE=1 FIESTA_TEST_REQUIRE_GIT=1 FIESTA_TEST_REQUIRE_TAGS=1 ./hack/gotest.sh test 'internal.gallery~2of3'
gotest: slice 2 of 3 of ./internal/gallery/ runs 35 of 102 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 111.338s
suite: test.internal.gallery~2of3 took 153s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m47.600000s 0m1.950000s
--- shard 3 of 52 ---
==> shard 3 of 52: make test.internal.gallery~3of3
FIESTA_TEST_REQUIRE_NODE=1 FIESTA_TEST_REQUIRE_GIT=1 FIESTA_TEST_REQUIRE_TAGS=1 ./hack/gotest.sh test 'internal.gallery~3of3'
gotest: slice 3 of 3 of ./internal/gallery/ runs 34 of 102 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 97.621s
suite: test.internal.gallery~3of3 took 139s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m36.140000s 0m1.560000s
--- shard 4 of 52 ---
==> shard 4 of 52: make test.cmd.fiesta~1of2
FIESTA_TEST_REQUIRE_NODE=1 FIESTA_TEST_REQUIRE_GIT=1 FIESTA_TEST_REQUIRE_TAGS=1 ./hack/gotest.sh test 'cmd.fiesta~1of2'
gotest: slice 1 of 2 of ./cmd/fiesta/ runs 7 of 12 tests
ok amadan.net/rastrillo/fiesta/cmd/fiesta 102.005s
suite: test.cmd.fiesta~1of2 took 144s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m36.040000s 0m2.900000s
--- shard 5 of 52 ---
==> shard 5 of 52: make test.cmd.fiesta~2of2
FIESTA_TEST_REQUIRE_NODE=1 FIESTA_TEST_REQUIRE_GIT=1 FIESTA_TEST_REQUIRE_TAGS=1 ./hack/gotest.sh test 'cmd.fiesta~2of2'
gotest: slice 2 of 2 of ./cmd/fiesta/ runs 5 of 12 tests
ok amadan.net/rastrillo/fiesta/cmd/fiesta 107.719s
suite: test.cmd.fiesta~2of2 took 146s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m37.350000s 0m2.910000s
--- shard 6 of 52 ---
==> shard 6 of 52: make test.internal.www~1of2
FIESTA_TEST_REQUIRE_NODE=1 FIESTA_TEST_REQUIRE_GIT=1 FIESTA_TEST_REQUIRE_TAGS=1 ./hack/gotest.sh test 'internal.www~1of2'
gotest: slice 1 of 2 of ./internal/www/ runs 8 of 18 tests
ok amadan.net/rastrillo/fiesta/internal/www 84.936s
suite: test.internal.www~1of2 took 146s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m46.330000s 0m3.630000s
--- shard 7 of 52 ---
==> shard 7 of 52: make test.internal.www~2of2
FIESTA_TEST_REQUIRE_NODE=1 FIESTA_TEST_REQUIRE_GIT=1 FIESTA_TEST_REQUIRE_TAGS=1 ./hack/gotest.sh test 'internal.www~2of2'
gotest: slice 2 of 2 of ./internal/www/ runs 10 of 18 tests
ok amadan.net/rastrillo/fiesta/internal/www 84.433s
suite: test.internal.www~2of2 took 141s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m40.490000s 0m2.860000s
--- shard 8 of 52 ---
==> shard 8 of 52: make staticcheck
go run honnef.co/go/tools/cmd/staticcheck@v0.8.2 -tags browser ./...
go: downloading honnef.co/go/tools v0.8.2
go: downloading golang.org/x/tools v0.51.0
go: downloading github.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c
go: downloading golang.org/x/exp/typeparams v0.0.0-20231108232855-2478ac86f678
go: downloading golang.org/x/exp v0.0.0-20231110203233-9a3e6036ecaa
suite: staticcheck took 91s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m39.710000s 0m6.140000s
--- shard 9 of 52 ---
==> shard 9 of 52: make govulncheck
go run golang.org/x/vuln/cmd/govulncheck@v1.8.0 ./...
go: downloading golang.org/x/vuln v1.8.0
go: downloading golang.org/x/telemetry v0.0.0-20260908163034-4bcc4b2ee518
go: downloading golang.org/x/mod v0.41.0
go: downloading golang.org/x/tools v0.50.0
go: downloading golang.org/x/sync v0.23.0
=== Symbol Results ===
No vulnerabilities found.
Your code is affected by 0 vulnerabilities.
This scan also found 2 vulnerabilities in packages you import and 5
vulnerabilities in modules you require, but your code doesn't appear to call
these vulnerabilities.
Use '-show verbose' for more details.
suite: govulncheck took 74s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m20.020000s 0m2.750000s
--- shard 10 of 52 ---
==> shard 10 of 52: make gitleaks
go run github.com/zricethezav/gitleaks/v8@v8.30.1 git --no-banner --redact --log-opts='--full-history --diff-filter=tuxdb HEAD' .
go: downloading github.com/zricethezav/gitleaks/v8 v8.30.1
go: downloading github.com/Masterminds/sprig/v3 v3.3.0
go: downloading github.com/charmbracelet/lipgloss v0.5.0
go: downloading github.com/fatih/semgroup v1.2.0
go: downloading github.com/gitleaks/go-gitdiff v0.9.1
go: downloading github.com/h2non/filetype v1.1.3
go: downloading github.com/mholt/archives v0.1.2
go: downloading github.com/spf13/viper v1.19.0
go: downloading github.com/BobuSumisu/aho-corasick v1.0.3
go: downloading github.com/hashicorp/go-version v1.7.0
go: downloading golang.org/x/exp v0.0.0-20250218142911-aa4b98e5adaa
go: downloading github.com/spf13/cobra v1.9.1
go: downloading github.com/rs/zerolog v1.33.0
go: downloading golang.org/x/sync v0.11.0
go: downloading github.com/muesli/termenv v0.15.1
go: downloading github.com/lucasb-eyer/go-colorful v1.2.0
go: downloading github.com/mattn/go-runewidth v0.0.14
go: downloading github.com/muesli/reflow v0.2.1-0.20210115123740-9e1d0d53df68
go: downloading gopkg.in/yaml.v3 v3.0.1
go: downloading github.com/fsnotify/fsnotify v1.8.0
go: downloading github.com/mitchellh/mapstructure v1.5.0
go: downloading github.com/sagikazarmark/slog-shim v0.1.0
go: downloading github.com/spf13/afero v1.12.0
go: downloading github.com/spf13/pflag v1.0.6
go: downloading github.com/spf13/cast v1.7.1
go: downloading github.com/magiconair/properties v1.8.9
go: downloading gopkg.in/ini.v1 v1.67.0
go: downloading github.com/subosito/gotenv v1.6.0
go: downloading github.com/pelletier/go-toml/v2 v2.2.3
go: downloading github.com/hashicorp/hcl v1.0.0
go: downloading github.com/Masterminds/semver/v3 v3.3.0
go: downloading dario.cat/mergo v1.0.1
go: downloading github.com/google/uuid v1.6.0
go: downloading github.com/Masterminds/goutils v1.1.1
go: downloading github.com/huandu/xstrings v1.5.0
go: downloading github.com/mitchellh/copystructure v1.2.0
go: downloading github.com/shopspring/decimal v1.4.0
go: downloading golang.org/x/crypto v0.35.0
go: downloading github.com/rivo/uniseg v0.2.0
go: downloading golang.org/x/text v0.22.0
go: downloading github.com/STARRY-S/zip v0.2.1
go: downloading github.com/andybalholm/brotli v1.1.2-0.20250424173009-453214e765f3
go: downloading github.com/bodgit/sevenzip v1.6.0
go: downloading github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707
go: downloading github.com/klauspost/compress v1.17.11
go: downloading github.com/klauspost/pgzip v1.2.6
go: downloading github.com/minio/minlz v1.0.0
go: downloading github.com/nwaples/rardecode/v2 v2.1.0
go: downloading github.com/pierrec/lz4/v4 v4.1.21
go: downloading github.com/sorairolake/lzip-go v0.3.5
go: downloading github.com/therootcompany/xz v1.0.1
go: downloading github.com/ulikunitz/xz v0.5.12
go: downloading github.com/mitchellh/reflectwalk v1.0.2
go: downloading github.com/aymanbagabas/go-osc52/v2 v2.0.1
go: downloading github.com/mattn/go-isatty v0.0.20
go: downloading golang.org/x/sys v0.30.0
go: downloading github.com/mattn/go-colorable v0.1.14
go: downloading github.com/bodgit/windows v1.0.1
go: downloading go4.org v0.0.0-20230225012048-214862532bf5
go: downloading github.com/hashicorp/go-multierror v1.1.1
go: downloading github.com/hashicorp/golang-lru/v2 v2.0.7
go: downloading github.com/bodgit/plumbing v1.3.0
go: downloading github.com/hashicorp/errwrap v1.1.0
[90m7:01PM[0m [32mINF[0m [1mUnknown SCM platform. Use --platform to include links in findings.[0m [36mhost=[0mamadan.net
[90m7:01PM[0m [32mINF[0m [1m25 commits scanned.[0m
[90m7:01PM[0m [32mINF[0m [1mscanned ~7736515 bytes (7.74 MB) in 7.07s[0m
[90m7:01PM[0m [32mINF[0m [1mno leaks found[0m
suite: gitleaks took 81s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m31.840000s 0m4.320000s
--- shard 11 of 52 ---
==> shard 11 of 52: make browser.internal.harness
./hack/gotest.sh browser 'internal.harness'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
ok amadan.net/rastrillo/fiesta/internal/harness 17.757s
suite: browser.internal.harness took 76s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m19.710000s 0m4.210000s
--- shard 12 of 52 ---
==> shard 12 of 52: make browser.check~1of14
./hack/gotest.sh browser 'check~1of14'
gotest: on ip-172-31-27-155, driving /opt/amadan-browser/chromium-1234/chrome-linux/chrome
gotest: slice 1 of 14 of ./check/ runs 9 of 287 tests
ok amadan.net/rastrillo/fiesta/check 101.549s
suite: browser.check~1of14 took 179s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m54.300000s 0m9.460000s
--- shard 13 of 52 ---
==> shard 13 of 52: make browser.check~2of14
./hack/gotest.sh browser 'check~2of14'
gotest: on ip-172-31-27-155, driving /opt/amadan-browser/chromium-1234/chrome-linux/chrome
gotest: slice 2 of 14 of ./check/ runs 13 of 287 tests
ok amadan.net/rastrillo/fiesta/check 95.883s
suite: browser.check~2of14 took 168s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m43.110000s 0m8.730000s
--- shard 14 of 52 ---
==> shard 14 of 52: make browser.check~3of14
./hack/gotest.sh browser 'check~3of14'
gotest: on ip-172-31-27-155, driving /opt/amadan-browser/chromium-1234/chrome-linux/chrome
gotest: slice 3 of 14 of ./check/ runs 15 of 287 tests
ok amadan.net/rastrillo/fiesta/check 100.925s
suite: browser.check~3of14 took 178s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m51.610000s 0m9.420000s
--- shard 15 of 52 ---
==> shard 15 of 52: make browser.check~4of14
./hack/gotest.sh browser 'check~4of14'
gotest: on ip-172-31-27-155, driving /opt/amadan-browser/chromium-1234/chrome-linux/chrome
gotest: slice 4 of 14 of ./check/ runs 17 of 287 tests
ok amadan.net/rastrillo/fiesta/check 106.100s
suite: browser.check~4of14 took 179s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m50.850000s 0m11.340000s
--- shard 16 of 52 ---
==> shard 16 of 52: make browser.check~5of14
./hack/gotest.sh browser 'check~5of14'
gotest: on ip-172-31-27-155, driving /opt/amadan-browser/chromium-1234/chrome-linux/chrome
gotest: slice 5 of 14 of ./check/ runs 22 of 287 tests
ok amadan.net/rastrillo/fiesta/check 80.264s
suite: browser.check~5of14 took 155s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m32.550000s 0m8.000000s
--- shard 17 of 52 ---
==> shard 17 of 52: make browser.check~6of14
./hack/gotest.sh browser 'check~6of14'
gotest: on ip-172-31-27-155, driving /opt/amadan-browser/chromium-1234/chrome-linux/chrome
gotest: slice 6 of 14 of ./check/ runs 22 of 287 tests
ok amadan.net/rastrillo/fiesta/check 83.110s
suite: browser.check~6of14 took 159s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m42.200000s 0m11.490000s
--- shard 18 of 52 ---
==> shard 18 of 52: make browser.check~7of14
./hack/gotest.sh browser 'check~7of14'
gotest: on ip-172-31-27-155, driving /opt/amadan-browser/chromium-1234/chrome-linux/chrome
gotest: slice 7 of 14 of ./check/ runs 22 of 287 tests
ok amadan.net/rastrillo/fiesta/check 83.330s
suite: browser.check~7of14 took 156s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m37.140000s 0m9.330000s
--- shard 19 of 52 ---
==> shard 19 of 52: make browser.check~8of14
./hack/gotest.sh browser 'check~8of14'
gotest: on ip-172-31-27-155, driving /opt/amadan-browser/chromium-1234/chrome-linux/chrome
gotest: slice 8 of 14 of ./check/ runs 23 of 287 tests
ok amadan.net/rastrillo/fiesta/check 57.880s
suite: browser.check~8of14 took 134s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m31.430000s 0m7.610000s
--- shard 20 of 52 ---
==> shard 20 of 52: make browser.check~9of14
./hack/gotest.sh browser 'check~9of14'
gotest: on ip-172-31-27-155, driving /opt/amadan-browser/chromium-1234/chrome-linux/chrome
gotest: slice 9 of 14 of ./check/ runs 23 of 287 tests
ok amadan.net/rastrillo/fiesta/check 79.010s
suite: browser.check~9of14 took 154s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m37.240000s 0m12.350000s
--- shard 21 of 52 ---
==> shard 21 of 52: make browser.check~10of14
./hack/gotest.sh browser 'check~10of14'
gotest: on ip-172-31-27-155, driving /opt/amadan-browser/chromium-1234/chrome-linux/chrome
gotest: slice 10 of 14 of ./check/ runs 24 of 287 tests
ok amadan.net/rastrillo/fiesta/check 86.418s
suite: browser.check~10of14 took 163s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m44.030000s 0m11.180000s
--- shard 22 of 52 ---
==> shard 22 of 52: make browser.check~11of14
./hack/gotest.sh browser 'check~11of14'
gotest: on ip-172-31-27-155, driving /opt/amadan-browser/chromium-1234/chrome-linux/chrome
gotest: slice 11 of 14 of ./check/ runs 24 of 287 tests
ok amadan.net/rastrillo/fiesta/check 70.616s
suite: browser.check~11of14 took 149s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m34.840000s 0m9.430000s
--- shard 23 of 52 ---
==> shard 23 of 52: make browser.check~12of14
./hack/gotest.sh browser 'check~12of14'
gotest: on ip-172-31-27-155, driving /opt/amadan-browser/chromium-1234/chrome-linux/chrome
gotest: slice 12 of 14 of ./check/ runs 24 of 287 tests
ok amadan.net/rastrillo/fiesta/check 73.890s
suite: browser.check~12of14 took 151s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m35.030000s 0m10.080000s
--- shard 24 of 52 ---
==> shard 24 of 52: make browser.check~13of14
./hack/gotest.sh browser 'check~13of14'
gotest: on ip-172-31-27-155, driving /opt/amadan-browser/chromium-1234/chrome-linux/chrome
gotest: slice 13 of 14 of ./check/ runs 24 of 287 tests
ok amadan.net/rastrillo/fiesta/check 78.201s
suite: browser.check~13of14 took 155s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m39.210000s 0m11.950000s
--- shard 25 of 52 ---
==> shard 25 of 52: make browser.check~14of14
./hack/gotest.sh browser 'check~14of14'
gotest: on ip-172-31-27-155, driving /opt/amadan-browser/chromium-1234/chrome-linux/chrome
gotest: slice 14 of 14 of ./check/ runs 25 of 287 tests
ok amadan.net/rastrillo/fiesta/check 81.385s
suite: browser.check~14of14 took 156s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m42.400000s 0m12.340000s
--- shard 26 of 52 ---
==> shard 26 of 52: make browser.internal.gallery~1of20
./hack/gotest.sh browser 'internal.gallery~1of20'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 1 of 20 of ./internal/gallery/ runs 9 of 184 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 82.629s
suite: browser.internal.gallery~1of20 took 152s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m36.550000s 0m4.950000s
--- shard 27 of 52 ---
==> shard 27 of 52: make browser.internal.gallery~2of20
./hack/gotest.sh browser 'internal.gallery~2of20'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 2 of 20 of ./internal/gallery/ runs 9 of 184 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 100.623s
suite: browser.internal.gallery~2of20 took 172s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m42.060000s 0m6.480000s
--- shard 28 of 52 ---
==> shard 28 of 52: make browser.internal.gallery~3of20
./hack/gotest.sh browser 'internal.gallery~3of20'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 3 of 20 of ./internal/gallery/ runs 9 of 184 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 105.989s
suite: browser.internal.gallery~3of20 took 177s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m43.690000s 0m4.980000s
--- shard 29 of 52 ---
==> shard 29 of 52: make browser.internal.gallery~4of20
./hack/gotest.sh browser 'internal.gallery~4of20'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 4 of 20 of ./internal/gallery/ runs 9 of 184 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 90.974s
suite: browser.internal.gallery~4of20 took 163s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m43.060000s 0m6.470000s
--- shard 30 of 52 ---
==> shard 30 of 52: make browser.internal.gallery~5of20
./hack/gotest.sh browser 'internal.gallery~5of20'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 5 of 20 of ./internal/gallery/ runs 10 of 184 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 89.687s
suite: browser.internal.gallery~5of20 took 163s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m43.950000s 0m6.000000s
--- shard 31 of 52 ---
==> shard 31 of 52: make browser.internal.gallery~6of20
./hack/gotest.sh browser 'internal.gallery~6of20'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 6 of 20 of ./internal/gallery/ runs 9 of 184 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 106.013s
suite: browser.internal.gallery~6of20 took 176s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m43.450000s 0m7.040000s
--- shard 32 of 52 ---
==> shard 32 of 52: make browser.internal.gallery~7of20
./hack/gotest.sh browser 'internal.gallery~7of20'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 7 of 20 of ./internal/gallery/ runs 10 of 184 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 100.402s
suite: browser.internal.gallery~7of20 took 177s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m52.560000s 0m7.740000s
--- shard 33 of 52 ---
==> shard 33 of 52: make browser.internal.gallery~8of20
./hack/gotest.sh browser 'internal.gallery~8of20'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 8 of 20 of ./internal/gallery/ runs 10 of 184 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 102.110s
suite: browser.internal.gallery~8of20 took 176s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m49.310000s 0m6.970000s
--- shard 34 of 52 ---
==> shard 34 of 52: make browser.internal.gallery~9of20
./hack/gotest.sh browser 'internal.gallery~9of20'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 9 of 20 of ./internal/gallery/ runs 8 of 184 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 95.847s
suite: browser.internal.gallery~9of20 took 173s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m43.560000s 0m6.570000s
--- shard 35 of 52 ---
==> shard 35 of 52: make browser.internal.gallery~10of20
./hack/gotest.sh browser 'internal.gallery~10of20'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 10 of 20 of ./internal/gallery/ runs 9 of 184 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 93.903s
suite: browser.internal.gallery~10of20 took 171s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m37.290000s 0m6.230000s
--- shard 36 of 52 ---
==> shard 36 of 52: make browser.internal.gallery~11of20
./hack/gotest.sh browser 'internal.gallery~11of20'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 11 of 20 of ./internal/gallery/ runs 9 of 184 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 89.703s
suite: browser.internal.gallery~11of20 took 165s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m28.820000s 0m5.330000s
--- shard 37 of 52 ---
==> shard 37 of 52: make browser.internal.gallery~12of20
./hack/gotest.sh browser 'internal.gallery~12of20'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 12 of 20 of ./internal/gallery/ runs 9 of 184 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 104.703s
suite: browser.internal.gallery~12of20 took 178s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
1m2.360000s 0m5.440000s
--- shard 38 of 52 ---
==> shard 38 of 52: make browser.internal.gallery~13of20
./hack/gotest.sh browser 'internal.gallery~13of20'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 13 of 20 of ./internal/gallery/ runs 9 of 184 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 91.281s
suite: browser.internal.gallery~13of20 took 163s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m39.150000s 0m3.970000s
--- shard 39 of 52 ---
==> shard 39 of 52: make browser.internal.gallery~14of20
./hack/gotest.sh browser 'internal.gallery~14of20'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 14 of 20 of ./internal/gallery/ runs 9 of 184 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 93.615s
suite: browser.internal.gallery~14of20 took 166s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m34.880000s 0m5.600000s
--- shard 40 of 52 ---
==> shard 40 of 52: make browser.internal.gallery~15of20
./hack/gotest.sh browser 'internal.gallery~15of20'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 15 of 20 of ./internal/gallery/ runs 10 of 184 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 102.085s
suite: browser.internal.gallery~15of20 took 174s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m44.140000s 0m9.040000s
--- shard 41 of 52 ---
==> shard 41 of 52: make browser.internal.gallery~16of20
./hack/gotest.sh browser 'internal.gallery~16of20'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 16 of 20 of ./internal/gallery/ runs 10 of 184 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 100.558s
suite: browser.internal.gallery~16of20 took 174s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m50.340000s 0m6.160000s
--- shard 42 of 52 ---
==> shard 42 of 52: make browser.internal.gallery~17of20
./hack/gotest.sh browser 'internal.gallery~17of20'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 17 of 20 of ./internal/gallery/ runs 10 of 184 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 98.815s
suite: browser.internal.gallery~17of20 took 172s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m59.750000s 0m6.750000s
--- shard 43 of 52 ---
==> shard 43 of 52: make browser.internal.gallery~18of20
./hack/gotest.sh browser 'internal.gallery~18of20'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 18 of 20 of ./internal/gallery/ runs 9 of 184 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 108.677s
suite: browser.internal.gallery~18of20 took 181s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m41.760000s 0m8.030000s
--- shard 44 of 52 ---
==> shard 44 of 52: make browser.internal.gallery~19of20
./hack/gotest.sh browser 'internal.gallery~19of20'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 19 of 20 of ./internal/gallery/ runs 9 of 184 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 111.146s
suite: browser.internal.gallery~19of20 took 183s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m41.390000s 0m8.390000s
--- shard 45 of 52 ---
==> shard 45 of 52: make browser.internal.gallery~20of20
./hack/gotest.sh browser 'internal.gallery~20of20'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 20 of 20 of ./internal/gallery/ runs 8 of 184 tests
ok amadan.net/rastrillo/fiesta/internal/gallery 104.488s
suite: browser.internal.gallery~20of20 took 178s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m46.070000s 0m9.680000s
--- shard 46 of 52 ---
==> shard 46 of 52: make browser.internal.gallery.history
./hack/gotest.sh browser 'internal.gallery.history'
gotest: on ip-172-31-27-155, driving /opt/amadan-browser/chromium-1234/chrome-linux/chrome
ok amadan.net/rastrillo/fiesta/internal/gallery/history 66.671s
suite: browser.internal.gallery.history took 129s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m36.680000s 0m4.050000s
--- shard 47 of 52 ---
==> shard 47 of 52: make browser.internal.www~1of5
./hack/gotest.sh browser 'internal.www~1of5'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 1 of 5 of ./internal/www/ runs 6 of 34 tests
ok amadan.net/rastrillo/fiesta/internal/www 96.594s
suite: browser.internal.www~1of5 took 165s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m51.480000s 0m4.030000s
--- shard 48 of 52 ---
==> shard 48 of 52: make browser.internal.www~2of5
./hack/gotest.sh browser 'internal.www~2of5'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 2 of 5 of ./internal/www/ runs 9 of 34 tests
ok amadan.net/rastrillo/fiesta/internal/www 88.260s
suite: browser.internal.www~2of5 took 155s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m54.040000s 0m4.230000s
--- shard 49 of 52 ---
==> shard 49 of 52: make browser.internal.www~3of5
./hack/gotest.sh browser 'internal.www~3of5'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 3 of 5 of ./internal/www/ runs 6 of 34 tests
ok amadan.net/rastrillo/fiesta/internal/www 95.455s
suite: browser.internal.www~3of5 took 161s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m55.180000s 0m5.470000s
--- shard 50 of 52 ---
==> shard 50 of 52: make browser.internal.www~4of5
./hack/gotest.sh browser 'internal.www~4of5'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 4 of 5 of ./internal/www/ runs 6 of 34 tests
ok amadan.net/rastrillo/fiesta/internal/www 96.781s
suite: browser.internal.www~4of5 took 163s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m56.070000s 0m4.050000s
--- shard 51 of 52 ---
==> shard 51 of 52: make browser.internal.www~5of5
./hack/gotest.sh browser 'internal.www~5of5'
gotest: on ip-172-31-27-155, driving the harness's own choice (/usr/local/bin/chromium)
gotest: slice 5 of 5 of ./internal/www/ runs 7 of 34 tests
ok amadan.net/rastrillo/fiesta/internal/www 89.540s
suite: browser.internal.www~5of5 took 157s; CPU, user then system, of this shell and then of the role:
0m0.000000s 0m0.000000s
0m53.420000s 0m4.340000s
Intent
diff --git a/.amadan/ci.d/40-history b/.amadan/ci.d/40-history| new file mode 100755 |
| index 0000000..12ec97b |
| --- /dev/null |
| +++ b/.amadan/ci.d/40-history |
| @@ -0,0 +1,2 @@ |
| +#!/bin/sh |
| +exec make history |
diff --git a/.amadan/ci.d/50-suite@15 b/.amadan/ci.d/50-suite@52| similarity index 100% |
| rename from .amadan/ci.d/50-suite@15 |
| rename to .amadan/ci.d/50-suite@52 |
diff --git a/Makefile b/Makefile| index 7298694..b7c9091 100644 |
| --- a/Makefile |
| +++ b/Makefile |
| @@ -1,4 +1,4 @@ |
| -.PHONY: ci suite suite-roles gofmt no-rastrillo size root staticcheck govulncheck gitleaks browser site |
| +.PHONY: ci suite suite-roles gofmt no-rastrillo size history root staticcheck govulncheck gitleaks browser heights-sweep site |
| # CGO_ENABLED=0 across the tree is an acceptance criterion, not a |
| # preference: Fiesta is CSS, scripts and the Go that embeds and checks |
| @@ -36,8 +36,10 @@ export GOTMPDIR ?= $(BIN)/tmp |
| # |
| # The cheap checks go first and alone, so a stray gofmt still costs |
| # seconds, and the size gate next, so a bundle over the line is the |
| -# first thing a red run says. Everything else is the suite. |
| -ci: gofmt no-rastrillo size suite |
| +# first thing a red run says. history then makes a runner's shallow |
| +# clone whole, once, before the suite's shards share it. Everything |
| +# else is the suite. |
| +ci: gofmt no-rastrillo size history suite |
| # The suite's roles. Under amadan, .amadan/ci.d/50-suite@N runs N |
| # shards at once and each runs one role, so the run takes as long as |
| @@ -48,17 +50,24 @@ ci: gofmt no-rastrillo size suite |
| # the two agree before a runner ever sees them. |
| # |
| # The slowest role is how long the suite takes, so the slow packages |
| -# are roles of their own. The browser packages are a role each. A |
| -# package written pkg@n is n roles, each a slice of its tests, dealt |
| -# by hack/gotest.sh. In a role's name a / in the path is a . so |
| -# check/drives@2 would become browser.check.drives~1of2 and ~2of2. |
| -# The tasks that add a browser package append here and rename |
| -# 50-suite@N to match. |
| -BROWSER_PKGS := internal/harness check@3 internal/gallery@4 internal/gallery/history internal/www@2 |
| +# are roles of their own, sliced until every role runs in under a |
| +# minute on a runner. The browser packages are a role each, and |
| +# ROOT_SPLIT's packages leave root's go test ./... for test.<pkg> roles. |
| +# A package written pkg@n is n roles, each a slice of its tests, dealt |
| +# by hack/gotest.sh by the seconds in hack/test-weights.txt. In a |
| +# role's name a / in the path is a . so internal/gallery@2 becomes |
| +# browser.internal.gallery~1of2 and ~2of2. A test too long for a slice |
| +# of its own is dealt into parts by the test itself (the height drives, |
| +# the two axe sweeps, the HTML tab's renders, the floors drive). The |
| +# tasks that add a browser package append here and rename 50-suite@N |
| +# to match. |
| +BROWSER_PKGS := internal/harness check@14 internal/gallery@20 internal/gallery/history internal/www@5 |
| +ROOT_SPLIT := internal/gallery@3 cmd/fiesta@2 internal/www@2 |
| go-roles = $(foreach p,$(2),$(if $(findstring @,$(p)),$(call go-slices,$(1),$(subst /,.,$(word 1,$(subst @, ,$(p)))),$(word 2,$(subst @, ,$(p)))),$(1).$(subst /,.,$(p)))) |
| go-slices = $(foreach k,$(shell seq 1 $(3)),$(1).$(2)~$(k)of$(3)) |
| BROWSER_ROLES := $(call go-roles,browser,$(BROWSER_PKGS)) |
| -SUITE := root staticcheck govulncheck gitleaks $(BROWSER_ROLES) |
| +TEST_ROLES := $(call go-roles,test,$(ROOT_SPLIT)) |
| +SUITE := root $(TEST_ROLES) staticcheck govulncheck gitleaks $(BROWSER_ROLES) |
| suite: |
| @MAKE='$(MAKE)' ./hack/suite.sh $(SUITE) |
| @@ -104,13 +113,20 @@ FIESTA_TEST_REQUIRE_NODE ?= 1 |
| # fiesta release -candidate is tested in a scratch repository, and a |
| # runner without git would otherwise pass with the guard untested. |
| FIESTA_TEST_REQUIRE_GIT ?= 1 |
| -TEST_ENV := FIESTA_TEST_REQUIRE_NODE=$(FIESTA_TEST_REQUIRE_NODE) FIESTA_TEST_REQUIRE_GIT=$(FIESTA_TEST_REQUIRE_GIT) |
| -# root builds, vets and tests every package. The browser drives are |
| -# behind the browser tag and run as roles of their own. |
| +# FIESTA_TEST_REQUIRE_TAGS does the same for the release tags: |
| +# releases_test.go decides what to check from them, and a clone with |
| +# none would pass it having compared nothing. make history fetches them |
| +# into a runner's shallow clone. |
| +FIESTA_TEST_REQUIRE_TAGS ?= 1 |
| +TEST_ENV := FIESTA_TEST_REQUIRE_NODE=$(FIESTA_TEST_REQUIRE_NODE) FIESTA_TEST_REQUIRE_GIT=$(FIESTA_TEST_REQUIRE_GIT) FIESTA_TEST_REQUIRE_TAGS=$(FIESTA_TEST_REQUIRE_TAGS) |
| +# root builds, vets and tests every package but ROOT_SPLIT's, whose |
| +# tests run in the test.<pkg> roles; `make root $(TEST_ROLES)` is the |
| +# old whole. The browser drives are behind the browser tag and run as |
| +# roles of their own. |
| root: |
| go build ./... |
| go vet ./... |
| - $(TEST_ENV) go test -count=1 ./... |
| + $(TEST_ENV) ./hack/gotest.sh root '$(ROOT_SPLIT)' |
| # test.<pkg>[~<k>of<n>]: one package of plain tests as a role of its own, |
| # for a package too slow to sit inside root. |
| @@ -155,8 +171,32 @@ GOVULNCHECK := golang.org/x/vuln/cmd/govulncheck@v1.8.0 |
| govulncheck: |
| go run $(GOVULNCHECK) ./... |
| +# history makes a shallow clone whole: a runner clones one commit deep |
| +# and with no tags, so gitleaks below would scan one commit while |
| +# promising the history, and releases_test.go would find no tag to |
| +# hold releases/ to. The whole history rather than a range of the |
| +# pushed commits: a runner cannot name where a push began, and |
| +# Fiesta's history is small (fetching it into a fresh one-commit clone |
| +# took a third of a second). Its own step before the suite, because |
| +# the suite's shards share one .git and two fetches at once would race |
| +# for its locks. On a runner (AMADAN_CI) it also fetches the release |
| +# tags into a clone that is already whole, since a tag pushed after a |
| +# commit was fetched is not always followed; elsewhere a whole clone |
| +# is left alone, so make ci by hand never fetches. |
| +history: |
| + @if [ "$$(git rev-parse --is-shallow-repository)" = true ]; then \ |
| + echo "history: a shallow clone; fetching the whole history and the release tags"; \ |
| + git fetch -q --unshallow origin 'refs/tags/v*:refs/tags/v*'; \ |
| + elif [ -n "$${AMADAN_CI:-}" ]; then \ |
| + git fetch -q origin 'refs/tags/v*:refs/tags/v*'; \ |
| + fi |
| + @if [ "$$(git rev-parse --is-shallow-repository)" != false ]; then echo "history: the clone is still shallow"; exit 1; fi |
| + @echo "history: $$(git rev-list --count HEAD) commits; release tags: $$(git tag -l 'v*' | paste -sd' ' -)" |
| + |
| # The whole git history, not only the tree: a key committed and deleted |
| # a year ago is still in every clone, and this repository is public. |
| +# A shallow clone is refused rather than scanned, since its history is |
| +# one commit; make history comes first in ci for that. |
| # --redact keeps anything it does find out of the CI log. |
| # |
| # The history of HEAD, not of every ref: gitleaks' default is |
| @@ -167,6 +207,7 @@ govulncheck: |
| # replaces wholesale. |
| GITLEAKS := github.com/zricethezav/gitleaks/v8@v8.30.1 |
| gitleaks: |
| + @if [ "$$(git rev-parse --is-shallow-repository)" != false ]; then echo "gitleaks: this clone is shallow, so its history is one commit; run make history first"; exit 1; fi |
| go run $(GITLEAKS) git --no-banner --redact --log-opts='--full-history --diff-filter=tuxdb HEAD' . |
| # The browser drives, one package or slice per browser.<pkg>[~<k>of<n>] |
| @@ -189,6 +230,27 @@ export FULL_CHROMIUM := check internal/gallery/history |
| browser.%: | $(BIN)/tmp |
| ./hack/gotest.sh browser '$*' |
| +# heights-sweep measures the preview heights table on this machine's |
| +# Chromium and fonts and prints it as Go source for heights.go: both |
| +# height drives over all twelve locales (FIESTA_SWEEP=full), at once, |
| +# each into its own log, printed when both are done. It reports; the |
| +# drives still fail a short box. Not in ci: it takes a quarter of an |
| +# hour. To calibrate on a runner, push a throwaway branch whose one |
| +# commit makes .amadan/ci.d this single step, 10-heights-sweep, being |
| +# `exec make heights-sweep`; the table is in that run's log |
| +# (amadan ci status rastrillo/fiesta -branch <it> -json). Then delete |
| +# the branch. |
| +HEIGHTS_SWEEP := FitTheirContent FitAtThePhonesNarrowest |
| +heights-sweep: | $(BIN)/tmp |
| + @d=$$(mktemp -d) && status=0 && pids= && \ |
| + for drive in $(HEIGHTS_SWEEP); do \ |
| + FIESTA_SWEEP=full ./hack/gotest.sh browser internal.gallery -timeout 30m -v -run "^TestPreviewFrameHeights$${drive}1of" > "$$d/$$drive.log" 2>&1 & \ |
| + pids="$$pids $$!"; \ |
| + done; \ |
| + for pid in $$pids; do wait "$$pid" || status=1; done; \ |
| + for drive in $(HEIGHTS_SWEEP); do echo "==> $$drive"; cat "$$d/$$drive.log"; done; \ |
| + rm -rf "$$d"; exit $$status |
| + |
| # site assembles fiesta.rastrillo.org into .build/site/public from the |
| # committed releases and the site source: each release under its exact |
| # version, _redirects sending /M/* to the newest M.y.z, and at the root |
diff --git a/check/coverage_browser_test.go b/check/coverage_browser_test.go| index d51904a..c698292 100644 |
| --- a/check/coverage_browser_test.go |
| +++ b/check/coverage_browser_test.go |
| @@ -343,7 +343,41 @@ type floorReading struct { |
| // that the server served every state's page on every leg, and that |
| // the manifest lists more than a hundred, so an emptied manifest or a |
| // loop that skipped cannot pass by measuring nothing. |
| -func TestEveryExampleStateMeetsTheFloors(t *testing.T) { |
| +// |
| +// It is two tests, so the suite's slices can run them on shards of |
| +// their own: the phone's leg, and the two desktop legs together, since |
| +// the no-script leg is held to what the scripted one measured. Every |
| +// leg's checks are as they were. Of the checks over every leg, the |
| +// pinned gaps are checked in both, each for what it measures, and an |
| +// idiom no state renders is looked for where the inventory says it is |
| +// rendered: a narrow: idiom on the phone, every other one at 1280. |
| +// That last is stricter than the one test, which accepted an idiom |
| +// seen on the phone alone; and both tests need a caret and an icon. |
| +func TestEveryExampleStateMeetsTheFloorsOnAPhone(t *testing.T) { |
| + meetTheFloors(t, true, []floorLeg{ |
| + {"390 touch", 390, 844, true, 44, false}, |
| + }) |
| +} |
| + |
| +func TestEveryExampleStateMeetsTheFloorsWithAMouse(t *testing.T) { |
| + meetTheFloors(t, false, []floorLeg{ |
| + {"1280 mouse", 1280, 900, false, 32, false}, |
| + {"1280 mouse, no scripts", 1280, 900, false, 32, true}, |
| + }) |
| +} |
| + |
| +// floorLeg is one way the floors drive loads every state. |
| +type floorLeg struct { |
| + name string |
| + w, h int64 |
| + coarse bool |
| + floor float64 |
| + noscript bool |
| +} |
| + |
| +// meetTheFloors drives legs, the phone's or the desktop's, over every |
| +// state. |
| +func meetTheFloors(t *testing.T, phone bool, legs []floorLeg) { |
| states, srv := floorPages(t) |
| want := manifestStates(t) |
| if len(want) <= 100 { |
| @@ -352,17 +386,7 @@ func TestEveryExampleStateMeetsTheFloors(t *testing.T) { |
| seen := map[string]int{} |
| withScripts := map[string]map[string]bool{} |
| var carets, icons int |
| - for _, leg := range []struct { |
| - name string |
| - w, h int64 |
| - coarse bool |
| - floor float64 |
| - noscript bool |
| - }{ |
| - {"390 touch", 390, 844, true, 44, false}, |
| - {"1280 mouse", 1280, 900, false, 32, false}, |
| - {"1280 mouse, no scripts", 1280, 900, false, 32, true}, |
| - } { |
| + for _, leg := range legs { |
| t.Run(leg.name, func(t *testing.T) { |
| var opts []harness.Option |
| if leg.coarse { |
| @@ -486,10 +510,14 @@ func TestEveryExampleStateMeetsTheFloors(t *testing.T) { |
| if strings.HasPrefix(e.Probe, "elsewhere:") { |
| continue |
| } |
| + narrow := strings.HasPrefix(e.Probe, "narrow:") |
| + if narrow && !phone { |
| + continue |
| + } |
| sel := strings.TrimPrefix(strings.TrimPrefix(e.Probe, "narrow:"), "modal:") |
| _, pinned := floorGaps[sel] |
| switch { |
| - case seen[sel] == 0 && !pinned: |
| + case seen[sel] == 0 && !pinned && narrow == phone: |
| t.Errorf("no example state renders %s (%s), and it is not a pinned gap; the examples lost the idiom", sel, e.Idiom) |
| case seen[sel] > 0 && pinned: |
| t.Errorf("%s (%s) is pinned as a gap in floorGaps but an example renders it; drop the pin", sel, e.Idiom) |
diff --git a/hack/gotest.sh b/hack/gotest.sh| index 1ab0366..8b8ef25 100755 |
| --- a/hack/gotest.sh |
| +++ b/hack/gotest.sh |
| @@ -1,13 +1,14 @@ |
| #!/bin/sh |
| # One go test role of the suite. |
| # |
| -# hack/gotest.sh browser <name> go test -tags browser, one package |
| -# hack/gotest.sh test <name> go test, one package |
| +# hack/gotest.sh browser <name> [go test flags] go test -tags browser, one package |
| +# hack/gotest.sh test <name> [go test flags] go test, one package |
| # hack/gotest.sh root '<split>' go test ./... minus the split packages |
| # |
| # <name> is a package path with its slashes written as dots |
| # (internal.designsystem), optionally ~<k>of<n> to run only the k-th of |
| -# n slices of its tests. |
| +# n slices of its tests. Flags after the name go to go test, after the |
| +# role's own, and only for a whole package: a slice is its own -run. |
| # |
| # A slice is dealt from the package's own test list: heaviest first, |
| # each onto the lightest slice so far, by the seconds in |
| @@ -76,6 +77,7 @@ esac |
| mode=$1 |
| name=$2 |
| +shift 2 |
| rel=$name |
| slice= |
| case "$name" in |
| @@ -157,7 +159,11 @@ run() { |
| if [ -z "$slice" ]; then |
| # shellcheck disable=SC2086 # an empty $tags is no argument at all |
| - run go test $tags -timeout "$timeout" "$pkg" -count=1 |
| + run go test $tags -timeout "$timeout" "$pkg" -count=1 "$@" |
| +fi |
| +if [ $# -gt 0 ]; then |
| + echo "gotest: flags for go test are for a whole package, not slice $slice of $pkg" |
| + exit 2 |
| fi |
| k=${slice%of*} |
diff --git a/hack/suite.sh b/hack/suite.sh| index c558607..bc73c20 100755 |
| --- a/hack/suite.sh |
| +++ b/hack/suite.sh |
| @@ -34,11 +34,21 @@ if [ "$count" -ne "$roles" ]; then |
| exit 1 |
| fi |
| +# A shard says how long its role took and how much CPU it used, the |
| +# second line of times being what the role ran: on a shared box the wall |
| +# time says how busy the box was as much as how heavy the role is, and |
| +# the CPU is what hack/test-weights.txt and the slice counts are judged |
| +# by on a box of another size. |
| i=0 |
| for role; do |
| if [ "$i" -eq "$shard" ]; then |
| echo "==> shard $shard of $count: make $role" |
| - exec $make --no-print-directory "$role" |
| + start=$(date +%s) |
| + status=0 |
| + $make --no-print-directory "$role" || status=$? |
| + echo "suite: $role took $(($(date +%s) - start))s; CPU, user then system, of this shell and then of the role:" |
| + times |
| + exit "$status" |
| fi |
| i=$((i + 1)) |
| done |
diff --git a/hack/test-weights.txt b/hack/test-weights.txt| index 3d37d55..9962b34 100644 |
| --- a/hack/test-weights.txt |
| +++ b/hack/test-weights.txt |
| @@ -1,103 +1,219 @@ |
| # Seconds per test, for hack/gotest.sh's slice deal (<mode> <pkg> |
| # <test> <seconds>, mode browser or test): the heaviest test goes to |
| -# the lightest slice first. Only tests of 2s or more are worth listing; |
| -# any test not here weighs 1. A stale number costs balance, never |
| -# coverage: every listed test still runs in exactly one slice. Refresh |
| -# from a `go test -v` run: |
| -# grep '^--- ' log | sed -E 's/^--- [A-Z]+: ([^ ]+) \(([0-9.]+)s\)/\1 \2/' |
| -# Measured 2026-10-27 on docs-t1, one full run of check with full |
| -# Chromium, every test at 2s or more. |
| -browser check TestEveryExampleStateMeetsTheFloors 58 |
| -browser check TestEveryThemeRendersAsItDidExceptDay 45 |
| +# the lightest slice first. Only tests of 2s or more are worth listing |
| +# (anything from 1.5s is listed as 2); any test not here weighs 1. A |
| +# stale number costs balance, never coverage: every listed test still |
| +# runs in exactly one slice. Refresh from a `go test -json` run, every |
| +# top-level test's Elapsed (mode test for an untagged run): |
| +# jq -r 'select(.Action == "pass" and .Test and (.Test | contains("/") | not) and .Elapsed >= 1.5) |
| +# | "browser \(.Package | ltrimstr("amadan.net/rastrillo/fiesta/")) \(.Test) \([.Elapsed | round, 2] | max)"' |
| +# Measured 2026-10-29 on docs-t1 (16 cores, shared), from one |
| +# `go test -json -tags browser` run of the five browser packages at |
| +# once, the split tests timed again after splitting, and one untagged |
| +# `go test -json ./...` for the test lines. Re-measure from a fleet run |
| +# once Fiesta's CI is routed there: the fleet is arm64 with its own |
| +# Chromium and fonts, and these weights are docs-t1's. |
| +browser check TestEveryThemeRendersAsItDidExceptDay 44 |
| +browser check TestEveryExampleStateMeetsTheFloorsWithAMouse 40 |
| browser check TestTheReleaseRendersAsTheSource 38 |
| -browser check TestEveryThemeAndShellRendersAsTheBaseline 37 |
| +browser check TestEveryThemeAndShellRendersAsTheBaseline 36 |
| browser check TestBusyButtonDrive 22 |
| +browser check TestEveryExampleStateMeetsTheFloorsOnAPhone 22 |
| +browser check TestAnyAccentKeepsTheFillLabelReadable 19 |
| browser check TestTheLightStaysWithinItsFrameBudget 17 |
| -browser check TestTheProfileMenuIsADropUpAtTheRailsFoot 17 |
| -browser check TestAnyAccentKeepsTheFillLabelReadable 13 |
| +browser check TestTheProfileMenuIsADropUpAtTheRailsFoot 16 |
| browser check TestFlatThemesFollowAnAppsOverrides 13 |
| -browser check TestThePhoneIndexWorksWithScripts 10 |
| browser check TestTheReleaseScriptsDriveTheJourneys 10 |
| -browser check TestThePhoneIndexWorksWithNoScript 9 |
| browser check TestASwitchSlidesAndSnaps 8 |
| +browser check TestThePhoneIndexWorksWithScripts 8 |
| browser check TestADropdownOpensUnderItsOwnSummary 7 |
| +browser check TestAPostItemsSpinnerNeverMovesTheMenu 7 |
| browser check TestAReleasedKnobSettlesWithAFirmCurve 7 |
| browser check TestARowMenuOpensBesideItsOwnKebab 7 |
| -browser check TestAPostItemsSpinnerNeverMovesTheMenu 6 |
| browser check TestAPreferenceToggleForgetsSettledCards 6 |
| browser check TestKeysAndTouchLeavePointerMode 6 |
| -browser check TestTheConsoleBarIsTheSameCard 6 |
| -browser check TestTheConsoleFoldsItsBarAndIndexesItsRail 6 |
| +browser check TestThePhoneIndexWorksWithNoScript 6 |
| browser check TestTheProfileMenuOnThePhoneIndex 6 |
| browser check TestACardBreathes 5 |
| browser check TestEveryTapTargetIsAtLeast44Pixels 5 |
| browser check TestLightJSOnAnOldThemeWritesOntoNothingItPaints 5 |
| +browser check TestSelectRoomIsCutOnlyByWhatClipsTheList 5 |
| browser check TestTheSheenFollowsADraggedKnob 5 |
| -browser check TestTheTopbarMenuIsACardOverThePage 5 |
| browser check TestALeaveAfterAQueuedMoveNeverRelightsACard 4 |
| browser check TestATblockHeadDragsLikeASwitch 4 |
| browser check TestBothSpellingsComputeTheSameStyles 4 |
| browser check TestReducedMotionIsTotal 4 |
| +browser check TestTheBulkBarNeverWrapsItsText 4 |
| browser check TestTheSlideKnowsWhichWayItIsGoing 4 |
| browser check TestTheWholeRowIsTheTarget 4 |
| -browser check TestTheWideLayoutIsUnchanged 4 |
| browser check TestAFragmentLinkLandsBelowTheBackStrip 3 |
| -browser check TestAModalCoversTheTopbarOnAPhone 3 |
| browser check TestANarrowRowGivesItsTitleTheWidth 3 |
| -browser check TestEscapeClosesTheWholeCard 3 |
| +browser check TestAReadonlyOrDisabledDateFieldCannotBeChangedThroughTheBox 3 |
| +browser check TestAnOptionUnderADisabledGroupCannotBeChosen 3 |
| browser check TestMediaGridFramesAreEvenAndCaptionsStayInTheTile 3 |
| +browser check TestSelectFollowsRequiredAndExtras 3 |
| +browser check TestSelectKeepsItsDividerAndPrompt 3 |
| +browser check TestSelectKeepsPaceWithTyping 3 |
| +browser check TestSelectPromptClearsToItsPlaceholder 3 |
| +browser check TestSelectRebuildKeepsWhatIsInFlight 3 |
| +browser check TestSelectTypingIsCheap 3 |
| browser check TestTheBackControlReusesHistoryWhenItCanProveWhatIsBehind 3 |
| -browser check TestTheBulkBarNeverWrapsItsText 3 |
| -browser check TestTheConsoleRailFitsTheViewport 3 |
| +browser check TestTheConsoleBarIsTheSameCard 3 |
| +browser check TestTheConsoleFoldsItsBarAndIndexesItsRail 3 |
| browser check TestTheLightFollowsThePointerAndSettles 3 |
| +browser check TestTheTopbarMenuIsACardOverThePage 3 |
| +browser check TestTheWideLayoutIsUnchanged 3 |
| +browser check TestTomorrowParsesWhateverTheScriptsTiming 3 |
| +browser check TestADetailListsLabelSitsOnItsValuesFirstLine 2 |
| +browser check TestAFormResetRepaintsTheBox 2 |
| +browser check TestAModalCoversTheTopbarOnAPhone 2 |
| +browser check TestAPITokensFactsStayVisibleOnAPhone 2 |
| +browser check TestAResetInTheTickAfterABlurKeepsTheDefault 2 |
| +browser check TestARowsKebabSitsAtItsTrailingEdge 2 |
| +browser check TestASchemeFlipLandsWhileHovered 2 |
| +browser check TestAToggleDuringASettleTakesItsOwnCurve 2 |
| +browser check TestControlsInARowKeepTheirOwnBoxes 2 |
| +browser check TestDaysDerivedTokensReproduceTheirDocumentedColours 2 |
| +browser check TestEnterCommitsWithoutSubmittingTheForm 2 |
| +browser check TestEveryButtonLabelIsVisible 2 |
| browser check TestMediaGridTileIsTheTarget 2 |
| +browser check TestOneAccentOverrideRecoloursDay 2 |
| browser check TestPaginationIsOneBalancedLineOnAPhone 2 |
| -browser check TestSelectFollowsRequiredAndExtras 2 |
| -browser check TestSelectKeepsItsDividerAndPrompt 2 |
| -browser check TestSelectKeepsPaceWithTyping 2 |
| +browser check TestPrintDrawsBordersAndNoDepth 2 |
| +browser check TestSegTabsKeepAGapBelowThem 2 |
| +browser check TestSelectFocusSelectsThePick 2 |
| browser check TestSelectOpensUpWhenThereIsNoRoomBelow 2 |
| browser check TestSelectOptionClassStylesRowAndBox 2 |
| -browser check TestSelectPlacement 2 |
| -browser check TestSelectRebuildKeepsWhatIsInFlight 2 |
| browser check TestSelectRebuildsWhenItsOptionsChange 2 |
| -browser check TestSelectStepsAsideWhenReplaced 2 |
| -browser check TestSelectTypingIsCheap 2 |
| +browser check TestSelectScanEvent 2 |
| browser check TestTheCalendarDocksAndItsDaysAreTaps 2 |
| -browser check TestTheCardFitsTheSmallestPhone 2 |
| -browser check TestTheConsoleDegradesTheWayItSaysItDoesWithoutHas 2 |
| +browser check TestTheConsoleRailFitsTheViewport 2 |
| +browser check TestTheFilterChipsRemoveFillsItsEnd 2 |
| +browser check TestTheHeaderRuleIsTheThemesTintedHairline 2 |
| +browser check TestTheKnobIsLitAndTheTrackSheenFollowsIt 2 |
| +browser check TestThePhoneHeaderStripIs56Pixels 2 |
| browser check TestTheTrackColourFollowsTheKnob 2 |
| -# Measured 2026-10-29 on docs-t1, one run of internal/gallery with |
| -# /usr/bin/chromium, every test at 2s or more. |
| -browser internal/gallery TestPreviewFrameHeightsFitAtThePhonesNarrowest 290 |
| -browser internal/gallery TestPreviewFrameHeightsFitTheirContent 83 |
| -browser internal/gallery TestEveryExamplePassesAxe 55 |
| -browser internal/gallery TestTheHTMLTabRendersExactlyLikeThePreview 104 |
| -browser internal/gallery TestA11yScansTheGallery 34 |
| +browser check TestWithoutRelativeColourDayKeepsItsCobalt 2 |
| +browser internal/gallery TestPreviewFrameHeightsFitAtThePhonesNarrowest1of12 30 |
| +browser internal/gallery TestPreviewFrameHeightsFitTheirContent1of3 29 |
| +browser internal/gallery TestPreviewFrameHeightsFitTheirContent2of3 28 |
| +browser internal/gallery TestPreviewFrameHeightsFitTheirContent3of3 28 |
| +browser internal/gallery TestPreviewFrameHeightsFitAtThePhonesNarrowest2of12 26 |
| +browser internal/gallery TestPreviewFrameHeightsFitAtThePhonesNarrowest3of12 25 |
| +browser internal/gallery TestTheHTMLTabRendersExactlyLikeThePreviewEn390 25 |
| +browser internal/gallery TestTheHTMLTabRendersExactlyLikeThePreviewEn900 25 |
| +browser internal/gallery TestPreviewFrameHeightsFitAtThePhonesNarrowest10of12 24 |
| +browser internal/gallery TestPreviewFrameHeightsFitAtThePhonesNarrowest11of12 24 |
| +browser internal/gallery TestPreviewFrameHeightsFitAtThePhonesNarrowest12of12 24 |
| +browser internal/gallery TestPreviewFrameHeightsFitAtThePhonesNarrowest4of12 24 |
| +browser internal/gallery TestPreviewFrameHeightsFitAtThePhonesNarrowest7of12 24 |
| +browser internal/gallery TestPreviewFrameHeightsFitAtThePhonesNarrowest8of12 24 |
| +browser internal/gallery TestPreviewFrameHeightsFitAtThePhonesNarrowest9of12 24 |
| +browser internal/gallery TestTheHTMLTabRendersExactlyLikeThePreviewJa390 24 |
| +browser internal/gallery TestTheHTMLTabRendersExactlyLikeThePreviewJa900 24 |
| +browser internal/gallery TestPreviewFrameHeightsFitAtThePhonesNarrowest5of12 23 |
| +browser internal/gallery TestPreviewFrameHeightsFitAtThePhonesNarrowest6of12 23 |
| +browser internal/gallery TestEveryExamplePassesAxeDayEn 21 |
| +browser internal/gallery TestEveryExamplePassesAxePlainEn 15 |
| +browser internal/gallery TestEveryExamplePassesAxeDayAr 13 |
| +browser internal/gallery TestEveryExamplePassesAxeSignalEn 13 |
| +browser internal/gallery TestA11yScansTheGalleryDay 12 |
| +browser internal/gallery TestA11yScansTheGalleryPlain 12 |
| +browser internal/gallery TestA11yScansTheGallerySignal 12 |
| +browser internal/gallery TestTheLeakSweepSeesEscapedPunctuation 12 |
| browser internal/gallery TestThePreviewDefaultIsMonotoneInStageWidth 12 |
| -browser internal/gallery TestThePreviewWidgetIsUsableOnAPhone 10 |
| +browser internal/gallery TestThePreviewWidgetIsUsableOnAPhone 11 |
| +browser internal/gallery TestTheBarsLanguageMenuStaysOnScreen 9 |
| browser internal/gallery TestAFrameStopsLookingForItsNextDocument 8 |
| -browser internal/gallery TestTheBarsLanguageMenuStaysOnScreen 8 |
| -browser internal/gallery TestEveryLookupIsTraced 7 |
| -browser internal/gallery TestTheGalleryHasNoHorizontalScrollAt320 6 |
| +browser internal/gallery TestEveryLookupIsTraced 8 |
| +browser internal/gallery TestTheGalleryHasNoHorizontalScrollAt320 7 |
| +browser internal/gallery TestNoEnglishProseReachesATranslatedPage 6 |
| +browser internal/gallery TestA11yScansTheCodePanels 5 |
| browser internal/gallery TestA11yScansThePhoneViews 5 |
| browser internal/gallery TestAMovedSectionsOldLinkReachesIt 5 |
| browser internal/gallery TestMobileIsThePageAtItsRealSize 5 |
| -browser internal/gallery TestA11yScansTheCodePanels 4 |
| +browser internal/gallery TestThePinnedBarFitsItsReservation 5 |
| +browser internal/gallery TestEveryFrameDocumentCarriesNoindexAndItsPaddingRule 4 |
| +browser internal/gallery TestEveryPageIsAWholeLocalisedDocument 4 |
| +browser internal/gallery TestEveryProseKeyIsTranslated 4 |
| +browser internal/gallery TestFocusIsNeverUnderThePinnedBar 4 |
| browser internal/gallery TestPositionSurvivesASwitch 4 |
| +browser internal/gallery TestTheAssetDeclarationDrivesTheTree 4 |
| +browser internal/gallery TestTheDemoIsFourDocumentsInTheShippedShell 4 |
| browser internal/gallery TestTheDisplaySettingsMenuOnAPhone 4 |
| -browser internal/gallery TestThePinnedBarFitsItsReservation 4 |
| +browser internal/gallery TestThePasswordScreenCarriesItsWarning 4 |
| browser internal/gallery TestAFrameHidesWhileItNavigates 3 |
| browser internal/gallery TestAFrameNeverShowsTheOtherSchemeWhileItLoads 3 |
| browser internal/gallery TestAFrameShowsNothingUntilItsDocumentIsStyled 3 |
| +browser internal/gallery TestAShellDemoLaysOutAsAnAppWould 3 |
| browser internal/gallery TestDesktopPansFromTheInlineStart 3 |
| -browser internal/gallery TestEveryFrameDocumentCarriesNoindexAndItsPaddingRule 3 |
| -browser internal/gallery TestFocusIsNeverUnderThePinnedBar 3 |
| +browser internal/gallery TestEveryLanguageIsReachableFromThePhoneIndex 3 |
| browser internal/gallery TestNothingPaintsOverTheBarOrTheBackStrip 3 |
| browser internal/gallery TestPreviewWidgetDrivesTheWholeJourney 3 |
| -browser internal/gallery TestTheAssetDeclarationDrivesTheTree 3 |
| browser internal/gallery TestTheCopyButtonCopiesAnnouncesAndFailsSafely 3 |
| -browser internal/gallery TestTheDemoIsFourDocumentsInTheShippedShell 3 |
| -browser internal/gallery TestThePasswordScreenCarriesItsWarning 3 |
| -browser internal/gallery TestAShellDemoLaysOutAsAnAppWould 2 |
| +browser internal/gallery TestADemoOpenedInItsOwnTabKeepsTheScheme 2 |
| +browser internal/gallery TestAFrameSubmitsNothing 2 |
| browser internal/gallery TestAStoredSchemeIsOnTheGalleryPageBeforeFirstPaint 2 |
| +browser internal/gallery TestEveryPagesH1IsItsOwnTitle 2 |
| +browser internal/gallery TestNoCodeBlockScrollsSidewaysOnAPhone 2 |
| +browser internal/gallery TestNoPageCarriesTheSameIdTwice 2 |
| +browser internal/gallery TestNoTitleOrStateLabelCarriesAnEmDash 2 |
| +browser internal/gallery TestTheDemoApplicationWorksWithNoScript 2 |
| +browser internal/gallery TestTheDemoRequestsRowsAreClickableEdgeToEdge 2 |
| +browser internal/gallery TestTheDisplaySettingsKeepYourPlace 2 |
| +browser internal/gallery TestTheGallerySwitchLandsOnTheNextFrame 2 |
| +browser internal/gallery TestTheGallerysOwnControlsAreExactlyTheFloor 2 |
| browser internal/gallery TestThePhoneIndexAndTheWayBack 2 |
| +browser internal/gallery TestTheWayBackFromAShellDemoIsATarget 2 |
| +browser internal/gallery TestTitlesUseTheApprovedSeparators 2 |
| +browser internal/www TestTheLandingPassesAxe 16 |
| +browser internal/www TestTheLanguageMenuOpensOnAPhone 14 |
| +browser internal/www TestTheLandingHasNoHorizontalScrollAt320 13 |
| +browser internal/www TestEveryLinkInTheSiteResolves 12 |
| +browser internal/www TestTheSampleFormsStayOnThePage 10 |
| +browser internal/www TestArabicReadsRightToLeft 9 |
| +browser internal/www TestCopy 9 |
| +browser internal/www TestTheSchemeMenuMarksTheRestoredScheme 9 |
| +browser internal/www TestTheThemeComesBackBeforeFirstPaint 9 |
| +browser internal/www TestABrokenStoredThemeFallsBackToDay 8 |
| +browser internal/www TestTheBuntingMovesOnlyWhenMotionIsAllowed 8 |
| +browser internal/www TestTheLandingReadsWithNoScript 8 |
| +browser internal/www TestTheSiteServesFromItsOwnOrigin 8 |
| +browser internal/www TestEveryPageLinksEveryLanguage 7 |
| +browser internal/www TestEverySamplePaneAndCourseSwitches 7 |
| +browser internal/www TestTheBuntingFollowsTheTheme 7 |
| +browser internal/www TestTheSampleSwitchDrags 7 |
| +browser internal/www TestTheStillBuntingMatchesTheMovingOne 7 |
| +browser internal/www TestPagesHasARootAndTheDesignSystem 6 |
| +browser internal/www TestSiteBuildAcceptsThePages 6 |
| +browser internal/www TestStaticFilesAreCopied 6 |
| +browser internal/www TestAHiddenSiteFileIsRefused 5 |
| +browser internal/www TestEveryLanguageHasAPage 5 |
| +browser internal/www TestTheDesignSystemIsMounted 5 |
| +test internal/gallery TestTheLeakSweepSeesEscapedPunctuation 12 |
| +test internal/gallery TestEveryLookupIsTraced 8 |
| +test internal/gallery TestNoEnglishProseReachesATranslatedPage 7 |
| +test internal/gallery TestEveryFrameDocumentCarriesNoindexAndItsPaddingRule 4 |
| +test internal/gallery TestEveryPageIsAWholeLocalisedDocument 4 |
| +test internal/gallery TestEveryProseKeyIsTranslated 4 |
| +test internal/gallery TestTheAssetDeclarationDrivesTheTree 4 |
| +test internal/gallery TestTheDemoIsFourDocumentsInTheShippedShell 4 |
| +test internal/gallery TestThePasswordScreenCarriesItsWarning 4 |
| +test internal/gallery TestNoPageCarriesTheSameIdTwice 3 |
| +test internal/gallery TestEveryPagesH1IsItsOwnTitle 2 |
| +test internal/gallery TestNoTitleOrStateLabelCarriesAnEmDash 2 |
| +test internal/gallery TestTitlesUseTheApprovedSeparators 2 |
| +test cmd/fiesta TestSiteWritesWhatBuildReturns 12 |
| +test cmd/fiesta TestGalleryRefusesADirectoryHoldingARelease 9 |
| +test cmd/fiesta TestGalleryRefusesALinkInsideItsDirectory 9 |
| +test cmd/fiesta TestGalleryRefusesALinkedDirectoryHoldingARelease 9 |
| +test cmd/fiesta TestGalleryWritesTheTreeAtItsMount 9 |
| +test cmd/fiesta TestGalleryRefusesARelease 4 |
| +test internal/www TestEveryLinkInTheSiteResolves 10 |
| +test internal/www TestSiteBuildAcceptsThePages 6 |
| +test internal/www TestEveryLanguageHasAPage 5 |
| +test internal/www TestEveryPageLinksEveryLanguage 5 |
| +test internal/www TestPagesHasARootAndTheDesignSystem 5 |
| +test internal/www TestStaticFilesAreCopied 5 |
| +test internal/www TestTheDesignSystemIsMounted 5 |
| +test internal/www TestAHiddenSiteFileIsRefused 4 |
diff --git a/internal/gallery/a11y_browser_test.go b/internal/gallery/a11y_browser_test.go| index 8f3424a..e9f835f 100644 |
| --- a/internal/gallery/a11y_browser_test.go |
| +++ b/internal/gallery/a11y_browser_test.go |
| @@ -202,6 +202,10 @@ func report(t *testing.T, where string, vs []axe.Violation) int { |
| return len(vs) |
| } |
| +// axeLegs are TestEveryExamplePassesAxe's legs: the themes in English, |
| +// and day in Arabic. Every theme is scanned by one of them. |
| +var axeLegs = []struct{ theme, locale string }{{"day", "en"}, {"plain", "en"}, {"signal", "en"}, {"day", "ar"}} |
| + |
| // TestEveryExamplePassesAxe is the gate: every frame document frameFiles |
| // names, scanned in day, plain and signal with data-theme light and |
| // dark, and in Arabic day, with the WCAG 2.2 AA tag set. Expected: no |
| @@ -210,28 +214,39 @@ func report(t *testing.T, where string, vs []axe.Violation) int { |
| // what a person reviews. Preservation: part 3's gate, iterating |
| // frameFiles so a document a later task adds is scanned the day it |
| // lands. |
| -func TestEveryExamplePassesAxe(t *testing.T) { |
| - rig := harness.New(t, func(string) http.Handler { return newTree(t, nil) }) |
| - ctx, cancel := context.WithTimeout(rig.Context(), 18*time.Minute) |
| - defer cancel() |
| - s := newTestSite(t, fiesta.FS(), mountPath) |
| - m := s.m |
| - |
| - legs := []struct { |
| - theme string |
| - locale string |
| - }{{"day", "en"}, {"plain", "en"}, {"signal", "en"}, {"day", "ar"}} |
| +// |
| +// It is one test per leg, so the suite's slices can run the legs on |
| +// shards of their own: TestEveryExamplePassesAxeDayEn and the three |
| +// after it. Each checks that every theme is scanned by some leg, and |
| +// that its own leg scanned something, which the one test checked of |
| +// all four together. |
| +func TestEveryExamplePassesAxeDayEn(t *testing.T) { examplesPassAxe(t, 0) } |
| +func TestEveryExamplePassesAxePlainEn(t *testing.T) { examplesPassAxe(t, 1) } |
| +func TestEveryExamplePassesAxeSignalEn(t *testing.T) { examplesPassAxe(t, 2) } |
| +func TestEveryExamplePassesAxeDayAr(t *testing.T) { examplesPassAxe(t, 3) } |
| + |
| +// examplesPassAxe is TestEveryExamplePassesAxe's leg i. |
| +func examplesPassAxe(t *testing.T, i int) { |
| + if len(axeLegs) != 4 { |
| + t.Fatalf("axeLegs has %d legs and there are four tests, one per leg; a leg needs a test of its own", len(axeLegs)) |
| + } |
| for _, theme := range fiesta.ThemeNames() { |
| found := false |
| - for _, l := range legs { |
| + for _, l := range axeLegs { |
| found = found || l.theme == theme |
| } |
| if !found { |
| t.Fatalf("theme %s is scanned by no leg", theme) |
| } |
| } |
| + rig := harness.New(t, func(string) http.Handler { return newTree(t, nil) }) |
| + ctx, cancel := context.WithTimeout(rig.Context(), 18*time.Minute) |
| + defer cancel() |
| + s := newTestSite(t, fiesta.FS(), mountPath) |
| + m := s.m |
| + |
| scans, failed := 0, 0 |
| - for _, leg := range legs { |
| + for _, leg := range axeLegs[i : i+1] { |
| name := leg.theme + " " + leg.locale |
| incomplete := map[string]int{} |
| start := time.Now() |
diff --git a/internal/gallery/code_browser_test.go b/internal/gallery/code_browser_test.go| index 84f2be1..5895668 100644 |
| --- a/internal/gallery/code_browser_test.go |
| +++ b/internal/gallery/code_browser_test.go |
| @@ -345,12 +345,37 @@ const disclosureState = `k => new Promise(done => { |
| // so this is what makes pasting it safe. It runs in English and again |
| // in Japanese, on the samples as day/ja shows and copies them: the |
| // words change, and with them where a line may break. |
| -func TestTheHTMLTabRendersExactlyLikeThePreview(t *testing.T) { |
| +// |
| +// It is one test per language and width, so the suite's slices can run |
| +// them on shards of their own: the En900 test and the three after it. |
| +// Each builds every state's two copies in both languages, so the count |
| +// of states the formatter changes is checked whole as before, then |
| +// renders its own language at its own width; and each runs both |
| +// controls first, which the one test ran once. |
| +func TestTheHTMLTabRendersExactlyLikeThePreviewEn900(t *testing.T) { |
| + htmlTabRendersLikeThePreview(t, "en", 900) |
| +} |
| + |
| +func TestTheHTMLTabRendersExactlyLikeThePreviewEn390(t *testing.T) { |
| + htmlTabRendersLikeThePreview(t, "en", 390) |
| +} |
| + |
| +func TestTheHTMLTabRendersExactlyLikeThePreviewJa900(t *testing.T) { |
| + htmlTabRendersLikeThePreview(t, "ja", 900) |
| +} |
| + |
| +func TestTheHTMLTabRendersExactlyLikeThePreviewJa390(t *testing.T) { |
| + htmlTabRendersLikeThePreview(t, "ja", 390) |
| +} |
| + |
| +// htmlTabRendersLikeThePreview is TestTheHTMLTabRendersExactlyLikeThePreview |
| +// in one language at one width. |
| +func htmlTabRendersLikeThePreview(t *testing.T, only string, onlyWidth int64) { |
| s := newTestSite(t, fiesta.FS(), mountPath) |
| theme := fiesta.ThemeNames()[0] |
| docs := map[string]string{} |
| var serveIn func(locale, name string, e example.Entry, body string) string |
| - type job struct{ name, raw, fmt string } |
| + type job struct{ locale, name, raw, fmt string } |
| var jobs []job |
| serve := func(name string, e example.Entry, body string) string { |
| return serveIn("en", name, e, body) |
| @@ -398,7 +423,7 @@ func TestTheHTMLTabRendersExactlyLikeThePreview(t *testing.T) { |
| base := "/fmt/" + name |
| docs[base+"-raw.html"] = serveIn(locale, name, e, raw) |
| docs[base+"-fmt.html"] = serveIn(locale, name, e, formatted) |
| - jobs = append(jobs, job{name, base + "-raw.html", base + "-fmt.html"}) |
| + jobs = append(jobs, job{locale, name, base + "-raw.html", base + "-fmt.html"}) |
| } |
| } |
| } |
| @@ -461,8 +486,14 @@ func TestTheHTMLTabRendersExactlyLikeThePreview(t *testing.T) { |
| } |
| var compared, states int |
| for _, width := range []int64{900, 390} { |
| + if width != onlyWidth { |
| + continue |
| + } |
| mustRun(t, ctx, chromedp.EmulateViewport(width, 900)) |
| for _, j := range jobs { |
| + if j.locale != only { |
| + continue |
| + } |
| want, got := read(j.raw), read(j.fmt) |
| compared++ |
| states += len(want) |
| @@ -471,7 +502,10 @@ func TestTheHTMLTabRendersExactlyLikeThePreview(t *testing.T) { |
| } |
| } |
| } |
| - t.Logf("compared %d samples in %d states, each rendered as written and as formatted", compared, states) |
| + if compared == 0 { |
| + t.Fatalf("no %s sample was compared at %dpx", only, onlyWidth) |
| + } |
| + t.Logf("compared %d %s samples at %dpx in %d states, each rendered as written and as formatted", compared, only, onlyWidth, states) |
| } |
| // sameReadings compares a document's readings state by state, and says |
diff --git a/internal/gallery/heights_browser_test.go b/internal/gallery/heights_browser_test.go| index 7a36f51..efcd3dd 100644 |
| --- a/internal/gallery/heights_browser_test.go |
| +++ b/internal/gallery/heights_browser_test.go |
| @@ -435,15 +435,85 @@ func printTable(t *testing.T, desk, mob map[string]map[string]int, deskFit, mobF |
| t.Logf("the table as measured:\n%s", b.String()) |
| } |
| -// TestPreviewFrameHeightsFitTheirContent: on the Overview and every |
| -// example page of day/en (every locale under FIESTA_SWEEP=full), at a |
| -// 1500px window, every frame's document against its box at Desktop |
| -// and, with Mobile chosen, against --ds-hm; then, at each, what the box |
| -// must hold with each of its menus and panels open. A stage frame |
| -// counts like any other: its document is 100dvh tall, so one that fits |
| -// reads equal to its box. The wide drive grants every frame |
| -// Rastrillo's 48px for content at rest, and none for what opens. |
| -func TestPreviewFrameHeightsFitTheirContent(t *testing.T) { |
| +// The two drives are the gallery's longest tests, so each is dealt into |
| +// parts, one top-level test apiece, which the suite's slices can put on |
| +// shards of their own. A drive's work is its (locale, page) pairs, in |
| +// locale order and heightPages order within each, and the i-th pair |
| +// goes to part i mod n: the parts are a partition of the drive, each |
| +// pair read in exactly one of them. The checks the drive made once over |
| +// everything it read are made where what they check was read: the |
| +// self-check with en/button.html, the demo with en/index.html, each |
| +// sidebar-shell example on every page it is read; and that something |
| +// was measured and opened, in every part, which is only stricter. |
| +// |
| +// Under FIESTA_SWEEP=full a drive is one part again, its first, so the |
| +// table it prints is the whole table; the other parts skip and say so. |
| +// make heights-sweep runs both drives that way. |
| +const ( |
| + wideHeightParts = 3 |
| + narrowHeightParts = 12 |
| +) |
| + |
| +// heightsUnit is one page of one locale, a drive's unit of work. |
| +type heightsUnit struct{ locale, file string } |
| + |
| +// heightsPart is part k of n of a drive over locales: its pairs, dealt |
| +// as above. |
| +func heightsPart(t *testing.T, locales []string, k, n int) []heightsUnit { |
| + t.Helper() |
| + if sweepFull() { |
| + if k > 1 { |
| + t.Skipf("FIESTA_SWEEP=full reads the whole drive in part 1 of %d, so the table it prints is whole", n) |
| + } |
| + n = 1 |
| + } |
| + var out []heightsUnit |
| + i := 0 |
| + pages := heightPages(t) |
| + for _, locale := range locales { |
| + for _, file := range pages { |
| + if i%n == k-1 { |
| + out = append(out, heightsUnit{locale, file}) |
| + } |
| + i++ |
| + } |
| + } |
| + if len(out) == 0 { |
| + t.Fatalf("part %d of %d of %d pages in %d locale(s) was dealt nothing", k, n, len(pages), len(locales)) |
| + } |
| + return out |
| +} |
| + |
| +// sidebarShellPages are the examples whose frames hold a sidebar shell: |
| +// each, wherever a narrow part reads it, must read as one, which is |
| +// what proves the 48px of grace goes where it should. A name here the |
| +// manifest no longer has fails, so the list cannot go quietly stale. |
| +var sidebarShellPages = []string{"layout-sidebar.html", "layout-sidebar-page.html", "shell-sidebar.html"} |
| + |
| +// TestPreviewFrameHeightsFitTheirContent1of3 to 3of3: on the Overview |
| +// and every example page of day/en (every locale under |
| +// FIESTA_SWEEP=full), at a 1500px window, every frame's document |
| +// against its box at Desktop and, with Mobile chosen, against --ds-hm; |
| +// then, at each, what the box must hold with each of its menus and |
| +// panels open. A stage frame counts like any other: its document is |
| +// 100dvh tall, so one that fits reads equal to its box. The wide drive |
| +// grants every frame Rastrillo's 48px for content at rest, and none for |
| +// what opens. Preservation: the one wide drive, dealt into parts. |
| +func TestPreviewFrameHeightsFitTheirContent1of3(t *testing.T) { |
| + wideHeights(t, 1, wideHeightParts) |
| +} |
| + |
| +func TestPreviewFrameHeightsFitTheirContent2of3(t *testing.T) { |
| + wideHeights(t, 2, wideHeightParts) |
| +} |
| + |
| +func TestPreviewFrameHeightsFitTheirContent3of3(t *testing.T) { |
| + wideHeights(t, 3, wideHeightParts) |
| +} |
| + |
| +// wideHeights is part k of n of the wide drive. |
| +func wideHeights(t *testing.T, k, n int) { |
| + units := heightsPart(t, wideLocales(), k, n) |
| rig := harness.New(t, func(string) http.Handler { return treeHandler(t) }) |
| deadline := 12 * time.Minute |
| if sweepFull() { |
| @@ -453,6 +523,7 @@ func TestPreviewFrameHeightsFitTheirContent(t *testing.T) { |
| defer cancel() |
| desk, mob := map[string]map[string]int{}, map[string]map[string]int{} |
| measured, opened, short := 0, 0, 0 |
| + readDemo := false |
| keep := func(m map[string]int, key string, n int) { |
| if n > m[key] { |
| m[key] = n |
| @@ -469,74 +540,140 @@ func TestPreviewFrameHeightsFitTheirContent(t *testing.T) { |
| } |
| fit[key] = ok |
| } |
| - for _, locale := range wideLocales() { |
| - desk[locale], mob[locale] = map[string]int{}, map[string]int{} |
| - for _, file := range heightPages(t) { |
| - where := fmt.Sprintf("%s/%s at 1500px", locale, file) |
| - openPage(t, ctx, where, pageURL(rig, locale, file), 1500, 1000) |
| - eagerly(t, ctx, where) |
| - desktop := readHeights(t, ctx, where+" Desktop") |
| - for key, r := range desktop { |
| - measured++ |
| - keep(desk[locale], key, r.Need) |
| - fits(deskFit, key, r) |
| - if shortfall(t, where+" Desktop", key, r, 48) { |
| - short++ |
| - } |
| + for _, u := range units { |
| + locale, file := u.locale, u.file |
| + if desk[locale] == nil { |
| + desk[locale], mob[locale] = map[string]int{}, map[string]int{} |
| + } |
| + where := fmt.Sprintf("%s/%s at 1500px", locale, file) |
| + openPage(t, ctx, where, pageURL(rig, locale, file), 1500, 1000) |
| + eagerly(t, ctx, where) |
| + desktop := readHeights(t, ctx, where+" Desktop") |
| + for key, r := range desktop { |
| + measured++ |
| + keep(desk[locale], key, r.Need) |
| + fits(deskFit, key, r) |
| + if shortfall(t, where+" Desktop", key, r, 48) { |
| + short++ |
| } |
| - for key, o := range readOpen(t, ctx, where+" Desktop", "--ds-h") { |
| - opened++ |
| - keep(desk[locale], key, o.Need) |
| - deskFit[key] = false |
| - if openShortfall(t, where+" Desktop", key, o, desktop[key].Box) { |
| - short++ |
| - } |
| + } |
| + for key, o := range readOpen(t, ctx, where+" Desktop", "--ds-h") { |
| + opened++ |
| + keep(desk[locale], key, o.Need) |
| + deskFit[key] = false |
| + if openShortfall(t, where+" Desktop", key, o, desktop[key].Box) { |
| + short++ |
| } |
| - clickAll(t, ctx, where, clickedMobile, "Mobile") |
| - mobileSettle(t, ctx, where) |
| - mobile := readHeights(t, ctx, where+" Mobile") |
| - for key, r := range mobile { |
| - keep(mob[locale], key, r.Need) |
| - fits(mobFit, key, r) |
| - if shortfall(t, where+" Mobile", key, r, 48) { |
| - short++ |
| - } |
| + } |
| + clickAll(t, ctx, where, clickedMobile, "Mobile") |
| + mobileSettle(t, ctx, where) |
| + mobile := readHeights(t, ctx, where+" Mobile") |
| + for key, r := range mobile { |
| + keep(mob[locale], key, r.Need) |
| + fits(mobFit, key, r) |
| + if shortfall(t, where+" Mobile", key, r, 48) { |
| + short++ |
| } |
| - for key, o := range readOpen(t, ctx, where+" Mobile", "--ds-hm") { |
| - keep(mob[locale], key, o.Need) |
| - mobFit[key] = false |
| - if openShortfall(t, where+" Mobile", key, o, mobile[key].Box) { |
| - short++ |
| - } |
| + } |
| + for key, o := range readOpen(t, ctx, where+" Mobile", "--ds-hm") { |
| + keep(mob[locale], key, o.Need) |
| + mobFit[key] = false |
| + if openShortfall(t, where+" Mobile", key, o, mobile[key].Box) { |
| + short++ |
| } |
| - if file == "button.html" && locale == "en" { |
| - selfCheck(t, ctx, where+" Mobile", "--ds-hm", mobile) |
| - clickAll(t, ctx, where, clickedDesktop, "Desktop") |
| - selfCheck(t, ctx, where+" Desktop", "--ds-h", desktop) |
| + } |
| + if file == "button.html" && locale == "en" { |
| + selfCheck(t, ctx, where+" Mobile", "--ds-hm", mobile) |
| + clickAll(t, ctx, where, clickedDesktop, "Desktop") |
| + selfCheck(t, ctx, where+" Desktop", "--ds-h", desktop) |
| + } |
| + if file == "index.html" && locale == "en" { |
| + if _, ok := desk["en"]["demo"]; !ok { |
| + t.Fatal("the Overview's demo was not measured") |
| } |
| + readDemo = true |
| } |
| } |
| if measured == 0 || opened == 0 { |
| t.Fatalf("%d frames measured and %d opened; the sweep read too little", measured, opened) |
| } |
| - if _, ok := desk["en"]["demo"]; !ok { |
| - t.Fatal("the Overview's demo was not measured") |
| - } |
| - t.Logf("%d frames measured and %d opened over %d locale(s); %d short (enforced: %v)", measured, opened, len(desk), short, heightsEnforced) |
| + t.Logf("part %d: %d pages, %d frames measured and %d opened over %d locale(s); %d short (enforced: %v; the Overview's demo read here: %v)", k, len(units), measured, opened, len(desk), short, heightsEnforced, readDemo) |
| if sweepFull() { |
| printTable(t, desk, mob, deskFit, mobFit) |
| } |
| } |
| -// TestPreviewFrameHeightsFitAtThePhonesNarrowest: the same with Mobile |
| -// chosen at a 320px viewport, where a Mobile frame lays out at the |
| -// stage's width and text that fits a line at 390px wraps, over en, ar, |
| -// ru and ga (all twelve under the sweep), closed and open. Only a frame |
| -// holding a sidebar shell gets the 48px of grace, as Rastrillo's |
| -// narrow drive grants it: the rail is 100dvh tall and no box can fit |
| -// it; every other frame gets none. The control: every frame is laid |
| -// out under 390px, at its stage's width. |
| -func TestPreviewFrameHeightsFitAtThePhonesNarrowest(t *testing.T) { |
| +// TestPreviewFrameHeightsFitAtThePhonesNarrowest1of12 to 12of12: the same |
| +// with Mobile chosen at a 320px viewport, where a Mobile frame lays out |
| +// at the stage's width and text that fits a line at 390px wraps, over |
| +// en, ar, ru and ga (all twelve under the sweep), closed and open. Only |
| +// a frame holding a sidebar shell gets the 48px of grace, as |
| +// Rastrillo's narrow drive grants it: the rail is 100dvh tall and no |
| +// box can fit it; every other frame gets none. The control: every frame |
| +// is laid out under 390px, at its stage's width. Preservation: the one |
| +// narrow drive, dealt into parts. |
| +func TestPreviewFrameHeightsFitAtThePhonesNarrowest1of12(t *testing.T) { |
| + narrowHeights(t, 1, narrowHeightParts) |
| +} |
| + |
| +func TestPreviewFrameHeightsFitAtThePhonesNarrowest2of12(t *testing.T) { |
| + narrowHeights(t, 2, narrowHeightParts) |
| +} |
| + |
| +func TestPreviewFrameHeightsFitAtThePhonesNarrowest3of12(t *testing.T) { |
| + narrowHeights(t, 3, narrowHeightParts) |
| +} |
| + |
| +func TestPreviewFrameHeightsFitAtThePhonesNarrowest4of12(t *testing.T) { |
| + narrowHeights(t, 4, narrowHeightParts) |
| +} |
| + |
| +func TestPreviewFrameHeightsFitAtThePhonesNarrowest5of12(t *testing.T) { |
| + narrowHeights(t, 5, narrowHeightParts) |
| +} |
| + |
| +func TestPreviewFrameHeightsFitAtThePhonesNarrowest6of12(t *testing.T) { |
| + narrowHeights(t, 6, narrowHeightParts) |
| +} |
| + |
| +func TestPreviewFrameHeightsFitAtThePhonesNarrowest7of12(t *testing.T) { |
| + narrowHeights(t, 7, narrowHeightParts) |
| +} |
| + |
| +func TestPreviewFrameHeightsFitAtThePhonesNarrowest8of12(t *testing.T) { |
| + narrowHeights(t, 8, narrowHeightParts) |
| +} |
| + |
| +func TestPreviewFrameHeightsFitAtThePhonesNarrowest9of12(t *testing.T) { |
| + narrowHeights(t, 9, narrowHeightParts) |
| +} |
| + |
| +func TestPreviewFrameHeightsFitAtThePhonesNarrowest10of12(t *testing.T) { |
| + narrowHeights(t, 10, narrowHeightParts) |
| +} |
| + |
| +func TestPreviewFrameHeightsFitAtThePhonesNarrowest11of12(t *testing.T) { |
| + narrowHeights(t, 11, narrowHeightParts) |
| +} |
| + |
| +func TestPreviewFrameHeightsFitAtThePhonesNarrowest12of12(t *testing.T) { |
| + narrowHeights(t, 12, narrowHeightParts) |
| +} |
| + |
| +// narrowHeights is part k of n of the narrow drive. |
| +func narrowHeights(t *testing.T, k, n int) { |
| + units := heightsPart(t, narrowLocales(), k, n) |
| + pages := map[string]bool{} |
| + for _, file := range heightPages(t) { |
| + pages[file] = true |
| + } |
| + shellPage := map[string]bool{} |
| + for _, file := range sidebarShellPages { |
| + if !pages[file] { |
| + t.Fatalf("%s is listed as a sidebar shell's page, but the manifest has no such example", file) |
| + } |
| + shellPage[file] = true |
| + } |
| rig := harness.New(t, func(string) http.Handler { return treeHandler(t) }) |
| deadline := 15 * time.Minute |
| if sweepFull() { |
| @@ -545,61 +682,66 @@ func TestPreviewFrameHeightsFitAtThePhonesNarrowest(t *testing.T) { |
| ctx, cancel := context.WithTimeout(rig.Context(), deadline) |
| defer cancel() |
| measured, opened, short, shells := 0, 0, 0, 0 |
| + locales := map[string]bool{} |
| var raise []string |
| - for _, locale := range narrowLocales() { |
| - for _, file := range heightPages(t) { |
| - where := fmt.Sprintf("%s/%s at 320px", locale, file) |
| - openPage(t, ctx, where, pageURL(rig, locale, file), 320, 844) |
| - eagerly(t, ctx, where) |
| - clickAll(t, ctx, where, clickedMobile, "Mobile") |
| - mobileSettle(t, ctx, where) |
| - var wide []string |
| - mustRun(t, ctx, chromedp.Evaluate(`[...document.querySelectorAll(".ds-view")].filter(v => { |
| - const f = v.querySelector(".ds-view__frame"); return !(f.offsetWidth < 390 && Math.abs(f.offsetWidth - v.clientWidth) <= 1); }).map(v => v.querySelector(".ds-view__frame").title)`, &wide)) |
| - if len(wide) > 0 { |
| - t.Fatalf("%s: frames %v are not at the stage's narrow width; the readings would be of the 390px frame", where, wide) |
| + for _, u := range units { |
| + locale, file := u.locale, u.file |
| + locales[locale] = true |
| + where := fmt.Sprintf("%s/%s at 320px", locale, file) |
| + openPage(t, ctx, where, pageURL(rig, locale, file), 320, 844) |
| + eagerly(t, ctx, where) |
| + clickAll(t, ctx, where, clickedMobile, "Mobile") |
| + mobileSettle(t, ctx, where) |
| + var wide []string |
| + mustRun(t, ctx, chromedp.Evaluate(`[...document.querySelectorAll(".ds-view")].filter(v => { |
| + const f = v.querySelector(".ds-view__frame"); return !(f.offsetWidth < 390 && Math.abs(f.offsetWidth - v.clientWidth) <= 1); }).map(v => v.querySelector(".ds-view__frame").title)`, &wide)) |
| + if len(wide) > 0 { |
| + t.Fatalf("%s: frames %v are not at the stage's narrow width; the readings would be of the 390px frame", where, wide) |
| + } |
| + got := readHeights(t, ctx, where) |
| + need := map[string]int{} |
| + pageShells := 0 |
| + for key, r := range got { |
| + measured++ |
| + grace := 0 |
| + if r.Shell { |
| + grace, shells, pageShells = 48, shells+1, pageShells+1 |
| + } |
| + need[key] = r.Need |
| + if shortfall(t, where, key, r, grace) { |
| + short++ |
| } |
| - got := readHeights(t, ctx, where) |
| - need := map[string]int{} |
| - for key, r := range got { |
| - measured++ |
| - grace := 0 |
| - if r.Shell { |
| - grace, shells = 48, shells+1 |
| - } |
| - need[key] = r.Need |
| - if shortfall(t, where, key, r, grace) { |
| - short++ |
| - } |
| + } |
| + if shellPage[file] && pageShells == 0 { |
| + t.Errorf("%s: no frame read as a sidebar shell, so the drive cannot say the grace went where it should", where) |
| + } |
| + for key, o := range readOpen(t, ctx, where, "--ds-hm") { |
| + opened++ |
| + if o.Need > need[key] { |
| + need[key] = o.Need |
| } |
| - for key, o := range readOpen(t, ctx, where, "--ds-hm") { |
| - opened++ |
| - if o.Need > need[key] { |
| - need[key] = o.Need |
| - } |
| - if openShortfall(t, where, key, o, got[key].Box) { |
| - short++ |
| - } |
| + if openShortfall(t, where, key, o, got[key].Box) { |
| + short++ |
| } |
| - for key, n := range need { |
| - grace := 0 |
| - if got[key].Shell { |
| - grace = 48 |
| - } |
| - if n > got[key].Box+grace { |
| - raise = append(raise, fmt.Sprintf("%s %q: %d,", locale, key, (n+20+9)/10*10)) |
| - } |
| + } |
| + for key, n := range need { |
| + grace := 0 |
| + if got[key].Shell { |
| + grace = 48 |
| } |
| - if file == "button.html" && locale == "en" { |
| - selfCheck(t, ctx, where, "--ds-hm", got) |
| + if n > got[key].Box+grace { |
| + raise = append(raise, fmt.Sprintf("%s %q: %d,", locale, key, (n+20+9)/10*10)) |
| } |
| } |
| + if file == "button.html" && locale == "en" { |
| + selfCheck(t, ctx, where, "--ds-hm", got) |
| + } |
| } |
| - if measured == 0 || opened == 0 || shells == 0 { |
| - t.Fatalf("%d frames measured, %d opened, %d of them sidebar shells; the sweep read too little to say the grace went where it should", measured, opened, shells) |
| + if measured == 0 || opened == 0 { |
| + t.Fatalf("%d frames measured and %d opened; the sweep read too little", measured, opened) |
| } |
| sort.Strings(raise) |
| - t.Logf("%d frames measured and %d opened at 320px over %d locale(s), %d sidebar shells given grace; %d short (enforced: %v)", measured, opened, len(narrowLocales()), shells, short, heightsEnforced) |
| + t.Logf("part %d: %d pages, %d frames measured and %d opened at 320px over %d locale(s), %d sidebar shells given grace; %d short (enforced: %v)", k, len(units), measured, opened, len(locales), shells, short, heightsEnforced) |
| if len(raise) > 0 { |
| t.Logf("Mobile entries to raise, as locale key: height:\n\t%s", strings.Join(raise, "\n\t")) |
| } |
diff --git a/internal/gallery/pages_a11y_browser_test.go b/internal/gallery/pages_a11y_browser_test.go| index 4b5d4eb..e0a6ed4 100644 |
| --- a/internal/gallery/pages_a11y_browser_test.go |
| +++ b/internal/gallery/pages_a11y_browser_test.go |
| @@ -7,6 +7,7 @@ import ( |
| "fmt" |
| "net/http" |
| "sort" |
| + "strings" |
| "testing" |
| "time" |
| @@ -41,7 +42,7 @@ func a11yPages(s *site) []string { |
| return append(out, "button.html", "dropdown.html", "signin.html", "format-money.html", "layout-sidebar.html", "modal.html", "demo.html", "demo-modal.html") |
| } |
| -// TestA11yScansTheGallery is the gate over the gallery's own pages, |
| +// TestA11yScansTheGalleryDay, Plain and Signal are the gate over the gallery's own pages, |
| // which carry the design system's explanations around the samples: |
| // a11yPages in day, plain and signal, with data-theme light and dark, |
| // in English and in Arabic, at 1280 (where the rail, the bar and the |
| @@ -58,14 +59,30 @@ func a11yPages(s *site) []string { |
| // lead. The rail's accessible name removed is not one: two unnamed |
| // navigation landmarks are axe's landmark-unique, a best-practice rule |
| // outside the WCAG 2.2 AA tag set this gate runs. |
| -func TestA11yScansTheGallery(t *testing.T) { |
| +// |
| +// It is one test per theme, so the suite's slices can run them on |
| +// shards of their own. Each checks that the themes are exactly the |
| +// three the tests name, so a theme added later cannot go unscanned, |
| +// and counts its own scans, as the one test counted all of them. |
| +func TestA11yScansTheGalleryDay(t *testing.T) { a11yScansTheGallery(t, "day") } |
| +func TestA11yScansTheGalleryPlain(t *testing.T) { a11yScansTheGallery(t, "plain") } |
| +func TestA11yScansTheGallerySignal(t *testing.T) { a11yScansTheGallery(t, "signal") } |
| + |
| +// a11yScanThemes are the themes the three tests above scan, one each. |
| +var a11yScanThemes = []string{"day", "plain", "signal"} |
| + |
| +// a11yScansTheGallery is the gallery's pages scanned in one theme. |
| +func a11yScansTheGallery(t *testing.T, only string) { |
| + if got := fiesta.ThemeNames(); strings.Join(got, " ") != strings.Join(a11yScanThemes, " ") { |
| + t.Fatalf("the themes are %v and the gallery's pages are scanned in %v, one test each; a theme needs a test of its own", got, a11yScanThemes) |
| + } |
| rig := harness.New(t, func(string) http.Handler { return treeHandler(t) }) |
| ctx, cancel := context.WithTimeout(rig.Context(), 18*time.Minute) |
| defer cancel() |
| pages := a11yPages(newTestSite(t, fiesta.FS(), mountPath)) |
| scans, failed := 0, 0 |
| - for _, theme := range fiesta.ThemeNames() { |
| + for _, theme := range []string{only} { |
| for _, locale := range []string{"en", "ar"} { |
| name := theme + " " + locale |
| incomplete := map[string]int{} |
| @@ -108,7 +125,7 @@ func TestA11yScansTheGallery(t *testing.T) { |
| t.Logf("%s: %d pages scanned in %s; needs review, axe could not decide: %v", name, len(pages), time.Since(start).Round(time.Second), ids) |
| } |
| } |
| - if want := len(fiesta.ThemeNames()) * 2 * len(pages) * 2; scans != want { |
| + if want := 2 * len(pages) * 2; scans != want { |
| t.Fatalf("%d scans, want %d", scans, want) |
| } |
| t.Logf("%d scans, %d violations", scans, failed) |
diff --git a/releases_test.go b/releases_test.go| index 557f12a..1e09ab7 100644 |
| --- a/releases_test.go |
| +++ b/releases_test.go |
| @@ -136,6 +136,41 @@ func isTagged(t *testing.T, version string) bool { |
| return strings.TrimSpace(string(gitOut(t, "tag", "-l", "v"+version))) != "" |
| } |
| +// requireReleaseTags fails a run that is required to see the release |
| +// tags and does not. Both tests below read the tags to decide what to |
| +// check, and a clone without them (a runner's one-commit clone, which |
| +// git gives no tags) would pass them on nothing: the tag check over no |
| +// tag, and the fresh-dist check on a release that is in fact tagged. |
| +// FIESTA_TEST_REQUIRE_TAGS is set by the Makefile, as the gate's other |
| +// requirements are, and make history fetches the tags into a shallow |
| +// clone before the suite runs. Required, git must see a vX.Y.Z tag for |
| +// every directory under releases/ but the current version's, which may |
| +// be an untagged candidate, and at least one tag in all. |
| +func requireReleaseTags(t *testing.T) { |
| + t.Helper() |
| + if os.Getenv("FIESTA_TEST_REQUIRE_TAGS") == "" { |
| + return |
| + } |
| + tags := map[string]bool{} |
| + for _, line := range strings.Split(strings.TrimSpace(string(gitOut(t, "tag", "-l", "v*"))), "\n") { |
| + if line != "" { |
| + tags[line] = true |
| + } |
| + } |
| + if len(tags) == 0 { |
| + t.Fatal("FIESTA_TEST_REQUIRE_TAGS is set and git sees no release tag; fetch them (make history does, in a shallow clone) or unset it to run without") |
| + } |
| + entries, err := os.ReadDir(releasesDir) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + for _, e := range entries { |
| + if e.IsDir() && e.Name() != Version && !tags["v"+e.Name()] { |
| + t.Errorf("FIESTA_TEST_REQUIRE_TAGS is set and %s/%s has no tag v%s in this clone; fetch the tags", releasesDir, e.Name(), e.Name()) |
| + } |
| + } |
| +} |
| + |
| // TestAnUntaggedReleaseIsAFreshDist: until v<Version> is tagged, the |
| // release directory is exactly what build.Dist makes from the tree |
| // today, byte for byte, with nothing missing and nothing extra. The |
| @@ -145,6 +180,7 @@ func isTagged(t *testing.T, version string) bool { |
| // instead, and a later change to the tree waits for the next version. |
| func TestAnUntaggedReleaseIsAFreshDist(t *testing.T) { |
| requireThisRepository(t) |
| + requireReleaseTags(t) |
| if isTagged(t, Version) { |
| t.Skipf("v%s is tagged, so releases/%s is held to its tag by TestPublishedReleasesMatchTheirTags; a change to the tree is the next version's", Version, Version) |
| } |
| @@ -191,6 +227,7 @@ var releaseTag = regexp.MustCompile(`^v([0-9]+\.[0-9]+\.[0-9]+)$`) |
| // this repository, or the test fails. |
| func TestPublishedReleasesMatchTheirTags(t *testing.T) { |
| requireThisRepository(t) |
| + requireReleaseTags(t) |
| out := gitOut(t, "tag", "-l", "v*") |
| var tags []string |
| for _, line := range strings.Split(strings.TrimSpace(string(out)), "\n") { |