amadan
Promise a release a year only once its tag holds it
Every release directory got a year's immutable cache, the untagged candidate included. A site shipped with a candidate, the candidate rebuilt with fiesta release -candidate, and the site shipped again would leave every browser that had loaded /1.1.0/ holding the first bytes for a year at the same address. SHA256SUMS cannot prevent it: each build is checked on its own, and both pass. So the year is opt-in. By default site.Build gives each release the five minutes /M/ has, still open to other origins, and every build but the one that ships keeps that default. Rules.Published names the releases that get the year, and Build refuses a name that is not one of its releases, or is named twice, rather than drop it. Build reads no repository, so whether a release is published is checked by the caller: fiesta site -published X.Y.Z,... -git REPO compares each against its tag vX.Y.Z there, file for file, the stamp included, and refuses before anything is built or written, naming every difference. The comparison is site.TagDifferences, and TestPublishedReleasesMatchTheirTags now runs it too, so the check that guards releases/ on every push and the one that guards the ship cannot drift apart. -published without -git, and -git alone, are refused. Requiring a tag was chosen over trusting a list typed at ship time: a version named by hand is exactly the mistake this prevents. Tests. New, each red by assertion against a stub that kept the old behaviour and parsed the flags without acting on them: TestAnUnpublishedBuildPromisesNoReleaseAYear, TestAPublishedVersionMustBeAReleaseHere and TestSitePromisesAYearOnlyToTaggedReleases (through fiesta site, in a scratch repository holding the committed 1.0.0 and a candidate: no -git, -git alone, a non-version, no tag, a candidate, a tagged release with a file changed, one added, one removed and one made a link with its sums rewritten to agree, a tag holding no release, and a tagged release missing; then 1.0.0 gets a year and the candidate five minutes). Adapted: the whole-file headers test now has a candidate group and is renamed for it; TestTheSitesHeaders checks the built site twice, with nothing published and with every release but the current one published. TestPublishedReleasesMatchTheirTags is a preservation test: on real data it still names a changed, a missing and an extra file. Mutations, each red and reverted: every release promised a year; the tag check skipped; changed bytes, extra files, missing files, links, an absent tag, a tag without the directory and a missing directory each ignored; Build taking a duplicate or an unknown version; fiesta site not passing the list to Build. Weights: TestSitePromisesAYearOnlyToTaggedReleases 13 seconds on docs-t1; TestTheSitesHeaders 6, now that it builds twice.
Human
Prompt
Important: an untagged candidate gets immutable caching (internal/site/site.go:313). Shipping a candidate tree, rebuilding the candidate, and shipping again would leave browsers holding old bytes for a year at the same URL. The site must only promise immutability for bytes that are published. Approach I'd suggest; improve it if you find better: by default, release directories get the same short rule as /1/* (public, max-age=300, still with CORS and CORP), so dev, test and candidate builds are always safe. The year-long immutable rule is opt-in, through a flag naming the published versions. Red first: a candidate-only build must emit no immutable rule. A publish build with a version that has no tag, or whose bytes differ from the tag, must refuse. Test that through the Go entry point.
From this commit's own message. The prompt ledger has no entries for it — which is what a squashed, rebased or amended commit looks like, and what every commit looks like in a repo without a ledger.
Summary
Releases get five minutes unless fiesta site -published names them, and each named release must match its tag file for file.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
10 files changed,
+736
−130
cmd/fiesta/main.go+49 −11cmd/fiesta/publish_test.go+263 −0cmd/fiesta/release_test.go+42 −35hack/test-weights.txt+2 −1internal/site/site.go+54 −19internal/site/site_test.go+61 −9internal/site/tagged.go+167 −0internal/www/headers_test.go+70 −20internal/www/serve_test.go+20 −2releases_test.go+8 −33
Intent
diff --git a/cmd/fiesta/main.go b/cmd/fiesta/main.go| index 841ed28..be08b42 100644 |
| --- a/cmd/fiesta/main.go |
| +++ b/cmd/fiesta/main.go |
| @@ -29,14 +29,18 @@ |
| // an unpublished candidate may be rebuilt until it is tagged. See |
| // release. |
| // |
| -// fiesta site -releases DIR -site DIR -out DIR |
| +// fiesta site -releases DIR -site DIR -out DIR [-published X.Y.Z,... -git REPO] |
| // |
| // assembles the site from the releases and the site source, exactly |
| // what internal/site.Build returns, into DIR/public/: each release |
| // under its exact version, _redirects sending /M/* to the newest |
| // M.y.z and the design system's former addresses to their pages, |
| -// _headers caching each release for a year and /M/ for five minutes, |
| -// and the pages at the root. The pages are what internal/www assembles |
| +// _headers caching /M/ and each release for five minutes, and the |
| +// pages at the root. A release named in -published is cached for a |
| +// year instead, as immutable, and each must match its tag vX.Y.Z in |
| +// the git repository REPO, file for file, or nothing is written: only |
| +// a tag's bytes can never change. make site-publish passes both for |
| +// the site that ships. The pages are what internal/www assembles |
| // from the site source: the landing page, the design system under |
| // /design-system/, and the source's static/ files. DIR/public/ is the |
| // tree to ship, and the stamp sits above it in DIR, so nothing hidden |
| @@ -70,7 +74,8 @@ import ( |
| ) |
| const usage = "usage: fiesta build -out DIR | fiesta gallery -out DIR [-mount /path] | " + |
| - "fiesta release -version X.Y.Z [-releases DIR] [-candidate] | fiesta site -releases DIR -site DIR -out DIR" |
| + "fiesta release -version X.Y.Z [-releases DIR] [-candidate] | " + |
| + "fiesta site -releases DIR -site DIR -out DIR [-published X.Y.Z,... -git REPO]" |
| func main() { |
| if err := run(os.Args[1:], os.Stderr); err != nil { |
| @@ -167,6 +172,8 @@ func siteCmd(args []string, stderr io.Writer) error { |
| releases := flags.String("releases", "", "the directory of committed releases, one X.Y.Z/ each") |
| siteDir := flags.String("site", "", "the site source: its static/ is copied to the site's root") |
| out := flags.String("out", "", "directory to write the site into") |
| + published := flags.String("published", "", "comma-separated releases browsers may keep for a year, each matching its tag vX.Y.Z in -git") |
| + repo := flags.String("git", "", "the git repository whose tags hold the -published releases") |
| if ok, err := parse(flags, args, stderr); !ok { |
| return err |
| } |
| @@ -175,7 +182,20 @@ func siteCmd(args []string, stderr io.Writer) error { |
| return errors.New(f.name + " is required\n" + usage) |
| } |
| } |
| - return writeSite(*releases, *siteDir, *out) |
| + var versions []string |
| + if *published != "" { |
| + versions = strings.Split(*published, ",") |
| + } |
| + // Each needs the other: a -git with nothing published is a ship |
| + // command missing its list, and a list with no repository has |
| + // nothing to be checked against. |
| + switch { |
| + case len(versions) > 0 && *repo == "": |
| + return errors.New("-published needs -git, the repository whose tags hold those releases\n" + usage) |
| + case len(versions) == 0 && *repo != "": |
| + return errors.New("-git is only for -published, and nothing is published\n" + usage) |
| + } |
| + return writeSite(*releases, *siteDir, *out, versions, *repo) |
| } |
| // write builds the release and puts it under out/<version>/, a |
| @@ -206,16 +226,34 @@ func writeGallery(out, mount string) error { |
| // keeps the command's own mark out too. |
| const shipDir = "public" |
| -// writeSite assembles the pages from the site source, then the site |
| -// from the releases, those pages and their rules, and puts it in |
| -// out/public/, after claiming out. A fault in the site source is |
| -// reported on its own, before the releases are read. |
| -func writeSite(releases, siteDir, out string) error { |
| +// writeSite checks each published release against its tag in repo, |
| +// assembles the pages from the site source, then the site from the |
| +// releases, those pages and their rules, and puts it in out/public/, |
| +// after claiming out. A release that is not its tag's is refused |
| +// before anything is built or written, every difference named: the |
| +// year's cache is a promise the bytes never change, and only a tag |
| +// keeps it. A fault in the site source is reported on its own, before |
| +// the releases are read. |
| +func writeSite(releases, siteDir, out string, published []string, repo string) error { |
| + var differences []string |
| + for _, version := range published { |
| + diffs, err := site.TagDifferences(repo, version, filepath.Join(releases, version)) |
| + if err != nil { |
| + return fmt.Errorf("refusing to publish %s: could not compare it with its tag: %w", version, err) |
| + } |
| + differences = append(differences, diffs...) |
| + } |
| + if len(differences) > 0 { |
| + return fmt.Errorf("refusing to publish %s, since browsers would keep it for a year: %s", |
| + strings.Join(published, ", "), strings.Join(differences, "; ")) |
| + } |
| pages, err := www.Pages(fiesta.FS(), os.DirFS(siteDir)) |
| if err != nil { |
| return err |
| } |
| - files, err := site.Build(os.DirFS(releases), www.FS(pages), www.Rules(pages)) |
| + rules := www.Rules(pages) |
| + rules.Published = published |
| + files, err := site.Build(os.DirFS(releases), www.FS(pages), rules) |
| if err != nil { |
| return err |
| } |
diff --git a/cmd/fiesta/publish_test.go b/cmd/fiesta/publish_test.go| new file mode 100644 |
| index 0000000..2e24ebd |
| --- /dev/null |
| +++ b/cmd/fiesta/publish_test.go |
| @@ -0,0 +1,263 @@ |
| +package main |
| + |
| +import ( |
| + "bytes" |
| + "crypto/sha256" |
| + "encoding/hex" |
| + "io" |
| + "io/fs" |
| + "maps" |
| + "os" |
| + "path/filepath" |
| + "regexp" |
| + "sort" |
| + "strings" |
| + "testing" |
| + |
| + "amadan.net/rastrillo/fiesta" |
| + "amadan.net/rastrillo/fiesta/internal/site" |
| +) |
| + |
| +// releaseRule is a release's path line in _headers and the |
| +// Cache-Control under it. |
| +var releaseRule = regexp.MustCompile(`(?m)^/([0-9]+\.[0-9]+\.[0-9]+)/\*\n Cache-Control: (.*)$`) |
| + |
| +// releaseCaches reads the Cache-Control _headers gives each release, |
| +// by version, from a site fiesta site wrote under out. |
| +func releaseCaches(t *testing.T, out string) map[string]string { |
| + t.Helper() |
| + data, err := os.ReadFile(filepath.Join(out, shipDir, "_headers")) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + caches := map[string]string{} |
| + for _, m := range releaseRule.FindAllStringSubmatch(string(data), -1) { |
| + caches[m[1]] = m[2] |
| + } |
| + return caches |
| +} |
| + |
| +// copyRelease copies the committed release version, every file and |
| +// the stamp, into releases/version. |
| +func copyRelease(t *testing.T, version, releases string) { |
| + t.Helper() |
| + src := filepath.Join("..", "..", "releases", version) |
| + err := filepath.WalkDir(src, func(p string, d fs.DirEntry, err error) error { |
| + if err != nil || d.IsDir() { |
| + return err |
| + } |
| + rel, err := filepath.Rel(src, p) |
| + if err != nil { |
| + return err |
| + } |
| + data, err := os.ReadFile(p) |
| + if err != nil { |
| + return err |
| + } |
| + dest := filepath.Join(releases, version, rel) |
| + if err := os.MkdirAll(filepath.Dir(dest), 0o755); err != nil { |
| + return err |
| + } |
| + return os.WriteFile(dest, data, 0o644) |
| + }) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| +} |
| + |
| +// resum rewrites dir's SHA256SUMS from the files beside it, as fiesta |
| +// release writes it, so a changed release still passes site.Release |
| +// and only its tag can tell it apart. |
| +func resum(t *testing.T, dir string) { |
| + t.Helper() |
| + var names []string |
| + err := filepath.WalkDir(dir, func(p string, d fs.DirEntry, err error) error { |
| + if err != nil || d.IsDir() { |
| + return err |
| + } |
| + rel, err := filepath.Rel(dir, p) |
| + if err != nil { |
| + return err |
| + } |
| + if rel = filepath.ToSlash(rel); rel != "SHA256SUMS" && rel != site.StampName { |
| + names = append(names, rel) |
| + } |
| + return nil |
| + }) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + sort.Strings(names) |
| + var sums bytes.Buffer |
| + for _, name := range names { |
| + data, err := os.ReadFile(filepath.Join(dir, filepath.FromSlash(name))) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + sum := sha256.Sum256(data) |
| + sums.WriteString(hex.EncodeToString(sum[:]) + " " + name + "\n") |
| + } |
| + if err := os.WriteFile(filepath.Join(dir, "SHA256SUMS"), sums.Bytes(), 0o644); err != nil { |
| + t.Fatal(err) |
| + } |
| +} |
| + |
| +// TestSitePromisesAYearOnlyToTaggedReleases: fiesta site keeps every |
| +// release five minutes, as /M/, unless -published names it, and a |
| +// release named there must be its tag's in the -git repository, file |
| +// for file, or nothing is written. A candidate shipped, rebuilt and |
| +// shipped again would otherwise leave browsers holding its first bytes |
| +// for a year at the same address. |
| +// |
| +// The repository holds the committed 1.0.0 and a candidate of the |
| +// current version. Refused, each by name and before anything is |
| +// written: -published without -git, -git without -published, a |
| +// version that is no X.Y.Z, a release with no tag, and a tagged release |
| +// with a file changed, one added, one taken away and one made a link |
| +// after the tag, its sums rewritten to agree; a tag holding no such |
| +// release; and a tagged release missing from the tree. Published once |
| +// its tag holds it, 1.0.0 gets a year and the candidate still five |
| +// minutes. |
| +func TestSitePromisesAYearOnlyToTaggedReleases(t *testing.T) { |
| + repo := scratchRepo(t) |
| + releases := filepath.Join(repo, "releases") |
| + copyRelease(t, "1.0.0", releases) |
| + if err := fiestaRelease(t, releases, fiesta.Version, false); err != nil { |
| + t.Fatal(err) |
| + } |
| + siteDir := filepath.Join(t.TempDir(), "site") |
| + siteFixture(t, siteDir) |
| + git(t, repo, "add", "releases") |
| + git(t, repo, "commit", "-q", "-m", "releases") |
| + out := filepath.Join(t.TempDir(), "site") |
| + siteWith := func(extra ...string) error { |
| + t.Helper() |
| + return run(append([]string{"site", "-releases", releases, "-site", siteDir, "-out", out}, extra...), io.Discard) |
| + } |
| + |
| + if err := siteWith(); err != nil { |
| + t.Fatalf("fiesta site with nothing published: %v", err) |
| + } |
| + brief := map[string]string{"1.0.0": "public, max-age=300", fiesta.Version: "public, max-age=300"} |
| + if got := releaseCaches(t, out); !maps.Equal(got, brief) { |
| + t.Errorf("with nothing published, _headers caches the releases %v; want %v", got, brief) |
| + } |
| + written, err := os.ReadFile(filepath.Join(out, shipDir, "_headers")) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + // unwritten fails if a refused run touched the site on disk. |
| + unwritten := func(what string) { |
| + t.Helper() |
| + if got, err := os.ReadFile(filepath.Join(out, shipDir, "_headers")); err != nil || !bytes.Equal(got, written) { |
| + t.Errorf("fiesta site refused %s but rewrote the site: %v", what, err) |
| + } |
| + } |
| + |
| + for _, c := range []struct { |
| + what string |
| + args []string |
| + want []string |
| + }{ |
| + {"-published without -git", []string{"-published", "1.0.0"}, []string{"-published needs -git"}}, |
| + {"-git without -published", []string{"-git", repo}, []string{"-git is only for -published"}}, |
| + {"a version that is no X.Y.Z", []string{"-published", "1.0", "-git", repo}, []string{`"1.0" is not a version`}}, |
| + {"a release with no tag", []string{"-published", "1.0.0", "-git", repo}, []string{"no tag v1.0.0"}}, |
| + {"a candidate", []string{"-published", fiesta.Version, "-git", repo}, []string{"no tag v" + fiesta.Version}}, |
| + } { |
| + err := siteWith(c.args...) |
| + if err == nil { |
| + t.Errorf("fiesta site accepted %s", c.what) |
| + continue |
| + } |
| + for _, want := range c.want { |
| + if !strings.Contains(err.Error(), want) { |
| + t.Errorf("the refusal of %s does not say %q: %v", c.what, want, err) |
| + } |
| + } |
| + unwritten(c.what) |
| + } |
| + |
| + git(t, repo, "tag", "-a", "-m", "Fiesta 1.0.0", "v1.0.0") |
| + one := filepath.Join(releases, "1.0.0") |
| + licence := filepath.Join(one, "LICENSE") |
| + data, err := os.ReadFile(licence) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := os.WriteFile(licence, append(data, "\nchanged after the tag\n"...), 0o644); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := os.WriteFile(filepath.Join(one, "extra.txt"), []byte("added after the tag\n"), 0o644); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := os.Remove(filepath.Join(one, "busy.js")); err != nil { |
| + t.Fatal(err) |
| + } |
| + // A link to the very bytes the tag holds is still not the tag's |
| + // file: what it points at can change after the check. |
| + notice := filepath.Join(one, "NOTICE") |
| + noticeBytes, err := os.ReadFile(notice) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + elsewhere := filepath.Join(t.TempDir(), "NOTICE") |
| + if err := os.WriteFile(elsewhere, noticeBytes, 0o644); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := os.Remove(notice); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := os.Symlink(elsewhere, notice); err != nil { |
| + t.Fatal(err) |
| + } |
| + resum(t, one) |
| + err = siteWith("-published", "1.0.0", "-git", repo) |
| + if err == nil { |
| + t.Error("fiesta site published a release whose bytes differ from its tag") |
| + } else { |
| + for _, want := range []string{"LICENSE differs from tag v1.0.0", "SHA256SUMS differs from tag v1.0.0", |
| + "extra.txt is not in tag v1.0.0", "busy.js is in tag v1.0.0 and missing", "NOTICE is not a regular file"} { |
| + if !strings.Contains(err.Error(), want) { |
| + t.Errorf("the refusal does not say %q: %v", want, err) |
| + } |
| + } |
| + } |
| + unwritten("a release changed after its tag") |
| + |
| + if err := os.Remove(notice); err != nil { |
| + t.Fatal(err) |
| + } |
| + git(t, repo, "checkout", "-q", "--", "releases") |
| + if err := os.Remove(filepath.Join(one, "extra.txt")); err != nil { |
| + t.Fatal(err) |
| + } |
| + |
| + // A tag that holds no such release, and a tagged release taken out |
| + // of the tree, are refused as well. |
| + git(t, repo, "tag", "-a", "-m", "not a release", "v2.0.0") |
| + aside := filepath.Join(t.TempDir(), "1.0.0") |
| + if err := os.Rename(one, aside); err != nil { |
| + t.Fatal(err) |
| + } |
| + for _, c := range []struct{ version, want string }{ |
| + {"2.0.0", "tag v2.0.0 holds no releases/2.0.0/"}, |
| + {"1.0.0", "v1.0.0 is tagged but " + one + " is missing"}, |
| + } { |
| + err := siteWith("-published", c.version, "-git", repo) |
| + if err == nil || !strings.Contains(err.Error(), c.want) { |
| + t.Errorf("fiesta site -published %s: %v; want a refusal saying %q", c.version, err, c.want) |
| + } |
| + unwritten("-published " + c.version) |
| + } |
| + if err := os.Rename(aside, one); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := siteWith("-published", "1.0.0", "-git", repo); err != nil { |
| + t.Fatalf("fiesta site refused 1.0.0, which its tag holds: %v", err) |
| + } |
| + want := map[string]string{"1.0.0": "public, max-age=31536000, immutable", fiesta.Version: "public, max-age=300"} |
| + if got := releaseCaches(t, out); !maps.Equal(got, want) { |
| + t.Errorf("with 1.0.0 published, _headers caches the releases %v; want %v", got, want) |
| + } |
| +} |
diff --git a/cmd/fiesta/release_test.go b/cmd/fiesta/release_test.go| index 7140402..7c7461c 100644 |
| --- a/cmd/fiesta/release_test.go |
| +++ b/cmd/fiesta/release_test.go |
| @@ -370,41 +370,7 @@ func TestSiteWritesWhatBuildReturns(t *testing.T) { |
| t.Fatal(err) |
| } |
| siteDir := filepath.Join(root, "site") |
| - if err := os.MkdirAll(filepath.Join(siteDir, "static"), 0o755); err != nil { |
| - t.Fatal(err) |
| - } |
| - if err := os.WriteFile(filepath.Join(siteDir, "static", "x.css"), []byte("body{margin:0}\n"), 0o644); err != nil { |
| - t.Fatal(err) |
| - } |
| - // The landing page's catalogs, one per language, each complete and |
| - // valued by its keys: a stand-in for site/landing/strings/ that |
| - // www.Strings accepts, so the fixture is a whole site source. |
| - catalogs := filepath.Join(siteDir, "landing", "strings") |
| - if err := os.MkdirAll(catalogs, 0o755); err != nil { |
| - t.Fatal(err) |
| - } |
| - catalog := map[string]string{} |
| - for _, key := range www.RequiredKeys { |
| - catalog[key] = key |
| - } |
| - stub, err := json.Marshal(catalog) |
| - if err != nil { |
| - t.Fatal(err) |
| - } |
| - for _, lang := range www.Languages { |
| - if err := os.WriteFile(filepath.Join(catalogs, lang+".json"), stub, 0o644); err != nil { |
| - t.Fatal(err) |
| - } |
| - } |
| - // The landing page's template, the repository's own: what the |
| - // catalogs fill in. |
| - tmpl, err := os.ReadFile(filepath.Join("..", "..", "site", "landing", "page.html.tmpl")) |
| - if err != nil { |
| - t.Fatal(err) |
| - } |
| - if err := os.WriteFile(filepath.Join(siteDir, "landing", "page.html.tmpl"), tmpl, 0o644); err != nil { |
| - t.Fatal(err) |
| - } |
| + siteFixture(t, siteDir) |
| out := filepath.Join(root, "out", "site") |
| if err := run([]string{"site", "-releases", releases, "-site", siteDir, "-out", out}, io.Discard); err != nil { |
| t.Fatalf("fiesta site: %v", err) |
| @@ -462,3 +428,44 @@ func TestSiteWritesWhatBuildReturns(t *testing.T) { |
| t.Errorf("-out holds %d entries, want the stamp and %s: %v", len(entries), shipDir, err) |
| } |
| } |
| + |
| +// siteFixture writes a whole site source into siteDir: one static |
| +// file, the landing page's catalogs and the repository's own template. |
| +func siteFixture(t *testing.T, siteDir string) { |
| + t.Helper() |
| + if err := os.MkdirAll(filepath.Join(siteDir, "static"), 0o755); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := os.WriteFile(filepath.Join(siteDir, "static", "x.css"), []byte("body{margin:0}\n"), 0o644); err != nil { |
| + t.Fatal(err) |
| + } |
| + // The landing page's catalogs, one per language, each complete and |
| + // valued by its keys: a stand-in for site/landing/strings/ that |
| + // www.Strings accepts, so the fixture is a whole site source. |
| + catalogs := filepath.Join(siteDir, "landing", "strings") |
| + if err := os.MkdirAll(catalogs, 0o755); err != nil { |
| + t.Fatal(err) |
| + } |
| + catalog := map[string]string{} |
| + for _, key := range www.RequiredKeys { |
| + catalog[key] = key |
| + } |
| + stub, err := json.Marshal(catalog) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + for _, lang := range www.Languages { |
| + if err := os.WriteFile(filepath.Join(catalogs, lang+".json"), stub, 0o644); err != nil { |
| + t.Fatal(err) |
| + } |
| + } |
| + // The landing page's template, the repository's own: what the |
| + // catalogs fill in. |
| + tmpl, err := os.ReadFile(filepath.Join("..", "..", "site", "landing", "page.html.tmpl")) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := os.WriteFile(filepath.Join(siteDir, "landing", "page.html.tmpl"), tmpl, 0o644); err != nil { |
| + t.Fatal(err) |
| + } |
| +} |
diff --git a/hack/test-weights.txt b/hack/test-weights.txt| index 6cd0e5c..068394b 100644 |
| --- a/hack/test-weights.txt |
| +++ b/hack/test-weights.txt |
| @@ -239,6 +239,7 @@ test internal/gallery TestEveryPagesH1IsItsOwnTitle 2 |
| test internal/gallery TestNoTitleOrStateLabelCarriesAnEmDash 2 |
| test internal/gallery TestTitlesUseTheApprovedSeparators 2 |
| test cmd/fiesta TestSiteWritesWhatBuildReturns 12 |
| +test cmd/fiesta TestSitePromisesAYearOnlyToTaggedReleases 13 |
| test cmd/fiesta TestGalleryRefusesADirectoryHoldingARelease 9 |
| test cmd/fiesta TestGalleryRefusesALinkInsideItsDirectory 9 |
| test cmd/fiesta TestGalleryRefusesALinkedDirectoryHoldingARelease 9 |
| @@ -255,5 +256,5 @@ test internal/www TestPagesHasARootAndTheDesignSystem 5 |
| test internal/www TestStaticFilesAreCopied 5 |
| test internal/www TestTheDesignSystemIsMounted 5 |
| test internal/www TestTheRedirectsFileIsWithinCarlosLimits 5 |
| -test internal/www TestTheSitesHeaders 5 |
| +test internal/www TestTheSitesHeaders 6 |
| test internal/www TestAHiddenSiteFileIsRefused 4 |
diff --git a/internal/site/site.go b/internal/site/site.go| index 84dc587..73aa7fb 100644 |
| --- a/internal/site/site.go |
| +++ b/internal/site/site.go |
| @@ -99,8 +99,9 @@ const ( |
| redirectsMaxLine = 2048 |
| ) |
| -// Rules are what the pages ask of the site's rules files, as |
| -// internal/www works them out from the pages it assembled. |
| +// Rules are what the site's rules files say beyond the release |
| +// rewrites: what the pages ask for, as internal/www works it out from |
| +// the pages it assembled, and which releases are published. |
| type Rules struct { |
| // Redirects are _redirects lines, written one a line as given, |
| // after the release rewrites, so a page's rule never decides a /M/ |
| @@ -110,6 +111,16 @@ type Rules struct { |
| // carries a hash of their bytes, so the bytes at that address can |
| // never change. _headers lets browsers keep each for a year. |
| Immutable []string |
| + // Published are releases, by version, whose directories are their |
| + // tags', so their bytes can never change: _headers lets browsers |
| + // keep each for a year. Every other release gets /M/'s five |
| + // minutes, because until it is tagged it may be rebuilt under the |
| + // same name, and a browser told it never changes would keep the |
| + // old bytes. Empty is the safe default for every build but the one |
| + // that ships. Build checks only that each is a release it holds; |
| + // that it matches its tag is the caller's to check |
| + // (TagDifferences), since Build reads no repository. |
| + Published []string |
| } |
| // less orders versions numerically, so 1.10.0 is newer than 1.9.3. |
| @@ -133,9 +144,10 @@ func (v version) less(w version) bool { |
| // /M/* to that major's newest release: `/M/* /M.y.z/:splat 200`; |
| // then rules.Redirects, the pages' rules, one a line as given, so |
| // a page's rule never decides a /M/ URL first; |
| -// - _headers, a five-minute cache on each major's /M/*, a year's |
| -// immutable one on each release's /M.y.z/*, both open to other |
| -// origins, then a year's on each of rules.Immutable (headersFor); |
| +// - _headers, a five-minute cache on each major's /M/* and on each |
| +// release's /M.y.z/* that is not in rules.Published, a year's |
| +// immutable one on each that is, all open to other origins, then |
| +// a year's on each of rules.Immutable (headersFor); |
| // - every page, at the path it has under pages. |
| // |
| // Anything else is an error, and the first error is the whole answer: |
| @@ -147,8 +159,9 @@ func (v version) less(w version) bool { |
| // not, and a page under a version-shaped segment would answer for that |
| // release. Both are refused, and so is a page named _redirects or |
| // _headers, which are the rules files. So is a redirect that is not |
| -// one line of its file, an immutable address that is not a page's, |
| -// and rules that take either file past the edge's limits. |
| +// one line of its file, an immutable address that is not a page's, a |
| +// published version that is not a release here or is named twice, and |
| +// rules that take either file past the edge's limits. |
| func Build(releases fs.FS, pages fs.FS, rules Rules) (map[string][]byte, error) { |
| out := map[string][]byte{} |
| entries, err := fs.ReadDir(releases, ".") |
| @@ -256,7 +269,17 @@ func Build(releases fs.FS, pages fs.FS, rules Rules) (map[string][]byte, error) |
| return nil, err |
| } |
| - headers, err := headersFor(versions, majors, rules.Immutable, out) |
| + published := map[string]bool{} |
| + for _, p := range rules.Published { |
| + switch { |
| + case published[p]: |
| + return nil, fmt.Errorf("the published version %q is named twice", p) |
| + case !slices.ContainsFunc(versions, func(v version) bool { return v.name == p }): |
| + return nil, fmt.Errorf("the published version %q is not a release under releases/", p) |
| + } |
| + published[p] = true |
| + } |
| + headers, err := headersFor(versions, majors, published, rules.Immutable, out) |
| if err != nil { |
| return nil, err |
| } |
| @@ -265,10 +288,11 @@ func Build(releases fs.FS, pages fs.FS, rules Rules) (map[string][]byte, error) |
| } |
| // headersFor is the site's _headers: a short cache on each major's |
| -// /M/, a year on each release's /M.y.z/, both open to other origins, |
| -// and a year on each immutable page, in that order. Every file it |
| -// does not name keeps the edge's no-cache, the pages included, so a |
| -// deploy shows on the next load. |
| +// /M/ and on each release's /M.y.z/ that is not published, a year on |
| +// each that is, all open to other origins, and a year on each |
| +// immutable page, in that order. Every file it does not name keeps the |
| +// edge's no-cache, the pages included, so a deploy shows on the next |
| +// load. |
| // |
| // Each immutable address must be a page of the site, not a release's |
| // file, written so the edge reads it back as that page's path: rooted, |
| @@ -276,7 +300,7 @@ func Build(releases fs.FS, pages fs.FS, rules Rules) (map[string][]byte, error) |
| // percent escape), as a query or fragment, or as the line's end |
| // (white space). Otherwise the rule would name nothing, or something |
| // else, and the page would quietly keep no-cache. |
| -func headersFor(versions []version, majors []int, immutable []string, out map[string][]byte) ([]byte, error) { |
| +func headersFor(versions []version, majors []int, published map[string]bool, immutable []string, out map[string][]byte) ([]byte, error) { |
| ordered := slices.Clone(versions) |
| slices.SortFunc(ordered, func(a, b version) int { |
| switch { |
| @@ -292,7 +316,8 @@ func headersFor(versions []version, majors []int, immutable []string, out map[st |
| // Majors, then releases, then pages: none of their paths overlap |
| // today, but where one ever does, the later and more specific rule |
| - // is the one that wins. |
| + // is the one that wins. The two kinds of release never share a |
| + // path, so their order between themselves decides nothing. |
| majorRules := headerGroup{ |
| kind: "majors", |
| why: []string{ |
| @@ -305,13 +330,22 @@ func headersFor(versions []version, majors []int, immutable []string, out map[st |
| for _, m := range ascending { |
| majorRules.paths = append(majorRules.paths, fmt.Sprintf("/%d/*", m)) |
| } |
| + candidateRules := headerGroup{ |
| + kind: "unpublished releases", |
| + why: []string{"An untagged release may be rebuilt under its name, so browsers keep it five minutes."}, |
| + headers: append([]header{{"Cache-Control", cacheBriefly}}, crossOrigin...), |
| + } |
| releaseRules := headerGroup{ |
| - kind: "releases", |
| + kind: "published releases", |
| why: []string{"A published release never changes, so browsers keep it for a year."}, |
| headers: append([]header{{"Cache-Control", cacheForever}}, crossOrigin...), |
| } |
| for _, v := range ordered { |
| - releaseRules.paths = append(releaseRules.paths, "/"+v.name+"/*") |
| + if published[v.name] { |
| + releaseRules.paths = append(releaseRules.paths, "/"+v.name+"/*") |
| + } else { |
| + candidateRules.paths = append(candidateRules.paths, "/"+v.name+"/*") |
| + } |
| } |
| pageRules := headerGroup{ |
| kind: "immutable pages", |
| @@ -333,7 +367,7 @@ func headersFor(versions []version, majors []int, immutable []string, out map[st |
| pageRules.paths = append(pageRules.paths, p) |
| } |
| slices.Sort(pageRules.paths) |
| - return headersFile([]headerGroup{majorRules, releaseRules, pageRules}) |
| + return headersFile([]headerGroup{majorRules, candidateRules, releaseRules, pageRules}) |
| } |
| // Release reads one release directory and checks it against its |
| @@ -441,8 +475,9 @@ const ( |
| // The Cache-Control values Build writes. immutable stops a browser |
| // revalidating even on a reload, so it goes only where the bytes at |
| -// the address can never change: a release, and a name that carries |
| -// its bytes' hash. |
| +// the address can never change: a published release, and a name that |
| +// carries its bytes' hash. A candidate release gets cacheBriefly, as |
| +// /M/ does. |
| const ( |
| cacheForever = "public, max-age=31536000, immutable" |
| cacheBriefly = "public, max-age=300" |
diff --git a/internal/site/site_test.go b/internal/site/site_test.go| index c149faf..e49a0ad 100644 |
| --- a/internal/site/site_test.go |
| +++ b/internal/site/site_test.go |
| @@ -540,14 +540,16 @@ const ( |
| " Cross-Origin-Resource-Policy: cross-origin\n" |
| ) |
| -// TestTheHeadersCacheAReleaseForAYearAndAMajorForMinutes: _headers |
| -// gives each major a five-minute cache, since /M/ moves to every new |
| -// M.x, each release a year's immutable one, since a published release |
| -// never changes, both with the cross-origin headers, and each |
| -// immutable page a year's cache alone. The whole file is compared, so |
| -// the order is pinned too: majors, then releases, then pages, and |
| -// nothing for the rest of the site, which keeps the edge's no-cache. |
| -func TestTheHeadersCacheAReleaseForAYearAndAMajorForMinutes(t *testing.T) { |
| +// TestTheHeadersCacheAPublishedReleaseForAYearAndTheRestForMinutes: |
| +// _headers gives each major a five-minute cache, since /M/ moves to |
| +// every new M.x, and so each release not published, since a candidate |
| +// may be rebuilt under its name; each published release a year's |
| +// immutable one, since its tag holds it; all with the cross-origin |
| +// headers; and each immutable page a year's cache alone. The whole |
| +// file is compared, so the order is pinned too: majors, unpublished |
| +// releases, published ones, then pages, and nothing for the rest of |
| +// the site, which keeps the edge's no-cache. |
| +func TestTheHeadersCacheAPublishedReleaseForAYearAndTheRestForMinutes(t *testing.T) { |
| pages := pagesOf(map[string]string{ |
| "index.html": "<h1>Fiesta</h1>\n", |
| "a/x.0123456789abcdef.css": "body{}\n", |
| @@ -559,6 +561,7 @@ func TestTheHeadersCacheAReleaseForAYearAndAMajorForMinutes(t *testing.T) { |
| }) |
| out, err := site.Build(releasesOf("1.0.0", "1.10.0", "1.9.3", "2.0.0"), pages, site.Rules{ |
| Immutable: []string{"/a/y.fedcba9876543210.js", "/a/x.0123456789abcdef.css"}, |
| + Published: []string{"2.0.0", "1.0.0", "1.9.3"}, |
| }) |
| if err != nil { |
| t.Fatalf("Build: %v", err) |
| @@ -574,10 +577,12 @@ func TestTheHeadersCacheAReleaseForAYearAndAMajorForMinutes(t *testing.T) { |
| "/1/*\n" + shortLived + cors + |
| "/2/*\n" + shortLived + cors + |
| "\n" + |
| + "# An untagged release may be rebuilt under its name, so browsers keep it five minutes.\n" + |
| + "/1.10.0/*\n" + shortLived + cors + |
| + "\n" + |
| "# A published release never changes, so browsers keep it for a year.\n" + |
| "/1.0.0/*\n" + yearLong + cors + |
| "/1.9.3/*\n" + yearLong + cors + |
| - "/1.10.0/*\n" + yearLong + cors + |
| "/2.0.0/*\n" + yearLong + cors + |
| "\n" + |
| "# Each of these names carries a hash of its bytes, so it never changes.\n" + |
| @@ -588,6 +593,53 @@ func TestTheHeadersCacheAReleaseForAYearAndAMajorForMinutes(t *testing.T) { |
| } |
| } |
| +// TestAnUnpublishedBuildPromisesNoReleaseAYear: a build told of no |
| +// published release, which is every build but the one that ships, |
| +// keeps each release five minutes, as /M/, and writes no immutable |
| +// rule at all when no page asks for one. A candidate shipped, rebuilt |
| +// and shipped again under the same name must not leave browsers |
| +// holding the first bytes for a year. |
| +func TestAnUnpublishedBuildPromisesNoReleaseAYear(t *testing.T) { |
| + out, err := site.Build(releasesOf("1.0.0", "1.1.0", "2.0.0"), pagesOf(nil), site.Rules{}) |
| + if err != nil { |
| + t.Fatalf("Build: %v", err) |
| + } |
| + headers := string(out["_headers"]) |
| + if strings.Contains(headers, "immutable") || strings.Contains(headers, "31536000") { |
| + t.Errorf("_headers promises a release a year with nothing published:\n%s", headers) |
| + } |
| + for _, v := range []string{"1.0.0", "1.1.0", "2.0.0"} { |
| + if !strings.Contains(headers, "/"+v+"/*\n"+shortLived+cors) { |
| + t.Errorf("_headers does not keep /%s/* five minutes, open to other origins:\n%s", v, headers) |
| + } |
| + } |
| +} |
| + |
| +// TestAPublishedVersionMustBeAReleaseHere: a published version names a |
| +// release directory Build holds, once, spelled as that directory is. |
| +// Anything else is refused by name rather than dropped, since a |
| +// version promised and quietly not written is a release shipped |
| +// without the year's cache it was meant to have. |
| +func TestAPublishedVersionMustBeAReleaseHere(t *testing.T) { |
| + for _, published := range [][]string{ |
| + {"1.2.0"}, |
| + {"v1.0.0"}, |
| + {"1.0"}, |
| + {""}, |
| + {"1.0.0", "1.0.0"}, |
| + } { |
| + _, err := site.Build(releasesOf("1.0.0", "1.1.0"), pagesOf(nil), site.Rules{Published: published}) |
| + if err == nil { |
| + t.Errorf("Build accepted the published versions %q", published) |
| + } else if want := strconv.Quote(published[len(published)-1]); !strings.Contains(err.Error(), want) { |
| + t.Errorf("the refusal of %q does not name %s: %v", published, want, err) |
| + } |
| + } |
| + if _, err := site.Build(releasesOf("1.0.0", "1.1.0"), pagesOf(nil), site.Rules{Published: []string{"1.0.0"}}); err != nil { |
| + t.Errorf("Build refused a release it holds as published: %v", err) |
| + } |
| +} |
| + |
| // TestANewReleaseAddsItsOwnHeadersRule: the rules come from the |
| // releases built, so cutting 1.1.0 adds /1.1.0/* and nothing else, |
| // and /1/* stays one rule. |
diff --git a/internal/site/tagged.go b/internal/site/tagged.go| new file mode 100644 |
| index 0000000..2d98f8e |
| --- /dev/null |
| +++ b/internal/site/tagged.go |
| @@ -0,0 +1,167 @@ |
| +package site |
| + |
| +import ( |
| + "bytes" |
| + "context" |
| + "errors" |
| + "fmt" |
| + "io/fs" |
| + "os" |
| + "os/exec" |
| + "path/filepath" |
| + "sort" |
| + "strings" |
| + "time" |
| +) |
| + |
| +// A release is published when its tag vX.Y.Z holds its directory: from |
| +// then on the bytes under /X.Y.Z/ never change, which is the one thing |
| +// that lets _headers tell browsers to keep them for a year. Before the |
| +// tag the directory is a candidate that fiesta release -candidate may |
| +// rebuild under the same name, and a browser told it never changes |
| +// would keep the old bytes. TagDifferences is the check that stands |
| +// between the two: the release tests hold every tagged directory to it |
| +// on every push, and fiesta site -published refuses a version that |
| +// fails it, so the site can only promise what a tag holds. |
| + |
| +// gitTimeout bounds each git call TagDifferences makes, so a git that |
| +// hangs is a refusal rather than a build that never finishes. |
| +const gitTimeout = 30 * time.Second |
| + |
| +// TagDifferences compares the release directory dir with what the tag |
| +// v<version> holds under releases/<version>/ in the git repository |
| +// repo: the same files, byte for byte, the stamp included, with nothing |
| +// missing and nothing extra. It returns one line per difference, each |
| +// naming the file at fault, and none when the two agree. An absent |
| +// tag, a tag holding no such directory, and a missing dir are |
| +// differences too. The error is for git failing to answer, which is |
| +// never taken as agreement. |
| +func TagDifferences(repo, version, dir string) ([]string, error) { |
| + // Checked before it is joined into a path or a ref, so "../x" or |
| + // "1.0.0^" can name nothing but itself. |
| + if !strictVersion.MatchString(version) { |
| + return []string{fmt.Sprintf("%q is not a version X.Y.Z", version)}, nil |
| + } |
| + tag := "v" + version |
| + ref := "refs/tags/" + tag |
| + if _, err := gitIn(repo, "rev-parse", "-q", "--verify", ref+"^{commit}"); err != nil { |
| + var exit *exec.ExitError |
| + if errors.As(err, &exit) && exit.ExitCode() == 1 { |
| + return []string{fmt.Sprintf("there is no tag %s, so %s is not published", tag, version)}, nil |
| + } |
| + return nil, err |
| + } |
| + // --full-tree, so the paths are the repository's own wherever in |
| + // it repo points; -z, so a path is never quoted or split. |
| + prefix := "releases/" + version + "/" |
| + listing, err := gitIn(repo, "ls-tree", "-r", "-z", "--full-tree", ref, "--", prefix) |
| + if err != nil { |
| + return nil, err |
| + } |
| + tagged := map[string]string{} // path under the release to its blob |
| + for _, entry := range strings.Split(strings.TrimSuffix(string(listing), "\x00"), "\x00") { |
| + if entry == "" { |
| + continue |
| + } |
| + meta, name, ok := strings.Cut(entry, "\t") |
| + fields := strings.Fields(meta) |
| + if !ok || len(fields) != 3 { |
| + return nil, fmt.Errorf("git ls-tree %s answered %q", tag, entry) |
| + } |
| + tagged[strings.TrimPrefix(name, prefix)] = fields[2] |
| + } |
| + if len(tagged) == 0 { |
| + return []string{fmt.Sprintf("tag %s holds no %s, so it is not a release tag", tag, prefix)}, nil |
| + } |
| + local, err := regularFiles(dir) |
| + switch { |
| + case errors.Is(err, fs.ErrNotExist): |
| + return []string{fmt.Sprintf("%s is tagged but %s is missing: a published release is never removed", tag, dir)}, nil |
| + case err != nil: |
| + return nil, err |
| + } |
| + var diffs []string |
| + for _, name := range sortedKeys(tagged) { |
| + path, ok := local[name] |
| + if !ok { |
| + diffs = append(diffs, fmt.Sprintf("%s/%s is in tag %s and missing", dir, name, tag)) |
| + continue |
| + } |
| + if path == "" { |
| + diffs = append(diffs, fmt.Sprintf("%s/%s is not a regular file, and a release holds nothing else", dir, name)) |
| + continue |
| + } |
| + want, err := gitIn(repo, "cat-file", "blob", tagged[name]) |
| + if err != nil { |
| + return nil, err |
| + } |
| + got, err := os.ReadFile(path) |
| + if err != nil { |
| + return nil, err |
| + } |
| + if !bytes.Equal(got, want) { |
| + diffs = append(diffs, fmt.Sprintf("%s/%s differs from tag %s: a published release never changes; a fix is the next version", dir, name, tag)) |
| + } |
| + } |
| + for _, name := range sortedKeys(local) { |
| + if _, ok := tagged[name]; !ok { |
| + diffs = append(diffs, fmt.Sprintf("%s/%s is not in tag %s: a published release gains nothing after the tag", dir, name, tag)) |
| + } |
| + } |
| + return diffs, nil |
| +} |
| + |
| +// regularFiles lists every file under dir by its slash path relative |
| +// to dir, mapped to its path on disk, or to "" for anything that is not |
| +// a regular file, so the caller can name it rather than read through |
| +// it. A missing dir is fs.ErrNotExist. |
| +func regularFiles(dir string) (map[string]string, error) { |
| + if _, err := os.Stat(dir); err != nil { |
| + return nil, err |
| + } |
| + files := map[string]string{} |
| + err := filepath.WalkDir(dir, func(p string, d fs.DirEntry, err error) error { |
| + if err != nil || d.IsDir() { |
| + return err |
| + } |
| + rel, err := filepath.Rel(dir, p) |
| + if err != nil { |
| + return err |
| + } |
| + files[filepath.ToSlash(rel)] = "" |
| + if d.Type().IsRegular() { |
| + files[filepath.ToSlash(rel)] = p |
| + } |
| + return nil |
| + }) |
| + return files, err |
| +} |
| + |
| +// sortedKeys is m's keys in order, so differences come out the same |
| +// way every run. |
| +func sortedKeys(m map[string]string) []string { |
| + keys := make([]string, 0, len(m)) |
| + for k := range m { |
| + keys = append(keys, k) |
| + } |
| + sort.Strings(keys) |
| + return keys |
| +} |
| + |
| +// gitIn runs git in repo and returns its standard output. |
| +func gitIn(repo string, args ...string) ([]byte, error) { |
| + ctx, cancel := context.WithTimeout(context.Background(), gitTimeout) |
| + defer cancel() |
| + cmd := exec.CommandContext(ctx, "git", append([]string{"-C", repo}, args...)...) |
| + var stderr strings.Builder |
| + cmd.Stderr = &stderr |
| + out, err := cmd.Output() |
| + if err != nil { |
| + if ctx.Err() != nil { |
| + return nil, fmt.Errorf("git %s: no answer within %s", args[0], gitTimeout) |
| + } |
| + // Wrapped, so a caller can still read the exit code. |
| + return nil, fmt.Errorf("git %s: %w: %s", args[0], err, strings.TrimSpace(stderr.String())) |
| + } |
| + return out, nil |
| +} |
diff --git a/internal/www/headers_test.go b/internal/www/headers_test.go| index a9a6d18..6c49d8e 100644 |
| --- a/internal/www/headers_test.go |
| +++ b/internal/www/headers_test.go |
| @@ -12,6 +12,7 @@ import ( |
| "errors" |
| "fmt" |
| "net/http" |
| + "os" |
| "path" |
| "regexp" |
| "slices" |
| @@ -245,22 +246,55 @@ func versionLess(a, b string) int { |
| } |
| // TestTheSitesHeaders reads the built site's _headers as the edge |
| -// does, and holds it to what the site needs: |
| +// does, twice: as every build but the one that ships makes it, with no |
| +// release published, and as fiesta site -published makes it, here with |
| +// every release but the current one published, so one tree holds both |
| +// kinds. Each is held to what the site needs: |
| // |
| // - exactly its rules: /M/* for each major, five minutes, since /M/ |
| -// moves to each new M.x; /X.Y.Z/* for each release, a year and |
| -// immutable, since a published release never changes; both with |
| -// the cross-origin headers; and each of the design system's hashed |
| -// assets, a year and immutable, alone. In that order, general |
| -// before specific. |
| -// - every file the browser can be served immutable is a release's or |
| -// a hashed asset, and every hashed asset is immutable; every other |
| -// page gets no Cache-Control from the file, so the edge's no-cache |
| -// stays. |
| +// moves to each new M.x; /X.Y.Z/* for each release not published, |
| +// five minutes too, since a candidate may be rebuilt under its |
| +// name; /X.Y.Z/* for each published release, a year and immutable, |
| +// since its tag holds it; all with the cross-origin headers; and |
| +// each of the design system's hashed assets, a year and immutable, |
| +// alone. In that order, general before specific. |
| +// - every file the browser can be served immutable is a published |
| +// release's or a hashed asset, and every one of those is |
| +// immutable; every other page gets no Cache-Control from the file, |
| +// so the edge's no-cache stays. |
| // - every rule's path matches at least one built file. |
| // - the file is one `carlos ship` packs, under the edge's limits. |
| func TestTheSitesHeaders(t *testing.T) { |
| - tree := builtSite(t) |
| + entries, err := os.ReadDir("../../releases") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + var older []string |
| + for _, e := range entries { |
| + if versionDir.MatchString(e.Name()) && e.Name() != fiesta.Version { |
| + older = append(older, e.Name()) |
| + } |
| + } |
| + if len(older) == 0 { |
| + t.Fatalf("releases/ holds no release but %s, so the publish build would have nothing published to check", fiesta.Version) |
| + } |
| + pages := sitePages(t) |
| + for _, build := range []struct { |
| + name string |
| + published []string |
| + }{ |
| + {"nothing published", nil}, |
| + {"all but the current release published", older}, |
| + } { |
| + t.Run(build.name, func(t *testing.T) { |
| + siteHeadersHold(t, buildSite(t, pages, build.published), build.published) |
| + }) |
| + } |
| +} |
| + |
| +// siteHeadersHold is TestTheSitesHeaders' checks on one built tree, |
| +// whose published releases are published. |
| +func siteHeadersHold(t *testing.T, tree map[string][]byte, published []string) { |
| data, ok := tree["_headers"] |
| if !ok { |
| t.Fatal("the site has no _headers") |
| @@ -269,6 +303,14 @@ func TestTheSitesHeaders(t *testing.T) { |
| if err != nil { |
| t.Fatalf("carlos ship would refuse _headers: %v", err) |
| } |
| + // releaseCache is what a release's files must get: a year only |
| + // once it is published. |
| + releaseCache := func(v string) string { |
| + if slices.Contains(published, v) { |
| + return yearLong |
| + } |
| + return fiveMinute |
| + } |
| var versions []string |
| majors := map[string]bool{} |
| @@ -286,8 +328,8 @@ func TestTheSitesHeaders(t *testing.T) { |
| } |
| slices.SortFunc(versions, versionLess) |
| slices.Sort(hashed) |
| - if !slices.Contains(versions, fiesta.Version) || len(hashed) == 0 { |
| - t.Fatalf("the site holds releases %v and %d hashed assets; the checks below need the current release and the gallery's assets", versions, len(hashed)) |
| + if !slices.Contains(versions, fiesta.Version) || !slices.Contains(versions, "1.0.0") || len(hashed) == 0 { |
| + t.Fatalf("the site holds releases %v and %d hashed assets; the checks below need 1.0.0, the current release and the gallery's assets", versions, len(hashed)) |
| } |
| majorList := make([]string, 0, len(majors)) |
| for m := range majors { |
| @@ -304,8 +346,12 @@ func TestTheSitesHeaders(t *testing.T) { |
| for _, m := range majorList { |
| wants = append(wants, want{"/" + m + "/*", withCache(crossOrigin, fiveMinute)}) |
| } |
| - for _, v := range versions { |
| - wants = append(wants, want{"/" + v + "/*", withCache(crossOrigin, yearLong)}) |
| + for _, cache := range []string{fiveMinute, yearLong} { |
| + for _, v := range versions { |
| + if releaseCache(v) == cache { |
| + wants = append(wants, want{"/" + v + "/*", withCache(crossOrigin, cache)}) |
| + } |
| + } |
| } |
| for _, p := range hashed { |
| wants = append(wants, want{p, withCache(nil, yearLong)}) |
| @@ -320,10 +366,10 @@ func TestTheSitesHeaders(t *testing.T) { |
| t.Errorf("_headers line %d: %s %v; want %s %v", rules[i].line, rules[i].path, rules[i].set, wants[i].path, wants[i].set) |
| } |
| } |
| - t.Logf("_headers: %d majors, %d releases, %d hashed assets", len(majorList), len(versions), len(hashed)) |
| + t.Logf("_headers: %d majors, %d releases (%d published), %d hashed assets", len(majorList), len(versions), len(published), len(hashed)) |
| }) |
| - t.Run("immutable is a release or a hashed asset", func(t *testing.T) { |
| + t.Run("immutable is a published release or a hashed asset", func(t *testing.T) { |
| for name, body := range tree { |
| if name == "_redirects" || name == "_headers" { |
| continue |
| @@ -332,7 +378,11 @@ func TestTheSitesHeaders(t *testing.T) { |
| got := lookupHeaders(rules, "", path.Clean("/"+name)) |
| cc := got.Get("Cache-Control") |
| switch { |
| - case versionDir.MatchString(first) || fingerprinted(name, body): |
| + case versionDir.MatchString(first): |
| + if want := releaseCache(first); cc != want { |
| + t.Errorf("/%s gets Cache-Control %q; want %q", name, cc, want) |
| + } |
| + case fingerprinted(name, body): |
| if cc != yearLong { |
| t.Errorf("/%s gets Cache-Control %q; it never changes, so want %q", name, cc, yearLong) |
| } |
| @@ -376,8 +426,8 @@ func TestTheSitesHeaders(t *testing.T) { |
| // Rules match before the rewrite, so /1/ is /1/*'s, not |
| // its release's. |
| {"/" + major + "/fiesta.css", fiveMinute, "*"}, |
| - {"/" + fiesta.Version + "/fiesta.css", yearLong, "*"}, |
| - {"/1.0.0/i18n/es.json", yearLong, "*"}, |
| + {"/" + fiesta.Version + "/fiesta.css", releaseCache(fiesta.Version), "*"}, |
| + {"/1.0.0/i18n/es.json", releaseCache("1.0.0"), "*"}, |
| {hashed[0], yearLong, ""}, |
| {"/design-system/", "no-cache", ""}, |
| {"/", "no-cache", ""}, |
diff --git a/internal/www/serve_test.go b/internal/www/serve_test.go| index 0cab78d..42e6813 100644 |
| --- a/internal/www/serve_test.go |
| +++ b/internal/www/serve_test.go |
| @@ -31,14 +31,32 @@ func siteSource(t *testing.T) *os.Root { |
| // builtSite is the whole site as the edge would receive it: site.Build |
| // over the committed releases, the pages www.Pages assembles from the |
| -// real site source and their rules, as fiesta site builds it. |
| +// real site source and their rules, as fiesta site builds it with no |
| +// release published. |
| func builtSite(t *testing.T) map[string][]byte { |
| + t.Helper() |
| + return buildSite(t, sitePages(t), nil) |
| +} |
| + |
| +// sitePages are the pages www.Pages assembles from the real site |
| +// source, the slow half of builtSite, for a test that builds twice. |
| +func sitePages(t *testing.T) map[string][]byte { |
| t.Helper() |
| pages, err := www.Pages(fiesta.FS(), siteSource(t).FS()) |
| if err != nil { |
| t.Fatalf("Pages: %v", err) |
| } |
| - tree, err := site.Build(os.DirFS("../../releases"), www.FS(pages), www.Rules(pages)) |
| + return pages |
| +} |
| + |
| +// buildSite is site.Build over the committed releases and pages, with |
| +// the releases named in published given a year, as fiesta site |
| +// -published builds the site that ships. |
| +func buildSite(t *testing.T, pages map[string][]byte, published []string) map[string][]byte { |
| + t.Helper() |
| + rules := www.Rules(pages) |
| + rules.Published = published |
| + tree, err := site.Build(os.DirFS("../../releases"), www.FS(pages), rules) |
| if err != nil { |
| t.Fatalf("site.Build: %v", err) |
| } |
diff --git a/releases_test.go b/releases_test.go| index 012c350..a4f5d51 100644 |
| --- a/releases_test.go |
| +++ b/releases_test.go |
| @@ -217,7 +217,7 @@ var releaseTag = regexp.MustCompile(`^v([0-9]+\.[0-9]+\.[0-9]+)$`) |
| // TestPublishedReleasesMatchTheirTags: for every tag vX.Y.Z, the |
| // current version included once it is tagged, releases/X.Y.Z/ exists |
| // in the working tree and holds exactly the files the tag holds under |
| -// that path, byte for byte, the stamp included. A missing directory, |
| +// that path, byte for byte, the stamp included (site.TagDifferences). A missing directory, |
| // a missing file, a changed file and an extra file each fail: a |
| // published release never changes, and this is the check that says so |
| // on every push after the tag. |
| @@ -243,39 +243,14 @@ func TestPublishedReleasesMatchTheirTags(t *testing.T) { |
| continue |
| } |
| version := m[1] |
| - prefix := releasesDir + "/" + version + "/" |
| - listing := gitOut(t, "ls-tree", "-r", "--name-only", tag, "--", prefix) |
| - tagged := map[string]bool{} |
| - for _, line := range strings.Split(strings.TrimSpace(string(listing)), "\n") { |
| - if line != "" { |
| - tagged[strings.TrimPrefix(line, prefix)] = true |
| - } |
| - } |
| - if len(tagged) == 0 { |
| - t.Errorf("tag %s holds no %s, so it is not a release tag", tag, prefix) |
| - continue |
| - } |
| - dir := filepath.Join(releasesDir, version) |
| - got, ok := filesUnder(t, dir) |
| - if !ok { |
| - t.Errorf("%s is tagged but %s is missing from the tree: a published release is never removed", tag, dir) |
| - continue |
| - } |
| - for _, name := range sortedNames(tagged) { |
| - b, ok := got[name] |
| - if !ok { |
| - t.Errorf("%s/%s is in tag %s and missing from the tree", dir, name, tag) |
| - continue |
| - } |
| - want := gitOut(t, "show", tag+":"+prefix+name) |
| - if !bytes.Equal(b, want) { |
| - t.Errorf("%s/%s differs from tag %s: a published release never changes; a fix is the next version", dir, name, tag) |
| - } |
| + // The comparison fiesta site -published makes before it lets |
| + // browsers keep a release for a year, so the two cannot drift. |
| + diffs, err := site.TagDifferences(".", version, filepath.Join(releasesDir, version)) |
| + if err != nil { |
| + t.Fatalf("comparing %s with its tag: %v", version, err) |
| } |
| - for _, name := range sortedNames(got) { |
| - if !tagged[name] { |
| - t.Errorf("%s/%s is not in tag %s: a published release gains nothing after the tag", dir, name, tag) |
| - } |
| + for _, d := range diffs { |
| + t.Error(d) |
| } |
| } |
| } |