rastrillo / native Public

Merge branch 'fix/release-assets' into main

Merge branch 'fix/release-assets' into main

claude 1 week ago 73aec852f9c4b0281c288d4d7893a55bd1c418c2 parent 529f356parent 7c54dd5

3 files changed, +133 −43

docs/mac-updates.md +48 −19
tools/publish_mac_update.py +24 −16
tools/tests/test_mac_updates.py +61 −8

CI failed — run details

CI log

=== test ===
swift test
Fetching binary artifact https://github.com/sparkle-project/Sparkle/releases/download/2.9.6/Sparkle-for-Swift-Package-Manager.zip from cache
error: failed validating archive from 'https://github.com/sparkle-project/Sparkle/releases/download/2.9.6/Sparkle-for-Swift-Package-Manager.zip' which is required by binary target 'Sparkle': could not find executable for 'unzip'
[0/1] Planning build
make: *** [Makefile:4: test] Error 1

amadan: step test failed: exit status 2

amadan: job failed at step test — its output is above, not at the tail of this log

landed from fix/release-assets

diff --git a/docs/mac-updates.md b/docs/mac-updates.md

index 7cf5269..393b3ca 100644

--- a/docs/mac-updates.md

+++ b/docs/mac-updates.md

@@ -89,27 +89,55 @@ Do not change the visibility of any application source repository. Initialize it

https://raw.amadan.net/oficina/native-releases/main/<app>/appcast.xml

```

-Archives use full immutable Git commit SHA URLs. Amadan raw hosting caps each

-file at 100 MiB; its Git receive request limit is 128 MiB. The publisher rejects

-oversize archives and pushes one archive at a time. Public raw responses need no

-application credentials. Pages hosting and expiring private download URLs are

-not used.

+Git holds the feeds and nothing else. Each archive is an Amadan release asset,

+stored in object storage, on a release tagged `<app>-<build>`:

+```

+https://amadan.net/oficina/native-releases/-/releases/<app>-<build>/<archive>.zip

+```

+That URL redirects to a short-lived signed download, which Sparkle follows.

+Never commit an archive: every zip in Git stays in history for good and is

+copied by every clone. On 2026-10-26 a repack of about 1.1 GB of them took

+amadan.net down for ten minutes (`oficina/native-releases#1`). Assets can be up

+to 500 MB each, and `release_mac.py` still caps archives at 100 MiB.

Run `tools/publish_mac_update.py RELEASE --config CONFIG --repo ARTIFACT_CLONE

--sparkle-tools TOOLS` to validate and print a plan. Add `--publish` only when the

concrete release is ready to publish. It verifies the archive signature, actual

archived app metadata, manifest, Keychain public key, monotonic build number,

-clean main checkout and correct origin. It pushes the archive first, downloads

-it anonymously by immutable SHA URL and checks length and SHA-256. Only then does

-it generate and push the appcast using Sparkle's official tool. The generated

-version must match the signed app. Existing feed items are preserved; deltas are

-disabled to keep upload sizes bounded.

-If publication stops after pushing an archive, the old feed stays usable. This

-first implementation deliberately refuses to overwrite that archive on rerun.

-Inspect the committed archive and failure, then either publish a new higher build

-or perform a reviewed manual feed recovery referencing that exact verified SHA.

-Do not force-push or replace a published archive.

+clean main checkout and correct origin. It pushes the release tag and uploads the

+archive with `amadan release create`. Then it downloads the archive anonymously

+and checks its length and SHA-256. Only after that does it generate and push the

+appcast, using Sparkle's official tool. The generated version must match the

+signed app. Existing feed items are preserved; deltas are disabled to keep

+upload sizes bounded.

+`amadan release create` needs writer access to the artifact repository. Set

+`AMADAN_HOME` to the publishing account's credentials (an agent publishes as its

+agent account), and use a Git credential for that same account to push the tag

+and feed.

+If publication stops before the feed commit, the old feed stays usable. Rerun the

+same command. The tag is reused, and an asset cannot be replaced, so the second

+upload is refused. The publisher accepts that refusal only when the public

+download already matches the signed archive, and then publishes the feed. A

+different archive for a build already uploaded needs a new build number. If it

+stopped while writing the feed, before its commit, discard the half-written feed

+and rerun. The publisher only starts on a clean checkout, so this removes nothing

+else: `git reset --hard HEAD`, then `git clean -f -- <app>/appcast.xml` for an

+app's first feed. If only the final feed push failed, the feed commit is already

+verified. Push it with `git push origin HEAD:refs/heads/main`, and do not rerun,

+because a rerun refuses a build the local feed already lists.

+The hub does not yet make an asset's bytes atomic against two simultaneous

+uploads of the same name. Only one publisher may run for a build at a time.

+Sparkle still checks each download's EdDSA signature, so replaced bytes fail

+to install rather than installing.

+Older feed items point at zips committed before this change, by full commit SHA

+on `raw.amadan.net`. They keep working. Removing those zips from history is a

+separate decision, for the repository's owner.

Users of pre-Sparkle builds need one initial installation of a Sparkle-enabled

release. Subsequent releases can update in place. The first publication should

@@ -119,7 +147,8 @@ include a real update/relaunch check before broad distribution.

`make ci` includes Swift tests, Mac/iOS companion builds, Go race tests, relocated

resource packaging and release-boundary tests. Publication tests check that

-metadata tampering, archive tampering, failed public download and generated

-version mismatch cannot publish a feed; successful publication must verify the

-archive between the two pushes. Independent mutation probes confirmed these

+metadata tampering, archive tampering, failed public download, a refused upload

+without a matching asset and generated version mismatch cannot publish a feed.

+They also check that no archive is ever added to Git, and that the archive is

+uploaded and verified between the tag push and the feed push. Independent mutation probes confirmed these

checks detect removal of the corresponding guards.

diff --git a/tools/publish_mac_update.py b/tools/publish_mac_update.py

index 059ef6f..469c584 100644

--- a/tools/publish_mac_update.py

+++ b/tools/publish_mac_update.py

@@ -1,8 +1,8 @@

"""Publish a prepared Sparkle release to an existing public Amadan artifact repo.

Without --publish, validate and print the publication plan without changing the repo.

-Archives are committed and pushed individually; the feed is pushed only after an

-anonymous download verifies the immutable archive's complete length and digest.

+The archive is uploaded as an Amadan release asset, never committed to Git; the feed

+is pushed only after an anonymous download verifies the asset's length and digest.

"""

import argparse

import hashlib

@@ -13,6 +13,7 @@ import plistlib

import zipfile

import shutil

import subprocess

+import sys

import tempfile

import time

from urllib.parse import quote, urlsplit

@@ -102,28 +103,34 @@ def publish(release, config_path, repo, tools, do_publish=False):

raise ValueError('origin must point to the configured Amadan artifact repository')

if git(repo, 'status', '--porcelain') or git(repo, 'branch', '--show-current') != 'main':

raise ValueError('artifact repository must be clean and on main')

- relative_archive = Path(app) / 'releases' / archive_name

relative_feed = Path(app) / 'appcast.xml'

check_build(repo / relative_feed, manifest['build'])

- if (repo / relative_archive).exists():

- raise ValueError('release archive path already exists; use a new build number')

subprocess.run([str(tools / 'sign_update'), '--account', config['keychain_account'], '--verify', str(archive), manifest['signature']], check=True)

- plan = dict(repository=expected_remote, archive=str(relative_archive), feed=config['feed_url'], build=manifest['build'])

+ # Zips committed to Git stay in history for good and are copied by every clone.

+ # On 2026-10-26 a repack of 1.1 GB of them took amadan.net down (native-releases#1),

+ # so archives are release assets in object storage and Git holds only the feeds.

+ tag = f"{app}-{manifest['build']}"

+ prefix = f'{expected_remote}/-/releases/{tag}/'

+ plan = dict(repository=expected_remote, release=tag, download=prefix + quote(archive_name), feed=config['feed_url'], build=manifest['build'])

if not do_publish:

return plan

git(repo, 'fetch', 'origin', 'main')

if git(repo, 'rev-parse', 'HEAD') != git(repo, 'rev-parse', 'origin/main'):

raise ValueError('main must exactly match the fetched origin/main before publication')

- target = repo / relative_archive

- target.parent.mkdir(parents=True, exist_ok=True)

- shutil.copy2(archive, target)

- git(repo, 'add', '--', str(relative_archive))

- git(repo, 'commit', '-m', f"Release {app} build {manifest['build']} archive")

- archive_commit = git(repo, 'rev-parse', 'HEAD')

- prefix = f'https://raw.amadan.net/{namespace}/{repository}/{archive_commit}/{app}/releases/'

- # One archive per push stays below Amadan's 128 MiB git receive limit.

- git(repo, 'push', 'origin', 'HEAD:refs/heads/main')

+ # A release needs a pushed tag. A rerun keeps the tag an earlier run made.

+ if not git(repo, 'tag', '--list', tag):

+ git(repo, 'tag', tag, 'HEAD')

+ git(repo, 'push', 'origin', f'refs/tags/{tag}')

+ with tempfile.TemporaryDirectory(prefix='oficina-release-') as directory:

+ notes = Path(directory) / 'notes.md'

+ notes.write_text(f"{info.get('CFBundleName', app)} {manifest['version']} (build {manifest['build']})\n")

+ upload = subprocess.run(['amadan', 'release', 'create', '-hub', f'https://{urlsplit(expected_remote).netloc}',

+ '-repo', f'{namespace}/{repository}', '-notes-file', str(notes), tag, str(archive)])

+ # An asset is immutable once uploaded, so a rerun's upload is refused. That is only

+ # acceptable when the public download already has exactly the signed bytes.

verify_download(prefix + quote(archive_name), length, digest)

+ if upload.returncode != 0:

+ print(f'release upload exited {upload.returncode}, but the published asset matches the signed archive', file=sys.stderr)

with tempfile.TemporaryDirectory(prefix='oficina-appcast-') as directory:

staging = Path(directory)

shutil.copy2(archive, staging / archive_name)

@@ -142,11 +149,12 @@ def publish(release, config_path, repo, tools, do_publish=False):

enclosures = [node for node in tree.iter('enclosure') if node.get('url') == prefix + quote(archive_name)]

if len(enclosures) != 1 or enclosures[0].get(SPARKLE + 'edSignature') != manifest['signature'] or enclosures[0].get('length') != str(length):

raise ValueError('generated appcast does not reference the verified signed archive')

+ (repo / relative_feed).parent.mkdir(parents=True, exist_ok=True)

shutil.copy2(output, repo / relative_feed)

git(repo, 'add', '--', str(relative_feed))

git(repo, 'commit', '-m', f"Publish {app} build {manifest['build']} appcast")

git(repo, 'push', 'origin', 'HEAD:refs/heads/main')

- return dict(plan, archive_commit=archive_commit, feed_commit=git(repo, 'rev-parse', 'HEAD'))

+ return dict(plan, feed_commit=git(repo, 'rev-parse', 'HEAD'))

def main():

diff --git a/tools/tests/test_mac_updates.py b/tools/tests/test_mac_updates.py

index b7b03c0..f82bb8e 100644

--- a/tools/tests/test_mac_updates.py

+++ b/tools/tests/test_mac_updates.py

@@ -4,6 +4,7 @@ import hashlib

import json

from pathlib import Path

import plistlib

+import subprocess

import sys

import tempfile

import unittest

@@ -130,6 +131,9 @@ class PublicationBoundaryTests(unittest.TestCase):

self.events = []

self.generated_build = '3'

self.download_error = None

+ self.upload_returncode = 0

+ self.existing_tag = ''

+ self.downloads = []

self.addCleanup(patch.stopall)

patch('publish_mac_update.git', side_effect=self.git).start()

patch('publish_mac_update.subprocess.check_output', return_value=self.values['public_ed_key']).start()

@@ -152,11 +156,15 @@ class PublicationBoundaryTests(unittest.TestCase):

return 'main'

if args[0] == 'rev-parse':

return 'a' * 40

+ if args[:2] == ('tag', '--list'):

+ return self.existing_tag

return ''

def tool(self, command, **kwargs):

tool = Path(command[0]).name

- self.events.append((tool,))

+ self.events.append((tool, *command[1:3]) if tool == 'amadan' else (tool,))

+ if tool == 'amadan':

+ return subprocess.CompletedProcess(command, self.upload_returncode)

if tool != 'generate_appcast':

return

prefix = command[command.index('--download-url-prefix') + 1]

@@ -164,6 +172,7 @@ class PublicationBoundaryTests(unittest.TestCase):

output.write_text(f'<rss xmlns:sparkle="http://www.andymatuschak.org/xml-namespaces/sparkle"><channel><item><sparkle:version>{self.generated_build}</sparkle:version><enclosure url="{prefix}{self.archive.name}" sparkle:edSignature="test-signature" length="{self.manifest["length"]}"/></item></channel></rss>')

def download(self, *args):

+ self.downloads.append(args)

self.events.append(('verify_download',))

if self.download_error:

raise self.download_error

@@ -174,6 +183,9 @@ class PublicationBoundaryTests(unittest.TestCase):

def pushes(self):

return [event for event in self.events if event[:2] == ('git', 'push')]

+ def feed_pushes(self):

+ return [event for event in self.pushes() if event[-1] == 'HEAD:refs/heads/main']

def test_manifest_cannot_claim_a_newer_build_than_the_signed_archive(self):

self.manifest['build'] = '4'

self.write_manifest()

@@ -208,13 +220,15 @@ class PublicationBoundaryTests(unittest.TestCase):

self.assertEqual(plan['build'], '3')

self.assertFalse((self.repo / 'meet/releases').exists())

self.assertEqual(self.feed.read_text(), self.original_feed)

- self.assertFalse(any(event[0] == 'git' and event[1] in ('fetch', 'add', 'commit', 'push') for event in self.events))

+ self.assertFalse(any(event[0] == 'git' and event[1] in ('fetch', 'tag', 'add', 'commit', 'push') for event in self.events))

+ self.assertNotIn(('amadan', 'release', 'create'), self.events)

+ self.assertEqual(plan['download'], 'https://amadan.net/oficina/native-releases/-/releases/meet-3/meet-3.zip')

def test_failed_public_download_leaves_existing_feed_untouched(self):

self.download_error = ValueError('download checksum mismatch')

with self.assertRaisesRegex(ValueError, 'checksum mismatch'):

self.publish()

- self.assertEqual(len(self.pushes()), 1)

+ self.assertEqual(self.feed_pushes(), [])

self.assertNotIn(('generate_appcast',), self.events)

self.assertEqual(self.feed.read_text(), self.original_feed)

@@ -222,18 +236,57 @@ class PublicationBoundaryTests(unittest.TestCase):

self.generated_build = '4'

with self.assertRaisesRegex(ValueError, 'build does not match'):

self.publish()

- self.assertEqual(len(self.pushes()), 1)

+ self.assertEqual(self.feed_pushes(), [])

self.assertEqual(self.feed.read_text(), self.original_feed)

def test_archive_is_publicly_verified_before_feed_publication(self):

self.publish()

- pushes = [index for index, event in enumerate(self.events) if event[:2] == ('git', 'push')]

- self.assertEqual(len(pushes), 2)

+ tag = self.events.index(('git', 'push', 'origin', 'refs/tags/meet-3'))

+ uploaded = self.events.index(('amadan', 'release', 'create'))

verified = self.events.index(('verify_download',))

generated = self.events.index(('generate_appcast',))

- self.assertLess(pushes[0], verified)

+ feed = self.events.index(('git', 'push', 'origin', 'HEAD:refs/heads/main'))

+ self.assertLess(tag, uploaded)

+ self.assertLess(uploaded, verified)

self.assertLess(verified, generated)

- self.assertLess(generated, pushes[1])

+ self.assertLess(generated, feed)

+ def test_archive_is_never_committed_to_git(self):

+ # native-releases#1: zips in Git history took amadan.net down.

+ self.publish()

+ added = [event for event in self.events if event[:2] == ('git', 'add')]

+ self.assertEqual(added, [('git', 'add', '--', 'meet/appcast.xml')])

+ self.assertFalse((self.repo / 'meet/releases').exists())

+ def test_verification_checks_the_release_asset_against_the_signed_bytes(self):

+ self.publish()

+ self.assertEqual(self.downloads, [('https://amadan.net/oficina/native-releases/-/releases/meet-3/meet-3.zip', self.manifest['length'], self.manifest['sha256'])])

+ def test_first_release_of_an_app_creates_its_feed(self):

+ self.feed.unlink()

+ self.feed.parent.rmdir()

+ self.publish()

+ self.assertIn('<sparkle:version>3</sparkle:version>', self.feed.read_text())

+ def test_rerun_reuses_the_tag_an_earlier_run_pushed(self):

+ self.existing_tag = 'meet-3'

+ self.publish()

+ self.assertFalse(any(event[:2] == ('git', 'tag') and event[2] != '--list' for event in self.events))

+ self.assertIn(('git', 'push', 'origin', 'refs/tags/meet-3'), self.events)

+ def test_rerun_accepts_refused_upload_only_when_asset_matches(self):

+ # An asset cannot be replaced, so a rerun's upload fails; the public bytes decide.

+ self.upload_returncode = 1

+ self.assertEqual(self.publish()['build'], '3')

+ self.assertEqual(len(self.feed_pushes()), 1)

+ def test_failed_upload_without_a_matching_asset_publishes_nothing(self):

+ self.upload_returncode = 1

+ self.download_error = OSError('HTTP Error 404: Not Found')

+ with self.assertRaises(OSError):

+ self.publish()

+ self.assertEqual(self.feed_pushes(), [])

+ self.assertEqual(self.feed.read_text(), self.original_feed)

if __name__ == '__main__':