carlos / platform Public

What happens

A box stages a new app release by reading the whole build into memory. stageFrom (internal/release/adopt.go) calls GetQualified, which returns the artifact as a []byte (S3Store.Get → io.ReadAll), hashes it with etagOf, and hands the bytes to StageArtifact → WriteFileAtomic.

Every root pass that stages runs under MemoryMax=256M (carlos-stage.service, carlos-adopt.service). io.ReadAll holds about twice the file while its buffer grows, and the Go runtime and AWS SDK sit on top. A ~99 MB build peaked at 256 MB and the pass was killed (seen on a deployment's staging box, 2026-10-09).

What it costs

• A console promote is noticed within seconds and relayed to carlos-stage.service. That pass is killed, so the release waits for the next hourly carlos-adopt.timer pass.
• With the stock units, the hourly pass is capped at 256 MB too, so a large enough build never goes live at all.
• The deployment that found this survives only because someone raised carlos-adopt.service to 512 MB by hand, and its CI promotes direct-bucket, so it never takes the stage path. Moving its CI to the console path (#229's deploy role) would make every deploy fail fast and then wait up to an hour.

Fix

Stream the artifact into the version store: open it with OpenObject, copy it into the temp file beside the destination while hashing, check the sha256, then chmod and rename. On a mismatch or a read error, remove the temp file and leave the destination untouched, as today. Keep the migration hint on a missing key (an OpenQualified beside GetQualified). Memory then stays flat whatever the build size, and 256 MB stays a sensible cap.

Out of scope here, worth their own look: edge.ensureExtracted (internal/edge/static.go) and vet also read whole artifacts with GetQualified.

Stopgap for deployments until the fix ships

# /etc/systemd/system/carlos-stage.service.d/memory.conf
[Service]
MemoryMax=512M

then systemctl daemon-reload. Remove it (and any adopt-pass equivalent) once boxes run a release with the fix.