> 2026-10-05: steps 2 and 4 are superseded by docs/superpowers/specs/2026-10-04-scanner-host-rollout-design.md. clamd runs on its own box, the switch is carlos system antivirus enable, and CARLOSSTORESCANNER is deprecated. Steps 5 to 8 and the watch list still stand.

Rolling object-storage scanning out to Carloku

Not TDD tasks — an ordered operational sequence. Do not start it until the implementation plan (2026-09-01-object-storage-virus-scanning.md) is merged and on main. Each step states how to know it worked.

Spec: docs/superpowers/specs/2026-09-01-object-storage-virus-scanning-design.md

Tito first throughout, for the reason the console-as-an-app rollout gives: smaller blast radius, and Tito's console going down does not take Carloku with it.

What is actually being rolled out

An opt-in scanner that is inert until three separate things are true: CARLOS_STORE_SCANNER is set on the deployment, a member has requested scanning for their app, and an operator has granted it. Shipping the binary changes nothing on its own, which is what makes this rollout low-risk and why step 1 is safe to do first and sit on.

The failure mode worth naming up front: the ClamAV engine reads tenant object bytes. Nothing reads anything until an operator grants a specific app, but the capability arrives with the binary, and the sequence below puts the disclosure before the first grant rather than after it.

---

1. Ship the binary with the scanner unset

Deploy main to Tito, then Carloku, with CARLOS_STORE_SCANNER unset.

How to know it worked: carlos store scan status on any app answers engine: none and the operator doors 503 naming CARLOS_STORE_SCANNER. No scan pass appears in the console log. carlos store status is unchanged for every existing app, and no scan.json exists anywhere:

aws s3 ls s3://<deployment-bucket>/apps/ --recursive | grep scan.json

should print nothing. If it prints anything, stop — a sweep ran when nothing should have.

2. Stand up clamd on the Tito console box

This is the tito/carlos-infrastructure half. Per spec §10:

• a clamav-daemon systemd unit, socket at /run/clamav/clamd.ctl
• freshclam on a timer — a stale signature database is the silent failure mode of this whole feature; a clean verdict from three-month-old signatures reads exactly like a fresh one
• StreamMaxLength matched to whatever CARLOS_CLAMD_MAX_BYTES will be
• the console unit's user added to the clamav group
• roughly 1–1.5 GB of RAM for resident signatures, held whether or not anything is being scanned — check the box has it rather than assuming
• resource control on the unit, which matters more than the headroom:

``ini [Service] MemoryMax=2G # the kernel kills clamd, never the console MemoryHigh=1600M CPUQuota=50% Nice=10 ``

This runs on the CONSOLE box. If clamd drives that into the OOM killer we do not lose scanning, we lose deploys, routing and config for every app on the deployment. With MemoryMax on clamd's own cgroup the kernel kills clamd and the console carries on; without it the two compete for the same memory and the wrong one can lose.

• the reload spike, which is the number that catches people. Current ClamAV defaults to ConcurrentDatabaseReload yes, holding the old and new databases in memory at once so scanning never pauses during a freshclam update — briefly around doubling it. Either size for ~3 GB of headroom, or set ConcurrentDatabaseReload no and size for steady state. The second trade is nearly free: it blocks scanning during a reload rather than doubling memory, and a pass that fails a tick resumes on the next one

How to know it worked:

systemctl is-active clamav-daemon
sudo -u <console-user> test -w /run/clamav/clamd.ctl && echo reachable
echo -n 'zVERSION' | sudo -u <console-user> socat - UNIX-CONNECT:/run/clamav/clamd.ctl

The last prints ClamAV <engine>/<sig-version>/<sig-date>. Check the date is today or yesterday. If freshclam has never run, it will not be.

3. Grant the console its S3 read and tag permissions

Add s3:GetObject, s3:GetObjectTagging and s3:PutObjectTagging on arn:aws:s3:::<BucketPrefix>* to the Tito console's role. It already lists those buckets for the economics storage pass; this adds read and tag.

How to know it worked: step 5's first sweep completes. There is no useful intermediate check that does not involve reading a tenant object, and doing that by hand before a member has opted in is exactly what this feature is supposed to require consent for.

4. Turn the engine on, still with nothing opted in

Set CARLOS_STORE_SCANNER=clamav, CARLOS_CLAMD_SOCKET=/run/clamav/clamd.ctl and CARLOS_CLAMD_MAX_BYTES on the Tito console. Restart.

The other three have defaults and only need setting deliberately: CARLOS_CLAMD_MAX_SIG_AGE (168h — when scan status starts warning), CARLOS_STORE_SCAN_BUDGET (60s of scanning per app per hourly tick) and CARLOS_STORE_SCAN_INTERVAL (24h floor before a completed sweep re-runs; a sweep still in progress ignores it, which is how a bucket bigger than one budget ever finishes).

Setting CARLOS_STORE_SCANNER without CARLOS_STORE_BACKEND is refused at boot, so a typo here fails loudly rather than leaving a scanner with nothing to scan.

How to know it worked: the console log shows the scan pass starting and then finding nothing to do. carlos store scan status on any app now answers engine: clamav, requested: no. No bucket has been read — the pass skips every app without an enabled block, and there are none.

This is the state to sit in for a day. A scan pass that wakes hourly and correctly does nothing is the cheapest possible proof that the enumeration and the gating work.

5. First real store: one Tito app you own

Pick an app whose objects are plaintext and whose data you own — not a member's. Then:

carlos store scan request --plaintext
carlos store scan grant
carlos store scan sweep
carlos store scan status

How to know it worked: status reports a completed sweep with a last complete timestamp and a signature version matching step 2.

Read the arithmetic carefully, because it is deliberately not the obvious one. scanned and unscanned are not complements of each other against objects — they are two independent facts:

objects   = scanned + skipped + denied + failed
unscanned = skipped + denied + failed          (NOT objects - scanned)

An object too large to stream was never judged, so it counts as both skipped and unscanned. unscanned: 0 is therefore only the right expectation on a store where nothing was skipped, denied or failed — likely on a small plaintext app, but do not treat a non-zero number as a bug. It is the feature: the second sentence of the audit answer.

status also splits coverage into verdicts reached during this sweep and verdicts carried forward from an earlier one. On a first sweep everything is fresh; on a re-sweep of an unchanged bucket everything is carried forward, which is correct and is why the engine database version is printed as what it is now rather than as a claim about every object beneath it.

Then verify the two properties that are easy to assume and worth proving:

• Verdicts are on the objects. aws s3api get-object-tagging on any scanned key shows carlos-scan, carlos-scan-version and carlos-scan-at, and the etag matches the object's current one.
• A rewrite invalidates the verdict. Overwrite one object, then run carlos store scan sweep — that object must be re-scanned rather than reading back its old verdict, which shows up as the fresh count going up. If it is carried forward instead, the version binding is not working and stop the rollout: verdicts that outlive the bytes they judged are worse than no verdicts, because they are trusted.

The tag records the object's ETag and its last-modified time. Both halves matter: a single-part upload's ETag is the content MD5, and chosen-prefix MD5 collisions are practical, so an attacker who controls two uploads could otherwise hand a payload the ETag of the clean object it replaced.

6. Prove the destructive paths before any member sees this

On a throwaway app in Tito:

• carlos store scan grant, then destroy the app. Confirm no scan.json remains under its prefix and the app name is immediately claimable again — an orphan sidecar makes a name permanently unusable and is the failure this feature's teardown path exists to prevent.
• carlos store scan enforce on, then unset CARLOS_STORE_SCANNER and restart. Confirm carlos store scan enforce off still works with no engine built. This is the deadlock draft 3 of the spec would have shipped; prove it is gone on a real deployment, not only in a unit test.

7. Disclose before the first member grant

Before granting scanning on any store CARLOS does not own, the member has to know what they are consenting to:

• the deployment will read their object bytes to scan them
• a clean verdict on a client-side-encrypted store means nothing, and they should request --client-encrypted if that is their shape
• verdicts are eventually consistent — an object is readable before it is scanned, and there is no upload hook

The scan request verb prints this, which is the point of making the attestation an explicit flag. But a member who has not run the verb has not read it, so an operator granting on someone's behalf must have had the conversation first.

8. Carloku, same sequence

Repeat steps 2 through 6 on Carloku. Then stop and wait: on Carloku the first grant should be driven by an actual member request, not by an operator enabling it speculatively. This feature exists to answer a compliance question, and the person with the compliance question is the one who should ask for it.

Watch after each grant

• Signature age. scan status warns past CARLOS_CLAMD_MAX_SIG_AGE. Treat the warning as an incident, not a notice — every clean verdict written while it was stale is worth less than it looks.
• Sweep duration against the budget. If a sweep spans many ticks, the audit answer's T is as old as the sweep is long. Raise CARLOS_STORE_SCAN_BUDGET or shorten the interval, and say the real number when anyone quotes coverage.
• skipped climbing. Over-ceiling objects are not scanned and are not clean. A store that is mostly skips has not been meaningfully scanned and the compliance answer must say so.
• denied appearing at all. It means the console could not read those objects — an SSE-KMS key policy that does not cover it, or an object owned by another account in an adopted bucket. It is re-issued every sweep rather than cached, so a permissions fix repairs it on the next tick with nothing to clean up. A denied count that stays flat after a fix means the fix did not work.
• Console box memory, now carrying clamd's resident signatures — and the reload spike, not just the steady state (step 2).

Rollback

Unset CARLOS_STORE_SCANNER and restart. The pass stops, the doors 503, and every existing verdict tag stays on its object — harmless, and still readable if the engine comes back. Nothing needs undoing in a bucket.

The one thing that does not roll back by unsetting the variable is an active enforce deny, which lives in the app's IAM policy. Clear it first with carlos store scan enforce off (which works with no engine built, by design) before unsetting, or an app keeps a Deny statement referring to a tag nothing will ever write again. Harmless in practice — the tag will not appear, so nothing is denied — but it is a stale statement in a live policy and it should not be left behind.