scanner-host: a clamd box for a deployment's console

> 🤖 Written by an LLM (Claude), per factor X.

This module makes one small box that runs clamd, the ClamAV scanning daemon, so the console can virus-scan the objects in app stores. The console reads each object from S3 and streams it to this box. The box reads nothing itself. Its role has Session Manager and nothing else, with an explicit Deny on every SSM parameter read, so a compromised clamd cannot fetch a tenant object, the scan tag key or the release key. There is no SSH. You reach the box over SSM Session Manager.

module "scanner" {
  source                   = "git::https://github.com/carlosframework/platform.git//infra/modules/scanner-host?ref=main"
  name                     = "carlos-flagship-scanner"
  ami_id                   = module.host_ie.ami_id
  client_security_group_id = module.host_ie.security_group_id
  region                   = "eu-west-1"
}

Pass the console box's own AMI, so the two boxes are built from the same base. Pass the console box's security group as client_security_group_id. It is the only thing allowed to reach clamd, which authenticates nobody, so that group is the whole fence. The module creates the scanner's security group in the same VPC. It admits TCP 3310 from that one group and never from a CIDR.

If you replace the box, the address changes

A replaced box gets a new private IP, and the console does not follow it. After any replacement, update CARLOS_CLAMD_ADDR on the console and restart it. Until you do, every sweep fails on its tick, and carlos store scan status and carlos system antivirus status say so. The console itself stays up.

Replacement is always explicit. A newer AMI, an edited init script or a new stream_max_bytes does nothing to a running box. To apply any of them:

tofu apply -replace=module.<name>.aws_instance.scanner

Wiring the console

The module has two outputs the console needs. Each becomes a console environment variable.

| Output | Console variable | |---|---| | clamd_addr (<private_ip>:3310) | CARLOS_CLAMD_ADDR | | stream_max_bytes | CARLOS_CLAMD_MAX_BYTES |

The console also needs CARLOS_SCAN_TAG_KEY. It signs every verdict the console writes, and the console refuses to boot with an address and no key. Generate 32 random bytes as hex, once per deployment. Keep the only copy in your secret store, never in git, tofu state or a transcript. If you lose it, every stored verdict becomes unverifiable.

The console's own role also needs store_scanning = true on its region-host, which lets it read and tag store objects. See that module's README.

Wire the console only after the box has passed the readiness check and the four acceptance checks below. Nothing the console can ask tells a tested engine from an untested one. The console must also run a build that records an archive over the engine's limits as not scanned. A console without that change would record such an archive as infected.

Wiring an address does not switch scanning on. Switch it on with carlos system antivirus enable.

How the port is kept shut

Port 3310 is open only while a self-test has passed against the clamd process that is running now. A failed test, a clamd restart or a ClamAV package upgrade closes the port until the test passes again. The console cannot check this for you, because antivirus status and enable only ask clamd for its version, and an engine that signs a truncated scan as clean answers that perfectly well.

To get there, clamd never listens on TCP. It listens on the unix socket /run/clamav/clamd.ctl only. scanner-tcp.socket listens on the box's private IPv4 address on port 3310 and hands each connection to scanner-tcp.service, which carries it to the unix socket. Both units are bound to scanner-selftest.service, which is bound to clamav-daemon.service. When the self-test unit goes inactive, the port closes and connections in flight are cut.

The self-test builds ten archive fixtures, streams each to clamd, and requires the right class of answer. An archive that expands past the limits must answer with a limit hit. An OK there is a failure. It runs every time clamd starts, on whatever ClamAV version is installed at that moment.

Why the two limits are equal and fixed

clamd runs with MaxFileSize 100M, MaxScanSize 100M and AlertExceedsMax yes. These are constants in this release, not variables.

AlertExceedsMax yes makes clamd answer Heuristics.Limits.Exceeded.<limit> FOUND when an archive expands past a limit. The console records that as not scanned. With the default no, clamd scans part of the archive and answers OK.

That is not enough on its own. ClamAV 1.4.3 trims a zip member that expands past MaxFileSize, scans the trimmed copy, and raises no limit. With the two limits equal, the trimmed copy is as large as the whole scan budget, so scanning it always trips MaxScanSize, which does alert. The fixtures and expected answers in the self-test are sized for 100 MiB, so changing the limit means deriving both from it.

Measured on 2026-10-05 against Debian trixie's clamd (ClamAV 1.4.3), with StreamMaxLength at 25M and every fixture streamed with clamdscan --stream. Each member is a repeating 16 KiB random block, distinct per member, so clamd's clean cache cannot short-circuit it. "Limit" means a Heuristics.Limits.Exceeded.* reply. The self-test does not count Heuristics.Limits.Exceeded.MaxScanTime as one. That reply is a timeout, not a limit, and it fails the test.

| Fixture | MaxFileSize 25M, MaxScanSize 100M | Both 100M (this module) | Self-test requires | |---|---|---|---| | small clean file | OK | OK | OK | | EICAR | infected | infected | infected | | zip, one member expanding to 500 MiB | OK (trimmed, silent) | Limit | Limit | | zip, a zip inside holding that member | OK (silent) | Limit | Limit | | zip, 30 distinct 20 MiB members | Limit | Limit | Limit | | gzip of 500 MiB | Limit | Limit | Limit | | tar.gz, 5 x 40 MiB | Limit | Limit | Limit | | bzip2, xz, tar.xz of the same | Limit | Limit | not in self-test | | zip, 500 MiB member then EICAR | infected | Limit | Limit | | zip, 60 MiB member then EICAR | infected | infected | infected | | zip, one 60 MiB member | OK (trimmed, silent) | OK (fully scanned) | OK |

The first column is what an unequal pair does: three fixtures were signed clean over content nobody read.

stream_max_bytes is the one limit you can change. It is clamd's StreamMaxLength, the largest object it accepts on a stream. It defaults to 26214400 (25 MiB). It must be a whole number of MiB, between 8 MiB and 100 MiB. The floor keeps the self-test's fixtures, about 5 MB each, inside the stream. The ceiling keeps a whole stream inside the scan budget. The console's CARLOS_CLAMD_MAX_BYTES must equal it. Use the stream_max_bytes output rather than typing the number twice.

What a clean verdict does not cover

A zip member written with zip64 local headers is not extracted at all by ClamAV 1.4.3 (local header - stream out of file), and EICAR inside one reads OK under every configuration. Encrypted archives are the same. This is parser coverage, not a limit, and no clamd setting changes it. A clean verdict means clamd found nothing in what it could open. It does not mean the object is safe.

Checking that a box is ready

First boot takes several minutes. It installs packages, downloads about 110 MB of signatures and runs the self-test. In an SSM session on the box:

cloud-init status --wait
cat /var/lib/scanner-host/ready
systemctl is-active scanner-selftest.service scanner-tcp.socket
journalctl -u scanner-selftest.service --no-pager | tail -14

You should see status: done, a ClamAV version line, active twice, and then ten pass lines and scanner-selftest: PASS. The init script writes each step to /var/log/cloud-init-output.log, and copies it to the serial console when it can, ending in SCANNER-HOST READY <version> or SCANNER-HOST FAILED: <reason>. To read the console copy without a session:

aws ec2 get-console-output --instance-id <scanner-instance-id> --latest --output text | grep 'SCANNER-HOST'

If the box failed, fix the module and replace the box. Do not patch the box.

Acceptance checks

Run all four on the first real box in a deployment, before the console is wired, and record the results. Each one proves a part of the invariant that first boot cannot. Run the commands as root in an SSM session on the box, except the third.

1. Killing clamd closes the port, and it reopens only after the self-test passes.

systemctl kill -s KILL clamav-daemon.service
sleep 3
echo "after kill: $(ss -ltnH 'sport = :3310' | wc -l) listeners, selftest $(systemctl is-active scanner-selftest.service)"
for i in $(seq 1 60); do [ "$(systemctl is-active scanner-tcp.socket)" = active ] && break; sleep 10; done
echo "recovered: socket $(systemctl is-active scanner-tcp.socket), selftest $(systemctl is-active scanner-selftest.service), $(ss -ltnH 'sport = :3310' | wc -l) listeners"

Expected: after kill: 0 listeners, with the self-test anything but active. Then recovered: socket active, selftest active, 1 listeners. If the port is still open right after the kill, or never reopens, the unit dependencies do not work on this systemd. Fix the module and replace the box.

2. A failed self-test keeps the port shut. The script honours SCANNER_SELFTEST_FORCE_FAIL=1. Set it in a drop-in for this check only.

mkdir -p /etc/systemd/system/scanner-selftest.service.d
printf '[Service]\nEnvironment=SCANNER_SELFTEST_FORCE_FAIL=1\n' > /etc/systemd/system/scanner-selftest.service.d/99-force-fail.conf
systemctl daemon-reload
systemctl restart clamav-daemon.service
sleep 90
systemctl is-active clamav-daemon.service scanner-selftest.service scanner-tcp.socket
ss -ltnH 'sport = :3310' | wc -l

rm /etc/systemd/system/scanner-selftest.service.d/99-force-fail.conf
systemctl daemon-reload
systemctl restart clamav-daemon.service
for i in $(seq 1 60); do [ "$(systemctl is-active scanner-tcp.socket)" = active ] && break; sleep 10; done
systemctl is-active clamav-daemon.service scanner-selftest.service scanner-tcp.socket
ss -ltnH 'sport = :3310' | wc -l

Expected from the first print: active, failed, then inactive or failed, then 0. Expected from the second: active three times, then 1.

3. The scan protocol works through the proxy, from the console's box. Run this in bash in an SSM session on the console box. /dev/tcp is a bash feature. It streams the EICAR test string, a harmless standard test file that every engine flags, to the scanner.

exec 3<>/dev/tcp/<scanner-private-ip>/3310
printf 'zINSTREAM\0\0\0\0\104X5O!P%%@AP[4\\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*\0\0\0\0' >&3
timeout 20 head -c 200 <&3 | tr -d '\0'; echo

\0\0\0\104 is the 68-byte chunk length (octal 104 is 68). The four zero bytes after the payload end the stream. Expected: stream: Eicar-Test-Signature FOUND. Also confirm the fence from a box that is not the console's: the same connect attempt should time out.

4. The box comes back from a reboot with the port open. First boot cannot show this, because basic.target and sockets.target are already reached by the time the init script runs. A systemd ordering cycle would only appear on a later boot.

aws ec2 reboot-instances --instance-ids <scanner-instance-id>

Wait five minutes, then in an SSM session on the box:

systemctl is-active clamav-daemon.service scanner-selftest.service scanner-tcp.socket
ss -ltnH 'sport = :3310' | wc -l
journalctl -b --no-pager | grep -ci "ordering cycle"
systemd-analyze verify /etc/systemd/system/scanner-selftest.service /etc/systemd/system/scanner-tcp.socket /etc/systemd/system/scanner-tcp.service 2>&1 | grep -v '^$' | head

Expected: active three times, 1, 0, and nothing from systemd-analyze verify about these three units.

Sizing

The default is t4g.medium. clamd holds about 1.5 GB of signatures in memory, and about twice that for a short time while it loads a fresh set. A t4g.small or smaller cannot hold two databases at once. tofu plan refuses that pairing unless you set concurrent_database_reload = false, which pauses scanning during each reload.

clamd runs under a cgroup ceiling (memory_max, default 3G, with a throttle at memory_high, default 2560M). Under memory pressure the kernel kills clamd, never the SSM agent, which is the only way onto the box. systemd restarts clamd, and the self-test re-runs before the port reopens. The root volume is 20 GiB, encrypted gp3.

Network

The box needs a route out. It downloads packages, the SSM agent (from the regional bucket for region) and signature updates. With subnet_id null, the module uses the default subnet of the default VPC, and refuses at plan time unless the client security group is in the default VPC. Otherwise pass a subnet_id in the client group's VPC. Egress is open. There is no ingress except TCP 3310 from the client group.

Variables and outputs

Variables are described in variables.tf. Outputs: clamd_addr, stream_max_bytes, instance_id, security_group_id.