worktree-scanner-host
Squashed from worktree-scanner-host into main.
30 files changed,
+3558
−11
docs/site/cli.md+9 −9docs/superpowers/plans/2026-09-01-carloku-scanning-rollout.md+2 −0docs/superpowers/plans/2026-10-05-scanner-host-rollout.md+1507 −0docs/superpowers/specs/2026-10-04-scanner-host-rollout-design.md+489 −0docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST-CODE.md+17 −0docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST-R2.md+35 −0docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST-R3.md+19 −0docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST-R4.md+26 −0docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST-R5.md+35 −0docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST.md+23 −0docs/superpowers/specs/CODEX-REVIEW-BRIEF-SCANNER-HOST-CODE.md+59 −0docs/superpowers/specs/CODEX-REVIEW-BRIEF-SCANNER-HOST.md+58 −0infra/modules/README.md+21 −0infra/modules/region-host/README.md+4 −0infra/modules/region-host/main.tf+35 −0infra/modules/region-host/tests/store_scanning.tftest.hcl+63 −0infra/modules/region-host/variables.tf+6 −0infra/modules/scanner-host/README.md+181 −0infra/modules/scanner-host/files/scanner-selftest+146 −0infra/modules/scanner-host/files/scanner-selftest.service+23 −0infra/modules/scanner-host/files/scanner-tcp.service+15 −0infra/modules/scanner-host/files/scanner-tcp.socket+26 −0infra/modules/scanner-host/main.tf+157 −0infra/modules/scanner-host/outputs.tf+19 −0infra/modules/scanner-host/scanner-init.sh.tftpl+174 −0infra/modules/scanner-host/tests/scanner_host.tftest.hcl+247 −0infra/modules/scanner-host/variables.tf+63 −0infra/modules/scanner-host/versions.tf+9 −0internal/clamd/clamd.go+27 −2internal/clamd/clamd_test.go+63 −0
CI passed — run details
diff --git a/docs/site/cli.md b/docs/site/cli.md| index e458de9ab..813b0ae14 100644 |
| --- a/docs/site/cli.md |
| +++ b/docs/site/cli.md |
| @@ -1070,7 +1070,7 @@ otherwise leave the denial in place with no way to clear it. |
| numbers — how many objects were scanned, and how many were not. They are not |
| complements. An object too large to stream past the engine was never judged at |
| all, and a bucket full of them could report zero infections while nothing had |
| -actually been looked at. A sweep still in progress says so and says which |
| +actually been looked at. An archive that expands past the engine's limits is counted as not scanned, the same as an object too large to stream. A sweep still in progress says so and says which |
| numbers are stale; a first sweep that has not finished claims no coverage at |
| all. Signature-database age is printed, and called out when it is old, because |
| a clean verdict from three-month-old definitions is a weaker claim than a |
| @@ -1084,16 +1084,18 @@ how many it carried forward, so the two are never confused. |
| objects are recorded and still readable. `carlos store status` says which of |
| the two you have. |
| -The operator side is six environment variables on the console: |
| +The operator side is six environment variables on the console. Setting `CARLOS_CLAMD_ADDR` wires an engine. It does not switch scanning on. The switch is `carlos system antivirus enable`. |
| | variable | default | meaning | |
| | --- | --- | --- | |
| -| `CARLOS_STORE_SCANNER` | unset | `clamav` is the only engine this build accepts. Unset means no scanning at all. | |
| -| `CARLOS_CLAMD_SOCKET` | `/run/clamav/clamd.ctl` | the unix socket to reach `clamd` on | |
| -| `CARLOS_CLAMD_MAX_BYTES` | 25 MB | objects larger than this are recorded as skipped without being fetched; mirror `clamd`'s own `StreamMaxLength` | |
| +| `CARLOS_CLAMD_ADDR` | unset | where to reach `clamd`: `host:port`, or a socket path beginning `/`. Unset means no scanning at all. A [scanner host](https://github.com/carlosframework/platform/tree/main/infra/modules/scanner-host) module output gives you this value. | |
| +| `CARLOS_SCAN_TAG_KEY` | unset | required once an address is set, and the console refuses to boot without it. It signs every verdict, so keep it: without it, stored verdicts cannot be verified. | |
| +| `CARLOS_CLAMD_MAX_BYTES` | 25 MiB | objects larger than this are recorded as skipped without being fetched. Set it equal to `clamd`'s own `StreamMaxLength`. | |
| | `CARLOS_CLAMD_MAX_SIG_AGE` | 168h | how old the signature database may get before `scan status` warns | |
| | `CARLOS_STORE_SCAN_BUDGET` | 60s | how much scanning one app gets per hourly tick | |
| -| `CARLOS_STORE_SCAN_INTERVAL` | 24h | the floor before a *completed* sweep is run again. A sweep still in progress ignores it — that is how a bucket bigger than one budget ever finishes. | |
| +| `CARLOS_STORE_SCAN_INTERVAL` | 24h | the floor before a *completed* sweep is run again. A sweep still in progress ignores it. That is how a bucket bigger than one budget ever finishes. | |
| + |
| +`CARLOS_CLAMD_SOCKET` is still accepted as another name for `CARLOS_CLAMD_ADDR`. `CARLOS_STORE_SCANNER=clamav` is deprecated: it still wires the default socket `/run/clamav/clamd.ctl` and seeds the switch once, and the console logs a warning on every boot while it is set. `CARLOS_SCAN_TAG_KEY_PREVIOUS` is optional and holds the old key while you rotate. |
| The budget arithmetic is worth stating rather than discovering. An hourly tick |
| at 60s is 24 minutes of scanning per app per day; at a few hundred objects a |
| @@ -1102,9 +1104,7 @@ The default is honest for small and medium stores. A deployment with large ones |
| raises the budget or shortens the tick, and `scan status` says when a sweep has |
| spanned more than one tick rather than implying its numbers are current. |
| -Setting `CARLOS_STORE_SCANNER` without `CARLOS_STORE_BACKEND` is refused at |
| -boot — a scanner with no object stores to scan is a misconfiguration, and one |
| -loud failure beats a pass that silently finds nothing forever. |
| +Setting `CARLOS_CLAMD_ADDR` without `CARLOS_STORE_BACKEND` is refused at boot. A scanner with no object stores to scan is a misconfiguration, and one loud failure beats a pass that silently finds nothing forever. |
| Wiring an engine is not the same as switching scanning on. The deployment-wide |
| switch is `carlos system antivirus` — see [carlos system](#carlos-system). |
diff --git a/docs/superpowers/plans/2026-09-01-carloku-scanning-rollout.md b/docs/superpowers/plans/2026-09-01-carloku-scanning-rollout.md| index 8a2cdd57c..0f164be27 100644 |
| --- a/docs/superpowers/plans/2026-09-01-carloku-scanning-rollout.md |
| +++ b/docs/superpowers/plans/2026-09-01-carloku-scanning-rollout.md |
| @@ -1,3 +1,5 @@ |
| +> **2026-10-05:** steps 2 and 4 are superseded by docs/superpowers/specs/2026-10-04-scanner-host-rollout-design.md. clamd runs on its own box, the switch is carlos system antivirus enable, and CARLOS_STORE_SCANNER is deprecated. Steps 5 to 8 and the watch list still stand. |
| + |
| # Rolling object-storage scanning out to Carloku |
| **Not TDD tasks — an ordered operational sequence.** Do not start it until the |
diff --git a/docs/superpowers/plans/2026-10-05-scanner-host-rollout.md b/docs/superpowers/plans/2026-10-05-scanner-host-rollout.md| new file mode 100644 |
| index 000000000..37b079565 |
| --- /dev/null |
| +++ b/docs/superpowers/plans/2026-10-05-scanner-host-rollout.md |
| @@ -0,0 +1,1507 @@ |
| +# Scanner Host Rollout Implementation Plan |
| + |
| +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. |
| + |
| +**Goal:** Object-storage virus scanning switched on, with no member granted, on carloku.com and then on the Tito platform, each with clamd on its own box in the home region. |
| + |
| +**Architecture:** A new `scanner-host` OpenTofu module builds a clamd box whose TCP port is open only while a self-test has passed against the running clamd. `region-host` gains an opt-in read-and-tag grant for the console. One small Go change records clamd's limit replies as unscanned. Then a release, and an operational sequence per deployment: apply, acceptance checks on the box, console roll, key, wiring, enable, proof. |
| + |
| +**Tech Stack:** Go (`internal/clamd`), OpenTofu 1.12 with `tofu test` and `mock_provider`, Debian trixie, ClamAV 1.4, systemd (`systemd-socket-proxyd`), AWS (EC2, IAM, SSM), the `carlos` and `amadan` CLIs. |
| + |
| +**Spec:** `docs/superpowers/specs/2026-10-04-scanner-host-rollout-design.md` (draft 6). Read it first; section numbers below (§) refer to it. |
| + |
| +## Global Constraints |
| + |
| +- **Rulings D1–D8 in spec §2 are fixed.** Do not reopen them. |
| +- `MaxFileSize` and `MaxScanSize` are both `100M`, a constant, not a variable (§3.2). |
| +- The invariant (§3.2): TCP 3310 is open only while a self-test has passed against the clamd process running now. clamd itself never listens on TCP. |
| +- The scanner box's role: Session Manager only, explicit Deny on the four parameter-read actions, no S3, no KMS (§3.1). |
| +- `CARLOS_STORE_SCANNER` stays unset everywhere. It is deprecated. |
| +- The scan tag key never appears in git, tofu state, argv, a terminal or a transcript (§6.1). |
| +- **Nothing live changes without Paul's go for that step.** Every `tofu apply`, console roll, env edit, restart, `antivirus enable`, merge, mirror push and release is preceded by showing him exactly what will run. Tasks 6–12 are marked **GATED**, as are the merge and mirror steps of Task 5. |
| +- Every tofu apply is `-target`ed. Never apply a whole root (flagship has standing `user_data` drift that restarts boxes; Tito's main cannot plan). |
| +- Carloku first, then Tito. Tito waits for #201 phase 2 to have landed. |
| +- User-facing prose (READMEs, `docs/site`, release notes) goes through the `copy-review` skill before it is written to a file. No em dashes in copy: full stop and a new sentence. |
| +- No GitHub issues. Track on an amadan discussion (`amadan` skill for the house rules). |
| +- Go: use the default build cache. Scratch goes under `$TMPDIR/scanner-host/`, never `/tmp`. Delete it at the end. |
| +- AWS profiles: `keymail` is the flagship account (console box `i-092c0c1eea75723cb`, eu-west-1, bucket `carlos-flagship-271376211898`). `titogo` is Tito's (console box `i-025a0b70f3d93304d`, eu-west-1). Assumed-role profiles need the Bash sandbox off for that command. |
| +- Commits end with `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>`. |
| + |
| +## Review Focus |
| + |
| +1. **An engine upgrade that changes limit behaviour.** A later ClamAV package could answer `OK` for an over-limit archive. Expected: the port closes and stays closed. Pinned by Task 7's forced-fail acceptance check and by the self-test running on every clamd start (Task 3). |
| +2. **A reboot of the scanner box.** Expected: it comes back scanning with no hand. First boot cannot show an ordering cycle. Pinned by Task 7's reboot check. |
| +3. **A signature name that merely contains the limit phrase.** `Win.Heuristics.Limits.Exceeded.Fake FOUND` must stay infected. Pinned by Task 1's tests. |
| +4. **A console restarted on the old binary with the scanner wired.** Expected: wiring stops before any variable is written. Pinned by Task 8's `/proc/<pid>/exe` check and Task 9's precondition. |
| +5. **A rerun of the key step.** Expected: the existing key survives; a second run cannot replace it and orphan every verdict. Pinned by `Overwrite:false` in Task 9. |
| + |
| +## File Structure |
| + |
| +Platform repo (this worktree, branch `worktree-scanner-host`): |
| + |
| +| File | Responsibility | |
| +|---|---| |
| +| `internal/clamd/clamd.go` (modify) | `parseReply`: limit replies become size-exceeded or error | |
| +| `internal/clamd/clamd_test.go` (modify) | tests for the above | |
| +| `infra/modules/region-host/variables.tf`, `main.tf` (modify) | `store_scanning` and the `-store-scan` policy | |
| +| `infra/modules/region-host/tests/store_scanning.tftest.hcl` (create) | plan-time tests for the grant | |
| +| `infra/modules/scanner-host/{versions,variables,main,outputs}.tf` (create) | the box, its group, its role | |
| +| `infra/modules/scanner-host/scanner-init.sh.tftpl` (create) | first-boot script | |
| +| `infra/modules/scanner-host/files/scanner-selftest` (create) | the self-test | |
| +| `infra/modules/scanner-host/files/scanner-selftest.service`, `scanner-tcp.socket`, `scanner-tcp.service` (create) | the gate | |
| +| `infra/modules/scanner-host/tests/scanner_host.tftest.hcl` (create) | plan-time tests | |
| +| `infra/modules/scanner-host/README.md` (create), `infra/modules/README.md`, `infra/modules/region-host/README.md`, `docs/site/cli.md`, `docs/superpowers/plans/2026-09-01-carloku-scanning-rollout.md` (modify) | docs | |
| + |
| +Other repos: `~/github.com/carlosframework/carloku-infrastructure` (`region-ie.tf`, new `scanner.tf`), `~/github.com/teamtito/carlos-infrastructure` (`object-store.tf`, new `scanner.tf`). |
| + |
| +--- |
| + |
| +### Task 1: clamd client records limit replies as unscanned |
| + |
| +**Files:** |
| +- Modify: `internal/clamd/clamd.go` (`parseReply`, near line 268; the `StatusSizeExceeded` comment near line 38) |
| +- Test: `internal/clamd/clamd_test.go` |
| + |
| +**Interfaces:** |
| +- Consumes: `replyFixture(t, reply string) Result` (existing test helper), `signatureOf(reply string) string` (existing). |
| +- Produces: no new names. `parseReply` now returns `StatusSizeExceeded` for limit heuristics other than scan time, and `StatusError` for `Heuristics.Limits.Exceeded.MaxScanTime`. `objstore.verdictFor` is unchanged and already maps these to `VerdictSkipped` and `VerdictFailed`. |
| + |
| +- [ ] **Step 1: Write the failing tests.** Append to `internal/clamd/clamd_test.go`: |
| + |
| +```go |
| +// With AlertExceedsMax on, clamd reports an archive it could only partly scan |
| +// as a FOUND whose signature names the limit. That is not an infection and it |
| +// is not clean: nothing was judged, so it is the same status as an object too |
| +// large to stream. Recording it clean would sign a verdict over bytes nobody |
| +// read; recording it infected would make enforce block large honest archives. |
| +func TestLimitHeuristicIsSizeExceededNotInfected(t *testing.T) { |
| + for _, limit := range []string{"MaxScanSize", "MaxFileSize", "MaxFiles", "MaxRecursion"} { |
| + reply := "stream: Heuristics.Limits.Exceeded." + limit + " FOUND\x00" |
| + res := replyFixture(t, reply) |
| + if res.Status != StatusSizeExceeded { |
| + t.Errorf("%s: status = %v, want size-exceeded", limit, res.Status) |
| + } |
| + if res.Signature != "" { |
| + t.Errorf("%s: a limit is not a signature, got %q", limit, res.Signature) |
| + } |
| + if !strings.Contains(res.Raw, limit) { |
| + t.Errorf("%s: the engine's own words did not survive: %q", limit, res.Raw) |
| + } |
| + } |
| +} |
| + |
| +// Older engines name the limit without saying which one. |
| +func TestBareLimitHeuristicIsSizeExceeded(t *testing.T) { |
| + res := replyFixture(t, "stream: Heuristics.Limits.Exceeded FOUND\x00") |
| + if res.Status != StatusSizeExceeded { |
| + t.Fatalf("status = %v, want size-exceeded", res.Status) |
| + } |
| +} |
| + |
| +// A scan that ran out of TIME depends on load, not on the bytes, so it must be |
| +// retried. StatusError becomes a failed verdict, which every sweep re-issues; |
| +// size-exceeded becomes skipped, which is carried forward until the object |
| +// changes. Filing a timeout under skipped would exclude the object for good. |
| +func TestScanTimeLimitIsAnErrorSoItIsRetried(t *testing.T) { |
| + res := replyFixture(t, "stream: Heuristics.Limits.Exceeded.MaxScanTime FOUND\x00") |
| + if res.Status != StatusError { |
| + t.Fatalf("status = %v, want error", res.Status) |
| + } |
| + if !strings.Contains(res.Raw, "MaxScanTime") { |
| + t.Fatalf("the engine's own words did not survive: %q", res.Raw) |
| + } |
| +} |
| + |
| +// The match is on the SIGNATURE, as a prefix. A real signature that happens to |
| +// carry the phrase elsewhere in its name is still an infection, and so is one |
| +// that only begins with something similar. |
| +func TestLookalikeSignaturesAreStillInfected(t *testing.T) { |
| + for _, sig := range []string{ |
| + "Win.Heuristics.Limits.Exceeded.Fake", |
| + "Heuristics.Limits.ExceededButNot", |
| + "Heuristics.Phishing.Email.SpoofedDomain", |
| + "Eicar-Signature", |
| + } { |
| + res := replyFixture(t, "stream: "+sig+" FOUND\x00") |
| + if res.Status != StatusInfected { |
| + t.Errorf("%s: status = %v, want infected", sig, res.Status) |
| + } |
| + if res.Signature != sig { |
| + t.Errorf("signature = %q, want %q", res.Signature, sig) |
| + } |
| + } |
| +} |
| +``` |
| + |
| +- [ ] **Step 2: Run them and watch them fail.** |
| + |
| +Run: `go test ./internal/clamd -run 'Limit|Lookalike' -count=1` |
| +Expected: FAIL. `TestLimitHeuristicIsSizeExceededNotInfected`, `TestBareLimitHeuristicIsSizeExceeded` and `TestScanTimeLimitIsAnErrorSoItIsRetried` report `status = infected`. `TestLookalikeSignaturesAreStillInfected` passes already; it is there to stay passing. |
| + |
| +- [ ] **Step 3: Implement.** In `internal/clamd/clamd.go`, add beside the timeout constants: |
| + |
| +```go |
| +// clamd's names for "I stopped before the end". With AlertExceedsMax yes it |
| +// reports them as FOUND, which parseReply must not read as an infection. |
| +const ( |
| + limitHeuristic = "Heuristics.Limits.Exceeded" |
| + limitHeuristicScanTime = "Heuristics.Limits.Exceeded.MaxScanTime" |
| +) |
| + |
| +// isLimitHeuristic reports whether sig names a scan limit rather than malware. |
| +// A prefix on the signature, with the dot, so neither a longer first component |
| +// nor a real signature carrying the phrase elsewhere can match. |
| +func isLimitHeuristic(sig string) bool { |
| + return sig == limitHeuristic || strings.HasPrefix(sig, limitHeuristic+".") |
| +} |
| +``` |
| + |
| +and replace the `FOUND` case of `parseReply`: |
| + |
| +```go |
| + case strings.HasSuffix(reply, " FOUND"): |
| + sig := signatureOf(reply) |
| + switch { |
| + case sig == limitHeuristicScanTime: |
| + // Load-dependent, so retried: an error becomes a failed verdict, |
| + // which every sweep re-issues. |
| + res.Status = StatusError |
| + case isLimitHeuristic(sig): |
| + // Deterministic for these bytes and this config, and nothing was |
| + // judged: the same fact as an object too large to stream. |
| + res.Status = StatusSizeExceeded |
| + default: |
| + res.Status = StatusInfected |
| + res.Signature = sig |
| + } |
| +``` |
| + |
| +Extend the `StatusSizeExceeded` doc comment with one sentence: `It also covers clamd's limit heuristics (an archive that expands past MaxScanSize, MaxFileSize, MaxFiles or MaxRecursion), which arrive as FOUND when AlertExceedsMax is on.` |
| + |
| +- [ ] **Step 4: Run the package.** |
| + |
| +Run: `go test ./internal/clamd -count=1` then `go vet ./internal/clamd` |
| +Expected: `ok`, no vet output. |
| + |
| +- [ ] **Step 5: Run what consumes it.** |
| + |
| +Run: `go test ./internal/objstore -run 'Clam|Scan|Verdict' -count=1` |
| +Expected: `ok` (no behaviour change there; this proves nothing asserted the old mapping). |
| + |
| +- [ ] **Step 6: Commit.** |
| + |
| +```bash |
| +git add internal/clamd/clamd.go internal/clamd/clamd_test.go |
| +git commit -m "clamd: a limit heuristic is unscanned, and a scan timeout is retried" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" |
| +``` |
| + |
| +--- |
| + |
| +### Task 2: `region-host` grants the console read-and-tag, opt-in |
| + |
| +**Files:** |
| +- Modify: `infra/modules/region-host/variables.tf` (after `store_user_path`), `infra/modules/region-host/main.tf` (after `host_store`; and the `host_store` comment) |
| +- Create: `infra/modules/region-host/tests/store_scanning.tftest.hcl` |
| + |
| +**Interfaces:** |
| +- Produces: variable `store_scanning` (bool, default `false`); resource `aws_iam_role_policy.host_store_scan[0]` named `"${var.iam_name}-store-scan"`. Task 7 targets `module.host_ie.aws_iam_role_policy.host_store_scan`. |
| + |
| +- [ ] **Step 1: Write the failing test.** Create `infra/modules/region-host/tests/store_scanning.tftest.hcl`: |
| + |
| +```hcl |
| +# Plan-time tests for the scanning grant. mock_provider means no AWS account |
| +# and no credentials: these assert what the module WOULD create. |
| +mock_provider "aws" {} |
| + |
| +override_data { |
| + target = data.aws_vpcs.default |
| + values = { ids = ["vpc-0aaaaaaaaaaaaaaaa"] } |
| +} |
| + |
| +variables { |
| + name = "carlos-test-ie" |
| + sg_description = "carlos test host" |
| + iam_name = "carlos-test-host" |
| + instance_type = "t4g.small" |
| + root_volume_size = 20 |
| + user_data = "#!/bin/sh\n" |
| + deployment_bucket_arn = "arn:aws:s3:::carlos-test-platform" |
| + data_bucket_arn = "arn:aws:s3:::carlos-test-ie" |
| + ssm_parameter_arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/carlos/*" |
| + store_bucket_prefix = "carlos-store-test-" |
| +} |
| + |
| +run "off_by_default" { |
| + command = plan |
| + assert { |
| + condition = length(aws_iam_role_policy.host_store_scan) == 0 |
| + error_message = "store_scanning defaults to false and must create nothing" |
| + } |
| +} |
| + |
| +run "on_grants_exactly_read_and_tag" { |
| + command = plan |
| + variables { |
| + store_scanning = true |
| + } |
| + assert { |
| + condition = length(aws_iam_role_policy.host_store_scan) == 1 |
| + error_message = "store_scanning = true must create the scan policy" |
| + } |
| + assert { |
| + condition = aws_iam_role_policy.host_store_scan[0].name == "carlos-test-host-store-scan" |
| + error_message = "the policy is named <iam_name>-store-scan" |
| + } |
| + assert { |
| + condition = toset(jsondecode(aws_iam_role_policy.host_store_scan[0].policy).Statement[0].Action) == toset([ |
| + "s3:GetObject", "s3:GetObjectTagging", "s3:PutObjectTagging", |
| + ]) |
| + error_message = "exactly the three actions the scanner calls, no more" |
| + } |
| + assert { |
| + condition = jsondecode(aws_iam_role_policy.host_store_scan[0].policy).Statement[0].Resource == "arn:aws:s3:::carlos-store-test-*/*" |
| + error_message = "scoped to objects under the store prefix" |
| + } |
| +} |
| + |
| +run "refused_without_a_store_prefix" { |
| + command = plan |
| + variables { |
| + store_scanning = true |
| + store_bucket_prefix = "" |
| + } |
| + expect_failures = [aws_iam_role_policy.host_store_scan] |
| +} |
| +``` |
| + |
| +- [ ] **Step 2: Run it and watch it fail.** |
| + |
| +```bash |
| +export TF_DATA_DIR="$TMPDIR/scanner-host/tf-region-host" |
| +tofu -chdir=infra/modules/region-host init -backend=false -input=false |
| +tofu -chdir=infra/modules/region-host test |
| +``` |
| +Expected: FAIL, `Reference to undeclared resource ... "aws_iam_role_policy" "host_store_scan"`. (The first `init` downloads the AWS provider into `TF_DATA_DIR`, on disk, not `/tmp`. If a mock data source makes an unrelated precondition fail, add an `override_data` for it in this file and say so in the commit; do not weaken the module.) |
| + |
| +- [ ] **Step 3: Implement.** In `variables.tf`, after `store_user_path`: |
| + |
| +```hcl |
| +variable "store_scanning" { |
| + description = "Let this host's role READ and TAG objects under store_bucket_prefix, which is what the console's virus scanner needs (s3:GetObject, s3:GetObjectTagging, s3:PutObjectTagging). Off by default: without it the console provisions stores and never reads tenant data. Set it only on the box that runs the console, and only on a deployment that runs a scanner (infra/modules/scanner-host). Requires store_bucket_prefix." |
| + type = bool |
| + default = false |
| +} |
| +``` |
| + |
| +In `main.tf`, directly after the `host_store` resource: |
| + |
| +```hcl |
| +# ---- object storage scanning (opt-in) -------------------------------------- |
| +# |
| +# The one grant that lets this role read tenant object bytes, so it is its own |
| +# policy behind its own switch: a deployment that has not turned scanning on |
| +# plans no change here, and the sentence in host_store above ("it does not |
| +# read tenant data") stays true for it. |
| +# |
| +# Exactly the calls the scanner makes. HeadObject is authorised by GetObject; |
| +# ListObjectsV2 is already in host_store. Store buckets use SSE-S3, so there is |
| +# no KMS statement. PutObjectTagging is how a verdict is written; the store |
| +# user boundary deliberately lacks it, so an app cannot write one itself. |
| +resource "aws_iam_role_policy" "host_store_scan" { |
| + count = var.store_scanning ? 1 : 0 |
| + name = "${var.iam_name}-store-scan" |
| + role = aws_iam_role.host.id |
| + policy = jsonencode({ |
| + Version = "2012-10-17" |
| + Statement = [ |
| + { |
| + Effect = "Allow" |
| + Action = ["s3:GetObject", "s3:GetObjectTagging", "s3:PutObjectTagging"] |
| + Resource = "arn:aws:s3:::${var.store_bucket_prefix}*/*" |
| + }, |
| + ] |
| + }) |
| + lifecycle { |
| + precondition { |
| + condition = var.store_bucket_prefix != "" |
| + error_message = "store_scanning needs store_bucket_prefix: with no prefix there are no object stores on this deployment to scan, and the grant would name every bucket in the account." |
| + } |
| + } |
| +} |
| +``` |
| + |
| +In the `host_store` comment, change `No GetObject: the console provisions stores, it does not read tenant data, and the economics sweep sizes buckets with ListBucket.` to add: ` (Scanning is the exception, and it is a separate policy behind store_scanning, below.)` |
| + |
| +- [ ] **Step 4: Run the tests and the formatter.** |
| + |
| +Run: `tofu -chdir=infra/modules/region-host test` then `tofu fmt -check -recursive infra/modules/region-host` |
| +Expected: `3 passed, 0 failed`; fmt prints nothing. |
| + |
| +- [ ] **Step 5: Commit.** Confirm `git status --short infra/` shows no `.terraform` or lock file, then: |
| + |
| +```bash |
| +git add infra/modules/region-host/variables.tf infra/modules/region-host/main.tf infra/modules/region-host/tests/store_scanning.tftest.hcl |
| +git commit -m "region-host: store_scanning, the console's opt-in read-and-tag grant" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" |
| +``` |
| + |
| +--- |
| + |
| +### Task 3: the `scanner-host` module |
| + |
| +**Files:** |
| +- Create: `infra/modules/scanner-host/versions.tf`, `variables.tf`, `main.tf`, `outputs.tf`, `scanner-init.sh.tftpl`, `files/scanner-selftest`, `files/scanner-selftest.service`, `files/scanner-tcp.socket`, `files/scanner-tcp.service`, `tests/scanner_host.tftest.hcl` |
| + |
| +**Interfaces:** |
| +- Consumes: nothing from earlier tasks. |
| +- Produces: module inputs `name`, `ami_id`, `client_security_group_id`, `region`, `instance_type`, `stream_max_bytes`, `memory_max`, `memory_high`, `concurrent_database_reload`, `subnet_id`; outputs `clamd_addr`, `instance_id`, `security_group_id`, `stream_max_bytes`. On the box: `/usr/local/sbin/scanner-selftest`, units `scanner-selftest.service`, `scanner-tcp.socket`, `scanner-tcp.service`, marker `/var/lib/scanner-host/ready`, console-log lines `SCANNER-HOST READY` / `SCANNER-HOST FAILED: <reason>`, and the environment switch `SCANNER_SELFTEST_FORCE_FAIL=1`. Tasks 7, 9, 11 and 12 use these names. |
| + |
| +- [ ] **Step 1: Write the failing tests.** Create `infra/modules/scanner-host/tests/scanner_host.tftest.hcl`: |
| + |
| +```hcl |
| +# Plan-time tests. mock_provider: no account, no credentials. |
| +mock_provider "aws" {} |
| + |
| +override_data { |
| + target = data.aws_security_group.client |
| + values = { vpc_id = "vpc-0aaaaaaaaaaaaaaaa" } |
| +} |
| + |
| +override_data { |
| + target = data.aws_vpc.client |
| + values = { default = true } |
| +} |
| + |
| +variables { |
| + name = "carlos-test-scanner" |
| + ami_id = "ami-0123456789abcdef0" |
| + client_security_group_id = "sg-0123456789abcdef0" |
| + region = "eu-west-1" |
| +} |
| + |
| +run "the_fence" { |
| + command = plan |
| + assert { |
| + condition = length(aws_security_group.scanner.ingress) == 1 |
| + error_message = "exactly one way in" |
| + } |
| + assert { |
| + condition = alltrue([ |
| + for i in aws_security_group.scanner.ingress : |
| + i.from_port == 3310 && i.to_port == 3310 && i.protocol == "tcp" |
| + && i.security_groups == toset(["sg-0123456789abcdef0"]) |
| + && try(length(i.cidr_blocks), 0) == 0 && try(length(i.ipv6_cidr_blocks), 0) == 0 |
| + ]) |
| + error_message = "3310 from the client security group only, never a CIDR" |
| + } |
| + assert { |
| + condition = aws_security_group.scanner.vpc_id == "vpc-0aaaaaaaaaaaaaaaa" |
| + error_message = "the group lands in the client group's VPC" |
| + } |
| +} |
| + |
| +run "the_box" { |
| + command = plan |
| + assert { |
| + condition = aws_instance.scanner.metadata_options[0].http_tokens == "required" |
| + error_message = "IMDSv2 only" |
| + } |
| + assert { |
| + condition = aws_instance.scanner.root_block_device[0].encrypted |
| + error_message = "encrypted root volume" |
| + } |
| + assert { |
| + condition = aws_instance.scanner.instance_type == "t4g.medium" |
| + error_message = "t4g.medium by default" |
| + } |
| + assert { |
| + # base64 is 4 characters per 3 bytes, so 21800 characters is under 16 KiB. |
| + condition = length(base64gzip(local.init_script)) < 21800 |
| + error_message = "EC2 refuses user_data over 16 KiB, and the script is sent gzipped to stay well under it" |
| + } |
| +} |
| + |
| +run "the_role_cannot_read_parameters" { |
| + command = plan |
| + assert { |
| + condition = jsondecode(aws_iam_role_policy.no_parameters.policy).Statement[0].Effect == "Deny" |
| + error_message = "an explicit Deny, because the managed SSM policy allows GetParameter* on *" |
| + } |
| + assert { |
| + condition = toset(jsondecode(aws_iam_role_policy.no_parameters.policy).Statement[0].Action) == toset([ |
| + "ssm:GetParameter", "ssm:GetParameters", "ssm:GetParameterHistory", "ssm:GetParametersByPath", |
| + ]) |
| + error_message = "all four parameter-read actions" |
| + } |
| + assert { |
| + condition = jsondecode(aws_iam_role_policy.no_parameters.policy).Statement[0].Resource == "*" |
| + error_message = "on every parameter: this box is allowed none" |
| + } |
| +} |
| + |
| +run "the_engine_config" { |
| + command = plan |
| + assert { |
| + condition = strcontains(local.init_script, "setkey MaxFileSize 100M") && strcontains(local.init_script, "setkey MaxScanSize 100M") |
| + error_message = "both limits 100M: equal is what makes a trimmed member trip the scan budget" |
| + } |
| + assert { |
| + condition = strcontains(local.init_script, "setkey AlertExceedsMax yes") |
| + error_message = "limits must alert" |
| + } |
| + assert { |
| + condition = strcontains(local.init_script, "setkey StreamMaxLength 26214400") |
| + error_message = "StreamMaxLength is stream_max_bytes" |
| + } |
| + assert { |
| + condition = !strcontains(local.init_script, "setkey TCPSocket") && !strcontains(local.init_script, "setkey TCPAddr") |
| + error_message = "clamd itself never listens on TCP; the gated proxy does" |
| + } |
| + assert { |
| + condition = output.stream_max_bytes == 26214400 |
| + error_message = "the console's CARLOS_CLAMD_MAX_BYTES comes from this output" |
| + } |
| +} |
| + |
| +run "small_box_with_concurrent_reload_is_refused" { |
| + command = plan |
| + variables { |
| + instance_type = "t4g.small" |
| + } |
| + expect_failures = [aws_instance.scanner] |
| +} |
| + |
| +run "small_box_without_concurrent_reload_is_fine" { |
| + command = plan |
| + variables { |
| + instance_type = "t4g.small" |
| + concurrent_database_reload = false |
| + } |
| + assert { |
| + condition = strcontains(local.init_script, "setkey ConcurrentDatabaseReload no") |
| + error_message = "the reload setting reaches clamd.conf" |
| + } |
| +} |
| + |
| +run "stream_limit_too_small_for_the_fixtures" { |
| + command = plan |
| + variables { |
| + stream_max_bytes = 1048576 |
| + } |
| + expect_failures = [var.stream_max_bytes] |
| +} |
| + |
| +run "stream_limit_above_the_scan_budget" { |
| + command = plan |
| + variables { |
| + stream_max_bytes = 209715200 |
| + } |
| + expect_failures = [var.stream_max_bytes] |
| +} |
| + |
| +run "another_vpc_needs_a_subnet" { |
| + command = plan |
| + override_data { |
| + target = data.aws_vpc.client |
| + values = { default = false } |
| + } |
| + expect_failures = [aws_instance.scanner] |
| +} |
| +``` |
| + |
| +- [ ] **Step 2: Run it and watch it fail.** |
| + |
| +```bash |
| +export TF_DATA_DIR="$TMPDIR/scanner-host/tf-scanner-host" |
| +tofu -chdir=infra/modules/scanner-host init -backend=false -input=false |
| +tofu -chdir=infra/modules/scanner-host test |
| +``` |
| +Expected: FAIL at init or test: the module has no configuration yet. |
| + |
| +- [ ] **Step 3: `versions.tf` and `variables.tf`.** |
| + |
| +`versions.tf`: |
| + |
| +```hcl |
| +terraform { |
| + required_version = ">= 1.10" |
| + required_providers { |
| + aws = { |
| + source = "hashicorp/aws" |
| + version = ">= 6.0" |
| + } |
| + } |
| +} |
| +``` |
| + |
| +`variables.tf`: |
| + |
| +```hcl |
| +variable "name" { |
| + description = "Name for the instance, its security group, its IAM role and its instance profile (e.g. \"carlos-flagship-scanner\")." |
| + type = string |
| +} |
| + |
| +variable "ami_id" { |
| + description = "Debian arm64 AMI. Pass the console box's own image, module.<host>.ami_id, so the two are built from the same base. Ignored after creation: a newer image never replaces the box by itself." |
| + type = string |
| +} |
| + |
| +variable "client_security_group_id" { |
| + description = "Security group of the ONE box allowed to reach clamd: the console's. clamd has no authentication at all, so this group is the whole fence. The scanner's own group is created in this group's VPC." |
| + type = string |
| +} |
| + |
| +variable "region" { |
| + description = "AWS region the box runs in (e.g. \"eu-west-1\"). The SSM agent package is served per region and the box must fetch its own." |
| + type = string |
| +} |
| + |
| +variable "instance_type" { |
| + description = "EC2 instance type (arm64 Graviton). clamd holds about 1.5 GB of signatures resident and briefly about twice that while reloading them, so t4g.medium is the default. A t4g.small works only with concurrent_database_reload = false." |
| + type = string |
| + default = "t4g.medium" |
| +} |
| + |
| +variable "stream_max_bytes" { |
| + description = "Largest object clamd accepts on a stream (StreamMaxLength), in bytes. Set the console's CARLOS_CLAMD_MAX_BYTES to this module's stream_max_bytes output: the two must be equal. Between 8 MiB and 100 MiB: the box's self-test streams fixtures of about 5 MB, and a whole stream must fit the 100 MiB scan budget." |
| + type = number |
| + default = 26214400 |
| + validation { |
| + condition = var.stream_max_bytes >= 8388608 && var.stream_max_bytes <= 104857600 |
| + error_message = "stream_max_bytes must be between 8388608 (8 MiB) and 104857600 (100 MiB)." |
| + } |
| +} |
| + |
| +variable "memory_max" { |
| + description = "clamd's cgroup ceiling (systemd MemoryMax). Under memory pressure the kernel kills clamd, never the SSM agent that is the only way onto the box." |
| + type = string |
| + default = "3G" |
| +} |
| + |
| +variable "memory_high" { |
| + description = "clamd's throttle point (systemd MemoryHigh), about 85% of memory_max." |
| + type = string |
| + default = "2560M" |
| +} |
| + |
| +variable "concurrent_database_reload" { |
| + description = "Keep scanning while freshclam's new signatures load, at the cost of holding both databases in memory at once. False pauses scanning during each reload and fits a smaller box." |
| + type = bool |
| + default = true |
| +} |
| + |
| +variable "subnet_id" { |
| + description = "Subnet to launch in. Null (the default) is the default subnet of the default VPC, and is refused unless the client security group lives in the default VPC. The subnet needs a route to the internet: the box fetches signatures and packages." |
| + type = string |
| + default = null |
| +} |
| +``` |
| + |
| +- [ ] **Step 4: `main.tf` and `outputs.tf`.** |
| + |
| +`main.tf`: |
| + |
| +```hcl |
| +# 🤖 A CARLOS scanner host: one box running clamd for a deployment's console. |
| +# |
| +# The console reads each object from S3 and streams it here. This box reads |
| +# nothing for itself: its role has Session Manager and nothing else, so a |
| +# compromised clamd cannot fetch a tenant object or a key. |
| +# |
| +# What makes it more than "install clamav" is in scanner-init.sh.tftpl and |
| +# files/: TCP 3310 is open only while a self-test has passed against the clamd |
| +# process running now. See README.md. |
| + |
| +data "aws_security_group" "client" { |
| + id = var.client_security_group_id |
| +} |
| + |
| +data "aws_vpc" "client" { |
| + id = data.aws_security_group.client.vpc_id |
| +} |
| + |
| +locals { |
| + # Fixed in this release. The self-test's fixtures and expected answers are |
| + # sized for it; making it a variable means deriving both. |
| + scan_limit = "100M" |
| + |
| + small_box = can(regex("^t4g\\.(nano|micro|small)$", var.instance_type)) |
| + |
| + init_script = templatefile("${path.module}/scanner-init.sh.tftpl", { |
| + region = var.region |
| + stream_max_bytes = var.stream_max_bytes |
| + scan_limit = local.scan_limit |
| + memory_max = var.memory_max |
| + memory_high = var.memory_high |
| + concurrent_reload = var.concurrent_database_reload ? "yes" : "no" |
| + selftest_b64 = base64encode(file("${path.module}/files/scanner-selftest")) |
| + selftest_unit_b64 = base64encode(file("${path.module}/files/scanner-selftest.service")) |
| + tcp_socket_b64 = base64encode(file("${path.module}/files/scanner-tcp.socket")) |
| + tcp_service_b64 = base64encode(file("${path.module}/files/scanner-tcp.service")) |
| + }) |
| +} |
| + |
| +resource "aws_security_group" "scanner" { |
| + name = var.name |
| + description = "clamd, reachable from the console box only" |
| + vpc_id = data.aws_security_group.client.vpc_id |
| + |
| + # The whole fence. clamd authenticates nobody, so nothing but the console's |
| + # own group may reach it, and never a CIDR. |
| + ingress { |
| + description = "clamd INSTREAM from the console" |
| + from_port = 3310 |
| + to_port = 3310 |
| + protocol = "tcp" |
| + security_groups = [var.client_security_group_id] |
| + } |
| + |
| + # Signatures (freshclam) and packages (apt). No SSH ingress exists: admin |
| + # access is SSM Session Manager. |
| + egress { |
| + from_port = 0 |
| + to_port = 0 |
| + protocol = "-1" |
| + cidr_blocks = ["0.0.0.0/0"] |
| + } |
| + |
| + tags = { Name = var.name } |
| +} |
| + |
| +resource "aws_iam_role" "scanner" { |
| + name = var.name |
| + assume_role_policy = jsonencode({ |
| + Version = "2012-10-17" |
| + Statement = [{ |
| + Effect = "Allow" |
| + Principal = { Service = "ec2.amazonaws.com" } |
| + Action = "sts:AssumeRole" |
| + }] |
| + }) |
| +} |
| + |
| +resource "aws_iam_role_policy_attachment" "scanner_ssm" { |
| + role = aws_iam_role.scanner.name |
| + policy_arn = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore" |
| +} |
| + |
| +# AmazonSSMManagedInstanceCore grants ssm:GetParameter* on "*". Left alone, |
| +# this box could read the deployment's scan tag key and its release key. An |
| +# explicit Deny beats the managed Allow and names only the parameter-read |
| +# actions, so Session Manager is untouched. Same shape as region-host's |
| +# host_ssm_parameter_scope, with no parameter allowed at all. |
| +resource "aws_iam_role_policy" "no_parameters" { |
| + name = "${var.name}-no-parameters" |
| + role = aws_iam_role.scanner.id |
| + policy = jsonencode({ |
| + Version = "2012-10-17" |
| + Statement = [ |
| + { |
| + Effect = "Deny" |
| + Action = [ |
| + "ssm:GetParameter", |
| + "ssm:GetParameters", |
| + "ssm:GetParameterHistory", |
| + "ssm:GetParametersByPath", |
| + ] |
| + Resource = "*" |
| + }, |
| + ] |
| + }) |
| +} |
| + |
| +resource "aws_iam_instance_profile" "scanner" { |
| + name = var.name |
| + role = aws_iam_role.scanner.name |
| +} |
| + |
| +resource "aws_instance" "scanner" { |
| + ami = var.ami_id |
| + instance_type = var.instance_type |
| + subnet_id = var.subnet_id |
| + vpc_security_group_ids = [aws_security_group.scanner.id] |
| + iam_instance_profile = aws_iam_instance_profile.scanner.name |
| + |
| + # Gzipped: the script carries the self-test and three units, and EC2 caps |
| + # user_data at 16 KiB. cloud-init unpacks it. |
| + user_data_base64 = base64gzip(local.init_script) |
| + user_data_replace_on_change = false |
| + |
| + metadata_options { |
| + http_endpoint = "enabled" |
| + http_tokens = "required" |
| + http_put_response_hop_limit = 1 |
| + } |
| + |
| + root_block_device { |
| + volume_size = 20 |
| + volume_type = "gp3" |
| + encrypted = true |
| + } |
| + |
| + tags = { Name = var.name } |
| + |
| + # A newer image or an edited script never replaces the box by itself. |
| + # Changing either, including stream_max_bytes, is an explicit |
| + # `tofu apply -replace=module.<name>.aws_instance.scanner`. A replaced box |
| + # has a new private IP: update the console's CARLOS_CLAMD_ADDR. |
| + lifecycle { |
| + ignore_changes = [ami, user_data, user_data_base64] |
| + |
| + precondition { |
| + condition = var.subnet_id != null || data.aws_vpc.client.default |
| + error_message = "The client security group is not in the default VPC, so the scanner cannot use the default subnet: its security group and its instance would be in different VPCs. Pass subnet_id naming a subnet in the client group's VPC." |
| + } |
| + |
| + precondition { |
| + condition = !(local.small_box && var.concurrent_database_reload) |
| + error_message = "A t4g.small or smaller cannot hold two signature databases at once. Either use t4g.medium, or set concurrent_database_reload = false (scanning pauses during each reload)." |
| + } |
| + } |
| +} |
| +``` |
| + |
| +`outputs.tf`: |
| + |
| +```hcl |
| +output "clamd_addr" { |
| + description = "Where the console reaches clamd: set it as CARLOS_CLAMD_ADDR. A private address, reachable only from the client security group. It changes if the box is replaced." |
| + value = "${aws_instance.scanner.private_ip}:3310" |
| +} |
| + |
| +output "stream_max_bytes" { |
| + description = "clamd's StreamMaxLength in bytes: set it as CARLOS_CLAMD_MAX_BYTES on the console." |
| + value = var.stream_max_bytes |
| +} |
| + |
| +output "instance_id" { |
| + description = "EC2 instance id of the scanner box, for SSM." |
| + value = aws_instance.scanner.id |
| +} |
| + |
| +output "security_group_id" { |
| + description = "Security group id of the scanner box." |
| + value = aws_security_group.scanner.id |
| +} |
| +``` |
| + |
| +- [ ] **Step 5: the self-test.** Create `infra/modules/scanner-host/files/scanner-selftest` (mode 0755 in git: `chmod +x` before `git add`): |
| + |
| +```bash |
| +#!/bin/bash |
| +# 🤖 scanner-selftest: prove, on THIS engine, that clamd never answers OK for |
| +# an archive it could only partly scan. |
| +# |
| +# scanner-selftest.service runs this every time clamd starts, and the TCP |
| +# listener is bound to that unit, so a failure here keeps port 3310 shut. It |
| +# talks to clamd on its unix socket: the port it guards is not open yet. |
| +# |
| +# Why it exists: with ClamAV's packaged limits, a zip member that expands past |
| +# MaxFileSize is trimmed and the trimmed copy is scanned and answered OK, with |
| +# no limit raised. The box is configured so that cannot happen (both limits |
| +# equal, AlertExceedsMax on). This checks that the configuration still does |
| +# what it did when it was chosen, on whatever version is installed today. |
| +# |
| +# Fixtures are sized for a 100 MiB scan limit. Members are written from a temp |
| +# file so their zip headers are ordinary; streamed members get zip64 headers, |
| +# which clamd does not open at all. |
| +set -u |
| +sock=/run/clamav/clamd.ctl |
| + |
| +if [ "${SCANNER_SELFTEST_FORCE_FAIL:-}" = 1 ]; then |
| + echo "scanner-selftest: FAIL (forced, for the acceptance check)" |
| + exit 1 |
| +fi |
| + |
| +d=$(mktemp -d /var/tmp/scanner-selftest.XXXXXX) || exit 1 |
| +trap 'rm -rf "$d"' EXIT |
| +echo "LocalSocket $sock" > "$d/clamdscan.conf" |
| + |
| +python3 - "$d" <<'PY' || { echo "scanner-selftest: FAIL (could not build fixtures)"; exit 1; } |
| +import io, gzip, os, random, sys, tarfile, zipfile |
| +d = sys.argv[1] |
| +fx = os.path.join(d, 'fx') |
| +os.mkdir(fx) |
| +EICAR = rb'X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*' |
| + |
| +def block(seed): |
| + # 1 MiB: a 16 KiB random block repeated. It compresses about 140 to 1 and |
| + # differs per seed, so clamd's clean cache cannot wave a member through. |
| + r = random.Random(seed) |
| + return bytes(r.getrandbits(8) for _ in range(16384)) * 64 |
| + |
| +def bigfile(name, mb, seed): |
| + p = os.path.join(d, name) |
| + blk = block(seed) |
| + with open(p, 'wb') as f: |
| + for _ in range(mb): |
| + f.write(blk) |
| + return p |
| + |
| +def mkzip(name, build): |
| + with zipfile.ZipFile(os.path.join(fx, name), 'w', zipfile.ZIP_DEFLATED) as z: |
| + build(z) |
| + |
| +open(os.path.join(fx, '01-clean.txt'), 'wb').write(b'hello world\n') |
| +open(os.path.join(fx, '02-eicar.txt'), 'wb').write(EICAR) |
| +big = bigfile('big.bin', 500, 1) |
| +mid = bigfile('mid.bin', 60, 5) |
| +mkzip('03-zip-one-500.zip', lambda z: z.write(big, 'big.bin')) |
| +with zipfile.ZipFile(os.path.join(d, 'inner.zip'), 'w', zipfile.ZIP_DEFLATED) as z: |
| + z.write(big, 'big.bin') |
| +mkzip('04-zip-nested-500.zip', lambda z: z.write(os.path.join(d, 'inner.zip'), 'inner.zip')) |
| +def many(z): |
| + for i in range(30): |
| + z.write(bigfile('m.bin', 20, 100 + i), 'm%02d.bin' % i) |
| +mkzip('05-zip-30x20.zip', many) |
| +with gzip.open(os.path.join(fx, '06-gzip-500.gz'), 'wb') as g: |
| + blk = block(2) |
| + for _ in range(500): |
| + g.write(blk) |
| +with tarfile.open(os.path.join(fx, '07-targz-5x40.tar.gz'), 'w:gz') as t: |
| + for i in range(5): |
| + data = block(200 + i) * 40 |
| + ti = tarfile.TarInfo('t%d.bin' % i) |
| + ti.size = len(data) |
| + t.addfile(ti, io.BytesIO(data)) |
| +def big_then_eicar(z): |
| + z.write(big, 'big.bin') |
| + z.writestr('eicar.com', EICAR) |
| +mkzip('08-zip-500-then-eicar.zip', big_then_eicar) |
| +def mid_then_eicar(z): |
| + z.write(mid, 'mid.bin') |
| + z.writestr('eicar.com', EICAR) |
| +mkzip('09-zip-60-then-eicar.zip', mid_then_eicar) |
| +mkzip('10-zip-one-60.zip', lambda z: z.write(mid, 'mid.bin')) |
| +PY |
| + |
| +# What each fixture must answer. "limit" is the one that matters: an OK there |
| +# is a truncated scan signed clean. |
| +want() { |
| + case $1 in |
| + 01 | 10) echo ok ;; |
| + 02 | 09) echo infected ;; |
| + *) echo limit ;; |
| + esac |
| +} |
| + |
| +fail=0 |
| +for f in "$d"/fx/*; do |
| + n=$(basename "$f") |
| + exp=$(want "${n%%-*}") |
| + reply=$(clamdscan --config-file="$d/clamdscan.conf" --stream --no-summary "$f" 2>&1 | tail -1) |
| + case $reply in |
| + *"Heuristics.Limits.Exceeded"*" FOUND") got=limit ;; |
| + *" FOUND") got=infected ;; |
| + *": OK") got=ok ;; |
| + *) got=error ;; |
| + esac |
| + if [ "$got" = "$exp" ]; then |
| + echo "scanner-selftest: pass $n ($got)" |
| + else |
| + echo "scanner-selftest: FAIL $n want $exp, got $got: $reply" |
| + fail=1 |
| + fi |
| +done |
| + |
| +if [ $fail -eq 0 ]; then |
| + echo "scanner-selftest: PASS" |
| +else |
| + echo "scanner-selftest: FAIL" |
| +fi |
| +exit $fail |
| +``` |
| + |
| +- [ ] **Step 6: the three units.** |
| + |
| +`files/scanner-selftest.service`: |
| + |
| +```ini |
| +# 🤖 Runs the self-test against the clamd that is running now. BindsTo makes it |
| +# go inactive the moment clamd stops or restarts, and the TCP listener is bound |
| +# to THIS unit, so the port follows the test, not the boot. |
| +[Unit] |
| +Description=Prove this clamd never answers OK for an over-limit archive |
| +BindsTo=clamav-daemon.service |
| +After=clamav-daemon.service |
| + |
| +[Service] |
| +Type=oneshot |
| +RemainAfterExit=yes |
| +ExecStart=/usr/local/sbin/scanner-selftest |
| +TimeoutStartSec=1800 |
| + |
| +[Install] |
| +WantedBy=multi-user.target |
| +``` |
| + |
| +`files/scanner-tcp.socket` (`@PRIVATE_IP@` is replaced at first boot): |
| + |
| +```ini |
| +# 🤖 clamd on TCP, for the console. Open only behind a passed self-test. |
| +# |
| +# DefaultDependencies=no is load-bearing. A socket unit is ordered before |
| +# sockets.target by default, sockets.target before basic.target, and every |
| +# service (the self-test included) after basic.target. A socket that must also |
| +# come AFTER the self-test closes that loop, and systemd breaks an ordering |
| +# cycle by dropping a job, which could leave this port shut after a reboot. |
| +# So it opts out, restores what it still needs, and is started by |
| +# multi-user.target and by clamd's Wants=, never by sockets.target. |
| +[Unit] |
| +Description=clamd on TCP, open only behind a passed self-test |
| +DefaultDependencies=no |
| +After=sysinit.target |
| +Requires=scanner-selftest.service |
| +After=scanner-selftest.service |
| +BindsTo=scanner-selftest.service |
| +Conflicts=shutdown.target |
| +Before=shutdown.target |
| + |
| +[Socket] |
| +ListenStream=@PRIVATE_IP@:3310 |
| + |
| +[Install] |
| +WantedBy=multi-user.target |
| +``` |
| + |
| +`files/scanner-tcp.service`: |
| + |
| +```ini |
| +# 🤖 Carries each TCP connection to clamd's unix socket. Bound to the self-test |
| +# like the socket is, so connections in flight are cut when the test's unit |
| +# goes inactive. |
| +[Unit] |
| +Description=Proxy TCP 3310 to clamd's unix socket |
| +Requires=scanner-selftest.service |
| +After=scanner-selftest.service |
| +BindsTo=scanner-selftest.service |
| + |
| +[Service] |
| +ExecStart=/usr/lib/systemd/systemd-socket-proxyd /run/clamav/clamd.ctl |
| +DynamicUser=yes |
| +SupplementaryGroups=clamav |
| +PrivateTmp=yes |
| +NoNewPrivileges=yes |
| +``` |
| + |
| +- [ ] **Step 7: the first-boot script.** Create `infra/modules/scanner-host/scanner-init.sh.tftpl`. It is a `templatefile`: `$${...}` would be needed for a literal bash `${...}`, so the bash below uses `$name` and `$(...)` forms only. Keep it that way. |
| + |
| +```bash |
| +#!/bin/bash |
| +# 🤖 First-boot init for a CARLOS scanner host (Debian arm64). Rendered by |
| +# infra/modules/scanner-host. Each step says where it got to on the serial |
| +# console, so `aws ec2 get-console-output` answers "how far did it get". |
| +# |
| +# The order is the design: |
| +# 1. SSM agent, because with no SSH it is the only way in to fix the rest. |
| +# 2. Packages installed but NOT started. |
| +# 3. Config, the self-test and the gate units. |
| +# 4. First signatures, alone, before anything else touches the database. |
| +# 5. Start: clamd, then the self-test, then (only on a pass) the port. |
| +# 6. Verify from the outside of the gate, and say READY or FAILED. |
| +set -euo pipefail |
| + |
| +say() { echo "scanner-init: $*" | tee /dev/console; } |
| +fail() { say "SCANNER-HOST FAILED: $*"; exit 1; } |
| +trap 'fail "line $LINENO"' ERR |
| +export DEBIAN_FRONTEND=noninteractive |
| + |
| +# ---- 1. the way in --------------------------------------------------------- |
| +say "1/6 SSM agent" |
| +apt-get update -q |
| +apt-get install -qy curl unattended-upgrades |
| +curl -fsSL -o /var/tmp/amazon-ssm-agent.deb \ |
| + "https://s3.${region}.amazonaws.com/amazon-ssm-${region}/latest/debian_arm64/amazon-ssm-agent.deb" |
| +dpkg -i /var/tmp/amazon-ssm-agent.deb |
| +rm -f /var/tmp/amazon-ssm-agent.deb |
| +systemctl enable --now amazon-ssm-agent |
| +systemctl enable --now unattended-upgrades |
| + |
| +# ---- 2. packages, not started ---------------------------------------------- |
| +# Debian's postinst starts clamav-freshclam and the daemon. Neither may run |
| +# before step 3 has configured it, and freshclam's service would fight the |
| +# foreground run in step 4 for the database lock. |
| +say "2/6 clamav packages" |
| +printf '#!/bin/sh\nexit 101\n' > /usr/sbin/policy-rc.d |
| +chmod +x /usr/sbin/policy-rc.d |
| +apt-get install -qy clamav-daemon clamav-freshclam clamdscan python3 iproute2 |
| +rm -f /usr/sbin/policy-rc.d |
| + |
| +# ---- 3. config, self-test, gate -------------------------------------------- |
| +say "3/6 config and gate" |
| +conf=/etc/clamav/clamd.conf |
| +# clamd has no include directory: replace each key in place. |
| +setkey() { |
| + sed -i "/^$1[[:space:]]/d" "$conf" |
| + echo "$1 $2" >> "$conf" |
| +} |
| +setkey StreamMaxLength ${stream_max_bytes} |
| +setkey AlertExceedsMax yes |
| +setkey MaxFileSize ${scan_limit} |
| +setkey MaxScanSize ${scan_limit} |
| +setkey ConcurrentDatabaseReload ${concurrent_reload} |
| +# Debian 13 mounts /tmp as tmpfs: extraction of up to the scan limit per scan |
| +# belongs on disk, not in memory. |
| +setkey TemporaryDirectory /var/tmp |
| +# clamd itself never listens on TCP. The gated proxy does. |
| +sed -i '/^TCPSocket[[:space:]]/d;/^TCPAddr[[:space:]]/d' "$conf" |
| + |
| +echo '${selftest_b64}' | base64 -d > /usr/local/sbin/scanner-selftest |
| +chmod 0755 /usr/local/sbin/scanner-selftest |
| +echo '${selftest_unit_b64}' | base64 -d > /etc/systemd/system/scanner-selftest.service |
| +echo '${tcp_service_b64}' | base64 -d > /etc/systemd/system/scanner-tcp.service |
| + |
| +tok=$(curl -fsS -X PUT http://169.254.169.254/latest/api/token \ |
| + -H 'X-aws-ec2-metadata-token-ttl-seconds: 300') |
| +ip=$(curl -fsS -H "X-aws-ec2-metadata-token: $tok" \ |
| + http://169.254.169.254/latest/meta-data/local-ipv4) |
| +case $ip in |
| + [0-9]*.[0-9]*.[0-9]*.[0-9]*) ;; |
| + *) fail "no private IPv4 from instance metadata: '$ip'" ;; |
| +esac |
| +echo '${tcp_socket_b64}' | base64 -d | sed "s/@PRIVATE_IP@/$ip/" \ |
| + > /etc/systemd/system/scanner-tcp.socket |
| +grep -q "^ListenStream=$ip:3310\$" /etc/systemd/system/scanner-tcp.socket \ |
| + || fail "the listener is not bound to the private address" |
| + |
| +mkdir -p /etc/systemd/system/clamav-daemon.service.d |
| +cat > /etc/systemd/system/clamav-daemon.service.d/10-scanner.conf <<EOF |
| +# Every start of clamd, an automatic restart included, asks for the gate again. |
| +[Unit] |
| +Wants=scanner-tcp.socket |
| + |
| +# The kernel kills clamd under memory pressure, never the SSM agent. |
| +[Service] |
| +MemoryMax=${memory_max} |
| +MemoryHigh=${memory_high} |
| +Restart=on-failure |
| +EOF |
| +systemctl daemon-reload |
| + |
| +# ---- 4. first signatures --------------------------------------------------- |
| +say "4/6 first signatures" |
| +# NotifyClamd would complain that no clamd is running yet. It is not. |
| +sed '/^NotifyClamd/d' /etc/clamav/freshclam.conf > /run/freshclam-first.conf |
| +freshclam --config-file=/run/freshclam-first.conf |
| +rm -f /run/freshclam-first.conf |
| +for db in main daily bytecode; do |
| + ls /var/lib/clamav/$db.c[vl]d >/dev/null 2>&1 || fail "no $db signature database after freshclam" |
| +done |
| +systemctl enable --now clamav-freshclam |
| + |
| +# ---- 5. start, in the order the units enforce ------------------------------ |
| +say "5/6 clamd, self-test, listener (several minutes)" |
| +systemctl enable clamav-daemon.service scanner-selftest.service scanner-tcp.socket |
| +systemctl start clamav-daemon.socket clamav-daemon.service |
| +# Blocks until the self-test has finished. Fails if it failed. |
| +systemctl start scanner-tcp.socket \ |
| + || fail "self-test or listener did not start: $(systemctl is-active clamav-daemon.service scanner-selftest.service scanner-tcp.socket | tr '\n' ' ')" |
| + |
| +# ---- 6. verify from outside the gate --------------------------------------- |
| +say "6/6 verify" |
| +[ "$(systemctl is-active scanner-selftest.service)" = active ] || fail "self-test is not active" |
| +[ "$(systemctl is-active scanner-tcp.socket)" = active ] || fail "listener is not active" |
| +bound=$(ss -ltnH 'sport = :3310' | awk '{print $4}' | sort -u | tr '\n' ' ') |
| +[ "$bound" = "$ip:3310 " ] || fail "3310 is bound to '$bound', want only $ip:3310" |
| +exec 3<>"/dev/tcp/$ip/3310" |
| +printf 'zVERSION\0' >&3 |
| +ver=$(timeout 20 head -c 200 <&3 | tr -d '\0') |
| +exec 3>&- |
| +case $ver in |
| + ClamAV*) ;; |
| + *) fail "no VERSION answer through the listener: '$ver'" ;; |
| +esac |
| +mkdir -p /var/lib/scanner-host |
| +echo "$ver" > /var/lib/scanner-host/ready |
| +say "SCANNER-HOST READY $ver" |
| +``` |
| + |
| +- [ ] **Step 8: Run the tests and the formatter.** |
| + |
| +Run: `tofu -chdir=infra/modules/scanner-host test` then `tofu fmt -check -recursive infra/modules/scanner-host` then `bash -n infra/modules/scanner-host/files/scanner-selftest` |
| +Expected: `9 passed, 0 failed`; fmt and `bash -n` print nothing. If an assertion fails because a mocked attribute is null or unknown at plan time (the `ingress` block attributes are the likely one), adjust the assertion's `try(...)`, never the module's behaviour, and say which in the commit. |
| + |
| +- [ ] **Step 9: Check the rendered script is valid bash and has no stray template syntax.** |
| + |
| +```bash |
| +mkdir -p "$TMPDIR/scanner-host/render" |
| +cat > "$TMPDIR/scanner-host/render/main.tf" <<'EOF' |
| +output "script" { |
| + value = templatefile("${path.module}/../../../infra/modules/scanner-host/scanner-init.sh.tftpl", { |
| + region = "eu-west-1", stream_max_bytes = 26214400, scan_limit = "100M", |
| + memory_max = "3G", memory_high = "2560M", concurrent_reload = "yes", |
| + selftest_b64 = "QQ==", selftest_unit_b64 = "QQ==", tcp_socket_b64 = "QQ==", tcp_service_b64 = "QQ==", |
| + }) |
| +} |
| +EOF |
| +``` |
| + |
| +Adjust the relative path so it resolves to this worktree's template (an absolute path is fine), then: |
| + |
| +Run: `tofu -chdir="$TMPDIR/scanner-host/render" init -input=false >/dev/null && tofu -chdir="$TMPDIR/scanner-host/render" apply -auto-approve >/dev/null && tofu -chdir="$TMPDIR/scanner-host/render" output -raw script > "$TMPDIR/scanner-host/render/init.sh" && bash -n "$TMPDIR/scanner-host/render/init.sh" && grep -c 'setkey MaxFileSize 100M' "$TMPDIR/scanner-host/render/init.sh"` |
| +Expected: `1`, and no bash syntax error. |
| + |
| +- [ ] **Step 10: Commit.** Confirm no `.terraform` directory or lock file is staged. |
| + |
| +```bash |
| +chmod +x infra/modules/scanner-host/files/scanner-selftest |
| +git add infra/modules/scanner-host |
| +git commit -m "scanner-host: a clamd box whose port is open only behind a passed self-test" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" |
| +``` |
| + |
| +--- |
| + |
| +### Task 4: docs, through copy review |
| + |
| +**Files:** |
| +- Create: `infra/modules/scanner-host/README.md` |
| +- Modify: `infra/modules/README.md` (new `## scanner-host` section), `infra/modules/region-host/README.md` (one paragraph on `store_scanning`), `docs/site/cli.md` (the operator table near line 1088 and the "What `status` will and will not claim" paragraph near line 1071), `docs/superpowers/plans/2026-09-01-carloku-scanning-rollout.md` (a dated note at the top) |
| + |
| +**Interfaces:** |
| +- Consumes: every name Task 3 produces. |
| + |
| +- [ ] **Step 1: Draft, do not write.** Use the `humane-docs` skill for the drafting voice. Content each piece must carry: |
| + - **`scanner-host/README.md`**: what the box is and that it reads nothing itself; the usage block (`module "scanner"` with `ami_id = module.host_ie.ami_id`, `client_security_group_id = module.host_ie.security_group_id`, `region`); the two outputs the console needs and the variables they become (`clamd_addr` to `CARLOS_CLAMD_ADDR`, `stream_max_bytes` to `CARLOS_CLAMD_MAX_BYTES`) plus `CARLOS_SCAN_TAG_KEY`; replacement changes the private IP (spec §3.6), first; the invariant in one paragraph (§3.2); why the two limits are equal and fixed, with the table from §3.4; what a clean verdict does not cover (zip64 and encrypted members); how to check a box is ready (`cloud-init status --wait`, `/var/lib/scanner-host/ready`, `systemctl is-active scanner-selftest.service scanner-tcp.socket`); the four acceptance checks from §3.3 with their commands (Task 7 has them); sizing; that the subnet needs a route out. |
| + - **`infra/modules/README.md`**: a short `## scanner-host` section in the style of the `deployment-buckets` one, with the variable table and outputs. |
| + - **`region-host/README.md`**: `store_scanning` in three sentences: what it grants, that it is off by default, that it goes on the console's box only. |
| + - **`docs/site/cli.md`**: the operator table is stale. Replace the `CARLOS_STORE_SCANNER` and `CARLOS_CLAMD_SOCKET` rows with `CARLOS_CLAMD_ADDR` (`host:port`, or a socket path beginning `/`) and `CARLOS_SCAN_TAG_KEY` (required once an address is set; signs every verdict), say the switch is `carlos system antivirus enable`, and change "six environment variables" to the true count. In the `status` paragraph add one sentence: an archive that expands past the engine's limits is counted as not scanned, the same as an object too large to stream. |
| + - **Rollout plan note**, at the top: `> **2026-10-05:** steps 2 and 4 are superseded by docs/superpowers/specs/2026-10-04-scanner-host-rollout-design.md. clamd runs on its own box, the switch is carlos system antivirus enable, and CARLOS_STORE_SCANNER is deprecated. Steps 5 to 8 and the watch list still stand.` (Internal plan text, not user-facing: no copy review for this one.) |
| + |
| +- [ ] **Step 2: Copy review.** Invoke the `copy-review` skill with every drafted string except the rollout-plan note. Wait for Paul's edits. Apply them verbatim. |
| + |
| +- [ ] **Step 3: Write the files** with the approved text. |
| + |
| +- [ ] **Step 4: Verify the docs gate.** |
| + |
| +Run: `go test ./internal/docsite -count=1` |
| +Expected: `ok`. |
| + |
| +- [ ] **Step 5: Commit.** |
| + |
| +```bash |
| +git add infra/modules/scanner-host/README.md infra/modules/README.md infra/modules/region-host/README.md docs/site/cli.md docs/superpowers/plans/2026-09-01-carloku-scanning-rollout.md |
| +git commit -m "docs: the scanner host, store_scanning, and the scanner's real variables" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" |
| +``` |
| + |
| +--- |
| + |
| +### Task 5: gate, review, discussion, merge |
| + |
| +**Files:** none new. |
| + |
| +- [ ] **Step 1: Full gate.** |
| + |
| +Run: `go build ./... && go vet ./... && go test ./... -count=1` |
| +Expected: all `ok`. If the activator's socket tests fail on path length or `/tmp` fills, use the `-exec` wrapper recorded in memory (`gate-tmp-exec-wrapper`); do not skip packages. Report any failure with its output. |
| + |
| +- [ ] **Step 2: Cross-compile, as CI does.** |
| + |
| +Run: `GOOS=darwin GOARCH=arm64 go build ./... && GOOS=windows GOARCH=amd64 go build ./...` |
| +Expected: no output. |
| + |
| +- [ ] **Step 3: Open the tracking discussion.** Invoke the `amadan` skill, then create a discussion on `carlos/platform` titled `Scanner host: object-storage scanning on Carloku and Tito`, linking the spec, this plan and the branch, and recording the two follow-ups from spec §11 (the zip64 and encrypted-archive disclosure; a configurable scan limit). Note its number; later tasks post progress there. |
| + |
| +- [ ] **Step 4: Astra code review.** The session model is Opus, so the reviewer is Astra via `codex` (memory `codex-second-reader` has the working command; set `model_reasoning_effort="high"` and read the header line). Commit a brief as `docs/superpowers/specs/CODEX-REVIEW-BRIEF-SCANNER-HOST-CODE.md` asking it to review `origin/main..HEAD` against the spec, with special attention to: the systemd dependency design in `files/`, the first-boot script's failure paths, the `parseReply` change, and the IAM statements. Commit findings unedited as `CODEX-FINDINGS-SCANNER-HOST-CODE.md`. Verify every citation before accepting a finding (`receiving-code-review` skill). Fix, re-run the affected tests, and repeat until the verdict is accept. |
| + |
| +- [ ] **Step 5: Push the branch.** `git push origin worktree-scanner-host`, then `amadan ci status carlos/platform` until green. |
| + |
| +- [ ] **Step 6 (GATED): merge.** Show Paul the diffstat and the review verdict. On his go: `amadan branch merge -squash` per the `amadan` skill and memory `carlos-platform-forge`. Record the main SHA in the discussion. |
| + |
| +- [ ] **Step 7 (GATED): move the GitHub mirror.** Only a fast-forward: |
| + |
| +```bash |
| +git fetch https://github.com/carlosframework/platform.git main |
| +git merge-base --is-ancestor FETCH_HEAD origin/main && git push https://github.com/carlosframework/platform.git origin/main:refs/heads/main |
| +``` |
| +Expected: the push succeeds. If the ancestor check fails, stop and tell Paul; never force. |
| + |
| +--- |
| + |
| +### Task 6 (GATED): release v0.23.0 |
| + |
| +Carloku pins modules to release tags, so the modules reach it through a release. The release carries everything on main since v0.22.2, not only this work, so **whether and when to cut it is Paul's call**. Ask before starting. |
| + |
| +- [ ] **Step 1:** List what the release contains: `git log --oneline v0.22.2..origin/main`. Show Paul. Wait for his go. |
| +- [ ] **Step 2:** Follow `README.md` "Releasing the platform" exactly (steps 1 to 9 and the public-side verification), with memory `platform-release-recipe` for the traps. Release notes go through `copy-review`. The operators' section says: object-storage scanning needs a scanner host (`infra/modules/scanner-host`) and `store_scanning` on the console's `region-host`; a console must run this release before it is pointed at a scanner host, because older consoles record an over-limit archive as infected. |
| +- [ ] **Step 3:** Push the tag to the GitHub mirror: `git push https://github.com/carlosframework/platform.git v0.23.0`. |
| +- [ ] **Step 4:** Do **not** roll boxes as part of this plan. The box binary is unaffected; only consoles need the new build (Tasks 8 and 12). If Paul wants the fleet on v0.23.0 too, that is the README's last step and his separate go. |
| + |
| +--- |
| + |
| +### Task 7 (GATED): Carloku, the scanner box |
| + |
| +**Files:** |
| +- `~/github.com/carlosframework/carloku-infrastructure/region-ie.tf` (modify: `host_ie`'s `?ref=`, add `store_scanning = true`) |
| +- `~/github.com/carlosframework/carloku-infrastructure/scanner.tf` (create) |
| +- `~/github.com/carlosframework/carloku-infrastructure/outputs.tf` (modify) |
| + |
| +- [ ] **Step 1: Branch and edit.** In that repo: `git checkout -b scanner-host origin/main`. In `region-ie.tf`, change `host_ie`'s source to `?ref=v0.23.0` (that module block only; the other three regions stay on v0.21.0) and add, after `store_bucket_prefix`: |
| + |
| +```hcl |
| + # Object-storage virus scanning (platform spec 2026-10-04): lets THIS box's |
| + # role read and tag store objects. The console is the only thing that scans. |
| + store_scanning = true |
| +``` |
| + |
| +Create `scanner.tf`: |
| + |
| +```hcl |
| +# The flagship's scanner host: clamd for the console's object-storage virus |
| +# scanning, on its own box so its signatures and CPU stay off the control |
| +# plane. Reachable from the IE host's security group only. The module's README |
| +# has the readiness and acceptance checks; run them before wiring the console. |
| +module "scanner" { |
| + source = "git::https://github.com/carlosframework/platform.git//infra/modules/scanner-host?ref=v0.23.0" |
| + name = "carlos-flagship-scanner" |
| + ami_id = module.host_ie.ami_id |
| + client_security_group_id = module.host_ie.security_group_id |
| + region = "eu-west-1" |
| +} |
| +``` |
| + |
| +Append to `outputs.tf`: |
| + |
| +```hcl |
| +output "scanner_clamd_addr" { |
| + description = "CARLOS_CLAMD_ADDR for the flagship console." |
| + value = module.scanner.clamd_addr |
| +} |
| + |
| +output "scanner_stream_max_bytes" { |
| + description = "CARLOS_CLAMD_MAX_BYTES for the flagship console." |
| + value = module.scanner.stream_max_bytes |
| +} |
| + |
| +output "scanner_instance_id" { |
| + description = "The scanner box, for SSM." |
| + value = module.scanner.instance_id |
| +} |
| +``` |
| + |
| +- [ ] **Step 2: Plan, targeted.** (Sandbox off for the `keymail` profile.) |
| + |
| +```bash |
| +AWS_PROFILE=keymail tofu init -upgrade -input=false |
| +AWS_PROFILE=keymail tofu plan -input=false -out="$TMPDIR/scanner-host/flagship.plan" \ |
| + -target=module.scanner \ |
| + -target='module.host_ie.aws_iam_role_policy.host_store_scan' |
| +``` |
| +Expected: creations only: the scanner's security group, role, policy attachment, `no_parameters` policy, instance profile and instance, plus `host_store_scan[0]`. **No `update in-place`, no `replace`, no `user_data` line on any existing box.** If there is one, stop. |
| + |
| +- [ ] **Step 3: Show Paul the plan** (`tofu show "$TMPDIR/scanner-host/flagship.plan"`), and separately the full untargeted plan's summary line so he sees what the ref bump leaves pending on `host_ie`. Wait for his go. |
| + |
| +- [ ] **Step 4: Apply that plan file.** `AWS_PROFILE=keymail tofu apply "$TMPDIR/scanner-host/flagship.plan"`. Record the three outputs. Commit the branch, push, open a PR on carloku-infrastructure listing what was applied and what the bump leaves unapplied. |
| + |
| +- [ ] **Step 5: Wait for first boot.** Create the SSM helper used from here on: |
| + |
| +```bash |
| +mkdir -p "$TMPDIR/scanner-host" |
| +cat > "$TMPDIR/scanner-host/ssm-run.sh" <<'EOF' |
| +#!/bin/sh |
| +# usage: ssm-run.sh <profile> <region> <instance-id> '<command>' |
| +# Runs one shell command on a box and prints its stdout, stderr and status. |
| +set -eu |
| +p=$1; r=$2; i=$3; c=$4 |
| +id=$(aws --profile "$p" --region "$r" ssm send-command --instance-ids "$i" \ |
| + --document-name AWS-RunShellScript --parameters "$(jq -nc --arg c "$c" '{commands:[$c]}')" \ |
| + --query Command.CommandId --output text) |
| +aws --profile "$p" --region "$r" ssm wait command-executed --command-id "$id" --instance-id "$i" 2>/dev/null || true |
| +aws --profile "$p" --region "$r" ssm get-command-invocation --command-id "$id" --instance-id "$i" \ |
| + --query '[Status,StandardOutputContent,StandardErrorContent]' --output text |
| +EOF |
| +chmod +x "$TMPDIR/scanner-host/ssm-run.sh" |
| +``` |
| + |
| +Then poll the serial console until it says one or the other (first boot takes several minutes: packages, about 300 MB of signatures, the self-test): |
| + |
| +Run: `aws --profile keymail --region eu-west-1 ec2 get-console-output --instance-id <scanner-id> --latest --output text | grep 'SCANNER-HOST'` |
| +Expected: `SCANNER-HOST READY ClamAV 1.4.x/<sig>/<date>`, with the date today or yesterday. On `SCANNER-HOST FAILED: <reason>`, read the `scanner-init:` lines above it and fix the module (back to Task 3), not the box. |
| + |
| +- [ ] **Step 6: Readiness.** |
| + |
| +Run: `"$TMPDIR/scanner-host/ssm-run.sh" keymail eu-west-1 <scanner-id> 'cloud-init status --wait; cat /var/lib/scanner-host/ready; systemctl is-active scanner-selftest.service scanner-tcp.socket; journalctl -u scanner-selftest.service --no-pager | tail -14; aa-status 2>/dev/null | grep -c /usr/sbin/clamd; journalctl -k --no-pager | grep -c "apparmor=\"DENIED\".*clamd"'` |
| +Expected: `status: done`; the VERSION line; `active` twice; ten `pass` lines and `scanner-selftest: PASS`; then `1` or more (clamd's AppArmor profile is loaded) and `0` (it denied clamd nothing, so the `/var/tmp` local include took). |
| + |
| +Then prove a package reconfigure leaves `clamd.conf` alone (debconf is frozen off; with it on, the postinst would rewrite `StreamMaxLength`): |
| + |
| +Run: `... ssm-run.sh keymail eu-west-1 <scanner-id> 'debconf-show clamav-daemon | grep "clamav-daemon/debconf"; dpkg-reconfigure -fnoninteractive clamav-daemon; grep -E "^(StreamMaxLength|MaxFileSize|MaxScanSize|AlertExceedsMax|TemporaryDirectory) " /etc/clamav/clamd.conf; systemctl restart clamav-daemon.service; for i in $(seq 1 60); do [ "$(systemctl is-active scanner-tcp.socket)" = active ] && break; sleep 10; done; echo "after reconfigure: socket $(systemctl is-active scanner-tcp.socket), selftest $(systemctl is-active scanner-selftest.service), $(ss -ltnH "sport = :3310" | wc -l) listeners"'` |
| +Expected: a line ending `clamav-daemon/debconf: false`; these five lines, in any order: `StreamMaxLength 25M`, `AlertExceedsMax yes`, `MaxFileSize 100M`, `MaxScanSize 100M`, `TemporaryDirectory /var/tmp` (init appends them in that order, so expect that one, but any order is a pass); then `after reconfigure: socket active, selftest active, 1 listeners`. The reconfigure moves Debian's `extend.conf` drop-in aside (postinst, debconf off), so the restart also proves clamd starts on the module's own drop-in. Anything else: stop, fix in Task 3, replace the box. |
| + |
| +- [ ] **Step 7: Acceptance check, kill.** The port follows the test. |
| + |
| +Run: `... ssm-run.sh keymail eu-west-1 <scanner-id> 'systemctl kill -s KILL clamav-daemon.service; sleep 3; echo "after kill: $(ss -ltnH "sport = :3310" | wc -l) listeners, selftest $(systemctl is-active scanner-selftest.service)"; for i in $(seq 1 60); do [ "$(systemctl is-active scanner-tcp.socket)" = active ] && break; sleep 10; done; echo "recovered: socket $(systemctl is-active scanner-tcp.socket), selftest $(systemctl is-active scanner-selftest.service), $(ss -ltnH "sport = :3310" | wc -l) listeners"'` |
| +Expected: `after kill: 0 listeners, selftest` anything but `active`; then `recovered: socket active, selftest active, 1 listeners`. If the port is still open right after the kill, or never reopens, the dependency design is wrong on this systemd: stop, fix in Task 3, replace the box. |
| + |
| +- [ ] **Step 8: Acceptance check, forced fail.** A failed test keeps the port shut. |
| + |
| +Run, in one command: write `/etc/systemd/system/scanner-selftest.service.d/99-force-fail.conf` containing `[Service]` and `Environment=SCANNER_SELFTEST_FORCE_FAIL=1`; `systemctl daemon-reload`; `systemctl restart clamav-daemon.service`; wait 90 seconds; print `systemctl is-active clamav-daemon.service scanner-selftest.service scanner-tcp.socket` and the listener count; then remove the drop-in, `daemon-reload`, `systemctl restart clamav-daemon.service`, wait until `scanner-tcp.socket` is active (up to 10 minutes), and print the same again. |
| +Expected first print: `active`, `failed`, `inactive` (or `failed`), `0` listeners. Second: `active` three times, `1` listener. |
| + |
| +- [ ] **Step 9: Acceptance check, the protocol through the proxy, from the console box.** |
| + |
| +Run: `... ssm-run.sh keymail eu-west-1 i-092c0c1eea75723cb 'exec 3<>/dev/tcp/<scanner-private-ip>/3310; printf "zINSTREAM\0\0\0\0\104X5O!P%%@AP[4\\PZX54(P^)7CC)7}\$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!\$H+H*\0\0\0\0" >&3; timeout 20 head -c 200 <&3 | tr -d "\0"; echo'` |
| +(`\0\0\0\104` is the 68-byte chunk length, octal 104 = 68; the four zero bytes after the payload end the stream.) |
| +Expected: `stream: Eicar-Test-Signature FOUND`. If the nested quoting fights you, send the same bytes with a three-line `python3` socket script instead; the bytes are what matter. Also confirm the fence from somewhere that is not the console box: the same connect attempt from another fleet box times out. |
| + |
| +- [ ] **Step 10: Acceptance check, reboot.** |
| + |
| +Run: `aws --profile keymail --region eu-west-1 ec2 reboot-instances --instance-ids <scanner-id>`, wait five minutes, then `... ssm-run.sh keymail eu-west-1 <scanner-id> 'systemctl is-active clamav-daemon.service scanner-selftest.service scanner-tcp.socket; ss -ltnH "sport = :3310" | wc -l; journalctl -b --no-pager | grep -ci "ordering cycle"; systemd-analyze verify /etc/systemd/system/scanner-selftest.service /etc/systemd/system/scanner-tcp.socket /etc/systemd/system/scanner-tcp.service 2>&1 | grep -v "^$" | head'` |
| +Expected: `active` three times, `1`, `0`, and nothing from `systemd-analyze verify` about these three units. |
| + |
| +- [ ] **Step 11: Record** all four results in the discussion, with the VERSION line. |
| + |
| +--- |
| + |
| +### Task 8 (GATED): Carloku, the console on the new build |
| + |
| +- [ ] **Step 1:** A clean checkout of the `v0.23.0` tag (scratch clone under `~/.claude/tmp/carlos-build/`, as memory `platform-roll-recipe` describes). `git status --porcelain` must print nothing. |
| +- [ ] **Step 2:** Show Paul the command. On his go, from `carloku-infrastructure`: `GOFLAGS=-buildvcs=false scripts/roll-console.sh <checkout> flagship`. |
| +Expected: it builds arm64, ships, promotes, adopts, restarts, and its own check prints the running executable as `.../versions/<sha>/console` for the SHA it built. **If that check fails, stop: nothing gets wired.** |
| +- [ ] **Step 3:** `carlos system antivirus status --console https://console.carloku.com` |
| +Expected: it answers, and says no engine is wired. (Wiring is Task 9.) |
| + |
| +--- |
| + |
| +### Task 9 (GATED): Carloku, key and wiring |
| + |
| +- [ ] **Step 1: Preconditions, both.** Task 7's readiness and four acceptance checks are recorded as passed. Task 8's executable check passed for the build with the Task 1 change: `git merge-base --is-ancestor <task-1-commit-on-main> v0.23.0` exits 0. |
| + |
| +- [ ] **Step 2: Find the console's environment file.** |
| + |
| +Run: `... ssm-run.sh keymail eu-west-1 i-092c0c1eea75723cb 'systemctl cat console.service | grep -i EnvironmentFile; systemctl show -p User --value console.service'` |
| +Expected: one `EnvironmentFile=` path (written `<envfile>` below) and the unit's user. |
| + |
| +- [ ] **Step 3: Generate the key straight into SSM.** It never touches argv, a file or the terminal: |
| + |
| +```bash |
| +openssl rand -hex 32 \ |
| + | jq -Rc '{Name:"/carlos/scan-tag-key",Type:"SecureString",Value:.,Overwrite:false,Description:"Signs every object-storage scan verdict. Losing it makes every stored verdict unverifiable."}' \ |
| + | aws --profile keymail --region eu-west-1 ssm put-parameter --cli-input-json file:///dev/stdin |
| +``` |
| +Expected: `"Version": 1`. `Overwrite:false` means a rerun fails with `ParameterAlreadyExists` instead of replacing the key; that failure is correct, leave the key alone. |
| + |
| +- [ ] **Step 4: Show Paul** the three variables (the key as a name only), the file, and that the next step restarts the console for a few seconds. Wait for his go. |
| + |
| +- [ ] **Step 5: Write the variables on the box.** The box fetches the key itself, so it never crosses the SSM command channel: |
| + |
| +```sh |
| +set -eu |
| +f=<envfile> |
| +cp -p "$f" "$f.bak-scan-$(date +%Y%m%d)" |
| +key=$(aws ssm get-parameter --region eu-west-1 --name /carlos/scan-tag-key --with-decryption --query Parameter.Value --output text) |
| +[ ${#key} -eq 64 ] || { echo "key is not 64 hex chars"; exit 1; } |
| +grep -v '^CARLOS_CLAMD_ADDR=\|^CARLOS_CLAMD_MAX_BYTES=\|^CARLOS_SCAN_TAG_KEY=\|^CARLOS_CLAMD_SOCKET=' "$f" > "$f.new" |
| +printf 'CARLOS_CLAMD_ADDR=%s\nCARLOS_CLAMD_MAX_BYTES=%s\nCARLOS_SCAN_TAG_KEY=%s\n' '<scanner_clamd_addr>' '<scanner_stream_max_bytes>' "$key" >> "$f.new" |
| +chmod --reference="$f" "$f.new"; chown --reference="$f" "$f.new"; mv "$f.new" "$f" |
| +grep -c '^CARLOS_CLAMD_ADDR=\|^CARLOS_CLAMD_MAX_BYTES=\|^CARLOS_SCAN_TAG_KEY=' "$f" |
| +grep -c '^CARLOS_STORE_SCANNER=' "$f" || true |
| +``` |
| +Expected: `3`, then `0`. Nothing prints the key. |
| + |
| +- [ ] **Step 6: Restart and check.** |
| + |
| +Run: `... ssm-run.sh keymail eu-west-1 i-092c0c1eea75723cb 'systemctl restart console; sleep 8; systemctl is-active console; readlink /proc/$(systemctl show -p MainPID --value console)/exe; journalctl -u console --since "-1min" --no-pager | grep -i "clamd\|scan\|fatal" | tail -5'` |
| +Expected: `active`; the same `versions/<sha>/console`; no `fatal` line. |
| +**If it is not active:** `cp -p <envfile>.bak-scan-<date> <envfile> && systemctl reset-failed console && systemctl restart console`, confirm `active`, stop, and tell Paul with the journal lines. |
| + |
| +- [ ] **Step 7: Status, then enable.** |
| + |
| +Run: `carlos system antivirus status --console https://console.carloku.com` |
| +Expected: engine wired, answering, signature date today or yesterday, policy off. |
| +Show Paul. On his go: `carlos system antivirus enable --console https://console.carloku.com`, then `status` again. |
| +Expected: policy on. No app is granted, so the hourly pass does nothing. |
| + |
| +--- |
| + |
| +### Task 10 (GATED): Carloku, the proof |
| + |
| +On a throwaway app Paul's account owns. Ask Paul which account to create it under, and confirm before creating and before destroying. |
| + |
| +- [ ] **Step 1: Make the app and its store.** Create an app (name `scan-proof-<date>`), then `carlos store create --app <app>` and `carlos store status --app <app>` to learn the bucket name. |
| +- [ ] **Step 2: Build the fixtures locally** by running the fixture half of `infra/modules/scanner-host/files/scanner-selftest` (the `python3` heredoc) into `$TMPDIR/scanner-host/fx`. Upload five with the operator profile: `01-clean.txt`, `02-eicar.txt`, `03-zip-one-500.zip`, `08-zip-500-then-eicar.zip`, `09-zip-60-then-eicar.zip`: |
| +`aws --profile keymail s3 cp "$TMPDIR/scanner-host/fx/<file>" s3://<bucket>/proof/<file>` |
| +- [ ] **Step 3: Request, grant, sweep.** |
| + |
| +``` |
| +carlos store scan request --app <app> --plaintext |
| +carlos store scan grant --app <app> |
| +carlos store scan sweep --app <app> |
| +carlos store scan status --app <app> |
| +``` |
| +Expected (spec §7 steps 1 and 2): a completed sweep; 5 objects; `01` clean; `02` and `09` infected; `03` and `08` skipped and counted unscanned. **If `03` or `08` reads clean, stop the rollout:** `carlos system antivirus disable`, and tell Paul. |
| + |
| +- [ ] **Step 4: The tags.** For `01` and `03`: `aws --profile keymail s3api get-object-tagging --bucket <bucket> --key proof/<file>` and `aws --profile keymail s3api head-object --bucket <bucket> --key proof/<file> --query ETag`. |
| +Expected: `carlos-scan`, `carlos-scan-version`, `carlos-scan-at` present; the version tag carries the object's current ETag; `01` says clean, `03` says skipped. |
| + |
| +- [ ] **Step 5: A rewrite invalidates the verdict.** Overwrite `proof/01-clean.txt` with different bytes, `sweep`, `status`. |
| +Expected: the fresh count goes up by one; it is re-scanned, not carried forward. **If carried forward, stop the rollout.** |
| + |
| +- [ ] **Step 6: Enforce survives the engine going away.** `carlos store scan enforce --app <app> on`. Then, with Paul's go (two more console restarts): on the box, back up `<envfile>`, remove the `CARLOS_CLAMD_ADDR` line, restart; run `carlos store scan enforce --app <app> off`. |
| +Expected: it succeeds with no engine built. Restore the line from the backup, restart, `antivirus status` shows the engine wired again. |
| + |
| +- [ ] **Step 7: Teardown.** Destroy the app. |
| +Expected: `aws --profile keymail s3 ls s3://carlos-flagship-271376211898/apps/ --recursive | grep '<app>.*scan.json'` prints nothing, and the app name can be created again at once (create it, then destroy it). |
| + |
| +- [ ] **Step 8:** Record every result in the discussion. Scanning stays enabled on Carloku with no member granted. |
| + |
| +--- |
| + |
| +### Task 11 (GATED): Tito, the scanner box and the grant |
| + |
| +Do not start until Task 10 has passed and #201 phase 2 has landed (check memory `tito-console-store-boundary` and the teamtito repo). Tito is a third party: every step here is shown to Paul first. |
| + |
| +**Files:** `~/github.com/teamtito/carlos-infrastructure/object-store.tf` (modify), `scanner.tf` (create), `outputs.tf` (modify) |
| + |
| +- [ ] **Step 1: Branch from the line that can plan.** `git fetch origin`, then branch `scanner-host` from the tip the last targeted apply ran from (`store-boundary-201`, or whatever has superseded it; `main` cannot plan while the Canada box is only on `region-ca`). Say which tip in the PR. |
| + |
| +- [ ] **Step 2: Edit.** In `object-store.tf`, add to `host_ie_object_store`'s `Statement` list: |
| + |
| +```hcl |
| + { |
| + # Object-storage virus scanning (platform spec 2026-10-04): the console |
| + # reads each object to stream it to the scanner host, and writes the |
| + # verdict as tags. Inert until a member requests scanning for a store |
| + # and an operator grants it. |
| + Sid = "StoreScan" |
| + Effect = "Allow" |
| + Action = ["s3:GetObject", "s3:GetObjectTagging", "s3:PutObjectTagging"] |
| + Resource = [ |
| + "arn:aws:s3:::carlos-store-tito-*/*", |
| + "arn:aws:s3:::titogo-uploads-665106592142/*", |
| + ] |
| + }, |
| +``` |
| + |
| +Create `scanner.tf` as in Task 7 Step 1 with `name = "carlos-tito-scanner"`, and the three outputs. |
| + |
| +- [ ] **Step 3: Check the box can read its key.** In `region-ie.tf`, read `host_ie`'s `ssm_parameter_arn`. If `/carlos/scan-tag-key` falls outside it, the console box could not fetch the key in Task 12; add that one parameter ARN in the smallest way that file allows and include the resource in the targeted plan. Say so to Paul. |
| + |
| +- [ ] **Step 4: Plan, targeted.** |
| + |
| +```bash |
| +AWS_PROFILE=titogo tofu init -upgrade -input=false |
| +AWS_PROFILE=titogo tofu plan -input=false -out="$TMPDIR/scanner-host/tito.plan" \ |
| + -target=module.scanner -target=aws_iam_role_policy.host_ie_object_store |
| +``` |
| +Expected: the scanner's six resources created; `host_ie_object_store` updated in place with exactly the `StoreScan` statement added (read the policy diff: any other difference is drift from #201, stop and ask). No change to any instance. |
| + |
| +- [ ] **Step 5:** Show Paul the plan. On his go, apply the plan file, push the branch, open the PR. |
| +- [ ] **Step 6:** Task 7 Steps 5 to 11 on Tito's scanner box (profile `titogo`, console box `i-025a0b70f3d93304d`): first boot, readiness, and all four acceptance checks. |
| + |
| +--- |
| + |
| +### Task 12 (GATED): Tito, console, key, wiring, enable |
| + |
| +- [ ] **Step 1:** Roll Tito's console to the same build as Task 8: `GOFLAGS=-buildvcs=false scripts/roll-console.sh <checkout> tito`, with Paul's go. Its executable check must pass. Before rolling, diff Tito's console environment against the flagship's for variables the new build requires (the 2026-09-11 lesson): `... ssm-run.sh` on each console box, `cut -d= -f1 <envfile> | sort`, and compare. |
| +- [ ] **Step 2:** Task 9 Steps 1 to 7 in Tito's account (profile `titogo`, region `eu-west-1`, `/carlos/scan-tag-key`, console `https://carlos.tito.io`). A **new** key: never copy the flagship's. |
| +- [ ] **Step 3:** Stop at `antivirus status` (wired, answering, fresh signatures) and `antivirus enable`. **No throwaway app, no grant, no sweep on Tito** (spec §8). |
| +- [ ] **Step 4:** Record in the discussion. |
| + |
| +--- |
| + |
| +### Task 13: close out |
| + |
| +- [ ] **Step 1:** Post the final state to the discussion: both deployments enabled, no store granted, the two follow-ups open. |
| +- [ ] **Step 2:** Update memory: a `scanner-host-rollout` entry (what is live, box ids, where the keys are, the replacement rule, the follow-ups) and a pointer line in `MEMORY.md`; correct `tito-console-store-boundary` if Tito's policy changed. |
| +- [ ] **Step 3:** Delete `$TMPDIR/scanner-host/` and the scratch clone if this plan made it. Remove `.scratch-codex-review.sh` from the worktree. Leave nothing under `/tmp`. |
| +- [ ] **Step 4:** Write the handoff (`handoff` skill). |
diff --git a/docs/superpowers/specs/2026-10-04-scanner-host-rollout-design.md b/docs/superpowers/specs/2026-10-04-scanner-host-rollout-design.md| new file mode 100644 |
| index 000000000..0739dc3fd |
| --- /dev/null |
| +++ b/docs/superpowers/specs/2026-10-04-scanner-host-rollout-design.md |
| @@ -0,0 +1,489 @@ |
| +# Scanner host: getting object-storage scanning running on Carloku and Tito |
| + |
| +**Status:** draft 6. Closes Astra review 1 (`CODEX-FINDINGS-SCANNER-HOST.md`, |
| +findings 1–5, all accepted), review 2 (`CODEX-FINDINGS-SCANNER-HOST-R2.md`, |
| +findings 6–8: 7 and 8 accepted, 6 declined with the reason in §4.3) and |
| +review 3 (`CODEX-FINDINGS-SCANNER-HOST-R3.md`, finding 9, accepted and tested |
| +against a real engine, §3.4) and review 4 (`CODEX-FINDINGS-SCANNER-HOST-R4.md`, |
| +findings 10 and 11, both accepted; 10 reshaped how the box listens, §3.2) and |
| +review 5 (`CODEX-FINDINGS-SCANNER-HOST-R5.md`, finding 12, accepted). Review 5 |
| +found the gating design sound at spec level once 12 is fixed, and left restart |
| +timing, connection cut-off and EICAR transport to the real-box acceptance |
| +checks in §3.3. Accepted findings are marked **(codex N)**. |
| +**Builds on:** `2026-09-01-object-storage-virus-scanning-design.md` (the sweep, |
| +verdict tags, enforcement) and `2026-09-02-scan-on-demand-design.md` §9 (clamd |
| +off the console box, reachable over TCP). |
| +**Supersedes:** steps 2 and 4 of `../plans/2026-09-01-carloku-scanning-rollout.md` |
| +(clamd on the console box, `CARLOS_STORE_SCANNER` as the switch). Its steps 5–8 |
| +and its watch list still stand. |
| + |
| +--- |
| + |
| +## 1. Where things stand |
| + |
| +The code is shipped and live. Both consoles run builds that carry the sweep, |
| +`carlos store scan request|status|grant|revoke|sweep|enforce` and |
| +`carlos system antivirus status|enable|disable` (Carloku `7edd4b33`, Tito |
| +`fe2088e3`). `CARLOS_CLAMD_ADDR` already takes `host:port` as well as a socket |
| +path (`2eaf2aae`, in every release since v0.18.0). |
| + |
| +What neither deployment has is a clamd to talk to, permission for the console |
| +to read and tag store objects, or a scan tag key. This spec supplies those three |
| +things, plus one small client change (§4.3) that stops a partly scanned archive |
| +being signed clean. **(codex 4)** |
| + |
| +## 2. Decisions |
| + |
| +| # | Decision | Why | |
| +|---|---|---| |
| +| D1 | clamd runs on its own box, not the console box | Its ~1.5 GB of resident signatures (about 3 GB during a reload) leave the control plane. Carloku's console box is a `t4g.large` that also serves IE apps. | |
| +| D2 | One box per deployment, in the home region only | Only the console talks to clamd, and every store bucket is in the one region `CARLOS_STORE_REGION` names (`cmd/console/main.go:546-585`). A box elsewhere would add cross-region transfer, not remove it. | |
| +| D3 | Console to clamd is plain TCP inside the VPC, closed by security group | clamd has no TLS and no authentication. Same-VPC, single-source security group is the same trust model as the console's own S3 traffic. TLS (stunnel or a dialler change) or Tailscale are later options, not v1. **(Paul, 2026-10-04: "A is fine for now")** | |
| +| D4 | The console keeps fetching from S3 and streams to clamd | Scan-on-demand §9.4: tenant-data credentials stay on one box. The scanner box gets no S3 or parameter grants. | |
| +| D5 | Both pieces are platform modules | Carloku consumes the platform the way a self-hoster does. Hand-written root resources would be a workaround and would duplicate clamd's setup per root. | |
| +| D6 | Carloku first, then Tito | Tito is a third party, and its console was only unpinned on 2026-10-03 with #201 phase 2 pending. Prove it on our own flagship first. | |
| +| D7 | Scope ends at "engine wired, switched on, no member granted" | Rollout plan steps 7–8: a member grant follows a member's request and the disclosure, never an operator's speculation. | |
| +| D8 | An object clamd only partly scanned, with no malware signature matched, is recorded **unscanned**, never clean, and never infected on the strength of the limit alone | A signed clean verdict over bytes nobody read is the failure the verdict design exists to prevent; flagging large legitimate archives infected would make enforce block them. Unscanned fits the existing audit arithmetic (`unscanned = skipped + denied + failed`). A real signature match is positive evidence and stays **infected** even when the scan was partial (§4.3). **(Paul, 2026-10-04, on codex 4)** | |
| + |
| +**Out of scope:** the unmerged scan-on-demand API (`worktree-store-scan-on-demand`), |
| +TLS to clamd, alerting on signature age, GuardDuty, per-region scanning. If |
| +store buckets ever become per-region, scanning would carry tenant bytes out of |
| +their region; the answer then is a regional scanner that fetches for itself, |
| +which reopens D4. That is a new decision, not more boxes. |
| + |
| +## 3. The `scanner-host` module (`infra/modules/scanner-host`) |
| + |
| +### 3.1 Resources |
| + |
| +- **`aws_instance`**: `ami_id` from the caller (the caller passes |
| + `module.<host>.ami_id`, as Tito's `svc-pdf.tf` does), `instance_type` default |
| + `t4g.medium`, 20 GiB encrypted gp3 root, IMDSv2 required |
| + (`http_tokens = "required"`), `user_data_replace_on_change = false`, |
| + `lifecycle { ignore_changes = [ami, user_data] }`, so a newer image or an |
| + edited script never silently replaces the box; changing either (including |
| + `stream_max_bytes`) is an explicit `-replace`. Optional `subnet_id`; null |
| + means the default subnet, as `region-host` does, and a precondition then |
| + requires the client security group's VPC to be the default VPC. |
| +- **`aws_security_group`**: ingress TCP 3310 from `client_security_group_id` |
| + **only**. No other ingress, no SSH. Egress open, for `freshclam` and apt. It |
| + lands in the client security group's VPC (`vpc_id` from a data source on that |
| + group), so the two can never end up in different VPCs. |
| +- **IAM role and instance profile**: `AmazonSSMManagedInstanceCore` attached, |
| + for Session Manager. That managed policy grants `ssm:GetParameter*` on `*`, |
| + which on its own would let the box read `/carlos/scan-tag-key` and |
| + `/carlos/release-key`. So the role also carries an explicit Deny on |
| + `ssm:GetParameter`, `GetParameters`, `GetParameterHistory` and |
| + `GetParametersByPath` on `*`, the same shape as `region-host`'s |
| + `host_ssm_parameter_scope` (`main.tf:418-450`) with no allowed parameter at |
| + all. Session Manager's channel actions are untouched. No bucket, no KMS. The |
| + box cannot fetch tenant data or the key even if clamd were compromised. |
| + **(codex 1)** |
| + |
| +### 3.2 How the box listens: the invariant |
| + |
| +**TCP 3310 is open only while a self-test has passed against the clamd process |
| +that is running now.** Not "the boot script checked once": a failed test, a |
| +clamd restart, or a ClamAV package upgrade closes the port until the test |
| +passes again. The console's own checks cannot stand in for this, because |
| +`antivirus status` and `enable` only ask clamd for `VERSION` |
| +(`internal/objstore/clamav.go:108`, `internal/console/apisystem.go:85,133`), |
| +and an engine that signs truncated scans clean answers `VERSION` perfectly |
| +well. **(codex 10)** |
| + |
| +So clamd itself never listens on TCP. Three units: |
| + |
| +- **`clamav-daemon`**, Debian's own unit and socket activation, **untouched** |
| + except a drop-in for `MemoryMax` (variable, default `3G`), `MemoryHigh` at |
| + about 85% of it, and `Restart=on-failure`. It listens on the unix socket |
| + `/run/clamav/clamd.ctl` only; `clamd.conf` has no `TCPSocket` and no |
| + `TCPAddr`. That is why the earlier replacement unit is gone: socket |
| + activation handing clamd only the unix socket is now exactly what is wanted. |
| + (Finding 2's problem no longer arises.) A drop-in `Wants=scanner-tcp.socket` |
| + makes every start of clamd, including an automatic restart, ask for the |
| + gate below to be re-run. |
| +- **`scanner-selftest.service`**, `Type=oneshot`, `RemainAfterExit=yes`, |
| + `BindsTo=clamav-daemon.service`, `After=clamav-daemon.service`. Runs |
| + `/usr/local/sbin/scanner-selftest` against the unix socket. Being bound to |
| + clamd, it goes inactive the moment clamd stops or restarts. |
| +- **`scanner-tcp.socket`** and **`scanner-tcp.service`**: the socket listens on |
| + `<private IPv4>:3310` (from IMDSv2, written into the unit at first boot; |
| + never `0.0.0.0`) and hands connections to `systemd-socket-proxyd |
| + /run/clamav/clamd.ctl`, running as a dedicated user in group `clamav`. Both |
| + carry `Requires=scanner-selftest.service`, `After=` it and `BindsTo=` it. So |
| + the port cannot open before the test passes, and it closes, cutting |
| + connections in flight, when the test's unit goes inactive. |
| + |
| + The socket unit sets **`DefaultDependencies=no`**. A socket unit is ordered |
| + `Before=sockets.target` by default, `sockets.target` comes before |
| + `basic.target`, and every service (the self-test included) comes after |
| + `basic.target`; a socket that must also come *after* the self-test closes |
| + that loop, and systemd breaks an ordering cycle by dropping a job, which |
| + could leave the port shut after a reboot. So the unit opts out, restores what |
| + it still needs by hand (`After=sysinit.target`, `Conflicts=shutdown.target`, |
| + `Before=shutdown.target`), and is started by `WantedBy=multi-user.target` and |
| + clamd's `Wants=`, never by `sockets.target`. **(codex 12)** |
| + |
| +`clamd.conf` keys, each rewritten idempotently (clamd has no include |
| +directory): |
| + |
| + - `StreamMaxLength <stream_max_bytes / 1 MiB>M`. Written in whole MiB, not |
| + bytes: Debian's debconf round trip strips an `M` and appends one, so a bare |
| + byte count would become a 25 TiB limit on the first configure (final |
| + review I1). `clamav-daemon/debconf` is also set false after install, so |
| + later upgrades leave `clamd.conf` alone. |
| + - `AlertExceedsMax yes`. With the default `no`, an archive that expands past |
| + `MaxScanSize`, `MaxFileSize`, `MaxFiles` or `MaxRecursion` is partly |
| + scanned and answered `OK`. With `yes` clamd answers |
| + `Heuristics.Limits.Exceeded.<limit> FOUND`, which §4.3 records as |
| + unscanned (D8). **(codex 4)** |
| + - `MaxFileSize 100M` and `MaxScanSize 100M`: **the same value, and fixed in |
| + this release.** `AlertExceedsMax` alone is not enough. ClamAV's zip |
| + extractor trims a member that expands past `MaxFileSize` and scans the |
| + trimmed copy without raising any limit (`libclamav/unzip.c:222-226`, |
| + 1.4.3). With the two limits equal, the trimmed copy is as large as the |
| + whole scan budget, so scanning it always trips `MaxScanSize`, which does |
| + alert. Tested, §3.4. **(codex 9)** It is a constant, not a variable, |
| + because the self-test's fixtures and expected answers are sized for |
| + 100 MiB; a configurable limit means deriving both from it, which is a |
| + follow-up (§11). **(codex 11)** |
| + - `ConcurrentDatabaseReload` from a variable, default `yes`. |
| + |
| +### 3.3 `user_data` |
| + |
| +A templated script (`scanner-init.sh.tftpl`), `set -eu`, each step logged to |
| +the serial console so `aws ec2 get-console-output` shows how far it got. |
| + |
| +1. **SSM agent first.** The AMI is the base Debian image `region-host` boots |
| + from (`outputs.tf:26`), not a configured host, and Debian ships no SSM agent. |
| + Install the arm64 `amazon-ssm-agent` `.deb` from the regional bucket and |
| + enable it, exactly as `host-init.sh.tftpl` does. With SSH closed this is the |
| + only way onto the box, so it goes before anything that can fail. A |
| + `region` variable feeds the URL. Also `unattended-upgrades`, as region hosts |
| + have. **(codex 5)** |
| +2. **Install without starting.** Write a `policy-rc.d` that exits 101, then |
| + `apt-get install clamav-daemon clamav-freshclam clamdscan python3`, then |
| + remove the `policy-rc.d`. Debian's postinst would otherwise start |
| + `clamav-freshclam` (`clamav-freshclam.postinst.in:333`) and the daemon |
| + before they are configured. **(codex 3)** |
| +3. **Write `clamd.conf`'s keys, the drop-in, the self-test script and the |
| + three units** of §3.2. `systemctl daemon-reload`. |
| +4. **First signatures.** `freshclam` once in the foreground, with no |
| + `clamav-freshclam` service running (step 2 kept it stopped), so the two never |
| + contend for the lock or the update log. Refuse to continue unless `main`, |
| + `daily` and `bytecode` (`.cvd` or `.cld`) exist. Then enable and start |
| + `clamav-freshclam` (hourly checks by default). **(codex 3)** |
| +5. **Enable and start** `clamav-daemon`, `scanner-selftest.service` and |
| + `scanner-tcp.socket`. The ordering in §3.2 does the rest: clamd loads, the |
| + self-test runs, and only on a pass does the port open. |
| +6. **Verify, and say so.** `systemctl is-active scanner-selftest.service |
| + scanner-tcp.socket` both `active`; `zVERSION` answers on |
| + `<private ip>:3310`; `ss -ltn` shows 3310 bound to the private IP only. |
| + Write `/var/lib/scanner-host/ready` and print `SCANNER-HOST READY` plus the |
| + `zVERSION` line. Any failure prints `SCANNER-HOST FAILED: <which check>` and |
| + exits non-zero, with the port closed by construction. |
| + |
| +**The self-test** (`/usr/local/sbin/scanner-selftest`) builds the §3.4 |
| +fixtures with `python3` in a temp directory (members written from a temp file, |
| +so their zip headers are ordinary rather than zip64, which clamd would skip), |
| +streams each to the unix socket with `clamdscan --stream`, and requires the |
| +class of reply in the table's last column. **An over-limit fixture answering |
| +`OK` is a failure.** It re-proves the behaviour on whatever ClamAV version the |
| +image or a later upgrade installs, rather than trusting what 1.4.3 did. |
| + |
| +**Acceptance checks for the module**, run on the first real box before any |
| +console is wired (§6.2), and recorded in the README: |
| + |
| +- kill clamd: 3310 closes at once, and reopens only after the self-test has |
| + re-run and passed; |
| +- force the self-test to fail (an environment override in a drop-in, used only |
| + for this check): clamd comes up, 3310 **stays closed**, the unit shows |
| + failed; remove the override and restart: it opens; |
| +- an INSTREAM scan of EICAR through `<private ip>:3310` from the console box |
| + answers `Eicar-Test-Signature FOUND`, so the proxy carries the protocol; |
| +- **reboot the box**: it comes back with the self-test passed and 3310 open, |
| + `systemd-analyze verify` on the three units is clean, and `journalctl -b` |
| + has no `ordering cycle` line. First boot cannot show this, because |
| + `basic.target` and `sockets.target` are already reached by the time |
| + `user_data` runs. **(codex 12)** |
| + |
| +### 3.4 What the limits do, measured |
| + |
| +Run 2026-10-05 against Debian trixie's clamd (ClamAV 1.4.3), `StreamMaxLength |
| +25M`, `AlertExceedsMax yes`, every fixture streamed with `clamdscan --stream`. |
| +Members are a repeating 16 KiB random block, distinct per member, so the clean |
| +cache cannot short-circuit them. "Limit" means a |
| +`Heuristics.Limits.Exceeded.*` reply. |
| + |
| +| Fixture | `MaxFileSize 25M`, `MaxScanSize 100M` | Both 100M (this spec) | Self-test requires | |
| +|---|---|---|---| |
| +| small clean file | OK | OK | OK | |
| +| EICAR | infected | infected | infected | |
| +| zip, one member expanding to 500 MiB | **OK** (trimmed, silent) | Limit | Limit | |
| +| zip, a zip inside holding that member | **OK** (silent) | Limit | Limit | |
| +| zip, 30 distinct 20 MiB members | Limit | Limit | Limit | |
| +| gzip of 500 MiB | Limit | Limit | Limit | |
| +| tar.gz, 5 × 40 MiB | Limit | Limit | Limit | |
| +| bzip2, xz, tar.xz of the same | Limit | Limit | not in self-test | |
| +| zip, 500 MiB member then EICAR | infected | Limit | Limit | |
| +| zip, 60 MiB member then EICAR | infected | infected | infected | |
| +| zip, one 60 MiB member | **OK** (trimmed, silent) | OK (fully scanned) | OK | |
| + |
| +The first column is finding 9, observed: three fixtures signed clean over |
| +content nobody read. The second is what this spec configures. |
| + |
| +**Known and not fixed: zip64 local headers.** A zip member written with zip64 |
| +local headers is not extracted at all by 1.4.3 (`local header - stream out of |
| +file`), and EICAR inside one reads `OK` under every config. That is parser |
| +coverage, not a limit, and no clamd setting changes it. Encrypted archives are |
| +the same class. So a clean verdict means "clamd found nothing in what it could |
| +open", never "this object is safe", and the disclosure a member reads before |
| +opting in (rollout plan step 7) should say so in those words. Recorded in §11 |
| +as a follow-up; it does not block this rollout, because it is true of the |
| +engine wherever it runs. |
| + |
| +### 3.5 Variables and outputs |
| + |
| +| Variable | Default | Notes | |
| +|---|---|---| |
| +| `name` | — | Prefixes every resource | |
| +| `ami_id` | — | Debian arm64, from `region-host` | |
| +| `client_security_group_id` | — | The console box's group | |
| +| `instance_type` | `t4g.medium` | `t4g.small` only with `concurrent_database_reload = false` | |
| +| `stream_max_bytes` | `26214400` | Must equal the console's `CARLOS_CLAMD_MAX_BYTES`. Validation: a whole number of MiB, between 8 MiB and 100 MiB, so the self-test's fixtures (about 5 MB compressed) always fit the stream and a whole stream always fits the scan budget **(codex 11)** | |
| +| `memory_max` | `"3G"` | clamd's cgroup ceiling | |
| +| `concurrent_database_reload` | `true` | | |
| +| `subnet_id` | `null` | | |
| +| `region` | — | For the SSM agent download URL | |
| + |
| +Validation: `instance_type` beginning `t4g.small`/`t4g.micro`/`t4g.nano` with |
| +`concurrent_database_reload = true` is refused with a message naming the reload |
| +spike. |
| + |
| +Outputs: `clamd_addr` (`"<private_ip>:3310"`), `instance_id`, |
| +`security_group_id`, `stream_max_bytes`. |
| + |
| +### 3.6 Replacement |
| + |
| +A replaced box gets a new private IP, so `CARLOS_CLAMD_ADDR` must be updated by |
| +hand. Accepted: replacements are explicit (`-replace`) and the failure is loud |
| +(sweeps fail per tick, `scan status` and `antivirus status` say so, the console |
| +stays up). The README says this in its first section. |
| + |
| +## 4. The console's read-and-tag grant |
| + |
| +### 4.1 `region-host` (Carloku) |
| + |
| +- New variable `store_scanning` (bool, default `false`). A precondition refuses |
| + `true` with an empty `store_bucket_prefix`. |
| +- When `true`, a **separate** `aws_iam_role_policy`, `${iam_name}-store-scan`: |
| + `s3:GetObject`, `s3:GetObjectTagging`, `s3:PutObjectTagging` on |
| + `arn:aws:s3:::${store_bucket_prefix}*/*`. These are exactly the calls the |
| + scanner makes (`HeadObject` is authorised by `GetObject`; `ListObjectsV2` is |
| + already granted by `host_store`). Store buckets use SSE-S3, so no KMS grant. |
| +- Separate, so an opted-out deployment plans zero changes, and so the existing |
| + `host_store` comment ("No GetObject: the console provisions stores, it does not |
| + read tenant data") stays true. That comment gains one line pointing at |
| + `store_scanning`. |
| +- The store-user boundary is untouched. Apps still cannot tag their own objects. |
| + |
| +### 4.2 Tito (`object-store.tf`) |
| + |
| +A new statement `StoreScan` in `host_ie_object_store` with the same three |
| +actions on `arn:aws:s3:::carlos-store-tito-*/*` and |
| +`arn:aws:s3:::titogo-uploads-665106592142/*`. Tito keeps its own store policy |
| +(#201 settled this), so `store_scanning` is not used there. |
| + |
| +### 4.3 The client change: limits exceeded is unscanned (D8) |
| + |
| +`internal/clamd/clamd.go` `parseReply` (`:273`) maps any `FOUND` to |
| +`StatusInfected`. With `AlertExceedsMax yes`, a limit hit arrives as |
| +`stream: Heuristics.Limits.Exceeded.MaxScanSize FOUND` (likewise `.MaxFileSize`, |
| +`.MaxFiles`, `.MaxRecursion`, `.MaxScanTime`). Add two cases **before** the |
| +`FOUND` case, both keeping the raw reply: |
| + |
| +- signature exactly `Heuristics.Limits.Exceeded.MaxScanTime` becomes |
| + `StatusError`. A timeout depends on load, not on the bytes, so it must be |
| + retried. `verdictFor` maps `StatusError` to `VerdictFailed`, which |
| + `clamav.go:249-251` re-issues every sweep rather than carrying forward. |
| + **(codex 7)** |
| +- any other signature beginning `Heuristics.Limits.Exceeded` becomes |
| + `StatusSizeExceeded`. These limits are deterministic for given bytes and |
| + config. `clamav.go:374-383` maps it to `VerdictSkipped`, so it lands in |
| + `skipped` and `unscanned`, never `clean` or `infected`, and like an oversize |
| + object it is carried forward until the bytes change. |
| + |
| +**Declined: codex 6** (a malware match hiding an earlier limit hit). When |
| +clamd reports a real match, the client records infected, whatever limit was |
| +also hit. That is correct under D8: the object is known to contain malware, |
| +whatever else went unread. Setting `HeuristicScanPrecedence yes` to surface the |
| +limit instead would turn a known infection into `unscanned`, which is worse. |
| +When nothing matches, clamd answers with the limit heuristic, so the case D8 |
| +exists for is covered. Which of the two an archive gets depends on what clamd |
| +reaches before the budget runs out (§3.4: a 500 MiB member ahead of EICAR |
| +answers the limit, a 60 MiB member ahead of it answers EICAR). Both are honest |
| +and neither is clean. §7 step 2 tests both. |
| + |
| +Tests in `clamd_test.go`: `MaxScanTime` maps to error; each of the other four |
| +limit names maps to size-exceeded; |
| +`Eicar-Signature FOUND` still maps to infected; a signature that merely |
| +contains the phrase elsewhere (`Win.Heuristics.Limits.Exceeded.Fake`) still |
| +maps to infected, so the match is a prefix on the signature, not a substring of |
| +the reply. The docs for `carlos store scan status` that explain `skipped` gain |
| +one sentence naming over-limit archives. |
| + |
| +Because this is console code, **both consoles roll to a build containing it |
| +before they are wired** (§6.2). A console without it, wired to a clamd with |
| +`AlertExceedsMax yes`, would record over-limit archives as infected. |
| + |
| +## 5. Release and consumption |
| + |
| +- The module change, `scanner-host` and the §4.3 client change land on amadan main through a branch and |
| + `amadan branch merge -squash`, then the GitHub mirror is pushed forward |
| + (fast-forward checked first), then **v0.23.0** is cut per the README's |
| + "Releasing the platform" recipe and pushed to the mirror. |
| +- **Carloku** (`carloku-infrastructure`): move **only `host_ie`'s** `?ref=` to |
| + v0.23.0, set `store_scanning = true`, add `module "scanner"` with |
| + `client_security_group_id = module.host_ie.security_group_id`. The other three |
| + regions stay on v0.21.0. Every apply is `-target`ed |
| + (`module.host_ie.aws_iam_role_policy.host_store_scan`, `module.scanner`), and |
| + the plan is read for any `user_data` line before it runs, because a |
| + `user_data` change stops and starts a box. Any other `host_ie` diff the |
| + v0.21.0 to v0.23.0 bump shows is listed in the PR and not applied. |
| +- **Tito** (`teamtito/carlos-infrastructure`): a branch from the |
| + `store-boundary-201` line (main cannot plan while the Canada box is only on |
| + `region-ca`), `module "scanner"` at `?ref=v0.23.0`, the `StoreScan` |
| + statement. Targeted applies. Tito pins `region-host` to a SHA; that pin does |
| + not move. |
| + |
| +## 6. Wiring the console |
| + |
| +### 6.1 The scan tag key |
| + |
| +- 32 random bytes, hex. Generated once per deployment, never in git, tofu state |
| + or a transcript: generated and written in one pipeline. |
| +- Durable copy: SSM SecureString `/carlos/scan-tag-key` in the deployment's |
| + account (inside the `/carlos/*` path the box can already read). Losing it makes |
| + every stored verdict unverifiable. |
| +- `CARLOS_SCAN_TAG_KEY_PREVIOUS` stays unset; rotation exists and is not needed. |
| + |
| +### 6.2 The console's environment |
| + |
| +**Two preconditions, both checked before any variable is written.** |
| + |
| +*The scanner box is ready.* Over SSM on the scanner instance: `cloud-init |
| +status --wait` reports `done` with no error; `/var/lib/scanner-host/ready` |
| +exists; `systemctl is-active scanner-selftest.service scanner-tcp.socket` says |
| +`active` twice; and the module's acceptance checks (§3.3) have been run on |
| +this box. Nothing the console can ask tells a tested engine from an untested |
| +one, so this is checked on the box, on **each** deployment, Tito included. |
| +**(codex 10)** |
| + |
| +*The console **process** runs a build containing §4.3.* Promoting only moves the channel |
| +pointer (`internal/release/release.go:718`); adoption is separate and can defer |
| +(`adopt.go:956`); and `x-carlos-version` comes from the edge's route state |
| +(`internal/edge/edge.go:2192`), not from the running process. So neither a |
| +promote nor the header proves it. The check is the one `roll-console.sh` |
| +already makes (`scripts/roll-console.sh:93-97`): over SSM, `readlink |
| +/proc/$(systemctl show -p MainPID --value console)/exe` must resolve to |
| +`versions/<sha>/console` for the exact SHA built. Roll with `roll-console.sh` |
| +(it ships, promotes, adopts, restarts and runs that check); if the check fails, |
| +**stop**: nothing is wired. Boxes are untouched. On Tito, the same from Tito's |
| +bucket. **(codex 8)** |
| + |
| +First confirm over SSM, read-only, which file `console.service` loads. Then, |
| +after a dated backup of that file: |
| + |
| +``` |
| +CARLOS_CLAMD_ADDR=<module.scanner.clamd_addr> |
| +CARLOS_CLAMD_MAX_BYTES=<module.scanner.stream_max_bytes> |
| +CARLOS_SCAN_TAG_KEY=<from /carlos/scan-tag-key> |
| +``` |
| + |
| +`systemctl restart console`, then repeat the `/proc/<pid>/exe` check (same SHA, |
| +new PID), and `carlos system antivirus status` answers with the engine wired. |
| +If it fails to start: restore the backup, `systemctl reset-failed console && |
| +systemctl restart console` (a start-limited unit ignores try-restart; seen |
| +2026-09-11). |
| + |
| +`CARLOS_STORE_SCANNER` stays unset. It is deprecated, and the switch is the |
| +record (`system/antivirus.json`), so the restart wires the engine without |
| +turning it on. |
| + |
| +### 6.3 Switching on |
| + |
| +`carlos system antivirus status`: engine wired, clamd answering, signature date |
| +today or yesterday. Then `carlos system antivirus enable`. With no app granted |
| +the hourly pass runs and correctly does nothing. Sit in that state until §7 |
| +passes. |
| + |
| +## 7. Proving it on Carloku |
| + |
| +On a throwaway app we own, with a plaintext store: |
| + |
| +1. Upload one clean object and the EICAR test file. `scan request --plaintext`, |
| + `grant`, `sweep`. **Pass:** `scan status` shows a completed sweep, one clean, |
| + one infected; both objects' tags verify under this deployment's key and carry |
| + the object's current ETag. |
| +2. Upload three archives from the §3.4 fixture set and `sweep`: |
| + - the zip with one member expanding to 500 MiB. **Pass:** `skipped` and |
| + `unscanned`, never `clean` or `infected`; its tag carries no clean |
| + verdict. **(codex 4, codex 9)** |
| + - the zip with that member followed by EICAR. **Pass:** `skipped` and |
| + `unscanned` (clamd stops at the limit before reaching EICAR). |
| + - the zip with a 60 MiB member followed by EICAR. **Pass:** `infected` |
| + (D8: a real match stands). |
| + |
| + If any of the three reads `clean`, stop. |
| +3. Overwrite the clean object, `sweep`. **Pass:** it is re-scanned, not carried |
| + forward. **If it is carried forward, stop the rollout:** a verdict that |
| + outlives its bytes is worse than none. |
| +4. `enforce on`. Remove `CARLOS_CLAMD_ADDR`, restart. **Pass:** `enforce off` |
| + still works with no engine built. Restore the address, restart. |
| +5. Destroy the app. **Pass:** no `scan.json` remains under its prefix and the |
| + name is claimable at once. |
| + |
| +Then stop. Scanning stays enabled; no member store is granted. |
| + |
| +## 8. Tito |
| + |
| +After §7 passes and after #201 phase 2 has landed (so two changes do not edit |
| +Tito's console environment on the same day): §5's Tito apply, the Tito console |
| +roll and the scanner-box readiness check (§6.2's two preconditions), §6 in |
| +Tito's account, `antivirus status`, `enable`. **Tito's proof stops at status and |
| +enable.** §7 exercises platform code identical to Carloku's, and a throwaway app |
| +in a customer's deployment needs the customer's say-so. |
| + |
| +## 9. Docs |
| + |
| +- `infra/modules/scanner-host/README.md`: what it is, the three outputs the |
| + console needs, replacement (§3.6), the invariant (§3.2), the limits and why they are equal (§3.4), the acceptance checks (§3.3), and the sizing rule. |
| +- `infra/modules/region-host/README.md`: `store_scanning`. |
| +- A dated note at the top of `../plans/2026-09-01-carloku-scanning-rollout.md`: |
| + steps 2 and 4 are superseded by this spec, `CARLOS_STORE_SCANNER` is |
| + deprecated. |
| +- An amadan discussion on `carlos/platform` tracking the rollout. |
| + |
| +## 10. Cost |
| + |
| +One `t4g.medium` plus 20 GiB gp3 per deployment: about $25 a month each, $50 |
| +for the pair. |
| + |
| +## 11. What would change this design |
| + |
| +- `t4g.medium` turning out too small for the reload spike in practice: raise |
| + the instance type, or set `concurrent_database_reload = false`. |
| +- A member asking for scanning on a client-encrypted store: no change here, the |
| + existing `--client-encrypted` path handles it. |
| +- Store buckets becoming per-region: see §2, out of scope. |
| +- **zip64 and encrypted archive members are not opened by clamd** (§3.4), so a |
| + clean verdict never covers them. Follow-ups, each needing its own decision: |
| + say so in the `scan request` disclosure (user-facing copy, so copy review); |
| + consider `AlertEncrypted yes` with a mapping to unscanned, the way §4.3 maps |
| + limits. |
| +- Making the scan limit configurable: the self-test's fixture sizes and |
| + expected answers must then be derived from it (codex 11). Fixed at 100 MiB |
| + until someone needs otherwise. |
| +- Raising clamd's limits later: objects already recorded `skipped` under the |
| + old limits are carried forward until their bytes change, as oversize objects |
| + are today. A forced re-sweep of skipped objects is a follow-up if anyone |
| + raises a limit, not part of this rollout. |
diff --git a/docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST-CODE.md b/docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST-CODE.md| new file mode 100644 |
| index 000000000..34fae5280 |
| --- /dev/null |
| +++ b/docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST-CODE.md |
| @@ -0,0 +1,17 @@ |
| +13. **P2 — The self-test accepts timeouts as proof that archive limits work.** [scanner-selftest:104](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/scanner-host/infra/modules/scanner-host/files/scanner-selftest:104) |
| + |
| + The classifier accepts `Heuristics.Limits.Exceeded.MaxScanTime FOUND` as a successful `limit` result. That proves only that scanning timed out, not that the deterministic archive limits prevented silent truncation. A regressed engine can therefore pass during transient load and later answer `OK` when that load subsides. ClamAV generates this reply specifically on timeout. [ClamAV 1.4.3 source](https://github.com/Cisco-Talos/clamav/blob/clamav-1.4.3/libclamav/others.c#L1152-L1156) |
| + |
| + The substring match also accepts `Win.Heuristics.Limits.Exceeded.Fake FOUND`. Both misclassifications were reproduced using the actual classifier. |
| + |
| + **Fix:** Parse the signature and accept only the expected deterministic limit names. Treat `MaxScanTime` as failure or retry it within a bounded policy; reject lookalikes. Add regression cases for both. |
| + |
| +14. **P3 — A logging failure bypasses the required FAILED marker.** [scanner-init.sh.tftpl:15](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/scanner-host/infra/modules/scanner-host/scanner-init.sh.tftpl:15) |
| + |
| + If `tee /dev/console` fails inside `say()`, `set -e` terminates the script without invoking the non-inherited `ERR` trap. Simulating that failure with the actual function definitions produced exit status 1 but no `SCANNER-HOST FAILED:` marker. The prescribed serial-output polling can consequently receive neither terminal marker. |
| + |
| + **Fix:** Handle logging failures explicitly, or inherit the trap with `set -E` and make `fail()` emit its marker through a fallback that does not depend on successful serial-console logging. |
| + |
| +Reviewed `092c5a85`; no files changed. Shell syntax, OpenTofu formatting and diff checks passed. Go, provider-backed and live-box tests were not run. |
| + |
| +accept with fixes |
| \ No newline at end of file |
diff --git a/docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST-R2.md b/docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST-R2.md| new file mode 100644 |
| index 000000000..fca8cce29 |
| --- /dev/null |
| +++ b/docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST-R2.md |
| @@ -0,0 +1,35 @@ |
| +| Round-1 finding | Disposition | Reason | |
| +|---|---|---| |
| +| 1. Scanner role can read parameters | **Closed** | §3.1 now requires an explicit deny covering all four parameter-read actions on every resource. | |
| +| 2. Masked socket prevents clamd startup | **Closed** | §3.2 now requires a complete replacement service without `Requires=` or `Also=` referencing the socket. | |
| +| 3. Foreground FreshClam conflicts with package-started daemon | **Closed** | Package starts are suppressed; the foreground initialization completes before the updater service starts. | |
| +| 4. Incomplete scans can be recorded as clean | **Partly closed** | `AlertExceedsMax` and the new mapping address limit-only replies. A limit can still be hidden by a malware signature, violating D8; see finding 6. | |
| +| 5. SSM administration lacks an installed agent | **Closed** | Installing and enabling the arm64 SSM agent is now the first bootstrap step. | |
| + |
| +Reviewed draft 2 at `05752422`. These are design dispositions; no files changed or live tests performed. |
| + |
| +6. **P2 — §§3.2, 4.3, 7: A malware signature can hide an earlier limit hit.** |
| + |
| + ClamAV’s default `HeuristicScanPrecedence no` allows scanning to continue after a heuristic detection and gives a subsequent malware detection precedence. Limit indicators use that heuristic path: [libclamav/others.c:1045,1354](https://raw.githubusercontent.com/Cisco-Talos/clamav/clamav-1.4.3/libclamav/others.c). The precedence behavior is documented in the [trixie configuration manual](https://manpages.debian.org/trixie/clamav-daemon/clamd.conf.5.en.html#HeuristicScanPrecedence). |
| + |
| + An archive with an oversized member followed by a detectable EICAR member can therefore produce a malware `FOUND` reply without the limit prefix. The proposed case never matches; [internal/clamd/clamd.go:280](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/scanner-host/internal/clamd/clamd.go:280) still classifies it as infected, and [internal/objstore/clamav.go:364](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/scanner-host/internal/objstore/clamav.go:364) signs that verdict. This contradicts D8’s required result for an object known to have been partly scanned. The zeros-only proof does not exercise this combination. |
| + |
| + **Fix:** Ensure an encountered limit survives to the client—configure and test heuristic precedence, or carry completion evidence separately from the selected signature. Add a mixed oversized-member/EICAR archive to §7 and require `unscanned`. |
| + |
| +7. **P2 — §4.3: Mapping scan-time exhaustion to `skipped` makes a temporary timeout permanent.** |
| + |
| + `StatusSizeExceeded` becomes `VerdictSkipped`, but [internal/objstore/clamav.go:249](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/scanner-host/internal/objstore/clamav.go:249) retries only `denied` and `failed`. An unchanged, signed `skipped` verdict is returned indefinitely. [internal/objstore/sweep.go:241](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/scanner-host/internal/objstore/sweep.go:241) calls the engine only for `unknown`. |
| + |
| + Consequently, one `MaxScanTime` result during scanner contention permanently excludes that object from scanning, even after recovery or a timeout increase. Raising extraction limits likewise does not retry archives skipped under the previous settings. |
| + |
| + **Fix:** Preserve D8’s `unscanned` classification while defining retry behavior for engine-limit results. For example, retry skipped objects on subsequent sweeps, retaining the existing size check before fetching. Add a two-sweep test: limit refusal first, successful scan second, with unchanged object bytes. |
| + |
| +8. **P2 — §6.2: Ship/promote does not establish the new console’s running-version precondition.** |
| + |
| + Direct promotion only writes the channel pointer: [internal/release/release.go:718](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/scanner-host/internal/release/release.go:718). Adoption happens separately and can defer or fail staging: [internal/release/adopt.go:956](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/scanner-host/internal/release/adopt.go:956). Restarting `console.service` before adoption therefore restarts the old executable, now with the scanner wired. |
| + |
| + The header check occurs after wiring and has no explicit expected-build or success-status gate. Moreover, the edge supplies `X-Carlos-Version` from its route state: [internal/edge/edge.go:2192](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/scanner-host/internal/edge/edge.go:2192). For an unstamped console unit, adoption can advance that state without restarting the actual console process; see [internal/host/agent.go:2898](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/scanner-host/internal/host/agent.go:2898). |
| + |
| + **Fix:** Before changing scanner environment variables, require adoption of the exact console artifact containing §4.3, completion of the console process restart, and a successful response identifying that build. Make failure stop the sequence before wiring or enabling scanning. |
| + |
| +**Verdict: accept with fixes.** |
| \ No newline at end of file |
diff --git a/docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST-R3.md b/docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST-R3.md| new file mode 100644 |
| index 000000000..5414e2e37 |
| --- /dev/null |
| +++ b/docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST-R3.md |
| @@ -0,0 +1,19 @@ |
| +| Finding | Disposition | Reason | |
| +|---|---|---| |
| +| 6 | **Declined-and-sound** | A matched malware signature can take precedence over a limit heuristic with `HeuristicScanPrecedence no`. Recording that result as infected satisfies revised D8. [ClamAV implementation](https://github.com/Cisco-Talos/clamav/blob/clamav-1.4.3/libclamav/others.c#L1271-L1360). | |
| +| 7 | **Partly closed** | §4.3 maps `MaxScanTime` to `StatusError` → `VerdictFailed`, which subsequent sweeps retry. Previously skipped objects still remain skipped after configuration limits increase; §11 explicitly defers that part. | |
| +| 8 | **Closed** | §6.2 now requires adoption, restart and verification of the actual running executable against the exact build SHA before wiring, then repeats the executable check after wiring. | |
| + |
| +Reviewed draft 3 at `ae1cb880`. No files changed or live scans performed. |
| + |
| +9. **P2 — §§3.2, 4.3 and 7: ZIP extraction can silently truncate despite `AlertExceedsMax yes`.** |
| + |
| + ClamAV 1.4.3’s deflate extraction loop stops when expanded output exceeds `MaxFileSize`, sets `Z_STREAM_END`, and subsequently scans the truncated temporary file. That branch does **not** record a limit heuristic. See [`libclamav/unzip.c:209–227`](https://github.com/Cisco-Talos/clamav/blob/clamav-1.4.3/libclamav/unzip.c#L209-L227) and [`:326–335`](https://github.com/Cisco-Talos/clamav/blob/clamav-1.4.3/libclamav/unzip.c#L326-L335). |
| + |
| + For example, the spec permits `MaxFileSize=26214400` with `MaxScanSize=400M`. A small ZIP containing a 500 MiB zero-filled member is truncated to approximately 25 MiB before scanning. The scanned prefix stays below both limits, so the result can be `OK`. The proposed prefix mapping never runs: [internal/clamd/clamd.go:278](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/scanner-host/internal/clamd/clamd.go:278) classifies it as clean, and [internal/objstore/clamav.go:364](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/scanner-host/internal/objstore/clamav.go:364) records and signs it. |
| + |
| + This is a newly identified mechanism that defeats the accepted `AlertExceedsMax` remedy. It violates D8 without involving malware precedence. The mixed archive can also return infected from its separate EICAR member without ever exercising limit-heuristic precedence. |
| + |
| + **Fix:** Require a tested engine/configuration that reports this truncation as unscanned. One configuration to validate is `MaxFileSize` sufficiently above `MaxScanSize`, so the truncated output itself exceeds the aggregate limit. Add separate real-engine fixtures for a single oversized ZIP member and aggregate expansion across smaller members; require their raw limit replies before accepting §7. |
| + |
| +**Verdict: accept with fixes.** |
| \ No newline at end of file |
diff --git a/docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST-R4.md b/docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST-R4.md| new file mode 100644 |
| index 000000000..3d68ee7af |
| --- /dev/null |
| +++ b/docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST-R4.md |
| @@ -0,0 +1,26 @@ |
| +| Finding | Disposition | Reason | |
| +|---|---|---| |
| +| 9 | **Closed for the specified 100 MiB configuration** | The recorded real-engine results address the reported truncation. Source inspection supports them: ClamAV accounts for the enclosing archive before checking the extracted member’s size, and checks limits before consulting the clean cache. See [`scanners.c:3918`](https://github.com/Cisco-Talos/clamav/blob/clamav-1.4.3/libclamav/scanners.c#L3918) and [`others.c:1081`](https://github.com/Cisco-Talos/clamav/blob/clamav-1.4.3/libclamav/others.c#L1081-L1126). | |
| +| 7, remaining part | **Explicitly deferred, not fixed** | §11 acknowledges that increasing limits does not retry unchanged `skipped` objects. That matches `internal/objstore/clamav.go:249–258`. This is an identified follow-up outside the initial rollout, not a new finding. | |
| + |
| +Reviewed `51985b0d`. No files changed or live scans performed. |
| + |
| +10. **P2 — §§3.2, 6.3, 8: A failed self-test does not prevent the scanner passing the rollout’s readiness checks.** |
| + |
| + Step 6 starts clamd before step 7 runs the self-test. Exiting the bootstrap script non-zero does not stop that already-running service; cloud-init’s user-data stage does not block boot. See [cloud-init’s final-stage documentation](https://docs.cloud-init.io/en/latest/explanation/boot.html#final). |
| + |
| + The subsequent checks cannot distinguish this state from success: [`internal/objstore/clamav.go:108`](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/scanner-host/internal/objstore/clamav.go:108) obtains provenance using only `VERSION`; [`internal/console/apisystem.go:85`](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/scanner-host/internal/console/apisystem.go:85) uses that for status; enable checks only that a scanner is wired at line 133. |
| + |
| + Consequently, an engine that returns `OK` for an over-limit fixture can fail its self-test yet remain reachable, report fresh signatures, and pass Tito’s prescribed `status`/`enable` proof. Withholding the script’s final version line does not close this path. |
| + |
| + **Fix:** Require successful bootstrap and self-test completion on each scanner instance before §6 wiring, including Tito. Make self-test failure stop the scanner and prevent it restarting into service without a successful test. |
| + |
| +11. **P2 — §§3.2–3.4: Fixed-size fixtures reject valid non-default limits.** |
| + |
| + §3.3 permits `max_scan_bytes = 33554432` with the default 25 MiB stream limit. But §3.4 requires the ZIP containing one 60 MiB member to return `OK`. With both engine limits at 32 MiB, its extraction instead encounters the limit: [`libclamav/unzip.c:209–227`](https://github.com/Cisco-Talos/clamav/blob/clamav-1.4.3/libclamav/unzip.c#L209-L227) trims the member, and [`libclamav/others.c:1081–1086`](https://github.com/Cisco-Talos/clamav/blob/clamav-1.4.3/libclamav/others.c#L1081-L1086) reports the exhausted aggregate budget. |
| + |
| + A correctly configured scanner therefore fails the mandatory self-test for an explicitly permitted module input. The recorded expectations apply to the 100 MiB default, not the full variable contract. |
| + |
| + **Fix:** Derive fixture sizes and expected outcomes from the configured limits, keeping compressed streams below `stream_max_bytes`; alternatively, restrict this release to the tested limits. |
| + |
| +**Verdict: accept with fixes.** |
| \ No newline at end of file |
diff --git a/docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST-R5.md b/docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST-R5.md| new file mode 100644 |
| index 000000000..6417cdc74 |
| --- /dev/null |
| +++ b/docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST-R5.md |
| @@ -0,0 +1,35 @@ |
| +| Finding | Disposition | Reason | |
| +|---|---|---| |
| +| 10 | **Closed for the original bypass** | The listener now requires a successful self-test, and both listener and proxy are bound to that test’s lifetime. §6.2 also requires scanner-side readiness before wiring. A separate boot-ordering defect remains below. | |
| +| 11 | **Closed** | Both scan limits are fixed at 100 MiB; §3.5 bounds the configurable stream limit so the specified fixtures fit. | |
| + |
| +Reviewed draft 5 at `edee53d4`. No files changed or live-box tests performed. |
| + |
| +12. **P2 — §§3.2–3.3: The gated socket creates a boot-ordering cycle with systemd’s default dependencies.** |
| + |
| + The [specified socket dependency](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/scanner-host/docs/superpowers/specs/2026-10-04-scanner-host-rollout-design.md:109) orders `scanner-tcp.socket` after `scanner-selftest.service`, but neither unit disables default dependencies. On reboot, this produces the following cycle, where each arrow means “must start before”: |
| + |
| + ```text |
| + scanner-tcp.socket |
| + → sockets.target |
| + → basic.target |
| + → scanner-selftest.service |
| + → scanner-tcp.socket |
| + ``` |
| + |
| + **Code evidence:** systemd adds `Before=sockets.target` to sockets in [`src/core/socket.c:262`](https://github.com/systemd/systemd/blob/v257/src/core/socket.c#L262), adds `After=basic.target` to services in [`src/core/service.c:712`](https://github.com/systemd/systemd/blob/v257/src/core/service.c#L712), and orders `basic.target` after `sockets.target` in [`units/basic.target:13`](https://github.com/systemd/systemd/blob/v257/units/basic.target#L13). |
| + |
| + Systemd resolves an ordering cycle by deleting a dispensable job or rejecting the transaction; it cannot execute the specified startup sequence intact. See [`src/core/transaction.c:351–395`](https://github.com/systemd/systemd/blob/v257/src/core/transaction.c#L351). Consequently, the scanner can remain unavailable after reboot. Initial cloud-init startup and §3.3’s kill/restart checks can miss this because `basic.target` and `sockets.target` have already completed. |
| + |
| + **Fix:** Specify `DefaultDependencies=no` for `scanner-tcp.socket`, restore its initialization and shutdown dependencies explicitly, and arrange activation through clamd or `multi-user.target` without ordering it before `sockets.target`. Retain the self-test dependencies. Add a reboot acceptance check and verify the complete boot transaction for ordering cycles. |
| + |
| +The other requested mechanisms do not produce additional findings: |
| + |
| +- **Self-test state:** `Type=oneshot` delays ordered dependents until successful completion. `RemainAfterExit=yes` retains that success as `active (exited)`; it neither overrides `BindsTo=` nor preserves success across reboot. A failed test cannot authorize the listener. [Service semantics](https://manpages.debian.org/trixie/systemd/systemd.service.5.en.html). |
| +- **Clamd restart and package restart:** `BindsTo=` plus `After=` stops dependents when clamd becomes inactive or fails; explicit restarts also propagate. Applying this to **both** TCP units removes the listener and terminates the proxy holding established connections. `Requires=` alone does not cover unexpected deactivation. [Dependency semantics](https://manpages.debian.org/trixie/systemd/systemd.unit.5.en.html). |
| +- **Automatic recovery:** With the default `RestartMode=normal`, automatic restart creates a new start transaction with dependency restart propagation. That transaction processes `Wants=scanner-tcp.socket`, which pulls in the self-test again. This is supported by [`service.c:2405–2418`](https://github.com/systemd/systemd/blob/v257/src/core/service.c#L2405) and [`transaction.c:937–947`](https://github.com/systemd/systemd/blob/v257/src/core/transaction.c#L937). |
| +- **INSTREAM:** `systemd-socket-proxyd` forwards bidirectional stream bytes between TCP and Unix sockets. The client’s [INSTREAM implementation](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/scanner-host/internal/clamd/clamd.go:195) uses byte framing, without descriptor passing, so the proxy supports its protocol. [Proxy semantics](https://manpages.debian.org/trixie/systemd/systemd-socket-proxyd.8.en.html). |
| + |
| +After fixing the boot cycle, the gating design is sound at spec level. Actual restart timing, connection termination and EICAR transport belong to the real-box acceptance checks. |
| + |
| +**Verdict: accept with fixes.** |
| \ No newline at end of file |
diff --git a/docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST.md b/docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST.md| new file mode 100644 |
| index 000000000..b66bfcfa6 |
| --- /dev/null |
| +++ b/docs/superpowers/specs/CODEX-FINDINGS-SCANNER-HOST.md |
| @@ -0,0 +1,23 @@ |
| +Reviewed `d1b85d63`; no files changed. |
| + |
| +1. **P1 — §3.1: The scanner role can read SSM parameters, violating D4.** |
| + `AmazonSSMManagedInstanceCore` includes `ssm:GetParameter` and `ssm:GetParameters` on `*`. Attaching it alone does not deliver “no parameter grants.” The existing module documents this exact problem and supplies an explicit deny: [infra/modules/region-host/main.tf:418](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/scanner-host/infra/modules/region-host/main.tf:418). The [AWS policy document](https://docs.aws.amazon.com/aws-managed-policy/latest/reference/AmazonSSMManagedInstanceCore.html) confirms the permissions. |
| + **Fix:** Explicitly deny all parameter-read actions on the scanner role while retaining Session Manager’s channel permissions. |
| + |
| +2. **P1 — §3.2, steps 6–7: Masking the socket leaves clamd requiring a masked unit.** |
| + Debian trixie’s [clamd/clamav-daemon.service.in:4](https://sources.debian.org/src/clamav/1.4.3%2Bdfsg-1/clamd/clamav-daemon.service.in/#L4) contains `Requires=clamav-daemon.socket`; its install section also names that socket. The proposed memory/restart drop-in does not remove this dependency. Starting the service after masking the socket therefore fails. Debian’s [packaging bug discussion](https://bugs-devel.debian.org/cgi-bin/bugreport.cgi?bug=1042377%3Bmsg%3D10) explicitly identifies removing the dependency as part of this workaround. |
| + **Fix:** Specify a complete service override without the socket dependency and `Also=` entry. Stop the service and socket, mask the socket, reload systemd, then start clamd and verify both listeners. |
| + |
| +3. **P1 — §3.2, steps 1–2: The foreground FreshClam invocation conflicts with the package-started daemon.** |
| + Debian’s [debian/clamav-freshclam.postinst.in:333](https://sources.debian.org/src/clamav/1.4.3%2Bdfsg-1/debian/clamav-freshclam.postinst.in/#L333) starts `clamav-freshclam` during installation. The next prescribed step launches another instance using the same configuration and locked update log. That invocation can fail before initialization reaches the clamd configuration. |
| + **Fix:** Suppress package service starts during installation, or explicitly stop FreshClam before its foreground run. Keep clamd and its socket stopped during configuration; restart the updater after database initialization succeeds. |
| + |
| +4. **P2 — §3.2: Matching the size limits does not prevent incomplete scans being recorded as clean.** |
| + `MaxScanSize` counts expanded archive content, and `MaxFileSize` also applies to contained files. A compressed object below 25 MiB can exceed either limit after extraction. With `AlertExceedsMax` defaulting to `no`, the proposed settings do not guarantee that this becomes a size-refusal response. See the [trixie configuration manual](https://manpages.debian.org/trixie/clamav-daemon/clamd.conf.5.en.html#AlertExceedsMax). The client maps an `OK` response to clean at [internal/clamd/clamd.go:273](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/scanner-host/internal/clamd/clamd.go:273), which is then signed and tagged at [internal/objstore/clamav.go:364](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/scanner-host/internal/objstore/clamav.go:364). |
| + **Fix:** Define explicit handling for extraction-limit results and add an archive exceeding those limits to §7. `AlertExceedsMax yes` prevents silent success, but the current client classifies its `FOUND` response as infected; either document that policy or add a distinct skipped mapping. |
| + |
| +5. **P2 — §§3.1–3.2: The only administration path lacks an installed SSM agent.** |
| + `module.<host>.ami_id` returns the base Debian image, not the configured console host: [infra/modules/region-host/outputs.tf:26](/home/paulca/amadan.net/carlos/platform/.claude/worktrees/scanner-host/infra/modules/region-host/outputs.tf:26). Debian images do not include SSM Agent by default, according to [AWS’s installation instructions](https://docs.aws.amazon.com/systems-manager/latest/userguide/agent-install-deb.html). The proposed script installs only ClamAV packages. With SSH closed, the promised SSM checks and recovery path will be unavailable. |
| + **Fix:** Install and enable the arm64 SSM agent during bootstrap, and require successful Systems Manager registration before wiring the console. |
| + |
| +**Verdict: reject.** |
| \ No newline at end of file |
diff --git a/docs/superpowers/specs/CODEX-REVIEW-BRIEF-SCANNER-HOST-CODE.md b/docs/superpowers/specs/CODEX-REVIEW-BRIEF-SCANNER-HOST-CODE.md| new file mode 100644 |
| index 000000000..4e16c870c |
| --- /dev/null |
| +++ b/docs/superpowers/specs/CODEX-REVIEW-BRIEF-SCANNER-HOST-CODE.md |
| @@ -0,0 +1,59 @@ |
| +# Review brief: scanner host, the code |
| + |
| +You are reviewing the implementation of an infrastructure spec you already |
| +reviewed six times as a design. Your output is a findings report, not an edit. |
| +Do not modify any file. |
| + |
| +## What to read |
| + |
| +1. The spec: `docs/superpowers/specs/2026-10-04-scanner-host-rollout-design.md` |
| + (draft 6). Decisions D1-D8 in §2 are fixed owner rulings: report a mechanism |
| + that fails one, do not re-argue one. Your earlier design findings are in |
| + `CODEX-FINDINGS-SCANNER-HOST*.md` beside it. |
| +2. The plan: `docs/superpowers/plans/2026-10-05-scanner-host-rollout.md`. |
| + Tasks 1-4 are what was built. Tasks 7 and 9-10 show how the module and its |
| + outputs are used on real boxes. |
| +3. The code, as the branch's own changes against main. The merge-base is the |
| + commit the branch started from; review `git diff <merge-base>..HEAD` for: |
| + - `internal/clamd/clamd.go`, `clamd_test.go` |
| + - `infra/modules/region-host/{main,variables}.tf`, `tests/store_scanning.tftest.hcl` |
| + - `infra/modules/scanner-host/` (all of it: `main.tf`, `variables.tf`, |
| + `outputs.tf`, `scanner-init.sh.tftpl`, `files/*`, `tests/*`, `README.md`) |
| + - `docs/site/cli.md`, `infra/modules/README.md`, `infra/modules/region-host/README.md` |
| + Ignore everything that arrived from main through the merge commit. |
| +4. Debian trixie's `clamav` 1.4.3 packaging and ClamAV 1.4.3 sources, as you |
| + know them or can fetch them. Earlier rounds relied on `libclamav/unzip.c`, |
| + `others.c`, `debian/usr.sbin.clamd`, `clamav-daemon.config.in`, |
| + `clamav-daemon.postinst.in`, `clamav-daemon.service.in`. |
| + |
| +## What matters most |
| + |
| +- **The invariant** (spec §3.2): TCP 3310 is open only while a self-test has |
| + passed against the clamd process running now. Trace the three units and the |
| + clamd drop-in through: first boot, reboot, clamd crash and auto-restart, |
| + explicit restart, a package upgrade (which now runs with |
| + `clamav-daemon/debconf` false), a freshclam signature reload, a failed |
| + self-test, and a direct restart of the self-test. Name any state in which the |
| + port is open without a passed test, or never reopens after one. |
| +- **The first-boot script** (`scanner-init.sh.tftpl`): every failure path must |
| + print `SCANNER-HOST FAILED: <reason>` and exit non-zero, and READY must never |
| + print with the invariant unmet. Check `set -euo pipefail` and the ERR trap |
| + (not inherited by functions), template escaping (`${...}` is templatefile), |
| + the AppArmor local include and reload, the debconf freeze and its position |
| + relative to the package's own `clamd.conf` generation, and the copied |
| + `ExecStartPre` lines. |
| +- **The self-test**: can it pass while the property it guards fails? |
| +- **The Go change**: every consumer of `clamd.Status` after the change. |
| +- **IAM**: least privilege on both roles; the scanner role's Deny. |
| +- **Tests**: does each `tofu test` run and Go test pin what it claims? |
| +- **The docs**: anything an operator following the README or `cli.md` would |
| + get wrong because the docs and the code disagree. |
| +- **The acceptance checks** in plan Task 7: could any pass on a box that is |
| + actually wrong? |
| + |
| +## Output format |
| + |
| +Number findings from 13 (1-12 were the design rounds). For each: severity |
| +(P1 blocks applying this to a live deployment, P2 should fix first, P3 nice to |
| +have), file:line, what goes wrong concretely, and a fix. End with one line: |
| +accept, accept with fixes, or reject. |
diff --git a/docs/superpowers/specs/CODEX-REVIEW-BRIEF-SCANNER-HOST.md b/docs/superpowers/specs/CODEX-REVIEW-BRIEF-SCANNER-HOST.md| new file mode 100644 |
| index 000000000..a1eb299b6 |
| --- /dev/null |
| +++ b/docs/superpowers/specs/CODEX-REVIEW-BRIEF-SCANNER-HOST.md |
| @@ -0,0 +1,58 @@ |
| +# Review brief: scanner host rollout spec |
| + |
| +You are reviewing an infrastructure and rollout spec independently of its |
| +author. Your output is a findings report, not an edit. Do not modify any file. |
| + |
| +## What to read |
| + |
| +1. The spec under review: |
| + `docs/superpowers/specs/2026-10-04-scanner-host-rollout-design.md`. |
| +2. Its parents: `docs/superpowers/specs/2026-09-01-object-storage-virus-scanning-design.md` |
| + (§4, §5, §10) and `docs/superpowers/specs/2026-09-02-scan-on-demand-design.md` §9, |
| + and the rollout plan `docs/superpowers/plans/2026-09-01-carloku-scanning-rollout.md`. |
| +3. The code it relies on, in this worktree: `cmd/console/main.go` |
| + (`storeScannerFor`, `scannerWiring`, `storeBackend`), `internal/clamd/clamd.go`, |
| + `internal/objstore/clamav.go`, `scanner.go`, `sweep.go`, `scantag.go`, |
| + `internal/console/scanpass.go`, `internal/console/apisystem.go`, `system.go`, |
| + `infra/modules/region-host/main.tf`, `variables.tf`, `outputs.tf`. |
| +4. Debian trixie's `clamav-daemon` and `clamav-freshclam` packaging behaviour |
| + (socket activation, `clamd.conf` keys `TCPSocket`, `TCPAddr`, |
| + `StreamMaxLength`, `MaxScanSize`, `MaxFileSize`, `ConcurrentDatabaseReload`) |
| + as you know it. |
| + |
| +## Fixed decisions (do not re-argue; report only a mechanism that fails one) |
| + |
| +D1 to D7 in spec §2. In particular D3: plain TCP inside the VPC, fenced by a |
| +single-source security group, was chosen by the owner. D7: scope ends at |
| +"engine wired, switched on, no member granted". |
| + |
| +## What matters most |
| + |
| +- **Does the console actually work with what §4 and §6 supply?** Check every |
| + S3 action the scanner, sweep, tagging, enforce and teardown paths call |
| + against the §4.1 grant and the existing `host_store` policy. A missing action |
| + shows up as `denied` counts or a failed pass in production; name it. |
| +- **Boot refusals.** Every environment variable the console requires once a |
| + clamd address is set, and whether §6.2 supplies them all. Anything that would |
| + crash-loop the console (the 2026-09-11 Tito outage was exactly this). |
| +- **The module.** Can the security group, VPC/subnet preconditions, IMDSv2, |
| + `ignore_changes`, and the user_data sequence (freshclam before clamd, masking |
| + the socket unit, binding the private IP) actually produce a clamd answering on |
| + `<private_ip>:3310` and nowhere else? Anything that leaves clamd on 0.0.0.0, |
| + or a box that reports healthy while clamd is not listening on TCP. |
| +- **StreamMaxLength vs CARLOS_CLAMD_MAX_BYTES.** Is §3.2's rule right for how |
| + `internal/clamd` and `objstore/clamav.go` decide to skip an over-size object? |
| + Any off-by-one or unit mismatch (MB vs MiB, bytes vs "25M"). |
| +- **The proof in §7.** Does each step test what it claims? Would EICAR be |
| + reported as infected through this code path? Is step 3's unwire/rewire safe |
| + given how `system/antivirus.json` and `enforce` behave? |
| +- **The rollout mechanics in §5 and §6.** Targeted applies, the partial ref |
| + bump, key generation and storage. Anything that loses the key, leaks it, or |
| + restarts a box unintentionally. |
| + |
| +## Output format |
| + |
| +Number each finding. For each: severity (P1 blocks rollout, P2 should fix |
| +before rollout, P3 nice to have), the spec section, the code citation |
| +(file:line) that proves it, what goes wrong concretely, and a suggested fix. |
| +End with a one-line verdict: accept, accept with fixes, or reject. |
diff --git a/infra/modules/README.md b/infra/modules/README.md| index 8526879fe..3553d493e 100644 |
| --- a/infra/modules/README.md |
| +++ b/infra/modules/README.md |
| @@ -115,6 +115,27 @@ only the fleet, with *no* bucket or parameter grants; statelessness is |
| imposed, not promised). Generalise it only when a second deployment |
| wants one. |
| +## scanner-host |
| + |
| +One box that runs clamd for the console's object-storage virus scanning, on its own host so signature updates and scan CPU stay off the control plane. The console streams each object to it over TCP 3310. The box reads no buckets and no parameters: its role has Session Manager and an explicit Deny on SSM parameter reads. The security group admits 3310 from the console box's group and nothing else, and there is no SSH. |
| + |
| +Port 3310 is open only while a self-test has passed against the clamd process running now. A replaced box gets a new private IP, so update the console's `CARLOS_CLAMD_ADDR` after any `-replace`. The module's `README.md` has the readiness check, four acceptance checks to run before wiring a console, and what a clean verdict does not cover. |
| + |
| +| Variable | Description | |
| +|---|---| |
| +| `name` | Name for the instance, its security group, its IAM role and its instance profile (e.g. `carlos-flagship-scanner`). | |
| +| `ami_id` | Debian arm64 AMI. Pass the console box's own, `module.<host>.ami_id`. Ignored after creation: a newer image never replaces the box. | |
| +| `client_security_group_id` | Security group of the one box allowed to reach clamd: the console's. clamd has no authentication, so this group is the whole fence. | |
| +| `region` | AWS region the box runs in. The box fetches its SSM agent package from that region's bucket. | |
| +| `instance_type` | arm64 instance type. Default `t4g.medium`. A `t4g.small` works only with `concurrent_database_reload = false`. | |
| +| `stream_max_bytes` | clamd's `StreamMaxLength`, in bytes. Default `26214400` (25 MiB), between 8 MiB and 100 MiB. The console's `CARLOS_CLAMD_MAX_BYTES` must equal it. | |
| +| `memory_max` | clamd's cgroup ceiling (systemd `MemoryMax`). Default `3G`. | |
| +| `memory_high` | clamd's throttle point (systemd `MemoryHigh`). Default `2560M`. | |
| +| `concurrent_database_reload` | Keep scanning while new signatures load, at the cost of holding both databases in memory. Default `true`. | |
| +| `subnet_id` | Subnet to launch in. Null (default) is the default subnet of the default VPC, and is refused unless the client security group is in the default VPC. The subnet needs a route out. | |
| + |
| +Outputs: `clamd_addr` (set as the console's `CARLOS_CLAMD_ADDR`), `stream_max_bytes` (set as `CARLOS_CLAMD_MAX_BYTES`), `instance_id`, `security_group_id`. |
| + |
| ## region-remote-host |
| What an off-cloud CARLOS host (customer fleets and tito's hq) |
diff --git a/infra/modules/region-host/README.md b/infra/modules/region-host/README.md| index b10c3a8c0..bf3c5d067 100644 |
| --- a/infra/modules/region-host/README.md |
| +++ b/infra/modules/region-host/README.md |
| @@ -102,6 +102,10 @@ platform-update script, re-run `carlos bootstrap` so any new unit is |
| installed and armed; a stage relay that is not armed fails silently, which |
| looks exactly like a deploy that did nothing. |
| +## Object-storage scanning |
| + |
| +`store_scanning = true` lets this host's role read and tag objects under `store_bucket_prefix` (`s3:GetObject`, `s3:GetObjectTagging` and `s3:PutObjectTagging`), which is what the console's virus scanner needs. It is off by default, so a deployment that has not switched scanning on plans no change and its console still never reads tenant data. Set it on the console's box only, and only on a deployment that runs a [scanner-host](../scanner-host/README.md). It requires `store_bucket_prefix`. |
| + |
| ## Checking it worked |
| ```sh |
diff --git a/infra/modules/region-host/main.tf b/infra/modules/region-host/main.tf| index 97a13e100..762e08b93 100644 |
| --- a/infra/modules/region-host/main.tf |
| +++ b/infra/modules/region-host/main.tf |
| @@ -624,6 +624,8 @@ resource "aws_iam_role_policy" "host_store" { |
| # ListBucket on the bucket and DeleteObject under it; teardown needs |
| # DeleteBucket. No GetObject: the console provisions stores, it does not |
| # read tenant data, and the economics sweep sizes buckets with ListBucket. |
| + # (Scanning is the exception, and it is a separate policy behind |
| + # store_scanning, below.) |
| { |
| Effect = "Allow" |
| Action = ["s3:CreateBucket", "s3:DeleteBucket", "s3:ListBucket", "s3:GetBucketLocation"] |
| @@ -683,3 +685,36 @@ resource "aws_iam_role_policy" "host_store" { |
| ] |
| }) |
| } |
| + |
| +# ---- object storage scanning (opt-in) -------------------------------------- |
| +# |
| +# The one grant that lets this role read tenant object bytes, so it is its own |
| +# policy behind its own switch: a deployment that has not turned scanning on |
| +# plans no change here, and the sentence in host_store above ("it does not |
| +# read tenant data") stays true for it. |
| +# |
| +# Exactly the calls the scanner makes. HeadObject is authorised by GetObject; |
| +# ListObjectsV2 is already in host_store. Store buckets use SSE-S3, so there is |
| +# no KMS statement. PutObjectTagging is how a verdict is written; the store |
| +# user boundary deliberately lacks it, so an app cannot write one itself. |
| +resource "aws_iam_role_policy" "host_store_scan" { |
| + count = var.store_scanning ? 1 : 0 |
| + name = "${var.iam_name}-store-scan" |
| + role = aws_iam_role.host.id |
| + policy = jsonencode({ |
| + Version = "2012-10-17" |
| + Statement = [ |
| + { |
| + Effect = "Allow" |
| + Action = ["s3:GetObject", "s3:GetObjectTagging", "s3:PutObjectTagging"] |
| + Resource = "arn:aws:s3:::${var.store_bucket_prefix}*/*" |
| + }, |
| + ] |
| + }) |
| + lifecycle { |
| + precondition { |
| + condition = var.store_bucket_prefix != "" |
| + error_message = "store_scanning needs store_bucket_prefix: with no prefix there are no object stores on this deployment to scan, and the grant would name every bucket in the account." |
| + } |
| + } |
| +} |
diff --git a/infra/modules/region-host/tests/store_scanning.tftest.hcl b/infra/modules/region-host/tests/store_scanning.tftest.hcl| new file mode 100644 |
| index 000000000..16a9ed0e9 |
| --- /dev/null |
| +++ b/infra/modules/region-host/tests/store_scanning.tftest.hcl |
| @@ -0,0 +1,63 @@ |
| +# Plan-time tests for the scanning grant. mock_provider means no AWS account |
| +# and no credentials: these assert what the module WOULD create. |
| +mock_provider "aws" {} |
| + |
| +override_data { |
| + target = data.aws_vpcs.default |
| + values = { ids = ["vpc-0aaaaaaaaaaaaaaaa"] } |
| +} |
| + |
| +variables { |
| + name = "carlos-test-ie" |
| + sg_description = "carlos test host" |
| + iam_name = "carlos-test-host" |
| + instance_type = "t4g.small" |
| + root_volume_size = 20 |
| + user_data = "#!/bin/sh\n" |
| + deployment_bucket_arn = "arn:aws:s3:::carlos-test-platform" |
| + data_bucket_arn = "arn:aws:s3:::carlos-test-ie" |
| + ssm_parameter_arn = "arn:aws:ssm:eu-west-1:123456789012:parameter/carlos/*" |
| + store_bucket_prefix = "carlos-store-test-" |
| +} |
| + |
| +run "off_by_default" { |
| + command = plan |
| + assert { |
| + condition = length(aws_iam_role_policy.host_store_scan) == 0 |
| + error_message = "store_scanning defaults to false and must create nothing" |
| + } |
| +} |
| + |
| +run "on_grants_exactly_read_and_tag" { |
| + command = plan |
| + variables { |
| + store_scanning = true |
| + } |
| + assert { |
| + condition = length(aws_iam_role_policy.host_store_scan) == 1 |
| + error_message = "store_scanning = true must create the scan policy" |
| + } |
| + assert { |
| + condition = aws_iam_role_policy.host_store_scan[0].name == "carlos-test-host-store-scan" |
| + error_message = "the policy is named <iam_name>-store-scan" |
| + } |
| + assert { |
| + condition = toset(jsondecode(aws_iam_role_policy.host_store_scan[0].policy).Statement[0].Action) == toset([ |
| + "s3:GetObject", "s3:GetObjectTagging", "s3:PutObjectTagging", |
| + ]) |
| + error_message = "exactly the three actions the scanner calls, no more" |
| + } |
| + assert { |
| + condition = jsondecode(aws_iam_role_policy.host_store_scan[0].policy).Statement[0].Resource == "arn:aws:s3:::carlos-store-test-*/*" |
| + error_message = "scoped to objects under the store prefix" |
| + } |
| +} |
| + |
| +run "refused_without_a_store_prefix" { |
| + command = plan |
| + variables { |
| + store_scanning = true |
| + store_bucket_prefix = "" |
| + } |
| + expect_failures = [aws_iam_role_policy.host_store_scan] |
| +} |
diff --git a/infra/modules/region-host/variables.tf b/infra/modules/region-host/variables.tf| index 75f16cdd5..dfdd197c9 100644 |
| --- a/infra/modules/region-host/variables.tf |
| +++ b/infra/modules/region-host/variables.tf |
| @@ -206,6 +206,12 @@ variable "store_user_path" { |
| default = "/carlos-store/" |
| } |
| +variable "store_scanning" { |
| + description = "Let this host's role READ and TAG objects under store_bucket_prefix, which is what the console's virus scanner needs (s3:GetObject, s3:GetObjectTagging, s3:PutObjectTagging). Off by default: without it the console provisions stores and never reads tenant data. Set it only on the box that runs the console, and only on a deployment that runs a scanner (infra/modules/scanner-host). Requires store_bucket_prefix." |
| + type = bool |
| + default = false |
| +} |
| + |
| variable "relay" { |
| description = "Open the platform STUN/TURN relay's ports (udp+tcp 3478, udp 49152-65535). Off by default; a box also needs /etc/carlos/relay-masters before the unit starts." |
| type = bool |
diff --git a/infra/modules/scanner-host/README.md b/infra/modules/scanner-host/README.md| new file mode 100644 |
| index 000000000..803fd079f |
| --- /dev/null |
| +++ b/infra/modules/scanner-host/README.md |
| @@ -0,0 +1,181 @@ |
| +# scanner-host: a clamd box for a deployment's console |
| + |
| +> 🤖 Written by an LLM (Claude), per factor X. |
| + |
| +This module makes one small box that runs clamd, the ClamAV scanning daemon, so the console can virus-scan the objects in app stores. The console reads each object from S3 and streams it to this box. The box reads nothing itself. Its role has Session Manager and nothing else, with an explicit Deny on every SSM parameter read, so a compromised clamd cannot fetch a tenant object, the scan tag key or the release key. There is no SSH. You reach the box over SSM Session Manager. |
| + |
| +```hcl |
| +module "scanner" { |
| + source = "git::https://github.com/carlosframework/platform.git//infra/modules/scanner-host?ref=main" |
| + name = "carlos-flagship-scanner" |
| + ami_id = module.host_ie.ami_id |
| + client_security_group_id = module.host_ie.security_group_id |
| + region = "eu-west-1" |
| +} |
| +``` |
| + |
| +Pass the console box's own AMI, so the two boxes are built from the same base. Pass the console box's security group as `client_security_group_id`. It is the only thing allowed to reach clamd, which authenticates nobody, so that group is the whole fence. The module creates the scanner's security group in the same VPC. It admits TCP 3310 from that one group and never from a CIDR. |
| + |
| +## If you replace the box, the address changes |
| + |
| +A replaced box gets a new private IP, and the console does not follow it. After any replacement, update `CARLOS_CLAMD_ADDR` on the console and restart it. Until you do, every sweep fails on its tick, and `carlos store scan status` and `carlos system antivirus status` say so. The console itself stays up. |
| + |
| +Replacement is always explicit. A newer AMI, an edited init script or a new `stream_max_bytes` does nothing to a running box. To apply any of them: |
| + |
| +```sh |
| +tofu apply -replace=module.<name>.aws_instance.scanner |
| +``` |
| + |
| +## Wiring the console |
| + |
| +The module has two outputs the console needs. Each becomes a console environment variable. |
| + |
| +| Output | Console variable | |
| +|---|---| |
| +| `clamd_addr` (`<private_ip>:3310`) | `CARLOS_CLAMD_ADDR` | |
| +| `stream_max_bytes` | `CARLOS_CLAMD_MAX_BYTES` | |
| + |
| +The console also needs `CARLOS_SCAN_TAG_KEY`. It signs every verdict the console writes, and the console refuses to boot with an address and no key. Generate 32 random bytes as hex, once per deployment. Keep the only copy in your secret store, never in git, tofu state or a transcript. If you lose it, every stored verdict becomes unverifiable. |
| + |
| +The console's own role also needs `store_scanning = true` on its `region-host`, which lets it read and tag store objects. See that module's README. |
| + |
| +Wire the console only after the box has passed the readiness check and the four acceptance checks below. Nothing the console can ask tells a tested engine from an untested one. The console must also run a build that records an archive over the engine's limits as not scanned. A console without that change would record such an archive as infected. |
| + |
| +Wiring an address does not switch scanning on. Switch it on with `carlos system antivirus enable`. |
| + |
| +## How the port is kept shut |
| + |
| +Port 3310 is open only while a self-test has passed against the clamd process that is running now. A failed test, a clamd restart or a ClamAV package upgrade closes the port until the test passes again. The console cannot check this for you, because `antivirus status` and `enable` only ask clamd for its version, and an engine that signs a truncated scan as clean answers that perfectly well. |
| + |
| +To get there, clamd never listens on TCP. It listens on the unix socket `/run/clamav/clamd.ctl` only. `scanner-tcp.socket` listens on the box's private IPv4 address on port 3310 and hands each connection to `scanner-tcp.service`, which carries it to the unix socket. Both units are bound to `scanner-selftest.service`, which is bound to `clamav-daemon.service`. When the self-test unit goes inactive, the port closes and connections in flight are cut. |
| + |
| +The self-test builds ten archive fixtures, streams each to clamd, and requires the right class of answer. An archive that expands past the limits must answer with a limit hit. An `OK` there is a failure. It runs every time clamd starts, on whatever ClamAV version is installed at that moment. |
| + |
| +## Why the two limits are equal and fixed |
| + |
| +clamd runs with `MaxFileSize 100M`, `MaxScanSize 100M` and `AlertExceedsMax yes`. These are constants in this release, not variables. |
| + |
| +`AlertExceedsMax yes` makes clamd answer `Heuristics.Limits.Exceeded.<limit> FOUND` when an archive expands past a limit. The console records that as not scanned. With the default `no`, clamd scans part of the archive and answers `OK`. |
| + |
| +That is not enough on its own. ClamAV 1.4.3 trims a zip member that expands past `MaxFileSize`, scans the trimmed copy, and raises no limit. With the two limits equal, the trimmed copy is as large as the whole scan budget, so scanning it always trips `MaxScanSize`, which does alert. The fixtures and expected answers in the self-test are sized for 100 MiB, so changing the limit means deriving both from it. |
| + |
| +Measured on 2026-10-05 against Debian trixie's clamd (ClamAV 1.4.3), with `StreamMaxLength` at 25M and every fixture streamed with `clamdscan --stream`. Each member is a repeating 16 KiB random block, distinct per member, so clamd's clean cache cannot short-circuit it. "Limit" means a `Heuristics.Limits.Exceeded.*` reply. The self-test does not count `Heuristics.Limits.Exceeded.MaxScanTime` as one. That reply is a timeout, not a limit, and it fails the test. |
| + |
| +| Fixture | `MaxFileSize 25M`, `MaxScanSize 100M` | Both 100M (this module) | Self-test requires | |
| +|---|---|---|---| |
| +| small clean file | OK | OK | OK | |
| +| EICAR | infected | infected | infected | |
| +| zip, one member expanding to 500 MiB | **OK** (trimmed, silent) | Limit | Limit | |
| +| zip, a zip inside holding that member | **OK** (silent) | Limit | Limit | |
| +| zip, 30 distinct 20 MiB members | Limit | Limit | Limit | |
| +| gzip of 500 MiB | Limit | Limit | Limit | |
| +| tar.gz, 5 x 40 MiB | Limit | Limit | Limit | |
| +| bzip2, xz, tar.xz of the same | Limit | Limit | not in self-test | |
| +| zip, 500 MiB member then EICAR | infected | Limit | Limit | |
| +| zip, 60 MiB member then EICAR | infected | infected | infected | |
| +| zip, one 60 MiB member | **OK** (trimmed, silent) | OK (fully scanned) | OK | |
| + |
| +The first column is what an unequal pair does: three fixtures were signed clean over content nobody read. |
| + |
| +`stream_max_bytes` is the one limit you can change. It is clamd's `StreamMaxLength`, the largest object it accepts on a stream. It defaults to 26214400 (25 MiB). It must be a whole number of MiB, between 8 MiB and 100 MiB. The floor keeps the self-test's fixtures, about 5 MB each, inside the stream. The ceiling keeps a whole stream inside the scan budget. The console's `CARLOS_CLAMD_MAX_BYTES` must equal it. Use the `stream_max_bytes` output rather than typing the number twice. |
| + |
| +## What a clean verdict does not cover |
| + |
| +A zip member written with zip64 local headers is not extracted at all by ClamAV 1.4.3 (`local header - stream out of file`), and EICAR inside one reads `OK` under every configuration. Encrypted archives are the same. This is parser coverage, not a limit, and no clamd setting changes it. A clean verdict means clamd found nothing in what it could open. It does not mean the object is safe. |
| + |
| +## Checking that a box is ready |
| + |
| +First boot takes several minutes. It installs packages, downloads about 110 MB of signatures and runs the self-test. In an SSM session on the box: |
| + |
| +```sh |
| +cloud-init status --wait |
| +cat /var/lib/scanner-host/ready |
| +systemctl is-active scanner-selftest.service scanner-tcp.socket |
| +journalctl -u scanner-selftest.service --no-pager | tail -14 |
| +``` |
| + |
| +You should see `status: done`, a ClamAV version line, `active` twice, and then ten `pass` lines and `scanner-selftest: PASS`. The init script writes each step to `/var/log/cloud-init-output.log`, and copies it to the serial console when it can, ending in `SCANNER-HOST READY <version>` or `SCANNER-HOST FAILED: <reason>`. To read the console copy without a session: |
| + |
| +```sh |
| +aws ec2 get-console-output --instance-id <scanner-instance-id> --latest --output text | grep 'SCANNER-HOST' |
| +``` |
| + |
| +If the box failed, fix the module and replace the box. Do not patch the box. |
| + |
| +## Acceptance checks |
| + |
| +Run all four on the first real box in a deployment, before the console is wired, and record the results. Each one proves a part of the invariant that first boot cannot. Run the commands as root in an SSM session on the box, except the third. |
| + |
| +**1. Killing clamd closes the port, and it reopens only after the self-test passes.** |
| + |
| +```sh |
| +systemctl kill -s KILL clamav-daemon.service |
| +sleep 3 |
| +echo "after kill: $(ss -ltnH 'sport = :3310' | wc -l) listeners, selftest $(systemctl is-active scanner-selftest.service)" |
| +for i in $(seq 1 60); do [ "$(systemctl is-active scanner-tcp.socket)" = active ] && break; sleep 10; done |
| +echo "recovered: socket $(systemctl is-active scanner-tcp.socket), selftest $(systemctl is-active scanner-selftest.service), $(ss -ltnH 'sport = :3310' | wc -l) listeners" |
| +``` |
| + |
| +Expected: `after kill: 0 listeners`, with the self-test anything but `active`. Then `recovered: socket active, selftest active, 1 listeners`. If the port is still open right after the kill, or never reopens, the unit dependencies do not work on this systemd. Fix the module and replace the box. |
| + |
| +**2. A failed self-test keeps the port shut.** The script honours `SCANNER_SELFTEST_FORCE_FAIL=1`. Set it in a drop-in for this check only. |
| + |
| +```sh |
| +mkdir -p /etc/systemd/system/scanner-selftest.service.d |
| +printf '[Service]\nEnvironment=SCANNER_SELFTEST_FORCE_FAIL=1\n' > /etc/systemd/system/scanner-selftest.service.d/99-force-fail.conf |
| +systemctl daemon-reload |
| +systemctl restart clamav-daemon.service |
| +sleep 90 |
| +systemctl is-active clamav-daemon.service scanner-selftest.service scanner-tcp.socket |
| +ss -ltnH 'sport = :3310' | wc -l |
| + |
| +rm /etc/systemd/system/scanner-selftest.service.d/99-force-fail.conf |
| +systemctl daemon-reload |
| +systemctl restart clamav-daemon.service |
| +for i in $(seq 1 60); do [ "$(systemctl is-active scanner-tcp.socket)" = active ] && break; sleep 10; done |
| +systemctl is-active clamav-daemon.service scanner-selftest.service scanner-tcp.socket |
| +ss -ltnH 'sport = :3310' | wc -l |
| +``` |
| + |
| +Expected from the first print: `active`, `failed`, then `inactive` or `failed`, then `0`. Expected from the second: `active` three times, then `1`. |
| + |
| +**3. The scan protocol works through the proxy, from the console's box.** Run this in bash in an SSM session on the console box. `/dev/tcp` is a bash feature. It streams the EICAR test string, a harmless standard test file that every engine flags, to the scanner. |
| + |
| +```bash |
| +exec 3<>/dev/tcp/<scanner-private-ip>/3310 |
| +printf 'zINSTREAM\0\0\0\0\104X5O!P%%@AP[4\\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*\0\0\0\0' >&3 |
| +timeout 20 head -c 200 <&3 | tr -d '\0'; echo |
| +``` |
| + |
| +`\0\0\0\104` is the 68-byte chunk length (octal 104 is 68). The four zero bytes after the payload end the stream. Expected: `stream: Eicar-Test-Signature FOUND`. Also confirm the fence from a box that is not the console's: the same connect attempt should time out. |
| + |
| +**4. The box comes back from a reboot with the port open.** First boot cannot show this, because `basic.target` and `sockets.target` are already reached by the time the init script runs. A systemd ordering cycle would only appear on a later boot. |
| + |
| +```sh |
| +aws ec2 reboot-instances --instance-ids <scanner-instance-id> |
| +``` |
| + |
| +Wait five minutes, then in an SSM session on the box: |
| + |
| +```sh |
| +systemctl is-active clamav-daemon.service scanner-selftest.service scanner-tcp.socket |
| +ss -ltnH 'sport = :3310' | wc -l |
| +journalctl -b --no-pager | grep -ci "ordering cycle" |
| +systemd-analyze verify /etc/systemd/system/scanner-selftest.service /etc/systemd/system/scanner-tcp.socket /etc/systemd/system/scanner-tcp.service 2>&1 | grep -v '^$' | head |
| +``` |
| + |
| +Expected: `active` three times, `1`, `0`, and nothing from `systemd-analyze verify` about these three units. |
| + |
| +## Sizing |
| + |
| +The default is `t4g.medium`. clamd holds about 1.5 GB of signatures in memory, and about twice that for a short time while it loads a fresh set. A `t4g.small` or smaller cannot hold two databases at once. `tofu plan` refuses that pairing unless you set `concurrent_database_reload = false`, which pauses scanning during each reload. |
| + |
| +clamd runs under a cgroup ceiling (`memory_max`, default `3G`, with a throttle at `memory_high`, default `2560M`). Under memory pressure the kernel kills clamd, never the SSM agent, which is the only way onto the box. systemd restarts clamd, and the self-test re-runs before the port reopens. The root volume is 20 GiB, encrypted gp3. |
| + |
| +## Network |
| + |
| +The box needs a route out. It downloads packages, the SSM agent (from the regional bucket for `region`) and signature updates. With `subnet_id` null, the module uses the default subnet of the default VPC, and refuses at plan time unless the client security group is in the default VPC. Otherwise pass a `subnet_id` in the client group's VPC. Egress is open. There is no ingress except TCP 3310 from the client group. |
| + |
| +## Variables and outputs |
| + |
| +Variables are described in `variables.tf`. Outputs: `clamd_addr`, `stream_max_bytes`, `instance_id`, `security_group_id`. |
diff --git a/infra/modules/scanner-host/files/scanner-selftest b/infra/modules/scanner-host/files/scanner-selftest| new file mode 100755 |
| index 000000000..f9affe585 |
| --- /dev/null |
| +++ b/infra/modules/scanner-host/files/scanner-selftest |
| @@ -0,0 +1,146 @@ |
| +#!/bin/bash |
| +# 🤖 scanner-selftest: prove, on THIS engine, that clamd never answers OK for |
| +# an archive it could only partly scan. |
| +# |
| +# scanner-selftest.service runs this every time clamd starts, and the TCP |
| +# listener is bound to that unit, so a failure here keeps port 3310 shut. It |
| +# talks to clamd on its unix socket: the port it guards is not open yet. |
| +# |
| +# Why it exists: with ClamAV's packaged limits, a zip member that expands past |
| +# MaxFileSize is trimmed and the trimmed copy is scanned and answered OK, with |
| +# no limit raised. The box is configured so that cannot happen (both limits |
| +# equal, AlertExceedsMax on). This checks that the configuration still does |
| +# what it did when it was chosen, on whatever version is installed today. |
| +# |
| +# Fixtures are sized for a 100 MiB scan limit. Members are written from a temp |
| +# file so their zip headers are ordinary; streamed members get zip64 headers, |
| +# which clamd does not open at all. |
| +set -u |
| +sock=/run/clamav/clamd.ctl |
| + |
| +if [ "${SCANNER_SELFTEST_FORCE_FAIL:-}" = 1 ]; then |
| + echo "scanner-selftest: FAIL (forced, for the acceptance check)" |
| + exit 1 |
| +fi |
| + |
| +d=$(mktemp -d /var/tmp/scanner-selftest.XXXXXX) || exit 1 |
| +trap 'rm -rf "$d"' EXIT |
| +echo "LocalSocket $sock" > "$d/clamdscan.conf" |
| + |
| +python3 - "$d" <<'PY' || { echo "scanner-selftest: FAIL (could not build fixtures)"; exit 1; } |
| +import io, gzip, os, random, sys, tarfile, zipfile |
| +d = sys.argv[1] |
| +fx = os.path.join(d, 'fx') |
| +os.mkdir(fx) |
| +EICAR = rb'X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*' |
| + |
| +def block(seed): |
| + # 1 MiB: a 16 KiB random block repeated. It compresses about 140 to 1 and |
| + # differs per seed, so clamd's clean cache cannot wave a member through. |
| + r = random.Random(seed) |
| + return bytes(r.getrandbits(8) for _ in range(16384)) * 64 |
| + |
| +def bigfile(name, mb, seed): |
| + p = os.path.join(d, name) |
| + blk = block(seed) |
| + with open(p, 'wb') as f: |
| + for _ in range(mb): |
| + f.write(blk) |
| + return p |
| + |
| +def mkzip(name, build): |
| + with zipfile.ZipFile(os.path.join(fx, name), 'w', zipfile.ZIP_DEFLATED) as z: |
| + build(z) |
| + |
| +open(os.path.join(fx, '01-clean.txt'), 'wb').write(b'hello world\n') |
| +open(os.path.join(fx, '02-eicar.txt'), 'wb').write(EICAR) |
| +big = bigfile('big.bin', 500, 1) |
| +mid = bigfile('mid.bin', 60, 5) |
| +mkzip('03-zip-one-500.zip', lambda z: z.write(big, 'big.bin')) |
| +with zipfile.ZipFile(os.path.join(d, 'inner.zip'), 'w', zipfile.ZIP_DEFLATED) as z: |
| + z.write(big, 'big.bin') |
| +mkzip('04-zip-nested-500.zip', lambda z: z.write(os.path.join(d, 'inner.zip'), 'inner.zip')) |
| +def many(z): |
| + for i in range(30): |
| + z.write(bigfile('m.bin', 20, 100 + i), 'm%02d.bin' % i) |
| +mkzip('05-zip-30x20.zip', many) |
| +with gzip.open(os.path.join(fx, '06-gzip-500.gz'), 'wb') as g: |
| + blk = block(2) |
| + for _ in range(500): |
| + g.write(blk) |
| +with tarfile.open(os.path.join(fx, '07-targz-5x40.tar.gz'), 'w:gz') as t: |
| + for i in range(5): |
| + data = block(200 + i) * 40 |
| + ti = tarfile.TarInfo('t%d.bin' % i) |
| + ti.size = len(data) |
| + t.addfile(ti, io.BytesIO(data)) |
| +def big_then_eicar(z): |
| + z.write(big, 'big.bin') |
| + z.writestr('eicar.com', EICAR) |
| +mkzip('08-zip-500-then-eicar.zip', big_then_eicar) |
| +def mid_then_eicar(z): |
| + z.write(mid, 'mid.bin') |
| + z.writestr('eicar.com', EICAR) |
| +mkzip('09-zip-60-then-eicar.zip', mid_then_eicar) |
| +mkzip('10-zip-one-60.zip', lambda z: z.write(mid, 'mid.bin')) |
| +PY |
| + |
| +# What each fixture must answer. "limit" is the one that matters: an OK there |
| +# is a truncated scan signed clean. |
| +want() { |
| + case $1 in |
| + 01 | 10) echo ok ;; |
| + 02 | 09) echo infected ;; |
| + *) echo limit ;; |
| + esac |
| +} |
| + |
| +# >>> classify |
| +# What a reply says: ok, infected, limit, timeout or error. The signature is |
| +# read as internal/clamd reads it: drop the trailing " FOUND", keep what follows |
| +# the last ": " (clamdscan puts the file path there, not "stream"). A limit is |
| +# Heuristics.Limits.Exceeded itself or a name under it, dot included, so a |
| +# lookalike such as Win.Heuristics.Limits.Exceeded.Fake is malware. MaxScanTime |
| +# is a timeout: it says the engine ran out of time, not that a limit held, so |
| +# it never matches what a fixture wants. |
| +classify() { |
| + local sig |
| + case $1 in |
| + *" FOUND") |
| + sig=${1% FOUND} |
| + sig=${sig##*: } |
| + case $sig in |
| + Heuristics.Limits.Exceeded.MaxScanTime) echo timeout ;; |
| + Heuristics.Limits.Exceeded | Heuristics.Limits.Exceeded.*) echo limit ;; |
| + *) echo infected ;; |
| + esac |
| + ;; |
| + *": OK") echo ok ;; |
| + *) echo error ;; |
| + esac |
| +} |
| +# <<< classify |
| + |
| +fail=0 |
| +for f in "$d"/fx/*; do |
| + n=$(basename "$f") |
| + exp=$(want "${n%%-*}") |
| + reply=$(clamdscan --config-file="$d/clamdscan.conf" --stream --no-summary "$f" 2>&1 | tail -1) |
| + got=$(classify "$reply") |
| + if [ "$got" = "$exp" ]; then |
| + echo "scanner-selftest: pass $n ($got)" |
| + elif [ "$got" = timeout ]; then |
| + echo "scanner-selftest: FAIL $n timeout, not a verdict (the engine is too slow or overloaded): $reply" |
| + fail=1 |
| + else |
| + echo "scanner-selftest: FAIL $n want $exp, got $got: $reply" |
| + fail=1 |
| + fi |
| +done |
| + |
| +if [ $fail -eq 0 ]; then |
| + echo "scanner-selftest: PASS" |
| +else |
| + echo "scanner-selftest: FAIL" |
| +fi |
| +exit $fail |
diff --git a/infra/modules/scanner-host/files/scanner-selftest.service b/infra/modules/scanner-host/files/scanner-selftest.service| new file mode 100644 |
| index 000000000..b4dc8e677 |
| --- /dev/null |
| +++ b/infra/modules/scanner-host/files/scanner-selftest.service |
| @@ -0,0 +1,23 @@ |
| +# 🤖 Runs the self-test against the clamd that is running now. BindsTo makes it |
| +# go inactive the moment clamd stops or restarts, and the TCP listener is bound |
| +# to THIS unit, so the port follows the test, not the boot. |
| +[Unit] |
| +Description=Prove this clamd never answers OK for an over-limit archive |
| +BindsTo=clamav-daemon.service |
| +After=clamav-daemon.service |
| +# Restarting this unit by hand after a failure reopens the port on a pass. |
| +# Wants= adds no ordering; the socket orders itself after this unit. |
| +Wants=scanner-tcp.socket |
| + |
| +[Service] |
| +Type=oneshot |
| +RemainAfterExit=yes |
| +ExecStart=/usr/local/sbin/scanner-selftest |
| +TimeoutStartSec=1800 |
| +# A killed test leaves its fixtures (about 1.2 GB) behind in /var/tmp. A |
| +# private /var/tmp goes with the unit. clamd never reads the fixture paths: |
| +# every one is streamed. |
| +PrivateTmp=yes |
| + |
| +[Install] |
| +WantedBy=multi-user.target |
diff --git a/infra/modules/scanner-host/files/scanner-tcp.service b/infra/modules/scanner-host/files/scanner-tcp.service| new file mode 100644 |
| index 000000000..65de5601a |
| --- /dev/null |
| +++ b/infra/modules/scanner-host/files/scanner-tcp.service |
| @@ -0,0 +1,15 @@ |
| +# 🤖 Carries each TCP connection to clamd's unix socket. Bound to the self-test |
| +# like the socket is, so connections in flight are cut when the test's unit |
| +# goes inactive. |
| +[Unit] |
| +Description=Proxy TCP 3310 to clamd's unix socket |
| +Requires=scanner-selftest.service |
| +After=scanner-selftest.service |
| +BindsTo=scanner-selftest.service |
| + |
| +[Service] |
| +ExecStart=/usr/lib/systemd/systemd-socket-proxyd /run/clamav/clamd.ctl |
| +DynamicUser=yes |
| +SupplementaryGroups=clamav |
| +PrivateTmp=yes |
| +NoNewPrivileges=yes |
diff --git a/infra/modules/scanner-host/files/scanner-tcp.socket b/infra/modules/scanner-host/files/scanner-tcp.socket| new file mode 100644 |
| index 000000000..ace642994 |
| --- /dev/null |
| +++ b/infra/modules/scanner-host/files/scanner-tcp.socket |
| @@ -0,0 +1,26 @@ |
| +# 🤖 clamd on TCP, for the console. Open only behind a passed self-test. |
| +# |
| +# DefaultDependencies=no is load-bearing. A socket unit is ordered before |
| +# sockets.target by default, sockets.target before basic.target, and every |
| +# service (the self-test included) after basic.target. A socket that must also |
| +# come AFTER the self-test closes that loop, and systemd breaks an ordering |
| +# cycle by dropping a job, which could leave this port shut after a reboot. |
| +# So it opts out, restores what it still needs, and is started by |
| +# multi-user.target and by clamd's Wants=, never by sockets.target. |
| +[Unit] |
| +Description=clamd on TCP, open only behind a passed self-test |
| +DefaultDependencies=no |
| +After=sysinit.target |
| +Requires=scanner-selftest.service |
| +After=scanner-selftest.service |
| +BindsTo=scanner-selftest.service |
| +Conflicts=shutdown.target |
| +Before=shutdown.target |
| + |
| +[Socket] |
| +ListenStream=@PRIVATE_IP@:3310 |
| +# Binds even if the private address is not up yet at boot. |
| +FreeBind=yes |
| + |
| +[Install] |
| +WantedBy=multi-user.target |
diff --git a/infra/modules/scanner-host/main.tf b/infra/modules/scanner-host/main.tf| new file mode 100644 |
| index 000000000..0fa89e1b3 |
| --- /dev/null |
| +++ b/infra/modules/scanner-host/main.tf |
| @@ -0,0 +1,157 @@ |
| +# 🤖 A CARLOS scanner host: one box running clamd for a deployment's console. |
| +# |
| +# The console reads each object from S3 and streams it here. This box reads |
| +# nothing for itself: its role has Session Manager and nothing else, so a |
| +# compromised clamd cannot fetch a tenant object or a key. |
| +# |
| +# What makes it more than "install clamav" is in scanner-init.sh.tftpl and |
| +# files/: TCP 3310 is open only while a self-test has passed against the clamd |
| +# process running now. See README.md. |
| + |
| +data "aws_security_group" "client" { |
| + id = var.client_security_group_id |
| +} |
| + |
| +data "aws_vpc" "client" { |
| + id = data.aws_security_group.client.vpc_id |
| +} |
| + |
| +locals { |
| + # Fixed in this release. The self-test's fixtures and expected answers are |
| + # sized for it; making it a variable means deriving both. |
| + scan_limit = "100M" |
| + |
| + small_box = can(regex("^t4g\\.(nano|micro|small)$", var.instance_type)) |
| + |
| + init_script = templatefile("${path.module}/scanner-init.sh.tftpl", { |
| + region = var.region |
| + stream_max_mib = var.stream_max_bytes / 1048576 |
| + scan_limit = local.scan_limit |
| + memory_max = var.memory_max |
| + memory_high = var.memory_high |
| + concurrent_reload = var.concurrent_database_reload ? "yes" : "no" |
| + selftest_b64 = base64encode(file("${path.module}/files/scanner-selftest")) |
| + selftest_unit_b64 = base64encode(file("${path.module}/files/scanner-selftest.service")) |
| + tcp_socket_b64 = base64encode(file("${path.module}/files/scanner-tcp.socket")) |
| + tcp_service_b64 = base64encode(file("${path.module}/files/scanner-tcp.service")) |
| + }) |
| +} |
| + |
| +resource "aws_security_group" "scanner" { |
| + name = var.name |
| + description = "clamd, reachable from the console box only" |
| + vpc_id = data.aws_security_group.client.vpc_id |
| + |
| + # The whole fence. clamd authenticates nobody, so nothing but the console's |
| + # own group may reach it, and never a CIDR. |
| + ingress { |
| + description = "clamd INSTREAM from the console" |
| + from_port = 3310 |
| + to_port = 3310 |
| + protocol = "tcp" |
| + security_groups = [var.client_security_group_id] |
| + } |
| + |
| + # Signatures (freshclam) and packages (apt). No SSH ingress exists: admin |
| + # access is SSM Session Manager. |
| + egress { |
| + from_port = 0 |
| + to_port = 0 |
| + protocol = "-1" |
| + cidr_blocks = ["0.0.0.0/0"] |
| + } |
| + |
| + tags = { Name = var.name } |
| +} |
| + |
| +resource "aws_iam_role" "scanner" { |
| + name = var.name |
| + assume_role_policy = jsonencode({ |
| + Version = "2012-10-17" |
| + Statement = [{ |
| + Effect = "Allow" |
| + Principal = { Service = "ec2.amazonaws.com" } |
| + Action = "sts:AssumeRole" |
| + }] |
| + }) |
| +} |
| + |
| +resource "aws_iam_role_policy_attachment" "scanner_ssm" { |
| + role = aws_iam_role.scanner.name |
| + policy_arn = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore" |
| +} |
| + |
| +# AmazonSSMManagedInstanceCore grants ssm:GetParameter* on "*". Left alone, |
| +# this box could read the deployment's scan tag key and its release key. An |
| +# explicit Deny beats the managed Allow and names only the parameter-read |
| +# actions, so Session Manager is untouched. Same shape as region-host's |
| +# host_ssm_parameter_scope, with no parameter allowed at all. |
| +resource "aws_iam_role_policy" "no_parameters" { |
| + name = "${var.name}-no-parameters" |
| + role = aws_iam_role.scanner.id |
| + policy = jsonencode({ |
| + Version = "2012-10-17" |
| + Statement = [ |
| + { |
| + Effect = "Deny" |
| + Action = [ |
| + "ssm:GetParameter", |
| + "ssm:GetParameters", |
| + "ssm:GetParameterHistory", |
| + "ssm:GetParametersByPath", |
| + ] |
| + Resource = "*" |
| + }, |
| + ] |
| + }) |
| +} |
| + |
| +resource "aws_iam_instance_profile" "scanner" { |
| + name = var.name |
| + role = aws_iam_role.scanner.name |
| +} |
| + |
| +resource "aws_instance" "scanner" { |
| + ami = var.ami_id |
| + instance_type = var.instance_type |
| + subnet_id = var.subnet_id |
| + vpc_security_group_ids = [aws_security_group.scanner.id] |
| + iam_instance_profile = aws_iam_instance_profile.scanner.name |
| + |
| + # Gzipped: the script carries the self-test and three units, and EC2 caps |
| + # user_data at 16 KiB. cloud-init unpacks it. |
| + user_data_base64 = base64gzip(local.init_script) |
| + user_data_replace_on_change = false |
| + |
| + metadata_options { |
| + http_endpoint = "enabled" |
| + http_tokens = "required" |
| + http_put_response_hop_limit = 1 |
| + } |
| + |
| + root_block_device { |
| + volume_size = 20 |
| + volume_type = "gp3" |
| + encrypted = true |
| + } |
| + |
| + tags = { Name = var.name } |
| + |
| + # A newer image or an edited script never replaces the box by itself. |
| + # Changing either, including stream_max_bytes, is an explicit |
| + # `tofu apply -replace=module.<name>.aws_instance.scanner`. A replaced box |
| + # has a new private IP: update the console's CARLOS_CLAMD_ADDR. |
| + lifecycle { |
| + ignore_changes = [ami, user_data, user_data_base64] |
| + |
| + precondition { |
| + condition = var.subnet_id != null || data.aws_vpc.client.default |
| + error_message = "The client security group is not in the default VPC, so the scanner cannot use the default subnet: its security group and its instance would be in different VPCs. Pass subnet_id naming a subnet in the client group's VPC." |
| + } |
| + |
| + precondition { |
| + condition = !(local.small_box && var.concurrent_database_reload) |
| + error_message = "A t4g.small or smaller cannot hold two signature databases at once. Either use t4g.medium, or set concurrent_database_reload = false (scanning pauses during each reload)." |
| + } |
| + } |
| +} |
diff --git a/infra/modules/scanner-host/outputs.tf b/infra/modules/scanner-host/outputs.tf| new file mode 100644 |
| index 000000000..d734de6a1 |
| --- /dev/null |
| +++ b/infra/modules/scanner-host/outputs.tf |
| @@ -0,0 +1,19 @@ |
| +output "clamd_addr" { |
| + description = "Where the console reaches clamd: set it as CARLOS_CLAMD_ADDR. A private address, reachable only from the client security group. It changes if the box is replaced." |
| + value = "${aws_instance.scanner.private_ip}:3310" |
| +} |
| + |
| +output "stream_max_bytes" { |
| + description = "clamd's StreamMaxLength in bytes: set it as CARLOS_CLAMD_MAX_BYTES on the console." |
| + value = var.stream_max_bytes |
| +} |
| + |
| +output "instance_id" { |
| + description = "EC2 instance id of the scanner box, for SSM." |
| + value = aws_instance.scanner.id |
| +} |
| + |
| +output "security_group_id" { |
| + description = "Security group id of the scanner box." |
| + value = aws_security_group.scanner.id |
| +} |
diff --git a/infra/modules/scanner-host/scanner-init.sh.tftpl b/infra/modules/scanner-host/scanner-init.sh.tftpl| new file mode 100644 |
| index 000000000..2d8bad14a |
| --- /dev/null |
| +++ b/infra/modules/scanner-host/scanner-init.sh.tftpl |
| @@ -0,0 +1,174 @@ |
| +#!/bin/bash |
| +# 🤖 First-boot init for a CARLOS scanner host (Debian arm64). Rendered by |
| +# infra/modules/scanner-host. Each step says where it got to on the serial |
| +# console, so `aws ec2 get-console-output` answers "how far did it get". |
| +# |
| +# The order is the design: |
| +# 1. SSM agent, because with no SSH it is the only way in to fix the rest. |
| +# 2. Packages installed but NOT started. |
| +# 3. Config, the self-test and the gate units. |
| +# 4. First signatures, alone, before anything else touches the database. |
| +# 5. Start: clamd, then the self-test, then (only on a pass) the port. |
| +# 6. Verify from the outside of the gate, and say READY or FAILED. |
| +set -euo pipefail |
| + |
| +# Logging must never be what stops the script: a failed write inside say() |
| +# would exit under set -e without the ERR trap (functions do not inherit it), |
| +# so without a FAILED marker. stdout goes to cloud-init's log; the serial |
| +# console copy is best effort. |
| +say() { |
| + echo "scanner-init: $*" || true |
| + { echo "scanner-init: $*" > /dev/console; } 2>/dev/null || true |
| +} |
| +# A failure in step 2 could leave policy-rc.d behind, and then nobody fixing |
| +# the box over SSM could start a service. Remove it on every way out. |
| +fail() { rm -f /usr/sbin/policy-rc.d || true; say "SCANNER-HOST FAILED: $*"; exit 1; } |
| +trap 'fail "line $LINENO"' ERR |
| +export DEBIAN_FRONTEND=noninteractive |
| + |
| +# ---- 1. the way in --------------------------------------------------------- |
| +say "1/6 SSM agent" |
| +apt-get update -q |
| +apt-get install -qy curl unattended-upgrades |
| +curl -fsSL -o /var/tmp/amazon-ssm-agent.deb \ |
| + "https://s3.${region}.amazonaws.com/amazon-ssm-${region}/latest/debian_arm64/amazon-ssm-agent.deb" |
| +dpkg -i /var/tmp/amazon-ssm-agent.deb |
| +rm -f /var/tmp/amazon-ssm-agent.deb |
| +systemctl enable --now amazon-ssm-agent |
| +systemctl enable --now unattended-upgrades |
| + |
| +# ---- 2. packages, not started ---------------------------------------------- |
| +# Debian's postinst starts clamav-freshclam and the daemon. Neither may run |
| +# before step 3 has configured it, and freshclam's service would fight the |
| +# foreground run in step 4 for the database lock. |
| +say "2/6 clamav packages" |
| +printf '#!/bin/sh\nexit 101\n' > /usr/sbin/policy-rc.d |
| +chmod +x /usr/sbin/policy-rc.d |
| +apt-get install -qy clamav-daemon clamav-freshclam clamdscan python3 iproute2 |
| +rm -f /usr/sbin/policy-rc.d |
| + |
| +# ---- 3. config, self-test, gate -------------------------------------------- |
| +say "3/6 config and gate" |
| +conf=/etc/clamav/clamd.conf |
| +[ -f "$conf" ] || fail "the package wrote no $conf" |
| +# With debconf on (the package default), the postinst regenerates clamd.conf |
| +# on every configure, unattended upgrades included. The file it generated at |
| +# install is the base; from here on it is ours. With debconf off, the postinst |
| +# leaves an existing clamd.conf alone. |
| +echo 'clamav-daemon clamav-daemon/debconf boolean false' | debconf-set-selections |
| +# clamd has no include directory: replace each key in place. The ERR trap is |
| +# not inherited by functions, so a failure in here reports itself. |
| +setkey() { |
| + { sed -i "/^$1[[:space:]]/d" "$conf" && echo "$1 $2" >> "$conf"; } \ |
| + || fail "could not set $1 in $conf" |
| +} |
| +# In M, never bytes: if the postinst ever does regenerate this file, it reads |
| +# the number as megabytes and appends M (26214400 would become 25 TiB). |
| +setkey StreamMaxLength ${stream_max_mib}M |
| +setkey AlertExceedsMax yes |
| +setkey MaxFileSize ${scan_limit} |
| +setkey MaxScanSize ${scan_limit} |
| +setkey ConcurrentDatabaseReload ${concurrent_reload} |
| +# Debian 13 mounts /tmp as tmpfs: extraction of up to the scan limit per scan |
| +# belongs on disk, not in memory. |
| +setkey TemporaryDirectory /var/tmp |
| +# clamd itself never listens on TCP. The gated proxy does. |
| +sed -i '/^TCPSocket[[:space:]]/d;/^TCPAddr[[:space:]]/d' "$conf" |
| + |
| +# Debian's AppArmor profile for clamd is enforced and allows temp files under |
| +# /tmp only, and clamd writes every INSTREAM to a temp file. Without this, |
| +# every scan errors and the self-test fails. The profile includes this local |
| +# file. It is loaded here, before clamd first starts. |
| +if [ -e /etc/apparmor.d/usr.sbin.clamd ]; then |
| + mkdir -p /etc/apparmor.d/local |
| + cat > /etc/apparmor.d/local/usr.sbin.clamd <<'EOF' |
| +# scanner-host: clamd's TemporaryDirectory is /var/tmp. |
| +/var/tmp/ rw, |
| +/var/tmp/** krw, |
| +EOF |
| + if command -v apparmor_parser >/dev/null && [ -d /sys/kernel/security/apparmor ]; then |
| + apparmor_parser -r /etc/apparmor.d/usr.sbin.clamd |
| + fi |
| +fi |
| + |
| +echo '${selftest_b64}' | base64 -d > /usr/local/sbin/scanner-selftest |
| +chmod 0755 /usr/local/sbin/scanner-selftest |
| +echo '${selftest_unit_b64}' | base64 -d > /etc/systemd/system/scanner-selftest.service |
| +echo '${tcp_service_b64}' | base64 -d > /etc/systemd/system/scanner-tcp.service |
| + |
| +tok=$(curl -fsS -X PUT http://169.254.169.254/latest/api/token \ |
| + -H 'X-aws-ec2-metadata-token-ttl-seconds: 300') |
| +ip=$(curl -fsS -H "X-aws-ec2-metadata-token: $tok" \ |
| + http://169.254.169.254/latest/meta-data/local-ipv4) |
| +case $ip in |
| + [0-9]*.[0-9]*.[0-9]*.[0-9]*) ;; |
| + *) fail "no private IPv4 from instance metadata: '$ip'" ;; |
| +esac |
| +echo '${tcp_socket_b64}' | base64 -d | sed "s/@PRIVATE_IP@/$ip/" \ |
| + > /etc/systemd/system/scanner-tcp.socket |
| +grep -q "^ListenStream=$ip:3310\$" /etc/systemd/system/scanner-tcp.socket \ |
| + || fail "the listener is not bound to the private address" |
| + |
| +mkdir -p /etc/systemd/system/clamav-daemon.service.d |
| +cat > /etc/systemd/system/clamav-daemon.service.d/10-scanner.conf <<EOF |
| +# Every start of clamd, an automatic restart included, asks for the gate again. |
| +[Unit] |
| +Wants=scanner-tcp.socket |
| + |
| +# The kernel kills clamd under memory pressure, never the SSM agent. |
| +[Service] |
| +MemoryMax=${memory_max} |
| +MemoryHigh=${memory_high} |
| +Restart=on-failure |
| + |
| +# The same two lines as Debian's extend.conf drop-in, which the postinst moves |
| +# aside on its first configure run with debconf off: an upgrade or a |
| +# dpkg-reconfigure, whichever comes first. |
| +ExecStartPre=-/bin/mkdir -p /run/clamav |
| +ExecStartPre=/bin/chown clamav /run/clamav |
| +EOF |
| +systemctl daemon-reload |
| + |
| +# ---- 4. first signatures --------------------------------------------------- |
| +say "4/6 first signatures" |
| +# NotifyClamd would complain that no clamd is running yet. It is not. |
| +sed '/^NotifyClamd/d' /etc/clamav/freshclam.conf > /run/freshclam-first.conf |
| +# ClamAV's mirror often rate-limits a new address. Three tries, a minute apart. |
| +tries=1 |
| +until freshclam --config-file=/run/freshclam-first.conf; do |
| + [ $tries -lt 3 ] || fail "first freshclam failed 3 times, 60s apart (rate-limited by ClamAV's mirror?)" |
| + say "freshclam try $tries failed, retrying in 60s" |
| + tries=$((tries + 1)) |
| + sleep 60 |
| +done |
| +rm -f /run/freshclam-first.conf |
| +for db in main daily bytecode; do |
| + ls /var/lib/clamav/$db.c[vl]d >/dev/null 2>&1 || fail "no $db signature database after freshclam" |
| +done |
| +systemctl enable --now clamav-freshclam |
| + |
| +# ---- 5. start, in the order the units enforce ------------------------------ |
| +say "5/6 clamd, self-test, listener (several minutes)" |
| +systemctl enable clamav-daemon.service scanner-selftest.service scanner-tcp.socket |
| +systemctl start clamav-daemon.socket clamav-daemon.service |
| +# Blocks until the self-test has finished. Fails if it failed. |
| +systemctl start scanner-tcp.socket \ |
| + || fail "self-test or listener did not start: $(systemctl is-active clamav-daemon.service scanner-selftest.service scanner-tcp.socket | tr '\n' ' ')" |
| + |
| +# ---- 6. verify from outside the gate --------------------------------------- |
| +say "6/6 verify" |
| +[ "$(systemctl is-active scanner-selftest.service)" = active ] || fail "self-test is not active" |
| +[ "$(systemctl is-active scanner-tcp.socket)" = active ] || fail "listener is not active" |
| +bound=$(ss -ltnH 'sport = :3310' | awk '{print $4}' | sort -u | tr '\n' ' ') |
| +[ "$bound" = "$ip:3310 " ] || fail "3310 is bound to '$bound', want only $ip:3310" |
| +exec 3<>"/dev/tcp/$ip/3310" |
| +printf 'zVERSION\0' >&3 |
| +ver=$(timeout 20 head -c 200 <&3 | tr -d '\0') |
| +exec 3>&- |
| +case $ver in |
| + ClamAV*) ;; |
| + *) fail "no VERSION answer through the listener: '$ver'" ;; |
| +esac |
| +mkdir -p /var/lib/scanner-host |
| +echo "$ver" > /var/lib/scanner-host/ready |
| +say "SCANNER-HOST READY $ver" |
diff --git a/infra/modules/scanner-host/tests/scanner_host.tftest.hcl b/infra/modules/scanner-host/tests/scanner_host.tftest.hcl| new file mode 100644 |
| index 000000000..18047639f |
| --- /dev/null |
| +++ b/infra/modules/scanner-host/tests/scanner_host.tftest.hcl |
| @@ -0,0 +1,247 @@ |
| +# Plan-time tests. mock_provider: no account, no credentials. |
| +mock_provider "aws" {} |
| + |
| +override_data { |
| + target = data.aws_security_group.client |
| + values = { vpc_id = "vpc-0aaaaaaaaaaaaaaaa" } |
| +} |
| + |
| +override_data { |
| + target = data.aws_vpc.client |
| + values = { default = true } |
| +} |
| + |
| +variables { |
| + name = "carlos-test-scanner" |
| + ami_id = "ami-0123456789abcdef0" |
| + client_security_group_id = "sg-0123456789abcdef0" |
| + region = "eu-west-1" |
| +} |
| + |
| +run "the_fence" { |
| + command = plan |
| + assert { |
| + condition = length(aws_security_group.scanner.ingress) == 1 |
| + error_message = "exactly one way in" |
| + } |
| + assert { |
| + condition = alltrue([ |
| + for i in aws_security_group.scanner.ingress : |
| + i.from_port == 3310 && i.to_port == 3310 && i.protocol == "tcp" |
| + && i.security_groups == toset(["sg-0123456789abcdef0"]) |
| + && try(length(i.cidr_blocks), 0) == 0 && try(length(i.ipv6_cidr_blocks), 0) == 0 |
| + ]) |
| + error_message = "3310 from the client security group only, never a CIDR" |
| + } |
| + assert { |
| + condition = aws_security_group.scanner.vpc_id == "vpc-0aaaaaaaaaaaaaaaa" |
| + error_message = "the group lands in the client group's VPC" |
| + } |
| +} |
| + |
| +run "the_box" { |
| + command = plan |
| + assert { |
| + condition = aws_instance.scanner.metadata_options[0].http_tokens == "required" |
| + error_message = "IMDSv2 only" |
| + } |
| + assert { |
| + condition = aws_instance.scanner.root_block_device[0].encrypted |
| + error_message = "encrypted root volume" |
| + } |
| + assert { |
| + condition = aws_instance.scanner.instance_type == "t4g.medium" |
| + error_message = "t4g.medium by default" |
| + } |
| + assert { |
| + # base64 is 4 characters per 3 bytes, so 21800 characters is under 16 KiB. |
| + condition = length(base64gzip(local.init_script)) < 21800 |
| + error_message = "EC2 refuses user_data over 16 KiB, and the script is sent gzipped to stay well under it" |
| + } |
| +} |
| + |
| +run "the_role_cannot_read_parameters" { |
| + command = plan |
| + assert { |
| + condition = jsondecode(aws_iam_role_policy.no_parameters.policy).Statement[0].Effect == "Deny" |
| + error_message = "an explicit Deny, because the managed SSM policy allows GetParameter* on *" |
| + } |
| + assert { |
| + condition = toset(jsondecode(aws_iam_role_policy.no_parameters.policy).Statement[0].Action) == toset([ |
| + "ssm:GetParameter", "ssm:GetParameters", "ssm:GetParameterHistory", "ssm:GetParametersByPath", |
| + ]) |
| + error_message = "all four parameter-read actions" |
| + } |
| + assert { |
| + condition = jsondecode(aws_iam_role_policy.no_parameters.policy).Statement[0].Resource == "*" |
| + error_message = "on every parameter: this box is allowed none" |
| + } |
| +} |
| + |
| +run "the_engine_config" { |
| + command = plan |
| + assert { |
| + condition = strcontains(local.init_script, "setkey MaxFileSize 100M") && strcontains(local.init_script, "setkey MaxScanSize 100M") |
| + error_message = "both limits 100M: equal is what makes a trimmed member trip the scan budget" |
| + } |
| + assert { |
| + condition = strcontains(local.init_script, "setkey AlertExceedsMax yes") |
| + error_message = "limits must alert" |
| + } |
| + assert { |
| + condition = strcontains(local.init_script, "setkey StreamMaxLength 25M") |
| + error_message = "StreamMaxLength is stream_max_bytes, written in M so a debconf rewrite cannot turn it into 25 TiB" |
| + } |
| + assert { |
| + condition = strcontains(local.init_script, "echo 'clamav-daemon clamav-daemon/debconf boolean false' | debconf-set-selections") |
| + error_message = "package upgrades never regenerate clamd.conf" |
| + } |
| + assert { |
| + condition = alltrue([ |
| + strcontains(local.init_script, "cat > /etc/apparmor.d/local/usr.sbin.clamd <<'EOF'"), |
| + strcontains(local.init_script, "\n/var/tmp/ rw,\n/var/tmp/** krw,\n"), |
| + strcontains(local.init_script, "apparmor_parser -r /etc/apparmor.d/usr.sbin.clamd"), |
| + strcontains(local.init_script, "setkey TemporaryDirectory /var/tmp"), |
| + ]) |
| + error_message = "clamd's AppArmor profile allows its TemporaryDirectory, /var/tmp, or every scan errors" |
| + } |
| + assert { |
| + condition = strcontains(local.init_script, "sed -i '/^TCPSocket[[:space:]]/d;/^TCPAddr[[:space:]]/d' \"$conf\"") |
| + error_message = "any TCPSocket or TCPAddr line the package wrote is deleted" |
| + } |
| + assert { |
| + condition = !strcontains(local.init_script, "setkey TCPSocket") && !strcontains(local.init_script, "setkey TCPAddr") |
| + error_message = "clamd itself never listens on TCP; the gated proxy does" |
| + } |
| + assert { |
| + condition = output.stream_max_bytes == 26214400 |
| + error_message = "the console's CARLOS_CLAMD_MAX_BYTES comes from this output" |
| + } |
| +} |
| + |
| +run "small_box_with_concurrent_reload_is_refused" { |
| + command = plan |
| + variables { |
| + instance_type = "t4g.small" |
| + } |
| + expect_failures = [aws_instance.scanner] |
| +} |
| + |
| +run "small_box_without_concurrent_reload_is_fine" { |
| + command = plan |
| + variables { |
| + instance_type = "t4g.small" |
| + concurrent_database_reload = false |
| + } |
| + assert { |
| + condition = strcontains(local.init_script, "setkey ConcurrentDatabaseReload no") |
| + error_message = "the reload setting reaches clamd.conf" |
| + } |
| +} |
| + |
| +run "stream_limit_too_small_for_the_fixtures" { |
| + command = plan |
| + variables { |
| + stream_max_bytes = 1048576 |
| + } |
| + expect_failures = [var.stream_max_bytes] |
| +} |
| + |
| +run "stream_limit_above_the_scan_budget" { |
| + command = plan |
| + variables { |
| + stream_max_bytes = 209715200 |
| + } |
| + expect_failures = [var.stream_max_bytes] |
| +} |
| + |
| +run "stream_limit_not_a_whole_mib" { |
| + command = plan |
| + variables { |
| + stream_max_bytes = 26214401 |
| + } |
| + expect_failures = [var.stream_max_bytes] |
| +} |
| + |
| +run "the_units" { |
| + command = plan |
| + assert { |
| + condition = alltrue([ |
| + for line in [ |
| + "DefaultDependencies=no", |
| + "Requires=scanner-selftest.service", |
| + "BindsTo=scanner-selftest.service", |
| + "After=scanner-selftest.service", |
| + "ListenStream=@PRIVATE_IP@:3310", |
| + "FreeBind=yes", |
| + ] : strcontains(file("${path.module}/files/scanner-tcp.socket"), "\n${line}\n") |
| + ]) |
| + error_message = "the socket opts out of default dependencies, is bound to the self-test and listens on the private address only" |
| + } |
| + assert { |
| + condition = alltrue([ |
| + for line in [ |
| + "Requires=scanner-selftest.service", |
| + "BindsTo=scanner-selftest.service", |
| + "After=scanner-selftest.service", |
| + ] : strcontains(file("${path.module}/files/scanner-tcp.service"), "\n${line}\n") |
| + ]) |
| + error_message = "the proxy is bound to the self-test, so connections in flight are cut with it" |
| + } |
| + assert { |
| + condition = alltrue([ |
| + for line in [ |
| + "BindsTo=clamav-daemon.service", |
| + "After=clamav-daemon.service", |
| + "Wants=scanner-tcp.socket", |
| + "Type=oneshot", |
| + "RemainAfterExit=yes", |
| + "PrivateTmp=yes", |
| + ] : strcontains(file("${path.module}/files/scanner-selftest.service"), "\n${line}\n") |
| + ]) |
| + error_message = "the self-test follows clamd, asks for the gate, stays active after a pass, and keeps its fixtures in a private /var/tmp" |
| + } |
| + assert { |
| + # Comments may sit between the heredoc line and [Unit]; the Wants= line |
| + # must be the first line of the [Unit] section. |
| + condition = can(regex("10-scanner\\.conf <<EOF\\n(#[^\\n]*\\n)*\\[Unit\\]\\nWants=scanner-tcp\\.socket\\n", local.init_script)) |
| + error_message = "every start of clamd, an automatic restart included, asks for the gate again" |
| + } |
| +} |
| + |
| +run "the_selftest_classifier" { |
| + command = plan |
| + assert { |
| + condition = !strcontains(file("${path.module}/files/scanner-selftest"), "*\"Heuristics.Limits.Exceeded\"*") |
| + error_message = "no substring glob: it accepted MaxScanTime and lookalikes such as Win.Heuristics.Limits.Exceeded.Fake as a limit" |
| + } |
| + assert { |
| + condition = strcontains(file("${path.module}/files/scanner-selftest"), "Heuristics.Limits.Exceeded.MaxScanTime) echo timeout ;;") |
| + error_message = "a scan timeout is its own class, which no fixture wants" |
| + } |
| + assert { |
| + condition = strcontains(file("${path.module}/files/scanner-selftest"), "Heuristics.Limits.Exceeded | Heuristics.Limits.Exceeded.*) echo limit ;;") |
| + error_message = "a limit is the heuristic itself or a name under it, dot included, as internal/clamd reads it" |
| + } |
| +} |
| + |
| +run "logging_cannot_kill_init" { |
| + command = plan |
| + assert { |
| + condition = !strcontains(local.init_script, "tee /dev/console") |
| + error_message = "a failed console write would exit under set -e with no FAILED marker" |
| + } |
| + assert { |
| + condition = strcontains(local.init_script, "{ echo \"scanner-init: $*\" > /dev/console; } 2>/dev/null || true") |
| + error_message = "the console copy of each line is best effort" |
| + } |
| +} |
| + |
| +run "another_vpc_needs_a_subnet" { |
| + command = plan |
| + override_data { |
| + target = data.aws_vpc.client |
| + values = { default = false } |
| + } |
| + expect_failures = [aws_instance.scanner] |
| +} |
diff --git a/infra/modules/scanner-host/variables.tf b/infra/modules/scanner-host/variables.tf| new file mode 100644 |
| index 000000000..3624478cf |
| --- /dev/null |
| +++ b/infra/modules/scanner-host/variables.tf |
| @@ -0,0 +1,63 @@ |
| +variable "name" { |
| + description = "Name for the instance, its security group, its IAM role and its instance profile (e.g. \"carlos-flagship-scanner\")." |
| + type = string |
| +} |
| + |
| +variable "ami_id" { |
| + description = "Debian arm64 AMI. Pass the console box's own image, module.<host>.ami_id, so the two are built from the same base. Ignored after creation: a newer image never replaces the box by itself." |
| + type = string |
| +} |
| + |
| +variable "client_security_group_id" { |
| + description = "Security group of the ONE box allowed to reach clamd: the console's. clamd has no authentication at all, so this group is the whole fence. The scanner's own group is created in this group's VPC." |
| + type = string |
| +} |
| + |
| +variable "region" { |
| + description = "AWS region the box runs in (e.g. \"eu-west-1\"). The SSM agent package is served per region and the box must fetch its own." |
| + type = string |
| +} |
| + |
| +variable "instance_type" { |
| + description = "EC2 instance type (arm64 Graviton). clamd holds about 1.5 GB of signatures resident and briefly about twice that while reloading them, so t4g.medium is the default. A t4g.small works only with concurrent_database_reload = false." |
| + type = string |
| + default = "t4g.medium" |
| +} |
| + |
| +variable "stream_max_bytes" { |
| + description = "Largest object clamd accepts on a stream (StreamMaxLength), in bytes. Set the console's CARLOS_CLAMD_MAX_BYTES to this module's stream_max_bytes output: the two must be equal. Between 8 MiB and 100 MiB: the box's self-test streams fixtures of about 5 MB, and a whole stream must fit the 100 MiB scan budget." |
| + type = number |
| + default = 26214400 |
| + validation { |
| + condition = var.stream_max_bytes >= 8388608 && var.stream_max_bytes <= 104857600 |
| + error_message = "stream_max_bytes must be between 8388608 (8 MiB) and 104857600 (100 MiB)." |
| + } |
| + validation { |
| + condition = var.stream_max_bytes % 1048576 == 0 |
| + error_message = "stream_max_bytes must be a whole number of MiB (a multiple of 1048576), because clamd.conf holds it in M." |
| + } |
| +} |
| + |
| +variable "memory_max" { |
| + description = "clamd's cgroup ceiling (systemd MemoryMax). Under memory pressure the kernel kills clamd, never the SSM agent that is the only way onto the box." |
| + type = string |
| + default = "3G" |
| +} |
| + |
| +variable "memory_high" { |
| + description = "clamd's throttle point (systemd MemoryHigh), about 85% of memory_max." |
| + type = string |
| + default = "2560M" |
| +} |
| + |
| +variable "concurrent_database_reload" { |
| + description = "Keep scanning while freshclam's new signatures load, at the cost of holding both databases in memory at once. False pauses scanning during each reload and fits a smaller box." |
| + type = bool |
| + default = true |
| +} |
| + |
| +variable "subnet_id" { |
| + description = "Subnet to launch in. Null (the default) is the default subnet of the default VPC, and is refused unless the client security group lives in the default VPC. The subnet needs a route to the internet: the box fetches signatures and packages." |
| + type = string |
| + default = null |
| +} |
diff --git a/infra/modules/scanner-host/versions.tf b/infra/modules/scanner-host/versions.tf| new file mode 100644 |
| index 000000000..c028066c2 |
| --- /dev/null |
| +++ b/infra/modules/scanner-host/versions.tf |
| @@ -0,0 +1,9 @@ |
| +terraform { |
| + required_version = ">= 1.10" |
| + required_providers { |
| + aws = { |
| + source = "hashicorp/aws" |
| + version = ">= 6.0" |
| + } |
| + } |
| +} |
diff --git a/internal/clamd/clamd.go b/internal/clamd/clamd.go| index a6e51630c..97c46787a 100644 |
| --- a/internal/clamd/clamd.go |
| +++ b/internal/clamd/clamd.go |
| @@ -39,6 +39,7 @@ const ( |
| // because it is larger than StreamMaxLength. This is deliberately NOT |
| // StatusError: nothing failed, the object simply was not judged, and an |
| // audit that cannot tell those apart is not worth keeping. |
| + // It also covers clamd's limit heuristics (an archive that expands past MaxScanSize, MaxFileSize, MaxFiles or MaxRecursion), which arrive as FOUND when AlertExceedsMax is on. |
| StatusSizeExceeded Status = "size-exceeded" |
| // StatusError is the engine reading the stream and failing on it. |
| StatusError Status = "error" |
| @@ -77,6 +78,11 @@ const ( |
| // the same order of magnitude and the frame header is 4 bytes, so this |
| // keeps the framing overhead under a thousandth. |
| chunkSize = 64 * 1024 |
| + |
| + // clamd's names for "I stopped before the end". With AlertExceedsMax yes it |
| + // reports them as FOUND, which parseReply must not read as an infection. |
| + limitHeuristic = "Heuristics.Limits.Exceeded" |
| + limitHeuristicScanTime = "Heuristics.Limits.Exceeded.MaxScanTime" |
| ) |
| func (t Timeouts) withDefaults() Timeouts { |
| @@ -265,6 +271,13 @@ func (c *Client) Version(ctx context.Context) (string, error) { |
| return c.readReply(ctx, conn) |
| } |
| +// isLimitHeuristic reports whether sig names a scan limit rather than malware. |
| +// A prefix on the signature, with the dot, so neither a longer first component |
| +// nor a real signature carrying the phrase elsewhere can match. |
| +func isLimitHeuristic(sig string) bool { |
| + return sig == limitHeuristic || strings.HasPrefix(sig, limitHeuristic+".") |
| +} |
| + |
| // parseReply maps clamd's one-line answers onto a Status. |
| // |
| // The size check comes first and deliberately: the refusal arrives as |
| @@ -278,8 +291,20 @@ func parseReply(reply string) Result { |
| case strings.HasSuffix(reply, " OK"): |
| res.Status = StatusClean |
| case strings.HasSuffix(reply, " FOUND"): |
| - res.Status = StatusInfected |
| - res.Signature = signatureOf(reply) |
| + sig := signatureOf(reply) |
| + switch { |
| + case sig == limitHeuristicScanTime: |
| + // Load-dependent, so retried: an error becomes a failed verdict, |
| + // which every sweep re-issues. |
| + res.Status = StatusError |
| + case isLimitHeuristic(sig): |
| + // Deterministic for these bytes and this config, and nothing was |
| + // judged: the same fact as an object too large to stream. |
| + res.Status = StatusSizeExceeded |
| + default: |
| + res.Status = StatusInfected |
| + res.Signature = sig |
| + } |
| default: |
| res.Status = StatusError |
| } |
diff --git a/internal/clamd/clamd_test.go b/internal/clamd/clamd_test.go| index aa88665e8..f15fc1702 100644 |
| --- a/internal/clamd/clamd_test.go |
| +++ b/internal/clamd/clamd_test.go |
| @@ -306,3 +306,66 @@ func TestDialRefusesEmptyAddress(t *testing.T) { |
| t.Fatalf("err = %v", err) |
| } |
| } |
| + |
| +// With AlertExceedsMax on, clamd reports an archive it could only partly scan |
| +// as a FOUND whose signature names the limit. That is not an infection and it |
| +// is not clean: nothing was judged, so it is the same status as an object too |
| +// large to stream. Recording it clean would sign a verdict over bytes nobody |
| +// read; recording it infected would make enforce block large honest archives. |
| +func TestLimitHeuristicIsSizeExceededNotInfected(t *testing.T) { |
| + for _, limit := range []string{"MaxScanSize", "MaxFileSize", "MaxFiles", "MaxRecursion"} { |
| + reply := "stream: Heuristics.Limits.Exceeded." + limit + " FOUND\x00" |
| + res := replyFixture(t, reply) |
| + if res.Status != StatusSizeExceeded { |
| + t.Errorf("%s: status = %v, want size-exceeded", limit, res.Status) |
| + } |
| + if res.Signature != "" { |
| + t.Errorf("%s: a limit is not a signature, got %q", limit, res.Signature) |
| + } |
| + if !strings.Contains(res.Raw, limit) { |
| + t.Errorf("%s: the engine's own words did not survive: %q", limit, res.Raw) |
| + } |
| + } |
| +} |
| + |
| +// Older engines name the limit without saying which one. |
| +func TestBareLimitHeuristicIsSizeExceeded(t *testing.T) { |
| + res := replyFixture(t, "stream: Heuristics.Limits.Exceeded FOUND\x00") |
| + if res.Status != StatusSizeExceeded { |
| + t.Fatalf("status = %v, want size-exceeded", res.Status) |
| + } |
| +} |
| + |
| +// A scan that ran out of TIME depends on load, not on the bytes, so it must be |
| +// retried. StatusError becomes a failed verdict, which every sweep re-issues; |
| +// size-exceeded becomes skipped, which is carried forward until the object |
| +// changes. Filing a timeout under skipped would exclude the object for good. |
| +func TestScanTimeLimitIsAnErrorSoItIsRetried(t *testing.T) { |
| + res := replyFixture(t, "stream: Heuristics.Limits.Exceeded.MaxScanTime FOUND\x00") |
| + if res.Status != StatusError { |
| + t.Fatalf("status = %v, want error", res.Status) |
| + } |
| + if !strings.Contains(res.Raw, "MaxScanTime") { |
| + t.Fatalf("the engine's own words did not survive: %q", res.Raw) |
| + } |
| +} |
| + |
| +// The match is on the SIGNATURE, as a prefix. A real signature that happens to |
| +// carry the phrase elsewhere in its name is still an infection, and so is one |
| +// that only begins with something similar. |
| +func TestLookalikeSignaturesAreStillInfected(t *testing.T) { |
| + for _, sig := range []string{ |
| + "Win.Heuristics.Limits.Exceeded.Fake", |
| + "Heuristics.Limits.ExceededButNot", |
| + "Heuristics.Phishing.Email.SpoofedDomain", |
| + "Eicar-Signature", |
| + } { |
| + res := replyFixture(t, "stream: "+sig+" FOUND\x00") |
| + if res.Status != StatusInfected { |
| + t.Errorf("%s: status = %v, want infected", sig, res.Status) |
| + } |
| + if res.Signature != sig { |
| + t.Errorf("signature = %q, want %q", res.Signature, sig) |
| + } |
| + } |
| +} |