🤖 CARLOS infra modules

> 🤖 Written by an LLM (Claude), per factor X.

Reusable OpenTofu modules for a CARLOS deployment. Every consumer reaches them the same way — by git source URL — including the deployments Paul operates, whose roots live in the private platform-infrastructure repo. There are no stacks in this repo to copy from, which is deliberate: nothing deployment-specific belongs here.

source = "git::https://github.com/carlosframework/platform.git//infra/modules/region-host?ref=main"

The division of labour is unchanged by modularisation: tofu owns boxes, buckets, IAM, and static DNS; the platform owns apps, releases, instances, and dynamic DNS. Nothing account-shaped (tenant hostnames, routes) may ever enter tofu state — tofu apply must always be safe. Providers, backends, and DNS records stay in your root module: they are deployment-specific, the modules are not.

deployment-buckets

The durable pair: the deployment bucket (apps, releases, channels, instance records, host config — routing metadata only, nothing customer-shaped) and one regional data bucket (parked databases, WAL streams, leases, blobs — the residency promise). Both get versioning, AES256 SSE, a full public-access block, and prevent_destroy. Boxes are disposable; these buckets are not.

| Variable | Description | |---|---| | deployment_bucket_name | Name of the deployment bucket. | | data_bucket_name | Name of the regional data bucket. |

Outputs: deployment_bucket, deployment_bucket_arn, data_bucket, data_bucket_arn.

Adding a second region later means a second deployment-buckets-shaped data bucket; for now each deployment is one region, so the module is a pair.

region-host

One region of a deployment: a Debian arm64 (Graviton) box running carlos agent, behind an Elastic IP. Provides the AMI lookup, a security group admitting 80/443 only (no SSH — admin access is SSM Session Manager, and the carlos binary arrives from the deployment bucket, not over scp), an IAM role scoped to exactly the deployment bucket + this region's data bucket + the deployment's SSM parameter prefix, the instance profile with AmazonSSMManagedInstanceCore attached, the instance (encrypted gp3 root), and the EIP.

Residency is enforced by construction: a host in another region gets a different module instance whose role does not include this region's data bucket, so a wrong-region wake fails loudly at the IAM layer.

| Variable | Description | |---|---| | name | Security group name + Name tag (e.g. carlos-flagship-ie). | | sg_description | SG description (changing it forces SG replacement — keep it stable). | | iam_name | IAM role + instance profile name; inline policy is <iam_name>-buckets. | | instance_type | arm64 instance type, e.g. t4g.small. | | root_volume_size | Root gp3 volume size, GiB. | | user_data | First-boot init script content (pass file("${path.module}/host-init.sh")). Changes never replace the instance. | | deployment_bucket_arn | ARN of the deployment bucket. | | data_bucket_arn | ARN of THIS region's data bucket. | | vpc_id | VPC to place the host in. Null (default) uses the region's default VPC — see the warning below. Set it together with subnet_id. | | subnet_id | Subnet to launch in; required when vpc_id is set, and must belong to it. Null (default) lets EC2 pick the default VPC's default subnet. | | ssm_parameter_arn | SSM parameter ARN pattern the host may read, e.g. arn:aws:ssm:<region>:<account>:parameter/carlos/*. Bounds parameter reads only in combination with scope_ssm_parameter_reads — see below. | | scope_ssm_parameter_reads | Deny every SSM parameter read outside ssm_parameter_arn. Default true. Set false only if this deployment's hosts genuinely read parameters outside that pattern — see below. | | route53_zone_ids | Hosted zones the box may publish ACME DNS-01 challenge TXTs into (wildcard certs). Bare Z... ids; empty = no Route53 access. | | steering_zone_ids | Hosted zones the DNS-steering converger may write latency A records into, plus the health checks that withdraw a dead edge. Bare Z... ids; empty = no steering grant (the default — steering also needs CARLOS_STEERING_ZONE_ID in the box env). | | steering_record_names | Name bound on that grant, e.g. ["*.oncarlos.com"]. Empty (default) grants A-record UPSERT/DELETE on the WHOLE zone, apex included — set it. | | ami_name_pattern | AMI name filter (default debian-13-arm64-*). | | ami_owner | AMI owner account (default 136693071363, Debian). | | alerts_email | Email for the host's MemAvailablePercent < 15% for 3 min alarm (memory-pressure spec §2.4). The module creates a same-region SNS topic + subscription per host (an alarm cannot notify a topic in another region). Empty (default): no topic, no alarm. Set it only once the box is publishing — missing data is treated as breaching. The cloudwatch:PutMetricData grant on namespace Carlos/Host is unconditional, so the agent's memwatch duty publishes either way. | | enable_auto_recovery | Arm the EC2 auto-recovery alarm (HA ladder rung 0) — a system-status-check failure recovers the instance onto healthy hardware, same instance id, EIP and volumes. Default true; set false only where an external supervisor owns recovery. |

Outputs: eip_public_ip (point your DNS here), instance_id, security_group_id (for sibling services that admit fleet traffic only), role_arn, role_name, ami_id.

README.md in the module directory is the other half: what to put on the box once tofu has made it, and what to tell it. That runbook used to live only in the private infrastructure repo, which is how one deployment came to triple its fleet with no committed way to bring a box up.

Two things to know before you apply

Default VPCs. With vpc_id unset the module builds the security group in the region's default VPC and lets EC2 pick that VPC's default subnet. Plenty of accounts no longer have one — deleting them is ordinary CIS hygiene — and without a default VPC the apply fails half-way through, after the buckets, the role and the instance profile already exist, with VPCIdNotSpecified: No default VPC for this user. The module now checks at plan time and tells you to pass vpc_id/subnet_id, which is the whole fix.

The SSM grant is wider than ssm_parameter_arn suggests. Session Manager is the only admin path to these boxes, so the module attaches the AWS-managed AmazonSSMManagedInstanceCore policy — and that policy grants ssm:GetParameter* on Resource: "*". On its own, the module's own scoped Allow bounds nothing: every host could read every SSM parameter in the account, which on one deployment included the root release-signing key, readable by an internet-facing edge box. scope_ssm_parameter_reads (default true) adds an explicit Deny on the four parameter-read actions for everything outside ssm_parameter_arn. It touches no other action, so Session Manager is unaffected. If your boxes read a parameter outside that pattern, widen the pattern rather than turning the deny off.

Not a module: Tito's svc/pdf box stays in Tito's own root (svc-pdf.tf) — it is deliberately deployment-local (a stateless Gotenberg whose SG admits only the fleet, with no bucket or parameter grants; statelessness is imposed, not promised). Generalise it only when a second deployment wants one.

scanner-host

One box that runs clamd for the console's object-storage virus scanning, on its own host so signature updates and scan CPU stay off the control plane. The console streams each object to it over TCP 3310. The box reads no buckets and no parameters: its role has Session Manager and an explicit Deny on SSM parameter reads. The security group admits 3310 from the console box's group and nothing else, and there is no SSH.

Port 3310 is open only while a self-test has passed against the clamd process running now. A replaced box gets a new private IP, so update the console's CARLOS_CLAMD_ADDR after any -replace. The module's README.md has the readiness check, four acceptance checks to run before wiring a console, and what a clean verdict does not cover.

| Variable | Description | |---|---| | name | Name for the instance, its security group, its IAM role and its instance profile (e.g. carlos-flagship-scanner). | | ami_id | Debian arm64 AMI. Pass the console box's own, module.<host>.ami_id. Ignored after creation: a newer image never replaces the box. | | client_security_group_id | Security group of the one box allowed to reach clamd: the console's. clamd has no authentication, so this group is the whole fence. | | region | AWS region the box runs in. The box fetches its SSM agent package from that region's bucket. | | instance_type | arm64 instance type. Default t4g.medium. A t4g.small works only with concurrent_database_reload = false. | | stream_max_bytes | clamd's StreamMaxLength, in bytes. Default 26214400 (25 MiB), between 8 MiB and 100 MiB. The console's CARLOS_CLAMD_MAX_BYTES must equal it. | | memory_max | clamd's cgroup ceiling (systemd MemoryMax). Default 3G. | | memory_high | clamd's throttle point (systemd MemoryHigh). Default 2560M. | | concurrent_database_reload | Keep scanning while new signatures load, at the cost of holding both databases in memory. Default true. | | subnet_id | Subnet to launch in. Null (default) is the default subnet of the default VPC, and is refused unless the client security group is in the default VPC. The subnet needs a route out. |

Outputs: clamd_addr (set as the console's CARLOS_CLAMD_ADDR), stream_max_bytes (set as CARLOS_CLAMD_MAX_BYTES), instance_id, security_group_id.

region-remote-host

What an off-cloud CARLOS host (customer fleets and tito's hq) needs from AWS when there is no EC2 instance profile to lean on: a region data bucket plus a scoped IAM user + access key, limited to this region's data bucket always, plus the deployment bucket only when deployment_bucket_arn is set — the same wrong-region-fails-at-IAM residency property region-host gives cloud hosts. Fleet-API consumers (issue #73) omit deployment_bucket_arn entirely: the box reaches the platform's deployment bucket over the console's edge with its tunnel token, not a direct IAM grant. Consumed by customer fleets (paulca/pi-fleet is the live example) and the platform's tito hq; the module is reusable as-is for both.

| Variable | Description | |---|---| | region_code | Short region code this host serves (e.g. pi, hq). | | data_bucket_name | Name of this region's data bucket (created here). | | deployment_bucket_arn | ARN of the existing deployment bucket the host reads releases/status from and writes heartbeats to. |

Outputs: bucket_name, access_key_id, secret_access_key (sensitive).

Self-hosting from your own repo

A minimal root module — backend + providers + the two module blocks + a DNS record. State backend, AWS profile, names, and DNS are yours; the shapes are these modules.

terraform {
  required_version = ">= 1.10"
  backend "s3" {
    bucket       = "my-tfstate-<account-id>"
    key          = "carlos/terraform.tfstate"
    region       = "eu-west-1"
    use_lockfile = true
  }
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }
  }
}

provider "aws" {
  region = "eu-west-1"
}

locals { account = "123456789012" }

module "buckets" {
  source                 = "git::https://github.com/carlosframework/platform.git//infra/modules/deployment-buckets?ref=main"
  deployment_bucket_name = "carlos-myco-platform-${local.account}"
  data_bucket_name       = "carlos-myco-ie-${local.account}"
}

module "host_ie" {
  source                = "git::https://github.com/carlosframework/platform.git//infra/modules/region-host?ref=main"
  name                  = "carlos-myco-ie"
  sg_description        = "carlos myco IE host"
  iam_name              = "carlos-myco-host-ie"
  instance_type         = "t4g.small"
  root_volume_size      = 20
  user_data             = file("${path.module}/host-init.sh")
  deployment_bucket_arn = module.buckets.deployment_bucket_arn
  data_bucket_arn       = module.buckets.data_bucket_arn
  ssm_parameter_arn     = "arn:aws:ssm:eu-west-1:${local.account}:parameter/carlos/*"
}

resource "aws_route53_record" "ie" {
  zone_id = data.aws_route53_zone.mine.zone_id
  name    = "ie.carlos.example.com"
  type    = "A"
  ttl     = 300
  records = [module.host_ie.eip_public_ip]
}

Then tofu init && tofu plan. Write your own host-init.sh — the module takes its content, and the box-side steps it needs to end up performing are in modules/region-host/README.md.

Pin ?ref= to a commit, not to main. The examples above say main so that they keep working as the modules move, and that is exactly the wrong setting for a live deployment: a routine tofu apply would adopt whatever landed upstream since your last one. With one region host that is a small risk; with six instantiating the same module it is a fleet-wide change nobody decided to make. Pin it, and treat bumping the pin as its own deliberate act:

source = "git::https://github.com/carlosframework/platform.git//infra/modules/region-host?ref=f975b47cc88a7c033daab6453111298d3d86d401"

More than one region

Six regions is the same file six times if you write it that way. One deployment measured its own: 743 lines across six region files, 118 each, of which 41 differed — about 65% pure duplication, and the duplicated part included a value that has to be different in every file.

That value is sibling_staging_arns, whose only correctness property is "every region's data bucket except my own". Maintained by hand it is O(n²) edits with nothing checking them: adding one region means editing all six, and the failure it invites — a region staging into its own bucket — is silent.

Both problems dissolve if regions are data rather than files. One map, one for_each on each module block, and the mesh as a computed expression:

locals {
  account = "123456789012"

  # The only thing that changes when you add or remove a region.
  regions = {
    ie = { aws_region = "eu-west-1", instance_type = "t4g.medium" }
    uk = { aws_region = "eu-west-2", instance_type = "t4g.small" }
    au = { aws_region = "ap-southeast-2", instance_type = "t4g.small" }
  }
}

# One provider alias per region, because a bucket and a box are created in
# the region they live in. This is the one part that stays literal: provider
# blocks cannot be generated.
provider "aws" {
  alias  = "ie"
  region = local.regions.ie.aws_region
}
# ...one per region.

module "region" {
  for_each = local.regions
  source   = "git::https://github.com/carlosframework/platform.git//infra/modules/region-host?ref=<sha>"
  providers = {
    aws = aws.${each.key}   # see the note below
  }

  name             = "carlos-myco-${each.key}"
  sg_description   = "carlos myco ${upper(each.key)} host"
  iam_name         = "carlos-myco-host-${each.key}"
  instance_type    = each.value.instance_type
  root_volume_size = 20
  user_data        = file("${path.module}/host-init.sh")

  deployment_bucket_arn = module.buckets.deployment_bucket_arn
  data_bucket_arn       = aws_s3_bucket.data[each.key].arn
  ssm_parameter_arn     = "arn:aws:ssm:${each.value.aws_region}:${local.account}:parameter/carlos/*"

  # The mesh: every region's data bucket except this one's. Computed, so it
  # cannot be wrong, and adding a region does not touch this line.
  sibling_staging_arns = [
    for code, _ in local.regions : aws_s3_bucket.data[code].arn if code != each.key
  ]
}

Adding a region becomes a three-line data change instead of six file edits, and a region staging into its own bucket becomes structurally impossible rather than something you assert afterwards with a script.

Two honest caveats. Provider aliases cannot be interpolated — providers = { aws = aws.${each.key} } is not valid HCL — so in practice you either keep one small module block per region that passes its own provider into a shared inner module, or generate the root from your own templating. And deployment-buckets makes a deployment+data pair, while a deployment has exactly one deployment bucket, so the per-region data buckets in the sketch above are plain aws_s3_bucket resources with the four settings deployment-buckets would have applied (versioning, encryption, public access block, ownership controls).

Neither caveat changes the shape: regions are data, the mesh is computed, and the thing you edit when the fleet changes is a map.