amadan

carlos / platform Public

harness: host up (step 2): host-setup.sh, hssm.py, host user-data, role policy, run log

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
claude dbc77d9b3e18ca9b586da1aac285c4ba5b933c3c parent 6b905c1
7 files changed, +298 −0
  • harness/RESOURCES.md +5 −0
  • harness/RUNLOG.md +41 −0
  • harness/aws/role-policy.json +20 −0
  • harness/aws/trust.json +1 −0
  • harness/host-setup.sh +116 −0
  • harness/host-user-data.sh +8 −0
  • harness/hssm.py +107 −0
diff --git a/harness/RESOURCES.md b/harness/RESOURCES.md
index d36d7161b..98db93995 100644
--- a/harness/RESOURCES.md
+++ b/harness/RESOURCES.md
@@ -5,3 +5,8 @@ Appended as each is created. Teardown commands: harness/TEARDOWN.md.
| Created (UTC) | Type | ID / name | Notes |
|---|---|---|---|
+| 2026-10-20T00:40:20Z | s3 bucket | carlos-harness-640012953448 | public access blocked, lifecycle 3 days, ttl 2026-10-20T12:40Z |
+| 2026-10-20T00:40:37Z | iam role | carlos-harness | trust ec2; AmazonSSMManagedInstanceCore + inline carlos-harness-bucket (harness/aws/role-policy.json) |
+| 2026-10-20T00:40:41Z | iam instance profile | carlos-harness | holds role carlos-harness |
+| 2026-10-20T00:40:50Z | security group | sg-0b9cf4c2a508b7cec (carlos-harness) | vpc-02212d67a9fca3b20; no inbound; egress tcp 80, 443 only |
+| 2026-10-20T00:41:14Z | ec2 instance (host) | i-06a1ec144aa5226fb | t4g.xlarge, ami-071babd89cea53fa3, subnet-0975dda9ac0e65347, profile carlos-harness, 60 GiB gp3 (DeleteOnTermination), terminate on shutdown, shutdown -h 12:40 UTC |
diff --git a/harness/RUNLOG.md b/harness/RUNLOG.md
index cc3609b8d..943909b0c 100644
--- a/harness/RUNLOG.md
+++ b/harness/RUNLOG.md
@@ -45,3 +45,44 @@ linux/arm64 binaries: `GOOS=linux GOARCH=arm64 CGO_ENABLED=0 go build -tags harn
-ldflags "-s -w -X .../internal/version.Build=harness-<sha>"` for `.` and `./cmd/console`.
**Step 1: GREEN** (2026-10-20T00:39:52Z).
+
+## Step 2: the host up
+
+AWS (profile kass, eu-west-1, sandbox off on DOCS-T1); every ID is in RESOURCES.md:
+
+- bucket `carlos-harness-640012953448`: public access blocked, lifecycle 3 days, tags.
+- role and instance profile `carlos-harness`: `AmazonSSMManagedInstanceCore` plus
+ `harness/aws/role-policy.json` (harness bucket Get/Put/Delete/List; explicit Deny
+ on `carlos-dev-*` buckets, `ssm:GetParameter*` on `/carlos/*`, and
+ `ec2:*`, `iam:*`, `ssm:SendCommand`, `ssm:StartSession`).
+- security group `sg-0b9cf4c2a508b7cec`: no inbound; egress tcp 80, 443 only.
+- host `i-06a1ec144aa5226fb`: t4g.xlarge, ami-071babd89cea53fa3, subnet-0975dda9ac0e65347,
+ 60 GiB gp3, IMDSv2 required, hop limit 1, terminate on shutdown,
+ user-data `harness/host-user-data.sh` (SSM agent; `shutdown -h 12:40` UTC).
+- TTL tag on everything: `ttl=2026-10-20T12:40Z`.
+
+Host setup: `python3 harness/hssm.py harness/host-setup.sh` (incus 6.0, btrfs pool
+`harness` 45 GiB, bridge `hbr0` 203.0.113.1/24 with NAT and DHCP .100-.200, profile
+`box` = privileged + nesting + `lxc.apparmor.profile=unconfined` + 2 CPU / 1 GiB /
+20 GiB, nftables table `inet harness` with the IMDS DNAT and the guard, unit
+`harness-guard.service`). `incus admin init --minimal`'s `incusbr0` and `default`
+dir pool were deleted; the default profile now uses hbr0 and the harness pool.
+
+Check (`python3 harness/hssm.py $TMPDIR/harness-build/check2.sh`, on host, 00:5x UTC):
+
+```
+== incus list (before) -> empty
+== shutdown --show -> Shutdown scheduled for Tue 2026-10-20 12:40:00 UTC
+scratch container (profile box, privileged), eth0 203.0.113.131/24:
+imds token PUT: URLError [Errno 111] Connection refused # DNAT -> 203.0.113.1:8169, no stand-in yet
+tcp 169.254.169.254:80 -> refused
+tcp 169.254.169.253:53 -> timeout (dropped)
+tcp 169.254.170.2:80 -> timeout (dropped)
+tcp host VPC ip 172.31.5.119:22 -> timeout (dropped)
+tcp 203.0.113.1:22 -> timeout (dropped)
+tcp 172.31.0.2:53 (VPC resolver) -> timeout (dropped)
+tcp deb.debian.org:443 (provisioning egress) -> CONNECTED
+== incus list (after) -> empty (scratch deleted)
+```
+
+The scratch container never reached the host role. **Step 2: GREEN** (2026-10-20T00:45:27Z).
diff --git a/harness/aws/role-policy.json b/harness/aws/role-policy.json
new file mode 100644
index 000000000..e687b9856
--- /dev/null
+++ b/harness/aws/role-policy.json
@@ -0,0 +1,20 @@
+{
+ "Version": "2012-10-17",
+ "Statement": [
+ {"Sid": "HarnessBucketObjects", "Effect": "Allow",
+ "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
+ "Resource": "arn:aws:s3:::carlos-harness-640012953448/*"},
+ {"Sid": "HarnessBucketList", "Effect": "Allow",
+ "Action": ["s3:ListBucket"],
+ "Resource": "arn:aws:s3:::carlos-harness-640012953448"},
+ {"Sid": "DenyDevDeploymentBuckets", "Effect": "Deny",
+ "Action": "s3:*",
+ "Resource": ["arn:aws:s3:::carlos-dev-*", "arn:aws:s3:::carlos-dev-*/*"]},
+ {"Sid": "DenyCarlosParameters", "Effect": "Deny",
+ "Action": ["ssm:GetParameter", "ssm:GetParameters", "ssm:GetParametersByPath", "ssm:GetParameterHistory"],
+ "Resource": "arn:aws:ssm:*:640012953448:parameter/carlos/*"},
+ {"Sid": "DenyControlPlane", "Effect": "Deny",
+ "Action": ["ec2:*", "iam:*", "ssm:SendCommand", "ssm:StartSession"],
+ "Resource": "*"}
+ ]
+}
diff --git a/harness/aws/trust.json b/harness/aws/trust.json
new file mode 100644
index 000000000..6c9b6b847
--- /dev/null
+++ b/harness/aws/trust.json
@@ -0,0 +1 @@
+{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"ec2.amazonaws.com"},"Action":"sts:AssumeRole"}]}
diff --git a/harness/host-setup.sh b/harness/host-setup.sh
new file mode 100644
index 000000000..eb2ccfbcd
--- /dev/null
+++ b/harness/host-setup.sh
@@ -0,0 +1,116 @@
+#!/bin/bash
+# Harness host setup (step 2). Run as root on the harness host, once.
+#
+# incus with a btrfs pool, the bridge hbr0 on 203.0.113.0/24 (TEST-NET-3, review
+# B2: outside every instance unit's IPAddressDeny), the IMDS DNAT to the stand-in
+# on 203.0.113.1:8169, and the nftables guard:
+# - containers never reach the host's own addresses except the stand-ins,
+# DNS and DHCP on 203.0.113.1;
+# - containers never reach 169.254.0.0/16 (real IMDS) or the VPC;
+# - egress beyond the bridge: tcp 80/443 while provisioning (apt, the incus
+# image server), nothing once harness-egress-close.sh has run.
+set -euo pipefail
+export DEBIAN_FRONTEND=noninteractive
+
+apt-get update -q
+apt-get install -y -q incus btrfs-progs nftables jq python3 curl ca-certificates >/dev/null
+
+incus admin init --minimal
+if ! incus storage show harness >/dev/null 2>&1; then
+ incus storage create harness btrfs size=45GiB
+fi
+if ! incus network show hbr0 >/dev/null 2>&1; then
+ incus network create hbr0 ipv4.address=203.0.113.1/24 ipv4.nat=true ipv6.address=none \
+ ipv4.dhcp.ranges=203.0.113.100-203.0.113.200 dns.domain=harness.test
+fi
+
+# --minimal made a default dir pool and an incusbr0 bridge the guard does not
+# cover: every container goes on hbr0 and the harness pool instead.
+incus profile device remove default eth0 2>/dev/null || true
+incus profile device remove default root 2>/dev/null || true
+incus profile device add default root disk path=/ pool=harness
+incus profile device add default eth0 nic network=hbr0 name=eth0
+incus network delete incusbr0 2>/dev/null || true
+incus storage delete default 2>/dev/null || true
+
+# The box profile: privileged, nested, AppArmor unconfined (review: verify day one),
+# sized as the dev boxes' t4g.micro.
+incus profile create box 2>/dev/null || true
+incus profile edit box <<'EOF'
+description: carlos harness box (t4g.micro shape)
+config:
+ security.privileged: "true"
+ security.nesting: "true"
+ raw.lxc: lxc.apparmor.profile=unconfined
+ limits.cpu: "2"
+ limits.memory: 1GiB
+devices:
+ root:
+ path: /
+ pool: harness
+ type: disk
+ size: 20GiB
+ eth0:
+ name: eth0
+ network: hbr0
+ type: nic
+EOF
+
+# nftables guard, in its own table so incus's table is untouched.
+mkdir -p /etc/harness
+cat > /etc/harness/guard.nft <<'EOF'
+table inet harness
+delete table inet harness
+table inet harness {
+ set egress_ports { type inet_service; elements = { 80, 443 } }
+
+ chain pre {
+ type nat hook prerouting priority dstnat - 10; policy accept;
+ iifname "hbr0" ip daddr 169.254.169.254 tcp dport 80 dnat ip to 203.0.113.1:8169
+ }
+
+ chain input {
+ type filter hook input priority filter - 10; policy accept;
+ iifname != "hbr0" accept
+ udp dport 67 accept
+ ip daddr != 203.0.113.1 counter drop
+ ct state established,related accept
+ tcp dport { 53, 8100, 8169 } accept
+ udp dport { 53, 67 } accept
+ icmp type echo-request accept
+ counter drop
+ }
+
+ chain forward {
+ type filter hook forward priority filter - 10; policy accept;
+ iifname != "hbr0" accept
+ oifname "hbr0" accept
+ ip daddr { 169.254.0.0/16, 172.16.0.0/12, 10.0.0.0/8, 192.168.0.0/16, 100.64.0.0/10 } counter drop
+ tcp dport @egress_ports accept
+ counter drop
+ }
+}
+EOF
+nft -f /etc/harness/guard.nft
+
+# Re-apply after incus restarts its own rules or the host reboots.
+cat > /etc/systemd/system/harness-guard.service <<'EOF'
+[Unit]
+Description=carlos harness nftables guard
+After=incus.service nftables.service
+Wants=incus.service
+
+[Service]
+Type=oneshot
+RemainAfterExit=yes
+ExecStart=/usr/sbin/nft -f /etc/harness/guard.nft
+ExecStartPost=/bin/sh -c 'test -f /etc/harness/egress-closed && /usr/sbin/nft flush set inet harness egress_ports || true'
+
+[Install]
+WantedBy=multi-user.target
+EOF
+systemctl daemon-reload
+systemctl enable harness-guard.service >/dev/null
+
+incus image list images: debian/13/cloud arm64 -c lf --format csv | head -3
+echo "host-setup: done"
diff --git a/harness/host-user-data.sh b/harness/host-user-data.sh
new file mode 100644
index 000000000..a45c9f70f
--- /dev/null
+++ b/harness/host-user-data.sh
@@ -0,0 +1,8 @@
+#!/bin/sh
+# carlos harness host: SSM agent (stock Debian has none) and the TTL self-shutdown.
+set -e
+timedatectl set-timezone UTC || true
+shutdown -h 12:40 "carlos harness TTL 2026-10-20T12:40Z"
+curl -fsSL -o /tmp/ssm.deb https://s3.eu-west-1.amazonaws.com/amazon-ssm-eu-west-1/latest/debian_arm64/amazon-ssm-agent.deb
+dpkg -i /tmp/ssm.deb
+systemctl enable --now amazon-ssm-agent
diff --git a/harness/hssm.py b/harness/hssm.py
new file mode 100644
index 000000000..d7adb3b60
--- /dev/null
+++ b/harness/hssm.py
@@ -0,0 +1,107 @@
+#!/usr/bin/env python3
+"""Run a shell script on the harness host through real SSM, from DOCS-T1.
+
+ hssm.py [--timeout S] [--quiet] FILE|- run FILE (or stdin) as root on the host
+ hssm.py put LOCAL KEY copy LOCAL to s3://<harness bucket>/KEY
+ hssm.py url KEY print a presigned GET for KEY (1 h)
+
+The host is found by its tags (purpose=carlos-harness, Name=carlos-harness-host,
+state running); the script refuses to act on anything else. Full stdout and
+stderr go to the harness bucket under ssm-logs/, because SSM's inline output is
+cut at 24,000 characters; this prints the full log from there.
+
+Copies root-runner.py's pattern (aws CLI, profile kass, eu-west-1). AWS calls
+need the DOCS-T1 sandbox off.
+"""
+import json, os, subprocess, sys, tempfile, time
+
+PROFILE, REGION = "kass", "eu-west-1"
+BUCKET = "carlos-harness-640012953448"
+
+
+def aws(*a, check=True):
+ r = subprocess.run(["aws", "--profile", PROFILE, "--region", REGION, *a, "--output", "json"],
+ capture_output=True, text=True)
+ if check and r.returncode:
+ raise SystemExit(f"aws {' '.join(a[:3])}: {r.stderr.strip()}")
+ return json.loads(r.stdout) if r.stdout.strip() else {}
+
+
+def host():
+ d = aws("ec2", "describe-instances", "--filters", "Name=tag:purpose,Values=carlos-harness",
+ "Name=tag:Name,Values=carlos-harness-host", "Name=instance-state-name,Values=running")
+ ids = [i["InstanceId"] for r in d["Reservations"] for i in r["Instances"]]
+ if len(ids) != 1:
+ raise SystemExit(f"want exactly one running harness host, found {ids}")
+ return ids[0]
+
+
+def fetch(prefix):
+ d = tempfile.mkdtemp(dir=os.environ.get("TMPDIR"))
+ try:
+ subprocess.run(["aws", "--profile", PROFILE, "--region", REGION, "s3", "cp", "--recursive", "--quiet",
+ f"s3://{BUCKET}/{prefix}", d], check=False)
+ out, err = "", ""
+ for root, _, files in os.walk(d):
+ for f in files:
+ with open(os.path.join(root, f), errors="replace") as fh:
+ if f == "stdout":
+ out += fh.read()
+ elif f == "stderr":
+ err += fh.read()
+ return out, err
+ finally:
+ subprocess.run(["rm", "-rf", d], check=False)
+
+
+def run(script, timeout=1800, quiet=False):
+ inst = host()
+ prefix = f"ssm-logs/{time.strftime('%Y%m%dT%H%M%S')}-{os.getpid()}"
+ cid = aws("ssm", "send-command", "--instance-ids", inst, "--document-name", "AWS-RunShellScript",
+ "--timeout-seconds", str(timeout), "--output-s3-bucket-name", BUCKET, "--output-s3-key-prefix", prefix,
+ "--parameters", json.dumps({"commands": [script], "executionTimeout": [str(timeout)]}))["Command"]["CommandId"]
+ time.sleep(2)
+ while True:
+ d = aws("ssm", "get-command-invocation", "--command-id", cid, "--instance-id", inst, check=False)
+ st = d.get("Status")
+ if st in ("Success", "Failed", "TimedOut", "Cancelled"):
+ break
+ time.sleep(3)
+ out, err = d.get("StandardOutputContent", ""), d.get("StandardErrorContent", "")
+ if len(out) >= 24000 or len(err) >= 8000:
+ full_out, full_err = fetch(prefix)
+ out, err = full_out or out, full_err or err
+ sys.stdout.write(out)
+ if err.strip() and not quiet:
+ sys.stderr.write(err)
+ print(f"[hssm] {inst} {cid} {st} exit={d.get('ResponseCode')}", file=sys.stderr)
+ return 0 if st == "Success" else 1
+
+
+def main():
+ a = sys.argv[1:]
+ timeout, quiet = 1800, False
+ while a and a[0].startswith("--"):
+ if a[0] == "--timeout":
+ timeout = int(a[1]); a = a[2:]
+ elif a[0] == "--quiet":
+ quiet = True; a = a[1:]
+ else:
+ raise SystemExit(__doc__)
+ if not a:
+ raise SystemExit(__doc__)
+ if a[0] == "put":
+ subprocess.run(["aws", "--profile", PROFILE, "--region", REGION, "s3", "cp", "--quiet", a[1],
+ f"s3://{BUCKET}/{a[2]}"], check=True)
+ return 0
+ if a[0] == "url":
+ r = subprocess.run(["aws", "--profile", PROFILE, "--region", REGION, "s3", "presign",
+ f"s3://{BUCKET}/{a[1]}", "--expires-in", "3600"], capture_output=True, text=True, check=True)
+ print(r.stdout.strip())
+ return 0
+ script = sys.stdin.read() if a[0] == "-" else open(a[0]).read()
+ return run(script, timeout, quiet)
+
+
+if __name__ == "__main__":
+ sys.exit(main())