isolated exec: the probe goes ready, and each instance gets its own /dev/shm (#126)
RestrictSUIDSGID= dropped from the four instance templates (it makes openat2 ENOSYS on systemd 257); control-unreachable lists and dials instead of Lstat; PrivateDevices= was binding the host's /dev/shm into every same-uid instance, so each unit now gets a private tmpfs, with a host fixture and a shm-private probe check. Spec revision 15, R15. Verified ready on dev. mqueue and abstract sockets: #178. Squashed from isolated-exec-probe-fixes into main.
12 files changed,
+229
−22
docs/superpowers/specs/2026-09-11-isolated-exec-design.md+7 −3internal/host/bootstrap.go+5 −1internal/host/units/carlos-instance-probe.service+12 −1internal/host/units/carlos-instance-seed@.service+12 −1internal/host/units/carlos-instance-sidecar@.service+12 −1internal/host/units/carlos-instance@.service+12 −1internal/host/units_isolation_test.go+9 −0internal/instancectl/peer_linux.go+11 −2internal/instancectl/peer_other.go+1 −1internal/instancectl/server.go+3 −2internal/instanceexec/probe.go+41 −6internal/instanceexec/probe_test.go+104 −3
CI passed — run details
diff --git a/docs/superpowers/specs/2026-09-11-isolated-exec-design.md b/docs/superpowers/specs/2026-09-11-isolated-exec-design.md| index 03c033f38..48d9efe55 100644 |
| --- a/docs/superpowers/specs/2026-09-11-isolated-exec-design.md |
| +++ b/docs/superpowers/specs/2026-09-11-isolated-exec-design.md |
| @@ -1,11 +1,12 @@ |
| # Isolated exec — every exec-backed instance in its own namespace |
| -**Date:** 2026-09-11 · **Status:** revision 14; codex verdict after |
| +**Date:** 2026-09-11 · **Status:** revision 15; codex verdict after |
| round 11 is **accept with corrections**, and the one correction is |
| applied here (54 findings over 11 rounds, all accepted; §13); a |
| whole-branch final review (plan 2) found six further items, fixed in |
| revision 13 (§13); plan 3's task 4 review settled the requested-stop |
| -close, recorded in revision 14 (§6.5, §13 R13); awaiting Paul's review · **Requested by:** discussion |
| +close, recorded in revision 14 (§6.5, §13 R13); the first roll's probe |
| +found two defects, fixed in revision 15 (§4.1, §8.5, §13 R15); awaiting Paul's review · **Requested by:** discussion |
| #126 (Tito, with box evidence), approved in direction by Paul on |
| 2026-09-11 |
| @@ -630,6 +631,8 @@ ReadWritePaths=/data/instances/%I /run/carlos/instances/%I |
| ProtectHome=yes |
| PrivateTmp=yes |
| PrivateDevices=yes |
| +# Its own /dev/shm; PrivateDevices= would bind the host's (R15). |
| +TemporaryFileSystem=/dev/shm:mode=1777,nosuid,nodev |
| # /sys/fs/cgroup read-only: the edge's delegated subtree is carlos-owned, |
| # and its cgroup.kill / cgroup.freeze would otherwise be one write away |
| # for a same-uid tenant (codex R9-1). |
| @@ -650,7 +653,7 @@ ProtectHostname=yes |
| RemoveIPC=yes |
| LockPersonality=yes |
| RestrictRealtime=yes |
| -RestrictSUIDSGID=yes |
| +# No RestrictSUIDSGID=: it refuses openat2 outright (R15). |
| MemoryDenyWriteExecute=yes |
| SystemCallArchitectures=native |
| @@ -2131,3 +2134,4 @@ rollback runs the old binary). |
| R11-1 the probe fixture comes from tmpfiles at bootstrap, and the control service verifies its ownership and mode before accepting a verdict (§3.2, §8.5). |
| - R12 (final review, plan 2): cgroup-ro targets the edge's delegated cgroup.procs; DAC precedes the mount flag so a root-owned file answers EACCES. |
| - R13 (task 4 review, plan 3): after a stop the activator requested, a unit reporting a foreign or absent invocation proves the old invocation over without the cgroup gate (§6.5); the observer never stops an invocation it did not start once its own stop was requested. |
| +- R15 (first roll, 2026-09-12; fixed 2026-09-25): the probe on dev's boxes answered `ready: false` twice, and the review of the fix found a third gap. (a) `RestrictSUIDSGID=yes` makes `openat2` fail with ENOSYS on systemd 257, because systemd cannot read the mode inside `open_how` and so blocks the whole call; every `instancefs` open in the namespace is an `openat2`. The line is dropped from all four templates. What it guarded is covered another way: with no capabilities an instance can only make a set-id file owned by `carlos`, which gains nothing for another `carlos` process; `NoNewPrivileges=yes` voids it inside the namespace; and no other uid can reach one (the tenant trees are `nosuid`, and `PrivateTmp=`'s `/tmp` and `/var/tmp` sit under a root 0700 directory, `/var/tmp` being on the root filesystem and not `nosuid`). (b) `control-unreachable` checked `Lstat`, but `InaccessiblePaths=` leaves a mode 000 root-owned node at the path. The check is now what a tenant could do, a listing of the directory and a dial of the socket, and passes only when both are refused for permission or absence; a refused connection or a timeout proves the path reachable and fails. (c) `PrivateDevices=` bind-mounts the host's `/dev/shm`, so every instance, all uid `carlos`, shared one; each unit now gets its own tmpfs there, and the probe's `shm-private` check plants a file in the host's `/dev/shm` (the control service's tmpfiles fixture) and fails if it is visible. POSIX message queues and abstract unix sockets are still box-wide; they are a follow-up (§11). |
diff --git a/internal/host/bootstrap.go b/internal/host/bootstrap.go| index 89d232ffc..b04bd7fde 100644 |
| --- a/internal/host/bootstrap.go |
| +++ b/internal/host/bootstrap.go |
| @@ -100,7 +100,11 @@ const tmpfilesConf = "d /run/carlos 2750 " + serviceUser + " " + socketsGroup + |
| // root:carlos 0640 that an isolated instance must NOT be able to read. |
| // Written here, not by the control service, whose ProtectSystem=strict |
| // makes /etc read-only to it. |
| - "f /etc/carlos/probe-secret 0640 root " + serviceUser + " - probe\n" |
| + "f /etc/carlos/probe-secret 0640 root " + serviceUser + " - probe\n" + |
| + // And one in the host's /dev/shm, which an isolated instance must NOT |
| + // see: its own /dev/shm is a private tmpfs (#126). Without a file |
| + // here, an empty listing inside the probe would prove nothing. |
| + "f /dev/shm/carlos-probe 0644 root root - probe\n" |
| // journaldConfName is the journald drop-in every box gets (discussion #111). |
| // |
diff --git a/internal/host/units/carlos-instance-probe.service b/internal/host/units/carlos-instance-probe.service| index f8878fe1e..c45b75f53 100644 |
| --- a/internal/host/units/carlos-instance-probe.service |
| +++ b/internal/host/units/carlos-instance-probe.service |
| @@ -46,6 +46,10 @@ InaccessiblePaths=/etc/carlos /run/carlos-control |
| ProtectHome=yes |
| PrivateTmp=yes |
| PrivateDevices=yes |
| +# PrivateDevices= bind-mounts the HOST's /dev/shm into the private /dev, |
| +# and every instance is uid carlos, so a sibling's shared memory would be |
| +# one open away. Each unit gets its own, charged to its own cgroup. |
| +TemporaryFileSystem=/dev/shm:mode=1777,nosuid,nodev |
| # /sys/fs/cgroup read-only: the edge's delegated subtree is carlos-owned, |
| # and its cgroup.kill / cgroup.freeze would otherwise be one write away |
| # for a same-uid tenant (codex R9-1). |
| @@ -66,7 +70,14 @@ ProtectHostname=yes |
| RemoveIPC=yes |
| LockPersonality=yes |
| RestrictRealtime=yes |
| -RestrictSUIDSGID=yes |
| +# No RestrictSUIDSGID=: systemd cannot read the mode inside openat2's |
| +# open_how, so it refuses openat2 outright (ENOSYS), and every instancefs |
| +# open in here is an openat2 (#126, first roll; systemd 257). What it |
| +# guarded is covered: with no capabilities an instance can only make a |
| +# set-id file owned by carlos, which gains nothing for another carlos |
| +# process, NoNewPrivileges= voids it in here, and no other uid can reach |
| +# one (the tenant trees are nosuid, PrivateTmp's sit under a root 0700 |
| +# directory). |
| MemoryDenyWriteExecute=yes |
| SystemCallArchitectures=native |
diff --git a/internal/host/units/carlos-instance-seed@.service b/internal/host/units/carlos-instance-seed@.service| index 711b51b24..7dd47f549 100644 |
| --- a/internal/host/units/carlos-instance-seed@.service |
| +++ b/internal/host/units/carlos-instance-seed@.service |
| @@ -73,6 +73,10 @@ InaccessiblePaths=/etc/carlos /run/carlos-control |
| ProtectHome=yes |
| PrivateTmp=yes |
| PrivateDevices=yes |
| +# PrivateDevices= bind-mounts the HOST's /dev/shm into the private /dev, |
| +# and every instance is uid carlos, so a sibling's shared memory would be |
| +# one open away. Each unit gets its own, charged to its own cgroup. |
| +TemporaryFileSystem=/dev/shm:mode=1777,nosuid,nodev |
| # /sys/fs/cgroup read-only: the edge's delegated subtree is carlos-owned, |
| # and its cgroup.kill / cgroup.freeze would otherwise be one write away |
| # for a same-uid tenant (codex R9-1). |
| @@ -93,7 +97,14 @@ ProtectHostname=yes |
| RemoveIPC=yes |
| LockPersonality=yes |
| RestrictRealtime=yes |
| -RestrictSUIDSGID=yes |
| +# No RestrictSUIDSGID=: systemd cannot read the mode inside openat2's |
| +# open_how, so it refuses openat2 outright (ENOSYS), and every instancefs |
| +# open in here is an openat2 (#126, first roll; systemd 257). What it |
| +# guarded is covered: with no capabilities an instance can only make a |
| +# set-id file owned by carlos, which gains nothing for another carlos |
| +# process, NoNewPrivileges= voids it in here, and no other uid can reach |
| +# one (the tenant trees are nosuid, PrivateTmp's sit under a root 0700 |
| +# directory). |
| MemoryDenyWriteExecute=yes |
| SystemCallArchitectures=native |
diff --git a/internal/host/units/carlos-instance-sidecar@.service b/internal/host/units/carlos-instance-sidecar@.service| index 8b65c85f0..88f3be41f 100644 |
| --- a/internal/host/units/carlos-instance-sidecar@.service |
| +++ b/internal/host/units/carlos-instance-sidecar@.service |
| @@ -69,6 +69,10 @@ InaccessiblePaths=/etc/carlos /run/carlos-control |
| ProtectHome=yes |
| PrivateTmp=yes |
| PrivateDevices=yes |
| +# PrivateDevices= bind-mounts the HOST's /dev/shm into the private /dev, |
| +# and every instance is uid carlos, so a sibling's shared memory would be |
| +# one open away. Each unit gets its own, charged to its own cgroup. |
| +TemporaryFileSystem=/dev/shm:mode=1777,nosuid,nodev |
| # /sys/fs/cgroup read-only: the edge's delegated subtree is carlos-owned, |
| # and its cgroup.kill / cgroup.freeze would otherwise be one write away |
| # for a same-uid tenant (codex R9-1). |
| @@ -89,7 +93,14 @@ ProtectHostname=yes |
| RemoveIPC=yes |
| LockPersonality=yes |
| RestrictRealtime=yes |
| -RestrictSUIDSGID=yes |
| +# No RestrictSUIDSGID=: systemd cannot read the mode inside openat2's |
| +# open_how, so it refuses openat2 outright (ENOSYS), and every instancefs |
| +# open in here is an openat2 (#126, first roll; systemd 257). What it |
| +# guarded is covered: with no capabilities an instance can only make a |
| +# set-id file owned by carlos, which gains nothing for another carlos |
| +# process, NoNewPrivileges= voids it in here, and no other uid can reach |
| +# one (the tenant trees are nosuid, PrivateTmp's sit under a root 0700 |
| +# directory). |
| MemoryDenyWriteExecute=yes |
| SystemCallArchitectures=native |
diff --git a/internal/host/units/carlos-instance@.service b/internal/host/units/carlos-instance@.service| index 01b3063a1..2baae24e8 100644 |
| --- a/internal/host/units/carlos-instance@.service |
| +++ b/internal/host/units/carlos-instance@.service |
| @@ -69,6 +69,10 @@ InaccessiblePaths=/etc/carlos /run/carlos-control |
| ProtectHome=yes |
| PrivateTmp=yes |
| PrivateDevices=yes |
| +# PrivateDevices= bind-mounts the HOST's /dev/shm into the private /dev, |
| +# and every instance is uid carlos, so a sibling's shared memory would be |
| +# one open away. Each unit gets its own, charged to its own cgroup. |
| +TemporaryFileSystem=/dev/shm:mode=1777,nosuid,nodev |
| # /sys/fs/cgroup read-only: the edge's delegated subtree is carlos-owned, |
| # and its cgroup.kill / cgroup.freeze would otherwise be one write away |
| # for a same-uid tenant (codex R9-1). |
| @@ -89,7 +93,14 @@ ProtectHostname=yes |
| RemoveIPC=yes |
| LockPersonality=yes |
| RestrictRealtime=yes |
| -RestrictSUIDSGID=yes |
| +# No RestrictSUIDSGID=: systemd cannot read the mode inside openat2's |
| +# open_how, so it refuses openat2 outright (ENOSYS), and every instancefs |
| +# open in here is an openat2 (#126, first roll; systemd 257). What it |
| +# guarded is covered: with no capabilities an instance can only make a |
| +# set-id file owned by carlos, which gains nothing for another carlos |
| +# process, NoNewPrivileges= voids it in here, and no other uid can reach |
| +# one (the tenant trees are nosuid, PrivateTmp's sit under a root 0700 |
| +# directory). |
| MemoryDenyWriteExecute=yes |
| SystemCallArchitectures=native |
diff --git a/internal/host/units_isolation_test.go b/internal/host/units_isolation_test.go| index 25dd4d829..8ebeb9588 100644 |
| --- a/internal/host/units_isolation_test.go |
| +++ b/internal/host/units_isolation_test.go |
| @@ -135,6 +135,7 @@ func TestTmpfilesCreatesTheInstanceTrees(t *testing.T) { |
| "d /run/carlos/logs 0700 carlos carlos -", |
| "d /run/carlos-control 0750 root carlos -", |
| "f /etc/carlos/probe-secret 0640 root carlos - probe", |
| + "f /dev/shm/carlos-probe 0644 root root - probe", |
| } { |
| if !strings.Contains(tmpfilesConf, want+"\n") { |
| t.Errorf("tmpfilesConf lacks %q", want) |
| @@ -220,6 +221,7 @@ func TestInstanceTemplatesShareOneNamespace(t *testing.T) { |
| for _, want := range []string{ |
| "ProtectSystem=strict", "TemporaryFileSystem=/data:ro", "TemporaryFileSystem=/run/carlos:ro", |
| "TemporaryFileSystem=/var/cache:ro", "TemporaryFileSystem=/var/lib:ro", "InaccessiblePaths=/etc/carlos", |
| + "TemporaryFileSystem=/dev/shm:mode=1777,nosuid,nodev", |
| "ProtectControlGroups=yes", "ProtectProc=invisible", "ProcSubset=pid", "PrivatePIDs=yes", |
| "RestrictNamespaces=yes", "NoNewPrivileges=yes", "CapabilityBoundingSet=", |
| "IPAddressDeny=169.254.169.254/32 fd00:ec2::254/128", "MemoryDenyWriteExecute=yes", |
| @@ -228,6 +230,13 @@ func TestInstanceTemplatesShareOneNamespace(t *testing.T) { |
| t.Errorf("namespace block lacks %q", want) |
| } |
| } |
| + // RestrictSUIDSGID= makes openat2 answer ENOSYS, and every instancefs |
| + // open inside the namespace is an openat2 (#126, first roll). |
| + for _, line := range strings.Split(ref, "\n") { |
| + if strings.HasPrefix(line, "RestrictSUIDSGID=") { |
| + t.Errorf("namespace block carries %q, which breaks openat2", line) |
| + } |
| + } |
| for name, wants := range map[string][]string{ |
| "carlos-instance@.service": {"ExecStart=/opt/carlos/carlos instance-exec run %I", "ExecStopPost=/opt/carlos/carlos instance-exit run %I", |
| "BindPaths=/data/instances/%I /run/carlos/instances/%I", "StandardOutput=file:/run/carlos/logs/%I.run.sock", |
diff --git a/internal/instancectl/peer_linux.go b/internal/instancectl/peer_linux.go| index af226bb21..0e4085b5d 100644 |
| --- a/internal/instancectl/peer_linux.go |
| +++ b/internal/instancectl/peer_linux.go |
| @@ -159,8 +159,17 @@ func bindControlSocket(path string) (net.Listener, error) { |
| return ln, nil |
| } |
| -// fixtureIsRootCarlos0640 checks the probe fixture tmpfiles stages. |
| -func fixtureIsRootCarlos0640() error { |
| +// probeFixturesOK checks the two probe fixtures tmpfiles stages: the |
| +// secret the probe must not read, and the host /dev/shm file it must not |
| +// see. Either one missing would make its check pass vacuously. |
| +func probeFixturesOK() error { |
| + shm, err := os.Lstat("/dev/shm/carlos-probe") |
| + if err != nil { |
| + return fmt.Errorf("probe shm fixture absent: %v", err) |
| + } |
| + if sst, ok := shm.Sys().(*syscall.Stat_t); !ok || sst.Uid != 0 || !shm.Mode().IsRegular() { |
| + return errors.New("probe shm fixture: not a root-owned regular file") |
| + } |
| fi, err := os.Stat("/etc/carlos/probe-secret") |
| if err != nil { |
| return fmt.Errorf("probe fixture absent: %v", err) |
diff --git a/internal/instancectl/peer_other.go b/internal/instancectl/peer_other.go| index 30cc02d32..17c373f1d 100644 |
| --- a/internal/instancectl/peer_other.go |
| +++ b/internal/instancectl/peer_other.go |
| @@ -12,4 +12,4 @@ func productionPeerOK() func(net.Conn) error { |
| } |
| func activatedListener() (net.Listener, bool, error) { return nil, false, errors.ErrUnsupported } |
| func bindControlSocket(string) (net.Listener, error) { return nil, errors.ErrUnsupported } |
| -func fixtureIsRootCarlos0640() error { return errors.ErrUnsupported } |
| +func probeFixturesOK() error { return errors.ErrUnsupported } |
diff --git a/internal/instancectl/server.go b/internal/instancectl/server.go| index 57ec1b0ce..580cbfe86 100644 |
| --- a/internal/instancectl/server.go |
| +++ b/internal/instancectl/server.go |
| @@ -44,7 +44,8 @@ type Server struct { |
| // ProbeOut is where the probe unit's StandardOutput=truncate: lands. |
| ProbeOut string |
| // FixtureOK verifies /etc/carlos/probe-secret is root:carlos 0640 |
| - // before a probe verdict is accepted (design §8.5, codex R11-1). |
| + // and /dev/shm/carlos-probe is a root-owned file before a probe |
| + // verdict is accepted (design §8.5, codex R11-1, #126 R15). |
| FixtureOK func() error |
| // ProbeEvery is the re-run cadence (five minutes in production). |
| ProbeEvery time.Duration |
| @@ -80,7 +81,7 @@ func Run(args []string) { |
| PeerOK: productionPeerOK(), |
| CgroupRoot: "/sys/fs/cgroup", |
| ProbeOut: filepath.Join(controlsock.Dir, "probe.out"), |
| - FixtureOK: fixtureIsRootCarlos0640, |
| + FixtureOK: probeFixturesOK, |
| ProbeEvery: 5 * time.Minute, |
| Logf: log.Printf, |
| } |
diff --git a/internal/instanceexec/probe.go b/internal/instanceexec/probe.go| index 21d41e3e4..05bd909d4 100644 |
| --- a/internal/instanceexec/probe.go |
| +++ b/internal/instanceexec/probe.go |
| @@ -7,6 +7,7 @@ import ( |
| "errors" |
| "flag" |
| "fmt" |
| + "net" |
| "os" |
| "path/filepath" |
| "sort" |
| @@ -15,6 +16,7 @@ import ( |
| "time" |
| "github.com/carlosframework/platform/internal/cliflag" |
| + "github.com/carlosframework/platform/internal/controlsock" |
| "github.com/carlosframework/platform/internal/instancectl" |
| "github.com/carlosframework/platform/internal/instancefs" |
| ) |
| @@ -28,8 +30,10 @@ type probeEnv struct { |
| varCache string // /var/cache |
| varLib string // /var/lib |
| etcCarlos string // /etc/carlos |
| + devShm string // /dev/shm, which must be this unit's own |
| procRoot string // /proc |
| - controlDir string // /run/carlos-control, which must be invisible from in here |
| + controlDir string // /run/carlos-control, which must be unusable from in here |
| + controlSock string // the control socket inside it, which must not dial |
| cgroupProcs string // the edge's delegated cgroup.procs, to test writability on |
| pid func() int |
| writable func(path string) error |
| @@ -76,7 +80,7 @@ func RunProbe(args []string) { |
| func productionProbeEnv() probeEnv { |
| return probeEnv{ |
| dataRoot: "/data", runRoot: "/run/carlos", varCache: "/var/cache", varLib: "/var/lib", |
| - etcCarlos: "/etc/carlos", procRoot: "/proc", controlDir: "/run/carlos-control", |
| + etcCarlos: "/etc/carlos", devShm: "/dev/shm", procRoot: "/proc", controlDir: controlsock.Dir, controlSock: controlsock.Path, |
| // The edge's own delegated cgroup.procs, not the probe's: Delegate= |
| // on carlos-edge.service chowns this file to carlos, so DAC passes |
| // and only the mount's ro flag can refuse the write, making EROFS |
| @@ -175,18 +179,31 @@ func runChecks(e probeEnv) instancectl.ProbeReport { |
| mro, mroErr := e.mountOption(cgroupMountpoint, "ro") |
| pass("cgroup-ro-mount", fmt.Sprintf("MountOptionOf(%s, ro) = %v, %v", cgroupMountpoint, mro, mroErr), mro && mroErr == nil) |
| - // The control socket directory is invisible from in here: InaccessiblePaths= |
| + // The control socket is unreachable from in here: InaccessiblePaths= |
| // in the shared namespace block (design §5.2) makes a connect from |
| // inside any instance impossible outright, not merely refused by the |
| - // peer check on the other end. |
| - _, cerr := os.Lstat(e.controlDir) |
| - pass("control-unreachable", "control socket directory visible", cerr != nil) |
| + // peer check on the other end. The path itself still answers Lstat, |
| + // because systemd hides it under a mode 000 root-owned node rather |
| + // than removing it, so the check is what a tenant could actually do: |
| + // list the directory, or dial the socket (#126, first roll). |
| + // Both must be refused for permission or absence: a refused |
| + // connection or a timeout means the path was reachable. |
| + _, cerr := os.ReadDir(e.controlDir) |
| + kerr := dialUnix(e.controlSock) |
| + pass("control-unreachable", fmt.Sprintf("readdir=%v dial=%v", cerr, kerr), unreachable(cerr) && unreachable(kerr)) |
| // /etc/carlos cannot be opened and its fixture cannot be read. |
| _, derr := os.ReadDir(e.etcCarlos) |
| _, ferr := os.ReadFile(filepath.Join(e.etcCarlos, "probe-secret")) |
| pass("etc-carlos", fmt.Sprintf("readdir=%v readfile=%v", derr, ferr), derr != nil && ferr != nil) |
| + // /dev/shm is this unit's own tmpfs, not the host's, which PrivateDevices= |
| + // would bind in: the host's holds the fixture /dev/shm/carlos-probe, so |
| + // an empty listing here means a sibling's shared memory is out of reach |
| + // (#126, R15). |
| + s1, sok := exactly(e.devShm) |
| + pass("shm-private", s1, sok) |
| + |
| // A symlink planted in the probe's own directory is refused with ELOOP. |
| serr := e.symlinkCheck(filepath.Join(e.dataRoot, "instances", instancectl.ProbeDirName)) |
| pass("symlink-eloop", fmt.Sprint(serr), serr == nil) |
| @@ -313,3 +330,21 @@ func realSymlinkCheck(dir string) error { |
| } |
| return err |
| } |
| + |
| +// dialUnix connects to a unix socket and closes the connection at once, |
| +// returning nil only if the connect succeeded. |
| +func dialUnix(path string) error { |
| + c, err := net.DialTimeout("unix", path, 2*time.Second) |
| + if err != nil { |
| + return err |
| + } |
| + return c.Close() |
| +} |
| + |
| +// unreachable reports whether err is a refusal for permission or absence: |
| +// the answers a hidden path gives. Any other error, a refused connection |
| +// among them, means the path itself was reached. |
| +func unreachable(err error) bool { |
| + return errors.Is(err, syscall.EACCES) || errors.Is(err, syscall.EPERM) || |
| + errors.Is(err, syscall.ENOENT) || errors.Is(err, syscall.ENOTDIR) |
| +} |
diff --git a/internal/instanceexec/probe_test.go b/internal/instanceexec/probe_test.go| index e03e54f12..d8b28a9e7 100644 |
| --- a/internal/instanceexec/probe_test.go |
| +++ b/internal/instanceexec/probe_test.go |
| @@ -4,6 +4,7 @@ package instanceexec |
| import ( |
| "errors" |
| + "net" |
| "os" |
| "path/filepath" |
| "strings" |
| @@ -34,12 +35,14 @@ func goodEnv(t *testing.T) probeEnv { |
| varCache := mk("var/cache") |
| varLib := mk("var/lib") |
| etc := filepath.Join(root, "etc", "carlos") // deliberately absent: opening it fails |
| + shm := mk("dev/shm") |
| proc := mk("proc") |
| mk("proc/1") |
| mk("proc/self") |
| return probeEnv{ |
| - dataRoot: data, runRoot: run, varCache: varCache, varLib: varLib, etcCarlos: etc, procRoot: proc, |
| - controlDir: filepath.Join(root, "run", "carlos-control"), // deliberately absent: Lstat fails |
| + dataRoot: data, runRoot: run, varCache: varCache, varLib: varLib, etcCarlos: etc, devShm: shm, procRoot: proc, |
| + controlDir: filepath.Join(root, "run", "carlos-control"), // deliberately absent: ReadDir and dial fail |
| + controlSock: filepath.Join(root, "run", "carlos-control", "control.sock"), |
| cgroupProcs: filepath.Join(root, "sys", "cgroup.procs"), |
| pid: func() int { return 1 }, |
| writable: func(string) error { return syscall.EROFS }, |
| @@ -71,7 +74,7 @@ func TestProbePassesOnAnIsolatedNamespace(t *testing.T) { |
| if !r.Ready { |
| t.Fatalf("not ready: %+v", r.Lines) |
| } |
| - for _, c := range []string{"data-tree", "run-tree", "var-empty", "pid1", "cgroup-ro", "cgroup-ro-mount", "control-unreachable", "etc-carlos", "symlink-eloop", "nosymfollow", "imds-v4", "imds-v6", "loopback", "imds-tcp"} { |
| + for _, c := range []string{"data-tree", "run-tree", "var-empty", "pid1", "cgroup-ro", "cgroup-ro-mount", "control-unreachable", "etc-carlos", "shm-private", "symlink-eloop", "nosymfollow", "imds-v4", "imds-v6", "loopback", "imds-tcp"} { |
| if v := verdict(r, c); v != "ok" { |
| t.Errorf("%s = %s", c, v) |
| } |
| @@ -142,6 +145,104 @@ func TestProbeFailsWhenTheControlSocketDirectoryIsVisible(t *testing.T) { |
| } |
| } |
| +// TestProbePassesOnAnInaccessibleControlNode pins the shape |
| +// InaccessiblePaths= really leaves: a mode 000 directory that still |
| +// answers Lstat. The first probe on a box failed here because it checked |
| +// Lstat; what a tenant cannot do is list it or dial through it (#126). |
| +func TestProbePassesOnAnInaccessibleControlNode(t *testing.T) { |
| + if os.Geteuid() == 0 { |
| + t.Skip("root reads a mode 000 directory") |
| + } |
| + e := goodEnv(t) |
| + if err := os.MkdirAll(e.controlDir, 0o700); err != nil { |
| + t.Fatal(err) |
| + } |
| + if err := os.Chmod(e.controlDir, 0); err != nil { |
| + t.Fatal(err) |
| + } |
| + t.Cleanup(func() { os.Chmod(e.controlDir, 0o700) }) |
| + if _, err := os.Lstat(e.controlDir); err != nil { |
| + t.Fatalf("fixture: Lstat = %v, want the node to exist", err) |
| + } |
| + r := runChecks(e) |
| + if v := verdict(r, "control-unreachable"); v != "ok" { |
| + t.Errorf("control-unreachable = %s on a mode 000 node: %+v", v, r.Lines) |
| + } |
| +} |
| + |
| +// TestProbeFailsWhenTheControlSocketDials: a directory that refuses a |
| +// listing but still lets a connect through is not unreachable. |
| +func TestProbeFailsWhenTheControlSocketDials(t *testing.T) { |
| + if os.Geteuid() == 0 { |
| + t.Skip("root reads a mode 0300 directory") |
| + } |
| + e := goodEnv(t) |
| + if err := os.MkdirAll(e.controlDir, 0o700); err != nil { |
| + t.Fatal(err) |
| + } |
| + l, err := net.Listen("unix", e.controlSock) |
| + if err != nil { |
| + t.Fatalf("listen at %s (a socket test wants a short TMPDIR): %v", e.controlSock, err) |
| + } |
| + defer l.Close() |
| + go func() { |
| + for { |
| + c, err := l.Accept() |
| + if err != nil { |
| + return |
| + } |
| + c.Close() |
| + } |
| + }() |
| + if err := os.Chmod(e.controlDir, 0o300); err != nil { |
| + t.Fatal(err) |
| + } |
| + t.Cleanup(func() { os.Chmod(e.controlDir, 0o700) }) |
| + r := runChecks(e) |
| + if r.Ready || verdict(r, "control-unreachable") != "fail" { |
| + t.Errorf("dialable control socket yet ready=%v control-unreachable=%s", r.Ready, verdict(r, "control-unreachable")) |
| + } |
| +} |
| + |
| +// TestProbeFailsOnAStaleControlSocket: a socket file with nothing |
| +// listening refuses the connection, which proves the path was reached. |
| +func TestProbeFailsOnAStaleControlSocket(t *testing.T) { |
| + if os.Geteuid() == 0 { |
| + t.Skip("root reads a mode 0300 directory") |
| + } |
| + e := goodEnv(t) |
| + if err := os.MkdirAll(e.controlDir, 0o700); err != nil { |
| + t.Fatal(err) |
| + } |
| + l, err := net.Listen("unix", e.controlSock) |
| + if err != nil { |
| + t.Fatalf("listen at %s (a socket test wants a short TMPDIR): %v", e.controlSock, err) |
| + } |
| + l.(*net.UnixListener).SetUnlinkOnClose(false) |
| + l.Close() |
| + if err := os.Chmod(e.controlDir, 0o300); err != nil { |
| + t.Fatal(err) |
| + } |
| + t.Cleanup(func() { os.Chmod(e.controlDir, 0o700) }) |
| + r := runChecks(e) |
| + if r.Ready || verdict(r, "control-unreachable") != "fail" { |
| + t.Errorf("stale control socket yet ready=%v control-unreachable=%s", r.Ready, verdict(r, "control-unreachable")) |
| + } |
| +} |
| + |
| +// TestProbeFailsWhenTheHostShmIsVisible: PrivateDevices= alone binds the |
| +// host's /dev/shm, where tmpfiles plants the fixture (#126, R15). |
| +func TestProbeFailsWhenTheHostShmIsVisible(t *testing.T) { |
| + e := goodEnv(t) |
| + if err := os.WriteFile(filepath.Join(e.devShm, "carlos-probe"), []byte("probe"), 0o644); err != nil { |
| + t.Fatal(err) |
| + } |
| + r := runChecks(e) |
| + if r.Ready || verdict(r, "shm-private") != "fail" { |
| + t.Errorf("host /dev/shm visible yet ready=%v shm-private=%s", r.Ready, verdict(r, "shm-private")) |
| + } |
| +} |
| + |
| func TestProbeFailsWhenEtcCarlosIsReadable(t *testing.T) { |
| e := goodEnv(t) |
| if err := os.MkdirAll(e.etcCarlos, 0o755); err != nil { |