carlos / platform Public

Specify maintenance crypto lease and process boundaries

Paul Campbell pushed by paul@keymail.dev 1 month ago 69b8aa900ea03fb7313e14ab5ab77ef8e20884a1 parent e899a47

1 file changed, +23 −1

implementation/maintenance-exec.md +23 −1

CI passed — run details

diff --git a/implementation/maintenance-exec.md b/implementation/maintenance-exec.md

index 7d0d4c236..5bd6721b6 100644

--- a/implementation/maintenance-exec.md

+++ b/implementation/maintenance-exec.md

@@ -1,6 +1,6 @@

# Fenced maintenance for an existing exec instance

-Status: draft for independent Claude Fable review. Platform discussion117/task14; base dcf0cad333405aa8990af62c0871bd6ef63d96ba. This is an implementation proposal, not authorization to run a maintenance operation. User authorization covers implementing the missing supported platform command as part of Oficina readiness. Concrete deployment, fresh owner approval and operation receipts remain required.

+Status: revised draft; first independent Claude Fable invocation terminated at provider quota429 with no verdict (reset2026-09-08T12:55:31.309Z). No approval is implied. Platform discussion117/task14; base dcf0cad333405aa8990af62c0871bd6ef63d96ba. This is an implementation proposal, not authorization to run a maintenance operation. User authorization covers implementing the missing supported platform command as part of Oficina readiness. Concrete deployment, fresh owner approval and operation receipts remain required.

## Source and scope

@@ -54,6 +54,28 @@ Resume is a separately approved operation, not `clear maintenance`. It names an

Only after the checker succeeds may the worker publish resume-ready evidence, install the pinned serving release/config and arrange the lease-to-serving replication handoff. Clear the fence through one guarded transition bound to this evidence; old/new worker races cannot resurrect a superseded resume. Admit HTTP only after the exact new process and replication are healthy and report the approved version; retain a durable resumed receipt. If process startup/replication/version verification fails, return to fenced/failed, not ready. The implementation must decide and test the precise lease handoff before code approval; simply releasing and hoping the next wake uses the approved pins is insufficient.

+## Concrete implementation seams for the next review

+These decisions replace the corresponding open mechanism questions above; the complete consumer inventory and final resume transition still need to be checked against implementation source before code approval.

+**Private transport does not require a console decryption key.** Actual appconfig/crypto.go deliberately puts CARLOS_SECRETS_PUBKEY on the encrypting side and CARLOS_SECRETS_KEY on decrypting hosts. The console must not gain the private key for maintenance. Add a shared purpose-aware envelope primitive in internal/maintenance, reusing the established ECDH-P256/HKDF-SHA256/AES-GCM shape but a new HKDF info string `carlos-maintenance-v1`; authenticated data includes canonical length-prefixed version, direction, account, app, primary host, request ID, attempt ID and recipient fingerprint. Generate a fresh ephemeral key and nonce per envelope. Unknown version/purpose/direction/recipient or changed binding refuses before plaintext is used. Golden vectors and cross-purpose mutation tests are required; old appconfig Seal/Open remain byte-compatible.

+The preparing CLI generates a per-request P-256 result key pair and saves its private key in its exclusive0600 local pending-operation file before upload. The owner's reviewed request includes the public recipient fingerprint. Request payload encrypts to the existing deployment secrets public key; the worker decrypts it with the existing host key. Private stdout encrypts to the CLI result public key. Console and bucket persist only ciphertext and authenticated metadata; the console can authorize delivery but cannot decrypt results. The CLI requires both a current human-owner API session and the saved recipient private key to produce the private result file. Lost local result keys make that result unavailable; no automatic replay or server-side plaintext fallback. Any operator-level recovery of an idempotent receipt is a new, approved request. Pending local keys and payloads are outside repository/terminal logs; cleanup does not silently destroy the only copy before the caller durably saves a result.

+**Environment pinning is private.** Public approval shows the selected named/default config revisions and a randomized snapshot-envelope digest, never a plaintext secret digest. A worker preflight captures the exact merged instanceEnv map into an authenticated encrypted snapshot, retaining both config source revisions and the box-local overlay identity. The owner approval binds that snapshot ciphertext and record revision. Before execution the worker compares the decrypted approved map with the current merged map in memory and refuses changes; it does not reload a newer key into the command. The snapshot is the command's environment. No EnsureAdminToken side effect or restoration of a missing overlay occurs. Adoption/config writers must preserve the pinned snapshot while the fence is active; checking only manifest generation omits local overlay changes and is insufficient.

+**Checked stop is a distinct result, not a synonym for Quiesced.** Add a per-host maintenance fence map inside activator.Service, protected by the same mutex that inserts waking entries. `FenceAndStop(ctx, host, generation)` sets the local admission barrier before inspecting a waking/awake/stopping entry, then waits for that exact entry's terminal result. Every path through begin, scheduled wake and BeginSidecarRequest consults the barrier, including stale Route callers. Extend the internal stop result to record sidecar Stop/Exited, main Stop/Exited and replication Close outcomes before removing the entry; ordinary stop logging behavior stays compatible, but the maintenance caller refuses on any missing/failed evidence. A missing entry is only part of the proof: strict cgroup emptiness and absence of an active replication owner must also hold. On agent restart an outstanding executing attempt is ambiguous, and checked cleanup is required before any retry/resume. CAS request generations prevent an old local completion from clearing a new barrier.

+**Checkpointing retains the same lease.** Add `(*lease.Held).Checkpoint(ctx, dbPath) (CheckpointReceipt, error)` as an exclusive, non-consuming operation. It uses an extracted common replica-store constructor, opens the single temporary store, calls SyncAndWait, closes it with checked errors and returns while the Held renewer remains active. No ReleaseLease occurs on success or failure. Checkpoint cannot run concurrently with itself, Replicate or Release, or on a spent handle; use an explicit ownership/state guard rather than relying on caller memory. Receipt is emitted only after sync AND store close AND final lease-health check. A failure leaves the handle owned but unusable for command execution until the worker records and resolves the failure. The eventual serving `Replicate` transfer retains its existing ownership semantics; existing callers are not silently changed to keep leases forever.

+Expose a conservative monotonic safe-until deadline from lease acquisition/renewal, captured before the corresponding remote request and shortened by a fixed termination margin. Maintenance must stop and prove its process tree empty before that deadline. A renewal failure or elapsed deadline signals immediate termination even if the wall clock moves backwards. Tests use shortened TTLs and a blocked renewal to prove termination precedes takeover. No new lease acquisition can be reported as proof an old process died.

+**Maintenance requires strict process containment.** Current cgroup.Manager.Start deliberately falls back to spawn-then-move and even an unaccounted spawn; its source notes grandchildren can remain outside the host cgroup. That fallback is acceptable accounting behavior but is not the maintenance boundary. Add a separate strict start path requiring Linux cgroup v2, CLONE_INTO_CGROUP, a writable delegated host subtree and usable cgroup.kill/events. It never falls back or starts before containment is established. A missing capability means unsupported, not a less safe command. Use the existing tier limits; failed limit/containment setup refuses. Kernel cgroup.events populated=0 is required after termination; PID/Wait alone does not cover descendants. Use cgroup.kill for the complete subtree on timeout/lease loss and verify emptiness within the remaining safe lease window. Keep ordinary Start behavior unchanged.

+The maintenance capability preflight also requires the platform-managed agent unit to explicitly enforce KillMode=control-group, SendSIGKILL=yes and a bounded TimeoutStopSec. The checked effective unit, not an assumed systemd default, is part of the deployed capability receipt. Parent-death protection is set on the direct command, but is not claimed to cover grandchildren. Agent main-process death must cause systemd to kill the entire delegated subtree before restart; test actual systemd behavior on the isolated Linux platform canary, including a descendant holding the output pipe. Without that deployed proof this class remains disabled. OOM/forced-kill and agent-restart tests must prove no surviving writer is mistaken for a clean stop. This is containment against process lifecycle failure under the existing trusted app/shared-uid model, not protection against a malicious instance intentionally migrating out of its delegated cgroup.

+**Resume must use the retained lease and a scoped internal permit.** The proposed seam is `Service.ResumeFenced` taking an already-held lease and a worker-created opaque permit bound to record/attempt/generation, approved serving artifact and environment snapshot. Ordinary callers cannot construct the permit. It starts replication and the exact pinned serving process while all public admission remains fenced; no Restore, Seed or live-link resolution is performed. It checks health and the app's actual reported version before the worker CASes the matching durable resume outcome. Failed CAS, version mismatch or process/replication failure invokes checked stop and retains the durable fence. The remaining design question is how the per-instance approved serving pin stays authoritative through concurrent channel promotion and the durable-clear/local-ready race; a bare release-and-next-wake or transient local pin is expressly rejected. This must be resolved in the final review, not deferred into a future rollout caveat.

## Implementation and release gates

1. Resolve the explicit seams above in a final reviewed design: complete record/admission consumer inventory; checked stop/replica-close evidence; lease-retaining checkpoint; cgroup/parent-death proof; environment binding; exact resume handoff. These are design work, not instructions to assume the current API already supplies them.