carlos / platform Public

guard-armed box: one refused roll holds the channel's activation forever (no completion/cancel/exclusion path) open

One refused roll on a guard-armed box wedges its channel. platform-d8's deployment harness found this on 2026-10-20 (harness-build RUNLOG, run on cc90d542 and d940c09f, G4).

After a guard-armed box refuses a router update, every later promote or rollback on that channel is refused: "activation <id> is held for router/<channel>. It is released when every reserved box reports completion or is excluded by a termination fence or quarantine". In the code:

  • settleGuard (internal/host/guardtransaction.go:126-145) withdraws root's marker on a box that never changed, but records no completion;
  • the agent's retry re-stages the same refused candidate forever;
  • cancel-activation refuses a published activation (internal/console/hostactivation.go:333);
  • ReleaseActivation accepts only completion or the D7 exclusions (internal/hostactors/activation_complete.go:13-56).

Consequence: fixing forward is impossible without quarantining or fencing the box, and every box sharing the channel is frozen too.

Dev trial mitigation: the release carries the rootaws sentinel fix (claim-empty-cohort-done 8376a468), which was the cause of the refusal, and G4 must pass in the harness first. G4's failure action is stop-and-hold until a recovery ruling.

Before arming production boxes: we need a recorded "refused, never changed" outcome that releases the activation, plus a tested operator path for a stuck activation.

Related defect, from platform-d8's harness on 2026-10-20: after a refused roll, the agent deletes its router cache and never fetches it again, because "refused" is terminal (internal/host/systemstate.go:77, systemparticipation.go:43). That contradicts settleGuard's comment that the box is "held for the agent's retry" (guardtransaction.go:126-131). So nothing ever retries, and the activation stays held.

Recovery proven in the harness and ruled an attended operator recovery for the dev trial:

  1. Stop carlos-system-update.path before the promote.
  2. Stage the release binary, verified against the signed sums and the published identity.
  3. Run that binary's own system apply once, with the service's environment.

That applies the release. guard-exec starts the edge on it, the box reports activation_complete, and the roster's held activation is released. This is the road for guard-armed boxes whose installed updater predates the stdout-only guarded run (claim-empty-cohort-done 8a313548).

Before arming production boxes: decide what "refused" should mean on an armed box (retry, or a recorded outcome that releases the activation), and ship the recovery as a supported command.