carlos / platform Public

restore-verify: fixed control/restore-verify.json key loses verdicts with N boxes — needs per-box keys Issue open

Migrated from github.com/carlosframework/platform issue #241, opened 2026-08-20. GitHub issues are closed for this repo; see AGENTS.md, "Work is tracked on amadan, not GitHub".

Still outstanding: restore-verify's fixed control key loses verdicts with more than one box; it needs a per-box key.

---

Found by the multi-region bucket-writer audit (spec 2026-08-20, stage 1) and sharpened by its review.

internal/dbverify publishes hibernate-restore-verify verdicts to the fixed key control/restore-verify.json. Two defects when more than one box runs the verify timer against the same deployment bucket:

  1. Deterministic verdict loss, no race required: Merge (dbverify.go:71-83) drops every prior host not present in this run's fleet, and fleet comes from the local box's own registry (HibernatingHosts). Two boxes alternately wipe each other's verdicts on every publish. Record.Box is a single scalar — the record structurally cannot describe two boxes.
  2. CAS retry clobbers bodies: the publish loop marshals the body BEFORE the retry loop and re-fetches only the etag on conflict (dbverify.go:132,137), so a losing box lands its stale pre-race merge over the winner's. The function's own comment claims it uses the usage.UpsertEdgeDay recipe — that recipe re-Gets AND re-merges inside the loop; this code does not.

Ruling (recorded in the audit doc): per-box keys — control/restore-verify/<box>.json, same shape as the registry snapshots (#158's fix). A re-merge-inside-retry fix alone does NOT work: defect 1 makes the loss deterministic regardless of CAS correctness. Readers (console/status surfaces, if any) union the per-box records.

Not urgent-urgent: the verify service is env-gated (CARLOS_HIBERNATE_BUCKET) and deployments run one platform box today. It must land before a second platform box arms the verify timer — i.e. before multi-region stage 5.