amadan
Steering: a trashed app's latency records are never withdrawn — the converger reads records, and a trashed app keeps its record Issue open
Migrated from github.com/carlosframework/platform issue #367, opened 2026-08-29. GitHub issues are closed for this repo; see AGENTS.md, "Work is tracked on amadan, not GitHub".
Still outstanding: A trashed app's Route53 latency records are never withdrawn, so they survive the full auto-purge window.
---
Found while verifying #340 and the static steering default on flagship (both rolled at c3f021d, 2026-08-29).
What happens
Trash an app whose record carries steering: "latency" and its Route53 latency records are never withdrawn. Measured:
| action | serving | latency records | |---|---|---| | carlos apps delete | 404 at every edge after ~6 min | 4, still there 10 min later | | delete the record instead | 404 | 0 after ~6 min |
So the converger's withdrawal path works perfectly — it is specifically the trash path that never reaches it.
Why
steeredRoutesFromRecords (internal/host/steering_loop.go) builds the desired set from provision.ListAllRecords — bucket records, not the local registry. That is deliberate and correct: dc30c28 made records the deployment's truth precisely so a housekeeper lease migration could not delete a live steered host's records and NXDOMAIN it.
But a trashed app keeps its record. The trash marker retires the route (#340, now on every box); the record stays until the 30-day auto-purge destroys it. Everything driven off records therefore keeps acting on a trashed app — and the steering converger is exactly such a consumer.
This is the same shape as #340 itself: a pass that reads records and never asks whether the app is in the trash.
Severity, and why it is worth fixing now
Self-healing but slow: once the cooloff purge removes the record, the converger withdraws the records on its next pass. So the litter lasts 30 days per deleted app, not forever. The name resolves to real edges and 404s, so there is no misdirection and no wrong content.
What changes the calculus is that static platform hosts are now steered by default (#355). Before that, exactly two hosts on flagship were steered and both were hand-set, so this was reachable but rare. Now every deleted static app leaves four health-checked record sets behind for a month.
Suggested fix
Give steeredRoutesFromRecords the same trash awareness provision.Reconcile now has: skip records whose app carries a valid trash marker. Two details worth carrying over from #340's fix:
- The marker's two answers are not the same answer. Here the safe direction is the opposite of minting's: an unreadable marker must NOT withdraw DNS (withdrawal is the disruptive direction for a live host), so only a marker positively read and present should drop a record from the desired set.
ListAllRecordsis deliberately fail-closed — a partial read returns an error and no records. Any tombstone read added alongside it needs the same posture, or a bucket blip becomes a fleet-wide NXDOMAIN.
Cross-refs #340, #306.