carlos / platform Public
Tenant Email, Plan 2 of 5: Senders
> For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.
Goal: Turn the one implicit per-app sender into an explicit, named, individually counted and individually pausable sender record, with no new capability for anyone: apps/<acct>/<app>/email/senders/app.json replaces email.json under a read-fallback migration, one SenderNames function owns every AWS name and its bound, a reverse index at email/callers/<iamuser>.json makes an IAM caller resolvable back to its sender, SES events are counted per sender in a new mail_sender_counts table, and the cap, the pause and the status all move onto the sender while the app-level wire fields keep saying exactly what they say today.
Architecture: internal/mail gains sender.go (the record, the names, the caller index, SenderOfCaller) and keeps Record as the app sender's in-memory view, so every existing walk caller compiles unchanged; Get/Put become the app sender's accessors with the legacy key as a read-only fallback that hands back an empty etag, which makes every writer's Put a create by construction. The console's counts move to a second table keyed by (account, app, sender, day), backfilled once from mail_counts; recordAndEnforceMail resolves the caller to a sender first and only falls back to today's domain-wide loop when it cannot. The status reply grows a senders array beside its unchanged top-level fields. PerInstance, PauseKindPerInstance, Host, Address, Identity and TenantARN land on the record with nothing in this plan setting them: they are step 3's and step 5's to write.
Tech Stack: Go 1.26, internal/mail, internal/console (SQLite store, SES event ingest, the email API doors), internal/release (Store, NewDirStore, ErrCASConflict, AppPrefix), aws-sdk-go-v2/service/sesv2 v1.66.6, root-package mail_cli.go.
Spec: docs/superpowers/specs/2026-09-15-tenant-email-senders-design.md — §11 step 2 is the whole of this plan's mandate. §5 (the record, the names, the caller index, the migration), §8 (attribution by caller, mail_sender_counts, caps, pauses, status) and §7's one sentence "Associations are wired in step 2, not here" are in scope. §6 and the rest of §7 are not: no instance senders, no external identities, no address claims, no per-instance config layer, no svcrole.Mail, no new doors.
Two parts of §8 are deferred to step 3, and that is a ruling rather than an oversight. §8's "SES's own pauses" paragraph asks for three things this plan does not build: EnsureTenant storing TenantArn on the record, SetTenantSending/TenantStatus on the provisioner with UpdateReputationEntityCustomerManagedStatus becoming the pause verb for a sender that has a tenant, and status reading GetReputationEntity so that an SES-imposed pause is reported apart from a CARLOS one (SendingStatusAggregate, CustomerManagedStatus and AwsSesManagedStatus kept distinct, per the spike at §4.1 item 4). They are deferred because all three are only reachable once a send is attributed to a tenant, which no send is until step 3 emits X-SES-TENANT: adopting the tenant pause verb now would move every existing app's pause onto a control SES does not yet apply to them, and reporting an SES pause distinctly would report a field that is null for every sender in the fleet. SetKeyActive stays the pause verb for this release, which is what it already is. The TenantARN field lands on the record in task 1 with nothing setting it, so step 3 has somewhere to write.
Global Constraints
- Run
gofmt -won every file you touched before running the gate. Several code blocks in this plan are pasted with trailing comments that gofmt will realign (task 1'sSenderstruct, task 2'slongAcct/longApppair, task 5'sCallerIndex).gofmt -l .only NAMES an unformatted file, so a plan followed literally goes red on the gate's last step rather than being fixed by it. - The gate before every commit:
go build ./... && go vet ./... && go test ./... -count=1all clean, andgofmt -l .empty over the whole tree. Env:GOCACHE=/tmp/claude-1001/gocache GOFLAGS=-mod=mod GOPROXY=off. Socket-binding tests needTMPDIR=/tmp/g. Go commands may need the sandbox disabled; "read-only file system" from the toolchain is the tell. - This repo lands through
amadan branch merge, never a GitHub PR. Branchworktree-tenant-email-senders. CI is.amadan/ci.d/, triggered by a push to theamadanremote; a branch pushed only tooriginhas been gated by nothing. - No capability change. Every task must leave the tree shippable on its own: an app that can send today sends after each commit, with the same credential, the same cap and the same delivered variables. The one deliberate behaviour change is task 11's attribution narrowing, which is a roll note, not a capability.
- Wire-shape changes are additive. Live
email.jsonobjects exist in production buckets and must still parse after every task. Thesendersarray is added beside the top-level reply fields, never instead of them, so an older CLI keeps working. - Existing AWS names never change.
UserName,StandaloneUserName,TenantNameandConfigSetNamerender exactly what they render today for the app sender;SenderNamesis a superset whose app-sender branch is pinned byte-identical by a test.RefreshPoliciesrecomputesStandaloneUserNamerather than readingsa.IAMUser(walk.go:364), so a changed rule would write a policy for a new principal and leave the live one stale. PerInstanceandPauseKindPerInstanceexist on the record and nothing in this plan sets them. The refusals that read them (task 12) are written and tested against a hand-set flag, because they are cheap now and load-bearing in step 5.- Copy: no em dashes. Full stop and a new sentence, lower case. Comments are full sentences saying why, never process history; log lines use
->between values. - Task 14 adds member-facing CLI copy. Run the
copy-reviewgate on the new strings before they land, as every user-facing string in this repo does. - Commit after every task with the trailer
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>.
---
File Structure
| File | Responsibility | |---|---| | internal/mail/sender.go (create) | Sender, Scope*/Source*/SenderAppID/PauseKindPerInstance, SendersPrefix, SenderKey, SenderNames, SenderNameSet, the caller index, SenderOfCaller, ResolveSenderNames | | internal/mail/sender_test.go (create) | record round-trip, the bounds enumeration, the caller index rules, SenderOfCaller | | internal/mail/mail.go (modify) | Key -> the sender key, LegacyKey, Get's fallback with the empty etag, Put always writing senders/app.json, GetSender/PutSender/ListSenders, Record.sender passthrough | | internal/mail/sweep.go (create) | SweepLegacyRecords: the one-shot email.json delete, a release later, off by default | | internal/mail/walk.go (modify) | caller index written after EnsureUser and deleted by RevokeStandalone/Teardown; tenant associations in EnsureDomain; disassociate before DeleteIdentity; SetSenderPaused | | internal/mail/provisioner.go, memprovisioner.go, aws.go (modify) | AssociateTenantResource / DisassociateTenantResource | | internal/mail/domain.go (modify) | ConfigSetARN(identityARN, set) extracted from VerifiedConfigSetARNs | | internal/console/store.go (modify) | mail_sender_counts DDL, the one-shot backfill behind PRAGMA user_version, DropLegacyMailCounts | | internal/console/mailcounts.go (modify) | RecordMailEvent/MailCounts take a sender; enforceMailLimits/resumeStaleCapPause work per sender | | internal/console/mailevents.go (modify) | mail_sender_counts joins mailAppTables; the explicit day case in mailAppTimeColumn and carryMailApp | | internal/console/sesevents.go (modify) | recordAndEnforceMail resolves the sender first | | internal/console/apimail.go (modify) | emailReply gains senders; the status door resumes the app sender | | mail_cli.go (modify) | apiEmailSender, apiEmailReply.Senders, a line per sender in printEmailStatus | | cmd/console/main.go (modify) | the legacy sweeps behind CARLOS_MAIL_LEGACY_SWEEP |
---
Task 1: The sender record
Files:
- Create:
internal/mail/sender.go,internal/mail/sender_test.go
Interfaces:
- Consumes:
Declared,GrantBlock,Provisioned,Standalone(internal/mail/mail.go:35,:46,:57,:95) unchanged;release.AppPrefix(internal/release/release.go:212). - Produces: ``
go const ( ScopeApp = "app" ScopeInstance = "instance" SourceDomain = "domain" SourceAddress = "address" SenderAppID = "app" PauseKindPerInstance = "per-instance" ) type Sender struct { ... } func SendersPrefix(acct, app string) string // apps/<acct>/<app>/email/senders/ func SenderKey(acct, app, id string) string // .../senders/<id>.json func ValidSenderID(id string) bool``
- [ ] Step 1: Write the failing test
internal/mail/sender_test.go (create):
package mail
import (
"encoding/json"
"strings"
"testing"
"time"
)
// The app sender is the shape every app has had since before senders
// existed, written under its own key. The four blocks move inside it
// unchanged, so a Sender decodes the same Declared/Grant/Provisioned/
// Standalone an email.json object carries.
func TestSenderCarriesTheFourBlocksUnchanged(t *testing.T) {
now := time.Date(2026, 9, 15, 12, 0, 0, 0, time.UTC)
sn := &Sender{
Schema: Schema, ID: SenderAppID, Slug: "", Scope: ScopeApp, Source: SourceDomain,
Declared: &Declared{EnvPrefix: "HARBOURMAIL_SMTP", From: "noreply@x.com", At: now, By: "owner@example.test"},
Grant: &GrantBlock{Domains: []string{"x.com"}, DailyCap: 50, Source: "tier:free", At: now, By: "tier"},
Provisioned: &Provisioned{IAMUser: "carlos-mail-bag-hm", KeyID: "AKIA1", Active: true, At: now},
Standalone: []*Standalone{{Name: "laptop", IAMUser: "carlos-mail-bag-hm+s+laptop", KeyID: "AKIA2", At: now, By: "owner@example.test"}},
}
body, err := json.Marshal(sn)
if err != nil {
t.Fatal(err)
}
// Nothing this sender does not carry appears on the wire: per_instance,
// host, address, identity, tenant_arn and salt are all omitempty, so an
// app sender's object is the four blocks plus its own identity.
for _, absent := range []string{"per_instance", "host", "address", "identity", "tenant_arn", "salt"} {
if strings.Contains(string(body), absent) {
t.Errorf("an app sender's wire form carries %q: %s", absent, body)
}
}
var back Sender
if err := json.Unmarshal(body, &back); err != nil {
t.Fatal(err)
}
if back.Declared.EnvPrefix != "HARBOURMAIL_SMTP" || back.Grant.DailyCap != 50 ||
back.Provisioned.KeyID != "AKIA1" || len(back.Standalone) != 1 || back.Standalone[0].Name != "laptop" {
t.Fatalf("round trip lost a block: %+v", back)
}
if back.ID != SenderAppID || back.Scope != ScopeApp || back.Source != SourceDomain {
t.Fatalf("round trip lost the sender's identity: %+v", back)
}
}
func TestSenderKeysSitUnderTheAppsOwnPrefix(t *testing.T) {
if got := SendersPrefix("bag", "harbourmail"); got != "apps/bag/harbourmail/email/senders/" {
t.Fatalf("SendersPrefix = %q", got)
}
if got := SenderKey("bag", "harbourmail", SenderAppID); got != "apps/bag/harbourmail/email/senders/app.json" {
t.Fatalf("SenderKey = %q", got)
}
// An id is a bucket key segment, so a traversal or a separator is
// refused at the one gate every caller passes through.
for _, bad := range []string{"", "..", "a/b", "A", "x.json", strings.Repeat("a", 65)} {
if ValidSenderID(bad) {
t.Errorf("ValidSenderID(%q) = true", bad)
}
}
for _, good := range []string{"app", "book.example.com", "_carlos-1-abc123"} {
if !ValidSenderID(good) {
t.Errorf("ValidSenderID(%q) = false", good)
}
}
}- [ ] Step 2: Run it to verify it fails
go test -run 'TestSenderCarriesTheFourBlocksUnchanged|TestSenderKeysSitUnderTheAppsOwnPrefix' ./internal/mail/Expected: build failure, undefined: Sender.
- [ ] Step 3: Write the code
internal/mail/sender.go (create):
// The sender (senders design §5): a From address, a credential, a cap, a
// pause and a set of counts, owned by one record.
//
// Before this file an app had exactly one implicit sender and everything was
// keyed by (account, app): email.json, the IAM user, the DailyCap, the pause,
// the mail_counts rows. Two requests do not fit that shape (#147 sends as an
// address under an identity CARLOS does not own, #148 gives one instance per
// customer its own domain, credential, cap and pause), so the sender becomes
// a thing with a name. This step gives every app exactly ONE sender, "app",
// which is what it already had. Nothing here creates a second one.
package mail
import (
"regexp"
"github.com/carlosframework/platform/internal/release"
)
// The two scopes and the two sources a sender can have. Scope says what the
// sender belongs to; Source says where its authority to send comes from. An
// app sender on a domain the account verified is the only combination this
// release produces; the others are named here because SenderNames, the
// status reply and the caller index all branch on them, and a closed set
// that is written down once is a set nothing invents a third member of.
const (
ScopeApp = "app"
ScopeInstance = "instance"
SourceDomain = "domain"
SourceAddress = "address"
)
// SenderAppID is the id of the sender every app has always had. It is a
// constant rather than a literal because it is written into every migrated
// record, read back by SenderOfCaller and compared in a dozen places.
const SenderAppID = "app"
// PauseKindPerInstance is the pause an app sender carries once its app has
// moved to per-instance senders (design §6.3). It is never a cap pause and
// never auto-resumes: resumeStaleCapPause reactivates every key it touches,
// and re-arming the app credential that every instance still holds is
// exactly what per-instance mode exists to prevent. Nothing in this release
// sets it; the refusals that read it are in place so step 5 cannot land
// without them.
const PauseKindPerInstance = "per-instance"
// Sender is apps/<acct>/<app>/email/senders/<id>.json, one object per
// sender, each with its own CAS etag so two writers never contend over a
// sibling's state.
type Sender struct {
Schema int `json:"schema"`
ID string `json:"id"` // "app", or provision.InstanceID(host)
Slug string `json:"slug"` // the AWS-safe name component, see SenderNames
Scope string `json:"scope"` // ScopeApp | ScopeInstance
Host string `json:"host,omitempty"`
Source string `json:"source"` // SourceDomain | SourceAddress
// The four blocks keep their current shapes and move inside the sender
// unchanged, which is what lets a legacy email.json decode straight into
// one and what keeps every walk in walk.go compiling against Record.
Declared *Declared `json:"declared,omitempty"`
Grant *GrantBlock `json:"grant,omitempty"`
Provisioned *Provisioned `json:"provisioned,omitempty"`
Standalone []*Standalone `json:"standalone,omitempty"`
Identity string `json:"identity,omitempty"` // external identity ARN (#147, step 3)
Address string `json:"address,omitempty"` // the exact address, lowercased (#147, step 3)
TenantARN string `json:"tenant_arn,omitempty"` // from CreateTenant
// PerInstance marks an app sender whose app has moved to per-instance
// senders (design §6.3). Nothing in this release sets it. It lives on
// the app sender rather than being derived from a prefix listing
// because a listing cannot decide the question without racing an
// app-sender enable, and the transition has to be one CAS write.
PerInstance bool `json:"per_instance,omitempty"`
// Salt disambiguates a slug collision. The slug is 40 bits of a hash, so
// two senders can in principle want the same IAM user name; the caller
// index detects that and ResolveSenderNames recomputes the slug with a
// fresh salt, which has to be stored or the name would not be stable
// across the next enable.
Salt string `json:"salt,omitempty"`
}
// SendersPrefix is where an app's senders live. Inside apps/<acct>/<app>/ on
// purpose: release.PurgeApp's prefix sweep reaches it, so a purged app
// leaves no sender behind.
func SendersPrefix(acct, app string) string {
return release.AppPrefix(acct, app) + "email/senders/"
}
func SenderKey(acct, app, id string) string { return SendersPrefix(acct, app) + id + ".json" }
// senderIDRe is deliberately stricter than a host name: this string becomes
// a bucket key segment and half an IAM user name, and the only safe place to
// refuse a separator or a traversal is the one gate every caller passes
// through. It accepts a host (provision.InstanceID returns the host itself
// when it is short enough) and the underscore-prefixed digest form
// InstanceID falls back to for a long one.
var senderIDRe = regexp.MustCompile(`^_?[a-z0-9]([a-z0-9.-]{0,62}[a-z0-9])?$`)
// ValidSenderID reports whether id is usable as a sender id. ".json" is
// refused along with every other dotted suffix that would collide with the
// key's own extension only incidentally: what actually refuses it is the
// check below, which is why it is spelled out.
func ValidSenderID(id string) bool {
if id == "" || len(id) > 64 || id == "." || id == ".." {
return false
}
if !senderIDRe.MatchString(id) {
return false
}
return !hasJSONSuffix(id)
}
func hasJSONSuffix(id string) bool { return len(id) > 5 && id[len(id)-5:] == ".json" }- [ ] Step 4: Run it to verify it passes
go test -run 'TestSenderCarriesTheFourBlocksUnchanged|TestSenderKeysSitUnderTheAppsOwnPrefix' ./internal/mail/Expected: PASS.
- [ ] Step 5: Gate and commit
go build ./... && go vet ./... && go test ./... -count=1 && gofmt -l .
git add internal/mail/sender.go internal/mail/sender_test.go
git commit -m "mail: the sender record, its scopes, sources and key shape (plan 2, task 1)"---
Task 2: SenderNames and the IAM and SES bounds
Files:
- Modify:
internal/mail/sender.go,internal/mail/sender_test.go
Interfaces:
- Consumes:
UserName(policy.go:27),StandaloneUserName(policy.go:92),TenantName(policy.go:84),ConfigSetName(walk.go:375). - Produces: ``
go type SenderNameSet struct { Slug, IAMUser, Tenant, ConfigSet string } func SenderNames(acct, app, senderID, salt string) (SenderNameSet, error) func (n SenderNameSet) StandaloneUser(acct, app, credential string) (string, error) var ErrNameTooLong = errors.New("mail: name exceeds the AWS bound")``
- [ ] Step 1: Write the failing test
internal/mail/sender_test.go (append):
// The bounds, enumerated. SES caps a tenant name and a configuration-set
// name at 64 characters of [A-Za-z0-9_-]; IAM caps a user name at 64. An
// account sqid is at most 16 lowercase letters and an app name at most 32
// of [a-z0-9-], so the app sender always fits and the instance sender only
// fits because the app component is truncated to 24 with the hash carrying
// uniqueness.
func TestSenderNamesEnumeratesTheBounds(t *testing.T) {
const longAcct = "abcdefghijklmnop" // 16, the longest legal account
const longApp = "a123456789012345678901234567890b" // 32, the longest legal app
if len(longAcct) != 16 || len(longApp) != 32 {
t.Fatalf("the fixture names are not at the bound: %d %d", len(longAcct), len(longApp))
}
// The app sender's names are today's names, byte for byte.
app, err := SenderNames(longAcct, longApp, SenderAppID, "")
if err != nil {
t.Fatalf("app sender at the bound: %v", err)
}
if app.Slug != "" {
t.Errorf("app sender slug = %q, want empty", app.Slug)
}
if app.IAMUser != UserName(longAcct, longApp) {
t.Errorf("IAMUser = %q, want %q — an existing name must never change", app.IAMUser, UserName(longAcct, longApp))
}
if app.Tenant != TenantName(longAcct) || app.ConfigSet != ConfigSetName(longAcct) {
t.Errorf("tenant/set = %q/%q, want %q/%q", app.Tenant, app.ConfigSet, TenantName(longAcct), ConfigSetName(longAcct))
}
if len(app.IAMUser) > 64 || len(app.Tenant) > 64 || len(app.ConfigSet) > 64 {
t.Errorf("app sender over a bound: %d/%d/%d", len(app.IAMUser), len(app.Tenant), len(app.ConfigSet))
}
// The instance sender at the same bound. 12 + 16 + 1 + 24 + 3 + 8 = 64.
inst, err := SenderNames(longAcct, longApp, "book.customer.example.com", "")
if err != nil {
t.Fatalf("instance sender at the bound: %v", err)
}
if len(inst.Slug) != 8 {
t.Fatalf("slug = %q, want 8 characters", inst.Slug)
}
if len(inst.IAMUser) != 64 {
t.Fatalf("IAMUser = %q (%d), want exactly 64 at the bound", inst.IAMUser, len(inst.IAMUser))
}
if !strings.Contains(inst.IAMUser, "+i+"+inst.Slug) {
t.Errorf("IAMUser = %q, want it to end in +i+<slug>", inst.IAMUser)
}
if len(inst.Tenant) > 64 || !strings.HasSuffix(inst.Tenant, "-"+inst.Slug) {
t.Errorf("tenant = %q (%d)", inst.Tenant, len(inst.Tenant))
}
for _, name := range []string{inst.IAMUser, inst.Tenant, inst.ConfigSet} {
for _, r := range name {
if !strings.ContainsRune("abcdefghijklmnopqrstuvwxyz0123456789-_+.", r) {
t.Errorf("name %q holds %q, which SES or IAM does not accept", name, r)
}
}
}
// The slug is a function of (account, app, sender, salt) and nothing
// else, so a re-enable computes the same name.
again, _ := SenderNames(longAcct, longApp, "book.customer.example.com", "")
if again.Slug != inst.Slug {
t.Fatalf("slug is not stable: %q then %q", inst.Slug, again.Slug)
}
salted, _ := SenderNames(longAcct, longApp, "book.customer.example.com", "0f0f")
if salted.Slug == inst.Slug {
t.Fatal("a salt did not move the slug, so a collision could never be resolved")
}
// A standalone credential's name is unchanged where it fits and refused
// where it does not. carlos-mail- (12) + 16 + 1 + 32 + +s+ (3) + 31
// reaches 95 against IAM's 64, which nothing checked before this.
if _, err := app.StandaloneUser(longAcct, longApp, "a123456789012345678901234567890"); !errors.Is(err, ErrNameTooLong) {
t.Fatalf("the longest standalone name on the longest app = %v, want ErrNameTooLong", err)
}
short, err := SenderNames("bag", "hm", SenderAppID, "")
if err != nil {
t.Fatal(err)
}
got, err := short.StandaloneUser("bag", "hm", "laptop")
if err != nil {
t.Fatal(err)
}
if got != StandaloneUserName("bag", "hm", "laptop") {
t.Fatalf("StandaloneUser = %q, want %q — RefreshPolicies recomputes this name and must get the live one",
got, StandaloneUserName("bag", "hm", "laptop"))
}
}Add "errors" to the test file's imports.
- [ ] Step 2: Run it to verify it fails
go test -run TestSenderNamesEnumeratesTheBounds ./internal/mail/Expected: build failure, undefined: SenderNames.
- [ ] Step 3: Write the code
internal/mail/sender.go (append; add crypto/sha256, encoding/base32, errors, fmt, strings to the imports):
// The AWS bounds this file exists to respect. IAM caps a user name at 64
// characters and SES caps a tenant name and a configuration-set name at 64
// of [A-Za-z0-9_-]. The first draft of the senders design got this wrong:
// carlos-mail-<acct>-<app>+i+<instance-id> reaches 104 characters when the
// instance id is the host itself, which it is for any host of 40 characters
// or less.
const (
iamUserNameMax = 64
sesNameMax = 64
// senderAppComponentMax truncates the app name inside a slugged sender's
// names. 12 + 16 + 1 + 24 + 3 + 8 = 64 exactly, at the longest legal
// account and app. Uniqueness comes from the hash, not from the app
// component, so truncating it costs nothing but readability.
senderAppComponentMax = 24
// senderSlugLen is 40 bits of base32. Short enough to fit, long enough
// that a collision is rare, and detected either way by the caller index.
senderSlugLen = 8
)
// ErrNameTooLong is the refusal at create for a name no AWS bound can hold.
// It is refused rather than truncated because a truncated name would
// collide silently with its neighbour, and because the only name this can
// happen to today is a standalone credential's, whose name the member chose
// and can choose differently.
var ErrNameTooLong = errors.New("mail: name exceeds the AWS bound")
// SenderNameSet is every AWS name one sender owns.
type SenderNameSet struct {
Slug string // "" for the app sender, 8 base32 characters otherwise
IAMUser string
Tenant string
ConfigSet string
}
// SenderNames is THE rule for what a sender's AWS resources are called, and
// there is exactly one of it so that a bounds test can enumerate every form.
//
// The app sender's names are unchanged, deliberately and permanently:
// carlos-mail-<acct>-<app>, tenant and configuration set carlos-<acct>.
// Renaming an SES tenant means destroying it and recreating it along with
// every association it carries, and rewriting an IAM user name means a new
// principal with the old one left live and stale. So the rule is that
// existing shapes never change and a name that would exceed a bound is
// refused at create.
//
// Configuration sets stay per account even for a slugged sender: an identity
// has only one default set, and SES has no tenant-level default, so
// per-sender sets on a shared identity are impossible (design §3).
func SenderNames(acct, app, senderID, salt string) (SenderNameSet, error) {
if !release.ValidAccount(acct) {
return SenderNameSet{}, fmt.Errorf("mail: invalid account %q", acct)
}
if !release.ValidApp(app) {
return SenderNameSet{}, fmt.Errorf("mail: invalid app %q", app)
}
if !ValidSenderID(senderID) {
return SenderNameSet{}, fmt.Errorf("mail: invalid sender id %q", senderID)
}
n := SenderNameSet{Tenant: TenantName(acct), ConfigSet: ConfigSetName(acct)}
if senderID == SenderAppID {
n.IAMUser = UserName(acct, app)
return n, checkNames(n)
}
n.Slug = senderSlug(acct, app, senderID, salt)
short := app
if len(short) > senderAppComponentMax {
short = short[:senderAppComponentMax]
}
n.IAMUser = "carlos-mail-" + acct + "-" + short + "+i+" + n.Slug
n.Tenant = "carlos-" + acct + "-" + short + "-" + n.Slug
return n, checkNames(n)
}
// StandaloneUser is the name of one displayed credential belonging to this
// sender, bound-checked. The rendered string is byte-identical to
// StandaloneUserName for an app sender, because RefreshPolicies recomputes
// the name rather than reading sa.IAMUser (walk.go:364): a rule that
// rendered anything else would write a policy for a principal that does not
// exist and leave the live one stale.
func (n SenderNameSet) StandaloneUser(acct, app, credential string) (string, error) {
if !validStandaloneName(credential) {
return "", fmt.Errorf("mail: invalid credential name %q", credential)
}
name := StandaloneUserName(acct, app, credential)
if n.Slug != "" {
name = n.IAMUser + "+s+" + credential
}
if len(name) > iamUserNameMax {
return "", fmt.Errorf("mail: credential name %q makes the IAM user %d characters, over IAM's %d: choose a shorter name: %w",
credential, len(name), iamUserNameMax, ErrNameTooLong)
}
return name, nil
}
func checkNames(n SenderNameSet) error {
if len(n.IAMUser) > iamUserNameMax {
return fmt.Errorf("mail: IAM user %q is %d characters, over IAM's %d: %w", n.IAMUser, len(n.IAMUser), iamUserNameMax, ErrNameTooLong)
}
for _, name := range []string{n.Tenant, n.ConfigSet} {
if len(name) > sesNameMax {
return fmt.Errorf("mail: SES name %q is %d characters, over SES's %d: %w", name, len(name), sesNameMax, ErrNameTooLong)
}
}
return nil
}
// senderSlug is 40 bits of base32 over (account, app, sender, salt),
// lowercased so it reads like every other name this package renders. Nothing
// parses it back, which is why the caller index exists.
func senderSlug(acct, app, id, salt string) string {
sum := sha256.Sum256([]byte(acct + "/" + app + "/" + id + "/" + salt))
enc := base32.StdEncoding.WithPadding(base32.NoPadding).EncodeToString(sum[:])
return strings.ToLower(enc[:senderSlugLen])
}- [ ] Step 4: Run it to verify it passes
go test -run TestSenderNames ./internal/mail/Expected: PASS.
- [ ] Step 5: Gate and commit
go build ./... && go vet ./... && go test ./... -count=1 && gofmt -l .
git add internal/mail
git commit -m "mail: SenderNames owns every AWS name and its bound (plan 2, task 2)"---
Task 3: The migration to senders/app.json
Files:
- Modify:
internal/mail/mail.go(Keyat:114,Getat:130,Putat:154),internal/mail/mail_test.go(TestKeyIsUnderTheAppPrefixat:13-17, rewritten),internal/mail/walk.go(Teardown's delete at:1297)
Interfaces:
- Produces: ``
go func Key(acct, app string) string // apps/<acct>/<app>/email/senders/app.json func LegacyKey(acct, app string) string // apps/<acct>/<app>/email.json func Get(ctx context.Context, s release.Store, acct, app string) (*Record, string, error) // etag "" when served from LegacyKey func Put(ctx context.Context, s release.Store, acct, app string, rec *Record, etag string) (string, error) // always writes Key func GetSender(ctx context.Context, s release.Store, acct, app, id string) (*Sender, string, error) func PutSender(ctx context.Context, s release.Store, acct, app string, sn *Sender, etag string) (string, error) func ListSenders(ctx context.Context, s release.Store, acct, app string) ([]*Sender, error)``
- [ ] Step 1: Write the failing test
internal/mail/mail_test.go, REPLACING TestKeyIsUnderTheAppPrefix (:13-17), which pins Key at the old object and goes red the moment Key moves. It is the reason this task is not additive:
func TestKeyIsUnderTheAppPrefix(t *testing.T) {
if got, want := Key("bag", "harbourmail"), "apps/bag/harbourmail/email/senders/app.json"; got != want {
t.Fatalf("Key = %q, want %q", got, want)
}
// The pre-senders object keeps its name, because Get still reads it and
// the sweep still has to find it.
if got, want := LegacyKey("bag", "harbourmail"), "apps/bag/harbourmail/email.json"; got != want {
t.Fatalf("LegacyKey = %q, want %q", got, want)
}
}internal/mail/mail_test.go (append):
// The migration (design §5). Get prefers senders/app.json and falls back to
// email.json; when it served the fallback it returns an EMPTY etag, so every
// writer's Put — and they all pass back the etag they were given — is a
// create by construction, and a lost race is ErrCASConflict exactly as it is
// today.
func TestGetFallsBackToEmailJSONWithAnEmptyETag(t *testing.T) {
ctx := context.Background()
s := release.NewDirStore(t.TempDir())
now := time.Date(2026, 9, 15, 12, 0, 0, 0, time.UTC)
legacy := &Record{Schema: Schema, App: "harbourmail",
Declared: &Declared{EnvPrefix: "HARBOURMAIL_SMTP", From: "noreply@x.com", At: now, By: "owner@example.test"},
Provisioned: &Provisioned{IAMUser: "carlos-mail-bag-harbourmail", KeyID: "AKIA1", Active: true, At: now},
}
body, err := json.MarshalIndent(legacy, "", " ")
if err != nil {
t.Fatal(err)
}
if _, err := s.Put(ctx, LegacyKey("bag", "harbourmail"), append(body, '\n'), ""); err != nil {
t.Fatal(err)
}
rec, etag, err := Get(ctx, s, "bag", "harbourmail")
if err != nil || rec == nil {
t.Fatalf("Get = %+v %v", rec, err)
}
if etag != "" {
t.Fatalf("etag = %q, want empty so the next Put is a create", etag)
}
if rec.Provisioned.KeyID != "AKIA1" {
t.Fatalf("the fallback lost the record: %+v", rec)
}
// Put always writes the sender key and never the legacy one.
if _, err := Put(ctx, s, "bag", "harbourmail", rec, etag); err != nil {
t.Fatalf("Put after the fallback: %v", err)
}
if _, _, err := s.Get(ctx, Key("bag", "harbourmail")); err != nil {
t.Fatalf("senders/app.json was not written: %v", err)
}
// email.json is left in place, unwritten, for one release.
raw, _, err := s.Get(ctx, LegacyKey("bag", "harbourmail"))
if err != nil || !bytes.Contains(raw, []byte("AKIA1")) {
t.Fatalf("email.json was disturbed: %q %v", raw, err)
}
// Once the sender object exists it is preferred, with a real etag.
rec2, etag2, err := Get(ctx, s, "bag", "harbourmail")
if err != nil || rec2 == nil || etag2 == "" {
t.Fatalf("Get after the migration = %+v %q %v", rec2, etag2, err)
}
if rec2.Provisioned.KeyID != "AKIA1" || rec2.Declared.From != "noreply@x.com" {
t.Fatalf("the migrated record lost a block: %+v", rec2)
}
// A second writer still holding the fallback's empty etag loses the race
// rather than clobbering the migrated object.
if _, err := Put(ctx, s, "bag", "harbourmail", rec, ""); !errors.Is(err, release.ErrCASConflict) {
t.Fatalf("a create over an existing sender = %v, want ErrCASConflict", err)
}
}
// An app with neither object is "no email", a state and not an error.
func TestGetWithNothingWrittenIsNotAnError(t *testing.T) {
rec, etag, err := Get(context.Background(), release.NewDirStore(t.TempDir()), "bag", "harbourmail")
if rec != nil || etag != "" || err != nil {
t.Fatalf("Get = %+v %q %v", rec, etag, err)
}
}
// ListSenders answers for an app that has never been migrated, because the
// status reply reads it on every call and an unmigrated app must not look
// like an app with no senders at all.
func TestListSendersAnswersFromTheLegacyRecord(t *testing.T) {
ctx := context.Background()
s := release.NewDirStore(t.TempDir())
now := time.Date(2026, 9, 15, 12, 0, 0, 0, time.UTC)
body, _ := json.Marshal(&Record{Schema: Schema, App: "harbourmail",
Declared: &Declared{EnvPrefix: "HARBOURMAIL_SMTP", From: "noreply@x.com", At: now, By: "o@example.test"}})
if _, err := s.Put(ctx, LegacyKey("bag", "harbourmail"), body, ""); err != nil {
t.Fatal(err)
}
got, err := ListSenders(ctx, s, "bag", "harbourmail")
if err != nil || len(got) != 1 {
t.Fatalf("ListSenders = %+v %v, want one sender", got, err)
}
if got[0].ID != SenderAppID || got[0].Scope != ScopeApp || got[0].Source != SourceDomain {
t.Fatalf("the legacy record did not read back as the app sender: %+v", got[0])
}
}Add "bytes", "context", "encoding/json", "errors", "testing", "time" and "github.com/carlosframework/platform/internal/release" to mail_test.go's imports as needed.
- [ ] Step 2: Run it to verify it fails
go test -run 'TestGetFallsBack|TestListSendersAnswers' ./internal/mail/Expected: build failure, undefined: LegacyKey.
- [ ] Step 3: Write the code
internal/mail/mail.go, replacing Key, Get and Put (:114, :130-162):
// Key is the app sender's object. It moved from email.json to
// email/senders/app.json when senders got names (design §5); LegacyKey is
// still read, and still on disk, for one release.
func Key(acct, app string) string { return SenderKey(acct, app, SenderAppID) }
// LegacyKey is the pre-senders record. Read-only from here: Get falls back
// to it and hands back an EMPTY etag when it did, so the writer's Put is a
// create by construction and a lost race is ErrCASConflict exactly as it is
// today. It is left in place, unwritten, until SweepLegacyRecords deletes it
// a release later. A console rolled back after the first sender write loses
// the writes made in between, which is in the roll notes.
func LegacyKey(acct, app string) string { return release.AppPrefix(acct, app) + "email.json" }
// Get reads the app sender as a Record — the view every walk in walk.go
// works with. Absent is (nil, "", nil): "no email" is a state, not an error.
//
// The returned Record keeps a private handle on the Sender it was decoded
// from, so a read-modify-write through Put preserves the fields Record has
// no name for (PerInstance, TenantARN, Identity, Address, Salt). Without
// that, an ordinary rotate would silently clear a flag step 3 or step 5
// wrote.
func Get(ctx context.Context, s release.Store, acct, app string) (*Record, string, error) {
sn, etag, err := GetSender(ctx, s, acct, app, SenderAppID)
if err != nil || sn == nil {
return nil, "", err
}
return sn.record(app), etag, nil
}
// Put writes the app sender at the caller's CAS position; etag "" is
// create-only. It ALWAYS writes the sender key, never the legacy one.
func Put(ctx context.Context, s release.Store, acct, app string, rec *Record, etag string) (string, error) {
return PutSender(ctx, s, acct, app, rec.sender(), etag)
}
// GetSender reads one sender. The app sender falls back to LegacyKey with an
// empty etag; any other sender has no legacy form and is simply absent.
func GetSender(ctx context.Context, s release.Store, acct, app, id string) (*Sender, string, error) {
if !release.ValidAccount(acct) {
return nil, "", fmt.Errorf("mail: invalid account %q", acct)
}
if !release.ValidApp(app) {
return nil, "", fmt.Errorf("mail: invalid app %q", app)
}
if !ValidSenderID(id) {
return nil, "", fmt.Errorf("mail: invalid sender id %q", id)
}
raw, etag, err := s.Get(ctx, SenderKey(acct, app, id))
if err == nil {
var sn Sender
if err := json.Unmarshal(raw, &sn); err != nil {
return nil, "", fmt.Errorf("mail: %s: %w", SenderKey(acct, app, id), err)
}
return &sn, etag, nil
}
if !errors.Is(err, os.ErrNotExist) {
return nil, "", err
}
if id != SenderAppID {
return nil, "", nil
}
raw, _, err = s.Get(ctx, LegacyKey(acct, app))
if err != nil {
if errors.Is(err, os.ErrNotExist) {
return nil, "", nil
}
return nil, "", err
}
var rec Record
if err := json.Unmarshal(raw, &rec); err != nil {
return nil, "", fmt.Errorf("mail: %s: %w", LegacyKey(acct, app), err)
}
// The empty etag is the whole of the migration's safety: it makes the
// caller's Put a create, so two consoles racing the first sender write
// cannot both win.
return rec.sender(), "", nil
}
// PutSender writes one sender at the caller's CAS position.
func PutSender(ctx context.Context, s release.Store, acct, app string, sn *Sender, etag string) (string, error) {
if !ValidSenderID(sn.ID) {
return "", fmt.Errorf("mail: invalid sender id %q", sn.ID)
}
sn.Schema = Schema
if sn.Scope == "" {
sn.Scope = ScopeApp
}
if sn.Source == "" {
sn.Source = SourceDomain
}
body, err := json.MarshalIndent(sn, "", " ")
if err != nil {
return "", err
}
return s.Put(ctx, SenderKey(acct, app, sn.ID), append(body, '\n'), etag)
}
// ListSenders reads every sender an app has, app sender first and the rest
// by id. An app that has never been migrated answers with the one sender its
// legacy record describes, because the status reply reads this on every call
// and an unmigrated app must not read as an app with no senders at all.
func ListSenders(ctx context.Context, s release.Store, acct, app string) ([]*Sender, error) {
keys, err := s.List(ctx, SendersPrefix(acct, app))
if err != nil {
return nil, err
}
var rest []*Sender
var appSender *Sender
for _, k := range keys {
if !strings.HasSuffix(k, ".json") {
continue
}
raw, _, err := s.Get(ctx, k)
if err != nil {
continue
}
var sn Sender
if json.Unmarshal(raw, &sn) != nil {
continue
}
if sn.ID == SenderAppID {
appSender = &sn
continue
}
rest = append(rest, &sn)
}
if appSender == nil {
appSender, _, err = GetSender(ctx, s, acct, app, SenderAppID)
if err != nil {
return nil, err
}
}
sort.Slice(rest, func(i, j int) bool { return rest[i].ID < rest[j].ID })
out := make([]*Sender, 0, len(rest)+1)
if appSender != nil {
out = append(out, appSender)
}
return append(out, rest...), nil
}internal/mail/mail.go, Record (:105) gains the passthrough and the two converters:
type Record struct {
Schema int `json:"schema"`
App string `json:"app"`
Declared *Declared `json:"declared,omitempty"`
Grant *GrantBlock `json:"grant,omitempty"`
Provisioned *Provisioned `json:"provisioned,omitempty"`
Standalone []*Standalone `json:"standalone,omitempty"`
// from is the sender this Record was decoded from, unexported so it
// never reaches the wire and never changes what a Record marshals to.
// It exists so a read-modify-write through Put carries forward the
// fields Record has no name for: an ordinary rotate must not clear a
// PerInstance flag or a TenantARN that something else wrote.
from *Sender
}
// sender renders this record as the app sender, preserving whatever the
// record was decoded from. A Record built from nothing (Declare's first
// call) gets a fresh app sender.
func (r *Record) sender() *Sender {
sn := r.from
if sn == nil {
sn = &Sender{ID: SenderAppID, Scope: ScopeApp, Source: SourceDomain}
}
sn.Schema = Schema
sn.Declared, sn.Grant, sn.Provisioned, sn.Standalone = r.Declared, r.Grant, r.Provisioned, r.Standalone
return sn
}
// record is the inverse: the four blocks as the walks read them, with the
// sender kept for the write back.
func (s *Sender) record(app string) *Record {
return &Record{Schema: Schema, App: app, Declared: s.Declared, Grant: s.Grant,
Provisioned: s.Provisioned, Standalone: s.Standalone, from: s}
}
// sender on a legacy Record: the pre-senders object IS the app sender, with
// the identity fields it never carried filled in.
func (r *Record) legacySender() *Sender {
return &Sender{Schema: Schema, ID: SenderAppID, Scope: ScopeApp, Source: SourceDomain,
Declared: r.Declared, Grant: r.Grant, Provisioned: r.Provisioned, Standalone: r.Standalone}
}In GetSender's legacy branch use rec.legacySender() rather than rec.sender().
internal/mail/walk.go, Teardown (:1296-1300): delete both keys.
if rec != nil {
// Both objects: the sender the app is on, and the legacy one the
// migration deliberately leaves standing until the sweep. Leaving
// email.json behind here would resurrect the app's whole email
// record on the next Get.
if err := s.Delete(ctx, Key(acct, app)); err != nil {
return err
}
if err := s.Delete(ctx, LegacyKey(acct, app)); err != nil {
return err
}
}Add "sort" and "strings" to mail.go's imports.
- [ ] Step 4: Run it to verify it passes
go test ./internal/mail/ -count=1Expected: PASS, including every existing walk test unchanged.
- [ ] Step 5: Gate and commit
go build ./... && go vet ./... && go test ./... -count=1 && gofmt -l .
git add internal/mail
git commit -m "mail: the app sender is senders/app.json, with email.json as a read fallback (plan 2, task 3)"---
Task 4: The one-shot legacy sweep, off by default
Files:
- Create:
internal/mail/sweep.go,internal/mail/sweep_test.go - Modify:
cmd/console/main.go(besidego srv.StartAlertWatcher(...),:334)
Interfaces:
- Produces: ``
go func SweepLegacyRecords(ctx context.Context, s release.Store, enabled bool) (deleted int, err error)`Deletesapps/<acct>/<app>/email.jsonfor every app that already has asenders/app.json, and only then.enabled == falsewalks nothing and returns(0, nil)`.
- [ ] Step 1: Write the failing test
internal/mail/sweep_test.go (create):
package mail
import (
"context"
"encoding/json"
"errors"
"os"
"testing"
"time"
"github.com/carlosframework/platform/internal/release"
)
// The sweep is what finally removes email.json, a release after the
// migration stopped writing it. It is off by default and it never deletes a
// legacy record for an app that has not been migrated, because that record
// is still the only copy of that app's email.
func TestSweepLegacyRecordsOnlyRemovesMigratedApps(t *testing.T) {
ctx := context.Background()
s := release.NewDirStore(t.TempDir())
now := time.Date(2026, 9, 15, 12, 0, 0, 0, time.UTC)
seed := func(app string, migrated bool) {
body, _ := json.Marshal(&Record{Schema: Schema, App: app,
Declared: &Declared{EnvPrefix: "P", From: "noreply@x.com", At: now, By: "o@example.test"}})
if _, err := s.Put(ctx, LegacyKey("bag", app), body, ""); err != nil {
t.Fatal(err)
}
if migrated {
if _, err := PutSender(ctx, s, "bag", app, &Sender{ID: SenderAppID, Scope: ScopeApp, Source: SourceDomain}, ""); err != nil {
t.Fatal(err)
}
}
}
seed("migrated", true)
seed("untouched", false)
// Off by default: the flag is the whole guard, and a console that has
// not been told to sweep sweeps nothing.
if n, err := SweepLegacyRecords(ctx, s, false); n != 0 || err != nil {
t.Fatalf("the sweep ran while disabled: %d %v", n, err)
}
if _, _, err := s.Get(ctx, LegacyKey("bag", "migrated")); err != nil {
t.Fatalf("the disabled sweep deleted something: %v", err)
}
n, err := SweepLegacyRecords(ctx, s, true)
if err != nil {
t.Fatal(err)
}
if n != 1 {
t.Fatalf("deleted %d, want 1", n)
}
if _, _, err := s.Get(ctx, LegacyKey("bag", "migrated")); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("the migrated app's email.json survived: %v", err)
}
if _, _, err := s.Get(ctx, LegacyKey("bag", "untouched")); err != nil {
t.Fatalf("an unmigrated app's only record was deleted: %v", err)
}
// Running it twice is a no-op, which is what makes it safe at every start.
if n, err := SweepLegacyRecords(ctx, s, true); n != 0 || err != nil {
t.Fatalf("a second sweep = %d %v, want 0 and no error", n, err)
}
}- [ ] Step 2: Run it to verify it fails
go test -run TestSweepLegacyRecords ./internal/mail/Expected: build failure, undefined: SweepLegacyRecords.
- [ ] Step 3: Write the code
internal/mail/sweep.go (create):
// The one-shot sweep that finishes the senders migration (design §5).
package mail
import (
"context"
"errors"
"fmt"
"log"
"os"
"path"
"strings"
"github.com/carlosframework/platform/internal/release"
)
// SweepLegacyRecords deletes apps/<acct>/<app>/email.json for every app that
// already has a senders/app.json, and nothing else.
//
// It is guarded by a flag that defaults OFF, and it is written now rather
// than a release from now because the thing that makes it safe is knowing,
// while the migration is fresh in view, exactly which object it may remove.
// The rule is one sentence: a legacy record is deletable when, and only
// when, the sender object that replaced it exists. An unmigrated app's
// email.json is still the only copy of its email and is left alone, so a
// deployment that turns the flag on early loses nothing.
//
// A failure on one app is logged and the sweep continues: one unreadable
// object must not stop the other five hundred, and every deletion is
// independent.
//
// It walks with ListDirs, twice, and never with List. List(ctx, "apps/") on
// a production bucket returns EVERY object under every app — every release
// artifact of every version of every app — to find the handful of keys
// ending in /email.json, and it would do it at console start. ListDirs is
// the delimiter-aware listing (release/store.go:50-58): one page of
// accounts, one page of apps per account, then two Gets per app.
func SweepLegacyRecords(ctx context.Context, s release.Store, enabled bool) (int, error) {
if !enabled {
return 0, nil
}
accts, err := s.ListDirs(ctx, "apps/")
if err != nil {
return 0, fmt.Errorf("mail: sweep legacy records: %w", err)
}
deleted := 0
for _, acctPrefix := range accts {
// ListDirs hands back a FULL prefix ending in "/", never a bare
// name, so the account is the last segment of it.
acct := path.Base(strings.TrimSuffix(acctPrefix, "/"))
if !release.ValidAccount(acct) {
continue
}
apps, err := s.ListDirs(ctx, acctPrefix)
if err != nil {
log.Printf("mail: sweep legacy records: listing %s -> %v", acctPrefix, err)
continue
}
for _, appPrefix := range apps {
app := path.Base(strings.TrimSuffix(appPrefix, "/"))
if !release.ValidApp(app) {
continue
}
if _, _, err := s.Get(ctx, SenderKey(acct, app, SenderAppID)); err != nil {
if !errors.Is(err, os.ErrNotExist) {
log.Printf("mail: sweep legacy records: reading %s -> %v", SenderKey(acct, app, SenderAppID), err)
}
continue // not migrated: this object is still the only copy
}
legacy := LegacyKey(acct, app)
if _, _, err := s.Get(ctx, legacy); err != nil {
if !errors.Is(err, os.ErrNotExist) {
log.Printf("mail: sweep legacy records: reading %s -> %v", legacy, err)
}
continue // nothing to remove, which is the steady state
}
if err := s.Delete(ctx, legacy); err != nil {
log.Printf("mail: sweep legacy records: deleting %s -> %v", legacy, err)
continue
}
deleted++
}
}
return deleted, nil
}Imports: path rather than path/filepath, since these are bucket keys and not OS paths. fmt is still used by the first wrap.
cmd/console/main.go: there is no variable holding the release store to reuse. The Config literal starts at :265 and passes Releases: openStore() inline (:269), and srv.cfg is unexported (internal/console/server.go:680), so the sweep cannot read it back off the server. Hoist it:
// One release.Store, named, because the senders sweep below needs the
// same one the server got.
rel := openStore()
srv, err := console.NewServer(console.Config{
Origin: origin,
Keymail: envOr("CONSOLE_KEYMAIL", "https://keymail.dev"),
Registry: envOr("CARLOS_REGISTRY", "/data/registry.db"),
Releases: rel,Leave the rest of the literal untouched, including the Bucket comment below Releases that mentions openStore() by name.
Then, immediately after go srv.StartAlertWatcher(context.Background()) (:334):
// The senders migration's tail (tenant-email senders design §5): once a
// release has passed with every console writing senders/app.json, an
// operator sets CARLOS_MAIL_LEGACY_SWEEP=1 and the next start deletes the
// email.json objects that have already been replaced. Off by default, and
// it never touches an app that has not been migrated.
if os.Getenv("CARLOS_MAIL_LEGACY_SWEEP") == "1" {
go func() {
n, err := mail.SweepLegacyRecords(context.Background(), rel, true)
if err != nil {
log.Printf("console: mail legacy sweep: %v", err)
return
}
log.Printf("console: mail legacy sweep -> %d email.json objects removed", n)
}()
}Add the internal/mail import if cmd/console/main.go does not already have it.
- [ ] Step 4: Run it to verify it passes
go test -run TestSweepLegacyRecords ./internal/mail/ && go build ./...Expected: PASS and a clean build.
- [ ] Step 5: Gate and commit
go build ./... && go vet ./... && go test ./... -count=1 && gofmt -l .
git add internal/mail cmd/console
git commit -m "mail: the one-shot email.json sweep, behind a flag that defaults off (plan 2, task 4)"---
Task 5: The caller reverse index
Files:
- Modify:
internal/mail/sender.go,internal/mail/sender_test.go,internal/mail/walk.go(Enableat:669-673,CreateStandaloneat:1032-1035,RevokeStandaloneat:1094-1105,Teardownat:1139-1150)
Interfaces:
- Consumes:
release.ErrCASConflict,getDomainClaimETag's idempotence rule (domainclaim.go:119-142) as the pattern. - Produces: ``
go type CallerIndex struct { Schema int; IAMUser, Account, App, Sender string; CreatedAt time.Time } func CallerKey(iamUser string) string // email/callers/<iamuser>.json func GetCallerIndex(ctx context.Context, s release.Store, iamUser string) (*CallerIndex, error) func PutCallerIndex(ctx context.Context, s release.Store, iamUser, acct, app, sender string, now time.Time) error func DeleteCallerIndex(ctx context.Context, s release.Store, iamUser, acct, app, sender string) error func ResolveSenderNames(ctx context.Context, s release.Store, acct, app string, sn *Sender, now time.Time) (SenderNameSet, bool, error) var ErrCallerClaimed = errors.New("mail: caller name belongs to another sender")``
- [ ] Step 1: Write the failing test
internal/mail/sender_test.go (append):
// The caller index is create-only with the domain claim's idempotence rule:
// the same (account, app, sender) is success, a different one is a
// collision. It is what makes a slugged IAM user name resolvable back to
// the sender that owns it, since nothing parses a hash.
func TestCallerIndexIsCreateOnlyAndIdempotentForItsOwnSender(t *testing.T) {
ctx := context.Background()
s := release.NewDirStore(t.TempDir())
now := time.Date(2026, 9, 15, 12, 0, 0, 0, time.UTC)
const user = "carlos-mail-bag-hm+i+abcd1234"
if err := PutCallerIndex(ctx, s, user, "bag", "hm", "book.example.com", now); err != nil {
t.Fatal(err)
}
if err := PutCallerIndex(ctx, s, user, "bag", "hm", "book.example.com", now); err != nil {
t.Fatalf("a re-enable of the same sender was refused by the index it holds: %v", err)
}
if err := PutCallerIndex(ctx, s, user, "bag", "hm", "other.example.com", now); !errors.Is(err, ErrCallerClaimed) {
t.Fatalf("a different sender at the same name = %v, want ErrCallerClaimed", err)
}
got, err := GetCallerIndex(ctx, s, user)
if err != nil || got == nil || got.Account != "bag" || got.App != "hm" || got.Sender != "book.example.com" {
t.Fatalf("GetCallerIndex = %+v %v", got, err)
}
// Deletion is owner-gated, the way releaseDomainClaim is: a stranger
// cannot remove an entry it does not hold.
if err := DeleteCallerIndex(ctx, s, user, "bag", "hm", "someone.else"); err != nil {
t.Fatalf("a mismatched delete should be a quiet no-op: %v", err)
}
if got, _ := GetCallerIndex(ctx, s, user); got == nil {
t.Fatal("a mismatched delete removed the entry")
}
if err := DeleteCallerIndex(ctx, s, user, "bag", "hm", "book.example.com"); err != nil {
t.Fatal(err)
}
if got, _ := GetCallerIndex(ctx, s, user); got != nil {
t.Fatalf("the entry survived its own delete: %+v", got)
}
// Deleting again is a quiet no-op, so a retried teardown is not an error.
if err := DeleteCallerIndex(ctx, s, user, "bag", "hm", "book.example.com"); err != nil {
t.Fatal(err)
}
}
// A 40-bit slug needs collision detection, and the index is where it
// happens: the losing sender recomputes its slug with a salt that is stored
// on its record so the name stays stable across the next enable.
func TestResolveSenderNamesRetriesACollisionWithASalt(t *testing.T) {
ctx := context.Background()
s := release.NewDirStore(t.TempDir())
now := time.Date(2026, 9, 15, 12, 0, 0, 0, time.UTC)
sn := &Sender{ID: "book.example.com", Scope: ScopeInstance, Source: SourceDomain, Host: "book.example.com"}
first, err := SenderNames("bag", "hm", sn.ID, "")
if err != nil {
t.Fatal(err)
}
// Somebody else is already at the unsalted name.
if err := PutCallerIndex(ctx, s, first.IAMUser, "bag", "other", "book.example.com", now); err != nil {
t.Fatal(err)
}
names, salted, err := ResolveSenderNames(ctx, s, "bag", "hm", sn, now)
if err != nil {
t.Fatalf("ResolveSenderNames: %v", err)
}
if !salted {
t.Fatal("the collision did not report that the record must be written back with its new salt")
}
if sn.Salt == "" {
t.Fatal("no salt was stored on the record, so the next enable would collide again")
}
if names.IAMUser == first.IAMUser {
t.Fatalf("the slug did not move: %q", names.IAMUser)
}
got, err := GetCallerIndex(ctx, s, names.IAMUser)
if err != nil || got == nil || got.App != "hm" {
t.Fatalf("the index at the new name = %+v %v", got, err)
}
// The app sender takes the same path and never needs a salt.
appSender := &Sender{ID: SenderAppID, Scope: ScopeApp, Source: SourceDomain}
appNames, saltedApp, err := ResolveSenderNames(ctx, s, "bag", "hm", appSender, now)
if err != nil || saltedApp || appSender.Salt != "" {
t.Fatalf("the app sender was salted: %+v %v %v", appNames, saltedApp, err)
}
if appNames.IAMUser != UserName("bag", "hm") {
t.Fatalf("the app sender's name changed: %q", appNames.IAMUser)
}
}Add "context" and "github.com/carlosframework/platform/internal/release" to sender_test.go's imports.
- [ ] Step 2: Run it to verify it fails
go test -run 'TestCallerIndex|TestResolveSenderNames' ./internal/mail/Expected: build failure, undefined: PutCallerIndex.
- [ ] Step 3: Write the code
internal/mail/sender.go (append; add crypto/rand, encoding/hex, encoding/json, os, time and context to the imports):
// callersPrefix is the reverse index from an IAM caller name to the sender
// that owns it. GLOBAL, not under apps/<acct>/<app>/, because SES hands the
// ingest a bare user name with no account attached and the ingest has
// nowhere to start looking. The price of that placement is that
// release.PurgeApp's prefix sweep never reaches it, so every entry has an
// explicit delete: without one, a stale entry would attribute SES events to
// a purged app and insert counts for it.
const callersPrefix = "email/callers/"
func CallerKey(iamUser string) string { return callersPrefix + iamUser + ".json" }
// CallerIndex is the wire shape at CallerKey(IAMUser).
type CallerIndex struct {
Schema int `json:"schema"` // 1
IAMUser string `json:"iam_user"`
Account string `json:"account"`
App string `json:"app"`
Sender string `json:"sender"`
CreatedAt time.Time `json:"created_at"`
}
// ErrCallerClaimed means this IAM name already belongs to a different
// sender. For a slugged name that is a hash collision and
// ResolveSenderNames recomputes with a salt; for an unslugged one it is a
// genuine conflict and the caller surfaces it.
var ErrCallerClaimed = errors.New("mail: caller name belongs to another sender")
// GetCallerIndex reads one entry. Absent is (nil, nil), matching every other
// read in this package.
func GetCallerIndex(ctx context.Context, s release.Store, iamUser string) (*CallerIndex, error) {
raw, _, err := s.Get(ctx, CallerKey(iamUser))
if err != nil {
if errors.Is(err, os.ErrNotExist) {
return nil, nil
}
return nil, err
}
var c CallerIndex
if err := json.Unmarshal(raw, &c); err != nil {
return nil, fmt.Errorf("mail: caller index %s: %w", iamUser, err)
}
if c.Schema > 1 {
return nil, fmt.Errorf("mail: caller index %s: schema %d is newer than this binary understands, upgrade carlos", iamUser, c.Schema)
}
return &c, nil
}
// PutCallerIndex claims iamUser for one sender. Create-only, with
// claimDomain's rule (domainclaim.go:119): the same triple is idempotent
// success, any other triple is ErrCallerClaimed.
//
// Written right AFTER EnsureUser, which runs on every enable, so a re-enable
// converges it. Writing it before MintKey would be no better and writing it
// before EnsureUser would be worse: a crash between the index write and the
// user's creation would strand the name against a principal that does not
// exist.
func PutCallerIndex(ctx context.Context, s release.Store, iamUser, acct, app, sender string, now time.Time) error {
body, err := json.Marshal(CallerIndex{Schema: 1, IAMUser: iamUser, Account: acct, App: app, Sender: sender, CreatedAt: now})
if err != nil {
return err
}
for attempt := 0; ; attempt++ {
cur, err := GetCallerIndex(ctx, s, iamUser)
if err != nil {
return err
}
if cur == nil {
if _, err := s.Put(ctx, CallerKey(iamUser), body, ""); err == nil {
return nil
} else if errors.Is(err, release.ErrCASConflict) && attempt == 0 {
continue // lost the create race, re-read and judge the winner
} else {
return err
}
}
if cur.Account != acct || cur.App != app || cur.Sender != sender {
return ErrCallerClaimed
}
return nil // already ours
}
}
// DeleteCallerIndex removes an entry, but only when it is held by this
// sender, the same owner gate releaseDomainClaim applies. An entry already
// absent is a quiet no-op, so a retried teardown is not an error.
func DeleteCallerIndex(ctx context.Context, s release.Store, iamUser, acct, app, sender string) error {
cur, err := GetCallerIndex(ctx, s, iamUser)
if err != nil || cur == nil {
return err
}
if cur.Account != acct || cur.App != app || cur.Sender != sender {
return nil
}
return s.Delete(ctx, CallerKey(iamUser))
}
// ResolveSenderNames renders a sender's names AND claims them in the caller
// index, recomputing a slugged name with a fresh salt on a collision.
//
// It reports whether the salt moved, because a moved salt has to be written
// back to the sender record under its own CAS or the next enable computes a
// different name again and the estate grows an orphan.
func ResolveSenderNames(ctx context.Context, s release.Store, acct, app string, sn *Sender, now time.Time) (SenderNameSet, bool, error) {
salted := false
for attempt := 0; attempt < 4; attempt++ {
names, err := SenderNames(acct, app, sn.ID, sn.Salt)
if err != nil {
return SenderNameSet{}, salted, err
}
err = PutCallerIndex(ctx, s, names.IAMUser, acct, app, sn.ID, now)
if err == nil {
sn.Slug = names.Slug
return names, salted, nil
}
if !errors.Is(err, ErrCallerClaimed) {
return SenderNameSet{}, salted, err
}
if names.Slug == "" {
// An unslugged name is deterministic, so a conflict here is not
// a hash collision: another account or app is genuinely using
// this principal, and a salt would not help.
return SenderNameSet{}, salted, fmt.Errorf("mail: IAM user %s is already claimed by another sender: %w", names.IAMUser, err)
}
buf := make([]byte, 4)
if _, err := rand.Read(buf); err != nil {
return SenderNameSet{}, salted, err
}
sn.Salt = hex.EncodeToString(buf)
salted = true
}
return SenderNameSet{}, salted, fmt.Errorf("mail: could not find a free IAM name for sender %s/%s/%s after four attempts", acct, app, sn.ID)
}internal/mail/walk.go, Enable, right after EnsureUser (:670-673):
user := UserName(acct, app)
arn, err := p.EnsureUser(ctx, user, UserPath, arns, configSets, names)
if err != nil {
return nil, fmt.Errorf("mail: ensure user %s: %w", user, err)
}
// The reverse index, right after EnsureUser and before MintKey (design
// §5). EnsureUser runs on every enable, so a re-enable converges the
// index; putting it after MintKey would strand a sender forever when a
// crash landed between the two.
if err := PutCallerIndex(ctx, s, user, acct, app, SenderAppID, now); err != nil {
return nil, fmt.Errorf("mail: index caller %s: %w", user, err)
}CreateStandalone, after its EnsureUser (:1033-1035):
if _, err := p.EnsureUser(ctx, user, UserPath, arns, configSets, names); err != nil {
return StandaloneCreds{}, err
}
// A standalone credential sends for the app sender, so its caller name
// resolves to the same sender the delivered credential does.
if err := PutCallerIndex(ctx, s, user, acct, app, SenderAppID, now); err != nil {
return StandaloneCreds{}, fmt.Errorf("mail: index caller %s: %w", user, err)
}Also in CreateStandalone, before StandaloneUserName is called (:1032), bound-check the name so an over-long one is refused at create rather than at IAM:
senderNames, err := SenderNames(acct, app, SenderAppID, "")
if err != nil {
return StandaloneCreds{}, err
}
user, err := senderNames.StandaloneUser(acct, app, name)
if err != nil {
return StandaloneCreds{}, err
}RevokeStandalone (:1096-1104), before DeleteUser:
if sa.Name == name {
found = true
if err := DeleteCallerIndex(ctx, s, sa.IAMUser, acct, app, SenderAppID); err != nil {
return err
}
if err := p.DeleteUser(ctx, sa.IAMUser); err != nil {
return err
}
continue
}Teardown (:1139-1150), beside each DeleteUser:
if rec != nil {
if rec.Provisioned != nil {
// The index sits outside apps/<acct>/<app>/, so PurgeApp's
// prefix sweep never reaches it. Without this delete a stale
// entry would attribute SES events to a purged app and insert
// counts for it.
if err := DeleteCallerIndex(ctx, s, rec.Provisioned.IAMUser, acct, app, SenderAppID); err != nil {
return err
}
if err := p.DeleteUser(ctx, rec.Provisioned.IAMUser); err != nil {
return err
}
}
for _, sa := range rec.Standalone {
if err := DeleteCallerIndex(ctx, s, sa.IAMUser, acct, app, SenderAppID); err != nil {
return err
}
if err := p.DeleteUser(ctx, sa.IAMUser); err != nil {
return err
}
}
}An account is only ever deleted once it holds no apps (handleDeleteAccount, internal/console/ownership.go:82, and DeleteAccount re-checks it in its own transaction), so the account reap reaches the index through each app's Teardown and needs no sweep of its own. That is a ruling, recorded here so the next reader does not go looking for one.
- [ ] Step 4: Run it to verify it passes
go test ./internal/mail/ -count=1Expected: PASS.
- [ ] Step 5: Gate and commit
go build ./... && go vet ./... && go test ./... -count=1 && gofmt -l .
git add internal/mail
git commit -m "mail: the caller reverse index, written after EnsureUser and deleted with the principal (plan 2, task 5)"---
Task 6: SenderOfCaller
Files:
- Modify:
internal/mail/sender.go,internal/mail/sender_test.go
Interfaces:
- Consumes:
AppOfCaller(policy.go:42),GetCallerIndex. - Produces: ``
go func SenderOfCaller(ctx context.Context, s release.Store, caller string) (acct, app, sender string, ok bool)``
- [ ] Step 1: Write the failing test
internal/mail/sender_test.go (append):
// Parse first, index second. Every app enabled today short-circuits Enable
// before MintKey, so it has no index entry until it next rotates: a
// Get-only resolver would silently stop attributing their events the moment
// it shipped.
func TestSenderOfCallerParsesBeforeItReads(t *testing.T) {
ctx := context.Background()
s := release.NewDirStore(t.TempDir())
now := time.Date(2026, 9, 15, 12, 0, 0, 0, time.UTC)
// No index entry exists at all, and the delivered and standalone names
// still resolve, to the app sender.
for _, caller := range []string{
UserName("bag", "harbourmail"),
StandaloneUserName("bag", "harbourmail", "laptop"),
"arn:aws:iam::123456789012:user/carlos-mail/carlos-mail-bag-harbourmail",
} {
acct, app, sender, ok := SenderOfCaller(ctx, s, caller)
if !ok || acct != "bag" || app != "harbourmail" || sender != SenderAppID {
t.Errorf("SenderOfCaller(%q) = %q %q %q %v", caller, acct, app, sender, ok)
}
}
// A name nobody CARLOS minted stays unknown, and costs no read.
for _, caller := range []string{"", "someone-else", "carlos-mail-bag-harbourmail+s+"} {
if _, _, _, ok := SenderOfCaller(ctx, s, caller); ok {
t.Errorf("SenderOfCaller(%q) claimed to know it", caller)
}
}
// A slugged name is unparseable by construction and comes from the index.
names, err := SenderNames("bag", "harbourmail", "book.example.com", "")
if err != nil {
t.Fatal(err)
}
if _, _, _, ok := SenderOfCaller(ctx, s, names.IAMUser); ok {
t.Fatal("a slugged name resolved before it was ever indexed")
}
if err := PutCallerIndex(ctx, s, names.IAMUser, "bag", "harbourmail", "book.example.com", now); err != nil {
t.Fatal(err)
}
acct, app, sender, ok := SenderOfCaller(ctx, s, names.IAMUser)
if !ok || acct != "bag" || app != "harbourmail" || sender != "book.example.com" {
t.Fatalf("SenderOfCaller(slug) = %q %q %q %v", acct, app, sender, ok)
}
}- [ ] Step 2: Run it to verify it fails
go test -run TestSenderOfCallerParsesBeforeItReads ./internal/mail/Expected: build failure, undefined: SenderOfCaller.
- [ ] Step 3: Write the code
internal/mail/sender.go (append; add log and strings if not already imported):
// SenderOfCaller resolves the IAM identity SES stamps on an event
// (mail.tags["ses:caller-identity"]) to the sender that owns it.
//
// PARSE FIRST, in two steps, and the order is load-bearing. AppOfCaller
// answers for carlos-mail-<acct>-<app> and its +s+ standalones, which is
// every sender that exists today, and it costs nothing. Only a slugged +i+
// name, which is a hash and unparseable by construction, reaches the index
// Get. Every app enabled today short-circuits Enable before MintKey
// (walk.go:677), so it has no index entry until it next rotates: a Get-only
// resolver would silently stop writing the by_sender log rows that
// sesevents.go writes now, for every app in the fleet, on the roll.
//
// An unreadable index entry is logged and answers "unknown", which puts the
// event back on the domain-wide fallback rather than dropping it.
func SenderOfCaller(ctx context.Context, s release.Store, caller string) (string, string, string, bool) {
if acct, app, ok := AppOfCaller(caller); ok {
return acct, app, SenderAppID, true
}
if !strings.Contains(caller, "+i+") {
return "", "", "", false
}
name := caller
if i := strings.LastIndex(name, "/"); i >= 0 {
name = name[i+1:]
}
idx, err := GetCallerIndex(ctx, s, name)
if err != nil {
log.Printf("mail: caller index %s -> %v", name, err)
return "", "", "", false
}
if idx == nil || !release.ValidAccount(idx.Account) || !release.ValidApp(idx.App) || !ValidSenderID(idx.Sender) {
return "", "", "", false
}
return idx.Account, idx.App, idx.Sender, true
}- [ ] Step 4: Run it to verify it passes
go test -run TestSenderOfCaller ./internal/mail/Expected: PASS.
- [ ] Step 5: Gate and commit
go build ./... && go vet ./... && go test ./... -count=1 && gofmt -l .
git add internal/mail
git commit -m "mail: SenderOfCaller parses first and reads the index only for a slug (plan 2, task 6)"---
Task 7: Tenant resource associations
Files:
- Modify:
internal/mail/provisioner.go(:49),internal/mail/memprovisioner.go(:40,:86),internal/mail/aws.go(sesAPIat:54, afterEnsureTenantat:182),internal/mail/domain.go(VerifiedConfigSetARNsat:164),internal/mail/walk.go(EnsureDomain's association block goes in AFTERPutDomainat:280, besideRefreshPoliciesat:297;Teardownat:1190-1208),internal/mail/memprovisioner_test.go,internal/mail/walk_domain_test.go
THE ASSOCIATION GOES AFTER PutDomain, AND THAT PLACEMENT IS LOAD-BEARING. Do not associate beside EnsureIdentity (:202-205) or beside EnsureConfigSet (:213-216). Teardown's orphan-claim sweep calls p.DeleteIdentity (walk.go:1288) on a claim that has NO domain record, so it never reaches the disassociate this task adds to the region loop — and once MemProvisioner.DeleteIdentity refuses an associated identity, an association written before the record exists makes that sweep unreachable. Associating only after the record is durable means an orphan claim never holds an associated identity, so the sweep still deletes it.
Interfaces:
- Produces: ``
go // Provisioner AssociateTenantResource(ctx context.Context, tenant, resourceARN string) error DisassociateTenantResource(ctx context.Context, tenant, resourceARN string) error // mail func ConfigSetARN(identityARN, configSet string) (string, bool)``
- [ ] Step 1: Write the failing test
internal/mail/walk_domain_test.go (append):
// Step 2 associates each account's EXISTING tenant with its identities and
// its configuration set (design §7, "Associations are wired in step 2, not
// here"). Associating a resource does not modify it and changes no
// behaviour on its own: no send carries X-SES-TENANT yet. It is done now so
// that step 3 can rely on it, and because SES validates the association on
// every tenant-attributed send.
func TestEnsureDomainAssociatesTheIdentityAndTheSetWithTheAccountsTenant(t *testing.T) {
ctx := context.Background()
s, p := release.NewDirStore(t.TempDir()), NewMemProvisioner("1")
now := time.Date(2026, 9, 15, 12, 0, 0, 0, time.UTC)
if _, err := Declare(ctx, s, "bag", "harbourmail", "HARBOURMAIL_SMTP",
"noreply@harbourmail.bag.oncarlos.com", "owner@example.test", now); err != nil {
t.Fatal(err)
}
rec, _, err := EnsureDomain(ctx, s, p, dnsrec.NewMemWriter(), testDeployment(),
"bag", "harbourmail", "harbourmail.bag.oncarlos.com", now)
if err != nil {
t.Fatal(err)
}
tenant := TenantName("bag")
identity := rec.Regions["eu-west-1"].IdentityARN
set, ok := ConfigSetARN(identity, ConfigSetName("bag"))
if !ok {
t.Fatalf("no configuration-set ARN composed from %q", identity)
}
for _, arn := range []string{identity, set} {
if !p.TenantResources[tenant][arn] {
t.Errorf("%s is not associated with %s: %v", arn, tenant, p.TenantResources[tenant])
}
}
// A re-run associates the same pair again without error: every verb on
// this seam is idempotent, which is what makes the walk re-runnable.
if _, _, err := EnsureDomain(ctx, s, p, dnsrec.NewMemWriter(), testDeployment(),
"bag", "harbourmail", "harbourmail.bag.oncarlos.com", now); err != nil {
t.Fatalf("re-run: %v", err)
}
// An identity cannot be deleted while an association exists (design §3),
// so teardown removes the association first. MemProvisioner models that
// refusal, because without it the first real teardown would fail.
if err := Teardown(ctx, s, p, dnsrec.NewMemWriter(), testDeployment(), "bag", "harbourmail"); err != nil {
t.Fatalf("Teardown: %v", err)
}
if p.TenantResources[tenant][identity] {
t.Error("the identity association survived teardown, so the identity could never be deleted")
}
// The configuration set is per ACCOUNT and outlives any one app, so its
// association is never removed by an app's teardown.
if !p.TenantResources[tenant][set] {
t.Error("an app's teardown removed the account's configuration-set association")
}
}internal/mail/memprovisioner_test.go (append):
// The fake refuses to delete an identity that is still associated with a
// tenant, because SES does. A walk that got the order wrong would pass
// against a permissive fake and fail on the first real teardown.
func TestMemProvisionerRefusesToDeleteAnAssociatedIdentity(t *testing.T) {
ctx := context.Background()
p := NewMemProvisioner("1")
id, err := p.EnsureIdentity(ctx, "eu-west-1", "x.com", "mail-eu-west-1.x.com")
if err != nil {
t.Fatal(err)
}
if err := p.AssociateTenantResource(ctx, "carlos-bag", id.ARN); err != nil {
t.Fatal(err)
}
if err := p.DeleteIdentity(ctx, "eu-west-1", "x.com"); err == nil {
t.Fatal("an associated identity was deleted")
}
if err := p.DisassociateTenantResource(ctx, "carlos-bag", id.ARN); err != nil {
t.Fatal(err)
}
// Disassociating twice is a success, like every other verb on this seam.
if err := p.DisassociateTenantResource(ctx, "carlos-bag", id.ARN); err != nil {
t.Fatal(err)
}
if err := p.DeleteIdentity(ctx, "eu-west-1", "x.com"); err != nil {
t.Fatal(err)
}
}- [ ] Step 2: Run it to verify it fails
go test -run 'TestEnsureDomainAssociates|TestMemProvisionerRefusesToDeleteAnAssociated' ./internal/mail/Expected: build failure, p.TenantResources undefined.
- [ ] Step 3: Write the code
internal/mail/provisioner.go, in the Provisioner interface after SetIdentityConfigSet (:66):
// AssociateTenantResource associates an identity, a configuration set or
// a template with a tenant. Associating does not modify the resource and
// one resource may belong to many tenants, so this is safe to call on an
// identity another product owns. It has exactly one side effect on the
// owner: an identity cannot be deleted while an association exists,
// which is why DisassociateTenantResource has to run before
// DeleteIdentity. Idempotent, like every verb here.
AssociateTenantResource(ctx context.Context, tenant, resourceARN string) error
// DisassociateTenantResource removes one association. "Already gone" is
// success.
DisassociateTenantResource(ctx context.Context, tenant, resourceARN string) errorinternal/mail/memprovisioner.go:
// TenantResources is tenant -> resource ARN -> associated. An identity
// with an association cannot be deleted, mirroring SES, because a walk
// that tore down in the wrong order would pass against a permissive fake
// and fail on the first real teardown.
TenantResources map[string]map[string]boolinitialised in NewMemProvisioner as TenantResources: map[string]map[string]bool{}, plus:
func (p *MemProvisioner) AssociateTenantResource(_ context.Context, tenant, resourceARN string) error {
p.mu.Lock()
defer p.mu.Unlock()
if err := p.takeFailure("AssociateTenantResource"); err != nil {
return err
}
if p.TenantResources[tenant] == nil {
p.TenantResources[tenant] = map[string]bool{}
}
p.TenantResources[tenant][resourceARN] = true
return nil
}
func (p *MemProvisioner) DisassociateTenantResource(_ context.Context, tenant, resourceARN string) error {
p.mu.Lock()
defer p.mu.Unlock()
if err := p.takeFailure("DisassociateTenantResource"); err != nil {
return err
}
delete(p.TenantResources[tenant], resourceARN)
return nil
}and DeleteIdentity (:191) grows the refusal:
func (p *MemProvisioner) DeleteIdentity(_ context.Context, region, domain string) error {
p.mu.Lock()
defer p.mu.Unlock()
if err := p.takeFailure("DeleteIdentity"); err != nil {
return err
}
got, ok := p.Identities[identityKey(region, domain)]
if ok {
for tenant, resources := range p.TenantResources {
if resources[got.ARN] {
return fmt.Errorf("mail: identity %s is still associated with tenant %s", domain, tenant)
}
}
}
delete(p.Identities, identityKey(region, domain))
return nil
}internal/mail/aws.go: sesAPI (:54) gains the two operations, and after EnsureTenant (:182):
// AssociateTenantResource associates an identity or a configuration set with
// a tenant. This is the call the file header said nothing here made yet: SES
// validates the association on every tenant-attributed send, so it must exist
// before any send carries X-SES-TENANT. Associating changes nothing about the
// resource itself, so it is safe on an identity another product owns.
func (p *awsProvisioner) AssociateTenantResource(ctx context.Context, tenant, resourceARN string) error {
_, err := p.ses.CreateTenantResourceAssociation(ctx, &sesv2.CreateTenantResourceAssociationInput{
TenantName: aws.String(tenant),
ResourceArn: aws.String(resourceARN),
})
if err != nil && !isAlreadyExists(err) {
return fmt.Errorf("mail: associate %s with tenant %q: %w", resourceARN, tenant, err)
}
return nil
}
// DisassociateTenantResource removes one association. It has to run before
// DeleteEmailIdentity: SES refuses to delete an identity while an
// association exists, which is also the cross-team contract for a borrowed
// identity (design §7) — while CARLOS's association stands, the owner cannot
// delete their own identity.
func (p *awsProvisioner) DisassociateTenantResource(ctx context.Context, tenant, resourceARN string) error {
_, err := p.ses.DeleteTenantResourceAssociation(ctx, &sesv2.DeleteTenantResourceAssociationInput{
TenantName: aws.String(tenant),
ResourceArn: aws.String(resourceARN),
})
if err != nil && !isNotFound(err) {
return fmt.Errorf("mail: disassociate %s from tenant %q: %w", resourceARN, tenant, err)
}
return nil
}Update the file's TENANT ATTRIBUTION header (:19-25) and EnsureTenant's PRECONDITION FOR SENDING, NOT YET MET HERE paragraph (above :182) to say that EnsureDomain now wires the associations, and record the spike's answer (design §4.1): the association is inert for the identity's owner in every respect except deletion, which is confirmed against the live API, and ses:TenantName IS enforced over SMTP. No send carries X-SES-TENANT in this release all the same, because nothing in step 2 composes an outgoing message or writes a tenant-conditioned policy. Step 3 is where both land.
internal/mail/domain.go, extract the ARN composition VerifiedConfigSetARNs (:170-174) does inline and call it from there:
// ConfigSetARN composes a configuration set's ARN from an identity ARN in
// the same account and region, which is the only place this package has
// those two values. Reports false for an ARN it cannot parse, so a caller
// never builds a resource string out of a malformed one.
func ConfigSetARN(identityARN, configSet string) (string, bool) {
if identityARN == "" || configSet == "" {
return "", false
}
parts := strings.Split(identityARN, ":")
if len(parts) != 6 {
return "", false
}
return strings.Join(parts[:5], ":") + ":configuration-set/" + configSet, true
}internal/mail/walk.go, EnsureDomain: BOTH associations go in one block immediately after PutDomain succeeds (:280-282) and immediately before the GRANT BEFORE BIND comment and RefreshPolicies (:297). tenant (:181), ident (:202) and configSet (:213) are all still in scope there.
// The account's tenant gets this identity, and its configuration set
// (design §7). Association does not modify the resource and one resource
// may belong to many tenants, so this changes nothing anyone can observe
// today. It is wired now because SES validates the association on every
// tenant-attributed send and step 3 sends that way.
//
// AFTER PutDomain, not beside EnsureIdentity, and that ordering is the
// whole of what keeps Teardown's orphan-claim sweep working. An
// association is the one thing that stops an identity being deleted, and
// the sweep (walk.go:1288) deletes the identity behind a claim that has
// NO record — so it never runs the disassociate below. Associate before
// the record is durable and a crash between the two leaves an identity
// nothing can ever reap.
if err := p.AssociateTenantResource(ctx, tenant, ident.ARN); err != nil {
return nil, nil, fmt.Errorf("mail: associate identity %s with tenant %s: %w", domain, tenant, err)
}
if configSet != "" {
if setARN, ok := ConfigSetARN(ident.ARN, configSet); ok {
if err := p.AssociateTenantResource(ctx, tenant, setARN); err != nil {
return nil, nil, fmt.Errorf("mail: associate configuration set %s with tenant %s: %w", configSet, tenant, err)
}
}
}internal/mail/walk.go, Teardown's region loop (:1190-1208), before DeleteIdentity:
// SES refuses to delete an identity while a tenant association
// exists, so the association goes first. The account's
// configuration set is NOT disassociated: it is per account and
// outlives any one app.
if st.IdentityARN != "" {
if err := p.DisassociateTenantResource(ctx, TenantName(acct), st.IdentityARN); err != nil {
return err
}
}
if err := p.DeleteIdentity(ctx, region, cur.Domain); err != nil {
return err
}- [ ] Step 4: Run it to verify it passes
go test ./internal/mail/ -count=1Expected: PASS, and in particular these two stay green, because they are the ones the new refusal can break:
--- PASS: TestTeardownOrphanSweepDestroysTheIdentityBeforeReleasing
--- PASS: TestTeardownOrphanSweepDoesNotCrossAccountsOnAMatchingAppNameIf either of those goes red, the fix is the ORDERING, never the fake. Both live in internal/mail/walk_domainclaim_test.go (:493 and :589). Each injects p.FailNext["EnsureConfigSet"] (:503, :601) so EnsureDomain dies after EnsureIdentity and before any record is written, then expects Teardown to succeed (:516, :615) by destroying the orphaned identity. If they fail with "identity is still associated with tenant", the association was written too early: move it back to after PutDomain. Do NOT weaken MemProvisioner.DeleteIdentity's refusal, and do not make the orphan sweep disassociate first — the refusal is what SES actually does (design §4.1, confirmed against the live API), and a fake that permits the delete would pass here and fail on the first real teardown.
- [ ] Step 5: Gate and commit
go build ./... && go vet ./... && go test ./... -count=1 && gofmt -l .
git add internal/mail
git commit -m "mail: associate each account's tenant with its identities and its configuration set (plan 2, task 7)"---
Task 8: mail_sender_counts, its backfill and the legacy drop
Files:
- Modify:
internal/console/store.go(the create loop at:92-297, the ALTER loop at:300-370),internal/console/mailcounts_test.go
Interfaces:
- Produces: ``
go func (s *Store) DropLegacyMailCounts(enabled bool) error`plus themailsendercountstable and a one-shot backfill guarded byPRAGMA user_version`.
- [ ] Step 1: Write the failing test
internal/console/mailcounts_test.go (append):
// SQLite cannot widen a primary key, so per-sender counts are a new table,
// backfilled once from mail_counts with sender = 'app'. The backfill runs at
// most once for the life of the database: a second migrate must not
// resurrect rows a purge or a rename has since removed.
func TestMailSenderCountsAreBackfilledExactlyOnce(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "console.db")
st, err := Open(path)
if err != nil {
t.Fatal(err)
}
if _, err := st.db.Exec(`INSERT INTO mail_counts (account, app, day, sends, bounces, complaints, rejects, deliveries)
VALUES ('bag', 'harbourmail', '2026-09-14', 7, 1, 0, 0, 6)`); err != nil {
t.Fatal(err)
}
// Force the backfill to run again by winding the marker back, which is
// what a console rolled back and forward would look like.
if _, err := st.db.Exec(`PRAGMA user_version = 0`); err != nil {
t.Fatal(err)
}
if err := st.migrate(); err != nil {
t.Fatal(err)
}
var sends, deliveries int
if err := st.db.QueryRow(`SELECT sends, deliveries FROM mail_sender_counts
WHERE account = 'bag' AND app = 'harbourmail' AND sender = 'app' AND day = '2026-09-14'`).Scan(&sends, &deliveries); err != nil {
t.Fatalf("the backfill did not land: %v", err)
}
if sends != 7 || deliveries != 6 {
t.Fatalf("backfilled %d sends and %d deliveries, want 7 and 6", sends, deliveries)
}
// A row removed after the backfill stays removed across a restart.
if _, err := st.db.Exec(`DELETE FROM mail_sender_counts WHERE account = 'bag'`); err != nil {
t.Fatal(err)
}
if err := st.migrate(); err != nil {
t.Fatal(err)
}
var n int
if err := st.db.QueryRow(`SELECT count(*) FROM mail_sender_counts WHERE account = 'bag'`).Scan(&n); err != nil {
t.Fatal(err)
}
if n != 0 {
t.Fatalf("the backfill resurrected %d deleted rows", n)
}
// The legacy table is dropped only when an operator asks, a release later.
if err := st.DropLegacyMailCounts(false); err != nil {
t.Fatal(err)
}
if err := st.db.QueryRow(`SELECT count(*) FROM mail_counts`).Scan(&n); err != nil {
t.Fatalf("mail_counts was dropped while the flag was off: %v", err)
}
if err := st.DropLegacyMailCounts(true); err != nil {
t.Fatal(err)
}
if err := st.db.QueryRow(`SELECT count(*) FROM mail_counts`).Scan(&n); err == nil {
t.Fatal("mail_counts survived its own drop")
}
if err := st.Close(); err != nil {
t.Fatal(err)
}
}Add "path/filepath" to the test file's imports. The constructor is Open, not OpenStore (internal/console/store.go:53).
- [ ] Step 2: Run it to verify it fails
go test -run TestMailSenderCountsAreBackfilledExactlyOnce ./internal/console/Expected: no such table: mail_sender_counts.
- [ ] Step 3: Write the code
internal/console/store.go, in the create loop immediately after the mail_counts statement (:209):
// Per-SENDER counts (senders design §8). A new table rather than a
// widened key, because SQLite cannot ALTER a primary key. Every count
// is written here from now on; mail_counts stays, unwritten, and is
// dropped a release later by DropLegacyMailCounts.
//
// day is a UTC date string, not a timestamp, exactly as mail_counts'
// is — which is why this table cannot join mailAppTables blindly: the
// rename, purge and reap sweeps key on mailAppTimeColumn and there is
// no such column here.
`CREATE TABLE IF NOT EXISTS mail_sender_counts (
account TEXT NOT NULL,
app TEXT NOT NULL,
sender TEXT NOT NULL,
day TEXT NOT NULL, -- UTC YYYY-MM-DD
sends INTEGER NOT NULL DEFAULT 0,
bounces INTEGER NOT NULL DEFAULT 0,
complaints INTEGER NOT NULL DEFAULT 0,
rejects INTEGER NOT NULL DEFAULT 0,
deliveries INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (account, app, sender, day)
);`,and after the ALTER loop, before migrateAppsKey() (:371):
// The one-shot backfill of mail_sender_counts from mail_counts (senders
// design §8). It runs AFTER the ALTER list because mail_counts.deliveries
// is added there, and it runs at most once for the life of the database:
// a backfill repeated at every start would resurrect rows a purge or a
// rename had since removed, since mail_counts is not swept by anything.
// PRAGMA user_version is the marker because the console has no other
// place to keep one and this is the first thing that has needed it.
if err := s.backfillMailSenderCounts(); err != nil {
return err
}// mailCountsBackfillVersion is the user_version this migration stamps. The
// next migration that needs a one-shot marker takes 2 and reads this
// constant rather than the literal.
const mailCountsBackfillVersion = 1
func (s *Store) backfillMailSenderCounts() error {
var version int
if err := s.db.QueryRow(`PRAGMA user_version`).Scan(&version); err != nil {
return err
}
if version >= mailCountsBackfillVersion {
return nil
}
if _, err := s.db.Exec(`INSERT OR IGNORE INTO mail_sender_counts
(account, app, sender, day, sends, bounces, complaints, rejects, deliveries)
SELECT account, app, 'app', day, sends, bounces, complaints, rejects, deliveries FROM mail_counts`); err != nil {
return err
}
// PRAGMA takes no bound parameter, and the value is a constant in this
// file rather than anything a caller supplies.
_, err := s.db.Exec(fmt.Sprintf(`PRAGMA user_version = %d`, mailCountsBackfillVersion))
return err
}
// DropLegacyMailCounts removes the pre-senders counts table. Guarded by a
// flag that defaults off and driven by an operator a release after the
// backfill, for the same reason mail.SweepLegacyRecords is: the rollback
// window has to close before the old shape goes, and dropping it early would
// leave a rolled-back console counting into a table that is not there.
func (s *Store) DropLegacyMailCounts(enabled bool) error {
if !enabled {
return nil
}
_, err := s.db.Exec(`DROP TABLE IF EXISTS mail_counts`)
return err
}cmd/console/main.go, inside the CARLOS_MAIL_LEGACY_SWEEP == "1" block from task 4, after the bucket sweep:
if err := st.DropLegacyMailCounts(true); err != nil {
log.Printf("console: mail legacy sweep: dropping mail_counts -> %v", err)
}- [ ] Step 4: Run it to verify it passes
go test -run TestMailSenderCounts ./internal/console/Expected: PASS.
- [ ] Step 5: Gate and commit
go build ./... && go vet ./... && go test ./... -count=1 && gofmt -l .
git add internal/console cmd/console
git commit -m "console: mail_sender_counts, its one-shot backfill and the legacy drop (plan 2, task 8)"---
Task 9: RecordMailEvent and MailCounts take a sender
Files:
- Modify:
internal/console/mailcounts.go(RecordMailEventat:106,MailCountsat:120, and its OWN caller insideenforceMailLimitsat:230),internal/console/sesevents.go(:523),internal/console/apimail.go(:403),internal/console/mailcounts_test.go(:83,:88,:91),internal/console/mailevents_test.go(:165,:306,:418),internal/console/sesevents_test.go(:295,:778)
There are FOUR production call sites, not three. enforceMailLimits reads the counts itself at mailcounts.go:230 — c, err := s.store.MailCounts(ctx, acct, app, day) — in the same file the signature changes in, and the package does not compile without it.
Interfaces:
- Produces: ``
go func (s *Store) RecordMailEvent(ctx context.Context, acct, app, sender, day, evType string) error func (s *Store) MailCounts(ctx context.Context, acct, app, sender, day string) (MailCounts, error) // sender "" sums every sender``
- [ ] Step 1: Write the failing test
internal/console/mailcounts_test.go (append):
// Counts are per sender from here on. An empty sender reads the app's whole
// day across every sender, which is what the status reply's top-level
// figures and the roll-up both need.
func TestMailCountsAreKeptAndReadPerSender(t *testing.T) {
fx := newConsoleFixture(t)
ctx := context.Background()
const day = "2026-09-15"
for i := 0; i < 3; i++ {
if err := fx.st.RecordMailEvent(ctx, release.OpsAccount, "harbourmail", mail.SenderAppID, day, mail.EventSend); err != nil {
t.Fatal(err)
}
}
for i := 0; i < 2; i++ {
if err := fx.st.RecordMailEvent(ctx, release.OpsAccount, "harbourmail", "book.example.com", day, mail.EventSend); err != nil {
t.Fatal(err)
}
}
if err := fx.st.RecordMailEvent(ctx, release.OpsAccount, "harbourmail", "book.example.com", day, mail.EventBounce); err != nil {
t.Fatal(err)
}
app, err := fx.st.MailCounts(ctx, release.OpsAccount, "harbourmail", mail.SenderAppID, day)
if err != nil {
t.Fatal(err)
}
if app.Sends != 3 || app.Bounces != 0 {
t.Fatalf("app sender = %+v, want 3 sends and no bounces", app)
}
one, err := fx.st.MailCounts(ctx, release.OpsAccount, "harbourmail", "book.example.com", day)
if err != nil {
t.Fatal(err)
}
if one.Sends != 2 || one.Bounces != 1 {
t.Fatalf("instance sender = %+v, want 2 sends and 1 bounce", one)
}
all, err := fx.st.MailCounts(ctx, release.OpsAccount, "harbourmail", "", day)
if err != nil {
t.Fatal(err)
}
if all.Sends != 5 || all.Bounces != 1 {
t.Fatalf("every sender = %+v, want 5 sends and 1 bounce", all)
}
// A day with no rows is a legitimate zero, not an error and not an
// ErrNoRows the caller has to know about.
quiet, err := fx.st.MailCounts(ctx, release.OpsAccount, "harbourmail", "", "2026-01-01")
if err != nil || quiet != (MailCounts{}) {
t.Fatalf("a quiet day = %+v %v", quiet, err)
}
}- [ ] Step 2: Run it to verify it fails
go test -run TestMailCountsAreKeptAndReadPerSender ./internal/console/Expected: build failure, not enough arguments in call to fx.st.RecordMailEvent.
- [ ] Step 3: Write the code
internal/console/mailcounts.go, replacing RecordMailEvent and MailCounts:
// RecordMailEvent increments the one counter evType names for (acct, app,
// sender, day). An event type this function does not know is silently
// ignored — mirrored from mail.ParseEvent's own tolerance of unknown types,
// for the same reason: SES adding a type must never turn into a 500 that
// makes SNS retry forever.
//
// Writes mail_sender_counts alone. mail_counts stays on disk, unwritten,
// until DropLegacyMailCounts removes it a release later.
func (s *Store) RecordMailEvent(ctx context.Context, acct, app, sender, day, evType string) error {
col, ok := mailCountColumn(evType)
if !ok {
return nil
}
if sender == "" {
sender = mail.SenderAppID
}
q := fmt.Sprintf(`INSERT INTO mail_sender_counts (account, app, sender, day, %s)
VALUES (?, ?, ?, ?, 1)
ON CONFLICT(account, app, sender, day) DO UPDATE SET %s = %s + 1`, col, col, col)
_, err := s.db.ExecContext(ctx, q, acct, app, sender, day)
return err
}
// MailCounts returns one sender's counts for one UTC day. An EMPTY sender
// reads every sender the app has, which is what the status reply's
// unchanged top-level figures mean and what a roll-up of an app in
// per-instance mode needs. A day with no rows at all is a legitimate zero.
func (s *Store) MailCounts(ctx context.Context, acct, app, sender, day string) (MailCounts, error) {
q := `SELECT COALESCE(SUM(sends), 0), COALESCE(SUM(bounces), 0), COALESCE(SUM(complaints), 0),
COALESCE(SUM(rejects), 0), COALESCE(SUM(deliveries), 0)
FROM mail_sender_counts WHERE account = ? AND app = ? AND day = ?`
args := []any{acct, app, day}
if sender != "" {
q += ` AND sender = ?`
args = append(args, sender)
}
var c MailCounts
// An aggregate always returns exactly one row, so there is no ErrNoRows
// branch to get wrong and a quiet day reads as zeroes by construction.
if err := s.db.QueryRowContext(ctx, q, args...).Scan(
&c.Sends, &c.Bounces, &c.Complaints, &c.Rejects, &c.Deliveries); err != nil {
return MailCounts{}, err
}
return c, nil
}Call sites, each taking the app sender because that is the only sender any of them can be about today:
internal/console/sesevents.go:523->s.store.RecordMailEvent(ctx, a.Sqid, app, mail.SenderAppID, day, ev.Type)(task 11 replaces this line properly).internal/console/apimail.go:403->s.store.MailCounts(ctx, acct.Sqid, app, "", day): the top-level reply fields describe the app as a whole and keep doing so.internal/console/mailcounts.go:230, insideenforceMailLimits->s.store.MailCounts(ctx, acct, app, "", day).""and notmail.SenderAppID, deliberately:enforceMailLimitsstill takes no sender in this task, so the only cap it can enforce is the app's whole day, which is exactly what it enforces today. Task 12 gives the function a sender and this line becomess.store.MailCounts(ctx, acct, app, sender, day).internal/console/mailcounts_test.go:83,:88,:91-> addmail.SenderAppIDafter the app.internal/console/mailevents_test.go:165,:306,:418andinternal/console/sesevents_test.go:295,:778-> add""after the app, since each asserts the app's whole day.
- [ ] Step 4: Run it to verify it passes
go test ./internal/console/ -count=1Expected: PASS.
- [ ] Step 5: Gate and commit
go build ./... && go vet ./... && go test ./... -count=1 && gofmt -l .
git add internal/console
git commit -m "console: mail counts are recorded and read per sender (plan 2, task 9)"---
Task 10: The rename, purge and reap sweeps take the new table
Files:
- Modify:
internal/console/mailevents.go(mailAppTablesat:94,carryMailAppat:114,mailAppTimeColumnat:170),internal/console/mailevents_test.go
Interfaces:
- Produces:
mailAppTablesgains"mail_sender_counts";mailAppTimeColumn("mail_sender_counts") == "day";carryMailAppcompares a date string for adaytable and a unix stamp for every other.
- [ ] Step 1: Write the failing test
internal/console/mailevents_test.go (append):
// A sender's counts belong to the app, not to its name. A rename carries
// them; a purge drops them, so the next app of that name starts at zero;
// an account reap takes the lot. The comparison is on the day STRING,
// because this table has no timestamp column for mailAppTimeColumn to name.
func TestSenderCountsFollowTheAppAndDieWithIt(t *testing.T) {
fx := newConsoleFixture(t)
ctx := context.Background()
now := time.Date(2026, 9, 15, 12, 0, 0, 0, time.UTC)
yesterday := now.AddDate(0, 0, -1).UTC().Format("2006-01-02")
today := now.UTC().Format("2006-01-02")
seed := func(app, sender, day string) {
t.Helper()
if err := fx.st.RecordMailEvent(ctx, release.OpsAccount, app, sender, day, mail.EventSend); err != nil {
t.Fatal(err)
}
}
countRows := func(app string) int {
t.Helper()
var n int
if err := fx.st.db.QueryRow(`SELECT count(*) FROM mail_sender_counts WHERE account = ? AND app = ?`,
release.OpsAccount, app).Scan(&n); err != nil {
t.Fatal(err)
}
return n
}
if _, err := fx.st.CreateAppIn(OpsAccountID, "jb-book", "bill@tito.io", now); err != nil {
t.Fatal(err)
}
seed("jb-book", mail.SenderAppID, today)
seed("jb-book", "book.example.com", today)
// A row from BEFORE the app row was created belongs to a predecessor of
// the name and must not travel, the rule RenameAppIn already applies to
// audit rows.
seed("jb-book", mail.SenderAppID, yesterday)
tx, err := fx.st.db.Begin()
if err != nil {
t.Fatal(err)
}
if err := carryMailApp(ctx, tx, release.OpsAccount, "jb-book", release.OpsAccount, "jb-reader", now.Unix()); err != nil {
t.Fatal(err)
}
if err := tx.Commit(); err != nil {
t.Fatal(err)
}
if got := countRows("jb-reader"); got != 2 {
t.Fatalf("the rename carried %d rows, want the 2 from today", got)
}
if got := countRows("jb-book"); got != 0 {
t.Fatalf("%d rows were left under the old name", got)
}
tx, err = fx.st.db.Begin()
if err != nil {
t.Fatal(err)
}
if err := forgetMailApp(ctx, tx, release.OpsAccount, "jb-reader"); err != nil {
t.Fatal(err)
}
if err := tx.Commit(); err != nil {
t.Fatal(err)
}
if got := countRows("jb-reader"); got != 0 {
t.Fatalf("a purge left %d rows for the next app of that name", got)
}
seed("jb-reader", mail.SenderAppID, today)
tx, err = fx.st.db.Begin()
if err != nil {
t.Fatal(err)
}
if err := forgetMailAccount(ctx, tx, release.OpsAccount); err != nil {
t.Fatal(err)
}
if err := tx.Commit(); err != nil {
t.Fatal(err)
}
if got := countRows("jb-reader"); got != 0 {
t.Fatalf("the account reap left %d rows", got)
}
}- [ ] Step 2: Run it to verify it fails
go test -run TestSenderCountsFollowTheAppAndDieWithIt ./internal/console/Expected: the rename carried 0 rows, want the 2 from today.
- [ ] Step 3: Write the code
internal/console/mailevents.go:
// mailAppTables are the tables keyed by an app's (account sqid, name) that
// belong to that app alone: a name that is freed must not hand them on.
//
// mail_sender_counts joins the list (senders design §8), which also closes
// the gap where a rename left an app's counts behind under its old name.
// It cannot join blindly, though: every sweep here keys on a TIMESTAMP
// column and this table has a day string instead, so it gets the explicit
// case in mailAppTimeColumn and carryMailApp below. mail_counts is
// deliberately not here: nothing writes it any more and
// DropLegacyMailCounts removes it whole.
var mailAppTables = []string{"mail_events", "mail_event_hooks", "mail_event_deliveries", "mail_sender_counts"}mailAppTimeColumn (:170):
// mailAppTimeColumn is the column that says when a row came to be. Most are
// unix seconds; mail_sender_counts keys on a UTC date string, which compares
// correctly against another ISO date and against nothing else.
func mailAppTimeColumn(table string) string {
switch table {
case "mail_events":
return "received_at"
case "mail_event_hooks":
return "set_at"
case "mail_sender_counts":
return "day"
}
return "created_at"
}
// mailAppTimeSince renders createdAt in whatever mailAppTimeColumn(table)
// compares against: a unix stamp for a timestamp column, a UTC date string
// for the one day column. Comparing a unix integer against a 'YYYY-MM-DD'
// string in SQLite is not an error, it is simply always false, so getting
// this wrong would silently carry nothing.
func mailAppTimeSince(table string, createdAt int64) any {
if mailAppTimeColumn(table) == "day" {
return time.Unix(createdAt, 0).UTC().Format("2006-01-02")
}
return createdAt
}carryMailApp's UPDATE (:134-137):
if _, err := tx.ExecContext(ctx, `UPDATE `+table+` SET account = ?, app = ? WHERE account = ? AND app = ? AND `+
mailAppTimeColumn(table)+` >= ?`, toSqid, toApp, fromSqid, fromApp, mailAppTimeSince(table, createdAt)); err != nil {
return err
}- [ ] Step 4: Run it to verify it passes
go test ./internal/console/ -count=1Expected: PASS.
- [ ] Step 5: Gate and commit
go build ./... && go vet ./... && go test ./... -count=1 && gofmt -l .
git add internal/console
git commit -m "console: the rename, purge and reap sweeps take mail_sender_counts on its day column (plan 2, task 10)"---
Task 11: Attribution by caller in recordAndEnforceMail
Files:
- Modify:
internal/console/sesevents.go(recordAndEnforceMailat:466-535),internal/console/mailevents_test.go(:163-168),internal/console/sesevents_test.go
Interfaces:
- Consumes:
mail.SenderOfCaller,mail.GetSender,s.mailAppLive(sesevents.go:558uses it for the log row). - Produces: no signature change.
recordAndEnforceMailcounts once against the resolved sender, only for a live app, and falls back to the domain loop only when the caller is unknown, skipping any app in per-instance mode.
- [ ] Step 1: Write the failing test
internal/console/mailevents_test.go, replace the counts assertion inside TestTheMailLogGoesToTheAppSESSaysSentIt (:163-168) with:
// THE VISIBLE CHANGE, and it belongs in the roll note (design §8).
// Before senders, every app on a shared domain was charged for every
// event on that domain. Now only the app SES named is charged, so a
// sibling's cap stops being consumed by another app's sends.
day := now.UTC().Format("2006-01-02")
if c, _ := fx.st.MailCounts(ctx, release.OpsAccount, "jb-book", "", day); c.Bounces != 1 {
t.Errorf("jb-book bounces = %d, want the 1 SES attributed to it", c.Bounces)
}
if c, _ := fx.st.MailCounts(ctx, release.OpsAccount, "jb-site", "", day); c.Bounces != 0 {
t.Errorf("jb-site bounces = %d, want 0 — a sibling on the same domain is no longer charged", c.Bounces)
}and, further down in the same test, after the two unnamed-caller events are recorded (:190), append:
// An unknown caller keeps today's behaviour exactly: the domain is all
// there is to go on, so every app on it is charged.
for _, app := range []string{"jb-book", "jb-site"} {
if c, _ := fx.st.MailCounts(ctx, release.OpsAccount, app, "", day); c.Bounces < 2 {
t.Errorf("%s bounces = %d, want the two domain-attributed ones", app, c.Bounces)
}
}and, at the very end of the same test (before its closing brace at :206), pin the gate the stray case now needs. The stray event is already there at :172-174: its caller is mail.UserName(release.OpsAccount, "not-an-app"), which parses cleanly and so takes the named branch.
// The stray caller parses into a well-formed account and app, so it is
// "named" — but "not-an-app" is not a live app, and a count row keyed on
// a name no app answers to is a row nothing reads and nothing reaps.
if c, _ := fx.st.MailCounts(ctx, release.OpsAccount, "not-an-app", "", day); c.Bounces != 0 {
t.Errorf("a name that is not an app was given %d bounces", c.Bounces)
}internal/console/mailevents_test.go (append a new test):
// An event whose caller names a slugged sender is charged to that sender
// alone, resolved through the caller index rather than by parsing.
func TestAnIndexedCallerIsChargedToItsOwnSender(t *testing.T) {
fx := newConsoleFixture(t)
ctx := context.Background()
now := time.Date(2026, 9, 15, 12, 0, 0, 0, time.UTC)
if _, err := fx.st.CreateAppIn(OpsAccountID, "jb-book", "bill@tito.io", now); err != nil {
t.Fatal(err)
}
names, err := mail.SenderNames(release.OpsAccount, "jb-book", "book.example.com", "")
if err != nil {
t.Fatal(err)
}
if err := mail.PutCallerIndex(ctx, fx.rel, names.IAMUser, release.OpsAccount, "jb-book", "book.example.com", now); err != nil {
t.Fatal(err)
}
ev := bounceTo("ann@example.com", "m-instance", now)
ev.Caller = names.IAMUser
fx.srv.recordAndEnforceMail(ctx, ev, "sns-instance")
day := now.UTC().Format("2006-01-02")
one, _ := fx.st.MailCounts(ctx, release.OpsAccount, "jb-book", "book.example.com", day)
if one.Bounces != 1 {
t.Fatalf("the named sender's counts = %+v, want 1 bounce", one)
}
appSender, _ := fx.st.MailCounts(ctx, release.OpsAccount, "jb-book", mail.SenderAppID, day)
if appSender.Bounces != 0 {
t.Fatalf("the app sender was charged for an instance sender's bounce: %+v", appSender)
}
}- [ ] Step 2: Run it to verify it fails
go test -run 'TestTheMailLogGoesToTheAppSESSaysSentIt|TestAnIndexedCallerIsCharged' ./internal/console/Expected: jb-site bounces = 1, want 0.
- [ ] Step 3: Write the code
internal/console/sesevents.go, recordAndEnforceMail's body from :493 to :534:
// ATTRIBUTION BY CALLER (senders design §8). Every sender has its own
// IAM user and SES stamps its name on Send, Bounce, Complaint, Delivery
// and Reject, so where the sending domain can only name every app that
// shares it, the caller names the one sender that actually sent. The
// resolver parses before it reads, so an app enabled before senders
// existed still resolves with no index entry of its own.
callerAcct, callerApp, callerSender, named := mail.SenderOfCaller(ctx, s.cfg.Releases, ev.Caller)
if named {
// The COUNT is gated on the app being live, exactly as the log row
// already is (recordMailMessage, sesevents.go:558). A caller name
// can parse into an account and an app that is not a live app —
// mail.UserName(acct, "not-an-app") is a well-formed name for an
// app that was trashed, or never existed — and counting it would
// insert a mail_sender_counts row keyed on a name no app answers
// to, which nothing would ever read or reap. Returning rather than
// falling through is deliberate: SES named a sender, so the
// domain-wide loop is not the right answer for it either.
if !s.mailAppLive(callerAcct, callerApp) {
return
}
s.recordMailMessage(ctx, callerAcct, callerApp, key, ev, true, now)
// Counted ONCE, against the sender SES named, and the domain loop
// below is skipped entirely. This is the visible change in the roll
// note: a sibling app on the same domain is no longer charged for
// this send, which is the intent, but a tenant watching its own
// numbers will see them move.
if err := s.store.RecordMailEvent(ctx, callerAcct, callerApp, callerSender, day, ev.Type); err != nil {
log.Printf("console: ses events: recording %s for %s/%s/%s -> %v", ev.Type, callerAcct, callerApp, callerSender, err)
return
}
if _, reason, err := s.enforceMailLimits(ctx, callerAcct, callerApp, callerSender, at); err != nil {
log.Printf("console: ses events: enforcing limits for %s/%s/%s -> %v", callerAcct, callerApp, callerSender, err)
} else if reason != "" {
log.Printf("console: ses events: paused %s/%s/%s -> %s", callerAcct, callerApp, callerSender, reason)
}
return
}
// The fallback, unchanged: SES named nobody CARLOS minted, so the
// sending domain is all there is to go on and every app on it is
// charged, against its app sender.
for _, a := range accts {
domains, err := mail.ListDomains(ctx, s.cfg.Releases, a.Sqid)
if err != nil {
log.Printf("console: ses events: listing domains for %s: %v", a.Sqid, err)
continue
}
for _, d := range domains {
if !domainMatchesEvent(d, ev.Domain) {
continue
}
logHere := domainVerifiedForEvent(d, ev.Domain)
for _, app := range d.Apps {
// An app in per-instance mode is skipped (design §8). Its
// app sender is deliberately dead — no instance sends
// through it — so charging it for a domain-wide event it
// cannot have caused would pause a credential nobody holds
// and hide the instance that actually sent. A read failure
// is not a reason to drop the event: fall through and
// charge, which is today's behaviour.
if sn, _, err := mail.GetSender(ctx, s.cfg.Releases, a.Sqid, app, mail.SenderAppID); err != nil {
log.Printf("console: ses events: reading the app sender for %s/%s -> %v", a.Sqid, app, err)
} else if sn != nil && sn.PerInstance {
continue
}
if logHere {
s.recordMailMessage(ctx, a.Sqid, app, key, ev, false, now)
}
if err := s.store.RecordMailEvent(ctx, a.Sqid, app, mail.SenderAppID, day, ev.Type); err != nil {
log.Printf("console: ses events: recording %s for %s/%s: %v", ev.Type, a.Sqid, app, err)
continue
}
if _, reason, err := s.enforceMailLimits(ctx, a.Sqid, app, mail.SenderAppID, at); err != nil {
log.Printf("console: ses events: enforcing limits for %s/%s: %v", a.Sqid, app, err)
} else if reason != "" {
log.Printf("console: ses events: paused %s/%s: %s", a.Sqid, app, reason)
}
}
}
}Update the function's doc comment (:459-465): the counts are no longer domain-wide where SES names the sender, and the paragraph that says "The counts keep their domain-wide attribution untouched" is now wrong and must be replaced with the rule above plus the roll note.
enforceMailLimits's sender parameter arrives in task 12; until then this task's call sites compile against the four-argument form by dropping the sender argument, or task 12 is merged into this one. Take the first option and keep the tree green: write these two calls as s.enforceMailLimits(ctx, callerAcct, callerApp, at) and s.enforceMailLimits(ctx, a.Sqid, app, at) in this task, and let task 12 add the argument.
- [ ] Step 4: Run it to verify it passes
go test ./internal/console/ -count=1Expected: PASS.
- [ ] Step 5: Gate and commit
go build ./... && go vet ./... && go test ./... -count=1 && gofmt -l .
git add internal/console
git commit -m "console: SES events are counted against the sender SES named, not every app on the domain (plan 2, task 11)"---
Task 12: Caps and pauses per sender
Files:
- Modify:
internal/mail/walk.go(SetPausedat:962),internal/console/mailcounts.go(resumeStaleCapPauseat:160,enforceMailLimitsat:204, and theMailCountsread at:230),internal/console/apimail.go(handleAPIEmailStatusat:652),internal/console/sesevents.go(the two calls from task 11),internal/mail/walk_lifecycle_test.go,internal/console/mailcounts_test.go
Both signatures have existing test callers, and there are fifteen of them. They are not listed as an afterthought: resumeStaleCapPause's four do not just need an extra argument, they need rewriting, because each one currently fetches a *mail.Record and asserts on it.
enforceMailLimits, eleven callers ininternal/console/mailcounts_test.goat:98,:119,:139,:169,:209,:221,:264,:276,:304,:311,:341. Each is mechanical: insertmail.SenderAppIDafter"harbourmail", sofx.srv.enforceMailLimits(ctx, release.OpsAccount, "harbourmail", now)becomesfx.srv.enforceMailLimits(ctx, release.OpsAccount, "harbourmail", mail.SenderAppID, now).resumeStaleCapPause, four callers in the same file at:385,:407,:426,:448, one per test:TestResumeStaleCapPauseClearsACapPauseFromYesterday,...LeavesATodayCapPauseAlone,...NeverTouchesARatePause,...NeverTouchesAKindlessPause. Each is preceded by amail.Getand each asserts ongot.Provisioned, so themail.Getbecomes amail.GetSenderand the sender argument goes in beside it. The whole of the first one, rewritten, is the shape for all four:
func TestResumeStaleCapPauseClearsACapPauseFromYesterday(t *testing.T) {
fx, now := mailFixture(t)
pauseDirectly(t, fx, "harbourmail", mail.PauseKindCap, now.AddDate(0, 0, -1))
sn, _, err := mail.GetSender(context.Background(), fx.rel, release.OpsAccount, "harbourmail", mail.SenderAppID)
if err != nil {
t.Fatal(err)
}
got, resumed, err := fx.srv.resumeStaleCapPause(context.Background(), release.OpsAccount, "harbourmail", mail.SenderAppID, sn, now)
if err != nil {
t.Fatal(err)
}
if !resumed {
t.Fatal("did not resume a cap pause applied on an earlier UTC day")
}
if !got.Provisioned.Active {
t.Fatal("resumed record still reports Active == false")
}
if got.Provisioned.PauseKind != "" {
t.Fatalf("PauseKind = %q after resume, want cleared", got.Provisioned.PauseKind)
}
} The three assertion bodies are unchanged, because *mail.Sender and *mail.Record expose Provisioned identically. pauseDirectly (:360) keeps using mail.Get/mail.Put and needs no change: task 3 made those the app sender's accessors, so it is already writing senders/app.json.
Interfaces:
- Produces: ``
go // mail func SetSenderPaused(ctx context.Context, s release.Store, p Provisioner, acct, app, sender string, paused bool, kind string, now time.Time) (*Sender, error) func SetPaused(...) (*Record, error) // unchanged signature, delegates with SenderAppID var ErrPerInstance = errors.New("mail: the app sender is in per-instance mode") // console func (s *Server) enforceMailLimits(ctx context.Context, acct, app, sender string, now time.Time) (bool, string, error) func (s *Server) resumeStaleCapPause(ctx context.Context, acct, app, sender string, sn *mail.Sender, now time.Time) (*mail.Sender, bool, error)``
- [ ] Step 1: Write the failing test
internal/mail/walk_lifecycle_test.go (append):
// An app sender in per-instance mode refuses to resume. Nothing sets the
// flag in this release, but the refusal has to be in place before anything
// can: resumeStaleCapPause reactivates every key it touches, including every
// standalone, so a cap-paused app sender would otherwise re-arm on the next
// UTC day roll the credential every instance still holds.
func TestPerInstanceModeRefusesAResume(t *testing.T) {
ctx, s, p, pub, now := enableFixture(t)
if _, err := Enable(ctx, s, p, pub, testDeployment(), "bag", "harbourmail", now); err != nil {
t.Fatal(err)
}
if _, err := SetPaused(ctx, s, p, "bag", "harbourmail", true, PauseKindCap, now); err != nil {
t.Fatal(err)
}
sn, etag, err := GetSender(ctx, s, "bag", "harbourmail", SenderAppID)
if err != nil {
t.Fatal(err)
}
sn.PerInstance = true
sn.Provisioned.PauseKind = PauseKindPerInstance
if _, err := PutSender(ctx, s, "bag", "harbourmail", sn, etag); err != nil {
t.Fatal(err)
}
if _, err := SetPaused(ctx, s, p, "bag", "harbourmail", false, "", now); !errors.Is(err, ErrPerInstance) {
t.Fatalf("resume in per-instance mode = %v, want ErrPerInstance", err)
}
// Pausing harder is still allowed: the refusal is about re-arming a
// credential, not about turning one off.
if _, err := SetPaused(ctx, s, p, "bag", "harbourmail", true, PauseKindRate, now); err != nil {
t.Fatalf("pausing an app sender in per-instance mode was refused: %v", err)
}
}Add "errors" to walk_lifecycle_test.go's imports. Its block (:1-10) is context, testing, then appconfig, dnsrec, release — there is no errors in it today, so the errors.Is above does not compile as written.
internal/console/mailcounts_test.go (append):
// The cap applies per sender, so one sender's breach pauses that sender and
// leaves its siblings sending.
func TestTheCapIsEnforcedPerSender(t *testing.T) {
fx, now := mailFixture(t)
ctx := context.Background()
day := now.UTC().Format("2006-01-02")
for i := 0; i < 50; i++ {
if err := fx.st.RecordMailEvent(ctx, release.OpsAccount, "harbourmail", mail.SenderAppID, day, mail.EventSend); err != nil {
t.Fatal(err)
}
}
// A second sender's traffic does not count towards the first one's cap.
for i := 0; i < 50; i++ {
if err := fx.st.RecordMailEvent(ctx, release.OpsAccount, "harbourmail", "book.example.com", day, mail.EventSend); err != nil {
t.Fatal(err)
}
}
paused, reason, err := fx.srv.enforceMailLimits(ctx, release.OpsAccount, "harbourmail", mail.SenderAppID, now)
if err != nil {
t.Fatal(err)
}
if !paused || !strings.Contains(reason, "daily cap 50") {
t.Fatalf("app sender = %v %q, want a cap pause", paused, reason)
}
sn, _, err := mail.GetSender(ctx, fx.rel, release.OpsAccount, "harbourmail", mail.SenderAppID)
if err != nil || sn.Provisioned.Active || sn.Provisioned.PauseKind != mail.PauseKindCap {
t.Fatalf("the app sender's record = %+v %v", sn.Provisioned, err)
}
// The day roll clears a cap pause, still, and only a cap pause.
fresh, resumed, err := fx.srv.resumeStaleCapPause(ctx, release.OpsAccount, "harbourmail", mail.SenderAppID, sn, now.AddDate(0, 0, 1))
if err != nil || !resumed || fresh == nil || !fresh.Provisioned.Active {
t.Fatalf("the day roll did not clear the cap pause: %+v %v %v", fresh, resumed, err)
}
}- [ ] Step 2: Run it to verify it fails
go test -run 'TestPerInstanceModeRefusesAResume' ./internal/mail/
go test -run 'TestTheCapIsEnforcedPerSender' ./internal/console/Expected: build failures, undefined: ErrPerInstance and too many arguments.
- [ ] Step 3: Write the code
internal/mail/walk.go, SetPaused becomes a delegate and SetSenderPaused does the work:
// ErrPerInstance refuses a resume aimed at an app sender whose app has moved
// to per-instance senders (design §6.3). Nothing sets PerInstance in this
// release; the refusal exists so that nothing can set it later without this
// guard already being in place and tested.
var ErrPerInstance = errors.New("mail: the app sender is in per-instance mode")
// SetPaused stops or resumes the APP sender, which is every sender an app
// has until instance senders exist. Its signature is unchanged so the
// operator door and the limiter keep calling it exactly as they did.
func SetPaused(ctx context.Context, s release.Store, p Provisioner,
acct, app string, paused bool, kind string, now time.Time) (*Record, error) {
sn, err := SetSenderPaused(ctx, s, p, acct, app, SenderAppID, paused, kind, now)
if err != nil {
return nil, err
}
return sn.record(app), nil
}
// SetSenderPaused toggles one sender's keys, never its policy. A policy
// rewrite would have to be undone correctly to resume, and a half-failed
// rewrite leaves a grant that is neither the old one nor the new one;
// deactivating a key is atomic at IAM and trivially reversible.
//
// BOTH of the delivered user's keys are toggled, because an app
// mid-rotation still holds the previous credential, and every standalone
// credential of this sender is toggled too — each is its own IAM user, so
// there is no two-key ceiling interaction, and skipping them would leave a
// tenant paused for abuse still sending through a laptop credential.
//
// A RESUME is refused while PerInstance is set: the app's instances still
// hold this credential and re-arming it is exactly what per-instance mode
// exists to prevent. A pause is always allowed.
func SetSenderPaused(ctx context.Context, s release.Store, p Provisioner,
acct, app, sender string, paused bool, kind string, now time.Time) (*Sender, error) {
sn, etag, err := GetSender(ctx, s, acct, app, sender)
if err != nil {
return nil, err
}
if sn == nil || sn.Provisioned == nil {
return nil, ErrNotProvisioned
}
if !paused && sn.PerInstance {
return nil, fmt.Errorf("mail: %s/%s: resume its instance senders instead: %w", acct, app, ErrPerInstance)
}
user := sn.Provisioned.IAMUser
for _, id := range []string{sn.Provisioned.KeyID, sn.Provisioned.PrevKeyID} {
if id == "" {
continue
}
if err := p.SetKeyActive(ctx, user, id, !paused); err != nil {
return nil, err
}
}
for _, sa := range sn.Standalone {
if err := p.SetKeyActive(ctx, sa.IAMUser, sa.KeyID, !paused); err != nil {
return nil, err
}
}
sn.Provisioned.Active = !paused
if paused {
sn.Provisioned.PauseKind = kind
} else {
sn.Provisioned.PauseKind = ""
}
sn.Provisioned.At = now
if _, err := PutSender(ctx, s, acct, app, sn, etag); err != nil {
return nil, err
}
return sn, nil
}internal/console/mailcounts.go, resumeStaleCapPause and enforceMailLimits move onto the sender:
func (s *Server) resumeStaleCapPause(ctx context.Context, acct, app, sender string, sn *mail.Sender, now time.Time) (*mail.Sender, bool, error) {
if sn == nil || sn.Provisioned == nil || sn.Provisioned.Active {
return sn, false, nil
}
// A RATE pause, a MANUAL pause and a PER-INSTANCE pause all fall through
// untouched. The first two never auto-resume by design; the third must
// not, because resuming reactivates every key including the standalones
// and the app credential every instance still holds.
if sn.Provisioned.PauseKind != mail.PauseKindCap || sn.PerInstance {
return sn, false, nil
}
pausedDay := sn.Provisioned.At.UTC().Format("2006-01-02")
today := now.UTC().Format("2006-01-02")
if pausedDay >= today {
return sn, false, nil
}
if _, err := mail.SetSenderPaused(ctx, s.cfg.Releases, s.cfg.MailProvisioner, acct, app, sender, false, "", now); err != nil {
return nil, false, err
}
if a, err := s.store.AccountBySqid(acct); err != nil {
log.Printf("console: mail resume audit: resolving account %s: %v", acct, err)
} else if a != nil {
s.audit(a, app, "ses", "mail.resumed", fmt.Sprintf("%s/%s: daily cap pause cleared on the UTC day roll", app, sender))
}
fresh, _, err := mail.GetSender(ctx, s.cfg.Releases, acct, app, sender)
if err != nil {
return nil, false, err
}
return fresh, true, nil
}
func (s *Server) enforceMailLimits(ctx context.Context, acct, app, sender string, now time.Time) (bool, string, error) {
sn, _, err := mail.GetSender(ctx, s.cfg.Releases, acct, app, sender)
if err != nil {
return false, "", err
}
if sn == nil || sn.Provisioned == nil {
return false, "", nil
}
if !sn.Provisioned.Active {
fresh, resumed, err := s.resumeStaleCapPause(ctx, acct, app, sender, sn, now)
if err != nil {
return false, "", err
}
if !resumed {
return false, "", nil
}
sn = fresh
if sn == nil || sn.Provisioned == nil || !sn.Provisioned.Active {
return false, "", nil
}
}
day := now.UTC().Format("2006-01-02")
c, err := s.store.MailCounts(ctx, acct, app, sender, day)
if err != nil {
return false, "", err
}
cap := TierCap("")
if sn.Grant != nil && sn.Grant.DailyCap > 0 {
cap = sn.Grant.DailyCap
}
// ... the switch over cap, BounceThreshold and ComplaintThreshold is
// unchanged ...
if _, err := mail.SetSenderPaused(ctx, s.cfg.Releases, s.cfg.MailProvisioner, acct, app, sender, true, kind, now); err != nil {
return false, "", err
}
if a, err := s.store.AccountBySqid(acct); err != nil {
log.Printf("console: mail pause audit: resolving account %s: %v", acct, err)
} else if a != nil {
s.audit(a, app, "ses", "mail.paused", fmt.Sprintf("%s/%s: %s", app, sender, reason))
}
return true, reason, nil
}internal/console/apimail.go, handleAPIEmailStatus (:644-657): read the app sender, resume it, and re-read the record for emailReply.
sn, _, err := mail.GetSender(ctx, s.cfg.Releases, acct.Sqid, app, mail.SenderAppID)
if err == nil {
if _, resumed, rerr := s.resumeStaleCapPause(ctx, acct.Sqid, app, mail.SenderAppID, sn, time.Now()); rerr != nil {
err = rerr
} else if resumed {
rec, _, err = mail.Get(ctx, s.cfg.Releases, acct.Sqid, app)
}
}
if err != nil {
log.Printf("console: api email status resume %s/%s: %v", acct.Sqid, app, err)
jsonError(w, http.StatusBadGateway, "could not read the email record — try again")
return
}internal/console/sesevents.go: restore the sender argument on the two enforceMailLimits calls from task 11 — s.enforceMailLimits(ctx, callerAcct, callerApp, callerSender, at) in the named branch and s.enforceMailLimits(ctx, a.Sqid, app, mail.SenderAppID, at) in the fallback loop.
internal/console/mailcounts_test.go: update the fifteen existing callers enumerated under Files above — the eleven enforceMailLimits calls take mail.SenderAppID after the app, and the four resumeStaleCapPause tests are rewritten to the shape shown there.
- [ ] Step 4: Run it to verify it passes
go test ./internal/mail/ ./internal/console/ -count=1Expected: PASS.
- [ ] Step 5: Gate and commit
go build ./... && go vet ./... && go test ./... -count=1 && gofmt -l .
git add internal/mail internal/console
git commit -m "mail, console: the cap, the pause and the day-roll resume are per sender (plan 2, task 12)"---
Task 13: emailReply gains a senders array
Files:
- Modify:
internal/console/apimail.go(emailReplyat:377-432),internal/console/apimail_test.go
Interfaces:
- Produces:
emailReply's map gains"senders": []map[string]any{...}, never null, with the top-level fields unchanged.
- [ ] Step 1: Write the failing test
internal/console/apimail_test.go (append; if the file does not exist, create it in package console with the imports the fixture needs):
// The reply gains a senders array and changes nothing else, so an older CLI
// reading only the top-level fields keeps working byte for byte.
func TestEmailReplyCarriesEverySenderAndKeepsItsTopLevelFields(t *testing.T) {
fx, now := mailFixture(t)
ctx := context.Background()
day := now.UTC().Format("2006-01-02")
if err := fx.st.RecordMailEvent(ctx, release.OpsAccount, "harbourmail", mail.SenderAppID, day, mail.EventSend); err != nil {
t.Fatal(err)
}
if err := fx.st.RecordMailEvent(ctx, release.OpsAccount, "harbourmail", "book.example.com", day, mail.EventSend); err != nil {
t.Fatal(err)
}
// A second sender, written straight to its own object the way step 5's
// instance enable will.
if _, err := mail.PutSender(ctx, fx.rel, release.OpsAccount, "harbourmail", &mail.Sender{
ID: "book.example.com", Scope: mail.ScopeInstance, Source: mail.SourceDomain, Host: "book.example.com",
Declared: &mail.Declared{EnvPrefix: "HARBOURMAIL_SMTP", From: "noreply@x.com", At: now, By: "o@example.test"},
Grant: &mail.GrantBlock{Domains: []string{"x.com"}, DailyCap: 1000, Source: "operator", At: now, By: "op"},
Provisioned: &mail.Provisioned{IAMUser: "u2", KeyID: "AKIA9", Active: true, At: now},
}, ""); err != nil {
t.Fatal(err)
}
acct, err := fx.st.AccountBySqid(release.OpsAccount)
if err != nil || acct == nil {
t.Fatalf("AccountBySqid = %v %v", acct, err)
}
rec, _, err := mail.Get(ctx, fx.rel, release.OpsAccount, "harbourmail")
if err != nil {
t.Fatal(err)
}
out, err := fx.srv.emailReply(ctx, acct, "harbourmail", rec)
if err != nil {
t.Fatal(err)
}
// Unchanged: the top-level fields still describe the app.
if out["active"] != true || out["daily_cap"] != 50 {
t.Fatalf("the top-level fields moved: active=%v daily_cap=%v", out["active"], out["daily_cap"])
}
counts, _ := out["counts"].(map[string]any)
if counts["sends"] != 2 {
t.Fatalf("top-level sends = %v, want the app's whole day across both senders", counts["sends"])
}
senders, ok := out["senders"].([]map[string]any)
if !ok || len(senders) != 2 {
t.Fatalf("senders = %#v, want two", out["senders"])
}
if senders[0]["id"] != mail.SenderAppID || senders[0]["scope"] != mail.ScopeApp {
t.Fatalf("the app sender is not first: %#v", senders[0])
}
if senders[1]["id"] != "book.example.com" || senders[1]["daily_cap"] != 1000 || senders[1]["host"] != "book.example.com" {
t.Fatalf("the second sender = %#v", senders[1])
}
first, _ := senders[0]["counts"].(map[string]any)
second, _ := senders[1]["counts"].(map[string]any)
if first["sends"] != 1 || second["sends"] != 1 {
t.Fatalf("per-sender counts = %v / %v, want one send each", first["sends"], second["sends"])
}
// Never null, so a client reads it rather than testing for absence.
bare := fx.srv.mustEmailReplyForAppWithNoSenders(t, ctx, acct)
if _, ok := bare["senders"].([]map[string]any); !ok {
t.Fatalf("senders on a bare app = %#v, want an empty array", bare["senders"])
}
}Drop the last three lines if no bare-app helper exists; the senders == nil case is covered by initialising the slice with make.
- [ ] Step 2: Run it to verify it fails
go test -run TestEmailReplyCarriesEverySender ./internal/console/Expected: senders = <nil>, want two.
- [ ] Step 3: Write the code
internal/console/apimail.go, in emailReply before the returned map:
// The senders array (senders design §8). The top-level fields above keep
// describing the APP sender and the app's whole day, so an older CLI
// still works; this is the per-sender view beside them.
all, err := mail.ListSenders(ctx, s.cfg.Releases, acct.Sqid, app)
if err != nil {
return nil, err
}
senders := make([]map[string]any, 0, len(all))
for _, sn := range all {
sc, err := s.store.MailCounts(ctx, acct.Sqid, app, sn.ID, day)
if err != nil {
return nil, err
}
entry := map[string]any{
"id": sn.ID,
"scope": sn.Scope,
"source": sn.Source,
"active": sn.Provisioned != nil && sn.Provisioned.Active,
"daily_cap": mailEffectiveCap(sn.record(app)),
"counts": map[string]any{
"sends": sc.Sends, "bounces": sc.Bounces,
"complaints": sc.Complaints, "rejects": sc.Rejects,
"deliveries": sc.Deliveries,
},
}
if sn.Host != "" {
entry["host"] = sn.Host
}
if sn.Address != "" {
entry["address"] = sn.Address
}
if sn.Provisioned != nil {
entry["key_id"] = sn.Provisioned.KeyID
if sn.Provisioned.PauseKind != "" {
entry["pause_kind"] = sn.Provisioned.PauseKind
}
}
senders = append(senders, entry)
}and "senders": senders, in the returned map. mailEffectiveCap takes a *mail.Record, so sn.record(app) needs record exported as Record() on *mail.Sender, or mailEffectiveCap gains a sibling that takes a *mail.GrantBlock. Take the second: add
// mailCapFor is mailEffectiveCap for one sender's grant. Both read the same
// rule, because the cap a status reply names must be the cap the limiter
// enforces (mailcounts.go's enforceMailLimits).
func mailCapFor(g *mail.GrantBlock) int {
if g != nil && g.DailyCap > 0 {
return g.DailyCap
}
return TierCap("")
}and have mailEffectiveCap(rec) call it.
- [ ] Step 4: Run it to verify it passes
go test ./internal/console/ -count=1Expected: PASS.
- [ ] Step 5: Gate and commit
go build ./... && go vet ./... && go test ./... -count=1 && gofmt -l .
git add internal/console
git commit -m "console: the email status reply carries a senders array beside its unchanged fields (plan 2, task 13)"---
Task 14: carlos email status prints a line per sender
Files:
- Modify:
mail_cli.go(apiEmailReplyat:249,printEmailStatusat:1129),mail_cli_test.go
Interfaces:
- Produces: ``
go type apiEmailSender struct { ID, Scope, Source, Host, Address string; Active bool; PauseKind string; DailyCap int; KeyID string; Counts apiEmailCounts } // apiEmailReply gains: Senders []apiEmailSenderjson:"senders"``
- [ ] Step 1: Write the failing test
mail_cli_test.go (append):
// A member with more than one sender sees one line each, and a member with
// only the app sender sees exactly what they saw before.
func TestPrintEmailStatusRendersALinePerSender(t *testing.T) {
var buf bytes.Buffer
printEmailStatus(&buf, apiEmailReply{
App: "harbourmail", DailyCap: 50, Active: true,
Provisioned: &apiEmailProvisioned{KeyID: "AKIA1"},
Counts: apiEmailCounts{Sends: 12},
Senders: []apiEmailSender{
{ID: "app", Scope: "app", Source: "domain", Active: true, DailyCap: 50, KeyID: "AKIA1",
Counts: apiEmailCounts{Sends: 4}},
{ID: "book.example.com", Scope: "instance", Source: "domain", Host: "book.example.com",
Active: false, PauseKind: "cap", DailyCap: 1000, KeyID: "AKIA9",
Counts: apiEmailCounts{Sends: 8, Bounces: 1}},
},
})
out := buf.String()
for _, want := range []string{"app", "book.example.com", "4/50", "8/1000"} {
if !strings.Contains(out, want) {
t.Errorf("status output is missing %q:\n%s", want, out)
}
}
if !strings.Contains(strings.ToUpper(out), "PAUSED") {
t.Errorf("a paused sender is not called out:\n%s", out)
}
}
// An older console answers with no senders array at all, and the output is
// the output it has always been.
func TestPrintEmailStatusWithNoSendersArrayIsUnchanged(t *testing.T) {
var buf bytes.Buffer
printEmailStatus(&buf, apiEmailReply{
App: "harbourmail", DailyCap: 50, Active: true,
Provisioned: &apiEmailProvisioned{KeyID: "AKIA1"},
Counts: apiEmailCounts{Sends: 12},
})
if strings.Contains(buf.String(), "senders") {
t.Errorf("a reply with no senders array printed a senders heading:\n%s", buf.String())
}
}- [ ] Step 2: Run it to verify it fails
go test -run TestPrintEmailStatus ./Expected: build failure, unknown field Senders.
- [ ] Step 3: Write the code
mail_cli.go, beside apiEmailCounts (:239):
// apiEmailSender is one sender in the status reply (tenant-email senders
// design §8). The reply's top-level fields keep describing the app sender
// and the app's whole day, so an older console that sends no senders array
// leaves this empty and the output is what it has always been.
type apiEmailSender struct {
ID string `json:"id"`
Scope string `json:"scope"`
Source string `json:"source"`
Host string `json:"host,omitempty"`
Address string `json:"address,omitempty"`
Active bool `json:"active"`
PauseKind string `json:"pause_kind,omitempty"`
DailyCap int `json:"daily_cap"`
KeyID string `json:"key_id,omitempty"`
Counts apiEmailCounts `json:"counts"`
}apiEmailReply gains Senders []apiEmailSender json:"senders"` after Counts`.
printEmailStatus, after the today: line (:1231-1232) and before the drift warning:
// One line per sender, and only when there is more than the app sender
// to say something about: an app with one sender has already had its
// whole story told by the lines above, and repeating it would be noise.
if len(res.Senders) > 1 {
fmt.Fprintln(w, " senders:")
for _, sn := range res.Senders {
what := sn.ID
if sn.Host != "" {
what = sn.Host
} else if sn.Address != "" {
what = sn.Address
}
state := c.ok("active")
if !sn.Active {
state = "paused"
if sn.PauseKind != "" {
state += " (" + sn.PauseKind + ")"
}
}
fmt.Fprintf(w, " %s %s: %s, %d/%d sends, %d bounces, %d complaints\n",
sn.Scope, what, state, sn.Counts.Sends, sn.DailyCap, sn.Counts.Bounces, sn.Counts.Complaints)
}
}The strings this task adds are member-facing. Extract them and run the copy-review gate before this commit lands, then apply the reviewed wording verbatim.
- [ ] Step 4: Run it to verify it passes
go test -run TestPrintEmailStatus ./Expected: PASS.
- [ ] Step 5: Gate and commit
go build ./... && go vet ./... && go test ./... -count=1 && gofmt -l .
git add mail_cli.go mail_cli_test.go
git commit -m "carlos email status prints a line per sender (plan 2, task 14)"---
Self-review
Spec coverage. §5's record, the slug and SenderNames with the IAM and SES bounds, the app-component truncation and the standalone refusal: tasks 1, 2. §5's migration, senders/app.json preferred with email.json served at an empty etag and Put always writing the sender key, plus the flag-guarded sweep a release later: tasks 3, 4. §5's caller reverse index, written after EnsureUser, create-only with claimDomain's idempotence rule, the stored salt on collision, deleted by RevokeStandalone and Teardown: task 5. §8's SenderOfCaller, parse first and Get only for a slug: task 6. §7's one step-2 sentence, tenant associations for each account's existing tenant: task 7. §8's mail_sender_counts, its DDL, its backfill and the explicit day-based case in the three sweeps: tasks 8, 9, 10. §8's attribution and the roll note: task 11. §8's caps, pauses and the day-roll resume per sender: task 12. §8's senders array and the per-sender status line: tasks 13, 14.
Out of scope and deliberately absent. No instance senders, no per-instance config layer, no ConfigAck change, no svcrole.Mail, no new doors, no external identities, no address claims, no provisioner refusal wrapper, no X-SES-TENANT anywhere, no <P>_CONFIG_SET. PerInstance, PauseKindPerInstance, Host, Address, Identity and TenantARN land on the record and nothing sets them.
Three rulings this plan takes. (0) Task 7's associations go in after PutDomain, not beside EnsureIdentity. An association is the one thing that stops an identity being deleted, and Teardown's orphan-claim sweep deletes the identity behind a claim that has no record, so it never reaches the disassociate. Associating only once the record is durable is what keeps that sweep able to do its job. (1) DisassociateTenantResource is built in task 7 even though §7 lists it among step 3's verbs: SES refuses to delete an identity while an association exists, so associating in step 2 without disassociating would break Teardown on the first real run, which is a capability regression and not a deferral. (2) The caller index's "account reap" deletion is Teardown's, because the console refuses to delete an account that still holds apps (handleDeleteAccount, internal/console/ownership.go:82; DeleteAccount re-checks inside its own transaction), so every index entry an account holds goes through a per-app teardown. No separate account sweep is built.
Placeholder scan. None left. Task 4 hoists rel := openStore() itself, with the real lines shown, because cmd/console/main.go:269 passes Releases: openStore() inline and srv.cfg is unexported; task 8 names the constructor as Open (internal/console/store.go:53). Task 13's optional bare-app helper is marked droppable. Everything else carries code.
Type consistency. Sender, SenderAppID, ScopeApp, ScopeInstance, SourceDomain, SourceAddress, PauseKindPerInstance, SenderKey, SendersPrefix, ValidSenderID, SenderNameSet, SenderNames, StandaloneUser, ErrNameTooLong, CallerIndex, CallerKey, GetCallerIndex, PutCallerIndex, DeleteCallerIndex, ErrCallerClaimed, ResolveSenderNames, SenderOfCaller, LegacyKey, GetSender, PutSender, ListSenders, SweepLegacyRecords, ConfigSetARN, AssociateTenantResource, DisassociateTenantResource, SetSenderPaused, ErrPerInstance, mailCapFor, mailAppTimeSince, DropLegacyMailCounts, apiEmailSender are spelled identically across every task that produces or consumes them.
The roll note this plan owes. Three lines for the release that carries it. Counts narrow from domain-wide to sender-wide, so a tenant watching its own numbers will see them move. A console rolled back after the first sender write loses the writes made in between, so a pause or a rotation in that window has to be redone. CARLOS_MAIL_LEGACY_SWEEP stays unset until the release after this one.